chore(ci): migrate Appsec fastapi tests (#12196)

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

---------

Co-authored-by: Laplie Anderson <[email protected]>
Co-authored-by: Brett Langdon <[email protected]>

chore(tracing): avoid using deprecated code internally [3.0] [backpor…

…ts 2.20] (#12182)

Backports: #12113

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

fix(profiling): fix SystemError when collecting memory profiler event…

…s [backport 2.19] (#12125)

Backports #12075 to 2.19

We added locking to the memory profiler to address crashes. These locks
are mostly "try" locks, meaning we bail out if we can't acquire them
right away. This was done defensively to mitigate the possibility of
deadlock until we fully understood why the locks are needed and could
guarantee their correctness. But as a result of using try locks, the
`iter_events` function in particular can fail if the memory profiler
lock
is contended when it tries to collect profiling events. The function
then returns NULL, leading to SystemError exceptions because we don't
set an error.

Even if we set an error, returning NULL isn't the right thing to do.
It'll basically mean we wait until the next profile iteration, still
accumulating events in the same buffer, and try again to upload the
events. So we're going to get multiple iteration's worth of events. The
right thing to do is take the lock unconditionally in `iter_events`. We
can allocate the new tracker outside the memory allocation profiler lock
so that we don't need to worry about reentrancy/deadlock issues if
we start profiling that allocation. Then, the only thing we do under the
lock is swap out the global tracker, so it's safe to take the lock
unconditionally.

Fixes #11831

TODO - regression test?

fix(openai): support Python 3.12, 3.13 [backport #12014 to 2.19] (#12083

)

Backports #12014 to 2.19.

Resolves #11994.

We were previously not testing our OpenAI integration with Python
versions >= 3.12 (at the time, OpenAI did not support those newer Python
versions). We recently saw that our bytecode wrapping for OpenAI methods
broke in Python 3.13, so this PR attempts to address that:
- Switch out our bytecode wrapping with `wrapt.wrap_function_wrapper()`
to make our patching compatible with newer Python versions.
- Add Python 3.12 and 3.13 to tested versions for the OpenAI
integration.
- Implements unpatching and adds patching tests

We previously were using direct bytecode wrapping in the OpenAI
integration, which was not supported for Python 3.13, and instead
switched to using `wrapt.wrap_function_wrapper()` to wrap OpenAI
methods. This meant that we needed to change wrapping formats,
including:
- use function/attribute names (string) rather than references as inputs
to the wrapping functions
- replace nested functions with `@with_traced_module()` to pass the
traced OpenAI module reference between traced methods
- implement unpatching
- add patching tests (note this is messy because we support both v0 and
v1 openai versions which have separate method names)

**Note**: the issue in #11994 was only reproducible via the
`AsyncOpenAI.Moderations.create(), AsyncOpenAI.Embeddings.create()`
endpoints (chat and completions were not affected). However to avoid any
risk introduced by the now unsupported bytecode wrapping, we are
changing the entire OpenAI integration to use
`wrapt.wrap_function_wrapper()` instead.

Since ddtrace 3.0 will be removing support for Python 3.7, we are also
dropping support accordingly in this PR. This also coincides with
removing support for OpenAI v0 since v0 was last released more than a 16
months ago, and requires a large maintenance burden and pollutes the
codebase. Note that this PR will just drop testing support, but a future
PR will remove tracing support for v0 in the form of a refactor/cleanup
of the OpenAI integration.

Azure OpenAI requests are changed in latest versions of OpenAI, which
fail our cassette tests (recorded in previous versions of OpenAI).
However on manual testing, our openai integration is unchanged in
support for newer versions. With consideration for time/effort to
produce new cassette test files which is time consuming, we are going to
skip cassette tests for Azure endpoints with latest openai versions
until we improve our testing framework entirely (move off cassette
files). Here are the manual traces submitted with Azure OpenAI for
reference:
<img width="578" alt="Screenshot 2025-01-23 at 2 20 00 PM"
src="https://github.com/user-attachments/assets/19f34edc-4c20-4cfd-804f-3e7bce26f4df"
/>
<img width="545" alt="Screenshot 2025-01-23 at 2 20 14 PM"
src="https://github.com/user-attachments/assets/a58bc888-6bee-4d67-9fda-0661aed14423"
/>

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

chore(integrations): internalize get_version, patch, and unpatch for …

…all integrations [3.0] (#11916)

Requires: [chore: avoids using deprecated modules in tests and log
injection [3.0
prep]](#11835).

Follow up to: #11918

This change internalizes the following functions:
- `ddtrace.contrib.<integration_name>.patch()`
- `ddtrace.contrib.<integration_name>.unpatch()`
- `ddtrace.contrib.<integration_name>.get_version()`

This change also internalizes all integrations that only expose
`patch()`, `unpatch()`, and `get_version()` functions.

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

chore(integrations): internalize get_version, patch, and unpatch for …

…all integrations [3.0] (#11916)

Requires: [chore: avoids using deprecated modules in tests and log
injection [3.0
prep]](#11835).

Follow up to: #11918

This change internalizes the following functions:
- `ddtrace.contrib.<integration_name>.patch()`
- `ddtrace.contrib.<integration_name>.unpatch()`
- `ddtrace.contrib.<integration_name>.get_version()`

This change also internalizes all integrations that only expose
`patch()`, `unpatch()`, and `get_version()` functions.

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

fix(profiling): reset all profiling c++ mutexes on fork [backport 2.1…

…9] (#11993)

Backport d855c4a from #11768 to 2.19.

I'm not sure why it took so long to surface this defect, but it turns
out that stack v2 can deadlock applications because not all mutices are
reset.

The repro in #11762 appears to be pretty durable. I need to investigate
it a bit more in order to distill it down into a native stress test we
can use moving forward. In practice, this patch suppresses the noted
behavior in the repro.

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

Co-authored-by: David Sanchez <[email protected]>

fix(celery): stop closing prerun_span too soon to account for Celery …

…chains scenario [backport 2.17] (#11805)

Backport e8aab65 from #11498 to 2.17.

We've made a few changes to handle celery context recently, including:
#10676

In particular the goal of
#10676 was to handle a
scenario where a long running task may run into an exception, preventing
it from closing.

Unfortunately, this scenario did not account for cases where tasks are
chained and may not close until later.

See: #11479 and
#11624

With this PR, the sample app in
#11479 would attach the
celery specific span back to the root span.

I also need to add tests for the chains scenario.

Related to AIDM-494

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

Co-authored-by: wantsui <[email protected]>

chore(ci): pin `greenlet` version to 3.1.0 for 3.8 [backport 2.19] (#…

…11956)

Backport 560aa89 from #11953 to 2.19.

## Context
Failing tests slipped into the 2.19 branch, one of them being [this
gevent test
failure](https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-py/-/jobs/765798135).
This was addressed in a fix PR to the main branch, and was [backported
to 2.19](#11920). However,
once backporting this fix, [ci_visibility tests on python 3.8 started to
fail
consistently](https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-py/-/jobs/765777070).

After some digging, it turns out that in the backported fix for gevent,
the [greenlet dependency was bumped from 3.0.3 to
3.1.1](https://github.com/DataDog/dd-trace-py/pull/11920/files#diff-939f705fbaed2d1386345b41d7c68406808ecbc5609022a59505453df9ba4b63),
which according to greenlet documentation, [3.1.0 was the last release
to support 3.7 and
3.8](https://greenlet.readthedocs.io/en/latest/changes.html#id2). Looks
like pip didn't catch this during the upgrade because the [minimum
runtime version is still pinned to 3.7 in
greenlet](https://github.com/python-greenlet/greenlet/blob/3.1.1/setup.py#L263).

## Description
This PR pins greenlet to 3.1.0 for python<=3.8, and will use latest for
3.9+. The failing ci_visibility tests will pass on 3.8 now.

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

Co-authored-by: erikayasuda <[email protected]>

chore(ci): pin `greenlet` version to 3.1.0 for 3.8 [backport 2.19] (#…

…11956)

Backport 560aa89 from #11953 to 2.19.

## Context
Failing tests slipped into the 2.19 branch, one of them being [this
gevent test
failure](https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-py/-/jobs/765798135).
This was addressed in a fix PR to the main branch, and was [backported
to 2.19](#11920). However,
once backporting this fix, [ci_visibility tests on python 3.8 started to
fail
consistently](https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-py/-/jobs/765777070).

After some digging, it turns out that in the backported fix for gevent,
the [greenlet dependency was bumped from 3.0.3 to
3.1.1](https://github.com/DataDog/dd-trace-py/pull/11920/files#diff-939f705fbaed2d1386345b41d7c68406808ecbc5609022a59505453df9ba4b63),
which according to greenlet documentation, [3.1.0 was the last release
to support 3.7 and
3.8](https://greenlet.readthedocs.io/en/latest/changes.html#id2). Looks
like pip didn't catch this during the upgrade because the [minimum
runtime version is still pinned to 3.7 in
greenlet](https://github.com/python-greenlet/greenlet/blob/3.1.1/setup.py#L263).

## Description
This PR pins greenlet to 3.1.0 for python<=3.8, and will use latest for
3.9+. The failing ci_visibility tests will pass on 3.8 now.

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

Co-authored-by: erikayasuda <[email protected]>