Port native malloc allocation profiling from async-profiler

[jbachorik]

⚠️ Code origin notice: The core implementation (mallocTracer.cpp, mallocTracer.h) was directly ported from async-profiler (Apache-2.0). All source files carry the original Copyright The async-profiler authors header. Datadog modifications are attributed separately in each file.

What does this PR do?:
Ports the native malloc allocation profiler from async-profiler and integrates it with the Datadog JFR pipeline. When enabled via nativemem=<interval>, the profiler intercepts malloc, calloc, realloc, posix_memalign, and aligned_alloc across all loaded native libraries using GOT patching, and emits profiler.Malloc JFR events with Java stack traces. The free function is hooked to forward correctly through the GOT but free events are not recorded — sampled mallocs mean most frees would match nothing, and the immense event volume with no stack traces provides no actionable insight.

Changes:

• mallocTracer.cpp/h — ported from async-profiler; GOT-patching hooks, Poisson byte-interval sampling with PID rate-limiting, nested-malloc detection for musl compatibility
• flightRecorder.cpp/h — recordMallocSample() for profiler.Malloc JFR events with profiling context (spanId, localRootSpanId, contextAttributes)
• jfrMetadata.cpp/h — new profiler.Malloc (T_MALLOC) event type definition with weight and context fields
• profiler.cpp/h — BCI_NATIVE_MALLOC path in recordSample, dlopen_hook patching of newly loaded libraries, CSTACK_VM promotion when VMStructs available
• hotspot/hotspotSupport.cpp — BCI_NATIVE_MALLOC case in eventTypeFromBCI and walkJavaStack
• jvmSupport.cpp — BCI_NATIVE_MALLOC allowed in walkJavaStack assert
• arguments.cpp/h — nativemem=<bytes> argument parsing
• codeCache.cpp/h — im_posix_memalign / im_aligned_alloc import IDs
• event.h — MallocEvent struct with weight field
• vmEntry.h — BCI_NATIVE_MALLOC = -20 frame type constant
• doc/architecture/NativeMemoryProfiling.md — architecture document

Motivation:
Native heap allocations (malloc/free) are a significant source of memory pressure and latency in JVM applications that rely on JNI, off-heap buffers, or native libraries. This feature gives users visibility into native allocation patterns alongside existing JVM heap profiling.

Additional Notes:

Upstream source: mallocTracer.cpp and mallocTracer.h are a port of the equivalent files from async-profiler. The porting involved:

• Replacing async-profiler's internal types and helpers with Datadog profiler equivalents
• Routing events through the Datadog JFR pipeline (recordSample) instead of async-profiler's own serialisation
• Adapting the patchLibraries loop to use Datadog's CodeCache / UnloadProtection API

Stack walking: Native malloc events have no signal context (ucontext == NULL). CSTACK_VM (HotSpot VMStructs + JavaFrameAnchor) is the only mode that can produce meaningful Java stack traces in this situation. CSTACK_DEFAULT is the initial default; at profiler start it is promoted to CSTACK_VM when VMStructs are available. On JVMs where VMStructs are unavailable the profiler stays at CSTACK_DEFAULT.

Sampling: Uses Poisson-interval sampling (shouldSample()) with a lock-free CAS loop. A PID controller (updateConfiguration()) periodically adjusts the interval to maintain ~100 samples/second. Each sample carries a statistical weight reflecting the Poisson sampling probability.

No free event tracking: Free calls are hooked (to forward through the GOT correctly) but not recorded. With Poisson sampling on mallocs, most frees correspond to unsampled allocations and would produce meaningless events. The volume of free calls with no stack traces provides no actionable insight.

Reentrancy: Allocations made by the profiler itself during recording (stack walking, JFR buffer writes) will re-enter the hooks. This is a deliberate design trade-off (no TLS guard) documented in the source — it does not cause infinite recursion but may produce minor double-accounting.

How to test the change?:
Automated integration tests covering malloc sampling:

./utils/run-docker-tests.sh --mount --tests="*NativememProfilerTest*"

Tests pass for both cstack=vm and cstack=vmx variants of:

• NativememProfilerTest#shouldRecordMallocSamples

For Datadog employees:

• If this PR touches code that signs or publishes builds or packages, or handles
  credentials of any kind, I've requested a review from @DataDog/security-design-and-guidance.
• This PR doesn't touch any of that.
• JIRA: PROF-13909

Unsure? Have a question? Request a review!

[dd-octo-sts]

CI Test Results

Run: #24854441810 | Commit: 7e0a8ce | Duration: 24m 56s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK	glibc-aarch64/debug	glibc-amd64/debug	musl-aarch64/debug	musl-amd64/debug
8	-	✅	-	-
8-ibm	-	✅	-	-
8-j9	✅	✅	-	-
8-librca	-	-	✅	✅
8-orcl	-	✅	-	-
11	-	✅	-	-
11-j9	✅	✅	-	-
11-librca	-	-	✅	✅
17	✅	✅	-	-
17-graal	✅	✅	-	-
17-j9	✅	✅	-	-
17-librca	-	-	✅	✅
21	✅	✅	-	-
21-graal	✅	✅	-	-
21-librca	-	-	✅	✅
25	✅	✅	-	-
25-graal	✅	✅	-	-
25-librca	-	-	✅	✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0

---

Updated: 2026-04-23 19:53:53 UTC

[pr-commenter]

Integration Tests

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 8d7dcd3c

[jbachorik]

@copilot resolve the merge conflicts in this pull request

[Copilot]

@copilot resolve the merge conflicts in this pull request

Resolved: I merged origin/main into this PR branch, fixed all merge conflicts, and pushed the merge commit 9e1f33f.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• groovy.jfrog.io
  • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.xml/javax.xml.namespace=ALL-UNNAMED --add-opens=java.base/java.time=ALL-UNNAMED -XX:MaxMetaspaceSize=384m -XX:+HeapDumpOnOutOfMemoryError -Xms256m -Xmx512m -Dfile.encoding=UTF-8 (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[Copilot]

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.