Skip to content

Java profiler remote enablement - #907

Draft
zhengyu123 wants to merge 16 commits into
mainfrom
zgu/remote_enablement
Draft

zhengyu123 wants to merge 16 commits into
mainfrom
zgu/remote_enablement

Conversation

@zhengyu123

@zhengyu123 zhengyu123 commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?:

Motivation:

Additional Notes:

How to test the change?:

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles
    credentials of any kind, I've requested a security review (run the dd:platform-security-review
    skill, or file a request via the PSEC review form).
    bewaire also runs automatically on every PR.
  • This PR doesn't touch any of that.
  • JIRA: PROF-16142

Unsure? Have a question? Request a review!

@dd-octo-sts

dd-octo-sts Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #41750446348 | Commit: 7ebf7f5 | Duration: 25m 8s (longest job)

❌ 2 of 32 test jobs failed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - ✅ - -
8-ibm - ✅ - -
8-j9 ✅ ✅ - -
8-librca - - ✅ ✅
8-orcl - ✅ - -
11 - ✅ - -
11-j9 ✅ ✅ - -
11-librca - - ❌ ❌
17 ✅ ✅ - -
17-graal ✅ ✅ - -
17-j9 ✅ ✅ - -
17-librca - - ✅ ✅
21 ✅ ✅ - -
21-graal ✅ ✅ - -
21-librca - - ✅ ✅
25 ✅ ✅ - -
25-graal ✅ ✅ - -
25-librca - - ✅ ✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

❌ Failing tests

Test Cells Message
JvmtiEventTimingTest.agentOnLoadSeesNativeBindingsBeforeFirstStart__ musl-11-librca-debug-aarch64, musl-11-librca-debug-amd64 VMStructs::NativeMethodBind never resolved getMemoryUsage0: NATIVE_METHOD_BIND was not enabled when VMInit bound it, so

Retried 2 of 32 cells.

Failed Jobs

Summary: Total: 32 | Passed: 30 | Failed: 2


Updated: 2026-11-04 01:24:20 UTC

@dd-octo-sts

dd-octo-sts Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 36048bc1

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Pipelines

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 36048bc | Docs | Give us feedback!

zhengyu123 and others added 7 commits October 27, 2026 18:00
Resolved the include conflict in hotspot/jitCodeCache.cpp by keeping both
common.h (TEST_LOG, this branch) and counters.h (Counters::, main).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- CallTraceStorage allocates no tables until acquire(): the profiler holds
  none before the first start(). processTraces() on released storage hands
  the processor only the dropped-trace placeholder instead of crashing.
- start(): the nativemem-only failure path and the stack-buffer calloc
  failure path now close thread registration and release recording memory,
  like the "no engine activated" path.
- Set DebugNonSafepoints at agent load on every HotSpot; COMPILED_METHOD_LOAD
  is only enabled on the first start(), and its replay cannot add debug info
  to code compiled before then.
- recordTrace0()/recordQueueEnd0() do nothing while the profiler is idle, so
  they no longer re-allocate the dictionaries stop() released.
- patch_pthread_create() re-checks already patched libraries only when
  glibc's unload count (dlpi_subs) changed since the last full pass.
- Move the per-platform MallocHooker patch/unpatch into mallocTracer_linux.cpp
  and mallocTracer_macos.cpp.
- Add missing Datadog copyright lines.
- NativeMem::logLive() prints live bytes per category (total in KB) after
  Profiler::init() and before Profiler::start(); ProfilerLifecycleTest prints
  its totals in KB.
- Update the affected architecture docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zhengyu123 zhengyu123 changed the title Java profiler remote enablement (minimize memory footprint in idle mode) Java profiler remote enablement Nov 3, 2026
zhengyu123 and others added 3 commits November 3, 2026 21:15
Conflict resolution:
- Thread registration: take main's gated registration (#912: in-flight
  counters, _engines_ready/_thread_tables_stable, ThreadTablesFreeze) and drop
  this branch's _thread_registration_open and CTimer::_registration_open
  gates, which it supersedes. unregisterThreadInEngines() keeps its NULL
  engine checks: the pthread_create hook is persistent here, so a thread can
  exit before the first start().
- CTimer: take main's ctimer.h/ctimer_linux.cpp.
- Call-trace buffers: keep acquireRecordingMemory() and the reallocation after
  stop() freed the buffers, with main's capacity-based sizing;
  releaseRecordingMemory() now accounts and resets _calltrace_buffer_capacity.
- start() error paths: main's rollBackStart() plus releaseRecordingMemory().
- Keep the persistent JVMTI event setup next to main's allocstack resolution.
- Docs and AGENTS.md: combine both sides; describe main's registration gates.
- allocStackDiagnostics_ut: acquire call-trace storage in the fixture; the
  storage holds no tables until a recording starts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dd-octo-sts

dd-octo-sts Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Scan-Build Report

User:runner@runnervm48cm3
Working Directory:/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:make -j4 all
Clang Version:Ubuntu clang version 18.1.3 (1ubuntu1)
Date:Wed Nov 4 00:57:28 2026

Bug Summary

Bug TypeQuantityDisplay?
All Bugs1
Logic error
Dereference of null pointer1

Reports

Bug Group Bug Type ▾ File Function/Method Line Path Length
Logic errorDereference of null pointerperfEvents_linux.cppunregisterThread72026

zhengyu123 and others added 3 commits November 4, 2026 13:43
- CTimer::stop(): sweep the timers inside releaseTimerTable()'s freeze, after
  the drain, so a registration that outlived the registration drain cannot
  publish a timer that the free then loses; without a drain, keep the table
  and sweep without the freeze.
- LivenessTracker: track() drops a sample whose table generation (read before
  the stack capture) moved, i.e. a callback that straddled a stop and may
  carry a call_trace_id from released call-trace storage.
- start(): a scope guard releases recording memory on every failure after
  acquireRecordingMemory(), including the replaceCpuEngine() path that leaked.
- Intern gate (renamed from the tracer-API gate) also covers
  ObjectSampler::recordAllocation(); it opens before the engines start, so a
  late allocation callback cannot re-allocate the released class map.
- releaseRecordingMemory(): free the stripe buffers and release call-trace
  storage outside lockAll(); release() tolerates a concurrent put().
- macOS malloc patch/restore: use a library only while dyld lists its path at
  the cache's slide; the name-based pin also succeeds after a reload elsewhere.
- pthread_create re-check: re-hook only a slot holding the real pthread_create
  or the library's lazy-binding stub, not a third-party hook.
- clearLeakTagsOnObjects(): MUST_POSSESS_CAPABILITY means nothing to clear.
- recordQueueEnd0: ignore null names instead of interning "".
- Fix stale comments; update the architecture docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ching

- Profiler::recordingGeneration(), bumped by releaseRecordingMemory() under
  lockAll(). ObjectSampler reads it (and the liveness table generation)
  before resolving the class id; recordJVMTISample() compares it under its
  stripe lock and drops the sample if it moved, so an allocation callback
  paused across stop/start cannot emit a class id from the previous
  recording.
- macOS isCurrentImage(): use dladdr() on the cache's Mach-O header instead of
  iterating dyld image indexes, which shift under concurrent unloads and could
  make a loaded library look gone (skipping its restore and dropping its
  entries). Drop the now unused CodeCache::textBase().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zhengyu123 zhengyu123 added the sphinx:critical Sphinx: critical — human review required label Nov 5, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sphinx:critical Sphinx: critical — human review required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant