Skip to content

Release pageheap_lock while HugePageFiller unbacks free pages. - #1038

Draft
copybara-service[bot] wants to merge 1 commit into
masterfrom
test_982709352
Draft

copybara-service[bot] wants to merge 1 commit into
masterfrom
test_982709352

Conversation

@copybara-service

Copy link
Copy Markdown

Release pageheap_lock while HugePageFiller unbacks free pages.

ReleaseCandidates held pageheap_lock across every madvise issued for a
candidate hugepage, stalling all page-level allocation and deallocation for
the duration of the syscall. Unback with unback_without_lock_ instead, as the
treatment and HugeCache release paths already do.

While a tracker's free pages are being unbacked it is taken off the filler
lists (being_released_, counted in n_in_flight_release_), so no other thread
can allocate from it, collapse it, or move it between lists; it is accounted as
a partially released hugepage for the duration. The pages about to be unbacked
are added to unmapped_ before the lock is dropped, as HugeCache does, so
readers such as PageAllocator::ShrinkToUsageLimitSlow never see reclaimed
memory as backed. Candidates are pinned with HugePageTreatmentType::kRelease
so a concurrent Put that empties one cannot free it while ReleaseCandidates
still holds a pointer; such trackers are parked on fully_freed_trackers_ and
drained by the caller (HugePageAwareAllocator::DrainFreedTrackers). Candidates
are re-validated after each lock drop, since an earlier candidate's unback may
have changed their state; the debug-only sort-order assertion no longer holds
for the same reason.

Pages stay free, not allocated, while in flight, so pages_allocated_ and
free_pages() remain truthful. PageTracker::ReleaseFree updates
released_count_ per run so it matches released_by_page_ whenever the lock is
held. HandleFullyFreedTracker now records the lifetime sample and resets the
anon VMA name for parked trackers too.

ReleaseCandidates held pageheap_lock across every madvise issued for a
candidate hugepage, stalling all page-level allocation and deallocation for
the duration of the syscall.  Unback with unback_without_lock_ instead, as the
treatment and HugeCache release paths already do.

While a tracker's free pages are being unbacked it is taken off the filler
lists (being_released_, counted in n_in_flight_release_), so no other thread
can allocate from it, collapse it, or move it between lists; it is accounted as
a partially released hugepage for the duration.  The pages about to be unbacked
are added to unmapped_ before the lock is dropped, as HugeCache does, so
readers such as PageAllocator::ShrinkToUsageLimitSlow never see reclaimed
memory as backed.  Candidates are pinned with HugePageTreatmentType::kRelease
so a concurrent Put that empties one cannot free it while ReleaseCandidates
still holds a pointer; such trackers are parked on fully_freed_trackers_ and
drained by the caller (HugePageAwareAllocator::DrainFreedTrackers).  Candidates
are re-validated after each lock drop, since an earlier candidate's unback may
have changed their state; the debug-only sort-order assertion no longer holds
for the same reason.

Pages stay free, not allocated, while in flight, so pages_allocated_ and
free_pages() remain truthful.  PageTracker::ReleaseFree updates
released_count_ per run so it matches released_by_page_ whenever the lock is
held.  HandleFullyFreedTracker now records the lifetime sample and resets the
anon VMA name for parked trackers too.

PiperOrigin-RevId: 982709352

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant