Release pageheap_lock while HugePageFiller unbacks free pages. - #1038
Draft
copybara-service[bot] wants to merge 1 commit into
Draft
copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
ReleaseCandidates held pageheap_lock across every madvise issued for a candidate hugepage, stalling all page-level allocation and deallocation for the duration of the syscall. Unback with unback_without_lock_ instead, as the treatment and HugeCache release paths already do. While a tracker's free pages are being unbacked it is taken off the filler lists (being_released_, counted in n_in_flight_release_), so no other thread can allocate from it, collapse it, or move it between lists; it is accounted as a partially released hugepage for the duration. The pages about to be unbacked are added to unmapped_ before the lock is dropped, as HugeCache does, so readers such as PageAllocator::ShrinkToUsageLimitSlow never see reclaimed memory as backed. Candidates are pinned with HugePageTreatmentType::kRelease so a concurrent Put that empties one cannot free it while ReleaseCandidates still holds a pointer; such trackers are parked on fully_freed_trackers_ and drained by the caller (HugePageAwareAllocator::DrainFreedTrackers). Candidates are re-validated after each lock drop, since an earlier candidate's unback may have changed their state; the debug-only sort-order assertion no longer holds for the same reason. Pages stay free, not allocated, while in flight, so pages_allocated_ and free_pages() remain truthful. PageTracker::ReleaseFree updates released_count_ per run so it matches released_by_page_ whenever the lock is held. HandleFullyFreedTracker now records the lifetime sample and resets the anon VMA name for parked trackers too. PiperOrigin-RevId: 982709352
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release pageheap_lock while HugePageFiller unbacks free pages.
ReleaseCandidates held pageheap_lock across every madvise issued for a
candidate hugepage, stalling all page-level allocation and deallocation for
the duration of the syscall. Unback with unback_without_lock_ instead, as the
treatment and HugeCache release paths already do.
While a tracker's free pages are being unbacked it is taken off the filler
lists (being_released_, counted in n_in_flight_release_), so no other thread
can allocate from it, collapse it, or move it between lists; it is accounted as
a partially released hugepage for the duration. The pages about to be unbacked
are added to unmapped_ before the lock is dropped, as HugeCache does, so
readers such as PageAllocator::ShrinkToUsageLimitSlow never see reclaimed
memory as backed. Candidates are pinned with HugePageTreatmentType::kRelease
so a concurrent Put that empties one cannot free it while ReleaseCandidates
still holds a pointer; such trackers are parked on fully_freed_trackers_ and
drained by the caller (HugePageAwareAllocator::DrainFreedTrackers). Candidates
are re-validated after each lock drop, since an earlier candidate's unback may
have changed their state; the debug-only sort-order assertion no longer holds
for the same reason.
Pages stay free, not allocated, while in flight, so pages_allocated_ and
free_pages() remain truthful. PageTracker::ReleaseFree updates
released_count_ per run so it matches released_by_page_ whenever the lock is
held. HandleFullyFreedTracker now records the lifetime sample and resets the
anon VMA name for parked trackers too.