Skip to content

Discard partial compiler output when recovering package set upgrades - #809

Merged
f-f merged 1 commit into
masterfrom
fix/package-set-interrupted-recovery
Oct 18, 2026
Merged

f-f merged 1 commit into
masterfrom
fix/package-set-interrupted-recovery

Conversation

@thomashoneyman

Copy link
Copy Markdown
Member

If a package set upgrade gets interrupted, then it can leave the disk in a bad state which blocks further updates because the baseline package set no longer compiles, as described in #808.

Because a leftover output-backup indicates that an upgrade didn't finish, I think we should fix this by clearing the compiler output before deleting that backup, and then rely on the existing package synchronization and compiling the baseline package set to rebuild it.

I tested this locally after setting up the same case that caused the server to fail, and reproduced the failure, and then used this fix to go back to a clean state.

This fixes the recovery path in the case a job got interrupted, but it doesn’t do anything about the OOM (still not exactly sure why that happened).

@thomashoneyman
thomashoneyman force-pushed the fix/package-set-interrupted-recovery branch 2 times, most recently from a457c8b to 6a261de Compare October 18, 2026 00:04

@f-f f-f left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏

@f-f
f-f force-pushed the fix/package-set-interrupted-recovery branch 2 times, most recently from 9b6b8ad to 892a860 Compare October 18, 2026 15:07
Clear partial compiler output when an upgrade leaves a backup behind, and remove backups after completed rollbacks to preserve incremental compilation.
@f-f
f-f force-pushed the fix/package-set-interrupted-recovery branch from 892a860 to b565594 Compare October 18, 2026 15:30
@f-f
f-f merged commit 16ed34a into master Oct 18, 2026
7 checks passed
@f-f
f-f deleted the fix/package-set-interrupted-recovery branch October 18, 2026 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Interrupted package set upgrades can leave partial compiler output and block further updates

2 participants