Fix git-lfs file smudging in zone source accessors#13
Fix git-lfs file smudging in zone source accessors#13lilyinstarlight merged 7 commits intotecnix-gcsfrom
Conversation
d18d8db to
41340f3
Compare
There was a problem hiding this comment.
I'm still iffy on continuing to put tree shas into the git accessor and fixing up later
I'd rather a commit sha plus path to retrieve be passed then peel back to tree sha for path early on in accessor for what it needs (and I guess fingerprint on that), and then otherwise operate on commit sha for everything. That way we'd only be trying to fixup path rather than having to fix up both commit sha and path with options everywhere
But I don't have bandwidth today/tomorrow for doing that and this is still a step towards fixing the mess, thank you!
Apparently git-lfs does this internally. Hit this in CI.
lilyinstarlight
left a comment
There was a problem hiding this comment.
Confirmed with further testing that pulling from LFS objects gitdir for checked-out zones, fetching from git-lfs API for non-checked-out zones, and pulling from nix git-lfs cache all work correctly!

Fix git-lfs file smudging in zone source accessors
Problem
Zone source accessors (
getZoneStorePath,mountZoneByTreeSha,getZoneFromCheckout) were created withsmudgeLfs = false, causing git-lfs pointer files to be copied into the Nix store as-is instead of being fetched and replaced with their real content. This broke builds for zones containing LFS-tracked files (e.g.//areas/platforms/query-engine), with errors like:internal/gen/catalog/generated_schema_shopify.go:1:1: expected 'package', found version
The root cause is two-fold:
smudgeLfswas never enabled for zone accessorslfs::Fetch::shouldFetch()usesGIT_ATTR_CHECK_INCLUDE_COMMITwhich requires a commit OID to resolve.gitattributes— so LFS attribute lookup would silently fail regardlessSolution
lfsCommitRevtoGitAccessorOptions— when set, this commit hash is passed tolfs::Fetchinstead of the tree hash, allowing correct.gitattributeslookup when the accessor is created from a tree SHAsmudgeLfs = trueon all three zone accessor call sites, passing the world commit hash (from--tectonix-git-sha) vialfsCommitRevFiles Changed
src/libfetchers/include/nix/fetchers/git-utils.hh— addlfsCommitRevtoGitAccessorOptionssrc/libfetchers/git-utils.cc— uselfsCommitRevinlfs::Fetchconstructor; include inmakeFingerprintsrc/libexpr/eval.cc— enablesmudgeLfs = true+lfsCommitRevon all zone accessor call sites