Skip to content

Use snapshotted package metadata - #134

Open
mini-1235 wants to merge 1 commit into
RoboStack:masterfrom
mini-1235:codex/snapshot-package-metadata
Open

Use snapshotted package metadata#134
mini-1235 wants to merge 1 commit into
RoboStack:masterfrom
mini-1235:codex/snapshot-package-metadata

Conversation

@mini-1235

Copy link
Copy Markdown
Contributor

Summary

  • load package.xml from each package's URL and ref in rosdistro_snapshot.yaml
  • use snapshot manifests for recursive dependency discovery
  • make snapshot membership and package enumeration independent of the current live rosdistro
  • preserve the existing live-rostdistro behavior when no snapshot is configured
  • add package-agnostic regression coverage for source, version, manifest, dependency, recipe, and membership consistency

Problem

Vinca already used snapshot data for a recipe's source URL, ref, and version, but it continued to use the current live rosdistro cache for package.xml, package membership, package enumeration, and recursive dependency walking. If package metadata changed after the snapshot was created, a generated recipe could therefore combine old source with new dependencies.

A concrete example is moveit_py in Rolling:

With a 2.14.1 snapshot and live 2.15.0 metadata, Vinca omitted the ROS vendor package required by the pinned source and the build failed while looking for pybind11_vendorConfig.cmake.

Fix

When a non-empty snapshot is active, this patch makes it authoritative for package-level data. Pinned manifests are fetched through the existing GitHub, GitLab, or archive paths and cached. Recursive ROS dependencies are then derived from those manifests, with package conditions evaluated and non-ROS dependencies left for normal conda dependency resolution.

The regression tests use synthetic snapshot and live releases with deliberately different dependencies. They verify that the generated recipe contains only the dependency required by the pinned source and that an empty snapshot still follows live rosdistro.

Fixes #93.

Validation

  • pytest -q vinca/ — 117 passed
  • ruff check vinca/distro.py vinca/main.py vinca/test_snapshot_metadata.py
  • ruff format --check vinca/distro.py vinca/main.py vinca/test_snapshot_metadata.py
  • live probe against the pinned Rolling MoveIt 2.14.1 release manifest confirmed pybind11_vendor is selected and live-only pybind11-dev is absent

Signed-off-by: Maurice <mauricepurnawan@gmail.com>
@mini-1235

Copy link
Copy Markdown
Contributor Author

I am seeing the same issue in RoboStack/ros-rolling#31, which is what led me to open this PR. However, I noticed that this has been a longstanding issue since last year, with little activity so far.

For now, I am opening this as a draft to get the maintainers' thoughts and see whether there is interest in pursuing this approach before I spend more time manually validating it :)

@traversaro

Copy link
Copy Markdown
Member

However, I noticed that this has been a longstanding issue since last year, with little activity so far.

And that is why this PR is really great, thanks a lot for working on this!

@traversaro

Copy link
Copy Markdown
Member

For now, I am opening this as a draft to get the maintainers' thoughts and see whether there is interest in pursuing this approach before I spend more time manually validating it :)

It is definitely interesting, this issue hit me several time in the past as well.

@mini-1235

mini-1235 commented Aug 17, 2026

Copy link
Copy Markdown
Contributor Author

For now, I am opening this as a draft to get the maintainers' thoughts and see whether there is interest in pursuing this approach before I spend more time manually validating it :)

It is definitely interesting, this issue hit me several time in the past as well.

Great, I can take another look at this issue once I have cleared the PRs I need to prioritize, probably by the end of August. I will let you know when this is ready to review

@mini-1235

Copy link
Copy Markdown
Contributor Author

@traversaro I validated this today by generating the recipe locally both before and after this PR, using the moveit_py case mentioned in the description.

As shown in the image, the only difference between the two generated recipes is the addition of pybind11-vendor, which is the expected result.

image

I also asked my agent to open a draft PR in my fork to make the comparison easier to review. The reproduction is available here: mini-1235/ros-rolling#1

Based on this validation, I am convinced that this fixes the issue. Let me know what you think.

@mini-1235
mini-1235 marked this pull request as ready for review August 27, 2026 12:06
@traversaro

Copy link
Copy Markdown
Member

Great, thanks a lot! This fixes a really big bug. Good for me, but given how tricky this is, I would wait also for @Tobias-Fischer thumbs up on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove use of self._distro (that uses latest rosdistro data) if snapshot is used

2 participants