This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
# Once per clone: let git regenerate the Manifest whenever a pull, checkout or
# rebase moves Project.toml (compat bumps from a release cut elsewhere).
# Without this every such pull leaves the Manifest illegal. See .githooks/.
git config core.hooksPath .githooks
# Run all tests — ReTestItems: every @testitem in test/*_tests.jl, in parallel
# worker processes (RETESTITEMS_NWORKERS overrides the count in test/runtests.jl;
# 0 runs serially in-process, which is what you want when debugging)
julia --project=. -e 'using Pkg; Pkg.test()'
# Run only the items whose name matches a regex. An item is named after its
# file: ransac_homography_tests.jl -> "ransac_homography". There is no
# include-one-file workflow any more — a @testitem cannot be `include`d
# standalone, it must be run through ReTestItems.
julia --project=. -e 'using Pkg; Pkg.test(; test_args = ["ransac_homography|lift"])'
# Run tests interactively with Revise (ReTestItems and the item's test deps
# must be loadable, e.g. from the global environment)
julia --project=.
julia> using Revise, ReTestItems, VisualGeometryToolkitEstimation
julia> runtests("test/ransac_homography_tests.jl") # one file
julia> runtests(VisualGeometryToolkitEstimation; name = r"^lift") # items by name
# Strict run: turns any upstream deprecation into a test failure at the
# commit that introduces it rather than at the release that removes it.
# Clean as of 2026-09-21 — run it before a release, and fix what it
# reports in the commit that owns the code.
julia --project=. -e 'using Pkg; Pkg.test("VisualGeometryToolkitEstimation"; julia_args=["--depwarn=error"])'test/quality_gates_tests.jl holds the Aqua / ExplicitImports / JET ratchets.
Each bound there is the value the package meets TODAY and may only ever move
DOWN; a change that trips one is a defect in the change, not in the bound.
configs/default.toml is the config of record, adopted from BlobBoards'
mechanism (2026-09-03): RansacConfig(), every core local-optimization
strategy's keyword defaults, every find_* pipeline's scoring table (MSAC at a threshold: a
per-class convention, not a tuned value) and local_optimization_strategy, and
fit_division_camera/fit_robot_world_hand_eye's knobs all READ that file. There is
no second set of Julia literals — src/config_defaults.jl parses it at
precompile time (include_dependency, so an edit forces a recompile; restart a
Revise session to see one) and exposes _dflt(table, key), which every
default expression calls. A default that is not in the file throws BY DESIGN.
- Override one value: a keyword —
find_homography(p; scoring = MsacScoring(HomographyProblem; threshold = 2.0)),RansacConfig(max_trials = 500),Irls{WelschShape}(efficiency_constant = 1.2, max_iter = 40).outlier_halfwidthis no keyword of the core entries (ArgumentError): it belongs to a marginal scorer supplied asscoring. - Use another file:
RansacConfig(path)loads a file's[ransac]table andoptimization_strategy_from_toml(table; class, tags = OPTIMIZATION_STRATEGY_TAGS)a strategy table; absent keys inherit the shipped file, present keys override, UNKNOWN keys/tags are anArgumentError, never ignored. (There is no whole-file loader: thePipelineConfig/RvgConfigpair,load_configandwrite_configwere deleted 2026-09-21 — only tests used them.) - Select a strategy in TOML:
local_optimization_strategy = { type = "irls", loss = "tls" }(no width key: follow the scorer; orefficiency_constant = m/loss_width = w) — tags in theTagRegistryvaluesOPTIMIZATION_STRATEGY_TAGS/LOSS_TAGS(src/config_toml.jl,scoring_glue.jl). The core registry holds"none"(nothing),"irls","triggs"and"terminal_lo"(TerminalLO, the terminal placement:{ type = "terminal_lo", inner = { … } }); LocalOptimizationDoneRight's strategies load through an explicit merge,tags = merge(VisualGeometryToolkitEstimation.OPTIMIZATION_STRATEGY_TAGS, LocalOptimizationDoneRight.LODR_STRATEGY_TAGS), which also resolvesefficiency_constant = "cross_validated"(the core refuses it, naming the merge). A loss the mechanism cannot run (a Barron shape without its α) is refused at load. - Write one back:
config_dictis derived fromfieldnames, andtest/config_toml_tests.jlasserts completeness + the strategy round trip.
Adding a tunable means: field on the struct → key in default.toml → default
expression _dflt(...). The completeness test fails until all three exist.
Revise.jl caveats:
- Compact one-liner redefinitions sometimes not picked up; use
VisualGeometryToolkitEstimation.eval(quote ... end)as workaround - Struct changes always require REPL restart
- Full test suite via
Pkg.test()in fresh process avoids Revise issues
VisualGeometryToolkitEstimation is the composable RANSAC: ONE generic loop on
VisualGeometryToolkitCore's scoring and local-optimization contracts, the RANSAC
problems of the stack, and the core local optimizations Irls{K} and Triggs{K}
(nothing is no local optimization). Out of the box every find_* runs its class's scorer ([pipelines.<class>] scoring: VGC's MSAC) with Irls{TLSShape} following the scorer's threshold; per-class
thresholds are conventions (H/F/radial 3 px, E 1 px, PnP 8 px, hand-eye
0.01·L). Paper scorers (VisualGeometryToolkitCore's StudentTUniformScoring)
and strategies (LocalOptimizationDoneRight's HQ, Gnc, StudentTUniform,
…) implement the contracts downstream and are supplied as scoring = …, local_optimization_strategy = …. Depends ONLY on VisualGeometryToolkitCore (geometry types,
solvers, the problem protocol, the contracts, the robust-statistics toolkit,
robust losses, the BCD and chart-LM cores); it must never depend on
RANSACScoringDoneRight or LocalOptimizationDoneRight.
The include order in src/VisualGeometryToolkitEstimation.jl is the authority; this tree
mirrors it. The generic block-coordinate-descent loop, the problem protocol,
the scoring and local-optimization contracts, the MSAC/RANSAC scorers, the
chart-rows contract and the Student-t / posterior-weighting toolkit are NOT
here — they are VisualGeometryToolkitCore's (robust/block_coordinate_descent.jl,
robust/lm_core.jl, robust/problem.jl, robust/scoring_contract.jl,
robust/lo_contract.jl, robust/baseline_scorers.jl, robust/rows.jl,
robust/scale.jl, robust/mixture.jl, robust/posterior.jl,
robust/losses.jl). What remains here is the RANSAC loop, the problem
implementations (with their refine! and chart-rows providers), the core
strategies and the fitting pipelines.
src/
├── VisualGeometryToolkitEstimation.jl # Entry point, imports, exports, include ordering
├── interface.jl # Shared function stubs
├── config_defaults.jl # configs/default.toml → `_dflt` (before every config-carrying struct)
├── estimators/ransac/
│ ├── types.jl # RansacConfig, RansacMetadata/RansacEstimate, DltWorkspace, LoBuffers, RansacWorkspace{K,M,T,W,L}, RansacResultBuffers
│ ├── samplers.jl # UniformSampler, ProsacSampler, GaussianSampler
│ ├── interface.jl # AbstractRansacProblem{K} API, refine_workspace, lo_workspace, RansacWorkspace constructors
│ ├── scoring_glue.jl # core strategy types (Irls{K,W}, Triggs{K,W}) + width modes, PosteriorLikelihood, OPTIMIZATION_STRATEGY_TAGS
│ ├── loop.jl # the contract-only RANSAC loop: ransac/ransac!, _try_model!, locally_optimize, _finalize
│ ├── local_optimization.jl # the core strategies' locally_optimize!, PosteriorIrlsMethod, consensus_posterior
│ ├── hypotheses.jl # draw_hypotheses: the RANSAC trial, repeated, no scoring
│ ├── extent_traits.jl # ExtentTrait / BoundedProblem validity-region wrapper
│ ├── problems/
│ │ ├── correspondence.jl # AbstractCorrespondenceProblem, MinimalSolveProblem + its aliases (Homography/FundMat/Essential/AbsolutePoseProblem)
│ │ ├── p3p.jl # AbsolutePoseProblem gates, residuals, PnP chart-LM refit
│ │ ├── bearing_pose.jl # BearingPoseProblem: pose from unit bearings (+ ray origins, pixel whitening), refine-only if non-central
│ │ ├── homography.jl # HomographyProblem
│ │ ├── radial_homography.jl # RadialHomographyProblem
│ │ ├── radial_camera.jl # RadialCameraProblem (distortion-free board verification)
│ │ ├── fundmat.jl # FundMatProblem
│ │ ├── essential.jl # EssentialProblem
│ │ ├── sampson_lm.jl # the chart-LM rows of every Sampson-constraint problem (F, E, H, circle, radial camera, rotation)
│ │ ├── relpose_bundle.jl # RelposeBundleProblem, the bundle on NLLSsolver
│ │ ├── line.jl # LineFittingProblem, InhomLineFittingProblem
│ │ ├── hand_eye.jl # HandEyeProblem
│ │ ├── pool.jl # the pool-frame problems of a pool record; PoolPair(rng, problem, raw)
│ │ └── manifold_providers.jl # ManifoldLiftProvider: the chart problems' VGC chart-rows contract (rows_provider, lift_*) (included after covariance.jl)
│ └── covariance.jl # solver_covariance: the one constraint-sandwich covariance (included after problems/hand_eye.jl)
├── fitting/
│ ├── _pipeline.jl # _ransac_pipeline / _refine_pipeline — shared bodies of every find_* / refine_*
│ ├── line.jl # fit_line
│ ├── homography.jl # find_homography, refine_homography
│ ├── radial_homography.jl # find_radial_homography, refine_radial_homography
│ ├── radial_camera.jl # find_radial_camera, refine_radial_camera
│ ├── fit_radial_camera.jl # fit_radial_camera
│ ├── division_camera_step.jl # division_camera_step, DivisionCameraState, DivisionCameraError
│ ├── fit_division_camera.jl # fit_division_camera
│ ├── fit_camera.jl # fit_camera (linear: no guess), pspline_grid, CameraFitMetadata (a lens family, a SplineLens camera's backward + forward, or a NoncentralSplineModel onto a source over a region)
│ ├── refine_camera.jl # refine_camera (the chart LM from a guess), approximate_camera (fit_camera, then refine_camera), optical_axis
│ ├── essential.jl # find_essential, refine_essential
│ ├── pose.jl # find_absolute_pose, refine_absolute_pose
│ ├── fundmat.jl # find_fundmat, refine_fundmat
│ ├── hand_eye.jl # find_hand_eye, refine_hand_eye
│ ├── robot_world_hand_eye.jl # fit_robot_world_hand_eye, RobotWorldHandEyeCalibration
│ ├── refine_robot_world_hand_eye.jl # refine_robot_world_hand_eye
│ └── residual_mixture.jl # fit_residual_mixture, MixtureEmConfig (a held model's residuals → mixture EM → MAP labels)
└── config_toml.jl # TOML loaders, OPTIMIZATION_STRATEGY_TAGS/LOSS_TAGS registries, writer (included LAST)
Include order in src/VisualGeometryToolkitEstimation.jl is dependency-aware:
- interface.jl — shared function stubs
- config_defaults.jl — parses configs/default.toml; every keyword default reads it
- estimators/ransac/ — framework (types → samplers → interface → scoring_glue → loop → local_optimization → hypotheses → extent_traits)
- estimators/ransac/problems/ — RANSAC problem implementations (correspondence →
p3p → homography → radial_homography → radial_camera → fundmat →
essential → circle → ellipse → rotation → sampson_lm → relpose_bundle →
line → hand_eye → pool), then
estimators/ransac/covariance.jl (its
solver_covariancedispatches on the problem types above), then problems/manifold_providers.jl (needs the per-problem chart/residual helpers) - fitting/*.jl — _pipeline, then the pipelines and
fits (depend on the RANSAC problems);
division_camera_step.jlprecedesfit_division_camera.jl, which followsresidual_mixture.jl(itsMixtureEmConfig),robot_world_hand_eye.jlprecedesrefine_robot_world_hand_eye.jl - config_toml.jl — LAST: its loaders and writer dispatch on every config-carrying type
scoring_glue.jl loads before the problem files. Problem-specific
specializations go in problems/.
- Functions:
snake_case(mahalanobis_distances!,minimal_sample_size,locally_optimize) - Types:
CamelCase(HomographyProblem,RansacWorkspace,RansacEstimate) - Constants:
UPPERCASE_SNAKE_CASE - Section dividers:
# ============================================================================= - Private helpers: Prefix with
_(_try_model!,_adaptive_count,_seed_posterior_state)
The rule is the stack's (conventions/signatures.md gives the rationale): the
model or shape first, then the data it acts on or is measured against, then
parameters.
project(camera, point3d),sampson_distance(H, u₁, u₂),signed_distance(line, point)- An accessor takes the one object:
minimal_sample_size(problem),model_type(problem) - A GIVEN model leads in task verbs and estimators too:
find_absolute_pose(camera, pairs),AbsolutePoseProblem(camera, pairs). - A REQUIRED tool applied to the data leads (a kernel, scorer, network, the
method being evaluated); a policy with a default value stays last
(
ransac(problem, scoring; local_optimization_strategy = …)).
Mutation, types and RNGs follow Julia's style guide exactly (the stack's
rule; conventions/signatures.md quotes the manual). Base's order: (1) a function argument, (2) an
IO — an RNG takes this slot, (3) the mutated input(s), (4) a type, (5) inputs not
mutated, (6) key, (7) value, (8) everything else, (9) varargs, (10) keywords.
The model-first rule above, and the RANSAC problem-first exception below, order
the inputs within (5) only.
- A function that modifies a Julia object argument ends in
!, and only a function argument, an IO or an RNG precede what it mutates:mahalanobis_distances!(d, problem, model),score!(W, scoring, scores, residuals, n),refine!(ws, problem, θ, weights),block_coordinate_descent!(state, likelihood, method, prob, θ₀),consensus_s2!(ws, problem, scoring, model). An output the caller reads back is the leading mutated argument of a!function (score_model!(buf, scoring, problem, θ): the caller readsbuf). - Not mutation: advancing an RNG or a RANSAC sampler's state (so
ransac,locally_optimizeand thefind_*pipelines keep their names), and a scratch buffer supplied as a KEYWORD whose contents don't matter afterwards (theirworkspace; Base'ssort(v; scratch)). - A stopping criterion is a pure query (
stop_reason): per-iteration mutation (best-tracking, a schedule's rung advance, the counters a criterion reads) goes inpost_iteration!, which the loop runs first. - A type argument is positional, never a keyword: it leads a non-mutating call
and follows the mutated inputs of a
!call. An optional type is an optional leading positional, written as separate methods:RansacWorkspace([T = Float32,] P, n_data)and every otherRansacWorkspaceform (Tthe per-point storage eltype; there are no per-family*_workspacewrappers). - An RNG leads:
draw_sample!(rng, indices, problem). - A reorder or rename makes every old call fail loudly (
MethodError/UndefVarError) and adds no method ambiguity.
Exception — the RANSAC family is problem-first among the inputs it does
not mutate: the problem (or context) carries the data, the model is the
hypothesis after it; a mutated workspace or state still leads. This includes
locally_optimize and every wrapper that holds a problem.
solve(problem, sample_indices)test_model(problem, model, sample_indices),constraint_with_jacobians(problem, model, i),solver_covariance(problem, model, sample_indices)refine!(ws, problem, θ, weights)— the local-optimization step from the current model (excluded points haveweights[i] = 0): the model right after the problem, like every siblinglocally_optimize(problem, θ₀),ransac(problem, scoring)
The other exceptions are VGC's: a method on a function owned elsewhere keeps the owner's order; a symmetric relation takes either order.
The conventions every stack package follows — naming and the estimator verbs,
signatures, data arguments, coordinate frames and normalization ("frame in =
frame out"), pixel origins, bearings, Jacobians, units and the stack
contracts — live ONCE, in VisualGeometryToolkitCore's conventions/ folder
(conventions/README.md indexes it). Read them before changing code here.
This file carries no copy of them, only how they land in this package. Where
the folder is:
- in a checkout of the stack's monorepo:
../VisualGeometryToolkitCore/conventions/(the root CLAUDE.md imports it); - in VisualGeometryToolkitCore's own repository:
conventions/; - from Julia, the installed copy:
VisualGeometryToolkitCore.conventions_dir(), i.e.joinpath(pkgdir(VisualGeometryToolkitCore), "conventions").
The agent-docs test checks that this file points there and holds no copy.
The rule is the stack's (conventions/frames.md); how it lands here:
In this package:
A RANSAC problem owns its normalization. It is built once at construction
(CorrespondenceNormalization(M, cs)), the gates and the minimal solves run on the
normalized view, and refine!(ws, problem, θ, w) uses the problem's
normalization. It never calls a public estimator, or a one-step form that
normalizes again. find_* returns residuals, scales, inlier sets and
covariances in the caller's units: px, or scene length for hand-eye.
- NEVER add "Co-Authored-By: Claude" or similar AI attribution to commit messages
- NEVER add "Generated with Claude Code" or similar footers
- Write clean, concise commit messages focused on the change itself
Export the user API only — the pipelines, the RANSAC entry points, the problems
a user builds, the LO strategies a user picks. The machinery (workspaces,
samplers, tag tables, plumbing types, extension points, research arms) is
public. RVG re-exports nothing: a name is exported only by the package that
owns it (VisualGeometryToolkitCore). Numbered category
subsections with - dividers:
# -----------------------------------------------------------------------------
# 1. PIPELINES — robust estimation end to end
# -----------------------------------------------------------------------------
export find_homography, find_fundmat, find_essential, ...
# -----------------------------------------------------------------------------
# PUBLIC, not exported — the machinery
# -----------------------------------------------------------------------------
public DltWorkspace, RansacWorkspace, ...Group related names together (types, then functions). Add brief comments where the purpose isn't obvious.
Use @btime or @benchmark from BenchmarkTools.jl. Never use @time for benchmarking (includes JIT compilation on first run).
using BenchmarkTools
@btime my_function($input) # $ interpolates to avoid benchmarking global access