VisualGeometryToolkitCore (main module)
│
┌────────────┬───────────┼────────────┬─────────────┐
│ │ │ │ │
▼ ▼ ▼ ▼ ▼
base/ robust/ transforms ┌──────────┐ ┌──────────┐
(types, (losses, (matrices, │Primitives│ │ Cameras │
frames, BCD loop, maps, warp, └──────────┘ └──────────┘
units, LM core) batched) ▲ │
FitResult) └──────────────┘
│
▼
Base types & interface
(base/interface.jl, base/types.jl)
Submodules: Primitives, Cameras, the small Synthetic evaluation module,
Pools (the benchmark pool record) and SpatialQueries (allocation-free
R-tree queries); everything else is included directly into the main module in the
order below. Feature detection, scale space and descriptor matching live in
VisualGeometryToolkitFeatures; RANSAC problem types live in VisualGeometryToolkitEstimation.
The include order in src/VisualGeometryToolkitCore.jl is dependency-driven. The
authoritative list is the include block in that file (each line carries a
comment saying what it provides); this is the shape of it.
include("base/units.jl") # unit constants px, mm, dpi; px, pd, dpi, pt registered at __init__ — no dependencies
include("base/size2.jl") # Size2
include("base/feature_coords.jl") # _strip_coords, the feature-coordinate compute boundary
include("base/frames.jl") # Frame{N} tag hierarchy (before types.jl: CovMat/PrecMat frame params)
include("base/image_frames.jl") # 2-D image frames (pixel/image/gnomonic/retinal/distorted/target-plane) + FrameDependence
include("base/types.jl") # interface.jl + interface_functions.jl, AbstractDetection, conic matrices
include("base/correspondence_set.jl") # Correspondences columnar container + sources/targets protocol
include("base/coordinates.jl") # to_affine / to_euclidean
include("base/return_code.jl") # ReturnCode, StopRule: the termination record (before FitResult, whose core holds it)
include("base/fit_result.jl") # FitResult{P,M} — the one iterative-fit return type
include("base/termination_accessors.jl") # retcode, stop_rule, converged, found_model, retcode_symbol on a FitResult
include("base/utils.jl") # geometry utilities
include("base/config_toml.jl") # TOML config skeleton shared by every package with a configuration of record
include("base/device.jl") # default_device — the family's one KA backend resolver (CUDA ext registers)
include("base/svd_workspace.jl") # SvdWorkspace + svd_nullvec!
include("base/propagate.jl") # propagate(J, Σ) + whitening policies
include("base/manifold.jl") # the Manifold contract (retract + transform_covariance); instances in geometry/manifolds/Why first? These define the foundational types (AbstractDetection,
HomEllipseMat, Size2, Uncertain, FitResult) and the interface stubs
every later file extends.
include("robust/losses.jl") # AbstractLossShape family (LOSSES.md)
include("robust/width_axes.jl") # WidthAxis tags, width_axis ⇄ width_axis_name, to_native(::LossWidth)
include("robust/estimation.jl") # RANSAC/LO interface hierarchy (types + contracts)
include("robust/block_coordinate_descent.jl") # block_coordinate_descent! + extension verbs
include("robust/rows.jl") # the chart-rows contract (rows_provider, lift_* verbs), RowsWorkspace, task_buffers
include("robust/lm_core.jl") # the chart LM: NLLSsolver on the row protocol (ManifoldLmWorkspace)
include("robust/problem.jl") # the constraint-problem protocol (mahalanobis_distances!, refine!)
include("robust/scale.jl") # IGPrior, fit_scale, the consensus Student-t scale
include("robust/mixture.jl") # StudentTMixture, GauInlier, decision_boundary
include("robust/posterior.jl") # PosteriorWeightState, the nuisance policies, mixture_ecm!
include("robust/normaliser.jl") # truncated_log_normaliser
include("robust/scoring_contract.jl") # the scoring contract, ScoreBuffers, score_model!
include("robust/baseline_scorers.jl") # RansacScoring, MsacScoring
include("robust/lo_contract.jl") # the local-optimization and external-refit contracts, loss_atWhy here? Contracts, the robust-statistics toolkit and the baseline
scorers — no geometry. VisualGeometryToolkitEstimation implements the problem protocol and
runs its one RANSAC loop over these verbs; RANSACScoringDoneRight implements
the scoring contract; LocalOptimizationDoneRight implements the
local-optimization contract and owns the lifts and continuation schedules. The
pool-selection verbs (robust/pool_selection.jl: best_pool_model,
select_on_grid!) are included after the Pools submodule they read.
include("geometry/transforms/matrices.jl") # HomographyMat (dense algebra) + AbstractMatrixMap
include("geometry/transforms/euclidean_map.jl") # EuclideanMap for poses
include("geometry/transforms/similarity_map.jl") # SimilarityMap (x ↦ s·R·x + t)
include("geometry/transforms/lie.jl") # SO(3)/SE(3) exp/log
include("geometry/transforms/rotation_fits.jl") # Kabsch/Wahba, chordal meanWhy before Primitives? matrices.jl defines HomographyMat and the
AbstractMatrixMap parent; EuclideanMap is used by Primitives (3D conics)
and Cameras, and SimilarityMap reuses its rotation checks. Their actions on
primitives come after Primitives (ct_contravariant.jl).
include("geometry/primitives/primitives.jl")
using .PrimitivesWhy here? Primitives uses the conic matrix types from step 1 and
EuclideanMap from step 3; the remaining transforms need Ellipse from it.
include("geometry/transforms/conversions.jl") # to_homogeneous, affine_part(::StaticMatrix)
include("geometry/transforms/ct_contravariant.jl") # CT maps callable on Circle/Ellipse/Line2D
include("geometry/transforms/coord_maps.jl") # canonical-frame AffineMap constructors
include("geometry/transforms/region_maps.jl") # canonical_map, logpolar_map
include("geometry/transforms/homography_map.jl") # HomographyMap (points/lines/conics)
include("geometry/transforms/propagate_framed.jl") # frame-labeled propagate over framed maps
include("geometry/transforms/bounded_map.jl") # BoundedMap (output extent decorator)include("geometry/solvers/solvers.jl") # forward declarations for backend-provided solvers
include("geometry/solvers/p3p_native.jl") # native P3P (PoseLib port)
include("geometry/solvers/essential_5pt_native.jl") # native 5-pt essential
include("geometry/solvers/homography_solvers.jl") # homography_4pt, DLT, Hartley frames
include("geometry/solvers/homography_jacobian.jl") # analytic Jacobian of homography_4pt
include("geometry/solvers/fundmat.jl") # F/E matrices, 7-pt, motion()
include("geometry/solvers/tls_line.jl") # the total-least-squares line (moved up from RVG, taxonomy v2)
include("geometry/radial_models.jl") # AbstractRadialModel — the plane→pencil family (BEFORE the 5-pt radial solver)
include("geometry/solvers/homography_5pt_radial.jl") # 5-pt one-sided radial homography
include("geometry/forms.jl") # slot-labeled epipolar forms; frame changes are ∘ (EpipolarMap, TransferForm ↔ HomographyMap)
include("geometry/manifolds/subset.jl") # SubsetManifold — restrict() (before the instances, which extend it)
include("geometry/manifolds/essential.jl") # EssentialManifold5/6
include("geometry/manifolds/fundmat.jl") # FundMatManifold7
include("geometry/manifolds/homography.jl") # HomographyManifold8 + sl(3) basis
include("geometry/manifolds/rigid.jl") # RigidManifold6, LeftRigidManifold6, RigidExpManifold6
include("geometry/manifolds/sphere.jl") # SphereManifold (a model up to scale: RadialCamera)
include("geometry/manifolds/product.jl") # EuclideanManifold (the ambient chart), ProductManifold
include("geometry/manifolds/line.jl") # Line2DManifold
include("geometry/manifolds/covariance.jl") # implicit_covariance: the estimate's covariance (chart, blocks)
include("geometry/confidence_regions.jl") # confidence ellipses, epipolar bands
include("geometry/model_validity.jl") # test_sample/test_model/test_consensus
include("geometry/solvers/solver_layer.jl") # the MinimalSolver protocol + registry (the normalizations: homography_solvers.jl)The three covariance verbs (GTSAM's vocabulary in parentheses): propagate
through a map (a function's Jacobian, H·Σ·Hᵀ), transform_covariance
between charts of one model — the ambient space is the chart
EuclideanManifold(ch) (TransformCovariance) — and implicit_covariance
of an estimate: a minimal solve, or the whitened least squares on n ≥ minimal
(Marginals).
include("geometry/transforms/warp.jl") # image_warp adapter + KernelAbstractions warp kernels
include("geometry/transforms/pointwise.jl") # pointwise!: one broadcast driver, every backend
include("numerics/bspline.jl") # the scalar Cox–de Boor basis, B-spline surfaces (grid camera models)
include("numerics/special.jl") # device-safe lgamma/digamma/trigamma, normal CDF
include("numerics/quadrature.jl") # Gauss–Hermite (probabilists') and Hammersley points
include("numerics/small_lu.jl") # in-register partial-pivot LU for fixed-size systems
include("numerics/ka_reduce.jl") # KernelAbstractions workgroup tree reduction
include("numerics/thin_plate_spline.jl") # the thin-plate spline: host-side solver, isbits evaluators
include("base/raster.jl") # array plotting without Makie (masks, debug frames)
include("geometry/primitives/bbox.jl") # shape → SpatialIndexing.Rect (needs frames + dilate)
include("geometry/spatial_queries.jl") # SpatialQueries submodule: allocation-free R-tree querieswarp.jl defines the single adapter image_warp(math, output_rect) that wraps
a math-space Transformation into a BoundedMap callable from Julia
(row, col) indices. VGC math is always COLMAP; the array-side conventions
(_RC_SWAP + COLMAP↔Julia Translation(±0.5)) are baked in with zero kwargs.
include("geometry/cameras/cameras.jl") # models, lenses, imaging chain, calibration, rigs, stereo
using .Cameras
include("raw/raw_mosaic.jl") # the sensor image: RawMosaic, the one phase rule, channelview(::RawMosaic), bayer_plane_map(m, k)
include("raw/linear_raw.jl") # LinearRaw: the one linearisation over the mosaic's photosites in one KA kernel; the cell images' maps
include("synthetic/synthetic.jl") # evaluation metrics (pose_error, imc_pose_error, mean_average_accuracy)
include("pools/pools.jl") # Pools submodule: PoolPair (the pool record, NormalizedImageFrame) + StereoRawPair
include("visualization_stubs.jl") # `function f end` stubs the Makie/GLMakie extensions fill
include("quality/naming.jl") # NamingViolation + the check_naming/naming_scope stubs (the linter: ext/VisualGeometryToolkitCoreTestExt, loads with Test)
include("quality/test_helpers.jl") # stubs: rows_cost, check_rows_jacobian, test_naming (TestExt); test_quality_gates (QualityExt: Test + Aqua + ExplicitImports + JET)
include("quality/tag_registry.jl") # snake_case, TagRegistry (the one TOML tag → type map) and its source/config scans
include("quality/stack_contracts.jl") # the dependency allow-list (configs/stack_dependencies.toml), private-name scan, CONTRACT_GENERICS
include("quality/agent_docs.jl") # agent_doc_violations: what CLAUDE.md, AGENTS.md and .claude/ may not carry
include("quality/conventions.jl") # conventions_dir, convention_files: the one copy of the stack conventionsWhy last? The sensor image (raw/, moved from VisualGeometryToolkitFeatures in 0.9.2)
follows Cameras for pixel_centre, the one index → COLMAP conversion its photosite
offsets read, and the warp section for ImageCore's channelview. Cameras uses the solvers (P3P, radial calibration), the batched
maps (lens kernels), and the B-spline numerics (grid models). check_naming
depends on nothing in the package; it sits last because nothing in the package
calls it — every stack package's quality test does. stack_contracts.jl is
the very last file: CONTRACT_GENERICS lists functions from every section above.
- Primitives: self-contained geometric primitives
- Cameras: camera models, lenses, projection, calibration, rigs
- Synthetic: evaluation metrics on synthetic data
- Pools: the benchmark pool record and the raw-pair interface (its
accessor names stay out of
using VisualGeometryToolkitCore)
- Transforms (
geometry/transforms/*.jl): NOT a submodule because Primitives needs the matrix types (loaded before it) andHomographyMapneedsEllipse(loaded after it). - Robust (
robust/*.jl), the solvers (geometry/solvers/*.jl) and the rest ofgeometry/outside the submodules (forms, radial models, manifolds, confidence regions, model validity): contracts and kernels consumed by the Cameras submodule and by downstream packages.
| Function | Defined in | Implemented by |
|---|---|---|
centroid |
base/interface_functions.jl |
Primitives (Ellipse, Circle, Rect, Disc) |
project |
base/interface_functions.jl |
Cameras (project(camera, X)) |
signed_distance |
base/interface_functions.jl |
Primitives (signed_distance(plane, p); distance is the unsigned generic, the hyperplane method deprecated) |
closest_point |
base/interface_functions.jl |
Primitives (closest_point(plane, p)) |
dilate |
base/interface_functions.jl |
Primitives (Circle, Ellipse) |
intersects |
base/interface_functions.jl |
Primitives (Rect, Circle) |
normal |
base/interface_functions.jl |
Primitives (Hyperplane, Ellipse3D) |
boundary_point |
base/interface_functions.jl |
Primitives (Ellipse, Ellipse3D) |
AffineMap(target) |
coord_maps.jl |
Rect, Ellipse, Circle, FeatureFrames — canonical [-1,1]² → target frame |
EuclideanMap(PixelFrame{A}, PixelFrame{B}) |
coord_maps.jl |
Image-origin conversion (:opencv/:colmap/:julia) — a labeled, unitless rigid map (Translation(::Symbol, ::Symbol) was removed in 0.7) |
coeffs |
Primitives (feature_frames.jl) |
Paraboloid3D{T,P} → SVector{P,T} |
Compile-time dispatch via trait types defined in base/traits.jl (loaded by
base/interface.jl), plus FrameDependence in base/image_frames.jl:
| Trait | Values | Used by |
|---|---|---|
ConicTrait |
CircleTrait, EllipseTrait, ParabolaTrait, HyperbolaTrait, DegenerateTrait |
classify_conic (Primitives) |
HasBoundaryTrait |
HasBoundary, NoBoundary |
Primitives (boundary-sampled overlap_area fallback) |
OverlapMethod |
AnalyticOverlap, BoundarySampled |
overlap_method / overlap_area (Primitives) |
BoundsTrait |
HasBounds, NoBounds |
extent on BoundedCamera, BoundedMap |
FrameScaleTrait |
IsotropicScale, ShapeFactorScale |
FeatureFrame accessors (Primitives feature_frames.jl) |
AbstractSolverBackend |
NativeSolver, PoseLibSolver |
MinimalSolver verb layer, positional solver kernels |
FrameDependence |
GeometryDefined, ModelDefined |
frame_transition, sphere sections (Cameras) |
The transform class (Euclidean ⊂ similarity ⊂ affine ⊂ projective) is not a
trait: the map types carry it — EuclideanMap, SimilarityMap (and the
rotation and uniform-scaling LinearMaps through it), CoordinateTransformations'
affines, HomographyMap — and each acts on a primitive in its class
(ct_contravariant.jl, homography_map.jl). A HomographyMap is always
projective: a circle maps to a Conic. Matrices (to_homogeneous(map), a
plain SMatrix) are algebra only. The affine class is stated once, as
_AffinePlanarMap (homography_map.jl): CoordinateTransformations' affines and
_StructuredAffineMap, the stack's own affine-class types, each with an
AffineMap(m); the homography lift, the ∘ bridges and fuse read it.
# In Primitives (feature_frames.jl)
# Dispatch on FrameScaleTrait: one accessor, one method per scale kind
characteristic_scale(f::FeatureFrame) = _characteristic_scale(frame_scale(f), f)
_characteristic_scale(::ShapeFactorScale, f) = sqrt(abs(det(shape_factor(f))))
_characteristic_scale(::IsotropicScale, f) = f.σA single function image_warp(math, output_rect) wraps a math-space
Transformation into a BoundedMap callable from Julia (row, col) indices.
The composition is FIXED (no kwargs) — VGC math is always COLMAP, the array
boundaries are always Julia, and image_warp always bakes the same bridge:
_RC_SWAP ∘ Translation(+0.5) ∘ math ∘ Translation(-0.5) ∘ _RC_SWAP
Math construction is the caller's responsibility — the two-step pattern is:
math = canonical_map(geometry, 64; bound=1.0) # COLMAP-math patch transform
bm = image_warp(math, 64) # array-callable BoundedMapLog-polar patches use the analogous chain via logpolar_map(geom, n, inner_radius).
Any other math chain works the same way — a K-based homography just
becomes image_warp(H, output_rect).
Both canonical_map and logpolar_map also accept an explicit
HyperRectangle{2} as the patch domain (for non-square or non-standard
extents); the size-form is sugar over the rect-form for the common case.
| File | Purpose |
|---|---|
src/VisualGeometryToolkitCore.jl |
Entry point, exports, include ordering |
src/base/interface.jl |
Loads traits.jl + interface_functions.jl; camera/detection interface stubs |
src/base/traits.jl |
Holy Traits (ConicTrait, HasBoundaryTrait, OverlapMethod, BoundsTrait, FrameScaleTrait, AbstractSolverBackend) |
src/base/types.jl |
Loads interface.jl, defines AbstractDetection, Uncertain, conic matrices |
src/base/units.jl |
unit constants, the stack's four units registered into FlexUnits' registry, parse_unit/unit_string |
src/base/size2.jl |
Size2 type |
src/base/feature_coords.jl |
_strip_coords: pixels in px, lengths in mm |
src/geometry/transforms/matrices.jl |
HomographyMat (dense algebraic homography) and AbstractMatrixMap |
src/geometry/transforms/euclidean_map.jl |
EuclideanMap for camera poses |
src/geometry/transforms/similarity_map.jl |
SimilarityMap (x ↦ s·R·x + t), exact under ∘ and inv |
src/geometry/transforms/homography_map.jl |
HomographyMap for point/line/conic transforms |
src/geometry/transforms/warp.jl |
image_warp adapter + image_warp_batch_to! batched primitive |
src/base/correspondence_set.jl |
Correspondence containers (Correspondences, Correspondence, Attributed) |
src/base/fit_result.jl |
FitResult{P,M}, the iterative-fit return type |
src/geometry/primitives/primitives.jl |
Primitives submodule entry |
src/robust/losses.jl |
Robust-loss family (AbstractLossShape) |
src/geometry/cameras/cameras.jl |
Cameras submodule entry |
-
Most models are a
Lens(projection, distortion)of existing parts or aLiftLens(radial)— no new type (see the Lens Families guide). -
A new projection or distortion term is a LEAF (
cameras/lenses.jl): a struct of numbers and tuples of numbers with its number type as the last type parameter — its coefficients, their names, its zero, its record tag and payload all follow from the fields (add it to_LEAVESinlens_chart.jlto read it back) — and its map: a term's_distort(or_deltawithisdelta), a projection's_project_ray/_lift_point. A lens of a new KIND implements the contract on rays directly:project(lens::MyLens, X::StaticVector{3}) -> Vec2 # ray → image point backproject(lens::MyLens, xd::StaticVector{2}) -> Vec3 # image point → unit ray
NaN where the lens does not image, generic in the number type.
-
Everything else is derived, with no per-lens methods: the perspective views (
distort_normalized,undistort_normalized, the functor,InverseLens),transform_deriv/transform_deriv_params/project_with_jacobians,projected_radius/view_angle,unproject,imaging_transform(affine_part(K) ∘ lens), the batches (pointwise!) and warps,RayMap, the framed stages, and — from the leaves —parameter_vector/with_parameters/ names / bounds (a leaf with a bounded coefficient adds its_bound_pairsmethod), the record, the manifold and_precision.
- Define struct in
primitives/directory - Implement the accessor interface (
centroid,area,boundary_point) andhas_boundary(orframe_scalefor a feature frame) where applicable — traits only when a dispatch site consumes them - Implement interface functions:
centroid,boundary_point, etc. - Add to exports in
primitives.jl
A transform class is a MAP type (a CoordinateTransformations.Transformation,
<: AbstractMatrixMap when it has a matrix form), never a matrix type:
- Store the class's parameters, with
Src/Dstframe tags asEuclideanMapandSimilarityMapdo (relabel/unlabel,invswapping them,∘checking the junction with_check_junction). - Give it its exact
∘andinvwithin the class,to_homogeneous(a plainSMatrix),transform_deriv, and aHomographyMaplift storing its analytic inverse. - Add actions on primitives only where the result's type changes
(
ct_contravariant.jl); everything else goes through the coarser class.