VisualGeometryToolkitEval benchmarks robust estimators: it summarizes pose errors (finite medians, the mean average accuracy (mAA), bootstrap intervals), tunes hyperparameters by a cross-validation configured in TOML, and evaluates a RANSAC scorer composed with a local optimizer. It is the evaluation package of the VisualGeometryToolkit stack, shared by the stack's research projects.
pkg> add VisualGeometryToolkitEvalThe package requires Julia 1.13 or later.
The example summarizes the pose errors of seven image pairs, then tunes one threshold by the package's coarse-to-fine cross-validation.
using VisualGeometryToolkitEval.Stats: finite_median, eligible_maa, bootstrap_ci
using VisualGeometryToolkitEval.Tuning: tuning_spec, grid_spec
using VisualGeometryToolkitEval.Evaluation: coarse_fine_cv
using Random: Xoshiro
using TOML
# Pose errors in degrees of seven image pairs; NaN marks a pair without a model.
errors = [0.4, 1.2, NaN, 0.8, 3.5, 0.6, 2.1]
finite_median(errors) # 1.0
eligible_maa(errors; max_error = 10.0) # ≈ 0.73: mAA up to 10 degrees; NaN counts as 0
bootstrap_ci(finite_median, Xoshiro(1), errors) # 95% BCa interval of the median
# A tuning specification, as a paper's TOML config states it. The keys left out
# take their defaults from the package's configs/tuning.toml.
config = TOML.parse("""
[tuning.threshold]
type = "coarse_fine"
coarse = { type = "spaced", window = { type = "fixed", lo = 0.1, hi = 10.0 }, spacing = { type = "log_per_decade", per_decade = 4 } }
cross_validation = { scene_split = { type = "registered", name = "phototourism" } }
""")
spec = tuning_spec(config, "threshold")
# A stand-in for the validation errors at threshold x; they are smallest at x = 2.
validation_errors(x) = [abs(log(x / 2)) + 0.1 * k for k in 1:5]
result = coarse_fine_cv(validation_errors, grid_spec(spec.scheme.coarse), spec.scheme;
method_name = :threshold)
result[:x_star_fine] # ≈ 2.0, the tuned threshold
result[:boundary_hit] # false: the optimum is inside the windowGetting started continues from here.
- Getting started: the first examples.
- Guide:
the experimental protocol, the
[tuning]schema, the composition evaluator, reproducible figures, paper projects, and the defaults and file formats. - API Reference: one page per submodule.
The evaluator is written against the contracts of VisualGeometryToolkitCore, so it runs any scorer with any local optimizer. It builds on VisualGeometryToolkitCore, VisualGeometryToolkitIO and VisualGeometryToolkitEstimation; a project that evaluates particular scorers or optimizers depends on their packages itself. The portal links every package of the toolkit.
VisualGeometryToolkitEval is part of VisualGeometryToolkit. Cite the toolkit. Its CITATION.cff is the source of this entry, and GitHub's "Cite this repository" button exports it:
@software{pritts2026vgtk,
author = {Pritts, James and Sauer, Hendrik},
title = {{VisualGeometryToolkit.jl}: Julia Packages for Multiple-View Geometry},
url = {https://github.com/VisualGeometryToolkit/VisualGeometryToolkit.jl},
doi = {10.5281/zenodo.23548372},
year = {2026}
}Cite also the papers behind the methods you use; the portal lists them.
MIT