Skip to content

Latest commit

 

History

History
872 lines (784 loc) · 76.1 KB

File metadata and controls

872 lines (784 loc) · 76.1 KB

VisualGeometryToolkitFeatures changelog

0.9.11

A patch on VisualGeometryToolkitCore 0.13.1 ([compat] VisualGeometryToolkitCore "0.13.1").

  • ONNX Runtime's CUDA build lives in the artifact store (Core's install_built_artifact: the store VGTK_ARTIFACT_STORE, the site profile's, the preference's), as the LoMa weights do, never a cluster's depot: the same conda-forge files and the same pinned tree, built once per store, under the store's per-tree lock, so the jobs of one machine that probe CUDA together build it once. A machine whose depot holds the tree moves it with Core's migrate_artifacts (or builds it again into the store); where no store can be had (the depot refused, none named), a depot's own tree is still read, with a warning, so the CUDA path does not fall to the CPU for want of a store. The CUDA 12 runtime and cuDNN it loads stay CUDA_Runtime_jll's and CUDNN_jll's own lazy artifacts, which Pkg installs into the depot like every JLL's.

0.9.10

A patch on VisualGeometryToolkitCore 0.13.0 ([compat] VisualGeometryToolkitCore "0.13").

  • The LoMa ONNX weights live in the artifact store (Core's store_file: <store>/files/<sha256>/<file>, the store VGTK_ARTIFACT_STORE, the site profile's, the preference's), never a cluster's depot; with the depot store they stay in this package's scratch space, as before. Same files, same sha256 check.
  • The verified download of the ONNX Runtime CUDA packages is Core's download_verified (this package's copy is removed); the Downloads dependency is dropped.
  • Tests: the tilted detection's scale-space hash pins are kept per CPU family, a (features, covariances) pair each with the machines that reproduce it (pair 1 the machine of the factoring, the cluster's poseidon and hephaestus; pair 2 the cluster's nereus and GitHub's hosted runners); a machine that matches neither prints its two hashes. The detector is unchanged.

0.9.9

A patch on VisualGeometryToolkitCore 0.12.0 ([compat] VisualGeometryToolkitCore "0.12"), whose radiometric types moved to VisualRadiometryToolkitCore. No code of this package changes.

0.9.8

A patch, additive: the Förstner F-test's critical values computed once per configuration.

  • FoerstnerTests() and detect_foerstner(…; f_tests) (seeded and image-wide, every input): a table of the F-test's critical values per window half-width ρ, keyed by everything they depend on (the gradient taps, ρ, the pixel noise's structure, significance_level), each computed on its first use and read by every later call. The caller makes one per detect configuration (a set of photographs). Without it (the default, nothing) every call computes them again, as before; the rows are bit-identical either way.
  • Why: the values are dense host linear algebra (_residual_dof; on a mosaic's bilinear green image eight lattice phases and centre parities per ρ) and a seeded call on a chrome photograph has one ρ per distinct seed scale, so on the GPU they were nearly all of a RAW chrome photograph's detect time (RC's chrome detect, Sony FX3 fisheye 30 cm set, one B200: the seeded call 5.9–9.0 s of 6.3–9.4 s per photograph, the GPU mostly idle).

0.9.7

A patch: on VisualGeometryToolkitCore 0.11 and VisualGeometryToolkitEstimation 0.12 ([compat] "0.11", "0.12"). Features' refinement runs Core's device LM (static_lm), whose success rule Core 0.11.0 keeps (stationary and accepted), and it reads no host-LM verdict; no behaviour or API changes.

0.9.6

A patch for SFOP's wrong detections on the RAW chrome photographs: a polarity gate, the sensor model's noise in SFOP's λ₂ test, and the detectors on a RAW mosaic at the raw resolution (the bilinear green image).

  • polarity = :none | :dark | :light on detect_sfop and the seeded detect_foerstner(img, seeds), additive, ASIFT's vocabulary and sign (one sign for the package, polarity.jl; ASIFT's parse now shared). The structure tensor is sign-invariant, so SFOP took dark spots for circular points: on Features 0.9.4's RAW green cells, 111 dark-polarity :circular rows over the 668 photographs of the five FX3 chrome sets (99 tamron chrome-siemens, 11 fisheye siemens, 1 fisheye april). The sign is the scale-normalised Laplacian of Gaussian at the detection's σ, positive at a dark blob, tested ONE-SIDED at significance_level under the pixel noise: a candidate is kept only on significant evidence of the requested polarity, so a window whose sign the noise could have given (on flat background between dots) is kept for neither (a missed blob, never a wrong one). SFOP gates before its NMS (ASIFT's order). The seeded call needs the pixel noise for the test (noise, or a gain-carrying sensor_model); polarity on the image-wide search is refused. :none is the default, and there every output is 0.9.5's.
  • Duplicates are not removed. One blob seen from two seeds (SFOP finds a dot at two scales more than a half-octave band apart, which the NMS keeps both of) gives two rows; the consumer resolves them by its own geometry (an assignment to the target's points).
  • detect_sfop(L::LinearRaw; sensor_model): under a gain-carrying sensor model (L linearised with it) the λ₂ significance test reads each pixel's variance (sampled on each octave's grid at the pyramid's own pixel map, the floor scaled by the variance under the window's Gaussian) instead of one noise_sigma, so a bright region's shot noise raises no blob. Without a gain, noise_sigma as before.
  • Recall against 0.9.5 (detect_sfop(L; polarity = :light), then the seeded detect_foerstner, green cells): on 295 photographs of the five sets no 0.9.5 :circular row that is significantly light is lost. Of 82 935 rows, 82 826 remain (none gained). Each of the 95 not found again within 0.5 px has a significantly dark seed (68) or one of a sign not significant (27); the other 14 are one row of a pair within 1 px whose seed is dark or not significant, its twin kept. Every pair within 1 px of two significantly light seeds keeps both rows (92 of 92). An opt-in real-data item holds this on 12 of the photographs (test/data/sfop_recall_095.tsv, 0.9.5's rows).
  • detect_sfop(m::RawMosaic; sensor_model) and detect_foerstner([T], m::RawMosaic, seeds; sensor_model), additive: the detectors on a RAW mosaic's green photosites at the RAW resolution, positions and covariances in raw COLMAP pixels. The green cells (LinearRaw{:green_cells}, unchanged) decimate the green quincunx to the cell lattice and alias its diagonal detail: on DSC06837 (fisheye chrome-april) they register 187 of 289 dots. The image is the bilinear green on the raw grid, the normalized convolution of the usable green photosites with the cross kernel — a green photosite its own value, a red or blue one the mean of its usable green 4-neighbours — one KernelAbstractions kernel, Float64 sums, its variance Σw²σ²/(Σw)² per pixel. A SATURATED photosite is censored: a normal sample at the white level (1), its variance the noise law's there (VisualGeometryToolkitCore 0.10.7's value_variance, the new compat floor), flagged. A DEFECT of the sensor model is missing: it drops out, and a pixel with no usable green under its cross (a defective green) is the mean of the usable greens of its 3×3 neighbourhood (its diagonal greens); only a pixel with none in the 3×3 (a defect cluster) has no value, never filled, and every window reading it fails. SFOP runs on it with its 2× upsampled first octave at input_blur = 1/√3 raw px (the photosite's box, variance 1/12, plus the interpolation's, ¼ on average), its λ₂ and polarity tests under the photosites' noise (the white-equivalent 2σ² per pixel: one green photosite per two raw pixels). The seeded Förstner estimates on the raw-grid image with its own doubling (ip_fop.m step 1) and propagates the noise from the PHOTOSITES, J_k = Σ_q J[:, q] W[q, k] through the bilinear weights, and its F-test's effective degrees of freedom are those of that noise (four separable terms of the interpolation; about 15 % below white input pixels'), each checked by Monte Carlo. On synthetic dots (radius 1.5 raw px) on a Bayer mosaic: NEES 1.93, 96 % coverage, spread 0.014 raw px against the green cells' 0.021, bias alike; a dot with a saturated core is found and localised; a defective green at a dot's centre costs it ≈ 0.06 px RMS (the fallback reads the diagonal greens). Pixel-valued keywords are the image's pixels: on the raw grid, twice the cells' values for the same physical scale (RC 0.10.2 states derivative_sigma = 2.0, precision_threshold = 1.0). The image-wide search on a mosaic is refused (seed it with detect_sfop(m)), and so are input_blur, upsample and seed_sigma (the image's own). Known limit: G1 and G2 are taken to respond alike (the sensor model carries no per-channel response): a G1/G2 gain mismatch stays in the image as a checkerboard of its amplitude (measure it with flat fields, the characterisation's E5, if it matters). Data (RC 0.10.2's detect + Hungarian index, FX3 chrome sets, each frame under its sensor model; green cells → raw grid): DSC06837 (fisheye chrome-april) 187 → 283 of 289 dots assigned, 0 mislabels; every RC-registered dot of DSC06825 (166 with a saturated photosite near the core) and DSC06837 found again; per set, assigned dots (dustbin): fisheye_10cm_ag 41 943 (1 433) → 44 284 (5 553), fisheye_2710_si 26 457 (2 608; 21 photographs refused) → 33 283 (4 817; none refused), tamron_17cm_si 32 368 (691) → 32 368 (1 416), tamron_20261108_ag 40 187 (107) → 40 999 (6 816), tamron_largeaperture 54 908 (778) → 54 910 (1 459), 14mm_central30cm 39 993 (263) → 40 744 (3 692); no mislabel refusal in either. The dustbin (detections left unassigned, the consumer's to drop) grows with the raw grid's finer windows.
  • Investigated, not changed: the three dim top-rim dots on DSC06962 with no SFOP candidate are not dropped by the significance test or the NMS: no candidate exists before the NMS. The squashed rim dots (≈ 4 × 1.5 cells) have a precision still rising at the finest searched level (w = 69, 34, 19 … over levels 1, 2, 3 of the upsampled octave), so there is no maximum in scale. Treating the first level as the scale range's bound recovers them (and adds 21 pool candidates image-wide); not shipped, since a missed blob is preferred to a wrong one.

0.9.5

A patch: RAW images stored as Float16 or Float32 (VisualGeometryToolkitCore 0.10.5, now the compat floor), and a known issue of detect_image_circle under a sensor model, documented.

  • load_raw([T], path; …) and load_raw([T], mosaic; …), additive: the values' element type T (a Base.IEEEFloat), positional and leading (the stack's signature rule), forwarded to Core 0.10.5's LinearRaw(T, mosaic; mode, gain, sensor_model). Without it the type is Core's default (Float64, bit for bit 0.9.4's values), read off Core's public linearview (for every mode). The display demosaic is stored as T too (computed in Float64, rounded once; a gain and white balance that could overflow T refused).

  • load_raw's cache key names the element type (any but Float64, whose entries keep the plain key: every entry before 0.9.5 is Float64), resolved from Core's default first: a cached Float64 decode is never served for a Float16 request, nor for a default request after Core's default changes.

  • No reduction over a RAW's values accumulates in Float16 (the audit, below). Two fixes:

    • box_downsample of a Float16 image (or Gray{Float16}, …) summed its blocks in Float16; it now sums and scales in Float64 and rounds each output once to Float16 (a block of 2048 and four 0.25s: 2049/9, not 2048/9).
    • detect_foerstner(Float16, …) would have accumulated the squared gradient's window sums in Float16; a precision other than Float32 or Float64 is refused (ArgumentError). A Float16 image is read in T.

    Audited, unchanged (each reads a Float16 image widened): SFOP and ASIFT (the pyramid's Float32 input, _asift_input); the seeded and image-wide Förstner (ka_upload(T, …), the variances Float32 → T); variance_stabilize (the Gaussian part and its mean in Float64); detect_image_circle (the frames' maximum in Float32, the bin medians order statistics, the bins' mean variance summed in Float64, profiles and drops in Float64); gaussian_blur (Float64).

  • Known issue, not fixed here: detect_image_circle under a sensor model can take the halo or barrel-reflection light beyond the rim for the rim (since 0.9.2; the default path, without a sensor model, is unaffected). With Uncertain cells (a RAW read with a Poisson–Gaussian model) each bin's noise is the model's alone, the sensor's photon and read noise. Past the coverage the FX3's fisheye has faint light (the lens barrel's reflections, a halo, 0.2–1 % of white) whose drops lie far above that noise (8–14 % of them beyond 4σ), so each angle's outermost edge, the first fit's, lies out there: a = 1328.6, 1366.2 and 1481.8 raw px on the f/4, f/22 and f/32 sets (the f/32 centre 97 px low), against 1086.5, 1340.4 and 1333.8 without a model, in Float64 and Float16 storage alike (the 14 mm diffuser set agrees). Measure the coverage without a sensor model (the default path is unaffected) until the rule that picks the coverage under a model is fixed in its own patch. The opt-in real-data item holds the three sets as @test_broken.

0.9.4

A patch: detect_foerstner's F-test at the effective degrees of freedom (a label change: below), RAW inputs for detect_sfop and the seeded detect_foerstner, and the image-circle deprecation text.

  • detect_foerstner's F-test at the residual sums' effective degrees of freedom (difference 4 from ip_fop.m, docs guide). F = Ω_corner / Ω_circular was tested against F(redundancy, redundancy), the window's doubled-pixel count less 2, as if the doubled, derivative-filtered samples were independent; at that count the test's level was not the stated one (on white noise 20 % of the windows labelled circular and 25 % corner at 0.999). It is now tested against F(ν_corner, ν_circular), each ν = (tr Q)²/tr Q² (Satterthwaite) of the residual sum as the quadratic form Iᵀ Q I of the input pixels about the window centre, per window half-width (ν ≈ 87 against 2023 at σ = 6; the estimate's two parameters neglected, a conservative test). Both the image-wide search and the seeded windows. On the THORLABS chrome photographs the SFOP seeds between dots, on line artefacts and on letter strokes (F ≈ 1–3) were labelled circular; they are now unknown. Labels change wherever F lies between the two critical values; positions and covariances do not change. At the image-wide default window (ρ = 6, ν ≈ 12) a noise-free checkerboard X-junction is now unknown at 0.999, a corner at 0.99 (docs guide).
  • detect_sfop(L::LinearRaw) and the seeded detect_foerstner(T, L::LinearRaw, seeds; sensor_model), additive: SFOP on a RAW's green cells (their values, the cell grid) and the seeded windows under the frame's sensor model, its covariance as the image-wide call's on L (bit for bit at the same windows). RC's chrome evaluation detects its RAW sets this way.
  • detect_image_circle's deprecation warning says "deprecated" for the hull fit's keywords (min_hole_area, smoothing_weight, border_margin), so @test_deprecated and a user's --depwarn=yes session recognise it (it read only "is ignored"; tested under --depwarn=yes, red first).
  • detect_sfop keeps a maximum beside a failed window. The extraction's edge test (ASIFT's) was meant to be off in SFOP's precision mode, by an edge threshold of floatmax; it was not: next to a −∞ neighbour (a window that failed the λ₂ test or reached the border) the curvature is −∞ and tr²/det is NaN (or det < 0 across a diagonal neighbour), so the maximum was dropped. The edge test is now skipped in the precision mode: such maxima are detections.
  • detect_sfop's precision is each detection's own. Adjacent octaves share a level, so two octaves can find a peak at one position and scale with different w; the table of w was keyed by position and scale alone, and the row the NMS kept could report the other octave's w. It is keyed by the octave too.
  • detect_sfop's nb_octaves default is min(3, sift_nb_octaves(size(img)...; upsample)): an image too small for 3 octaves is searched on the octaves it holds (it was an ArgumentError from gaussian_scale_space); a nb_octaves beyond them, asked for, is still refused.
  • The seeded detect_foerstner(T, img, seeds) takes its own keywords: device, noise (sensor_model on a LinearRaw), derivative, derivative_sigma and significance_level. The image-wide search's (method, noise_sigma, integration, integration_sigma, precision_threshold, roundness_threshold) were accepted and ignored; they are now an ArgumentError naming them. The seeds are typed: detections of a Disc, OrientedDisc, EllipticFrame or AffineFrame, each also uncertain, checked per seed (so Detection[] and a vector of mixed frames are seed vectors); a point seed, which has no window, is an ArgumentError naming its frame (it was a MethodError of characteristic_scale, inside the call). detect_foerstner(L, seeds) takes the default precision, as detect_foerstner(img, seeds).
  • detect_image_circle: the same outputs, faster (bit for bit, checked on every real set, the synthetic field stop and a flat image against 0.9.3's code): the bin median of a bin of more than 32 samples by a radix selection (O(m) sweeps, no scratch, every backend) instead of the O(m²) rank count, the same two order statistics bit for bit; the saturation mask's warp skipped when no pixel is saturated (its samples are all 0); the empirical noise's drop noise at unit pixel noise computed once per bin row count, not once per angle.
  • detect_image_circle's docs say what the code does (no code change): a bin more than half saturated is FLAGGED (:saturated in rim.bin_reason) and keeps its median in the profile, read like any other; 0.9.2's and 0.9.3's docs and this CHANGELOG's 0.9.2 entry said such bins were skipped. Also stated: a column's pixel count has no floor of one pixel; the :auto ladder's fallback (the finest binning tried); a zero noise used as it is. These are candidates for an owner-reviewed revision (.release/wave11-removals.md).
  • Scheduled for wave 11: gaussian_gradient's derivative::Symbol becomes a type (by dispatch); public since 0.9.1, so unchanged in this patch (.release/wave11-removals.md).

0.9.3

A patch: detect_image_circle's rim radii through the log-polar map's differential and inverse (Core 0.10.3).

Changed

  • detect_image_circle's rim radii are in pixels through the log-polar map's differential and inverse (Core's new transform_deriv(::LogPolarToCartesian, p) and inv(::LogPolarToCartesian); needs that Core release). 0.9.2 measured an edge's drop over "the samples to ≥ 2 px": on the log grid that span's sample count steps where the radial spacing crosses 2/n px, and 0.9.2 corrected it by hand, dividing each log drop by its span's length. Now:
    • the drop at radius r is across exactly 2 px about it, its ends r ± 1 px found through the inverse map and the profile interpolated between columns: continuous in r, no span-length step; the per-span normalisation is gone;
    • the edge is the log slope's sub-sample peak in the grid coordinate, taken to pixels by the forward map;
    • each bin's noise is scaled by its own pixel count, rows × arc from the map's differential (each column is at most a pixel thick): before, the empirical noise was one per angle, so the significance test and the bin rule's edge uncertainties now use each bin's pixels. RimProfile.noise is now each angle's noise PER PIXEL (a bin of m pixels has noise/√m). The return type and every keyword are unchanged. Real-data coverages (raw px) move within their σ on the 14 mm, f/4 and f/22 sets; the start invariance holds (14 mm: sub-pixel and ±25 px starts within 0.25 px); the f/32 set's coverage is now its reference, owner-verified on the rim including the mount side (centre (2136.997, 1439.331), semi-axes 1333.768 and 1315.245 raw px; the earlier reference sat 3.5 px high), checked as every other set is; the synthetic field stop is at 1310.29 (truth 1309.6, a binary disc: the lit pixels' outer boundary is within √2/2 px). Requires VisualGeometryToolkitCore 0.10.3 (its compat floor).

Not noted at release (added in 0.9.4)

  • RimProfile.noise changed meaning, a change for any code that reads it: in 0.9.2 it was each angle's noise scale of its BINS (the model's median bin noise, or the empirical one of the profile's samples); from 0.9.3 it is each angle's noise PER PIXEL σ₁ (a bin of m pixels has σ₁/√m, m its rows × its cells' arc). The field's name and type are unchanged.

0.9.2

A patch, additive (owner rulings 2026-11-14: the lens coverage is the rim's outer edge; no rim margin).

  • detect_image_circle measures the lens COVERAGE, where its light ends, on the rim itself. The method:

    • the frames' per-pixel maximum is resampled on a log-polar grid about the image centre (or a caller's initial model), using VisualGeometryToolkitCore's logpolar_map / image_warp! at ≤ 1 px in radius and in arc;
    • every (angle bin, radius) is reduced to its median by one KernelAbstractions kernel, on device (default default_device()); off-frame and saturated bins are skipped;
    • every edge of each angle's outer envelope (the brightest the profile gets at or beyond each radius: where the light ends it does not come back) is a candidate: a run of drops significant against the angle's own noise at the family-wise level significance, at its steepest log drop per pixel (the log floored at the angle's darkest level), refined to the sub-sample peak; a rim that falls in two steps gives both (owner ruling);
    • the RANSAC ellipse (or circle) is fitted on one candidate per angle, the same scorer as before: each angle's outermost edge (where its light ends), then each angle's edge nearest that model (stray light, or a step off the model, gives way), fitted again: the coverage.

    What changes:

    • No MSER and no hull. On a soft rim (an APS-C lens on the full frame, a ~230 px ramp) the hull fit landed mid-ramp; the coverage is at the ramp's foot.
    • The return type is unchanged: circle, fit (now on the candidates), contour (each angle's chosen edge, the rim's points), cut and rim_gap.
    • Added to the result: covariance (the coverage's, in Core's ambient_coordinates of the measured plane: s²·solver_covariance on the inliers; nothing for a Circle, whose problem provides none), center_scale, axes_scale (the standard deviations, in the result's pixels) and rim::RimProfile (the profiles, the candidates, the per-angle edges and the noise statistics).
    • New keywords: initial, rim_band = (0.8, 0.0), rim_angles = :auto (the binning whose coverage is the most precise, chosen per image), significance = 0.01 (the edge test's family-wise level per angle) and device.
    • The noise statistics follow the frames' TYPE: Uncertain RAW cells (a sensor model with a variance law) give the model's per-bin noise (SensorModelNoise, with a model-vs-data check); plain images give the data's (EmpiricalNoise).
    • New public names: RimProfile, SensorModelNoise, EmpiricalNoise.
  • Deprecated, inert:

    • detect_image_circle's min_hole_area (no longer required), smoothing_weight and border_margin are hull-fit keywords. They are accepted with a deprecation warning and ignored, and are REMOVED in 0.10.0 (wave 11: listed in the monorepo's .release/wave11-removals.md).
    • Every other keyword is live.
  • Removed (internal):

    • the VisualGeometryToolkitFeaturesImageSmoothExt extension and its ImageSmooth weak dependency (the L0 smoothing fed the MSER path);
    • the DelaunayTriangulation dependency (the hull).
  • Not noted at release (added in 0.9.4): inlier_mask(fit) indexes other points. In 0.9.1 fit was the hull's RANSAC and inlier_mask(fit) indexed contour. From 0.9.2 fit is the RANSAC on one candidate per angle that has any, and inlier_mask(fit) indexes THOSE points (in angle order); contour holds the inliers' edges only, so contour[inlier_mask(fit)] is no longer meaningful (and generally a BoundsError or the wrong points).

  • Not noted at release (corrected in 0.9.4): a bin more than half saturated was described above as skipped; it is flagged (:saturated) and keeps its median.

0.9.1 (2026-11-14)

A patch, additive, on VisualGeometryToolkitCore 0.10.

  • New: gaussian_gradient(img, σ[, mask]; derivative = :gaussian2d) -> (gx, gy), exported: an image's gradient at scale σ (∂/∂column, ∂/∂row, per pixel), the derivative of a Gaussian along one axis and the Gaussian across, zero padded — the Förstner operator's own gradient, which detect_foerstner now calls (one implementation; its results are unchanged, bit for bit). One KernelAbstractions kernel, on the image's device (CPU or GPU). With a mask it is the gradient of gaussian_blur's normalized convolution blur(img·mask)/blur(mask): nothing bleeds in from outside the mask, and the gradient is 0 there.
  • New: gaussian_scale_space(img; …) -> GaussianScaleSpace, exported (and level_sigma, public): an image's Gaussian scale space, built by ASIFT's own pyramid (its taps, its fused tiled blur, Lowe's downsample, the 2× upsampled first octave), level for level what asift_detect builds for the untilted warp. ASIFT now holds its levels in one (AsiftPlan.space, replacing the fields gauss, dims, pyr_taps, pyr_r, nb_octaves, nb_scales, o_first, seed_sigma); its detections are unchanged, bit for bit (pinned by a test).
  • AsiftPlan's type parameters and fields (AsiftPlan{D, A2, A3, A4, AI, AU, AU8, GT, KN, SP}, was {D, A1, A2, …, KN}: the blur taps' A1 is gone, the scale space's SP is new; the 8 fields above are now plan.space): not breaking: the moved fields were never documented API; owner ruling 2026-11-14.
  • New: detect_sfop(img; noise_sigma, significance_level, precision_threshold, …), exported: SFOP scale selection (Förstner, Dickscheid & Schindler 2009) for circular points (α = 90°) on that scale space — the precision (N(σ) − 2)·λ₂/Ω per level, its 3-D maxima (ASIFT's extraction), the λ₂ significance test derived from the image noise (no empirical factor), T_w from precision_threshold, ASIFT's elliptical NMS. Rows Detection(Disc(x, σ), (; precision)).
  • New: detect_foerstner(img, seeds): the Förstner window estimation at given windows (each seed's centre and scale σ, the box window of integration_sigma = σ), the same rows as detect_foerstner(img); at the windows detect_foerstner(img) chose it returns its rows bit for bit.
  • Measured limitation, accepted (owner, 2026-11-14): at a dot radius of about 1.8 px the window SFOP selects is 0.0008 px RMS worse than the best fixed window (0.0092 vs 0.0084 px, synthetic discs, σi 1.41). Three window rules were tested and failed (the smallest a-posteriori variance, a window ∝ the radius, Gaussian weighting) and Farid–Simoncelli derivatives had no effect: the limit is the window estimator's pixel locking. Stage 2 keeps the selected window.
  • detect_foerstner is faster (results unchanged, bit for bit): the squared gradient stays on the device and the window centres are compacted there (12 MP photograph on a B200: 587 → 28 ms; on 32 CPU threads 1.2 → 0.42 s).

0.9.0 (2026-11-13)

A breaking release of the wave after RC 0.8.0, on VisualGeometryToolkitCore 0.10 ([compat] "0.10.0").

Breaking

  • detect_foerstner's rows carry covariance_status (their attributes are (; label, window, F, covariance_status)): how the frame's covariance was obtained or why there is none — :propagated, :propagated_measured / :propagated_estimated / :propagated_specified, :a_posteriori, or :unclassified on an :unknown row (a NaN covariance). The producer states its covariance policy; no consumer substitutes one.
  • detect_foerstner(...; noise): the covariance propagated from the image noise. With noise = σn (iid pixel noise, grey levels of an 8-bit image) the covariance is σn² J Jᵀ, J the exact first-order (implicit-function) derivative of the estimate with respect to every input pixel, through the detector's own doubling, derivative taps, squared gradient and window estimate, exact at the image borders; one KernelAbstractions kernel on the CPU and any GPU. Its Monte Carlo calibration (NEES₂ ≈ 2, χ²₂ 95 % coverage) is a test. noise = nothing (the default) keeps the reference's a-posteriori Ω/redundancy · N⁻¹, marked :a_posteriori; it is not calibrated (Förstner study: NEES 0.01–25). noise_sigma, the precision test's level, defaults to noise when that is given (else the reference's 2).
  • The sensor model (VisualGeometryToolkitCore 0.10's SensorModel, owner rulings A and D1–D13):
    • load_raw(...; sensor_model) hands the frame's model to Core's LinearRaw, the one black subtraction (its black, its defects, and under a Poisson–Gaussian or relative noise each value's variance); refused with the display demosaic. A cached decode is keyed by Core's content_digest(model); the absent model keeps the plain key.
    • detect_foerstner(L::LinearRaw; sensor_model): fed only under a gain (Poisson–Gaussian): J diag(var) Jᵀ from the uncertain image's variances; read noise alone, relative noise or none → :a_posteriori.
    • variance_stabilize(L, sensor_model) (was the keywords electrons_per_dn, read_noise, retired with a pointed refusal): the variance plane and Core's poisson_slope; a model without a gain is an ArgumentError naming the missing gain.
  • The continuous refinement's LM stop settings are Core's (StaticLmStops, [lm], the one owner): the ContinuousRefinement keywords and TOML keys max_iter, converged_tol2 and cost_floor are retired (a pointed refusal); success is Core's one rule (the point is stationary).
  • CspondSet is CorrespondenceSet (VisualGeometryToolkitCore 0.10's terminology: Cspond → Correspondence, csponds → correspondences); the old name is refused by Core's naming check, with no alias.
  • A detection's propagate(m, d) is Core's one conversion carried through m, propagate(m, center(d)): the centre is now a Point2 (was an SVector{2}), the covariance the frame's position block as before.
  • A Förstner point's error ellipse is Ellipse(d.value; sigma) (Core 0.10 deleted the detection forwarders; a PointFrame detection has no region, so Ellipse(d) is a MethodError naming why).
  • Terminology (the owner's ruling of 2026-11-13): "keypoint" and "measurement" are out of every name, docstring and page — a keypoint is a detection. Internal names renamed: _keypoint_cov → _detection_cov, _loma_keypoints → _loma_points; the benchmark benchmark/mc/keypoint_uncertainty_mc.jl → benchmark/mc/detection_uncertainty_mc.jl; the test items "keypoint covariance: …" → "detection covariance: …".

New

  • onnx_run(session, inputs) / onnx_run(f, session), public: a run of an onnx_session whose CUDA arena growth is measured around it and charged to the session, so the ASIFT detector's device plans leave exactly the room the arena may still take (was the private _onnx_run, LoMa's runs only). The f form takes any call that runs the session (a device-tensor CAPI.Run), for BlobBoards' descriptor runs.

Fixed

  • extract_capture_settings no longer compiles per frame count (patch 0.8.4's fix, carried): a capture's identities merge field by field (StructArrays' components), not by splatting one tuple per frame into zip (a 171-photo set took 518 s, ~90 % LLVM). The other splats of a data-length collection go too: detect_foerstner collects its rows, _score_eltype takes a collection, compare_images links its axes through Makie's vector method.
  • detect_foerstner(Float64, img) computes in Float64 from the input on: a Float64 image was rounded to Float32 before the upload, so the Float64 path ran on Float32 grey values.
  • The continuous refinement stays inside the level (refinement = ContinuousRefinement()): its LM box is also bounded by the level's extent, the detector's keep test; a point whose extremum lies beyond the level's edge ends LeftBasin with its start, never outside the canvas.
  • A continuous refinement held on its box is LeftBasin, the start returned (the refinement's basin policy): Core's stationarity rule counts a point a bound holds as stationary, so a solve whose extremum lies beyond the displacement bound or the level's edge would end Success at the box, a biased point; any point on the box is now LeftBasin with its start, whatever the LM's verdict (as it was when the box-held solve did not converge).
  • detect_image_circle checks smoothing_weight before it reads a frame, with its other keywords.
  • variance_stabilize reads Core's green_plane(L): it no longer searches the planes for the green one by comparing frame maps with ===.
  • The RAW decoder's fallbacks are Core's require_extension: without the decoder's extension they name the package to load; with it loaded, a call it has no method for raises that MethodError (it said the extension was missing).

Internal

  • The continuous refinement hands its scale to Core's static_lm (its s), solving for the displacement, instead of rescaling the coordinates itself (the owner's no-duplication ruling): the damping floors are the same; the step stop reads the displacement in level units.
  • The detection covariance of a continuously refined point builds the refinement's field taps once per chunk, not per detection. A non-CPU backend builds each ASIFT kernel once (only the CPU has a static form).
  • The Förstner propagation's one-axis pipeline columns are the adjoints ASIFT's covariance already has (_correlate_adj, the zero-padded correlation's, of which the symmetric blur's _blur_adj is now the special case, bit for bit; _up_adj with an optional bounded edge).

0.8.4 (unreleased)

A patch: a compile-time fix, no API change, no result changes.

  • Fixed: extract_capture_settings no longer compiles per capture size. It merged the frames' body and lens identities by splatting one tuple per frame into zip, so each frame count compiled a zip of that arity, its LLVM time super-linear in it: a 171-photo set took 518 s, about 90 % of it compiling (decoding takes ~0.2 s a photo). The identities now merge field by field over DeviceIdentity's three fields, over the distinct identities in frame order (the first stated field kept, a disagreement refused, as before); tested at 500 frames after a warm-up at 300.
  • No other splat of a data-length collection: detect_foerstner collects its detections (was [rows...], one argument per detected point, per image), the stacked descriptor index finds its score type over a collection of galleries (was one argument per gallery), and compare_images links its axes through Makie's vector method.
  • Tests: the matcher items take their reference matrix on the CPU, and pairwise_matrix on the default device is checked against the CPU within its derived rounding (17 Float32 ulps); the default device is a GPU once another item has loaded CUDA in the process.

0.8.3 (unreleased)

A patch, on VisualGeometryToolkitCore 0.9.5 ([compat] VisualGeometryToolkitCore "0.9.5"). Additive; the detector's default is unchanged.

  • Julia 1.13 (the owner's ruling of 2026-11-13: the whole stack requires 1.13): [compat] julia = "1.13" in the package and in every environment of its workspace; CI runs 1.13 only.

  • Every [compat] floor raised to the gated resolution (the owner's ruling of 2026-11-13): a stack dependency at its registry latest or its in-wave version, a third-party one at the version the release gate resolved (a floor only, the semver upper bound kept); the monorepo's Downgrade workflow is deleted.

  • New: ContinuousRefinement, exported, and asift_detect(img; refinement = ContinuousRefinement()) (also asift_detect!): an opt-in second refinement of every detection on a continuous model of its DoG response — the spatial gradient from Farid and Simoncelli's matched prefilter and first-derivative filters (IEEE TIP 2004), the scale row from the cardinal cubic spline of the unfiltered slices, Lagrange in scale — iterated by Core's device LM (static_lm) in a KernelAbstractions kernel on the CPU and any GPU, from the one-step refinement's point. The same candidates; a keypoint whose continuous refinement fails keeps the one-step point. With uncertainty = FeatureCovariance() the refined keypoints' covariances are recomputed at the refined points from the model's own derivatives. Measured on noise-free synthetic blobs (benchmark/continuous_refine): the largest centre bias over a sub-pixel period falls from 0.02–0.14 to 0.0001–0.0005 level pixels, the one-step refinement's half-sample jump is gone, the scale error is the plain spline's (≤ 0.0041 in ln σ at octaves 1–3), and the covariance's NEES₂ is 1.94–2.09. Its bounds and tolerances: configs/continuous_refinement.toml. The default refinement = nothing is the detector of 0.8.2.

  • The one-step refinement's emission and the continuous one's share one map (_level_to_canvas, _slice_sigma): the same arithmetic, the same bits for _extract_at!.

  • The spline window sweep, the interpolation prefilter, the weight derivatives and the positive-definiteness test are Core 0.9.5's (spline_window_sums, cubic_bspline_prefilter_taps, cubic_bspline_weight_derivatives, is_positive_definite); nothing of them is restated here.

0.8.2 (2026-11-13)

A patch: on VisualGeometryToolkitEstimation 0.10 (the weak dependency of detect_image_circle's fit: [compat] VisualGeometryToolkitEstimation "0.10" in the package and test environments). No code change: Features never called the SplineLens refine that Estimation 0.10.0 retired.

0.8.1 (unreleased)

A patch, on VisualGeometryToolkitCore 0.9.3 ([compat] VisualGeometryToolkitCore 0.9.3).

  • detect_image_circle's extension checks are VisualGeometryToolkitCore's require_extension. The private _require_extension is deleted. The message without the extension is unchanged. The hooks' fallbacks (_fit_image_circle, _l0_smooth!) now raise the MethodError of arguments that the loaded extension does not take, where they used to name the package to load even when it was loaded.

0.8.0 (2026-11-12)

Breaking. The sensor image moves to VisualGeometryToolkitCore 0.9.2 ([compat] VisualGeometryToolkitCore 0.9.2), and this package keeps only what needs LibRaw: load_raw_mosaic/LibRawDecoder (producing Core's RawMosaic), load_raw and the display demosaic. The moved definitions are deleted here, and by the owner's ruling (2026-11-12) this package re-exports NO name of Core's — the moved ones and its three older re-exports alike: each is exported by Core alone.

  • Breaking: no longer exported (or declared public) here — use VisualGeometryToolkitCore's (using VisualGeometryToolkitCore, or VgtkCore.X):

    • RawMosaic — use VisualGeometryToolkitCore.RawMosaic;
    • LinearRaw — use VisualGeometryToolkitCore.LinearRaw;
    • BayerCellFrame — use VisualGeometryToolkitCore.BayerCellFrame;
    • raw_frame_map — use VisualGeometryToolkitCore.raw_frame_map;
    • bayer_plane_map — use VisualGeometryToolkitCore.bayer_plane_map;
    • green_cell_sites — use VisualGeometryToolkitCore.green_cell_sites;
    • raw_coordinates — use VisualGeometryToolkitCore.raw_coordinates;
    • raw_provenance — use VisualGeometryToolkitCore.raw_provenance;
    • cfa_string — use VisualGeometryToolkitCore.cfa_string;
    • photosite_colors — use VisualGeometryToolkitCore.photosite_colors;
    • feature_frame — use VisualGeometryToolkitCore.feature_frame (the Detection method is still added here);
    • Cspond — use VisualGeometryToolkitCore.Cspond;
    • csponds — use VisualGeometryToolkitCore.csponds (the NNMatch method is still added here).
  • Breaking: the private _linear is gone: a photosite's linear value is Core's linearview(m)[i, j].

  • New: the display demosaic of a mosaic in memory, load_raw(mosaic::RawMosaic; mode = :demosaic, gain, white_balance): LibRaw's AHD on the mosaic's photosites (libraw_open_bayer, copied to the host), its colour phase and its black and white levels, so a mosaic corrected per photosite is displayed through the one demosaic (no second one). Bit for bit the demosaic of the same buffer opened by LibRaw (a test); a mosaic whose levels are not whole codes (LibRaw holds integers) is an ArgumentError, and LibRaw's own reading of the buffer (phase, levels) is checked against the mosaic's. load_raw(mosaic; mode = :green_cells | :bin2) is Core's LinearRaw(mosaic; mode, gain).

  • variance_stabilize finds the green-cell plane through Core's public frame maps (the plane whose bayer_plane_map is raw_frame_map(L), the cell grid; the display image refused there) in place of the raw module's private accessors.

  • CACHE_FORMAT 4: a cached LinearRaw was stored as VisualGeometryToolkitFeatures.LinearRaw; the type is Core's now, so earlier entries are misses (decoded again), never a load that fails.

  • Adapt is no longer a dependency (it served the mosaic's device form, now Core's). VisualGeometryToolkitCore 0.9.2's dependency allow-list names VisualGeometryToolkitEstimation for this package (the weak dependency below).

  • New: detect_image_circle([rng,] [T = Ellipse,] frames; smoothing_weight = 0, outlier_halfwidth, min_hole_area, border_margin = 5) (exported; moved from RadiometricCamera, the same signature and, on its tests, the same results bit for bit): the image circle of a lens measured from photographs — the frames' per-pixel maximum as N0f8, its dark MSERs (detect_mser, max_area = 1, duplicates kept), every region boundary less the holes smaller than min_hole_area (VisualGeometryToolkitCore's signed_area) and the border's vertices, their convex hull, and a RANSAC Ellipse or Circle through the hull's vertices off the border, scored by the Student-t + uniform marginal at outlier_halfwidth. Returns (; circle, fit, contour, cut, rim_gap) or nothing. A RAW frame (a RawMosaic, or LinearRaw(mosaic; mode = :green_cells)) is measured on its green cells, with the keywords in green-cell pixels (a pipeline config's resolution), and only the result is mapped: its circle and contour come back in the raw's pixels through the cells' raw_frame_map, and fit's estimate with them, while its rms and residuals stay in green-cell pixels. A cell image on a device is read to the host (the result bit for bit the host's, a GPU test).

  • New dependencies: DelaunayTriangulation (the convex hull) and Random. New weak dependencies and extensions: VisualGeometryToolkitEstimation (VisualGeometryToolkitFeaturesEstimationExt: the fit, find_ellipse / find_circle; without it detect_image_circle is an ArgumentError naming the package) and ImageSmooth (VisualGeometryToolkitFeaturesImageSmoothExt: the optional L0 smoothing, smoothing_weight > 0).

0.7.0 (2026-11-12)

  • New: propagate(m, d::Detection{<:Uncertain}): the uncertain centre of a detection through a map (its centre and the position block of its frame covariance, carried by VisualGeometryToolkitCore's propagate; moved from PanoramaStitching).

  • Device buffers are released through VisualGeometryToolkitCore's free_device_buffers! (an evicted ASIFT plan's, the matcher's score blocks), no longer KernelAbstractions' unsafe_free!, which CUDA.jl does not extend: on a GPU they were not released until the collector ran. Now they are, at once.

  • ASIFT with FeatureCovariance drops a detection whose covariance is singular or NaN (the owner's ruling, 2026-11-12), exactly as it drops a failed sub-pixel refinement, before the cross-warp NMS, and counts it in plan.refine_dropped. Every returned keypoint's covariance is finite and positive definite. Without uncertainty nothing changes.

  • Internal: every kernel is instantiated through VisualGeometryToolkitCore's one launch policy, OnBackend(device; workgroupsize)(kernel) (the owner's ruling, 2026-11-12): the ASIFT, cross-warp NMS, matching, Förstner, SSC and RAW cell kernels, each at the workgroup it had (_launch! keeps only the synchronize). Results and timings unchanged.

  • Breaking: pairwise_matrix and nn_match default to default_device() (the stack's GPU-first rule): pairwise_matrix(metric, d1, d2, device = default_device()) and nn_match(…; device = default_device()) run on an accelerator when one is functional, else on KA.CPU() as before. nn_match(…; device = nothing) is a TypeError: give the backend, or leave the keyword out. match stays the host call.

  • Breaking: RawMosaic{A} <: AbstractMatrix{UInt16} is its photosite matrix (the owner's raw-image decision; BayeredImages' idea, no dependency on it). size, indexing, IndexStyle and parent are the held array's (m.raw, any AbstractMatrix{UInt16}, a device array included), so an array or image function reads a mosaic unchanged; what one makes (similar, copy, a broadcast, a reduction) is a plain array of the held array's kind. A mosaic is now == to its photosite matrix, iterates over its photosites and prints one summary line. The type parameters are RawMosaic{A, M} (was RawMosaic{M}). copy, ==, isequal and isapprox are the held array's own (a device mosaic's copy(m) is a device array, its comparison no scalar read). Equality is the photosite matrix's, by design: two mosaics (or windows) with the same photosites and another colour phase, window origin or metadata are == (and hash alike); compare photosite_colors, window_origin and metadata for the rest.

  • load_raw's cache key includes the decoder library's version (libraw_version() for LibRawDecoder, the _decoder_version(decoder) hook; nothing for a decoder that states none), so after a LibRaw_jll upgrade a cached decode (its black levels, white level, crop) is a miss, not the old library's values.

  • New: windows of a mosaic. m[rows, columns] (unit ranges, a copy) and view(m, rows, columns) (no copy) are mosaics of the same sensor: the new field window_origin (the constructor's keyword, (0, 0) as decoded) is the window's offset into the visible area, so its colours are the parent's at those photosites for either parity of its start, and raw_frame_map(m::RawMosaic) (new) is that offset. A LinearRaw made from a window maps onto the parent's photosites exactly (its raw_frame_map and bayer_plane_map include the offset).

  • New: raw_coordinates(img, x), THE accessor of an image's sensor-pixel coordinates: raw_frame_map(img)(x), in the visible raw's COLMAP frame — the identity on a decoded mosaic, x ↦ 2x on the cell grid of a LinearRaw; the display demosaic is refused, a plain matrix has no method. A detection on the green cells reports through it (the half-scale bug of writing cell positions as raw pixels). The design note: the guide's "Raw images" page.

  • New: a mosaic on the device. adapt(CuArray, m) or adapt(backend, m) (Adapt, a new dependency; a KernelAbstractions backend) move the photosites and keep the mosaic; LinearRaw(m; mode) runs its cell kernel on parent(m)'s device and leaves the image and the saturation flags there. Breaking: LinearRaw{Mode, I, S, G, M} holds image::AbstractArray{Float64, 3} and saturated::AbstractMatrix{Bool} (an Array and a BitMatrix on the host, as before); CACHE_FORMAT is 3, so decodes cached under the earlier layout are a miss.

  • Internal: one launch path for the ASIFT kernels on every backend (the owner's one-kernel policy; Core's check_duplication flagged the _scheduled method pair): every backend's kernels launch through _ScheduledKernel (formerly _CPUKernel), built on the device's static form (KernelAbstractions' CPU(static = true) on the CPU, the device itself elsewhere) and on the device; the CPU's static/dynamic launches are unchanged. _chunk_warps' backend test is a free-memory reserve (an allocation, the same kernels on every backend): a DUPLICATION.toml [[backend]] entry with that reason.

  • New: onnx_session(path; execution_provider = :auto, threads, gpu_mem_limit), public: the package's ONNX Runtime session builder (formerly the internal _onnx_session), for another package's network: the provider resolved as LoMa's is (resolve_execution_provider; on CUDA, conda-forge's ONNX Runtime build, which runs on the B200's sm_100), ONNX Runtime's intra-op pool the caller's threads, and on CUDA the session's arena capped at the keyword gpu_mem_limit bytes (default 8 GiB, LoMa's). The detector's plans now leave free the room the live CUDA sessions' arenas may still take, each session's cap less what its arena already holds (measured at its creation and around LoMa's runs; the free memory the plans start from already excludes it), rather than 8 GiB per session ever created. A session released or collected reserves nothing, and the CUDA probe's one-op session (built by onnx_cuda_status, so by any :auto or :cuda resolution) reserves nothing and is released at once: before, the reserve only grew, the probe alone added 8 GiB, and a descriptor and a matcher on a 24 GB device left the detector one warp per chunk (slower ASIFT, the same detections). BlobBoards' descriptor network moves onto it.

  • Breaking: asift_detect's detection keyword mask is exclusion_mask (the image and the StabilizedImage methods; the semantics are unchanged: a marked pixel excludes every detection whose centre lies within three of its scales, and a StabilizedImage's saturation joins it). A call with mask is an ArgumentError naming exclusion_mask. gaussian_blur(img, σ, mask)'s positional weight map is another thing and keeps its name.

  • Faster: asift_detect(…; exclusion_mask) decides each detection's clearance from a summed-area table of the mask, read over the detection's disc, instead of a full-frame Euclidean distance transform per call: the same decisions (the same distance rule), ~6× faster and ~6.6× less memory on a 24 MP frame (also every asift_detect(::StabilizedImage), whose saturation is its mask).

  • Changed: StabilizedImage(image, saturated, white) holds a Matrix{Float64} image and a BitMatrix mask as given (another element type is converted, once): variance_stabilize no longer copies the transformed frame and the saturation mask a second time.

  • Breaking: a LinearRaw's mode is its type, LinearRaw{:green_cells}, LinearRaw{:bin2} or LinearRaw{:demosaic}, and its sample geometry one field. The fields plane_maps, green_offsets and green_dn are L.geometry = (; plane_maps, green_offsets, green_dn), nothing for the display demosaic (and only for it); the mode field is gone (L isa LinearRaw{:bin2}, or raw_provenance(L).mode). The geometric accessors (raw_frame_map, bayer_plane_map, green_cell_sites, variance_stabilize) are methods of the cell images, the display image's their refusals (the same ArgumentError), with no run-time mode test; the green-cell plane is chosen by dispatch.

  • New: load_raw(path; …, cache), the decoded-RAW cache: with cache a FileCache (open_cache, the JLD2 extension), each RAW is decoded once per content and settings — the entry is keyed by the file's SHA-256, the mode, the gain, the white balance and the decoder's type (cache_key) — and a hit returns the LinearRaw as decoded, value for value. One cache for the stack's one RAW path: PanoramaStitching's example kept its own (code-sum PNGs and a JSON index, decoded_frames), which goes for it. cache = nothing (the default) decodes every call.

  • Fixed: the stabilized image's noise threshold (asift_detect(::StabilizedImage; noise_sigmas)) read the noise σ of every DoG slice, the two boundary slices included, while the extremum search reads only the interior ones (2 … NS − 1, since the interior-slice search): the threshold was set by a slice no extremum comes from, so too high, and faint extrema were dropped. It is the noisiest searched slice's σ now: a lower threshold, more detections at the same noise_sigmas.

  • Changed: LoMa on ONNX Runtime's CUDA provider no longer releases the ASIFT detector's plans at every call (a full GC, the pool reclaimed and the plan rebuilt at the next detection). Each CUDA session's arena is capped (gpu_mem_limit 8 GiB, grown by request), and the detector's plans size their resident warps from the free device memory less that cap for every CUDA session of the process; the plans cached before a session is created are released once, then.

  • Internal: Core's device queries. The chunk sizing's device_free_bytes and the plan release's pool reclaim are Core's (device_free_bytes, reclaim_device!, answered by its CUDA extension); the CUDA 12 cuDNN probe reads its version with the module's one _library_version.

  • Faster: load_raw(...; mode = :demosaic) through LibRaw copies the visible photosites once, into the display image: its checks of the mosaic's layout no longer copy the mosaic too.

  • New: the stack's one image viewer, compare_images(entries; titles, default, size) (exported; the Makie extension): a menu over named entries, each a tuple of images (or a function returning one, loaded when chosen), one axis per image, the axes linked, each image drawn by Core's orientation rule (imshow!). PanoramaStitching's compare_makie.jl keeps only which renders form a pair.

  • New: the one display encoding, srgb_image([T = N0f8], img; exposure, white_balance, background, tone) (exported), and linear_to_srgb (public), the IEC 61966-2-1 OETF beside srgb_to_linear. A linear H × W × 3 image (or one plane) to RGB{T} (Gray{T}): exposure, a white-balance factor per plane, an optional tone curve (default none), clamp, sRGB, N0f8 or N0f16; a NaN value is the background. A broadcast, so a device image is encoded on the device. The copies of the encoder in PanoramaStitching's save_srgb, RadiometricCamera's write_pngs and write_decoded, and BlobBoards' hero script become calls to it.

  • Breaking: each plane of a Bayer image is a frame-labelled map onto the raw, bayer_plane_map(L, c); the bare offsets are gone. BayerCellFrame is now BayerCellFrame{C}, the frame of plane C (:R, :G, :B): the planes are indexed alike but sampled at different points of the cell. bayer_plane_map(L, c) (exported) is plane c's Core SimilarityMap{…, BayerCellFrame{C}, PixelFrame{:colmap}}, its translation onto the cell grid composed with raw_frame_map(L), so the ×2 is stated once; it sends every sample onto the centre of its photosites exactly, for all four phases and both origin parities. The green plane's map IS raw_frame_map(L), whose source is now BayerCellFrame{:G} (the cell grid). A warp reads plane c through inv(bayer_plane_map(L, c)) ∘ math, like any other map, all planes in one call through Core's image_warp!(dst, warps, src); a composition that joins two planes' frames is refused (MethodError). Removed: the LinearRaw fields plane_offsets (a plane's offset is translation(inv(raw_frame_map(L)) ∘ bayer_plane_map(L, c))) and frame_map (raw_frame_map(L)); the new field plane_maps holds the maps. raw_provenance(L).frame_map.from reads "BayerCellFrame{:G}" (was "BayerCellFrame"). plane_offsets and PanoramaStitching's raw_scale are retired names.

  • ASIFT's result no longer depends on the threads: the same image gives the same keypoints, in the same order, run to run. The extrema pool fills through an atomic counter, in the order the threads (or a GPU's warps) reach it, and the cross-warp NMS breaks ties in its quantized response by pool index, so on many threads both the order of the detections and, at a tie, which of two survived varied between runs (a panorama's frames detected twice reproduced on 12 of 38 frames with spread_strongest after, 36 of 38 with a top-n cut, in a 32-thread session). The pool is now sorted into a canonical order before the NMS (by position, scale and response, then every other field, the covariance's included: a total order), on the host, and the detections come back in it. cross_warp_nms sorts its input the same way, so its survivors no longer depend on the order of the pool handed in, and on the pool asift_detect(…; nms = :none) returns it keeps exactly what the in-pipeline NMS keeps (no longer "up to ties"). spread_strongest was already a function of its input's order; given the canonical detections it is reproducible too. Breaking (order): a caller that relied on the pool's warp-by-warp order now gets the canonical order.

  • New: an exclusion mask, asift_detect(img; exclusion_mask) — an H×W Bool matrix, true where detection is not allowed (clipped highlights, a vignetted rim): a detection whose centre lies within three of its scales (the DoG's support) of a masked pixel is dropped, on Detections, raw features and covariances alike. There was no way to exclude clipped regions of an ordinary (tone-mapped) image, so keypoints landed on clipped window panes. The rule is the one asift_detect(::StabilizedImage) applied to sensor saturation, now in one place: that method masks s.saturated through exclusion_mask (joined with a caller's exclusion_mask), and takes raw = true, which it refused while the masking read Detections.

  • New: tilt_schedule(name), ASIFT's tilts by name; asift_detect(img; tilts = :untilted) restricts ASIFT to the untilted view (t = 1). :imas25 (the default of asift_detect and AsiftPlan) is IMAS-25; :untilted is the identity warp alone, plain DoG-SIFT, exactly the identity warp's detections under :imas25. On real panorama frames a tilted warp's keypoints localize at about 0.95 px rms, the untilted view's at about 0.3 px, and the untilted schedule gave the best seams. tilts still takes a vector of (t, φ) pairs. Breaking: tilts = nothing (the old spelling of the default) is gone: give :imas25 or omit it. The private IMAS-25 helper is retired (tilt_schedule(:imas25)).

  • Every ASIFT detection is refined; an extremum whose refinement fails is dropped, never returned on the octave's grid. The extrema were searched on all S + 2 DoG slices of an octave, the two boundary slices included, whose extrema have no slice above or below and were emitted unrefined (integer positions on the octave's grid, 2^octave input pixels; NaN covariance), and an extremum whose sub-pixel refinement failed (a singular or non-finite step, a walk off the interior) was emitted unrefined too. On a Tamron panorama frame 1451 of the 3077 detections (47 %) were unrefined: 1382 from boundary slices, 69 failed refinements. Now only the interior slices are searched, as Lowe's SIFT does (the boundary slices' scales are the adjacent octaves' interior slices), and a failed refinement is dropped and counted (plan.refine_dropped). The same frame gives 2099 detections, all refined. On a synthetic field of blobs at known sub-pixel centres (IMAS-25, every pooled detection): rms error 0.52 → 0.17 px (octave −1 0.46 → 0.08, octave 0 0.60 → 0.28, octave 1 0.65 → 0.39); two renders 1.4° apart, the matched detections' rms 0.30 → 0.16 px. The tilted views' back-projection carries no bias (every warp's mean error below 0.04 px).

  • clear_asift_plans!() returns the memory to the device (CUDA.jl's pool is reclaimed), and a LoMaDescriptor or LoMaMatcher on ONNX Runtime's CUDA provider calls it before it runs: the cached plan (most of the device, sized from what was free) stayed resident after detection, and ONNX Runtime, which allocates outside CUDA.jl's pool, then failed to allocate in a long-lived process. The next detection rebuilds its plan (+0.04 s on a 2128 × 1424 frame on the RTX 4090; the release itself 0.3 s: a full collection, since unsafe_free! frees a buffer only once the arrays derived from it during a detection are collected, and they held a 10.6 GiB plan in use until then). A plan whose pyramid buffers no longer fit (another allocator grew after the chunk was sized from the free memory) halves its resident warps and retries instead of failing; an explicit chunk is never shrunk.

  • New: variance_stabilize(L::LinearRaw; electrons_per_dn, read_noise) -> StabilizedImage, the detection image of a RAW. The generalized Anscombe transform of the green cells (:green_cells, or :bin2's G plane) for Poisson–Gaussian noise, from the photosites' own code spans (white − black of each green), the gain, the conversion gain (electrons per code) and the read noise (codes): after it the noise has σ ≈ 1 from a few electrons to the white level. Neither noise parameter is in a RAW's metadata (LibRaw reports neither): they are stated per camera and ISO. LinearRaw gains the field green_dn (the two greens' white − black; nothing for :demosaic).

  • New: asift_detect(s::StabilizedImage; noise_sigmas = 3.0, kwargs...): the DoG contrast threshold in noise σ. The DoG's noise σ for unit iid input is derived from the pyramid's own impulse response (the root energy per sample phase) in its noisiest slice, the finest octave's finest (0.137 for the default upsampled pyramid, seed blur 1.6): every coarser slice and tilted view is held to more than noise_sigmas of its own σ. A detection whose centre lies within three of its scales of a saturated pixel is dropped (masked, not clamped). intensity_thr stays the threshold of asift_detect on an arbitrary image, which has no noise model.

  • New: spread_strongest(detections, image_size, n; tolerance = 0.1, device), about n of the strongest detections spread over the image — Suppression via Square Covering (SSC) of Bailo et al. (Pattern Recognition Letters 106, 2018), ported from the authors' code (https://github.com/BAILOOL/ANMS-Codes, MIT License, Copyright (c) 2018 Oleksandr Bailo; credited in the docstring and the README's Citing section) and parallelised on KernelAbstractions: one implementation on the CPU and CUDA. The sequential covering is computed as per-cell maxima (two atomic reductions, no sort) and synchronous rounds of the greedy independent set of the cells in score order, and returns the authors' sequential result exactly, ties included (test/data/ssc_bailo.toml is their C++'s output). Where the authors' integer width search misses the tolerance band, the width is bisected between the bracketing integers, so the count lands in the band. It replaces strongest (removed), the global top n by |response|, which on a frame with a bright window and a dim interior kept nearly every detection in the window; LoMaDescriptor now describes spread_strongest(detections, size(img), max_detections). A caller of strongest(d, n) calls spread_strongest(d, image_size, n).

  • LoMa on CUDA runs on the B200 (sm_100) too. Microsoft's GPU builds of ONNX Runtime (1.30 and 1.31, CUDA 12 and 13, and the CUDA plugin EP) carry device code for sm_60 to sm_90 and sm_120 only, no sm_100 and no PTX, so on a B200 every kernel failed (cudaErrorNoKernelImageForDevice) and :auto fell back to the CPU. The CUDA path now runs conda-forge's build of ONNX Runtime 1.31.1 (onnxruntime-cpp, cuda129; sm_70 to sm_120 with sm_100), merged with the libraries it links (libprotobuf, libabseil, libre2) into one artifact, onnxruntime_cuda in the new Artifacts.toml, pinned by its tree hash and installed by the first CUDA probe from the four conda-forge files it pins by sha256 (Pkg cannot unpack a .conda; Julia's own 7z does), 0.34 GB once per depot; ONNXRunTime's own GPU download is no longer used. The CUDA 12 runtime and cuDNN stay the JLLs' (the runtime now at least 12.9, the build's floor; libcusparse.so.12 is loaded too). The CPU path is unchanged (ONNXRunTime's Microsoft 1.30 build). loma_provenance records onnxruntime_build on CUDA. New dependencies: the stdlibs Tar and p7zip_jll.

  • LoMa runs on ONNX Runtime's CUDA provider wherever it can, with no CUDA setup. execution_provider = :auto is the new default of LoMaDescriptor and LoMaMatcher (was :cpu): CUDA when a CUDA device is visible and the active environment holds CUDA_Runtime_jll and CUDNN_jll, otherwise the CPU with a warning naming why. :cpu and :cuda stay explicit; an explicit :cuda that cannot run is an error naming why. ONNX Runtime's GPU build needs CUDA's 12.x runtime and cuDNN 9, which CUDA.jl 5.11 (runtime 13 on a CUDA 13 driver) and cuDNN.jl (1.4 does not resolve beside CUDA_Runtime_Discovery 2) could not give it: the package now takes the CUDA 12 builds of CUDA_Runtime_jll's and CUDNN_jll's artifacts (lazy: downloaded on the first CUDA session), selected by their cuda tag and loaded for ONNX Runtime alone, beside whatever runtime CUDA.jl uses. The two JLLs are weak dependencies pinned by [compat] (CUDA_Runtime_jll 0.21 - 0.26, CUDNN_jll 9.20 - 9) and never loaded as modules (CUDNN_jll would load the host CUDA's build under the same soname); an environment without them never resolves or downloads them. Neither CUDA.jl nor cuDNN.jl is needed for LoMa on the GPU any more; a cuDNN or CUDA 12 library already loaded from another file is refused by name. Needs a driver for CUDA ≥ 12 (≥ 525.60; Blackwell needs ≥ 570).

  • New (public): onnx_cuda_status() (an OnnxCudaStatus: available, or the reason why not; the driver, CUDA runtime and cuDNN versions) and resolve_execution_provider(execution_provider), :auto's answer, for a caller that keys a cache on the provider.

  • ONNX Runtime's threads are the process's, not the machine's. A new threads keyword of both constructors (default Threads.nthreads(), the stack's measure of the allotment: a Slurm job starts Julia with --threads=$SLURM_CPUS_PER_TASK) sizes ONNX Runtime's intra-op pool; the inter-op pool is 1 (the graphs run sequentially). ONNX Runtime sized its pool from all cores and pinned each thread to one, which inside a Slurm cgroup oversubscribed the allotment (six 28-CPU jobs drove a 112-thread machine to a load of 673) and failed to pin (pthread_setaffinity_np failed); an explicit size disables the pinning.

  • New (public): loma_provenance(l) of a descriptor or matcher: the model file and its sha256, the provider used, the threads, ONNX Runtime's version and, on CUDA, the driver, runtime and cuDNN versions.

  • ONNXRunTime compat is ~2.1: the sessions are built through its C API (its load_inference takes no session options), so a new minor version is checked before it is admitted.

  • Measured 2026-11-10 (RTX 4090, 32 threads, two 2128 × 1424 panorama frames, 2048 detections each, :b + :b128): both descriptions and the match in 0.11 s on CUDA against 3.46 s on the CPU. CUDA against the CPU on six frames: descriptors within 3.2e-3 of their largest component, 1103 of the 1109 matches of 15 frame pairs identical.

  • The one RAW path. Every RAW → image step of the stack lives in this module (load_raw, load_raw_mosaic, LinearRaw; the LibRaw_jll extension decodes), and VisualGeometryToolkitCore's check_raw_decoding flags any other decoding path (rawpy, LibRaw C calls, dcraw, a demosaic or a CFA interpolation, a decoded RAW re-read as a PNG); the quality item runs it on this package.

  • New: mode = :green_cells, THE image for geometry and radiometry: each 2×2 Bayer cell's two green photosites' mean, half resolution, one plane: :bin2's green plane, bit for bit, from the same kernel. A cell is saturated when either green is.

  • New: the exact frame map of the cell grid. BayerCellFrame (the cell images' COLMAP frame) and raw_frame_map(L), a Core SimilarityMap{2, …, BayerCellFrame, PixelFrame{:colmap}}, x ↦ 2x onto the visible raw for every Bayer phase and visible-origin parity (an odd margin changes only the colour phase). inv is its inverse. A calibration matrix moves with it through Core's reframed_calibration(raw_frame_map(L), K).

  • New: green_cell_sites(L, row, column), a green-cell value's footprint: its two diagonal green photosites' raw COLMAP centres, at (±½, ∓½) from the cell centre in an RGGB or BGGR cell and (±½, ±½) in GRBG or GBRG (LinearRaw.green_offsets).

  • New: mode = :demosaic, for display figures only: LibRaw's AHD demosaic (libraw_dcraw_process, linear: gamma 1, no auto-brightening, no maximum adjustment, 16 bits, raw colour, white balance 1) of the visible area, with an optional white_balance and the gain applied in Julia. It carries no frame map: raw_frame_map and green_cell_sites refuse it, and its plane_offsets is empty (no pixel is a sample). Through load_raw(path; mode = :demosaic) only.

  • The EXIF flip is never applied, in any mode (the demosaic zeroes LibRaw's flip before it copies the image out); a demosaic whose output is not the visible area's size is refused.

  • New: provenance. LinearRaw carries its frame_map; raw_provenance(L) (public) gives the decoder, LibRaw's version, the mode, the gain and the frame map as plain data.

  • Breaking: the raw-grid :green mode is removed (the green photosites at full resolution, zeros at red and blue): read :green_cells, or the mosaic itself. With it LinearRaw.sampled is removed (every pixel of every remaining mode carries its sample).

  • Breaking: RawpyDecoder and its PythonCall extension are removed, with the package's CondaPkg.toml (rawpy, exifread) and the PythonCall weak dependency: LibRawDecoder is the one decoder.

  • The two-axis matches!(ax_A, ax_B, cs) and matches(img_A, img_B, cs) take weighted csponds: a cspond is bare (opaque) or Attributed(src => tgt, (; weight)), its weight (Core's weight_of, in [0, 1]) the alpha of its markers.

0.6.1 (2026-11-09)

  • On VisualGeometryToolkitCore 0.8 (compat "0.8"); nothing else changes.

0.6.0 (2026-11-07)

  • The LoMa model digest and cache_key are VisualGeometryToolkitCore's sha256_hex (the same values); the SHA dependency is dropped.
  • Breaking, with VisualGeometryToolkitCore 0.7: a Detection's and a DescribedFeature's blob shape is shape_matrix(d) (Core's renamed generic; moment_matrix is retired). d.Σ is unchanged.
  • LoMaDescriptor takes an image of any size: (descriptor)(img, detections) with img a 3 × H × W RGB channelview of any real element type and the detections in its COLMAP pixels. An image that is not the graph's fixed LOMA_IMAGE_SIZE² (public, 784) is resampled to it per channel by Core's bilinear image_warp! (the COLMAP scaling, Float64 math, Float32 taps: PanoramaStitching's hand resampling, bit for bit); a 3 × 784 × 784 image is taken as before. An image smaller than 784 on either side (an upscale) is resampled with its border replicated: the network's outermost pixel centres map within half a pixel past the image's, and their taps clamp to the border pixel instead of blending in the fill. The keypoint chart reads the image's own size, so the detections are never rescaled and the described features carry them as given (PanoramaStitching rescaled its frames to 784² by hand, moment matrices included, which the descriptor never read).
  • strongest(detections, n) (public), the selection LoMaDescriptor applies (formerly the private _strongest): the n largest |response|, in detector order. Ties at the cut now keep the earlier detections (a stable partial selection, O(N) as the former partialsortperm, which left that choice unspecified). BlobBoards' cap keeps the same set but re-sorts it by response.
  • box_downsample takes a 2-D image and any real or colour element type, averaging in the element type's float (Float32 for a Float32, Gray{Float32} or 8-bit image: BlobBoards' supersampling filter, bit for bit; Float64 for a Float64 or integer image, as before). A Float32 3-D image now gives a Float32 result (formerly Float64).

0.5.0 (2026-11-06)

VisualGeometryToolkitCore 0.6 (compat "0.6"). Breaking with Core's CovMat, which now holds a static matrix (full, diagonal or isotropic).

  • Detections' covariances are Core 0.6's CovMat{2, T, M}; the spelled-out CovMat{N, T, N², F} types are gone.
  • CACHE_FORMAT (2) is part of every cache key, so a JLD2 cache entry of the earlier CovMat layout is a miss, not a failed load.

0.4.2 (2026-11-04)

Additive: RawMosaic.metadata keeps its type and every field it had.

  • The RAW's EXIF as Core's typed values: exposure_settings, lens_settings, body_identity, lens_identity and capture_time (public) of a RawMosaic or its metadata, and extract_capture_settings(frames; lens = nothing), a capture's CaptureSettings (one body; a stated lens overrides EXIF's claim, with a warning when they disagree: an adapted lens is misnamed, the FX3 with Sigma EF lenses claims a Sony "DT" lens).
  • load_raw_metadata(path; decoder): the EXIF fields alone; LibRaw reads the headers without unpacking the photosites.
  • metadata gains make, body_serial, lens_make, lens_model, lens_serial and focus_distance_m (nothing where absent; a non-finite or non-positive focus distance too, never a distance). LibRaw: the lens from libraw_get_lensinfo (its leading fields mirrored, the offset checked); no body serial or focus distance. rawpy: the EXIF tags.
  • Compat: VisualGeometryToolkitCore 0.5.4.

0.4.1 (2026-11-04)

  • New load_raw(path; mode, gain = 1, decoder = LibRawDecoder()): a camera RAW file (any Bayer format LibRaw reads) as exact linear values, (raw − black[c]) / (white − black[c]) × gain, camera raw colour, white balance 1, in the sensor's orientation (no EXIF flip; the raw's COLMAP frame). Ported from RadiometricCamera's convert_raw.py. Modes :bin2 (2×2 Bayer quads: R, the mean of the two greens, B; half size) and :green (the green photosites alone on the raw grid). Its fullres demosaic is not provided. Returns a LinearRaw: image, sampled, saturated (a quad is saturated when ANY of its photosites is at the white level, raw ≥ white, no margin), plane_offsets (each plane's true sample position: the green at the quad centre, R and B (±¼, ±¼) binned pixels from it) and the EXIF metadata. Rounded to 16 bits, :bin2 equals convert_raw.py --mode bin2's linear PNG value for value.
  • New load_raw_mosaic, RawMosaic, LinearRaw(mosaic; mode, gain): the decode and the processing apart; the processing is all Julia, one KernelAbstractions kernel per mode. A RawMosaic holds the colour pattern (square, at most 4 × 4, validated) and the visible area's origin; each photosite's colour index is photosite_colors(mosaic), the pattern tiled from the raw origin (rawpy's raw_colors_visible, which the rawpy decoder checks against it).
  • Black levels: one rule. LibRaw's per-channel black levels with a 2×2 spatial pattern folded in; a spatial pattern with no per-channel form is refused, naming the file, never dropped. rawpy exposes only the per-channel levels (its black_level_per_channel drops such a pattern), so RawpyDecoder() cannot apply the rule; its docstring says so.
  • Two decoders, both weak dependencies: LibRawDecoder() (new extension VisualGeometryToolkitFeaturesLibRawExt on LibRaw_jll 0.22.2: a ccall layer whose mirrored libraw_data_t layout is checked against LibRaw's accessors on every file, refusing a LibRaw other than 0.22) and RawpyDecoder() (new extension VisualGeometryToolkitFeaturesPythonExt on PythonCall; rawpy and exifread declared in the new CondaPkg.toml, reading one open file object). Without the extension loaded, load_raw names the package to load.