A patch on VisualGeometryToolkitCore 0.13.1 ([compat] VisualGeometryToolkitCore "0.13.1").
- ONNX Runtime's CUDA build lives in the artifact store (Core's
install_built_artifact: the storeVGTK_ARTIFACT_STORE, the site profile's, the preference's), as the LoMa weights do, never a cluster's depot: the same conda-forge files and the same pinned tree, built once per store, under the store's per-tree lock, so the jobs of one machine that probe CUDA together build it once. A machine whose depot holds the tree moves it with Core'smigrate_artifacts(or builds it again into the store); where no store can be had (the depot refused, none named), a depot's own tree is still read, with a warning, so the CUDA path does not fall to the CPU for want of a store. The CUDA 12 runtime and cuDNN it loads stay CUDA_Runtime_jll's and CUDNN_jll's own lazy artifacts, which Pkg installs into the depot like every JLL's.
A patch on VisualGeometryToolkitCore 0.13.0 ([compat] VisualGeometryToolkitCore "0.13").
- The LoMa ONNX weights live in the artifact store (Core's
store_file:<store>/files/<sha256>/<file>, the storeVGTK_ARTIFACT_STORE, the site profile's, the preference's), never a cluster's depot; with the depot store they stay in this package's scratch space, as before. Same files, same sha256 check. - The verified download of the ONNX Runtime CUDA packages is Core's
download_verified(this package's copy is removed); the Downloads dependency is dropped. - Tests: the tilted detection's scale-space hash pins are kept per CPU family, a (features, covariances) pair each with the machines that reproduce it (pair 1 the machine of the factoring, the cluster's poseidon and hephaestus; pair 2 the cluster's nereus and GitHub's hosted runners); a machine that matches neither prints its two hashes. The detector is unchanged.
A patch on VisualGeometryToolkitCore 0.12.0 ([compat] VisualGeometryToolkitCore "0.12"), whose radiometric types moved to
VisualRadiometryToolkitCore. No code of this package changes.
A patch, additive: the Förstner F-test's critical values computed once per configuration.
FoerstnerTests()anddetect_foerstner(…; f_tests)(seeded and image-wide, every input): a table of the F-test's critical values per window half-width ρ, keyed by everything they depend on (the gradient taps, ρ, the pixel noise's structure,significance_level), each computed on its first use and read by every later call. The caller makes one per detect configuration (a set of photographs). Without it (the default,nothing) every call computes them again, as before; the rows are bit-identical either way.- Why: the values are dense host linear algebra (
_residual_dof; on a mosaic's bilinear green image eight lattice phases and centre parities per ρ) and a seeded call on a chrome photograph has one ρ per distinct seed scale, so on the GPU they were nearly all of a RAW chrome photograph's detect time (RC's chrome detect, Sony FX3 fisheye 30 cm set, one B200: the seeded call 5.9–9.0 s of 6.3–9.4 s per photograph, the GPU mostly idle).
A patch: on VisualGeometryToolkitCore 0.11 and VisualGeometryToolkitEstimation 0.12 ([compat] "0.11", "0.12").
Features' refinement runs Core's device LM (static_lm), whose success rule Core 0.11.0 keeps (stationary and
accepted), and it reads no host-LM verdict; no behaviour or API changes.
A patch for SFOP's wrong detections on the RAW chrome photographs: a polarity gate, the sensor model's noise in SFOP's λ₂ test, and the detectors on a RAW mosaic at the raw resolution (the bilinear green image).
polarity = :none | :dark | :lightondetect_sfopand the seededdetect_foerstner(img, seeds), additive, ASIFT's vocabulary and sign (one sign for the package,polarity.jl; ASIFT's parse now shared). The structure tensor is sign-invariant, so SFOP took dark spots for circular points: on Features 0.9.4's RAW green cells, 111 dark-polarity:circularrows over the 668 photographs of the five FX3 chrome sets (99 tamron chrome-siemens, 11 fisheye siemens, 1 fisheye april). The sign is the scale-normalised Laplacian of Gaussian at the detection's σ, positive at a dark blob, tested ONE-SIDED atsignificance_levelunder the pixel noise: a candidate is kept only on significant evidence of the requested polarity, so a window whose sign the noise could have given (on flat background between dots) is kept for neither (a missed blob, never a wrong one). SFOP gates before its NMS (ASIFT's order). The seeded call needs the pixel noise for the test (noise, or a gain-carryingsensor_model); polarity on the image-wide search is refused.:noneis the default, and there every output is 0.9.5's.- Duplicates are not removed. One blob seen from two seeds (SFOP finds a dot at two scales more than a half-octave band apart, which the NMS keeps both of) gives two rows; the consumer resolves them by its own geometry (an assignment to the target's points).
detect_sfop(L::LinearRaw; sensor_model): under a gain-carrying sensor model (Llinearised with it) the λ₂ significance test reads each pixel's variance (sampled on each octave's grid at the pyramid's own pixel map, the floor scaled by the variance under the window's Gaussian) instead of onenoise_sigma, so a bright region's shot noise raises no blob. Without a gain,noise_sigmaas before.- Recall against 0.9.5 (
detect_sfop(L; polarity = :light), then the seededdetect_foerstner, green cells): on 295 photographs of the five sets no 0.9.5:circularrow that is significantly light is lost. Of 82 935 rows, 82 826 remain (none gained). Each of the 95 not found again within 0.5 px has a significantly dark seed (68) or one of a sign not significant (27); the other 14 are one row of a pair within 1 px whose seed is dark or not significant, its twin kept. Every pair within 1 px of two significantly light seeds keeps both rows (92 of 92). An opt-in real-data item holds this on 12 of the photographs (test/data/sfop_recall_095.tsv, 0.9.5's rows). detect_sfop(m::RawMosaic; sensor_model)anddetect_foerstner([T], m::RawMosaic, seeds; sensor_model), additive: the detectors on a RAW mosaic's green photosites at the RAW resolution, positions and covariances in raw COLMAP pixels. The green cells (LinearRaw{:green_cells}, unchanged) decimate the green quincunx to the cell lattice and alias its diagonal detail: on DSC06837 (fisheye chrome-april) they register 187 of 289 dots. The image is the bilinear green on the raw grid, the normalized convolution of the usable green photosites with the cross kernel — a green photosite its own value, a red or blue one the mean of its usable green 4-neighbours — one KernelAbstractions kernel, Float64 sums, its varianceΣw²σ²/(Σw)²per pixel. A SATURATED photosite is censored: a normal sample at the white level (1), its variance the noise law's there (VisualGeometryToolkitCore 0.10.7'svalue_variance, the new compat floor), flagged. A DEFECT of the sensor model is missing: it drops out, and a pixel with no usable green under its cross (a defective green) is the mean of the usable greens of its 3×3 neighbourhood (its diagonal greens); only a pixel with none in the 3×3 (a defect cluster) has no value, never filled, and every window reading it fails. SFOP runs on it with its 2× upsampled first octave atinput_blur = 1/√3raw px (the photosite's box, variance 1/12, plus the interpolation's, ¼ on average), its λ₂ and polarity tests under the photosites' noise (the white-equivalent2σ²per pixel: one green photosite per two raw pixels). The seeded Förstner estimates on the raw-grid image with its own doubling (ip_fop.m step 1) and propagates the noise from the PHOTOSITES,J_k = Σ_q J[:, q] W[q, k]through the bilinear weights, and its F-test's effective degrees of freedom are those of that noise (four separable terms of the interpolation; about 15 % below white input pixels'), each checked by Monte Carlo. On synthetic dots (radius 1.5 raw px) on a Bayer mosaic: NEES 1.93, 96 % coverage, spread 0.014 raw px against the green cells' 0.021, bias alike; a dot with a saturated core is found and localised; a defective green at a dot's centre costs it ≈ 0.06 px RMS (the fallback reads the diagonal greens). Pixel-valued keywords are the image's pixels: on the raw grid, twice the cells' values for the same physical scale (RC 0.10.2 statesderivative_sigma = 2.0,precision_threshold = 1.0). The image-wide search on a mosaic is refused (seed it withdetect_sfop(m)), and so areinput_blur,upsampleandseed_sigma(the image's own). Known limit: G1 and G2 are taken to respond alike (the sensor model carries no per-channel response): a G1/G2 gain mismatch stays in the image as a checkerboard of its amplitude (measure it with flat fields, the characterisation's E5, if it matters). Data (RC 0.10.2's detect + Hungarian index, FX3 chrome sets, each frame under its sensor model; green cells → raw grid): DSC06837 (fisheye chrome-april) 187 → 283 of 289 dots assigned, 0 mislabels; every RC-registered dot of DSC06825 (166 with a saturated photosite near the core) and DSC06837 found again; per set, assigned dots (dustbin): fisheye_10cm_ag 41 943 (1 433) → 44 284 (5 553), fisheye_2710_si 26 457 (2 608; 21 photographs refused) → 33 283 (4 817; none refused), tamron_17cm_si 32 368 (691) → 32 368 (1 416), tamron_20261108_ag 40 187 (107) → 40 999 (6 816), tamron_largeaperture 54 908 (778) → 54 910 (1 459), 14mm_central30cm 39 993 (263) → 40 744 (3 692); no mislabel refusal in either. The dustbin (detections left unassigned, the consumer's to drop) grows with the raw grid's finer windows.- Investigated, not changed: the three dim top-rim dots on DSC06962 with no SFOP candidate are not dropped by the significance test or the NMS: no candidate exists before the NMS. The squashed rim dots (≈ 4 × 1.5 cells) have a precision still rising at the finest searched level (w = 69, 34, 19 … over levels 1, 2, 3 of the upsampled octave), so there is no maximum in scale. Treating the first level as the scale range's bound recovers them (and adds 21 pool candidates image-wide); not shipped, since a missed blob is preferred to a wrong one.
A patch: RAW images stored as Float16 or Float32 (VisualGeometryToolkitCore 0.10.5, now the compat floor), and a
known issue of detect_image_circle under a sensor model, documented.
-
load_raw([T], path; …)andload_raw([T], mosaic; …), additive: the values' element typeT(aBase.IEEEFloat), positional and leading (the stack's signature rule), forwarded to Core 0.10.5'sLinearRaw(T, mosaic; mode, gain, sensor_model). Without it the type is Core's default (Float64, bit for bit 0.9.4's values), read off Core's publiclinearview(for every mode). The display demosaic is stored asTtoo (computed in Float64, rounded once; a gain and white balance that could overflowTrefused). -
load_raw's cache key names the element type (any butFloat64, whose entries keep the plain key: every entry before 0.9.5 is Float64), resolved from Core's default first: a cached Float64 decode is never served for a Float16 request, nor for a default request after Core's default changes. -
No reduction over a RAW's values accumulates in Float16 (the audit, below). Two fixes:
box_downsampleof aFloat16image (orGray{Float16}, …) summed its blocks in Float16; it now sums and scales in Float64 and rounds each output once toFloat16(a block of 2048 and four 0.25s: 2049/9, not 2048/9).detect_foerstner(Float16, …)would have accumulated the squared gradient's window sums in Float16; a precision other thanFloat32orFloat64is refused (ArgumentError). A Float16 image is read inT.
Audited, unchanged (each reads a Float16 image widened): SFOP and ASIFT (the pyramid's
Float32input,_asift_input); the seeded and image-wide Förstner (ka_upload(T, …), the variancesFloat32→T);variance_stabilize(the Gaussian part and its mean in Float64);detect_image_circle(the frames' maximum in Float32, the bin medians order statistics, the bins' mean variance summed in Float64, profiles and drops in Float64);gaussian_blur(Float64). -
Known issue, not fixed here:
detect_image_circleunder a sensor model can take the halo or barrel-reflection light beyond the rim for the rim (since 0.9.2; the default path, without a sensor model, is unaffected). WithUncertaincells (a RAW read with a Poisson–Gaussian model) each bin's noise is the model's alone, the sensor's photon and read noise. Past the coverage the FX3's fisheye has faint light (the lens barrel's reflections, a halo, 0.2–1 % of white) whose drops lie far above that noise (8–14 % of them beyond 4σ), so each angle's outermost edge, the first fit's, lies out there: a = 1328.6, 1366.2 and 1481.8 raw px on the f/4, f/22 and f/32 sets (the f/32 centre 97 px low), against 1086.5, 1340.4 and 1333.8 without a model, in Float64 and Float16 storage alike (the 14 mm diffuser set agrees). Measure the coverage without a sensor model (the default path is unaffected) until the rule that picks the coverage under a model is fixed in its own patch. The opt-in real-data item holds the three sets as@test_broken.
A patch: detect_foerstner's F-test at the effective degrees of freedom (a label change: below), RAW inputs for
detect_sfop and the seeded detect_foerstner, and the image-circle deprecation text.
detect_foerstner's F-test at the residual sums' effective degrees of freedom (difference 4 fromip_fop.m, docs guide).F = Ω_corner / Ω_circularwas tested againstF(redundancy, redundancy), the window's doubled-pixel count less 2, as if the doubled, derivative-filtered samples were independent; at that count the test's level was not the stated one (on white noise 20 % of the windows labelled circular and 25 % corner at 0.999). It is now tested againstF(ν_corner, ν_circular), eachν = (tr Q)²/tr Q²(Satterthwaite) of the residual sum as the quadratic formIᵀ Q Iof the input pixels about the window centre, per window half-width (ν ≈ 87 against 2023 at σ = 6; the estimate's two parameters neglected, a conservative test). Both the image-wide search and the seeded windows. On the THORLABS chrome photographs the SFOP seeds between dots, on line artefacts and on letter strokes (F ≈ 1–3) were labelled circular; they are now unknown. Labels change wherever F lies between the two critical values; positions and covariances do not change. At the image-wide default window (ρ = 6, ν ≈ 12) a noise-free checkerboard X-junction is now unknown at 0.999, a corner at 0.99 (docs guide).detect_sfop(L::LinearRaw)and the seededdetect_foerstner(T, L::LinearRaw, seeds; sensor_model), additive: SFOP on a RAW's green cells (their values, the cell grid) and the seeded windows under the frame's sensor model, its covariance as the image-wide call's onL(bit for bit at the same windows). RC's chrome evaluation detects its RAW sets this way.detect_image_circle's deprecation warning says "deprecated" for the hull fit's keywords (min_hole_area,smoothing_weight,border_margin), so@test_deprecatedand a user's--depwarn=yessession recognise it (it read only "is ignored"; tested under--depwarn=yes, red first).detect_sfopkeeps a maximum beside a failed window. The extraction's edge test (ASIFT's) was meant to be off in SFOP's precision mode, by an edge threshold offloatmax; it was not: next to a −∞ neighbour (a window that failed the λ₂ test or reached the border) the curvature is −∞ andtr²/detis NaN (ordet < 0across a diagonal neighbour), so the maximum was dropped. The edge test is now skipped in the precision mode: such maxima are detections.detect_sfop's precision is each detection's own. Adjacent octaves share a level, so two octaves can find a peak at one position and scale with differentw; the table ofwwas keyed by position and scale alone, and the row the NMS kept could report the other octave'sw. It is keyed by the octave too.detect_sfop'snb_octavesdefault ismin(3, sift_nb_octaves(size(img)...; upsample)): an image too small for 3 octaves is searched on the octaves it holds (it was anArgumentErrorfromgaussian_scale_space); anb_octavesbeyond them, asked for, is still refused.- The seeded
detect_foerstner(T, img, seeds)takes its own keywords:device,noise(sensor_modelon aLinearRaw),derivative,derivative_sigmaandsignificance_level. The image-wide search's (method,noise_sigma,integration,integration_sigma,precision_threshold,roundness_threshold) were accepted and ignored; they are now anArgumentErrornaming them. The seeds are typed: detections of aDisc,OrientedDisc,EllipticFrameorAffineFrame, each also uncertain, checked per seed (soDetection[]and a vector of mixed frames are seed vectors); a point seed, which has no window, is anArgumentErrornaming its frame (it was aMethodErrorofcharacteristic_scale, inside the call).detect_foerstner(L, seeds)takes the default precision, asdetect_foerstner(img, seeds). detect_image_circle: the same outputs, faster (bit for bit, checked on every real set, the synthetic field stop and a flat image against 0.9.3's code): the bin median of a bin of more than 32 samples by a radix selection (O(m) sweeps, no scratch, every backend) instead of the O(m²) rank count, the same two order statistics bit for bit; the saturation mask's warp skipped when no pixel is saturated (its samples are all 0); the empirical noise's drop noise at unit pixel noise computed once per bin row count, not once per angle.detect_image_circle's docs say what the code does (no code change): a bin more than half saturated is FLAGGED (:saturatedinrim.bin_reason) and keeps its median in the profile, read like any other; 0.9.2's and 0.9.3's docs and this CHANGELOG's 0.9.2 entry said such bins were skipped. Also stated: a column's pixel count has no floor of one pixel; the:autoladder's fallback (the finest binning tried); a zero noise used as it is. These are candidates for an owner-reviewed revision (.release/wave11-removals.md).- Scheduled for wave 11:
gaussian_gradient'sderivative::Symbolbecomes a type (by dispatch); public since 0.9.1, so unchanged in this patch (.release/wave11-removals.md).
A patch: detect_image_circle's rim radii through the log-polar map's differential and inverse (Core 0.10.3).
detect_image_circle's rim radii are in pixels through the log-polar map's differential and inverse (Core's newtransform_deriv(::LogPolarToCartesian, p)andinv(::LogPolarToCartesian); needs that Core release). 0.9.2 measured an edge's drop over "the samples to ≥ 2 px": on the log grid that span's sample count steps where the radial spacing crosses 2/n px, and 0.9.2 corrected it by hand, dividing each log drop by its span's length. Now:- the drop at radius r is across exactly 2 px about it, its ends r ± 1 px found through the inverse map and the profile interpolated between columns: continuous in r, no span-length step; the per-span normalisation is gone;
- the edge is the log slope's sub-sample peak in the grid coordinate, taken to pixels by the forward map;
- each bin's noise is scaled by its own pixel count, rows × arc from the map's differential (each column is at most a
pixel thick): before, the empirical noise was one per angle, so the significance test and the bin rule's edge
uncertainties now use each bin's pixels.
RimProfile.noiseis now each angle's noise PER PIXEL (a bin of m pixels hasnoise/√m). The return type and every keyword are unchanged. Real-data coverages (raw px) move within their σ on the 14 mm, f/4 and f/22 sets; the start invariance holds (14 mm: sub-pixel and ±25 px starts within 0.25 px); the f/32 set's coverage is now its reference, owner-verified on the rim including the mount side (centre (2136.997, 1439.331), semi-axes 1333.768 and 1315.245 raw px; the earlier reference sat 3.5 px high), checked as every other set is; the synthetic field stop is at 1310.29 (truth 1309.6, a binary disc: the lit pixels' outer boundary is within √2/2 px). Requires VisualGeometryToolkitCore 0.10.3 (its compat floor).
RimProfile.noisechanged meaning, a change for any code that reads it: in 0.9.2 it was each angle's noise scale of its BINS (the model's median bin noise, or the empirical one of the profile's samples); from 0.9.3 it is each angle's noise PER PIXEL σ₁ (a bin of m pixels hasσ₁/√m, m its rows × its cells' arc). The field's name and type are unchanged.
A patch, additive (owner rulings 2026-11-14: the lens coverage is the rim's outer edge; no rim margin).
-
detect_image_circlemeasures the lens COVERAGE, where its light ends, on the rim itself. The method:- the frames' per-pixel maximum is resampled on a log-polar grid about the image centre (or a caller's
initialmodel), using VisualGeometryToolkitCore'slogpolar_map/image_warp!at ≤ 1 px in radius and in arc; - every (angle bin, radius) is reduced to its median by one KernelAbstractions kernel, on
device(defaultdefault_device()); off-frame and saturated bins are skipped; - every edge of each angle's outer envelope (the brightest the profile gets at or beyond each radius: where the light
ends it does not come back) is a candidate: a run of drops significant against the angle's own noise at the
family-wise level
significance, at its steepest log drop per pixel (the log floored at the angle's darkest level), refined to the sub-sample peak; a rim that falls in two steps gives both (owner ruling); - the RANSAC ellipse (or circle) is fitted on one candidate per angle, the same scorer as before: each angle's outermost edge (where its light ends), then each angle's edge nearest that model (stray light, or a step off the model, gives way), fitted again: the coverage.
What changes:
- No MSER and no hull. On a soft rim (an APS-C lens on the full frame, a ~230 px ramp) the hull fit landed mid-ramp; the coverage is at the ramp's foot.
- The return type is unchanged:
circle,fit(now on the candidates),contour(each angle's chosen edge, the rim's points),cutandrim_gap. - Added to the result:
covariance(the coverage's, in Core'sambient_coordinatesof the measured plane: s²·solver_covarianceon the inliers;nothingfor aCircle, whose problem provides none),center_scale,axes_scale(the standard deviations, in the result's pixels) andrim::RimProfile(the profiles, the candidates, the per-angle edges and the noise statistics). - New keywords:
initial,rim_band = (0.8, 0.0),rim_angles = :auto(the binning whose coverage is the most precise, chosen per image),significance = 0.01(the edge test's family-wise level per angle) anddevice. - The noise statistics follow the frames' TYPE:
UncertainRAW cells (a sensor model with a variance law) give the model's per-bin noise (SensorModelNoise, with a model-vs-data check); plain images give the data's (EmpiricalNoise). - New public names:
RimProfile,SensorModelNoise,EmpiricalNoise.
- the frames' per-pixel maximum is resampled on a log-polar grid about the image centre (or a caller's
-
Deprecated, inert:
detect_image_circle'smin_hole_area(no longer required),smoothing_weightandborder_marginare hull-fit keywords. They are accepted with a deprecation warning and ignored, and are REMOVED in 0.10.0 (wave 11: listed in the monorepo's.release/wave11-removals.md).- Every other keyword is live.
-
Removed (internal):
- the
VisualGeometryToolkitFeaturesImageSmoothExtextension and itsImageSmoothweak dependency (the L0 smoothing fed the MSER path); - the
DelaunayTriangulationdependency (the hull).
- the
-
Not noted at release (added in 0.9.4):
inlier_mask(fit)indexes other points. In 0.9.1fitwas the hull's RANSAC andinlier_mask(fit)indexedcontour. From 0.9.2fitis the RANSAC on one candidate per angle that has any, andinlier_mask(fit)indexes THOSE points (in angle order);contourholds the inliers' edges only, socontour[inlier_mask(fit)]is no longer meaningful (and generally a BoundsError or the wrong points). -
Not noted at release (corrected in 0.9.4): a bin more than half saturated was described above as skipped; it is flagged (
:saturated) and keeps its median.
A patch, additive, on VisualGeometryToolkitCore 0.10.
- New:
gaussian_gradient(img, σ[, mask]; derivative = :gaussian2d) -> (gx, gy), exported: an image's gradient at scale σ (∂/∂column, ∂/∂row, per pixel), the derivative of a Gaussian along one axis and the Gaussian across, zero padded — the Förstner operator's own gradient, whichdetect_foerstnernow calls (one implementation; its results are unchanged, bit for bit). One KernelAbstractions kernel, on the image's device (CPU or GPU). With amaskit is the gradient ofgaussian_blur's normalized convolutionblur(img·mask)/blur(mask): nothing bleeds in from outside the mask, and the gradient is 0 there. - New:
gaussian_scale_space(img; …) -> GaussianScaleSpace, exported (andlevel_sigma, public): an image's Gaussian scale space, built by ASIFT's own pyramid (its taps, its fused tiled blur, Lowe's downsample, the 2× upsampled first octave), level for level whatasift_detectbuilds for the untilted warp. ASIFT now holds its levels in one (AsiftPlan.space, replacing the fieldsgauss,dims,pyr_taps,pyr_r,nb_octaves,nb_scales,o_first,seed_sigma); its detections are unchanged, bit for bit (pinned by a test). AsiftPlan's type parameters and fields (AsiftPlan{D, A2, A3, A4, AI, AU, AU8, GT, KN, SP}, was{D, A1, A2, …, KN}: the blur taps'A1is gone, the scale space'sSPis new; the 8 fields above are nowplan.space): not breaking: the moved fields were never documented API; owner ruling 2026-11-14.- New:
detect_sfop(img; noise_sigma, significance_level, precision_threshold, …), exported: SFOP scale selection (Förstner, Dickscheid & Schindler 2009) for circular points (α = 90°) on that scale space — the precision(N(σ) − 2)·λ₂/Ωper level, its 3-D maxima (ASIFT's extraction), the λ₂ significance test derived from the image noise (no empirical factor),T_wfromprecision_threshold, ASIFT's elliptical NMS. RowsDetection(Disc(x, σ), (; precision)). - New:
detect_foerstner(img, seeds): the Förstner window estimation at given windows (each seed's centre and scale σ, the box window ofintegration_sigma = σ), the same rows asdetect_foerstner(img); at the windowsdetect_foerstner(img)chose it returns its rows bit for bit. - Measured limitation, accepted (owner, 2026-11-14): at a dot radius of about 1.8 px the window SFOP selects is 0.0008 px RMS worse than the best fixed window (0.0092 vs 0.0084 px, synthetic discs, σi 1.41). Three window rules were tested and failed (the smallest a-posteriori variance, a window ∝ the radius, Gaussian weighting) and Farid–Simoncelli derivatives had no effect: the limit is the window estimator's pixel locking. Stage 2 keeps the selected window.
detect_foerstneris faster (results unchanged, bit for bit): the squared gradient stays on the device and the window centres are compacted there (12 MP photograph on a B200: 587 → 28 ms; on 32 CPU threads 1.2 → 0.42 s).
A breaking release of the wave after RC 0.8.0, on VisualGeometryToolkitCore 0.10 ([compat] "0.10.0").
detect_foerstner's rows carrycovariance_status(their attributes are(; label, window, F, covariance_status)): how the frame's covariance was obtained or why there is none —:propagated,:propagated_measured/:propagated_estimated/:propagated_specified,:a_posteriori, or:unclassifiedon an:unknownrow (a NaN covariance). The producer states its covariance policy; no consumer substitutes one.detect_foerstner(...; noise): the covariance propagated from the image noise. Withnoise = σn(iid pixel noise, grey levels of an 8-bit image) the covariance isσn² J Jᵀ,Jthe exact first-order (implicit-function) derivative of the estimate with respect to every input pixel, through the detector's own doubling, derivative taps, squared gradient and window estimate, exact at the image borders; one KernelAbstractions kernel on the CPU and any GPU. Its Monte Carlo calibration (NEES₂ ≈ 2, χ²₂ 95 % coverage) is a test.noise = nothing(the default) keeps the reference's a-posterioriΩ/redundancy · N⁻¹, marked:a_posteriori; it is not calibrated (Förstner study: NEES 0.01–25).noise_sigma, the precision test's level, defaults tonoisewhen that is given (else the reference's 2).- The sensor model (VisualGeometryToolkitCore 0.10's
SensorModel, owner rulings A and D1–D13):load_raw(...; sensor_model)hands the frame's model to Core'sLinearRaw, the one black subtraction (its black, its defects, and under a Poisson–Gaussian or relative noise each value's variance); refused with the display demosaic. A cached decode is keyed by Core'scontent_digest(model); the absent model keeps the plain key.detect_foerstner(L::LinearRaw; sensor_model): fed only under a gain (Poisson–Gaussian):J diag(var) Jᵀfrom the uncertain image's variances; read noise alone, relative noise or none →:a_posteriori.variance_stabilize(L, sensor_model)(was the keywordselectrons_per_dn,read_noise, retired with a pointed refusal): the variance plane and Core'spoisson_slope; a model without a gain is anArgumentErrornaming the missing gain.
- The continuous refinement's LM stop settings are Core's (
StaticLmStops,[lm], the one owner): theContinuousRefinementkeywords and TOML keysmax_iter,converged_tol2andcost_floorare retired (a pointed refusal); success is Core's one rule (the point is stationary). CspondSetisCorrespondenceSet(VisualGeometryToolkitCore 0.10's terminology:Cspond→Correspondence,csponds→correspondences); the old name is refused by Core's naming check, with no alias.- A detection's
propagate(m, d)is Core's one conversion carried throughm,propagate(m, center(d)): the centre is now aPoint2(was anSVector{2}), the covariance the frame's position block as before. - A Förstner point's error ellipse is
Ellipse(d.value; sigma)(Core 0.10 deleted the detection forwarders; aPointFramedetection has no region, soEllipse(d)is a MethodError naming why). - Terminology (the owner's ruling of 2026-11-13): "keypoint" and "measurement" are out of every name, docstring
and page — a keypoint is a detection. Internal names renamed:
_keypoint_cov→_detection_cov,_loma_keypoints→_loma_points; the benchmarkbenchmark/mc/keypoint_uncertainty_mc.jl→benchmark/mc/detection_uncertainty_mc.jl; the test items "keypoint covariance: …" → "detection covariance: …".
onnx_run(session, inputs)/onnx_run(f, session), public: a run of anonnx_sessionwhose CUDA arena growth is measured around it and charged to the session, so the ASIFT detector's device plans leave exactly the room the arena may still take (was the private_onnx_run, LoMa's runs only). Thefform takes any call that runs the session (a device-tensorCAPI.Run), for BlobBoards' descriptor runs.
extract_capture_settingsno longer compiles per frame count (patch 0.8.4's fix, carried): a capture's identities merge field by field (StructArrays'components), not by splatting one tuple per frame intozip(a 171-photo set took 518 s, ~90 % LLVM). The other splats of a data-length collection go too:detect_foerstnercollects its rows,_score_eltypetakes a collection,compare_imageslinks its axes through Makie's vector method.detect_foerstner(Float64, img)computes inFloat64from the input on: aFloat64image was rounded toFloat32before the upload, so theFloat64path ran onFloat32grey values.- The continuous refinement stays inside the level (
refinement = ContinuousRefinement()): its LM box is also bounded by the level's extent, the detector's keep test; a point whose extremum lies beyond the level's edge endsLeftBasinwith its start, never outside the canvas. - A continuous refinement held on its box is
LeftBasin, the start returned (the refinement's basin policy): Core's stationarity rule counts a point a bound holds as stationary, so a solve whose extremum lies beyond the displacement bound or the level's edge would end Success at the box, a biased point; any point on the box is nowLeftBasinwith its start, whatever the LM's verdict (as it was when the box-held solve did not converge). detect_image_circlecheckssmoothing_weightbefore it reads a frame, with its other keywords.variance_stabilizereads Core'sgreen_plane(L): it no longer searches the planes for the green one by comparing frame maps with===.- The RAW decoder's fallbacks are Core's
require_extension: without the decoder's extension they name the package to load; with it loaded, a call it has no method for raises that MethodError (it said the extension was missing).
- The continuous refinement hands its scale to Core's
static_lm(itss), solving for the displacement, instead of rescaling the coordinates itself (the owner's no-duplication ruling): the damping floors are the same; the step stop reads the displacement in level units. - The detection covariance of a continuously refined point builds the refinement's field taps once per chunk, not per detection. A non-CPU backend builds each ASIFT kernel once (only the CPU has a static form).
- The Förstner propagation's one-axis pipeline columns are the adjoints ASIFT's covariance already has
(
_correlate_adj, the zero-padded correlation's, of which the symmetric blur's_blur_adjis now the special case, bit for bit;_up_adjwith an optional bounded edge).
A patch: a compile-time fix, no API change, no result changes.
- Fixed:
extract_capture_settingsno longer compiles per capture size. It merged the frames' body and lens identities by splatting one tuple per frame intozip, so each frame count compiled a zip of that arity, its LLVM time super-linear in it: a 171-photo set took 518 s, about 90 % of it compiling (decoding takes ~0.2 s a photo). The identities now merge field by field overDeviceIdentity's three fields, over the distinct identities in frame order (the first stated field kept, a disagreement refused, as before); tested at 500 frames after a warm-up at 300. - No other splat of a data-length collection:
detect_foerstnercollects its detections (was[rows...], one argument per detected point, per image), the stacked descriptor index finds its score type over a collection of galleries (was one argument per gallery), andcompare_imageslinks its axes through Makie's vector method. - Tests: the matcher items take their reference matrix on the CPU, and
pairwise_matrixon the default device is checked against the CPU within its derived rounding (17 Float32 ulps); the default device is a GPU once another item has loaded CUDA in the process.
A patch, on VisualGeometryToolkitCore 0.9.5 ([compat] VisualGeometryToolkitCore "0.9.5"). Additive; the detector's
default is unchanged.
-
Julia 1.13 (the owner's ruling of 2026-11-13: the whole stack requires 1.13):
[compat] julia = "1.13"in the package and in every environment of its workspace; CI runs 1.13 only. -
Every
[compat]floor raised to the gated resolution (the owner's ruling of 2026-11-13): a stack dependency at its registry latest or its in-wave version, a third-party one at the version the release gate resolved (a floor only, the semver upper bound kept); the monorepo's Downgrade workflow is deleted. -
New:
ContinuousRefinement, exported, andasift_detect(img; refinement = ContinuousRefinement())(alsoasift_detect!): an opt-in second refinement of every detection on a continuous model of its DoG response — the spatial gradient from Farid and Simoncelli's matched prefilter and first-derivative filters (IEEE TIP 2004), the scale row from the cardinal cubic spline of the unfiltered slices, Lagrange in scale — iterated by Core's device LM (static_lm) in a KernelAbstractions kernel on the CPU and any GPU, from the one-step refinement's point. The same candidates; a keypoint whose continuous refinement fails keeps the one-step point. Withuncertainty = FeatureCovariance()the refined keypoints' covariances are recomputed at the refined points from the model's own derivatives. Measured on noise-free synthetic blobs (benchmark/continuous_refine): the largest centre bias over a sub-pixel period falls from 0.02–0.14 to 0.0001–0.0005 level pixels, the one-step refinement's half-sample jump is gone, the scale error is the plain spline's (≤ 0.0041 in ln σ at octaves 1–3), and the covariance's NEES₂ is 1.94–2.09. Its bounds and tolerances: configs/continuous_refinement.toml. The defaultrefinement = nothingis the detector of 0.8.2. -
The one-step refinement's emission and the continuous one's share one map (
_level_to_canvas,_slice_sigma): the same arithmetic, the same bits for_extract_at!. -
The spline window sweep, the interpolation prefilter, the weight derivatives and the positive-definiteness test are Core 0.9.5's (
spline_window_sums,cubic_bspline_prefilter_taps,cubic_bspline_weight_derivatives,is_positive_definite); nothing of them is restated here.
A patch: on VisualGeometryToolkitEstimation 0.10 (the weak dependency of detect_image_circle's fit: [compat]
VisualGeometryToolkitEstimation "0.10" in the package and test environments). No code change: Features never called
the SplineLens refine that Estimation 0.10.0 retired.
A patch, on VisualGeometryToolkitCore 0.9.3 ([compat] VisualGeometryToolkitCore 0.9.3).
detect_image_circle's extension checks are VisualGeometryToolkitCore'srequire_extension. The private_require_extensionis deleted. The message without the extension is unchanged. The hooks' fallbacks (_fit_image_circle,_l0_smooth!) now raise theMethodErrorof arguments that the loaded extension does not take, where they used to name the package to load even when it was loaded.
Breaking. The sensor image moves to VisualGeometryToolkitCore 0.9.2 ([compat] VisualGeometryToolkitCore 0.9.2), and
this package keeps only what needs LibRaw: load_raw_mosaic/LibRawDecoder (producing Core's RawMosaic), load_raw
and the display demosaic. The moved definitions are deleted here, and by the owner's ruling (2026-11-12) this package
re-exports NO name of Core's — the moved ones and its three older re-exports alike: each is exported by Core alone.
-
Breaking: no longer exported (or declared public) here — use VisualGeometryToolkitCore's (
using VisualGeometryToolkitCore, orVgtkCore.X):RawMosaic— useVisualGeometryToolkitCore.RawMosaic;LinearRaw— useVisualGeometryToolkitCore.LinearRaw;BayerCellFrame— useVisualGeometryToolkitCore.BayerCellFrame;raw_frame_map— useVisualGeometryToolkitCore.raw_frame_map;bayer_plane_map— useVisualGeometryToolkitCore.bayer_plane_map;green_cell_sites— useVisualGeometryToolkitCore.green_cell_sites;raw_coordinates— useVisualGeometryToolkitCore.raw_coordinates;raw_provenance— useVisualGeometryToolkitCore.raw_provenance;cfa_string— useVisualGeometryToolkitCore.cfa_string;photosite_colors— useVisualGeometryToolkitCore.photosite_colors;feature_frame— useVisualGeometryToolkitCore.feature_frame(theDetectionmethod is still added here);Cspond— useVisualGeometryToolkitCore.Cspond;csponds— useVisualGeometryToolkitCore.csponds(theNNMatchmethod is still added here).
-
Breaking: the private
_linearis gone: a photosite's linear value is Core'slinearview(m)[i, j]. -
New: the display demosaic of a mosaic in memory,
load_raw(mosaic::RawMosaic; mode = :demosaic, gain, white_balance): LibRaw's AHD on the mosaic's photosites (libraw_open_bayer, copied to the host), its colour phase and its black and white levels, so a mosaic corrected per photosite is displayed through the one demosaic (no second one). Bit for bit the demosaic of the same buffer opened by LibRaw (a test); a mosaic whose levels are not whole codes (LibRaw holds integers) is anArgumentError, and LibRaw's own reading of the buffer (phase, levels) is checked against the mosaic's.load_raw(mosaic; mode = :green_cells | :bin2)is Core'sLinearRaw(mosaic; mode, gain). -
variance_stabilizefinds the green-cell plane through Core's public frame maps (the plane whosebayer_plane_mapisraw_frame_map(L), the cell grid; the display image refused there) in place of the raw module's private accessors. -
CACHE_FORMAT4: a cachedLinearRawwas stored asVisualGeometryToolkitFeatures.LinearRaw; the type is Core's now, so earlier entries are misses (decoded again), never a load that fails. -
Adapt is no longer a dependency (it served the mosaic's device form, now Core's). VisualGeometryToolkitCore 0.9.2's dependency allow-list names VisualGeometryToolkitEstimation for this package (the weak dependency below).
-
New:
detect_image_circle([rng,] [T = Ellipse,] frames; smoothing_weight = 0, outlier_halfwidth, min_hole_area, border_margin = 5)(exported; moved from RadiometricCamera, the same signature and, on its tests, the same results bit for bit): the image circle of a lens measured from photographs — the frames' per-pixel maximum asN0f8, its dark MSERs (detect_mser,max_area = 1, duplicates kept), every region boundary less the holes smaller thanmin_hole_area(VisualGeometryToolkitCore'ssigned_area) and the border's vertices, their convex hull, and a RANSACEllipseorCirclethrough the hull's vertices off the border, scored by the Student-t + uniform marginal atoutlier_halfwidth. Returns(; circle, fit, contour, cut, rim_gap)ornothing. A RAW frame (aRawMosaic, orLinearRaw(mosaic; mode = :green_cells)) is measured on its green cells, with the keywords in green-cell pixels (a pipeline config's resolution), and only the result is mapped: its circle and contour come back in the raw's pixels through the cells'raw_frame_map, andfit's estimate with them, while itsrmsand residuals stay in green-cell pixels. A cell image on a device is read to the host (the result bit for bit the host's, a GPU test). -
New dependencies: DelaunayTriangulation (the convex hull) and Random. New weak dependencies and extensions: VisualGeometryToolkitEstimation (
VisualGeometryToolkitFeaturesEstimationExt: the fit,find_ellipse/find_circle; without itdetect_image_circleis anArgumentErrornaming the package) and ImageSmooth (VisualGeometryToolkitFeaturesImageSmoothExt: the optional L0 smoothing,smoothing_weight > 0).
-
New:
propagate(m, d::Detection{<:Uncertain}): the uncertain centre of a detection through a map (its centre and the position block of its frame covariance, carried by VisualGeometryToolkitCore'spropagate; moved from PanoramaStitching). -
Device buffers are released through VisualGeometryToolkitCore's
free_device_buffers!(an evicted ASIFT plan's, the matcher's score blocks), no longer KernelAbstractions'unsafe_free!, which CUDA.jl does not extend: on a GPU they were not released until the collector ran. Now they are, at once. -
ASIFT with
FeatureCovariancedrops a detection whose covariance is singular or NaN (the owner's ruling, 2026-11-12), exactly as it drops a failed sub-pixel refinement, before the cross-warp NMS, and counts it inplan.refine_dropped. Every returned keypoint's covariance is finite and positive definite. Withoutuncertaintynothing changes. -
Internal: every kernel is instantiated through VisualGeometryToolkitCore's one launch policy,
OnBackend(device; workgroupsize)(kernel)(the owner's ruling, 2026-11-12): the ASIFT, cross-warp NMS, matching, Förstner, SSC and RAW cell kernels, each at the workgroup it had (_launch!keeps only the synchronize). Results and timings unchanged. -
Breaking:
pairwise_matrixandnn_matchdefault todefault_device()(the stack's GPU-first rule):pairwise_matrix(metric, d1, d2, device = default_device())andnn_match(…; device = default_device())run on an accelerator when one is functional, else onKA.CPU()as before.nn_match(…; device = nothing)is aTypeError: give the backend, or leave the keyword out.matchstays the host call. -
Breaking:
RawMosaic{A} <: AbstractMatrix{UInt16}is its photosite matrix (the owner's raw-image decision; BayeredImages' idea, no dependency on it).size, indexing,IndexStyleandparentare the held array's (m.raw, anyAbstractMatrix{UInt16}, a device array included), so an array or image function reads a mosaic unchanged; what one makes (similar,copy, a broadcast, a reduction) is a plain array of the held array's kind. A mosaic is now==to its photosite matrix, iterates over its photosites and prints one summary line. The type parameters areRawMosaic{A, M}(wasRawMosaic{M}).copy,==,isequalandisapproxare the held array's own (a device mosaic'scopy(m)is a device array, its comparison no scalar read). Equality is the photosite matrix's, by design: two mosaics (or windows) with the same photosites and another colour phase, window origin or metadata are==(and hash alike); comparephotosite_colors,window_originandmetadatafor the rest. -
load_raw's cache key includes the decoder library's version (libraw_version()forLibRawDecoder, the_decoder_version(decoder)hook;nothingfor a decoder that states none), so after a LibRaw_jll upgrade a cached decode (its black levels, white level, crop) is a miss, not the old library's values. -
New: windows of a mosaic.
m[rows, columns](unit ranges, a copy) andview(m, rows, columns)(no copy) are mosaics of the same sensor: the new fieldwindow_origin(the constructor's keyword,(0, 0)as decoded) is the window's offset into the visible area, so its colours are the parent's at those photosites for either parity of its start, andraw_frame_map(m::RawMosaic)(new) is that offset. ALinearRawmade from a window maps onto the parent's photosites exactly (itsraw_frame_mapandbayer_plane_mapinclude the offset). -
New:
raw_coordinates(img, x), THE accessor of an image's sensor-pixel coordinates:raw_frame_map(img)(x), in the visible raw's COLMAP frame — the identity on a decoded mosaic,x ↦ 2xon the cell grid of aLinearRaw; the display demosaic is refused, a plain matrix has no method. A detection on the green cells reports through it (the half-scale bug of writing cell positions as raw pixels). The design note: the guide's "Raw images" page. -
New: a mosaic on the device.
adapt(CuArray, m)oradapt(backend, m)(Adapt, a new dependency; a KernelAbstractions backend) move the photosites and keep the mosaic;LinearRaw(m; mode)runs its cell kernel onparent(m)'s device and leaves the image and the saturation flags there. Breaking:LinearRaw{Mode, I, S, G, M}holdsimage::AbstractArray{Float64, 3}andsaturated::AbstractMatrix{Bool}(anArrayand aBitMatrixon the host, as before);CACHE_FORMATis 3, so decodes cached under the earlier layout are a miss. -
Internal: one launch path for the ASIFT kernels on every backend (the owner's one-kernel policy; Core's
check_duplicationflagged the_scheduledmethod pair): every backend's kernels launch through_ScheduledKernel(formerly_CPUKernel), built on the device's static form (KernelAbstractions'CPU(static = true)on the CPU, the device itself elsewhere) and on the device; the CPU's static/dynamic launches are unchanged._chunk_warps' backend test is a free-memory reserve (an allocation, the same kernels on every backend): a DUPLICATION.toml[[backend]]entry with that reason. -
New:
onnx_session(path; execution_provider = :auto, threads, gpu_mem_limit), public: the package's ONNX Runtime session builder (formerly the internal_onnx_session), for another package's network: the provider resolved as LoMa's is (resolve_execution_provider; on CUDA, conda-forge's ONNX Runtime build, which runs on the B200's sm_100), ONNX Runtime's intra-op pool the caller'sthreads, and on CUDA the session's arena capped at the keywordgpu_mem_limitbytes (default 8 GiB, LoMa's). The detector's plans now leave free the room the live CUDA sessions' arenas may still take, each session's cap less what its arena already holds (measured at its creation and around LoMa's runs; the free memory the plans start from already excludes it), rather than 8 GiB per session ever created. A session released or collected reserves nothing, and the CUDA probe's one-op session (built byonnx_cuda_status, so by any:autoor:cudaresolution) reserves nothing and is released at once: before, the reserve only grew, the probe alone added 8 GiB, and a descriptor and a matcher on a 24 GB device left the detector one warp per chunk (slower ASIFT, the same detections). BlobBoards' descriptor network moves onto it. -
Breaking:
asift_detect's detection keywordmaskisexclusion_mask(the image and theStabilizedImagemethods; the semantics are unchanged: a marked pixel excludes every detection whose centre lies within three of its scales, and aStabilizedImage's saturation joins it). A call withmaskis anArgumentErrornamingexclusion_mask.gaussian_blur(img, σ, mask)'s positional weight map is another thing and keeps its name. -
Faster:
asift_detect(…; exclusion_mask)decides each detection's clearance from a summed-area table of the mask, read over the detection's disc, instead of a full-frame Euclidean distance transform per call: the same decisions (the same distance rule), ~6× faster and ~6.6× less memory on a 24 MP frame (also everyasift_detect(::StabilizedImage), whose saturation is its mask). -
Changed:
StabilizedImage(image, saturated, white)holds aMatrix{Float64}image and aBitMatrixmask as given (another element type is converted, once):variance_stabilizeno longer copies the transformed frame and the saturation mask a second time. -
Breaking: a
LinearRaw's mode is its type,LinearRaw{:green_cells},LinearRaw{:bin2}orLinearRaw{:demosaic}, and its sample geometry one field. The fieldsplane_maps,green_offsetsandgreen_dnareL.geometry = (; plane_maps, green_offsets, green_dn),nothingfor the display demosaic (and only for it); themodefield is gone (L isa LinearRaw{:bin2}, orraw_provenance(L).mode). The geometric accessors (raw_frame_map,bayer_plane_map,green_cell_sites,variance_stabilize) are methods of the cell images, the display image's their refusals (the sameArgumentError), with no run-time mode test; the green-cell plane is chosen by dispatch. -
New:
load_raw(path; …, cache), the decoded-RAW cache: withcacheaFileCache(open_cache, the JLD2 extension), each RAW is decoded once per content and settings — the entry is keyed by the file's SHA-256, the mode, the gain, the white balance and the decoder's type (cache_key) — and a hit returns theLinearRawas decoded, value for value. One cache for the stack's one RAW path: PanoramaStitching's example kept its own (code-sum PNGs and a JSON index,decoded_frames), which goes for it.cache = nothing(the default) decodes every call. -
Fixed: the stabilized image's noise threshold (
asift_detect(::StabilizedImage; noise_sigmas)) read the noise σ of every DoG slice, the two boundary slices included, while the extremum search reads only the interior ones (2 … NS − 1, since the interior-slice search): the threshold was set by a slice no extremum comes from, so too high, and faint extrema were dropped. It is the noisiest searched slice's σ now: a lower threshold, more detections at the samenoise_sigmas. -
Changed: LoMa on ONNX Runtime's CUDA provider no longer releases the ASIFT detector's plans at every call (a full GC, the pool reclaimed and the plan rebuilt at the next detection). Each CUDA session's arena is capped (
gpu_mem_limit8 GiB, grown by request), and the detector's plans size their resident warps from the free device memory less that cap for every CUDA session of the process; the plans cached before a session is created are released once, then. -
Internal: Core's device queries. The chunk sizing's
device_free_bytesand the plan release's pool reclaim are Core's (device_free_bytes,reclaim_device!, answered by its CUDA extension); the CUDA 12 cuDNN probe reads its version with the module's one_library_version. -
Faster:
load_raw(...; mode = :demosaic)through LibRaw copies the visible photosites once, into the display image: its checks of the mosaic's layout no longer copy the mosaic too. -
New: the stack's one image viewer,
compare_images(entries; titles, default, size)(exported; the Makie extension): a menu over named entries, each a tuple of images (or a function returning one, loaded when chosen), one axis per image, the axes linked, each image drawn by Core's orientation rule (imshow!). PanoramaStitching'scompare_makie.jlkeeps only which renders form a pair. -
New: the one display encoding,
srgb_image([T = N0f8], img; exposure, white_balance, background, tone)(exported), andlinear_to_srgb(public), the IEC 61966-2-1 OETF besidesrgb_to_linear. A linearH × W × 3image (or one plane) toRGB{T}(Gray{T}): exposure, a white-balance factor per plane, an optional tone curve (default none), clamp, sRGB,N0f8orN0f16; a NaN value is thebackground. A broadcast, so a device image is encoded on the device. The copies of the encoder in PanoramaStitching'ssave_srgb, RadiometricCamera'swrite_pngsandwrite_decoded, and BlobBoards' hero script become calls to it. -
Breaking: each plane of a Bayer image is a frame-labelled map onto the raw,
bayer_plane_map(L, c); the bare offsets are gone.BayerCellFrameis nowBayerCellFrame{C}, the frame of planeC(:R,:G,:B): the planes are indexed alike but sampled at different points of the cell.bayer_plane_map(L, c)(exported) is planec's CoreSimilarityMap{…, BayerCellFrame{C}, PixelFrame{:colmap}}, its translation onto the cell grid composed withraw_frame_map(L), so the ×2 is stated once; it sends every sample onto the centre of its photosites exactly, for all four phases and both origin parities. The green plane's map ISraw_frame_map(L), whose source is nowBayerCellFrame{:G}(the cell grid). A warp reads planecthroughinv(bayer_plane_map(L, c)) ∘ math, like any other map, all planes in one call through Core'simage_warp!(dst, warps, src); a composition that joins two planes' frames is refused (MethodError). Removed: theLinearRawfieldsplane_offsets(a plane's offset istranslation(inv(raw_frame_map(L)) ∘ bayer_plane_map(L, c))) andframe_map(raw_frame_map(L)); the new fieldplane_mapsholds the maps.raw_provenance(L).frame_map.fromreads"BayerCellFrame{:G}"(was"BayerCellFrame").plane_offsetsand PanoramaStitching'sraw_scaleare retired names. -
ASIFT's result no longer depends on the threads: the same image gives the same keypoints, in the same order, run to run. The extrema pool fills through an atomic counter, in the order the threads (or a GPU's warps) reach it, and the cross-warp NMS breaks ties in its quantized response by pool index, so on many threads both the order of the detections and, at a tie, which of two survived varied between runs (a panorama's frames detected twice reproduced on 12 of 38 frames with
spread_strongestafter, 36 of 38 with a top-n cut, in a 32-thread session). The pool is now sorted into a canonical order before the NMS (by position, scale and response, then every other field, the covariance's included: a total order), on the host, and the detections come back in it.cross_warp_nmssorts its input the same way, so its survivors no longer depend on the order of the pool handed in, and on the poolasift_detect(…; nms = :none)returns it keeps exactly what the in-pipeline NMS keeps (no longer "up to ties").spread_strongestwas already a function of its input's order; given the canonical detections it is reproducible too. Breaking (order): a caller that relied on the pool's warp-by-warp order now gets the canonical order. -
New: an exclusion mask,
asift_detect(img; exclusion_mask)— an H×WBoolmatrix,truewhere detection is not allowed (clipped highlights, a vignetted rim): a detection whose centre lies within three of its scales (the DoG's support) of a masked pixel is dropped, on Detections,rawfeatures and covariances alike. There was no way to exclude clipped regions of an ordinary (tone-mapped) image, so keypoints landed on clipped window panes. The rule is the oneasift_detect(::StabilizedImage)applied to sensor saturation, now in one place: that method maskss.saturatedthroughexclusion_mask(joined with a caller'sexclusion_mask), and takesraw = true, which it refused while the masking read Detections. -
New:
tilt_schedule(name), ASIFT's tilts by name;asift_detect(img; tilts = :untilted)restricts ASIFT to the untilted view (t = 1).:imas25(the default ofasift_detectandAsiftPlan) is IMAS-25;:untiltedis the identity warp alone, plain DoG-SIFT, exactly the identity warp's detections under:imas25. On real panorama frames a tilted warp's keypoints localize at about 0.95 px rms, the untilted view's at about 0.3 px, and the untilted schedule gave the best seams.tiltsstill takes a vector of(t, φ)pairs. Breaking:tilts = nothing(the old spelling of the default) is gone: give:imas25or omit it. The private IMAS-25 helper is retired (tilt_schedule(:imas25)). -
Every ASIFT detection is refined; an extremum whose refinement fails is dropped, never returned on the octave's grid. The extrema were searched on all S + 2 DoG slices of an octave, the two boundary slices included, whose extrema have no slice above or below and were emitted unrefined (integer positions on the octave's grid, 2^octave input pixels;
NaNcovariance), and an extremum whose sub-pixel refinement failed (a singular or non-finite step, a walk off the interior) was emitted unrefined too. On a Tamron panorama frame 1451 of the 3077 detections (47 %) were unrefined: 1382 from boundary slices, 69 failed refinements. Now only the interior slices are searched, as Lowe's SIFT does (the boundary slices' scales are the adjacent octaves' interior slices), and a failed refinement is dropped and counted (plan.refine_dropped). The same frame gives 2099 detections, all refined. On a synthetic field of blobs at known sub-pixel centres (IMAS-25, every pooled detection): rms error 0.52 → 0.17 px (octave −1 0.46 → 0.08, octave 0 0.60 → 0.28, octave 1 0.65 → 0.39); two renders 1.4° apart, the matched detections' rms 0.30 → 0.16 px. The tilted views' back-projection carries no bias (every warp's mean error below 0.04 px). -
clear_asift_plans!()returns the memory to the device (CUDA.jl's pool is reclaimed), and aLoMaDescriptororLoMaMatcheron ONNX Runtime's CUDA provider calls it before it runs: the cached plan (most of the device, sized from what was free) stayed resident after detection, and ONNX Runtime, which allocates outside CUDA.jl's pool, then failed to allocate in a long-lived process. The next detection rebuilds its plan (+0.04 s on a 2128 × 1424 frame on the RTX 4090; the release itself 0.3 s: a full collection, sinceunsafe_free!frees a buffer only once the arrays derived from it during a detection are collected, and they held a 10.6 GiB plan in use until then). A plan whose pyramid buffers no longer fit (another allocator grew after the chunk was sized from the free memory) halves its resident warps and retries instead of failing; an explicitchunkis never shrunk. -
New:
variance_stabilize(L::LinearRaw; electrons_per_dn, read_noise) -> StabilizedImage, the detection image of a RAW. The generalized Anscombe transform of the green cells (:green_cells, or:bin2's G plane) for Poisson–Gaussian noise, from the photosites' own code spans (white − blackof each green), the gain, the conversion gain (electrons per code) and the read noise (codes): after it the noise has σ ≈ 1 from a few electrons to the white level. Neither noise parameter is in a RAW's metadata (LibRaw reports neither): they are stated per camera and ISO.LinearRawgains the fieldgreen_dn(the two greens'white − black;nothingfor:demosaic). -
New:
asift_detect(s::StabilizedImage; noise_sigmas = 3.0, kwargs...): the DoG contrast threshold in noise σ. The DoG's noise σ for unit iid input is derived from the pyramid's own impulse response (the root energy per sample phase) in its noisiest slice, the finest octave's finest (0.137 for the default upsampled pyramid, seed blur 1.6): every coarser slice and tilted view is held to more thannoise_sigmasof its own σ. A detection whose centre lies within three of its scales of a saturated pixel is dropped (masked, not clamped).intensity_thrstays the threshold ofasift_detecton an arbitrary image, which has no noise model. -
New:
spread_strongest(detections, image_size, n; tolerance = 0.1, device), aboutnof the strongest detections spread over the image — Suppression via Square Covering (SSC) of Bailo et al. (Pattern Recognition Letters 106, 2018), ported from the authors' code (https://github.com/BAILOOL/ANMS-Codes, MIT License, Copyright (c) 2018 Oleksandr Bailo; credited in the docstring and the README's Citing section) and parallelised on KernelAbstractions: one implementation on the CPU and CUDA. The sequential covering is computed as per-cell maxima (two atomic reductions, no sort) and synchronous rounds of the greedy independent set of the cells in score order, and returns the authors' sequential result exactly, ties included (test/data/ssc_bailo.toml is their C++'s output). Where the authors' integer width search misses the tolerance band, the width is bisected between the bracketing integers, so the count lands in the band. It replacesstrongest(removed), the global topnby|response|, which on a frame with a bright window and a dim interior kept nearly every detection in the window;LoMaDescriptornow describesspread_strongest(detections, size(img), max_detections). A caller ofstrongest(d, n)callsspread_strongest(d, image_size, n). -
LoMa on CUDA runs on the B200 (sm_100) too. Microsoft's GPU builds of ONNX Runtime (1.30 and 1.31, CUDA 12 and 13, and the CUDA plugin EP) carry device code for sm_60 to sm_90 and sm_120 only, no sm_100 and no PTX, so on a B200 every kernel failed (
cudaErrorNoKernelImageForDevice) and:autofell back to the CPU. The CUDA path now runs conda-forge's build of ONNX Runtime 1.31.1 (onnxruntime-cpp,cuda129; sm_70 to sm_120 with sm_100), merged with the libraries it links (libprotobuf, libabseil, libre2) into one artifact,onnxruntime_cudain the newArtifacts.toml, pinned by its tree hash and installed by the first CUDA probe from the four conda-forge files it pins by sha256 (Pkg cannot unpack a.conda; Julia's own 7z does), 0.34 GB once per depot; ONNXRunTime's own GPU download is no longer used. The CUDA 12 runtime and cuDNN stay the JLLs' (the runtime now at least 12.9, the build's floor;libcusparse.so.12is loaded too). The CPU path is unchanged (ONNXRunTime's Microsoft 1.30 build).loma_provenancerecordsonnxruntime_buildon CUDA. New dependencies: the stdlibsTarandp7zip_jll. -
LoMa runs on ONNX Runtime's CUDA provider wherever it can, with no CUDA setup.
execution_provider = :autois the new default ofLoMaDescriptorandLoMaMatcher(was:cpu): CUDA when a CUDA device is visible and the active environment holdsCUDA_Runtime_jllandCUDNN_jll, otherwise the CPU with a warning naming why.:cpuand:cudastay explicit; an explicit:cudathat cannot run is an error naming why. ONNX Runtime's GPU build needs CUDA's 12.x runtime and cuDNN 9, which CUDA.jl 5.11 (runtime 13 on a CUDA 13 driver) and cuDNN.jl (1.4 does not resolve beside CUDA_Runtime_Discovery 2) could not give it: the package now takes the CUDA 12 builds of CUDA_Runtime_jll's and CUDNN_jll's artifacts (lazy: downloaded on the first CUDA session), selected by theircudatag and loaded for ONNX Runtime alone, beside whatever runtime CUDA.jl uses. The two JLLs are weak dependencies pinned by[compat](CUDA_Runtime_jll0.21 - 0.26,CUDNN_jll9.20 - 9) and never loaded as modules (CUDNN_jll would load the host CUDA's build under the same soname); an environment without them never resolves or downloads them. Neither CUDA.jl nor cuDNN.jl is needed for LoMa on the GPU any more; a cuDNN or CUDA 12 library already loaded from another file is refused by name. Needs a driver for CUDA ≥ 12 (≥ 525.60; Blackwell needs ≥ 570). -
New (public):
onnx_cuda_status()(anOnnxCudaStatus: available, or the reason why not; the driver, CUDA runtime and cuDNN versions) andresolve_execution_provider(execution_provider),:auto's answer, for a caller that keys a cache on the provider. -
ONNX Runtime's threads are the process's, not the machine's. A new
threadskeyword of both constructors (defaultThreads.nthreads(), the stack's measure of the allotment: a Slurm job starts Julia with--threads=$SLURM_CPUS_PER_TASK) sizes ONNX Runtime's intra-op pool; the inter-op pool is 1 (the graphs run sequentially). ONNX Runtime sized its pool from all cores and pinned each thread to one, which inside a Slurm cgroup oversubscribed the allotment (six 28-CPU jobs drove a 112-thread machine to a load of 673) and failed to pin (pthread_setaffinity_np failed); an explicit size disables the pinning. -
New (public):
loma_provenance(l)of a descriptor or matcher: the model file and its sha256, the provider used, the threads, ONNX Runtime's version and, on CUDA, the driver, runtime and cuDNN versions. -
ONNXRunTimecompat is~2.1: the sessions are built through its C API (itsload_inferencetakes no session options), so a new minor version is checked before it is admitted. -
Measured 2026-11-10 (RTX 4090, 32 threads, two 2128 × 1424 panorama frames, 2048 detections each,
:b+:b128): both descriptions and the match in 0.11 s on CUDA against 3.46 s on the CPU. CUDA against the CPU on six frames: descriptors within 3.2e-3 of their largest component, 1103 of the 1109 matches of 15 frame pairs identical. -
The one RAW path. Every RAW → image step of the stack lives in this module (
load_raw,load_raw_mosaic,LinearRaw; the LibRaw_jll extension decodes), and VisualGeometryToolkitCore'scheck_raw_decodingflags any other decoding path (rawpy, LibRaw C calls, dcraw, a demosaic or a CFA interpolation, a decoded RAW re-read as a PNG); the quality item runs it on this package. -
New:
mode = :green_cells, THE image for geometry and radiometry: each 2×2 Bayer cell's two green photosites' mean, half resolution, one plane::bin2's green plane, bit for bit, from the same kernel. A cell is saturated when either green is. -
New: the exact frame map of the cell grid.
BayerCellFrame(the cell images' COLMAP frame) andraw_frame_map(L), a CoreSimilarityMap{2, …, BayerCellFrame, PixelFrame{:colmap}},x ↦ 2xonto the visible raw for every Bayer phase and visible-origin parity (an odd margin changes only the colour phase).invis its inverse. A calibration matrix moves with it through Core'sreframed_calibration(raw_frame_map(L), K). -
New:
green_cell_sites(L, row, column), a green-cell value's footprint: its two diagonal green photosites' raw COLMAP centres, at(±½, ∓½)from the cell centre in an RGGB or BGGR cell and(±½, ±½)in GRBG or GBRG (LinearRaw.green_offsets). -
New:
mode = :demosaic, for display figures only: LibRaw's AHD demosaic (libraw_dcraw_process, linear: gamma 1, no auto-brightening, no maximum adjustment, 16 bits, raw colour, white balance 1) of the visible area, with an optionalwhite_balanceand the gain applied in Julia. It carries no frame map:raw_frame_mapandgreen_cell_sitesrefuse it, and itsplane_offsetsis empty (no pixel is a sample). Throughload_raw(path; mode = :demosaic)only. -
The EXIF flip is never applied, in any mode (the demosaic zeroes LibRaw's
flipbefore it copies the image out); a demosaic whose output is not the visible area's size is refused. -
New: provenance.
LinearRawcarries itsframe_map;raw_provenance(L)(public) gives the decoder, LibRaw's version, the mode, the gain and the frame map as plain data. -
Breaking: the raw-grid
:greenmode is removed (the green photosites at full resolution, zeros at red and blue): read:green_cells, or the mosaic itself. With itLinearRaw.sampledis removed (every pixel of every remaining mode carries its sample). -
Breaking:
RawpyDecoderand its PythonCall extension are removed, with the package'sCondaPkg.toml(rawpy, exifread) and the PythonCall weak dependency:LibRawDecoderis the one decoder. -
The two-axis
matches!(ax_A, ax_B, cs)andmatches(img_A, img_B, cs)take weighted csponds: a cspond is bare (opaque) orAttributed(src => tgt, (; weight)), its weight (Core'sweight_of, in [0, 1]) the alpha of its markers.
- On VisualGeometryToolkitCore 0.8 (compat
"0.8"); nothing else changes.
- The LoMa model digest and
cache_keyare VisualGeometryToolkitCore'ssha256_hex(the same values); the SHA dependency is dropped. - Breaking, with VisualGeometryToolkitCore 0.7: a
Detection's and aDescribedFeature's blob shape isshape_matrix(d)(Core's renamed generic;moment_matrixis retired).d.Σis unchanged. LoMaDescriptortakes an image of any size:(descriptor)(img, detections)withimga3 × H × WRGB channelview of any real element type and the detections in its COLMAP pixels. An image that is not the graph's fixedLOMA_IMAGE_SIZE² (public, 784) is resampled to it per channel by Core's bilinearimage_warp!(the COLMAP scaling, Float64 math, Float32 taps: PanoramaStitching's hand resampling, bit for bit); a3 × 784 × 784image is taken as before. An image smaller than 784 on either side (an upscale) is resampled with its border replicated: the network's outermost pixel centres map within half a pixel past the image's, and their taps clamp to the border pixel instead of blending in the fill. The keypoint chart reads the image's own size, so the detections are never rescaled and the described features carry them as given (PanoramaStitching rescaled its frames to 784² by hand, moment matrices included, which the descriptor never read).strongest(detections, n)(public), the selectionLoMaDescriptorapplies (formerly the private_strongest): thenlargest|response|, in detector order. Ties at the cut now keep the earlier detections (a stable partial selection, O(N) as the formerpartialsortperm, which left that choice unspecified). BlobBoards' cap keeps the same set but re-sorts it by response.box_downsampletakes a 2-D image and any real or colour element type, averaging in the element type's float (Float32for aFloat32,Gray{Float32}or 8-bit image: BlobBoards' supersampling filter, bit for bit;Float64for aFloat64or integer image, as before). AFloat323-D image now gives aFloat32result (formerlyFloat64).
VisualGeometryToolkitCore 0.6 (compat "0.6"). Breaking with Core's
CovMat, which now holds a static matrix (full, diagonal or isotropic).
- Detections' covariances are Core 0.6's
CovMat{2, T, M}; the spelled-outCovMat{N, T, N², F}types are gone. CACHE_FORMAT(2) is part of every cache key, so a JLD2 cache entry of the earlierCovMatlayout is a miss, not a failed load.
Additive: RawMosaic.metadata keeps its type and every field it had.
- The RAW's EXIF as Core's typed values:
exposure_settings,lens_settings,body_identity,lens_identityandcapture_time(public) of aRawMosaicor its metadata, andextract_capture_settings(frames; lens = nothing), a capture'sCaptureSettings(one body; a stated lens overrides EXIF's claim, with a warning when they disagree: an adapted lens is misnamed, the FX3 with Sigma EF lenses claims a Sony "DT" lens). load_raw_metadata(path; decoder): the EXIF fields alone; LibRaw reads the headers without unpacking the photosites.metadatagainsmake,body_serial,lens_make,lens_model,lens_serialandfocus_distance_m(nothingwhere absent; a non-finite or non-positive focus distance too, never a distance). LibRaw: the lens fromlibraw_get_lensinfo(its leading fields mirrored, the offset checked); no body serial or focus distance. rawpy: the EXIF tags.- Compat: VisualGeometryToolkitCore 0.5.4.
- New
load_raw(path; mode, gain = 1, decoder = LibRawDecoder()): a camera RAW file (any Bayer format LibRaw reads) as exact linear values,(raw − black[c]) / (white − black[c]) × gain, camera raw colour, white balance 1, in the sensor's orientation (no EXIF flip; the raw's COLMAP frame). Ported from RadiometricCamera'sconvert_raw.py. Modes:bin2(2×2 Bayer quads: R, the mean of the two greens, B; half size) and:green(the green photosites alone on the raw grid). Itsfullresdemosaic is not provided. Returns aLinearRaw:image,sampled,saturated(a quad is saturated when ANY of its photosites is at the white level,raw ≥ white, no margin),plane_offsets(each plane's true sample position: the green at the quad centre, R and B(±¼, ±¼)binned pixels from it) and the EXIFmetadata. Rounded to 16 bits,:bin2equalsconvert_raw.py --mode bin2's linear PNG value for value. - New
load_raw_mosaic,RawMosaic,LinearRaw(mosaic; mode, gain): the decode and the processing apart; the processing is all Julia, one KernelAbstractions kernel per mode. ARawMosaicholds the colourpattern(square, at most 4 × 4, validated) and the visible area's origin; each photosite's colour index isphotosite_colors(mosaic), the pattern tiled from the raw origin (rawpy'sraw_colors_visible, which the rawpy decoder checks against it). - Black levels: one rule. LibRaw's per-channel black levels with a 2×2
spatial pattern folded in; a spatial pattern with no per-channel form is
refused, naming the file, never dropped. rawpy exposes only the per-channel
levels (its
black_level_per_channeldrops such a pattern), soRawpyDecoder()cannot apply the rule; its docstring says so. - Two decoders, both weak dependencies:
LibRawDecoder()(new extensionVisualGeometryToolkitFeaturesLibRawExton LibRaw_jll 0.22.2: accalllayer whose mirroredlibraw_data_tlayout is checked against LibRaw's accessors on every file, refusing a LibRaw other than 0.22) andRawpyDecoder()(new extensionVisualGeometryToolkitFeaturesPythonExton PythonCall; rawpy and exifread declared in the newCondaPkg.toml, reading one open file object). Without the extension loaded,load_rawnames the package to load.