S1 GRD error on loading as DimTree

[felixcremer]

Opening the S1 GRD data as open_tree gives the following error:

v = "incidence_angle"
ERROR: ArgumentError: start and stop must be finite, got 0.0053354413248598576 and NaN
Stacktrace:
  [1] _linspace(start::Float64, stop::Float64, len::Int64)
    @ Base ./twiceprecision.jl:669
  [2] range_start_stop_length(start::Float64, stop::Float64, len::Int64)
    @ Base ./twiceprecision.jl:664
  [3] _range
    @ ./range.jl:167 [inlined]
  [4] range
    @ ./range.jl:150 [inlined]
  [5] _cdmspan
    @ ~/Documents/EOPF.jl/SentinelDataSource/dev/Rasters/src/sources/commondatamodel.jl:366 [inlined]
  [6] _cdmlookup(ds::ZarrDataset{…}, var::CommonDataModel.CFVariable{…}, attr::OrderedCollections.OrderedDict{…}, dimname::String, D::Type, index::Vector{…}, metadata::DimensionalData.Dimensions.Lookups.Metadata{…}, crs::Rasters.NoKW, mappedcrs::Rasters.NoKW)
    @ Rasters ~/Documents/EOPF.jl/SentinelDataSource/dev/Rasters/src/sources/commondatamodel.jl:305
  [7] _cdmlookup(ds::ZarrDataset{…}, dimname::String, D::Type, crs::Rasters.NoKW, mappedcrs::Rasters.NoKW)
    @ Rasters ~/Documents/EOPF.jl/SentinelDataSource/dev/Rasters/src/sources/commondatamodel.jl:283
  [8] _cdmdim(ds::ZarrDataset{…}, dimname::String, crs::Rasters.NoKW, mappedcrs::Rasters.NoKW)
    @ Rasters ~/Documents/EOPF.jl/SentinelDataSource/dev/Rasters/src/sources/commondatamodel.jl:231
  [9] (::Rasters.var"#1088#1089"{ZarrDatasets.ZarrVariable{…}, Rasters.NoKW, Rasters.NoKW, Tuple{…}})(i::Int64)
    @ Rasters ~/Documents/EOPF.jl/SentinelDataSource/dev/Rasters/src/sources/commondatamodel.jl:166
 [10] ntuple
    @ ./ntuple.jl:49 [inlined]
 [11] _dims(var::ZarrDatasets.ZarrVariable{Float32, 2, ZArray{…}, ZarrDataset{…}}, crs::Rasters.NoKW, mappedcrs::Rasters.NoKW)
    @ Rasters ~/Documents/EOPF.jl/SentinelDataSource/dev/Rasters/src/sources/commondatamodel.jl:167
 [12] (::Rasters.var"#77#78"{…})(var::ZarrDatasets.ZarrVariable{…})
    @ Rasters ~/Documents/EOPF.jl/SentinelDataSource/dev/Rasters/src/array.jl:384
 [13] _open(f::Rasters.var"#77#78"{…}, ::Rasters.Zarrsource, var::ZarrDatasets.ZarrVariable{…}; mod::Nothing, kw::@Kwargs{…})
    @ Rasters ~/Documents/EOPF.jl/SentinelDataSource/dev/Rasters/src/sources/commondatamodel.jl:94
 [14] _raster(ds::ZarrDatasets.ZarrVariable{…}; dims::Rasters.NoKW, refdims::Tuple{}, name::Rasters.NoKW, group::Rasters.NoKW, filename::String, metadata::Rasters.NoKW, missingval::Rasters.NoKW, crs::Rasters.NoKW, mappedcrs::Rasters.NoKW, source::Rasters.Zarrsource, replace_missing::Rasters.NoKW, coerce::typeof(convert), scaled::Rasters.NoKW, verbose::Bool, write::Bool, lazy::Bool, dropband::Bool, checkmem::Bool, raw::Bool, mod::Rasters.NoKW, f::typeof(identity))
    @ Rasters ~/Documents/EOPF.jl/SentinelDataSource/dev/Rasters/src/array.jl:365
 [15] _raster
    @ ~/Documents/EOPF.jl/SentinelDataSource/dev/Rasters/src/array.jl:334 [inlined]
 [16] #Raster#1080
    @ ~/Documents/EOPF.jl/SentinelDataSource/dev/Rasters/src/sources/commondatamodel.jl:72 [inlined]
 [17] open_tree(dataset::ZarrDataset{ZarrDataset{…}, Missing, ZGroup{…}})
    @ SentinelDataSource ~/Documents/EOPF.jl/SentinelDataSource/src/SentinelDataSource.jl:20
 [18] open_tree(dataset::ZarrDataset{ZarrDataset{…}, Missing, ZGroup{…}})
    @ SentinelDataSource ~/Documents/EOPF.jl/SentinelDataSource/src/SentinelDataSource.jl:24
 [19] open_tree(dataset::ZarrDataset{ZarrDataset{Nothing, Missing, ZGroup{…}}, Missing, ZGroup{Zarr.ConsolidatedStore{…}}})
    @ SentinelDataSource ~/Documents/EOPF.jl/SentinelDataSource/src/SentinelDataSource.jl:24
 [20] open_tree(dataset::ZarrDataset{Nothing, Missing, ZGroup{Zarr.ConsolidatedStore{Zarr.HTTPStore}}})
    @ SentinelDataSource ~/Documents/EOPF.jl/SentinelDataSource/src/SentinelDataSource.jl:24
 [21] top-level scope
    @ REPL[365]:1
Some type information was truncated. Use `show(err)` to see complete types.

[rafaqz]

NaN in lookups is undefined behaviour

[felixcremer]

Yeah most likely this is bad metadata

[felixcremer]

According to @konstntokas this is a metadata problem with a duplicated slant_range_time variable in the antenna_pattern group.

[konstntokas]

I created an issue to the CPM library (used for the conversion to Zarr): https://gitlab.eopf.copernicus.eu/cpm/eopf-cpm/-/issues/763

[konstntokas]

One test could be to download the Zarr product and rename (maybe with xarray) the dimension and see if it works. That would be proof that it is a format issue. @felixcremer

[konstntokas]

@felixcremer the made some changes to the CPM software; see https://gitlab.eopf.copernicus.eu/cpm/eopf-cpm/-/issues/763

Please try it out:

• check out the branch https://gitlab.eopf.copernicus.eu/cpm/eopf-cpm/-/tree/slc-noise-xml
• get an original Sentinel-1 GRD product from https://browser.dataspace.copernicus.eu (CDSE account is needed for the download)
• Convert the product to Zarr using the new branch of the cpm library.
• See if this resolves the reading issues in Julia.

If you have troubles converting the product, I could also have a look. If you think cpm makes problems, you can also jup into the discussion here: https://gitlab.eopf.copernicus.eu/cpm/eopf-cpm/-/issues/763 (CDSE account is needed)

[felixcremer]

@konstntokas thanks for providing the S1 data via email.

I tested the S1 data set and this is still erroring but I get a different error.

julia> s1path = "../data/S1A_IW_GRDH_1SDV_20250626T053421_20250626T053446_059813_076DB6_D372.zarr/"
"../data/S1A_IW_GRDH_1SDV_20250626T053421_20250626T053446_059813_076DB6_D372.zarr/"

julia> s1z = ZarrDataset(s1path);

julia> s1tree = SentinelDataSource.open_tree(s1z)
v = "line"
┌ Warning: cannot parse time units `nanoseconds since 2025-06-26 05:34:21.819152`: unknown units "nanoseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `nanoseconds since 2025-06-26 05:34:21.819152`: unknown units "nanoseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `nanoseconds since 2025-06-26 05:34:21.819152`: unknown units "nanoseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
v = "pixel"
v = "grd"
┌ Warning: cannot parse time units `nanoseconds since 2025-06-26 05:34:21.819152`: unknown units "nanoseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `nanoseconds since 2025-06-26 05:34:21.819152`: unknown units "nanoseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `nanoseconds since 2025-06-26 05:34:21.819152`: unknown units "nanoseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
v = "q0"
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
v = "pitch"
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
v = "roll"
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
v = "q2"
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
v = "yaw"
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
v = "wy"
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
v = "q1"
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
v = "wx"
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
v = "wz"
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
v = "q3"
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.750003`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
v = "swath"
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.371148`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.371148`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.371148`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
v = "incidence_angle"
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.371148`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.371148`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
┌ Warning: cannot parse time units `microseconds since 2025-06-26 05:34:22.371148`: unknown units "microseconds"
└ @ CommonDataModel ~/.julia/packages/CommonDataModel/82Ewz/src/cfvariable.jl:144
ERROR: DimensionMismatch: DimensionalData.Dimensions.Dim{:azimuth_time} and DimensionalData.Dimensions.Dim{:range} dims on the same axis.
Stacktrace:
  [1] _failed_comparedims(t::DimensionalData.Dimensions.Throw{Nothing}, msg_intro::String)
    @ DimensionalData.Dimensions ~/Documents/EOPF.jl/SentinelDataSource/dev/DimensionalData/src/Dimensions/primitives.jl:567
  [2] _dimsmismatchaction(err::DimensionalData.Dimensions.Throw{…}, a::DimensionalData.Dimensions.Dim{…}, b::DimensionalData.Dimensions.Dim{…})
    @ DimensionalData.Dimensions ~/Documents/EOPF.jl/SentinelDataSource/dev/DimensionalData/src/Dimensions/primitives.jl:560
  [3] #_comparedims2#89
    @ ~/Documents/EOPF.jl/SentinelDataSource/dev/DimensionalData/src/Dimensions/primitives.jl:607 [inlined]
  [4] _comparedims2
    @ ~/Documents/EOPF.jl/SentinelDataSource/dev/DimensionalData/src/Dimensions/primitives.jl:602 [inlined]
  [5] _comparedims2
    @ ~/Documents/EOPF.jl/SentinelDataSource/dev/DimensionalData/src/Dimensions/primitives.jl:592 [inlined]
  [6] macro expansion
    @ ~/Documents/EOPF.jl/SentinelDataSource/dev/DimensionalData/src/Dimensions/primitives.jl:0 [inlined]
  [7] _comparedims_gen(dt1::Tuple{…}, dts::Tuple{…}; kw::@Kwargs{…})
    @ DimensionalData.Dimensions ~/Documents/EOPF.jl/SentinelDataSource/dev/DimensionalData/src/Dimensions/primitives.jl:574
  [8] #_comparedims#69
    @ ~/Documents/EOPF.jl/SentinelDataSource/dev/DimensionalData/src/Dimensions/primitives.jl:572 [inlined]
  [9] comparedims(::Tuple{DimensionalData.Dimensions.Dim{…}}, ::Vararg{Any}; kw::@Kwargs{})
    @ DimensionalData.Dimensions ~/Documents/EOPF.jl/SentinelDataSource/dev/DimensionalData/src/Dimensions/primitives.jl:532
 [10] comparedims
    @ ~/Documents/EOPF.jl/SentinelDataSource/dev/DimensionalData/src/Dimensions/primitives.jl:532 [inlined]
 [11] _adddims!(dt::DimensionalData.DimTree, newdims::Tuple{…})
    @ DimensionalData ~/Documents/EOPF.jl/SentinelDataSource/dev/DimensionalData/src/tree/tree.jl:237
 [12] setindex!(dt::DimensionalData.DimTree, A::Rasters.Raster{…}, key::Symbol)
    @ DimensionalData ~/Documents/EOPF.jl/SentinelDataSource/dev/DimensionalData/src/tree/tree.jl:222
 [13] open_tree(dataset::ZarrDataset{ZarrDataset{…}, Missing, Zarr.ZGroup{…}})
    @ SentinelDataSource ~/Documents/EOPF.jl/SentinelDataSource/src/SentinelDataSource.jl:21
 [14] open_tree(dataset::ZarrDataset{ZarrDataset{…}, Missing, Zarr.ZGroup{…}})
    @ SentinelDataSource ~/Documents/EOPF.jl/SentinelDataSource/src/SentinelDataSource.jl:24
 [15] open_tree(dataset::ZarrDataset{ZarrDataset{…}, Missing, Zarr.ZGroup{…}})
    @ SentinelDataSource ~/Documents/EOPF.jl/SentinelDataSource/src/SentinelDataSource.jl:24
 [16] open_tree(dataset::ZarrDataset{Nothing, Missing, Zarr.ZGroup{Zarr.DirectoryStore}})
    @ SentinelDataSource ~/Documents/EOPF.jl/SentinelDataSource/src/SentinelDataSource.jl:24
 [17] top-level scope
    @ REPL[11]:1
Some type information was truncated. Use `show(err)` to see complete types.

[konstntokas]

This is how the new version looks as a xarray. Dataset.

[rafaqz]

@felixcremer guessing the dims may be added incorrectly here to get that mismatch

They need to be added per DimArray then the arrays are added to the tree separately?

It may be a bug in that _adddims! method, quite likely actually.

And the nanoseconds warnings may need fixing in CFTime.jl

[konstntokas]

@felixcremer do you think the branch in https://gitlab.eopf.copernicus.eu/cpm/eopf-cpm/-/issues/763 fixes the issue?

[konstntokas]

@felixcremer do you think the branch in https://gitlab.eopf.copernicus.eu/cpm/eopf-cpm/-/issues/763 fixes the issue?

@felixcremer please provide us with the current state of this issue, thanks.

[felixcremer]

This has been closed in the latest DimensionalData PR and I can open the data now as a DimTree.