Update ISMIP7 processing to handle OCX forcing - #978
Conversation
Handle the OCX scenario in the ismip7_forcing test group. OCX has no distinct ESM model: it uses RACMO2.3p2-ERA (atmosphere) and EN4 (ocean) at data version v1, with a named grid resolution (ocean-1000m) in the ocean file names. get_params() now applies OCX-specific overrides (sources, versions, ocean grid token) when scenario == OCX, so the [ismip7] model option is ignored and a single config file drives both the atmosphere and ocean_thermal test cases. Adds ismip7_forcing_ocx_gis.cfg and updates the user and developer docs.
Two fixes that block processing of forcing whose remapped files or source
directories differ from the CESM test data:
- Drop the ncremap-mangled 'time_bnds' variable when combining remapped
files. ncremap collides the bounds 'nv' dimension with the destination
mesh corner dimension and pads with a fill value that overflows time
decoding. The steps build their own xtime and never use time_bnds.
- Skip input files whose trailing token is not a year (e.g. climatology
averages like *_1978-2007_avg.nc) instead of crashing on int('avg').
Restructure processed-forcing output from
{output_base_path}/atmosphere_forcing/{model}_{scenario}/ (and the analogous
ocean_thermal_forcing path) to {output_base_path}/{group}/atmosphere/ and
{output_base_path}/{group}/ocean_thermal_forcing/, matching the directory
layout the ismip7_run test group globs for. 'group' is {model}_{scenario} for
the ESM scenarios and {scenario} (i.e. OCX) for OCX, whose atmosphere and
ocean sources differ but must share one directory. Documents the layout and
how to point ismip7_run's forcing_basepath / ocx_forcing_path at it.
…lation helper process_smb.py, process_temperature.py, process_smb_gradient.py, and process_temperature_gradient.py remapped the raw source file directly with ncremap, unlike process_runoff.py, ocean_thermal, and the fracture pathway, which all extrapolate fill/missing values on the source grid first. Without that step, no-data cells near the ice-sheet margin (e.g. ocean cells in the RACMO OCX source) leak the ~9.97e36 netCDF fill sentinel into valid neighboring cells during conservative/bilinear remapping. Add the same pre-remap extrapolation to the four affected steps. Also promote the previously-duplicated extrapolate_source() (from fracture/remap_utils.py, formerly re-implemented again in ocean_thermal/process_thermal_forcing.py and process_runoff.py) to a single shared compass.landice.tests.ismip7_forcing.remap_utils module, and update all five call sites to use it instead of per-file private copies.
extrapolate_source moved from fracture.remap_utils to the shared compass.landice.tests.ismip7_forcing.remap_utils module (828699b). Update the api.rst autosummary entry accordingly; this was breaking the strict Sphinx build (autosummary failed to import the moved symbol).
Enable the OCX (reanalysis) scenario for the Antarctic ice sheet: - ice_sheet_params: add an 'ais' OCX override (RACMO atmosphere at v1, reanalysis ocean at v1 with no model token) and an ocean_choice_layout flag for the idiosyncratic AIS OCX ocean layout. - process_thermal_forcing: assemble a list of ocean forcing sources so AIS OCX can process one or more ocean choices (main/cold/warm/vary), each in its own OCX_<choice> output directory; other cases are unchanged. - ismip7_forcing.cfg: add ocean_choice option (AIS OCX only). - remap_utils: write the extrapolated source file as CDF-5 so large 3D ocean thermal forcing fields aren't limited by the NetCDF3 per-variable size cap or HDF5 chunk limits that break ncremap. - add ismip7_forcing_ocx_ais.cfg example config.
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical output-layout and NetCDF encoding issues, along with API and input-validation issues, block approval.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR adds OCX reanalysis support to ISMIP7 forcing processing, including AIS/GrIS source handling, missing-cell extrapolation, and updated output organization.
Changes:
- Adds OCX configurations and AIS ocean-choice handling.
- Centralizes extrapolation before remapping.
- Updates forcing layouts and user/developer documentation.
File summaries
| File | Change / final review notes |
|---|---|
docs/users_guide/landice/test_groups/ismip7_forcing.rst |
Documents OCX inputs and outputs. Nit (1 vote): AIS ocean and atmosphere outputs use inconsistent directories, so the documented ocx_forcing_path is not directly runnable. |
docs/developers_guide/landice/test_groups/ismip7_forcing.rst |
Updates developer guidance. Nits (1 vote each): Qualify EN4 as GrIS-only, document the AIS choice layout, and update the stale fracture helper reference. |
docs/developers_guide/landice/api.rst |
Registers the shared remapping helper. |
compass/landice/tests/ismip7_forcing/remap_utils.py |
Adds shared extrapolation logic. Critical (1 vote): Source encodings may be incompatible with NETCDF3_64BIT_DATA; clear or replace them before writing temporary files. |
compass/landice/tests/ismip7_forcing/ocean_thermal/process_thermal_forcing.py |
Handles OCX ocean sources and choices. Critical (3 votes): AIS ocean output is not co-located with atmosphere output and cannot be consumed directly by ismip7_run. Moderate (2 votes): Invalid tokens combined with all bypass validation. |
compass/landice/tests/ismip7_forcing/ismip7_forcing.cfg |
Adds ocean-choice configuration. |
compass/landice/tests/ismip7_forcing/ismip7_forcing_ocx_gis.cfg |
Adds a GrIS OCX configuration. |
compass/landice/tests/ismip7_forcing/ismip7_forcing_ocx_ais.cfg |
Adds an AIS OCX configuration. |
compass/landice/tests/ismip7_forcing/ice_sheet_params.py |
Adds OCX parameter overrides. Nit (1 vote): The module description incorrectly says all OCX runs use EN4; qualify this for GrIS and document AIS choices. |
compass/landice/tests/ismip7_forcing/fracture/remap_utils.py |
Removes duplicated extrapolation code. Moderate (1 vote): Removing the documented helper path can break downstream imports; retain a compatibility re-export or explicitly handle the breaking change. |
compass/landice/tests/ismip7_forcing/fracture/process_lake_properties.py |
Uses the shared extrapolation helper. |
compass/landice/tests/ismip7_forcing/fracture/process_excess_melt.py |
Uses the shared extrapolation helper. |
compass/landice/tests/ismip7_forcing/atmosphere/process_temperature.py |
Adds OCX handling and extrapolation. |
compass/landice/tests/ismip7_forcing/atmosphere/process_temperature_gradient.py |
Adds OCX handling and extrapolation. |
compass/landice/tests/ismip7_forcing/atmosphere/process_smb.py |
Adds OCX handling and extrapolation. |
compass/landice/tests/ismip7_forcing/atmosphere/process_smb_gradient.py |
Adds OCX handling and extrapolation. |
compass/landice/tests/ismip7_forcing/atmosphere/process_runoff.py |
Adds OCX handling and extrapolation. |
Review details
Suppressed comments (5)
compass/landice/tests/ismip7_forcing/fracture/remap_utils.py:5
- This removes
fracture.remap_utils.extrapolate_source, which was a documented API entry before this PR. Internal imports are updated, but downstream callers importing that path will now getImportError; please keep a compatibility re-export here or explicitly handle this as a breaking API change.
import xarray as xr
from mpas_tools.io import write_netcdf
compass/landice/tests/ismip7_forcing/ice_sheet_params.py:36
- The module-level description says every OCX run uses EN4 for the ocean, but the AIS override below deliberately sets
ocean_modeltoNoneand uses per-choice ocean products. Qualify EN4 to GrIS and document the AIS choice layout so this description matchesget_params().
# Overrides applied for the OCX (reanalysis) scenario. OCX has no distinct
# ESM model: it uses fixed reanalysis products (RACMO for the atmosphere and
# EN4 for the ocean) at data version v1. When scenario is 'OCX' the [ismip7]
# model option is ignored and these sources are used instead.
docs/developers_guide/landice/test_groups/ismip7_forcing.rst:35
- This paragraph says OCX always sets
ocean_model = EN4, but_OCX_OVERRIDES['ais']sets it toNoneand uses the choice-based ocean layout; EN4 is only the GrIS source. Qualify these source names by ice sheet and document the AIS choice layout here, otherwise developers may construct the wrong filename/path.
When ``scenario = OCX``, ``get_params`` applies a set of OCX overrides on top
of the ice-sheet defaults: data version ``v1``, the ocean file-name grid token
(e.g. ``ocean-1000m``), and the fixed reanalysis sources (``atm_model`` =
``RACMO2.3p2-ERA`` and ``ocean_model`` = ``EN4``). The processing steps use
docs/developers_guide/landice/test_groups/ismip7_forcing.rst:37
- The helper was moved to
compass.landice.tests.ismip7_forcing.remap_utils, but the fracture section later in this same guide still documentsextrapolate_sourceunderfracture.remap_utils(lines 175-177). Update that reference so the developer guide does not direct readers to a symbol that no longer exists.
they are set, so the OCX ``model`` option is ignored. This keeps OCX handling
docs/users_guide/landice/test_groups/ismip7_forcing.rst:100
- This layout description is not true for AIS OCX: the changed AIS branch writes ocean output under
OCX_<choice>while atmosphere output is underOCX. Consequently, directingocx_forcing_pathto{output_base_path}/OCXdoes not provide a runnable AIS forcing set. Update the documented OCX layout after aligning the producer and consumer.
The ``group`` directory is ``{model}_{scenario}`` for the ESM scenarios and
``{scenario}`` (i.e. ``OCX``) for OCX, whose atmosphere and ocean use
different sources (``RACMO2.3p2-ERA`` and ``EN4``) but must share one
directory. To feed :ref:`landice_ismip7_run`, point its ``forcing_basepath``
at ``output_base_path`` for ESM scenarios, or its ``ocx_forcing_path`` at
``{output_base_path}/OCX`` for OCX.
- Files reviewed: 17/17 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Ensure variables do not inherit encoding from source files. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Make atmosphere ocx output structure parallel to ocean forcing, using symlinks to main atmosphere output directory. This makes the output structure compatible with ismip7_run.
Update
ismip7_forcingtest group to handle OCX forcing, which has a slightly different directory structure from other forcings. Extrapolate forcings into missing data cells before interpolating to MALI mesh. Also unify the structure forismip7_forcingoutput with that expected as input forismip7_run.Checklist
Testingin this PR) any testing that was used to verify the changes