Skip to content

Update ISMIP7 processing to handle OCX forcing - #978

Open
trhille wants to merge 8 commits into
MPAS-Dev:mainfrom
trhille:landice/ismip7_ocx_forcing
Open

Update ISMIP7 processing to handle OCX forcing#978
trhille wants to merge 8 commits into
MPAS-Dev:mainfrom
trhille:landice/ismip7_ocx_forcing

Conversation

@trhille

@trhille trhille commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Update ismip7_forcing test group to handle OCX forcing, which has a slightly different directory structure from other forcings. Extrapolate forcings into missing data cells before interpolating to MALI mesh. Also unify the structure for ismip7_forcing output with that expected as input for ismip7_run.

Checklist

  • User's Guide has been updated
  • Developer's Guide has been updated
  • Documentation has been built locally and changes look as expected
  • Document (in a comment titled Testing in this PR) any testing that was used to verify the changes

Handle the OCX scenario in the ismip7_forcing test group. OCX has no distinct
ESM model: it uses RACMO2.3p2-ERA (atmosphere) and EN4 (ocean) at data version
v1, with a named grid resolution (ocean-1000m) in the ocean file names.

get_params() now applies OCX-specific overrides (sources, versions, ocean grid
token) when scenario == OCX, so the [ismip7] model option is ignored and a
single config file drives both the atmosphere and ocean_thermal test cases.
Adds ismip7_forcing_ocx_gis.cfg and updates the user and developer docs.
Two fixes that block processing of forcing whose remapped files or source
directories differ from the CESM test data:

- Drop the ncremap-mangled 'time_bnds' variable when combining remapped
  files. ncremap collides the bounds 'nv' dimension with the destination
  mesh corner dimension and pads with a fill value that overflows time
  decoding. The steps build their own xtime and never use time_bnds.

- Skip input files whose trailing token is not a year (e.g. climatology
  averages like *_1978-2007_avg.nc) instead of crashing on int('avg').
Restructure processed-forcing output from
{output_base_path}/atmosphere_forcing/{model}_{scenario}/ (and the analogous
ocean_thermal_forcing path) to {output_base_path}/{group}/atmosphere/ and
{output_base_path}/{group}/ocean_thermal_forcing/, matching the directory
layout the ismip7_run test group globs for. 'group' is {model}_{scenario} for
the ESM scenarios and {scenario} (i.e. OCX) for OCX, whose atmosphere and
ocean sources differ but must share one directory. Documents the layout and
how to point ismip7_run's forcing_basepath / ocx_forcing_path at it.
…lation helper

process_smb.py, process_temperature.py, process_smb_gradient.py, and
process_temperature_gradient.py remapped the raw source file directly with
ncremap, unlike process_runoff.py, ocean_thermal, and the fracture pathway,
which all extrapolate fill/missing values on the source grid first. Without
that step, no-data cells near the ice-sheet margin (e.g. ocean cells in the
RACMO OCX source) leak the ~9.97e36 netCDF fill sentinel into valid
neighboring cells during conservative/bilinear remapping.

Add the same pre-remap extrapolation to the four affected steps.

Also promote the previously-duplicated extrapolate_source() (from
fracture/remap_utils.py, formerly re-implemented again in
ocean_thermal/process_thermal_forcing.py and process_runoff.py) to a single
shared compass.landice.tests.ismip7_forcing.remap_utils module, and update
all five call sites to use it instead of per-file private copies.
extrapolate_source moved from fracture.remap_utils to the shared
compass.landice.tests.ismip7_forcing.remap_utils module (828699b). Update
the api.rst autosummary entry accordingly; this was breaking the strict
Sphinx build (autosummary failed to import the moved symbol).
Enable the OCX (reanalysis) scenario for the Antarctic ice sheet:

- ice_sheet_params: add an 'ais' OCX override (RACMO atmosphere at v1,
  reanalysis ocean at v1 with no model token) and an ocean_choice_layout
  flag for the idiosyncratic AIS OCX ocean layout.
- process_thermal_forcing: assemble a list of ocean forcing sources so AIS
  OCX can process one or more ocean choices (main/cold/warm/vary), each in
  its own OCX_<choice> output directory; other cases are unchanged.
- ismip7_forcing.cfg: add ocean_choice option (AIS OCX only).
- remap_utils: write the extrapolated source file as CDF-5 so large 3D
  ocean thermal forcing fields aren't limited by the NetCDF3 per-variable
  size cap or HDF5 chunk limits that break ncremap.
- add ismip7_forcing_ocx_ais.cfg example config.
@trhille
trhille requested review from hollyhan and a lite review from Copilot September 11, 2026 14:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical output-layout and NetCDF encoding issues, along with API and input-validation issues, block approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR adds OCX reanalysis support to ISMIP7 forcing processing, including AIS/GrIS source handling, missing-cell extrapolation, and updated output organization.

Changes:

  • Adds OCX configurations and AIS ocean-choice handling.
  • Centralizes extrapolation before remapping.
  • Updates forcing layouts and user/developer documentation.
File summaries
File Change / final review notes
docs/users_guide/landice/test_groups/ismip7_forcing.rst Documents OCX inputs and outputs. Nit (1 vote): AIS ocean and atmosphere outputs use inconsistent directories, so the documented ocx_forcing_path is not directly runnable.
docs/developers_guide/landice/test_groups/ismip7_forcing.rst Updates developer guidance. Nits (1 vote each): Qualify EN4 as GrIS-only, document the AIS choice layout, and update the stale fracture helper reference.
docs/developers_guide/landice/api.rst Registers the shared remapping helper.
compass/landice/tests/ismip7_forcing/remap_utils.py Adds shared extrapolation logic. Critical (1 vote): Source encodings may be incompatible with NETCDF3_64BIT_DATA; clear or replace them before writing temporary files.
compass/landice/tests/ismip7_forcing/ocean_thermal/process_thermal_forcing.py Handles OCX ocean sources and choices. Critical (3 votes): AIS ocean output is not co-located with atmosphere output and cannot be consumed directly by ismip7_run. Moderate (2 votes): Invalid tokens combined with all bypass validation.
compass/landice/tests/ismip7_forcing/ismip7_forcing.cfg Adds ocean-choice configuration.
compass/landice/tests/ismip7_forcing/ismip7_forcing_ocx_gis.cfg Adds a GrIS OCX configuration.
compass/landice/tests/ismip7_forcing/ismip7_forcing_ocx_ais.cfg Adds an AIS OCX configuration.
compass/landice/tests/ismip7_forcing/ice_sheet_params.py Adds OCX parameter overrides. Nit (1 vote): The module description incorrectly says all OCX runs use EN4; qualify this for GrIS and document AIS choices.
compass/landice/tests/ismip7_forcing/fracture/remap_utils.py Removes duplicated extrapolation code. Moderate (1 vote): Removing the documented helper path can break downstream imports; retain a compatibility re-export or explicitly handle the breaking change.
compass/landice/tests/ismip7_forcing/fracture/process_lake_properties.py Uses the shared extrapolation helper.
compass/landice/tests/ismip7_forcing/fracture/process_excess_melt.py Uses the shared extrapolation helper.
compass/landice/tests/ismip7_forcing/atmosphere/process_temperature.py Adds OCX handling and extrapolation.
compass/landice/tests/ismip7_forcing/atmosphere/process_temperature_gradient.py Adds OCX handling and extrapolation.
compass/landice/tests/ismip7_forcing/atmosphere/process_smb.py Adds OCX handling and extrapolation.
compass/landice/tests/ismip7_forcing/atmosphere/process_smb_gradient.py Adds OCX handling and extrapolation.
compass/landice/tests/ismip7_forcing/atmosphere/process_runoff.py Adds OCX handling and extrapolation.
Review details

Suppressed comments (5)

compass/landice/tests/ismip7_forcing/fracture/remap_utils.py:5

  • This removes fracture.remap_utils.extrapolate_source, which was a documented API entry before this PR. Internal imports are updated, but downstream callers importing that path will now get ImportError; please keep a compatibility re-export here or explicitly handle this as a breaking API change.
import xarray as xr
from mpas_tools.io import write_netcdf

compass/landice/tests/ismip7_forcing/ice_sheet_params.py:36

  • The module-level description says every OCX run uses EN4 for the ocean, but the AIS override below deliberately sets ocean_model to None and uses per-choice ocean products. Qualify EN4 to GrIS and document the AIS choice layout so this description matches get_params().
# Overrides applied for the OCX (reanalysis) scenario. OCX has no distinct
# ESM model: it uses fixed reanalysis products (RACMO for the atmosphere and
# EN4 for the ocean) at data version v1. When scenario is 'OCX' the [ismip7]
# model option is ignored and these sources are used instead.

docs/developers_guide/landice/test_groups/ismip7_forcing.rst:35

  • This paragraph says OCX always sets ocean_model = EN4, but _OCX_OVERRIDES['ais'] sets it to None and uses the choice-based ocean layout; EN4 is only the GrIS source. Qualify these source names by ice sheet and document the AIS choice layout here, otherwise developers may construct the wrong filename/path.
When ``scenario = OCX``, ``get_params`` applies a set of OCX overrides on top
of the ice-sheet defaults: data version ``v1``, the ocean file-name grid token
(e.g. ``ocean-1000m``), and the fixed reanalysis sources (``atm_model`` =
``RACMO2.3p2-ERA`` and ``ocean_model`` = ``EN4``). The processing steps use

docs/developers_guide/landice/test_groups/ismip7_forcing.rst:37

  • The helper was moved to compass.landice.tests.ismip7_forcing.remap_utils, but the fracture section later in this same guide still documents extrapolate_source under fracture.remap_utils (lines 175-177). Update that reference so the developer guide does not direct readers to a symbol that no longer exists.
they are set, so the OCX ``model`` option is ignored. This keeps OCX handling

docs/users_guide/landice/test_groups/ismip7_forcing.rst:100

  • This layout description is not true for AIS OCX: the changed AIS branch writes ocean output under OCX_<choice> while atmosphere output is under OCX. Consequently, directing ocx_forcing_path to {output_base_path}/OCX does not provide a runnable AIS forcing set. Update the documented OCX layout after aligning the producer and consumer.
The ``group`` directory is ``{model}_{scenario}`` for the ESM scenarios and
``{scenario}`` (i.e. ``OCX``) for OCX, whose atmosphere and ocean use
different sources (``RACMO2.3p2-ERA`` and ``EN4``) but must share one
directory. To feed :ref:`landice_ismip7_run`, point its ``forcing_basepath``
at ``output_base_path`` for ESM scenarios, or its ``ocx_forcing_path`` at
``{output_base_path}/OCX`` for OCX.
  • Files reviewed: 17/17 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread compass/landice/tests/ismip7_forcing/remap_utils.py
trhille and others added 2 commits September 11, 2026 09:39
Ensure variables do not inherit encoding from source files.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Make atmosphere ocx output structure parallel to ocean forcing, using
symlinks to main atmosphere output directory. This makes the output
structure compatible with ismip7_run.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants