Porting plan: opera_utils functions into disp-xr¶
disp-xr does not depend on opera_utils (PRD decision, Sep 30, 2026). The DISP product logic it needs is ported into disp-xr modules, with attribution, and disp-xr owns and tests it from then on.
- Source:
opera-utilscommit97f77d9(upstreamopera-adt/opera-utilsmain, 2026-06-04). The port was read from a checkout at a fork commit,dff4b45, that adds only NISAR download modules on top of97f77d9; every ported file and test is identical in the two, so headers cite the public97f77d9. - Status of this spec: written by two readers of the source. The independent verification pass did not run; spot checks of the license, the filename regex, the module locations and upstream bugs 1–3 passed. Confirm every line range when porting.
- Tasks: each row names the task in todo.md that owns the port.
License and attribution¶
opera_utils is dual-licensed, BSD-3-Clause or Apache-2.0, at the user's choice (Copyright (c) 2022 California Institute of Technology, U.S. Government sponsorship acknowledged). disp-xr is BSD-3-Clause, so the ports are taken under BSD-3-Clause.
- Ship the opera_utils BSD-3-Clause text as
LICENSES/opera-utils-BSD-3-Clause.txt, and list it underlicense-filesinpyproject.tomlso it goes into the sdist, wheel and conda package. - Give each ported module a header:
Portions adapted from opera-utils (https://github.com/opera-adt/opera-utils, commit 97f77d9, <source path>), Copyright (c) 2022 California Institute of Technology, BSD-3-Clause; modified for disp-xr. - Do not use the Caltech or JPL names to endorse disp-xr.
- Ported tests carry the same header.
plate_motion.pyfollows MintPy code, and MintPy is GPL-3.0. Do not copy it. Reimplement plate motion from the published math (ITRF2014 plate motion model, Altamimi et al. 2017) and cite the paper.- A CI rule forbids
import opera_utils(ruffTID251banned-api), so the dependency cannot come back.
What is ported, and where¶
Verdicts: port = copy the logic with small changes; simplify = port the core and drop parts disp-xr does not need; rewrite = same behavior, new implementation on xarray.
Product parsing and search (T4.5, T5)¶
| opera_utils source | Verdict | disp-xr target | Changes |
|---|---|---|---|
constants.DISP_FILE_REGEX |
Port | product.py, one regex per ProductSpec |
The upstream regex hard-codes IW, a 5-digit frame and VV\|HH; confirm the NISAR pattern on a real granule |
disp/_product.py DispProduct, from_filename |
Simplify | product.py (DispGranule frozen dataclass) |
Drop the grid properties that read the S1 burst database; the grid comes from the opened file |
disp/_product.py DispProductStack (dedupe, sort) |
Rewrite | product.py (find_products returns a DataFrame) |
Compare versions with packaging.version, not strings |
disp/_enums.py dataset enums |
Rewrite | product.py (ProductSpec layer lists) |
Layer names per product; mark NISAR ionospheric_delay as already applied |
disp/_search.py search, _cmr.py fetch_cmr_pages, get_download_url, DispProduct.from_umm, constants.UrlType |
Port | search.py |
CMR short name from ProductSpec (S1 and NISAR); logger named disp_xr |
disp/_download.py subset download |
Simplify, opt-in | search.py (aws extra) |
Clip a bbox that leaves the frame instead of raising |
Remote open and credentials (T6)¶
| opera_utils source | Verdict | disp-xr target | Changes |
|---|---|---|---|
_remote.py open_file, open_h5, get_s3_fs, get_https_fs, is_remote_url, get_url_str |
Port | _remote.py (private, aws extra) |
Keep cache_type="first" and the 4 MiB HDF5 page size, which match the DISP file layout |
credentials.py (Earthdata login, temporary S3 credentials, TTL cache) |
Port | _credentials.py |
Expose a refreshable get_credentials callable; Earthdata S3 credentials expire after about 1 hour |
disp/_utils.py _clamp_chunk_dict |
Port | _chunks.py |
Shared by open, stack and export |
Ministack chaining (T7)¶
| opera_utils source | Verdict | disp-xr target | Changes |
|---|---|---|---|
disp/_rebase.py NaNPolicy |
Port | stack.py (public) |
— |
disp/_rebase.py rebase_timeseries |
Port | stack.py (private kernel) |
Keep the numerics so upstream tests apply; keep the input dtype (upstream forces float32); check that each crossover epoch exists |
disp/_rebase.py create_rebased_displacement |
Rewrite | stack.py (stack_disp) |
xr.apply_ufunc(dask="parallelized") over time; read reference times from ds.reference_time; fix the zero epoch label |
The timeseries_fitting branch already holds unattributed copies of these in stack.py. T4.0 merges the branch; T7 keeps one implementation and deletes rebase_disp.
Quality and reference (T10, T11)¶
| opera_utils source | Verdict | disp-xr target | Changes |
|---|---|---|---|
disp/_reformat.py combine_quality_masks |
Port | quality.py |
Backs the mask() presets |
disp/mintpy.py create_reliability_mask |
Simplify | quality.py (reliability) |
Return an xr.DataArray; no file output |
Harmonic-mean temporal coherence (_reformat.py, rebase_reference.py) |
Rewrite | quality.py (quality_summary) |
One value per ministack through groupby("reference_time") |
disp/_enums.py ReferenceMethod |
Port | reference.py |
Methods: none, point, median, border, high_coherence |
disp/_reference.py get_reference_values |
Simplify | reference.py |
Bounds-check the point; require a mask for high_coherence; median honors the good-pixel mask |
disp/_reference.py lon/lat to row/col, _product.py OutOfBoundsError |
Rewrite | geometry.py |
Through pyproj and ds.rio.transform() |
disp/_reference.py _get_border_pixels |
Rewrite | reference.py |
No dropna on a stacked dask array (it forces a compute) |
disp/rebase_reference.py find_reference_point |
Simplify | reference.py (method="auto") |
Deterministic pick by center of mass of high-quality candidates |
Geometry (T12)¶
| opera_utils source | Verdict | disp-xr target | Changes |
|---|---|---|---|
geometry.py stitch_geometry_layers, Layer, nodata table |
Rewrite | geometry/static.py |
Done (T12): open_static reads the frame-level DISP-S1-STATIC product (found in CMR after this plan was written); stitch_cslc_static merges CSLC-STATIC bursts with rioxarray for frames without it. The opera_utils import is gone |
geometry.py LOS up and 3-band LOS helpers |
Simplify | geometry/static.py |
Computed in open_static |
disp/create_incidence_range.py incidence and slant range |
Rewrite | geometry.py |
Array functions on the LOS dataset; wavelength and orbit height from ProductSpec |
Export (T14)¶
| opera_utils source | Verdict | disp-xr target | Changes |
|---|---|---|---|
disp/_reformat.py _get_zarr_encoding, _to_shard_dict |
Simplify | export/zarr.py |
Presets timeseries and maps; shards as a multiple of chunks with a divisibility check |
disp/_utils.py _get_netcdf_encoding |
Port | export/netcdf.py |
— |
disp/_utils.py round_mantissa |
Port, opt-in | export/_encoding.py |
Lossy; keep_bits=None by default so Cal/Val outputs stay exact |
disp/mintpy.py disp_nc_to_mintpy, metadata, HDF5 writer, static layers |
Rewrite | export/mintpy.py (mintpy extra) |
Take an xr.Dataset; platform, wavelength and height from ProductSpec; write chunk by chunk |
Plate motion (T13, new)¶
| opera_utils source | Verdict | disp-xr target | Changes |
|---|---|---|---|
disp/plate_motion.py |
Rewrite from the paper | corrections.py |
Done (T13.5): written from Altamimi et al. (2017); ECEF tested against pyproj, so upstream bugs 7–8 do not apply |
Not ported¶
reformat_stack (its steps become the disp_xr stack CLI command), the GeoTIFF rebase loop and stack reader/writer classes in rebase_reference.py, burst_frame_db lookups, create_virtual_stack (HDF5 VDS; VirtualiZarr covers this, PRD C3), last_per_ministack, flatten, and the rasterio-profile helpers.
Upstream bugs to fix in the ports¶
Each gets a regression test in disp-xr.
DispProductStacksorts versions as strings, so1.10ranks below1.9.search()drops the frame filter whenproduct_version=None.create_rebased_displacement(add_reference_time=True)labels the zero epoch with the first secondary time, giving a duplicate time coordinate.rebase_timeseriesaccumulates in float32 whatever the input dtype, and never checks that the crossover epoch exists.get_reference_values: the point method has no bounds check (negative indices wrap), the median ignoresgood_pixel_mask, and high_coherence with no mask silently uses every pixel.mintpy._write_hdf5leaves compression off for every dataset after the first; the metadata hard-codes Sentinel-1 platform, wavelength and height, and the velocity step loads the full cube.plate_motion.geodetic_to_ecefpasses radians to a transformer that expects degrees.plate_motion.runswaps latitude and longitude when unpacking the mesh grid.
Practices kept from opera_utils¶
- Processing order: corrections, then quality mask (both per product, before chaining), then chain, then spatial reference. Masked NaNs then propagate through the chain.
NaNPolicy.propagateis the default.- Chain with the whole time axis in one chunk and spatial chunks clamped to the array shape.
- Use
ds.rio.transform(), not theGeoTransformattribute, after subsetting. - Read
/identificationand/metadatawith h5py; xarray cannot decode their byte strings. - Zarr v3 output with inner chunks plus shards, dask rechunked to the shard shape before writing, and uncompressed coordinates.
Tests to port¶
All are synthetic and need no network: test_disp_rebase.py, test_disp_reference.py, the combine_quality_masks and encoding parts of test_disp_reformat.py, test_disp_utils.py (mantissa rounding, chunk clamping, NetCDF encoding), the filename and dedupe parts of test_disp_product.py, test_disp_search.py (mocked requests), test_disp_remote.py and test_credentials.py (monkeypatched).
mintpy.py, plate_motion.py, create_incidence_range.py and stitch_geometry_layers have no upstream tests; the ports need new ones.
DISP-NISAR differences the ports must handle¶
| Item | DISP-S1 | DISP-NISAR |
|---|---|---|
| Filename mode and polarization | IW, VV |
To confirm on a real granule |
| CMR short name | OPERA_L3_DISP-S1_V1 |
To confirm |
| Static geometry source | DISP-S1-STATIC per frame (CSLC-STATIC bursts as fallback) | DISP-NI-STATIC (not in CMR yet) |
| Wavelength, orbit height | 0.0555 m, about 693 km | about 0.238 m, about 747 km |
/corrections/ionospheric_delay |
Separate layer, not applied | Already applied to displacement |
| ASF S3 credential endpoint | OPERA endpoint | Unverified |
Layer names for the 11 displacement and quality layers are the same in both products.