Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

The constants decision — a design notebook

The attrs decision split the metadata world in two: lineage became the xmr_history record, and the physical constants — the numbers a measurement cannot be interpreted without — stayed flat, typed, individually addressable. Two promises about those constants are still broken or unmade: they do not reliably travel (one ordinary xarray operation strips them, and the failure surfaces steps later), and they are not discoverable (every constant carries curated prose and a unit in the vocabulary, yet nothing on the object surfaces it). This notebook’s thesis, reached the long way round: the library should stop trying to patch xarray’s metadata rules and instead stop keeping physics as metadata — the options differ in where in the object’s structure the constants live, and what each home costs in the repr, at the gates, and in a file.

The problem, live

simulate_fid stamps its output with the calibration the physics needs — the same keys a vendor loader writes. Inside .xmr chains they are safe; a notebook is not an .xmr chain:

{}

Everything is gone — including reference_frequency and carrier_ppm, without which a ppm axis cannot exist. Nothing says so; the failure arrives later, in a different cell, possibly a different day:

'to_ppm' requires the following missing attributes in `obj.attrs`: ['reference_frequency', 'carrier_ppm'].

To fix this, assign them using standard xarray methods:
    >>> obj = obj.assign_attrs({'reference_frequency': value})

The error is honest but names the symptom: the user is told to re-enter a value they already provided, with no hint which upstream line dropped it — #21’s insidious half. And the second promise breaks silently in the first cell’s output: to a scientist arriving from any vendor, 'reference_frequency': 120.66 is a bare float, while the curation written for exactly that reader is reachable only by importing library internals:

unit: MHz
The measured Larmor frequency of the target nucleus. This reflects the actual B0 field during the scan, not a theoretical constant. It serves as the denominator to convert frequency shifts (Hz) to parts-per-million (ppm). Maps to Bruker 'PVM_FrqRef' and potentially DICOM 'ImagingFrequency'(0018,0084) or 'TransmitterFrequency' (0018,9098).

What must travel — and what already does

Not every key on today’s objects is a constant this decision must carry:

Key on today’s objectsWhat it isIn this decision’s class?
reference_frequency, carrier_ppm, group_delay, b0_fieldNon-derivable acquisition constants — gates and converters need themYes
spectral_width, dead_timeDerivable: the time coordinate’s spacing and first sample — fit_amares already infers both from itNo — duplication to retire
unitsDescribes the data values (CF-style variable attr)No — stays an attr
sim_*, target_snr, apodization_lb, …Lineage — becomes xmr_historyNo — decision 02’s law

The second row is the thesis in miniature: spectral_width never had a travel problem, because it is not an annotation about the data — it is structure, the spacing of a coordinate, and xarray protects structure with everything it has. Four constants need a home today; the MRSI horizon multiplies the family (echo/repetition timing, flip angle, voxel geometry), so the mechanism must stay readable at, say, ten constants on a four-dimensional object. That object is this page’s measuring stick:

The keep_attrs family — ergonomics for the flag

The constants could stay attrs if the flag that protects attrs stopped being ceremony. Two ergonomic forms, prototyped. First, scoped sugar — with xmris.keep():

True

It works, and the name is friendlier — but count the ceremony. Scoped honestly, every cell that touches the data needs the wrapper, which is more ritual than the line it replaced; hoisted to one call at the top of the notebook, it is the anti-goal line wearing a nicer name. Sugar relocates the ceremony; it cannot remove it.

Second form: the library detects a notebook and sets the flag there, leaving scripts alone. Detection is real — get_ipython() exists exactly when running under Jupyter/IPython (it does right now, in this page’s kernel). But watch what the same three lines then do in the two environments:

in the notebook: {'reference_frequency': 120.66}
as a script:   {}

The pipeline that worked all week in Jupyter loses its constants the day it is promoted to a script — the failure mode moves to exactly the moment nobody is watching. And in the notebook, the flag is a session global: every other library’s objects in the same kernel now behave differently because xmris was imported — the third tenet inverted, and a direct contradiction of the attrs Resolution’s guarantee that xmris never sets global options. Worse, the whole family shares one deeper flaw — under the flag, attrs of combined objects are first-wins:

120.66 — first operand wins, silently
<xarray.core.options.set_options at 0x7fbebc9c3450>

A later to_ppm draws an axis that is wrong for half the signal, without a murmur. Whatever the ergonomics, keep_attrs converts silent loss into silent lies. The family is not the answer at any level of sugar.

The structural turn

xarray drops annotations; it protects structure — coordinates survive because science depends on them. So measure exactly what a coordinate-borne constant can rely on. The probe: a scalar (dimensionless) coordinate whose value is the constant and whose own attrs carry its unit — pushed through the operations a real analysis performs:

Loading...

Three facts, and they shape everything below. Object attrs survive almost nothing — the problem section, quantified. A coordinate’s value survives everything on this list, including the reduction that removes the spectral axis. A coordinate’s attrs survive everything except one function — xr.where (the function form; the everyday method da.where is safe), which rebuilds coordinates bare. So a constant stored as a coordinate value is bulletproof; a constant stored in a coordinate’s attrs has exactly one known leak. Three homes follow from this table — they differ in which side of it they sit on, and in what they do to the repr.

Option P — one coordinate per constant

The maximal-robustness corner: every constant is its own scalar coordinate (value = the constant, attrs = its unit via as_variable, Commandment 7’s machinery). Travel is the “value” column above — no leak at all — and disagreement gets the best semantics on this page:

cross-field sum:  constant kept? False
drifting series:  [120.66 120.68] ('repetition',)

Conflicting values are dropped — the combined object honestly has no single reference frequency, and the next gate refuses loudly. A drifting series is promoted to a per-scan array — the truth, recorded. This option fails somewhere else entirely. Put the horizon family on the measuring stick:

Loading...

Ten constants shoulder-to-shoulder with four real axes, indistinguishable from data coordinates at a glance. This is the review objection, on screen: at MRSI scale the repr — the single most-read surface in a notebook — becomes a wall. Mitigations were considered and don’t rescue it: insertion order is preserved through operations (the constants stay grouped at the end), but ten lines are ten lines; an xmr_ name prefix makes the grouping visible at the cost of uglier names everywhere (da.xmr_reference_frequency), and xarray offers no per-object way to fold coordinates away. Right physics, unreadable at scale.

Option C — the container coordinates

The geoscience stacks hit this exact wall and left a pattern: rioxarray carries an entire coordinate-reference system as one scalar coordinate (spatial_ref) whose attrs hold the fields — CF’s grid-mapping design, netCDF-native, proven at survey scale. The xmris shape is that pattern, twice — a two-line block that ends the coordinate list:

The measuring stick, again

Loading...

Ten constants — and the processing record beside them — in a two-line block that stays two lines whether the family grows to fifteen or fifty. In a Jupyter repr the xmr_acquisition entry expands to the full constants dict on one click; programmatically it is a plain mapping, one hop away, with explain() (below) as the curated view:

{'reference_frequency': 120.66, 'carrier_ppm': 4.7, 'group_delay': 68.0, 'b0_field': 7.0, 'echo_time': 0.012, 'repetition_time': 2.0, 'flip_angle': 90.0, 'voxel_dx': 2.5, 'voxel_dy': 2.5, 'voxel_dz': 8.0}

Travel is the matrix’s “attrs” column — everything except xr.where — plus the block itself surviving axis-dropping reductions (it outlives mean("time"), where axis-carried constants below die). The opening section’s failure, replayed end-to-end:

Loading...

No flag, no ceremony, no failure. Entry for an outsider is today’s documented plain-string path, unchanged — the first gate re-homes loose attrs into the container, and from then on they travel:

re-homed: {'reference_frequency': 120.66, 'carrier_ppm': 4.7}
recorded: {"schema": 1, "events": [{"op": "to_ppm"}]}
survives the next plain op? True

Pinned to the bottom, named as a block

The review asked for a guarantee: these must not read as data coordinates, and they must sit at the end of the list. Both are enforceable, and the enforcement was measured. xarray displays coordinates in insertion order — there is no auto-sorting — so mint_containers pins the block by dropping and re-assigning it last on every xmris return (a plain re-assignment updates in place and would not move it). Ordinary operations then preserve that order:

Loading...

Three cases can push the block off the bottom — a user’s own later assign_coords, groupby re-appending the grouped coordinate, and netCDF reload order (engine-determined). None of them is silent damage, and every one self-heals at the next xmris touch:

after a user assign_coords: ['frequency', 'xmr_acquisition', 'xmr_history', 'mask']
after the next xmris call:  ['frequency', 'mask', 'chemical_shift', 'xmr_acquisition', 'xmr_history']

The names do the other half of the review’s ask. The xmr_ prefix marks ownership at a glance — it is the accessor’s own name, so the mental link “xmr_* belongs to .xmr” is free — it renders the pair as one visual block above the Attributes section, and it cannot collide with any vendor’s axis names. The words after the prefix are the domain’s own:

CoordinateValue (what xarray compares)AttrsRead surface
xmr_acquisitioncalibration fingerprintthe constants, flat + typedone click in the repr · explain() · da.xmr_acquisition.attrs
xmr_historythe JSON envelope — 02’s format, verbatim—da.xmr.history()

When calibrations — or histories — disagree

The fingerprint value is what xarray compares, so conflicts behave like Option P’s — honest, and each container answers independently:

calibration kept? False | record kept? True
'to_ppm' requires the following missing attributes in `obj.attrs`: ['reference_frequency', 'carrier_ppm'].

Different constants → different fingerprints → the calibration is dropped whole and the next gate refuses loudly, instead of keep_attrs’ confidently wrong axis — while the record survives, because both operands carry the same history. The mirror case: same calibration, diverged processing —

calibration kept? True | record kept? False

The blend keeps its constants (they still hold) and honestly loses its lineage (no single history describes it) — exactly the right answer in both columns, and nothing in the library invented merge semantics to get it. A drifting series promotes the fingerprint to a per-scan array — visibly non-uniform, so a gate meeting a non-scalar container can refuse with “this series is not uniformly calibrated” (the exact message is decision 03’s schema work):

array(['f1be16af', 'd8954ad2'], dtype='<U8')

One honest cost, from the matrix’s one leak: xr.where (function form) strips the container’s attrs — but keeps its value, so the damage is detectable, not silent:

container present: True | constants left: {}

A gate that finds the container present but empty can say precisely what happened and how to fix it — a named, loud, one-function edge case (today xr.where already strips the units metadata Commandment 7 puts on every ppm axis, so this is an upstream wart worth an xarray issue regardless of this decision).

What survives a file

coordinate order after reload: ['frequency', 'xmr_history', 'xmr_acquisition']
Loading...

The container is a real netCDF variable carrying the constants as its attributes — the CF grid-mapping shape, self-describing under any tool’s ncdump, no xmris required to read it. (Reload order is the netCDF engine’s choice — one of the three slip cases above; the first xmris call re-pins the block.)

Fine print — costs and open ends, honestly
  • The 02 amendment, stated precisely. Decision 02 closed the record’s format (one JSON envelope, versioned, timestamp-free) and named the key xmr_history; this decision moves its home from da.attrs["xmr_history"] to the coordinate’s value — same name, same envelope, one line of 02’s Resolution to amend at the harvest. Everything else 02 decided (write-only for library math, da.xmr.history() as the reading surface, no invented merge semantics) carries over unchanged.

  • Stale attrs on a promoted container. After the drift concat, the per-scan fingerprints are truthful but the container’s attrs still show scan 1’s constants (first-wins on attrs is unfixable from library code). The gate’s non-scalar check is what makes this safe: a promoted container is unreadable until sliced back to one scan — drifted.isel(repetition=0) — which restores that scan’s fingerprint.

  • Contributor footprint. Two writers (simulate_fid, the Bruker loader) mint the block instead of assign_attrs; decision 02’s central decorator gains the re-pin (drop + assign last) and the container read; one reader helper behind requires_attrs (and the direct attrs.get reads in to_ppm, to_hz, fit_amares, remove_digital_filter) looks container-first-then-attrs, with the attrs fallback kept indefinitely — data written under today’s law keeps working, and outsider entry stays plain assign_attrs (the lazy re-homing makes the existing fix-lines correct). TestAttrsPreservation is untouched; one new parametrized test pins “the block is the last two coordinates” across the public API.

  • Prototype cheats. Only four functions are wrapped here (the real change is the central decorator, so every function records and re-pins); events carry just op (the real envelope carries params and version, per 02); mint_containers runs at the chain head (real writers mint at the source).

Option X — constants ride the axis they calibrate

The zero-repr-cost corner: reference_frequency and carrier_ppm calibrate the spectral axis, group_delay the time axis — so store each constant in the attrs of the dimension coordinate it belongs to. No new repr line at all, and a certain semantic beauty: the calibration lives on the thing it calibrates.

survives da*2: 120.66
survives mean('frequency'): False

Both edges cut against it. The constants die whenever their axis does — and an axis dies in legitimate workflows (integrating a peak, collapsing a spectral region into a map: exactly the fit-derived amplitude maps MRSI produces, which still deserve their acquisition context). It shares xr.where’s attrs leak without a surviving value to detect it by, and binary-op conflicts are first-wins (measured — the axis values agree, so xarray keeps the first operand’s attrs wholesale). And the class has members with no axis at all: b0_field today, echo/repetition timing tomorrow — which forces a second home and makes the whole design two mechanisms instead of one. The converters would also need to hand-carry constants across every axis swap (to_spectrum builds its output axis from scratch). Elegant for the axis constants it fits, structurally partial for the family.

Discoverability — explain()

Whatever the home, discoverability is one accessor method: walk the object, match every name against the vocabulary, surface the curation. Under the container it reads one level deeper — and the native repr already carries the constants one click away:

Loading...

Every recognized name answers for itself — readable wording, unit, inline — and the table doubles as an audit: the literal keys simulate_fid leaks past the vocabulary today light up as not in the xmris vocabulary, which is how a growing constants family stays curated instead of accumulating folklore.

Side by side

keep_attrs familyP — per-constant coordsC — container blockX — axis-carried
Survives plain opsonly under the flageverything measuredeverything but xr.wheredies with its axis
Cross-calibration sumfirst wins — silent liedropped → loud gatedropped → loud gatefirst wins — silent lie
Drifting series (concat)first scan’s valuesper-scan truthper-scan fingerprints, gate-detectablefirst scan’s values
Repr cost at MRSI scalenone+10 lines and growing+2 pinned lines, constantnone
The record can share the homenono — needs its own coord anywayyes — sibling coordinateno
Ceremony / sessionper-notebook or owns sessionnonenonenone
Covers axis-less constantsyesyesyesno — needs a second home
netCDF shapeheader attrs10 scalar variablesone CF grid-mapping-style blockaxis attrs
Script ≡ notebooknot under auto-detectyesyesyes
Migrationdocs onlywriters + gateswriters + gates + decorator re-pinwriters + gates + converters