The attrs decision — a design notebook

The roadmap promises that the object handed back can answer for every step that produced it. This notebook holds every candidate attrs strategy against that sentence — first the problem, demonstrated live against today’s main; then each option as code, the user experience and the contributor’s side of the same coin.

The problem, live¶

Today’s law is Commandment 3: preserve inbound attrs, then append the parameter you applied — apodization_lb = 5.0, not apodized = True. A short chain looks well-recorded:

import numpy as np
import xarray as xr

import xmris

fid = xmris.simulate_fid(
    amplitudes=[1.0, 0.6],
    chemical_shifts=[0.0, 5.2],
    reference_frequency=120.66,  # MHz
    n_points=1024,
)

spectrum = (
    fid.xmr.zero_fill(target_points=2048)
    .xmr.apodize_exp(lb=5.0)
    .xmr.to_spectrum()
    .xmr.phase(p0=20.0)
)
spectrum.attrs

Fifteen keys, physics and lineage shoulder to shoulder — already showing #64’s second worry, sprawl. But the record reads complete: the line broadening, the zero-fill, the phase are all there. It stays complete exactly as long as no step happens twice.

twice = fid.xmr.apodize_exp(lb=5.0).xmr.apodize_exp(lb=5.0)

print("the record claims lb =", twice.attrs["apodization_lb"])
print("the data says lb = 10:", bool(np.allclose(twice, fid.xmr.apodize_exp(lb=10.0))))

the record claims lb = 5.0
the data says lb = 10: True

The second application silently overwrote the first — no trace remains that this FID was apodized twice. With phasing, the record graduates from forgetting to lying:

spec = fid.xmr.to_spectrum()
rephased = spec.xmr.phase(p0=20.0).xmr.phase(p0=-5.0)

print("the record claims p0 =", rephased.attrs["phase_p0"])
print("the data carries 15°:", bool(np.allclose(rephased, spec.xmr.phase(p0=15.0))))

the record claims p0 = -5.0
the data carries 15°: True

Phases add: the object carries 15° of correction and its own record claims −5°. Anyone reproducing this result from the attrs reproduces the wrong spectrum.

The library already feels this strain. phase hand-rolls a one-off cross-step consistency check — it remembers which coordinate space the last phase was applied in, and warns when the next one happens somewhere else:

spec_ppm = spec.xmr.phase(p0=20.0).xmr.to_ppm()
_ = spec_ppm.xmr.phase(p0=5.0, dim="chemical_shift")  # ← warns

/home/runner/work/xmris/xmris/src/xmris/processing/phasing.py:85: UserWarning: Applying phase in 'chemical_shift', but previous phase operations were recorded in 'frequency'. Ensure your pivot value (0.0) matches the current dimension's units.
  warnings.warn(

That warning is proto-history: one function keeping a private memory of the step before it, because the object has no shared one. And finally, one step outside xmris loses everything — xarray drops attrs on most operations by default (#21):

(spectrum * 2).attrs

So the question splits in two: what shape should the record have (Options A–C below), and what preservation guarantee can the library honestly make (its own section — the answer is the same under every option).

Two kinds of attrs¶

Before comparing options, one observation that every option benefits from. Today’s ATTRS vocabulary mixes two populations that behave nothing alike:

Physics attrs are healthy: individually addressable, typed, gated at the door. Everything wrong in the previous section is confined to the lineage class. netCDF’s CF conventions draw exactly this line — physical metadata that must stay valid, separate from an append-only history attribute of processing steps. Every option below leaves physics attrs untouched; the decision is only about the record.

Option A — flat keys stay law¶

Keep Commandment 3 as written. The overwrite becomes documented semantics — last application wins — and #23 (provenance tracking) closes as won’t-do.

The user experience is the one already shown live above: readable single keys (apodization_lb: 5.0) that answer “what was applied last?” and nothing else — not how many times, not in what order, not what the effective total is. The re-phasing cell stays a lie.

The contributor side is today’s hand-rolled pattern, repeated in every function (src/xmris/processing/fid.py):

da_apodized = (da * weight).transpose(*da.dims).assign_attrs(da.attrs)

# Record lineage
da_apodized.attrs[ATTRS.apodization_lb] = lb

return da_apodized

Twenty functions, twenty appends, and any new function’s author must remember both halves — the assign_attrs copy and the append. Cheapest option by far: zero migration, no new machinery, TestAttrsPreservation untouched. Its price is that the hero sentence is false and stays false — the object answers for the last application of each step, unordered.

Option B — one structured history¶

Lineage moves into a single attr: xmr_history, one JSON string holding an append-only event log. Flat lineage keys are deleted from the vocabulary. One central decorator does all the bookkeeping — preservation and the event — so function bodies stop handling attrs entirely.

To make this option feelable rather than imagined, the cell below prototypes that decorator and grafts it onto today’s functions. These ~40 lines are also the honest size estimate of the central machinery:

import functools
import inspect
import json
from importlib.metadata import version

HISTORY_KEY = "xmr_history"  # would live in the vocabulary as ATTRS.history
EMPTY = json.dumps({"schema": 1, "events": []})

# Keys today's functions hand-append. Under option B the appends are deleted from
# the source; the prototype strips them from wrapped outputs to emulate that.
FLAT_LINEAGE = {
    "phase_p0", "phase_p1", "phase_pivot", "phase_pivot_coord",
    "apodization_lb", "apodization_gb", "zero_fill_target", "zero_fill_position",
    "baseline_method", "baseline_lam", "baseline_p", "baseline_iter",
    "group_delay_removed", "amares_amplitude_scale",
    "sim_amplitudes", "sim_dampings", "sim_frequencies_hz", "sim_chemical_shifts_ppm",
}


def records_history(func):
    """Prototype of the one central decorator: preserve attrs, append the event."""

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        bound = inspect.signature(func).bind(*args, **kwargs)
        bound.apply_defaults()
        data_args = [
            v for v in bound.arguments.values() if isinstance(v, (xr.DataArray, xr.Dataset))
        ]
        params = {
            k: v
            for k, v in bound.arguments.items()
            if not isinstance(v, (xr.DataArray, xr.Dataset))
        }

        result = func(*args, **kwargs)

        inbound = dict(data_args[0].attrs) if data_args else {}
        added = {k: v for k, v in result.attrs.items() if k not in FLAT_LINEAGE}
        result.attrs = {**inbound, **added}  # preservation by construction

        envelope = json.loads(result.attrs.get(HISTORY_KEY, EMPTY))
        envelope["events"].append({"op": func.__name__, "params": params, "v": version("xmris")})
        result.attrs[HISTORY_KEY] = json.dumps(envelope, default=str)
        return result

    return wrapper

# Graft the prototype onto the live library, notebook-locally: the .xmr methods are
# thin delegators to these module-level names (Commandment 9), so rebinding them
# gives the real chained UX without touching src/.
import xmris.core.accessor as _accessor

for _name in ("zero_fill", "apodize_exp", "to_spectrum", "phase"):
    setattr(_accessor, _name, records_history(getattr(_accessor, _name)))

simulate_fid = records_history(xmris.simulate_fid)  # history starts at birth

What the user sees¶

The same chain as the opening — with the double apodization and the re-phasing left in deliberately:

fid_b = simulate_fid(
    amplitudes=[1.0, 0.6],
    chemical_shifts=[0.0, 5.2],
    reference_frequency=120.66,
    n_points=1024,
)

spectrum_b = (
    fid_b.xmr.zero_fill(target_points=2048)
    .xmr.apodize_exp(lb=5.0)
    .xmr.apodize_exp(lb=5.0)  # the same step twice — now on the record
    .xmr.to_spectrum()
    .xmr.phase(p0=20.0)
    .xmr.phase(p0=-5.0)  # re-phased — both calls visible, order kept
)
spectrum_b.attrs

The attrs dict collapses to its physics plus one record key. That xmr_history string is the honest cost of this option: at a glance it is a JSON blob. The reading surface would be a da.xmr.history() method — prototyped here as a function:

import pandas as pd


def history(da: xr.DataArray) -> pd.DataFrame:
    """What `da.xmr.history()` would return."""
    events = json.loads(da.attrs[HISTORY_KEY])["events"]
    return pd.DataFrame(
        {
            "op": [e["op"] for e in events],
            "params": [
                ", ".join(f"{k}={v!r}" for k, v in e["params"].items() if v is not None)
                for e in events
            ],
            "xmris": [e["v"] for e in events],
        }
    )


history(spectrum_b)

Every step, in order, with its parameters — including the repeated ones the flat record swallowed. The hero sentence, answered by a method call.

What survives a file¶

The record must travel with the data, so the shape is chosen for serialization: netCDF attrs cannot hold nested dicts, and a length-1 list-of-strings attr silently collapses to a scalar on an xarray round-trip — one JSON string is the engine-proof form.

import tempfile
from pathlib import Path

# netCDF holds no complex values natively (that is the component dim's business,
# out of scope here) — the attrs are what we are round-tripping.
magnitude = spectrum_b.copy(data=np.abs(spectrum_b.values))

with tempfile.TemporaryDirectory() as tmp:
    path = Path(tmp) / "spectrum.nc"
    magnitude.to_netcdf(path)
    loaded = xr.load_dataarray(path)

assert loaded.attrs[HISTORY_KEY] == spectrum_b.attrs[HISTORY_KEY]
history(loaded)

A colleague opening this file next year gets the full processing story out of the .nc itself.

One deliberate absence: no timestamps. Events record op, params and the xmris version — nothing wall-clock. Identical inputs must produce identical objects, or every assert_identical in the executable docs (this project’s differentiator) starts failing:

rerun = (
    fid_b.xmr.zero_fill(target_points=2048)
    .xmr.apodize_exp(lb=5.0)
    .xmr.apodize_exp(lb=5.0)
    .xmr.to_spectrum()
    .xmr.phase(p0=20.0)
    .xmr.phase(p0=-5.0)
)
xr.testing.assert_identical(spectrum_b, rerun)
print("bit-identical rerun — the record is deterministic")

bit-identical rerun — the record is deterministic

What the contributor writes¶

Function bodies lose all attrs bookkeeping. Today’s apodize_exp closes with three lines of hand-rolled lineage; under option B it becomes:

@computes_in(TIME_DIMS)
@records_history  # ← preservation + the event, centrally
def apodize_exp(da: xr.DataArray, dim: str = DIMS.time, lb: float = 1.0) -> xr.DataArray:
    _check_dims(da, dim, "apodize_exp")
    weight = np.exp(-np.pi * lb * da.coords[dim])
    return (da * weight).transpose(*da.dims)  # no attrs handling at all

The wider footprint of the change:

• Vocabulary: the 14 flat lineage terms in ATTRS (plus simulate_fid’s literal sim_* keys) are deleted; one term is added (history = "xmr_history" — namespaced, so it cannot collide with user or CF attrs the way baseline_method can today).

• Commandment 3 rewrite (draft): Preserve inbound coordinates and attributes. Physics attrs stay flat and typed. What the function did is one appended event in the history — written by the decorator, never by hand. Banned: state flags, hand-appended lineage keys, and any math that branches on the history. The last clause makes the record write-only for library logic; advisory reads stay legal — phase’s coordinate-space warning becomes a peek at the history’s tail instead of a private key.

• Tests: TestAttrsPreservation survives unchanged (preservation is now structural, the test becomes a regression guard); one new test pins the round trip and event order.

• fit_amares appends a single fit_amares event to the Dataset — amares_amplitude_scale becomes one of its params instead of a vocabulary term (the fit-Dataset chapter of the data-model schema, #28).

Option C — hybrid: flat “latest” keys plus the history¶

Keep the history from option B as the audit record, and additionally keep flat keys as a quick-glance surface for the latest application. The attrs would render like:

>>> spectrum.attrs
{'reference_frequency': 120.66,
 'carrier_ppm': 0.0,
 'apodization_lb': 5.0,        # ← latest application only
 'phase_p0': -5.0,             # ← still claims −5° while the data carries 15°
 'xmr_history': '{"schema": 1, "events": [...]}'}

The contributor delta over B is small — the decorator mirrors each event’s params into flat keys. The problems are what the mirror means:

• “Latest” is not “effective state”: after two lb=5 apodizations the mirror reads 5.0; after the re-phasing it reads -5.0. The quick-glance surface keeps the lie that motivated the change, now beside the record that contradicts it.

• The mirror duplicates the history’s last event — redundant information that must be kept consistent, and a second record format frozen at 1.0.

• Its only justification would be readers — and today there are none (the grep above). A consumer that appears later can be served by adding the mirror back, a non-breaking change; removing it later is the breaking one.

The preservation guarantee — the same answer under every option¶

Whatever the record’s shape, what may a user actually rely on? Inside xmris the answer can be structural: the decorator restores inbound attrs onto every result, so even attrs-dropping operations inside a function body cannot lose them (this is #21’s “systematic mechanism”). Outside xmris, the library cannot patch xarray, and pretending otherwise would be a false promise. The escape hatch already exists upstream:

with xr.set_options(keep_attrs=True):
    doubled = spectrum_b * 2

doubled.attrs == spectrum_b.attrs

Side by side¶