Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 111 additions & 0 deletions rework_summaries/tb-04-transform2d-normalize_changes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# tb-04-transform2d-normalize

Makes `Transform2D` input normalization opt-in instead of unconditional, and
replaces two tests that claimed to cover it but did not.

## Why

`Transform2D.fit` rescaled `xin` and `yin` to `[-1, 1]` before fitting, stored
the rescaling as a `pretransform`, and carried `1/scale` into `gradient()` as a
chain-rule factor. Upstream's version is five lines with none of that.

Enabling it by default is not upstreamable for two reasons:

1. **It changes the public meaning of `.matrix`.** The coefficients are in
rescaled coordinates, not input coordinates. The existing upstream test
`test_fit_gives_correct_matrix` had to be given `normalize=False` to keep
passing — that edit is the clearest evidence the change is observable.
2. It silently alters the fitted transforms of every instrument in the IRDB.

So it is now off by default and selected per trace.

## What the rescaling is actually worth

Measured, because the original justification did not survive checking.

| Input coordinates | degree | unnormalized | normalized |
|---|---:|---:|---:|
| focal-plane mm, centred (-50..50, -25..25) | 4 | 1.2e-14 | 1.6e-14 |
| focal-plane mm, centred | 6 | 2.8e-14 | 1.9e-13 |
| pixels, uncentred (0..2048) | 4 | 5.8e-11 | 8.7e-14 |
| pixels, uncentred | 6 | 5.8e-11 | 8.9e-14 |

(worst absolute residual of the fitted surface)

The rescaling is a **safeguard for poorly scaled inputs**, worth ~650x when
coordinates run far from the origin, and **immaterial — if anything marginally
worse — on the centred focal-plane mm that ScopeSim actually fits.** Both
routes reach machine precision there, which is also why turning it on does not
move an existing trace.

This corrects the rationale I started with, which was that `fit_degree: 6` in
`ZShooter_SPEC.yaml` made conditioning important. At degree 6 in centred mm the
unnormalized fit is the better of the two.

## Where the switch lives, and why it is not a `!SIM` flag

`fit_normalize` is a plain entry in `SpectralTrace._class_params`, defaulting
to `False`, exactly like the `fit_degree` that sits beside it:

```python
"trace_flux_jacobian": "!SIM.spectral.trace_flux_jacobian",
"fit_degree": 4,
"fit_normalize": False,
```

It is resolved in `compute_interpolation_functions` and passed to the six
`Transform2D.fit` calls.

A global `!SIM.spectral.*` bang-string was tried first and rejected. The fit
happens during `SpectralTrace.__init__`, and several tests construct trace
lists with `cmds=None` while `scopesim.rc.__currsys__` is patched to a bare
`MagicMock` (`test_OpticalTrain.py`'s `patch_globals`). Any bang-key resolved
at construction time fails there, which is why `trace_flux_jacobian` gets away
with it — it is only read later, inside `_trace_flux_scale`.

Per-trace is also the right scope on the merits: this is a property of one
fit, not of the simulation. `fit_degree` is the precedent, already overridden
for exactly one instrument in `ZShooter_SPEC.yaml`.

**Config impact:** ZShooter and X-shooter must set `fit_normalize: true` on
their trace effects to keep today's numbers. See the companion irdb branch.

## Tests

Two existing tests were named for normalization but passed with it on *or* off,
so they were not testing it:

- `test_fit_preserves_high_dynamic_range_coefficients` — now explicitly
`normalize=False`; it is really a float64-dtype and accuracy test.
- `test_gradient_with_normalized_fit_uses_input_units` — now explicitly
`normalize=True`, so the `dpretransform` chain-rule factor is actually
exercised. With `normalize=False` that factor is 1 and the test is vacuous.

New:

- `test_normalization_conditions_poorly_scaled_inputs` — the uncentred case,
asserting >100x improvement.
- `test_normalization_is_immaterial_for_centred_focal_plane_mm` — both routes
below 1e-12, documenting that enabling it is safe.
- `TestSpectralTraceFitNormalize` — the parameter defaults off, opting in
populates `pretransform_x`/`dpretransform_x`, both routes agree on the mapped
wavelength and slit coordinates to 1e-9, and gradients agree to 1e-7, i.e.
stay in per-mm units either way.

## Deliberately not included

`fit2matrix` builds the coefficient matrix as float64 where upstream used
float32. That is left unconditional and unchanged by this branch: float32 for a
degree-4-or-higher 2D polynomial coefficient matrix is indefensible regardless
of normalization, and it deserves its own one-line PR with residual numbers
rather than being bundled into a behaviour toggle.

`xilam2xy_fit`, `xy2xilam_fit` and `_xiy2xlam_fit` in the same module are dead
— zero call sites anywhere including tests — but they are upstream code
(oczoske, 2021) and untouched by this fork, so removing them is upstream's
call, not ours.

## Verification

`scopesim/tests`: 4 failed, 1382 passed. Same four pre-existing failures as on
`main`.
19 changes: 12 additions & 7 deletions scopesim/effects/spectral_trace_list_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ class SpectralTrace:
"description": "<no description>",
"trace_flux_jacobian": "!SIM.spectral.trace_flux_jacobian",
"fit_degree": 4,
"fit_normalize": False,
}

def __init__(self, trace_tbl, cmds=None, **kwargs):
Expand Down Expand Up @@ -202,12 +203,16 @@ def compute_interpolation_functions(self):
self.wave_max = quantify(np.max(lam_arr), u.um).value

degree = self.meta["fit_degree"]
self.xy2xi = Transform2D.fit(x_arr, y_arr, xi_arr, degree=degree)
self.xy2lam = Transform2D.fit(x_arr, y_arr, lam_arr, degree=degree)
self.xilam2x = Transform2D.fit(xi_arr, lam_arr, x_arr, degree=degree)
self.xilam2y = Transform2D.fit(xi_arr, lam_arr, y_arr, degree=degree)
self._xiy2x = Transform2D.fit(xi_arr, y_arr, x_arr, degree=degree)
self._xiy2lam = Transform2D.fit(xi_arr, y_arr, lam_arr, degree=degree)
# Rescaling the inputs to [-1, 1] conditions the Vandermonde matrix
# for coordinates far from the origin. It also changes what .matrix
# means, so like fit_degree it is a per-trace setting, default off.
normalize = self.meta["fit_normalize"]
self.xy2xi = Transform2D.fit(x_arr, y_arr, xi_arr, degree=degree, normalize=normalize)
self.xy2lam = Transform2D.fit(x_arr, y_arr, lam_arr, degree=degree, normalize=normalize)
self.xilam2x = Transform2D.fit(xi_arr, lam_arr, x_arr, degree=degree, normalize=normalize)
self.xilam2y = Transform2D.fit(xi_arr, lam_arr, y_arr, degree=degree, normalize=normalize)
self._xiy2x = Transform2D.fit(xi_arr, y_arr, x_arr, degree=degree, normalize=normalize)
self._xiy2lam = Transform2D.fit(xi_arr, y_arr, lam_arr, degree=degree, normalize=normalize)
if "slit_dispersion_colname" in self.meta:
center = xi_arr == 0
derivative = self.table[self.meta["slit_dispersion_colname"]].quantity.to_value(u.um / u.arcsec)
Expand Down Expand Up @@ -1158,7 +1163,7 @@ def __call__(self, x, y, grid=False, **kwargs):
return result

@classmethod
def fit(cls, xin, yin, xout, degree=4, normalize=True):
def fit(cls, xin, yin, xout, degree=4, normalize=False):
"""Determine polynomial fits."""
xin = np.asarray(xin, dtype=float)
yin = np.asarray(yin, dtype=float)
Expand Down
68 changes: 66 additions & 2 deletions scopesim/tests/tests_effects/test_SpectralTraceListUtils.py
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ def test_fit_preserves_high_dynamic_range_coefficients(self):
+ 0.2 * x
)

tf2d = Transform2D.fit(x, y, z, degree=4)
tf2d = Transform2D.fit(x, y, z, degree=4, normalize=False)

assert tf2d.matrix.dtype == np.float64
assert np.max(np.abs(tf2d(x, y) - z)) < 1e-5
Expand All @@ -274,7 +274,7 @@ def test_gradient_with_normalized_fit_uses_input_units(self):
y = y_grid.ravel()
z = 3 + 2 * x + 5 * y + 7 * x * y + 11 * y**2

tf2d = Transform2D.fit(x, y, z, degree=2)
tf2d = Transform2D.fit(x, y, z, degree=2, normalize=True)
dz_dx, dz_dy = tf2d.gradient()

assert dz_dx(x, y) == pytest.approx(2 + 7 * y, rel=1e-10, abs=1e-10)
Expand Down Expand Up @@ -343,3 +343,67 @@ def test_works_with_non_square_image(self):
interps = make_image_interpolations(hdul, kx=1, ky=1)
imginterp = interps[0](yy, xx, grid=False)
assert np.allclose(imginterp, img)

def test_normalization_conditions_poorly_scaled_inputs(self):
# Inputs far from the origin are where rescaling to [-1, 1] pays off.
# ScopeSim's focal-plane mm are centred, so this is a safeguard for
# other coordinate conventions rather than a general improvement.
x_grid, y_grid = np.meshgrid(np.linspace(0, 2048, 40), np.linspace(0, 2048, 40), indexing="ij")
x, y = x_grid.ravel(), y_grid.ravel()
xs, ys = (x - 1024)/2048, (y - 1024)/2048
z = 1. + .35*(ys + .02*xs + .1*ys**2 + .03*xs*ys + .005*ys**3)

plain = Transform2D.fit(x, y, z, degree=4, normalize=False)
scaled = Transform2D.fit(x, y, z, degree=4, normalize=True)

plain_residual = np.max(np.abs(plain(x, y) - z))
scaled_residual = np.max(np.abs(scaled(x, y) - z))
assert scaled_residual < plain_residual / 100

def test_normalization_is_immaterial_for_centred_focal_plane_mm(self):
# The coordinates ScopeSim actually fits. Both routes reach machine
# precision, so enabling the flag must not move anybody's trace.
x_grid, y_grid = np.meshgrid(np.linspace(-50, 50, 40), np.linspace(-25, 25, 40), indexing="ij")
x, y = x_grid.ravel(), y_grid.ravel()
z = 1. + .35*(y/50 + .02*x/100 + .1*(y/50)**2)

plain = Transform2D.fit(x, y, z, degree=4, normalize=False)
scaled = Transform2D.fit(x, y, z, degree=4, normalize=True)

assert np.max(np.abs(plain(x, y) - z)) < 1e-12
assert np.max(np.abs(scaled(x, y) - z)) < 1e-12


class TestSpectralTraceFitNormalize:
def test_defaults_to_the_upstream_unnormalized_fit(self):
spt = SpectralTrace(_tilted_linear_trace_table())
assert spt.meta["fit_normalize"] is False
assert spt.xy2lam.pretransform_x is None

def test_opting_in_rescales_the_fit_inputs(self):
spt = SpectralTrace(_tilted_linear_trace_table(), fit_normalize=True)
assert spt.xy2lam.pretransform_x is not None
assert spt.xy2lam.dpretransform_x != 1.0

def test_both_routes_agree_on_this_trace(self):
# Focal-plane mm are centred, so the two fits must describe the same
# trace; enabling the option is not allowed to move anybody's spectrum.
table = _tilted_linear_trace_table()
plain = SpectralTrace(table.copy())
scaled = SpectralTrace(table.copy(), fit_normalize=True)

x_mm = np.linspace(1.2, 2.8, 17)
y_mm = np.linspace(-0.08, 0.08, 17)
assert scaled.xy2lam(x_mm, y_mm) == pytest.approx(plain.xy2lam(x_mm, y_mm), abs=1e-9)
assert scaled.xy2xi(x_mm, y_mm) == pytest.approx(plain.xy2xi(x_mm, y_mm), abs=1e-9)

def test_gradients_are_in_input_units_either_way(self):
# The chain-rule factor is what keeps the normalized gradient in
# per-mm rather than per-rescaled-unit.
table = _tilted_linear_trace_table()
plain = SpectralTrace(table.copy())
scaled = SpectralTrace(table.copy(), fit_normalize=True)

x_mm, y_mm = np.linspace(1.2, 2.8, 9), np.linspace(-0.08, 0.08, 9)
for plain_grad, scaled_grad in zip(plain.xy2lam.gradient(), scaled.xy2lam.gradient()):
assert scaled_grad(x_mm, y_mm) == pytest.approx(plain_grad(x_mm, y_mm), abs=1e-7)
Loading