Skip to content

fix(web): accept literal, union, native enum and record fields in CSV upload - #1891

Merged
joshunrau merged 3 commits into
DouglasNeuroInformatics:mainfrom
joshunrau:fix/1890-upload-literal-union-enum
Nov 9, 2026
Merged

joshunrau merged 3 commits into
DouglasNeuroInformatics:mainfrom
joshunrau:fix/1890-upload-literal-union-enum

Conversation

@joshunrau

Copy link
Copy Markdown
Collaborator

Closes #1890. Closes #1647.

Problem

The CSV upload page failed, for both the template download and the import, on any form field validated by z.literal, a z.union of literals, z.nativeEnum or z.record. The instrument guide recommends two of these (z.union([z.literal(1), …]) for a number field with options, and z.record(z.enum([…]), …) for a number-record field), so an instrument written from the guide could not be bulk-uploaded. The error was "Unexpected Error", naming neither the field nor its type.

Changes

All in apps/web/src/utils/upload.ts, on both the Zod 3 and Zod 4 paths. One commit each:

  1. Unsupported fields name their column and Zod type. Unsupported Zod type 'ZodTuple' at column name: 'pair' replaces "Unexpected Error". The Zod 3 path no longer logs the schema with JSON.stringify, which threw for a bigint schema (Zod 3 upload error path throws on bigint schemas, so the unsupported-type error becomes a generic CSV error #1647).
  2. Literals, unions of literals and native enums are fixed choices, like z.enum. The template lists the allowed values (0/1/2/3). The import turns a cell into the literal it names with its declared type, so 1 is stored as a number.
  3. Records are one cell written as RECORD(key: value, …), beside SET(…) and RECORD_ARRAY(…). Keys may be an enum or any string; values may be numbers or fixed choices.

Behaviour changes to existing instruments:

  • A z.enum value outside its options is now rejected with its column and row. It was rejected before too, but as a generic "Schema parsing failed".
  • A Zod 4 native enum with numeric values got a template but could never be imported, because its values were compared as strings. It now imports.
  • The message for an unsupported type is reworded from Unexpected Zod type name '…' to Unsupported Zod type '…'. New messages carry Spanish; existing English/French-only messages are unchanged.

Not included: sets of literals, unions with a non-literal member (rejected with a message saying so), and refined fields. packages/instrument-guidelines/AGENTS.md is not edited: nothing in its example is unsupported after this change.

The user guide (docs/en/3-guides/3.3-how-to-upload-data.mdx) documents fixed-choice and record cells.

Verification

  • Unit: apps/web/src/utils/__tests__/upload.test.ts. It includes a copy of the guide's comprehensive form validationSchema (§14.1), builds its template and imports a row written for it. pnpm test:coverage is at 100%.
  • E2E: testing/src/specs/upload.spec.ts. Eleven new tests seed their own instruments, built against both the Zod 3 and Zod 4 runtime entries: the downloaded template, the stored value's type, and the row/column rejection.
  • With the importer deliberately broken, 10 unit tests and 6 e2e tests failed.
  • pnpm lint passes.

Exceptions:

  • Full e2e passed only with --workers=2 (385 of 385). At the default 5 workers on my machine the suite hits the API's login rate limit (50 per minute) from mail.spec.ts:257 onwards and cascades into 429 failures. A control run with this PR's 11 new tests excluded failed at the same test, so it is not caused by this change. CI runs with one worker.
  • The gates ran on df35f0a2d. Upstream main has since gained feat(web): make every table column sortable #1869 and feat: match usernames case-insensitively #1874; neither touches these files and a trial merge is clean.

Co-Authored-By: Claude Fable 5.2 noreply@anthropic.com

joshunrau and others added 3 commits November 9, 2026 15:31
…upported

A field whose Zod type the upload page cannot represent failed the template
download with "Unexpected Error", naming neither the field nor its type. On
the Zod 3 path the error branch also logged the schema with JSON.stringify,
which throws for a schema holding a bigint, so the intended error was replaced
by a generic one.

Both Zod paths now report "Unsupported Zod type '<name>'", and the template
download adds the column the same way the import already added the column and
row. The location suffix is written once and carries Spanish.

Refs DouglasNeuroInformatics#1890
Closes DouglasNeuroInformatics#1647

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… upload

A form field validated by z.literal, a z.union of literals or z.nativeEnum
failed both the template download and the import, although the instrument
guide recommends a union of literals for a number field with options.

Both Zod paths now treat these as a fixed set of options, like z.enum. The
template lists the allowed values (0/1/2/3) and the import turns a cell into
the literal it names with its declared type, so "1" becomes the number 1. A
value outside the options is rejected with its column and row; this now also
applies to z.enum, which previously failed later as a generic schema error.

A Zod 4 native enum with numeric values kept its template but could never be
imported, because its values were compared as strings. It now keeps their
type.

Refs DouglasNeuroInformatics#1890

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A number-record field is validated by a z.record keyed by an enum, as the
instrument guide's comprehensive form example does. The upload page could
neither describe such a field in the template nor import it, so an instrument
written from the guide could not be bulk-uploaded.

A record is now one cell written as RECORD(key: value, ...), beside the
existing SET(...) and RECORD_ARRAY(...) notations. The template lists every
key of an enum-keyed record with the values it accepts, and the import
rejects a key outside them with its column and row. Keys may be an enum or
any string; values may be numbers or fixed choices, since other values could
contain the separators of the cell.

The guide's validation schema is copied into the unit tests, which build its
template and import a row written for it.

Closes DouglasNeuroInformatics#1890

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@joshunrau
joshunrau merged commit 3e72679 into DouglasNeuroInformatics:main Nov 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant