Make batch API parseable with web-standard FormData #769

The batch endpoint (POST /v1/objects:batch/{usecase}/{scopes}/) uses multipart/form-data, but all per-operation data lives in custom per-part headers:

• Requests: the server reads x-sn-batch-operation-kind, x-sn-batch-operation-key, and insert metadata (content-type, x-sn-expiration, x-sn-origin, x-snme-*, …) from each part's headers (objectstore-server/src/extractors/batch.rs). Content-Disposition is ignored.
• Responses: every part is name=part. Index, status, key, kind, and object metadata are in x-sn-batch-operation-* and metadata headers (objectstore-server/src/endpoints/batch.rs, objectstore-server/src/multipart.rs). Parts stream in completion order, so the index header is the only way to correlate a result with its operation.

The web FormData API (new FormData() + fetch for requests, Response.formData() for responses, also used by Node via undici) only exposes each part's name, filename, content type, and value. JS clients in Node, browsers, or Workers have to hand-roll a byte-level multipart encoder and parser, as getsentry/sentry-api-schema#133 does for the Sentry CLI's snapshot upload.

The protocol itself is valid. RFC 2046 §5.1 allows non-Content- headers in body parts, though it says they "must not be depended on", and FormData drops them by design. The unquoted name=part is also valid (RFC 2183 allows token values).

Tested on Node 22.23 (undici 6.x) against the current response shape:

• Response.formData() throws Failed to parse body as FormData because undici before 7.17 only accepts quoted names. Node 24.13+ and 25.3+ accept name=part (fix fd parsing unquoted attribute values nodejs/undici#4662).
• With the name quoted, it parses, but every entry is named part, all per-part headers are dropped, and parts without a filename come back as strings (even with a Content-Type), which corrupts binary GET payloads.

So quoting the name alone does not make today's format usable through FormData. That needs a format that carries the metadata in form fields.

Raised by BYK in review of getsentry/sentry-api-schema#133.

Proposal

Opt-in FormData-native format that carries metadata in form fields instead of part headers:

• Request: a first part name="operations" (application/json) with an ordered list, e.g. [{"kind":"insert","key":"a/b","headers":{"content-type":"image/png","x-sn-expiration":"ttl:30d"}}, {"kind":"head","key":"c"}]. The headers map keeps existing names so the server can reuse Metadata::from_insert_headers. Insert bodies follow as parts named by operation index (name="0", with a filename and Blob), so a client just does form.append("0", blob, "0"). Because the manifest comes first, the server can still stream body parts.
• Response: for each completed operation, a JSON part name="<index>" such as {"status":201,"kind":"insert","key":"a/b","headers":{…}} (error body included on failure). For a GET with a payload, a following part name="<index>.body" with filename and Content-Type set, so parsers return a File and binary stays intact. Completion-order streaming is preserved.
• Quoted names: all part names and filenames in the new format are quoted (name="0"), so Node 22 and other parsers based on undici before 7.17 accept the response.
• Negotiation: the server detects the request format from a leading operations part. The client opts into the new response format explicitly (e.g. ?format=form or an Accept profile parameter). Without it, the server keeps today's header format, so the Rust client and the in-progress Python batch client (feat(py-client): Implement "many" api for batch requests #546) are unaffected.

Separate from this, the current format could also quote its name=part. That is optional compatibility cleanup for strict parsers. It does not help JS clients read the metadata.

Alternatives

• Status quo: keep the header protocol and maintain a hand-rolled parser in each JS client (~200 lines that must track the server).
• Metadata in name/filename only (e.g. name="insert:3", filename=<key>): can't carry expiration or custom metadata, and filename escaping differs between implementations.
• JSON body with base64 payloads: ~33% size overhead and no streaming.
• Emit both formats at once: breaks existing clients, which expect one part per operation with an index header.

Open questions

• Should the new format replace the per-part headers or sit alongside them?
• Response.formData() buffers the whole response; should responses in the new format have a lower size cap?

via @BYK, @lcian.

--

View Session [Sentry]