Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
c6d1543
Merge remote-tracking branch 'origin/main' into codex/tywrap-value-co…
bbopen Sep 20, 2026
fbe931e
fix(codec): validate terminal JSON integer metadata
bbopen Sep 20, 2026
fa09797
feat(runtime): await Python coroutines in Node and Pyodide
bbopen Sep 20, 2026
d0a990e
fix(runtime): preserve timeout diagnostics and async cleanup
bbopen Sep 20, 2026
b39ca6e
Merge commit 'fbe931e' into codex/tywrap-coroutines-final-integration…
bbopen Sep 20, 2026
12a93a1
Merge commit 'd0a990e' into codex/tywrap-coroutines-final-integration…
bbopen Sep 20, 2026
b741337
test(pyodide): prove generated coroutine on real WASM
bbopen Sep 20, 2026
755616f
fix(codec): validate set subclass integers
bbopen Sep 20, 2026
0486e7b
feat(runtime): await Python coroutines in Node and Pyodide
bbopen Sep 20, 2026
78d3656
fix(runtime): preserve timeout diagnostics and async cleanup
bbopen Sep 20, 2026
bfbc6ef
Merge commit '755616f' into codex/tywrap-coroutines-final-integration…
bbopen Sep 20, 2026
93032cd
Merge commit '78d3656' into codex/tywrap-coroutines-final-integration…
bbopen Sep 20, 2026
1e8717b
fix(runtime): reap retired Python children after forced shutdown
bbopen Sep 20, 2026
677c06e
Merge commit '1e8717b' into codex/tywrap-coroutines-final-integration…
bbopen Sep 20, 2026
6b56a96
style(runtime): satisfy shutdown lint rules
bbopen Sep 20, 2026
3393850
test(pyodide): verify real coroutine cancellation and reuse
bbopen Sep 20, 2026
6847a73
Merge branch 'codex/tywrap-coroutines-20260920' into codex/tywrap-cor…
bbopen Sep 20, 2026
b380418
fix(runtime): fence pool replacement until child cleanup succeeds
bbopen Sep 20, 2026
e640be2
Merge branch 'codex/tywrap-coroutines-20260920' into codex/tywrap-cor…
bbopen Sep 20, 2026
8879171
test(pool): wait for retired cleanup before replacement
bbopen Sep 20, 2026
8967a8a
Merge branch 'codex/tywrap-coroutines-20260920' into codex/tywrap-cor…
bbopen Sep 20, 2026
11a1db7
fix(pool): retain failed cleanup from unpublished workers
bbopen Sep 20, 2026
6d5fed9
Merge branch 'codex/tywrap-coroutines-20260920' into codex/tywrap-cor…
bbopen Sep 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/maintainers/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ not edit it directly. Regenerate it from `runtime/tywrap_bridge_core.py` with
the repository script. Pyodide uses JSON codec envelopes because pyarrow is not
available in WASM.

The bridge awaits Python call results under the event loop owned by each
backend. See [coroutine calls](./coroutines.md) for timeout and disposal rules.

`HttpBridge` sends the same RPC and codec concepts to a remote Python service.
The transport owns HTTP concerns while the codec and return validator preserve
the client-side invariants.
Expand Down
55 changes: 55 additions & 0 deletions docs/maintainers/coroutines.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Coroutine calls

A `call` request returns one result or one error envelope. The bridge awaits a
Python awaitable before it serializes the result. It rejects generators and
async generators. It does not expose tasks, streams, retries, or cached results.

## Node subprocess

The Python JSONL server handles requests in order. A synchronous call runs
directly. For an awaitable result, `handle_call()` wraps it in a coroutine and
uses `asyncio.run()`. Python 3.10 through 3.12 create and close one event loop
for that call. Pending background tasks do not survive the call.

The call must return an awaitable that can run on this new loop. A Future bound
to another loop fails in the normal Python error envelope. A synchronous
function that needs a running loop before it returns must create that loop
itself or use an `async def` function.

After a request reaches stdin, a timeout or abort retires that Python process.
The transport rejects other requests on the same process and starts a new
process for later calls. Replacement runs the configured worker warmup. Python
state held only in the retired process is lost. A request that times out before
its first write does not retire the process.

Retirement sends SIGTERM first. If the child stays alive, it sends SIGKILL after
one second and waits for the child to exit. A failed kill blocks replacement of
that process generation. The pool keeps the worker slot until disposal succeeds.
If cleanup fails, the pool rejects new leases and keeps the child handle for a
later disposal retry.

Disposal kills the process and rejects pending requests. The caller receives one
timeout, abort, or disposal error. A retired process cannot send a late response
into a replacement process. A timeout does not cancel Python code in place;
process retirement stops code that does not cooperate with cancellation.

## Pyodide

Pyodide owns its
[WebLoop](https://pyodide.org/en/0.28.1/usage/api/python-api/webloop.html). The
bootstrap registers a task by request ID before it returns the task to
JavaScript. The transport awaits the task and destroys its Pyodide proxies after
the task settles. Timeout, abort, and disposal call `Task.cancel()` for that
request. The task map removes settled and cancelled tasks.

Cancellation is cooperative. Python receives `CancelledError` at an await. The
caller receives one timeout, abort, or disposal error. The transport consumes a
late task rejection without sending another response.

CPU-bound Python on the browser thread can block both the JavaScript timer and
the Pyodide event loop. This transport cannot stop it at a deadline. Pyodide
documents
[interrupt buffers](https://pyodide.org/en/0.28.1/usage/api/js-api.html#pyodide.setInterruptBuffer)
for web workers. The bridge does not install an interrupt buffer or terminate a
worker. Host applications must manage their own worker lifecycle if they need
that limit.
2 changes: 1 addition & 1 deletion docs/maintainers/value-contracts.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Migration for users with large Python integers: convert them to strings in Pytho

The [fixed proof fixtures](value-contract-fixtures.v2.json) cover integer boundaries, binary16 values, unions, fixed tuples, and nested paths. The focused tests also compare every finite binary16 storage word with an independent float32 bit reference. They test signed zero with `Object.is`.

The Python serializer checks plain integers, NumPy scalar integers, and model dumps during its existing value walk. A JSON ndarray or sparse matrix checks integer data before it creates an envelope. The TypeScript decoder converts float16 Arrow storage during extraction, before it reshapes the ndarray. Torch uses that same ndarray path. These checks do not walk the full decoded payload again for schema validation.
The Python serializer checks plain integers, NumPy scalar integers, and model dumps during its existing value walk. It also checks terminal estimator parameters and Series name metadata before JSON output. A JSON ndarray or sparse matrix checks integer data before it creates an envelope. Sparse metadata checks shapes and indices. The TypeScript decoder converts float16 Arrow storage during extraction, before it reshapes the ndarray. Torch uses that same ndarray path. These checks do not walk Arrow payloads again for schema validation.

## Exact bigint extension for #337

Expand Down
Loading
Loading