Praxis mode 8/8: Run the live check and decide the default #1725

Activity

1. huang195

   self-assigned this

   on Nov 7, 2026
2. huang195

   added

   laptopThe local (laptop) tool: install, abctl, the numbersThe local (laptop) tool: install, abctl, the numbers

   on Nov 7, 2026
3. rubambiza

   added this to Rossoctl Issue Prioritizationon Nov 7, 2026
4. github-project-automation

   moved this to New/ToDo in Rossoctl Issue Prioritizationon Nov 7, 2026
5. huang195

   added a parent issue on Nov 7, 2026

   • Tracker: Move the laptop data plane to Praxis when the upstream gaps close #1714
6. huang195 commented on Nov 10, 2026

   

   huang195

   on Nov 10, 2026

   Last edited by huang195

   MemberAuthor

   More actions

   Criteria for the default, written before the live check.

   Praxis mode can become the default only when every criterion passes:

   • C1 Feature parity. Every feature that a default laptop install has works in praxis mode: tool-prune, inference-router (agentop server), script plugins (agentop script), agentop tools. Status today: fails by design (D-B).
   • C2 Facts. For the same live session, agentop shows in praxis mode: one session per agent run, the title, every timeline row, the tool call names, the subagent blocks, input/output/cache tokens equal to the usage in the raw capture, and cost equal to direct mode's price for those tokens. Zero rows with a gateway_* error kind, apart from the fail-closed step.
   • C3 Latency. 200 interleaved pairs, ~1 MiB body. Praxis p50 minus direct p50 is at most 5 ms. Praxis p95 minus direct p95 is at most 20 ms.
   • C4 Resources. On a long session (at least 40 inference turns and at least 2 subagents): the peak RSS of cortex plus cortex-gateway is at most 1.5 times the peak RSS of direct-mode cortex. Their CPU seconds summed are at most 2 times direct-mode cortex CPU seconds.
   • C5 Streaming. Claude Code SSE and Codex (chatgpt.com /backend-api/codex/responses) stream frame by frame in praxis mode: the time to the first frame is within 100 ms of direct mode.
   • C6 Fail closed. Kill cortex-gateway: the next agent request gets 503 with a gateway_unavailable row. The supervisor restarts the gateway, and requests succeed again within 10 s.
   • C7 One network proxy. The conversion is complete: cortex-gateway is the only network proxy on the laptop, and no traffic goes through the Go forward proxy. Until C7 passes, praxis mode stays behind the --praxis flag (agentop setup --praxis), even when C1-C6 pass. Status today: fails (phases B-D of Tracker: Move the laptop data plane to Praxis when the upstream gaps close #1714 are open).

   Condition to go back to direct mode (after a future flip): set praxis.enabled: false and restart when any one of these occurs:

   • an agent request with no row in the session store;
   • a gateway_unavailable row outside a Cortex or gateway restart;
   • a measured p50 added latency over 20 ms;
   • a cortex plus cortex-gateway RSS over 2 times direct mode;
   • a parser parity test red on main for more than 7 days.

   Source: docs/superpowers/plans/2026-11-10-praxis-mode-8-live-check.md. D-B: praxis mode turns off every outbound plugin by design, so C1 fails today.
7. huang195

   added a commit that references this issue on Nov 10, 2026

   fix: Keep the scratch Cortex off the caller's proxy and stop on empty…

   ...

   a35eb2c
8. github-project-automation

   moved this from New/ToDo to In progress in Rossoctl Issue Prioritizationon Nov 10, 2026
9. huang195

   added 2 commits that reference this issue on Nov 11, 2026

   docs: Add the plan for the praxis-mode live check (#1725)

   ...

   1356384

   feat: Add a dev script for the praxis-mode live check

   ...

   62932dd
10. huang195

    closed this as completedin #1775on Nov 11, 2026
11. github-project-automation

    moved this from In progress to Done in Rossoctl Issue Prioritizationon Nov 11, 2026
12. huang195

    added a commit that references this issue on Nov 11, 2026

    fix: Record a client that leaves before the head as client_canceled

    ...

    d59ab28
13. huang195 commented on Nov 11, 2026

    

    huang195

    on Nov 11, 2026

    Last edited by huang195

    MemberAuthor

    More actions

    Live check result (2026-11-10)

    Criteria: comment, written before the check.

    Setup:

    • Build: main at ce712778 (after Feat: Add a dev script for the praxis-mode live check #1775), Praxis 0.8.0, run with scripts/dev/praxis-live-check.sh.
    • Two scratch instances side by side: direct on 497xx, praxis on 4971x.
    • Agent: Claude Code 2.1.322, claude-opus-5-5 through the ete-int LiteLLM gateway.
    • Codex: not installed, so its parts were not run.

    #	Criterion	Measured	Threshold	Result
    C1	Feature parity	tool-prune, inference-router, scripts and agentop tools are off in praxis mode	all work	fail (by design)
    C2	Facts	See below. One wrong error kind	same facts	pass, 1 difference
    C3	Latency, 200 interleaved pairs, ~1 MiB	p50 delta −22.5 ms, p95 delta −316.5 ms (praxis faster), 0 dropped	≤ +5 ms / ≤ +20 ms	pass
    C4	Resources, 48–50 turns, 8 subagents	RSS 1.46×, CPU 1.35×	≤ 1.5×, ≤ 2×	pass
    C5	First frame, Claude Code, 20 pairs	p50 delta −99.7 ms, 0 dropped	within 100 ms	pass
    C5	First frame, Codex	—	—	not run
    C6	Fail closed	503 at once, 5 gateway_unavailable rows, requests succeed again after 3.5 s	≤ 10 s	pass
    C7	One network proxy	two proxies	one	fail (phases B–D are open)

    C3: direct p50 252.4 ms, p95 1058.0 ms; praxis p50 229.9 ms, p95 741.5 ms. The upstream noise (direct p95 over 1 s) is much larger than the difference between the modes.

    C4 (peak RSS, CPU seconds; heap in use where measured):

    Mode	Process	Peak RSS	Heap in use	CPU s
    direct	cortex	106 MiB	55 MiB	3.82
    praxis	cortex	108 MiB	58 MiB	4.05
    praxis	cortex-gateway	47 MiB	not measured (no pprof)	1.09

    C5: direct p50 362.0 ms, praxis p50 262.3 ms.

    C2 (direct / praxis):

    • Sessions: one named session for each run, with the same title.
    • Inference rows: 50 / 48. Subagents: 8 / 8.
    • Tool call names: the same set (Agent, Bash, Glob, Grep, Read).
    • Tokens: equal to the raw usage on every row in both modes (0 mismatches).
    • Cost: the same price for each token kind in both modes (input $3.04/M, output $15.20/M, cache read $0.152/M, cache write $3.80/M).
    • No error rows in the agent sessions, apart from the fail-closed step.

    The two runs did not make the same calls, because the model does not repeat itself. So the row and tool counts are close, not equal.

    C2 difference: a client that stops before the response head is recorded as gateway_refused in praxis mode. Direct mode records client_canceled. Praxis 0.8.0 gives status 0 in that case, not 499. Fix: #1777.

    Live install (laptop, same day)

    The same build (main-latest, ce71277) was installed with install.sh --ref=main --praxis and used for about 25 minutes with Claude Code, Bob and OpenCode.

    • 209 Claude Code inference rows and all Bob rows matched the raw usage.
    • Ingest: 1,164 lines, 0 refused, 0 captures dropped.
    • A test set through both modes (sequential, parallel, a 1 MiB body, JSON, gzip, SSE, a stop in the middle of a stream, MCP tools/call, plain HTTP) recorded the same rows in both.

    One finding in both modes, not specific to praxis mode: when a client hangs up during the TLS handshake, the bridge arms a 30 s skip for that program (FailTransient, core/listener/forwardproxy/bridging.go). For those 30 s, that program's requests to that host are tunneled without decryption and leave no rows.

    Decision

    Direct mode stays the default. Praxis mode stays opt-in, behind --praxis.

    Blocking criteria:

    • C1 fails by design until the plugins are ported or retired.
    • C7 fails until phase C leaves one network proxy.

    C2–C6 pass. That is the first half of the phase D gate (#1714, "Gates"). The second half, no rollback condition in daily use, needs a period that is not decided yet.