Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -343,7 +343,7 @@ The standalone [evidence-review runbook](backend/evidence_review/README.md) docu

- Sidebar Jev reviews run through backend-owned `/api/ai/sidebar/*/evidence-reviews` contracts in research/pilot only, using backend-only `RADSYSX_TYPESAFE_AI_API_KEY`. They retain original answers, require independent text confirmation, and persist POSIX-private artifacts in `.ai-evidence/` beside the actual database (or absolute `RADSYSX_AI_EVIDENCE_DIR`). Ending voice preserves review; explicit cancel/account changes/logout stop it; source-history deletion cancels jobs and removes their artifacts. Clinical use and the qualified human evidence-quality study remain unapproved/pending.

- Typed chat and explicit Research do not require a Realtime connection. They use the account-selected standard Gemini/NVIDIA text/research model, owned HTTP text sessions and the same attestation/context/lifecycle authority. Voice remains optional; text-only requests never capture images. See `backend/clinical/AGENTS.md` and `viewer/assets/live/AGENTS.md` for the bounded worker and UI contracts.
- Typed chat and explicit Research do not require a Realtime connection. They use the account-selected standard Gemini/NVIDIA/Codex text/research model, owned HTTP text sessions and the same attestation/context/lifecycle authority. Voice remains optional. Codex Chat/Research can send one explicitly attached, previewed active-viewport JPEG; ordinary text requests never capture images automatically. Exact catalog image support, fresh context/attestation and transient pixels are required; saved receipts establish submission, not diagnostic validity. See `backend/clinical/AGENTS.md` and `viewer/assets/live/AGENTS.md` for the bounded worker and UI contracts.

## ChatGPT subscription access

Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ The standalone [evidence-review runbook](backend/evidence_review/README.md) docu

NVIDIA NIM is available for explicit evidence evaluation and opt-in PubMed research. Set backend-only `RADSYSX_NVIDIA_API_KEY`, `RADSYSX_RESEARCH_PROVIDER=nvidia_nim` and an exact `RADSYSX_NIM_RESEARCH_MODEL` to select NIM research; these environment settings supply the default when the account has no saved choice. In **Settings → Text & research models**, choose Gemini or NVIDIA NIM and an exact model from the dropdown. NVIDIA lists every ID returned by its hosted catalog, with a refresh control. Saving persists the choice for the signed-in account and ends its active sessions/tasks; the next text or research request uses it. Catalog membership does not verify tool support or access. The model catalog is `.venv/bin/python -m backend.evidence_review models --provider nvidia_nim`. See the [NIM runbook](roadmap/ai-backend/NIM_IMPLEMENTATION.md) for tested models, limits and failure evidence.

Typed **Send** and explicit **Research** work without Gemini Live/OpenAI Realtime. Confirm synthetic/deidentified content and choose the standard model in **Settings → Text & research models**. Only the question, bounded text history (chat only) and neutral viewer metadata are sent; image pixels require separate live sharing. **Connect voice** starts a separate voice conversation. See `roadmap/ai-backend/DESKTOP_AI_ACTIVATION.md`.
Typed **Send** and explicit **Research** work without Gemini Live/OpenAI Realtime. Confirm synthetic/deidentified content and choose the standard model in **Settings → Text & research models**. Only the question, bounded text history (chat only) and neutral viewer metadata are sent; Codex image input requires explicit **Attach current view**, preview, and Send/Research. This submits one active-viewport snapshot including visible overlays, not the full series; other providers retain separate live sharing. **Connect voice** starts a separate voice conversation. See `roadmap/ai-backend/DESKTOP_AI_ACTIVATION.md`.

ChatGPT/Codex subscription sign-in is available under desktop AI Settings for typed chat and public PubMed research. It uses isolated backend-owned Codex App Server and the OS keyring, not OpenAI API credentials or Realtime entitlement. Read `roadmap/ai-backend/CODEX_SUBSCRIPTION.md`; never copy the user's existing Codex auth.

Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -558,3 +558,7 @@ Security remediation and verification details are in [SECURITY_REMEDIATION.md](S
The standalone [evidence-review runbook](backend/evidence_review/README.md) documents private public/synthetic PubMed capture, Jev/Gemini/NIM replay, blind references and comparative reports. Run `.venv/bin/python -m backend.evidence_review --help`. The CLI remains independent of live conversation and never changes assistant answers. The sidebar now offers a separate explicit **Review evidence with Jev** action on completed PubMed research cards; see the [sidebar implementation runbook](roadmap/ai-backend/JEV_SIDEBAR_IMPLEMENTATION.md). Preview the exact claims and original abstracts, exclude claims as needed, and confirm public/synthetic text before TypeSafe receives anything. Saved claim-level judgments and resolved-model receipts establish what ran; the Settings configuration row alone does not. Clinical mode disables its network commands; local report/validation commands need no credentials. Software completion does not imply human-quality or live-provider acceptance.

NVIDIA NIM is available for explicit evidence evaluation and opt-in PubMed research. Set backend-only `RADSYSX_NVIDIA_API_KEY`, `RADSYSX_RESEARCH_PROVIDER=nvidia_nim` and an exact `RADSYSX_NIM_RESEARCH_MODEL` to select NIM research; these environment settings supply the default when the account has no saved choice. In **Settings → Text & research models**, choose Gemini or NVIDIA NIM and an exact model from the dropdown. NVIDIA lists every ID returned by its hosted catalog, with a refresh control. Saving persists the choice for the signed-in account and ends its active sessions/tasks; the next text or research request uses it. Catalog membership does not verify tool support or access. The model catalog is `.venv/bin/python -m backend.evidence_review models --provider nvidia_nim`. See the [NIM runbook](roadmap/ai-backend/NIM_IMPLEMENTATION.md) for tested models, limits and failure evidence.

### Attach an image to Codex Chat or Research

With a ChatGPT/Codex model selected, confirm synthetic/deidentified data and choose **Attach current view**. Expand **Preview image** to inspect the exact snapshot and visible measurement overlays, then type your question and choose **Send** or **Research**. The selected model must advertise image input; `gpt-6-astra` was verified on 2026-09-23. This works without Realtime. Each request receives only that captured viewport, not the whole series or continuous screen access. Attach again for another view. Saved image receipts record what was submitted; pixels are not saved in RadSysX history. See the [subscription runbook](roadmap/ai-backend/CODEX_SUBSCRIPTION.md).
2 changes: 1 addition & 1 deletion WARP.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ The standalone [evidence-review runbook](backend/evidence_review/README.md) docu

NVIDIA NIM is available for explicit evidence evaluation and opt-in PubMed research. Set backend-only `RADSYSX_NVIDIA_API_KEY`, `RADSYSX_RESEARCH_PROVIDER=nvidia_nim` and an exact `RADSYSX_NIM_RESEARCH_MODEL` to select NIM research; these environment settings supply the default when the account has no saved choice. In **Settings → Text & research models**, choose Gemini or NVIDIA NIM and an exact model from the dropdown. NVIDIA lists every ID returned by its hosted catalog, with a refresh control. Saving persists the choice for the signed-in account and ends its active sessions/tasks; the next text or research request uses it. Catalog membership does not verify tool support or access. The model catalog is `.venv/bin/python -m backend.evidence_review models --provider nvidia_nim`. See the [NIM runbook](roadmap/ai-backend/NIM_IMPLEMENTATION.md) for tested models, limits and failure evidence.

Typed **Send** and explicit **Research** work without Gemini Live/OpenAI Realtime. Confirm synthetic/deidentified content and choose the standard model in **Settings → Text & research models**. Only the question, bounded text history (chat only) and neutral viewer metadata are sent; image pixels require separate live sharing. **Connect voice** starts a separate voice conversation. See `roadmap/ai-backend/DESKTOP_AI_ACTIVATION.md`.
Typed **Send** and explicit **Research** work without Gemini Live/OpenAI Realtime. Confirm synthetic/deidentified content and choose the standard model in **Settings → Text & research models**. Only the question, bounded text history (chat only) and neutral viewer metadata are sent; Codex image input requires explicit **Attach current view**, preview, and Send/Research. This submits one active-viewport snapshot including visible overlays, not the full series; other providers retain separate live sharing. **Connect voice** starts a separate voice conversation. See `roadmap/ai-backend/DESKTOP_AI_ACTIVATION.md`.

ChatGPT/Codex subscription sign-in is available under desktop AI Settings for typed chat and public PubMed research through isolated, pinned Codex App Server. Subscription credentials stay in the OS keyring; Realtime remains API-key billed. Read `roadmap/ai-backend/CODEX_SUBSCRIPTION.md`.

Expand Down
8 changes: 5 additions & 3 deletions backend/clinical/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,13 +90,15 @@
## Voice-independent text sessions

- Own `ai_text.py` and `ai_text_routes.py`. POST `/text-sessions` allocates a synthetic/deidentified session using the owner's selected research provider/model, without Realtime setup or voice credentials. Additive `ai_text_sessions` marks transport/provider; existing sessions retain their voice identity. Text session DTOs have `mode=text`, null audio rates/live URL, and cannot attach a WebSocket.
- POST `/sessions/{id}/text-turns` requires signed unexpired `ai.run`, enabled research/pilot, allowed Origin, strict bounded 16 KiB JSON, current context and attestation. Chat accepts 2,000 characters, research 1,700. Idempotency keys bind action/text/context; duplicates return the original task. One active typed task per actor, two globally, with the shared subprocess cap and 120-second bound. Cancellation joins the worker, including cancellation before its first instruction. Context/account/model changes, close, expiry and shutdown stop work. Late responses cannot restore cancelled/deleted results.
- Chat uses the same fixed-endpoint native ChatNVIDIA/ChatGoogle adapter in a credential-isolated subprocess, with one `ainvoke` and no tools. Only neutral allowlisted numeric/modality/series metadata and bounded completed chat pairs enter its prompt; explicitly state no pixels or records were provided. Research uses the existing bounded DeepAgents graph with the public question and neutral modality/count only, never prior chat or arbitrary viewer strings. No image/vision inference is implied.
- POST `/sessions/{id}/text-turns` requires signed unexpired `ai.run`, enabled research/pilot, allowed Origin, strict bounded 720,000-byte turn JSON (session creation remains 16 KiB), current context and attestation. Chat accepts 2,000 characters, research 1,700. Idempotency keys bind action/text/context and image receipt; duplicates return the original task. One active typed task per actor, two globally, with the shared subprocess cap and 120-second bound. Cancellation joins the worker, including cancellation before its first instruction. Context/account/model changes, close, expiry and shutdown stop work. Late responses cannot restore cancelled/deleted results.
- Chat uses the same fixed-endpoint native ChatNVIDIA/ChatGoogle adapter in a credential-isolated subprocess, with one `ainvoke` and no tools. Only neutral allowlisted numeric/modality/series metadata and bounded completed chat pairs enter its prompt; explicitly state no pixels or records were provided. Research uses the existing bounded DeepAgents graph with the public question and neutral modality/count only, never prior chat or arbitrary viewer strings. These Gemini/NVIDIA lanes remain text-only; explicit Codex image input is described below.
- Persist transcript, progress, terminal tool results and dispatch model receipts. Text research uses normal `research_run` records and is eligible for explicit saved-result Jev review. During an already connected voice session the Research button directly schedules the existing research tool; normal typed Send still uses that voice conversation. The retired `/messages` stub stays retired.

## Subscription text/research

- `ai_codex.py` and `ai_codex_routes.py` own local per-actor Codex App Server stdio, browser login, status, model discovery and bounded text/PubMed execution. Require signed ai.run, enabled pilot/research, explicit write Origin, strict empty write bodies and private no-store errors. No generic RPC or browser tokens. Account mutations stop owned jobs. Clinical logout closes its process/pending login; subscription Sign out also clears its isolated keyring login.
- Workspace-pinned Codex 0.154.0 receives a private 0700 database-adjacent `.ai-codex/<actor-hash>` home, 077 umask, allowlisted environment and keyring-only forced ChatGPT auth. Never inherit API keys or the user's Codex config/auth. Fail closed without private storage/keyring.
- Every ephemeral thread/turn has no execution environments; shell, local images, plugins, hooks, memory and subagents are disabled. Verify exact model/provider, empty loaded instructions and read-only/network-disabled sandbox before sending text. Only bounded public PubMed dynamic calls are handled; deny all other requests. Eight tool calls, owned job/expiry deadline, final text only and ledger-backed citations. Unconfirmed cancellation terminates the child, including lost turn-start acknowledgements. Do not replay.
- Every ephemeral thread/turn has no execution environments; shell, filesystem-image tools, plugins, hooks, memory and subagents are disabled. Verify exact model/provider, empty loaded instructions and read-only/network-disabled sandbox before sending text or an explicit inline image. Only bounded public PubMed dynamic calls are handled; deny all other requests. Eight tool calls, owned job/expiry deadline, final text only and ledger-backed citations. Unconfirmed cancellation terminates the child, including lost turn-start acknowledgements. Do not replay.
- The codex research preference is authenticated catalog-backed and independent of voice. Gemini/NVIDIA research remains DeepAgents/LangGraph. Subscription uses the Codex harness; never mislabel its orchestration or bill it as an API key.

- `ai_view_image.py` owns explicit Codex viewport input: strict JPEG/base64, at most 512 KiB decoded, 768 px per edge, matching dimensions/target/context and capture age up to five minutes. Require the exact authenticated model to advertise image input before creating a job. Forward an inline App Server `image` data URL; never enable filesystem image, screen/computer or other autonomous tools. Pixels stay in memory for this one request and are omitted from persisted tool args, history and logs. Save only scope/dimensions/time/hash/model submission receipts. Prior chat images are historical text context and their pixels are never replayed. Keep visual observations separate from PubMed claims, and public queries free of identifiers. Voice session routes reject typed image attachments; clinical mode remains disabled.
Loading
Loading