Skip to content

fix(e2e): stabilize hosted-compatible release gates - #5885

Closed
cv wants to merge 7 commits into
mainfrom
fix/release-069-e2e-gates
Closed

cv wants to merge 7 commits into
mainfrom
fix/release-069-e2e-gates

Conversation

@cv

@cv cv commented Jun 26, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Fixes the first release-gate E2E failures found while running the v0.0.69 full nightly candidate: hosted-compatible CI onboarding now overrides explicit cloud selection, Deep Agents TUI checks skip non-Deep-Agents sandboxes before requiring expect, and JSON/live reachability probes tolerate CI warning and endpoint behavior.

Changes

  • Route NEMOCLAW_E2E_USE_HOSTED_INFERENCE=1 + NEMOCLAW_PROVIDER=cloud through the compatible endpoint while keeping normal explicit cloud behavior unchanged.
  • Preserve the hosted-compatible mode flag through the Vitest fixture env allowlist without passing source secrets by default.
  • Make Deep Agents TUI startup checks probe sandbox type before requiring/installing expect.
  • Avoid quota/flaky /v1/models preflights in live Vitest Hermes/sandbox-survival scenarios.
  • Skip provider-routed Model Router Vitest when the configured secret is not a public nvapi-* key.
  • Tolerate stderr warnings after JSON envelopes in legacy and Vitest sessions/agents E2E parsing.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: E2E/release-gate harness behavior only; no user-facing docs changed.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: self-review focused on CI-only NEMOCLAW_E2E_USE_HOSTED_INFERENCE routing and fixture secret filtering; source credentials remain excluded from default child env.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

  • PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
  • Git hooks passed during commit and push, or npx prek run --from-ref main --to-ref HEAD passes
  • Targeted tests pass for changed behavior
  • Full npm test passes (broad runtime changes only)
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Targeted verification run:

npm run build:cli
npm test -- --run src/lib/onboard/providers.test.ts test/deepagents-code-tui-startup-check.test.ts test/e2e-scenario/support-tests/hosted-inference.test.ts
bash -n test/e2e/e2e-cloud-experimental/checks/10-deepagents-code-tui-startup.sh test/e2e/test-sessions-agents-cli.sh
npm run typecheck:cli

Local hooks were attempted but the broad local test hook failed on unrelated environment-sensitive tests (Hermes tempdir mode and sandbox rlimit hook); the focused checks above cover this diff.


Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

  • New Features
    • Expanded hosted inference support with refined routing and safer credential/environment staging when running in trusted CI.
  • Bug Fixes
    • Improved hosted-inference readiness checks by using endpoint reachability probing instead of fixed URL response validation.
    • Reduced flakiness in JSON envelope handling and agents-list JSON parsing.
  • Tests
    • Added/updated unit and E2E coverage for hosted inference key precedence, env passthrough, sandbox skipping, and TUI startup behavior.
    • Improved CI startup checks to ensure expect is available when possible.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv cv self-assigned this Jun 26, 2026
@coderabbitai

coderabbitai Bot commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Hosted inference staging now rewrites compatible environment values and readiness checks use a reachability probe. Deep Agents startup now conditionally ensures expect, and session CLI JSON parsing retries multiple envelope candidates.

Changes

Hosted inference routing and probes

Layer / File(s) Summary
Hosted inference staging
src/lib/onboard/providers.ts, src/lib/onboard/providers.test.ts, test/e2e-scenario/fixtures/redaction.ts, test/e2e-scenario/support-tests/hosted-inference.test.ts
stageHostedInferenceSourceSecretEnv now stages hosted-compatible build selections, rewrites the provider to custom in hosted CI cases, preserves an explicit compatible credential when present, and keeps the new hosted-inference env key available to child processes.
Hosted endpoint probes
test/e2e-scenario/fixtures/clients/provider.ts, test/e2e-scenario/live/hermes-e2e.test.ts, test/e2e-scenario/live/sandbox-survival.test.ts, test/e2e-scenario/live/model-router-provider-routed-inference.test.ts
ProviderClient adds reachability probing, live hosted-inference checks switch to that probe, and the routed-inference scenario skips when the NVIDIA key lacks the nvapi- prefix.

Deep Agents startup check

Layer / File(s) Summary
Expect availability handling
test/e2e/e2e-cloud-experimental/checks/10-deepagents-code-tui-startup.sh, test/deepagents-code-tui-startup-check.test.ts
The startup check now verifies expect through ensure_expect_available(), and the test covers non-Deep-Agents sandboxes skipping the requirement.

Session JSON parsing

Layer / File(s) Summary
JSON envelope fallback
test/e2e-scenario/live/sessions-agents-cli.test.ts, test/e2e/test-sessions-agents-cli.sh
JSON envelope parsing now retries truncated candidates after parse failures, and shell JSON checks use json.JSONDecoder().raw_decode(...).

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • NVIDIA/NemoClaw#5672: Also changes src/lib/onboard/providers.ts’s hosted-inference staging logic and credential/env selection.
  • NVIDIA/NemoClaw#5805: Closely matches the hosted-inference staging and routing test updates in this PR.
  • NVIDIA/NemoClaw#5839: Related to the Deep Agents Code TUI startup check and its expect-handling changes.

Suggested labels

area: ci, area: e2e, area: onboarding, area: inference

Suggested reviewers

  • jyaunches

Poem

A bunny peeks at hosted light,
And hops by probe to check it right.
The shells say JSON, neat and clear,
While expect joins in when paths are near.
🐇 Thump-thump — all tests look bright.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and clearly reflects the main e2e change around hosted-compatible release gates.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/release-069-e2e-gates

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in the fix/release-069-e2e-... branch is 96%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.
File main fix/release-069-e2e-... 005ca78 +/-
nemoclaw/src/se...cret-scanner.ts — 100% —
nemoclaw/src/commands/slash.ts — 100% —
nemoclaw/src/li...bprocess-env.ts — 100% —
nemoclaw/src/bl...eprint/state.ts — 98% —
nemoclaw/src/onboard/config.ts — 98% —
nemoclaw/src/bl...int/snapshot.ts — 97% —
nemoclaw/src/bl...print/runner.ts — 95% —
nemoclaw/src/co...ration-state.ts — 94% —
nemoclaw/src/bl...ate-networks.ts — 94% —
nemoclaw/src/index.ts — 94% —

TypeScript / code-coverage/cli

The overall coverage in the fix/release-069-e2e-... branch is 47%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.
File main fix/release-069-e2e-... 005ca78 +/-
src/lib/state/o...oard-session.ts — 91% —
src/lib/actions...dbox/rebuild.ts — 73% —
src/lib/sandbox/config.ts — 72% —
src/lib/onboard/preflight.ts — 62% —
src/lib/shields/index.ts — 62% —
src/lib/actions...licy-channel.ts — 60% —
src/lib/state/sandbox.ts — 56% —
src/lib/policy/index.ts — 48% —
src/lib/onboard...er-gpu-patch.ts — 47% —
src/lib/onboard.ts — 19% —

Updated June 26, 2026 23:26 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.

@github-actions

github-actions Bot commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor

PR Review Advisor — Changes requested

Merge posture: Do not merge yet
Primary next action: Resolve or justify PRA-1: Source-of-truth review needed: Hosted inference live reachability probes.
Open items: 0 required · 8 warnings · 0 suggestions · 8 test follow-ups
Since last review: 2 prior items resolved · 6 still apply · 1 new item found

Action checklist

  • PRA-1 Resolve or justify: Source-of-truth review needed: Hosted inference live reachability probes
  • PRA-2 Resolve or justify: Source-of-truth review needed: Sessions/agents JSON envelope parsing
  • PRA-3 Resolve or justify: Source-of-truth review needed: Deep Agents TUI expect availability
  • PRA-4 Resolve or justify: Source-of-truth review needed: Hosted CI cloud/build provider override
  • PRA-5 Resolve or justify: Deep Agents TUI check still installs expect from live scenario code in test/e2e/e2e-cloud-experimental/checks/10-deepagents-code-tui-startup.sh:47
  • PRA-6 Resolve or justify: Provider reachability helper lacks a direct security contract test in test/e2e-scenario/fixtures/clients/provider.ts:218
  • PRA-7 Resolve or justify: Tolerant JSON envelope parsing still lacks an explicit contract in test/e2e-scenario/live/sessions-agents-cli.test.ts:181
  • PRA-8 Resolve or justify: PR description still claims hosted flag plus cloud is routed through compatible inference
  • PRA-T1 Add or justify test follow-up: Mocked behavioral coverage
  • PRA-T2 Add or justify test follow-up: Mocked behavioral coverage
  • PRA-T3 Add or justify test follow-up: Mocked behavioral coverage
  • PRA-T4 Add or justify test follow-up: Mocked behavioral coverage
  • PRA-T5 Add or justify test follow-up: Mocked behavioral coverage
  • PRA-T6 Add or justify test follow-up: Provider reachability helper lacks a direct security contract test
  • PRA-T7 Add or justify test follow-up: Acceptance clause
  • PRA-T8 Add or justify test follow-up: Hosted inference live reachability probes

Findings index

ID Severity Category Location Required action
PRA-1 Resolve/justify architecture — Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
PRA-2 Resolve/justify architecture — Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
PRA-3 Resolve/justify architecture — Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
PRA-4 Resolve/justify architecture — Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
PRA-5 Resolve/justify security test/e2e/e2e-cloud-experimental/checks/10-deepagents-code-tui-startup.sh:47 Prefer making `expect` a trusted workflow or runner prerequisite and fail with a clear missing-prerequisite message after the Deep Agents probe. If the in-script install is intentionally retained, add an in-code rationale explaining why the workflow/runner cannot own this prerequisite and add focused tests for the retained GitHub Actions install branch.
PRA-6 Resolve/justify tests test/e2e-scenario/fixtures/clients/provider.ts:218 Add focused support tests in `test/e2e-scenario/support-tests/e2e-clients.test.ts` using the existing `FakeRunner` to pin `probeReachability()` behavior.
PRA-7 Resolve/justify correctness test/e2e-scenario/live/sessions-agents-cli.test.ts:181 Extract the parser into a small tested helper or add focused tests beside this scenario that define the exact accepted envelope contract. If first-envelope-wins is intended, document that in the helper; otherwise reject multiple-envelope or malformed/truncated cases.
PRA-8 Resolve/justify scope — Either update the PR description/quality-gate text to match the reverted/current behavior, or reintroduce the intended code and tests proving `NEMOCLAW_E2E_USE_HOSTED_INFERENCE=1` plus explicit cloud uses the compatible endpoint while normal explicit cloud remains unchanged.
Review findings by urgency: 0 required fixes, 8 items to resolve/justify, 0 in-scope improvements

⚠️ Resolve or justify before merge

Investigate these in the current review; either fix them, explain why they are not applicable, or document the accepted risk.

PRA-1 Resolve/justify — Source-of-truth review needed: Hosted inference live reachability probes

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: Shell-helper tests cover no `/models`, no auth, HTTP `000`, and curl nonzero behavior. Missing direct tests should pin `ProviderClient.probeReachability()` argv, timeout validation, and redaction behavior.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: `hermes-e2e.test.ts` and `sandbox-survival.test.ts` now use `provider.probeReachability(...)`; `ci-compatible-inference.sh` comments document the no-auth/no-inference preflight rationale.

PRA-2 Resolve/justify — Source-of-truth review needed: Sessions/agents JSON envelope parsing

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as missing.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: No focused parser tests were added for leading diagnostics, trailing diagnostics, malformed JSON, truncated JSON, or multiple JSON envelopes.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: `parseJsonFromText()` suppresses parse errors while trying truncated candidates; the legacy shell parser uses `JSONDecoder().raw_decode(raw[offset:])`.

PRA-3 Resolve/justify — Source-of-truth review needed: Deep Agents TUI expect availability

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: A test now covers non-Deep-Agents skip before `expect`. Missing tests should cover Deep-Agents missing-prerequisite behavior and any retained GitHub Actions `sudo apt-get` branch.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: `ensure_expect_available()` is called only after the Deep Agents probe, but still installs `expect` via `sudo apt-get` when available on GitHub Actions.

PRA-4 Resolve/justify — Source-of-truth review needed: Hosted CI cloud/build provider override

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: If the override is intended, add a test for `NEMOCLAW_E2E_USE_HOSTED_INFERENCE=1` plus `NEMOCLAW_PROVIDER=cloud` proving compatible endpoint staging and explicit compatible credential precedence. If not intended, the existing Build-provider test is the relevant regression coverage once the PR text is corrected.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: `src/lib/onboard/providers.test.ts` contains `keeps explicit cloud provider selection on the Build provider path`; the current PR body says the opposite behavior is part of this change.

PRA-5 Resolve/justify — Deep Agents TUI check still installs expect from live scenario code

  • Location: test/e2e/e2e-cloud-experimental/checks/10-deepagents-code-tui-startup.sh:47
  • Category: security
  • Problem: `ensure_expect_available()` still runs `sudo apt-get update -qq` and `sudo apt-get install -y --no-install-recommends expect` when `GITHUB_ACTIONS=true`. This revision usefully moves that check after the Deep Agents sandbox probe and adds non-Deep-Agents skip coverage, but the Deep Agents path still mutates the host runner and fetches an unpinned OS package from repository-controlled live E2E code.
  • Impact: A credential-bearing live E2E lane can perform host package installation while executing PR-controlled test code, weakening the workflow trusted-code boundary and making installer provenance harder to audit for release-gate scenarios.
  • Recommended action: Prefer making `expect` a trusted workflow or runner prerequisite and fail with a clear missing-prerequisite message after the Deep Agents probe. If the in-script install is intentionally retained, add an in-code rationale explaining why the workflow/runner cannot own this prerequisite and add focused tests for the retained GitHub Actions install branch.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read `ensure_expect_available()` and confirm whether the Deep Agents path can still reach `sudo apt-get`; compare with `test/deepagents-code-tui-startup-check.test.ts`, which currently covers only the non-Deep-Agents skip-before-expect path.
  • Missing regression test: Add a helper-level test that simulates `NEMOCLAW_DCODE_PROBE:deepagents` with `expect` absent and verifies the intended behavior: either a clean missing-prerequisite failure without `apt-get`, or the explicit GitHub Actions install branch with mocked `sudo`/`apt-get` calls and no install attempt before the sandbox type probe.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read `ensure_expect_available()` and confirm whether the Deep Agents path can still reach `sudo apt-get`; compare with `test/deepagents-code-tui-startup-check.test.ts`, which currently covers only the non-Deep-Agents skip-before-expect path.
  • Evidence: The shell script defines `ensure_expect_available()` with `sudo apt-get update -qq && sudo apt-get install -y --no-install-recommends expect`; the new test `skips non-Deep-Agents sandboxes before requiring expect` proves only the skip path.

PRA-6 Resolve/justify — Provider reachability helper lacks a direct security contract test

  • Location: test/e2e-scenario/fixtures/clients/provider.ts:218
  • Category: tests
  • Problem: `ProviderClient.probeReachability()` centralizes the new unauthenticated provider reachability behavior, but the nearby `e2e-clients.test.ts` coverage still tests `requestJson()` and endpoint validation rather than this new helper's curl argv and redaction contract.
  • Impact: A future change could accidentally reintroduce `/models`, `Authorization` headers, redirect following, missing timeout validation, or query-secret artifact leakage in a credential-sensitive hosted inference preflight without a focused unit failure.
  • Recommended action: Add focused support tests in `test/e2e-scenario/support-tests/e2e-clients.test.ts` using the existing `FakeRunner` to pin `probeReachability()` behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect `test/e2e-scenario/support-tests/e2e-clients.test.ts` for calls to `provider.probeReachability`; current matches are only in live tests and the helper definition, not support tests.
  • Missing regression test: Add tests named for these behaviors: `provider reachability uses curl without auth headers, without /models, and without redirects`; `provider reachability rejects invalid connect and max timeouts before command construction`; and `provider reachability preserves caller probe options while redacting endpoint query values`.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect `test/e2e-scenario/support-tests/e2e-clients.test.ts` for calls to `provider.probeReachability`; current matches are only in live tests and the helper definition, not support tests.
  • Evidence: `hermes-e2e.test.ts` and `sandbox-survival.test.ts` now call `provider.probeReachability(...)`; `e2e-clients.test.ts` has `FakeRunner` tests for `requestJson()`, unsafe curl options, no redirects, and query redaction, but no direct `probeReachability()` tests.

PRA-7 Resolve/justify — Tolerant JSON envelope parsing still lacks an explicit contract

  • Location: test/e2e-scenario/live/sessions-agents-cli.test.ts:181
  • Category: correctness
  • Problem: `parseJsonFromText()` now scans for a JSON-looking line, tries the whole suffix, then repeatedly truncates at every `}` or `]` from the end until `JSON.parse()` succeeds. The shell E2E parser similarly switched to `JSONDecoder().raw_decode()`. This tolerates trailing diagnostics, but the accepted behavior for malformed, truncated, or multiple JSON envelopes is not tested or documented.
  • Impact: The E2E can pass on the first parseable prefix even if the command emitted a second JSON object, a malformed envelope, or diagnostics that indicate a real CLI/output contract regression.
  • Recommended action: Extract the parser into a small tested helper or add focused tests beside this scenario that define the exact accepted envelope contract. If first-envelope-wins is intended, document that in the helper; otherwise reject multiple-envelope or malformed/truncated cases.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read `parseJsonFromText()` in `sessions-agents-cli.test.ts` and `is_valid_json()` in `test/e2e/test-sessions-agents-cli.sh`; then check for nearby tests covering leading diagnostics, trailing diagnostics, malformed JSON, truncated JSON, and multiple JSON envelopes.
  • Missing regression test: Add behavior tests for `leading warning plus JSON object parses`, `pretty-printed JSON followed by stderr warning parses`, `truncated JSON fails`, `malformed JSON fails`, and either `multiple JSON envelopes are rejected` or `first JSON envelope wins and ignores later diagnostics`.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read `parseJsonFromText()` in `sessions-agents-cli.test.ts` and `is_valid_json()` in `test/e2e/test-sessions-agents-cli.sh`; then check for nearby tests covering leading diagnostics, trailing diagnostics, malformed JSON, truncated JSON, and multiple JSON envelopes.
  • Evidence: The TypeScript parser suppresses parse failures while trying truncated candidates; the legacy shell parser now uses `json.JSONDecoder().raw_decode(raw[offset:])`; no focused parser tests were added in this PR.

PRA-8 Resolve/justify — PR description still claims hosted flag plus cloud is routed through compatible inference

  • Location: not file-specific
  • Category: scope
  • Problem: The current diff no longer changes `src/lib/onboard/providers.ts`, and nearby tests still assert that explicit `NEMOCLAW_PROVIDER=cloud` stays on the Build provider path. The PR body, however, says `NEMOCLAW_E2E_USE_HOSTED_INFERENCE=1` plus `NEMOCLAW_PROVIDER=cloud` is routed through the compatible endpoint.
  • Impact: Reviewers and release-gate operators can believe a hosted-compatible cloud override is covered when the current code/tests preserve explicit cloud/build behavior, leading to incorrect environment selection or missed validation.
  • Recommended action: Either update the PR description/quality-gate text to match the reverted/current behavior, or reintroduce the intended code and tests proving `NEMOCLAW_E2E_USE_HOSTED_INFERENCE=1` plus explicit cloud uses the compatible endpoint while normal explicit cloud remains unchanged.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Compare the PR description against `src/lib/onboard/providers.ts` around `stageHostedInferenceSourceSecretEnv()` and the test named `keeps explicit cloud provider selection on the Build provider path` in `src/lib/onboard/providers.test.ts`.
  • Missing regression test: If the cloud override is intended, add a test that sets `NVIDIA_INFERENCE_API_KEY`, `NEMOCLAW_E2E_USE_HOSTED_INFERENCE=1`, and `NEMOCLAW_PROVIDER=cloud`, then asserts compatible endpoint staging and explicit `COMPATIBLE_API_KEY` precedence. If not intended, no code test is needed; update the PR text to remove the stale acceptance claim.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Compare the PR description against `src/lib/onboard/providers.ts` around `stageHostedInferenceSourceSecretEnv()` and the test named `keeps explicit cloud provider selection on the Build provider path` in `src/lib/onboard/providers.test.ts`.
  • Evidence: `stageHostedInferenceSourceSecretEnv()` only stages hosted custom for explicit/implicit custom paths; the existing providers test says explicit cloud stays on Build; `ci-compatible-inference.sh` preserves an existing `NEMOCLAW_PROVIDER` value when configuring compatible inference.

💡 In-scope improvements

These are lower-risk, not throwaway. Prefer fixing them in this PR when they are local to changed code; defer only with rationale or a linked follow-up.

  • None.
Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

  • PRA-T1 Mocked behavioral coverage — ProviderClient.probeReachability uses curl without Authorization headers, without /models, without -L, and writes only %{http_code}. The runtime scenarios are live E2E, but the changed security-sensitive helper and tolerant parsers can be pinned with small existing support-test fakes without executing external scripts, package managers, or live services.
  • PRA-T2 Mocked behavioral coverage — ProviderClient.probeReachability rejects invalid connectTimeoutSeconds and curlMaxTimeSeconds before command construction. The runtime scenarios are live E2E, but the changed security-sensitive helper and tolerant parsers can be pinned with small existing support-test fakes without executing external scripts, package managers, or live services.
  • PRA-T3 Mocked behavioral coverage — ProviderClient.probeReachability preserves artifactName/env/timeout options and redacts endpoint query values. The runtime scenarios are live E2E, but the changed security-sensitive helper and tolerant parsers can be pinned with small existing support-test fakes without executing external scripts, package managers, or live services.
  • PRA-T4 Mocked behavioral coverage — sessions/agents parser accepts leading diagnostics followed by a JSON object. The runtime scenarios are live E2E, but the changed security-sensitive helper and tolerant parsers can be pinned with small existing support-test fakes without executing external scripts, package managers, or live services.
  • PRA-T5 Mocked behavioral coverage — sessions/agents parser accepts pretty-printed JSON followed by a trailing warning. The runtime scenarios are live E2E, but the changed security-sensitive helper and tolerant parsers can be pinned with small existing support-test fakes without executing external scripts, package managers, or live services.
  • PRA-T6 Provider reachability helper lacks a direct security contract test — Add focused support tests in `test/e2e-scenario/support-tests/e2e-clients.test.ts` using the existing `FakeRunner` to pin `probeReachability()` behavior.
  • PRA-T7 Acceptance clause — No linked issue clauses were provided in the deterministic review context. — add test evidence or identify existing coverage. `linkedIssues` was empty, so there are no issue-body or issue-comment acceptance clauses to map. PR-body text was used only as untrusted scope evidence.
  • PRA-T8 Hosted inference live reachability probes — Shell-helper tests cover no `/models`, no auth, HTTP `000`, and curl nonzero behavior. Missing direct tests should pin `ProviderClient.probeReachability()` argv, timeout validation, and redaction behavior.. `hermes-e2e.test.ts` and `sandbox-survival.test.ts` now use `provider.probeReachability(...)`; `ci-compatible-inference.sh` comments document the no-auth/no-inference preflight rationale.
Since last review details

Current findings, using the urgency labels above:

PRA-1 Resolve/justify — Source-of-truth review needed: Hosted inference live reachability probes

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: Shell-helper tests cover no `/models`, no auth, HTTP `000`, and curl nonzero behavior. Missing direct tests should pin `ProviderClient.probeReachability()` argv, timeout validation, and redaction behavior.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: `hermes-e2e.test.ts` and `sandbox-survival.test.ts` now use `provider.probeReachability(...)`; `ci-compatible-inference.sh` comments document the no-auth/no-inference preflight rationale.

PRA-2 Resolve/justify — Source-of-truth review needed: Sessions/agents JSON envelope parsing

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as missing.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: No focused parser tests were added for leading diagnostics, trailing diagnostics, malformed JSON, truncated JSON, or multiple JSON envelopes.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: `parseJsonFromText()` suppresses parse errors while trying truncated candidates; the legacy shell parser uses `JSONDecoder().raw_decode(raw[offset:])`.

PRA-3 Resolve/justify — Source-of-truth review needed: Deep Agents TUI expect availability

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: A test now covers non-Deep-Agents skip before `expect`. Missing tests should cover Deep-Agents missing-prerequisite behavior and any retained GitHub Actions `sudo apt-get` branch.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: `ensure_expect_available()` is called only after the Deep Agents probe, but still installs `expect` via `sudo apt-get` when available on GitHub Actions.

PRA-4 Resolve/justify — Source-of-truth review needed: Hosted CI cloud/build provider override

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: If the override is intended, add a test for `NEMOCLAW_E2E_USE_HOSTED_INFERENCE=1` plus `NEMOCLAW_PROVIDER=cloud` proving compatible endpoint staging and explicit compatible credential precedence. If not intended, the existing Build-provider test is the relevant regression coverage once the PR text is corrected.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: `src/lib/onboard/providers.test.ts` contains `keeps explicit cloud provider selection on the Build provider path`; the current PR body says the opposite behavior is part of this change.

PRA-5 Resolve/justify — Deep Agents TUI check still installs expect from live scenario code

  • Location: test/e2e/e2e-cloud-experimental/checks/10-deepagents-code-tui-startup.sh:47
  • Category: security
  • Problem: `ensure_expect_available()` still runs `sudo apt-get update -qq` and `sudo apt-get install -y --no-install-recommends expect` when `GITHUB_ACTIONS=true`. This revision usefully moves that check after the Deep Agents sandbox probe and adds non-Deep-Agents skip coverage, but the Deep Agents path still mutates the host runner and fetches an unpinned OS package from repository-controlled live E2E code.
  • Impact: A credential-bearing live E2E lane can perform host package installation while executing PR-controlled test code, weakening the workflow trusted-code boundary and making installer provenance harder to audit for release-gate scenarios.
  • Recommended action: Prefer making `expect` a trusted workflow or runner prerequisite and fail with a clear missing-prerequisite message after the Deep Agents probe. If the in-script install is intentionally retained, add an in-code rationale explaining why the workflow/runner cannot own this prerequisite and add focused tests for the retained GitHub Actions install branch.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read `ensure_expect_available()` and confirm whether the Deep Agents path can still reach `sudo apt-get`; compare with `test/deepagents-code-tui-startup-check.test.ts`, which currently covers only the non-Deep-Agents skip-before-expect path.
  • Missing regression test: Add a helper-level test that simulates `NEMOCLAW_DCODE_PROBE:deepagents` with `expect` absent and verifies the intended behavior: either a clean missing-prerequisite failure without `apt-get`, or the explicit GitHub Actions install branch with mocked `sudo`/`apt-get` calls and no install attempt before the sandbox type probe.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read `ensure_expect_available()` and confirm whether the Deep Agents path can still reach `sudo apt-get`; compare with `test/deepagents-code-tui-startup-check.test.ts`, which currently covers only the non-Deep-Agents skip-before-expect path.
  • Evidence: The shell script defines `ensure_expect_available()` with `sudo apt-get update -qq && sudo apt-get install -y --no-install-recommends expect`; the new test `skips non-Deep-Agents sandboxes before requiring expect` proves only the skip path.

PRA-6 Resolve/justify — Provider reachability helper lacks a direct security contract test

  • Location: test/e2e-scenario/fixtures/clients/provider.ts:218
  • Category: tests
  • Problem: `ProviderClient.probeReachability()` centralizes the new unauthenticated provider reachability behavior, but the nearby `e2e-clients.test.ts` coverage still tests `requestJson()` and endpoint validation rather than this new helper's curl argv and redaction contract.
  • Impact: A future change could accidentally reintroduce `/models`, `Authorization` headers, redirect following, missing timeout validation, or query-secret artifact leakage in a credential-sensitive hosted inference preflight without a focused unit failure.
  • Recommended action: Add focused support tests in `test/e2e-scenario/support-tests/e2e-clients.test.ts` using the existing `FakeRunner` to pin `probeReachability()` behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect `test/e2e-scenario/support-tests/e2e-clients.test.ts` for calls to `provider.probeReachability`; current matches are only in live tests and the helper definition, not support tests.
  • Missing regression test: Add tests named for these behaviors: `provider reachability uses curl without auth headers, without /models, and without redirects`; `provider reachability rejects invalid connect and max timeouts before command construction`; and `provider reachability preserves caller probe options while redacting endpoint query values`.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect `test/e2e-scenario/support-tests/e2e-clients.test.ts` for calls to `provider.probeReachability`; current matches are only in live tests and the helper definition, not support tests.
  • Evidence: `hermes-e2e.test.ts` and `sandbox-survival.test.ts` now call `provider.probeReachability(...)`; `e2e-clients.test.ts` has `FakeRunner` tests for `requestJson()`, unsafe curl options, no redirects, and query redaction, but no direct `probeReachability()` tests.

PRA-7 Resolve/justify — Tolerant JSON envelope parsing still lacks an explicit contract

  • Location: test/e2e-scenario/live/sessions-agents-cli.test.ts:181
  • Category: correctness
  • Problem: `parseJsonFromText()` now scans for a JSON-looking line, tries the whole suffix, then repeatedly truncates at every `}` or `]` from the end until `JSON.parse()` succeeds. The shell E2E parser similarly switched to `JSONDecoder().raw_decode()`. This tolerates trailing diagnostics, but the accepted behavior for malformed, truncated, or multiple JSON envelopes is not tested or documented.
  • Impact: The E2E can pass on the first parseable prefix even if the command emitted a second JSON object, a malformed envelope, or diagnostics that indicate a real CLI/output contract regression.
  • Recommended action: Extract the parser into a small tested helper or add focused tests beside this scenario that define the exact accepted envelope contract. If first-envelope-wins is intended, document that in the helper; otherwise reject multiple-envelope or malformed/truncated cases.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read `parseJsonFromText()` in `sessions-agents-cli.test.ts` and `is_valid_json()` in `test/e2e/test-sessions-agents-cli.sh`; then check for nearby tests covering leading diagnostics, trailing diagnostics, malformed JSON, truncated JSON, and multiple JSON envelopes.
  • Missing regression test: Add behavior tests for `leading warning plus JSON object parses`, `pretty-printed JSON followed by stderr warning parses`, `truncated JSON fails`, `malformed JSON fails`, and either `multiple JSON envelopes are rejected` or `first JSON envelope wins and ignores later diagnostics`.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read `parseJsonFromText()` in `sessions-agents-cli.test.ts` and `is_valid_json()` in `test/e2e/test-sessions-agents-cli.sh`; then check for nearby tests covering leading diagnostics, trailing diagnostics, malformed JSON, truncated JSON, and multiple JSON envelopes.
  • Evidence: The TypeScript parser suppresses parse failures while trying truncated candidates; the legacy shell parser now uses `json.JSONDecoder().raw_decode(raw[offset:])`; no focused parser tests were added in this PR.

PRA-8 Resolve/justify — PR description still claims hosted flag plus cloud is routed through compatible inference

  • Location: not file-specific
  • Category: scope
  • Problem: The current diff no longer changes `src/lib/onboard/providers.ts`, and nearby tests still assert that explicit `NEMOCLAW_PROVIDER=cloud` stays on the Build provider path. The PR body, however, says `NEMOCLAW_E2E_USE_HOSTED_INFERENCE=1` plus `NEMOCLAW_PROVIDER=cloud` is routed through the compatible endpoint.
  • Impact: Reviewers and release-gate operators can believe a hosted-compatible cloud override is covered when the current code/tests preserve explicit cloud/build behavior, leading to incorrect environment selection or missed validation.
  • Recommended action: Either update the PR description/quality-gate text to match the reverted/current behavior, or reintroduce the intended code and tests proving `NEMOCLAW_E2E_USE_HOSTED_INFERENCE=1` plus explicit cloud uses the compatible endpoint while normal explicit cloud remains unchanged.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Compare the PR description against `src/lib/onboard/providers.ts` around `stageHostedInferenceSourceSecretEnv()` and the test named `keeps explicit cloud provider selection on the Build provider path` in `src/lib/onboard/providers.test.ts`.
  • Missing regression test: If the cloud override is intended, add a test that sets `NVIDIA_INFERENCE_API_KEY`, `NEMOCLAW_E2E_USE_HOSTED_INFERENCE=1`, and `NEMOCLAW_PROVIDER=cloud`, then asserts compatible endpoint staging and explicit `COMPATIBLE_API_KEY` precedence. If not intended, no code test is needed; update the PR text to remove the stale acceptance claim.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Compare the PR description against `src/lib/onboard/providers.ts` around `stageHostedInferenceSourceSecretEnv()` and the test named `keeps explicit cloud provider selection on the Build provider path` in `src/lib/onboard/providers.test.ts`.
  • Evidence: `stageHostedInferenceSourceSecretEnv()` only stages hosted custom for explicit/implicit custom paths; the existing providers test says explicit cloud stays on Build; `ci-compatible-inference.sh` preserves an existing `NEMOCLAW_PROVIDER` value when configuring compatible inference.

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard/providers.ts`:
- Around line 260-271: The build-selection staging path in providers.ts is
overriding an explicit COMPATIBLE_API_KEY with sourceKey because
hostedCompatibleBuildSelection now routes into the same write branch as implicit
cases. Update the staging logic around shouldStage and the subsequent write
branch in the provider-selection flow so that when compatibleKey is already
present, the code preserves that explicit credential instead of assigning
sourceKey; keep the existing precedence behavior tied to compatibleKey and only
populate sourceKey for implicit or non-compatible cases.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9030e8ed-f7b3-47e8-b49e-13f24a04bd1d

📥 Commits

Reviewing files that changed from the base of the PR and between c5902a8 and 34ced9b.

📒 Files selected for processing (11)
  • src/lib/onboard/providers.test.ts
  • src/lib/onboard/providers.ts
  • test/deepagents-code-tui-startup-check.test.ts
  • test/e2e-scenario/fixtures/redaction.ts
  • test/e2e-scenario/live/hermes-e2e.test.ts
  • test/e2e-scenario/live/model-router-provider-routed-inference.test.ts
  • test/e2e-scenario/live/sandbox-survival.test.ts
  • test/e2e-scenario/live/sessions-agents-cli.test.ts
  • test/e2e-scenario/support-tests/hosted-inference.test.ts
  • test/e2e/e2e-cloud-experimental/checks/10-deepagents-code-tui-startup.sh
  • test/e2e/test-sessions-agents-cli.sh

Comment thread src/lib/onboard/providers.ts Resolved
@github-actions

github-actions Bot commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: None
Optional E2E: hermes-e2e-vitest, sandbox-survival-vitest, sessions-agents-cli-vitest, model-router-provider-routed-inference-vitest, cloud-onboard-e2e, sessions-agents-cli-e2e

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • None. No merge-blocking E2E is required because this PR is tests-only: it changes E2E fixtures, support tests, live scenario tests, and shell test scripts, with no src/, installer, policy, onboarding state machine, credential runtime, sandbox lifecycle implementation, or deployment code changes.

Optional E2E

  • hermes-e2e-vitest (high): Optional validation for the changed Hermes live Vitest preflight and hosted reachability probe path.
  • sandbox-survival-vitest (high): Optional validation for the changed sandbox-survival hosted reachability preflight while preserving install/onboard, gateway restart, state survival, and inference.local coverage.
  • sessions-agents-cli-vitest (high): Optional validation for the changed Vitest sessions/agents JSON envelope parsing against the real CLI/gateway passthrough flow.
  • model-router-provider-routed-inference-vitest (high): Optional validation for the changed provider-routed model-router live test skip behavior and routed inference contract.
  • cloud-onboard-e2e (high): Optional validation because the changed Deep Agents Code TUI startup shell check is part of the cloud-onboard sandbox checks suite.
  • sessions-agents-cli-e2e (high): Optional validation for the changed legacy shell sessions/agents CLI E2E JSON parsing behavior.

New E2E recommendations

  • None.

@github-actions

github-actions Bot commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor

Vitest E2E Scenario Recommendation

Required Vitest E2E scenarios: model-router-provider-routed-inference-vitest, sandbox-survival-vitest, sessions-agents-cli-vitest, e2e-scenarios-all
Optional Vitest E2E scenarios: None

Dispatch required Vitest E2E scenarios:

  • gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=model-router-provider-routed-inference-vitest
  • gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=sandbox-survival-vitest
  • gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=sessions-agents-cli-vitest
  • gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref>

Workflow run

Full Vitest E2E advisor summary

Vitest E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required Vitest E2E scenarios

  • model-router-provider-routed-inference-vitest: Focused free-standing Vitest job wired for changed live test test/e2e-scenario/live/model-router-provider-routed-inference.test.ts.
    • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=model-router-provider-routed-inference-vitest
  • sandbox-survival-vitest: Focused free-standing Vitest job wired for changed live test test/e2e-scenario/live/sandbox-survival.test.ts.
    • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=sandbox-survival-vitest
  • sessions-agents-cli-vitest: Focused free-standing Vitest job wired for changed live test test/e2e-scenario/live/sessions-agents-cli.test.ts.
    • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=sessions-agents-cli-vitest
  • e2e-scenarios-all: Shared Vitest E2E fixture changes in provider client reachability and redaction/env allowlisting can affect multiple scenario paths, and several wired free-standing live Vitest tests were changed. Run the full Vitest scenario workflow fan-out to cover registry-backed scenarios plus the default-enabled free-standing jobs.
    • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref>

Optional Vitest E2E scenarios

  • None.

Relevant changed files

  • test/e2e-scenario/fixtures/clients/provider.ts
  • test/e2e-scenario/fixtures/redaction.ts
  • test/e2e-scenario/live/hermes-e2e.test.ts
  • test/e2e-scenario/live/model-router-provider-routed-inference.test.ts
  • test/e2e-scenario/live/sandbox-survival.test.ts
  • test/e2e-scenario/live/sessions-agents-cli.test.ts
  • test/e2e-scenario/support-tests/hosted-inference.test.ts

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@github-actions

github-actions Bot commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor

PR Review Advisor (Nemotron Ultra) — Informational

Merge posture: Informational / low confidence
Primary next action: Resolve or justify PRA-1: PR review advisor unavailable.
Open items: 0 required · 1 warning · 0 suggestions · 0 test follow-ups
Top item: PR review advisor unavailable

Action checklist

  • PRA-1 Resolve or justify: PR review advisor unavailable

Findings index

ID Severity Category Location Required action
PRA-1 Resolve/justify correctness — Re-run the PR Review Advisor or perform a manual review.
Review findings by urgency: 0 required fixes, 1 item to resolve/justify, 0 in-scope improvements

⚠️ Resolve or justify before merge

Investigate these in the current review; either fix them, explain why they are not applicable, or document the accepted risk.

PRA-1 Resolve/justify — PR review advisor unavailable

  • Location: not file-specific
  • Category: correctness
  • Problem: The automated advisor could not complete: Could not parse JSON from PR review advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/pr-review-advisor-nemotron-ultra/pr-review-advisor-retry-raw-output.txt
  • Impact: Automated review evidence is incomplete, so human review must cover the changed code manually.
  • Recommended action: Re-run the PR Review Advisor or perform a manual review.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the workflow logs and raw advisor artifact for the execution failure.
  • Missing regression test: No regression test recommendation is available because the advisor did not complete.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the workflow logs and raw advisor artifact for the execution failure.
  • Evidence: Could not parse JSON from PR review advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/pr-review-advisor-nemotron-ultra/pr-review-advisor-retry-raw-output.txt

💡 In-scope improvements

These are lower-risk, not throwaway. Prefer fixing them in this PR when they are local to changed code; defer only with rationale or a linked follow-up.

  • None.

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

cv added 5 commits June 26, 2026 15:43
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv

cv commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #5887 because force-push is disabled on PR branches and the replacement branch has a clean conventional commit history.

@cv cv closed this Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant