Skip to content

test(runner): pass an explicit per-test timeout and grace to program rows - #5450

Open
elezar wants to merge 1 commit into
mainfrom
test/5416-program-row-timeout/elezar
Open

elezar wants to merge 1 commit into
mainfrom
test/5416-program-row-timeout/elezar

Conversation

@elezar

@elezar elezar commented Nov 10, 2026

Copy link
Copy Markdown
Member

Summary

Program rows relied on the program's 900 s per-test default, because the runner passed neither --timeout nor --grace. One hung test could use much of the 45-minute run timeout. The nextest path stopped a test after 120 s.

  • Derive the per-test timeout (30 s x 4 = 120 s) from shared constants, and generate the nextest profile from them so the two paths cannot drift apart.
  • Pass --timeout and --grace (10 s, nextest's default) to the program.
  • Add a test for the generated nextest config.

Related Issue

Part of #5416 (first item only; the issue stays open). Groundwork for #5333, which propagates this budget into testkit's inner calls.

Testing

  • cargo fmt --check, cargo clippy --all-targets -- -D warnings and cargo test for tests/runner pass.
  • The full e2e lane was not run locally.

…rows

Program rows relied on the program's 900 s default, so one hung test could use much of the 45-minute run timeout. Derive the per-test timeout from the same constants as the nextest profile (120 s) and pass it with a 10 s grace to the program.

Part of #5416.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
@github-actions

Copy link
Copy Markdown

Compatibility review (advisory)

New or updated breaking changes; see the full report for all findings.

1. cli — No explicit risk acceptance found

openshell-test-runner run --program changes its child invocation from test-program run --junit PATH (900-second default) to test-program run --timeout 120 --grace 10 --junit PATH.

  • Impact: Existing prebuilt testkit suites with tests requiring 120–900 seconds now receive SIGTERM and report timeout failures with exit code 1, without a runner flag to restore their previous budget.
  • Action: Preserve the 900-second --program default in v0.1.8 and make the 120-second limit opt-in through a documented runner flag.
  • PR: test(runner): pass an explicit per-test timeout and grace to program rows #5450
Review details
  • Execution: complete
  • Candidate: PR #5450 (29a08e491d961743824200fac035e389f911d875)
  • Baseline: 7873aaa3b8a3529459034cfce3079077a1a1e67b (7873aaa3b8a3529459034cfce3079077a1a1e67b)
  • Reviewed: rust-sdk, python-sdk, go-sdk, typescript-sdk, api-behavior, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant