Skip to content

test: start interactive bash in tests without the developer's startup files - #361

Merged
ThomasK33 merged 2 commits into
mainfrom
test/hermetic-bash-in-tests
Nov 7, 2026
Merged

ThomasK33 merged 2 commits into
mainfrom
test/hermetic-bash-in-tests

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Nov 7, 2026 •

Copy link
Copy Markdown
Member

Why

A test that starts a plain /bin/bash runs the developer's ~/.bashrc. That file can print text, ask a question, or end the shell. Then the test types into a shell that is not the one it expects.

This broke the new test "answers a run whose catch-up replay goes on for longer than the client waits" from #359. In one checkout, mise printed a "Trust them?" prompt at shell start. The prompt read the first typed line. The test failed 3 of 3 runs, and its event log showed bash: syntax error near unexpected token 'do'. With /bin/bash --norc --noprofile -i the same steps passed.

lifecycle.test.ts already starts its shells that way. Nine other tests did not.

What

Test-only change. No file under src/ changes.

  • test/helpers.ts adds two exports:
    • HERMETIC_BASH: the command /bin/bash --norc --noprofile -i.
    • createBashSession(testHome, options): createSession with that command and HISTFILE=/dev/null, so the shell neither reads nor writes ~/.bash_history.
  • run.test.ts: 8 sessions use createBashSession. Seven started a plain /bin/bash. One ("does not log postamble cursor controls when shell echo is disabled") already passed --noprofile --norc but wrote ~/.bash_history.
  • batch.test.ts: 1 session ("reads the step array from --file") uses createBashSession.
  • slow-replay.test.ts: the fix: keep a live render waiting while its replay moves forward #359 test passes HERMETIC_BASH to its own helper. The test already sets HISTFILE=/dev/null in the env it passes.

Not changed:

  • The three lifecycle.test.ts tests that start bash --norc --noprofile -i by hand. They are already hermetic.
  • Interactive /bin/sh sessions: run.test.ts (3 tests), io-loop.test.ts (exec /bin/sh) and the --shell test in lifecycle.test.ts. Here /bin/sh is dash. A non-login interactive dash reads no file unless the developer exports ENV, so these tests have no startup-file dependency by default. I checked this with a scratch HOME that holds a .profile and a .bashrc. No /bin/sh session read either file.

Validation

All runs used Node 26.8.1, one worker, no retry. The host load average was 60 to 120, so many create calls failed with exit 6. I judged each red test by its assertion and reran the ones that failed in createSession.

Scratch HOME proof. A scratch HOME under /tmp held a .bashrc that appends a line to a marker file, then either (a) prints "Trust them?" and reads one line, or (b) prints do and runs exit 3. Each file was run with that HOME.

  • origin/main (c9f7152), variant (b), whole files: 9 tests fail and 9 marker lines appear. They are 7 in run.test.ts, 1 in batch.test.ts and the fix: keep a live render waiting while its replay moves forward #359 test in slow-replay.test.ts. Every failure is an assertion after create worked (expected 4 to be +0, the session had exited). The three /bin/sh tests and the sh -c tests pass and add no marker line.
  • origin/main, variant (a): the fix: keep a live render waiting while its replay moves forward #359 test fails with "timed out waiting for the printed lines". Two run.test.ts tests also fail on assertions (waits for the command to complete before reporting success, returns timedOut when run completion is not observed within timeout).
  • This branch, variants (a) and (b), whole files: run.test.ts 15/15, batch.test.ts 11/11, slow-replay.test.ts 5/5. The one exception is variant (b) in slow-replay.test.ts: one test that this PR does not change (it starts sh -c) failed in createSession with exit 6 under load, and it passed alone on rerun. The marker file got 0 lines, and no .bash_history appeared in either scratch HOME.
  • This branch, real HOME: the same three files pass 15/15, 11/11 and 5/5.
  • Mutation: removing --norc from HERMETIC_BASH makes returns immediately with --no-wait, reads the step array from --file and the fix: keep a live render waiting while its replay moves forward #359 test fail again under variant (b). Removing HISTFILE=/dev/null leaves the tests green but creates .bash_history in the scratch HOME.

Dogfood with the real CLI (npx tsx src/cli/main.ts, isolated --home under /tmp, variant (a) HOME): with /bin/bash, run 'echo hello-old' answered the prompt and printed no hello-old. With /bin/bash --norc --noprofile -i, the same run printed hello-new.

Gates. npm run format:check, npm run lint and npm run typecheck pass. npx vitest run test/unit passes except probeUsedFonts > stops between batches once the time budget is spent, a timing test that does not import test/helpers.ts. It failed once under load and passed twice alone. check.sh --drift-only reports "Covered code matches the sync record." Other integration files that import test/helpers.ts also passed: pty-basics and io-loop. In event-log and lifecycle, 3 tests failed in createSession (exit 6) under load, and all passed on rerun.


Generated with xum • Model: anthropic:claude-sonnet-5-5 • Thinking: xhigh

… files

A test that started a plain /bin/bash loaded ~/.bashrc and ~/.profile. A startup file that prompts, prints or ends the shell broke the commands the test typed (a mise 'Trust them?' prompt swallowed one). Start these shells with --norc --noprofile -i and HISTFILE=/dev/null, like lifecycle.test.ts does.
This test already passed --noprofile --norc, but not -i or HISTFILE=/dev/null, so it still wrote ~/.bash_history. Use createBashSession like the other run tests.
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-07T00:37:54.300590Z 3e6b118 Manual request
🔒 Security Review ✅ Completed 2026-11-07T00:39:31.006612Z 3e6b118 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 3e6b118017

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 3e6b118017

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33
ThomasK33 added this pull request to the merge queue Nov 7, 2026
Merged via the queue into main with commit 9ba4e9d Nov 7, 2026
27 checks passed
@ThomasK33
ThomasK33 deleted the test/hermetic-bash-in-tests branch November 7, 2026 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant