Skip to content

Security migration roll-up - #140

Merged
M3gA-Mind merged 246 commits into
tinyhumansai:mainfrom
senamakel:epic/8369-security-migration
Nov 3, 2026
Merged

M3gA-Mind merged 246 commits into
tinyhumansai:mainfrom
senamakel:epic/8369-security-migration

Conversation

@senamakel

@senamakel senamakel commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Prerequisite: TinyBus PR #116 must merge before this PR can merge. This branch pins TinyBus to its current head 0689566109edfc28c416ba927b2af4b88f1fb4cf, which provides Connection::accept_stream_from and Connection::downgrade; update the gitlink if the landed commit differs.

This is the upstream roll-up required by OpenHuman PR #8487. It moves native-shell lifecycle orchestration behind TinyBox's module boundary. The new TinyBus RunNativeShell operation keeps polling and cleanup inside TinyBox through completion or a yield point, returns finished output or a running process handle with its output cursor, and cancels/reaps work if the invocation is dropped. Hosts can continue a yielded process through the existing ReadOutput, Cancel, ReleaseOutput, and Close members.

Validation on this head: cargo test -p tinybox-bus -p tinybox-module passed (48 bus tests, 202 module tests, and bus doctests; 4 environment-dependent module e2e tests ignored). cargo check -p tinybox-module passed. The three new module tests cover completed output and cleanup, yielding with subsequent output polling, and cancellation cleanup after a dropped invocation.

senamakel and others added 30 commits October 26, 2026 05:48
Update the workspace package version and all internal path dependency
requirements from 0.2.2 to 0.2.3, with Cargo.lock regenerated to match.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
LocalGrantConfig now derives JsonSchema so OpenHuman config operations can
generate a schema from the shared contract type, with a test asserting the
schema is produced. The jail crate also gains helpers exposing the detected
backend name and whether Landlock is available.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
…g test

The schema test now serializes the generated schema to JSON and checks for a
properties key instead of inspecting the schemars object directly, and returns
a Result so serialization failures surface as test errors.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a runtime-neutral native shell client that runs commands and polls
bounded output, exposed alongside the existing bus types. The helper
depends on futures-util and futures-timer so it works without pulling in
a specific async runtime.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The timeout was read from request.shell after the shell had been moved into
start_native_shell_exec, so the deadline now uses a value captured up front.
This keeps the timeout clamping behaviour unchanged while avoiding the
use-after-move.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The timed_out flag was previously derived from cancellation_requested, so
caller-initiated cancellations were reported as timeouts; a dedicated flag now
tracks only deadline expiry. Read errors and failed executions also release the
output stream before returning, and the Failed state no longer leaks it.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reworked the NativeShellClient trait to return boxed futures instead of
using async fn, so it can be used as a trait object, and added a Sync
bound. Also fixed the timeout flag being set on cancellation rather than
on the delay branch, and documented the Transport error fields.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Replace the tokio test attribute with a plain test that drives the async
body through futures_executor::block_on, adding futures-executor as a
dev-dependency. This removes the tokio runtime requirement from the
native shell test.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Output batches containing chunks but reporting a next sequence at or
before the current cursor are now treated as an error, with the process
cancelled and its output released before returning. Cleanup failures are
also preserved instead of being dropped when another operation is
already failing, so the original error is no longer lost.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reworked the select arms and the running-state branch to use unit patterns and an empty block, and added an allow attribute for the long run_native_shell function so the process lifecycle stays in one auditable scope. The mock client methods now await a ready future to keep them genuinely async, and a missing semicolon in the test closure was fixed.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Extend the mock client with failure and cancellation switches so the
native shell tests can exercise two cleanup paths. One test asserts that
cancellation reaps the process before releasing and closing, and the
other asserts that a failed create still releases the resource
reservation.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reformatted the mock client's cancel check and the transport error
assertion so they fit within the configured line width. No test
behaviour changed.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Handle the ExecutionState::Failed variant carrying ExecutionFailure::TimedOut
separately so a timed out command releases its output and returns a result with
timed_out set, instead of falling through to the generic failure arm and
surfacing as an execution error.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a yield_after option to the native shell request and return a
NativeShellRunOutcome that either reports a finished process or hands back a
running one with its resource, process reference, cursor, and output collected
so far. This lets callers take over the cleanup lifecycle of long-running
shells instead of always waiting for exit.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The native shell runner now returns a Running outcome carrying the process,
cursor, and captured output when a configured yield_after duration elapses,
instead of blocking until the command exits. This lets callers poll or resume
long-running commands without holding the request open indefinitely.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add tests for the module timeout path returning collected output as a
finished result and for yielding preserving the process resource, cursor
and partial output. The mock client gains timed-out and stays-running
states, and existing tests now unwrap the new run outcome type.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add the ExecutionFailure import to the native shell test module so the
tests can reference the failure type.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Wrap the TinyBox product name in backticks within the NativeShellRunOutcome
doc comment so rustdoc renders it as code.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The mock client's four independent boolean flags are collapsed into a single
MockBehavior enum, so each test now selects one named scenario instead of
setting every flag explicitly. This makes the intended state of each test
obvious and prevents contradictory combinations.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The yield deadline is now dropped when it is not shorter than the shell
timeout, and the elapsed clock starts before the process is created so
early setup time counts toward the yield. Output batches that arrive
alongside a terminal state are skipped before the yield check, and
cancellation takes precedence over yielding so a cancelled run is never
reported as still running.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The crate-level doc comment now notes that the crate also provides runtime-neutral native-shell lifecycle orchestration over a caller-supplied client, so the summary matches what the module actually offers.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The function no longer trips the too_many_lines lint, so the attribute and its
explanatory comment were removed to keep the code free of unnecessary
suppressions.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
…achine

The native shell run loop was restructured into a dedicated state machine
that tracks the process lifecycle across polling, cancellation, and
timeout transitions. This makes the control flow easier to follow and
keeps the cleanup paths consistent when a run is interrupted or resumed.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
…bility

Capture the process handle returned by start_native_shell so it can be
passed to collect_native_shell, and make the fused cancellation future
mutable where it is polled. The previous bindings left the process
unused and the cancellation stream immutable, which prevented the shell
from being collected and the cancellation branch from being polled.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The terminal-state match arms in the poll loop were folded into a new
finish_native_shell helper that releases output and returns the finished
capture, leaving the loop to handle only the running and yield cases. The
redundant continue statements after cancellation and timeout are gone since
those branches now fall through to the same yield check.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Extend the mock client with failure behaviors for reserve, start, read, cancel, release, and close, plus failed-execution and invalid-cursor cases, and add tests asserting these errors are reported and resources cleaned up.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a mock behavior that reports a failed execution and also fails on
close, and use it in the cleanup test so the primary error is checked
when both the execution and the subsequent close fail. The cancelled
state check no longer depends on a prior cancel call, since the mock
behavior alone determines the reported state.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
…anup failures

Extend the mock client with timed-out and invalid-cursor cleanup failure
behaviors so the read path can block until a cancel arrives. Add tests
asserting that a timeout cancels the process and returns the collected
output, and that cancel and release failures after an invalid cursor are
surfaced as cleanup errors.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The mock client now returns Running instead of Cancelled when a cancelled or timed out behavior is polled without a cancellation flag, so tests can distinguish a pending execution from a cancelled one.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The capability wire snapshot test still expected contract version 2.2, so it now fails against the current serialization. Bump the expected version to 2.3 to match the contract the code emits.

Auto-committed-on: cuttlefish
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Brings in the OpenShell backend (tinyhumansai#136, tinyhumansai#138) so a module built from this
branch serves the backend OpenHuman's OpenShell driver creates. Keeps this
branch's 0.2.4 versions, its TinyBus pin (0689566, ABI-identical to tinyhumansai#116's
head) and its storage pins, and adds tinybox-openshell to the workspace
and lockfile.
The merge from main adds the openshell backend without a contract bump;
this branch already moves the contract to 2.3, so document it there and
add the 2.3 spec page.
The guard matched only #[allow(... unsafe_code ...)], so cfg_attr, expect,
warn, a multi-line attribute or a crate manifest override passed it. Treat
any non-comment mention of unsafe_code as a relaxation unless it is the one
file-scoped allowance in windows.rs or the workspace deny, add behavioral
tests for each spelling, and correct AGENTS.md, which still said no crate
relaxes the lint.
The six tinystoragedrivers crates pointed at a personal fork rev. The
canonical v2.1.2 commit main uses (6226b94) carries the same zstd advisory
fix; the fork's extra commits touch only crates tinybox does not build
(secrets, kv, queue, blob, fence), and every lockfile entry has the same
dependencies, so only the source changes.
Backticks around TinyBox in a doc comment, tests that return Result
instead of calling expect(), and assert_ne! for the empty-name check.
RunNativeShell maps a failure before the shell runs to the bus's backend
error; a request naming an unsupported backend exercises it.
A shell still running at its timeout is cancelled through the module's
own adapter and reported as a finished run that timed out.
The request helper names BTreeMap::default(), and the two tests that
expect a finished or a running shell return an error for the other
outcome instead of panicking.
@M3gA-Mind
M3gA-Mind marked this pull request as ready for review November 3, 2026 06:44

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes: 3 lane(s) blocking, worst finding is high.

Fix or reply to the findings below and push. The next review clears this automatically once they are gone — you should not need to dismiss anything by hand.

Diagnostics

Total spend: $0.0046

A member without `[lints] workspace = true` has no unsafe_code lint at all,
so the root deny never reaches it.
A zero timeout was floored for collection but sent as zero to the module,
which refuses it. Output chunks must now continue exactly at the cursor, so
a replayed or skipped chunk fails as an invalid cursor. The failing-cancel
test keeps its read pending so cancellation always wins the select.
…events honest

The jailed arm called run_command_observed, which supplies an empty map, and
the jail launcher then cleared the shell's approved environment. It now passes
that environment explicitly. A timed-out stream keeps its CPU limit event and
reports its exit only once cleanup is acknowledged. The lifecycle event test
uses cmd on Windows, like the test beside it.
…rlimit at create

RLIMIT_AS bounds each process, not the box, so a hard memory requirement is
now refused under the fallback. A fallback box is refused at create when no
trusted prlimit exists, instead of at its first exec. The cgroup CPU report
reads cpu.stat with a bash builtin rather than a cat found through PATH.
…ronment

A hard Network or Subprocess requirement now clears allow_net or
allow_subprocess, so a backend never launches with access its preflight said
was denied. The default owned spawn with an environment clears the command's
own environment and applies the approved map. The spawns without a map go to
spawn_owned_captured, so the child keeps the environment the command sets.
The README and lib comment describe the Windows backend as it now ships.
Profile names carry a hash of the full label and stay within 64 characters,
so labels that sanitize alike no longer share a SID. A CPU limit below the
smallest job rate is refused instead of being rounded up. A spawn without an
environment map passes the variables the command sets.
The storage crates moved back to tinyhumansai/tinystoragedrivers, but
deny.toml still allowed only the fork, so cargo-deny's source check failed.
Nothing in Cargo.lock uses the fork any more.
cmd ends echo lines with CRLF, and does not take `;` as a command separator,
so the yield test exited 1 there. Line endings are normalized and the yield
test waits with ping under cmd.
…error

An output-limit or cancelled collection returned early and skipped the CPU
report, so a throttled run lost its CpuQuota cause.
create and exec now resolve the trusted prlimit through one helper, and a
test covers each case of it, including the refusal when none exists.

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes: 2 lane(s) blocking, worst finding is high.

Fix or reply to the findings below and push. The next review clears this automatically once they are gone — you should not need to dismiss anything by hand.

Diagnostics

Total spend: $0.0004

Comment thread AGENTS.md Resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority: p1 Next. Wrong behaviour a user will hit, or a security weakness behind a condition.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants