From 4f4494e243743b5fd7c2f606a9488b345105893d Mon Sep 17 00:00:00 2001 From: Juniper Alanna <201364921+juniper-shopify@users.noreply.github.com> Date: Thu, 7 May 2026 15:48:23 -0400 Subject: [PATCH] Add LLM-generated detailed architecture documentation --- .claude/commands/intro.md | 2 + .../architecture/01-architecture-overview.md | 410 +++++++ .../architecture/02-dsl-users-guide.md | 1013 ++++++++++++++++ .../architecture/03-cog-reference.md | 1035 +++++++++++++++++ .../architecture/04-design-philosophy.md | 312 +++++ .../05-execution-engine-internals.md | 904 ++++++++++++++ .../architecture/06-metaprogramming-map.md | 597 ++++++++++ .../architecture/07-control-flow-reference.md | 490 ++++++++ .../08-infrastructure-and-events.md | 768 ++++++++++++ .../architecture/09-error-hierarchy.md | 289 +++++ .../architecture/10-writing-custom-cogs.md | 830 +++++++++++++ .../architecture/11-testing-guide.md | 766 ++++++++++++ .../12-known-issues-and-gotchas.md | 391 +++++++ 13 files changed, 7807 insertions(+) create mode 100644 .claude/commands/intro.md create mode 100644 internal/documentation/architecture/01-architecture-overview.md create mode 100644 internal/documentation/architecture/02-dsl-users-guide.md create mode 100644 internal/documentation/architecture/03-cog-reference.md create mode 100644 internal/documentation/architecture/04-design-philosophy.md create mode 100644 internal/documentation/architecture/05-execution-engine-internals.md create mode 100644 internal/documentation/architecture/06-metaprogramming-map.md create mode 100644 internal/documentation/architecture/07-control-flow-reference.md create mode 100644 internal/documentation/architecture/08-infrastructure-and-events.md create mode 100644 internal/documentation/architecture/09-error-hierarchy.md create mode 100644 internal/documentation/architecture/10-writing-custom-cogs.md create mode 100644 internal/documentation/architecture/11-testing-guide.md create mode 100644 internal/documentation/architecture/12-known-issues-and-gotchas.md diff --git a/.claude/commands/intro.md b/.claude/commands/intro.md new file mode 100644 index 00000000..994c8632 --- /dev/null +++ b/.claude/commands/intro.md @@ -0,0 +1,2 @@ +Read all of the documentation in `internal/documentation/architecture` in order to get a comprehensive understanding of this codebase + diff --git a/internal/documentation/architecture/01-architecture-overview.md b/internal/documentation/architecture/01-architecture-overview.md new file mode 100644 index 00000000..37d52a08 --- /dev/null +++ b/internal/documentation/architecture/01-architecture-overview.md @@ -0,0 +1,410 @@ +# Roast Architecture Overview + +> **Read this first.** This document provides the foundational mental model for +> the entire Roast framework. Every other document in this set assumes you've +> read and understood the concepts here. + +--- + +## What Is Roast? + +Roast is a **Ruby DSL for building structured AI workflows**. It orchestrates +four kinds of operations — shell commands, LLM conversations, AI coding agents, +and pure Ruby logic — into repeatable, composable pipelines. + +It is released as the **`roast-ai`** gem (entry point: `lib/roast-ai.rb`, which +simply requires `lib/roast.rb`). Invoked via `devx roast` (Shopify internal) or +`bundle exec roast`. Source code lives at `Shopify/roast`. + +Key runtime dependencies: `activesupport` (~> 8.0), `async` (>= 2.34), +`ruby_llm` (>= 1.8), `type_toolkit` (>= 0.0.5), `zeitwerk` (>= 2.6). Requires +Ruby >= 3.3.0. + +All source files in `lib/` are `typed: true` under Sorbet, but the gem has **no +`sorbet-runtime` dependency** — types are enforced at development time only, via +inline `#:` RBS annotations and RBI shim files (see +[06-metaprogramming-map.md](06-metaprogramming-map.md)). This was a deliberate +architectural decision (PR #476). + +--- + +## The Declarative Philosophy + +Roast is **declarative**: you describe _what_ should happen, and the framework +decides _when_ and _how_ to execute it. This is achieved through a strict +**two-phase lifecycle**: + +1. **`prepare!`** — Collect and bind. The framework reads your workflow + definition, gathers all `config {}` and `execute {}` blocks, resolves + configurations, and builds the execution plan. No cog actually runs yet. + +2. **`start!`** — Execute. The framework walks the execution plan, running each + cog in order, honoring async/sync settings, and producing outputs. + +This is analogous to **Terraform plan/apply**, **React's virtual DOM +reconciliation**, or **Rails migration DSL** — declaration is separated from +execution so the framework can analyze, validate, and optimize the plan before +committing to side effects. + +**The key implication**: All cog declarations are fixed at prepare time. The +`execute {}` block builds the cog stack during `prepare!`, not during `run!`. +You cannot conditionally create cogs at runtime. What you _can_ do is +conditionally provide input to cogs or skip them (via `skip!` in their input +blocks), but the set of cogs that _exist_ in a scope is determined before any +cog runs. + +The distinction: +- **`config {}` blocks** define _what cogs look like_ (their configuration). +- **`execute {}` blocks** define _what cogs exist and in what order_ (the execution plan). +- **Input blocks** (the `{ |my| ... }` passed to each cog invocation inside + `execute`) are **arbitrary Ruby code** that prepares a cog to run. These run at + execution time, not at prepare time. + +> **⚠️ Key Concept: What Input Blocks Really Are** +> +> An input block is NOT just "set some fields on an object." It is a full Ruby +> execution context. Its **primary** job is to populate an uninitialized Input +> instance with the values the cog needs, but you can do **any work** in it: +> set up the file system, load data, compute values, create temp files, +> transform outputs from prior cogs, etc. Think of it as a preparation phase +> where you do whatever work is needed _before_ the cog's own execution fires. +> +> For standard cogs (`cmd`, `chat`, `agent`), the cog's own execution — running +> the shell command, calling the LLM, invoking the agent CLI — happens **after** +> the input block returns and the framework validates the input. +> +> The `ruby` cog is the deliberate exception: it is a **no-op cog**. Its +> `execute` method does nothing except pass through the `value` field from its +> Input as its Output (`Output.new(input.value)`). The `ruby` cog exists so that +> you can write arbitrary Ruby code in an input block without needing a +> "real" cog underneath. All the actual work happens in the input block itself. +> It was named `ruby` (not `no-op`) because from the workflow author's +> perspective, it _looks like_ writing Ruby code that Roast executes — even +> though technically the execution is happening in the input context, not in +> `execute(input)`. + +**Source**: `lib/roast/workflow.rb` — `extract_dsl_procs!` (line 134) collects +procs via `instance_eval`; `prepare!` (line 46) evaluates them into configs and +cog stacks; `start!` (line 61) runs the execution manager. + +--- + +## The Three Evaluation Contexts + +This is the single most important concept in Roast. Master this, and everything +else follows. + +Roast defines **three empty classes** — `ConfigContext`, `ExecutionContext`, and +`CogInputContext` — onto which methods are **dynamically defined at runtime** +using `define_singleton_method`. Each class serves a different purpose, and **the +same method name does entirely different things depending on which context it +appears in.** + +For example, calling `agent(:analyze) { ... }`: +- In a **`config {}`** block → configures the agent cog named `:analyze` (sets model, temperature, etc.) +- In an **`execute {}`** block → declares an agent cog named `:analyze` and saves the block as its input proc +- In a **cog input block** → retrieves the output of the agent cog named `:analyze` (nil-safe) + +This triple-dispatch is the core of Roast's declarative design. Each context is +managed by a dedicated manager class: + +### ConfigContext (`lib/roast/config_context.rb`) + +**Source**: A single-line empty class: `class ConfigContext; end`. + +At prepare time, `ConfigManager` defines methods on the ConfigContext instance +via `define_singleton_method` (`config_manager.rb`, lines 86–96). For each +registered cog type, a method is created that dispatches to +`ConfigManager#on_config`. When you write: + +```ruby +config do + agent(:analyze) { model "claude-sonnet-4-20250514"; temperature 0.0 } +end +``` + +…the `agent(:analyze)` call routes to `on_config(Agent, :analyze, block)`, which +looks up or creates the name-scoped config for `Agent` cogs named `:analyze`, +then `instance_exec`s the block against that config object (so `model "..."` and +`temperature 0.0` are method calls on an `Agent::Config` instance). + +### ExecutionContext (`lib/roast/execution_context.rb`) + +**Source**: A single-line empty class: `class ExecutionContext; end`. + +At prepare time, `ExecutionManager` defines methods on the ExecutionContext +instance (`execution_manager.rb`, lines 186–197). For each registered cog type, +a method is created that dispatches to `ExecutionManager#on_execute`. It also +defines `outputs` and `outputs!` methods. When you write: + +```ruby +execute do + agent(:analyze) { |my| my.prompt = "Analyze this code" } +end +``` + +…the `agent(:analyze)` call routes to `on_execute(Agent, [:analyze], {}, +block)`, which creates a new `Cog` instance with the block saved as +`@cog_input_proc` (NOT evaluated yet), then pushes the cog onto the execution +stack. **The input block runs later**, during `Cog#run!`, not during `prepare!`. + +### CogInputContext (`lib/roast/cog_input_context.rb`) + +**Source**: Has hardcoded control flow methods (`skip!`, `fail!`, `next!`, +`break!`) and includes `Call::InputContext` and `Map::InputContext` modules. + +At construction time (which happens during `ExecutionManager#initialize`), +`CogInputManager` defines **three methods per registered cog type** on the +CogInputContext instance (`cog_input_manager.rb`, lines 40–51): +- `agent(:name)` — returns the output of the named cog, or `nil` on error (except `CogDoesNotExistError`, which always raises) +- `agent!(:name)` — returns the output or raises on any error; blocks if the cog is still running (async) +- `agent?(:name)` — returns `true`/`false` + +It also defines workflow parameter accessors: `target!`, `targets`, `arg?`, +`args`, `kwarg`, `kwarg!`, `kwarg?`, `kwargs`, `tmpdir`, `template`. + +And from the included modules: `from` (Call::InputContext), `collect` and +`reduce` (Map::InputContext). + +### Why Three Contexts? + +The separation enforces that: +- **Configuration** cannot accidentally create cogs or access outputs. +- **Declaration** cannot accidentally mutate config or access outputs. +- **Input evaluation** cannot accidentally create new cogs or change config. + +Each context is a **deep module** in the Ousterhout sense — its interface is +simple (call `agent(:name) { ... }`), but behind each call is a complex dispatch +chain through the corresponding manager class. The empty class definitions and +dynamic method binding are what make this possible. + +**For AI agents**: The RBI shim files (`sorbet/rbi/shims/lib/roast/`) are the +canonical documentation for every dynamically-defined method across all three +contexts. `config_context.rbi` (322 lines), `execution_context.rbi` (496 lines), +and `cog_input_context.rbi` (1,197 lines) document types, usage examples, and +cross-references for every method. See +[06-metaprogramming-map.md](06-metaprogramming-map.md) for the complete dynamic +method binding reference. + +--- + +## The Cog Taxonomy + +Every operation in Roast is a **cog** — a unit of work with a standardized +lifecycle. There are two families: + +### Standard Cogs (`lib/roast/cogs/`) + +| Cog | Purpose | Key Output Fields | +|-----|---------|-------------------| +| **`cmd`** | Run a shell command via `CommandRunner` | `.out`, `.err`, `.status` | +| **`chat`** | Single LLM conversation turn via `RubyLLM` | `.response`, `.session` | +| **`agent`** | AI coding agent invocation via CLI subprocess (Claude or Pi) | `.response`, `.session`, `.stats` | +| **`ruby`** | No-op cog — all work happens in the input block; `execute` just passes through `.value` | `.value` (delegates via `method_missing`) | + +### System Cogs (`lib/roast/system_cogs/`) + +| Cog | Purpose | Key Behavior | +|-----|---------|--------------| +| **`call`** | Invoke a named execution scope | Creates a child `ExecutionManager` for the named scope | +| **`map`** | Iterate over a collection (serial or parallel) | Creates one child EM per item; supports `Async::Semaphore` concurrency limiting | +| **`repeat`** | Loop until `break!` or `max_iterations` | Chains output → input across iterations | + +All cogs share the same base lifecycle: **Config → Input → Execute → Output**. +System cogs extend this with **Params** (set at declaration time), **Manager +modules** (mixed into `ExecutionManager` to orchestrate child scopes), and +**InputContext modules** (mixed into `CogInputContext` to provide `from`, +`collect`, `reduce`). + +Seven cogs are auto-registered by `Cog::Registry` on initialization +(`cog/registry.rb`, lines 24–30). Custom cogs can be added via the `use` +directive (see [10-writing-custom-cogs.md](10-writing-custom-cogs.md)). + +--- + +## The Execution Lifecycle + +Here is the complete end-to-end path from CLI invocation to workflow completion: + +### Phase 0: CLI Parsing (`lib/roast/cli.rb`) + +1. `CLI#execute` parses arguments, splitting at `--` into Roast flags and + workflow arguments. +2. Workflow arguments are parsed into `WorkflowParams`: positional file + paths/URLs → `targets`, bare words → `args` (Symbols), `key=value` pairs → + `kwargs` (Hash). +3. `Workflow.from_file(path, params)` is called. + +### Phase 1: Framework Bootstrap (`lib/roast/workflow.rb`, lines 18–30) + +``` +Sync do # Enter async event loop + Dir.mktmpdir("roast-") do |tmpdir| # Create ephemeral workspace + EventMonitor.start! # Begin event processing + workflow = Workflow.new(path, context) + workflow.prepare! + workflow.start! + EventMonitor.stop! + end # tmpdir auto-cleaned +end +``` + +`Workflow.new` reads the workflow file as a raw string, creates a fresh +`Cog::Registry` (auto-registering all 7 built-in cogs), and initializes empty +proc collection arrays. + +### Phase 2: Prepare (`lib/roast/workflow.rb`, lines 46–58) + +1. **`extract_dsl_procs!`**: `instance_eval(@workflow_definition)` on the + Workflow instance. This evaluates the top-level Ruby file, which calls + `config {}` (appending to `@config_procs`), `execute {}` (appending to + `@execution_procs`), and `use` (registering custom cogs in the registry). + None of the collected blocks are executed yet. + +2. **`ConfigManager.prepare!`**: Binds `global` and per-cog-type methods on + `ConfigContext`, then evaluates all `@config_procs` sequentially. Config + objects are populated. + +3. **`ExecutionManager.prepare!`**: Binds `outputs`/`outputs!` and per-cog-type + methods on `ExecutionContext`, then evaluates all `@execution_procs` + sequentially. This is when cog instances are created and pushed onto the cog + stack. The `WorkflowParams` object is passed as the initial `scope_value`. + +### Phase 3: Execute (`lib/roast/execution_manager.rb`, lines 87–116) + +1. Enter `Sync` block, annotate the task, begin `TaskContext` tracking. +2. **Iterate the cog stack**: For each cog: + - Resolve merged config via `ConfigManager.config_for` (4-layer cascade: + global → type-general → regex-matched → name-specific → validate). + - `cog.run!(barrier, config.deep_dup, input_context, scope_value.deep_dup, + scope_index)` — creates an async task. + - `cog_task.wait unless cog_config.async?` — sync cogs block here. +3. **`@barrier.wait`**: Process remaining async tasks via + `wait_for_task_with_exception_handling`. +4. **`compute_final_output`**: Eagerly evaluate the `outputs`/`outputs!` block + (or fall back to the last cog's output). +5. **`ensure`**: Stop barrier, compute final output (idempotent), end + TaskContext, clear running flag. + +--- + +## The Cog Lifecycle + +What happens inside a single `cog.run!` call +(`lib/roast/cog.rb`, lines 71–101): + +``` +barrier.async(finished: false) do |task| + TaskContext.begin_cog(self) + @config = config + input = self.class.input_class.new # ← empty Input instance + return_value = input_context.instance_exec(input, scope_value, scope_index, &@cog_input_proc) # ← YOUR CODE RUNS HERE + coerce_and_validate_input!(input, return_value) # ← validation + coercion + @output = execute(input) # ← THE COG'S OWN WORK +rescue SkipCog → @skipped = true (swallowed) +rescue FailCog → @failed = true; re-raise if abort_on_failure? +rescue Next, Break → @skipped = true; re-raise +rescue StandardError → @failed = true; re-raise +ensure + TaskContext.end +end +``` + +Note the separation: `instance_exec(..., &@cog_input_proc)` runs **your** input +block code. Then `execute(input)` runs **the cog's** work. For `cmd`, `chat`, and +`agent`, these are very different phases. For the `ruby` cog, `execute(input)` is +`Output.new(input.value)` — a literal no-op — so all meaningful work happens in +the input block. + +### Two-Phase Input Validation + +1. **`validate!`** is called first (optimistic — maybe the input block set all + fields directly via `my.prompt = "..."`). +2. If `InvalidInputError`: **`coerce(return_value)`** is called — this attempts + to interpret the block's return value (e.g., a returned string becomes the + prompt). +3. **`validate!`** is called again (mandatory — if still invalid, the cog fails). + +This two-phase design (`cog.rb`, lines 149–157) means workflow authors can +either set fields explicitly (`my.prompt = "..."`) or return a value from the +block (`{ "analyze this" }`) — both work. + +--- + +## Concurrency Model + +Roast uses **fiber-based cooperative concurrency** via the `async` gem. There are +no threads and no mutexes. + +- **`Async::Barrier`** manages groups of concurrent tasks within each + `ExecutionManager`. One barrier per EM instance. +- **Sync cogs** block the iteration loop: `cog_task.wait` is called immediately + after `cog.run!`, so the next cog doesn't start until the current one finishes. +- **Async cogs** (`config { agent(:name) { async! } }`) run in the background. + The iteration loop continues to the next cog immediately. Output access + (`agent!(:name)`) blocks until the async cog completes. +- **Parallel map** uses `Async::Semaphore` for concurrency limiting + (`map.rb`). `parallel(5)` runs up to 5 iterations concurrently; + `parallel!` runs all items concurrently. + +### Deep Copy Discipline + +Because multiple fibers may share the same objects, Roast applies `deep_dup` at +every boundary where data crosses between contexts. There are **12 identified +`deep_dup` sites** serving 5 purposes: +- **Config isolation**: Prevent one cog from mutating config shared by others. +- **Scope value isolation**: Prevent one cog from mutating the scope value seen + by subsequent cogs. +- **Output isolation**: Prevent downstream cogs from mutating shared output + (`cog_input_manager.rb`, line 78). +- **Event path isolation**: Snapshot fiber-local path at event creation time. +- **Session fork isolation**: Deep copy chat session messages for fork semantics. + +This is analogous to **Erlang's message-passing** — isolation by copying, not +sharing. + +### ⚠️ Critical: The Sync/Async Next Divergence + +This is the most subtle and important behavioral difference in the framework: + +- **Sync cog calls `next!`**: The exception flows through `cog_task.wait` → + exits the cog stack loop → propagates OUT of `run!` to the parent scope + (Map/Repeat manager). +- **Async cog calls `next!`**: The exception is caught in + `wait_for_task_with_exception_handling` → barrier is stopped → exception is + **swallowed**. The scope ends normally. +- **`break!` always propagates** regardless of sync/async. + +This divergence is an unavoidable consequence of cooperative concurrency: async +exceptions are "out-of-band" — they happen in a different fiber, and the only +way to communicate them back is through the barrier wait handler. See +[07-control-flow-reference.md](07-control-flow-reference.md) for the complete +propagation matrix. + +--- + +## Analogies to Other Systems + +| Roast Pattern | Analogy | Shared Principle | +|---|---|---| +| `config {}`/`execute {}` collection → `prepare!` → `start!` | Terraform plan/apply | Separate declaration from execution | +| `deep_dup` at every boundary | Erlang message-passing | Isolation by copying, not sharing | +| Flat scope namespace (`execute(:name)` callable from any depth) | CSS selectors, Make targets | Global addressability, no lexical scoping | +| Three evaluation contexts | DSL-specific MVC | Different views of the same entity per responsibility | +| JSON/number candidate extraction in Output mixins | Framework absorbs complexity | "Pulling complexity downwards" (Ousterhout) | +| Config merge cascade (global → type → regex → name) | CSS specificity | More-specific rules override less-specific | + +--- + +## Where to Go Next + +- **[02-dsl-users-guide.md](02-dsl-users-guide.md)** — How to write Roast + workflows (practical usage) +- **[03-cog-reference.md](03-cog-reference.md)** — Detailed per-cog reference + cards +- **[06-metaprogramming-map.md](06-metaprogramming-map.md)** — The complete + dynamic method binding reference (critical for AI agents) +- **[07-control-flow-reference.md](07-control-flow-reference.md)** — The full + control flow propagation matrix +- **Tutorial chapters** in the repo's tutorial directory cover hands-on basics + (ch1–ch9) diff --git a/internal/documentation/architecture/02-dsl-users-guide.md b/internal/documentation/architecture/02-dsl-users-guide.md new file mode 100644 index 00000000..b5aed638 --- /dev/null +++ b/internal/documentation/architecture/02-dsl-users-guide.md @@ -0,0 +1,1013 @@ +# Document 2: DSL Users Guide + +_How to write Roast workflows. The practical "using the tool" guide._ + +> **Prerequisites**: Read [01-architecture-overview.md](./01-architecture-overview.md) first. +> **Reference companion**: [03-cog-reference.md](./03-cog-reference.md) for per-cog config options, input fields, output fields, and defaults. + +--- + +## Table of Contents + +1. [Your First Workflow](#1-your-first-workflow) +2. [Cog Basics](#2-cog-basics) +3. [Configuring Cogs](#3-configuring-cogs) +4. [Chaining Cogs and Accessing Outputs](#4-chaining-cogs-and-accessing-outputs) +5. [Targets and Parameters](#5-targets-and-parameters) +6. [Control Flow](#6-control-flow) +7. [Reusable Scopes (call)](#7-reusable-scopes-call) +8. [Processing Collections (map)](#8-processing-collections-map) +9. [Iterative Workflows (repeat)](#9-iterative-workflows-repeat) +10. [Async Execution](#10-async-execution) +11. [Templates and Prompts](#11-templates-and-prompts) +12. [Session Management](#12-session-management) +13. [Common Idioms Quick-Reference](#13-common-idioms-quick-reference) + +--- + +## 1. Your First Workflow + +A Roast workflow is a Ruby file that defines two things: **configuration** (what cogs look like) and **execution** (what cogs exist and in what order). Every workflow has this basic structure: + +```ruby +#: self as Roast::Workflow +config do + # configure cog types here +end + +execute do + # declare and run cogs here +end +``` + +The `#: self as Roast::Workflow` annotation on line 1 is optional but recommended. It tells Sorbet (and your editor) that the top-level `self` in this file is a `Roast::Workflow` instance, enabling autocomplete for `config`, `execute`, and `use`. + +**Running a workflow:** + +```bash +devx roast my_workflow.rb # via devx +bundle exec roast my_workflow.rb # via bundler +``` + +**What happens when you run this:** + +1. `Workflow.from_file` reads the file as a raw string +2. `instance_eval` evaluates it on the Workflow instance — this collects the `config {}` and `execute {}` blocks as procs, but does **not** evaluate their contents yet +3. `prepare!` evaluates the config procs (binding cog configuration) and the execute procs (declaring cogs and building the execution stack) +4. `start!` runs the cogs in declaration order + +> **Source**: `Workflow.from_file` at `lib/roast/workflow.rb:18–30`, `extract_dsl_procs!` at line 134–136. + +**Key principle**: The `config {}` block runs at **prepare time**. The `execute {}` block also runs at prepare time (to declare cogs), but the individual cog input blocks within it run at **execution time**. This is the declarative two-phase lifecycle described in Document 1. + +--- + +## 2. Cog Basics + +### Declaring Cogs + +Cogs are declared inside the `execute` block. The method name is the cog type, the first argument is the cog's name (as a Symbol), and the block is the **input block**: + +```ruby +execute do + cmd(:list_files) { "ls -la" } + + chat(:analyze) do |my| + my.prompt = "Analyze these files: #{cmd!(:list_files).text}" + end + + agent(:review) do |my| + my.prompts = ["Review the analysis: #{chat!(:analyze).response}"] + end + + ruby(:summarize) do |my| + my.value = { analysis: chat!(:analyze).text, review: agent!(:review).text } + end +end +``` + +The four standard cog types are `cmd`, `chat`, `agent`, and `ruby`. The three system cog types (`call`, `map`, `repeat`) are covered in Sections 7–9. + +### The Input Block + +> **⚠️ Key Concept: The Input Block Is More Than a Setter** +> +> The input block is **arbitrary Ruby code** that runs before the cog does its +> own work. Its primary purpose is to populate an empty Input instance so the +> cog knows what to do, but you can (and often should) do other preparation +> work here: set up files, load data, compute intermediate values, transform +> outputs from prior cogs, etc. Think of it as "everything that needs to happen +> before this cog fires." +> +> For `cmd`, `chat`, and `agent` cogs, the cog's own execution (running a shell +> command, calling the LLM, invoking the agent) is a **separate step** that +> happens after your input block returns and the framework validates the input. +> +> The `ruby` cog is the **deliberate exception**: its `execute` method is a +> no-op that just passes through the `value` attribute. All real work happens +> inside the input block itself. See [Section 5 in the Cog Reference](./03-cog-reference.md#5-ruby-cog) +> for details on why this exists and how to use it. + +The input block receives up to three arguments: + +```ruby +chat(:name) do |my, scope_value, scope_index| + # my → the cog's Input object (e.g., Chat::Input) — starts EMPTY; your job is to fill it + # scope_value → the current scope's value (WorkflowParams at top-level, or item in a map/repeat) + # scope_index → the current scope's index (0 at top-level, iteration index in map/repeat) +end +``` + +The input block runs at **execution time**, inside `Cog.run!` (line 79 of `lib/roast/cog.rb`). It is evaluated via `instance_exec` on the `CogInputContext`, meaning you have access to all output accessors, workflow context methods, and control flow primitives within it. + +> **Source**: `input_context.instance_exec(input_instance, executor_scope_value, executor_scope_index, &@cog_input_proc)` at `lib/roast/cog.rb:79–81`. + +### Return Value Coercion + +You can set input fields in two ways: + +**Explicit** (via the `my` object): +```ruby +cmd(:greet) do |my| + my.command = "echo" + my.args = ["Hello", "World"] +end +``` + +**Implicit** (via block return value): +```ruby +cmd(:greet) { "echo Hello World" } +``` + +When the block returns a value, the framework attempts to **coerce** it into a valid input. The coercion rules vary by cog type: + +| Cog Type | Return Value | Coercion | +|----------|-------------|----------| +| `cmd` | `String` | Sets `command` | +| `cmd` | `Array` | First element → `command`, rest → `args` (safe from shell injection) | +| `chat` | `String` | Sets `prompt` | +| `agent` | `String` | Wraps in array → `prompts = [string]` | +| `agent` | `Array[String]` | Sets `prompts` (multi-prompt sequential invocation) | +| `ruby` | Any value | Sets `value` | +| `call` | Any value | Sets `value` | +| `map` | Enumerable | Converts to array → sets `items` | +| `repeat` | Any value | Sets `value` | + +The coercion mechanism is a two-phase validation process inside `coerce_and_validate_input!` (`lib/roast/cog.rb:149–157`): + +1. `validate!` — check if the input is already valid (maybe the block set fields explicitly) +2. If `InvalidInputError` → `coerce(return_value)` — try to interpret the return value +3. `validate!` again — if still invalid, the cog fails + +This means explicit setting always takes priority over return value coercion: if you set `my.command = "..."` and also return a string, the explicit value wins because `validate!` passes on the first attempt and coercion is never invoked. + +> **Source**: `Cog::Input` base class at `lib/roast/cog/input.rb`, per-cog coerce implementations in each cog's `Input` subclass. + +### Anonymous Cogs + +If you omit the name, the cog gets a random UUID name and is marked anonymous: + +```ruby +execute do + cmd { "echo hello" } # anonymous — exists in the store but can't be referenced by name +end +``` + +Anonymous cogs execute normally but their output cannot be accessed by other cogs (there's no stable name to reference). Use named cogs whenever you need to chain outputs. + +> **Source**: `Cog.generate_fallback_name` at `lib/roast/cog.rb:23` uses `Random.uuid.to_sym`. + +--- + +## 3. Configuring Cogs + +Configuration happens inside the `config {}` block. There are four levels of configuration specificity, applied in a merge cascade where more-specific values override less-specific ones. + +### Global Config + +Applies to **all** cogs of **all** types: + +```ruby +config do + global do + working_directory "/tmp" + abort_on_failure! + end +end +``` + +Global config seeds every cog's configuration. It uses `Cog::Config` (the base class), so only base-level options are available: `async!`/`no_async!`, `abort_on_failure!`/`no_abort_on_failure!`, `working_directory`, and hash-style `[]=` for arbitrary keys. + +> **Source**: `ConfigManager#bind_global` at `lib/roast/config_manager.rb:124–132`. Global values are extracted via `instance_variable_get(:@values)` in `config_for` (line 48) — the only place in the codebase that reaches into `@values` from outside the Config class. + +### Type-General Config + +Applies to **all** cogs of a **specific type**: + +```ruby +config do + chat do + model "gpt-4o" + temperature 0.0 + end + + agent do + async! + end +end +``` + +When you call `chat` (or any cog type method) with **no arguments**, the block configures the type-general config for that cog type. + +### Regex Config + +Applies to all cogs whose **name matches** a pattern: + +```ruby +config do + agent(/review_/) do + model "claude-sonnet-4-20250514" + async! + end + + chat(/analyze_/) do + temperature 0.0 + end +end +``` + +When you pass a `Regexp` as the first argument, the block configures a regex-scoped config. Multiple regex patterns can match the same cog — they are all merged in the order they appear. + +### Per-Cog Config + +Applies to a **single named cog**: + +```ruby +config do + chat(:summarize) do + model "gpt-4o-mini" + temperature 0.7 + end +end +``` + +When you pass a `Symbol` as the first argument, the block configures the name-scoped config for that specific cog. + +### The Merge Cascade + +When a cog runs, its final configuration is computed by merging all applicable configs in order: + +``` +1. Global config values (seed) +2. Type-general config (merge) +3. All matching regex configs (merge, in declaration order) +4. Name-specific config (merge) +5. validate! (whole-config validation) +``` + +At each step, `Hash#merge` is used — right-side values (more specific) win. The result is deep-dup'd before being passed to the cog, so no cog can mutate another's configuration. + +> **Source**: `ConfigManager#config_for` at `lib/roast/config_manager.rb:44–59`. + +### Reopenable Config Blocks + +You can have **multiple** `config {}` blocks in a workflow. They are collected as procs and evaluated sequentially during `prepare!`. This means later config blocks can override earlier ones: + +```ruby +config do + chat { model "gpt-4o-mini" } +end + +config do + chat(:important) { model "gpt-4o" } +end +``` + +Both blocks target the same underlying config objects — they accumulate. + +--- + +## 4. Chaining Cogs and Accessing Outputs + +### Naming Cogs + +Always name cogs you want to reference later: + +```ruby +execute do + cmd(:list) { "ls" } + chat(:analyze) { "Analyze: #{cmd!(:list).text}" } +end +``` + +The name is a `Symbol` passed as the first argument. Within the same execution scope, names must be unique — duplicate names raise `CogAlreadyDefinedError`. + +### The Three Accessor Variants + +Every registered cog type gets three accessor methods on the `CogInputContext`, usable inside any cog's input block: + +| Method | Behavior | Returns | +|--------|----------|---------| +| `cmd(:name)` | Tolerant — returns `nil` if the cog was skipped, failed, stopped, or hasn't run yet | `Cog::Output?` | +| `cmd!(:name)` | Strict — blocks on async cogs, raises on any error state | `Cog::Output` | +| `cmd?(:name)` | Boolean — `true` if the cog produced output | `bool` | + +**Critical**: All three methods **always** raise `CogDoesNotExistError` if the named cog doesn't exist in the current scope. The tolerant variant only suppresses state-related errors (skipped, failed, stopped, not-yet-run), never existence errors. This is intentional — a nonexistent cog name is likely a typo. + +**Blocking behavior**: `cmd!(:name)` calls `cog.wait` before checking state, which blocks the current fiber if the referenced cog is async and still running. This is how sync cogs naturally wait for async dependencies. + +> **Source**: `CogInputManager#cog_output!` at `lib/roast/cog_input_manager.rb:69–79`, `cog_output` at lines 54–61. + +### Output Deep Copy + +Every output access returns a **deep copy** (`cog.output.deep_dup`). This prevents downstream cogs from mutating shared output objects — critical for correctness in concurrent execution. + +### Convenience Methods on Outputs + +Each cog type's Output class includes mixin modules that provide convenience methods for common extraction patterns. The key methods: + +**Text extraction** (from `WithText`): +- `.text` — stripped string +- `.lines` — array of stripped lines + +**JSON extraction** (from `WithJson`): +- `.json` — parsed JSON (nil on error), keys symbolized +- `.json!` — parsed JSON (raises on error) + +**Number extraction** (from `WithNumber`): +- `.integer` / `.integer!` — extracted integer +- `.float` / `.float!` — extracted float + +**Per-cog output fields:** + +| Cog Type | Primary Output | Raw Text Source | +|----------|---------------|-----------------| +| `cmd` | `.out`, `.err`, `.status` | `out` | +| `chat` | `.response`, `.session` | `response` | +| `agent` | `.response`, `.session`, `.stats` | `response` | +| `ruby` | `.value` | N/A (no mixins) | + +> **See**: [03-cog-reference.md](./03-cog-reference.md) for exhaustive per-cog output field documentation. + +### Chaining Examples + +```ruby +execute do + cmd(:ls) { "ls -la" } + + # Access text output + chat(:analyze) { "Files:\n#{cmd!(:ls).text}" } + + # Parse JSON from output + cmd(:json) { "echo '{\"count\": 42}'" } + ruby(:result) { cmd!(:json).json![:count] } + + # Chain LLM outputs + chat(:draft) { "Write a summary" } + chat(:refine) { "Improve this: #{chat!(:draft).response}" } + + # Extract numbers from LLM responses + chat(:estimate) { "How many files?" } + ruby(:count) { chat!(:estimate).integer } +end +``` + +--- + +## 5. Targets and Parameters + +Roast workflows receive parameters from the command line. The CLI uses a `--` separator to distinguish roast arguments from workflow arguments: + +```bash +roast execute my_workflow.rb target1 target2 -- retry force name=Samantha +``` + +- **Before** `--`: targets (positional arguments for the workflow) and roast flags +- **After** `--`: workflow arguments parsed into `args` (simple flags) and `kwargs` (key=value pairs) + +### Accessing Parameters + +These methods are available in any cog input block, defined on the `CogInputContext` by `CogInputManager#bind_workflow_context` (`lib/roast/cog_input_manager.rb:82–104`): + +| Method | Returns | Behavior | +|--------|---------|----------| +| `target!` | `String` | Raises `ArgumentError` unless exactly 1 target | +| `targets` | `Array[String]` | Defensive copy (`.dup`) | +| `arg?(value)` | `bool` | Checks if flag is in args | +| `args` | `Array[Symbol]` | Defensive copy | +| `kwarg(key)` | `String?` | Returns `nil` if missing | +| `kwarg!(key)` | `String` | Raises `ArgumentError` if missing | +| `kwarg?(key)` | `bool` | Checks if key exists | +| `kwargs` | `Hash[Symbol, String]` | Defensive copy | +| `tmpdir` | `Pathname` | Auto-created temp directory, cleaned on exit | + +**Important**: All kwarg values are **strings**. There is no automatic type coercion from the CLI. If you need integers, parse them yourself: `kwarg!(:count).to_i`. + +### The Scope Value + +The top-level scope value is the `WorkflowParams` object itself. This means the second block parameter at the top level is the params object: + +```ruby +execute do + ruby(:info) do |_, params| + params.targets # same as calling targets + end +end +``` + +In called scopes (`call`, `map`, `repeat`), the second parameter is whatever value was passed into that scope. + +> **Source**: `WorkflowParams` is passed as `scope_value:` to the top-level `ExecutionManager` at `lib/roast/workflow.rb:54`. + +--- + +## 6. Control Flow + +Roast provides four control flow primitives, available in any cog input block (hardcoded on `CogInputContext` at `lib/roast/cog_input_context.rb:15–32`): + +### skip! + +Silently skip the current cog. The cog is marked as `skipped` and produces no output: + +```ruby +cmd(:optional) do + skip! unless arg?(:verbose) + "ls -la" +end +``` + +`skip!` raises `ControlFlow::SkipCog`, which is always caught inside `Cog.run!` and never propagates. The cog simply doesn't execute. + +### fail! + +Mark the current cog as failed. Optionally pass a message: + +```ruby +chat(:validate) do |my| + fail! "Input too large" if targets.length > 100 + my.prompt = "Validate: #{targets.join(', ')}" +end +``` + +`fail!` raises `ControlFlow::FailCog`. By default, failed cogs **abort the workflow** — `abort_on_failure?` defaults to `true`. To make a cog's failure non-fatal: + +```ruby +config do + chat(:validate) { no_abort_on_failure! } + # or equivalently: + chat(:validate) { continue_on_failure! } +end +``` + +When `abort_on_failure?` is `false`, the cog is marked failed but execution continues. Accessing a failed cog's output via `chat(:validate)` (tolerant) returns `nil`; via `chat!(:validate)` (strict) raises `CogFailedError`. + +> **Source**: `Cog.run!` catches `FailCog` at `lib/roast/cog.rb:87–92`, re-raises only if `config.abort_on_failure?`. + +### next! + +Skip to the next iteration in a `map` or `repeat` loop. In a `call` scope, terminates the scope early: + +```ruby +execute(:process_item) do + ruby(:check) do |_, item| + next! if item.nil? # skip nil items + item + end + chat(:analyze) { "Analyze: #{ruby!(:check).value}" } +end +``` + +`next!` raises `ControlFlow::Next`. The cog is marked `skipped` and the exception always propagates. In a serial `map`, it advances to the next item. In a `call`, it's caught and the scope returns early. + +**⚠️ Sync/Async Divergence**: `next!` behaves differently depending on whether the cog is sync or async. In a sync cog, the exception propagates out of `ExecutionManager.run!` to the parent scope (e.g., advancing a Map iteration). In an async cog, the exception is swallowed by the barrier handler — remaining cogs in the scope stop, but the parent scope is not signaled. See [07-control-flow-reference.md](./07-control-flow-reference.md) for the full propagation matrix. + +### break! + +Exit the current loop or scope entirely: + +```ruby +execute(:iterate) do + ruby(:check) do |_, value, index| + break! if index >= 10 + value + end +end +``` + +`break!` raises `ControlFlow::Break`. Like `next!`, the cog is marked `skipped` and the exception propagates. `break!` always propagates regardless of sync/async status. In a `repeat`, it exits the loop. In a `map`, it stops iteration. At the top-level, `Workflow.start!` catches it and terminates the workflow gracefully. + +### fail_on_error! (cmd-specific) + +For `cmd` cogs, a non-zero exit status is treated as a failure by default. To allow non-zero exits: + +```ruby +config do + cmd(:grep) { no_fail_on_error! } +end + +execute do + cmd(:grep) { "grep -c pattern file.txt" } + # Even if grep exits with 1 (no matches), the cog succeeds. + # Access exit code: cmd!(:grep).status.exitstatus +end +``` + +> **Source**: `Cogs::Cmd::Config` defines `fail_on_error!` / `no_fail_on_error!`. Default is `true`. + +--- + +## 7. Reusable Scopes (call) + +Named execution scopes let you group cogs into reusable units. + +### Defining a Scope + +Scopes are defined at the top level with `execute(:name)`: + +```ruby +execute(:greet) do + cmd(:echo) do |_, name| + ["echo", "Hello, #{name}!"] + end +end +``` + +Scope definitions can appear in any order — they are just named proc collections. They don't execute until called. + +### Calling a Scope + +Use the `call` system cog to invoke a scope: + +```ruby +execute do + call(:greeting, run: :greet) { "World" } +end +``` + +The `run:` keyword specifies which scope to invoke. The input block's return value becomes the scope's `scope_value`, accessible as the second parameter in that scope's cog blocks. + +The first argument (`:greeting`) is the name of this call cog — used to reference its output later. If you don't need the output, you can omit it (anonymous call). + +### Extracting Results with from() + +The output of a `call` cog wraps an `ExecutionManager`. To extract the actual result, use `from()`: + +```ruby +execute do + call(:result, run: :my_scope) { "input" } + + # Get the scope's final output (from outputs block or last cog) + ruby(:use_result) { from(call!(:result)) } + + # Access a specific inner cog's output + ruby(:inner_data) do + from(call!(:result)) { cmd!(:some_inner_cog).text } + end +end +``` + +**Without a block**: `from(call!(:name))` returns the scope's `final_output` — either the value from an `outputs`/`outputs!` block, or the output of the last cog in the scope. + +**With a block**: `from(call!(:name)) { ... }` evaluates the block in the **called scope's** `CogInputContext`, giving you access to that scope's cogs via `instance_exec`. The block receives `(final_output, scope_value, scope_index)` as arguments. + +> **Source**: `Call::InputContext#from` at `lib/roast/system_cogs/call.rb:147–157`. + +### Defining Scope Outputs + +By default, a scope's final output is the output of its last cog. You can override this with `outputs` or `outputs!`: + +```ruby +execute(:compute) do + cmd(:step_a) { "echo A" } + cmd(:step_b) { "echo B" } + + # Tolerant: accessing a skipped/failed cog returns nil + outputs { cmd!(:step_a).text } + + # Strict: accessing a skipped/failed cog raises an exception + outputs! { cmd!(:step_a).text } +end +``` + +You cannot define both `outputs` and `outputs!` in the same scope — `OutputsAlreadyDefinedError` is raised. The outputs block receives `(scope_value, scope_index)` as arguments and runs in the `CogInputContext`. + +**Key behavior**: The outputs block **always runs**, even after `break!` or `next!`. It runs in the `ensure` block of `ExecutionManager.run!`. This ensures the scope always produces a final output for chaining. See Section 9 for why this matters in repeat loops. + +> **Source**: `compute_final_output` at `lib/roast/execution_manager.rb:254–283`. + +--- + +## 8. Processing Collections (map) + +The `map` system cog iterates a scope over a collection of items. + +### Basic Map + +```ruby +execute(:process_word) do + chat(:define) do |_, word| + "Define the word: #{word}" + end +end + +execute do + map(:definitions, run: :process_word) { ["hello", "world", "roast"] } +end +``` + +Each item in the collection becomes the `scope_value` for one invocation of the named scope. The iteration index is passed as `scope_index`. + +### Setting Items Explicitly + +```ruby +execute do + map(:results, run: :process) do |my| + my.items = ["a", "b", "c"] + my.initial_index = 10 # first iteration gets index 10, not 0 + end +end +``` + +### Parallel Execution + +By default, map executes serially (one item at a time). Configure parallel execution: + +```ruby +config do + map(:results) { parallel 3 } # max 3 concurrent iterations + # or + map(:results) { parallel! } # unlimited concurrency + # or + map(:results) { no_parallel! } # explicitly serial (default) +end +``` + +| Config Call | `@values[:parallel]` | Behavior | +|------------|---------------------|----------| +| _(none)_ | _(absent, fetched as 1)_ | Serial | +| `parallel(3)` | `3` | Max 3 concurrent | +| `parallel(0)` | `nil` | Unlimited concurrency | +| `parallel!` | `nil` | Unlimited concurrency | +| `no_parallel!` | `1` | Explicitly serial | + +**Results are always ordered** regardless of completion order. Parallel map uses a `Hash` for thread-safe writes during concurrent execution, then reconstructs the ordered array afterward. + +> **Source**: `execute_map_in_parallel` at `lib/roast/system_cogs/map.rb:306–338`. Serial at lines 288–303. + +### Collecting Results with collect() + +`collect()` extracts the final output from each iteration into an array: + +```ruby +execute do + map(:results, run: :process) { ["a", "b", "c"] } + + # Without block: array of each iteration's final_output + ruby(:all) { collect(map!(:results)) } + + # With block: evaluate per-iteration in that iteration's CogInputContext + ruby(:texts) do + collect(map!(:results)) { chat!(:define).text } + end +end +``` + +**Without a block**: Returns `[final_output_0, final_output_1, ...]`. Iterations that didn't run (due to `break!`) appear as `nil`. + +**With a block**: The block is evaluated via `instance_exec` on each iteration's `CogInputContext`, receiving `(final_output, scope_value, scope_index)`. You can access any cog from within that iteration. + +> **Source**: `Map::InputContext#collect` at `lib/roast/system_cogs/map.rb:375–389`. + +### Reducing Results with reduce() + +`reduce()` aggregates iteration outputs into a single value: + +```ruby +execute do + map(:scores, run: :compute) { items } + + ruby(:total) do + reduce(map!(:scores), 0) do |sum, output, item, index| + sum + output.to_i + end + end +end +``` + +The block receives `(accumulator, final_output, scope_value, scope_index)`. Return the new accumulator value. + +**Nil-preservation**: If the block returns `nil`, the accumulator is **not** updated. This prevents accidental overwriting with nil. Iterations that didn't run (due to `break!`) are skipped entirely (via `.compact`). + +> **Source**: `Map::InputContext#reduce` at `lib/roast/system_cogs/map.rb:426–448`. + +### Accessing Individual Iterations + +```ruby +# Specific iteration (0-indexed, supports negative indices) +from(map!(:results).iteration(2)) + +# First and last +from(map!(:results).first) +from(map!(:results).last) + +# Check if iteration ran +map!(:results).iteration?(3) # → true/false +``` + +Each iteration accessor returns a `Call::Output`, so you use `from()` to extract data — exactly like accessing a single `call` cog's output. + +> **Source**: `Map::Output` at `lib/roast/system_cogs/map.rb:171–252`. + +### Control Flow in Map + +| Primitive | Serial Behavior | Parallel Behavior | +|-----------|----------------|-------------------| +| `next!` | Skip current item, continue to next | Skip current task, others continue | +| `break!` | Stop iteration, exit loop | Stop all tasks via `barrier.stop` | + +Unexecuted iterations appear as `nil` entries in the output. + +--- + +## 9. Iterative Workflows (repeat) + +The `repeat` system cog runs a scope in a loop, chaining each iteration's output into the next iteration's input. + +### Basic Repeat + +```ruby +execute(:refine) do + chat(:improve) do |_, draft| + "Improve this text: #{draft}" + end + outputs { chat!(:improve).response } +end + +execute do + repeat(:loop, run: :refine) { "Initial rough draft" } +end +``` + +**Iteration chaining**: The first iteration receives `"Initial rough draft"` as its `scope_value`. The outputs block produces a refined version. That refined version becomes the `scope_value` for the second iteration, and so on. + +> **Source**: `Repeat::Manager` at `lib/roast/system_cogs/repeat.rb:208–236`. The chaining is at line 228: `scope_value = em.final_output`. + +### Termination + +Repeat loops run indefinitely until explicitly stopped: + +**break!** — Exit the loop: +```ruby +execute(:iterate) do + ruby(:check) do |_, value, index| + break! if index >= 5 + value + 1 + end +end +``` + +**max_iterations** — Safety valve: +```ruby +execute do + repeat(:loop, run: :iterate) do |my| + my.value = 0 + my.max_iterations = 10 + end +end +``` + +If `max_iterations` is reached, the loop exits normally (not via exception). + +### The outputs Block Always Runs + +This is a critical design point: the `outputs` block runs even on iterations where `break!` or `next!` was called. It executes in the `ensure` block of `ExecutionManager.run!`. This guarantees that every iteration produces a `final_output`, which is necessary for the chaining mechanism. + +### Accessing Repeat Results + +```ruby +# Final value (last iteration's final_output) +repeat!(:loop).value + +# Specific iteration +from(repeat!(:loop).iteration(0)) +from(repeat!(:loop).first) +from(repeat!(:loop).last) + +# All iterations as a Map::Output (the Repeat→Map bridge) +collect(repeat!(:loop).results) { chat!(:improve).text } +reduce(repeat!(:loop).results, "") { |acc, output| acc + output.to_s } +``` + +The `.results` method returns a `Map::Output` wrapping all iterations — this is the **Repeat→Map bridge** that lets you reuse `collect` and `reduce` on repeat loop results. + +> **Source**: `Repeat::Output#results` at `lib/roast/system_cogs/repeat.rb:198–199` creates `Map::Output.new(@execution_managers)`. + +### State Machine Pattern + +For complex iterative workflows, use a Hash as the scope value to carry state between iterations: + +```ruby +execute(:guess) do + chat(:make_guess) do |_, state| + "The target is between 1 and 100. Previous guesses: #{state[:history].join(', ')}. Guess a number." + end + + ruby(:update) do |_, state| + guess = chat!(:make_guess).integer! + history = state[:history] + [guess] + break! if guess == state[:target] + { target: state[:target], history: history, session: chat!(:make_guess).session } + end + + outputs do |state| + ruby!(:update).value + end +end + +execute do + repeat(:game, run: :guess) do + { target: 42, history: [], session: nil } + end +end +``` + +--- + +## 10. Async Execution + +Any cog can be configured to run asynchronously in the background. + +### Configuring Async + +```ruby +config do + agent(:slow_review) { async! } + agent(/background_/) { async! } # regex-based + agent { async! } # all agents +end +``` + +### Behavior + +- **Async cog starts**: The cog begins executing in a background fiber. The next cog in the `execute` block starts immediately without waiting. +- **Sync cog blocks**: A sync cog (the default) blocks until it completes before the next cog starts. Sync cogs act as natural **execution barriers**. +- **Output access blocks**: Accessing an async cog's output (`agent!(:slow_review)`) blocks the current fiber until that cog completes. +- **Scope completion**: A scope does not finish until all its async cogs complete. The `@barrier.wait` at the end of `ExecutionManager.run!` ensures this. + +### Async vs Parallel Map + +These are different concurrency patterns: + +- **Async cogs**: Different tasks running concurrently within the **same scope** (e.g., review and lint happening at the same time) +- **Parallel map**: The **same task** running concurrently on **different items** (e.g., processing 10 files simultaneously) + +They can be combined: you can have async cogs inside a parallel map's scope. + +> **Source**: `cog_task.wait unless cog_config.async?` at `lib/roast/execution_manager.rb:104`. Barrier wait at line 108. + +### ⚠️ Control Flow Warning + +`next!` and `break!` behave differently in async cogs. In particular, `next!` from an async cog is **swallowed** by the barrier handler and does not propagate to the parent scope. Only `break!` propagates reliably from async contexts. See [07-control-flow-reference.md](./07-control-flow-reference.md) for details. + +--- + +## 11. Templates and Prompts + +The `template()` method renders ERB templates for building prompts or other text: + +```ruby +chat(:analyze) do + template("analysis_prompt", files: cmd!(:ls).lines, context: "production") +end +``` + +### Search Priority + +Given `template("greeting", name: "World")`, the framework searches for the template file in this order: + +1. Absolute path as-is (if `path.absolute?`) +2. `workflow_dir / "greeting"` +3. `workflow_dir / "greeting.erb"` +4. `workflow_dir / "greeting.md.erb"` +5. `workflow_dir / "prompts" / "greeting"` +6. `workflow_dir / "prompts" / "greeting.erb"` +7. `workflow_dir / "prompts" / "greeting.md.erb"` +8. `pwd / "greeting"` +9. `pwd / "greeting.erb"` +10. `pwd / "greeting.md.erb"` +11. `pwd / "prompts" / "greeting"` +12. `pwd / "prompts" / "greeting.erb"` +13. `pwd / "prompts" / "greeting.md.erb"` + +The first existing file wins. Templates are rendered with `ERB.new(content).result_with_hash(args)`. + +**Known issue**: `Pathname` does not expand `~` for home directory paths (tracked in issue #663). + +> **Source**: `CogInputManager#template` at `lib/roast/cog_input_manager.rb:182–223`. + +### Inline Prompts + +For simple prompts, use heredocs directly in the input block: + +```ruby +chat(:analyze) do + <<~PROMPT + Analyze the following files for potential issues: + #{cmd!(:ls).text} + + Focus on security and performance concerns. + PROMPT +end +``` + +The string return is coerced into the chat's `prompt` field automatically. + +--- + +## 12. Session Management + +Sessions allow LLM conversations to be continued or forked across cogs. + +### Chat Sessions + +```ruby +execute do + chat(:initial) { "What is Ruby?" } + + chat(:followup) do |my| + my.session = chat!(:initial).session + my.prompt = "Can you give an example?" + end +end +``` + +Setting `my.session` resumes the conversation from where the previous chat left off. The new prompt is added to the existing message history. + +**Fork semantics**: Sessions are deep-copied when accessed (`cog.output.deep_dup`). This means you can fork from the same point: + +```ruby +# Both continue from :initial independently +chat(:branch_a) do |my| + my.session = chat!(:initial).session + "Tell me about Rails" +end + +chat(:branch_b) do |my| + my.session = chat!(:initial).session + "Tell me about Sinatra" +end +``` + +**Cross-model**: You can resume a session with a different model than the original. The message history transfers; only the model changes. + +### Agent Sessions + +```ruby +execute do + agent(:first) { "Set up the project" } + + agent(:second) do |my| + my.session = agent!(:first).session + my.prompts = ["Now add tests"] + end +end +``` + +Agent sessions work via the CLI provider's `--fork-session` flag. The session string is a file path pointing to the serialized conversation state. + +--- + +## 13. Common Idioms Quick-Reference + +| Idiom | Example | Notes | +|-------|---------|-------| +| String return = implicit coercion | `cmd { "ls" }`, `chat { "prompt" }` | Block return value coerced to input | +| Array return for cmd = safe shell | `cmd { ["echo", user_input] }` | No shell interpolation | +| Multi-prompt agent | `agent { ["prompt 1", "prompt 2"] }` | Sequential invocations in same session | +| outputs as tolerant finalizer | `outputs { ruby!(:x).value }` | Returns nil for skipped/failed cogs | +| outputs! as strict finalizer | `outputs! { ruby!(:x).value }` | Raises for skipped/failed cogs | +| from() for scope bridging | `from(call!(:name))` | Extracts scope's final output | +| from() with block | `from(call!(:x)) { cmd!(:inner).text }` | Access inner scope's cogs | +| collect() for map results | `collect(map!(:x)) { chat!(:y).text }` | Per-iteration extraction | +| reduce() for aggregation | `reduce(map!(:x), 0) { \|sum, o\| sum + o }` | nil return preserves accumulator | +| Repeat→Map bridge | `collect(repeat!(:x).results)` | Reuses Map algebra on repeat results | +| State machine repeat | `repeat(:x, run: :y) { { state: ... } }` | Hash as scope value | +| Regex config for groups | `agent(/review_/) { async! }` | Pattern-based configuration | +| Sync cog as barrier | Place a sync cog after async ones | Blocks until all prior cogs finish | +| Template for prompts | `template("name", vars)` | 13-candidate search path | +| Session fork | `my.session = chat!(:a).session` | Deep copy = independent fork | +| tmpdir for ephemeral work | `tmpdir` → `Pathname` | Auto-created, auto-cleaned | +| Custom cog loading | `use "name"` or `use "name", from: "gem"` | See Section 14 note | + +**Loading custom cogs**: Custom cogs are loaded with `use` at the top level of the workflow file (outside `config` and `execute`). Local: `use "name"` loads from `cogs/name` relative to the workflow file. From a gem: `use "name", from: "gem_name"`. See [10-writing-custom-cogs.md](./10-writing-custom-cogs.md) for details. + +> **Source**: `Workflow#use` at `lib/roast/workflow.rb:105–126`. + +--- + +## See Also + +- [01-architecture-overview.md](./01-architecture-overview.md) — The architectural context for everything in this guide +- [03-cog-reference.md](./03-cog-reference.md) — Detailed per-cog reference cards (config options, input fields, output fields, defaults) +- [07-control-flow-reference.md](./07-control-flow-reference.md) — The complete sync/async propagation matrix and known edge cases +- [10-writing-custom-cogs.md](./10-writing-custom-cogs.md) — How to create your own cog types diff --git a/internal/documentation/architecture/03-cog-reference.md b/internal/documentation/architecture/03-cog-reference.md new file mode 100644 index 00000000..4c8a9f0a --- /dev/null +++ b/internal/documentation/architecture/03-cog-reference.md @@ -0,0 +1,1035 @@ +# Cog Reference + +> **Per-cog reference cards.** This document provides exhaustive detail on every +> cog type: configuration options, input fields, output fields, execute behavior, +> defaults, and source locations. Consult this when you need to know exactly what +> a cog can do and how it's configured. +> +> Prerequisite: [01-architecture-overview.md](01-architecture-overview.md) (for +> the base cog lifecycle: Config → Input → Execute → Output). + +--- + +## Table of Contents + +1. [Common Configuration (All Cogs)](#1-common-configuration-all-cogs) +2. [cmd Cog](#2-cmd-cog) +3. [chat Cog](#3-chat-cog) +4. [agent Cog](#4-agent-cog) +5. [ruby Cog](#5-ruby-cog) +6. [call System Cog](#6-call-system-cog) +7. [map System Cog](#7-map-system-cog) +8. [repeat System Cog](#8-repeat-system-cog) +9. [Config Merge Cascade](#9-config-merge-cascade) +10. [Output Mixin Modules](#10-output-mixin-modules) +11. [Boolean Default Patterns](#11-boolean-default-patterns) + +--- + +## 1. Common Configuration (All Cogs) + +**Source**: `lib/roast/cog/config.rb` + +Every cog inherits these options from `Cog::Config`. They are set inside +`config {}` blocks and are subject to the merge cascade (§9). + +### Async Execution + +| Method | Effect | Stored Value | +|---|---|---| +| `async!` | Run cog in background | `@values[:async] = true` | +| `no_async!` | Run cog synchronously (default) | `@values[:async] = false` | +| `sync!` | Alias for `no_async!` | `@values[:async] = false` | + +**Query**: `async?` → `!!@values[:async]` → defaults to **`false`** (sync). + +When async, the next cog in the stack starts immediately. Accessing the async +cog's output via `cog!(:name)` blocks the caller until the cog completes. + +### Abort On Failure + +| Method | Effect | Stored Value | +|---|---|---| +| `abort_on_failure!` | Abort workflow on cog failure (default) | `@values[:abort_on_failure] = true` | +| `no_abort_on_failure!` | Continue workflow on cog failure | `@values[:abort_on_failure] = false` | +| `continue_on_failure!` | Alias for `no_abort_on_failure!` | `@values[:abort_on_failure] = false` | + +**Query**: `abort_on_failure?` → `@values.fetch(:abort_on_failure, true)` → +defaults to **`true`** (abort). (`config.rb`, line 233) + +**Scope**: This setting only affects `ControlFlow::FailCog` exceptions (raised +via `fail!` or by `cmd` on non-zero exit with `fail_on_error?`). Unexpected +`StandardError` exceptions (e.g., `RuntimeError`, `NoMethodError`) **always +propagate** and abort the workflow regardless of this setting. + +### Working Directory + +| Method | Effect | +|---|---| +| `working_directory("/path")` | Run external commands in the specified directory | +| `use_current_working_directory!` | Use the directory from which Roast was invoked (default) | + +**Query**: `valid_working_directory` → returns `Pathname?` (`nil` = use cwd). +Raises `InvalidConfigError` if the path does not exist or is not a directory. + +**Important**: This only affects external commands invoked by a cog (via +`CommandRunner`). It does not change Roast's own process working directory. + +### The `field` Macro (For Custom Cogs) + +`Cog::Config` provides a class method `field(key, default, &validator)` that +generates a dual-purpose getter/setter and a `use_default_{key}!` reset method. +(`config.rb`, lines 110–128) + +⚠️ **Falsy value pitfall**: The getter uses `@values[key] || default.deep_dup`. +This means `false` and `nil` values **fall through to the default**. All +built-in boolean options avoid this macro and use direct `@values` manipulation +instead. See §11 for the full pattern catalog. + +### Hash-Style Access + +`config[key]` and `config[key] = value` provide direct `@values` hash access +for simple custom cog use cases. (`config.rb`, lines 61–77) + +--- + +## 2. cmd Cog + +**Source**: `lib/roast/cogs/cmd.rb` +**Purpose**: Execute a shell command via `CommandRunner`. + +### Config (`Cmd::Config`) + +| Method | Default | Pattern | Effect | +|---|---|---|---| +| `fail_on_error!` | **enabled** | `@values[:fail_on_error] != false` | Mark cog as failed on non-zero exit | +| `no_fail_on_error!` | | | Allow non-zero exit without failure | +| `show_stdout!` | disabled | `!!@values[:show_stdout]` | Print command stdout to console | +| `show_stderr!` | disabled | `!!@values[:show_stderr]` | Print command stderr to console | +| `display!` | | | Enable both stdout and stderr | +| `no_display!` / `quiet!` | | | Disable both stdout and stderr | + +**Queries**: `fail_on_error?`, `show_stdout?`, `show_stderr?`, `display?` + +Note the behavioral interaction: `fail_on_error?` marks the cog as failed (via +`ControlFlow::FailCog`), and then `abort_on_failure?` determines whether that +failure aborts the workflow. By default, both are true: a non-zero exit will +abort the workflow. + +### Input (`Cmd::Input`) + +| Field | Type | Required | Default | +|---|---|---|---| +| `command` | `String?` | Yes | `nil` | +| `args` | `Array[String]` | No | `[]` | +| `stdin` | `String?` | No | `nil` | + +**Validation**: `command.present?` is required. (`cmd.rb`, line 208) + +**Coercion** (`cmd.rb`, lines 222–230): +- `String` → sets `command` +- `Array` → first element becomes `command`, remaining become `args` (elements + coerced to strings via `.to_s`; uses `shift` which mutates the array) + +### Output (`Cmd::Output`) + +| Field | Type | Description | +|---|---|---| +| `out` | `String` | The command's stdout | +| `err` | `String` | The command's stderr | +| `status` | `Process::Status` | The exit status object | + +**Includes**: `WithJson`, `WithNumber`, `WithText` (see §10) + +**`raw_text`** → delegates to `out` (stdout is the "primary" output for text/JSON/number parsing) + +### Execute Behavior + +1. Creates streaming handlers based on `show_stdout?` / `show_stderr?` + (`cmd.rb`, lines 277–278) +2. Delegates to `CommandRunner.execute` with the command + args array, + `working_directory`, `stdin_content`, and handlers (`cmd.rb`, lines 280–287) +3. If the process exits non-zero AND `fail_on_error?` is true: raises + `ControlFlow::FailCog` (`cmd.rb`, lines 288–289) +4. Returns `Output.new(stdout, stderr, status)` + +**Source**: `lib/roast/command_runner.rb` handles the subprocess lifecycle: +`Bundler.with_unbundled_env` → `Open3.popen3` → concurrent stdout/stderr +reading via `Async` tasks → process cleanup with SIGTERM → SIGKILL fallback. + +--- + +## 3. chat Cog + +**Source**: `lib/roast/cogs/chat.rb`, `lib/roast/cogs/chat/config.rb`, +`lib/roast/cogs/chat/input.rb`, `lib/roast/cogs/chat/output.rb`, +`lib/roast/cogs/chat/session.rb` +**Purpose**: Single LLM conversation turn via the `RubyLLM` gem. No local +filesystem access, no local tools — only the model and any cloud-based tools or +MCP servers provided by the LLM provider. + +### Config (`Chat::Config`) + +#### Provider & Credentials + +| Method | Default | Effect | +|---|---|---| +| `provider(:openai)` | `:openai` | Set the LLM provider | +| `use_default_provider!` | | Reset to default provider | +| `api_key("sk-...")` | from ENV | Set an explicit API key | +| `use_api_key_from_environment!` | | Clear explicit key, fall back to `OPENAI_API_KEY` | +| `base_url("https://...")` | from ENV or `https://api.openai.com/v1` | Set the API base URL | +| `use_default_base_url!` | | Reset to env var or provider default | + +**Currently only one provider is supported**: `:openai`. The `PROVIDERS` hash +(`chat/config.rb`, lines 8–15) maps `:openai` to its env var names and defaults. + +**Validated getters**: `valid_provider!`, `valid_api_key!` (raises +`InvalidConfigError` if missing), `valid_base_url` (falls back through env → +provider default). + +#### Model & Temperature + +| Method | Default | Effect | +|---|---|---| +| `model("gpt-4o")` | `"gpt-4o-mini"` | Set the model name | +| `use_default_model!` | | Reset to provider default | +| `temperature(0.7)` | provider default (no explicit value) | Set temperature (0.0–1.0) | +| `use_default_temperature!` | | Remove explicit temperature | +| `verify_model_exists!` | disabled | Check model availability before invocation | +| `no_verify_model_exists!` / `assume_model_exists!` | | Skip model verification (default) | + +**Validated getters**: `valid_model` (returns model or provider default), +`valid_temperature` (returns `Float?`, `nil` means use provider default), +`verify_model_exists?` → `@values.fetch(:verify_model_exists, false)`. + +#### Display Options + +| Method | Default | Pattern | +|---|---|---| +| `show_prompt!` / `no_show_prompt!` | **disabled** | `@values.fetch(:show_prompt, false)` | +| `show_response!` / `no_show_response!` | **enabled** | `@values.fetch(:show_response, true)` | +| `show_stats!` / `no_show_stats!` | **enabled** | `@values.fetch(:show_stats, true)` | +| `display!` | | Enable all three | +| `no_display!` / `quiet!` | | Disable all three | + +**Query**: `display?` → `show_prompt? || show_response? || show_stats?` + +### Input (`Chat::Input`) + +| Field | Type | Required | Default | +|---|---|---|---| +| `prompt` | `String?` | Yes | `nil` | +| `session` | `Session?` | No | `nil` | + +**Validation**: Calls `valid_prompt!`, which raises `InvalidInputError` if +`prompt` is not `present?`. (`chat/input.rb`, lines 41–42, 68–71) + +**Coercion** (`chat/input.rb`, lines 53–56): +- `String` → sets `prompt` +- Other types → ignored (no coercion) + +Note: Chat's `coerce` does **not** call `super`, so the `coerce_ran?` flag is +never set. This is safe because Chat's `validate!` doesn't check `coerce_ran?`. + +**Helper methods**: `valid_prompt!` (raises if blank), `valid_session` (returns +session or `nil` — no raise on missing session). + +### Output (`Chat::Output`) + +| Field | Type | Description | +|---|---|---| +| `response` | `String` | The LLM's response text | +| `session` | `Session` | Conversation context for resumption | + +**Includes**: `WithJson`, `WithNumber`, `WithText` + +**`raw_text`** → delegates to `response` + +### Session (`Chat::Session`) + +**Source**: `lib/roast/cogs/chat/session.rb` + +The session captures the conversation message history for fork-style resumption. + +| Method | Behavior | +|---|---| +| `Session.from_chat(chat)` | Creates a session from a RubyLLM::Chat, **deep_dup**'ing all messages | +| `session.first(n=2)` | Returns a new session with only the first `n` messages (deep_dup'd) | +| `session.last(n=2)` | Returns a new session with only the last `n` messages (deep_dup'd) | +| `session.apply!(chat)` | Replaces the chat's `@messages` via `instance_variable_set` (deep_dup'd); also restores temperature if captured | + +**Fork semantics**: Every `apply!` call deep-copies the messages, so multiple +downstream cogs can fork from the same session state independently. + +⚠️ **`instance_variable_set` boundary**: `apply!` reaches into RubyLLM's +internals to replace `@messages` (`session.rb`, line 60). This is a fragile +coupling to RubyLLM's internal structure. + +### Execute Behavior + +1. Get or create a `RubyLLM::Context` (memoized per cog instance, configured + with `api_key` and `base_url`) (`chat.rb`, lines 71–76) +2. Create a new `RubyLLM::Chat` with the configured model, provider, and + `assume_model_exists` flag (`chat.rb`, lines 32–36) +3. Apply input session if present (fork semantics via deep_dup) (`chat.rb`, + line 37) +4. Set temperature via `chat.with_temperature` if configured (`chat.rb`, line 38) +5. Record `num_existing_messages` for display filtering (`chat.rb`, line 39) +6. `chat.ask(prompt)` — the actual LLM call (`chat.rb`, line 41) +7. Display new messages based on config (prompt → `[USER PROMPT]`, response → + `[LLM RESPONSE]`) (`chat.rb`, lines 42–53) +8. Display stats if enabled: model, temperature, input/output tokens. Temperature + is read via `chat.instance_variable_get(:@temperature)` (`chat.rb`, lines + 54–61) +9. Return `Output.new(Session.from_chat(chat), response.content)` + +⚠️ **`instance_variable_get` boundary**: Stats display reaches into RubyLLM's +`@temperature` internal (`chat.rb`, line 55). This is the second fragile +coupling to RubyLLM internals in this cog. + +--- + +## 4. agent Cog + +**Source**: `lib/roast/cogs/agent.rb`, `lib/roast/cogs/agent/config.rb`, +`lib/roast/cogs/agent/input.rb`, `lib/roast/cogs/agent/output.rb`, +`lib/roast/cogs/agent/provider.rb`, `lib/roast/cogs/agent/providers/claude.rb`, +`lib/roast/cogs/agent/providers/pi.rb` +**Purpose**: Invoke an AI coding agent on the local machine. The agent has full +filesystem access, local tools, and MCP servers. Session state is maintained +across invocations via session identifiers. + +### Error Hierarchy + +``` +Roast::Error + └── AgentCogError + ├── UnknownProviderError — invalid provider name + ├── MissingProviderError — no provider configured + └── MissingPromptError — no prompt provided +``` + +### Config (`Agent::Config`) + +#### Provider & Command + +| Method | Default | Effect | +|---|---|---| +| `provider(:claude)` | `:claude` | Set the agent provider | +| `use_default_provider!` | | Reset to default (`:claude`) | +| `command("claude")` | provider default | Override the CLI command | +| `use_default_command!` | | Reset to provider default | +| `model("claude-sonnet-4-20250514")` | provider default | Set the model name | +| `use_default_model!` | | Reset to provider default | + +**Valid providers**: `[:claude, :pi]` (`agent/config.rb`, line 8) + +**Validated getters**: `valid_provider!` (raises if invalid), `valid_command` +(returns `nil` for provider default), `valid_model` (returns `nil` for provider +default — uses `.presence`). + +#### System Prompt + +| Method | Default | Effect | +|---|---|---| +| `replace_system_prompt("...")` | none | Completely replace the agent's default system prompt | +| `no_replace_system_prompt!` | | Clear replacement (restore default) | +| `append_system_prompt("...")` | none | Append text to the agent's system prompt | +| `no_append_system_prompt!` | | Clear append text | + +`replace_system_prompt` and `append_system_prompt` **can be combined**: the +replacement is applied first, then the append is added to the end. + +**Validated getters**: `valid_replace_system_prompt`, `valid_append_system_prompt` +— both return `String?` via `.presence` (blank strings → `nil`). + +#### Permissions + +| Method | Default | Pattern | +|---|---|---| +| `apply_permissions!` / `no_skip_permissions!` | **enabled** | `@values.fetch(:apply_permissions, true)` | +| `no_apply_permissions!` / `skip_permissions!` | | Disable permissions | + +**Query**: `apply_permissions?` → defaults to `true`. + +When disabled, the agent is invoked with `--dangerously-skip-permissions` +(Claude) — this bypasses all permission checks. Pi does not support this flag. + +#### Display Options + +| Method | Default | Pattern | +|---|---|---| +| `show_prompt!` / `no_show_prompt!` | **disabled** | `@values.fetch(:show_prompt, false)` | +| `show_progress!` / `no_show_progress!` | **enabled** | `@values.fetch(:show_progress, true)` | +| `show_response!` / `no_show_response!` | **enabled** | `@values.fetch(:show_response, true)` | +| `show_stats!` / `no_show_stats!` | **enabled** | `@values.fetch(:show_stats, true)` | +| `display!` | | Enable all four | +| `no_display!` / `quiet!` | | Disable all four | + +**Query**: `display?` → any of the four enabled. + +#### Debug + +| Method | Effect | +|---|---| +| `dump_raw_agent_messages_to("filename")` | Dump raw agent messages to file (dev/debug) | + +**Validated getter**: `valid_dump_raw_agent_messages_to_path` → `Pathname?` + +### Input (`Agent::Input`) + +| Field | Type | Required | Default | +|---|---|---|---| +| `prompts` | `Array[String]` | Yes (at least one) | `[]` | +| `session` | `String?` | No | `nil` | + +**Validation**: Requires non-empty `prompts` with no blank entries. +(`agent/input.rb`, lines 46–48) + +**Coercion** (`agent/input.rb`, lines 61–67): +- `String` → `[string]` (single-element prompts array) +- `Array` → elements coerced via `.map(&:to_s)` + +**Convenience setter**: `prompt=(str)` → wraps in `[str]` (`agent/input.rb`, +line 71) + +**Multi-prompt semantics**: When multiple prompts are provided, each is sent to +the agent sequentially in the same session. The agent completes one prompt before +receiving the next. This is designed for "perform task, then summarize" patterns. + +### Output (`Agent::Output`) + +| Field | Type | Description | +|---|---|---| +| `response` | `String` | The agent's final response text | +| `session` | `String` | Session identifier for resumption | +| `stats` | `Stats` | Execution statistics | + +**Includes**: `WithJson`, `WithNumber`, `WithText` + +**`raw_text`** → delegates to `response` + +Note: `Agent::Output` defines `attr_reader` but **no constructor**. Construction +is handled by provider-specific Output subclasses (`Claude::Output`, +`Pi::Output`) that delegate to their respective `Result` classes. + +### Stats & Usage (`agent/stats.rb`, `agent/usage.rb`) + +| Class | Fields | +|---|---| +| `Stats` | `duration_ms`, `num_turns`, `usage` (aggregate), `model_usage` (Hash[String, Usage]) | +| `Usage` | `input_tokens`, `output_tokens`, `cost_usd` — all nullable | + +Both classes support the `+` operator for merging across multi-prompt +invocations. `Stats#to_s` produces human-readable output using +`ActiveSupport::NumberHelper` and `ActiveSupport::Duration`. + +### Providers + +Both providers work by **running CLI tools as subprocesses** via +`CommandRunner`. They are CLI wrappers, not SDK clients. This means: +- The agent CLI tool must be installed on the system +- Communication is via stdin (prompt) / stdout (streaming JSON) +- No in-process state sharing (clean isolation) +- Stats parsing depends on the CLI's output format + +#### Claude Provider (`agent/providers/claude.rb`) + +**Command**: `claude -p --verbose --output-format stream-json [--model MODEL] +[--system-prompt PROMPT] [--append-system-prompt PROMPT] [--fork-session +--resume SESSION] [--dangerously-skip-permissions]` + +Prompt is sent via stdin. Output is parsed line-by-line as JSON using a typed +message hierarchy (`Message`, `AssistantMessage`, `ResultMessage`, +`ToolUseMessage`, etc.). The final response text, session ID, and stats are +extracted from a `ResultMessage`. + +**Multi-prompt**: First prompt uses `fork_session: true` to create an +independent fork; subsequent prompts use `fork_session: false` to continue in +the same forked session. Stats are merged across invocations. + +#### Pi Provider (`agent/providers/pi.rb`) + +**Command**: `pi --mode json -p [--model MODEL] [--fork SESSION | --no-session]` + +Uses event-based message types (`session`, `turn_start`, `message_update`, +`message_end`, `tool_execution_start/end`, `agent_end`) instead of Claude's +typed message classes. Stats are accumulated manually across streaming events. + +**Key differences from Claude**: + +| Aspect | Claude | Pi | +|---|---|---| +| Output format | `--output-format stream-json` | `--mode json` | +| Session fork | `--fork-session --resume ID` | `--fork ID` | +| No session | (no flag) | `--no-session` | +| Permissions | `--dangerously-skip-permissions` | (not supported) | +| Stats | Single ResultMessage | Manual accumulation across events | +| Text streaming | From ResultMessage | Via `text_delta` / `text_end` events | + +### Execute Behavior + +1. Lazily initialize provider based on `config.valid_provider!` (memoized in + `@provider`) (`agent.rb`, lines 59–67) +2. Call `provider.invoke(input)` → returns provider-specific `Output` subclass + (`agent.rb`, line 50) +3. Display stats and session ID if `show_stats?` is enabled (`agent.rb`, + lines 51–52) +4. Return the Output + +--- + +## 5. ruby Cog + +**Source**: `lib/roast/cogs/ruby.rb` +**Purpose**: A **no-op cog** that gives you a "naked" input block. All real work +happens in the input block; `execute` just passes the value through unchanged. + +### Why It Exists + +Every cog in Roast has an input block — a full Ruby execution context where you +can write arbitrary code. For `cmd`, `chat`, and `agent`, the input block is +preparation for the cog's own action (running a command, calling an LLM, etc.). +But sometimes you want to write a chunk of Ruby logic — compute something, +transform data, set up files — without needing a "real" cog underneath. + +Without the `ruby` cog, you'd need to create a `chat` cog with a dummy prompt +like "repeat this string verbatim" just to get an input block to run your code +in. That would be cumbersome and wasteful. The `ruby` cog was created to provide +a **clean input block with no underlying action**. It's named `ruby` (not +`no-op`) because from the workflow author's perspective, it _looks like_ writing +Ruby code that Roast executes — even though technically the execution is +happening entirely in the input context. + +> **Key mental model**: For `cmd`/`chat`/`agent`, the input block is +> preparation and `execute(input)` is action. For `ruby`, the input block IS +> the action and `execute(input)` is just bookkeeping. + +### Config (`Ruby::Config`) + +Empty subclass of `Cog::Config`. Inherits only the common options (§1). + +### Input (`Ruby::Input`) + +| Field | Type | Required | Default | +|---|---|---|---| +| `value` | `untyped` | Conditional | `nil` | + +**Validation** (`ruby.rb`, line 31): Raises `InvalidInputError` if `value` is +`nil` AND `coerce_ran?` is `false`. After coercion, `nil` is a legitimate value. + +**Coercion** (`ruby.rb`, lines 43–46): Calls `super` (setting `@coerce_ran`), +then sets `@value = input_return_value` unconditionally. This means returning +anything from the input block — including `nil` — sets the value. + +### Output (`Ruby::Output`) + +| Field | Type | Description | +|---|---|---| +| `value` | `untyped` | The exact value passed through from input | + +**Does NOT include** `WithJson`, `WithNumber`, or `WithText` — there is no +`raw_text` implementation. + +#### Dynamic Method Dispatch (`method_missing`) + +The Ruby cog's output uses a three-level dispatch priority (`ruby.rb`, lines +135–145): + +1. **Value delegation**: If `value.respond_to?(name, false)` → delegates via + `value.public_send(name, ...)` +2. **Hash key access**: If `value.is_a?(Hash) && value.key?(name)` → returns the + hash value (or calls it if it's a `Proc`) +3. **Fallback**: `super` → standard `NoMethodError` + +`respond_to_missing?` mirrors this logic for correct introspection. + +#### Special Methods + +| Method | Behavior | +|---|---| +| `[](key)` | Direct hash key access — bypasses method dispatch | +| `call(*args)` | If value is a `Proc`: calls it directly. If value is a `Hash`: first arg must be a `Symbol` key → fetches the `Proc` at that key → calls it | + +### Execute Behavior + +**This is a no-op by design.** The entire method body is: + +```ruby +def execute(input) + Output.new(input.value) # ruby.rb, line 164 +end +``` + +No transformation. No side effects. No LLM call. No shell command. The `ruby` +cog's `execute` exists solely to satisfy the framework's `Config → Input → +Execute → Output` lifecycle contract. All meaningful work is expected to happen +in the input block, with the result either set explicitly via `my.value = ...` +or returned as the block's return value (which gets coerced to `value`). + +--- + +## 6. call System Cog + +**Source**: `lib/roast/system_cogs/call.rb` +**Purpose**: Invoke a named execution scope (defined with `execute(:name) { ... }`) +with a provided value and index. + +### Config (`Call::Config`) + +Empty subclass — no call-specific config options. Inherits only the common +options (§1). + +### Params (`Call::Params`) + +| Field | Type | Description | +|---|---|---| +| `run` | `Symbol` | The name of the execution scope to invoke | +| `name` | `Symbol?` | Optional cog name (auto-generated UUID if omitted) | + +Set at declaration time in the `execute {}` block: +`call(:result, run: :my_scope) { ... }` + +### Input (`Call::Input`) + +| Field | Type | Required | Default | +|---|---|---|---| +| `value` | `untyped` | Yes | `nil` | +| `index` | `Integer` | No | `0` | + +**Validation** (`call.rb`, line 58): Raises if `value.nil?` and `coerce_ran?` +is `false`. + +**Coercion** (`call.rb`, lines 65–67): Calls `super`, then sets `@value = +input_return_value` unless `@value.present?`. + +⚠️ **`present?` pitfall**: If you explicitly set `value` to `false`, `""`, or +`[]`, the coercion will overwrite it with the block's return value. + +### Output (`Call::Output`) + +Wraps an `ExecutionManager` instance (stored in private `@execution_manager`). + +**Primary access**: Use `from()` in the CogInputContext (see below). + +### Manager Module (`Call::Manager`) + +Mixed into `ExecutionManager`. Creates the system cog with an `on_execute` +callback that: + +1. Creates a new `ExecutionManager` for the named scope, passing + `scope_value: input.value` and `scope_index: input.index` (`call.rb`, + lines 96–104) +2. Calls `em.prepare!` then `em.run!` (`call.rb`, lines 105–106) +3. Catches both `ControlFlow::Next` and `ControlFlow::Break` identically — + ends the inner execution early and returns normally (`call.rb`, line 108) +4. Returns `Output.new(em)` (`call.rb`, line 113) + +### InputContext Module (`Call::InputContext`) + +Defines the `from()` method on `CogInputContext`: + +```ruby +from(call_cog_output) # Returns the scope's final_output directly +from(call_cog_output) { ... } # Evaluates block in the inner scope's CogInputContext +``` + +With a block, the block receives `(final_output, scope_value, scope_index)` and +is evaluated via `instance_exec` in the inner scope's `CogInputContext` — +meaning you can access inner-scope cog outputs: + +```ruby +from(call!(:my_call)) { cmd!(:inner_step).text } +``` + +**Implementation**: Uses `instance_variable_get(:@execution_manager)` to extract +the EM from the output — a pragmatic encapsulation bypass (`call.rb`, line 148). +The `scope_value` is `deep_dup`'d before being passed to the block (`call.rb`, +line 152). + +--- + +## 7. map System Cog + +**Source**: `lib/roast/system_cogs/map.rb` +**Purpose**: Execute a named scope once per item in a collection, either serially +or in parallel. + +### Config (`Map::Config`) + +| Method | Stored Value | `valid_parallel!` Returns | Behavior | +|---|---|---|---| +| *(default — no call)* | absent | `1` | **Serial** | +| `parallel(5)` | `5` | `5` | 5 concurrent iterations | +| `parallel(0)` | `nil` | `nil` | Unlimited concurrency | +| `parallel!` | `nil` | `nil` | Unlimited concurrency | +| `no_parallel!` | `1` | `1` | Serial | + +**Query**: `valid_parallel!` → `@values.fetch(:parallel, 1)`. Returns `nil` for +unlimited, `Integer` for limited. Raises `InvalidConfigError` if negative. +(`map.rb`, lines 86–92) + +**`validate!`**: Calls `valid_parallel!` on prepare (`map.rb`, line 71). + +### Params (`Map::Params`) + +| Field | Type | Description | +|---|---|---| +| `run` | `Symbol` | The named scope to invoke per item | +| `name` | `Symbol?` | Optional cog name | + +### Input (`Map::Input`) + +| Field | Type | Required | Default | +|---|---|---|---| +| `items` | `Array[untyped]` | Yes | `[]` | +| `initial_index` | `Integer` | No | `0` | + +**Validation** (`map.rb`, lines 144–146): Raises if `items.nil?`. Also raises if +`items.empty?` and `coerce_ran?` is false (to allow intentionally empty +collections after coercion). + +**Coercion** (`map.rb`, lines 155–159): If `@items` is not `present?`, +converts the return value: enumerable → `.to_a`, non-enumerable → +`Array.wrap(value)`. + +### Output (`Map::Output`) + +Wraps `Array[ExecutionManager?]` — `nil` entries represent iterations that did +not run (due to `break!`). + +| Method | Returns | Notes | +|---|---|---| +| `iteration(index)` | `Call::Output` | Wraps single EM. Raises `MapIterationDidNotRunError` for nil | +| `iteration?(index)` | `bool` | Check if iteration ran | +| `first` | `Call::Output` | Alias for `iteration(0)` | +| `last` | `Call::Output` | Alias for `iteration(-1)` | + +Supports negative indices (e.g., `iteration(-1)` for the last). + +**Error**: `MapIterationDidNotRunError < MapOutputAccessError < Roast::Error` + +### Manager Module (`Map::Manager`) + +Mixed into `ExecutionManager`. Dispatches to serial or parallel based on +`valid_parallel!`: + +#### Serial Execution (`map.rb`, lines 288–302) + +``` +items.each_with_index → create EM → prepare! → run! + rescue Next → continue to next item + rescue Break → stop iterating +nil-fill unexecuted slots in output array +``` + +**Key detail**: `ems.fill(nil, ems.length, items.length - ems.length)` ensures +the output array always matches the input items count. + +#### Parallel Execution (`map.rb`, lines 306–338) + +``` +Async::Barrier.new + Async::Semaphore.new(limit) if limited +items.map.with_index → (semaphore || barrier).async(finished: false) → create EM → prepare! → run! + rescue Next → continue (EM still stored) +barrier.wait → task.wait + rescue Break → barrier.stop + rescue StandardError → barrier.stop; re-raise +Reconstruct ordered Array from Hash +``` + +**Critical implementation details**: +1. **Hash storage**: `ems = {}` keyed by integer index — avoids concurrent Array + mutation issues. Reconstructed to Array via + `(0...items.length).map { |idx| ems[idx] }` (`map.rb`, line 334) +2. **`finished: false`**: Prevents auto-completion before barrier management + (`map.rb`, line 311) +3. **Break in parallel**: Caught during `barrier.wait`, stops all concurrent + tasks (`map.rb`, lines 326–328) +4. **Ensure cleanup**: `barrier&.stop` always called (`map.rb`, line 337) +5. **Ordering guaranteed**: Results are always in input order regardless of + completion order + +### InputContext Module (`Map::InputContext`) + +#### `collect(map_output, &block)` + +Without block: `ems.map { |em| em&.final_output }` — `nil` for unexecuted +iterations. + +With block: Evaluates block in each iteration's `CogInputContext` via +`instance_exec(final_output, scope_value, scope_index, &block)`. `nil` +iterations produce `nil` in the output array. (`map.rb`, lines 375–388) + +#### `reduce(map_output, initial_value = nil, &block)` + +Folds over `ems.compact` (skips nil iterations entirely). Block receives +`(accumulator, final_output, scope_value, scope_index)` and is evaluated in +each iteration's `CogInputContext`. (`map.rb`, lines 426–446) + +⚠️ **Nil-preservation**: If the block returns `nil`, the accumulator is NOT +updated (`map.rb`, lines 438–443). This prevents accidental nil-overwrites but +means you **cannot intentionally set the accumulator to `nil`**. + +--- + +## 8. repeat System Cog + +**Source**: `lib/roast/system_cogs/repeat.rb` +**Purpose**: Execute a named scope in a loop, feeding each iteration's output +as the next iteration's input. Terminated by `break!` or `max_iterations`. + +### Config (`Repeat::Config`) + +Empty subclass — no repeat-specific config options. Inherits only the common +options (§1). + +### Params (`Repeat::Params`) + +| Field | Type | Description | +|---|---|---| +| `run` | `Symbol` | The named scope to invoke for each iteration | +| `name` | `Symbol?` | Optional cog name | + +### Input (`Repeat::Input`) + +| Field | Type | Required | Default | +|---|---|---|---| +| `value` | `untyped` | Yes | `nil` | +| `index` | `Integer` | No | `0` | +| `max_iterations` | `Integer?` | No | `nil` (no limit) | + +**Validation** (`repeat.rb`, lines 71–73): Raises if `value.nil?` and +`coerce_ran?` is false. Raises if `max_iterations` is present and `< 1`. + +**Coercion** (`repeat.rb`, lines 81–83): Calls `super`, then sets `@value` +unless `@value.present?`. + +### Output (`Repeat::Output`) + +Wraps `Array[ExecutionManager]` — unlike Map, there are **no nil entries** +because only completed iterations are stored. + +| Method | Returns | Description | +|---|---|---| +| `value` | `untyped` | Last iteration's `final_output` (`@execution_managers.last&.final_output`) | +| `iteration(index)` | `Call::Output` | Wraps specific iteration's EM | +| `first` | `Call::Output` | Alias for `iteration(0)` | +| `last` | `Call::Output` | Alias for `iteration(-1)` | +| `results` | `Map::Output` | **Bridge to Map's collect/reduce** | + +The `results` method is the key design pattern: it wraps the iteration EMs in a +`Map::Output`, enabling reuse of `collect` and `reduce`: + +```ruby +collect(repeat!(:loop).results) # All iteration outputs as array +reduce(repeat!(:loop).results, 0) { |sum, out| sum + out.integer } # Aggregate +``` + +### Manager Module (`Repeat::Manager`) + +Mixed into `ExecutionManager`. Creates the system cog with an `on_execute` +callback that runs a Ruby `loop`: + +``` +scope_value = input.value.deep_dup # initial deep copy +loop do + create EM(scope: params.run, scope_value:, scope_index: ems.length) + em.prepare! → em.run! + scope_value = em.final_output # CHAIN: output feeds next iteration + break if max_iterations reached +rescue ControlFlow::Break → break +end +Output.new(ems) +``` + +(`repeat.rb`, lines 208–236) + +**Critical details**: +1. **Output chaining**: `scope_value = em.final_output` — the output of + iteration N becomes the input of iteration N+1 (`repeat.rb`, line 228) +2. **Initial deep copy**: `input.value.deep_dup` prevents mutation leakage + (`repeat.rb`, line 214) +3. **Auto-incrementing index**: `scope_index: ems.length` → 0, 1, 2, ... + (`repeat.rb`, line 224) +4. **max_iterations check after execution**: The iteration runs before the limit + check, so `max_iterations: 1` still executes one iteration (`repeat.rb`, + line 229) +5. **Only `Break` is caught**: `ControlFlow::Next` is NOT caught by the Repeat + manager (`repeat.rb`, lines 230–232) + +⚠️ **Known bug**: Since Repeat only catches `Break`, a synchronous cog calling +`next!` inside a repeat loop causes the `Next` exception to escape the repeat +entirely and propagate to the parent scope. See +[07-control-flow-reference.md](07-control-flow-reference.md) for details. + +### How Next Works in Repeat (Subtle) + +When `next!` is called inside a repeat iteration: +1. The inner EM's `wait_for_task_with_exception_handling` catches `Next` → + stops barrier → no re-raise (for async cogs) +2. For sync cogs, `Next` propagates out of `em.run!` and escapes the repeat + loop entirely (bug) +3. For async cogs, `em.run!` completes normally; `compute_final_output` runs; + `scope_value = em.final_output` picks up whatever was computed — the + "skipped" iteration still counts and feeds into the next iteration + +### Comparison: Map vs Repeat + +| Aspect | Map | Repeat | +|---|---|---| +| Item count | Predetermined (`items.length`) | Unbounded (until `break!` / `max_iterations`) | +| Nil entries | Yes (for break'd iterations) | No — only completed iterations stored | +| `iteration()` on nil | Raises `MapIterationDidNotRunError` | N/A | +| Output chaining | No (each iteration gets original item) | Yes (output N → input N+1) | +| `value` accessor | N/A | Returns last iteration's final_output | +| `results` bridge | N/A | Returns `Map::Output` for collect/reduce | + +--- + +## 9. Config Merge Cascade + +**Source**: `lib/roast/config_manager.rb`, `config_for` method (lines 44–59) + +When a cog runs, its configuration is assembled by merging four layers. Each +layer overrides the previous via `Config#merge` (Hash merge, right-side wins): + +``` +Step 1: Start with cog-type Config, seeded with global @values (deep_dup'd) +Step 2: Merge general config for this cog type (e.g., config { agent { ... } }) +Step 3: Merge each regexp-matched config (e.g., config { agent(/review/) { ... } }) +Step 4: Merge name-specific config (e.g., config { agent(:analyze) { ... } }) +Step 5: Call validate! on the merged result +``` + +**Detail on Step 1**: The global config's `@values` hash is extracted via +`instance_variable_get(:@values)` and deep_dup'd into a new cog-specific Config +instance (`config_manager.rb`, line 48). This is a "back-door" access pattern — +`ConfigManager` reaches into `Cog::Config`'s internal storage rather than using +public API, because global values may contain keys that are not declared on the +specific cog's Config subclass. + +**Regexp matching** (`config_manager.rb`, lines 50–52): All regexp-scoped +configs for the cog class are checked against the cog's name via +`pattern.match?(name.to_s)`. Multiple patterns can match — each matching +config is merged in iteration order. + +**Reopenable config blocks**: Multiple `config {}` blocks in a workflow are +collected and evaluated sequentially during `prepare!`. They all mutate the same +config objects, so later blocks override earlier ones within the same scope level. + +--- + +## 10. Output Mixin Modules + +**Source**: `lib/roast/cog/output.rb` + +All three modules depend on the implementing class providing a private +`raw_text` method that returns `String?`. + +### WithText + +| Method | Returns | Behavior | +|---|---|---| +| `text` | `String` | `raw_text.strip` | +| `lines` | `Array[String]` | `raw_text.lines.map(&:strip)` | + +### WithJson + +| Method | Returns | Behavior | +|---|---|---| +| `json!` | `Hash[Symbol, untyped]` | Parse JSON; raise `JSON::ParserError` on failure | +| `json` | `Hash[Symbol, untyped]?` | Parse JSON; return `nil` on failure | + +**All JSON keys are symbolized** (`symbolize_names: true`). + +**Empty input**: Returns `{}` if input is `nil` or blank (`output.rb`, line 23). + +**Results are memoized** in `@json` (`output.rb`, line 25). + +**Candidate extraction priority** (`output.rb`, lines 68–75): + +| Priority | Source | Scanning Order | +|---|---|---| +| 1 | Entire input string (stripped) | — | +| 2 | `` ```json `` code blocks | **Last first** | +| 3 | `` ``` `` code blocks (no language) | **Last first** | +| 4 | `` ``` `` code blocks (any non-json language) | **Last first** | +| 5 | JSON-like `{}`/`[]` blocks extracted from text | **Longest first** | + +**Why last-first?** LLMs tend to place their final, refined answer in the last +code block. Scanning last-first finds the most relevant answer faster. + +Each candidate is tried with `JSON.parse`; the first successful parse wins. If +all fail, raises `JSON::ParserError`. + +### WithNumber + +| Method | Returns | Behavior | +|---|---|---| +| `float!` | `Float` | Parse number; raise `ArgumentError` on failure | +| `float` | `Float?` | Parse number; return `nil` on failure | +| `integer!` | `Integer` | `float!.round` | +| `integer` | `Integer?` | `integer!` or `nil` on failure | + +**Results are memoized** in `@float` and `@integer`. + +**Candidate extraction priority** (`output.rb`, lines 230–249): + +| Priority | Source | Scanning Order | +|---|---|---| +| 1 | Entire string (stripped) | — | +| 2 | Each line | **Bottom-up** (last line first) | +| 3 | Number-pattern matches within each line | **Bottom-up, rightmost first** | + +**Normalization** (`output.rb`, lines 255–261): Strips currency symbols +(`$¢£€¥`), commas, underscores, and spaces. Validates against the pattern: +`-?\d+(?:\.\d*)?(?:[eE][+-]?\d+)?` + +Very permissive: handles `1,234.56`, `€1.23`, `1_000`, scientific notation. + +### Design Pattern + +Both JSON and number parsing use a **"generate candidates → try each → first +success wins"** strategy. This is intentionally resilient to messy LLM output +where actual data may be buried in surrounding prose or code blocks. The +framework absorbs this parsing complexity so workflow authors don't have to. + +--- + +## 11. Boolean Default Patterns + +The codebase uses **four distinct patterns** for boolean config defaults. This +inconsistency is a known artifact of organic development. Understanding the +patterns is essential for both reading existing code and writing custom cogs. + +| Pattern | Example | Default When Unset | Correct for Falsy? | +|---|---|---|---| +| `@values.fetch(:key, true)` | `abort_on_failure?` | `true` | ✅ Yes | +| `@values.fetch(:key, false)` | `show_prompt?` (chat, agent) | `false` | ✅ Yes | +| `!!@values[:key]` | `async?` | `false` | ✅ Yes | +| `@values[:key] != false` | `fail_on_error?` (cmd) | `true` | ⚠️ Only `false` | + +The `fetch` pattern is the clearest and most correct — it handles all falsy +values correctly. The `!!` pattern also works (nil → false). The `!= false` +pattern is correct for its specific use (only `false` is the negative case, +`nil` means "not set, use default true"), but it's the least obvious. + +The `field` macro's `@values[key] || default.deep_dup` pattern is **incorrect +for booleans** — see §1 for the pitfall. + +--- + +## Where to Go Next + +- **[01-architecture-overview.md](01-architecture-overview.md)** — Foundational + mental model (read first if you haven't) +- **[02-dsl-users-guide.md](02-dsl-users-guide.md)** — How to write workflows + using these cogs (practical usage) +- **[06-metaprogramming-map.md](06-metaprogramming-map.md)** — How the config, + execute, and input methods for each cog type are dynamically defined +- **[07-control-flow-reference.md](07-control-flow-reference.md)** — The + complete exception propagation matrix across all cog types +- **[10-writing-custom-cogs.md](10-writing-custom-cogs.md)** — How to create + your own cog types diff --git a/internal/documentation/architecture/04-design-philosophy.md b/internal/documentation/architecture/04-design-philosophy.md new file mode 100644 index 00000000..3910b79d --- /dev/null +++ b/internal/documentation/architecture/04-design-philosophy.md @@ -0,0 +1,312 @@ +# Document 4: Design Philosophy + +_Why the framework is built the way it is._ + +This document connects Roast's implementation decisions to software design principles and the architectural reviews that shaped them. It answers "why?" for every major pattern you'll encounter in the codebase. + +--- + +## 1. Declarative-First: Separate Declaration from Execution + +### The Principle + +Roast workflows are **declarative**: you describe _what_ should happen, and the framework decides _how_ and _when_ to execute it. This is enforced by the two-phase lifecycle: + +1. **Prepare** (`prepare!`) — Collect all declarations. Build the complete plan. +2. **Start** (`start!`) — Execute the plan sequentially. + +You cannot conditionally create cogs at runtime. The cog stack is fixed once `prepare!` completes. Input blocks (the `{ |my| ... }` passed to each cog call) run at execution time and _can_ be conditional — but the cog itself always exists. + +### Why This Design + +**Predictability over flexibility.** If cog creation were conditional on runtime data, the framework couldn't guarantee: +- That output references (`cmd!(:name)`) always resolve to a known cog +- That the config cascade is complete before any cog runs +- That async barriers know the full set of tasks they're managing + +The trade-off is that workflows can't dynamically branch. Instead, they use input-block `skip!` to bypass cogs that shouldn't execute in a given run. + +### The Terraform Analogy + +| Phase | Terraform | Roast | +|-------|-----------|-------| +| Collect | `.tf` file parsing | `extract_dsl_procs!` — `instance_eval` on workflow file | +| Plan | `terraform plan` | `ConfigManager.prepare!` + `ExecutionManager.prepare!` | +| Apply | `terraform apply` | `ExecutionManager.run!` | + +Source: `workflow.rb:46–58` (prepare!), `workflow.rb:61–73` (start!), `workflow.rb:134–136` (extract_dsl_procs!) + +### Why `extract_dsl_procs!` Doesn't Evaluate + +`workflow.rb:135` calls `instance_eval(@workflow_definition, ...)` on the Workflow instance. This runs the top-level workflow file — but the only methods available at that level are `config`, `execute`, and `use`. These methods _collect_ blocks into arrays (`@config_procs`, `@execution_procs`) without evaluating any of them. The actual evaluation happens later during each manager's `prepare!` call, when the blocks are `instance_eval`'d against the appropriate context. + +This two-stage collection ensures that all `use` declarations (which register custom cogs) complete before any config or execute block tries to reference them. + +--- + +## 2. Deep Copy at Every Boundary + +### The Principle + +Every time data crosses a component boundary, it is `deep_dup`'d. This gives each component an **independent copy** that can be mutated freely without affecting any other component. + +### The Complete Catalogue (13 sites, 5 purposes) + +**Config isolation (4 sites):** + +| Site | File:Line | Purpose | +|------|-----------|---------| +| Field getter fallback | `cog/config.rb:116` | `@values[key] \|\| default.deep_dup` — prevent mutation of class-level default | +| Field reset | `cog/config.rb:126` | `@values[key] = default.deep_dup` — same protection via `use_default_!` | +| Global values seeding | `config_manager.rb:48` | `@global_config.instance_variable_get(:@values).deep_dup` — isolate each cog's config from global | +| Config to cog | `execution_manager.rb:99` | `cog_config.deep_dup` — cog receives its own config copy | + +**Scope isolation (3 sites):** + +| Site | File:Line | Purpose | +|------|-----------|---------| +| Scope value to cog | `execution_manager.rb:101` | `@scope_value.deep_dup` — parallel cogs can't corrupt shared scope | +| Repeat iteration chaining | `system_cogs/repeat.rb:214` | `input.value.deep_dup` — iteration N+1 gets a clean copy of N's output | +| `from()` helper | `system_cogs/call.rb:152` | `em.instance_variable_get(:@scope_value).deep_dup` — consumer can't mutate producer's scope | + +**Output isolation (1 site):** + +| Site | File:Line | Purpose | +|------|-----------|---------| +| Output access | `cog_input_manager.rb:78` | `.output.deep_dup` — every accessor call returns an independent copy | + +**Event path isolation (1 site):** + +| Site | File:Line | Purpose | +|------|-----------|---------| +| Fiber path snapshot | `task_context.rb:24` | `Fiber[:path]&.deep_dup \|\| []` — event carries immutable path snapshot | + +**Session fork isolation (4 sites):** + +| Site | File:Line | Purpose | +|------|-----------|---------| +| From chat | `cogs/chat/session.rb:17` | `chat.messages.deep_dup` — new Session doesn't share message array | +| First N | `cogs/chat/session.rb:37` | `@messages.first(n).deep_dup` — truncated session is independent | +| Last N | `cogs/chat/session.rb:49` | `@messages.last(n).deep_dup` — same | +| Apply to chat | `cogs/chat/session.rb:60` | `@messages.deep_dup` — restoring a session doesn't consume it | + +### The Erlang Analogy + +This pattern mirrors Erlang's message-passing semantics: processes (fibers, in Roast) never share mutable state. Communication happens by copying data at every boundary. The cost is memory; the benefit is **total isolation without locks**. Since Roast uses cooperative (fiber-based) concurrency, there are no mutexes, no thread-safety concerns, no race conditions — as long as the deep copy discipline is maintained. + +### The Invariant + +**If you add a new boundary crossing where data flows between components, you must `deep_dup` at that boundary.** Violating this invariant will produce subtle bugs that only manifest under parallel execution. + +--- + +## 3. Config Layering & Nilability + +### Value Absence vs Value Presence + +The config merge cascade relies on a critical distinction: **a key that was never set** (absent from the `@values` hash) vs **a key that was explicitly set** (present, even if the value is `nil`). + +`Config#merge` (`cog/config.rb:47`) uses `Hash#merge`, which only overwrites keys present in the incoming hash. If a cog-type config never calls `model "gpt-4"`, the `:model` key is absent, and the global value flows through. If it explicitly calls `model "gpt-4"`, the key is present and overrides. + +This enables the CSS-like specificity cascade: + +``` +global → cog-type-general → regexp-matched → name-specific → validate! +``` + +Each layer only contributes the keys it explicitly sets. Everything else is inherited from the less-specific layer. + +### The `field` Macro's Falsy-Value Limitation + +`cog/config.rb:116`: `@values[key] || default.deep_dup` + +This means `false` and `nil` values fall through to the default. The built-in boolean configs (`async!`, `abort_on_failure!`) work around this by manipulating `@values` directly without going through the `field` macro. Custom cogs that need boolean fields should follow the same pattern. + +### Why `instance_variable_get` for Global Config? + +`config_manager.rb:48` uses `@global_config.instance_variable_get(:@values)` despite `Config` having a public `attr_reader :values` (config.rb:30). This is a vestigial pattern from before the accessor existed. It still works correctly but is technically redundant. + +--- + +## 4. Three Contexts as "Deep Modules" + +### Ousterhout's Principle (A Philosophy of Software Design, Ch. 4) + +A "deep module" has a simple interface that hides substantial complexity. The three evaluation contexts in Roast are exemplary deep modules: + +| Context | Interface | Hidden Complexity | +|---------|-----------|-------------------| +| `ConfigContext` | `agent(:name) { model "gpt-4o" }` | Dispatches through `ConfigManager.on_config`, resolves to correct config store (general/regexp/named), evaluates block via `instance_exec` against the config object | +| `ExecutionContext` | `agent(:name) { \|my\| my.prompt = "..." }` | Dispatches through `ExecutionManager.on_execute`, distinguishes standard cogs from system cogs, creates appropriate instances, saves input blocks, pushes to cog stack | +| `CogInputContext` | `agent!(:name).response` | Dispatches through `CogInputManager.cog_output!`, resolves cog from store, calls `wait` (blocking if async), validates state, returns `output.deep_dup` | + +The workflow author sees one simple API: call a cog-type method with a name and a block. The three contexts route that identical-looking call to completely different operations based on _where_ it appears in the workflow. + +### Why Blank Classes? + +`ConfigContext`, `ExecutionContext`, and `CogInputContext` are intentionally blank at the class level. All their methods are installed dynamically via `define_singleton_method` during `prepare!`. This means: + +1. **No method exists until it's needed** — typos produce clear `NoMethodError` at eval time +2. **The method set is customizable** — custom cogs automatically get their own methods in all three contexts via the Registry → bind loop +3. **IDE/Sorbet support** is provided via RBI shims that document the dynamic methods + +--- + +## 5. Pulling Complexity Downwards + +### Ousterhout's Principle (Ch. 7) + +"Pull complexity downwards": modules should absorb complexity on behalf of their consumers, even if it makes the module's implementation harder. + +**JSON extraction** (`cog/output.rb:55–131`): `WithJson#json!` doesn't just call `JSON.parse`. It tries: +1. The entire output string +2. `` ```json `` code blocks (last first) +3. Bare `` ``` `` code blocks (last first) +4. Any-language code blocks (last first) +5. `{ }` and `[ ]` patterns (longest first) + +The "last first" ordering exists because LLMs tend to refine their output — the final JSON block in a response is most likely to be the correct one. + +**Number extraction** (`cog/output.rb:214–261`): `WithNumber#float!` scans bottom-up because LLMs often explain their reasoning before stating a final answer. It strips currency symbols, digit separators, and validates against a strict regex before calling `Float()`. + +The workflow author simply calls `.json` or `.integer`. The framework absorbs all the messy reality of LLM output parsing. + +--- + +## 6. Error Hierarchy as Information Hiding + +### Consumer-Side vs Producer-Side Errors + +The error hierarchy embodies a deliberate separation: + +- **`CogError`** (producer-side): something went wrong _inside_ a cog (e.g., `CogAlreadyStartedError`). These are programming errors in the framework itself. +- **`CogOutputAccessError`** (consumer-side): something went wrong when _another component_ tried to access a cog's output. These are workflow-level conditions. + +`CogOutputAccessError` is NOT a subclass of `CogError`. This is intentional: `rescue CogError` catches only producer bugs. Consumer-side errors require their own handling. + +The consumer-side hierarchy further separates: +- `CogDoesNotExistError` — always fatal (programming error, raised even in tolerant mode) +- `CogNotYetRunError`, `CogSkippedError`, `CogStoppedError` — swallowed by `outputs`, raised by `outputs!` +- `CogFailedError` — always propagates (never swallowed by either variant) + +--- + +## 7. Flat Scope Namespace + +Execution scopes (`execute(:name) { ... }`) are **globally addressable**. A `call(:x, run: :my_scope)` at any depth can invoke any named scope defined at the top level. Scopes are not lexically nested — there's no concept of "local scope." + +### Why? + +1. **Reusability**: The same scope can be called from multiple places (map, repeat, or direct call) +2. **Simplicity**: No scope resolution rules, no shadowing, no inheritance chains +3. **Predictability**: Every scope name resolves to exactly one definition + +### Trade-off + +The risk is naming collisions. With a flat namespace, two scopes with the same name overwrite silently (the `@execution_procs[scope]` array just accumulates). This is analogous to CSS selectors or Make targets — simple addressing at the cost of global uniqueness discipline. + +--- + +## 8. Composition over Inheritance + +### The Repeat→Map Output Bridge + +`Repeat::Output#results` returns a `Map::Output` instance, wrapping the repeat's execution managers in map's collection interface. This means `collect()` and `reduce()` work identically on both map and repeat results. + +Rather than making Repeat inherit from Map or extracting a shared base class, the framework composes: Repeat's output _contains_ a Map output. The implementation is a one-line delegation that avoids duplicating collection logic. + +### Mixin Modules for Output Parsing + +`WithText`, `WithJson`, and `WithNumber` are modules that any cog can include. They compose with each other (a single output class can include all three) and with the cog's own output fields. The only contract: implement `raw_text` (private method returning the string to parse). + +--- + +## 9. Architectural Decisions Traced to Reviews + +### PR #485 — Fiber-Based Cooperative Concurrency + +Juniper mandated cooperative (fiber-based) concurrency over threads, with no default timeouts. This created the `Async::Barrier` model where cogs yield control voluntarily. The implication: the framework can never forcibly terminate a running cog. `break!` sets a flag and `barrier.stop` cancels pending tasks — but a cog in the middle of an HTTP request will complete before yielding. + +This is also why the sync/async `next!` divergence exists: `barrier.wait` is the only place where async exceptions are caught, so `next!` from an async cog gets swallowed at that boundary rather than propagating to the parent scope. + +### PR #428 — Config-Block-First + +Juniper rejected XDG_CONFIG_HOME, global config files, and environment-based configuration for cog behavior. All cog config belongs inside the workflow DSL. The only external config source is `ROAST_WORKING_DIRECTORY` (CLI-level, not cog-level). This keeps workflows self-contained and reproducible. + +### PR #476 — No sorbet-runtime in the DSL Layer + +Roast uses `typed: true` with inline RBS annotations (`#:`) and RBI shims for IDE support, but has zero `sorbet-runtime` dependency. Types are checked at development time (via `srb tc`) but never at runtime. `type_toolkit` provides lightweight runtime utilities. The rationale: runtime type checking in a DSL layer adds overhead and confusing error messages for workflow authors. + +### PR #783 — No Mutable Objects in Config @values + +Juniper rejected storing provider instances (mutable objects) in config `@values`. Config values must be primitives and strings only. The agent cog's provider is memoized on the Cog instance itself (`@provider ||= ...`), not in config. This ensures config remains copyable (via `deep_dup`) and serializable. + +--- + +## 10. The Four Workflow Archetypes + +Every Roast workflow maps to one of four structural patterns: + +### Pipeline + +Sequential processing: each cog's output feeds the next cog's input. + +``` +cmd(:fetch) → chat(:analyze) → cmd(:write) +``` + +Most workflows are pipelines. The cog stack is executed in order, and input blocks reference earlier outputs via `cmd!(:fetch).text`. + +### Fan-Out / Fan-In (Map-Reduce) + +Parallel processing of a collection, with aggregation: + +``` +cmd(:list) → map(:process, run: :item_scope) → outputs { reduce(map!(:process), ...) } +``` + +The `map` system cog fans out to N parallel executions. `collect()` and `reduce()` fan back in. + +### Iterative Refinement (Repeat) + +Loop until a condition is met, with each iteration refining the previous output: + +``` +repeat(:refine, run: :improve_step) { initial_draft } +``` + +State flows forward: output of iteration N becomes scope_value of iteration N+1. `break!` terminates. + +### Multi-Model Composition + +Different cogs use different AI models for different tasks within the same workflow: + +``` +config { + chat(:draft) { model "gpt-4o" } + agent(:review) { provider :claude } + chat(:summarize) { model "gpt-4o-mini" } +} +``` + +The config cascade enables per-cog model selection. Sessions can be forked between models (deep-copied message arrays applied to new providers). + +--- + +## Summary of Design Principles + +| Pattern | Principle | Source | +|---------|-----------|--------| +| Two-phase lifecycle | Separate declaration from execution | Terraform plan/apply | +| Deep copy at boundaries | Isolation without locks | Erlang message-passing | +| Config merge cascade | Specificity-based override | CSS cascade | +| Three blank contexts | Deep modules (simple interface, complex internals) | Ousterhout Ch. 4 | +| Output parsing mixins | Pull complexity downwards | Ousterhout Ch. 7 | +| Error hierarchy split | Consumer vs producer boundaries | Information hiding | +| Flat scope namespace | Global addressability | Make targets, CSS selectors | +| Repeat→Map bridge | Composition over inheritance | GoF, SOLID | +| No mutable config | Copyable, serializable state | Immutability discipline | +| Fiber-based concurrency | Cooperative scheduling, no locks | PR #485 | +| Config-block-first | Self-contained workflows | PR #428 | +| No sorbet-runtime | Dev-time checks, no runtime overhead | PR #476 | +| Primitives-only config | Deep-dup safety | PR #783 | diff --git a/internal/documentation/architecture/05-execution-engine-internals.md b/internal/documentation/architecture/05-execution-engine-internals.md new file mode 100644 index 00000000..0ac239f2 --- /dev/null +++ b/internal/documentation/architecture/05-execution-engine-internals.md @@ -0,0 +1,904 @@ +# Document 5: Execution Engine Internals + +_Deep reference for how the three managers orchestrate the full Roast lifecycle._ + +**Primary audience**: AI coding agents (critical for navigating code), Intern (after completing the learning path) + +--- + +## Overview + +The execution engine is a triad of collaborating managers: + +| Manager | Source | Context owned | Responsibility | +|---------|--------|---------------|----------------| +| `ConfigManager` | `lib/roast/config_manager.rb` | `ConfigContext` | Evaluate `config {}` blocks, assemble merged configs | +| `ExecutionManager` | `lib/roast/execution_manager.rb` | `ExecutionContext` | Evaluate `execute {}` blocks, run cog stack, manage async | +| `CogInputManager` | `lib/roast/cog_input_manager.rb` | `CogInputContext` | Provide runtime data access (output, params, control flow) | + +Each manager owns one blank context class and dynamically installs methods on it via `define_singleton_method`. The context instances are the surfaces where user-written DSL blocks are evaluated via `instance_eval` or `instance_exec`. + +--- + +## 1. ExecutionManager + +**File**: `lib/roast/execution_manager.rb` + +### 1.1 Class Structure + +``` +ExecutionManager + include SystemCogs::Call::Manager + include SystemCogs::Map::Manager + include SystemCogs::Repeat::Manager +``` + +The three system cog Manager modules are mixed in so they can access EM internals (`@cog_registry`, `@config_manager`, `@all_execution_procs`, `@workflow_context`) when creating child EMs. This is **intentionally not polymorphic** — system cog execution needs EM encapsulation, and making it polymorphic would leak those internals through a public interface. + +### 1.2 Constructor (lines 51–74) + +```ruby +def initialize( + cog_registry, # Cog::Registry — the 7 registered cog types + config_manager, # ConfigManager — shared across all EMs in a workflow + all_execution_procs, # Hash[Symbol?, Array[Proc]] — ALL scopes, flat namespace + workflow_context, # WorkflowContext — params, tmpdir, workflow_dir + scope: nil, # Symbol? — which scope's procs to evaluate + scope_value: nil, # untyped — passed to cogs as scope_value + scope_index: 0 # Integer — iteration counter (for map/repeat) +) +``` + +Creates fresh instances of: +- `Cog::Store` — name→cog mapping (uniqueness enforced) +- `Cog::Stack` — ordered execution queue (FIFO via `shift`) +- `ExecutionContext` — blank target for DSL method installation +- `CogInputManager` — immediately binds accessors on its `CogInputContext` +- `Async::Barrier` — task group for cooperative scheduling + +Also initializes: +- `@final_output = nil` — computed once, then frozen via flag +- `@final_output_computed = false` — idempotency guard + +**Key design point**: `@all_execution_procs` is the *same Hash reference* across every child EM in the workflow. Scopes are globally addressable — a scope defined at the workflow level is callable from any depth. The `scope:` parameter simply selects which key's procs to evaluate. + +### 1.3 prepare! (lines 77–85) + +```ruby +def prepare! + raise ExecutionManagerAlreadyPreparedError if preparing? || prepared? + @preparing = true + bind_outputs # install `outputs` and `outputs!` DSL methods + bind_registered_cogs # install cog type methods (agent, cmd, etc.) + my_execution_procs.each { |ep| @execution_context.instance_eval(&ep) } + @prepared = true +end +``` + +**Step by step**: + +1. **Guard**: Double-prepare raises immediately. Uses both `preparing?` and `prepared?` flags. +2. **`bind_outputs`** (lines 228–237): Installs `outputs` and `outputs!` as singleton methods on `@execution_context`. Both capture `method(:on_outputs)` / `method(:on_outputs!)` closures and delegate to them. +3. **`bind_registered_cogs`** (lines 182–183): Iterates `@cog_registry.cogs` (all 7 entries) and calls `bind_cog` for each. +4. **Evaluate execution procs**: The user's `execute {}` blocks run against the context. This is when all cog declarations actually execute — calling `agent(:name) { ... }` routes to `on_execute`, which creates cog instances and pushes them onto the stack. + +**`my_execution_procs`** (lines 163–167): Validates that `@all_execution_procs` has a key matching `@scope`. If not, raises `ExecutionScopeDoesNotExistError`. Returns the array of procs (or `[]` if the key exists but is nil). + +### 1.4 bind_cog (lines 187–197) + +```ruby +def bind_cog(cog_method_name, cog_class) + on_execute_method = method(:on_execute) + cog_method = proc do |*args, **kwargs, &cog_input_proc| + on_execute_method.call(cog_class, args, kwargs, cog_input_proc) + end + @execution_context.instance_eval do + raise IllegalCogNameError, cog_method_name if respond_to?(cog_method_name, true) + define_singleton_method(cog_method_name, cog_method) + end +end +``` + +**Pattern**: Captures a reference to `on_execute` via `method(:on_execute)`. Creates a proc that closes over both `cog_class` and the method reference. Installs it on the context via `define_singleton_method`. + +**Name conflict check**: `respond_to?(cog_method_name, true)` — the `true` includes private methods. A cog named `:freeze` would conflict with `Object#freeze`. Checked at prepare-time for early error surfacing. + +### 1.5 on_execute (lines 200–226) — The Dispatch Hub + +```ruby +def on_execute(cog_class, cog_args, cog_kwargs, cog_input_proc) + if cog_class <= SystemCog + cog_params = cog_class.params_class.new(*cog_args, **cog_kwargs) + cog_instance = if cog_class == SystemCogs::Call + create_call_system_cog(cog_params, cog_input_proc) + elsif cog_class == SystemCogs::Map + create_map_system_cog(cog_params, cog_input_proc) + elsif cog_class == SystemCogs::Repeat + create_repeat_system_cog(cog_params, cog_input_proc) + else + raise NotImplementedError, "No system cog manager defined for #{cog_class}" + end + else + cog_name = Array.wrap(cog_args).shift + if cog_name + anonymous = false + else + anonymous = true + cog_name = Cog.generate_fallback_name # Random.uuid.to_sym + end + cog_instance = cog_class.new(cog_name, cog_input_proc, anonymous:) + end + add_cog_instance(cog_instance) +end +``` + +**System cogs**: First constructs a `Params` object from the args/kwargs, then delegates to the appropriate Manager module method (e.g., `create_call_system_cog`). The Manager methods are defined in `SystemCogs::Call::Manager`, etc., which are `include`d at the top of `ExecutionManager`. + +**Standard cogs**: Extracts the name from the first positional arg. If no name is provided, generates a UUID-based anonymous name. Anonymous cogs exist in the Store but cannot be meaningfully referenced by users. + +**`add_cog_instance`** (lines 170–173): Inserts into both `@cogs` (Store — for lookup) and `@cog_stack` (Stack — for execution order). Store raises `CogAlreadyDefinedError` on duplicate names. + +### 1.6 run! (lines 87–116) — The Main Loop + +```ruby +def run! + raise ExecutionManagerNotPreparedError unless prepared? + raise ExecutionManagerCurrentlyRunningError if running? + + @running = true + Sync do |sync_task| + sync_task.annotate("ExecutionManager #{@scope}") + TaskContext.begin_execution_manager(self) + @cog_stack.each do |cog| + cog_config = @config_manager.config_for(cog.class, cog.name) + cog_task = cog.run!( + @barrier, + cog_config.deep_dup, + cog_input_context, + @scope_value.deep_dup, + @scope_index, + ) + cog_task.wait unless cog_config.async? + end + @barrier.wait { |task| wait_for_task_with_exception_handling(task) } + compute_final_output + ensure + @barrier.stop + compute_final_output + TaskContext.end + @running = false + end +end +``` + +**Detailed execution trace**: + +1. **Guards**: Must be prepared. Cannot re-enter. +2. **Sync**: Enters an Async event loop (or reuses the current fiber scheduler). +3. **TaskContext annotation**: Registers this EM in the fiber-local execution path for debugging/eventing. +4. **Iterate cog stack**: For each cog: + - a. **Resolve config**: `config_for(cog.class, cog.name)` runs the full 5-step merge cascade (see §2.3). + - b. **Deep dup config**: `cog_config.deep_dup` — prevents config mutation by one cog from leaking to the next. + - c. **Deep dup scope_value**: `@scope_value.deep_dup` — prevents scope value mutation by one cog from affecting subsequent cogs. + - d. **Launch cog**: `cog.run!` creates an async task on the barrier. Returns the `Async::Task` handle. + - e. **Wait if sync**: `cog_task.wait unless cog_config.async?` — blocks the fiber until the cog completes. This is what makes sync cogs sequential. +5. **Barrier wait**: After the loop, waits on remaining async tasks in completion order. Each task is processed through `wait_for_task_with_exception_handling`. +6. **Eager compute**: `compute_final_output` is called immediately so its result is available for chaining (e.g., in repeat loops). +7. **Ensure block**: Always runs: + - `@barrier.stop` — kills any still-running tasks (e.g., after Break) + - `compute_final_output` — idempotent second call ensures output is always computed + - `TaskContext.end` — pops the fiber-local path element + - `@running = false` — resets state + +### 1.7 Deep Dup Boundaries in run! + +Two explicit `deep_dup` calls on lines 99 and 101: + +| Site | What's copied | Why | +|------|---------------|-----| +| `cog_config.deep_dup` | The merged config object | Cog N modifying its config mustn't affect Cog N+1 | +| `@scope_value.deep_dup` | The current scope value | Cog N modifying the scope value mustn't affect Cog N+1 | + +Both are per-cog copies — called once for each cog in the stack. + +### 1.8 wait_for_task_with_exception_handling (lines 148–160) + +```ruby +def wait_for_task_with_exception_handling(task) + task.wait +rescue ControlFlow::Next + @barrier.stop +rescue ControlFlow::Break => e + @barrier.stop + compute_final_output + raise e +rescue StandardError => e + @barrier.stop + raise e +end +``` + +This is the **critical async exception handler**. It processes tasks that complete during `@barrier.wait`: + +| Exception | Behavior | Propagates? | +|-----------|----------|-------------| +| `ControlFlow::Next` | Stops barrier (kills remaining tasks) | **NO** — swallowed | +| `ControlFlow::Break` | Stops barrier, computes output | **YES** — re-raised | +| `StandardError` | Stops barrier | **YES** — re-raised | + +**⚠️ CRITICAL BEHAVIORAL DIFFERENCE**: This handler only processes tasks that were *async* (not already awaited via `cog_task.wait`). For sync cogs, exceptions propagate directly from `cog_task.wait` (line 104), exiting the `each` loop immediately. This means: + +- **Sync `next!`**: Propagates out of `run!` to the parent scope (Map/Repeat manager catches it) +- **Async `next!`**: Swallowed here — parent scope never knows + +- **Sync `break!`**: Propagates out of `run!` (via `cog_task.wait`) +- **Async `break!`**: Also propagates — re-raised after stopping barrier + +### 1.9 compute_final_output (lines 254–283) + +```ruby +def compute_final_output + return if @final_output_computed + @final_output_computed = true + outputs_proc = @outputs_bang || @outputs + + @final_output = if outputs_proc + @cog_input_manager.context.instance_exec(@scope_value, @scope_index, &outputs_proc) + else + last_cog_name = @cog_stack.last&.name + raise CogInputManager::CogDoesNotExistError, "no cogs defined in scope" unless last_cog_name + @cog_input_manager.send(:cog_output, last_cog_name) + end +rescue ControlFlow::SkipCog, ControlFlow::Next + # Swallowed — final_output becomes nil +rescue CogInputManager::CogNotYetRunError, CogInputManager::CogSkippedError, CogInputManager::CogStoppedError => e + raise e if @outputs_bang.present? + # Swallowed for `outputs` (tolerant) — final_output becomes nil +end +``` + +**Idempotent**: The `@final_output_computed` flag prevents re-computation. Called both eagerly (line 109) and in `ensure` (line 112). + +**Priority**: +1. If `outputs_proc` exists (from `outputs {}` or `outputs! {}`): evaluate it in `CogInputContext`, passing scope_value and scope_index. +2. If NO outputs proc: fall back to the last cog's output via `cog_output(last_cog_name)`. + +**`outputs` vs `outputs!`** (mutually exclusive — `OutputsAlreadyDefinedError` if both set): +- `outputs`: Tolerant. `CogNotYetRunError`, `CogSkippedError`, `CogStoppedError` are swallowed → `nil`. +- `outputs!`: Strict. Same exceptions re-raised to the caller. + +Both variants swallow `SkipCog` and `Next` — these are valid ways to "produce no output" from an outputs block. + +**Linchpin for Repeat**: `em.final_output` is what feeds `scope_value` for the next iteration of a repeat loop. The idempotent computation ensures it's always available even after Break. + +### 1.10 bind_outputs (lines 228–251) + +```ruby +def bind_outputs + on_outputs_method = method(:on_outputs) + on_outputs_bang_method = method(:on_outputs!) + method_to_bind = proc { |&outputs_proc| on_outputs_method.call(outputs_proc) } + bang_method_to_bind = proc { |&outputs_proc| on_outputs_bang_method.call(outputs_proc) } + @execution_context.instance_eval do + define_singleton_method(:outputs, method_to_bind) + define_singleton_method(:outputs!, bang_method_to_bind) + end +end +``` + +Both `on_outputs` (line 240) and `on_outputs!` (line 247) raise `OutputsAlreadyDefinedError` if either `@outputs` or `@outputs_bang` is already set. Only one outputs declaration per scope. + +### 1.11 Error Hierarchy + +``` +Roast::Error + ExecutionManagerError + ExecutionManagerNotPreparedError + ExecutionManagerAlreadyPreparedError + ExecutionManagerCurrentlyRunningError + ExecutionScopeDoesNotExistError + ExecutionScopeNotSpecifiedError + IllegalCogNameError + OutputsAlreadyDefinedError +``` + +--- + +## 2. ConfigManager + +**File**: `lib/roast/config_manager.rb` + +### 2.1 Internal Storage + +```ruby +@global_config = Cog::Config.new # target of `global {}` blocks +@general_configs = {} # Hash[singleton(Cog), Cog::Config] +@regexp_scoped_configs = {} # Hash[singleton(Cog), Hash[Regexp, Cog::Config]] +@name_scoped_configs = {} # Hash[singleton(Cog), Hash[Symbol, Cog::Config]] +``` + +Four levels of configuration storage, one per cascade tier. Each maps a cog *class* (not instance) to a config object. This means config is resolved per-class-and-name, not per-instance. + +### 2.2 prepare! (lines 23–31) + +```ruby +def prepare! + raise ConfigManagerAlreadyPreparedError if preparing? || prepared? + @preparing = true + bind_global # install `global {}` DSL method + bind_registered_cogs # install per-cog config methods + @config_procs.each { |cp| @config_context.instance_eval(&cp) } + @prepared = true +end +``` + +1. **`bind_global`** (lines 124–132): Installs `global` as a singleton method on `@config_context`. The global block runs against `@global_config` via `instance_exec`. +2. **`bind_registered_cogs`** (lines 81–82): For each registered cog type, installs a method with the same name on `@config_context`. +3. **Evaluate config procs**: All `config {}` blocks from the workflow are evaluated sequentially. Multiple config blocks accumulate — they don't replace each other. + +### 2.3 config_for (lines 44–59) — THE MERGE CASCADE + +This is the method called by `ExecutionManager.run!` before each cog runs: + +```ruby +def config_for(cog_class, name = nil) + raise ConfigManagerNotPreparedError unless prepared? + + # Step 1: Global seed — cog-specific config seeded with global values + config = cog_class.config_class.new(@global_config.instance_variable_get(:@values).deep_dup) + + # Step 2: General merge — type-wide defaults + config = config.merge(fetch_general_config(cog_class)) + + # Step 3: Regexp merge — pattern-matched overrides + @regexp_scoped_configs.fetch(cog_class, {}).select do |pattern, _| + pattern.match?(name.to_s) unless name.nil? + end.values.each { |cfg| config = config.merge(cfg) } + + # Step 4: Name merge — cog-specific overrides + unless name.nil? + name_scoped_config = fetch_name_scoped_config(cog_class, name) + config = config.merge(name_scoped_config) + end + + # Step 5: Validate + config.validate! + config +end +``` + +**Step 1 — The Global Config Back-Door**: Uses `instance_variable_get(:@values)` to extract the raw values hash from `@global_config`. This is the *only* place in the codebase that reaches into `@values` from outside `Cog::Config`. Why? Because global config is a base `Cog::Config`, not a cog-specific Config. It may contain keys (like `model` or `temperature`) that only make sense for specific cog types. The cog-specific `config_class.new(values)` accepts them into its own `@values` hash via its constructor. The `deep_dup` prevents mutation of the shared global config. + +**Steps 2–4**: Each `fetch_*` method lazily creates an empty config if none exists (`||=`). This means `config_for` always returns a valid config even for unconfigured cogs. The `merge` method on `Config` does a non-destructive merge: only keys present in the source overwrite keys in the target. + +**Step 5**: Calls `validate!` on the assembled config. Each cog-specific Config class can override `validate!` to enforce constraints (e.g., Map validates that `parallel` is non-negative). + +**Regexp matching note**: All matching patterns are applied in insertion order. If multiple patterns match, their configs are merged sequentially (last wins for any given key). + +### 2.4 on_config (lines 98–122) — Config Dispatch + +```ruby +def on_config(cog_class, cog_name_or_pattern, cog_config_proc) + config_object = case cog_name_or_pattern + when NilClass → fetch_general_config(cog_class) + when Regexp → fetch_regexp_scoped_config(cog_class, pattern) + when Symbol → fetch_name_scoped_config(cog_class, name) + else → raise ArgumentError + end + config_object.instance_exec(&cog_config_proc) if cog_config_proc +end +``` + +The proc runs in the context of the Config object itself. This is why calling `model("gpt-4o")` inside a config block works — it invokes the `model` setter on the Config instance. + +### 2.5 bind_cog (lines 86–96) + +```ruby +def bind_cog(cog_method_name, cog_class) + on_config_method = method(:on_config) + cog_method = proc do |cog_name_or_pattern = nil, &cog_config_proc| + on_config_method.call(cog_class, cog_name_or_pattern, cog_config_proc) + end + @config_context.instance_eval do + raise IllegalCogNameError, cog_method_name if respond_to?(cog_method_name, true) + define_singleton_method(cog_method_name, cog_method) + end +end +``` + +Same metaprogramming pattern as EM's `bind_cog`: capture method reference in closure, define on context. + +### 2.6 Error Hierarchy + +``` +Roast::Error + ConfigManagerError + ConfigManagerNotPreparedError + ConfigManagerAlreadyPreparedError + IllegalCogNameError +``` + +--- + +## 3. ExecutionManager ↔ ConfigManager Interaction + +The EM and CM have a **one-directional dependency**: EM calls CM, CM never calls EM. + +``` +Workflow.prepare! + → ConfigManager.prepare! (evaluates config blocks first) + → ExecutionManager.prepare! (evaluates execute blocks second) + +ExecutionManager.run! + → per cog: @config_manager.config_for(cog.class, cog.name) + → config.deep_dup (isolation before passing to cog) + → cog.run!(..., config, ...) +``` + +**Critical sequencing**: CM must be prepared before EM's execution procs run. The execution procs declare cogs; the config for those cogs must already be collected. However, `config_for` is lazy — it doesn't need the EM's cog list at all. It computes the merged config on-demand from the storage hashes that were populated during CM's `prepare!`. + +**Shared across child EMs**: All child EMs (created by Call, Map, Repeat managers) share the same `@config_manager` instance. Config is workflow-wide. + +--- + +## 4. CogInputManager + +**File**: `lib/roast/cog_input_manager.rb` + +### 4.1 Constructor (lines 19–27) + +```ruby +def initialize(cog_registry, cogs, workflow_context) + @cog_registry = cog_registry + @cogs = cogs # Cog::Store — THIS EM's cogs only + @workflow_context = workflow_context + @context = CogInputContext.new + bind_registered_cogs # immediately installs output accessors + bind_workflow_context # immediately installs target!, kwargs, etc. +end +``` + +Unlike EM and CM, CIM does **not** have a `prepare!` phase. It binds everything in the constructor because cog input blocks can reference any cog in the same scope at any time. + +**Important**: `@cogs` is the Store from *this* EM only. A cog in scope A cannot directly access a cog in scope B through the input context. Cross-scope access requires `from()`. + +### 4.2 The Three Output Accessors + +For each registered cog type (e.g., `:agent`), three methods are installed on `CogInputContext` (lines 40–51): + +| Installed method | Maps to | Behavior | +|-----------------|---------|----------| +| `agent(:name)` | `cog_output(name)` | Tolerant — returns nil on error | +| `agent?(:name)` | `cog_output?(name)` | Boolean — `!cog_output(name).nil?` | +| `agent!(:name)` | `cog_output!(name)` | Strict — raises on error | + +#### `cog_output!` (lines 69–79) — The Strict Path + +```ruby +def cog_output!(cog_name) + raise CogDoesNotExistError, cog_name unless @cogs.key?(cog_name) + + @cogs[cog_name].tap do |cog| + cog.wait # blocks if async task still running + raise CogSkippedError, cog_name if cog.skipped? + raise CogFailedError, cog_name if cog.failed? + raise CogStoppedError, cog_name if cog.stopped? + raise CogNotYetRunError, cog_name unless cog.succeeded? + end.output.deep_dup # ALWAYS deep copies output +end +``` + +**Blocking behavior**: `cog.wait` calls `@task&.wait` on the cog's Async task. If the cog is async and still running, this fiber yields until completion. This is how output access "implicitly awaits" async cogs. + +**State check order**: skipped → failed → stopped → not-yet-run. All four states are checked before output access. + +**Deep dup on access**: `output.deep_dup` ensures each accessor call returns an independent copy. Mutating the returned output never affects the source cog's stored output. + +#### `cog_output` (lines 54–61) — The Tolerant Path + +```ruby +def cog_output(cog_name) + cog_output!(cog_name) +rescue CogOutputAccessError => e + raise e if e.is_a?(CogDoesNotExistError) + nil +end +``` + +Delegates to `cog_output!` but rescues all `CogOutputAccessError` **except** `CogDoesNotExistError`. Design rationale: accessing a nonexistent cog is likely a typo (always an error), but accessing a cog that didn't produce output (skipped, failed, stopped, not yet run) is forgivable. + +#### `cog_output?` (lines 64–66) — The Boolean Path + +```ruby +def cog_output?(cog_name) + !cog_output(cog_name).nil? +end +``` + +Simply checks if the tolerant path returns non-nil. + +### 4.3 Workflow Context Accessors (lines 82–105) + +All bound via `define_singleton_method` on the `CogInputContext`: + +| Method | Implementation | Return type | +|--------|---------------|-------------| +| `target!` | Raises if not exactly 1 target; returns first | `String` | +| `targets` | `@workflow_context.params.targets.dup` | `Array[String]` | +| `arg?(value)` | `params.args.include?(value)` | `bool` | +| `args` | `params.args.dup` | `Array[Symbol]` | +| `kwarg(key)` | `params.kwargs[key]` | `String?` | +| `kwarg!(key)` | Raises if key missing | `String` | +| `kwarg?(key)` | `params.kwargs.include?(key)` | `bool` | +| `kwargs` | `params.kwargs.dup` | `Hash[Symbol, String]` | +| `tmpdir` | `Pathname.new(...).realpath` | `Pathname` | +| `template(path, args)` | 13-candidate resolution + ERB | `String` | + +**Defensive copying**: `targets`, `args`, `kwargs` all return `.dup`. Lighter than `deep_dup` because their contents are simple values (strings, symbols). + +### 4.4 Template Resolution (lines 182–223) — 13-Candidate Priority Stack + +Given `template("greeting", name: "World")`: + +``` +1. Absolute path as-is (if path.absolute?) +2. workflow_dir / path +3. workflow_dir / "#{path}.erb" +4. workflow_dir / "#{path}.md.erb" +5. workflow_dir / "prompts" / path +6. workflow_dir / "prompts" / "#{path}.erb" +7. workflow_dir / "prompts" / "#{path}.md.erb" +8. pwd / path +9. pwd / "#{path}.erb" +10. pwd / "#{path}.md.erb" +11. pwd / "prompts" / path +12. pwd / "prompts" / "#{path}.erb" +13. pwd / "prompts" / "#{path}.md.erb" +``` + +Uses `candidate_paths.find(&:exist?)` — first match wins. Renders with `ERB.new(resolved_path.read).result_with_hash(args)`. + +Raises `CogInputContext::ContextNotFoundError` if no candidate exists. + +**Known bug**: `Pathname` does NOT expand `~` for home directory. Tracked as [issue #663](https://github.com/Shopify/roast/issues/663). + +### 4.5 Error Hierarchy + +``` +Roast::Error + CogOutputAccessError ← NOTE: under Roast::Error, NOT under CogError + CogDoesNotExistError + CogNotYetRunError + CogSkippedError + CogFailedError + CogStoppedError + CogInputContext::ContextNotFoundError ← raised by template() and from() +``` + +**Intentional separation**: `CogOutputAccessError` is a *consumer-side* error (raised when accessing another cog's output). `CogError` is a *producer-side* error (raised within a cog's own lifecycle). They're separate hierarchies under `Roast::Error` because they serve different error handling audiences. + +--- + +## 5. Cog.run! Lifecycle (Detailed) + +**File**: `lib/roast/cog.rb`, lines 71–101 + +### 5.1 Method Signature + +```ruby +def run!(barrier, config, input_context, executor_scope_value, executor_scope_index) +``` + +Receives: +- `barrier` — the EM's `Async::Barrier` for task registration +- `config` — already deep_dup'd by EM +- `input_context` — the `CogInputContext` with all accessors bound +- `executor_scope_value` — already deep_dup'd by EM +- `executor_scope_index` — the iteration index + +### 5.2 Execution Trace + +```ruby +@task = barrier.async(finished: false) do |task| + task.annotate("Cog #{type}(:#{@name})") + TaskContext.begin_cog(self) + @config = config + input_instance = self.class.input_class.new + input_return = input_context.instance_exec( + input_instance, executor_scope_value, executor_scope_index, &@cog_input_proc + ) if @cog_input_proc + coerce_and_validate_input!(input_instance, input_return) + @output = execute(input_instance) +rescue ControlFlow::SkipCog + @skipped = true +rescue ControlFlow::FailCog => e + @failed = true + raise e if config.abort_on_failure? +rescue ControlFlow::Next, ControlFlow::Break => e + @skipped = true + raise e +rescue StandardError => e + @failed = true + raise e +ensure + TaskContext.end +end +``` + +**Step by step**: + +1. **barrier.async(finished: false)**: Creates an async task on the barrier. `finished: false` means the barrier won't consider this task done until it explicitly completes — required for proper barrier cleanup. +2. **TaskContext annotation**: Registers the cog in the fiber-local path for debugging. +3. **Config assignment**: `@config = config` — makes the merged config available to `execute`. +4. **Input creation**: `self.class.input_class.new` — creates a fresh Input instance. +5. **Input block evaluation**: `instance_exec` on the CogInputContext with three args: the input instance, scope_value, and scope_index. The `&@cog_input_proc` is the user's `{ |my, scope_value, index| ... }` block. +6. **Coerce and validate**: `coerce_and_validate_input!(input_instance, input_return)` — the two-phase validation. +7. **Execute**: Calls the cog-specific `execute(input_instance)` method, which returns an Output. + +### 5.3 Two-Phase Input Validation (lines 149–157) + +```ruby +def coerce_and_validate_input!(input, return_value) + input.validate! # Phase 1: optimistic — is input already valid? +rescue Cog::Input::InvalidInputError + input.coerce(return_value) # If not, try coercion from return value + input.validate! # Phase 2: mandatory — must be valid now +end +``` + +**Design**: Allows users to either set input fields explicitly (`my.prompt = "..."`) or rely on implicit coercion from the block's return value. The two-phase approach means explicit setting is checked first (fast path), coercion is only attempted if needed. + +### 5.4 Exception Handling Matrix + +| Exception | `@skipped` | `@failed` | Propagates? | Condition | +|-----------|-----------|----------|------------|-----------| +| `ControlFlow::SkipCog` | `true` | – | No | Always swallowed | +| `ControlFlow::FailCog` | – | `true` | Conditional | Only if `config.abort_on_failure?` (default: `true`) | +| `ControlFlow::Next` | `true` | – | Yes | Always re-raised | +| `ControlFlow::Break` | `true` | – | Yes | Always re-raised | +| `StandardError` | – | `true` | Yes | Always re-raised | + +### 5.5 State Queries + +| Method | Implementation | Notes | +|--------|---------------|-------| +| `started?` | `@task.present?` | True once `run!` is called | +| `skipped?` | `@skipped` | Set by SkipCog, Next, Break | +| `failed?` | `@failed \|\| !!@task&.failed?` | Explicit flag OR Async task failure | +| `stopped?` | `!!@task&.stopped?` | Async task was externally killed (barrier.stop) | +| `succeeded?` | `@output != nil && @task&.finished?` | The `!= nil` check is intentional — see below | + +**The `succeeded?` gotcha**: Uses `@output != nil` (explicit nil check, not `.present?`) because the Ruby cog's Output class delegates `method_missing` to its `.value`. If the value is `nil`, `.present?` would return `false` even though the Output *object* exists. The `!= nil` check tests the object reference itself. + +### 5.6 The `wait` Bare Rescue (lines 105–109) + +```ruby +def wait + @task&.wait +rescue + # Do nothing +end +``` + +Used by `CogInputManager#cog_output!` to ensure a cog's task is complete before accessing its output. The bare rescue swallows *all* exceptions (including Next, Break). This is safe because: +1. Exceptions from the cog's task have already been propagated through the barrier system. +2. The purpose of `wait` here is only to ensure completion, not to handle errors. +3. The actual error checking happens via the state queries (`skipped?`, `failed?`, etc.) in `cog_output!`. + +--- + +## 6. Registry, Store, Stack + +### 6.1 Cog::Registry (`lib/roast/cog/registry.rb`) + +```ruby +class Registry + def initialize + @cogs = {} + use(SystemCogs::Call) + use(SystemCogs::Map) + use(SystemCogs::Repeat) + use(Cogs::Cmd) + use(Cogs::Chat) + use(Cogs::Agent) + use(Cogs::Ruby) + end + + def use(cog_class) + name, klass = create_registration(cog_class) + cogs[name] = klass + end + + private + + def create_registration(cog_class) + cog_class_name = cog_class.name + raise CouldNotDeriveCogNameError if cog_class_name.nil? + [cog_class_name.demodulize.underscore.to_sym, cog_class] + end +end +``` + +**Auto-registration**: All 7 built-in cogs are registered at construction time. Registration derives the method name from the class name: `Roast::Cogs::Agent` → `:agent`, `Roast::SystemCogs::Map` → `:map`. + +**Custom cogs**: Use `registry.use(MyCustomCog)` to add new cog types. The custom cog's class name determines its DSL method name. + +**One per workflow**: A single Registry instance is created by `Workflow` and shared across all managers. + +### 6.2 Cog::Store (`lib/roast/cog/store.rb`) + +```ruby +class Store + delegate :[], :key?, to: :store + + def initialize + @store = {} + end + + def insert(cog) + raise CogAlreadyDefinedError, cog.name if store.key?(cog.name) + store[cog.name] = cog + end +end +``` + +**Uniqueness constraint**: Cannot insert two cogs with the same name. This is per-EM, not global. Two different scopes can have cogs with the same name because they have separate EMs with separate Stores. + +**Lookup**: `store[name]` for direct access, `store.key?(name)` for existence check. + +### 6.3 Cog::Stack (`lib/roast/cog/stack.rb`) + +```ruby +class Stack + delegate :each, :empty?, :last, :map, :push, :size, to: :@queue + + def initialize + @queue = [] + end + + def pop + @queue.shift # FIFO — pops from front + end +end +``` + +**FIFO ordering**: Cogs execute in the order they were declared. `push` appends to the end, `pop` (shift) removes from the front. + +**Note**: `run!` uses `each` (not `pop`) to iterate the stack. The `pop` method exists but is not used in the current execution path — it's for potential future use or external consumers. + +**`last`**: Used by `compute_final_output` to determine which cog's output is the default final output when no `outputs` block is declared. + +--- + +## 7. TaskContext — Fiber-Local Path Tracking + +**File**: `lib/roast/task_context.rb` + +```ruby +module TaskContext + extend self + + class PathElement + attr_reader :cog, :execution_manager + end + + def path + Fiber[:path]&.deep_dup || [] + end + + def begin_cog(cog) + begin_element(PathElement.new(cog:)) + end + + def begin_execution_manager(execution_manager) + begin_element(PathElement.new(execution_manager:)) + end + + def end + Event << { end: Fiber[:path]&.last } + el = Fiber[:path]&.pop + [el, path] + end + + private + + def begin_element(element) + Fiber[:path] = (Fiber[:path] || []) + [element] + Event << { begin: element } + path + end +end +``` + +**Fiber-local storage**: Uses `Fiber[:path]` (Ruby 3.2+ fiber storage) to maintain a per-fiber execution path. Each fiber has its own path — no cross-fiber contamination. + +**Path structure**: Array of `PathElement` objects, alternating between EMs and Cogs. The path grows as execution descends into scopes and cogs, and shrinks as they complete. + +**Event emission**: Both `begin_element` and `end` emit events via `Event <<`. This feeds the EventMonitor for logging and debugging. + +**Deep dup on read**: `path` returns `deep_dup` to prevent external mutation of the fiber-local state. + +--- + +## 8. WorkflowContext — Immutable Shared State + +**File**: `lib/roast/workflow_context.rb` + +```ruby +class WorkflowContext + attr_reader :params # WorkflowParams — targets, args, kwargs + attr_reader :tmpdir # String — Dir.mktmpdir path + attr_reader :workflow_dir # Pathname — directory containing the workflow file + + def initialize(params:, tmpdir:, workflow_dir:) + @params = params + @tmpdir = tmpdir + @workflow_dir = workflow_dir + end +end +``` + +**Effectively immutable**: Created once during `Workflow.from_file`, shared across all managers and child EMs. Contains: +- `params` — parsed CLI arguments (targets, positional args, keyword args) +- `tmpdir` — a unique temporary directory created for this workflow run, cleaned up after +- `workflow_dir` — the directory of the workflow `.rb` file, used for template resolution + +--- + +## 9. Complete Interaction Diagram + +``` +Workflow.from_file + │ + ├─ creates WorkflowContext (params, tmpdir, workflow_dir) + ├─ creates Cog::Registry (7 built-in cogs) + ├─ creates ConfigManager (registry, config_procs) + ├─ creates ExecutionManager (registry, CM, execution_procs, WC) + │ + ├─ prepare! + │ ├─ CM.prepare! + │ │ ├─ bind_global → ConfigContext gets `global` method + │ │ ├─ bind_registered_cogs → ConfigContext gets `agent`, `cmd`, ... methods + │ │ └─ evaluate config procs → on_config fills @general/@regexp/@name_scoped_configs + │ │ + │ └─ EM.prepare! + │ ├─ bind_outputs → ExecutionContext gets `outputs`, `outputs!` methods + │ ├─ bind_registered_cogs → ExecutionContext gets `agent`, `cmd`, ... methods + │ └─ evaluate execution procs → on_execute creates Cog instances → Store + Stack + │ └─ CogInputManager created in EM constructor + │ ├─ bind_registered_cogs → CogInputContext gets `agent`/`agent!`/`agent?` triplets + │ └─ bind_workflow_context → CogInputContext gets target!, kwargs, template, etc. + │ + └─ start! + └─ EM.run! + └─ for each cog in stack: + ├─ CM.config_for(cog.class, cog.name) → 5-step merge cascade + ├─ config.deep_dup + ├─ @scope_value.deep_dup + └─ cog.run!(barrier, config, CogInputContext, scope_value, scope_index) + ├─ input_context.instance_exec(input, sv, si, &input_proc) + ├─ coerce_and_validate_input! + └─ execute(input) → Output +``` + +--- + +## 10. Invariants for Contributors + +These invariants must be maintained when modifying the execution engine: + +1. **Deep dup at every boundary**: If you create a new boundary where data crosses from one cog/scope to another, you MUST `deep_dup` at that boundary. + +2. **Idempotent compute_final_output**: The flag pattern must be preserved. Calling it multiple times is safe and expected. + +3. **Config is resolved per-cog-invocation, not cached**: `config_for` is called fresh for each cog in the stack. Don't cache configs across cogs. + +4. **Execution procs are shared, not copied**: `@all_execution_procs` is the same Hash everywhere. Never mutate it after workflow construction. + +5. **CIM binds immediately**: Unlike EM/CM, the CogInputManager binds all methods in its constructor. New accessor types must be bound there. + +6. **Name conflict checks include private methods**: `respond_to?(name, true)` — don't forget the `true`. + +7. **System cog dispatch is intentionally not polymorphic**: Adding a new system cog requires adding a new `elsif` branch in `on_execute` AND a new Manager module inclusion at the top of EM. + +8. **Break always propagates; Next propagates only from sync**: This is the fundamental concurrency contract. Any change here affects all workflow control flow. diff --git a/internal/documentation/architecture/06-metaprogramming-map.md b/internal/documentation/architecture/06-metaprogramming-map.md new file mode 100644 index 00000000..7fe394d1 --- /dev/null +++ b/internal/documentation/architecture/06-metaprogramming-map.md @@ -0,0 +1,597 @@ +# Document 6: Metaprogramming Map + +> **Audience**: AI coding agents navigating the Roast codebase. +> **Purpose**: Map every dynamically-defined method to its installation site, its runtime behaviour, and the manager that creates it. + +--- + +## 1. The Core Pattern + +Roast's DSL is powered by a single metaprogramming pattern repeated across three manager classes. Each manager: + +1. Captures a reference to its own dispatch method via `method(:on_xxx)`. +2. Wraps it in a `proc` with the appropriate parameter signature. +3. Installs the proc on a context instance via `define_singleton_method`. + +The result: three "blank" context classes (`ConfigContext`, `ExecutionContext`, `CogInputContext`) acquire methods at runtime that route to completely different behavior depending on which manager installed them. + +### Source of the "blank" classes + +| Class | Source | Hardcoded content | +|-------|--------|-------------------| +| `ConfigContext` | `lib/roast/config_context.rb:6` | Empty class body | +| `ExecutionContext` | `lib/roast/execution_context.rb:6` | Empty class body | +| `CogInputContext` | `lib/roast/cog_input_context.rb:6–33` | 4 control flow methods + 2 module includes | + +--- + +## 2. ConfigContext — Methods Installed by ConfigManager + +**Installer**: `ConfigManager` (`lib/roast/config_manager.rb`) + +### 2.1 `global` method + +| Aspect | Detail | +|--------|--------| +| **Installed by** | `ConfigManager#bind_global` (line 124) | +| **Signature** | `global(&block)` | +| **Dispatch target** | `ConfigManager#on_global` (line 135) | +| **Behavior** | Evaluates `block` via `instance_exec` on `@global_config` (a `Cog::Config` instance) | +| **Available within** | `config { }` blocks only | + +**Installation code** (lines 124–132): +```ruby +def bind_global + on_global_method = method(:on_global) + method_to_bind = proc do |&global_proc| + on_global_method.call(global_proc) + end + @config_context.instance_eval do + define_singleton_method(:global, method_to_bind) + end +end +``` + +### 2.2 Per-cog-type methods (× 7 cog types) + +| Aspect | Detail | +|--------|--------| +| **Installed by** | `ConfigManager#bind_cog` (line 86) | +| **Method names** | `:agent`, `:chat`, `:cmd`, `:ruby`, `:call`, `:map`, `:repeat` | +| **Signature** | `cog_type(name_or_pattern = nil, &block)` | +| **Dispatch target** | `ConfigManager#on_config` (line 99) | +| **Dispatch routing** | `nil` → general config; `Regexp` → regexp-scoped config; `Symbol` → name-scoped config | +| **Block evaluation** | `config_object.instance_exec(&cog_config_proc)` | +| **Collision guard** | `respond_to?(cog_method_name, true)` → raises `IllegalCogNameError` (line 92) | + +**Installation code** (lines 86–96): +```ruby +def bind_cog(cog_method_name, cog_class) + on_config_method = method(:on_config) + cog_method = proc do |cog_name_or_pattern = nil, &cog_config_proc| + on_config_method.call(cog_class, cog_name_or_pattern, cog_config_proc) + end + @config_context.instance_eval do + raise IllegalCogNameError, cog_method_name if respond_to?(cog_method_name, true) + define_singleton_method(cog_method_name, cog_method) + end +end +``` + +### 2.3 Complete ConfigContext method table + +| Method | Installer | Dispatch | Lines | +|--------|-----------|----------|-------| +| `global` | `bind_global` | `on_global` | CM:124–132 | +| `agent` | `bind_cog` | `on_config(Cogs::Agent, ...)` | CM:86–96 | +| `chat` | `bind_cog` | `on_config(Cogs::Chat, ...)` | CM:86–96 | +| `cmd` | `bind_cog` | `on_config(Cogs::Cmd, ...)` | CM:86–96 | +| `ruby` | `bind_cog` | `on_config(Cogs::Ruby, ...)` | CM:86–96 | +| `call` | `bind_cog` | `on_config(SystemCogs::Call, ...)` | CM:86–96 | +| `map` | `bind_cog` | `on_config(SystemCogs::Map, ...)` | CM:86–96 | +| `repeat` | `bind_cog` | `on_config(SystemCogs::Repeat, ...)` | CM:86–96 | + +**Total**: 8 methods dynamically installed on each `ConfigContext` instance. + +--- + +## 3. ExecutionContext — Methods Installed by ExecutionManager + +**Installer**: `ExecutionManager` (`lib/roast/execution_manager.rb`) + +### 3.1 `outputs` and `outputs!` methods + +| Aspect | Detail | +|--------|--------| +| **Installed by** | `ExecutionManager#bind_outputs` (line 228) | +| **Signatures** | `outputs(&block)`, `outputs!(&block)` | +| **Dispatch targets** | `on_outputs` (line 240), `on_outputs!` (line 247) | +| **Behavior** | Stores the proc for later evaluation in `compute_final_output` | +| **Mutual exclusion** | Both raise `OutputsAlreadyDefinedError` if either is already set (line 241/249) | + +**Installation code** (lines 228–237): +```ruby +def bind_outputs + on_outputs_method = method(:on_outputs) + on_outputs_bang_method = method(:on_outputs!) + method_to_bind = proc { |&outputs_proc| on_outputs_method.call(outputs_proc) } + bang_method_to_bind = proc { |&outputs_proc| on_outputs_bang_method.call(outputs_proc) } + @execution_context.instance_eval do + define_singleton_method(:outputs, method_to_bind) + define_singleton_method(:outputs!, bang_method_to_bind) + end +end +``` + +### 3.2 Per-cog-type methods (× 7 cog types) + +| Aspect | Detail | +|--------|--------| +| **Installed by** | `ExecutionManager#bind_cog` (line 187) | +| **Method names** | `:agent`, `:chat`, `:cmd`, `:ruby`, `:call`, `:map`, `:repeat` | +| **Signature** | `cog_type(*args, **kwargs, &cog_input_proc)` | +| **Dispatch target** | `ExecutionManager#on_execute` (line 200) | +| **Collision guard** | `respond_to?(cog_method_name, true)` → raises `IllegalCogNameError` (line 193) | + +**Installation code** (lines 187–197): +```ruby +def bind_cog(cog_method_name, cog_class) + on_execute_method = method(:on_execute) + cog_method = proc do |*args, **kwargs, &cog_input_proc| + on_execute_method.call(cog_class, args, kwargs, cog_input_proc) + end + @execution_context.instance_eval do + raise IllegalCogNameError, cog_method_name if respond_to?(cog_method_name, true) + define_singleton_method(cog_method_name, cog_method) + end +end +``` + +### 3.3 on_execute dispatch routing (lines 200–226) + +```ruby +def on_execute(cog_class, cog_args, cog_kwargs, cog_input_proc) + if cog_class <= SystemCog + cog_params = cog_class.params_class.new(*cog_args, **cog_kwargs) + cog_instance = if cog_class == SystemCogs::Call + create_call_system_cog(cog_params, cog_input_proc) + elsif cog_class == SystemCogs::Map + create_map_system_cog(cog_params, cog_input_proc) + elsif cog_class == SystemCogs::Repeat + create_repeat_system_cog(cog_params, cog_input_proc) + else + raise NotImplementedError + end + else + cog_name = Array.wrap(cog_args).shift + anonymous = !cog_name + cog_name ||= Cog.generate_fallback_name + cog_instance = cog_class.new(cog_name, cog_input_proc, anonymous:) + end + add_cog_instance(cog_instance) +end +``` + +**Critical design note**: System cog dispatch is hardcoded `if/elsif`, NOT polymorphic. This is intentional — Manager modules need access to EM internals (`@cog_registry`, `@config_manager`, `@all_execution_procs`, `@workflow_context`) to create child EMs. + +### 3.4 Complete ExecutionContext method table + +| Method | Installer | Dispatch | Lines | +|--------|-----------|----------|-------| +| `outputs` | `bind_outputs` | `on_outputs` | EM:228–237 | +| `outputs!` | `bind_outputs` | `on_outputs!` | EM:228–237 | +| `agent` | `bind_cog` | `on_execute(Cogs::Agent, ...)` | EM:187–197 | +| `chat` | `bind_cog` | `on_execute(Cogs::Chat, ...)` | EM:187–197 | +| `cmd` | `bind_cog` | `on_execute(Cogs::Cmd, ...)` | EM:187–197 | +| `ruby` | `bind_cog` | `on_execute(Cogs::Ruby, ...)` | EM:187–197 | +| `call` | `bind_cog` | `on_execute(SystemCogs::Call, ...)` | EM:187–197 | +| `map` | `bind_cog` | `on_execute(SystemCogs::Map, ...)` | EM:187–197 | +| `repeat` | `bind_cog` | `on_execute(SystemCogs::Repeat, ...)` | EM:187–197 | + +**Total**: 9 methods dynamically installed on each `ExecutionContext` instance. + +--- + +## 4. CogInputContext — Methods Installed by CogInputManager + +**Installer**: `CogInputManager` (`lib/roast/cog_input_manager.rb`) + +### 4.1 Per-cog-type triplets (× 7 cog types = 21 methods) + +| Aspect | Detail | +|--------|--------| +| **Installed by** | `CogInputManager#bind_cog` (line 40) | +| **Method names per type** | `cog_type(:name)`, `cog_type!(:name)`, `cog_type?(:name)` | +| **Dispatch targets** | `cog_output` (line 54), `cog_output!` (line 69), `cog_output?` (line 64) | + +**Installation code** (lines 40–51): +```ruby +def bind_cog(cog_method_name) + cog_question_method_name = (cog_method_name.to_s + "?").to_sym + cog_bang_method_name = (cog_method_name.to_s + "!").to_sym + cog_output_method = method(:cog_output) + cog_output_question_method = method(:cog_output?) + cog_output_bang_method = method(:cog_output!) + @context.instance_eval do + define_singleton_method(cog_method_name, proc { |cog_name| cog_output_method.call(cog_name) }) + define_singleton_method(cog_question_method_name, proc { |cog_name| cog_output_question_method.call(cog_name) }) + define_singleton_method(cog_bang_method_name, proc { |cog_name| cog_output_bang_method.call(cog_name) }) + end +end +``` + +**Behavior of each accessor variant**: + +| Variant | Method | On success | On cog not found | On cog skipped/failed/stopped/not-run | +|---------|--------|------------|------------------|---------------------------------------| +| Tolerant | `cog_output` | `output.deep_dup` | Raises `CogDoesNotExistError` | Returns `nil` | +| Strict | `cog_output!` | `output.deep_dup` | Raises `CogDoesNotExistError` | Raises specific error | +| Boolean | `cog_output?` | `true` | Raises `CogDoesNotExistError` | `false` | + +**Blocking behavior**: All three call `cog.wait` (line 73) before state checks, which blocks the fiber if the cog is async and still running. + +### 4.2 Workflow context accessors (10 methods) + +| Aspect | Detail | +|--------|--------| +| **Installed by** | `CogInputManager#bind_workflow_context` (line 82) | +| **Timing** | Installed immediately in the constructor (line 26) | + +**Installation code** (lines 82–104): +```ruby +def bind_workflow_context + target_bang_method = method(:target!) + targets_method = method(:targets) + arg_question_method = method(:arg?) + args_method = method(:args) + kwarg_method = method(:kwarg) + kwarg_bang_method = method(:kwarg!) + kwarg_question_method = method(:kwarg?) + kwargs_method = method(:kwargs) + tmpdir_method = method(:tmpdir) + template_method = method(:template) + @context.instance_eval do + define_singleton_method(:target!, proc { target_bang_method.call }) + define_singleton_method(:targets, proc { targets_method.call }) + define_singleton_method(:arg?, proc { |value| arg_question_method.call(value) }) + define_singleton_method(:args, proc { args_method.call }) + define_singleton_method(:kwarg, proc { |key| kwarg_method.call(key) }) + define_singleton_method(:kwarg!, proc { |key| kwarg_bang_method.call(key) }) + define_singleton_method(:kwarg?, proc { |key| kwarg_question_method.call(key) }) + define_singleton_method(:kwargs, proc { kwargs_method.call }) + define_singleton_method(:tmpdir, proc { tmpdir_method.call }) + define_singleton_method(:template, proc { |path, args = {}| template_method.call(path, args) }) + end +end +``` + +### 4.3 Hardcoded control flow methods (NOT dynamically defined) + +These are defined directly in `CogInputContext` class body (`lib/roast/cog_input_context.rb:15–30`): + +| Method | Raises | Line | +|--------|--------|------| +| `skip!(message = nil)` | `ControlFlow::SkipCog` | 15 | +| `fail!(message = nil)` | `ControlFlow::FailCog` | 20 | +| `next!(message = nil)` | `ControlFlow::Next` | 25 | +| `break!(message = nil)` | `ControlFlow::Break` | 30 | + +### 4.4 Module-included methods (NOT dynamically defined) + +**From `SystemCogs::Call::InputContext`** (`lib/roast/system_cogs/call.rb:119–158`): + +| Method | Signature | Line | +|--------|-----------|------| +| `from` | `(call_cog_output, &block)` | 147 | + +**From `SystemCogs::Map::InputContext`** (`lib/roast/system_cogs/map.rb:342–448`): + +| Method | Signature | Line | +|--------|-----------|------| +| `collect` | `(map_cog_output, &block)` | 375 | +| `reduce` | `(map_cog_output, initial_value = nil, &block)` | 426 | + +### 4.5 Complete CogInputContext method table + +| Method | Source | Type | Lines | +|--------|--------|------|-------| +| `agent(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `agent!(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `agent?(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `chat(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `chat!(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `chat?(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `cmd(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `cmd!(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `cmd?(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `ruby(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `ruby!(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `ruby?(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `call(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `call!(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `call?(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `map(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `map!(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `map?(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `repeat(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `repeat!(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `repeat?(:name)` | `bind_cog` | Dynamic | CIM:40–51 | +| `target!` | `bind_workflow_context` | Dynamic | CIM:82–104 | +| `targets` | `bind_workflow_context` | Dynamic | CIM:82–104 | +| `arg?(:value)` | `bind_workflow_context` | Dynamic | CIM:82–104 | +| `args` | `bind_workflow_context` | Dynamic | CIM:82–104 | +| `kwarg(:key)` | `bind_workflow_context` | Dynamic | CIM:82–104 | +| `kwarg!(:key)` | `bind_workflow_context` | Dynamic | CIM:82–104 | +| `kwarg?(:key)` | `bind_workflow_context` | Dynamic | CIM:82–104 | +| `kwargs` | `bind_workflow_context` | Dynamic | CIM:82–104 | +| `tmpdir` | `bind_workflow_context` | Dynamic | CIM:82–104 | +| `template(path, args)` | `bind_workflow_context` | Dynamic | CIM:82–104 | +| `skip!(msg)` | Class body | Hardcoded | CIC:16 | +| `fail!(msg)` | Class body | Hardcoded | CIC:20 | +| `next!(msg)` | Class body | Hardcoded | CIC:25 | +| `break!(msg)` | Class body | Hardcoded | CIC:30 | +| `from(output, &blk)` | `Call::InputContext` module | Included | call.rb:147 | +| `collect(output, &blk)` | `Map::InputContext` module | Included | map.rb:375 | +| `reduce(output, init, &blk)` | `Map::InputContext` module | Included | map.rb:426 | + +**Total**: 37 methods available on each `CogInputContext` instance (21 dynamic triplets + 10 dynamic workflow accessors + 4 hardcoded control flow + 2 module includes from `Map::InputContext` + 1 module include from `Call::InputContext`). + +--- + +## 5. The Same Method Name, Three Behaviors + +This is the most confusing aspect of the codebase. The method `agent` (for example) exists on all three contexts but does completely different things: + +| Context | Call site | What happens | +|---------|-----------|--------------| +| `ConfigContext` | `config { agent(:name) { temperature 0.5 } }` | Fetches/creates a `Cog::Config` for `Cogs::Agent` scoped to `:name`, evaluates block on it | +| `ExecutionContext` | `execute { agent(:name) { "prompt" } }` | Creates a `Cogs::Agent` instance, saves the block as `@cog_input_proc`, pushes to stack | +| `CogInputContext` | `chat(:x) { agent(:name).response }` | Looks up the already-run `agent(:name)` cog, waits if async, returns `output.deep_dup` | + +The method name is shared because each context's methods are installed from the same cog registry iteration. But the *implementation* comes from entirely different managers. + +### Disambiguation rule for AI agents + +When you see `agent(:name)` (or any cog method) in source: +1. **Is the call site inside a `config { }` block?** → It routes to `ConfigManager#on_config`. +2. **Is it inside an `execute { }` block at the top level (not inside a cog's `{ }` block)?** → It routes to `ExecutionManager#on_execute`. +3. **Is it inside a cog's input block (the `{ }` passed to a cog declaration)?** → It routes to `CogInputManager#cog_output`. + +The trick: (2) and (3) are syntactically indistinguishable without understanding the nesting level. The `execute { }` block is evaluated on `ExecutionContext`, but the inner blocks (e.g., `agent(:x) { ... }`) are saved as procs and later evaluated on `CogInputContext` via `instance_exec`. + +--- + +## 6. Custom Cog Registration + +When a workflow calls `use :my_custom_cog`, the following chain executes: + +``` +Workflow#use (lib/roast/workflow.rb:105–126) + ├─ require path: @workflow_path.realdirpath.dirname.join("cogs/my_custom_cog") + ├─ Resolve class: "my_custom_cog".camelize.constantize → MyCustomCog + ├─ Validate: MyCustomCog < Roast::Cog + └─ @cog_registry.use(MyCustomCog) + └─ Registry#use (lib/roast/cog/registry.rb:53) + └─ create_registration: "MyCustomCog".demodulize.underscore.to_sym → :my_custom_cog + └─ @cogs[:my_custom_cog] = MyCustomCog +``` + +Once registered, the cog is treated identically to built-in cogs during `prepare!`: + +- `ConfigManager#bind_registered_cogs` → installs `my_custom_cog` on `ConfigContext` +- `ExecutionManager#bind_registered_cogs` → installs `my_custom_cog` on `ExecutionContext` +- `CogInputManager#bind_registered_cogs` → installs `my_custom_cog`, `my_custom_cog!`, `my_custom_cog?` on `CogInputContext` + +### Gem-based cogs + +```ruby +use :my_cog, from: "roast-my_cog" +``` + +This calls `require "roast-my_cog"` (the gem handles its own autoloading), then resolves and registers the class identically to local cogs. + +### Name derivation algorithm + +```ruby +cog_class.name.demodulize.underscore.to_sym +``` + +| Class | `.name` | `.demodulize` | `.underscore` | Final symbol | +|-------|---------|---------------|---------------|--------------| +| `Roast::Cogs::Agent` | `"Roast::Cogs::Agent"` | `"Agent"` | `"agent"` | `:agent` | +| `Roast::SystemCogs::Call` | `"Roast::SystemCogs::Call"` | `"Call"` | `"call"` | `:call` | +| `MyCustomCog` | `"MyCustomCog"` | `"MyCustomCog"` | `"my_custom_cog"` | `:my_custom_cog` | +| `Analyzers::CodeReview` | `"Analyzers::CodeReview"` | `"CodeReview"` | `"code_review"` | `:code_review` | + +**Important**: Only the final class name component matters. The module namespace is stripped. + +--- + +## 7. Name Collision Guards + +Both `ConfigManager#bind_cog` (line 92) and `ExecutionManager#bind_cog` (line 193) check: + +```ruby +raise IllegalCogNameError, cog_method_name if respond_to?(cog_method_name, true) +``` + +The `true` argument includes private methods in the check. This prevents custom cogs from shadowing: +- Built-in cog names (`:agent`, `:chat`, `:cmd`, `:ruby`, `:call`, `:map`, `:repeat`) +- Ruby `Object` methods (`:freeze`, `:class`, `:send`, `:object_id`, etc.) +- Previously registered custom cogs + +The check happens at **prepare time**, not at registration time. This means the error surfaces during `Workflow#prepare!` when `ConfigManager.prepare!` and `ExecutionManager.prepare!` iterate the registry. + +`CogInputManager#bind_cog` (line 40) does **not** perform this check — it relies on the upstream managers to have caught collisions first. + +--- + +## 8. Method Dispatch Through `instance_exec` and `instance_eval` + +### Where each context's methods are invoked + +| Context | Evaluation mechanism | Code location | +|---------|---------------------|---------------| +| `ConfigContext` | `@config_context.instance_eval(&config_proc)` | CM:29 | +| `ExecutionContext` | `@execution_context.instance_eval(&execution_proc)` | EM:83 | +| `CogInputContext` | `input_context.instance_exec(input, scope_value, scope_index, &cog_input_proc)` | Cog:79–81 | +| `CogInputContext` | `@cog_input_manager.context.instance_exec(@scope_value, @scope_index, &outputs_proc)` | EM:261 | +| `CogInputContext` | `em.cog_input_context.instance_exec(final_output, scope_value, scope_index, &block)` | call.rb:154 | +| `CogInputContext` | `em.cog_input_context.instance_exec(final_output, scope_value, scope_index, &block)` | map.rb:385 | +| `CogInputContext` | `em.cog_input_context.instance_exec(accumulator, final_output, scope_value, scope_index, &block)` | map.rb:437 | + +**Key distinction**: +- `instance_eval` (ConfigContext, ExecutionContext): Block receives no arguments; `self` is the context. +- `instance_exec` (CogInputContext): Block receives explicit arguments AND `self` is the context. This is why cog input blocks receive `|my, scope_value, scope_index|` as parameters while simultaneously having access to output accessors like `cmd!(:name)`. + +--- + +## 9. The `instance_variable_get` Back-Door Pattern + +Three locations in the codebase reach into objects via `instance_variable_get` rather than public accessors. AI agents must understand these are intentional, not code smells: + +| Location | Target | Ivar accessed | Why | +|----------|--------|---------------|-----| +| `ConfigManager#config_for` (line 48) | `@global_config` | `@values` | Global config may contain keys that don't belong to the cog-specific Config subclass. Using the public accessor would validate/reject unknown keys. | +| `Call::InputContext#from` (line 148) | `call_cog_output` | `@execution_manager` | Output objects are opaque to consumers. The `from` method needs EM access to retrieve `final_output` and `cog_input_context`. | +| `Map::InputContext#collect` (line 376) | `map_cog_output` | `@execution_managers` | Same pattern as above — the array of EMs is internal to the Output. | +| `Map::InputContext#reduce` (line 427) | `map_cog_output` | `@execution_managers` | Same as collect. | +| `Call::InputContext#from` (line 152) | `em` | `@scope_value` | Scope value is accessible via `attr_reader :scope_value`, but `@scope_index` is also an attr_reader. However, `instance_variable_get` is used for `@scope_value` to apply `.deep_dup` in the same call chain. | +| `Call::InputContext#from` (line 153) | `em` | `@scope_index` | To pass it to the block. | + +--- + +## 10. RBI Shim Files — Canonical Documentation + +The three RBI shim files serve as **the authoritative API reference** for dynamically-defined methods. They provide: +1. Sorbet type annotations for IDE integration +2. Extensive RDoc-style comments with usage examples +3. Type signatures for method parameters and return values + +| Shim file | Documenting | Total lines | +|-----------|-------------|-------------| +| `sorbet/rbi/shims/lib/roast/config_context.rbi` | 8 ConfigContext methods | 323 lines | +| `sorbet/rbi/shims/lib/roast/execution_context.rbi` | 9 ExecutionContext methods | 496 lines | +| `sorbet/rbi/shims/lib/roast/cog_input_context.rbi` | 37 CogInputContext methods | 1,198 lines | + +**Usage for AI agents**: When you need to understand what a DSL method does, look up its entry in the corresponding RBI file first. The comments there are written for human readers and are more comprehensive than the implementation code. + +--- + +## 11. SystemCog Manager Modules — Mixed Into ExecutionManager + +System cogs use `Manager` modules mixed into `ExecutionManager` to access its private state: + +| Module | Source | Mixed in at | Provides | +|--------|--------|-------------|----------| +| `SystemCogs::Call::Manager` | `lib/roast/system_cogs/call.rb:87–116` | EM line 7 | `create_call_system_cog` | +| `SystemCogs::Map::Manager` | `lib/roast/system_cogs/map.rb:255–338` | EM line 8 | `create_map_system_cog`, `create_execution_manager_for_map_item`, `execute_map_in_series`, `execute_map_in_parallel` | +| `SystemCogs::Repeat::Manager` | `lib/roast/system_cogs/repeat.rb` | EM line 9 | `create_repeat_system_cog` | + +These modules freely access EM instance variables (`@cog_registry`, `@config_manager`, `@all_execution_procs`, `@workflow_context`) because they execute in the EM's context after being included. + +### InputContext modules — Mixed into CogInputContext + +| Module | Source | Mixed in at | Provides | +|--------|--------|-------------|----------| +| `SystemCogs::Call::InputContext` | `lib/roast/system_cogs/call.rb:118–158` | CIC line 7 | `from` | +| `SystemCogs::Map::InputContext` | `lib/roast/system_cogs/map.rb:341–449` | CIC line 8 | `collect`, `reduce` | + +**Note**: There is no `Repeat::InputContext` module. Repeat cogs use `Map::Output` for their `.results` field, which enables reuse of `collect` and `reduce` without a separate module. + +--- + +## 12. The `Cog::Config` Field Macro + +The `field` macro (`lib/roast/cog/config.rb`) generates getter/setter/bang method triplets: + +```ruby +# Simplified from source: +def self.field(name, default: nil) + define_method(name) do |value = :__unset__| + if value == :__unset__ + @values[name] || default # ← NOTE: the || fallback + else + @values[name] = value + end + end + define_method(:"#{name}?") { !!(@values[name] || default) } + # ... bang and no_ methods +end +``` + +**Critical pitfall**: The `||` fallback means `false` and `nil` stored in `@values[name]` will be treated as "not set" and the default will be returned instead. This affects boolean fields — see the `abort_on_failure?` workaround which uses `@values.fetch(:abort_on_failure, true)` instead of the field macro pattern. + +--- + +## 13. Quick-Reference: "Where Does This Method Come From?" + +### Lookup algorithm for AI agents + +Given a method call `xyz(...)` in Roast DSL code: + +``` +1. Is it inside `config { ... }`? + └─ YES → ConfigContext method, installed by ConfigManager#bind_cog or bind_global + └─ NO → continue + +2. Is it at the TOP level of `execute { ... }` (not inside a cog block)? + └─ YES → ExecutionContext method, installed by ExecutionManager#bind_cog or bind_outputs + └─ NO → continue + +3. Is it inside a cog input block (agent(:x) { HERE }) or an outputs block? + └─ YES → CogInputContext method + ├─ Is it skip!/fail!/next!/break!? → Hardcoded in CogInputContext class body + ├─ Is it from/collect/reduce? → Module included on CogInputContext + ├─ Is it target!/targets/args/kwargs/tmpdir/template? → Dynamic via bind_workflow_context + └─ Is it a cog_type name? → Dynamic triplet via bind_cog +``` + +### File lookup table + +| If you see... | Look in... | +|---------------|-----------| +| `config { agent { ... } }` | `lib/roast/config_manager.rb` → `on_config` | +| `execute { agent(:x) { } }` | `lib/roast/execution_manager.rb` → `on_execute` | +| `chat(:x) { agent!(:y).response }` | `lib/roast/cog_input_manager.rb` → `cog_output!` | +| `config { global { } }` | `lib/roast/config_manager.rb` → `on_global` | +| `execute { outputs! { } }` | `lib/roast/execution_manager.rb` → `on_outputs!` | +| `from(call!(:x))` | `lib/roast/system_cogs/call.rb` → `InputContext#from` | +| `collect(map!(:x))` | `lib/roast/system_cogs/map.rb` → `InputContext#collect` | +| `reduce(map!(:x), 0)` | `lib/roast/system_cogs/map.rb` → `InputContext#reduce` | +| `skip!` / `fail!` / `next!` / `break!` | `lib/roast/cog_input_context.rb` (hardcoded) | +| `target!` / `targets` / `args` / `kwargs` | `lib/roast/cog_input_manager.rb` → `bind_workflow_context` | +| `template("path")` | `lib/roast/cog_input_manager.rb` → `#template` (line 182) | + +--- + +## 14. Timing: When Are Methods Installed? + +| Phase | What happens | Methods available after | +|-------|--------------|------------------------| +| `Workflow.new` | Registry populated (7 built-in cogs) | — | +| `extract_dsl_procs!` | `use` may add custom cogs to registry | — | +| `ConfigManager.prepare!` | `bind_global` + `bind_registered_cogs` on ConfigContext | ConfigContext fully armed | +| | Config procs evaluated (`instance_eval`) | — | +| `ExecutionManager.prepare!` | `bind_outputs` + `bind_registered_cogs` on ExecutionContext | ExecutionContext fully armed | +| | Execution procs evaluated (`instance_eval`) | — | +| `CogInputManager.new` (inside EM.new) | `bind_registered_cogs` + `bind_workflow_context` on CogInputContext | CogInputContext fully armed | + +**Critical implication**: Custom cogs registered via `use` in the workflow definition file are available on all three contexts because `extract_dsl_procs!` runs before any manager's `prepare!`. But you cannot conditionally register cogs — `use` is evaluated at the `Workflow` level before contexts exist. + +--- + +## 15. Method Count Summary + +| Context | Dynamic methods | Hardcoded methods | Module methods | Total | +|---------|----------------|-------------------|----------------|-------| +| ConfigContext | 8 | 0 | 0 | **8** | +| ExecutionContext | 9 | 0 | 0 | **9** | +| CogInputContext | 31 | 4 | 3 | **38** | +| **Grand total** | **48** | **4** | **3** | **55** | + +With 7 built-in cog types. Each additional custom cog adds: +- +1 method to ConfigContext +- +1 method to ExecutionContext +- +3 methods to CogInputContext (triplet) +- = **+5 methods per custom cog** diff --git a/internal/documentation/architecture/07-control-flow-reference.md b/internal/documentation/architecture/07-control-flow-reference.md new file mode 100644 index 00000000..08a31840 --- /dev/null +++ b/internal/documentation/architecture/07-control-flow-reference.md @@ -0,0 +1,490 @@ +# Document 7: Control Flow Reference + +> **Audience**: AI coding agents and developers working on or with Roast internals. +> **Purpose**: Exhaustive reference for the exception-based control flow system, including the complete propagation matrix, sync/async divergence, output access semantics, and known edge cases. +> **Source version**: Roast 1.1.0 (verified May 2026) + +--- + +## 1. The ControlFlow Exception Hierarchy + +All control flow signals inherit from `ControlFlow::Base < StandardError`, which is a **separate** hierarchy from `Roast::Error`. This separation is intentional: control flow signals are not errors — they are structured workflow directives. + +**Source**: `lib/roast/control_flow.rb:1–39` + +``` +ControlFlow::Base < StandardError +├── SkipCog — terminate this cog, mark it skipped (line 11) +├── FailCog — terminate this cog, mark it failed (line 16) +├── Next — terminate the current scope/iteration, advance (line 25) +└── Break — terminate the current scope/iteration AND all subsequent iterations (line 37) +``` + +### Summary Table + +| Exception | Raised By | Cog State Set | Propagation Rule | +|------------|-----------|---------------|------------------| +| `SkipCog` | `skip!(msg)` in input/execute blocks | `@skipped = true` | **Never** re-raised from `Cog#run!` | +| `FailCog` | `fail!(msg)` in input/execute blocks; `CommandRunner` when `fail_on_error?` | `@failed = true` | Re-raised **only if** `config.abort_on_failure?` (default: `true`) | +| `Next` | `next!(msg)` in input/execute blocks | `@skipped = true` | **Always** re-raised from `Cog#run!` | +| `Break` | `break!(msg)` in input/execute blocks | `@skipped = true` | **Always** re-raised from `Cog#run!` | + +**Key distinction**: `SkipCog` and `FailCog` are **cog-scoped** — they affect only the current cog. `Next` and `Break` are **scope-scoped** — they affect the entire execution scope (all subsequent cogs in the scope are abandoned). + +--- + +## 2. Layer-by-Layer Propagation + +Control flow exceptions pass through four distinct layers, each with its own catch/propagate policy. Understanding these layers is the key to understanding Roast's runtime behavior. + +### Layer 1: Cog.run! + +**Source**: `lib/roast/cog.rb:71–101` + +This is the innermost boundary. Every exception raised during cog input evaluation or execution is first caught here. + +```ruby +# Simplified from lib/roast/cog.rb:84–101 +rescue ControlFlow::SkipCog + @skipped = true + # SWALLOWED — does not propagate +rescue ControlFlow::FailCog => e + @failed = true + raise e if config.abort_on_failure? # conditional propagation +rescue ControlFlow::Next, ControlFlow::Break => e + @skipped = true + raise e # ALWAYS propagates +rescue StandardError => e + @failed = true + raise e # ALWAYS propagates +``` + +**Important**: The cog runs inside a `barrier.async(finished: false)` task (line 74). When an exception is re-raised here, it becomes the task's result. How it surfaces to the parent depends on whether the cog is synchronous or asynchronous — this is Layer 2. + +### Layer 2: ExecutionManager.run! + +**Source**: `lib/roast/execution_manager.rb:87–116` + +This is where the critical **sync/async divergence** occurs. The `run!` method iterates the cog stack and has two distinct exception paths. + +#### Sync Path (line 104) + +```ruby +cog_task.wait unless cog_config.async? +``` + +For synchronous cogs, `cog_task.wait` is called **immediately** after `cog.run!` returns the task. This call surfaces any exception the task raised, **directly in the cog stack iteration loop**. The exception propagates through the `@cog_stack.each` block, hits the `ensure` clause (line 110), and then propagates **out of `run!`** to the parent. + +#### Async Path (line 108) + +```ruby +@barrier.wait { |task| wait_for_task_with_exception_handling(task) } +``` + +For asynchronous cogs, the exception surfaces later, when the barrier drains completed tasks. The handler at line 148 determines what happens: + +```ruby +# lib/roast/execution_manager.rb:148–160 +def wait_for_task_with_exception_handling(task) + task.wait +rescue ControlFlow::Next + @barrier.stop # scope ends normally — Next is SWALLOWED +rescue ControlFlow::Break => e + @barrier.stop + compute_final_output # eagerly compute before re-raising + raise e # Break is RE-RAISED +rescue StandardError => e + @barrier.stop + raise e # errors are RE-RAISED +end +``` + +**The divergence**: For sync cogs, `Next` propagates out of `run!` (just like `Break`). For async cogs, `Next` is swallowed but `Break` propagates. This is the single most important behavioral distinction in the entire framework. + +#### The `ensure` Clause (lines 110–115) + +Regardless of how `run!` exits (normally or via exception), the `ensure` block always runs: + +```ruby +ensure + @barrier.stop # cancel any still-running async cogs + compute_final_output # set @final_output (idempotent) + TaskContext.end # clean up fiber-local path + @running = false +``` + +This guarantees that `final_output` is always computed, even when `break!` or an error terminates execution early. + +### Layer 3: System Cog Managers + +Each system cog type wraps `ExecutionManager.run!` with its own catch policy. + +#### Call::Manager + +**Source**: `lib/roast/system_cogs/call.rb:106–113` + +```ruby +em.prepare! +begin + em.run! +rescue ControlFlow::Next, ControlFlow::Break + # both swallowed — inner scope ends early, returns Output.new(em) normally +end +Output.new(em) +``` + +**Behavior**: Both `Next` and `Break` are treated identically — they simply end the called scope early. The call cog always returns normally to its parent scope. + +#### Map::Manager — Serial + +**Source**: `lib/roast/system_cogs/map.rb:288–303` + +```ruby +input.items.each_with_index do |item, index| + ems << em = create_execution_manager_for_map_item(...) + em.prepare! + em.run! +rescue ControlFlow::Next + # swallowed — loop continues to next item +rescue ControlFlow::Break + break # exits the each_with_index loop +end +``` + +**Behavior**: `Next` advances to the next collection item. `Break` exits the entire map loop. Both are handled cleanly. + +#### Map::Manager — Parallel + +**Source**: `lib/roast/system_cogs/map.rb:306–338` + +```ruby +# Per-task (inside barrier.async): +em.prepare! +em.run! +rescue ControlFlow::Next + # swallowed per-task — other parallel tasks continue + +# Barrier wait: +barrier.wait do |task| + task.wait +rescue ControlFlow::Break + barrier.stop # terminates all parallel tasks +rescue StandardError => e + barrier.stop + raise e +end +``` + +**Behavior**: `Next` is caught per-task (other iterations continue). `Break` stops all parallel iterations via the barrier. Note that `Next` from a sync cog inside a parallel map iteration propagates out of `em.run!` and is caught by the per-task rescue on line 316. + +#### Repeat::Manager + +**Source**: `lib/roast/system_cogs/repeat.rb:216–233` + +```ruby +loop do + ems << em = ExecutionManager.new(...) + em.prepare! + em.run! + scope_value = em.final_output + break if max_iterations.present? && ems.length >= max_iterations +rescue ControlFlow::Break + break # exits the loop +end +Output.new(ems) +``` + +**⚠️ Critical**: There is **no** `rescue ControlFlow::Next` clause. This has different consequences depending on sync/async: + +- **Async cog calls `next!`**: The inner `ExecutionManager` swallows it (Layer 2 async path). The iteration completes with whatever `final_output` was computed. The loop continues normally. **This is the expected behavior.** +- **Sync cog calls `next!`**: The inner `ExecutionManager` propagates it (Layer 2 sync path). The `Next` exception **escapes** the repeat loop entirely, propagating to the parent scope. **This is a known bug** (see Section 7). + +### Layer 4: Workflow.start! + +**Source**: `lib/roast/workflow.rb:61–73` + +```ruby +begin + @execution_manager.run! +rescue ControlFlow::Break + # treat break! as graceful termination +end +@completed = true +``` + +**⚠️ Critical**: There is **no** `rescue ControlFlow::Next`. Same consequences as Repeat: + +- **Async cog calls `next!` at top level**: Swallowed by the EM's barrier handler. Workflow ends normally. +- **Sync cog calls `next!` at top level**: `Next` escapes `Workflow.start!` as an unhandled exception. **This is a known bug** (see Section 7). + +--- + +## 3. The Sync/Async Divergence — Why It Exists + +This is not a bug in the design. It is an **unavoidable consequence** of fiber-based cooperative concurrency. + +When a **synchronous** cog raises `Next`, the exception flows "in-band" — through the normal Ruby call stack. `cog_task.wait` on line 104 surfaces it immediately in the same fiber as `ExecutionManager.run!`, and it propagates naturally through the `@cog_stack.each` loop. + +When an **asynchronous** cog raises `Next`, the exception is "out-of-band" — it occurs in a different fiber. The only mechanism to communicate it back is through the barrier's task completion handler (`wait_for_task_with_exception_handling`). This handler **cannot** re-raise `Next` into the parent fiber context in a way that would skip remaining stack entries — the parent fiber is in the middle of `@barrier.wait`, not `@cog_stack.each`. So `Next` is swallowed, and the barrier is stopped to prevent further async work. + +The architectural consequence: **`Next` is reliable only in async cogs or serial map iterations.** For sync cogs in repeat loops or at the top level, `Next` propagates beyond its intended scope. + +--- + +## 4. Complete Propagation Matrix + +This is the definitive reference. Each cell describes what happens when the given exception is raised by a cog in the given context. + +### SkipCog + +| Context | Behavior | +|---------|----------| +| Any cog (sync or async) | Swallowed in `Cog#run!` (line 84). Cog marked `@skipped = true`. No propagation. | +| `outputs` / `outputs!` block | Not raised by control flow — but see `CogSkippedError` in Section 5. | + +### FailCog + +| Context | `abort_on_failure? = false` | `abort_on_failure? = true` (default) | +|---------|----------------------------|--------------------------------------| +| Any cog | Swallowed in `Cog#run!` (line 88). Cog marked `@failed = true`. | Re-raised from `Cog#run!` (line 92). Propagates as `StandardError` through all layers. | +| Call scope | N/A | Not caught by Call::Manager — propagates to parent. | +| Map (serial) | N/A | Not caught by Map serial — propagates out of map. | +| Map (parallel) | N/A | Caught by barrier `rescue StandardError` (line 329) — barrier stops, re-raised. | +| Repeat | N/A | Not caught by Repeat::Manager — propagates out of repeat. | +| Top-level | N/A | Not caught by `Workflow.start!` — **unhandled exception**. | +| `outputs` block | N/A | `FailCog` is **not** in `compute_final_output`'s rescue clause — always propagates. | + +### Next + +| Context | Sync Cog | Async Cog | +|---------|----------|-----------| +| **In Call** | Propagates from `em.run!` → caught by Call::Manager (line 108). Scope ends early, returns normally. | Swallowed by inner EM's barrier handler (line 150). Scope ends normally. | +| **In Map (serial)** | Caught by `rescue ControlFlow::Next` (line 294). Loop advances to next item. | N/A (serial mode = all cogs are sync within each iteration's EM). | +| **In Map (parallel)** | Propagates from `em.run!` → caught by per-task `rescue ControlFlow::Next` (line 316). Other iterations continue. | Swallowed by inner EM's barrier handler. Task ends normally. Other iterations continue. | +| **In Repeat** | ⚠️ Propagates from `em.run!` → **NOT caught** by Repeat::Manager → **escapes repeat entirely** to parent scope. | Swallowed by inner EM's barrier handler. Iteration ends normally. `final_output` feeds next iteration. Loop continues. | +| **Top-level** | ⚠️ Propagates from `em.run!` → **NOT caught** by `Workflow.start!` → **unhandled exception**. | Swallowed by EM's barrier handler. Workflow ends normally. | +| **In `outputs` block** | Caught by `compute_final_output` rescue (line 268). `final_output = nil`. | Same. | + +### Break + +| Context | Sync Cog | Async Cog | +|---------|----------|-----------| +| **In Call** | Propagates from `em.run!` → caught by Call::Manager (line 108). Scope ends early, returns normally. | Re-raised by EM barrier handler (line 155) → caught by Call::Manager. Same result. | +| **In Map (serial)** | Caught by `rescue ControlFlow::Break` (line 297). Loop exits via `break`. | N/A (serial). | +| **In Map (parallel)** | Propagates from `em.run!` → ⚠️ goes to per-task fiber, not caught per-task → raised during `barrier.wait` → caught (line 326). Barrier stops. | Re-raised by inner EM barrier handler → caught by outer map `barrier.wait` (line 326). Barrier stops. | +| **In Repeat** | Caught by `rescue ControlFlow::Break` (line 231). Loop exits. | Re-raised by inner EM barrier handler → caught by Repeat (line 231). Loop exits. | +| **Top-level** | Caught by `Workflow.start!` (line 68). Graceful termination. | Re-raised by EM barrier handler → caught by `Workflow.start!`. Graceful termination. | +| **In `outputs` block** | ⚠️ **NOT caught** by `compute_final_output` — propagates through the `ensure` block. See Section 6. | Same. | + +--- + +## 5. Output Access Semantics + +When a cog input block or `outputs` block accesses another cog's output, a separate error hierarchy governs what happens. + +### The CogOutputAccessError Hierarchy + +**Source**: `lib/roast/cog_input_manager.rb:7–17` + +``` +CogOutputAccessError < Roast::Error (NOT under ControlFlow::Base!) +├── CogDoesNotExistError — no cog with that name was ever declared +├── CogNotYetRunError — cog exists but hasn't completed yet +├── CogSkippedError — cog was skipped (via skip!, next!, or break!) +├── CogFailedError — cog failed (via fail! or unhandled StandardError) +└── CogStoppedError — cog's Async task was externally stopped +``` + +**Design note**: This hierarchy is under `Roast::Error`, not `ControlFlow::Base`. Output access errors are **consumer-side** errors (the code trying to read the output encounters a problem), whereas control flow exceptions are **producer-side** signals (the code running the cog decides to skip/fail/advance). + +### The Three Accessor Methods + +**Source**: `lib/roast/cog_input_manager.rb:54–79` + +| Method | Blocks on async? | On error | Returns | +|--------|-------------------|----------|---------| +| `cog_type(:name)` (tolerant) | No | Catches all `CogOutputAccessError` **except** `CogDoesNotExistError` → returns `nil` | `Cog::Output?` | +| `cog_type!(:name)` (strict) | **Yes** — calls `cog.wait` (line 73) | Raises `CogSkippedError`, `CogFailedError`, `CogStoppedError`, `CogNotYetRunError` | `Cog::Output` | +| `cog_type?(:name)` (query) | No | Returns `false` for any access error except `CogDoesNotExistError` | `bool` | + +The strict accessor (`!`) performs a **blocking wait** on async cogs before checking state. This is what makes sync cogs act as implicit barriers — any `!` access to a still-running async cog pauses the current fiber until that cog completes. + +### State Check Order in `cog_output!` + +**Source**: `lib/roast/cog_input_manager.rb:69–79` + +```ruby +def cog_output!(cog_name) + raise CogDoesNotExistError, cog_name unless @cogs.key?(cog_name) + + @cogs[cog_name].tap do |cog| + cog.wait # block until complete + raise CogSkippedError, cog_name if cog.skipped? + raise CogFailedError, cog_name if cog.failed? + raise CogStoppedError, cog_name if cog.stopped? + raise CogNotYetRunError, cog_name unless cog.succeeded? + end.output.deep_dup # defensive copy on every access +end +``` + +The check order matters: `skipped?` → `failed?` → `stopped?` → `succeeded?`. A cog that was both `@skipped = true` and has a failed task will report as skipped (because `skipped?` is checked first). + +**Deep copy**: Every successful output access returns a `deep_dup` of the output object (line 78). This prevents the caller from mutating the original output, maintaining the framework's isolation guarantees. + +--- + +## 6. The `outputs` / `outputs!` Finalizer + +**Source**: `lib/roast/execution_manager.rb:254–283` + +The `compute_final_output` method runs in the `ensure` block of `ExecutionManager.run!` (line 112), guaranteeing it always executes. It is also called eagerly at line 109 (normal completion) and line 155 (before re-raising `Break`). The `@final_output_computed` flag (line 256) ensures idempotency. + +### Error Handling Matrix + +| Error Type | `outputs { ... }` (tolerant) | `outputs! { ... }` (strict) | +|------------|------------------------------|-----------------------------| +| `CogNotYetRunError` | Swallowed → `final_output = nil` | **Re-raised** | +| `CogSkippedError` | Swallowed → `final_output = nil` | **Re-raised** | +| `CogStoppedError` | Swallowed → `final_output = nil` | **Re-raised** | +| `CogDoesNotExistError` | **Not caught** → propagates | **Not caught** → propagates | +| `CogFailedError` | **Not caught** → propagates | **Not caught** → propagates | +| `ControlFlow::SkipCog` | Swallowed → `final_output = nil` | Swallowed → `final_output = nil` | +| `ControlFlow::Next` | Swallowed → `final_output = nil` | Swallowed → `final_output = nil` | +| `ControlFlow::FailCog` | **Not caught** → propagates | **Not caught** → propagates | +| `ControlFlow::Break` | **Not caught** → propagates | **Not caught** → propagates | + +**Source for rescue clauses**: lines 268–282. + +### Default Behavior (No `outputs` Block) + +If neither `outputs` nor `outputs!` is defined, `compute_final_output` falls back to: + +```ruby +last_cog_name = @cog_stack.last&.name +raise CogDoesNotExistError, "no cogs defined in scope" unless last_cog_name +@cog_input_manager.send(:cog_output, last_cog_name) +``` + +This uses the **tolerant** accessor on the last cog in the stack. If the last cog was skipped, failed, or stopped, `final_output` will be `nil` (not an exception). + +--- + +## 7. Known Bugs and Edge Cases + +### Bug 1: Repeat + Sync `next!` Escapes the Loop + +**Location**: `lib/roast/system_cogs/repeat.rb:216–233` + +**Problem**: `Repeat::Manager` only catches `ControlFlow::Break`. If a **synchronous** cog inside a repeat loop calls `next!`, the inner `ExecutionManager.run!` propagates the `Next` exception (Layer 2 sync path), and Repeat has no rescue for it. The exception escapes the entire repeat cog and propagates to the parent scope. + +**Expected behavior**: `next!` should advance to the next iteration (like it does for async cogs, where the inner EM swallows it). + +**Workaround**: Use `break!` to exit, or make the cog async. + +### Bug 2: Top-Level Sync `next!` Is Unhandled + +**Location**: `lib/roast/workflow.rb:61–73` + +**Problem**: `Workflow.start!` only catches `ControlFlow::Break`. A synchronous cog at the top level calling `next!` produces an unhandled `ControlFlow::Next` exception. + +**Expected behavior**: `next!` at the top level should terminate the workflow gracefully (like `break!` does). + +**Workaround**: Use `break!` instead of `next!` at the top level. + +### Bug 3: `outputs!` + `break!` Exception Masking + +**Location**: `lib/roast/execution_manager.rb:110–112, 254–282` + +**Problem**: When `break!` terminates a scope, `compute_final_output` runs in the `ensure` block. If the `outputs!` block accesses a cog that was skipped due to `break!`, `CogSkippedError` is raised. In Ruby, an exception raised in `ensure` **replaces** the original exception (`ControlFlow::Break`). The parent scope sees `CogSkippedError` instead of `Break`, which may prevent proper loop termination. + +**Expected behavior**: The `Break` exception should propagate to the parent for loop control, regardless of what happens in `outputs!`. + +**Workaround**: Use `outputs` (tolerant) instead of `outputs!` when `break!` may be called. + +### Bug 4: Empty Scope Without `outputs` Block + +**Location**: `lib/roast/execution_manager.rb:263–264` + +**Problem**: If a scope has zero cogs and no `outputs` block, `@cog_stack.last` is `nil`, and the code raises `CogDoesNotExistError` with "no cogs defined in scope". This error is **not** in the rescue clauses of `compute_final_output`, so it propagates. + +**Expected behavior**: An empty scope should produce `nil` as `final_output`. + +### Edge Case: `FailCog` in `outputs` Block + +**Source**: `lib/roast/execution_manager.rb:268–282` + +The rescue clauses in `compute_final_output` do **not** catch `ControlFlow::FailCog`. If the `outputs` block calls `fail!`, the exception propagates even from the tolerant `outputs` variant. This is intentional — calling `fail!` in a finalizer is a deliberate error signal that should not be silently swallowed. + +### Edge Case: `Break` in `outputs` Block + +`ControlFlow::Break` is also **not** caught by `compute_final_output`. Calling `break!` inside an `outputs` block propagates the break signal to the parent scope. This is by design — `break!` is meant to signal loop termination at any level. + +--- + +## 8. Control Flow Transparency in Helpers + +The three scope-bridging helpers — `from`, `collect`, and `reduce` — are **transparent** to control flow. They do not catch any `ControlFlow` exceptions. If a block passed to these helpers raises `skip!`, `fail!`, `next!`, or `break!`, that exception propagates directly to the calling context (typically an `outputs` block or a cog input block). + +| Helper | Source | Block context | +|--------|--------|---------------| +| `from(call_output) { ... }` | `lib/roast/system_cogs/call.rb:147–157` | Block runs in the **called scope's** `CogInputContext` | +| `collect(map_output) { ... }` | `lib/roast/system_cogs/map.rb:375–389` | Block runs in **each iteration's** `CogInputContext` | +| `reduce(map_output, init) { ... }` | `lib/roast/system_cogs/map.rb:426–448` | Block runs in **each iteration's** `CogInputContext` | + +**Practical consequence**: Calling `break!` inside a `collect` block in an `outputs` context will propagate `Break` out of `compute_final_output`, potentially masking the loop's normal termination. + +--- + +## 9. Decision Tree for Workflow Authors + +Use this to choose the right control flow primitive. + +``` +Want to skip JUST THIS COG? + └── YES → skip! + (cog becomes nil, workflow continues) + +Want to signal THIS COG FAILED? + └── YES → fail! + ├── abort_on_failure? = true → scope terminates (default) + └── abort_on_failure? = false → cog becomes nil, workflow continues + +Want to ADVANCE TO NEXT ITERATION? + └── YES → next! + ├── In map (serial or parallel) → ✅ works correctly + ├── In repeat with ASYNC cogs → ✅ works correctly + ├── In repeat with SYNC cogs → ⚠️ BUG: escapes repeat + ├── In call scope → ✅ ends scope early + └── At top level with SYNC cog → ⚠️ BUG: unhandled + +Want to EXIT THE LOOP ENTIRELY? + └── YES → break! + ├── In map → ✅ stops all iterations + ├── In repeat → ✅ exits loop + ├── In call → ✅ ends scope early + └── At top level → ✅ graceful workflow termination +``` + +### Safety Recommendations + +1. **Prefer `break!` over `next!` for sync cogs in repeat loops.** The `next!` bug means sync cogs cannot reliably advance repeat iterations. +2. **Prefer `outputs` over `outputs!` when `break!` may be called.** The exception-masking interaction means strict mode can interfere with loop control. +3. **Never call `fail!` inside an `outputs` block** unless you intend to terminate the scope with an error. It propagates even from tolerant `outputs`. +4. **`CogDoesNotExistError` is always fatal.** Neither the tolerant accessor (`cog_type(:name)`) nor the tolerant finalizer (`outputs`) will catch it. Always ensure cog names are correct. + +--- + +## 10. Quick-Reference: Exception × Layer Matrix + +A compact summary for fast lookup. + +| Exception | Cog.run! | EM sync | EM async handler | Call | Map serial | Map parallel | Repeat | Workflow.start! | +|-----------|----------|---------|------------------|------|------------|--------------|--------|-----------------| +| `SkipCog` | swallow | — | — | — | — | — | — | — | +| `FailCog` (abort=false) | swallow | — | — | — | — | — | — | — | +| `FailCog` (abort=true) | re-raise | propagate | re-raise | propagate | propagate | barrier stop, re-raise | propagate | **unhandled** | +| `Next` | re-raise | propagate | **swallow** | catch | catch (advance) | catch per-task | **⚠️ escapes** | **⚠️ unhandled** | +| `Break` | re-raise | propagate | re-raise | catch | catch (exit loop) | barrier stop | catch (exit loop) | catch (graceful) | +| `StandardError` | re-raise | propagate | re-raise | propagate | propagate | barrier stop, re-raise | propagate | propagate | + +**Legend**: "—" means the exception never reaches this layer. "swallow" means caught and not re-raised. "propagate" means exception passes through without being caught. "catch" means caught and handled appropriately. diff --git a/internal/documentation/architecture/08-infrastructure-and-events.md b/internal/documentation/architecture/08-infrastructure-and-events.md new file mode 100644 index 00000000..e350a09f --- /dev/null +++ b/internal/documentation/architecture/08-infrastructure-and-events.md @@ -0,0 +1,768 @@ +# Document 8: Infrastructure & Events + +The infrastructure layer solves a fundamental problem in concurrent workflow execution: +**how to correctly attribute output from parallel fibers to the right cog/scope in the +workflow hierarchy**. It does this through a 4-stage event pipeline, fiber-local path +tracking, global IO interception, and a two-layer logging design. + +--- + +## 1. The 4-Stage Event Pipeline + +``` +┌──────────────────────┐ +│ PRODUCERS │ +│ TaskContext (.begin) │ +│ Roast::Log (.info) │ +│ OutputRouter (.write)│ +└─────────┬────────────┘ + │ Event << { key: value } + ▼ +┌──────────────────────┐ +│ EVENT │ +│ path + payload + time│ +└─────────┬────────────┘ + │ EventMonitor.accept(event) + ▼ +┌──────────────────────┐ +│ EVENT MONITOR │ +│ Async::Queue consumer│ +│ handle_#{type}_event │ +└─────────┬────────────┘ + │ Roast::Log.logger.add(...) + ▼ +┌──────────────────────┐ +│ $stderr │ +│ (via Logger backend) │ +└──────────────────────┘ +``` + +### Why This Exists + +In a parallel map with 10 fibers all running cmd cogs simultaneously, each fiber's +stdout/stderr must be attributed to the correct `{:files}[3] -> cmd(:lint)` path. +Without this pipeline, output from concurrent fibers would interleave without +attribution, making parallel workflows undebuggable. + +### Three Producer Categories + +| Producer | Events Created | Source | +|----------|---------------|--------| +| **TaskContext** | `{begin: PathElement}`, `{end: PathElement}` | `lib/roast/task_context.rb` lines 28–41 | +| **Roast::Log** | `{debug: msg}`, `{info: msg}`, `{warn: msg}`, `{error: msg}`, `{fatal: msg}`, `{unknown: msg}` | `lib/roast/log.rb` lines 35–62 | +| **OutputRouter** | `{stdout: str}`, `{stderr: str}` | `lib/roast/output_router.rb` lines 52–63 | + +All events enter the pipeline through exactly ONE entry point: `Event << { key: value }` +(line 8 of `lib/roast/event.rb`). + +--- + +## 2. Event Class + +**Source**: `lib/roast/event.rb` (75 lines) + +### Structure + +```ruby +class Event + attr_reader :path # Array[TaskContext::PathElement] — snapshot of producer's path + attr_reader :payload # Hash[Symbol, untyped] — the event data + attr_reader :time # Time — when the event was CREATED (not processed) +end +``` + +### The Universal Emitter (line 8) + +```ruby +def self.<<(event) + EventMonitor.accept(Event.new(TaskContext.path, event)) +end +``` + +This is the ONLY way events enter the pipeline. `TaskContext.path` returns a `deep_dup` +of the current fiber's path, capturing the exact execution position at creation time. + +### Type Detection (lines 48–52) + +Key-intersection based, with log types taking priority: + +```ruby +def type + return :log if (LOG_TYPE_KEYS & @payload.keys).present? + (OTHER_TYPE_KEYS & @payload.keys).first || :unknown +end +``` + +**Priority rule**: If a payload contains BOTH a log key and an other key (e.g., +`{info: "msg", begin: element}`), the event type is `:log`. This prioritization +ensures that log messages are never misrouted. + +**Type keys**: +- `LOG_TYPE_KEYS`: `[:fatal, :error, :warn, :info, :debug, :unknown]` +- `OTHER_TYPE_KEYS`: `[:begin, :end, :stdout, :stderr]` + +### Severity Mapping (lines 55–65) + +```ruby +def log_severity + severity = case type + when :log + (LOG_TYPE_KEYS & @payload.keys).first || :unknown + when :stderr + :warn + else + :info + end + Logger::Severity.const_get(:LEVELS)[severity.to_s] +end +``` + +- Log events → severity from the key name (`:debug`=0, `:info`=1, `:warn`=2, etc.) +- stderr events → always `:warn` +- Everything else → `:info` + +### Delegate Pattern (line 38) + +```ruby +delegate :[], :key?, :keys, to: :payload +``` + +Events can be treated like hashes: `event[:begin]`, `event.key?(:stdout)`. + +--- + +## 3. EventMonitor + +**Source**: `lib/roast/event_monitor.rb` (162 lines) + +A **singleton module** (`extend self`) with two operational modes. This dual-mode +design is critical for testability. + +### Running Mode (during workflow execution) + +**Startup** (`start!`, line 24): +1. Raises `EventMonitorAlreadyStartedError` if already running +2. Calls `OutputRouter.enable!` — starts intercepting IO +3. Creates a new `Async::Queue` +4. Spawns a `transient: true` Async task as the consumer fiber +5. Consumer calls `OutputRouter.mark_as_output_fiber!` (prevents recursion — see §4) +6. Consumer loops: `@queue.pop` → `handle_event` → repeat until `nil` + +**Shutdown** (`stop!`, line 41): +1. Raises `EventMonitorNotRunningError` unless running +2. Calls `OutputRouter.disable!` — stops intercepting IO +3. Closes the queue (causes consumer to receive `nil` → break) +4. `@task.wait` — blocks until consumer drains remaining events + +**Reset** (`reset!`, line 51): +- Force-close without waiting. Used in tests to avoid hangs. + +### Not-Running Mode (before/after workflow, during tests) + +```ruby +def accept(event) + if running? + @queue.push(event) + else + handle_event(event) # synchronous fallback + end +end +``` + +When the monitor isn't running, events are handled synchronously in the current fiber. +This is why `Roast::Log.info` works even outside a workflow context (e.g., in tests, +in CLI setup code). + +### Handler Dispatch (lines 69–78) + +Dynamic dispatch via `send`: + +```ruby +def handle_event(event) + with_stubbed_class_method_returning(Time, :now, event.time) do + OutputRouter.mark_as_output_fiber! + handler_method_name = "handle_#{event.type}_event".to_sym + if respond_to?(handler_method_name, true) + send(handler_method_name, event) + else + handle_unknown_event(event) + end + end +end +``` + +### Handler Methods + +| Handler | Trigger | Behavior | +|---------|---------|----------| +| `handle_begin_event` | `:begin` | Logs "Starting" for cog begins; "🔥🔥🔥 Workflow Starting" for top-level EM | +| `handle_begin_workflow_event` | first `:begin` | Debug dump of WorkflowContext (targets, args, kwargs, tmpdir, workflow_dir, pwd) | +| `handle_end_event` | `:end` | Logs "Complete" for cog ends; "🔥🔥🔥 Workflow Complete" for top-level EM | +| `handle_log_event` | any log key | `Log.logger.add(severity, "path message")` | +| `handle_stdout_event` | `:stdout` | `Log.logger.info { "path ❯ content" }` (single chevron) | +| `handle_stderr_event` | `:stderr` | `Log.logger.warn { "path ❯❯ content" }` (double chevron) | +| `handle_unknown_event` | unrecognized | `Log.logger.unknown(event.inspect)` | + +### Time Preservation (line 70) + +```ruby +with_stubbed_class_method_returning(Time, :now, event.time) do + # ... handle event ... +end +``` + +This temporarily stubs `Time.now` to return the event's **creation time** (when the +producer fiber emitted it), not the **processing time** (when the consumer fiber handles +it). Without this, all events in a parallel burst would show the same "processed at" +timestamp instead of their actual creation times. + +**Implementation** (lines 151–161): Saves the original singleton method, replaces with +a proc returning the fixed value, calls the block, then restores the original in `ensure`. + +### Path Formatting (lines 138–148) + +```ruby +def format_path(event) + event.path.map do |element| + cog = element.cog + execution_manager = element.execution_manager + if cog.present? + "#{cog.type}#{cog.anonymous? ? "" : "(:#{cog.name})"}" + elsif execution_manager&.scope + "{:#{execution_manager.scope}}[#{execution_manager.scope_index}]" + end + end.compact.join(" -> ") +end +``` + +Produces human-readable paths: +- `{:files}[0] -> agent(:analyze)` — scoped EM iteration 0, agent cog named "analyze" +- `cmd(:build)` — named cog at top level +- `cmd` — anonymous cog (no name suffix) +- ` ` (empty string) — top-level EM with no scope (compacted out by `.compact`) + +### Error Hierarchy + +``` +StandardError + └── EventMonitorError (line 9) + ├── EventMonitorAlreadyStartedError (line 11) + └── EventMonitorNotRunningError (line 13) +``` + +**Important**: NOT under `Roast::Error`. A `rescue Roast::Error` will NOT catch these. +Same isolation pattern as `CommandRunnerError`. + +--- + +## 4. OutputRouter + +**Source**: `lib/roast/output_router.rb` (76 lines) + +### Purpose + +Intercept `$stdout.write` and `$stderr.write` calls from non-output fibers and route +them through the Event system for proper path attribution. Without this, any gem or +library that writes to stdout/stderr directly would produce unattributed output. + +### Activation (`enable!`, line 12) + +```ruby +def self.enable! + return false if enabled? + activate($stdout, :stdout) + activate($stderr, :stderr) + mark_as_output_fiber! + true +end +``` + +The `activate` method (line 49): +1. `alias_method :write_without_roast, :write` — saves the original write +2. `define_singleton_method(:write)` — installs the routing interceptor +3. The interceptor checks `router.output_fiber?` to decide routing + +### Deactivation (`disable!`, line 22) + +The `deactivate` method (line 68): +```ruby +def deactivate(stream) + sc = stream.singleton_class + sc.send(:remove_method, :write) # remove interceptor + sc.send(:alias_method, :write, WRITE_WITHOUT_ROAST) # restore original + sc.send(:remove_method, WRITE_WITHOUT_ROAST) # clean up alias +end +``` + +A clean three-step restore that leaves no traces on the singleton class. + +### Routing Logic (lines 52–63) + +```ruby +stream.define_singleton_method(:write) do |*args| + if router.output_fiber? + self.send(WRITE_WITHOUT_ROAST, *args) # direct pass-through + else + str = args.map(&:to_s).join + Event << case name + when :stdout then { stdout: str } + when :stderr then { stderr: str } + else { unknown: str } + end + end +end +``` + +**Decision rule**: If the current fiber IS the output fiber → write directly (no event). +If it's any other fiber → create an Event so the write gets attributed to the correct +path in the workflow hierarchy. + +### Fiber Identity (lines 37–43) + +```ruby +def self.output_fiber? + @output_fiber == Fiber.current +end + +def self.mark_as_output_fiber! + @output_fiber = Fiber.current +end +``` + +Only ONE fiber is the output fiber at a time. The EventMonitor's consumer fiber calls +`mark_as_output_fiber!` both at startup (line 30) and within each `handle_event` call +(line 71). This ensures that when the consumer writes to $stderr via `Log.logger`, those +writes bypass the interceptor and go directly to the real IO. + +### The WRITE_WITHOUT_ROAST Escape Hatch + +```ruby +WRITE_WITHOUT_ROAST = :write_without_roast +``` + +Any code that needs to bypass routing can call: +```ruby +$stdout.send(OutputRouter::WRITE_WITHOUT_ROAST, "direct output") +``` + +### ⚠️ The Circular Flow (Critical) + +This is the most subtle interaction in the infrastructure layer: + +``` +Roast::Log.info("msg") + → Event << { info: "msg" } + → EventMonitor.accept(event) + → @queue.push(event) + ... async queue transfer ... + → handle_event(event) + → handle_log_event(event) + → Roast::Log.logger.add(severity, "msg") + → Logger writes to $stderr + → OutputRouter.write intercepts + → output_fiber? == true ← BECAUSE we're in the EM consumer + → write_without_roast("msg") ← breaks the cycle +``` + +If `output_fiber?` didn't return `true` for the consumer fiber, the Logger's write +to $stderr would create another Event, which would be queued, which would be handled, +which would write to $stderr again → **infinite recursion**. + +The distinction between: +- `Roast::Log.info("msg")` — public API, creates Events, goes through the pipeline +- `Roast::Log.logger.add(severity, "msg")` — internal backend, writes directly to IO + +is **CRITICAL**. Calling `Roast::Log.logger.info("msg")` from application code would +bypass event attribution entirely. + +### ⚠️ Global State Modification + +`$stdout` and `$stderr` are process-global. OutputRouter patches their singleton classes. +If `disable!` is not called (e.g., test crashes), subsequent code could break. +This is why `EventMonitor.reset!` exists and is called in test teardown. + +--- + +## 5. TaskContext + +**Source**: `lib/roast/task_context.rb` (53 lines) + +A **singleton module** (`extend self`) that maintains per-fiber execution path stacks +using Ruby 3.2+ fiber-local variables. + +### PathElement (lines 8–19) + +```ruby +class PathElement + attr_reader :cog # Cog? + attr_reader :execution_manager # ExecutionManager? + + def initialize(cog: nil, execution_manager: nil) + @cog = cog + @execution_manager = execution_manager + end +end +``` + +One field or the other is set, never both. A path is a sequence of alternating EM and +Cog elements representing the current position in the workflow hierarchy. + +### Fiber[:path] Storage + +Ruby 3.2+ fiber-local variables: `Fiber[:path]` is an Array unique to each fiber. + +**Creation semantics**: +- `Fiber.new(storage: {})` creates a child fiber with an isolated empty path +- Child fibers without explicit `storage:` inherit the parent's storage reference +- Map parallel creates fibers with `storage: {}` for full isolation + +### Lifecycle Methods + +**`begin_cog(cog)`** (line 28): +```ruby +def begin_cog(cog) + begin_element(PathElement.new(cog:)) +end +``` + +**`begin_execution_manager(em)`** (line 33): +```ruby +def begin_execution_manager(execution_manager) + begin_element(PathElement.new(execution_manager:)) +end +``` + +**`end`** (line 38): +```ruby +def end + Event << { end: Fiber[:path]&.last } + el = Fiber[:path]&.pop + [el, path] +end +``` + +Returns `[popped_element, remaining_path_snapshot]`. + +### Path Isolation (line 48) — Critical Detail + +```ruby +def begin_element(element) + Fiber[:path] = (Fiber[:path] || []) + [element] + Event << { begin: element } + path +end +``` + +The expression `(Fiber[:path] || []) + [element]` creates a **NEW array**. This is +essential because child fibers that inherit the parent's `Fiber[:path]` reference would +otherwise see mutations to each other's paths. By replacing the array entirely (not +pushing to it), each fiber's subsequent path changes are isolated. + +### Path Deep-Dup (line 23) + +```ruby +def path + Fiber[:path]&.deep_dup || [] +end +``` + +Events get a snapshot of the path, not a live reference. This prevents the path from +changing between when the event is created and when the EventMonitor processes it +(which may be milliseconds later in a concurrent workflow). + +### Integration Points + +- `Cog.run!` line 76: `TaskContext.begin_cog(self)` / line 100: `TaskContext.end` +- `ExecutionManager.run!` line 94: `TaskContext.begin_execution_manager(self)` / line 113: `TaskContext.end` + +--- + +## 6. Logging + +**Source**: `lib/roast/log.rb` (99 lines), `lib/roast/log_formatter.rb` (55 lines) + +### The Two-Layer Design + +This is a common source of confusion. There are TWO ways to produce log output: + +| Layer | API | What It Does | When to Use | +|-------|-----|-------------|-------------| +| **Public** | `Roast::Log.info("msg")` | Creates an Event, goes through the pipeline | Application code, cog implementations | +| **Internal** | `Roast::Log.logger.info("msg")` | Writes directly to the Logger's IO device | EventMonitor handlers ONLY | + +Calling `Roast::Log.logger.info` from application code bypasses: +- Path attribution (no fiber context captured) +- Async queue ordering (writes immediately, not in event order) +- Time preservation (uses wall-clock time, not event creation time) + +### Roast::Log Module (`lib/roast/log.rb`) + +**Public methods** (lines 35–62): `debug`, `info`, `warn`, `error`, `fatal`, `unknown` +— all simply emit Events: +```ruby +def info(message) + Roast::Event << { info: message } +end +``` + +**Logger accessor** (line 65): `logger` — memoized stdlib Logger instance writing to $stderr. + +**Configuration**: +- `ROAST_LOG_LEVEL` env var (line 94) — sets minimum level; default `INFO` +- `attr_writer :logger` (line 33) — replace with any Logger-compatible object (e.g., `Rails.logger`) +- `reset!` (line 70) — clears memoized logger (used in tests) + +**TTY detection** (line 75): +```ruby +def tty? + return false unless @logger + logdev = @logger.instance_variable_get(:@logdev)&.dev + logdev&.respond_to?(:isatty) && logdev&.isatty +end +``` + +### LogFormatter (`lib/roast/log_formatter.rb`) + +Two output formats: + +**TTY format** (compact): +``` +• I, Starting agent(:analyze) +``` + +**Non-TTY format** (full): +``` +I, [2026-01-01T12:00:00.000000] INFO -- Starting agent(:analyze) +``` + +**ANSI colorization** (lines 29–44): +| Content | Color | +|---------|-------| +| Lines containing `❯❯` (stderr) | Yellow | +| Lines containing `❯` (stdout) | Default (no extra color) | +| ERROR, FATAL | Red | +| WARN | Orange (`#FF8C00`) | +| INFO | Bright | +| DEBUG | Faint | + +Uses the `Rainbow` gem with TTY-awareness (`@rainbow.enabled = tty`). + +**`msg2str`** (lines 46–54): Strips whitespace from String messages before calling +`super` (parent Logger::Formatter behavior). + +--- + +## 7. CLI & Invocation + +**Source**: `lib/roast/cli.rb` (119 lines) + +### Command Parsing Flow + +``` +argv = ["my_workflow.rb", "target1", "--", "--dry-run", "--model=gpt-4"] + │ │ + ├─ split_at_separator ─────────────────┤ + │ │ + ▼ ▼ +roast_args = ["my_workflow.rb", "target1"] extra_args = ["--dry-run", "--model=gpt-4"] + │ │ + ├─ OptionParser (-h/--help only) ──────│ + │ │ + ▼ ▼ +command dispatch parse_custom_workflow_args + │ │ + ▼ ▼ +resolve_workflow_path args: [:dry_run] kwargs: {model: "gpt-4"} +``` + +### split_at_separator (lines 109–116) + +Splits at `--` into `[roast_args, extra_args]`. If no `--`, extra_args is empty. + +### Command Dispatch (lines 18–34) + +Priority order: +1. `-h`/`--help` flag or `help` command → print help +2. `version` → print version string +3. `execute` → shift command, call `run_execute` +4. Implicit execute → if the command resolves to a file path, treat as workflow +5. Unknown → error message + help + exit(1) + +### resolve_workflow_path (lines 65–76) + +```ruby +def resolve_workflow_path(workflow_path) + roast_working_directory = Pathname.new(File.expand_path(ENV["ROAST_WORKING_DIRECTORY"] || Dir.pwd)) + path = Pathname.new(workflow_path) + resolved = if path.absolute? || path.exist? + path + else + roast_working_directory / path + end + resolved.realpath +rescue Errno::ENOENT + nil +end +``` + +Resolution rules: +- Absolute path or exists relative to cwd → use as-is +- Otherwise → join with `ROAST_WORKING_DIRECTORY` (defaulting to pwd) +- `.realpath` resolves symlinks +- Returns `nil` on file-not-found + +### parse_custom_workflow_args (lines 79–91) + +```ruby +def parse_custom_workflow_args(extra_args) + args = [] + kwargs = {} + extra_args.each do |arg| + arg = arg.sub(/^--?(?=[^-])/, "") # strip leading - or -- + if arg.include?("=") + key, value = arg.split("=", 2) # split at FIRST = only + kwargs[key.to_sym] = value if key + else + args << arg.to_sym + end + end + [args, kwargs] +end +``` + +**Conventions**: +- `--key=value` → kwarg `{key: "value"}` (value is always a String) +- `--flag` → arg `:flag` (Symbol) +- `--a=b=c` → `{a: "b=c"}` (splits at first `=` only) +- Leading `-` or `--` stripped by regex `/^--?(?=[^-])/` + +### run_execute (lines 39–59) + +The workflow entry point: +1. Validates workflow file argument present +2. Splits `args` into `workflow_path` and `targets` (remaining positionals) +3. Resolves path; errors on not-found +4. Parses custom args from `extra_args` +5. Constructs `WorkflowParams.new(targets, workflow_args, workflow_kwargs)` +6. `Dir.chdir(roast_working_directory)` — changes pwd for the duration +7. Calls `Workflow.from_file(real_workflow_path, workflow_params)` + +### ROAST_WORKING_DIRECTORY Environment Variable + +Used in TWO places: +- `resolve_workflow_path`: resolves relative paths against it +- `run_execute`: `Dir.chdir` into it for workflow execution + +Default: `Dir.pwd` (current working directory). + +--- + +## 8. WorkflowParams & WorkflowContext + +### WorkflowParams (`lib/roast/workflow_params.rb`, 22 lines) + +A simple value object: + +```ruby +class WorkflowParams + attr_reader :targets # Array[String] — positional args after workflow path + attr_reader :args # Array[Symbol] — flag-style workflow args (--dry-run → :dry_run) + attr_reader :kwargs # Hash[Symbol, String] — key=value workflow args (--model=gpt-4 → {model: "gpt-4"}) +end +``` + +**Important**: All kwarg values are Strings. No type coercion is performed. + +### WorkflowContext (`lib/roast/workflow_context.rb`, 22 lines) + +Immutable shared state constructed once per workflow invocation: + +```ruby +class WorkflowContext + attr_reader :params # WorkflowParams — the parsed invocation parameters + attr_reader :tmpdir # String — Dir.mktmpdir (unique temp directory per run) + attr_reader :workflow_dir # Pathname — dirname of the workflow .rb file +end +``` + +Constructed in `Workflow.from_file` and shared (by reference, never deep_dup'd) with +every ExecutionManager in the tree. WorkflowContext is the ONE exception to the +deep-copy-at-every-boundary rule — it's intentionally shared because it's immutable. + +--- + +## 9. Complete Data Flow Example + +A concrete trace of what happens when `cmd(:build)` writes to stdout in a parallel map: + +``` +1. cmd(:build) calls system("make") + → make writes "Compiling..." to fd 1 + +2. Ruby's IO captures it; $stdout.write("Compiling...") is called + +3. OutputRouter's interceptor fires (line 52) + → output_fiber? == false (we're in a map worker fiber) + → Event << { stdout: "Compiling..." } + +4. Event.new(TaskContext.path, { stdout: "Compiling..." }) + → path = [{em: map_em, scope: :targets, index: 3}, {cog: build_cog}].deep_dup + → time = Time.now (wall clock at creation) + +5. EventMonitor.accept(event) + → running? == true + → @queue.push(event) + +6. Consumer fiber pops event from queue (may be milliseconds later) + → handle_event(event) + → Time.now stubbed to event.time (step 4's timestamp) + → mark_as_output_fiber! (ensure our writes go through) + → handler = :handle_stdout_event + +7. handle_stdout_event(event) + → Roast::Log.logger.info { "{:targets}[3] -> cmd(:build) ❯ Compiling..." } + +8. Logger calls $stderr.write(formatted_line) + → OutputRouter intercepts + → output_fiber? == true (we ARE the consumer fiber) + → write_without_roast(formatted_line) ← direct to terminal + +9. User sees: + • I, {:targets}[3] -> cmd(:build) ❯ Compiling... +``` + +--- + +## 10. Test Infrastructure Integration + +### CaptureLogOutput Concern + +Included in ALL test cases: +- **setup**: Creates `@logger_output` (StringIO), sets `Roast::Log.logger` to write there +- **teardown**: On test FAILURE, dumps captured output to $stderr (debugging aid), then calls `Roast::Log.reset!` + +### EventMonitor in Tests + +- Tests use `EventMonitor.reset!` (not `stop!`) in teardown — avoids needing an async context +- The not-running synchronous mode means events from `Roast::Log.info` still work in tests +- No Async reactor needed for simple cog unit tests + +### original_streams_from_logger_output + +A test helper that reconstructs original stdout/stderr from captured logger output by: +1. Parsing lines with `❯` marker as stdout +2. Parsing lines with `❯❯` marker as stderr +3. Treating non-log-prefix lines as continuations of the previous stream + +--- + +## 11. Invariants for Contributors + +1. **All events go through `Event <<`** — never call `EventMonitor.accept` directly +2. **Only EventMonitor handlers call `Log.logger.*`** — application code uses `Roast::Log.*` +3. **OutputRouter must be enabled/disabled symmetrically** — `enable!` without `disable!` corrupts $stdout/$stderr globally +4. **`mark_as_output_fiber!` must be called in the consumer** — forgetting this causes infinite recursion +5. **`TaskContext.path` returns a deep_dup** — events capture a snapshot, never a live reference +6. **`begin_element` creates a new array** — push would mutate shared references across fibers +7. **Time stubbing in handle_event is non-negotiable** — removing it breaks all timing accuracy in parallel workflows +8. **EventMonitor errors are NOT under Roast::Error** — `rescue Roast::Error` is intentionally insufficient for infrastructure failures diff --git a/internal/documentation/architecture/09-error-hierarchy.md b/internal/documentation/architecture/09-error-hierarchy.md new file mode 100644 index 00000000..dfb371af --- /dev/null +++ b/internal/documentation/architecture/09-error-hierarchy.md @@ -0,0 +1,289 @@ +# Document 9: Error Hierarchy + +> **Audience**: AI agents (primary — for exception handling changes), Intern (reference) +> +> **Purpose**: Complete error tree with descriptions of when each error is raised, by whom, and whether it propagates or is caught internally. + +--- + +## 1. The Four Root Branches + +The Roast framework has **four independent exception hierarchies**, each rooted directly in `StandardError`: + +``` +StandardError +├── Roast::Error # lib/roast/error.rb:6 +│ └── (all framework errors — see §2) +│ +├── Roast::CommandRunner::CommandRunnerError # lib/roast/command_runner.rb:19 +│ ├── NoCommandProvidedError # lib/roast/command_runner.rb:21 +│ └── TimeoutError # lib/roast/command_runner.rb:23 +│ +├── Roast::EventMonitor::EventMonitorError # lib/roast/event_monitor.rb:9 +│ ├── EventMonitorAlreadyStartedError # lib/roast/event_monitor.rb:11 +│ └── EventMonitorNotRunningError # lib/roast/event_monitor.rb:13 +│ +└── Roast::ControlFlow::Base # lib/roast/control_flow.rb:7 + ├── SkipCog # lib/roast/control_flow.rb:11 + ├── FailCog # lib/roast/control_flow.rb:16 + ├── Next # lib/roast/control_flow.rb:25 + └── Break # lib/roast/control_flow.rb:37 +``` + +### Critical Design Consequence + +`rescue Roast::Error` will **NOT** catch: +- `CommandRunnerError` (infrastructure — external process failures) +- `EventMonitorError` (infrastructure — logging subsystem) +- `ControlFlow::*` (not "errors" at all — they are flow control signals) + +This is intentional. `Roast::Error` represents programming/configuration mistakes in the framework. The other branches represent orthogonal concerns. + +--- + +## 2. The Complete `Roast::Error` Tree + +``` +Roast::Error # lib/roast/error.rb:6 +│ +├── Workflow::WorkflowError # lib/roast/workflow.rb:6 +│ ├── WorkflowNotPreparedError # lib/roast/workflow.rb:8 +│ ├── WorkflowAlreadyPreparedError # lib/roast/workflow.rb:10 +│ ├── WorkflowAlreadyStartedError # lib/roast/workflow.rb:12 +│ └── InvalidLoadableReference # lib/roast/workflow.rb:14 +│ +├── ExecutionManager::ExecutionManagerError # lib/roast/execution_manager.rb:11 +│ ├── ExecutionManagerNotPreparedError # lib/roast/execution_manager.rb:13 +│ ├── ExecutionManagerAlreadyPreparedError # lib/roast/execution_manager.rb:15 +│ ├── ExecutionManagerCurrentlyRunningError # lib/roast/execution_manager.rb:17 +│ ├── ExecutionScopeDoesNotExistError # lib/roast/execution_manager.rb:19 +│ ├── ExecutionScopeNotSpecifiedError # lib/roast/execution_manager.rb:21 +│ ├── IllegalCogNameError # lib/roast/execution_manager.rb:23 +│ └── OutputsAlreadyDefinedError # lib/roast/execution_manager.rb:25 +│ +├── ConfigManager::ConfigManagerError # lib/roast/config_manager.rb:6 +│ ├── ConfigManagerNotPreparedError # lib/roast/config_manager.rb:7 +│ ├── ConfigManagerAlreadyPreparedError # lib/roast/config_manager.rb:8 +│ └── IllegalCogNameError # lib/roast/config_manager.rb:9 +│ +├── Cog::CogError # lib/roast/cog.rb:6 +│ └── CogAlreadyStartedError # lib/roast/cog.rb:8 +│ +├── Cog::Config::ConfigError # lib/roast/cog/config.rb:13 +│ └── InvalidConfigError # lib/roast/cog/config.rb:16 +│ +├── Cog::Input::InputError # lib/roast/cog/input.rb:20 +│ └── InvalidInputError # lib/roast/cog/input.rb:23 +│ +├── Cog::Registry::CogRegistryError # lib/roast/cog/registry.rb:9 +│ └── CouldNotDeriveCogNameError # lib/roast/cog/registry.rb:12 +│ +├── Cog::Store::CogAlreadyDefinedError # lib/roast/cog/store.rb:7 +│ +├── CogInputContext::CogInputContextError # lib/roast/cog_input_context.rb:10 +│ └── ContextNotFoundError # lib/roast/cog_input_context.rb:12 +│ +├── CogInputManager::CogOutputAccessError # lib/roast/cog_input_manager.rb:7 +│ ├── CogDoesNotExistError # lib/roast/cog_input_manager.rb:9 +│ ├── CogNotYetRunError # lib/roast/cog_input_manager.rb:11 +│ ├── CogSkippedError # lib/roast/cog_input_manager.rb:13 +│ ├── CogFailedError # lib/roast/cog_input_manager.rb:15 +│ └── CogStoppedError # lib/roast/cog_input_manager.rb:17 +│ +├── Map::MapOutputAccessError # lib/roast/system_cogs/map.rb:13 +│ └── MapIterationDidNotRunError # lib/roast/system_cogs/map.rb:19 +│ +├── Agent::AgentCogError # lib/roast/cogs/agent.rb:23 +│ ├── UnknownProviderError # lib/roast/cogs/agent.rb:26 +│ ├── MissingProviderError # lib/roast/cogs/agent.rb:29 +│ └── MissingPromptError # lib/roast/cogs/agent.rb:32 +│ +├── Agent::Providers::Claude::ClaudeInvocation::ClaudeInvocationError # ...claude_invocation.rb:10 +│ ├── ClaudeNotStartedError # ...claude_invocation.rb:12 +│ ├── ClaudeAlreadyStartedError # ...claude_invocation.rb:14 +│ ├── ClaudeNotCompletedError # ...claude_invocation.rb:16 +│ └── ClaudeFailedError # ...claude_invocation.rb:18 +│ +└── Agent::Providers::Pi::PiInvocation::PiInvocationError # ...pi_invocation.rb:10 + ├── PiNotStartedError # ...pi_invocation.rb:12 + ├── PiAlreadyStartedError # ...pi_invocation.rb:14 + ├── PiNotCompletedError # ...pi_invocation.rb:16 + └── PiFailedError # ...pi_invocation.rb:18 +``` + +**Total**: 4 root branches, 14 intermediate error classes, 38 leaf error classes. + +--- + +## 3. Key Design Observations + +### 3.1 Consumer-Side vs Producer-Side Separation + +`CogOutputAccessError` is **NOT** under `CogError`. This is deliberate: + +| Branch | Perspective | Meaning | +|--------|-------------|---------| +| `CogError` | Producer-side | Something went wrong *inside* the cog (e.g., started twice) | +| `CogOutputAccessError` | Consumer-side | Something went wrong when another cog *tried to access* this cog's output | + +A `rescue CogError` will never catch output access failures, and vice versa. The consumer doesn't need to know *why* a cog failed internally — only that its output is unavailable. + +### 3.2 Duplicate `IllegalCogNameError` + +Both `ExecutionManager` and `ConfigManager` define their own `IllegalCogNameError`. These are **not** the same class — they share a name but have different parents. Both are raised during `prepare!` when a cog name collides with an existing method on the respective context: + +- `ConfigManager::IllegalCogNameError` — raised at `config_manager.rb:92` +- `ExecutionManager::IllegalCogNameError` — raised at `execution_manager.rb:193` + +### 3.3 Infrastructure Errors Are Intentionally Isolated + +`CommandRunnerError` and `EventMonitorError` inherit directly from `StandardError`, not from `Roast::Error`. This means a `rescue Roast::Error` in application code will not inadvertently swallow infrastructure failures. If a command times out or the event monitor fails to start, those are operational errors that should propagate to the top level. + +### 3.4 ControlFlow Exceptions Are Not Errors + +The `ControlFlow::Base` branch represents **workflow steering signals**, not error conditions. They inherit from `StandardError` (not `Roast::Error`) because they must be raiseable/rescuable, but they carry no error semantics. See [Document 7: Control Flow Reference](07-control-flow-reference.md) for complete propagation semantics. + +--- + +## 4. Per-Error Reference + +### Lifecycle Guard Errors + +These prevent invalid state transitions. They indicate programmer error (calling methods in wrong order or calling them twice). + +| Error | Raised by | When | +|-------|-----------|------| +| `WorkflowNotPreparedError` | `Workflow#start!` (line 62) | `start!` called before `prepare!` | +| `WorkflowAlreadyPreparedError` | `Workflow#prepare!` (line 47) | `prepare!` called twice | +| `WorkflowAlreadyStartedError` | `Workflow#start!` (line 63) | `start!` called twice | +| `ExecutionManagerNotPreparedError` | `EM#run!` (line 88), `EM#execution_context` (line 140) | `run!` or context accessed before `prepare!` | +| `ExecutionManagerAlreadyPreparedError` | `EM#prepare!` (line 78) | `prepare!` called twice | +| `ExecutionManagerCurrentlyRunningError` | `EM#run!` (line 89) | `run!` called while already running | +| `ConfigManagerNotPreparedError` | `CM#config_for` (line 45) | Config accessed before `prepare!` | +| `ConfigManagerAlreadyPreparedError` | `CM#prepare!` (line 24) | `prepare!` called twice | +| `CogAlreadyStartedError` | `Cog#run!` (line 72) | `run!` called on a cog that already has a task | + +**Caught internally?** Never — these always propagate to the caller. + +--- + +### Registration & Naming Errors + +These occur during `prepare!` when the cog graph is being assembled. + +| Error | Raised by | When | +|-------|-----------|------| +| `InvalidLoadableReference` | `Workflow#resolve_and_validate_loadable` (line 117, 123) | `use` references a class that doesn't exist or isn't a valid Roast primitive | +| `CogAlreadyDefinedError` | `Cog::Store#push` (line 21) | Two cogs with the same name in the same scope | +| `CouldNotDeriveCogNameError` | `Cog::Registry#derive_name` (line 63) | A custom cog class can't be converted to a method name via `demodulize.underscore` | +| `IllegalCogNameError` (EM) | `EM#bind_cog` (line 193) | Cog name collides with existing method on ExecutionContext | +| `IllegalCogNameError` (CM) | `CM#bind_registered_cogs` (line 92) | Cog name collides with existing method on ConfigContext | + +**Caught internally?** Never — these always propagate. + +--- + +### Configuration Errors + +| Error | Raised by | When | +|-------|-----------|------| +| `InvalidConfigError` | `Config#validate!` overrides, `Config#working_directory` (lines 297–298), `Map::Config#valid_parallel!` (line 89), `Chat::Config#valid_api_key!` (line 102), `Chat::Config#valid_provider!` (line 41), `Agent::Config#valid_provider!` (line 41) | A config value is invalid or missing after the merge cascade completes | + +**Caught internally?** Never — propagates as a fatal workflow configuration error. + +--- + +### Input Errors + +| Error | Raised by | When | +|-------|-----------|------| +| `InvalidInputError` | `Input#validate!` overrides in every cog type | Input to a cog is missing or invalid **after** coercion has been attempted | + +**Caught internally?** YES — the first `validate!` failure triggers the coercion path (`coerce(return_value)`), after which `validate!` is called again. Only the *second* failure propagates. See `Cog#run!` and `Cog::Input#coerce_and_validate_input!`. + +--- + +### Output Access Errors + +These form the graduated tolerance model used by `CogInputManager`. + +| Error | Raised by | When | Tolerant mode (`cog_output`) | Strict mode (`cog_output!`) | +|-------|-----------|------|------------------------------|----------------------------| +| `CogDoesNotExistError` | `cog_output!` (line 70) | Referenced cog name doesn't exist in scope | **Re-raises** (line 58) | Raises | +| `CogNotYetRunError` | `cog_output!` (line 77) | Cog hasn't completed yet (sync cog not yet reached) | Swallows → `nil` | Raises | +| `CogSkippedError` | `cog_output!` (line 74) | Cog was skipped via `skip!` or stopped via `next!`/`break!` | Swallows → `nil` | Raises | +| `CogFailedError` | `cog_output!` (line 75) | Cog failed via `fail!` | Swallows → `nil` | Raises | +| `CogStoppedError` | `cog_output!` (line 76) | Cog was stopped by barrier interruption | Swallows → `nil` | Raises | + +**Also caught by `compute_final_output`**: The `outputs` block (non-bang) catches `CogNotYetRunError`, `CogSkippedError`, and `CogStoppedError` (line 274), swallowing them. `CogFailedError` is NOT caught by `compute_final_output` and always propagates. + +--- + +### Scope & Context Errors + +| Error | Raised by | When | +|-------|-----------|------| +| `ExecutionScopeDoesNotExistError` | `EM#run!` (line 164) | `run:` param references a scope name that was never declared | +| `ExecutionScopeNotSpecifiedError` | `Call::Manager` (line 94), `Map::Manager` (line 261), `Repeat::Manager` (line 211) | System cog created without `run:` parameter | +| `OutputsAlreadyDefinedError` | `EM#on_outputs` (line 241), `EM#on_outputs!` (line 248) | Two `outputs`/`outputs!` blocks declared in same scope | +| `ContextNotFoundError` | `CogInputManager#resolve_template_path` (line 219), `Call::InputContext#from` (line 149), `Map::InputContext#collect` (line 377), `Map::InputContext#reduce` (line 428) | Template file not found, or `from`/`collect`/`reduce` called with nil EM | +| `MapIterationDidNotRunError` | `Map::Output#iteration` (line 220) | Accessing a map iteration that was broken/never executed | + +**Caught internally?** `ContextNotFoundError` in template resolution propagates to `Cog#run!`. The others always propagate. + +--- + +### Agent Provider Errors + +| Error | Raised by | When | +|-------|-----------|------| +| `UnknownProviderError` | `Agent#execute` (line 66) | Config specifies an unrecognized provider name | +| `MissingProviderError` | (defined but not currently raised in source) | Reserved for future use | +| `MissingPromptError` | (defined but not currently raised in source) | Reserved for future use | +| `ClaudeAlreadyStartedError` | `ClaudeInvocation#start!` (line 77) | `start!` called on running invocation | +| `ClaudeNotStartedError` | `ClaudeInvocation#result!` (line 121) | `result!` called before `start!` | +| `ClaudeNotCompletedError` | `ClaudeInvocation#result!` (line 123) | `result!` called before process completes | +| `ClaudeFailedError` | `ClaudeInvocation#result!` (line 122), `#process_message` (line 172) | Claude process exits with error or reports error message | +| `PiAlreadyStartedError` | `PiInvocation#start!` (line 80) | Same as Claude equivalent | +| `PiNotStartedError` | `PiInvocation#result!` (line 128) | Same as Claude equivalent | +| `PiNotCompletedError` | `PiInvocation#result!` (line 130) | Same as Claude equivalent | +| `PiFailedError` | `PiInvocation#result!` (line 129) | Same as Claude equivalent | + +**Caught internally?** None are caught internally. These propagate through `Cog#run!`'s `rescue StandardError => e` on line 96, which sets `@failed = true` and re-raises. + +--- + +### Infrastructure Errors + +| Error | Raised by | When | +|-------|-----------|------| +| `NoCommandProvidedError` | `CommandRunner.execute` (line 62) | Empty args array passed to execute | +| `TimeoutError` | `CommandRunner.execute` (line 136) | Command exceeds configured timeout | +| `EventMonitorAlreadyStartedError` | `EventMonitor.start!` (line 25) | `start!` called when queue is already open | +| `EventMonitorNotRunningError` | `EventMonitor.stop!` (line 42) | `stop!` called when queue is already closed | + +**Caught internally?** `TimeoutError` is not caught — it propagates through the cmd cog's execute method. `EventMonitor` errors propagate to the CLI bootstrap. + +--- + +## 5. Error Handling Strategy Summary + +| Layer | What it catches | What propagates | +|-------|----------------|-----------------| +| `Cog#run!` | `SkipCog`, `FailCog` (conditionally), stores `@error` for all others | `Next`, `Break`, `StandardError` (if `abort_on_failure?`) | +| `ExecutionManager#run!` (sync) | Nothing — re-raises immediately after `wait` | Everything | +| `ExecutionManager#run!` (async barrier) | `Next` (swallowed), `Break` (stored + re-raised) | `Break`, other errors | +| `compute_final_output` | `SkipCog`, `Next` (in outputs eval), `CogNotYetRunError`, `CogSkippedError`, `CogStoppedError` | `CogFailedError`, `CogDoesNotExistError` | +| `Call::Manager` | `Next`, `Break` (both treated as scope termination) | Nothing further | +| `Map::Manager` (serial) | `Next` (continue), `Break` (stop iterations) | Nothing further | +| `Map::Manager` (parallel) | `Next` (per-fiber), `Break` (stops barrier) | Nothing further | +| `Repeat::Manager` | `Break` only | `Next` (⚠️ BUG — escapes loop) | +| `Workflow#start!` | `Break` (graceful termination) | `Next` (⚠️ BUG — unhandled) | + +--- + +## 6. See Also + +- [Document 7: Control Flow Reference](07-control-flow-reference.md) — Full propagation matrix for ControlFlow exceptions +- [Document 5: Execution Engine Internals](05-execution-engine-internals.md) — Where in the code each rescue lives +- [Document 12: Known Issues & Gotchas](12-known-issues-and-gotchas.md) — The Repeat+Next and top-level Next bugs diff --git a/internal/documentation/architecture/10-writing-custom-cogs.md b/internal/documentation/architecture/10-writing-custom-cogs.md new file mode 100644 index 00000000..d81b40a7 --- /dev/null +++ b/internal/documentation/architecture/10-writing-custom-cogs.md @@ -0,0 +1,830 @@ +# Document 10: Writing Custom Cogs + +_The extensibility guide for contributors building new cog types._ + +--- + +## 1. The Custom Cog Contract + +Every custom cog must satisfy a minimal contract to integrate with the Roast framework. The framework discovers, registers, configures, and executes cogs through standardized interfaces. + +### Required Elements + +| # | Requirement | Details | +|---|-------------|---------| +| 1 | **Named class** inheriting from `Roast::Cog` | Anonymous classes (`Class.new(Roast::Cog)`) have `name == nil`, which triggers `Cog::Registry::CouldNotDeriveCogNameError` at registration time (registry.rb line 63). | +| 2 | **Nested `Input` class** inheriting from `Cog::Input` | Must implement `validate!`. The base class's `validate!` raises `NotImplementedError` (input.rb line 35), so you **must** override it. Auto-discovered via `find_child_input_or_default` (cog.rb line 35). | +| 3 | **`execute(input)` method** | Protected method returning a `Cog::Output` instance (or subclass). The base class raises `NotImplementedError` (cog.rb line 143). | + +### Optional Elements + +| # | Element | Details | +|---|---------|---------| +| 4 | **Nested `Config` class** inheriting from `Cog::Config` | Auto-discovered via `find_child_config_or_default` (cog.rb line 30). Falls back to base `Cog::Config` (empty, no validation). | +| 5 | **Nested `Output` class** inheriting from `Cog::Output` | **NOT auto-discovered** — unlike Config and Input, there is no `find_child_output_or_default`. Your `execute` method constructs and returns the Output directly. | + +### The Auto-Discovery Asymmetry + +This is a critical design point that trips people up: + +``` + Auto-Discovered? Who Creates It? +Config ✅ Yes Framework (ConfigManager) +Input ✅ Yes Framework (Cog#run!, line 78) +Output ❌ No Your code (execute method) +``` + +**Why the asymmetry?** Config and Input are constructed by framework machinery before your code runs. The framework needs to find the right class to instantiate. Output is constructed by your `execute` method — you're already running, you know what to return. + +The auto-discovery mechanism uses `const_defined?` and `const_get` on the string `"#{cog_class.name}::Config"` (or `::Input`). The nested class must be defined **inside** your cog class to be found. + +--- + +## 2. Config: The `@values` Hash Contract + +### The Golden Rule + +**ALL configuration values MUST be stored in `@values`.** This is not a suggestion — the merge cascade depends on it. + +The `merge` method (config.rb line 47): + +```ruby +def merge(config_object) + self.class.new(values.merge(config_object.values)) +end +``` + +If you store a value in `@temperature` instead of `@values[:temperature]`, it will be **silently lost** when ConfigManager runs the 4-layer cascade (global → general → regexp → name-specific). Only `@values` survives the merge. + +### The `field` Macro + +For simple get/set fields with defaults, use the `field` class method (config.rb line 110): + +```ruby +class Config < Roast::Cog::Config + field :temperature, 0.7 + field :model, "gpt-4o-mini" do |value| + raise InvalidConfigError, "model must be a string" unless value.is_a?(String) + value # validator return value becomes the stored value + end +end +``` + +This generates two methods: + +1. **`temperature(*args)`** — Getter (no args) or setter (with arg) +2. **`use_default_temperature!`** — Explicitly resets to default + +#### ⚠️ The Falsy Value Pitfall + +The getter implementation (config.rb line 116): + +```ruby +@values[key] || default.deep_dup +``` + +This means: +- If `@values[:enabled] = false`, then `false || default` returns **the default**, not `false` +- If `@values[:enabled] = nil`, same problem +- `0` also triggers this (`0` is falsy in Ruby? No — `0` is truthy in Ruby, but this is still fragile thinking) + +**Actually**: Only `false` and `nil` are falsy in Ruby. So the pitfall affects: +- Boolean fields where `false` is a legitimate set value +- Fields where `nil` means "explicitly set to nil" vs "not configured" + +#### Workaround for Boolean Fields + +Don't use `field`. Use the imperative toggle pattern (same as base `Cog::Config` does for `async!` and `abort_on_failure!`): + +```ruby +def enable! + @values[:enabled] = true +end + +def disable! + @values[:enabled] = false +end + +def enabled? + !!@values[:enabled] +end +``` + +Or for "default true" booleans (like `abort_on_failure?`): + +```ruby +def enabled? + @values.fetch(:enabled, true) +end +``` + +### The `validate!` Method on Config + +`validate!` on Config is called **after** the merge cascade completes — i.e., on the fully-merged config object. This means: +- Your validation sees the final resolved values from all 4 layers +- You can validate combinations ("if provider is :claude, model must be specified") +- The base implementation is a no-op (config.rb line 25) — override only if needed + +### Hash-Style Access + +For simple cases where `field` is too rigid, Config supports direct hash syntax: + +```ruby +config[:my_key] = "value" # []= calls @values[key] = value +config[:my_key] # [] calls @values[key] +``` + +--- + +## 3. Input: The validate!/coerce Lifecycle + +### The Two-Phase Pattern + +The input lifecycle (executed in `Cog#run!`, lines 78–82): + +``` +1. Create Input instance (no-arg constructor) +2. Run input block via instance_exec (user code sets fields on Input + any other prep work) +3. Call validate! + ├── If passes: proceed to execute + └── If raises InvalidInputError: + 4. Call coerce(return_value_from_input_block) + 5. Call validate! again (mandatory — must pass or workflow crashes) +``` + +This is implemented in `coerce_and_validate_input!` (cog.rb line 149): + +```ruby +def coerce_and_validate_input!(input, return_value) + input.validate! +rescue Cog::Input::InvalidInputError + input.coerce(return_value) + input.validate! +end +``` + +### Implementing `validate!` + +The base class raises `NotImplementedError` — you **must** override. Your implementation should: + +```ruby +def validate! + raise InvalidInputError, "prompt is required" if prompt.nil? && !coerce_ran? +end +``` + +Key pattern: Use `coerce_ran?` to distinguish between the first validation (optimistic — can fail to trigger coercion) and the second validation (final — must accept the coerced state). + +**Why this matters**: Some inputs have fields that can legitimately be `nil` after coercion. Without `coerce_ran?`, the second `validate!` call would reject a valid nil. + +### Implementing `coerce` + +`coerce` is optional. The base implementation just sets `@coerce_ran = true` (input.rb line 49): + +```ruby +def coerce(input_return_value) + @coerce_ran = true +end +``` + +If you override it, **always call `super`** to maintain the `coerce_ran?` flag: + +```ruby +def coerce(input_return_value) + super # Sets @coerce_ran = true + @prompt = input_return_value.to_s if input_return_value +end +``` + +**What is `input_return_value`?** It's the return value from the user's input block: + +```ruby +chat(:analyze) { "What is the meaning of life?" } +# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this string is input_return_value +``` + +Standard cogs use coercion as a convenience mechanism: +- **cmd**: String → `command`, Array → first=`command` + rest=`args` +- **chat**: String → `prompt` +- **agent**: String → `[prompt]`, Array → `prompts` +- **ruby**: anything → `value` + +### The `coerce_ran?` Helper + +Private method (input.rb line 67). Returns `false` until `coerce` is called (which sets `@coerce_ran = true`). Used exclusively by `validate!` to determine whether this is the first or second validation pass. + +--- + +## 4. Output: Mixins and `raw_text` + +### The Output Base Class + +`Cog::Output` is minimal — it has no attrs, no constructor args, and just a private `raw_text` that raises `NotImplementedError` (output.rb line 308). Your custom Output adds whatever fields are appropriate: + +```ruby +class Output < Roast::Cog::Output + attr_reader :result, :metadata + + def initialize(result, metadata) + super() # IMPORTANT: call with empty parens (base has no-arg constructor) + @result = result + @metadata = metadata + end +end +``` + +### Including Mixin Modules + +Three optional modules provide parsing convenience methods: + +| Module | Methods | Requires | +|--------|---------|----------| +| `WithText` | `text`, `lines` | `raw_text` → String | +| `WithJson` | `json`, `json!` | `raw_text` → String? | +| `WithNumber` | `float`, `float!`, `integer`, `integer!` | `raw_text` → String? | + +To use them: + +```ruby +class Output < Roast::Cog::Output + include WithText + include WithJson + include WithNumber + + attr_reader :response + + def initialize(response) + super() + @response = response + end + + private + + def raw_text + @response + end +end +``` + +### The `raw_text` Contract + +All three modules call `raw_text` to get the source string for parsing. It's defined as a private method in the base Output class (raises `NotImplementedError`). If you include ANY mixin, you must implement `raw_text`. + +**Important**: The base `Output` class defines `raw_text` at line 308. Each mixin also declares it (e.g., `WithText` at line 297, `WithJson` at line 48). Ruby's method resolution means the **last include wins** for the abstract declaration, but since they all delegate to the same private method you define, this is fine in practice. Just implement one `raw_text` in your Output class. + +### JSON Candidate Priority + +`WithJson` uses a sophisticated multi-strategy extraction (output.rb lines 68–76): + +1. Entire input string (stripped) +2. ` ```json ` code blocks (**last first**) +3. ` ``` ` code blocks with no language (**last first**) +4. ` ```type ` code blocks with any other language (**last first**) +5. `{ }` or `[ ]` patterns (**longest first**) + +The "last first" strategy exists because LLMs often put their final answer at the end. The method tries each candidate with `JSON.parse` and returns the first that succeeds. + +### Number Candidate Priority + +`WithNumber` uses bottom-up scanning (output.rb lines 230–249): + +1. Entire input (stripped) +2. Each line from **bottom up** (stripped, empty lines removed) +3. Number patterns extracted from each line (bottom up, regex: `/-?[\d\s$¢£€¥.,_]+(?:[eE][+-]?\d+)?/`) + +Normalization strips `$¢£€¥,_` and whitespace, then validates the result matches `/\A-?\d+(?:\.\d*)?(?:[eE][+-]?\d+)?\z/`. + +--- + +## 5. Loading Custom Cogs + +### The `use` Directive + +`use` is called at the top level of a workflow file, and runs during `extract_dsl_procs!` (workflow.rb line 134) — **before** `prepare!`. This ensures custom cogs are registered and bound alongside built-in cogs. + +### Local Loading (no `from:`) + +```ruby +use "local" +``` + +Resolution path: +``` +workflow_path.realdirpath.dirname.join("cogs/local").to_s +``` + +So if your workflow is at `/app/workflows/my_flow.rb`, the cog must be at `/app/workflows/cogs/local.rb`. + +**Constraints**: +- The directory is always `cogs/` — not configurable +- The file must define the class at the top level (or use the full class name in `use`) + +### Gem Loading (with `from:`) + +```ruby +use "simple", from: "plugin_gem_example" +``` + +Steps (workflow.rb lines 106–125): +1. `require "plugin_gem_example"` — loads the gem's entry point +2. `"simple".camelize` → `"Simple"` +3. `Object.const_defined?("Simple")` — check existence +4. `"Simple".constantize` → resolves to the class +5. `Simple < Roast::Cog` — validate inheritance +6. `@cog_registry.use(Simple)` — register + +### Multi-Name Loading + +```ruby +use "simple", "MyCogNamespace::Other", from: "plugin_gem_example" +``` + +The gem is `require`d **once**. Each name is resolved and registered separately. This is efficient for gems that provide multiple cog types. + +### Name Derivation Algorithm + +`Cog::Registry#create_registration` (registry.rb line 65): + +```ruby +cog_class_name.demodulize.underscore.to_sym +``` + +Examples: +- `Simple` → `:simple` +- `MyCogNamespace::Other` → `:other` +- `MyCogs::MyCustomCog` → `:my_custom_cog` +- `Roast::Cogs::Chat` → `:chat` + +**Critical implication**: `demodulize` drops ALL namespacing. Two classes `A::Widget` and `B::Widget` both derive to `:widget`. Last `use` call wins (overwrites silently). + +### Overwrite Behavior + +`Registry#use` overwrites existing registrations without error (registry.rb line 55): + +```ruby +def use(cog_class) + name, klass = create_registration(cog_class) + cogs[name] = klass # Simple hash assignment — overwrites +end +``` + +This is by design: +- A gem can override a built-in (register a custom `:cmd` that replaces the standard one) +- The test suite explicitly validates this behavior + +### Name Collision With Ruby Methods + +The name collision check happens **at bind time** (during `prepare!`), not during `use`: + +```ruby +# config_manager.rb line 92 +raise IllegalCogNameError, cog_method_name if respond_to?(cog_method_name, true) + +# execution_manager.rb line 193 +raise IllegalCogNameError, cog_method_name if respond_to?(cog_method_name, true) +``` + +The `true` argument to `respond_to?` includes private methods. This catches names like `:freeze`, `:hash`, `:class`, `:object_id`, etc. + +**Consequence**: You can `use` a cog named `:freeze` without error — it only fails when `prepare!` tries to bind it to the contexts. The error is `ConfigManager::IllegalCogNameError` or `ExecutionManager::IllegalCogNameError` (two separate, unrelated classes with the same base name). + +--- + +## 6. The Plugin Gem Pattern + +### Minimal Gem Structure + +``` +my-cog-gem/ +├── lib/ +│ ├── my_cog_gem.rb # Entry point (require'd by `use ... from:`) +│ └── my_widget.rb # class MyWidget < Roast::Cog +├── my-cog-gem.gemspec +└── Gemfile +``` + +### Entry Point (`lib/my_cog_gem.rb`) + +Must make the cog class available in the global namespace: + +```ruby +# frozen_string_literal: true + +require "my_widget" +``` + +That's it. No special registration, no plugin API. The `use` directive handles registration after the `require`. + +### Gemspec + +Add `roast-ai` as a dependency: + +```ruby +Gem::Specification.new do |spec| + spec.name = "my-cog-gem" + # ... + spec.add_dependency("roast-ai") +end +``` + +### Workflow Usage + +```ruby +use "my_widget", from: "my_cog_gem" + +config do + my_widget { temperature 0.5 } +end + +execute do + my_widget(:thing) { |my| my.value = "hello" } +end +``` + +### Multiple Cogs Per Gem + +```ruby +# lib/my_cog_gem.rb +require "my_widget" +require "my_other_cog" + +# workflow.rb +use "my_widget", "my_other_cog", from: "my_cog_gem" +``` + +### Namespaced Cogs + +```ruby +# lib/other.rb +module MyCogNamespace + class Other < Roast::Cog + class Input < Roast::Cog::Input + def validate!; end + end + + def execute(input) + # ... + end + end +end + +# workflow.rb — use the full class name string: +use "MyCogNamespace::Other", from: "my_cog_gem" +# DSL name becomes: :other (demodulized + underscored) +``` + +--- + +## 7. Complete Custom Cog Example + +Here's a fully-realized custom cog demonstrating all contract points: + +```ruby +# typed: true +# frozen_string_literal: true + +class HttpFetch < Roast::Cog + class Config < Roast::Cog::Config + field :timeout, 30 do |value| + raise InvalidConfigError, "timeout must be positive" unless value.is_a?(Integer) && value > 0 + value + end + + def follow_redirects! + @values[:follow_redirects] = true + end + + def no_follow_redirects! + @values[:follow_redirects] = false + end + + def follow_redirects? + @values.fetch(:follow_redirects, true) + end + + def validate! + # Called after the full merge cascade + end + end + + class Input < Roast::Cog::Input + attr_accessor :url, :method_name, :headers + + def validate! + raise InvalidInputError, "url is required" if url.nil? && !coerce_ran? + end + + def coerce(input_return_value) + super # Sets @coerce_ran = true + case input_return_value + when String + @url = input_return_value + when Hash + @url = input_return_value[:url] + @method_name = input_return_value[:method] + @headers = input_return_value[:headers] + end + end + end + + class Output < Roast::Cog::Output + include WithText + include WithJson + include WithNumber + + attr_reader :body, :status_code, :response_headers + + def initialize(body, status_code, response_headers) + super() + @body = body + @status_code = status_code + @response_headers = response_headers + end + + private + + def raw_text + @body + end + end + + protected + + def execute(input) + # @config is set by the framework before execute runs + # Use @config to access merged configuration + response = perform_request( + url: input.url, + method: input.method_name || :get, + headers: input.headers || {}, + timeout: @config.timeout, + follow_redirects: @config.follow_redirects? + ) + Output.new(response.body, response.status, response.headers) + end + + private + + def perform_request(url:, method:, headers:, timeout:, follow_redirects:) + # Implementation here + end +end +``` + +Usage in a workflow: + +```ruby +use "http_fetch" # Loads from cogs/http_fetch.rb + +config do + http_fetch { timeout 60 } + http_fetch(:api_call) { no_follow_redirects! } +end + +execute do + http_fetch(:api_call) { "https://api.example.com/data" } + ruby(:result) { |my| my.value = http_fetch!(:api_call).json } +end +``` + +--- + +## 8. Testing Custom Cogs + +### The `run_cog` Helper + +Defined in `test/test_helper.rb` (line 117), this provides the complete async execution harness: + +```ruby +def run_cog(cog, config: nil, scope_value: nil, scope_index: 0) + config ||= cog.class.config_class.new + + Sync do + barrier = Async::Barrier.new + input_context = Roast::CogInputContext.new + Fiber[:path] = [Roast::TaskContext::PathElement.new(execution_manager: mock_execution_manager)] + + cog.run!(barrier, config, input_context, scope_value, scope_index) + barrier.wait + end + + cog +end +``` + +This sets up: +- An `Async::Barrier` (for the cooperative scheduling) +- A bare `CogInputContext` (no cog accessors bound — just control flow) +- A mock `TaskContext` path element (for event attribution) +- Runs the full `cog.run!` lifecycle and waits for completion + +### Writing Tests for a Custom Cog + +```ruby +require "test_helper" +require_relative "../path/to/http_fetch" + +class HttpFetchTest < ActiveSupport::TestCase + setup do + input_proc = proc { |my| my.url = "https://example.com" } + @cog = HttpFetch.new(:test_fetch, input_proc) + end + + test "successful fetch returns output with body" do + # Stub your external dependency + HttpFetch.any_instance.stubs(:perform_request).returns( + OpenStruct.new(body: '{"key":"value"}', status: 200, headers: {}) + ) + + run_cog(@cog) + + assert @cog.succeeded? + assert_equal 200, @cog.output.status_code + assert_equal({ key: "value" }, @cog.output.json) + end + + test "config cascade applies" do + config = HttpFetch::Config.new + config.timeout(60) + config.no_follow_redirects! + + HttpFetch.any_instance.stubs(:perform_request).returns( + OpenStruct.new(body: "ok", status: 200, headers: {}) + ) + + run_cog(@cog, config: config) + + assert @cog.succeeded? + end + + test "missing url fails validation" do + empty_proc = proc { } + cog = HttpFetch.new(:fail_fetch, empty_proc) + + assert_raises(Roast::Cog::Input::InvalidInputError) do + run_cog(cog) + end + end +end +``` + +### The TestCog Reference Implementation + +`test/support/test_cog.rb` provides the canonical minimal custom cog used across the test suite: + +```ruby +module TestCogSupport + class TestInput < Roast::Cog::Input + attr_accessor :value + + def validate! + raise InvalidInputError if value.nil? && !coerce_ran? + end + + def coerce(input_return_value) + super + @value = input_return_value + end + end + + class TestOutput < Roast::Cog::Output + attr_reader :value + + def initialize(value) + super() + @value = value + end + end + + class TestCog < Roast::Cog + class Config < Roast::Cog::Config; end + class Input < TestInput; end + + def execute(input) + TestOutput.new(input.value) + end + end +end +``` + +Key patterns to copy: +- `validate!` uses `coerce_ran?` — strict first, lenient after coercion +- `coerce` calls `super` — maintains the `coerce_ran?` flag +- `Output#initialize` calls `super()` — empty parens (base has no-arg constructor) +- `execute` returns an Output instance — never a raw value + +--- + +## 9. Gotchas and Edge Cases + +### 1. execute Must Return an Output (or nil) + +The framework stores whatever `execute` returns in `@output` (cog.rb line 83). If you return `nil`, `succeeded?` returns `false` (because it checks `@output != nil`). No error is raised — the cog just appears to have not completed. + +If you return something that isn't a `Cog::Output`, things will work until someone calls a method on the output that doesn't exist (like `.text` or `.json`), at which point you get a `NoMethodError`. + +### 2. Config Values Lost Outside @values + +```ruby +# WRONG — value lost during merge cascade +class Config < Roast::Cog::Config + attr_accessor :temperature +end + +# RIGHT — value preserved during merge +class Config < Roast::Cog::Config + field :temperature, 0.7 +end + +# ALSO RIGHT — manual @values storage +class Config < Roast::Cog::Config + def temperature(value = :_no_arg_) + if value == :_no_arg_ + @values[:temperature] || 0.7 + else + @values[:temperature] = value + end + end +end +``` + +### 3. Anonymous Classes Crash + +```ruby +# This raises CouldNotDeriveCogNameError +klass = Class.new(Roast::Cog) +registry.use(klass) # Error! klass.name is nil +``` + +Always use named classes. + +### 4. Namespace Collision Is Silent + +```ruby +use "namespace_a/widget", from: "gem_a" # registers as :widget +use "namespace_b/widget", from: "gem_b" # OVERWRITES :widget — no warning +``` + +The last `use` wins. There is no duplicate detection or warning. + +### 5. Reserved Names + +Any name that would collide with an existing method on `ConfigContext` or `ExecutionContext` will fail at bind time. Reserved names include all Object instance methods: `freeze`, `hash`, `class`, `object_id`, `send`, `dup`, `clone`, `nil?`, `tap`, `then`, `is_a?`, etc. + +Additionally, `global`, `outputs`, and `outputs!` are pre-bound by the managers. + +### 6. The `validate!` Returning `true` Pattern + +You'll see this in simple examples: + +```ruby +def validate! + true +end +``` + +This is valid — it means "all inputs are always valid, never trigger coercion." The framework only cares about whether `validate!` raises, not its return value. + +### 7. Output Mixin Memoization + +Both `WithJson` and `WithNumber` memoize their results: + +```ruby +@json ||= parse_json_with_fallbacks(input) +@float ||= parse_number_with_fallbacks(raw_text || "") +``` + +This means calling `.json` multiple times returns the same parsed result. But it also means if `raw_text` changes after first access (unlikely but possible), the cached value is stale. + +### 8. The `@config` Variable + +In `Cog#run!` (line 77), `@config = config` overwrites the default config created in the constructor (line 57). Your `execute` method always sees the fully-merged config. You never need to merge configs yourself — the framework has already done it. + +--- + +## 10. Checklist for Contributors + +When creating a new custom cog, verify: + +- [ ] Class has a proper name (not anonymous) +- [ ] Class inherits from `Roast::Cog` +- [ ] `Input` class is nested inside the cog class (for auto-discovery) +- [ ] `Input#validate!` is implemented (raises `InvalidInputError` on failure) +- [ ] `Input#coerce` calls `super` if overridden +- [ ] `Config` class stores all values in `@values` (not instance variables) +- [ ] `Config` class is nested inside the cog class (for auto-discovery) +- [ ] `Output#initialize` calls `super()` with empty parens +- [ ] `Output` implements `raw_text` if any mixin is included +- [ ] `execute` returns an Output instance (not `nil`, not a raw value) +- [ ] DSL name doesn't collide with Object methods or other registered cogs +- [ ] If using namespaces, you understand that only the demodulized name is used in the DSL + +--- + +## See Also + +- [01 Architecture Overview](01-architecture-overview.md) — The three evaluation contexts and cog lifecycle +- [03 Cog Reference](03-cog-reference.md) — Complete reference for all built-in cogs (as implementation examples) +- [05 Execution Engine Internals](05-execution-engine-internals.md) — How ConfigManager and ExecutionManager bind and run cogs +- [06 Metaprogramming Map](06-metaprogramming-map.md) — How `define_singleton_method` binds cog methods to contexts +- [12 Known Issues & Gotchas](12-known-issues-and-gotchas.md) — The falsy value pitfall and other fragilities diff --git a/internal/documentation/architecture/11-testing-guide.md b/internal/documentation/architecture/11-testing-guide.md new file mode 100644 index 00000000..9c02aeb5 --- /dev/null +++ b/internal/documentation/architecture/11-testing-guide.md @@ -0,0 +1,766 @@ +# Document 11: Testing Guide + +> **Audience**: Intern (primary), AI coding agents (secondary) +> **Purpose**: How to write and run tests for the Roast framework +> **Roast version**: 1.1.0 + +--- + +## 1. Test Stack + +| Dependency | Role | Require | +|------------|------|---------| +| `minitest ~> 5.0` | Test framework | `minitest/autorun` | +| `active_support` | `ActiveSupport::TestCase` base class | `active_support/test_case` | +| `mocha` | Mocking/stubbing (auto-verifying) | `mocha/minitest` | +| `simplecov` | Code coverage | `simplecov` (loaded first) | +| `vcr` | HTTP interaction recording/replay | `vcr` | +| `webmock` | HTTP stubbing (VCR hook) | `webmock` | +| `minitest-rg` | Colorized test output | `minitest/rg` | +| `guard-minitest` | Watch-mode auto-runner (development) | Optional | + +All test dependencies live in the Gemfile's development group. The entry point is `test/test_helper.rb`. + +--- + +## 2. Running Tests + +### Rake Tasks + +| Command | What It Does | +|---------|--------------| +| `rake minitest_fast` | Run all tests, skipping those marked `slow_test!` | +| `rake minitest_all` | Run all tests including slow tests (`ROAST_RUN_SLOW_TESTS=true`) | +| `rake test` | Alias for `minitest_all` | +| `rake rubocop` | Run RuboCop with autocorrect | +| `rake rubocop_ci` | Run RuboCop without autocorrect (CI mode) | +| `rake sorbet` | Run Sorbet type checker (`bin/srb tc`) | +| `rake` (default) | Runs `sorbet` → `rubocop` → `minitest_fast` | +| `rake check` | Runs `sorbet` → `rubocop` only (no tests) | + +**Source**: `Rakefile` (lines 1–56) + +### Running Individual Tests + +```bash +# Single test file +ruby -Itest -Ilib test/roast/cog_test.rb + +# Single test by name +ruby -Itest -Ilib test/roast/cog_test.rb -n "test_started?_returns_false_before_execution" + +# Pattern match +ruby -Itest -Ilib test/roast/cog_test.rb -n "/started/" +``` + +### Environment Variables + +| Variable | Effect | +|----------|--------| +| `ROAST_RUN_SLOW_TESTS=1` | Enable slow tests (otherwise skipped) | +| `RECORD_VCR=true` | Record real HTTP responses (uses real API keys) | +| `PRESERVE_SANDBOX=1` | Keep functional test tmpdirs after test run | +| `CI=true` | Enable colorized output, dump stdout/stderr on functional tests | +| `ROAST_LOG_LEVEL` | Set logger level during test execution | + +--- + +## 3. Test Directory Structure + +``` +test/ +├── test_helper.rb # All shared infrastructure +├── support/ +│ ├── improved_assertions.rb # Custom assertion extensions +│ └── test_cog.rb # Reference test cog implementation +├── roast/ # Unit tests (mirror lib/roast/) +│ ├── cog_test.rb # Roast::Cog base class +│ ├── cog/ # Cog infrastructure +│ │ ├── config_test.rb +│ │ ├── input_test.rb +│ │ ├── output_test.rb +│ │ ├── registry_test.rb +│ │ ├── stack_test.rb +│ │ └── store_test.rb +│ ├── cogs/ # Standard cogs +│ │ ├── cmd_test.rb +│ │ ├── ruby_test.rb +│ │ ├── chat/ +│ │ │ ├── config_test.rb +│ │ │ ├── input_test.rb +│ │ │ ├── output_test.rb +│ │ │ └── session_test.rb +│ │ └── agent/ +│ │ ├── config_test.rb +│ │ ├── input_test.rb +│ │ ├── output_test.rb +│ │ ├── provider_test.rb +│ │ ├── stats_test.rb +│ │ ├── usage_test.rb +│ │ └── providers/ +│ │ ├── claude_test.rb +│ │ ├── claude/ +│ │ │ ├── claude_invocation_test.rb +│ │ │ ├── message_test.rb +│ │ │ ├── tool_result_test.rb +│ │ │ ├── tool_use_test.rb +│ │ │ └── messages/ (9 message type tests) +│ │ ├── pi_test.rb +│ │ └── pi/ +│ │ ├── pi_invocation_test.rb +│ │ └── messages/ (2 message type tests) +│ ├── system_cogs/ # System cogs +│ │ ├── call_test.rb +│ │ ├── map_test.rb +│ │ └── repeat_test.rb +│ ├── execution_manager_test.rb # Core managers +│ ├── config_manager_test.rb +│ ├── cog_input_manager_test.rb +│ ├── cog_input_context_test.rb +│ ├── workflow_test.rb # Workflow lifecycle +│ ├── cli_test.rb # CLI unit tests +│ ├── cli_e2e_test.rb # CLI end-to-end +│ ├── command_runner_test.rb # Shell boundary +│ ├── event_test.rb # Event pipeline +│ ├── event_monitor_test.rb +│ ├── output_router_test.rb +│ ├── task_context_test.rb +│ ├── log_test.rb +│ ├── log_formatter_test.rb +│ └── system_cog_test.rb +├── examples/ # Functional tests +│ ├── support/ +│ │ └── functional_test.rb # FunctionalTest base class +│ └── functional/ +│ └── roast_examples_test.rb # 29 workflow integration tests +└── fixtures/ + ├── agent_transcripts/ # 5 .stdout.txt files + │ ├── agent_with_multiple_prompts_0.stdout.txt + │ ├── agent_with_multiple_prompts_1.stdout.txt + │ ├── agent_with_multiple_prompts_2.stdout.txt + │ ├── simple_agent.stdout.txt + │ └── simple_pi_agent.stdout.txt + └── vcr_cassettes/ # 2 HTTP recording files + ├── simple_chat.yml + └── temperature.yml +``` + +**Total**: 60 test files, including 2 support modules, 1 base class, and 57 actual test files. + +--- + +## 4. Test Helper Infrastructure + +All shared test infrastructure lives in `test/test_helper.rb` (248 lines). Everything is globally available to all test files. + +### 4.1 CaptureLogOutput (Concern) + +```ruby +# Included in ALL tests via ActiveSupport::TestCase.include(CaptureLogOutput) +module CaptureLogOutput + # setup: redirects Roast::Log.logger to a StringIO + # teardown: on failure, dumps captured output to $stderr; always calls Log.reset! +end +``` + +**What it does**: Every test gets its own isolated logger. Test output stays clean on success. On failure, the full log is dumped to stderr to aid debugging. + +**Access captured output**: `@logger_output.string` within any test. + +**Source**: `test/test_helper.rb` lines 34–56 + +### 4.2 Global Helper Methods + +| Method | Signature | Purpose | +|--------|-----------|---------| +| `slow_test!` | `() -> void` | Skip unless `ROAST_RUN_SLOW_TESTS` is set | +| `with_log_level` | `(Integer) { -> T } -> T` | Temporarily change logger level | +| `with_env` | `(String, String) { -> T } -> T` | Temporarily set an env var | +| `mock_execution_manager` | `(scope:, scope_index:, workflow_context:) -> Mock` | Stub EM with configurable scope | +| `create_workflow_context` | `(targets:, args:, kwargs:, tmpdir:, workflow_dir:) -> WorkflowContext` | Factory with handy defaults | +| `run_cog` | `(Cog, config:, scope_value:, scope_index:) -> Cog` | Full async execution harness | +| `use_command_runner_fixtures` | `(*Hash) -> void` | Sequential CommandRunner stub chain | +| `load_command_runner_fixture_file` | `(String, Symbol) -> String` | Load fixture by name + stream | +| `original_streams_from_logger_output` | `(logger_output:) -> [String, String]` | Reconstruct stdout/stderr from log markers | + +### 4.3 `run_cog` — The Integration Test Helper + +This is the most important helper. It runs a single cog through the **full async execution path**: + +```ruby +def run_cog(cog, config: nil, scope_value: nil, scope_index: 0) + config ||= cog.class.config_class.new + + Sync do + barrier = Async::Barrier.new + input_context = Roast::CogInputContext.new + Fiber[:path] = [Roast::TaskContext::PathElement.new(execution_manager: mock_execution_manager)] + + cog.run!(barrier, config, input_context, scope_value, scope_index) + barrier.wait + end + + cog +end +``` + +**What it provides**: +1. An `Async::Barrier` (required by `cog.run!`) +2. A bare `CogInputContext` (no bound cog accessors — input blocks see nothing) +3. A fiber-local `[:path]` with a mock PathElement (required by TaskContext) +4. Blocks until the cog completes (`barrier.wait`) +5. Returns the cog for state/output inspection + +**When to use it**: Unit-testing a cog's config → input → execute → output lifecycle in isolation. The cog runs as it would in production (inside an async barrier) but without the full ExecutionManager/ConfigManager/CogInputManager orchestration. + +**When NOT to use it**: When testing interactions between cogs (output chaining, control flow propagation, scope value passing). Use functional tests for those. + +**Source**: `test/test_helper.rb` lines 117–130 + +### 4.4 `use_command_runner_fixtures` — Sequential Stub Replay + +Sets up a Mocha stub for `CommandRunner.execute` that serves fixture files in order. Each invocation of CommandRunner gets the next fixture in sequence. + +```ruby +use_command_runner_fixtures( + { + fixture: "agent_transcripts/simple_agent", # Required: fixture name + exit_code: 0, # Optional (default: 0) + expected_args: ["claude", "-p", ...], # Optional: assert args match + expected_working_directory: Pathname("/tmp"), # Optional: assert working dir + expected_timeout: 30, # Optional: assert timeout + expected_stdin_content: "Hello", # Optional: assert stdin + }, + { fixture: "agent_transcripts/second_call" }, # Second invocation fixture +) +``` + +**How it works**: +1. Pre-loads all fixture files (`.stdout.txt`, `.stderr.txt`) +2. Creates mock `Process::Status` objects +3. Stubs `CommandRunner.execute` with a `with` block that: + - Asserts call count doesn't exceed fixture count + - Optionally asserts argument expectations per invocation + - Replays stdout/stderr line-by-line through the handler callbacks +4. Chains `.returns().then.returns()` for sequential return values + +**Fixture file resolution**: For fixture name `"agent_transcripts/simple_agent"`: +- Tries `test/fixtures/agent_transcripts/simple_agent.stdout.txt`, then `.stdout.log` +- Tries `test/fixtures/agent_transcripts/simple_agent.stderr.txt`, then `.stderr.log` +- Returns empty string if neither exists + +**Source**: `test/test_helper.rb` lines 174–215 + +### 4.5 `original_streams_from_logger_output` — Stream Reconstruction + +Parses the captured log output to separate original stdout and stderr content: + +```ruby +stdout, stderr = original_streams_from_logger_output +# or with explicit input: +stdout, stderr = original_streams_from_logger_output(logger_output: some_string) +``` + +**How it works**: The EventMonitor logs stdout lines with a `❯` marker and stderr lines with `❯❯`. This method: +1. Scans lines for the log prefix pattern (`/^[DIWEFA], \[/`) +2. Identifies `❯❯` lines as stderr, `❯` lines as stdout +3. Continuation lines (without log prefix) belong to the current stream +4. Returns `[stdout_string, stderr_string]` + +**When to use**: Functional tests where you need to verify what a workflow actually printed. Direct `capture_io` output should be empty (all output goes through EventMonitor), so this reconstructs what the user would have seen. + +**Source**: `test/test_helper.rb` lines 136–160 + +--- + +## 5. Test Patterns by Cog Type + +### 5.1 Unit Testing a Config Class + +```ruby +module Roast + module Cogs + class Cmd < Cog + class ConfigTest < ActiveSupport::TestCase + def setup + @config = Config.new + end + + test "fail_on_error? returns true by default" do + assert @config.fail_on_error? + end + + test "no_fail_on_error! sets fail_on_error to false" do + @config.no_fail_on_error! + refute @config.fail_on_error? + end + end + end + end +end +``` + +**Pattern**: Create a fresh config in `setup`, test each setter/getter pair, verify defaults. + +### 5.2 Unit Testing a Cog with `run_cog` + +```ruby +test "successful execution sets output" do + cog = TestCog.new(:my_cog, ->(_input, _scope, _idx) { "hello" }) + run_cog(cog) + + assert cog.succeeded? + assert_equal "hello", cog.output.value +end +``` + +**Pattern**: Instantiate cog with a name and input proc, call `run_cog`, assert state and output. + +### 5.3 Testing cmd Cogs with Fixtures + +```ruby +test "cmd cog captures stdout" do + use_command_runner_fixtures( + { fixture: "my_test_fixture", exit_code: 0 } + ) + + # Create and run the cmd cog... +end +``` + +**Pattern**: Set up fixtures → run cog → verify output fields (`.out`, `.err`, `.status`). + +### 5.4 Testing chat Cogs with VCR + +```ruby +test "simple chat completes" do + in_sandbox :simple_chat do + Roast::Workflow.from_file("examples/simple_chat.rb", EMPTY_PARAMS) + end +end +``` + +**Pattern**: Wrap in `in_sandbox` which activates the VCR cassette. The cassette name matches the workflow_id passed to `in_sandbox`. + +### 5.5 Testing agent Cogs with Transcript Fixtures + +```ruby +test "agent_with_multiple_prompts.rb workflow runs successfully" do + use_command_runner_fixtures( + { fixture: "agent_transcripts/agent_with_multiple_prompts_0", expected_args: [...], expected_stdin_content: "What is 2+2?" }, + { fixture: "agent_transcripts/agent_with_multiple_prompts_1", expected_args: [...], expected_stdin_content: "Now multiply that by 3" }, + { fixture: "agent_transcripts/agent_with_multiple_prompts_2", expected_args: [...], expected_stdin_content: "Now subtract 5" }, + ) + + stdout, stderr = in_sandbox :simple_agent do + Roast::Workflow.from_file("examples/agent_with_multiple_prompts.rb", EMPTY_PARAMS) + end + + assert_empty stdout + assert_empty stderr + + logged_stdout, _logged_stderr = original_streams_from_logger_output + assert_equal expected_output, logged_stdout +end +``` + +**Pattern**: Agent cogs shell out to `claude` CLI (or `pi`). The CommandRunner fixtures replay pre-recorded CLI stdout/stderr. Each prompt in a multi-prompt agent gets its own sequential fixture. Assertions verify: +1. Direct stdout/stderr is empty (all output routed through EventMonitor) +2. Reconstructed logged output matches expected content + +--- + +## 6. Functional Tests (FunctionalTest Base Class) + +**Source**: `test/examples/support/functional_test.rb` (60 lines) + +### 6.1 The `in_sandbox` Method + +```ruby +def in_sandbox(workflow_id, &block) +``` + +Creates an isolated test environment: + +1. **Creates a temporary directory** under `tmp/sandboxes/` +2. **Copies all examples** into the sandbox (`FileUtils.cp_r`) +3. **Sets up VCR**: Uses cassette matching `workflow_id.to_s` +4. **Configures credentials**: Real keys if `RECORD_VCR=true`, fake keys otherwise +5. **Captures IO**: Wraps block in `capture_io` +6. **Scrubs paths**: Replaces tmpdir with `/fake-testing-dir` for stable assertions +7. **CI dump**: If `CI=true`, prints raw stdout/stderr for debugging +8. **Returns**: `[stdout_string, stderr_string]` + +### 6.2 Sandbox Preservation + +When `PRESERVE_SANDBOX=1` is set, the tmpdir is **not cleaned up**. This lets you inspect the state after a test: + +```bash +PRESERVE_SANDBOX=1 ruby -Itest -Ilib test/examples/functional/roast_examples_test.rb -n "/simple_chat/" +ls tmp/sandboxes/simple_chat*/ +``` + +### 6.3 Recording New VCR Cassettes + +```bash +RECORD_VCR=true OPENAI_API_KEY=sk-real-key ruby -Itest -Ilib test/examples/functional/roast_examples_test.rb -n "/simple_chat/" +``` + +VCR will: +- Use the real API key (from environment) +- Record all HTTP interactions to `test/fixtures/vcr_cassettes/simple_chat.yml` +- Filter sensitive data (Authorization headers, cookies, URIs) + +### 6.4 Writing a New Functional Test + +1. Create an example workflow in `examples/my_workflow.rb` +2. If it uses `agent`: create fixture files in `test/fixtures/agent_transcripts/` +3. If it uses `chat`: record a VCR cassette with `RECORD_VCR=true` +4. Add the test to `test/examples/functional/roast_examples_test.rb`: + +```ruby +test "my_workflow.rb runs successfully" do + # Set up fixtures if needed + use_command_runner_fixtures(...) # for agent cogs + + stdout, stderr = in_sandbox :my_workflow do + Roast::Workflow.from_file("examples/my_workflow.rb", EMPTY_PARAMS) + end + + assert_empty stdout + assert_empty stderr + + logged_stdout, logged_stderr = original_streams_from_logger_output + # Assert expected output... +end +``` + +--- + +## 7. The TestCog Reference Implementation + +**Source**: `test/support/test_cog.rb` (35 lines) + +```ruby +module TestCogSupport + class TestInput < Roast::Cog::Input + attr_accessor :value + + def validate! + raise InvalidInputError if value.nil? && !coerce_ran? + end + + def coerce(input_return_value) + super # Sets @coerce_ran = true + @value = input_return_value + end + end + + class TestOutput < Roast::Cog::Output + attr_reader :value + + def initialize(value) + super() + @value = value + end + end + + class TestCog < Roast::Cog + class Config < Roast::Cog::Config; end + class Input < TestInput; end + + def execute(input) + TestOutput.new(input.value) + end + end +end +``` + +This is the **minimal complete implementation** of the cog contract. Use it as a template when: +- Writing tests that need a generic cog +- Creating a new custom cog (copy this structure) +- Understanding the Input validate!/coerce two-phase pattern + +Key points: +- `validate!` raises `InvalidInputError` when value is nil AND coercion hasn't run +- `coerce` calls `super` (mandatory — sets `coerce_ran?` flag), then sets value +- `execute` receives the validated Input, returns an Output instance +- Config is empty (inherits all base behavior) + +--- + +## 8. Custom Assertions + +**Source**: `test/support/improved_assertions.rb` (54 lines) + +### `assert_predicate_with_args` + +Extends Minitest's `assert_predicate` to accept arguments: + +```ruby +# Standard Minitest (no args): +assert_predicate cog, :started? + +# Extended (with args): +assert_predicate store, :include?, :my_cog +``` + +The override dispatches to the original `assert_predicate` when no args are provided, or to the extended version when args are present. + +### No `assert_received` + +Mocha provides **automatic mock verification**. If you set up an expectation (`expects(:method)`), Mocha verifies it was called during teardown. No manual `assert_received` needed. + +--- + +## 9. Type System (Sorbet) + +### Configuration + +| Setting | Value | Source | +|---------|-------|--------| +| Sigil level | `typed: true` for ALL 66 lib/ files | Each file's first line | +| `typed: false` files | **0** | — | +| Test files | Excluded from Sorbet | `sorbet/config`: `--ignore=test/` | +| Examples | Excluded from Sorbet | `sorbet/config`: `--ignore=examples/demo` | +| Runtime dependency | `type_toolkit >= 0.0.5` (NOT `sorbet-runtime`) | `roast-ai.gemspec` | +| Experimental features | `--enable-experimental-requires-ancestor`, `--enable-experimental-rbs-comments` | `sorbet/config` | + +### Running Sorbet + +```bash +bin/srb tc # Direct invocation +rake sorbet # Via Rake task +rake # Part of default task (sorbet + rubocop + minitest_fast) +``` + +### Inline RBS Annotations + +The project uses inline RBS comments (`#:` prefix) for type annotations instead of `sig` blocks: + +```ruby +#: (String, Symbol) -> String +def format_message(text, level) + ... +end + +@cogs = Cog::Store.new #: Cog::Store +``` + +**Stats**: 596 inline RBS annotations across 59 source files. + +### RBI Shim Files + +Three shim files document the dynamically-defined methods: + +| File | Lines | Purpose | +|------|-------|---------| +| `sorbet/rbi/shims/lib/roast/config_context.rbi` | 323 | ConfigContext dynamic methods | +| `sorbet/rbi/shims/lib/roast/execution_context.rbi` | 496 | ExecutionContext dynamic methods | +| `sorbet/rbi/shims/lib/roast/cog_input_context.rbi` | 1,198 | CogInputContext dynamic methods | +| **Total** | **2,017** | — | + +These shims serve dual purpose: Sorbet type information AND canonical API documentation. They contain extensive docstrings, usage examples, and cross-references. + +### `as untyped` Escape Hatch + +7 surgical uses where Sorbet's type system cannot express the actual runtime behavior: + +| File | Line | Reason | +|------|------|--------| +| `execution_manager.rb` | 204 | Suppress unknown-length splat warning | +| `cog/config.rb` | 111 | Default value type variance in field macro | +| `command_runner.rb` | 70 | Open3 complex return type | +| `output_router.rb` | 54 | `self` in singleton method context | +| `cogs/ruby.rb` | 112 | Proc reassignment for type narrowing | +| `cogs/ruby.rb` | 140 | Hash value access | +| `cogs/cmd.rb` | 280 | CommandRunner complex return type | + +### Generated RBS Files + +65 `.rbs` files in `sig/generated/` are auto-generated by `RBS::Inline` from the inline annotations. These stay in sync by re-running the RBS generation tool. + +--- + +## 10. Style Enforcement (RuboCop) + +### Configuration + +**Source**: `.rubocop.yml` + +```yaml +inherit_gem: + rubocop-shopify: rubocop.yml # Shopify house style + +plugins: + - rubocop-sorbet + - type_toolkit: + require_path: rubocop-type_toolkit + +AllCops: + TargetRubyVersion: 3.4 +``` + +### Key Rules + +| Rule | Setting | Effect | +|------|---------|--------| +| `Style/MethodCallWithArgsParentheses` | Enabled (except test/) | Must use parens for method calls with args in lib/ | +| `Sorbet/FalseSigil` | Enabled for `lib/**/*.rb` | Prevents `typed: false` from creeping in | +| `Sorbet/ConstantsFromStrings` | Todolist (4 files) | Legacy workflow files allowed to use `constantize` | +| `Sorbet/SelectByIsA` | Todolist (1 file) | One legacy `select { |x| x.is_a?(Foo) }` | + +### Exclusions + +| Path | Excluded From | +|------|---------------| +| `bin/*` | AllCops | +| `test/fixtures/**/*` | AllCops | +| `examples/**/*` | AllCops | +| `test/**/*.rb` | MethodCallWithArgsParentheses | +| `test/**/*` | Sorbet/FalseSigil | +| `examples/**/*` | Sorbet/FalseSigil | +| `lib/roast/sorbet_runtime_stub.rb` | Sorbet/FalseSigil (**vestigial** — file no longer exists) | + +### Running RuboCop + +```bash +rake rubocop # With autocorrect +rake rubocop_ci # Without autocorrect (CI mode) +rubocop # Direct invocation +rubocop --autocorrect-all # Fix everything possible +``` + +### Current State + +Only 5 offenses in `.rubocop_todo.yml` — all in legacy workflow files that predate the current architecture. The codebase is extremely clean. + +--- + +## 11. Testing Philosophy: Mirrors Architecture + +The test pyramid directly reflects the system's structural hierarchy: + +| Test Level | What It Tests | Tool | Corresponds To | +|------------|---------------|------|----------------| +| **Unit** | Single Config/Input/Output/cog | Direct instantiation | Individual cog contract | +| **Integration** | Cog within async execution | `run_cog` helper | CM + EM + CIM orchestration | +| **Functional** | Full workflow end-to-end | `FunctionalTest` + `in_sandbox` | CLI → `from_file` → `prepare!` → `start!` | + +### Fixture Strategy Reveals Trust Boundaries + +Each boundary gets its own isolation approach: + +| Boundary | Fixture Strategy | Why | +|----------|-----------------|-----| +| Shell (cmd) | `use_command_runner_fixtures` | Deterministic command replay | +| HTTP/API (chat) | VCR cassettes | Full request/response recording | +| Subprocess (agent) | Agent transcript fixtures | CLI stdout/stderr replay | + +### The Sixth Boundary Protection + +The FunctionalTest sandbox (copy to tmpdir + path scrubbing) is itself an instance of the framework's deep-copy-at-boundaries pattern — the same philosophy that drives `deep_dup` on config, output access, event paths, and sessions. + +--- + +## 12. Practical Recipes + +### Recipe: Adding a Unit Test for a New Config Option + +1. Open the corresponding config test file (e.g., `test/roast/cogs/chat/config_test.rb`) +2. Add tests for: default value, setter, negation, validated getter (if applicable) +3. Follow the `setup` → test pairs pattern + +### Recipe: Adding a New Functional Test Workflow + +1. Create `examples/my_new_workflow.rb` +2. Determine fixture needs: + - **cmd only**: No fixtures needed (direct execution in sandbox) + - **agent**: Record CLI output, save as `test/fixtures/agent_transcripts/my_fixture.stdout.txt` + - **chat**: Run once with `RECORD_VCR=true` to generate cassette +3. Add test in `test/examples/functional/roast_examples_test.rb` +4. Assert: `stdout`/`stderr` empty, then verify `original_streams_from_logger_output` + +### Recipe: Debugging a Failing Test + +```bash +# 1. Run with verbose logging +ROAST_LOG_LEVEL=0 ruby -Itest -Ilib test/roast/failing_test.rb + +# 2. Preserve sandbox for inspection +PRESERVE_SANDBOX=1 ruby -Itest -Ilib test/examples/functional/roast_examples_test.rb -n "/my_test/" + +# 3. Check captured log (automatically dumped on failure in test output) + +# 4. For parallel/async issues, add explicit barrier.wait or Sync blocks +``` + +### Recipe: Testing Control Flow + +Control flow exceptions (SkipCog, FailCog, Next, Break) propagate differently in sync vs async contexts. When testing: + +- **SkipCog**: Always swallowed at Layer 1 — test via `cog.skipped?` +- **FailCog**: Test `abort_on_failure?` behavior by setting config +- **Next/Break**: Must test within a system cog context (Map/Repeat) — use functional tests + +### Recipe: Verifying All Checks Pass + +```bash +rake # Runs: sorbet → rubocop → minitest_fast +``` + +This is the standard pre-commit verification. For full coverage including slow tests: + +```bash +rake test # Runs minitest_all (includes slow tests) +rake check && rake test # Full verification +``` + +--- + +## 13. VCR Configuration + +**Source**: `test/test_helper.rb` lines 229–248 + +```ruby +VCR.configure do |config| + config.cassette_library_dir = "test/fixtures/vcr_cassettes" + config.hook_into :webmock + + # Filters — replace real values with stable fakes in recordings + config.filter_sensitive_data("http://mytestingproxy.local/v1/chat/completions") { |i| i.request.uri } + config.filter_sensitive_data("my-token") { |i| i.request.headers["Authorization"].first } + config.filter_sensitive_data("") { |i| i.request.headers["Set-Cookie"] } + config.filter_sensitive_data("") { |i| i.response.headers["Set-Cookie"] } +end +``` + +Key points: +- **Playback mode** (default): Intercepts HTTP, matches against cassette, returns recorded response +- **Record mode** (`RECORD_VCR=true`): Passes through to real API, records everything +- **Sensitive data**: Automatically scrubbed in recordings (API keys, cookies, URIs) +- **Fake credentials**: During playback, env vars are set to dummy values (`"my-token"`, `"http://mytestingproxy.local/v1"`) + +--- + +## 14. Invariants for Test Contributors + +1. **Every test file requires `test_helper`** — No exceptions. This ensures CaptureLogOutput is active. +2. **Tests are namespaced under `Roast` module** — Mirrors the lib/ structure. +3. **All tests inherit from `ActiveSupport::TestCase`** — For setup/teardown hooks and CaptureLogOutput. +4. **EventMonitor.reset! in functional test setup/teardown** — Prevents cross-test contamination. +5. **Never assert on direct stdout/stderr in functional tests** — All output goes through EventMonitor. Use `original_streams_from_logger_output` instead. +6. **Fixture count must match invocation count** — `use_command_runner_fixtures` will fail-fast if CommandRunner is called more times than expected. +7. **VCR cassette name must match sandbox workflow_id** — The `in_sandbox` method uses the same string for both. +8. **Mocha auto-verifies** — Don't manually assert expectations. If `expects` is set, Mocha will fail the test if it's not called. +9. **No sorbet-runtime in tests** — Tests are excluded from Sorbet entirely (`--ignore=test/`). +10. **Path scrubbing makes assertions deterministic** — Always compare against `/fake-testing-dir` in functional test assertions. + +--- + +## See Also + +- [01 Architecture Overview](01-architecture-overview.md) — System structure that tests mirror +- [03 Cog Reference](03-cog-reference.md) — What each cog's Input/Output/Config looks like +- [10 Writing Custom Cogs](10-writing-custom-cogs.md) — TestCog as template, `run_cog` usage +- [12 Known Issues & Gotchas](12-known-issues-and-gotchas.md) — Edge cases that are hard to test diff --git a/internal/documentation/architecture/12-known-issues-and-gotchas.md b/internal/documentation/architecture/12-known-issues-and-gotchas.md new file mode 100644 index 00000000..f3b1dd73 --- /dev/null +++ b/internal/documentation/architecture/12-known-issues-and-gotchas.md @@ -0,0 +1,391 @@ +# Document 12: Known Issues & Gotchas + +> **Purpose**: Critical safety reading before modifying any part of the codebase. Everything that will trip you up — confirmed bugs, fragilities, documentation discrepancies, vestigial artifacts, and naming pitfalls. +> +> **Audience**: Everyone — intern, AI agents, new contributors. Read this before submitting your first PR. + +--- + +## 1. Confirmed Bugs + +### 1.1 Claude Dump Path Is Hardcoded + +**File**: `lib/roast/cogs/agent/providers/claude/message.rb:18` + +```ruby +File.write("./tmp/claude-messages.log", "#{json}\n", mode: "a") if raw_dump_file +``` + +The `raw_dump_file` parameter is accepted but **never used as the write target**. The write always goes to `./tmp/claude-messages.log` regardless of what path is passed. The parameter only controls whether any write occurs at all (truthiness gate). + +**Impact**: Any code passing a custom dump path will silently write to the wrong location. + +--- + +### 1.2 Top-Level Sync `next!` — Unhandled Exception + +**File**: `lib/roast/workflow.rb:68` + +`Workflow.start!` only rescues `ControlFlow::Break`: + +```ruby +rescue ControlFlow::Break +``` + +A synchronous cog that calls `next!` at the top level (not inside a `map` or `repeat`) will raise `ControlFlow::Next` through `cog_task.wait` (line 104 of execution_manager.rb), and since nothing catches it, the exception propagates as an unhandled `StandardError` crash. + +**Workaround**: Use `skip!` instead of `next!` at the top level. Or make the cog `async!` (which swallows `Next` in `wait_for_task_with_exception_handling`). + +--- + +### 1.3 Repeat + Sync `next!` — Escapes Loop Entirely + +**File**: `lib/roast/system_cogs/repeat.rb:230` + +`Repeat::Manager` only rescues `ControlFlow::Break`: + +```ruby +rescue ControlFlow::Break +``` + +A synchronous cog inside a `repeat` scope that calls `next!` will have the exception escape through the EM's sync `cog_task.wait`, then propagate out of `Repeat::Manager` entirely (since it doesn't rescue `Next`). Instead of advancing to the next iteration, it crashes the workflow. + +**Workaround**: Make the cog async, or use `skip!` + conditional logic instead of `next!`. + +--- + +### 1.4 Async `next!` Is Silently Swallowed + +**File**: `lib/roast/execution_manager.rb:150` + +```ruby +rescue ControlFlow::Next + # TODO: do something with the message passed to next! + @barrier.stop +``` + +When an async cog calls `next!`, it's caught by `wait_for_task_with_exception_handling` which stops the barrier but **does not re-raise**. This means `next!` in an async cog within a `map` or `repeat` will stop the current scope's remaining cogs but won't signal the parent to advance to the next iteration. + +**Impact**: The semantic difference between sync `next!` (propagates) and async `next!` (swallowed) is undocumented and surprising. The TODO comment confirms this is a known incomplete implementation. + +--- + +### 1.5 `outputs!` + `break!` — Exception Masking + +**File**: `lib/roast/execution_manager.rb:110–114` (the `ensure` block) + +When a cog calls `break!`: +1. `Break` propagates out of `run!` +2. The `ensure` block calls `compute_final_output` +3. If `outputs!` (strict mode) is configured and tries to access a cog that was skipped/never-ran due to the break, it raises `CogSkippedError` +4. The `CogSkippedError` from `ensure` **replaces** the original `Break` exception + +**Impact**: The workflow appears to crash with a `CogSkippedError` rather than cleanly breaking. The root cause (`break!`) is masked. + +**Workaround**: Use `outputs` (tolerant) instead of `outputs!` in scopes that may use `break!`. + +--- + +### 1.6 Empty Scope Without `outputs` — Crash + +**File**: `lib/roast/execution_manager.rb:264` + +```ruby +raise CogInputManager::CogDoesNotExistError, "no cogs defined in scope" unless last_cog_name +``` + +If a scope contains zero cogs and no `outputs` block is defined, `compute_final_output` raises `CogDoesNotExistError`. This error is NOT caught by any internal rescue clause and crashes the workflow. + +**Impact**: An empty `execute(:scope_name) do; end` followed by `call(run: :scope_name)` will crash. + +--- + +## 2. Fragilities + +### 2.1 Chat Accesses RubyLLM Internals + +**File**: `lib/roast/cogs/chat.rb:55` + +```ruby +temperature = chat.instance_variable_get(:@temperature) +``` + +The `chat` cog reaches into `RubyLLM::Chat`'s private instance variable to read the temperature. If `ruby_llm` renames or restructures this internal state, the `chat` cog silently gets `nil` temperature. + +**Risk level**: Medium. Any `ruby_llm` upgrade should verify this still works. + +--- + +### 2.2 `field` Macro Falsy-Value Bug + +**File**: `lib/roast/cog/config.rb:116` + +```ruby +@values[key] || default.deep_dup +``` + +The getter uses `||`, which means if a field is explicitly set to `false`, `nil`, or `0`, the getter returns the default instead of the stored value. + +**Affected fields**: Any custom cog using `field(:name, true)` where setting the field to `false` is a valid operation. The built-in boolean options (`async!`, `abort_on_failure!`, `fail_on_error!`) all use direct `@values` manipulation to avoid this — they do NOT use the `field` macro. + +**Workaround for custom cogs**: Use `@values.fetch(key, default.deep_dup)` instead of the field macro for boolean fields. + +--- + +### 2.3 `present?` Rejects Valid Falsy Values in Coercion + +**Files**: +- `lib/roast/system_cogs/call.rb:67`: `@value = input_return_value unless @value.present?` +- `lib/roast/system_cogs/repeat.rb:83`: `@value = input_return_value unless @value.present?` +- `lib/roast/system_cogs/map.rb:157`: `return if @items.present?` + +ActiveSupport's `present?` returns `false` for `false`, `""`, `[]`, `{}`, and `nil`. This means: +- A `call` scope value of `false` or `""` will be overwritten by the input block's return value +- A `repeat` initial value of `false` or `""` will be overwritten +- Map items set to `[]` (empty array) will be overwritten by coercion + +**Impact**: You cannot pass `false`, empty strings, or empty arrays as intentional scope values. + +--- + +### 2.4 `instance_variable_get` Boundary Crossings + +12 sites in the codebase reach across object boundaries via `instance_variable_get`: + +| File | Line | Target | Purpose | +|------|------|--------|---------| +| `config_manager.rb` | 48 | `@global_config.@values` | Seed new config with global values | +| `log.rb` | 78 | `@logger.@logdev` | Check if logger is writing to a stream | +| `call.rb` | 148 | `call_cog_output.@execution_manager` | `from()` scope bridging | +| `call.rb` | 152 | `em.@scope_value` | Extract scope value for `from()` | +| `call.rb` | 153 | `em.@scope_index` | Extract scope index for `from()` | +| `map.rb` | 376 | `map_cog_output.@execution_managers` | `collect()` iteration access | +| `map.rb` | 382 | `em.@scope_value` | Per-iteration value in `collect()` | +| `map.rb` | 383 | `em.@scope_index` | Per-iteration index in `collect()` | +| `map.rb` | 427 | `map_cog_output.@execution_managers` | `reduce()` iteration access | +| `map.rb` | 434 | `em.@scope_value` | Per-iteration value in `reduce()` | +| `map.rb` | 435 | `em.@scope_index` | Per-iteration index in `reduce()` | +| `chat.rb` | 55 | `chat.@temperature` | Read RubyLLM internal state | + +**Fragility assessment**: The `call.rb`/`map.rb` sites are internal (Roast accessing its own objects) so they break only if Roast itself is refactored. The `chat.rb` and `log.rb` sites depend on third-party internal structure and are genuinely fragile. + +--- + +### 2.5 `wait` Uses Bare Rescue + +**File**: `lib/roast/cog.rb:105–107` + +```ruby +def wait + @task&.wait +rescue +``` + +The bare `rescue` catches ALL exceptions (including `SignalException`, `SystemExit`, etc.) and silently swallows them. This means a cog that fails during async execution will appear to succeed if `wait` is called independently of `wait_for_task_with_exception_handling`. + +**Impact**: Direct `cog.wait` calls (as used in `CogInputManager` for blocking output access) will never raise — the cog's failure is only visible through its state (`failed?`, `stopped?`). + +--- + +## 3. Documentation Discrepancies + +### 3.1 `abort_on_failure` Default — RESOLVED + +The RBI shim previously documented `abort_on_failure` as "enabled by default" while the implementation defaulted to `false`. This has been **fixed** — the implementation now reads: + +```ruby +# lib/roast/cog/config.rb:233 +def abort_on_failure? + @values.fetch(:abort_on_failure, true) +end +``` + +The default is now `true` (abort on failure), matching the RBI documentation. + +--- + +## 4. Vestigial Artifacts + +### 4.1 Ghost RuboCop Exclusion + +**File**: `.rubocop.yml:33` + +```yaml +- "lib/roast/sorbet_runtime_stub.rb" +``` + +This file no longer exists (`ls` confirms). The exclusion is harmless but indicates dead configuration. It was likely removed when the project migrated from `sorbet-runtime` stubs to inline RBS comments. + +--- + +## 5. Naming & Convention Pitfalls + +### 5.1 `abort_on_failure` Only Affects `FailCog`, Not Real Errors + +The name suggests it controls behavior for any failure, but it **only** gates propagation of `ControlFlow::FailCog` exceptions (raised by the `fail!` DSL method). Real `StandardError` exceptions from cog execution always propagate regardless of this setting. + +--- + +### 5.2 Anonymous Cogs: UUID Names, Can't Be Referenced + +**File**: `lib/roast/cog.rb:22–23` + +```ruby +def generate_fallback_name + Random.uuid.to_sym +end +``` + +Cogs declared without a name (e.g., `cmd { "echo hello" }`) get a random UUID symbol as their name. They exist in the store and execute normally, but cannot be referenced by other cogs since the name is unpredictable. They are invisible to the output access API. + +--- + +### 5.3 All CLI Kwargs Are Strings — No Type Coercion + +**File**: `lib/roast/cli.rb:85–86` + +```ruby +key, value = arg.split("=", 2) +kwargs[key.to_sym] = value if key +``` + +The `value` is always a `String`. Passing `count=5` gives you `"5"`, not `5`. Passing `verbose=true` gives you `"true"`, not `true`. Workflow authors must coerce manually: + +```ruby +count = kwarg!(:count).to_i +verbose = kwarg(:verbose) == "true" +``` + +--- + +### 5.4 `demodulize` Drops ALL Namespacing — Collision Risk + +**File**: `lib/roast/cog/registry.rb:65` + +```ruby +cog_class_name.demodulize.underscore.to_sym +``` + +Loading `Billing::HttpFetch` and `Shipping::HttpFetch` both produce `:http_fetch`. The second silently overwrites the first in the registry — no warning, no error. + +--- + +### 5.5 Reserved Method Names Block Cog Registration + +Cogs cannot use names that conflict with existing methods on the context objects. The collision guard checks `respond_to?(name, true)` (the `true` includes private methods). This means names like `:send`, `:class`, `:object_id`, `:puts`, etc. will raise `IllegalCogNameError` at prepare time. + +--- + +## 6. Deep Copy Discipline — The 13 Sites + +Every boundary crossing must `deep_dup` to prevent shared-state corruption in concurrent fibers: + +| # | File | Line | What's Copied | Purpose | +|---|------|------|---------------|---------| +| 1 | `execution_manager.rb` | 99 | `cog_config` | Isolate per-cog config from mutations | +| 2 | `execution_manager.rb` | 101 | `@scope_value` | Isolate per-cog scope from mutations | +| 3 | `task_context.rb` | 24 | `Fiber[:path]` | Isolate event path per fiber | +| 4 | `cog/config.rb` | 116 | `default` (in field getter) | Prevent default object sharing | +| 5 | `cog/config.rb` | 126 | `default` (in use_default!) | Same | +| 6 | `config_manager.rb` | 48 | `@global_config.@values` | Isolate new config from global mutations | +| 7 | `cog_input_manager.rb` | 78 | `.output` | Prevent consumer from mutating producer's output | +| 8 | `call.rb` | 152 | `em.@scope_value` | Isolate `from()` extraction | +| 9 | `repeat.rb` | 214 | `input.value` | Isolate next iteration's input from current | +| 10 | `chat/session.rb` | 17 | `chat.messages` | Fork session (snapshot messages) | +| 11 | `chat/session.rb` | 37 | `@messages.first(n)` | Partial session fork | +| 12 | `chat/session.rb` | 49 | `@messages.last(n)` | Partial session fork | +| 13 | `chat/session.rb` | 60 | `@messages` | Restore session to chat object | + +**The invariant**: If you add a new site where data crosses from one fiber/scope/cog to another, you MUST `deep_dup` at that boundary. Forgetting this will cause intermittent, fiber-ordering-dependent data corruption that is extremely difficult to reproduce. + +--- + +## 7. Behavioral Surprises + +### 7.1 `outputs` Block Always Runs — Even After `break!` + +The `outputs`/`outputs!` block inside a scope executes in the `ensure` path of `compute_final_output`. This means it runs even when `break!` terminates the scope early. If your `outputs` block accesses cogs that were skipped by the break, use `outputs` (tolerant) not `outputs!` (strict). + +--- + +### 7.2 Sync vs Async Changes `next!` Semantics + +Toggling a cog between `async!` and sync changes whether `next!` propagates to the parent scope: +- **Sync**: `next!` propagates out of the EM, signaling the parent (map/repeat) to advance +- **Async**: `next!` is swallowed by the barrier handler, only stopping the current scope + +This means adding `async!` to a cog that uses `next!` will **silently break** the workflow's control flow without any error. + +--- + +### 7.3 `from()` Without a Block Returns an Untyped Wrapper + +`from(call!(:name))` without a block returns the raw `compute_final_output` result. Without `outputs`/`outputs!`, this is the last cog's `Output` object. The type is untyped — Sorbet cannot help here. + +`from(call!(:name)) { agent!(:inner).response }` with a block executes in a `CogInputContext` scoped to the inner EM, giving typed access to specific cog outputs. + +--- + +### 7.4 `map` Parallel Results Have `nil` Entries for Skipped Iterations + +When a parallel map iteration calls `skip!` or `next!`, its entry in the results is `nil`. The `collect` helper preserves these nils unless filtered explicitly: + +```ruby +collect(map!(:process)) { |output| output&.text } # may contain nils +collect(map!(:process)) { |output| output&.text }.compact # filtered +``` + +--- + +## 8. Development Environment Gotchas + +### 8.1 `ROAST_WORKING_DIRECTORY` Has Dual Usage + +This environment variable is used both by the CLI (to set the working directory for all cmd cogs) AND by the functional test infrastructure (to override the sandbox path). Setting it for testing purposes may inadvertently affect workflow behavior. + +--- + +### 8.2 Tests Exclude Sorbet + +Test files are excluded in `sorbet/config`. You get no type checking in tests. Mistakes in test helper type signatures are invisible until runtime. + +--- + +### 8.3 `RECORD_VCR=true` Uses Real API Credentials + +Running tests with `RECORD_VCR=true` sends real API requests. Be aware of: +- Rate limiting (OpenAI, Anthropic) +- Cost (token usage) +- Credential leakage in committed cassettes (filter sensitive data) + +--- + +## 9. Quick-Reference: What Each Gotcha Affects + +| Issue | Affects Writing Workflows | Affects Writing Cogs | Affects Framework Dev | +|-------|:---:|:---:|:---:| +| Claude dump path | | | ✓ | +| Top-level sync `next!` | ✓ | | | +| Repeat sync `next!` | ✓ | | | +| Async `next!` swallowed | ✓ | | | +| `outputs!` + `break!` | ✓ | | | +| Empty scope crash | ✓ | | | +| RubyLLM `instance_variable_get` | | | ✓ | +| `field` macro falsy values | | ✓ | | +| `present?` rejects falsy | ✓ | | | +| Anonymous cog names | ✓ | | | +| Kwargs are strings | ✓ | | | +| `demodulize` collision | | ✓ | | +| Deep copy discipline | | | ✓ | +| Sync/async `next!` divergence | ✓ | | | +| `outputs` always runs | ✓ | | | + +--- + +## See Also + +- [07-control-flow-reference.md](07-control-flow-reference.md) — Full propagation matrix for bugs 1.2–1.5 +- [03-cog-reference.md](03-cog-reference.md) — `field` macro details and boolean patterns +- [05-execution-engine-internals.md](05-execution-engine-internals.md) — `compute_final_output` and `wait_for_task_with_exception_handling` +- [10-writing-custom-cogs.md](10-writing-custom-cogs.md) — Workarounds for the `field` macro +- [06-metaprogramming-map.md](06-metaprogramming-map.md) — `instance_variable_get` site catalogue