Harness engineering

[joejstuart]

This is an implementation of OpenAI's harness engineering paper - https://openai.com/index/harness-engineering/

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: dd60ed2f-28fc-4d11-898b-29433e67f2c1

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

📝 Coding Plan

• Generate coding plan for human review comments

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Tip

You can validate your CodeRabbit configuration file in your editor.

If your editor has YAML language server, you can enable auto-completion and validation by adding # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json at the top of your CodeRabbit configuration file.

[qodo-code-review]

Review Summary by Qodo

Implement harness engineering with code quality enforcement and documentation

✨ Enhancement 📝 Documentation

Walkthroughs

Description

• Implement harness engineering framework with code quality enforcement
• Add comprehensive engineering documentation (principles, style, architecture)
• Integrate go-postconditions for automated sanity checks
• Create Architecture Decision Records (ADRs) and design documentation
• Simplify AGENTS.md and CLAUDE.md with focused guidance

Diagram

flowchart LR
  A["Engineering Framework"] --> B["Code Quality Checks"]
  A --> C["Documentation"]
  B --> D["make sanity targets"]
  B --> E["Complexity budgets"]
  C --> F["PRINCIPLES.md"]
  C --> G["ARCHITECTURE.md"]
  C --> H["Design Docs"]
  C --> I["ADRs"]
  D --> J["Pre-PR validation"]

Loading

File Changes

1. hack/check-docs-freshness.sh ✨ Enhancement +51/-0

Add documentation freshness validation script

hack/check-docs-freshness.sh

---

2. .cursor/rules/rule_filtering_process.mdc 📝 Documentation +6/-26

Simplify rule filtering documentation and remove incorrect logic

.cursor/rules/rule_filtering_process.mdc

---

3. .gitmodules Dependencies +3/-0

Add go-postconditions submodule for code quality

.gitmodules

---

View more (17)

4. AGENTS.md 📝 Documentation +82/-230

Restructure agent instructions with focused quick reference

AGENTS.md

---

5. CLAUDE.md 📝 Documentation +61/-12

Simplify Claude guidance with mandatory workflow

CLAUDE.md

---

6. Makefile ✨ Enhancement +32/-0

Add sanity check targets using postconditions

Makefile

---

7. docs/ARCHITECTURE.md 📝 Documentation +89/-0

Document module structure and layer dependencies

docs/ARCHITECTURE.md

---

8. docs/CODE_STYLE.md 📝 Documentation +147/-0

Define anti-patterns and code style guidelines

docs/CODE_STYLE.md

---

9. docs/PRINCIPLES.md 📝 Documentation +61/-0

Establish 10 core engineering principles

docs/PRINCIPLES.md

---

10. docs/REVIEW.md 📝 Documentation +51/-0

Create PR review checklist for humans and agents

docs/REVIEW.md

---

11. docs/decisions/ADR-001-cgo-disabled.md 📝 Documentation +37/-0

Document CGO_ENABLED=0 decision and consequences

docs/decisions/ADR-001-cgo-disabled.md

---

12. docs/decisions/ADR-template.md 📝 Documentation +17/-0

Provide template for future Architecture Decision Records

docs/decisions/ADR-template.md

---

13. docs/decisions/index.md 📝 Documentation +13/-0

Create ADR catalog and index

docs/decisions/index.md

---

14. docs/design-docs/index.md 📝 Documentation +15/-0

Create design documents index and process

docs/design-docs/index.md

---

15. docs/design-docs/package-filtering.md 📝 Documentation +337/-0

Document pluggable rule filtering system architecture

docs/design-docs/package-filtering.md

---

16. docs/design-docs/rule-filtering.md 📝 Documentation +491/-0

Document complete rule filtering process with examples

docs/design-docs/rule-filtering.md

---

17. docs/design-docs/vsa-architecture.md 📝 Documentation +542/-0

Document VSA layered architecture and workflows

docs/design-docs/vsa-architecture.md

---

18. docs/quality/baseline-report.md 📝 Documentation +50/-0

Record initial code quality baseline metrics

docs/quality/baseline-report.md

---

19. docs/quality/tech-debt-tracker.md 📝 Documentation +28/-0

Track complexity violations and ratchet schedule

docs/quality/tech-debt-tracker.md

---

20. make/postconditions Dependencies +1/-0

Add go-postconditions submodule for quality checks

make/postconditions

---

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (3) 📘 Rule violations (2) 📎 Requirement gaps (0)

1. Sanity target breaks checkout 🐞 Bug ⛯ Reliability

Description

make sanity (and related targets) executes make -f make/postconditions/Makefile, but the
repository checkout workflows don’t fetch submodules, so that Makefile won’t exist and the target
will fail. This makes the newly “required” sanity workflow non-functional in typical clones/CI.

Code

Makefile[R238-266]

+POSTCONDITIONS_MK := make/postconditions/Makefile
+
+.PHONY: sanity
+sanity: ## Run sanity checks (complexity, duplication, unused params)
+	@$(MAKE) -f $(POSTCONDITIONS_MK) check-sanity
+
+.PHONY: sanity-report
+sanity-report: ## Generate sanity report with summary
+	@$(MAKE) -f $(POSTCONDITIONS_MK) report-sanity
+
+.PHONY: sanity-file
+sanity-file: ## Run sanity checks on specific files (FILES=./path/to/file.go)
+	@$(MAKE) -f $(POSTCONDITIONS_MK) check-sanity FILES="$(FILES)"
+
+.PHONY: quick
+quick: ## Fast validation for small changes
+	@$(MAKE) -f $(POSTCONDITIONS_MK) quick
+
+.PHONY: refactor
+refactor: ## Validation for structural changes (includes unused function scan)
+	@$(MAKE) -f $(POSTCONDITIONS_MK) refactor
+
+.PHONY: behavior
+behavior: ## Validation for behavior changes (full quality + tests + coverage)
+	@$(MAKE) -f $(POSTCONDITIONS_MK) behavior
+
+.PHONY: prepr
+prepr: ## Pre-PR stabilization checks
+	@$(MAKE) -f $(POSTCONDITIONS_MK) prepr

Evidence

The Makefile directly shells out to another Makefile located inside a git submodule, but the
repository’s GitHub Actions checkout step does not enable submodules, meaning the submodule contents
(including its Makefile) will not be present when make sanity is run.

Makefile[236-266]
.gitmodules[1-3]
.github/workflows/lint.yaml[49-53]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`make sanity` invokes `make/postconditions/Makefile`, but common checkouts (including GitHub Actions) do not fetch submodules by default. This causes `make sanity` to fail with a missing Makefile.

## Issue Context
The repo now treats `make sanity` as a required quality gate (per AGENTS/CLAUDE/docs), so it must work in a fresh clone and in CI.

## Fix Focus Areas
- Add submodule initialization to CI checkout (e.g., `submodules: recursive` in `actions/checkout`) for workflows that will run sanity
 - .github/workflows/lint.yaml[49-53]
 - .github/workflows/checks-codecov.yaml[45-50]
 - .github/workflows/release.yaml[81-85]
- Add a Makefile guard that errors with a clear instruction when `make/postconditions/Makefile` is missing (or add a `make submodules` prerequisite)
 - Makefile[236-266]
- (Optional) Document the required `git submodule update --init --recursive` step near the `make sanity` instructions
 - AGENTS.md[16-27]
 - CLAUDE.md[12-31]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Unrequested make sanity framework 📘 Rule violation ✓ Correctness

Description

This PR introduces a new make/postconditions submodule and multiple new Makefile targets (e.g.,
sanity, behavior) that are not described in the PR scope. This appears to add new capabilities
beyond the stated implementation work and increases review/maintenance surface.

Code

Makefile[R236-267]

+##@ Code Quality (Postconditions)
+
+POSTCONDITIONS_MK := make/postconditions/Makefile
+
+.PHONY: sanity
+sanity: ## Run sanity checks (complexity, duplication, unused params)
+	@$(MAKE) -f $(POSTCONDITIONS_MK) check-sanity
+
+.PHONY: sanity-report
+sanity-report: ## Generate sanity report with summary
+	@$(MAKE) -f $(POSTCONDITIONS_MK) report-sanity
+
+.PHONY: sanity-file
+sanity-file: ## Run sanity checks on specific files (FILES=./path/to/file.go)
+	@$(MAKE) -f $(POSTCONDITIONS_MK) check-sanity FILES="$(FILES)"
+
+.PHONY: quick
+quick: ## Fast validation for small changes
+	@$(MAKE) -f $(POSTCONDITIONS_MK) quick
+
+.PHONY: refactor
+refactor: ## Validation for structural changes (includes unused function scan)
+	@$(MAKE) -f $(POSTCONDITIONS_MK) refactor
+
+.PHONY: behavior
+behavior: ## Validation for behavior changes (full quality + tests + coverage)
+	@$(MAKE) -f $(POSTCONDITIONS_MK) behavior
+
+.PHONY: prepr
+prepr: ## Pre-PR stabilization checks
+	@$(MAKE) -f $(POSTCONDITIONS_MK) prepr
+

Evidence

The compliance checklist prohibits adding features not requested; the PR description only states
implementing the harness engineering paper, while the diff adds a new quality-check framework via a
submodule and new Make targets.

CLAUDE.md
Makefile[236-267]
.gitmodules[1-3]
make/postconditions[1-1]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
This PR adds new build/quality infrastructure (a `make/postconditions` submodule and multiple Make targets) that is not reflected in the stated PR scope.

## Issue Context
Compliance requires avoiding unrequested features; the PR description only references implementing the harness engineering paper, but the diff adds new tooling and workflows.

## Fix Focus Areas
- Makefile[236-267]
- .gitmodules[1-3]
- make/postconditions[1-1]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Large unrelated docs rewrite 📘 Rule violation ✓ Correctness

Description

This PR substantially rewrites agent/docs guidance files (e.g., AGENTS.md, CLAUDE.md) rather
than keeping changes localized to the requested implementation. This increases the risk of
unintended scope expansion and makes review harder.

Code

AGENTS.md[R3-95]

+The `ec` CLI verifies artifacts and evaluates software supply chain policies using OPA/Rego rules.

-The `ec` (Enterprise Contract) CLI is a command-line tool for verifying artifacts and evaluating software supply chain policies. It validates container image signatures, provenance, and enforces policies across various types of software artifacts using Open Policy Agent (OPA)/Rego rules.
+## Quick Reference

-## Essential Commands
-
-### Building
+### Build & Test
```bash
-make build          # Build ec binary for current platform (creates dist/ec)
-make dist           # Build for all supported architectures
-make clean          # Remove build artifacts
-DEBUG_BUILD=1 make build  # Build with debugging symbols for gdb/dlv
-make debug-run      # Run binary with delve debugger (requires debug build)
+make build              # Build ec binary (dist/ec)
+make test               # Run all tests
+make acceptance         # Run acceptance tests (20m timeout)
+make lint-fix           # Fix linting issues
+make ci                 # Full CI suite

-### Testing
+### Code Quality (Required)

-make test           # Run all tests (unit, integration, generative)
-make acceptance     # Run acceptance tests (Cucumber/Gherkin, 20m timeout)
-make scenario_<name>  # Run single acceptance scenario (replace spaces with underscores)
-make feature_<name>   # Run all scenarios in a single feature file
-
-# Running specific tests
-go test -tags=unit ./internal/evaluator -run TestSpecificFunction
-cd acceptance && go test -test.run 'TestFeatures/scenario_name'
+make sanity             # Run sanity checks - MUST PASS before PR
+make sanity-report      # Generate detailed quality report
+make sanity-file FILES="./path/to/file.go"  # Check specific files
+```
+
+### Before Submitting Code
+1. `make sanity` - Must pass (complexity, duplication, unused params)
+2. `make test` - Must pass
+3. Follow [docs/PRINCIPLES.md](docs/PRINCIPLES.md)
+4. Review checklist in [docs/REVIEW.md](docs/REVIEW.md)
+
+## Documentation Map
+
+| Topic | Location |
+|-------|----------|
+| Architecture overview | [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) |
+| Engineering principles | [docs/PRINCIPLES.md](docs/PRINCIPLES.md) |
+| Code style rules | [docs/CODE_STYLE.md](docs/CODE_STYLE.md) |
+| PR review checklist | [docs/REVIEW.md](docs/REVIEW.md) |
+| Package filtering | [docs/design-docs/package-filtering.md](docs/design-docs/package-filtering.md) |
+| Rule filtering | [docs/design-docs/rule-filtering.md](docs/design-docs/rule-filtering.md) |
+| VSA architecture | [docs/design-docs/vsa-architecture.md](docs/design-docs/vsa-architecture.md) |
+| Tech debt tracker | [docs/quality/tech-debt-tracker.md](docs/quality/tech-debt-tracker.md) |
+
+## Key Constraints
+
+1. **Complexity budgets are enforced** - `make sanity` fails if exceeded
+2. **Simplicity over abstraction** - See [PRINCIPLES.md](docs/PRINCIPLES.md)
+3. **Match existing patterns** - Don't "improve" surrounding code
+4. **Parse at boundaries** - Validate input, trust internal data
+
+## Project Structure
+

+cmd/ # CLI commands (validate, fetch, inspect, track, etc.)
+internal/ # Private packages
+├── evaluator/ # Policy evaluation engine (filters.go, conftest_evaluator.go)
+├── validate/ # Validation logic
+│ └── vsa/ # VSA subsystem (9 layers)
+├── policy/ # Policy management
+├── attestation/ # In-toto attestations, SLSA provenance
+└── ...
+acceptance/ # Cucumber acceptance tests
+features/ # Gherkin feature files
+```
+
+## Testing
+
+| Type | Command | Timeout |
+|------|---------|---------|
+| Unit | go test -tags=unit ./... | 10s |
+| Integration | `go test -tags=integration ./...` | 15s |
+| Acceptance | `make acceptance` | 20m |
+| Single scenario | `make scenario_` | - |

-### Code Quality
+## Common Issues
+
+DNS resolution in tests - Add to /etc/hosts:
+ +127.0.0.1 apiserver.localhost +127.0.0.1 rekor.localhost +
+
+Go checksum mismatch:

-make lint           # Run all linters (golangci-lint, addlicense, tekton-lint)
-make lint-fix       # Auto-fix linting issues
-make ci             # Run full CI suite (test + lint-fix + acceptance)
+go env -w GOPROXY='https://proxy.golang.org,direct'

-## Architecture

-### Command Structure
-Main commands in cmd/:
-- validate - Validate container images, attestations, and policies
-- test - Test policies against data (similar to conftest)
-- fetch - Download and inspect attestations
-- inspect - Examine policy bundles and data
-- track - Track compliance status
-- sigstore - Sigstore-related operations
-- initialize - Initialize policy configurations

-### Core Components

-#### Policy Evaluation (internal/evaluator/)
-- Conftest Evaluator: Main evaluation engine using OPA/Rego
-- Pluggable Rule Filtering: Extensible system for filtering which rules run based on:

• • Pipeline intentions (build vs release vs production)
• • Include/exclude lists (collections, packages, specific rules)
• • Custom metadata criteria
    -- Result Processing: Complex rule result filtering with scoring, severity promotion/demotion, and effective time handling

-Key Implementation Details:
-- PolicyResolver interface provides comprehensive policy resolution for pre and post-evaluation filtering
-- UnifiedPostEvaluationFilter implements unified filtering logic
-- Sophisticated scoring system for include/exclude decisions (collections: 10pts, packages: 10pts per level, rules: +100pts, terms: +100pts)
-- Term-based filtering allows fine-grained control (e.g., tasks.required_untrusted_task_found:clamav-scan)
-- See .cursor/rules/rule_filtering_process.mdc and .cursor/rules/package_filtering_process.mdc for detailed documentation

-#### Attestation Handling (internal/attestation/)
-- Parsing and validation of in-toto attestations
-- SLSA provenance processing (supports both v0.2 and v1.0)
-- Integration with Sigstore for signature verification

-#### VSA (Verification Summary Attestation) (internal/validate/vsa/)
-VSA creates cryptographically signed attestations containing validation metadata and policy information after successful image validation.

-Layered Architecture:
-1. Core Interfaces (interfaces.go) - Fundamental VSA interfaces
-2. Service Layer (service.go) - High-level VSA processing orchestration
-3. Core Logic (vsa.go) - VSA data structures and predicate generation
-4. Attestation (attest.go) - DSSE envelope creation and signing
-5. Storage (storage*.go) - Abstract storage backends (local, Rekor)
-6. Retrieval (*_retriever.go) - VSA retrieval mechanisms
-7. Orchestration (orchestrator.go) - Complex VSA processing workflows
-8. Validation (validator.go) - VSA validation with policy comparison
-9. Command Interface (cmd/validate/vsa.go) - CLI for VSA validation

-Key Features:
-- Policy comparison and equivalence checking
-- DSSE envelope signature verification (enabled by default)
-- Multiple storage backends (local filesystem, Rekor transparency log)
-- VSA expiration checking with configurable thresholds
-- Batch validation from application snapshots with parallel processing

-See .cursor/rules/vsa_functionality.mdc for comprehensive documentation.

-#### Input Processing (internal/input/)
-- Multiple input sources: container images, files, Kubernetes resources
-- Automatic detection and parsing of different artifact types

-#### Policy Management (internal/policy/)
-- OCI-based policy bundle loading
-- Git repository policy fetching
-- Policy metadata extraction and rule discovery

-### Key Internal Packages
-- internal/signature/ - Container image signature verification
-- internal/image/ - Container image operations and metadata
-- internal/kubernetes/ - Kubernetes resource processing
-- internal/utils/ - Common utilities and helpers
-- internal/rego/ - Rego policy compilation and execution
-- internal/format/ - Output formatting (JSON, YAML, etc.)

-## Module Structure

-The project uses multiple Go modules:
-- Root module - Main CLI application
-- acceptance/ - Acceptance test module with Cucumber integration
-- tools/ - Development tools and utilities

-## Testing Strategy

-### Test Types
-- Unit tests (-tags=unit, 10s timeout) - Fast isolated tests
-- Integration tests (-tags=integration, 15s timeout) - Component integration
-- Generative tests (-tags=generative, 30s timeout) - Property-based testing
-- Acceptance tests (20m timeout) - End-to-end Cucumber scenarios with real artifacts

• • Use -persist flag to keep test environment after execution for debugging
• • Use -restore to run tests against persisted environment
• • Use -tags=@focus to run specific scenarios

-### Acceptance Test Framework
-- Uses Cucumber/Gherkin syntax for feature definitions in features/ directory
-- Steps implemented in Go using Godog framework
-- Self-contained test environment using Testcontainers
-- WireMock for stubbing HTTP APIs (Kubernetes apiserver, Rekor)
-- Snapshots stored in features/__snapshots__/ (update with UPDATE_SNAPS=true)

-## Development Environment

-### Required Tools
-- Go 1.24.4+
-- Make
-- Podman/Docker for container operations
-- Node.js for tekton-lint

-### Troubleshooting Common Issues

-1. Go checksum mismatch

• go env -w GOPROXY='https://proxy.golang.org,direct'

-2. Container failures - Ensure podman runs as user service, not system service

• systemctl status podman.socket podman.service
• systemctl disable --now podman.socket podman.service
• systemctl enable --user --now podman.socket podman.service

-3. Too many containers - Increase inotify watches

• echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p

-4. Key limits - Increase max keys

• echo kernel.keys.maxkeys=1000 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p

-5. Host resolution - Add to /etc/hosts:

• 127.0.0.1 apiserver.localhost
• 127.0.0.1 rekor.localhost

-## Key Configuration

-### Policy Sources
-Policies can be loaded from:
-- OCI registries: oci::quay.io/repo/policy:tag
-- Git repositories: git::https://github.com/repo//path
-- Local files/directories

-### Debug Mode
-- Use --debug flag or EC_DEBUG=1 environment variable
-- Debug mode preserves temporary ec-work-* directories for inspection

-## Special Considerations

-### CGO and DNS Resolution
-Binaries are built with CGO_ENABLED=0 for OS compatibility, which affects DNS resolution. The Go native resolver cannot resolve second-level localhost domains like apiserver.localhost, requiring manual /etc/hosts entries for acceptance tests.

-### Multi-Architecture Support
-The build system supports all major platforms and architectures. Use make dist to build for all supported targets or make dist/ec_<os>_<arch> for specific platforms.

-### Policy Rule Filtering System
-The evaluation system includes sophisticated rule filtering that operates at multiple levels:

-#### Pre-Evaluation Filtering (Package Level)
-1. Pipeline Intention Filtering (ECPolicyResolver only)

• • When pipeline_intention is set: only include packages with matching metadata
• • When not set: only include general-purpose rules (no pipeline_intention metadata)

-2. Rule-by-Rule Evaluation

• • Each rule is scored against include/exclude criteria
• • Scoring system: collections (10pts), packages (10pts/level), rules (+100pts), terms (+100pts)
• • Higher score determines inclusion/exclusion

-3. Package-Level Determination

• • If ANY rule in package is included → Package is included
• • Package runs through conftest evaluation

-#### Post-Evaluation Filtering (Result Level)
-- UnifiedPostEvaluationFilter processes all results using same PolicyResolver
-- Filters warnings, failures, exceptions, skipped results
-- Applies severity logic (promotion/demotion based on metadata)
-- Handles effective time filtering (future-effective failures → warnings)

-#### Term-Based Filtering
-Terms provide fine-grained control over specific rule instances:
-- Example: tasks.required_untrusted_task_found:clamav-scan (scores 210pts)
-- Can override general patterns like tasks.* (10pts)
-- Terms are extracted from result metadata during filtering

-### Working with Rule Filtering Code
-When modifying policy evaluation or filtering logic:
-1. Read .cursor/rules/package_filtering_process.mdc for architecture overview
-2. Read .cursor/rules/rule_filtering_process.mdc for detailed filtering flow
-3. Main filtering code is in internal/evaluator/filters.go
-4. Integration point is in internal/evaluator/conftest_evaluator.go

-### Working with VSA Code
-When modifying VSA functionality:
-1. Read .cursor/rules/vsa_functionality.mdc for complete documentation
-2. Understand the layered architecture (9 layers from interfaces to CLI)
-3. VSA code is in internal/validate/vsa/ directory
-4. CLI implementation in cmd/validate/vsa.go
-5. Signature verification is enabled by default and implemented via DSSE envelopes

-## Additional Documentation

-For detailed implementation guides, see:
-- .cursor/rules/package_filtering_process.mdc - Pluggable rule filtering system
-- .cursor/rules/rule_filtering_process.mdc - Complete rule filtering process
-- .cursor/rules/vsa_functionality.mdc - VSA architecture and workflows
+## When Stuck
+
+If code quality checks fail:
+1. Simplify the implementation
+2. Extract focused helper functions
+3. Use early returns to reduce nesting
+4. Do NOT add flags or config to work around limits
+
+For architecture questions, see docs/ARCHITECTURE.md.
+For detailed design docs, see docs/design-docs/.

</details>

Evidence

The compliance checklist requires avoiding unrelated refactors/improvements; the diff shows
large-scale restructuring and replacement of existing documentation content instead of minimal,
necessary edits.

CLAUDE.md
AGENTS.md[3-95]
CLAUDE.md[3-63]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The PR includes broad rewrites/reorganization of guidance documentation that may be unrelated to the requested implementation, increasing scope and review complexity.

## Issue Context
Compliance requires avoiding unrelated refactors/cleanups; documentation changes should be localized to what the feature needs.

## Fix Focus Areas
- AGENTS.md[3-95]
- CLAUDE.md[3-63]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

4. Package filtering doc mismatch 🐞 Bug ✧ Quality

Description

The new package-filtering design doc claims a package can be “excluded” when it contains only
excluded rules, but the implementation does not track excluded packages and only includes packages
that have at least one included rule. This documentation is inconsistent with the actual resolver
behavior and will mislead future changes to filtering logic.

Code

docs/design-docs/package-filtering.md[R170-173]

+### Phase 3: Package-Level Determination
+- If ANY rule in the package is included → Package is included
+- If NO rules are included but SOME rules are excluded → Package is excluded
+- If NO rules are included and NO rules are excluded → Package is not explicitly categorized

Evidence

The design doc describes package exclusion behavior that is not implemented. The resolver’s shared
package inclusion logic only marks a package included when it has an included rule and otherwise
does nothing (no excluded-package tracking).

docs/design-docs/package-filtering.md[170-173]
internal/evaluator/filters.go[623-643]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`docs/design-docs/package-filtering.md` documents a package-level exclusion mode that the actual resolver does not implement.

## Issue Context
The canonical implementation currently only includes packages with at least one included rule; packages with only excluded rules are simply not included.

## Fix Focus Areas
- Align doc Phase 3 text with implementation in `baseDeterminePackageInclusion`
 - docs/design-docs/package-filtering.md[170-173]
 - internal/evaluator/filters.go[623-643]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

View more (1)

5. Relative submodule URL brittle 🐞 Bug ⛯ Reliability

Description

The submodule is configured with a relative URL (../go-postconditions), which can break for
forks/mirrors where that relative repository path doesn’t exist, preventing submodule initialization
and therefore breaking the sanity targets. This is a portability hazard for contributors outside the
primary repo namespace.

Code

.gitmodules[R1-3]

+[submodule "make/postconditions"]
+	path = make/postconditions
+	url = ../go-postconditions

Evidence

The .gitmodules entry uses a relative URL, so submodule resolution depends on the parent repo’s
remote layout being compatible with that relative path.

.gitmodules[1-3]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The submodule URL is relative and may not resolve in forks/mirrors, breaking submodule init and downstream `make sanity`.

## Issue Context
Relative submodule URLs are sometimes intentional, but they impose constraints on how the parent repo is hosted/forked.

## Fix Focus Areas
- Consider changing to an explicit URL (or add documentation for fork users on overriding submodule URLs)
 - .gitmodules[1-3]
- If keeping relative URL, document contributor steps clearly near sanity instructions
 - AGENTS.md[16-27]
 - CLAUDE.md[12-31]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

ⓘ The new review experience is currently in Beta. Learn more

[qodo-code-review]

1. Sanity target breaks checkout 🐞 Bug ⛯ Reliability

make sanity (and related targets) executes make -f make/postconditions/Makefile, but the
repository checkout workflows don’t fetch submodules, so that Makefile won’t exist and the target
will fail. This makes the newly “required” sanity workflow non-functional in typical clones/CI.

Agent Prompt

## Issue description
`make sanity` invokes `make/postconditions/Makefile`, but common checkouts (including GitHub Actions) do not fetch submodules by default. This causes `make sanity` to fail with a missing Makefile.

## Issue Context
The repo now treats `make sanity` as a required quality gate (per AGENTS/CLAUDE/docs), so it must work in a fresh clone and in CI.

## Fix Focus Areas
- Add submodule initialization to CI checkout (e.g., `submodules: recursive` in `actions/checkout`) for workflows that will run sanity
  - .github/workflows/lint.yaml[49-53]
  - .github/workflows/checks-codecov.yaml[45-50]
  - .github/workflows/release.yaml[81-85]
- Add a Makefile guard that errors with a clear instruction when `make/postconditions/Makefile` is missing (or add a `make submodules` prerequisite)
  - Makefile[236-266]
- (Optional) Document the required `git submodule update --init --recursive` step near the `make sanity` instructions
  - AGENTS.md[16-27]
  - CLAUDE.md[12-31]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag	Coverage Δ
acceptance	54.86% <ø> (ø)
generative	18.14% <ø> (ø)
integration	26.99% <ø> (ø)
unit	68.66% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.