Skip to content

fix(workflows): confine workflow and step catalog cache writes - #4931

Open
mnriem wants to merge 2 commits into
github:mainfrom
mnriem:mnriem-catalog-cache-hardening
Open

mnriem wants to merge 2 commits into
github:mainfrom
mnriem:mnriem-catalog-cache-hardening

Conversation

@mnriem

@mnriem mnriem commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator

Description

Defensively harden workflow and step catalog cache I/O so repository-provided symlinks cannot redirect cache writes into unrelated files. This covers both JSON data and freshness metadata under .specify/workflows/.cache/ and .specify/workflows/steps/.cache/.

Both catalog types now use one shared cache helper. POSIX directory traversal and operations are anchored to no-follow directory descriptors; Windows uses native handles that reject reparse points and pin ancestors without delete sharing. Cache writes exclusively create temporary files and atomically replace destination entries instead of truncating their referents. Reads also refuse unsafe cache paths. Normal discovery, fresh-cache reuse, refresh, and safe stale/offline fallback are preserved; cache-write failures log warnings while discovery continues from fetched data.

The regression fixtures use isolated sentinel files, including committed symlinks cloned through Git and reached through workflow search / workflow step search with discovery-only catalogs. No archive download, installation, workflow execution, or executable payload is involved. Additional cases cover dangling links, hard links, ancestor and leaf swaps, failure cleanup, serialization failures, and real Windows filesystem behavior. Windows-specific cases run only on Windows jobs; Win32 APIs are not mocked.

This is defensive cache hardening only. It does not change catalog trust policy, advisory disposition, or CVE classification. Unrelated catalog domains are unchanged. The workflow reference documents the cache behavior.

Testing

  • Tested locally with uv run specify --help
  • Ran existing tests with uv sync && uv run pytest
  • Tested with a sample project (if applicable)

The existing-test checklist above was satisfied using the documented worktree-local equivalent: uv sync --extra test --quiet, followed by this worktree's .venv/bin/python -m pytest, to avoid another checkout's editable installation.

Command Result
uv sync --extra test --quiet Passed; installed the worktree-local test environment
uv run specify --help Passed
LC_ALL=en_US.UTF-8 .venv/bin/python -m pytest tests -q 9,749 passed, 46 skipped, 62 warnings on macOS / Python 3.13
LC_ALL=en_US.UTF-8 .venv/bin/python -m pytest tests/specify_cli/workflows -q 642 passed, 27 skipped
LC_ALL=en_US.UTF-8 .venv/bin/python -m pytest tests/test_workflows.py -q 747 passed
uvx ruff@0.15.0 check src tests --output-format concise Passed using the CI-pinned Ruff version
git --no-pager diff --check Passed
LC_ALL=en_US.UTF-8 .venv/bin/python -m pytest --collect-only -q 9,795 collected, up from the unchanged baseline of 9,707; no tests removed

Before-and-after evidence: before implementation, LC_ALL=en_US.UTF-8 .venv/bin/python -m pytest tests/specify_cli/workflows/test_catalog_cache.py -q ran the original 22-case regression set and produced 15 failed, 6 passed, 1 skipped. Failures included all four committed-symlink → clone → CLI-search cases and workflow cache ancestor redirection. After the fix, the same original cases produced 21 passed, 1 skipped with:

LC_ALL=en_US.UTF-8 .venv/bin/python -m pytest tests/specify_cli/workflows/test_catalog_cache.py -q -k 'cache_leaf_cannot_redirect_write or committed_symlink_clone_search_preserves_target or symlinked_cache_ancestor_cannot_redirect_write or regular_cache_fresh_refresh_and_stale_fallback'

Native Windows tests were not run on this macOS host. They are collected by the existing windows-latest Python 3.13/3.14 jobs, and their CI results remain a gate before merge.

AI Disclosure

  • I did not use AI assistance for this contribution
  • I did use AI assistance (fill in the disclosure below)

AI disclosure: Implemented on behalf of @mnriem with GitHub Copilot App / CLI using GPT-6.1 Sol (model ID: gpt-6.1-sol) in autonomous/autopilot mode; reasoning effort was not explicitly configured in the task. AI assistance covered repository investigation, reproduction, code and test generation, documentation, automated validation, the commit, and this PR description. The user bounded the scope, directed replacement of mocked Windows cases with native Windows-only tests, and explicitly authorized commit, fork push, and upstream PR creation. This is not a claim of human line-by-line review.

Anchor catalog cache traversal and atomically replace JSON data and metadata without following repository-provided symlinks. Preserve discovery, freshness checks, refresh, and safe stale-cache fallback.

Add isolated clone/search regressions, race and failure coverage, and native Windows filesystem tests. Document the defensive cache behavior.

Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 85ee2fff-5185-406e-b4e5-c961e0ed0938
Copilot AI balanced review requested due to automatic review settings October 12, 2026 15:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The low-level cross-platform filesystem security behavior warrants final human review and successful native Windows CI validation.

0 open findings

What changed in this PR

Hardens workflow and step catalog caches against symlink and filesystem-redirection attacks.

Changes:

  • Adds shared confined, atomic cache I/O for POSIX and Windows.
  • Migrates both catalog implementations to the helper.
  • Adds extensive regression tests and cache-behavior documentation.
File Description
src/​specify_cli/​workflows/​_catalog_cache.py Implements secure shared cache I/O.
src/​specify_cli/​workflows/​catalog/​_domain.py Secures workflow catalog caching.
src/​specify_cli/​workflows/​step/​catalog/​_domain.py Secures step catalog caching.
tests/​specify_cli/​workflows/​test_catalog_cache.py Covers normal, adversarial, and platform-specific behavior.
docs/​reference/​workflows.md Documents cache safety and fallback behavior.

🧠 Review effort: Balanced


💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Include FILE_LIST_DIRECTORY in directory handle access so no-delete sharing participates in Windows share checks. FILE_READ_ATTRIBUTES alone allows empty cache directories to be renamed while held, as demonstrated by the three native Windows CI failures.

Keep the native Windows rename, ancestor pinning, and reparse-point regressions unchanged.

Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 85ee2fff-5185-406e-b4e5-c961e0ed0938
Copilot AI balanced review requested due to automatic review settings October 12, 2026 16:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Native Windows handle semantics and adversarial filesystem race handling warrant final human review and gated CI confirmation.

0 open findings

🧠 Review effort: Balanced

@mnriem

mnriem commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator Author

Fixed the Windows CI failures in commit bc88e86. Attribute-only directory handles did not enforce Windows share checking. Adding FILE_LIST_DIRECTORY alongside FILE_READ_ATTRIBUTES makes withholding FILE_SHARE_DELETE actually block renames, including empty cache directories. The existing native tests and platform skips are unchanged.

CI run 38278468357 passed for the exact correction commit. All three previously failing native cases now pass on both Windows Python 3.13 and 3.14; each Windows suite completed with 11,411 passed and 612 skipped. All six Linux/macOS/Windows pytest jobs and the remaining applicable checks passed. Native CI validation is confirmed; final human review remains outstanding.

Posted on behalf of @mnriem by GitHub Copilot App / CLI using GPT-6.1 Sol (gpt-6.1-sol), autonomous/autopilot mode, with reasoning effort not explicitly configured. AI assistance covered the code correction, automated validation, and this summary.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants