Skip to content

build: migrate developer tasks to mise - #211

Open
binaryaaron wants to merge 25 commits into
mainfrom
binaryaaron/add-mise
Open

binaryaaron wants to merge 25 commits into
mainfrom
binaryaaron/add-mise

Conversation

@binaryaaron

@binaryaaron binaryaaron commented Jul 6, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Developer commands and CI duplicated task definitions across the Makefile and workflow files, which allowed local and CI behavior to drift. Mise provides one versioned task graph and pins the tools that execute it.

No linked issue required: maintainer-owned developer infrastructure migration.

Summary

  • migrate developer tasks and CI invocation to Mise
  • adopt the shared colon-delimited task vocabulary; public task names contain no hyphens or aliases
  • compose prerequisites and ordered stages with native Mise task references
  • keep the blocking ty check and DataDesigner 0.10 compatibility from current main; ty moves from 0.0.69 to 0.0.90 and Ruff from >=0.12.3,<1 to 0.17.1
  • require Mise 2026.10.15 or newer; pin uv 0.13.4, Ruff 0.17.1, and ty 0.0.90
  • add setup all for every locked dependency group and test:all for unit plus opt-in E2E suites
  • keep benchmark workflow orchestration on the current revision while checking arbitrary historical targets into isolated directories
  • publish releases from the run's triggering revision: a pushed v* tag runs the full release, workflow_dispatch keeps dry-run and create-gh-release and has no release-ref input, and tag-triggered releases fail when the tag and built package version differ (folded in from the former stacked PR ci(release): publish tagged releases from triggering ref #249, following Safe Synthesizer's tag-push strategy in Safe-Synthesizer refactor(graph): name the operation scope phase_scoped #361)
  • document the release sequence in DEVELOPMENT.md, including that a manual dry run works only on an existing release tag and that create-gh-release is not tied to dry-run
  • install each isolated benchmark target's locked development group before running measurement tooling
  • run wheel builds and documentation builds and deploys through repository-owned Mise tasks
  • harden Mise installation with the fingerprint-pinned, versioned signed installer and a file-backed unsigned fallback download

Command tree

Intent Command
Onboarding mise run setup [runtime|dev|docs|notebooks|all]
Dependency sync mise run deps:sync [runtime|dev|docs|notebooks|all]
Repository hooks mise run hooks:install
Static checks mise run check and mise run check:*
Default local gate mise run check ::: test
Tests mise run test, mise run test:all, mise run test:coverage, mise run test:e2e
Documentation mise run docs:build, mise run docs:serve, mise run docs:deploy VERSION
Notebooks mise run notebooks:execute
Build mise run build:wheel
Lock update mise run lock:update

Task names follow <domain>[:<action>[:<qualifier>...]]. Arguments select profiles or values. Commands containing check preserve tracked files. Mutating tasks state their outputs in mise tasks.

The default local gate stays explicit because it runs only static checks and unit tests; affected specialized suites remain separate. No validate task or removed Make compatibility alias is restored.

Task and environment boundaries

  • Mise owns pinned tools, environment variables, virtual-environment activation, and task composition.
  • uv owns Python dependency resolution and the contents of the project environment.
  • uv run --locked --group <profile> makes leaf tasks independently runnable and synchronizes the requested locked profile.
  • deps:sync remains the explicit command for preparing a profile without running another task.
  • setup remains the shared local and CI entry point. CI uses --no-hooks to skip repository hook installation.
  • Historical benchmark refs supply the target source only. Workflow YAML, local actions, and Mise tasks always come from the PR revision. Release jobs use one revision, the triggering ref, so orchestration, setup, build, and docs cannot diverge.

Base and compatibility

  • rebased onto main at 7605c13c
  • keep data-designer>=0.10.0,<0.11 and required-version = ">=0.13.4" for uv from main; the only pyproject.toml change is the Ruff and ty pins, and uv.lock was regenerated for them
  • retain only make help, make install-mise, and make setup
  • use make setup as the bootstrap path for machines without Mise
  • keep Ruff and ty versions aligned between Mise and the uv development environment
  • reject unknown dependency profiles through Mise usage validation
  • preserve the Python version requested by each CI matrix job

Migration audits

  • Ruff 0.17: enable E9, RUF100, UP015, UP017, UP035, and UP037; apply safe fixes; format and lint tracked Python files and rendered notebooks. The incidental library and tooling formatting edits are mechanical Ruff migrations with no intended behavior change.
  • uv 0.13: use locked synchronization for every dependency profile and locked project execution where applicable.
  • ty 0.0.90: preserve the blocking repository check and verify that the configured docs path discovers all rendered notebooks.
  • pre-commit: run the blocking aggregate check, including read-only lock verification; lock regeneration remains the explicit lock:update operation.
  • Make: confirm no repository automation or documentation depends on the removed forwarding aliases.

Security

GitHub Actions disables the unsigned fallback. When Mise must be installed, the pinned GPG-verified installer is required and setup fails if verification cannot complete. The installer fetches the pinned release key through HTTPS, verifies its fingerprint, downloads the signed installer from the exact pinned Mise release, and avoids dirmngr keyserver hangs. Local installation retains its documented unsigned fallback for slim development environments, but downloads the script completely to a temporary file before execution so curl retries cannot concatenate partial responses.

The installer runs the downloaded script only when the pinned release key signed it. Before commit 2592232, it only checked that the pinned key was present, so a second key added to the download could pass.

Scope of this check, in plain terms: automated review (Greptile and a Claude agent pass) found most of these installer and bootstrap issues, not a known attack. The threat is narrow. It matters only if someone can tamper with both downloads while Mise is being installed, for example through a TLS-intercepting proxy. It does not cover local installs without gpg (they use the unsigned fallback) or the tools Mise installs afterward. It is cheap to keep and makes the signed path do what it says; it is not a strong defense on its own.

The PyPI task validates credentials before building, requires an explicit version, keeps credentials out of process arguments, and supports an upload-free --dry-run.

Validation

  • mise tasks validate: all 26 tasks passed
  • mise run deps:sync all: every locked dependency group synchronized successfully
  • mise run --dry-run test:all: resolved to the unit suite followed by the opt-in E2E suite
  • mise run check passed (Ruff, ty, lock, formatting, SPDX); mise run test: 1801 passed, 2 skipped, run before the release commits were folded in. CI runs the full suite on the final head.
  • mise run docs:build: strict documentation build passed
  • mise run test -- tests/tools/test_mise_configuration.py -q: 25 focused workflow and tooling tests passed
  • benchmark shell regression coverage passed on Bash 5.2 and Bash 3.2.57
  • bash -n tools/install-mise.sh: signed-installer script syntax passed

@binaryaaron
binaryaaron force-pushed the binaryaaron/add-mise branch from 2dae1b6 to e7b17d2 Compare July 13, 2026 21:40
@binaryaaron
binaryaaron force-pushed the binaryaaron/add-mise branch from e7b17d2 to f1e7c30 Compare August 4, 2026 23:09
@binaryaaron
binaryaaron changed the base branch from main to binaryaaron/chore/blocking-ty August 4, 2026 23:09
@binaryaaron
binaryaaron force-pushed the binaryaaron/chore/blocking-ty branch from 6251891 to bb7e5c8 Compare August 10, 2026 17:09
@binaryaaron
binaryaaron force-pushed the binaryaaron/add-mise branch from 8f8ca82 to 8a8eeb4 Compare August 10, 2026 17:22
Base automatically changed from binaryaaron/chore/blocking-ty to main August 10, 2026 21:03
@binaryaaron
binaryaaron force-pushed the binaryaaron/add-mise branch 2 times, most recently from c32e672 to 42ba54c Compare August 11, 2026 23:54
@binaryaaron
binaryaaron marked this pull request as ready for review August 11, 2026 23:56
@binaryaaron
binaryaaron requested review from a team as code owners August 11, 2026 23:56
@greptile-apps

greptile-apps Bot commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The PR appears safe to merge; no new actionable issues were found.

Summary

Moves developer commands and CI to shared Mise tasks with pinned tools.

  • Developers and CI now share Mise tasks for common repository work.
  • Benchmark runs keep current orchestration and check out target code separately.
  • Pushing a version tag now builds and publishes that release from the tag.
  • The bootstrap script installs Mise through a pinned installer path.
  • Repository checks now include rendered notebooks and use pinned Ruff and ty versions.
  • Model-selection overrides now pass through the selection validators.

The latest documentation changes match the supplied tasks and workflows. The earlier benchmark and installer findings are addressed; no new actionable issues were found.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Local[Developer] --> Tasks[Mise tasks]
  CI[GitHub Actions] --> Tasks
  Tasks --> Setup[Setup pinned tools]
  Tasks --> Checks[Read-only checks]
  Tasks --> Tests[Unit and optional E2E tests]
  Tasks --> Build[Wheel and documentation builds]
  Setup --> UV[uv locked dependencies]
Loading

Reviews (15) · Last reviewed commit: "docs(dev): document the tag-push release..." · Reviewed by Greptile

Comment thread .github/workflows/benchmark-ci.yml Resolved
Comment thread .github/workflows/benchmark-ci.yml Resolved
Comment thread .github/workflows/release.yml Resolved
Comment thread tools/install-mise.sh Resolved
Comment thread .mise/tasks/benchmark Resolved
Comment thread .mise/tasks/benchmark Resolved

@andreatnvidia andreatnvidia left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Mise direction looks good. I found three blockers around historical releases, the signed bootstrap, and macOS benchmark execution. There’s also one non-blocking argument-forwarding bug in the same benchmark task.

Comment thread .github/workflows/benchmark-ci.yml Resolved
@binaryaaron

Copy link
Copy Markdown
Collaborator Author

@andreatnvidia re-review requested. The branch is rebased onto current main and force-pushed; your three blockers and the benchmark argument-forwarding issue were addressed earlier in 019efb4 (release checkout), 7ab8add (signed installer), and 32c79b5 (Bash 3.2 and quoting), and are unchanged by the rebase apart from the conflict resolutions below.

Changes since your review:

  • Rebase conflicts: main moved notebooks from a dependency group to an extra, so the notebooks profile, setup all and notebooks:execute use --extra notebooks, and build-notebooks.yml keeps main's GLiNER2 cache and wheel build through mise tasks. Workflow actions follow main's v7/v8 bumps.
  • Tool versions: uv 0.13.4 (main now requires >=0.13.4), ruff 0.17.1, ty 0.0.90, mise min_version 2026.10.15. Both lock files are regenerated.
  • Cooldown: minimum_release_age (renamed from install_before) goes from 7d to 3d, which admits uv 0.13.4.
  • ty 0.0.90 fixes: 18 diagnostics resolved. Seven @contextmanager return annotations became Generator, one redundant cast and one unused ignore were removed, and seven intentional pydantic/DataDesigner field overrides carry targeted ty: ignore comments.

Local validation: mise run check and mise run test (1578 tests) pass. CI is running on the new head. The stacked release follow-up (#249) is rebased onto this branch as well.

Comment thread tools/install-mise.sh Resolved

@andreatnvidia andreatnvidia left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on the signing-key issue Greptile flagged in the open inline thread. The installer checks that the pinned key is present, but also accepts signatures from other imported keys. Could we verify the successful signer against MISE_GPG_KEY and add a regression test for a bundle containing an extra key?

binaryaaron and others added 24 commits November 10, 2026 06:28
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Rebasing onto main raised the uv floor to 0.13.4, so the mise uv pin
moves from 0.12.3 to 0.13.4. The mise minimum_release_age (renamed from
install_before) drops from 7d to 3d, which admits uv 0.13.4. Within that
window the newest releases are uv 0.13.4, ruff 0.17.1, and ty 0.0.90;
mise min_version moves to 2026.10.15 and both lock files are regenerated.

ty 0.0.90 reports 18 diagnostics. Seven deprecated context-manager
Iterator annotations become Generator, one redundant cast and one unused
ignore are removed, and `asyncio.iscoroutinefunction` becomes the
`inspect` equivalent. The seven intentional pydantic and DataDesigner
field-narrowing overrides carry targeted `ty: ignore` comments. Ruff
0.17's UP037 finding in Detect is fixed. Workflow actions follow main's
v7/v8 bumps.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
The signed mise installer path only checked that the pinned key was in the
keyring, then ran `gpg --decrypt`, which accepts a signature from any key in
it. A bundle holding the pinned key plus a second key could run an installer
signed by the second key even with MISE_REQUIRE_SIGNED_INSTALL=1. The installer
now requires a VALIDSIG whose primary-key fingerprint equals MISE_GPG_KEY.

The setup action exported UV_PYTHON only within its own step, so later `uv run`
steps fell back to .python-version (3.11) in every matrix job. The action now
writes UV_PYTHON to GITHUB_ENV, and the test job asserts the interpreter
version before running tests.

Tests run the real installer against a stub curl serving a two-key bundle, and
cover the positive and rejected signers. They use a short scratch directory for
gpg-agent sockets, stop the agents afterward, and skip on gpg older than 2.2.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Main added or changed files after the previous rebase that trip the rules
this branch enables: unused noqa directives (RUF100), import order, three
`@contextmanager` return annotations that ty 0.0.90 flags as deprecated
(`Iterator` to `Generator`), one redundant cast, one unused `ty: ignore`, and
one intentional override of `BaseHTTPRequestHandler.server`.

No behavior change. The sync `_dispatch_chunk` that main removed is not
restored; the Ruff-only edits this branch made to it are dropped with it.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
@binaryaaron

Copy link
Copy Markdown
Collaborator Author

@andreatnvidia re-review requested. Both of your open threads are addressed in 2592232, and the branch is rebased onto current main (CI had not run since the earlier conflict; it now passes, including Test (3.11/3.12/3.13) with the new interpreter assert).

  • Installer signer: the signed path runs the payload only when gpg reports a valid signature from MISE_GPG_KEY. Two tests run the real script against a bundle with an extra key; the extra-key case fails on the old script. The same fix is open for Safe-Synthesizer in fix(tools): require the pinned key to sign the mise installer Safe-Synthesizer#820.
  • UV_PYTHON: the setup action persists it through GITHUB_ENV, and each matrix job asserts its interpreter before the tests.
  • Rebase: main removed the sync _dispatch_chunk; I took main's version of the three conflicting files and added one commit (8e06f96) with the Ruff and ty fixes that newer main code needs.
  • Scope: I added a note to the description that the signer check is narrow and mostly came from automated review.

Release follow-up #249 is rebased on this branch too.

@binaryaaron

Copy link
Copy Markdown
Collaborator Author

@andreatnvidia re-review requested. I folded the stacked release follow-up (#249) into this branch: two commits on top of the head you approved, 71ee4b09 (publish tagged releases from the triggering ref) and 6e7a216e (reject mismatched version tags). The head is now 6e7a216e; 8e06f96f and everything below it is unchanged.

What the two commits change, all in the release path:

  • release.yml: a pushed v* tag runs the full release; workflow_dispatch keeps dry-run and create-gh-release with no release-ref input. Every job uses its ordinary checkout, so the release-target directory and the project-directory arguments on build:wheel, docs:build and docs:deploy are gone. This closes the release.yml thread by removing the second checkout.
  • Tag-triggered releases fail when the tag and the built package version differ.
  • Tests and DEVELOPMENT.md updated to match (5 files, +96/-134).

The description is updated. I will close #249 once this lands on the branch.

Pushing a `v*` tag publishes to PyPI, creates the GitHub release, and deploys
docs with no confirmation step, and DEVELOPMENT.md only said that release.yml
handles publishing. Describe the sequence and the manual dry run, including
two limits found in review:

- The version comes from Git tags, so a commit without a tag builds a
  `+<hash>` version that PyPI and Test PyPI do not accept. There is no
  rehearsal before the tag exists; the manual dry run works on an existing
  release tag.
- `create-gh-release` defaults to true and is not gated on `dry-run`, so a
  dry run with the default still creates the GitHub release and deploys docs.

Also mention `publish:pypi` as a manual fallback that skips Test PyPI and the
tag check, state the minimum Mise version, run `mise trust` before
`mise run setup`, drop the uv prerequisite that Mise already provides, and fix
CONTRIBUTING's description of the commit hook, which runs `mise run check` on
every commit.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
@binaryaaron

Copy link
Copy Markdown
Collaborator Author

@andreatnvidia one addition to the re-review request. Your DEVELOPMENT.md comment from #249 (document the release sequence and the manual dry-run) was not addressed when #249 was folded into this branch. Commit 5ebd2f1 adds it to the Releases section, and I replied on your original thread.

A docs review while writing it found two things you may want to weigh in on:

  • A rehearsal before the tag exists does not work today. The version comes from Git tags, so an untagged commit builds a +<hash> version that PyPI and Test PyPI reject, and creating the tag starts the production release. The docs say so; the manual dry run works only on an existing release tag.
  • create-gh-release defaults to true and is not gated on dry-run, so a dry run at the default still creates the GitHub release and deploys docs. The docs say to turn it off.

The same commit also corrects setup guidance (minimum Mise version, mise trust before mise run setup, the hook description in CONTRIBUTING) and the PR description's stale tool versions.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants