ref(git): Fold GrantV1 and GrantV2 into one Grant with optional rules

README.md

@@ -16,7 +16,7 @@ The platform exposes four subsystems:
 
 - **Actions** — outbound SCM operations initiated by Sentry code. The `SourceCodeManager` class provides 70+ methods covering comments, reactions, pull requests, branches, git objects, reviews, and check runs. With more actions planned to be added as we port more use cases.
 - **Actions RPC** - outbound SCM commands exposed over the network. The `SourceCodeManager` is fully available over the network. This enables usage of the SCM for services outside the monolith.
-- **Git relay** - `git` fetches and pushes for services that hold no provider credentials. Seer signs a grant naming the organization, operation and repositories with `scm.git.grant.sign_grant` and an Ed25519 private key only Seer holds. `scm.git.server.GitServer` verifies the grant against the matching public keys, so whoever verifies grants cannot mint one, and checks each git smart-HTTP request against it before relaying the request to the provider with a token, streaming both bodies. A missing or unusable key on either side raises `scm.errors.GitGrantKeyError`, the server's misconfiguration rather than a refused grant. `require_private_key` and `require_public_keys` check the keys at startup, and `generate_private_key` makes a key for tests and local tools. `key_id` gives a public key's `kid`, the id each grant names its signing key by, so a log can name the key that signed a grant without quoting it.
+- **Git relay** - `git` fetches and pushes for services that hold no provider credentials. Seer signs a grant with `scm.git.grant.sign_grant` and an Ed25519 private key only Seer holds. A `Grant` names the organization and a list of rules, each a repository, an operation (`FetchRule` or `PushRule`) and the branches it may touch, or `any_object` for a fetch that may read any object. A push rule names its branches or full-match `branch_patterns`, and `create_only` limits it to new branches. A request passes if any rule allows it. `scm.git.server.GitServer` verifies the grant against the matching public keys, so whoever verifies grants cannot mint one, and checks each git smart-HTTP request against it before relaying the request to the provider with a token, streaming both bodies. For a push under a push rule it first reads the ref updates, and answers a push that updates any other ref, deletes a branch or carries push options with a status report git shows as rejected refs. A missing or unusable key on either side raises `scm.errors.GitGrantKeyError`, the server's misconfiguration rather than a refused grant. `require_private_key` and `require_public_keys` check the keys at startup, and `generate_private_key` makes a key for tests and local tools. `key_id` gives a public key's `kid`, the id each grant names its signing key by, so a log can name the key that signed a grant without quoting it.
 - **Event Stream** — inbound webhook processing. SCM service-providers push events which are deserialized into typed, provider-neutral dataclasses (`CheckRunEvent`, `CommentEvent`, `PullRequestEvent`) which are then dispatched to registered listener functions.
 
 ### Why

bin/git-server

@@ -50,14 +50,15 @@ Configuration:
         GITLAB_BASE_URL                      Instance URL (default: https://gitlab.com)
 
 Serving:
-    The server prints a fetch grant and a push grant for the repository, valid
-    for an hour, with commands that pass them to git:
+    The server prints a grant to fetch any object of the repository and one to
+    push its branch my-branch, valid for an hour, with commands that pass them
+    to git:
 
         git -c http.extraHeader="X-Scm-Git-Grant: $FETCH_GRANT" \\
             clone http://127.0.0.1:8080/OWNER/REPO.git
 
     The URL path is the repository's path, which the server uses as its external
-    id, so a grant names the repository as (provider, path).
+    id, so a grant's rule names the repository as (provider, path).
 
 Checks:
     --check starts the server on a free port and runs git against it:
@@ -68,16 +69,18 @@ Checks:
         for credentials
       - a repository the token cannot see fails with the provider's own
         message, which the relay passes on
-    With --push it also commits a 256 KiB file on a new branch, pushes it with
-    a push grant, sending the pack chunked as git does for a large push, and
-    then deletes the branch. This writes to the repository, so point it at a
-    test repository.
+    With --push it also commits a 256 KiB file, pushes it with a push grant for
+    a new branch, sending the pack chunked as git does for a large push, and
+    checks that the grant can neither push another branch nor delete its own.
+    It then deletes the branch with the provider token. This writes to the
+    repository, so point it at a test repository.
 
     git runs without the user's or the system's git config, so credential
     helpers and URL rewrites cannot hide a failure.
 """
 
 import argparse
+import base64
 import importlib.machinery
 import importlib.util
 import logging
@@ -100,10 +103,10 @@ from scm.errors import GitGrantKeyError
 from scm.git.grant import generate_private_key, public_key_of, sign_grant
 from scm.git.relay import GitRelay
 from scm.git.server import GitServer
-from scm.git.types import GRANT_HEADER, GetToken, GitOperation, Grant, Readable
+from scm.git.types import GRANT_HEADER, FetchRule, GetToken, Grant, PushRule, Readable, Rule
 from scm.providers.github.provider import GitHubProvider
 from scm.providers.gitlab.provider import GitLabProvider
-from scm.types import Provider, ProviderName, Repository, RepositoryId
+from scm.types import GetGitRemoteProtocol, Provider, ProviderName, Repository, RepositoryId
 
 logger = logging.getLogger(__name__)
 
@@ -278,17 +281,20 @@ def make_git_server(
     )
 
 
-def grant_for(private_key: Secret, operation: GitOperation, repository_id: RepositoryId) -> str:
+def grant_for(private_key: Secret, rule: Rule) -> str:
     grant = Grant(
         organization_id=ORGANIZATION_ID,
         referrer=REFERRER,
-        operation=operation,
-        repositories=[repository_id],
+        rules=[rule],
         expires_at=int(time.time()) + GRANT_TTL,
     )
     return sign_grant(private_key, grant)
 
 
+def fetch_grant_for(private_key: Secret, repository_id: RepositoryId) -> str:
+    return grant_for(private_key, FetchRule(repository=repository_id, any_object=True))
+
+
 class GitHandler(BaseHTTPRequestHandler):
     """Serves /<repository path>.git/<route>, as Sentry's endpoint serves a repository's git routes.
 
@@ -408,8 +414,8 @@ def serve(backend: Backend, args: argparse.Namespace) -> None:
         (args.host, args.port), make_git_server(backend, args.public_key, args.ca_bundle), backend.provider_name
     )
     url = f"{server.url}/{backend.repository['name']}.git"
-    fetch_grant = grant_for(args.private_key, "fetch", backend.repository_id)
-    push_grant = grant_for(args.private_key, "push", backend.repository_id)
+    fetch_grant = fetch_grant_for(args.private_key, backend.repository_id)
+    push_grant = grant_for(args.private_key, PushRule(repository=backend.repository_id, branches=["my-branch"]))
     logger.info(
         f"Serving {backend.provider_name} repository {backend.repository['name']} at {url}\n\n"
         f"Grants, valid for {GRANT_TTL // 60} minutes:\n"
@@ -474,11 +480,17 @@ class Checks:
         # git shows the text of a refused first request as "remote: <message>".
         return self.check(name, result, result.returncode != 0 and f"remote: {message}" in result.stderr)
 
+    def expect_rejection(self, name: str, result: subprocess.CompletedProcess[str], reason: str) -> bool:
+        # git shows a ref the server's status report refuses as "! [remote rejected] <ref> (<reason>)".
+        rejected = "[remote rejected]" in result.stderr and f"({reason})" in result.stderr
+        return self.check(name, result, result.returncode != 0 and rejected)
+
     def run(self) -> int:
         repository_id = self.backend.repository_id
         other_repository_id = (self.backend.provider_name, f"{self.backend.repository['name']}-other")
-        fetch_grant = grant_for(self.args.private_key, "fetch", repository_id)
-        push_grant = grant_for(self.args.private_key, "push", repository_id)
+        branch = f"scm-git-server-check/{int(time.time())}"
+        fetch_grant = fetch_grant_for(self.args.private_key, repository_id)
+        push_grant = grant_for(self.args.private_key, PushRule(repository=repository_id, branches=[branch]))
 
         server = self.start()
         url = f"{server.url}/{self.backend.repository['name']}.git"
@@ -490,16 +502,15 @@ class Checks:
         cloned = self.expect_success(
             "clone with a fetch grant", self.git(fetch_grant, "clone", "--depth=1", url, str(clone))
         )
-        branch = f"scm-git-server-check/{int(time.time())}"
         self.expect_refusal(
             "push with a fetch grant is refused",
             # Refused before the provider sees it, so any local repository will do.
             self.git(fetch_grant, "push", url, f"HEAD:refs/heads/{branch}", cwd=self.local_repository()),
-            "The git grant allows fetch, not push.",
+            "The git grant allows fetch on this repository, not push.",
         )
         self.expect_refusal(
             "a grant for another repository is refused",
-            self.git(grant_for(self.args.private_key, "fetch", other_repository_id), "ls-remote", url),
+            self.git(fetch_grant_for(self.args.private_key, other_repository_id), "ls-remote", url),
             "The git grant does not cover this repository.",
         )
         server.shutdown()
@@ -519,7 +530,7 @@ class Checks:
         missing = self.backend.repository_at(f"{self.backend.repository['name']}-does-not-exist")
         server = self.start(repository=missing)
         result = self.git(
-            grant_for(self.args.private_key, "fetch", repository_id_of(self.backend.provider_name, missing)),
+            fetch_grant_for(self.args.private_key, repository_id_of(self.backend.provider_name, missing)),
             "ls-remote",
             f"{server.url}/{missing['name']}.git",
         )
@@ -557,17 +568,46 @@ class Checks:
         (clone / "scm-git-server-check.bin").write_bytes(secrets.token_bytes(256 * 1024))
         self.git(push_grant, "add", "scm-git-server-check.bin", cwd=clone).check_returncode()
         self.git(push_grant, "commit", "-m", "scm-platform git-server check", cwd=clone).check_returncode()
+        self.expect_rejection(
+            "chunked push to a branch the grant does not list is rejected",
+            self.git(
+                push_grant, "-c", "http.postBuffer=65536", "push", url, f"HEAD:refs/heads/{branch}-unlisted", cwd=clone
+            ),
+            "the git grant does not cover this branch",
+        )
         pushed = self.expect_success(
             "chunked push with a push grant",
             self.git(push_grant, "-c", "http.postBuffer=65536", "push", url, f"HEAD:refs/heads/{branch}", cwd=clone),
         )
         if pushed:
-            self.expect_success(
-                f"delete the pushed branch {branch}",
+            self.expect_rejection(
+                "deleting the pushed branch with the push grant is rejected",
                 self.git(push_grant, "push", url, f":refs/heads/{branch}", cwd=clone),
+                "the git grant does not allow deleting a branch",
+            )
+            self.expect_success(
+                f"delete the pushed branch {branch} with the token", self.delete_directly(branch, clone)
             )
         server.shutdown()
 
+    def delete_directly(self, branch: str, clone: Path) -> subprocess.CompletedProcess[str]:
+        """Delete ``branch`` on the provider with the token Sentry would use, since a push grant cannot."""
+        provider = self.backend.fetch_provider(ORGANIZATION_ID, self.backend.repository)
+        assert isinstance(provider, GetGitRemoteProtocol)
+        remote = provider.get_git_remote()
+        token = self.backend.get_token(self.backend.repository, "push")
+        assert token is not None
+        credentials = base64.b64encode(f"{remote['basic_auth_username']}:{token.reveal()}".encode()).decode()
+        # Through the environment rather than the command line, which other processes can read.
+        env = {
+            **self.env,
+            "GIT_CONFIG_COUNT": "1",
+            "GIT_CONFIG_KEY_0": "http.extraHeader",
+            "GIT_CONFIG_VALUE_0": f"Authorization: Basic {credentials}",
+        }
+        command = ["git", "-c", "credential.helper=", "push", remote["url"], f":refs/heads/{branch}"]
+        return subprocess.run(command, cwd=clone, env=env, capture_output=True, text=True, timeout=300)
+
 
 def _indent(text: str) -> str:
     return "".join(f"     {line}\n" for line in text.splitlines())

src/scm/errors.py

@@ -3,6 +3,7 @@
 type ErrorCode = Literal[
     "git_grant_expired",
     "git_invalid_grant",
+    "git_malformed_request_body",
     "git_operation_not_granted",
     "git_provider_not_supported",
     "git_remote_not_https",
@@ -51,6 +52,7 @@
 ERROR_CODES: dict[ErrorCode, str] = {
     "git_grant_expired": "The git grant has expired.",
     "git_invalid_grant": "The git grant was missing, malformed or forged.",
+    "git_malformed_request_body": "The request body is not one the git endpoint can read.",
     "git_operation_not_granted": "The git grant does not allow this operation.",
     "git_provider_not_supported": "The repository's provider has no git remote.",
     "git_remote_not_https": "The repository's git remote is not an https URL.",
@@ -170,6 +172,10 @@ class GitInvalidGrant(GitSCMCodedError):
     code = "git_invalid_grant"
 
 
+class GitMalformedRequestBody(GitSCMCodedError):
+    code = "git_malformed_request_body"
+
+
 class GitOperationNotGranted(GitSCMCodedError):
     code = "git_operation_not_granted"
 

src/scm/git/grant.py

@@ -44,7 +44,7 @@
 # The only algorithm the verifier accepts, with parsed Ed25519 keys. A token that names HS256 and uses a public key,
 # which is no secret, as its HMAC key is refused, as is one that names ``none``.
 _ALGORITHM = "EdDSA"
-# Claims the JWT layer owns. Every other claim must be a Grant field.
+# Claims the JWT layer owns. Every other claim must be a field of the grant.
 _JWT_CLAIMS = frozenset({"aud", "exp", "iat"})
 
 
@@ -57,6 +57,7 @@ def sign_grant(private_key: Secret, grant: Grant, *, now: float | None = None) -
     """
     key = _load_private_key(private_key)
     claims: dict[str, Any] = msgspec.to_builtins(grant)
+    claims["version"] = grant.version
     claims["exp"] = claims.pop("expires_at")
     claims["iat"] = int(time.time() if now is None else now)
     claims["aud"] = GRANT_AUDIENCE
@@ -155,16 +156,24 @@ def verify_grant(public_keys: Sequence[str], token: str, *, now: float | None =
         raise GitInvalidGrant(detail=f"The git grant is invalid: {e}") from None
 
     fields = {name: value for name, value in claims.items() if name not in _JWT_CLAIMS}
+    # A grant from before there was a second version carries no version claim.
+    version = fields.pop("version", 1)
     try:
+        if type(version) is not int or version not in (1, 2):
+            raise msgspec.ValidationError(f"Unknown version {version!r}")
         grant = msgspec.convert({**fields, "expires_at": claims["exp"]}, type=Grant, strict=True)
+        if grant.version != version:
+            raise msgspec.ValidationError(
+                f"A version {version} grant cannot hold the fields of version {grant.version}"
+            )
     except msgspec.ValidationError as e:
         # The signature matched, so the holder of the private key wrote these claims; telling them why they are
         # invalid gives nothing away.
         raise GitInvalidGrant(detail=f"The git grant's payload is invalid: {e}") from None
 
     if grant.expires_at <= now:
         raise GitGrantExpired(detail="The git grant has expired.")
-    if not grant.repositories:
+    if grant.repositories is not None and not grant.repositories:
         raise GitInvalidGrant(detail="The git grant names no repositories.")
     return grant