Add a Gateway resource for Task network policy #549

Activity

1. rakyll

   added

   P0

   on Nov 2, 2026
2. yongruilin commented on Nov 2, 2026

   

   yongruilin

   on Nov 2, 2026

   Collaborator

   More actions

   +1

   On the two open questions:

   1. I prefer to have it. When a Task binds a Model, ax-server should automatically contribute the provider's https rule + credential injection, so standard tasks work out of the box without needing a policy manifest unless they require extra hosts (e.g. GitHub) or inbound access.
   2. For naming, how about NetworkPolicy or TrafficPolicy? K8s Gateway API and Substrate (egress gateway) both use Gateway for proxy infra, whereas IIUC this resource declares per-task traffic and credential rules.
3. yongruilin commented on Nov 2, 2026

   

   yongruilin

   on Nov 2, 2026

   Collaborator

   More actions

   This is part of the Task lifecycle. cc @siyuanfoundation
4. yongruilin

   self-assigned this

   on Nov 2, 2026
5. yongruilin commented on Nov 2, 2026

   

   yongruilin

   on Nov 2, 2026

   Last edited by yongruilin

   Collaborator

   More actions

   @rakyll Here's a proto sketch, named TrafficPolicy: in Substrate, "gateway" is the proxy that enforces these rules (agent-substrate/substrate#3258 is about swapping it), and NetworkPolicy collides with the Kubernetes NetworkPolicy Substrate puts on worker pools. Let me know if it makes sense?

   message TaskSpec {
     // ...
     // traffic_policy binds the TrafficPolicy whose rules allow the task's
     // network traffic. A task without one has no network access.
     TrafficPolicyRef traffic_policy = 8;
   }
   
   message TrafficPolicyRef {
     string name = 1;
   }
   
   // --- TrafficPolicy ---
   
   // TrafficPolicy declares which traffic may reach and leave the Tasks that bind
   // it. Both directions are allowlists: traffic that matches no rule is denied.
   message TrafficPolicy {
     string api_version = 1;
     string kind = 2;
     ObjectMeta metadata = 3;
     TrafficPolicySpec spec = 4;
   }
   
   message TrafficPolicySpec {
     // ingress must have no rules until Agent Substrate can enforce them per
     // actor (agent-substrate/substrate#431). Until then, `ax apply` rejects
     // non-empty ingress.rules and the router's cluster-wide allowlist decides
     // who may call a task.
     Ingress ingress = 1;
     Egress egress = 2;
   }
   
   // Ingress lists who may call the tasks that bind the policy.
   message Ingress {
     repeated IngressRule rules = 1;
   }
   
   // IngressRule lets callers whose SPIFFE ID is listed in from reach the listed
   // ports of the task.
   message IngressRule {
     IngressPeer from = 1;
     Ports ports = 2;
   }
   
   message IngressPeer {
     repeated string spiffe_ids = 1;
   }
   
   // Egress is copied into the egress policy of a bound task's actor on Agent
   // Substrate when the task is created; later edits or deletes of the policy
   // don't change existing tasks. Its rules are an unordered set: when several
   // match a destination, the most specific hostname decides (an exact name, then
   // "*.domain", then "*"), and no two rules may match the same hostname pattern
   // on the same port.
   message Egress {
     repeated EgressRule rules = 1;
   }
   
   // EgressRule sets exactly one of http, https, and tls_passthrough.
   message EgressRule {
     HttpRule http = 1;
     HttpsRule https = 2;
     TlsPassthroughRule tls_passthrough = 3;
   }
   
   // HttpRule allows cleartext HTTP to hostnames.
   message HttpRule {
     repeated string hostnames = 1;
     // ports defaults to 80.
     Ports ports = 2;
   }
   
   // HttpsRule allows HTTPS to hostnames. The egress gateway terminates TLS to
   // inspect each request, so the task must trust the gateway's CA; AX provides
   // it to tasks whose policy has an https rule.
   message HttpsRule {
     repeated string hostnames = 1;
     // ports defaults to 443.
     Ports ports = 2;
     // credentials are added by the egress gateway, so secrets never enter the
     // sandbox. They are not allowed on a rule with the hostname "*".
     repeated CredentialHeader credentials = 3;
   }
   
   // TlsPassthroughRule allows TLS to hostnames, forwarded without decryption.
   message TlsPassthroughRule {
     repeated string hostnames = 1;
     // ports is required.
     Ports ports = 2;
   }
   
   message Ports {
     repeated int32 numbers = 1;
   }
   
   // CredentialHeader replaces the value of a request header the task sends, such
   // as `x-api-key: placeholder`, with prefix followed by the secret's value.
   // Requests without the header are forwarded unchanged.
   message CredentialHeader {
     string header = 1;
     string prefix = 2;
     // secret_key is a key of a Kubernetes Secret in the namespace named after
     // the policy's atespace, as for Model.spec.secret_key.
     SecretKeyRef secret_key = 3;
   }

   • RPCs and ax apply/get/describe/delete trafficpolicy mirror Model; creating a task that binds a missing policy fails.
   • First PR ("Part of Add a Gateway resource for Task network policy #549") ships egress + credential injection (ate-secret://k8s.io/...) and rejects non-empty ingress.rules until Enhancing Network Ingress agent-substrate/substrate#431 lands.
   • Follow-ups: TaskSpec.model for automatic Model rules (Q1), plus setup-scoped egress for Workspace.
6. mastersingh24 commented on Nov 3, 2026

   

   mastersingh24

   on Nov 3, 2026

   Contributor

   More actions

   +1. We've run both directions on our fork for a few weeks, on Agent Substrate v0.5 and now v1.0.0-rc.1, so here's what we learned in case it helps the design.

   Egress. Ours is a list of host rules on the Task with injected credentials, which is close to the egress half here. Our triage agents call Vertex AI and the GKE API like this:

   egress:
     - hosts: [aiplatform.googleapis.com, container.googleapis.com]
       credentials:
         - header: Authorization
           prefix: "Bearer "
           credentialUri: ate-secret://k8s.io/default/ate-credentials/google-token/token

   It has worked well. The agent never holds the token, and the egress gateway swaps in a fresh one on every request. Three things came up:

   • +1 to the first open question. Nearly every task needs its model provider's rule, and we wrote it into every template by hand.
   • Copying the rules into the actor's policy at create time is fine, but it should be documented. Changing a host list means re-creating the task, and someone will expect an edit to apply.
   • Credential URIs name a Kubernetes Secret in Substrate's own scheme. Documenting the scheme next to the rule would save people a dig through Substrate's source.

   Ingress. Until per-actor ingress lands in Substrate, the router's CONNECT path already reaches any port in the actor (my last comment on #501). The cluster-wide allowlist decides who can do that. On v1.0 any listed client can reach any actor: we list ax-server, our incident watcher, a dashboard, our e2e runner, and kubectl ate proxy's atenet-client identity. So per-task from.spiffeIds would be a real improvement.

   One thing the proposal doesn't mention, and the reason we kept a small patch: waking up cleanly. When a request wakes a suspended task, the router waits for Substrate's wakeup probe, which asks the runner's /readyz. The runner reports ready before the agent's own server is listening again, so the first request after a resume could fail. Our patch lets the Task name its server port and makes /readyz wait until that port accepts connections. If ingress rules list ports anyway, the runner could take its readiness port from there, or the Task could name it separately. Either would let us drop our patch.

   Naming. TrafficPolicy reads well to us, and avoids both Substrate's "egress gateway" and Kubernetes NetworkPolicy.

   When this lands, we'll move our fork's egress and port fields onto it.
7. yongruilin

   added a parent issue on Nov 9, 2026

   • Epic: v1alpha1 API updates (Profile, Workspace, Task, Gateway) #563