Repository navigation
Conversation
ghchinoy
added a commit
to ghchinoy/ax
that referenced
this pull request
Nov 8, 2026
…e links, and actor UID (google#526, google#556)
Add residency, actor_uid, and outcome to TaskStatus and show RESIDENCY and OUTCOME in `ax get tasks`, and Residency, Outcome, and Actor UID in `ax describe task`. Residency names where the task's actor is in its Substrate lifecycle: queued, resuming, running, suspending, suspended, reverting, crashed, deleting, or deleted. Unlike phase, it tells a crashed or reverting actor apart from a task AX failed to provision. actor_uid is the actor's ate.actor.uid, which Substrate's logs and telemetry carry. The reconciler records both from the Actor that CreateActor, SuspendActor, and ResumeActor return, and the store keeps them, so each change AX makes is a stored change that state-change telemetry can observe. Reads still overlay the live actor, so a wake by the router or a recreated actor shows up on the next get. Phase is unchanged. outcome is defined but not set yet; command exit detection (google#346) will set it to succeeded, failed, or cancelled.
erain
force-pushed
the
task-residency-outcome
branch
from
November 10, 2026 16:47
2bf0e6b to
05bda21
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ax getshows only a task's phase, and the phase can't tell the cases operators ask about apart. A task whose actor Substrate marked crashed readsFailed, the same as a task whose template AX could not create. A reverting actor readsResuming. Nothing inTaskStatusnames the actor in Substrate's own telemetry, so ax-server records can't be joined with Substrate logs and events. Part of #13; see also #346 and #502.This PR adds three fields to
TaskStatusand shows them in the CLI:residency: where the task's actor is in its Substrate lifecycle. The values arequeued(no actor yet),resuming,running,suspending,suspended,reverting,crashed,deleting, anddeleted(the actor existed and is gone).actorUid: the actor's Substrate UID, theate.actor.uidon Substrate's logs and events. It changes if Substrate recreates the actor.outcome: how the command ended:succeeded,failed, orcancelled. Nothing sets it yet. Command exit detection (Task reports phase=Running and Ready=True indefinitely after its command has exited #346) will, and that change only needs to write the field.ax get tasksgainsRESIDENCYandOUTCOMEcolumns, andax describe taskgainsResidency,Outcome, andActor UID.phaseis unchanged.How the values are kept
Since #515, AX reads everything about the actor from Substrate on every get and list, and stores none of it.
residencyandactorUidare a deliberate exception. Planned state-change events (#13) will wrap the task store and emit an event whenever a write changes phase, residency, or outcome, so AX's own transitions have to show up as stored changes.CreateTaskstoresqueuedbefore it provisions the actor.CreateActor,SuspendActor, andResumeActorreturn.SuspendActornow returns the actor; its response always carried one.DeleteTaskstoresdeletingtogether with phaseTerminating.runningon the next get, and an actor deleted outside AX showsdeleted. When Substrate can't be reached, phase isUnknownand residency keeps its last recorded value.Keeping the stored residency in step with changes made outside AX, such as router wakes, is separate follow-up work and is not part of this PR.
Validation
go test ./.... The new tests cover:SuspendActorreturning the actor;postgres:17-alpine.ax get tasksandax describe taskfor residency, outcome, and an actor UID equal to the Substrate actor's.kubectl port-forwardto ax-server partway through, and every later test then failed to connect. ax-server neither restarted nor logged errors, and all eight tests passed when rerun.In the recording, a script drives the CLI against that cluster and fails on any unexpected state. It creates a task, resumes it, and reads the actor UID. It then deletes the worker pod hosting the actor: the task reads
Failedwith residencycrashed. Resuming reverts the actor torunning, and suspending givessuspended. The script and its setup notes are on a separate branch of my fork.This change was written with help from an AI coding assistant.