Checks
Controller Version
0.13.1
Deployment Method
Helm
Checks
To Reproduce
We started noticing a couple of days ago that runner listener pods are being recreated too frequently. We upgraded the controller version from 0.13.0 to 0.13.1, but we are still seeing the same behavior, so we are unsure which version this issue started in.
I am not sure how we can reproduce it, but I am going to provide the controller and listener logs to check.
Describe the bug
Runner scale set listener pods are frequently recreating with "reason":"Error"
Describe the expected behavior
Listener pod should be in a stable state - frequenty restart cause delay in CI/CD jobs.
Additional Context
|
if !errors.As(err, &expiredError) { |
--> this should only happen when the token expires, but we are not seeing any pattern like 30m, 60m.
In listener/controller pod logs - we are seeing below before it is being recreated. We have debug log level enabled for the controller, but don't see much information except below.
1768712941128 2026-01-18T05:09:01.128Z {"container":"listener","pod":"general-metadata-755cf869-listener","_entry":"I0118 05:09:00.959725 1 warnings.go:110] \"Warning: unknown field \\\"status.jobId\\\"\""}
1768712944385 2026-01-18T05:09:04.385Z {"container":"listener","pod":"general-metadata-755cf869-listener","_entry":"2026/01/18 05:09:04 Application returned an error: failed to get message: failed to get next message: actions error: StatusCode 200, AcivityId \"\": EOF"}
1768712945426 2026-01-18T05:09:05.426Z {"container":"manager","level":"info","pod":"arc-controller-67c9bf8ccb-nb528","_entry":"{\"severity\":\"info\",\"ts\":\"2026-01-18T05:09:05Z\",\"logger\":\"AutoscalingListener\",\"message\":\"Listener pod is terminated\",\"version\":\"0.13.1\",\"autoscalinglistener\":{\"name\":\"general-metadata-755cf869-listener\",\"namespace\":\"arc-system\"},\"namespace\":\"arc-system\",\"name\":\"general-metadata-755cf869-listener\",\"reason\":\"Error\",\"message\":\"\"}"}
1768712945426 2026-01-18T05:09:05.426Z {"container":"manager","level":"info","pod":"arc-controller-67c9bf8ccb-nb528","_entry":"{\"severity\":\"info\",\"ts\":\"2026-01-18T05:09:05Z\",\"logger\":\"AutoscalingListener\",\"message\":\"Deleting the listener pod\",\"version\":\"0.13.1\",\"autoscalinglistener\":{\"name\":\"general-metadata-755cf869-listener\",\"namespace\":\"arc-system\"},\"namespace\":\"arc-system\",\"name\":\"general-metadata-755cf869-listener\"}"}
Controller Logs
https://gist.github.com/rajesh-dhakad/2c8e2fc07746fab7a3deab71ab99e7bf
Runner Pod Logs
Checks
Controller Version
0.13.1
Deployment Method
Helm
Checks
To Reproduce
Describe the bug
Runner scale set listener pods are frequently recreating with "reason":"Error"
Describe the expected behavior
Listener pod should be in a stable state - frequenty restart cause delay in CI/CD jobs.
Additional Context
actions-runner-controller/cmd/ghalistener/listener/listener.go
Line 278 in d3ca9de
Controller Logs
Runner Pod Logs