Skip to content

chore(safego): remove unused WithRestartTimeout (fixes data race) - #1465

Open
kotwal-itpro wants to merge 1 commit into
jitsucom:newjitsufrom
kotwal-itpro:fix/safego-test-data-race
Open

kotwal-itpro wants to merge 1 commit into
jitsucom:newjitsufrom
kotwal-itpro:fix/safego-test-data-race

Conversation

@kotwal-itpro

Copy link
Copy Markdown

Problem

Execution.WithRestartTimeout mutates exec.restartTimeout from the caller's goroutine, while the spawned goroutine's panic-recovery defer reads that same field on restart. When a caller uses the documented RunWithRestart(f).WithRestartTimeout(t) builder pattern (as safego's own TestHandlePanicAndRestart did), the goroutine is already running by the time the setter fires — any panic in the interim races the write.

go test -race ./bulker/jitsubase/safego/ reproduces it deterministically on current newjitsu:

==================
WARNING: DATA RACE
Read at 0x... by goroutine 9:
  github.com/jitsucom/bulker/jitsubase/safego.(*Execution).run.func1.1()
      bulker/jitsubase/safego/safego.go:56 +0x70
  runtime.gopanic()
      runtime/panic.go:860 +0x128
  github.com/jitsucom/bulker/jitsubase/safego.(*Execution).run.func1()
      bulker/jitsubase/safego/safego.go:62 +0x78

Previous write at 0x... by goroutine 8:
  github.com/jitsucom/bulker/jitsubase/safego.(*Execution).WithRestartTimeout()
      bulker/jitsubase/safego/safego.go:68 +0x19c
  github.com/jitsucom/bulker/jitsubase/safego.TestHandlePanicAndRestart()
      bulker/jitsubase/safego/safego_test.go:24 +0x198
==================
    testing.go:1712: race detected during execution of test
--- FAIL: TestHandlePanicAndRestart (0.30s)
FAIL	github.com/jitsucom/bulker/jitsubase/safego	0.669s

safego.RunWithRestart / safego.Run are called in 20+ production paths across bulker/, ingest/, kafkabase/, sync-controller/, bulkerapp/, eventslog/, and jitsubase/, so the primitive is load-bearing. No current production caller happens to chain WithRestartTimeout, so the race does not fire in shipped code — but the pattern is publicly exported and documented, so any future caller (or a copy-paste from the existing test) trips it.

Fix (backward compatible)

  1. New constructor RunWithRestartTimeout(f func(), timeout time.Duration) *Execution — sets the timeout before the goroutine spawns, eliminating the need for the mutating builder pattern in the common case.
  2. run() captures restartTimeout into a function-local at spawn time, so a subsequent mutation via the deprecated setter (or any future mutation of the shared Execution) cannot race the goroutine's recovery handler.
  3. WithRestartTimeout retained for source-level backward compatibility and marked with a Deprecated: doc comment pointing callers at the race-free constructor.

Test

Updates TestHandlePanicAndRestart to use the new constructor and an atomic.Int32 counter (the previous plain-int shared counter was itself flagged by -race).

Verified

  • go test -race -count=5 ./bulker/jitsubase/safego/ — 5/5 pass (was consistently failing before)
  • go build ./... clean on jitsubase, kafkabase, bulkerlib, bulkerapp, ingest, sync-controller
  • No breaking API changes: Run, RunWithRestart, and WithRestartTimeout all keep their signatures; new RunWithRestartTimeout is additive

@kotwal-itpro
kotwal-itpro force-pushed the fix/safego-test-data-race branch from 5de640a to 7e33052 Compare August 30, 2026 01:39
@kotwal-itpro

Copy link
Copy Markdown
Author

Hey team, just checking in on this one — rebased against latest newjitsu today (was quite a way behind since safego hadn't been touched upstream). CI should re-run.

The fix captures restartTimeout at spawn time to eliminate the data race in WithRestartTimeout. safego is called in 20+ production paths across bulker, ingest, kafkabase, sync-controller, bulkerapp, and eventslog, so it seemed worth getting right. Happy to change the approach if you'd prefer a different sync primitive.

Thanks!

@kotwal-itpro
kotwal-itpro force-pushed the fix/safego-test-data-race branch from 7e33052 to 0bb049f Compare September 13, 2026 04:22
@kotwal-itpro

Copy link
Copy Markdown
Author

Rebased onto latest newjitsu to keep this current and mergeable (had drifted since the last sync). No changes to the fix itself — diff is identical, just replayed cleanly on top of recent upstream changes. CI should re-run on the new head.

@kotwal-itpro
kotwal-itpro force-pushed the fix/safego-test-data-race branch from 0bb049f to db98d43 Compare October 6, 2026 05:45
@kotwal-itpro

Copy link
Copy Markdown
Author

@absorbb @sahiltyagi-jitsu — would one of you have a few minutes to take a look at this? It's a small, self-contained fix (2 files in bulker/jitsubase/safego) for a data race in WithRestartTimeout: the restart timeout is now captured at spawn time instead of being read concurrently from the restarting goroutine. go test -race on the package passes.

I've rebased onto the latest newjitsu. CI hasn't run on this PR yet, presumably because it needs maintainer approval for a fork contribution, so approving the workflow run would be much appreciated. Happy to adjust anything. Thanks!

@absorbb

absorbb commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

@kotwal-itpro
i guess there is no need to keep WithRestartTimeout - it is not used anywhere

@kotwal-itpro
kotwal-itpro force-pushed the fix/safego-test-data-race branch from db98d43 to 19a12b4 Compare October 7, 2026 04:59
@kotwal-itpro kotwal-itpro changed the title fix(safego): eliminate WithRestartTimeout data race by capturing restartTimeout at spawn chore(safego): remove unused WithRestartTimeout (fixes data race) Oct 7, 2026
@kotwal-itpro

Copy link
Copy Markdown
Author

@absorbb good call, thanks. I've updated the PR to simply remove WithRestartTimeout: with no way to change restartTimeout after the goroutine starts, the race goes away with it. The test now builds the Execution directly with a 50ms restart timeout and uses an atomic counter, so go test -race passes. Net change: -5 lines in safego.go plus the test update.

WithRestartTimeout mutated Execution.restartTimeout after the goroutine
had already started, racing with the recover handler that reads it.
Nothing in the codebase calls it except its own test, so remove it
instead of patching the race. The test now constructs the Execution
directly with a short restart timeout and uses an atomic counter so it
passes go test -race.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kotwal-itpro
kotwal-itpro force-pushed the fix/safego-test-data-race branch from 19a12b4 to cbdef22 Compare October 7, 2026 05:00

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants