Skip to content

fuse: configurable MaxBackground from go-fuse, minor cleanups#222

Merged
EdSchouten merged 3 commits into
buildbarn:mainfrom
aron-muon:aron/fuse-max-background
May 13, 2026
Merged

fuse: configurable MaxBackground from go-fuse, minor cleanups#222
EdSchouten merged 3 commits into
buildbarn:mainfrom
aron-muon:aron/fuse-max-background

Conversation

@aron-muon
Copy link
Copy Markdown
Contributor

@aron-muon aron-muon commented May 3, 2026

  • expose maximum_background in FUSEMountConfiguration

    • Adds a maximum_background field to FUSEMountConfiguration. When set, it caps the FUSE asynchronous request queue (FUSE_INIT max_background); when zero, go-fuse's default of 12 is used.

    • The default of 12 suits bb_clientd, but is too low for bb_worker, where a single mount serves many concurrent actions. The kernel CongestionThreshold is max_background * 3/4, and once it's reached, reads beyond it queue in D-state, visible as the worker wedging while continuing to report actions as executing.

    • Our recommended value for bb_worker is 1024. Validated in production at Muon Space against full //... Rust builds.

  • bump bb-storage to upstream/main HEAD

    • Picks up buildbarn/bb-storage#332 ("Forward termination signals to child process"). Without it, processes running as PID 1 in containers ignore SIGTERM and the kubelet has to fall back to SIGKILL once the grace period expires. For bb_worker that means in-flight actions get killed and have to be re-executed elsewhere. Bumps both go.mod (gazelle's go_deps source) and the git_override in MODULE.bazel that controls bzlmod resolution.
  • load proto_library from @protobuf//bazel

    • Routine cleanup: @rules_proto//proto:defs.bzl is deprecated in favour of @protobuf//bazel:proto_library.bzl in newer protobuf module versions. Same change across all 14 pkg/proto/**/BUILD.bazel files.

Comment thread pkg/filesystem/virtual/configuration/fuse_mount_enabled.go Outdated
@aron-muon aron-muon force-pushed the aron/fuse-max-background branch 2 times, most recently from 3fa2050 to 7dae9bd Compare May 13, 2026 10:34
@aron-muon aron-muon changed the title fuse: bump MaxBackground from go-fuse default 12 to 1024 fuse: configurable MaxBackground from go-fuse, minor cleanups May 13, 2026
@aron-muon aron-muon marked this pull request as ready for review May 13, 2026 10:39
@aron-muon aron-muon requested a review from EdSchouten May 13, 2026 10:39
Copy link
Copy Markdown
Member

@EdSchouten EdSchouten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Aron,
LGTM, though I have a couple of tiny remarks. Let me know what you think!

Comment thread pkg/proto/configuration/filesystem/virtual/virtual.proto Outdated
Comment thread pkg/proto/configuration/filesystem/virtual/virtual.proto Outdated
aron-muon added 3 commits May 13, 2026 12:10
Add a maximum_background field to FUSEMountConfiguration. When set, it
caps the FUSE asynchronous request queue (FUSE_INIT max_background); when
zero, go-fuse's default of 12 is used.

The default of 12 suits bb_clientd, but is too low for bb_worker, where
a single mount serves many concurrent actions and the kernel
CongestionThreshold (max_background * 3/4) is reached almost as soon as
multiple actions begin reading. Reads beyond the threshold queue in
kernel D-state, which presents as the worker wedging while reporting
actions as still executing.

Recommended value for bb_worker is 1024.
Picks up commit d0c6f26 ("Forward termination signals to child process",
PR #332). Without that fix, processes running as PID 1 in containers
silently ignore SIGTERM and the kubelet has to fall back to SIGKILL
once the grace period expires. For bb_worker that means in-flight
actions are killed and the scheduler has to retry them on another worker.

Bumps both go.mod (gazelle's go_deps source) and the git_override
in MODULE.bazel that actually controls bzlmod resolution; they need
to stay in sync.
The @rules_proto//proto:defs.bzl source has been deprecated in favour
of @protobuf//bazel:proto_library.bzl in newer protobuf module versions.
This commit switches all proto_library load statements over.
@aron-muon aron-muon force-pushed the aron/fuse-max-background branch from 7dae9bd to 69ad305 Compare May 13, 2026 11:10
@EdSchouten EdSchouten merged commit ed02b7a into buildbarn:main May 13, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants