Repository navigation
Pin go-keyring's per-OS secret size caps behind #7707 - #7727
Conversation
`databricks auth login` fails on macOS with `store token: data passed to Set was too big` when the U2M access token is ~3250 bytes (#7707). zalando/go-keyring v0.2.8 caps the base64-encoded `security -i add-generic-password` line at 4096 bytes on macOS and the credential blob at 2560 bytes on Windows, and the CLI stores the whole token envelope in one keyring item, so the access token alone exceeds both caps. The test calls the real backend with a token of that size and asserts ErrSetDataTooBig. The size check runs before the keyring is contacted, so it is deterministic on CI runners with a locked keychain. It pins the current behavior so a fix has to flip it on purpose. Co-authored-by: Isaac <no-reply@databricks.com>
✅ Integration test reportCommit: 5df7c96
Top 3 slowest tests (at least 2 minutes):
|
simonfaltum
left a comment
There was a problem hiding this comment.
🤖 Automated review from simonfaltum's review agent (pr-review skill) at afcd304. Each finding was verified against the code; simonfaltum may follow up.
No blocking issues found, approving. 4 non-blocking comment(s) and 1 question(s) below.
This PR adds a macOS/Windows-only test that sends a 3250-byte access token through the real go-keyring v0.2.8 backend and asserts ErrSetDataTooBig. I checked the macOS (4096-byte command line after base64) and Windows (2560-byte raw blob) limits against the library source, and the test passes on both CI cells. I found nothing blocking, only four small comment/cleanup nits. One open question is whether a test that pins the bug is worth merging before the #7707 fix, or whether the regression test should land with the fix.
Questions
- This test asserts the current bug rather than the intended behavior, and only some fixes flip it: a fix inside
keyringStore.Put(chunking, storing only the refresh token) or a go-keyring bump that lifts the cap turns the macOS/Windows cells red, while a login-path fix (falling back to plaintext onErrSetDataTooBiginstoreLoginToken) leaves it green. It is also the only test that drives the real OS keyring; the rest of the package usesfakeBackend. Would you rather land the regression test with the #7707 fix, for example afakeBackendwith a size cap that asserts the token round-trips or login falls back, so it runs on every OS? Or is the goal to have CI catch go-keyring limit changes, in which case could the comment say that? (non-blocking)
| func TestKeyringStore_Put_RealBackendRejectsLargeToken(t *testing.T) { | ||
| store := NewKeyringStore() | ||
| account := keyringProbeAccountPrefix + uuid.New().String() | ||
| t.Cleanup(func() { |
There was a problem hiding this comment.
[nit] The t.Cleanup always runs store.Delete, which runs security delete-generic-password on macOS and a Credential Manager lookup on Windows. So every run contacts the real keyring, which contradicts the comment that the test does not need an unlocked keychain. On a locked keychain the Delete can wait up to the 3s timeout. Register the cleanup only when the write went through: err := store.Put(...); if err == nil { t.Cleanup(func() { _ = store.Delete(account) }) }. That also makes the 'only matters if the limit disappears' comment unnecessary.
| // The access token alone already exceeds both caps once base64-encoded, | ||
| // so the exact refresh token size is irrelevant. |
There was a problem hiding this comment.
[nit] go-keyring does not base64-encode on Windows. The 2560-byte cap applies to the raw JSON envelope, so access tokens from about 2.4 KB already fail there. Reword to something like "The access token alone exceeds both caps", and consider noting the lower Windows threshold.
| // Linux Secret Service has no such cap, hence the build constraint. This test | ||
| // pins the current behavior so a fix for #7707 has to flip it on purpose. |
There was a problem hiding this comment.
[nit] Only fixes that change what keyringStore.Put stores flip this test. A fix in the login path, for example falling back to plaintext on keyring.ErrSetDataTooBig in storeLoginToken (cmd/auth/login.go), would leave it green while users no longer see the error. Soften the comment to say it pins the storage-layer limit of go-keyring.
| // the same account store fine. | ||
| const issue7707AccessTokenBytes = 3250 | ||
|
|
||
| // The OS backends behind zalando/go-keyring cap the secret size and return |
There was a problem hiding this comment.
[nit] .agents/rules/comments.md asks for Go doc comments that start with the function name, but this one starts with "The OS backends behind zalando/go-keyring...". Start it with TestKeyringStore_Put_RealBackendRejectsLargeToken ....
There was a problem hiding this comment.
@simonfaltum This is not the case for test files.
simonfaltum
left a comment
There was a problem hiding this comment.
🤖 Automated review from simonfaltum's review agent (pr-review skill) at afcd304. Each finding was verified against the code; simonfaltum may follow up.
No blocking issues found, approving. 3 non-blocking comment(s) and 2 question(s) below.
This PR adds a test, built only on macOS and Windows, that sends a 3250-byte access token through the real go-keyring backend and asserts ErrSetDataTooBig. The size arithmetic and backend claims match go-keyring v0.2.8, and the test passed on the macOS and Windows CI cells. All checks are green. I left three nits: the comment's base64 wording, a cleanup that always calls the real keyring, and token fields that don't affect the result. I also asked two non-blocking questions: whether a fix in the login path would actually flip this test, and whether a real-keyring unit test is the right place for this.
Questions
- The doc comment says a fix for #7707 "has to flip it on purpose", but the test calls
keyringStore.Putdirectly. A fix in the login path, such as falling back to plaintext onErrSetDataTooBiginstoreLoginToken/applyLoginFallback, or a wrapper store that splits large secrets, would leave this test green. After such a fix, the test would still read as "large tokens are rejected". Would you rather (a) land the test together with the fix, flipped to assert that the store succeeds, or (b) reword the comment to say it only documents go-keyring's cap at thekeyringStorelayer? (non-blocking) - This would be the first unit test that calls the real OS keyring. Every other keyring test uses
fakeBackend, andcache.gosays the store factories exist "so unit tests can inject stubs without hitting the real OS keyring". If a go-keyring bump removes the cap, macOS and Windows developer runs would write a real secret to the keychain. On a locked CI keychain the test would fail with a 3sTimeoutErrorinstead of a clear message. Lint and the merge queue also run only on Linux, so this file is only built on PR macOS/Windows cells. Is a real-backend tripwire worth that, or would afakeBackend-based test of the CLI's handling (once #7707 is fixed) be enough? (non-blocking)
| _ = store.Delete(account) | ||
| }) | ||
|
|
||
| // The access token alone already exceeds both caps once base64-encoded, |
There was a problem hiding this comment.
[nit] go-keyring base64-encodes the secret only on macOS. keyring_windows.go checks the raw JSON with len(password) > 2560. On Windows, the 3250-byte token goes over the cap because of its raw size, not because of base64. A reader who changes the token size based on this comment would get the Windows cap wrong. Suggested wording: "The access token alone exceeds both caps (macOS after base64, Windows raw), so the exact refresh token size is irrelevant."
| func TestKeyringStore_Put_RealBackendRejectsLargeToken(t *testing.T) { | ||
| store := NewKeyringStore() | ||
| account := keyringProbeAccountPrefix + uuid.New().String() | ||
| t.Cleanup(func() { |
There was a problem hiding this comment.
[nit] t.Cleanup always runs store.Delete. On the expected path, that still runs security delete-generic-password on macOS or CredRead on Windows against the real store, even though the doc comment says the test does not need the keychain. On a locked or headless keychain, the cleanup can wait up to the 3s keyring timeout, and its error is thrown away. Register the cleanup only when Put returns nil (for example if err == nil { t.Cleanup(...) } before the assertion). That way the keyring is touched only in the case your comment describes.
|
|
||
| // The access token alone already exceeds both caps once base64-encoded, | ||
| // so the exact refresh token size is irrelevant. | ||
| tok := &oauth2.Token{ |
There was a problem hiding this comment.
[nit] TokenType, RefreshToken and Expiry don't affect the result. The 3250-byte access token alone makes the JSON larger than the roughly 2990-byte macOS budget and the 2560-byte Windows cap. Using &oauth2.Token{AccessToken: strings.Repeat("a", issue7707AccessTokenBytes)} matches how probeWithBackend builds its token, removes the time.Now() call, and lets you drop the comment on lines 41-42.
…eal keyring The first revision drove the real OS keyring from a unit test, which only built on the PR macOS and Windows cells, ran a delete against the keychain on every run, and would write a real secret if the cap were ever lifted. fakeBackend gains an optional size hook. The macOS rule (base64 secret on one `security -i` line, rejected over 4096 bytes) and the Windows rule (raw blob over 2560 bytes) are transcribed from go-keyring v0.2.8, so the test pins the same behavior on every OS without touching a keychain. The token shrinks to the access token alone, which exceeds both caps by itself. Co-authored-by: Isaac <no-reply@databricks.com>
The fakeBackend table test models go-keyring v0.2.8's caps and cannot notice when a bumped or replaced keyring library accepts larger secrets. Bring back the darwin/windows test against the real backend as that signal, with the review feedback folded in: it runs only when CI is set so local runs never depend on a keychain, the cleanup is registered only after a successful write so the expected path never contacts the keyring, the token is just the access token, and the failure message says which caps to re-verify. Co-authored-by: Isaac <no-reply@databricks.com>
Lookup reads a split entry again (25 ms doubling, 5 attempts) when its access token parts are missing or do not match, so a reader that catches another process writing or deleting the entry neither refreshes with a refresh token that process already used up nor restores a deleted login. Delete counts the parts and deletes them from the last, so a Delete that fails part way leaves parts that the next one still finds. A refresh whose part write fails still succeeds when the store kept the new refresh token: recoverStoreUpdate accepts that partial update, so losing the access token only costs the next command a refresh. auth login rejects #access-token- in any letter case, because Windows Credential Manager ignores case in credential names. On macOS, zalandoBackend rejects a secret that go-keyring is sure to reject without calling it, because go-keyring starts `security -i` before its size check and leaves the process unreaped. keyringEntry embeds a struct that Entry converts to and from again, so a field added to Entry cannot be dropped silently. Also: string slicing in putSplit, doc comments, the service name constant in the new tests, and the original wording of the comments from #7727 except where they became wrong. Co-authored-by: Isaac <no-reply@databricks.com>
Changes
Two tests pin how the CLI's keyring envelope fares against go-keyring's per-OS secret size caps. They carry different signals, which is why both exist:
TestKeyringStore_Put_OSSecretSizeCaps(every OS).fakeBackendgains an optional size hook, and the two rules are transcribed from go-keyring v0.2.8: on macOS the base64-encoded secret must fit a 4096-bytesecurity -icommand line, on Windows the raw credential blob must stay at or under 2560 bytes. A 3250-byte access token (the size from the issue below) is rejected under both rules, a 950-byte one round-trips, and the Linux Secret Service row has no cap. The test is keychain-agnostic and documents the thresholds and what the storage layer does with them.TestKeyringStore_Put_RealBackendRejectsLargeToken(macOS and Windows, CI only). Stores the same 3250-byte token through the real go-keyring backend and assertsErrSetDataTooBig. It is the only test that observes the real library, so it is what fails when a bumped or replaced keyring library accepts larger secrets and the modeled caps go stale. It skips unlessCIis set so local runs never depend on a keychain. go-keyring's size check fires before the OS keyring is contacted, so on the expected path nothing is written, and the cleanup is registered only after a successful write.No fix is included. A fix for the issue that changes what the keyring store writes must update these tests on purpose.
Why
#7707 reports
databricks auth loginfailing on macOS withstore token: data passed to Set was too bigfor workspaces whose U2M access tokens are about 3250 bytes. The reproduction confirmed the mechanism:add-generic-password -U -s <service> -a <account> -w <secret>intosecurity -i, rejecting the write when that line exceeds 4096 bytes (keyring_darwin.go). Upstreammasterstill has the same check.oauth2.Token(access token, refresh token, expiry) into one keyring item. A 3250-byte access token yields a ~3350-byte JSON envelope and a ~4540-byte command, so it is rejected even with no refresh token. The cap is reached at roughly a 2830-byte access token for a 10-character profile name.ProbeKeyringwrites a tiny token and succeeds, so the login path commits to the keyring and only the finalPutfails, which is why the error surfaces asstore token: ...instead of the plaintext fallback.Verified manually on macOS (outside CI): piping a 4538-byte
add-generic-passwordline intosecurity -istores a password truncated to 4038 bytes and reportsunknown commandfor the remaining bytes, so the tool reads at most 4095 bytes per line and go-keyring's check prevents a silently truncated token. The same 4480-byte secret passed as a command-line argument is stored in full, so the keychain item itself has no such limit; only thesecurity -ipath go-keyring uses does.No changelog fragment: test-only change.
Tests
go test ./libs/auth/storage/passes; the real-backend file is excluded there by its build constraint. golangci-lint andgo vetpass forGOOS=darwinandGOOS=windows.This pull request and its description were written by Isaac.