Log redacted+truncated HTTP bodies at debug level#187
Draft
parthban-db wants to merge 1 commit into
Draft
Conversation
|
Please ensure that the NEXT_CHANGELOG.md file is updated with any relevant changes. |
parthban-db
force-pushed
the
parthban-db/stack/bugbash-bug4-secret-logging
branch
from
b9aaa84 to
97b23bc
Compare
Restores HTTP body logging in the generated executeHttpCall / sendAndCheckError helpers, which had been dropped entirely to avoid leaking plaintext secrets (e.g. getSecret()). Bodies are now logged at debug the way the Go SDK does it: secret-bearing JSON fields (token, password, access_token, string_value, ...) are replaced with **REDACTED** and every value is truncated to debugTruncateBytes, so secrets no longer reach the logs while the debugging signal returns. Adds debugHeaders and debugTruncateBytes to ClientOptions (defaulting to false and 96, matching Go's DebugHeaders / DebugTruncateBytes). Headers are logged only when debugHeaders is true, and Authorization-family headers are always redacted to REDACTED. The redaction and truncation logic lives in a new, unit-tested @databricks/sdk-core/logger/debug module (onlyNBytes, redactedDumpBody, redactHeaders) exported under the ./logger/debug subpath. Regenerates utils.ts and client.ts across all packages to thread the new options and emit the redacted body/header logs. Adds core unit tests for the redaction helpers and an examples integration test asserting the body is logged (redacted) at debug and headers only appear when debugHeaders is set. Co-authored-by: Isaac
parthban-db
force-pushed
the
parthban-db/stack/bugbash-bug4-secret-logging
branch
from
97b23bc to
7a3446a
Compare
parthban-db
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🥞 Stacked PR
Use this link to review incremental changes.
Summary
Restores HTTP request/response body logging at debug level in the generated
executeHttpCall/sendAndCheckErrorhelpers, made secret-safe by redaction and truncation. This replaces the earlier approach of dropping the body from debug logs entirely. The behavior now matches the Go SDK: bodies are always logged at debug, but secret-bearing fields are redacted and every value is truncated.Why
executeHttpCallpreviously logged the full decoded response body with no redaction, so endpoints likegetSecret()leaked plaintext secrets into debug logs. The first fix dropped the body altogether, which also removed a useful debugging signal. The Go SDK keeps body logging on at debug and stays safe by redacting secret-bearing JSON keys (token,password,access_token,string_value, ...) to**REDACTED**and truncating every value todebugTruncateBytes. This PR ports that behavior, so bodies return without secrets reaching the logs.What changed
@databricks/sdk-core/logger/debugmodule (onlyNBytes,redactedDumpBody,redactHeaders,DEFAULT_DEBUG_TRUNCATE_BYTES), a hand-written, unit-tested port of the Go SDK'slogger/httplogredaction and truncation logic, exported under the./logger/debugsubpath.ClientOptionsgainsdebugHeaders(defaultfalse) anddebugTruncateBytes(default96), mirroring Go'sDebugHeaders/DebugTruncateBytes. Headers are logged only whendebugHeadersis true, and Authorization-family headers are always redacted toREDACTED. Bodies are logged regardless ofdebugHeaders, matching Go; streaming bodies log a<stream>sentinel and are never drained.utils.tsandclient.tsacross all packages to thread the new options and emit the redacted body/header logs.DATABRICKS_DEBUG_*for a future layer), and Authorization headers are always redacted (noDebugAuthorizationHeaderopt-in), which is the safer default given this was a secret-leak fix.How do you know it works?
Adds
packages/core/tests/logger/debug.test.tscovering each redact key, nested objects, the array trailer, UTF-8 boundary truncation, header redaction, and the non-JSON/empty-body fallbacks. Addspackages/examples/tests/debug-logging.test.ts, an end-to-end test through a generatedPostgresClientasserting the response body is logged with the secret field redacted (and not in plaintext) when a debug logger is supplied, nothing is logged with the defaultNoOpLogger, and headers appear only whendebugHeadersis enabled with Authorization redacted. Validated locally:npm run build(87/87),@databricks/sdk-coretests (370 passing),@databricks/sdk-options(8) and@databricks/sdk-examples(20) tests passing,npm run typecheck(95/95),npm run lint(95/95),npm run format:check(87/87).This pull request and its description were written by Isaac.