Skip to content

Extend the secret-literal report/repair script to DATABASE base URLs - #1741

Merged
helpcodeai merged 3 commits into
HelpCode-ai:mainfrom
rangarajan19:fix/1245-database-secret-repair
Nov 17, 2026
Merged

helpcodeai merged 3 commits into
HelpCode-ai:mainfrom
rangarajan19:fix/1245-database-secret-repair

Conversation

@rangarajan19

Copy link
Copy Markdown
Contributor

Part 3 of #1245: extends the secret-literal report/repair script to DATABASE base URLs, for connectors created before #1677.

What's here

  • scripts/secret-literals/report.ts — DATABASE added to the guarded type list. Finds a connector whose connection string still carries its password (a database_url_password finding, redacted — never the real password).
  • scripts/secret-literals/repair.ts — moves the credentials into encrypted auth_config, dry run by default, with a backup table (now including auth_config) before --apply. Skips a connector whose auth_config already holds a username or password, the same rule the live API follows (which one wins then differs by driver, so moving it could change who connects).
  • src/connectors/secret-literal-scan.util.ts — the DATABASE branch in scanConnector/planRepair. Both call straight into moveUrlCredentials/maskUrlPassword from database-url-credentials.util.ts (fix(connectors): keep database passwords out of the plaintext base URL #1677), so there's no new driver-specific logic here to drift from the live API's behavior.
  • docs/connectors/database.md — one line pointing at the script for rows that predate the fix.

Testing

  • secret-literal-scan.util.spec.ts: pure unit tests — finds a password and masks it in the report; moves credentials for postgres, mssql, oracle and hana; skips when authConfig already holds a username or password; skips an unparseable multi-host MongoDB URL.
  • secret-literal-repair.integration.spec.ts: extended the existing real-Postgres test with two DATABASE fixtures — one that gets repaired, one left alone because its authConfig already holds credentials. Covers report, dry run, --apply (backup table including the pre-move auth_config, the real encrypt/decrypt round trip), and a second run being a no-op.
  • Ran the integration test for real against a freshly migrated local Postgres (not just the pure unit tests): 3/3 pass.
  • npx tsc --noEmit and eslint clean on every touched file.
  • npx jest src/connectors/connectors.controller.spec.ts src/connectors/connectors.service.database-credentials.spec.ts src/connectors/database-url-credentials.util.spec.ts: all pass — confirms nothing in fix(connectors): keep database passwords out of the plaintext base URL #1677 broke.
  • llms-full.txt regenerated.

Not in this PR

Any change to the live API (ConnectorsController/ConnectorsService) — #1677 already covers every live read/write path. This is purely the offline cleanup for rows that predate it.

@rangarajan19

Copy link
Copy Markdown
Contributor Author

@helpcodeai This is ready for review — part 3 of #1245, the report/repair script extended to DATABASE base URLs, built on #1677. Happy to adjust anything you'd like changed.

@Coding-Professional Coding-Professional left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the DATABASE changes in the report, repair and scan utility at this commit (source review only; I did not run the branch). The apply path keeps connector locking, backup and updates in one transaction, and the printed plan lists changed fields rather than credential values. One non-blocking test-coverage suggestion below for the new auth-config decryption path.

if (!planRepair(row, ownTools)) continue;

if (c.type === 'DATABASE') {
const authConfig = openAuthConfigStrict(c.auth_config, encryptionKey());

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking coverage suggestion: could you add a DATABASE fixture whose auth_config cannot be decrypted, and assert that both dry run and --apply put it in result.skipped, leave base_url/auth_config byte-for-byte unchanged, and emit no credential values? An encrypted non-object JSON value would be another useful case.

The existing c-badkey integration fixture exercises env_vars; it does not cover openAuthConfigStrict added here. The implementation already skips an unreadable auth config, so this would lock in the credential-preservation behavior rather than fix a reproduced failure.

@helpcodeai helpcodeai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @rangarajan19, this is close. The security side looks good: no secrets, all values are bound parameters, the report runs read-only, and there are no outbound calls. A few small things before we merge:

  1. Rebase on main. #1740 was closed, but its change landed on main inside #1742 (v0.55.1). So you can rebase now. The branch merges cleanly.
  2. Docs wording. In docs/connectors/database.md, the new line says an old connector keeps its password in the connection string "until it is next saved". Since v0.55.1 the backend moves these passwords at every start, so that is no longer true. Please say that the backend moves them at start-up, and that the scripts are for checking and for instances that have not restarted yet. Then regenerate llms-full.txt.
  3. Query string in the report. For a DATABASE connector, scanDatabaseConnector prints the whole connection string with only the user password masked. A value like ?password=... or ?sslpassword=... in the query string would still be printed. Please drop the query string from the printed pattern (or mask its values), and add a test for it.
  4. Optional: a test for an auth_config that cannot be decrypted, so the repair skips that connector instead of failing.

When you push, I'll start CI and review again the same day.

Part 3 of HelpCode-ai#1245: a DATABASE connector created before HelpCode-ai#1677 still has
its password in plaintext in base_url. report.ts now finds it
(database_url_password finding, redacted); repair.ts moves it into
encrypted auth_config, dry run by default, with a backup table before
--apply. Both reuse moveUrlCredentials/maskUrlPassword from HelpCode-ai#1677
rather than reimplementing the move, so there is no new driver-specific
logic to drift from the live API's behavior. Skips a connector whose
auth_config already holds a username or password, same rule the API
follows (which one wins then differs by driver).
…r the start-up pass

- scanDatabaseConnector no longer prints a connection string's query
  string in the report's pattern: only the userinfo password was
  masked, so a value like ?password=... or ?sslpassword=... would
  have printed in full. Dropped entirely rather than selectively
  masked, with tests.
- docs/connectors/database.md: the line about an old connector's
  password no longer claims it stays until next saved -- the backend
  now moves it at every start (v0.55.1). Points at the scripts as the
  way to check an instance before its next restart, or for a dry run
  and a backup instead of the silent pass.
- Test for a DATABASE connector whose auth_config was encrypted with a
  different key: skipped by both report and repair, never overwritten,
  exactly like the existing env_vars case.
@rangarajan19
rangarajan19 force-pushed the fix/1245-database-secret-repair branch from 9b8d97a to 1ab695a Compare November 17, 2026 16:16
Masking before dropping the query string, not after: maskUrlPassword
finds the password by scanning up to the first "/", not the first "?",
so a password containing a raw, unencoded ? or # was being cut in two
by stripping the query string first, missing the password entirely.
Now masks the full string first, then drops the query string from the
already-masked result. Added a regression test with such a password.

Also reworded the database.md line once more to match the maintainer's
exact phrasing: the scripts are for checking what the start-up pass
could not move, and for an instance that has not restarted onto it yet.

@helpcodeai helpcodeai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @rangarajan19, this covers all four points:

  • Rebase: the branch is on v0.55.3.
  • Docs: they now describe the start-up pass, and llms-full.txt is regenerated.
  • Query string: dropped from the printed pattern, and you mask before you cut. Good catch on the raw ?/# in a password.
  • Tests: the undecryptable auth_config test is in.

Security re-check of the two new commits (source review):

  • The report prints a DATABASE finding only when a password was actually masked. passwordSpan takes the last @ before the first /, so a password containing @ is masked whole.
  • A password with a raw / is not found, so nothing is printed for that connector: a missed finding, not a leak.
  • No new outbound calls, no new dependencies, all SQL still uses bound parameters, and the report stays read-only.

One optional follow-up, not blocking: a ;password=... style parameter after the host (JDBC/MSSQL connection strings) would still print, because the cut only stops at ?/#. Cutting at ; as well would cover that. Happy to take it in a separate small PR if you like.

Approving. It will merge once CI is green.

@helpcodeai
helpcodeai enabled auto-merge (squash) November 17, 2026 16:41
@helpcodeai
helpcodeai merged commit 49e95ad into HelpCode-ai:main Nov 17, 2026
16 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 17, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants