Repository navigation
Extend the secret-literal report/repair script to DATABASE base URLs - #1741
Conversation
|
@helpcodeai This is ready for review — part 3 of #1245, the report/repair script extended to DATABASE base URLs, built on #1677. Happy to adjust anything you'd like changed. |
Coding-Professional
left a comment
There was a problem hiding this comment.
Reviewed the DATABASE changes in the report, repair and scan utility at this commit (source review only; I did not run the branch). The apply path keeps connector locking, backup and updates in one transaction, and the printed plan lists changed fields rather than credential values. One non-blocking test-coverage suggestion below for the new auth-config decryption path.
| if (!planRepair(row, ownTools)) continue; | ||
|
|
||
| if (c.type === 'DATABASE') { | ||
| const authConfig = openAuthConfigStrict(c.auth_config, encryptionKey()); |
There was a problem hiding this comment.
Non-blocking coverage suggestion: could you add a DATABASE fixture whose auth_config cannot be decrypted, and assert that both dry run and --apply put it in result.skipped, leave base_url/auth_config byte-for-byte unchanged, and emit no credential values? An encrypted non-object JSON value would be another useful case.
The existing c-badkey integration fixture exercises env_vars; it does not cover openAuthConfigStrict added here. The implementation already skips an unreadable auth config, so this would lock in the credential-preservation behavior rather than fix a reproduced failure.
helpcodeai
left a comment
There was a problem hiding this comment.
Thanks @rangarajan19, this is close. The security side looks good: no secrets, all values are bound parameters, the report runs read-only, and there are no outbound calls. A few small things before we merge:
- Rebase on main. #1740 was closed, but its change landed on main inside #1742 (v0.55.1). So you can rebase now. The branch merges cleanly.
- Docs wording. In
docs/connectors/database.md, the new line says an old connector keeps its password in the connection string "until it is next saved". Since v0.55.1 the backend moves these passwords at every start, so that is no longer true. Please say that the backend moves them at start-up, and that the scripts are for checking and for instances that have not restarted yet. Then regeneratellms-full.txt. - Query string in the report. For a DATABASE connector,
scanDatabaseConnectorprints the whole connection string with only the user password masked. A value like?password=...or?sslpassword=...in the query string would still be printed. Please drop the query string from the printedpattern(or mask its values), and add a test for it. - Optional: a test for an
auth_configthat cannot be decrypted, so the repair skips that connector instead of failing.
When you push, I'll start CI and review again the same day.
Part 3 of HelpCode-ai#1245: a DATABASE connector created before HelpCode-ai#1677 still has its password in plaintext in base_url. report.ts now finds it (database_url_password finding, redacted); repair.ts moves it into encrypted auth_config, dry run by default, with a backup table before --apply. Both reuse moveUrlCredentials/maskUrlPassword from HelpCode-ai#1677 rather than reimplementing the move, so there is no new driver-specific logic to drift from the live API's behavior. Skips a connector whose auth_config already holds a username or password, same rule the API follows (which one wins then differs by driver).
…r the start-up pass - scanDatabaseConnector no longer prints a connection string's query string in the report's pattern: only the userinfo password was masked, so a value like ?password=... or ?sslpassword=... would have printed in full. Dropped entirely rather than selectively masked, with tests. - docs/connectors/database.md: the line about an old connector's password no longer claims it stays until next saved -- the backend now moves it at every start (v0.55.1). Points at the scripts as the way to check an instance before its next restart, or for a dry run and a backup instead of the silent pass. - Test for a DATABASE connector whose auth_config was encrypted with a different key: skipped by both report and repair, never overwritten, exactly like the existing env_vars case.
9b8d97a to
1ab695a
Compare
Masking before dropping the query string, not after: maskUrlPassword finds the password by scanning up to the first "/", not the first "?", so a password containing a raw, unencoded ? or # was being cut in two by stripping the query string first, missing the password entirely. Now masks the full string first, then drops the query string from the already-masked result. Added a regression test with such a password. Also reworded the database.md line once more to match the maintainer's exact phrasing: the scripts are for checking what the start-up pass could not move, and for an instance that has not restarted onto it yet.
helpcodeai
left a comment
There was a problem hiding this comment.
Thanks @rangarajan19, this covers all four points:
- Rebase: the branch is on v0.55.3.
- Docs: they now describe the start-up pass, and
llms-full.txtis regenerated. - Query string: dropped from the printed pattern, and you mask before you cut. Good catch on the raw
?/#in a password. - Tests: the undecryptable
auth_configtest is in.
Security re-check of the two new commits (source review):
- The report prints a DATABASE finding only when a password was actually masked.
passwordSpantakes the last@before the first/, so a password containing@is masked whole. - A password with a raw
/is not found, so nothing is printed for that connector: a missed finding, not a leak. - No new outbound calls, no new dependencies, all SQL still uses bound parameters, and the report stays read-only.
One optional follow-up, not blocking: a ;password=... style parameter after the host (JDBC/MSSQL connection strings) would still print, because the cut only stops at ?/#. Cutting at ; as well would cover that. Happy to take it in a separate small PR if you like.
Approving. It will merge once CI is green.
Part 3 of #1245: extends the secret-literal report/repair script to DATABASE base URLs, for connectors created before #1677.
What's here
scripts/secret-literals/report.ts—DATABASEadded to the guarded type list. Finds a connector whose connection string still carries its password (adatabase_url_passwordfinding, redacted — never the real password).scripts/secret-literals/repair.ts— moves the credentials into encryptedauth_config, dry run by default, with a backup table (now includingauth_config) before--apply. Skips a connector whoseauth_configalready holds a username or password, the same rule the live API follows (which one wins then differs by driver, so moving it could change who connects).src/connectors/secret-literal-scan.util.ts— the DATABASE branch inscanConnector/planRepair. Both call straight intomoveUrlCredentials/maskUrlPasswordfromdatabase-url-credentials.util.ts(fix(connectors): keep database passwords out of the plaintext base URL #1677), so there's no new driver-specific logic here to drift from the live API's behavior.docs/connectors/database.md— one line pointing at the script for rows that predate the fix.Testing
secret-literal-scan.util.spec.ts: pure unit tests — finds a password and masks it in the report; moves credentials for postgres, mssql, oracle and hana; skips whenauthConfigalready holds a username or password; skips an unparseable multi-host MongoDB URL.secret-literal-repair.integration.spec.ts: extended the existing real-Postgres test with two DATABASE fixtures — one that gets repaired, one left alone because itsauthConfigalready holds credentials. Covers report, dry run,--apply(backup table including the pre-moveauth_config, the real encrypt/decrypt round trip), and a second run being a no-op.npx tsc --noEmitandeslintclean on every touched file.npx jest src/connectors/connectors.controller.spec.ts src/connectors/connectors.service.database-credentials.spec.ts src/connectors/database-url-credentials.util.spec.ts: all pass — confirms nothing in fix(connectors): keep database passwords out of the plaintext base URL #1677 broke.llms-full.txtregenerated.Not in this PR
Any change to the live API (
ConnectorsController/ConnectorsService) — #1677 already covers every live read/write path. This is purely the offline cleanup for rows that predate it.