Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/connectors/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ You can type the username and password into the connection string, as above. Whe
- An API request that sends `authConfig` replaces it whole, the moved username and password included.
- No export contains a password that has moved, an admin export included, so a connector restored from a backup needs it typed in again.
- A connection string the parser cannot read, such as a multi-host MongoDB URL typed with its credentials, keeps them in place; the password is shown as `********`.
- `scripts/secret-literals/report.ts` finds a DATABASE connector whose connection string still carries its password (and the same literal-secret cases other connector types have); `repair.ts` moves it, dry run by default, with a backup table before `--apply`. The backend also moves these at every start (below); the scripts are for checking what that pass could not move, and for an instance that has not restarted onto it yet.

Connectors saved before v0.55.0 kept `user:password@` in the stored connection string. The backend moves those too: every start checks the stored database connection strings once and moves the credentials it still finds, the same way a save does. This needs no setting and works the same on AnythingMCP Cloud and self-hosted. The backend log shows the result with connector ids and counts, never a connection string:

Expand Down
1 change: 1 addition & 0 deletions llms-full.txt

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

71 changes: 63 additions & 8 deletions packages/backend/scripts/secret-literals/repair.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,13 @@
* regenerated from the cleaned path;
* - config.baseUrlBaseline is cleaned the same way;
* - a catalog connector's own headers are left alone (they are rebuilt
* from the adapter's template when its variables change).
* from the adapter's template when its variables change);
* - a DATABASE connector's `user:password@` moves into auth_config
* (re-encrypted with ENCRYPTION_KEY, same column the live API now writes
* on every create/update — see database-url-credentials.util.ts), unless
* auth_config already holds a username or password: which one wins then
* differs by driver, so moving it could change who connects, exactly the
* rule the API itself follows.
*
* DRY RUN BY DEFAULT: prints what it would change (redacted) and writes
* nothing. With --apply it first copies every row it is about to change into
Expand All @@ -24,7 +30,8 @@
* in-memory tool registry: restart it (or reload the affected connectors).
*
* Rollback, using the backup table the run prints:
* UPDATE connectors c SET base_url=b.base_url, headers=b.headers, config=b.config, env_vars=b.env_vars
* UPDATE connectors c SET base_url=b.base_url, headers=b.headers, config=b.config,
* env_vars=b.env_vars, auth_config=b.auth_config
* FROM <backup> b WHERE b.kind='connector' AND b.id=c.id;
* UPDATE mcp_tools t SET name=b.name, description=b.description, parameters=b.parameters,
* endpoint_mapping=b.endpoint_mapping
Expand All @@ -38,8 +45,35 @@ import {
type ToolRow,
} from '../../src/connectors/secret-literal-scan.util';
import { openEnvVarsStrict, sealEnvVars } from '../../src/common/crypto/env-vars-at-rest';
import { decrypt, encrypt } from '../../src/common/crypto/encryption.util';

const GUARDED = `('REST','SOAP','GRAPHQL','MCP','ODATA')`;
const GUARDED = `('REST','SOAP','GRAPHQL','MCP','ODATA','DATABASE')`;

/**
* auth_config's own format (connectors.service.ts): a plain ciphertext
* string, no `$enc` wrapper and no AAD — unlike env_vars. `null` means "not
* decrypted yet" (the two-phase trick `planRepair` relies on); a value that
* fails to decrypt is reported as `null` too, so the caller can tell it
* apart from "genuinely holds nothing" and skip the row instead of writing
* an empty authConfig over a real one.
*/
function openAuthConfigStrict(value: unknown, encryptionKey: string): Record<string, unknown> | null {
if (value === null || value === undefined) return {};
if (typeof value !== 'string' || !value) return {};
try {
const parsed = JSON.parse(decrypt(value, encryptionKey));
return parsed && typeof parsed === 'object' && !Array.isArray(parsed) ? parsed : null;
} catch {
return null;
}
}

/** Read lazily, like env-vars-at-rest.ts's own key(): only a run that actually touches a DATABASE connector needs it set. */
function encryptionKey(): string {
const value = process.env.ENCRYPTION_KEY;
if (!value) throw new Error("ENCRYPTION_KEY is not set: a DATABASE connector's credentials cannot be moved");
return value;
}

export interface RepairOptions {
apply?: boolean;
Expand Down Expand Up @@ -87,7 +121,7 @@ export async function repair(client: Client, opts: RepairOptions = {}): Promise<
try {
const connectors = (
await client.query(
`SELECT c.id, c.organization_id, c.name, c.type::text AS type, c.base_url, c.headers, c.config, c.env_vars
`SELECT c.id, c.organization_id, c.name, c.type::text AS type, c.base_url, c.headers, c.config, c.env_vars, c.auth_config
FROM connectors c WHERE ${where} ${opts.apply ? 'FOR UPDATE' : ''}`,
params,
)
Expand Down Expand Up @@ -116,7 +150,8 @@ export async function repair(client: Client, opts: RepairOptions = {}): Promise<
const result: RepairResult = { plans: [], skipped: [] };
for (const c of connectors) {
// Planned without the variables first: most connectors have nothing to
// move, and their env_vars are then never decrypted.
// move, and their env_vars (or a DATABASE connector's auth_config) are
// then never decrypted.
const row: ConnectorRow = {
id: c.id,
organizationId: c.organization_id,
Expand All @@ -126,9 +161,22 @@ export async function repair(client: Client, opts: RepairOptions = {}): Promise<
headers: c.headers,
config: c.config,
envVars: null,
authConfig: null,
};
const ownTools = toolsByConnector.get(c.id) ?? [];
if (!planRepair(row, ownTools)) continue;

if (c.type === 'DATABASE') {
const authConfig = openAuthConfigStrict(c.auth_config, encryptionKey());

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking coverage suggestion: could you add a DATABASE fixture whose auth_config cannot be decrypted, and assert that both dry run and --apply put it in result.skipped, leave base_url/auth_config byte-for-byte unchanged, and emit no credential values? An encrypted non-object JSON value would be another useful case.

The existing c-badkey integration fixture exercises env_vars; it does not cover openAuthConfigStrict added here. The implementation already skips an unreadable auth config, so this would lock in the credential-preservation behavior rather than fix a reproduced failure.

if (authConfig === null) {
result.skipped.push({ connectorId: c.id, reason: 'auth_config could not be decrypted (ENCRYPTION_KEY?)' });
continue;
}
const plan = planRepair({ ...row, authConfig }, ownTools);
if (plan) result.plans.push(plan);
continue;
}

const env = openEnvVarsStrict(c.env_vars);
if (env === null) {
result.skipped.push({ connectorId: c.id, reason: 'env_vars could not be decrypted (ENCRYPTION_KEY?)' });
Expand All @@ -147,14 +195,16 @@ export async function repair(client: Client, opts: RepairOptions = {}): Promise<
await client.query(
`CREATE TABLE ${table} (
kind text NOT NULL, id text NOT NULL, base_url text, headers jsonb, config jsonb,
env_vars jsonb, name text, description text, parameters jsonb, endpoint_mapping jsonb,
env_vars jsonb, auth_config text, name text, description text, parameters jsonb, endpoint_mapping jsonb,
backed_up_at timestamptz NOT NULL DEFAULT now())`,
);
const connectorIds = result.plans.map((p) => p.connectorId);
const toolIds = result.plans.flatMap((p) => p.tools.map((t) => t.id));
await client.query(
`INSERT INTO ${table} (kind, id, base_url, headers, config, env_vars)
SELECT 'connector', id, base_url, headers, config, env_vars FROM connectors WHERE id = ANY($1)`,
// auth_config is copied as the ciphertext string it already is — no
// decrypt/re-encrypt round trip for the backup itself.
`INSERT INTO ${table} (kind, id, base_url, headers, config, env_vars, auth_config)
SELECT 'connector', id, base_url, headers, config, env_vars, auth_config FROM connectors WHERE id = ANY($1)`,
[connectorIds],
);
if (toolIds.length > 0) {
Expand All @@ -178,6 +228,11 @@ export async function repair(client: Client, opts: RepairOptions = {}): Promise<
if (plan.connector.envVars !== undefined) {
set('env_vars', sealEnvVars(plan.connector.envVars), true);
}
if (plan.connector.authConfig !== undefined) {
// auth_config is a plain text column (the ciphertext itself), not
// jsonb, unlike env_vars — see schema.prisma and connectors.service.ts.
set('auth_config', encrypt(JSON.stringify(plan.connector.authConfig), encryptionKey()));
}
if (sets.length > 0) {
values.push(plan.connectorId);
await client.query(`UPDATE connectors SET ${sets.join(', ')}, updated_at = now() WHERE id = $${values.length}`, values);
Expand Down
14 changes: 10 additions & 4 deletions packages/backend/scripts/secret-literals/report.ts
Original file line number Diff line number Diff line change
@@ -1,14 +1,20 @@
/**
* READ-ONLY report: connectors and tools whose base URL, headers, paths,
* query parameters, parameter defaults or names hold a secret-looking
* literal (see src/common/secret-literals.util.ts).
* literal (see src/common/secret-literals.util.ts), plus a DATABASE
* connector whose connection string still carries its password (see
* src/connectors/database-url-credentials.util.ts) — a `database_url_password`
* finding, for a row from before that move was added.
*
* Prints one JSON line per finding, then a summary. A finding names the
* connector, the workspace, the field and a redacted pattern
* (`bot<10 digits>:***(35)`); the secret itself is never printed.
* (`bot<10 digits>:***(35)`, or the DATABASE connection string with its
* password replaced by `********`); the secret itself is never printed.
*
* The queries run inside a READ ONLY transaction, so the script cannot
* change anything even by mistake. env_vars are not read.
* change anything even by mistake. env_vars and a DATABASE connector's
* auth_config are not read — the finding only looks at what is still
* visible in the plaintext column, not at whether repair could act on it.
*
* DATABASE_URL=postgres://… npx ts-node scripts/secret-literals/report.ts
*
Expand All @@ -24,7 +30,7 @@ import {
type ToolRow,
} from '../../src/connectors/secret-literal-scan.util';

const GUARDED = `('REST','SOAP','GRAPHQL','MCP','ODATA')`;
const GUARDED = `('REST','SOAP','GRAPHQL','MCP','ODATA','DATABASE')`;

export async function report(client: Client, out: (line: string) => void = console.log) {
await client.query('BEGIN READ ONLY');
Expand Down
Loading
Loading