Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,9 @@ endpoint. `authConfig` (OAuth and other stored credentials) is never exported:
re-authorize the connector after importing it. Secret environment variables
and headers are included only for admins signed in to the dashboard or using a
`MANAGEMENT_ADMIN` key. For other roles, and for any `MANAGEMENT_READ` key, they
are empty and named in `maskedEnvVars` / `maskedHeaders`. Per-tool `useProxy`, annotation
are empty and named in `maskedEnvVars` / `maskedHeaders`, and a secret still written in an HTTP
base URL (a webhook token, a key, `user:password@`) shows as `********`; see
[Secrets typed into the base URL](connectors/rest.md#secrets-typed-into-the-base-url). Per-tool `useProxy`, annotation
overrides and role access are not exported.

```bash
Expand Down
48 changes: 48 additions & 0 deletions docs/connectors/rest.md
Original file line number Diff line number Diff line change
Expand Up @@ -467,6 +467,54 @@ curl -s -X PUT http://localhost:4000/api/connectors/$CONNECTOR_ID/env-vars \

Use `{{VAR_NAME}}` in endpoint mappings. Parameters matching env var names are automatically stripped from the tool schema.

### Secrets typed into the base URL

Some services put the credential in the address itself: a Bitrix24 inbound webhook
(`https://portal.bitrix24.com/rest/1/<code>/`), a Slack, Discord, Microsoft Teams, Zapier or Make
webhook URL, a Telegram bot URL (`https://api.telegram.org/bot<token>`), a key in the query string
(`?api_key=…`, `?token=…`, `?sig=…`), or `https://user:password@host`. Whoever holds such a URL can
use it like a password.

When you save a REST, GraphQL, SOAP, OData or MCP connector with such a base URL, AnythingMCP moves
only the secret part into an encrypted variable and stores the rest of the URL with a reference:

```
typed: https://portal.bitrix24.com/rest/1/k2bx…vqgo/
stored: https://portal.bitrix24.com/rest/1/{{BITRIX_CRM_WEBHOOK_TOKEN}}/
```

- The variable is named after the connector (`<NAME>_WEBHOOK_TOKEN`, `<NAME>_PASSWORD`,
`<NAME>_API_KEY`, `TELEGRAM_BOT_TOKEN`) or reuses a variable that already holds the same value.
Its value is encrypted at rest and never returned by the connector API, to admins neither (only an
admin export with secrets carries it, like every other variable); the dashboard shows it as set. The save answers with `secretsMoved` and `secretsNotice`, naming the variable,
never the secret.
- Calls are unchanged: every call replaces `{{VAR}}` in the base URL with the variable's value as
stored, so the request goes to exactly the URL you typed. To change the secret, edit the variable,
or type the full URL again.
- The host and the rest of the path stay readable, so the connector page, error hints and logs still
show where the connector points.
- Headers and tool paths get the same treatment (`Authorization: Bearer <token>` becomes
`Bearer {{<NAME>_TOKEN}}`).

Connectors saved before a rule existed keep the secret in the stored base URL until they are
saved again. Until then the API shows it as `********`
(`https://portal.bitrix24.com/rest/1/********/`); saving the form with the URL as shown keeps the
stored secret and moves it into a variable, and a URL edited around `********` is refused, so type
the full address again. The backend also moves these at every start, the same way a save does,
except for long random path segments with no known shape (a save moves those and says so; the
start-up pass leaves them alone). This needs no setting and works the same on AnythingMCP Cloud and
self-hosted. The backend log shows the result with connector ids and counts, never a URL:

```
Base URL secrets: secrets of 2 connectors moved into encrypted variables (cm1…, cm2…); 0 left as is; 0 failed
```

"Left as is" are connectors that were saved while the pass ran; the next start checks them again.
"Failed" names connectors whose variables could not be decrypted (a changed `ENCRYPTION_KEY`) or
written; they stay unchanged, still masked in the API, and the next start tries again.
`scripts/secret-literals/report.ts` lists what is left, and `repair.ts` also cleans tool paths,
headers and tool names (dry run by default, with a backup table before `--apply`).

---

## Example: Full REST API Setup
Expand Down
52 changes: 51 additions & 1 deletion llms-full.txt

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

149 changes: 149 additions & 0 deletions packages/backend/src/common/secret-literals.util.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,155 @@ describe('findSecretLiterals', () => {
});
});

// Webhook credentials of each vendor's shape, assembled at runtime too.
const BITRIX_CODE = ['k2bx', 'a8g5', 'n4gh', 'vqgo'].join('');
const SLACK_HOOK = ['AbCdEfGh', 'IjKlMnOp', 'QrStUv12'].join('');
const DISCORD_TOKEN = ['Xy1_Ab2-Cd3Ef4Gh5', 'Ij6Kl7Mn8Op9Qr0St', 'Uv1Wx2Yz3Ab4Cd5Ef', 'Gh6Ij7Kl8Mn9Op0Q'].join('');
const TEAMS_HEX = ['0123456789ab', 'cdef01234567', '89abcdef'].join('');
const TEAMS_GUID = '1a2b3c4d-5e6f-7a8b-9c0d-1e2f3a4b5c6d';
const ZAPIER_CODE = ['ab', 'c1d', '2e'].join('');
const MAKE_TOKEN = ['h7k2m9p4q1', 'w8x3y6z5a0', 'b2c4d6e8f1'].join('');
const SIG = ['aB3dE5gH7jK9mN1pQ3', 'sT5vW7yZ9bC1eF3hJ5', 'kL7n'].join('');

describe('webhook URLs and URL passwords', () => {
/** The URL, the secret it carries, and the kind of finding. */
const POSITIVE: Array<[label: string, url: string, secret: string, kind: string]> = [
['Bitrix24 cloud portal', `https://portal.bitrix24.com/rest/1/${BITRIX_CODE}/`, BITRIX_CODE, 'webhook_token'],
['Bitrix24 .de portal with a method', `https://acme.bitrix24.de/rest/27/${BITRIX_CODE}/crm.deal.list.json`, BITRIX_CODE, 'webhook_token'],
['Bitrix24 .com.br portal, no trailing slash', `https://acme.bitrix24.com.br/rest/3/${BITRIX_CODE}`, BITRIX_CODE, 'webhook_token'],
['Bitrix24 self-hosted portal', `https://crm.example.com/rest/12/${BITRIX_CODE}/`, BITRIX_CODE, 'webhook_token'],
['Bitrix24 self-hosted, relative path', `/rest/12/${BITRIX_CODE}/crm.lead.add`, BITRIX_CODE, 'webhook_token'],
['Slack incoming webhook', `https://hooks.slack.com/services/T0A1B2C3D/B0E1F2G3H/${SLACK_HOOK}`, SLACK_HOOK, 'webhook_token'],
['Slack workflow trigger', `https://hooks.slack.com/workflows/T0A1B2C3D/A0E1F2G3H/456789012/${SLACK_HOOK}`, SLACK_HOOK, 'webhook_token'],
['Slack triggers URL', `https://hooks.slack.com/triggers/T0A1B2C3D/456789012/${SLACK_HOOK}`, SLACK_HOOK, 'webhook_token'],
['Discord webhook', `https://discord.com/api/webhooks/123456789012345678/${DISCORD_TOKEN}`, DISCORD_TOKEN, 'webhook_token'],
['Discord webhook, versioned, old host', `https://discordapp.com/api/v10/webhooks/123456789012345678/${DISCORD_TOKEN}/slack`, DISCORD_TOKEN, 'webhook_token'],
[
'Teams Office 365 connector',
`https://acme.webhook.office.com/webhookb2/${TEAMS_GUID}@${TEAMS_GUID}/IncomingWebhook/${TEAMS_HEX}/${TEAMS_GUID}`,
`${TEAMS_HEX}/${TEAMS_GUID}`,
'webhook_token',
],
[
'Teams connector, legacy host, V2 suffix',
`https://outlook.office.com/webhook/${TEAMS_GUID}@${TEAMS_GUID}/IncomingWebhook/${TEAMS_HEX}/${TEAMS_GUID}/V2abcDEF123/`,
`${TEAMS_HEX}/${TEAMS_GUID}/V2abcDEF123`,
'webhook_token',
],
[
'Power Automate workflow (sig=)',
`https://prod-12.westeurope.logic.azure.com:443/workflows/0a1b2c3d4e5f/triggers/manual/paths/invoke?api-version=2016-06-01&sp=%2Ftriggers%2Fmanual%2Frun&sv=1.0&sig=${SIG}`,
SIG,
'secret_query_param',
],
[
'Power Platform workflow (sig=)',
`https://default0a1b.2c.environment.api.powerplatform.com:443/powerautomate/automations/direct/workflows/0a1b2c3d/triggers/manual/paths/invoke?api-version=1&sp=%2Ftriggers%2Fmanual%2Frun&sv=1.0&sig=${SIG}`,
SIG,
'secret_query_param',
],
['Zapier catch hook', `https://hooks.zapier.com/hooks/catch/1234567/${ZAPIER_CODE}/`, ZAPIER_CODE, 'webhook_token'],
['Zapier catch hook, silent', `https://hooks.zapier.com/hooks/catch/1234567/${ZAPIER_CODE}/silent/`, ZAPIER_CODE, 'webhook_token'],
['Make webhook', `https://hook.eu1.make.com/${MAKE_TOKEN}`, MAKE_TOKEN, 'webhook_token'],
['Integromat webhook', `https://hook.integromat.com/${MAKE_TOKEN}`, MAKE_TOKEN, 'webhook_token'],
['Make on Celonis', `https://hook.us2.make.celonis.com/${MAKE_TOKEN}`, MAKE_TOKEN, 'webhook_token'],
['REST API with user:password@', 'https://apiuser:Pa55w0rd-1@api.example.com/v1', 'Pa55w0rd-1', 'url_password'],
['user:password@ with a port and an encoded character', 'https://svc:p%40ss42@erp.example.com:8443/odata/', 'p%40ss42', 'url_password'],
['password next to a {{VAR}} user name', 'https://{{API_USER}}:hunter2pass@api.example.com', 'hunter2pass', 'url_password'],
];

it.each(POSITIVE)('finds the secret of a %s', (_label, url, secret, kind) => {
const found = findSecretLiterals(url);
expect(found).toHaveLength(1);
expect(found[0].kind).toBe(kind);
expect(found[0].value).toBe(secret);
expect(url.slice(found[0].start, found[0].end)).toBe(secret);
expect(found[0].pattern).not.toContain(secret);
});

it('also finds the Telegram token and the Google Chat key and token it already knew', () => {
expect(findSecretLiterals(`https://api.telegram.org/bot${TG}`)[0].kind).toBe('telegram_bot_token');
const chatToken = ['3D', 'aBcDeFgHiJkLmNoPqRsT', '_uVwXyZ0123456789%3D'].join('');
const chat = findSecretLiterals(
`https://chat.googleapis.com/v1/spaces/AAAAbCdEfGh/messages?key=${GOOGLE}&token=${chatToken}`,
);
expect(chat.map((f) => f.value)).toEqual([GOOGLE, chatToken]);
});

it('describes a finding without the secret', () => {
const [bitrix] = findSecretLiterals(`https://portal.bitrix24.com/rest/1/${BITRIX_CODE}/`);
expect(bitrix.pattern).toBe('Bitrix24 webhook ***(16)');
const [password] = findSecretLiterals('https://apiuser:Pa55w0rd-1@api.example.com');
expect(password.pattern).toBe('user:***(10)@');
});

it.each([
// plain API base URLs
'https://api.example.com/v1',
'https://api.example.com/rest/v1/users',
'https://jira.example.com/rest/api/2/issue',
'https://acme.atlassian.net/rest/agile/1.0/board/123',
'https://shop.example.com/rest/V1/products',
'https://shop.example.com/rest/default/V1/orders',
'https://erp.example.com/rest/2.0/',
'https://erp.example.com/rest/1/crm.deal.list',
'https://erp.example.com/rest/1/abcdefghijklmnop/',
'https://erp.example.com/rest/1/0123456789abcdef/',
'https://erp.example.com/rest/1/1234567890123456/',
'https://erp.example.com/rest/1/k2bxa8g5n4ghvqgoextra/',
'https://portal.bitrix24.com/rest/',
'https://portal.bitrix24.com/rest/1/',
'https://portal.bitrix24.com/rest/api/crm.deal.list',
'https://portal.bitrix24.com/rest/1/department/',
'https://portal.bitrix24.com/rest/1/12345678/',
'https://notbitrix24.example.com/rest/1/abc/',
'https://slack.com/api/chat.postMessage',
'https://hooks.slack.com/services/',
// Slack's own docs example, assembled so that push protection does not take it for a leak
`https://hooks.slack.com/services/T00000000/B00000000/${'X'.repeat(24)}`,
'https://hooks.slack.com/services/T0A1B2C3D/B0E1F2G3H/short',
'https://discord.com/api/v10/channels/123456789012345678/messages',
'https://discord.com/api/webhooks/123456789012345678',
'https://discord.com/api/webhooks/123456789012345678/short',
'https://graph.microsoft.com/v1.0/teams/abc/channels',
'https://acme.webhook.office.com/webhookb2/abc',
'https://www.example.com/docs/IncomingWebhook/how-to-set-up-one',
'https://outlook.office.com/api/v2.0/me/messages',
'https://hooks.zapier.com/hooks/catch/',
'https://hooks.zapier.com/hooks/catch/1234567/',
'https://zapier.com/hooks/catch/1234567/abc1d2e/',
'https://www.make.com/en/help/tools/webhooks',
'https://eu1.make.com/api/v2/scenarios',
'https://hook.eu1.make.com/short',
'https://hook.eu1.make.com/{{MAKE_WEBHOOK_TOKEN}}',
'https://logic.azure.com/workflows/abc/triggers/manual/paths/invoke?api-version=2016-06-01',
// placeholders, masks and user names alone
'https://portal.bitrix24.com/rest/1/{{BITRIX24_WEBHOOK_CODE}}/',
'{{BITRIX24_WEBHOOK_URL}}',
'https://{{API_USER}}:{{API_PASSWORD}}@api.example.com',
'https://apiuser:{{API_PASSWORD}}@api.example.com',
'https://apiuser:********@api.example.com',
'https://apiuser@api.example.com',
'https://apiuser:@api.example.com',
'https://portal.bitrix24.com/rest/1/********/',
// an @ or a colon that is not userinfo
'https://api.example.com:8443?email=someone@example.com',
'https://api.example.com/users/someone@example.com',
'https://api.example.com/v1?from=a:b@c',
'mailto:someone@example.com',
'someone:secretish@example.com',
])('does not flag %s', (url) => {
expect(findSecretLiterals(url)).toEqual([]);
});

it('redacts a webhook URL quoted in free text', () => {
const text = `POST https://hooks.slack.com/services/T0A1B2C3D/B0E1F2G3H/${SLACK_HOOK} failed`;
expect(redactSecretLiterals(text)).toBe(
'POST https://hooks.slack.com/services/T0A1B2C3D/B0E1F2G3H/[REDACTED] failed',
);
});
});

describe('looksHighEntropy', () => {
it('rejects UUIDs, hex and short tokens', () => {
expect(looksHighEntropy('123e4567e89b12d3a456426614174000')).toBe(false);
Expand Down
Loading
Loading