diff --git a/.github/workflows/dify-plugin.yml b/.github/workflows/dify-plugin.yml new file mode 100644 index 0000000000..d26d582971 --- /dev/null +++ b/.github/workflows/dify-plugin.yml @@ -0,0 +1,40 @@ +name: Dify plugin + +on: + pull_request: + paths: + - integrations/dify/** + - .github/workflows/dify-plugin.yml + push: + branches: [main] + paths: + - integrations/dify/** + - .github/workflows/dify-plugin.yml + +permissions: + contents: read + +jobs: + check: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0 + with: + python-version: "3.12" + - run: python -m pip install -r requirements.txt pytest + working-directory: integrations/dify + - run: python -m pytest tests -q + working-directory: integrations/dify + - name: Package with the official CLI + run: | + curl --fail --location https://github.com/langgenius/dify-plugin-daemon/releases/download/0.6.15/dify-plugin-linux-amd64 --output "$RUNNER_TEMP/dify" + echo "d0a1e3b25bd6c5f6cd749cad2cf04817ea808344e6cc07a2c49bb0108019ab93 $RUNNER_TEMP/dify" | sha256sum --check + chmod +x "$RUNNER_TEMP/dify" + "$RUNNER_TEMP/dify" plugin package integrations/dify -o loom-0.1.0.difypkg + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: loom-dify-plugin + path: loom-0.1.0.difypkg diff --git a/docs/architecture/dify-adapter.md b/docs/architecture/dify-adapter.md new file mode 100644 index 0000000000..34e577e662 --- /dev/null +++ b/docs/architecture/dify-adapter.md @@ -0,0 +1,42 @@ +# Dify tools adapter + +The Dify plugin calls Loom through the Python SDK and stores no content locally. +Its three native tools map to `memory.set`, `memory.search` and `knowledge.search`. +The optional standalone package requires the 0.11 plugin SDK minor series and the +Loom 1.x SDK; the server and core SDK do not depend on Dify. + +## Invariants + +Provider credentials bind the endpoint, organization, namespace, token and agent. +A required User ID form parameter binds the end user outside the model's arguments. +Recall can read that user's memories across Loom agents; documents are shared by the +namespace. Both search operations pin `namespace_scope=this`. Separate namespaces +and token grants constrain mutually untrusted applications. Loom applies its usual +actor, provenance, conflict handling, rolling schema and erasure rules to these calls. + +The administrator controls the HTTP destination. The plugin checks its URL shape +and rejects embedded credentials, queries and fragments; deployment network policy +constrains which hosts the plugin runner can reach. Requests have finite timeouts +and one attempt. Tool errors expose no credentials or remote response detail. +The plugin creates no additional database, cache, model provider or telemetry path. + +Dify patches Python threading with gevent greenlets. Each request constructs, calls +and closes its SDK client inside the current hub's native thread pool, so overlapping +clients do not run asyncio loops on the same native thread. Credentials stay local to +that request. The SDK call waits at most 30 seconds; pool admission, client setup and +close have separate waits rather than a whole-invocation deadline. + +Dify installs the package from a local `.difypkg`. The official CLI packages source, +metadata, runtime requirements, icons, license, README and privacy statement. Its +ignore file excludes tests, environments, credentials and build state. CI invokes +the native tools against a fake SDK HTTP transport, checks overlapping native calls +over loopback HTTP, validates native tool schemas and builds the package. +Live tests run only with an explicit private Loom URL and +token; platform execution is documented separately from those SDK tests. + +## Source entry points + +- `integrations/dify/client.py`: scoped SDK calls and request failures. +- `integrations/dify/provider/loom.py`: credential validation. +- `integrations/dify/tools/`: native tool implementations and schemas. +- `.github/workflows/dify-plugin.yml`: tests and packaging. diff --git a/docs/features/integrations/frameworks.md b/docs/features/integrations/frameworks.md index 18b472a921..9b85df9e95 100644 --- a/docs/features/integrations/frameworks.md +++ b/docs/features/integrations/frameworks.md @@ -30,6 +30,7 @@ or needs another database, vector store or cache. | LangGraph | `LoomStore` | Python, `loom.integrations.langgraph` | `BaseStore` | | Strands Agents | `LoomMemoryStore` | Python, `loom.integrations.strands` | the `MemoryStore` protocol | | Strands Agents | `LoomMemoryStore` | TypeScript, `@clickhouse/loom/integrations/strands` | the same protocol, structurally | +| Dify | Remember, Recall, Search knowledge | `integrations/dify` standalone plugin | Native Dify tools | Any other framework that can call an HTTP endpoint or an MCP server uses Loom directly: see [agent integration](../../guides/agent-integration.md) and @@ -630,6 +631,24 @@ with `?toolset=core` and a bearer token, and expose `loom_memory_set` and `loom_ to its AI Agent. [The MCP client setup](./mcp-clients.md#other-editors-and-agents) describes that route. +## Dify + +The [Loom tools plugin](../../../integrations/dify/README.md) runs Remember, Recall and +Search knowledge as native Dify tools. Install its `.difypkg` from a local package +file and configure a reachable Loom URL, a scoped writer token and the deployment's +organization, namespace and agent. Bind the User ID form field to your app's user; +it is separate from model-generated arguments. + +Remember stores a fact, Recall searches that user's memories across agents, and +Search knowledge searches shared app documents. Both searches stay in the configured +namespace. The plugin calls the Python SDK, adds no store and keeps data in your Loom +deployment. [Privacy](../../../integrations/dify/PRIVACY.md) describes credentials, +network access, provider processing and retention. The package is tested through the +native plugin SDK against Loom; a Dify application and Dify Cloud have not been tested. + +The [MCP/OpenAPI setup](mcp-clients.md#hosted-builders-and-chat-apps) also works without +the package. + ## LangGraph `LoomStore` (`sdk/python/loom/integrations/langgraph.py`) is a LangGraph `BaseStore`, so diff --git a/docs/use-cases/agent-memory.md b/docs/use-cases/agent-memory.md index e9322c4d7b..7ceb66ff70 100644 --- a/docs/use-cases/agent-memory.md +++ b/docs/use-cases/agent-memory.md @@ -81,3 +81,6 @@ in Loom across agent runs and handoffs. For a visual workflow, [n8n nodes](../features/integrations/frameworks.md#n8n) connect an AI Agent to Loom conversation history, durable memories and knowledge search. + +Use the [Dify tools plugin](../features/integrations/frameworks.md#dify) to save facts, +recall a user's memories and search app documents from a workflow. diff --git a/integrations/dify/.difyignore b/integrations/dify/.difyignore new file mode 100644 index 0000000000..0344255abf --- /dev/null +++ b/integrations/dify/.difyignore @@ -0,0 +1,12 @@ +tests/ +__pycache__/ +*.pyc +.pytest_cache/ +.ruff_cache/ +.venv/ +.env* +.git/ +.github/ +.gitignore +*.difypkg +pyproject.toml diff --git a/integrations/dify/.gitignore b/integrations/dify/.gitignore new file mode 100644 index 0000000000..9a95cc6749 --- /dev/null +++ b/integrations/dify/.gitignore @@ -0,0 +1,5 @@ +__pycache__/ +.pytest_cache/ +.venv/ +.env* +*.difypkg diff --git a/integrations/dify/LICENSE b/integrations/dify/LICENSE new file mode 100644 index 0000000000..ebd8172dc3 --- /dev/null +++ b/integrations/dify/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for describing the origin of the Work and + reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or directory name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright 2026 ClickHouse, Inc. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/integrations/dify/PRIVACY.md b/integrations/dify/PRIVACY.md new file mode 100644 index 0000000000..e634fbf1fe --- /dev/null +++ b/integrations/dify/PRIVACY.md @@ -0,0 +1,22 @@ +# Privacy + +The plugin sends the configured organization, namespace, agent, end-user identifier, +memory keys, fact text and search queries to the Loom endpoint configured by the +Dify administrator. The token is sent in the HTTP Authorization header. + +Dify stores provider credentials and may retain workflow inputs, outputs and execution +logs under its own deployment settings. The plugin adds no telemetry, log files or +local data store. It does not send data to the plugin maintainers. + +Loom stores memories and documents in the user's Loom deployment. Its configured +providers may process the text for embeddings or memory extraction; the deployment +operator controls those providers and retention. Remember writes persist after the +workflow ends. Use Loom's deletion and erasure operations to remove stored data, and +configure Dify separately to remove workflow traces. + +This plugin makes authenticated requests to an administrator-configured URL. Treat +that destination and the Dify plugin runner as trusted; use network policy and scoped +tokens to constrain access. + +For privacy questions, use the source repository's security policy: +[ClickHouse/loom security](https://github.com/ClickHouse/loom/security/policy). diff --git a/integrations/dify/README.md b/integrations/dify/README.md new file mode 100644 index 0000000000..47f72ad497 --- /dev/null +++ b/integrations/dify/README.md @@ -0,0 +1,48 @@ +# Loom tools for Dify + +Remember facts, recall a user's memories and search documents in your Loom deployment. +The plugin calls the Loom Python SDK over HTTP. It adds no database, cache or local store. + +## Install and connect + +Use Dify's Plugins page to install the `.difypkg` from a local package file. +The plugin runs on Python 3.12 with `dify-plugin` 0.11 and `clickhouse-loom` 1.2 or later 1.x. + +Configure the Loom URL, a scoped writer token, organization, namespace and agent name +in the provider credentials. Prefer HTTPS and restrict the token to this namespace. +The plugin can reach the URL you configure from the Dify plugin runner; allow only +trusted Loom destinations using your deployment's network policy. Credentials belong +in Dify's credential fields, never model prompts, workflow text or committed files. + +Bind each tool's User ID form field to an application-controlled end-user identifier. +It must be non-empty and cannot be `-` or `*`. The model cannot supply this field. +Recall reads that user's memories across agents in the configured namespace. +Search knowledge reads app documents shared by the namespace, rather than private +user memories. Give mutually untrusted apps separate namespaces and token grants. + +- Remember takes a stable memory key and a text value. Loom resolves duplicates and conflicts. +- Recall takes a query and returns full memory results with their scores and abstention fields. +- Search knowledge takes a query and returns matching document chunks and source URIs. + +Searches stay in the configured namespace. A request failure fails the tool; the plugin +does not replace it with an empty result. Requests use 15-second HTTP timeouts and +one attempt, with a 30-second SDK call wait. Native pool admission and client setup +and close have separate waits; this is not a whole-invocation deadline. Credential validation +reads at most one memory row and discards it. The plugin makes no model calls. + +## Build and check + +```sh +uv venv --python 3.12 .venv +uv pip install --python .venv/bin/python -r requirements.txt pytest +.venv/bin/python -m pytest tests -q +dify plugin package . -o loom-0.1.0.difypkg +``` + +Tests use the actual SDK against a fake HTTP transport, concurrent native tool calls +against a loopback HTTP server, and a local Loom server when configured. +No local Dify application or Dify Cloud runtime has been tested. + +Source: [ClickHouse/loom](https://github.com/ClickHouse/loom/tree/main/integrations/dify). +See [privacy](PRIVACY.md). +Contact: [source repository issues](https://github.com/ClickHouse/loom/issues). diff --git a/integrations/dify/_assets/icon.svg b/integrations/dify/_assets/icon.svg new file mode 100644 index 0000000000..cbeb5995be --- /dev/null +++ b/integrations/dify/_assets/icon.svg @@ -0,0 +1,19 @@ + + + + + + + + + + + + + + + + + + diff --git a/integrations/dify/client.py b/integrations/dify/client.py new file mode 100644 index 0000000000..79b00315af --- /dev/null +++ b/integrations/dify/client.py @@ -0,0 +1,75 @@ +"""Scoped SDK requests run in native threads; credentials stay in provider settings.""" + +from typing import Any +from urllib.parse import urlsplit + +from gevent import get_hub +from loom import SyncAgent + + +def request( + credentials: dict[str, Any], op: str, body: dict[str, Any], **identity: Any +) -> dict[str, Any]: + fields = {} + for name in ("base_url", "api_key", "org", "namespace", "agent"): + value = credentials.get(name) + if not isinstance(value, str) or not value.strip() or value.strip() in ("-", "*"): + raise ValueError(f"Configure {name} in the Loom provider") + fields[name] = value.strip() + url = urlsplit(fields["base_url"]) + if ( + url.scheme not in ("http", "https") + or not url.hostname + or url.username + or url.password + or url.query + or url.fragment + ): + raise ValueError( + "Loom URL must be an HTTP(S) endpoint without embedded credentials or query" + ) + + def call() -> dict[str, Any] | Exception: + try: + with SyncAgent( + **fields, + timeout_s=15, + call_timeout_s=30, + retry={"max_attempts": 1}, + pacing=False, + consumer="dify-plugin", + ) as agent: + return agent.call(op, body, **identity) + except Exception as error: # noqa: BLE001 — Return failures without hub credential logging. + return error + + result = get_hub().threadpool.apply(call) + if isinstance(result, Exception): + raise result + return result + + +def user(parameters: dict[str, Any]) -> str: + value = parameters.get("user_id") + if not isinstance(value, str) or not value.strip() or value.strip() in ("-", "*"): + raise ValueError("Bind User ID to an application-controlled end-user identifier") + return value.strip() + + +def text(parameters: dict[str, Any], name: str) -> str: + value = parameters.get(name) + if not isinstance(value, str) or not value.strip(): + raise ValueError(f"{name} must not be empty") + return value + + +def limit(parameters: dict[str, Any], maximum: int = 500) -> int: + value = parameters.get("top_k", 5) + if ( + isinstance(value, bool) + or not isinstance(value, (int, float)) + or not 1 <= value <= maximum + or int(value) != value + ): + raise ValueError(f"top_k must be an integer from 1 to {maximum}") + return int(value) diff --git a/integrations/dify/main.py b/integrations/dify/main.py new file mode 100644 index 0000000000..35b5299da5 --- /dev/null +++ b/integrations/dify/main.py @@ -0,0 +1,4 @@ +from dify_plugin import DifyPluginEnv, Plugin + +if __name__ == "__main__": + Plugin(DifyPluginEnv(MAX_REQUEST_TIMEOUT=60)).run() diff --git a/integrations/dify/manifest.yaml b/integrations/dify/manifest.yaml new file mode 100644 index 0000000000..ffe805c521 --- /dev/null +++ b/integrations/dify/manifest.yaml @@ -0,0 +1,33 @@ +version: 0.1.0 +type: plugin +author: clickhouse +name: loom +repo: https://github.com/ClickHouse/loom +contact: feedback@clickhouse.com +tags: + - agent + - search +label: + en_US: Loom +description: + en_US: Memory and document search in your Loom deployment. +icon: icon.svg +resource: + memory: 268435456 + permission: {} +plugins: + tools: + - provider/loom.yaml +meta: + version: 0.0.1 + arch: + - amd64 + - arm64 + runner: + language: python + version: "3.12" + entrypoint: main + minimum_dify_version: 1.0.0 +created_at: "2026-11-16T20:57:26Z" +privacy: PRIVACY.md +verified: false diff --git a/integrations/dify/provider/__init__.py b/integrations/dify/provider/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/integrations/dify/provider/loom.py b/integrations/dify/provider/loom.py new file mode 100644 index 0000000000..c57400dd17 --- /dev/null +++ b/integrations/dify/provider/loom.py @@ -0,0 +1,16 @@ +from typing import Any + +from client import request +from dify_plugin import ToolProvider +from dify_plugin.errors.tool import ToolProviderCredentialValidationError +from loom import LoomError + + +class LoomProvider(ToolProvider): + def _validate_credentials(self, credentials: dict[str, Any]) -> None: + try: + request(credentials, "memory.list", {"limit": 1}) + except (LoomError, TimeoutError, ValueError): + raise ToolProviderCredentialValidationError( + "Loom connection failed; check the endpoint, identity and token grants" + ) from None diff --git a/integrations/dify/provider/loom.yaml b/integrations/dify/provider/loom.yaml new file mode 100644 index 0000000000..d0ab4dab4d --- /dev/null +++ b/integrations/dify/provider/loom.yaml @@ -0,0 +1,51 @@ +identity: + author: clickhouse + name: loom + label: + en_US: Loom + description: + en_US: Memory and document search in your Loom deployment. + icon: icon.svg +credentials_for_provider: + base_url: + type: text-input + required: true + label: + en_US: Loom URL + help: + en_US: Configure a restricted Loom writer token and its deployment identity. + api_key: + type: secret-input + required: true + label: + en_US: Loom token + help: + en_US: Configure a restricted Loom writer token and its deployment identity. + org: + type: text-input + required: true + label: + en_US: Organization + help: + en_US: Configure a restricted Loom writer token and its deployment identity. + namespace: + type: text-input + required: true + label: + en_US: Namespace + help: + en_US: Configure a restricted Loom writer token and its deployment identity. + agent: + type: text-input + required: true + label: + en_US: Agent name + help: + en_US: Configure a restricted Loom writer token and its deployment identity. +tools: + - tools/remember.yaml + - tools/recall.yaml + - tools/search_knowledge.yaml +extra: + python: + source: provider/loom.py diff --git a/integrations/dify/pyproject.toml b/integrations/dify/pyproject.toml new file mode 100644 index 0000000000..0a0ff1822b --- /dev/null +++ b/integrations/dify/pyproject.toml @@ -0,0 +1,8 @@ +[project] +name = "loom-dify-plugin" +version = "0.1.0" +requires-python = ">=3.12" +dependencies = ["dify-plugin>=0.11.1,<0.12.0", "clickhouse-loom>=1.2.0,<2"] + +[tool.pytest.ini_options] +testpaths = ["tests"] diff --git a/integrations/dify/requirements.txt b/integrations/dify/requirements.txt new file mode 100644 index 0000000000..4ffb130923 --- /dev/null +++ b/integrations/dify/requirements.txt @@ -0,0 +1,2 @@ +dify-plugin>=0.11.1,<0.12.0 +clickhouse-loom>=1.2.0,<2 diff --git a/integrations/dify/tests/conftest.py b/integrations/dify/tests/conftest.py new file mode 100644 index 0000000000..7cd9ff1859 --- /dev/null +++ b/integrations/dify/tests/conftest.py @@ -0,0 +1,6 @@ +"""Import the standalone plugin as Dify imports it from its package root.""" + +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) diff --git a/integrations/dify/tests/native_concurrency_case.py b/integrations/dify/tests/native_concurrency_case.py new file mode 100644 index 0000000000..488cf02c59 --- /dev/null +++ b/integrations/dify/tests/native_concurrency_case.py @@ -0,0 +1,183 @@ +"""Exercise Dify's import order and overlapping calls in a fresh interpreter.""" + +import dify_plugin # noqa: F401 — patch threading before importing the SDK and executor + +# isort: split + +import json +import sys +from concurrent.futures import ThreadPoolExecutor +from pathlib import Path + +from gevent import get_hub +from gevent.event import Event +from gevent.pool import Pool +from gevent.pywsgi import WSGIServer + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) + +from provider.loom import LoomProvider +from tools.recall import RecallTool +from tools.remember import RememberTool +from tools.search_knowledge import SearchKnowledgeTool + +SECOND = sys.argv[1] +PATHS = { + "recall": "/v1/memory.search", + "remember": "/v1/memory.set", + "knowledge": "/v1/knowledge.search", + "credentials": "/v1/memory.list", +} +assert SECOND in PATHS +calls = [] +first_entered, both_entered, release = Event(), Event(), Event() +hold = False + + +def marker(body, path): + return f"{body['org']}:{body.get('user_id', '-')}:{path}" + + +def application(environ, start_response): + path = environ["PATH_INFO"] + body = json.loads(environ["wsgi.input"].read(int(environ["CONTENT_LENGTH"]))) + calls.append((path, body, environ.get("HTTP_AUTHORIZATION"))) + if hold: + first_entered.set() + if len(calls) == 2: + both_entered.set() + assert release.wait(timeout=15), "Held HTTP response was not released" + refused = body.get("user_id") == "refused" + reply = ( + {"detail": "first-native-test-token"} + if refused + else {"stored": True, "marker": marker(body, path)} + if path == "/v1/memory.set" + else {"entries": [], "truncated": False, "marker": marker(body, path)} + if path == "/v1/memory.list" + else {"results": [], "marker": marker(body, path)} + ) + start_response("403 Forbidden" if refused else "200 OK", [("Content-Type", "application/json")]) + return [json.dumps(reply).encode()] + + +def invoke(kind, credentials, user_id): + if kind == "credentials": + LoomProvider().validate_credentials(credentials) + return None + cls = {"recall": RecallTool, "remember": RememberTool, "knowledge": SearchKnowledgeTool}[kind] + parameters = ( + {"user_id": user_id, "memory_key": "seat", "value": "aisle"} + if kind == "remember" + else {"user_id": user_id, "query": "seat", "top_k": 3} + ) + messages = list(cls.from_credentials(credentials).invoke(parameters)) + assert len(messages) == 1 + return messages[0].message.json_object + + +server = WSGIServer(("127.0.0.1", 0), application, log=None, spawn=Pool(4)) +pool = None +server.start() +first = { + "base_url": f"http://127.0.0.1:{server.server_port}", + "api_key": "first-native-test-token", + "org": "first-org", + "namespace": "first-namespace", + "agent": "first-agent", +} +second = dict( + first, + api_key="second-native-test-token", + org="second-org", + namespace="second-namespace", + agent="second-agent", +) +try: + assert invoke("recall", first, "serial") == { + "results": [], + "marker": "first-org:serial:/v1/memory.search", + } + print("SERIAL_NATIVE_IMPORT_PASS", flush=True) + calls.clear() + hold = True + pool = ThreadPoolExecutor(max_workers=2) + futures = [pool.submit(invoke, "recall", first, "alice")] + entered = first_entered.wait(timeout=5) + overlap = False + try: + if entered: + futures.append(pool.submit(invoke, SECOND, second, "bob")) + overlap = both_entered.wait(timeout=3) + finally: + release.set() + results, errors = [], [] + for future in futures: + try: + results.append(future.result(timeout=25)) + except Exception as exc: # noqa: BLE001 — report every invocation before asserting + errors.append({"type": type(exc).__name__, "message": str(exc)}) + print( + json.dumps( + { + "case": SECOND, + "entered": entered, + "overlap": overlap, + "calls": calls, + "results": results, + "errors": errors, + } + ), + flush=True, + ) + assert entered and overlap and not errors and len(calls) == len(results) == 2 + assert [path for path, _, _ in calls] == ["/v1/memory.search", PATHS[SECOND]] + for (path, body, authorization), credentials, user_id in zip( + calls, [first, second], ["alice", "-" if SECOND == "credentials" else "bob"], strict=True + ): + assert authorization == "Bearer " + credentials["api_key"] + assert body["org"] == credentials["org"] + assert body["namespace"] == credentials["namespace"] + assert body["user_id"] == user_id + assert body["agent"] == ("*" if path == "/v1/memory.search" else credentials["agent"]) + if path.endswith("search"): + assert body["namespace_scope"] == "this" and body["top_k"] == 3 + if path == "/v1/memory.search": + assert body["bodies"] == "full" + elif path == "/v1/memory.list": + assert body["limit"] == 1 + expected_second = ( + None + if SECOND == "credentials" + else {"stored": True, "marker": "second-org:bob:" + PATHS[SECOND]} + if SECOND == "remember" + else {"results": [], "marker": "second-org:bob:" + PATHS[SECOND]} + ) + assert results == [ + {"results": [], "marker": "first-org:alice:/v1/memory.search"}, + expected_second, + ] + hold = False + calls.clear() + try: + invoke("recall", first, "refused") + except ValueError as exc: + assert "Loom request failed" in str(exc) and first["api_key"] not in str(exc) + else: + raise AssertionError("A server refusal must fail the invocation") + assert invoke("recall", first, "after-failure") == { + "results": [], + "marker": "first-org:after-failure:/v1/memory.search", + } + assert [body["user_id"] for _, body, _ in calls] == ["refused", "after-failure"] +finally: + release.set() + if pool is not None: + pool.shutdown(wait=True, cancel_futures=True) + server.stop(timeout=1) + # This isolated child owns its hub; drain SDK work before ending the process. + native_pool = get_hub().threadpool + native_pool.join() + native_pool.kill() + +print("NATIVE_CONCURRENCY_PASS " + SECOND, flush=True) diff --git a/integrations/dify/tests/test_native_concurrency.py b/integrations/dify/tests/test_native_concurrency.py new file mode 100644 index 0000000000..e61d62b324 --- /dev/null +++ b/integrations/dify/tests/test_native_concurrency.py @@ -0,0 +1,33 @@ +"""Native monkey patches are isolated from pytest's import order.""" + +import os +import subprocess +import sys +from pathlib import Path + +import pytest + + +@pytest.mark.parametrize("second", ["recall", "remember", "knowledge", "credentials"]) +def test_native_invocations_overlap_with_scoped_credentials(second): + case = Path(__file__).with_name("native_concurrency_case.py") + result = subprocess.run( + [sys.executable, "-I", str(case), second], + env={"PATH": os.defpath, "HOME": os.environ["HOME"], "LOOM_ENV_FILE": ""}, + capture_output=True, + text=True, + timeout=45, + check=False, + ) + evidence = result.stdout + result.stderr + assert result.returncode == 0, evidence + assert "SERIAL_NATIVE_IMPORT_PASS" in result.stdout + assert "NATIVE_CONCURRENCY_PASS " + second in result.stdout + for diagnostic in ( + "Traceback (most recent call last):", + "Exception in thread", + "RuntimeWarning:", + "was never awaited", + "Task was destroyed", + ): + assert diagnostic not in evidence, evidence diff --git a/integrations/dify/tests/test_tools.py b/integrations/dify/tests/test_tools.py new file mode 100644 index 0000000000..9de804b6b0 --- /dev/null +++ b/integrations/dify/tests/test_tools.py @@ -0,0 +1,233 @@ +"""Native Dify tool invocation through the installed Loom SDK.""" + +import json +import os +from pathlib import Path + +import httpx +import pytest +import yaml +from provider.loom import LoomProvider +from tools.recall import RecallTool +from tools.remember import RememberTool +from tools.search_knowledge import SearchKnowledgeTool + +PLUGIN = Path(__file__).resolve().parents[1] + +CREDENTIALS = { + "base_url": "http://loom.invalid", + "api_key": "test-token", + "org": "o", + "namespace": "n", + "agent": "dify", +} + + +@pytest.fixture +def fake(monkeypatch): + calls = [] + reply = {"status": 200, "body": {"results": [{"content_excerpt": "aisle"}]}} + + async def send(self, request, **kwargs): + calls.append((request.url.path, json.loads(request.content), request.headers)) + return httpx.Response(reply["status"], json=reply["body"], request=request) + + monkeypatch.setattr(httpx.AsyncClient, "send", send) + return calls, reply + + +def output(cls, parameters, credentials=None): + tool = cls.from_credentials(credentials or CREDENTIALS) + messages = list(tool.invoke(parameters)) + assert len(messages) == 1 + return messages[0].message.json_object + + +def test_remember_uses_bound_identity_and_preserves_text(fake): + calls, reply = fake + reply["body"] = {"stored": True} + assert output(RememberTool, {"user_id": "alice", "memory_key": "seat", "value": "aisle"})[ + "stored" + ] + path, body, headers = calls[-1] + assert path == "/v1/memory.set" + assert { + k: body[k] for k in ("org", "namespace", "agent", "user_id", "memory_key", "value") + } == { + "org": "o", + "namespace": "n", + "agent": "dify", + "user_id": "alice", + "memory_key": "seat", + "value": "aisle", + } + assert headers["authorization"] == "Bearer test-token" + + +@pytest.mark.parametrize( + "cls,op", [(RecallTool, "memory.search"), (SearchKnowledgeTool, "knowledge.search")] +) +def test_searches_are_user_bound_and_namespace_local(fake, cls, op): + calls, _ = fake + assert output(cls, {"user_id": "alice", "query": "seat", "top_k": 3})["results"] + assert calls[-1][0] == "/v1/" + op + assert calls[-1][1]["namespace_scope"] == "this" + assert calls[-1][1]["user_id"] == "alice" + assert calls[-1][1]["top_k"] == 3 + if cls is RecallTool: + assert calls[-1][1]["bodies"] == "full" + assert calls[-1][1]["agent"] == "*" + + +@pytest.mark.parametrize("user_id", [None, "", " ", "-", "*", {}]) +def test_missing_users_are_refused_before_network(fake, user_id): + calls, _ = fake + with pytest.raises(ValueError, match="end-user"): + output(RecallTool, {"user_id": user_id, "query": "seat"}) + assert not calls + + +@pytest.mark.parametrize("top_k", [0, 501, 1.5, True, float("nan"), float("inf")]) +def test_invalid_limits_are_refused_without_a_request(fake, top_k): + calls, _ = fake + with pytest.raises(ValueError, match="top_k"): + output(RecallTool, {"user_id": "alice", "query": "seat", "top_k": top_k}) + assert not calls + + +def test_provider_checks_authentication_without_writing(fake): + calls, _ = fake + LoomProvider().validate_credentials(CREDENTIALS) + assert calls[-1][0] == "/v1/memory.list" + assert calls[-1][1]["limit"] == 1 + + +def test_server_refusal_does_not_become_success_or_expose_credentials(fake): + _, reply = fake + reply.update(status=403, body={"detail": "test-token"}) + with pytest.raises(ValueError, match="Loom request failed") as caught: + output(RecallTool, {"user_id": "alice", "query": "seat"}) + assert "test-token" not in str(caught.value) + with pytest.raises(Exception, match="Loom connection failed") as caught: + LoomProvider().validate_credentials(CREDENTIALS) + assert "test-token" not in str(caught.value) + + +def test_invalid_endpoint_is_refused_before_network(fake): + calls, _ = fake + for url in ["file:///tmp/data", "http://name:secret@host", "http://host/?token=secret"]: + with pytest.raises(ValueError, match="endpoint"): + output( + RecallTool, {"user_id": "alice", "query": "seat"}, dict(CREDENTIALS, base_url=url) + ) + assert not calls + + +def test_native_sdk_loads_each_tool_schema(monkeypatch): + from dify_plugin.entities.tool import ToolProviderConfiguration + + monkeypatch.chdir(PLUGIN) + config = ToolProviderConfiguration.model_validate( + yaml.safe_load(Path("provider/loom.yaml").read_text()) + ) + assert len(config.tools) == 3 + for tool in config.tools: + assert next(p for p in tool.parameters if p.name == "user_id").form.value == "form" + + +@pytest.mark.skipif( + not os.environ.get("LOOM_BASE_URL") or not os.environ.get("LOOM_API_KEY"), + reason="needs a private Loom server", +) +def test_native_tools_against_live_loom(): + import time + import uuid + + credentials = dict( + CREDENTIALS, + base_url=os.environ["LOOM_BASE_URL"], + api_key=os.environ["LOOM_API_KEY"], + org=os.environ["LOOM_ORG"], + namespace="dify-" + uuid.uuid4().hex[:8], + ) + stored = output( + RememberTool, + {"user_id": "alice", "memory_key": "seat", "value": "Alice prefers aisle seats"}, + credentials, + ) + assert stored.get("memory_id") or stored.get("stored") + for _ in range(30): + found = output(RecallTool, {"user_id": "alice", "query": "aisle seats"}, credentials) + if found.get("results"): + break + time.sleep(0.1) + assert any("aisle" in row["content_excerpt"].lower() for row in found["results"]) + assert ( + output(RecallTool, {"user_id": "bob", "query": "aisle seats"}, credentials)["results"] == [] + ) + + from client import request + + request( + credentials, + "knowledge.upsert", + {"source_uri": "guide", "content": "Aisle seats are in the travel guide."}, + ) + for _ in range(30): + documents = output( + SearchKnowledgeTool, {"user_id": "bob", "query": "aisle seats"}, credentials + ) + if documents.get("results"): + break + time.sleep(0.1) + assert documents["results"][0]["source_uri"] == "guide" + + +def test_knowledge_limit_matches_the_wire_cap(fake): + calls, _ = fake + with pytest.raises(ValueError, match="100"): + output(SearchKnowledgeTool, {"user_id": "alice", "query": "guide", "top_k": 101}) + assert not calls + + +def test_rate_limit_is_not_retried_or_exposed(fake): + calls, reply = fake + reply.update(status=429, body={"detail": "test-token"}) + with pytest.raises(ValueError, match="Loom request failed") as caught: + output(RecallTool, {"user_id": "alice", "query": "seat"}) + assert len(calls) == 1 and "test-token" not in str(caught.value) + + +@pytest.mark.parametrize( + "cls,parameters", + [ + (RememberTool, {"user_id": "alice", "memory_key": "seat", "value": "aisle"}), + (RecallTool, {"user_id": "alice", "query": "seat"}), + (SearchKnowledgeTool, {"user_id": "alice", "query": "guide"}), + ], +) +def test_sdk_deadline_becomes_a_sanitized_tool_failure(monkeypatch, cls, parameters): + from loom import SyncAgent + + def expired(*args, **kwargs): + raise TimeoutError("test-token") + + monkeypatch.setattr(SyncAgent, "call", expired) + with pytest.raises(ValueError, match="Loom request failed") as caught: + output(cls, parameters) + assert "test-token" not in str(caught.value) + + +def test_sdk_deadline_becomes_a_credential_validation_failure(monkeypatch): + from dify_plugin.errors.tool import ToolProviderCredentialValidationError + from loom import SyncAgent + + def expired(*args, **kwargs): + raise TimeoutError("test-token") + + monkeypatch.setattr(SyncAgent, "call", expired) + with pytest.raises( + ToolProviderCredentialValidationError, match="Loom connection failed" + ) as caught: + LoomProvider().validate_credentials(CREDENTIALS) + assert "test-token" not in str(caught.value) diff --git a/integrations/dify/tools/__init__.py b/integrations/dify/tools/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/integrations/dify/tools/recall.py b/integrations/dify/tools/recall.py new file mode 100644 index 0000000000..07034e6dbb --- /dev/null +++ b/integrations/dify/tools/recall.py @@ -0,0 +1,31 @@ +from collections.abc import Generator +from typing import Any + +from client import limit, request, text, user +from dify_plugin import Tool +from dify_plugin.entities.tool import ToolInvokeMessage +from loom import LoomError + + +class RecallTool(Tool): + def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage, None, None]: + user_id = user(tool_parameters) + try: + result = request( + self.runtime.credentials, + "memory.search", + { + "query": text(tool_parameters, "query"), + "top_k": limit(tool_parameters), + "namespace_scope": "this", + "bodies": "full", + "explain": False, + }, + user_id=user_id, + agent_id="*", + ) + except (LoomError, TimeoutError): + raise ValueError( + "Loom request failed; check the provider connection and token grants" + ) from None + yield self.create_json_message(result) diff --git a/integrations/dify/tools/recall.yaml b/integrations/dify/tools/recall.yaml new file mode 100644 index 0000000000..7f0110b5b6 --- /dev/null +++ b/integrations/dify/tools/recall.yaml @@ -0,0 +1,42 @@ +identity: + name: recall + author: clickhouse + label: + en_US: Recall +description: + human: + en_US: Search the configured end user's memories in this Loom namespace. + llm: Search the configured end user's memories in this Loom namespace. +parameters: + - name: user_id + type: string + required: true + label: + en_US: User ID + human_description: + en_US: Bind to the end-user identifier from your app. Do not let the model choose + it. + form: form + - name: query + type: string + required: true + label: + en_US: Query + human_description: + en_US: The text to search for. + llm_description: The text to search for. + form: llm + - name: top_k + type: number + required: false + default: 5 + min: 1 + max: 500 + label: + en_US: Result limit + human_description: + en_US: Number of results, from 1 to 500. + form: form +extra: + python: + source: tools/recall.py diff --git a/integrations/dify/tools/remember.py b/integrations/dify/tools/remember.py new file mode 100644 index 0000000000..d5c8874495 --- /dev/null +++ b/integrations/dify/tools/remember.py @@ -0,0 +1,27 @@ +from collections.abc import Generator +from typing import Any + +from client import request, text, user +from dify_plugin import Tool +from dify_plugin.entities.tool import ToolInvokeMessage +from loom import LoomError + + +class RememberTool(Tool): + def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage, None, None]: + user_id = user(tool_parameters) + try: + result = request( + self.runtime.credentials, + "memory.set", + { + "memory_key": text(tool_parameters, "memory_key"), + "value": text(tool_parameters, "value"), + }, + user_id=user_id, + ) + except (LoomError, TimeoutError): + raise ValueError( + "Loom request failed; check the provider connection and token grants" + ) from None + yield self.create_json_message(result) diff --git a/integrations/dify/tools/remember.yaml b/integrations/dify/tools/remember.yaml new file mode 100644 index 0000000000..bf8fcd7fb5 --- /dev/null +++ b/integrations/dify/tools/remember.yaml @@ -0,0 +1,40 @@ +identity: + name: remember + author: clickhouse + label: + en_US: Remember +description: + human: + en_US: Save a fact for the configured end user in Loom. + llm: Save a fact for the configured end user in Loom. +parameters: + - name: user_id + type: string + required: true + label: + en_US: User ID + human_description: + en_US: Bind to the end-user identifier from your app. Do not let the model choose + it. + form: form + - name: memory_key + type: string + required: true + label: + en_US: Memory key + human_description: + en_US: Stable key for this fact. + llm_description: Stable key for this fact. + form: llm + - name: value + type: string + required: true + label: + en_US: Value + human_description: + en_US: The fact to remember. + llm_description: The fact to remember. + form: llm +extra: + python: + source: tools/remember.py diff --git a/integrations/dify/tools/search_knowledge.py b/integrations/dify/tools/search_knowledge.py new file mode 100644 index 0000000000..cace8f4847 --- /dev/null +++ b/integrations/dify/tools/search_knowledge.py @@ -0,0 +1,28 @@ +from collections.abc import Generator +from typing import Any + +from client import limit, request, text, user +from dify_plugin import Tool +from dify_plugin.entities.tool import ToolInvokeMessage +from loom import LoomError + + +class SearchKnowledgeTool(Tool): + def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage, None, None]: + user_id = user(tool_parameters) + try: + result = request( + self.runtime.credentials, + "knowledge.search", + { + "query": text(tool_parameters, "query"), + "top_k": limit(tool_parameters, maximum=100), + "namespace_scope": "this", + }, + user_id=user_id, + ) + except (LoomError, TimeoutError): + raise ValueError( + "Loom request failed; check the provider connection and token grants" + ) from None + yield self.create_json_message(result) diff --git a/integrations/dify/tools/search_knowledge.yaml b/integrations/dify/tools/search_knowledge.yaml new file mode 100644 index 0000000000..5666240d52 --- /dev/null +++ b/integrations/dify/tools/search_knowledge.yaml @@ -0,0 +1,42 @@ +identity: + name: search_knowledge + author: clickhouse + label: + en_US: Search knowledge +description: + human: + en_US: Search documents shared by this Loom namespace. + llm: Search documents shared by this Loom namespace. +parameters: + - name: user_id + type: string + required: true + label: + en_US: User ID + human_description: + en_US: Bind to the end-user identifier from your app. Do not let the model choose + it. + form: form + - name: query + type: string + required: true + label: + en_US: Query + human_description: + en_US: The text to search for. + llm_description: The text to search for. + form: llm + - name: top_k + type: number + required: false + default: 5 + min: 1 + max: 100 + label: + en_US: Result limit + human_description: + en_US: Number of results, from 1 to 100. + form: form +extra: + python: + source: tools/search_knowledge.py diff --git a/llms.txt b/llms.txt index 2f6af30c41..bb677e32ae 100644 --- a/llms.txt +++ b/llms.txt @@ -145,6 +145,7 @@ Start with [the docs map](docs/index.md). To install Loom, a coding agent reads - [Claude API memory file tree](docs/architecture/claude-memory-adapter.md): The Claude API memory adapter implements the framework's Python memory-tool bases and TypeScript memory handlers through Loom knowledge documents. - [ClickHouse](docs/architecture/clickhouse.md): Durable Loom state lives in ClickHouse, except document bytes in an optional S3-compatible bucket and, under the Postgres memory authority, the Postgres tables: live memory, knowledge state and the records beside them. - [Code index architecture](docs/architecture/code-index-architecture.md): This page is the inside view of the code index: how files become chunks, symbols and edges, and how the two tiers stay current. +- [Dify tools adapter](docs/architecture/dify-adapter.md): The Dify plugin calls Loom through the Python SDK and stores no content locally. - [Document extraction](docs/architecture/document-extraction.md): An uploaded original is parsed into one shape, the extraction artifact, by an extractor service: a separate process with its own dependencies, never the Loom server. - [Document sources](docs/architecture/document-sources.md): A document source pulls files from a document store (a shared-folder store, a page store, a project tracker, a meeting store) into a namespace's documents and keeps them in step, on any shared namespace or collection. - [Failure and recovery](docs/architecture/failure-and-recovery.md): The primary write and every access decision fail closed. diff --git a/tests/unit/test_public_copy.py b/tests/unit/test_public_copy.py index 12b8512532..e45c406b20 100644 --- a/tests/unit/test_public_copy.py +++ b/tests/unit/test_public_copy.py @@ -144,10 +144,11 @@ def _banned_in(line: str) -> str: "paper citation": re.compile(r"arxiv|\bet al\.", re.IGNORECASE), "decision-record citation": re.compile(r"\b[Dd]ecisions? \d{3,5}\b|docs/decisions/\d"), "internal host": re.compile(r"\bubuntu-[0-9a-z]{2}(?![\w.])|\.ts\.net\b"), - # A personal address at the project's domain; the policy inboxes SECURITY.md and - # CODE_OF_CONDUCT.md name are public. + # Public security, conduct and feedback inboxes identify organizational roles, + # not individuals; personal addresses at the project's domain stay private. "personal address": re.compile( - r"\b(?!(?:security|conduct)@)[A-Za-z0-9._%+-]+@clickhouse\.com\b", re.IGNORECASE + r"\b(?!(?:security|conduct|feedback)@)[A-Za-z0-9._%+-]+@clickhouse\.com\b", + re.IGNORECASE, ), } @@ -316,6 +317,18 @@ def test_server_code_names_no_benchmark_dataset() -> None: ) +def test_personal_address_pattern_distinguishes_public_role_inboxes() -> None: + pattern = PATTERNS["personal address"] + for inbox in ("security", "conduct", "feedback"): + address = f"{inbox}@clickhouse.com" + assert pattern.search(address) is None + assert pattern.search(address.upper()) is None + for inbox in ("person", "person.feedback", "feedback-person", "person+feedback"): + address = f"{inbox}@clickhouse.com" + assert pattern.search(address) is not None + assert pattern.search(address.upper()) is not None + + def test_a_digest_catches_its_name_in_every_shape_and_nothing_else(monkeypatch) -> None: """With a made-up name's digest: prose, identifiers, compounds and hosts carry it; a longer word, an underscore form of a hyphenated name, and a spaced form do not."""