diff --git a/CHANGELOG.md b/CHANGELOG.md index 93d8147f..95fe4b54 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,11 @@ For each release we list user-facing changes grouped as **Added**, **Changed**, ### Fixed +- **Stored API keys no longer reach the browser in clear text.** A key kept in `config.json`, whether typed in Settings or written by hand, used to come back in the capabilities response and fill a plain text input. The OpenAI-compatible and LiteLLM-compatible `api_key` properties (in the model lists and in the stored `chat_model` and `inline_completion_model` entries), `claude_settings.api_key`, and `acp_settings.api_key` are now sent as `********`, and the Settings inputs are password fields. Saving the panel with the mask untouched keeps the stored key; typing a new value replaces it and clearing the field removes it. Extension providers can opt a property in with `LLMProviderProperty(..., secret=True)`. Keys supplied through environment variables were already withheld. +- **MCP credentials no longer reach the browser or the workspace in clear text.** The Claude MCP panel's server list now masks every `env` and `headers` value, which it never displayed. The MCP config editor writes a temporary copy of `mcp.json` into the workspace; `env` and `headers` values whose names look like credentials (`Authorization`, `*_TOKEN`, `X-Api-Key`, and similar) are now masked in that copy, and a mask saved back keeps the stored value. A mask with no stored value behind it, such as a renamed server, is refused with a 400 instead of being saved. Values in `args` and `url` are not masked. +- **The agent's file tools no longer read or write NBI's own config.** When JupyterLab starts in a directory that contains `~/.jupyter/nbi`, `read_file`, `write_to_file`, `insert_content` and `search_files` could expose API keys or edit `mcp.json`, bypassing the command allowlist the config API applies. They now refuse `config.json`, `mcp.json` and `user-data.json` in the NBI user and environment directories, and the editor's `nbi.mcp.temp.json`, including through a symlink or hard link. Chatbook `@`-mentions refuse them too. The shell tools are unchanged. +- **Shell tool output redacts stored credentials.** Output was scrubbed of environment secrets and well-known token prefixes only, so `cat ~/.jupyter/nbi/config.json` showed a LiteLLM or proxy key. The API keys in `config.json` and the credential-named `mcp.json` values are now redacted too. Placeholder keys that local servers document as "no key" (`not-needed`, `lm-studio`, `ollama`, and similar) are left alone, so those words are not blanked in output. +- **Claude CLI and Claude-mode error text no longer echoes credentials.** `claude mcp add` takes `env` and header values only as command-line flags, and the CLI's error output is forwarded to the browser, so those values are now scrubbed from it along with the credentials stored in `config.json` and `mcp.json`. The same scrub applies to Claude-mode stream errors shown in chat and in the inline editor. Passing a token to `claude mcp add` still exposes it in the process list while the CLI runs; the admin guide's new MCP server credentials section describes the trade-off. - **Chat Markdown images no longer load automatically** (#533). Images in user, model, and tool Markdown now display a visible, translatable blocked-image notice, preserving safe image descriptions. Both local and remote image URLs are blocked to prevent requests that could disclose chat context without a click. Links containing images become non-clickable content so the notice cannot act as a load-image link. Dedicated image results still render; notebook Markdown cells use a separate renderer. See [Chat Markdown images](docs/admin-guide.md#chat-markdown-images). ## [6.3.0] - 2026-10-28 diff --git a/docs/admin-guide.md b/docs/admin-guide.md index 0201a0cf..774349ab 100644 --- a/docs/admin-guide.md +++ b/docs/admin-guide.md @@ -190,7 +190,7 @@ Dedicated image results, such as figures delivered through image response events ## API-key handling -By default, custom-provider API keys (Anthropic, OpenAI-compatible, LiteLLM-compatible) are stored plaintext in `~/.jupyter/nbi/config.json`. This is acceptable for single-tenant developer workstations and unacceptable for multi-tenant clusters. +By default, custom-provider API keys (Anthropic, OpenAI-compatible, LiteLLM-compatible) are stored plaintext in `~/.jupyter/nbi/config.json`. The Settings dialog never receives a stored key: the server sends `********` in its place and keeps the stored key when the mask is saved back unchanged. This is acceptable for single-tenant developer workstations and unacceptable for multi-tenant clusters. Recommended approach for clusters: @@ -200,6 +200,12 @@ Recommended approach for clusters: `${ENV_VAR}`-style interpolation inside `config.json` is not currently supported. Tracked as a feature request. +### MCP server credentials + +Tokens in an MCP server's `env` or `headers` are stored in plaintext: in `mcp.json` for NBI's own MCP Servers tab, and in Claude's own config (`~/.claude.json` or `.mcp.json`) for the Claude-mode MCP Servers tab. NBI never sends those values to the browser in clear text. The Claude-mode server list masks them, and the `mcp.json` editor masks credential-named values and keeps the stored value when the mask is saved back. + +The Claude-mode **Add MCP server** form hands values to `claude mcp add` as command-line flags, because the CLI has no other input for them. For the second or so the CLI runs, another local user on the same machine can read them from the process list. NBI scrubs those values from the CLI's error output. On a shared machine, avoid typing a real token into that form. Instead, add the server without the secret and supply it another way, for example with a `${VAR}` reference in the value. Check that your Claude CLI version expands `${VAR}` in MCP `env` and `headers` before relying on it. + --- ## Self-hosted LLM endpoints diff --git a/notebook_intelligence/_claude_cli.py b/notebook_intelligence/_claude_cli.py index 8ad775e0..68ba945e 100644 --- a/notebook_intelligence/_claude_cli.py +++ b/notebook_intelligence/_claude_cli.py @@ -17,9 +17,13 @@ import asyncio import logging import os -from typing import Optional +from typing import Iterable, Optional -from notebook_intelligence.util import resolve_claude_cli_path +from notebook_intelligence.util import ( + _replace_values, + redact_env_secrets, + resolve_claude_cli_path, +) log = logging.getLogger(__name__) @@ -102,6 +106,7 @@ async def run_claude_cli( timeout: float = CLI_TIMEOUT_DEFAULT_SECONDS, label: str = "claude", env_overrides: Optional[dict[str, str]] = None, + secret_args: Optional[Iterable[str]] = None, ) -> str: """Run ``claude `` and return its stdout. @@ -111,6 +116,11 @@ async def run_claude_cli( Values pass via env, not argv, so they don't appear in the redacted DEBUG log. + ``secret_args`` are values that have to travel in argv (``claude mcp add`` + takes env and header values only as flags). The CLI's error text is + scrubbed of them before it is raised, since the handlers forward it to the + browser. + Raises: FileNotFoundError: when the CLI can't be resolved. TimeoutError: when the call exceeds ``timeout``. @@ -148,10 +158,24 @@ async def run_claude_cli( # The CLI's stderr is forwarded to the browser via the handlers' # `_error()`; scrub any injected secret that the underlying tool # might have echoed (git in verbose mode, curl 401 challenges, etc.). - raise ValueError(_scrub_env_overrides(raw, env_overrides)) + raise ValueError(_scrub_error_text(raw, env_overrides, secret_args)) return out +def _scrub_error_text( + text: str, + env_overrides: Optional[dict[str, str]], + secret_args: Optional[Iterable[str]], +) -> str: + """Scrub CLI error text of every credential this module knows about: + injected env values, secret argv values, then the shared redactor (process + env, stored credentials, known token prefixes).""" + out = _scrub_env_overrides(text, env_overrides) + if secret_args: + out = _replace_values(out, secret_args) + return redact_env_secrets(out) + + def _scrub_env_overrides(text: str, env_overrides: Optional[dict[str, str]]) -> str: """Replace any env_overrides values that appear in `text` with ``. diff --git a/notebook_intelligence/api.py b/notebook_intelligence/api.py index 39d57359..37236d78 100644 --- a/notebook_intelligence/api.py +++ b/notebook_intelligence/api.py @@ -1139,6 +1139,9 @@ class LLMProviderProperty: description: str value: str optional: bool = False + # A credential: the capabilities response masks it and the settings panel + # renders it as a password field. + secret: bool = False def to_dict(self): return asdict(self) diff --git a/notebook_intelligence/built_in_toolsets.py b/notebook_intelligence/built_in_toolsets.py index f135cff5..a0dc1c87 100644 --- a/notebook_intelligence/built_in_toolsets.py +++ b/notebook_intelligence/built_in_toolsets.py @@ -15,6 +15,7 @@ from notebook_intelligence.util import ( get_jupyter_root_dir, + is_protected_nbi_path, redact_env_secrets, safe_jupyter_path, ) @@ -26,6 +27,13 @@ # directly. _get_safe_path = safe_jupyter_path +# Returned to the model, so it stays English. The same text for every +# protected file: it says nothing about whether the file exists. +PROTECTED_NBI_FILE_MESSAGE = ( + "Error: Notebook Intelligence's own configuration files are not available " + "to this tool." +) + def _glob_pattern_escapes(pattern: str) -> bool: """True if a glob pattern would range outside the search root. @@ -588,6 +596,8 @@ async def search_files( # whether or not its target exists, so no outside-path # existence can be inferred. Matches read_file's gate. continue + if is_protected_nbi_path(safe_file): + continue # Safe to stat now: the path is confirmed inside the workspace. if safe_file.is_file(): seen.add(key) @@ -744,6 +754,8 @@ async def read_file( """ try: target_file = _get_safe_path(file_path) + if is_protected_nbi_path(target_file): + return PROTECTED_NBI_FILE_MESSAGE if not target_file.exists(): return f"File '{file_path}' does not exist" @@ -786,6 +798,8 @@ async def insert_content(file_path: str, line_number: int, content: str, **args) """ try: target_file = _get_safe_path(file_path) + if is_protected_nbi_path(target_file): + return PROTECTED_NBI_FILE_MESSAGE if not target_file.exists(): return f"File '{file_path}' does not exist" @@ -825,6 +839,8 @@ async def write_to_file(file_path: str, content: str, **args) -> str: """ try: target_file = _get_safe_path(file_path) + if is_protected_nbi_path(target_file): + return PROTECTED_NBI_FILE_MESSAGE # Create parent directories if they don't exist target_file.parent.mkdir(parents=True, exist_ok=True) diff --git a/notebook_intelligence/chatbook_mentions.py b/notebook_intelligence/chatbook_mentions.py index b0dc75a5..35d6870b 100644 --- a/notebook_intelligence/chatbook_mentions.py +++ b/notebook_intelligence/chatbook_mentions.py @@ -20,6 +20,7 @@ from notebook_intelligence.util import ( get_jupyter_root_dir, has_dangerous_text_codepoints, + is_protected_nbi_path, safe_jupyter_path, ) @@ -80,7 +81,12 @@ def _safe_relative_path( or any(part in skipped for part in parts) ): raise ValueError("unsafe mention path") - return safe_jupyter_path(value) + path = safe_jupyter_path(value) + # A workspace symlink can lead to NBI's own config, which the dot-directory + # check above does not see once the link is resolved. + if is_protected_nbi_path(path): + raise ValueError("unsafe mention path") + return path def list_filesystem_mentions( diff --git a/notebook_intelligence/claude.py b/notebook_intelligence/claude.py index 769d3361..5bdf8dda 100644 --- a/notebook_intelligence/claude.py +++ b/notebook_intelligence/claude.py @@ -31,7 +31,7 @@ import logging from claude_agent_sdk import AssistantMessage, PermissionResultAllow, PermissionResultDeny, ResultMessage, SdkMcpTool, TextBlock, ToolResultBlock, ToolUseBlock, UserMessage, create_sdk_mcp_server, ClaudeAgentOptions, ClaudeSDKClient, tool -from notebook_intelligence.util import ThreadSafeWebSocketConnector, _emit, get_jupyter_root_dir, import_litellm, resolve_claude_cli_path, safe_jupyter_path, terminate_process_tree +from notebook_intelligence.util import ThreadSafeWebSocketConnector, _emit, get_jupyter_root_dir, import_litellm, redact_env_secrets, resolve_claude_cli_path, safe_jupyter_path, terminate_process_tree from notebook_intelligence.inline_completion import ( extract_inline_completion, inline_completion_system_prompt, @@ -1343,14 +1343,17 @@ def completions(self, messages: list[dict], tools: list[dict] = None, response: # detect the failure and skip writing the partial buffer to the # user's cell. Re-raise so the caller logs and runs finish(). if response is not None: + # Scrubbed: the text goes to the browser, and SDK and proxy + # errors can echo part of the credential they rejected. + error_text = redact_env_secrets(str(e)) response.stream({ "choices": [{ "delta": { "role": "assistant", - "content": f"\n\n[Stream interrupted: {e}]" + "content": f"\n\n[Stream interrupted: {error_text}]" } }], - "nbi_stream_error": str(e) + "nbi_stream_error": error_text }) raise @@ -1563,7 +1566,7 @@ def _start_worker_thread(self) -> bool: return True except Exception as e: self._client_thread = None - log.error(f"Error occurred while connecting to Claude agent client: {str(e)}") + log.error(f"Error occurred while connecting to Claude agent client: {redact_env_secrets(str(e))}") self._set_status(ClaudeAgentClientStatus.FailedToConnect) self._connect_resolved.set() return False @@ -1879,12 +1882,12 @@ def _tok(key: str) -> int: )) in_flight_tools.clear() except Exception as e: - err_msg = f"Error communicating with Claude agent: {str(e)}" - log.error(err_msg) + error_text = redact_env_secrets(str(e)) + log.error(f"Error communicating with Claude agent: {error_text}") if not self._reconnect_required: trans = i18n.response_bundle(response) response.stream(MarkdownData( - trans.__("Error communicating with Claude agent: %1", str(e)) + trans.__("Error communicating with Claude agent: %1", error_text) )) finally: _emit(signal, {"id": event_id, "data": "query completed"}) @@ -2910,11 +2913,11 @@ async def handle_chat_request(self, request: ChatRequest, response: ChatResponse # worker thread died. Surface it so the user sees why instead of a # silent spinner stop. if isinstance(result, str) and result: - response.stream(MarkdownData(trans.__("**Claude agent error:** %1", result))) + response.stream(MarkdownData(trans.__("**Claude agent error:** %1", redact_env_secrets(result)))) except Exception as e: - log.error(f"Error while handling Claude chat request: {e}", exc_info=True) + log.error(f"Error while handling Claude chat request: {redact_env_secrets(str(e))}", exc_info=True) try: - response.stream(MarkdownData(trans.__("**Error:** %1", e))) + response.stream(MarkdownData(trans.__("**Error:** %1", redact_env_secrets(str(e))))) except Exception as stream_err: log.debug(f"Could not stream error to client (likely closed websocket): {stream_err}") finally: diff --git a/notebook_intelligence/claude_mcp_manager.py b/notebook_intelligence/claude_mcp_manager.py index 5818eaa2..73131775 100644 --- a/notebook_intelligence/claude_mcp_manager.py +++ b/notebook_intelligence/claude_mcp_manager.py @@ -29,6 +29,11 @@ from pathlib import Path from typing import Iterable, Literal, Optional +from notebook_intelligence.credential_mask import ( + credential_parts, + mask_mcp_server_credentials, +) + from notebook_intelligence._claude_cli import ( CLAUDE_SCOPES, reject_flag_smuggling, @@ -78,6 +83,11 @@ def to_dict(self) -> dict: "disabled_for_workspace": self.disabled_for_workspace, } + def to_wire_dict(self) -> dict: + """``to_dict`` for the browser: ``env`` and ``headers`` values are + masked, since they hold tokens and the panel never shows them.""" + return mask_mcp_server_credentials(self.to_dict()) + def _read_json(path: Path) -> Optional[dict]: try: @@ -277,8 +287,15 @@ async def add_server( tail.append("--") tail.extend(args) + # env and header values can only travel as flags, so the CLI's error + # text, which the handlers forward to the browser, is scrubbed of them. + secret_args = [ + part + for value in [*(env or {}).values(), *(headers or {}).values()] + for part in credential_parts(value) + ] async with self._write_lock: - await self._run_cli(tail) + await self._run_cli(tail, secret_args=secret_args) srv = self.get_server(name, scope) if srv is None: # Surfaced if Claude wrote elsewhere or our reads missed it. The CLI @@ -383,10 +400,13 @@ async def remove_server(self, name: str, scope: ClaudeMCPScope) -> None: # --- internals ----------------------------------------------------- - async def _run_cli(self, tail: list[str]) -> str: + async def _run_cli( + self, tail: list[str], secret_args: Optional[list[str]] = None + ) -> str: return await run_claude_cli( tail, cwd=str(self._working_dir), timeout=CLI_TIMEOUT_SECONDS, label="claude mcp", + secret_args=secret_args, ) diff --git a/notebook_intelligence/config.py b/notebook_intelligence/config.py index 99d9342b..8aca24d4 100644 --- a/notebook_intelligence/config.py +++ b/notebook_intelligence/config.py @@ -8,7 +8,13 @@ import tempfile from typing import Optional -from notebook_intelligence.util import get_claude_config_dir +from notebook_intelligence.util import ( + get_claude_config_dir, + invalidate_protected_paths_cache, + invalidate_stored_secret_cache, + register_protected_paths, +) +from notebook_intelligence.credential_mask import collect_stored_secrets from notebook_intelligence.feature_flags import ( ACP_SETTINGS_OVERRIDES, CHAT_MODEL_OVERRIDES, @@ -50,6 +56,14 @@ def __init__(self, options: Optional[dict] = None): self.user_config_file = os.path.join(self.nbi_user_dir, "config.json") self.env_mcp_file = os.path.join(self.nbi_env_dir, "mcp.json") self.user_mcp_file = os.path.join(self.nbi_user_dir, "mcp.json") + register_protected_paths([ + self.env_config_file, + self.user_config_file, + self.env_mcp_file, + self.user_mcp_file, + self.deprecated_env_config_file, + self.deprecated_user_config_file, + ]) self.env_config = {} self.user_config = {} self.env_mcp = {} @@ -69,6 +83,13 @@ def server_root_dir(self): return self.options.get('server_root_dir', '') def load(self): + try: + self._read_files() + finally: + # Redaction caches the credentials found here; see util.py. + invalidate_stored_secret_cache() + + def _read_files(self): if os.path.exists(self.env_config_file): with open(self.env_config_file, 'r') as file: self.env_config = json.load(file) @@ -109,6 +130,14 @@ def save(self): _atomic_write_json(self.user_config_file, self.user_config) _atomic_write_json(self.user_mcp_file, self.user_mcp) + invalidate_stored_secret_cache() + # The atomic write replaced the files, and with them their inodes. + invalidate_protected_paths_cache() + + def stored_secret_values(self) -> list: + """Credential values kept in config.json and mcp.json, for redacting + shell output (see ``util.set_stored_secret_provider``).""" + return collect_stored_secrets([self.user_config, self.env_config], self.mcp) def get(self, key, default=None): return self.user_config.get(key, self.env_config.get(key, default)) diff --git a/notebook_intelligence/credential_mask.py b/notebook_intelligence/credential_mask.py new file mode 100644 index 00000000..f4cb0ad4 --- /dev/null +++ b/notebook_intelligence/credential_mask.py @@ -0,0 +1,321 @@ +# Copyright (c) Mehmet Bektas + +"""Keep stored API keys out of the capabilities response. + +A key kept in config.json, whether typed in the settings panel or written by +hand, used to travel to the browser in clear text and fill a plain input. The +capabilities handler now sends ``SECRET_MASK`` in its place, and the config +handler maps the mask posted back onto the stored key. The panel can then +re-save its form untouched without overwriting the key it never saw. +""" + +from __future__ import annotations + +import copy +from typing import Any, Iterable, Optional + +from notebook_intelligence.util import _SECRET_ENV_NAME_PATTERNS + +SECRET_MASK = "********" + +# Property ids treated as credentials even when a provider did not set +# ``secret``, so an extension that names its key ``api_key`` cannot leak it by +# omission. +SECRET_PROPERTY_IDS = frozenset({"api_key"}) + + +def _is_secret_property(prop: dict) -> bool: + return bool(prop.get("secret")) or prop.get("id") in SECRET_PROPERTY_IDS + + +def mask_secret(value: Any) -> Any: + """The mask for a non-empty string; anything else unchanged. + + An empty value stays empty so the panel can tell "no key" from "a key". + """ + if isinstance(value, str) and value: + return SECRET_MASK + return value + + +def mask_settings_api_key(settings: Optional[dict]) -> Optional[dict]: + """A copy of a Claude or ACP settings dict with its ``api_key`` masked.""" + if not isinstance(settings, dict) or not settings.get("api_key"): + return settings + result = dict(settings) + result["api_key"] = mask_secret(result["api_key"]) + return result + + +def restore_masked_api_key(posted: Any, stored: Any) -> Any: + """Swap a posted mask for the stored ``api_key``. + + ``stored`` without a key yields an empty one: a mask with nothing behind it + must not be saved as though it were the key. + """ + if not isinstance(posted, dict) or posted.get("api_key") != SECRET_MASK: + return posted + result = dict(posted) + stored_key = stored.get("api_key", "") if isinstance(stored, dict) else "" + result["api_key"] = stored_key if isinstance(stored_key, str) else "" + return result + + +def mask_model_properties(models: list[dict]) -> list[dict]: + """Copies of the model dicts with each secret property's value masked. + + The ``secret`` flag is set on what is sent, so the panel renders a + password field for a property a provider left unflagged. + """ + masked = [] + for model in models: + properties = model.get("properties") + if not isinstance(properties, list): + masked.append(model) + continue + masked.append( + { + **model, + "properties": [ + {**prop, "value": mask_secret(prop.get("value")), "secret": True} + if _is_secret_property(prop) + else prop + for prop in properties + ], + } + ) + return masked + + +def mask_model_config(model_config: Any) -> Any: + """A copy of a stored ``chat_model`` / ``inline_completion_model`` entry + (``{provider, model, properties: [{id, value}]}``) with secret property + values masked. The capabilities response carries this entry as well as the + model list, and both must mask.""" + properties = model_config.get("properties") if isinstance(model_config, dict) else None + if not isinstance(properties, list): + return model_config + return { + **model_config, + "properties": [ + {**prop, "value": mask_secret(prop.get("value"))} + if isinstance(prop, dict) and _is_secret_property(prop) + else prop + for prop in properties + ], + } + + +def restore_masked_model_properties(posted: Any, stored: Any) -> Any: + """Swap posted property masks for the values stored for the same model. + + Only the model the config already names can legitimately hold a key, since + the server loads property values for the selected model alone. A mask for + any other model has no key behind it and becomes empty. + """ + if not isinstance(posted, dict) or not isinstance(posted.get("properties"), list): + return posted + same_model = ( + isinstance(stored, dict) + and stored.get("provider") == posted.get("provider") + and stored.get("model") == posted.get("model") + ) + stored_values = {} + if same_model and isinstance(stored.get("properties"), list): + stored_values = { + prop.get("id"): prop.get("value", "") + for prop in stored["properties"] + if isinstance(prop, dict) + } + properties = [] + for prop in posted["properties"]: + if isinstance(prop, dict) and prop.get("value") == SECRET_MASK: + prop = {**prop, "value": stored_values.get(prop.get("id"), "")} + properties.append(prop) + return {**posted, "properties": properties} + + +# MCP server entries carry credentials in two maps: stdio ``env`` and remote +# ``headers``. Names are matched case-insensitively with ``-`` read as ``_``, +# using the shell-output redactor's list plus the bare words an MCP config +# tends to use (``X-Api-Key``, ``MY_KEY``). +_MCP_SECRET_NAME_PATTERNS = (*_SECRET_ENV_NAME_PATTERNS, "KEY", "PASS") +_MCP_CREDENTIAL_FIELDS = ("env", "headers") + + +def _is_secret_name(name: Any) -> bool: + upper = str(name).upper().replace("-", "_") + return any(pattern in upper for pattern in _MCP_SECRET_NAME_PATTERNS) + + +def _mcp_servers(config: Any) -> dict: + servers = config.get("mcpServers") if isinstance(config, dict) else None + return servers if isinstance(servers, dict) else {} + + +def mask_mcp_server_credentials(server: dict) -> dict: + """A copy of a Claude MCP server dict with every ``env`` and ``headers`` + value masked. The panel lists names only, so nothing there needs a value.""" + result = dict(server) + for field in _MCP_CREDENTIAL_FIELDS: + values = result.get(field) + if isinstance(values, dict): + result[field] = {key: mask_secret(value) for key, value in values.items()} + return result + + +def mask_mcp_config(config: Any) -> Any: + """A copy of an ``mcp.json`` document with credential-named ``env`` and + ``headers`` values masked, for the editor that shows the file. + + Only names that look like credentials are masked, so settings such as + ``PYTHONPATH`` stay editable. ``args`` and ``url`` are left as they are: + a secret inside one cannot be told from the rest of the string. + """ + result = copy.deepcopy(config) + for server in _mcp_servers(result).values(): + if not isinstance(server, dict): + continue + for field in _MCP_CREDENTIAL_FIELDS: + values = server.get(field) + if isinstance(values, dict): + for key, value in values.items(): + if _is_secret_name(key): + values[key] = mask_secret(value) + return result + + +def has_masked_mcp_values(config: Any) -> bool: + """Whether a posted ``mcp.json`` carries any mask to restore.""" + return any( + isinstance(server, dict) + and any( + isinstance(server.get(field), dict) + and SECRET_MASK in server[field].values() + for field in _MCP_CREDENTIAL_FIELDS + ) + for server in _mcp_servers(config).values() + ) + + +class UnresolvedMaskError(ValueError): + """A posted mask has no stored value behind it. ``names`` lists each as + ``server.field.key`` so the handler can word the message in the user's + language.""" + + def __init__(self, names: list): + super().__init__( + "Masked MCP values with no saved value: " + ", ".join(names) + ) + self.names = list(names) + + +def restore_masked_mcp_config(posted: Any, stored: Any) -> Any: + """Swap masks in a posted ``mcp.json`` for the stored values. + + Raises ``UnresolvedMaskError`` naming each mask with no stored value + behind it (a renamed server, or a key added as the mask), so the save is + refused instead of persisting ``********`` as a credential. + """ + result = copy.deepcopy(posted) + stored_servers = _mcp_servers(stored) + unresolved = [] + for name, server in _mcp_servers(result).items(): + if not isinstance(server, dict): + continue + for field in _MCP_CREDENTIAL_FIELDS: + values = server.get(field) + if not isinstance(values, dict): + continue + stored_values = (stored_servers.get(name) or {}).get(field) + for key, value in values.items(): + if value != SECRET_MASK: + continue + if isinstance(stored_values, dict) and isinstance( + stored_values.get(key), str + ): + values[key] = stored_values[key] + else: + unresolved.append(f"{name}.{field}.{key}") + if unresolved: + raise UnresolvedMaskError(unresolved) + return result + + +def credential_parts(value: str) -> list: + """The value, plus the token of a ``Bearer abc`` style header value.""" + parts = [value] + scheme, _, token = value.partition(" ") + if token and scheme.isalpha(): + parts.append(token.strip()) + return parts + + +# What the OpenAI-compatible clients document as the "key" for a server that +# checks none (Ollama, LM Studio, vLLM, llama.cpp). They are not secrets, and +# redacting them would blank ordinary words wherever shell output mentions +# them. Compared lower-cased, with ``_`` read as ``-``. +PLACEHOLDER_API_KEYS = frozenset( + { + "dummy", + "dummy-api-key", + "dummy-key", + "empty", + "fake-key", + "lm-studio", + "lmstudio", + "no-key", + "no-key-required", + "no-api-key", + "none", + "not-needed", + "not-required", + "null", + "ollama", + "placeholder", + "sk-no-key-required", + } +) + + +def is_placeholder_api_key(value: str) -> bool: + return value.strip().lower().replace("_", "-") in PLACEHOLDER_API_KEYS + + +def collect_stored_secrets(configs: Iterable[Any], mcp: Any) -> list: + """Every credential value NBI keeps in its config, for output redaction. + + ``configs`` are config.json layers; ``mcp`` is the merged mcp.json. + """ + found = [] + + def add(value: Any) -> None: + if isinstance(value, str) and value and value != SECRET_MASK: + parts = credential_parts(value) + # `Bearer not-needed` is as much a placeholder as `not-needed`. + if not any(is_placeholder_api_key(part) for part in parts): + found.extend(parts) + + for config in configs: + if not isinstance(config, dict): + continue + for key in ("claude_settings", "acp_settings"): + settings = config.get(key) + if isinstance(settings, dict): + add(settings.get("api_key")) + for key in ("chat_model", "inline_completion_model"): + model = config.get(key) + properties = model.get("properties") if isinstance(model, dict) else None + for prop in properties if isinstance(properties, list) else []: + if isinstance(prop, dict) and prop.get("id") in SECRET_PROPERTY_IDS: + add(prop.get("value")) + for server in _mcp_servers(mcp).values(): + if not isinstance(server, dict): + continue + for field in _MCP_CREDENTIAL_FIELDS: + values = server.get(field) + if isinstance(values, dict): + for key, value in values.items(): + if _is_secret_name(key): + add(value) + return found diff --git a/notebook_intelligence/extension.py b/notebook_intelligence/extension.py index 93cba2dd..2e0e5d1f 100644 --- a/notebook_intelligence/extension.py +++ b/notebook_intelligence/extension.py @@ -98,7 +98,18 @@ from notebook_intelligence.built_in_toolsets import built_in_toolsets, toolset_description from notebook_intelligence.base_chat_participant import BaseChatParticipant from notebook_intelligence.llm_providers.property_labels import translate_model_properties -from notebook_intelligence.util import ThreadSafeWebSocketConnector, get_claude_config_dir, get_jupyter_root_dir, set_jupyter_root_dir, is_builtin_tool_enabled_in_env, is_provider_enabled_in_env, filter_models_by_enabled_providers, VALID_CODING_AGENT_LAUNCHERS, compute_effective_disabled_launchers, validate_coding_agent_launcher_ids, resolve_claude_cli_path, resolve_opencode_cli_path, resolve_pi_cli_path, resolve_copilot_cli_path, resolve_codex_cli_path, safe_anchor_uri, has_dangerous_text_codepoints, split_csv +from notebook_intelligence.credential_mask import ( + UnresolvedMaskError, + has_masked_mcp_values, + mask_mcp_config, + mask_model_config, + mask_model_properties, + mask_settings_api_key, + restore_masked_mcp_config, + restore_masked_api_key, + restore_masked_model_properties, +) +from notebook_intelligence.util import ThreadSafeWebSocketConnector, get_claude_config_dir, get_jupyter_root_dir, set_jupyter_root_dir, is_builtin_tool_enabled_in_env, is_provider_enabled_in_env, filter_models_by_enabled_providers, VALID_CODING_AGENT_LAUNCHERS, compute_effective_disabled_launchers, validate_coding_agent_launcher_ids, resolve_claude_cli_path, resolve_opencode_cli_path, resolve_pi_cli_path, resolve_copilot_cli_path, resolve_codex_cli_path, safe_anchor_uri, has_dangerous_text_codepoints, split_csv, set_stored_secret_provider from notebook_intelligence.context_factory import RuleContextFactory from notebook_intelligence.skillset import SKILL_NAME_REGEX from notebook_intelligence.chatbook_generate import ( @@ -741,14 +752,16 @@ def _build_setting_locks_response(string_overrides: dict) -> dict: def _scrub_credentials_for_wire( settings: dict, string_overrides: dict, api_key_lock: str = "claude_api_key" ) -> dict: - """Strip the api_key from the capabilities response when locked by env. + """Keep the api_key out of the capabilities response. The SDKs read their key env var directly (ANTHROPIC_API_KEY for Claude, OPENAI_API_KEY for the Codex ACP agent); surfacing the value through the frontend would - leak the credential. + leak the credential, so a key locked by env is blanked. A key stored in + config.json is masked instead: the panel shows that one is set, and + ConfigHandler maps the mask posted back onto the stored key. """ if not string_overrides.get(api_key_lock): - return settings + return mask_settings_api_key(settings) result = dict(settings or {}) result["api_key"] = "" return result @@ -871,26 +884,31 @@ def is_provider_enabled(provider_id: str) -> bool: # Filtered by the same predicate as `llm_providers` above: the # model lists walk every registered provider, so without this a # disabled provider kept shipping its models (#431). - "chat_models": translate_model_properties( - trans, - filter_models_by_enabled_providers( - ai_service_manager.chat_model_ids, is_provider_enabled - ), + "chat_models": mask_model_properties( + translate_model_properties( + trans, + filter_models_by_enabled_providers( + ai_service_manager.chat_model_ids, is_provider_enabled + ), + ) ), - "inline_completion_models": translate_model_properties( - trans, - filter_models_by_enabled_providers( - ai_service_manager.inline_completion_model_ids, is_provider_enabled - ), + "inline_completion_models": mask_model_properties( + translate_model_properties( + trans, + filter_models_by_enabled_providers( + ai_service_manager.inline_completion_model_ids, + is_provider_enabled, + ), + ) ), "embedding_models": filter_models_by_enabled_providers( ai_service_manager.embedding_model_ids, is_provider_enabled ), - "chat_model": nbi_config.chat_model, + "chat_model": mask_model_config(nbi_config.chat_model), "chat_model_supports_vision": _resolve_supports_vision( ai_service_manager ), - "inline_completion_model": nbi_config.inline_completion_model, + "inline_completion_model": mask_model_config(nbi_config.inline_completion_model), "embedding_model": nbi_config.embedding_model, "chat_participants": [], "store_github_access_token": nbi_config.store_github_access_token, @@ -1251,6 +1269,9 @@ def post(self): # Re-apply the env override after the user's POST so locked fields # stay pinned; non-locked fields keep the user's value. if key == "chat_model": + value = restore_masked_model_properties( + value, ai_service_manager.nbi_config.get("chat_model") + ) value = apply_string_overrides( value, self.string_overrides, CHAT_MODEL_OVERRIDES ) @@ -1258,6 +1279,9 @@ def post(self): continue has_model_change = True elif key == "inline_completion_model": + value = restore_masked_model_properties( + value, ai_service_manager.nbi_config.get("inline_completion_model") + ) value = apply_string_overrides( value, self.string_overrides, INLINE_COMPLETION_MODEL_OVERRIDES ) @@ -1276,6 +1300,11 @@ def post(self): stored = ai_service_manager.nbi_config.get("claude_settings") if isinstance(stored, dict): value = {**stored, **value} + # The capabilities response masks a stored key; a mask posted + # back means "unchanged", not a new key. + value = restore_masked_api_key( + value, ai_service_manager.nbi_config.get("claude_settings") + ) value = apply_claude_policies(value, self.feature_policies) value = apply_string_overrides( value, self.string_overrides, CLAUDE_SETTINGS_OVERRIDES @@ -1288,6 +1317,9 @@ def post(self): value = dict(value) value["api_key"] = "" elif key == "acp_settings": + value = restore_masked_api_key( + value, ai_service_manager.nbi_config.get("acp_settings") + ) value = apply_acp_policies(value, self.feature_policies) value = apply_string_overrides( value, self.string_overrides, ACP_SETTINGS_OVERRIDES @@ -1441,7 +1473,10 @@ def get(self): mcp_config = ai_service_manager.nbi_config.mcp.copy() if "mcpServers" not in mcp_config: mcp_config["mcpServers"] = {} - self.finish(json.dumps(mcp_config)) + # The editor writes this to a file in the workspace, where the agent's + # file tools and a sync client can reach it, so credential values are + # masked here and restored when the edited file is posted back. + self.finish(json.dumps(mask_mcp_config(mcp_config))) @tornado.web.authenticated def post(self): @@ -1470,6 +1505,9 @@ def post(self): # on every restart. Apply the same env-key denylist that # blocks PATH / LD_PRELOAD / etc. bypasses. allowlist = ai_service_manager.get_mcp_stdio_command_allowlist() + # Same source as the GET above, so a mask maps to what was shown. + if has_masked_mcp_values(data): + data = restore_masked_mcp_config(data, ai_service_manager.nbi_config.mcp) servers = data.get("mcpServers") if isinstance(data, dict) else None if isinstance(servers, dict): for name, server in servers.items(): @@ -1482,6 +1520,18 @@ def post(self): ai_service_manager.nbi_config.load() ai_service_manager.update_mcp_servers() self.finish(json.dumps({"status": "ok"})) + except UnresolvedMaskError as exc: + # A masked value with nothing saved behind it: refused, not saved. + trans = i18n.request_bundle(self) + self.set_status(400) + self.finish(json.dumps({ + "status": "error", + "message": trans.__( + "These MCP values are masked and have no saved value to keep: %1. Enter the real value.", + ", ".join(exc.names), + ), + })) + return except ValueError as exc: # Policy rejection: surface as HTTP 400 so the Settings UI # shows the operator's policy message instead of a generic @@ -1766,7 +1816,7 @@ class ClaudeMCPListHandler(ClaudeMCPBaseHandler): @tornado.web.authenticated def get(self): try: - servers = [s.to_dict() for s in self.manager.list_servers()] + servers = [s.to_wire_dict() for s in self.manager.list_servers()] except (FileNotFoundError, TimeoutError, ValueError) as e: self._error(e) return @@ -1787,7 +1837,7 @@ async def post(self): env=data.get("env"), headers=data.get("headers"), ) - self.finish(json.dumps({"server": srv.to_dict()})) + self.finish(json.dumps({"server": srv.to_wire_dict()})) except (FileNotFoundError, TimeoutError, ValueError) as e: self._error(e) @@ -1803,7 +1853,7 @@ def get(self, scope, name): "error": trans.__("MCP server '%1' not found in %2 scope", name, scope) })) return - self.finish(json.dumps({"server": srv.to_dict()})) + self.finish(json.dumps({"server": srv.to_wire_dict()})) @tornado.web.authenticated async def delete(self, scope, name): @@ -1842,7 +1892,7 @@ async def patch(self, scope, name): return try: srv = await self.manager.set_server_disabled(name=name, disabled=raw) - self.finish(json.dumps({"server": srv.to_dict()})) + self.finish(json.dumps({"server": srv.to_wire_dict()})) except (FileNotFoundError, TimeoutError, ValueError) as e: self._error(e) @@ -4787,6 +4837,9 @@ def initialize_ai_service( "string_overrides": string_overrides, "mcp_stdio_command_allowlist": mcp_command_allowlist, }) + # Shell-tool output is scrubbed of the credentials kept in config.json + # and mcp.json, which rarely carry a recognizable prefix. + set_stored_secret_provider(ai_service_manager.nbi_config.stored_secret_values) # Apply env-var perf policy/locks immediately at boot, mirroring the # comment above about acp_mode: don't wait for the first config POST # to see a force-on/force-off admin policy take effect. diff --git a/notebook_intelligence/llm_providers/litellm_compatible_llm_provider.py b/notebook_intelligence/llm_providers/litellm_compatible_llm_provider.py index 851d1414..c5545256 100644 --- a/notebook_intelligence/llm_providers/litellm_compatible_llm_provider.py +++ b/notebook_intelligence/llm_providers/litellm_compatible_llm_provider.py @@ -20,7 +20,7 @@ def __init__(self, provider: "LiteLLMCompatibleLLMProvider"): self._properties = [ LLMProviderProperty("model_id", "Model", "Model (must support streaming)", "", False), LLMProviderProperty("base_url", "Base URL", "Base URL", "", False), - LLMProviderProperty("api_key", "API key", "API key", "", True), + LLMProviderProperty("api_key", "API key", "API key", "", True, secret=True), LLMProviderProperty("context_window", "Context window", "Context window length", "", True), ] @@ -120,7 +120,7 @@ def __init__(self, provider: "LiteLLMCompatibleLLMProvider"): self._properties = [ LLMProviderProperty("model_id", "Model", "Model", "", False), LLMProviderProperty("base_url", "Base URL", "Base URL", "", False), - LLMProviderProperty("api_key", "API key", "API key", "", True), + LLMProviderProperty("api_key", "API key", "API key", "", True, secret=True), LLMProviderProperty("context_window", "Context window", "Context window length", "", True), ] diff --git a/notebook_intelligence/llm_providers/openai_compatible_llm_provider.py b/notebook_intelligence/llm_providers/openai_compatible_llm_provider.py index ba2e8f24..a9a517f4 100644 --- a/notebook_intelligence/llm_providers/openai_compatible_llm_provider.py +++ b/notebook_intelligence/llm_providers/openai_compatible_llm_provider.py @@ -33,7 +33,7 @@ def __init__(self, provider: "OpenAICompatibleLLMProvider"): super().__init__(provider) self._provider = provider self._properties = [ - LLMProviderProperty("api_key", "API key", "API key", "", False), + LLMProviderProperty("api_key", "API key", "API key", "", False, secret=True), LLMProviderProperty("model_id", "Model", "Model (must support streaming)", "", False), LLMProviderProperty("base_url", "Base URL", "Base URL", "", True), LLMProviderProperty("context_window", "Context window", "Context window length", "", True), @@ -134,7 +134,7 @@ def __init__(self, provider: "OpenAICompatibleLLMProvider"): super().__init__(provider) self._provider = provider self._properties = [ - LLMProviderProperty("api_key", "API key", "API key", "", False), + LLMProviderProperty("api_key", "API key", "API key", "", False, secret=True), LLMProviderProperty("model_id", "Model", "Model", "", False), LLMProviderProperty("base_url", "Base URL", "Base URL", "", True), LLMProviderProperty("context_window", "Context window", "Context window length", "", True), diff --git a/notebook_intelligence/locale/notebook_intelligence.pot b/notebook_intelligence/locale/notebook_intelligence.pot index 24c00923..19419cb3 100644 --- a/notebook_intelligence/locale/notebook_intelligence.pot +++ b/notebook_intelligence/locale/notebook_intelligence.pot @@ -2816,6 +2816,10 @@ msgstr "" msgid "The turn ended in an error." msgstr "" +#: notebook_intelligence/extension.py +msgid "These MCP values are masked and have no saved value to keep: %1. Enter the real value." +msgstr "" + #: src/components/claude-mcp-panel.tsx msgid "These are Claude Code's own MCP servers (read from %1, project %2, and the per-user-per-project block). Independent of the NBI MCP Servers tab." msgstr "" diff --git a/notebook_intelligence/util.py b/notebook_intelligence/util.py index 356ab09b..7d610408 100644 --- a/notebook_intelligence/util.py +++ b/notebook_intelligence/util.py @@ -7,6 +7,7 @@ import shutil import signal import subprocess +import sys import time from pathlib import Path from typing import Optional, Set @@ -48,6 +49,102 @@ def get_jupyter_root_dir() -> str: return _jupyter_root_dir +# The agent's file tools stay inside the workspace, but a workspace that +# contains the user's home directory also contains NBI's own config. Those +# files hold API keys and MCP commands, and a write bypasses the checks the +# config API applies, so the file tools refuse them. +NBI_MCP_TEMP_FILENAME = "nbi.mcp.temp.json" +_registered_protected_paths: set = set() +_protected_paths_version = 0 + +# Resolving every candidate costs about 200 us, and search_files asks once +# per matched file. The resolved set is rebuilt when the home directory, +# prefix or registrations change, and after a few seconds so a symlink that +# was repointed is noticed. +_PROTECTED_PATHS_CACHE_SECONDS = 5.0 +# (key, expires_at, resolved paths, (st_dev, st_ino) of the ones that exist) +_protected_paths_cache = None + + +def register_protected_paths(paths) -> None: + """Add files the agent's file tools must not touch (NBIConfig registers + the ones it reads and writes).""" + global _protected_paths_version + for path in paths: + if path: + _registered_protected_paths.add(str(path)) + _protected_paths_version += 1 + + +def _default_protected_paths() -> list: + home = os.path.expanduser("~") + env_dir = os.path.join(sys.prefix, "share", "jupyter", "nbi") + user_dir = os.path.join(home, ".jupyter", "nbi") + return [ + os.path.join(env_dir, "config.json"), + os.path.join(env_dir, "mcp.json"), + os.path.join(user_dir, "config.json"), + os.path.join(user_dir, "mcp.json"), + os.path.join(user_dir, "user-data.json"), + os.path.join(sys.prefix, "share", "jupyter", "nbi-config.json"), + os.path.join(home, ".jupyter", "nbi-config.json"), + os.path.join(home, ".jupyter", "nbi-data.json"), + ] + + +def is_protected_nbi_path(path) -> bool: + """True for NBI's config and credential files, and the MCP editor's + temp copy of them. Compares resolved paths, so a symlink to one counts.""" + try: + resolved = Path(path).expanduser().resolve() + except (OSError, RuntimeError, ValueError): + return False + if resolved.name == NBI_MCP_TEMP_FILENAME: + return True + paths, file_ids = _resolved_protected_paths() + if resolved in paths: + return True + # Same file under another name: a different-case spelling on a + # case-insensitive filesystem, or a hard link. The path match above + # cannot see either, the device and inode can. + try: + info = os.stat(resolved) + except (OSError, ValueError): + return False + return (info.st_dev, info.st_ino) in file_ids + + +def invalidate_protected_paths_cache() -> None: + """Drop the cached protected set; the next check rebuilds it. NBIConfig + calls this after a save, which replaces the files and so their inodes.""" + global _protected_paths_cache + _protected_paths_cache = None + + +def _resolved_protected_paths() -> tuple: + global _protected_paths_cache + key = (os.path.expanduser("~"), sys.prefix, _protected_paths_version) + now = time.monotonic() + cached = _protected_paths_cache + if cached is not None and cached[0] == key and now < cached[1]: + return cached[2], cached[3] + resolved = set() + file_ids = set() + for candidate in [*_default_protected_paths(), *_registered_protected_paths]: + try: + path = Path(candidate).expanduser().resolve() + resolved.add(path) + info = os.stat(path) + except FileNotFoundError: + continue + except (OSError, RuntimeError, ValueError): + continue + file_ids.add((info.st_dev, info.st_ino)) + resolved, file_ids = frozenset(resolved), frozenset(file_ids) + _protected_paths_cache = (key, now + _PROTECTED_PATHS_CACHE_SECONDS, resolved, file_ids) + return resolved, file_ids + + def safe_jupyter_path(path: str) -> Path: """Resolve ``path`` and confirm it stays inside ``jupyter_root_dir``. @@ -647,6 +744,62 @@ def _looks_like_secret_env_name(name: str) -> bool: ) +# Where `redact_env_secrets` asks for the credentials NBI keeps in its own +# config files. One slot, owned by the running extension: the latest +# registration wins, so a test or a second manager cannot pile up providers. +_stored_secret_provider = None + +# The shell tools redact line by line, and the provider walks the whole +# config each time. The result is reused for a few seconds; NBIConfig also +# clears it on every load and save, so the TTL only covers a hand edit that +# nothing has reloaded yet. +_STORED_SECRET_CACHE_SECONDS = 5.0 +_stored_secret_cache = None # (provider, expires_at, values) +_stored_secret_generation = 0 + + +def set_stored_secret_provider(provider) -> None: + """Register a zero-argument callable returning the stored credential + values to redact, or ``None`` to clear it.""" + global _stored_secret_provider + _stored_secret_provider = provider + invalidate_stored_secret_cache() + + +def invalidate_stored_secret_cache() -> None: + """Drop the cached credential values; the next redaction asks again.""" + global _stored_secret_cache, _stored_secret_generation + _stored_secret_generation += 1 + _stored_secret_cache = None + + +def _stored_secret_values() -> list: + global _stored_secret_cache + provider = _stored_secret_provider + if provider is None: + return [] + cached = _stored_secret_cache + now = time.monotonic() + # Keyed on the provider object too, so swapping it, even by patching the + # module attribute, never serves another provider's values. + if cached is not None and cached[0] is provider and now < cached[1]: + return cached[2] + generation = _stored_secret_generation + try: + values = [v for v in provider() if isinstance(v, str)] + except Exception: + # Redaction is defense in depth; a broken provider must not turn a + # shell tool result into an error. Not cached, so it is retried. + log.debug("Stored-secret provider failed", exc_info=True) + return [] + # A load or save that ran while the provider was walking the config bumped + # the generation; what was collected may predate it, so use it once and + # let the next call collect again. + if generation == _stored_secret_generation: + _stored_secret_cache = (provider, now + _STORED_SECRET_CACHE_SECONDS, values) + return values + + def redact_env_secrets(text: str) -> str: """Replace any process-env values that look like secrets with ````. Defense in depth for the shell-execute tools, which @@ -690,6 +843,9 @@ def redact_env_secrets(text: str) -> str: for name, value in os.environ.items() if value and _looks_like_secret_env_name(name) ] + # Credentials NBI itself stores (API keys, MCP headers) rarely carry a + # known prefix, so a `cat config.json` would otherwise show them. + sensitive_values.extend(_stored_secret_values()) out = _replace_values(text, sensitive_values) # Pass 2: redact known-prefix forms. Catches base64/derived shapes # plus tokens an upstream tool might emit that aren't in this diff --git a/src/components/settings-panel.tsx b/src/components/settings-panel.tsx index b12263ac..163a4512 100644 --- a/src/components/settings-panel.tsx +++ b/src/components/settings-panel.tsx @@ -117,6 +117,23 @@ function useNbiPolicies() { const OPENAI_COMPATIBLE_CHAT_MODEL_ID = 'openai-compatible-chat-model'; +/** + * What the server sends in place of a stored API key (see + * credential_mask.py). Posting it back leaves the stored key as it is. + */ +const SECRET_MASK = '********'; + +/** + * Select the mask when a key field gains focus, so the first keystroke or + * paste replaces it. Without this, typing appends to the mask and saves + * `********abc` as the key. + */ +function selectMask(event: React.FocusEvent): void { + if (event.target.value === SECRET_MASK) { + event.target.select(); + } +} + /** * A required provider property left empty. The value persists as an empty * string and requests then go out with that field blank, which the endpoint @@ -149,6 +166,7 @@ export function ModelPropertyList(props: { description: string; value: string; optional?: boolean; + secret?: boolean; }[]; providerId: string; modelId: string; @@ -187,6 +205,7 @@ export function ModelPropertyField(props: { description: string; value: string; optional?: boolean; + secret?: boolean; }; inputName: string; onChange: (value: string) => void; @@ -204,6 +223,8 @@ export function ModelPropertyField(props: { setTouched(true)} /> {showError && ( @@ -2243,6 +2265,8 @@ function SettingsPanelComponentClaude(props: any) { setApiKey(event.target.value)} /> @@ -2478,6 +2503,8 @@ function SettingsPanelComponentAcp(props: any) { setApiKey(event.target.value)} /> diff --git a/tests/test_cell_output_features_response.py b/tests/test_cell_output_features_response.py index 21055730..6924b052 100644 --- a/tests/test_cell_output_features_response.py +++ b/tests/test_cell_output_features_response.py @@ -5,6 +5,7 @@ from types import SimpleNamespace +from notebook_intelligence.credential_mask import SECRET_MASK from notebook_intelligence.extension import ( FEATURE_POLICY_NAMES, _build_cell_output_features_response, @@ -271,8 +272,14 @@ def test_response_includes_every_known_lock_name(self): class TestScrubCredentialsForWire: - def test_unlocked_settings_pass_through_unchanged(self): + def test_unlocked_key_is_masked_not_sent(self): settings = {"api_key": "sk-secret", "base_url": "https://x"} + masked = _scrub_credentials_for_wire(settings, {}) + assert masked == {"api_key": SECRET_MASK, "base_url": "https://x"} + assert settings["api_key"] == "sk-secret" + + def test_settings_without_a_key_pass_through_unchanged(self): + settings = {"api_key": "", "base_url": "https://x"} assert _scrub_credentials_for_wire(settings, {}) is settings def test_claude_key_scrubbed_when_env_locks_it(self): @@ -288,12 +295,9 @@ def test_claude_key_scrubbed_when_env_locks_it(self): def test_acp_key_scrubbed_with_its_own_lock_name(self): settings = {"api_key": "sk-openai"} # The Claude lock must not scrub the Codex key. - assert ( - _scrub_credentials_for_wire( - settings, {"claude_api_key": "sk-env"}, "acp_api_key" - ) - is settings - ) + assert _scrub_credentials_for_wire( + settings, {"claude_api_key": "sk-env"}, "acp_api_key" + ) == {"api_key": SECRET_MASK} scrubbed = _scrub_credentials_for_wire( settings, {"acp_api_key": "sk-env"}, "acp_api_key" ) diff --git a/tests/test_config_handler_claude_settings.py b/tests/test_config_handler_claude_settings.py index e4b7c4b7..d66596ea 100644 --- a/tests/test_config_handler_claude_settings.py +++ b/tests/test_config_handler_claude_settings.py @@ -356,3 +356,55 @@ def test_a_malformed_config_file_fails_without_overwriting_it(self, config): with open(config.user_config_file) as f: assert f.read() == '{"claude_settings": {"enabled": true,}}' + + +class TestApiKeyMaskRoundTrip: + """The capabilities response masks a stored key; saving it back must not + overwrite the key with the mask.""" + + def test_claude_key_survives_a_posted_mask(self, config): + from notebook_intelligence.credential_mask import SECRET_MASK + + _store(config, claude_settings={"enabled": True, "api_key": "sk-real"}) + + stored = _post(config, {"claude_settings": {**PANEL_PAYLOAD, "api_key": SECRET_MASK}}) + + assert stored["api_key"] == "sk-real" + + def test_claude_key_can_still_be_replaced_or_cleared(self, config): + _store(config, claude_settings={"api_key": "sk-real"}) + assert _post(config, {"claude_settings": {**PANEL_PAYLOAD, "api_key": "sk-new"}})["api_key"] == "sk-new" + assert _post(config, {"claude_settings": {**PANEL_PAYLOAD, "api_key": ""}})["api_key"] == "" + + def test_acp_key_survives_a_posted_mask(self, config): + from notebook_intelligence.credential_mask import SECRET_MASK + + _store(config, acp_settings={"enabled": False, "api_key": "sk-openai"}) + + _post(config, {"acp_settings": {"enabled": False, "api_key": SECRET_MASK}}) + + assert _on_disk(config)["acp_settings"]["api_key"] == "sk-openai" + + def test_model_property_key_survives_a_posted_mask(self, config): + from notebook_intelligence.credential_mask import SECRET_MASK + + _store(config, chat_model={ + "provider": "openai-compatible", + "model": "openai-compatible-chat-model", + "properties": [ + {"id": "api_key", "value": "sk-real"}, + {"id": "base_url", "value": "https://old"}, + ], + }) + + _post(config, {"chat_model": { + "provider": "openai-compatible", + "model": "openai-compatible-chat-model", + "properties": [ + {"id": "api_key", "value": SECRET_MASK}, + {"id": "base_url", "value": "https://new"}, + ], + }}) + + props = {p["id"]: p["value"] for p in _on_disk(config)["chat_model"]["properties"]} + assert props == {"api_key": "sk-real", "base_url": "https://new"} diff --git a/tests/test_credential_mask.py b/tests/test_credential_mask.py new file mode 100644 index 00000000..8c3b5096 --- /dev/null +++ b/tests/test_credential_mask.py @@ -0,0 +1,246 @@ +# Copyright (c) Mehmet Bektas + +"""A stored API key reaches the browser masked and survives a round trip.""" + +import json + +from notebook_intelligence.api import LLMProviderProperty +from notebook_intelligence.credential_mask import ( + SECRET_MASK, + mask_model_config, + mask_model_properties, + mask_secret, + mask_settings_api_key, + restore_masked_api_key, + restore_masked_model_properties, +) +from notebook_intelligence.llm_providers.litellm_compatible_llm_provider import ( + LiteLLMCompatibleLLMProvider, +) +from notebook_intelligence.llm_providers.openai_compatible_llm_provider import ( + OpenAICompatibleLLMProvider, +) + + +def _model(provider="openai-compatible", model="m", key="sk-real"): + return { + "provider": provider, + "id": model, + "properties": [ + {"id": "api_key", "value": key, "secret": True}, + {"id": "base_url", "value": "https://x", "secret": False}, + ], + } + + +class TestMasking: + def test_mask_secret_keeps_empty_values_empty(self): + assert mask_secret("") == "" + assert mask_secret(None) is None + assert mask_secret("sk-real") == SECRET_MASK + + def test_settings_key_masked_on_a_copy(self): + settings = {"api_key": "sk-real", "enabled": True} + assert mask_settings_api_key(settings) == { + "api_key": SECRET_MASK, + "enabled": True, + } + assert settings["api_key"] == "sk-real" + + def test_only_secret_properties_are_masked(self): + original = _model() + masked = mask_model_properties([original])[0] + values = {p["id"]: p["value"] for p in masked["properties"]} + assert values == {"api_key": SECRET_MASK, "base_url": "https://x"} + assert original["properties"][0]["value"] == "sk-real" + + def test_unflagged_api_key_property_is_masked_and_flagged(self): + model = { + "provider": "ext", + "id": "m", + "properties": [ + {"id": "api_key", "value": "sk-real"}, + {"id": "base_url", "value": "https://x"}, + ], + } + masked = mask_model_properties([model])[0]["properties"] + assert masked[0] == {"id": "api_key", "value": SECRET_MASK, "secret": True} + assert masked[1] == {"id": "base_url", "value": "https://x"} + + def test_blank_secret_property_stays_blank(self): + masked = mask_model_properties([_model(key="")])[0] + assert masked["properties"][0]["value"] == "" + + def test_built_in_api_key_properties_are_flagged_secret(self): + for provider in (OpenAICompatibleLLMProvider(), LiteLLMCompatibleLLMProvider()): + for model in provider.chat_models + provider.inline_completion_models: + assert model.get_property("api_key").secret is True + assert model.get_property("model_id").secret is False + + def test_property_to_dict_carries_the_flag(self): + assert LLMProviderProperty("k", "K", "K", "v", secret=True).to_dict()[ + "secret" + ] is True + + +class TestMaskModelConfig: + def test_masks_secret_properties_of_a_stored_entry(self): + stored = { + "provider": "litellm-compatible", + "model": "m", + "properties": [ + {"id": "api_key", "value": "sk-real"}, + {"id": "base_url", "value": "https://x"}, + ], + } + assert mask_model_config(stored) == { + "provider": "litellm-compatible", + "model": "m", + "properties": [ + {"id": "api_key", "value": SECRET_MASK}, + {"id": "base_url", "value": "https://x"}, + ], + } + assert stored["properties"][0]["value"] == "sk-real" + + def test_entries_without_properties_pass_through(self): + copilot = {"provider": "github-copilot", "model": "gpt-4.1"} + assert mask_model_config(copilot) is copilot + assert mask_model_config(None) is None + + def test_a_real_nbi_config_entry_is_masked(self, tmp_path): + from notebook_intelligence.config import NBIConfig + + config = NBIConfig.__new__(NBIConfig) + config._feature_policies, config._string_overrides = {}, {} + config.env_config = {} + config.user_config = { + "chat_model": { + "provider": "openai-compatible", + "model": "openai-compatible-chat-model", + "properties": [{"id": "api_key", "value": "sk-LEAK"}], + } + } + assert "sk-LEAK" in json.dumps(config.chat_model) + assert "sk-LEAK" not in json.dumps(mask_model_config(config.chat_model)) + + +class TestRestore: + def test_posted_mask_becomes_the_stored_key(self): + assert restore_masked_api_key( + {"api_key": SECRET_MASK, "enabled": False}, {"api_key": "sk-real"} + ) == {"api_key": "sk-real", "enabled": False} + + def test_new_key_is_kept(self): + assert restore_masked_api_key( + {"api_key": "sk-new"}, {"api_key": "sk-real"} + ) == {"api_key": "sk-new"} + + def test_cleared_key_stays_cleared(self): + assert restore_masked_api_key( + {"api_key": ""}, {"api_key": "sk-real"} + ) == {"api_key": ""} + + def test_mask_with_nothing_stored_is_never_saved(self): + assert restore_masked_api_key({"api_key": SECRET_MASK}, None) == { + "api_key": "" + } + + def test_model_mask_restored_for_the_same_model(self): + stored = { + "provider": "openai-compatible", + "model": "m", + "properties": [{"id": "api_key", "value": "sk-real"}], + } + posted = { + "provider": "openai-compatible", + "model": "m", + "properties": [ + {"id": "api_key", "value": SECRET_MASK}, + {"id": "base_url", "value": "https://y"}, + ], + } + restored = restore_masked_model_properties(posted, stored) + assert restored["properties"] == [ + {"id": "api_key", "value": "sk-real"}, + {"id": "base_url", "value": "https://y"}, + ] + + def test_model_mask_for_another_model_is_dropped(self): + stored = { + "provider": "openai-compatible", + "model": "m", + "properties": [{"id": "api_key", "value": "sk-real"}], + } + posted = { + "provider": "litellm-compatible", + "model": "other", + "properties": [{"id": "api_key", "value": SECRET_MASK}], + } + restored = restore_masked_model_properties(posted, stored) + assert restored["properties"] == [{"id": "api_key", "value": ""}] + + def test_model_without_properties_passes_through(self): + posted = {"provider": "github-copilot", "model": "gpt-4.1"} + assert restore_masked_model_properties(posted, None) is posted + + +class TestCapabilitiesResponse: + """GetCapabilitiesHandler.get is where the masks are applied.""" + + def test_no_stored_key_reaches_the_response(self, monkeypatch): + from unittest.mock import MagicMock + + from notebook_intelligence import extension as ext_module + + manager = MagicMock() + manager.chat_model_ids = [_model(key="sk-chat")] + manager.inline_completion_model_ids = [_model(key="sk-inline")] + manager.nbi_config.claude_settings = {"api_key": "sk-claude", "enabled": False} + manager.nbi_config.acp_settings = {"api_key": "sk-acp", "enabled": False} + # The stored model entries travel in the response too, apart from the + # model lists; a mock here would hide a leak through them. + manager.nbi_config.chat_model = { + "provider": "openai-compatible", + "model": "m", + "properties": [{"id": "api_key", "value": "sk-stored-chat"}], + } + manager.nbi_config.inline_completion_model = { + "provider": "openai-compatible", + "model": "m", + "properties": [{"id": "api_key", "value": "sk-stored-inline"}], + } + monkeypatch.setattr(ext_module, "ai_service_manager", manager) + # Capture the dict before it is serialized; most fields are mocks. + monkeypatch.setattr(ext_module.json, "dumps", lambda obj, *a, **k: obj) + handler = MagicMock() + handler.tour_config_path = "" + handler.string_overrides = {} + for name in ( + "disabled_tools", + "disabled_providers", + "disabled_coding_agent_launchers", + "additional_skipped_workspace_directories", + ): + setattr(handler, name, []) + + ext_module.GetCapabilitiesHandler.get(handler) + response = handler.finish.call_args[0][0] + + for models in (response["chat_models"], response["inline_completion_models"]): + key = next(p for p in models[0]["properties"] if p["id"] == "api_key") + assert key["value"] == SECRET_MASK + assert response["claude_settings"]["api_key"] == SECRET_MASK + assert response["acp_settings"]["api_key"] == SECRET_MASK + for stored in (response["chat_model"], response["inline_completion_model"]): + assert stored["properties"] == [{"id": "api_key", "value": SECRET_MASK}] + assert stored["provider"] == "openai-compatible" + for secret in ( + "sk-chat", + "sk-inline", + "sk-claude", + "sk-acp", + "sk-stored-chat", + "sk-stored-inline", + ): + assert secret not in repr(response) diff --git a/tests/test_credential_surfaces.py b/tests/test_credential_surfaces.py new file mode 100644 index 00000000..133afc1a --- /dev/null +++ b/tests/test_credential_surfaces.py @@ -0,0 +1,611 @@ +# Copyright (c) Mehmet Bektas + +"""Credentials NBI stores must not reach the browser, the workspace, the +agent's file tools, or shell output in clear text.""" + +import asyncio +import json +from types import SimpleNamespace +from unittest.mock import MagicMock + +import pytest + +import notebook_intelligence.built_in_toolsets as toolsets +from notebook_intelligence import extension as ext_module +from notebook_intelligence import util as util_mod +from notebook_intelligence.claude_mcp_manager import ClaudeMCPServer +from notebook_intelligence.credential_mask import ( + SECRET_MASK, + UnresolvedMaskError, + collect_stored_secrets, + mask_mcp_config, + restore_masked_mcp_config, +) +from notebook_intelligence.util import ( + is_protected_nbi_path, + redact_env_secrets, + set_stored_secret_provider, +) + +MCP = { + "mcpServers": { + "remote": { + "url": "https://mcp.example", + "headers": {"Authorization": "Bearer tok-abcdef123456", "X-Tenant": "acme"}, + }, + "local": { + "command": "uvx", + "env": {"GITHUB_TOKEN": "ghp-local-secret-1", "LOG_LEVEL": "verbose-debug"}, + }, + } +} + + +class TestClaudeMcpWire: + def _server(self): + return ClaudeMCPServer( + name="x", + scope="user", + transport="http", + url="https://m", + headers={"Authorization": "Bearer abc123", "X-Tenant": "acme"}, + env={"API_KEY": "k-secret-value"}, + ) + + def test_wire_dict_masks_every_env_and_header_value(self): + wire = self._server().to_wire_dict() + assert wire["headers"] == {"Authorization": SECRET_MASK, "X-Tenant": SECRET_MASK} + assert wire["env"] == {"API_KEY": SECRET_MASK} + assert wire["url"] == "https://m" + assert "abc123" not in json.dumps(wire) + + def test_internal_dict_keeps_the_values(self): + assert self._server().to_dict()["headers"]["Authorization"] == "Bearer abc123" + + +class TestMcpEditorMask: + def test_only_credential_named_values_are_masked(self): + masked = mask_mcp_config(MCP)["mcpServers"] + assert masked["remote"]["headers"] == { + "Authorization": SECRET_MASK, + "X-Tenant": "acme", + } + assert masked["local"]["env"] == { + "GITHUB_TOKEN": SECRET_MASK, + "LOG_LEVEL": "verbose-debug", + } + assert MCP["mcpServers"]["local"]["env"]["GITHUB_TOKEN"] == "ghp-local-secret-1" + + def test_masked_document_round_trips_to_the_original(self): + assert restore_masked_mcp_config(mask_mcp_config(MCP), MCP) == MCP + + def test_an_edited_value_replaces_the_stored_one(self): + posted = mask_mcp_config(MCP) + posted["mcpServers"]["local"]["env"]["GITHUB_TOKEN"] = "ghp-new-value-22" + restored = restore_masked_mcp_config(posted, MCP) + assert restored["mcpServers"]["local"]["env"]["GITHUB_TOKEN"] == "ghp-new-value-22" + + def test_a_mask_with_nothing_stored_is_refused(self): + posted = mask_mcp_config(MCP) + posted["mcpServers"]["renamed"] = posted["mcpServers"].pop("remote") + with pytest.raises(UnresolvedMaskError, match=r"renamed\.headers\.Authorization") as caught: + restore_masked_mcp_config(posted, MCP) + assert caught.value.names == ["renamed.headers.Authorization"] + + def test_non_server_documents_pass_through(self): + assert mask_mcp_config({}) == {} + assert restore_masked_mcp_config({"mcpServers": {}}, None) == {"mcpServers": {}} + + +class FakeConfig: + def __init__(self, mcp): + self.mcp = mcp + self.user_mcp = None + self.saved = False + + def load(self): + pass + + def save(self): + self.saved = True + + +def _handler(): + handler = MagicMock(spec=ext_module.MCPConfigFileHandler) + handler.request = MagicMock() + return handler + + +class TestMcpConfigHandler: + @pytest.fixture + def config(self, monkeypatch): + config = FakeConfig(json.loads(json.dumps(MCP))) + manager = SimpleNamespace( + nbi_config=config, + get_mcp_stdio_command_allowlist=lambda: [], + update_mcp_servers=lambda: None, + ) + monkeypatch.setattr(ext_module, "ai_service_manager", manager) + return config + + def test_get_sends_no_credential(self, config): + handler = _handler() + ext_module.MCPConfigFileHandler.get.__wrapped__(handler) + body = handler.finish.call_args[0][0] + assert "tok-abcdef123456" not in body + assert "ghp-local-secret-1" not in body + assert "acme" in body + + def test_saving_the_masked_document_keeps_the_stored_values(self, config): + handler = _handler() + handler.request.body = json.dumps(mask_mcp_config(MCP)).encode() + ext_module.MCPConfigFileHandler.post.__wrapped__(handler) + assert config.saved + assert config.user_mcp == MCP + + def test_an_unresolvable_mask_is_a_400_and_nothing_is_saved(self, config): + posted = mask_mcp_config(MCP) + posted["mcpServers"]["renamed"] = posted["mcpServers"].pop("remote") + handler = _handler() + handler.request.body = json.dumps(posted).encode() + ext_module.MCPConfigFileHandler.post.__wrapped__(handler) + handler.set_status.assert_called_with(400) + message = json.loads(handler.finish.call_args[0][0])["message"] + assert "renamed.headers.Authorization" in message + assert "Enter the real value" in message + assert not config.saved + + +@pytest.fixture +def home_root(tmp_path, monkeypatch): + """A workspace that is the home directory, as when JupyterLab starts in ~.""" + monkeypatch.setenv("HOME", str(tmp_path)) + nbi = tmp_path / ".jupyter" / "nbi" + nbi.mkdir(parents=True) + (nbi / "config.json").write_text('{"claude_settings": {"api_key": "sk-REAL-9999"}}') + (nbi / "mcp.json").write_text("{}") + (tmp_path / "notes.txt").write_text("api_key lives elsewhere\n") + monkeypatch.setattr(util_mod, "_jupyter_root_dir", str(tmp_path)) + return tmp_path + + +def _run(tool, **kwargs): + return asyncio.run(tool._tool_function(**kwargs)) + + +class TestFileToolsRefuseNbiConfig: + def test_read_file_refuses_config_and_still_reads_others(self, home_root): + result = _run(toolsets.read_file, file_path=".jupyter/nbi/config.json") + assert result == toolsets.PROTECTED_NBI_FILE_MESSAGE + assert "REAL" not in result + assert "elsewhere" in _run(toolsets.read_file, file_path="notes.txt") + + def test_write_tools_refuse_config(self, home_root): + target = ".jupyter/nbi/mcp.json" + assert _run(toolsets.write_to_file, file_path=target, content="x") == ( + toolsets.PROTECTED_NBI_FILE_MESSAGE + ) + assert _run( + toolsets.insert_content, file_path=target, line_number=1, content="x" + ) == toolsets.PROTECTED_NBI_FILE_MESSAGE + assert (home_root / ".jupyter/nbi/mcp.json").read_text() == "{}" + + def test_search_skips_config(self, home_root): + result = _run( + toolsets.search_files, + pattern="**/*", + directory=".", + content_pattern="api_key", + ) + assert "REAL" not in result + assert "notes.txt" in result + + def test_a_symlink_to_config_is_refused(self, home_root): + (home_root / "link.json").symlink_to(home_root / ".jupyter/nbi/config.json") + assert _run(toolsets.read_file, file_path="link.json") == ( + toolsets.PROTECTED_NBI_FILE_MESSAGE + ) + + def test_editor_temp_copy_is_refused_wherever_it_is(self, home_root): + (home_root / "nbi.mcp.temp.json").write_text("{}") + assert is_protected_nbi_path(home_root / "nbi.mcp.temp.json") + assert not is_protected_nbi_path(home_root / "notes.txt") + + +class TestStoredSecretRedaction: + @pytest.fixture(autouse=True) + def _clear(self): + yield + set_stored_secret_provider(None) + + def test_collects_keys_headers_and_secret_env_values(self): + values = collect_stored_secrets( + [ + { + "claude_settings": {"api_key": "sk-claude-1234"}, + "acp_settings": {"api_key": "acp-key-12345"}, + "chat_model": { + "properties": [ + {"id": "api_key", "value": "proxy-key-98765"}, + {"id": "base_url", "value": "https://proxy.example"}, + ] + }, + } + ], + MCP, + ) + assert { + "sk-claude-1234", + "acp-key-12345", + "proxy-key-98765", + "Bearer tok-abcdef123456", + "tok-abcdef123456", + "ghp-local-secret-1", + } <= set(values) + assert "https://proxy.example" not in values + assert "acme" not in values and "verbose-debug" not in values + + @pytest.mark.parametrize( + "placeholder", ["not-needed", "lm-studio", "Dummy-API-Key", "sk_no_key_required", "ollama"] + ) + def test_placeholder_keys_are_not_collected(self, placeholder): + values = collect_stored_secrets( + [ + { + "claude_settings": {"api_key": placeholder}, + "chat_model": { + "properties": [{"id": "api_key", "value": placeholder}] + }, + } + ], + {"mcpServers": {"s": {"headers": {"Authorization": f"Bearer {placeholder}"}}}}, + ) + assert placeholder not in values + + def test_placeholder_is_left_alone_in_shell_output(self): + set_stored_secret_provider( + lambda: collect_stored_secrets( + [{"claude_settings": {"api_key": "not-needed"}}], None + ) + ) + text = "the value is not-needed here" + assert redact_env_secrets(text) == text + + def test_a_real_key_next_to_a_placeholder_is_still_collected(self): + values = collect_stored_secrets( + [ + { + "claude_settings": {"api_key": "not-needed"}, + "acp_settings": {"api_key": "acp-key-12345"}, + } + ], + None, + ) + assert values == ["acp-key-12345"] + + def test_shell_output_loses_a_key_without_a_known_prefix(self): + text = '{"api_key": "litellm-master-key-9f8e7d6c"}' + assert "litellm-master-key" in redact_env_secrets(text) + set_stored_secret_provider(lambda: ["litellm-master-key-9f8e7d6c"]) + assert "litellm-master-key" not in redact_env_secrets(text) + + def test_a_failing_provider_does_not_break_redaction(self): + def boom(): + raise RuntimeError("config unreadable") + + set_stored_secret_provider(boom) + assert redact_env_secrets("hello") == "hello" + + +class TestCliErrorScrubbing: + """`claude mcp add` takes env and header values as flags, and the + handlers forward the CLI's error text to the browser.""" + + def _manager(self, tmp_path, monkeypatch, stderr): + from pathlib import Path + + from notebook_intelligence.claude_mcp_manager import ClaudeMCPManager + + home = tmp_path / "home" + home.mkdir() + monkeypatch.setattr(Path, "home", classmethod(lambda cls: home)) + monkeypatch.delenv("CLAUDE_CONFIG_DIR", raising=False) + monkeypatch.setattr( + "notebook_intelligence._claude_cli.resolve_claude_cli_path", + lambda: "/usr/local/bin/claude", + ) + + async def fake_subprocess(*argv, **kwargs): + proc = MagicMock() + + async def communicate(): + return (b"", stderr) + + proc.communicate = communicate + proc.returncode = 1 + return proc + + monkeypatch.setattr(asyncio, "create_subprocess_exec", fake_subprocess) + project = tmp_path / "project" + project.mkdir() + return ClaudeMCPManager(working_dir=str(project)) + + def test_header_and_env_values_are_scrubbed_from_the_error( + self, tmp_path, monkeypatch + ): + stderr = ( + b"error: invalid header 'Authorization: Bearer tok-abcdef123456'; " + b"env GITHUB_TOKEN=ghp-local-secret-1 rejected" + ) + manager = self._manager(tmp_path, monkeypatch, stderr) + with pytest.raises(ValueError) as caught: + asyncio.run( + manager.add_server( + name="srv", + scope="user", + transport="http", + command_or_url="https://mcp.example", + headers={"Authorization": "Bearer tok-abcdef123456"}, + env={"GITHUB_TOKEN": "ghp-local-secret-1"}, + ) + ) + message = str(caught.value) + assert "tok-abcdef123456" not in message + assert "ghp-local-secret-1" not in message + assert "invalid header" in message + + def test_stored_credentials_are_scrubbed_even_when_not_in_the_call( + self, tmp_path, monkeypatch + ): + monkeypatch.setattr( + "notebook_intelligence.util._stored_secret_provider", + lambda: ["proxy-master-key-0042"], + ) + manager = self._manager( + tmp_path, monkeypatch, b"401 Received API Key = proxy-master-key-0042" + ) + with pytest.raises(ValueError) as caught: + asyncio.run(manager.remove_server("srv", "user")) + assert "proxy-master-key-0042" not in str(caught.value) + + +class TestClaudeStreamErrorScrubbing: + def test_inline_stream_error_marker_is_scrubbed(self, monkeypatch): + from notebook_intelligence.claude import ClaudeChatModel + + monkeypatch.setattr( + "notebook_intelligence.util._stored_secret_provider", + lambda: ["proxy-master-key-0042"], + ) + model = ClaudeChatModel.__new__(ClaudeChatModel) + model._model_id = "claude-sonnet-test" + model._model_name = "Claude Sonnet (test)" + model._context_window = 200000 + model._supports_tools = True + model._client = MagicMock() + boom = RuntimeError("401 Received API Key = proxy-master-key-0042") + model._client.messages.stream = MagicMock(side_effect=boom) + response = MagicMock() + + with pytest.raises(RuntimeError): + model.completions( + messages=[{"role": "user", "content": "x"}], response=response + ) + + payload = response.stream.call_args_list[-1].args[0] + assert "proxy-master-key-0042" not in repr(payload) + assert "Stream interrupted" in payload["choices"][0]["delta"]["content"] + assert "proxy-master-key-0042" not in payload["nbi_stream_error"] + + +class TestRedactionCaches: + """The path check and the credential list are reused between calls; the + reuse must not outlive a change that matters.""" + + @pytest.fixture(autouse=True) + def _clear(self): + yield + set_stored_secret_provider(None) + + def _counting_provider(self, values): + calls = [] + + def provider(): + calls.append(1) + return list(values) + + return provider, calls + + def test_credentials_are_collected_once_across_many_lines(self): + provider, calls = self._counting_provider(["litellm-master-key-9f8e7d6c"]) + set_stored_secret_provider(provider) + for _ in range(50): + assert "" in redact_env_secrets("key litellm-master-key-9f8e7d6c") + assert len(calls) == 1 + + def test_invalidating_picks_up_a_changed_credential(self): + values = ["first-secret-value-1"] + set_stored_secret_provider(lambda: list(values)) + assert "first-secret-value-1" not in redact_env_secrets("first-secret-value-1") + values[:] = ["second-secret-value-2"] + util_mod.invalidate_stored_secret_cache() + assert "second-secret-value-2" not in redact_env_secrets("second-secret-value-2") + + def test_config_save_and_load_invalidate(self, tmp_path): + from notebook_intelligence.config import NBIConfig + + config = NBIConfig.__new__(NBIConfig) + config.nbi_user_dir = str(tmp_path / "nbi") + config.user_config_file = str(tmp_path / "nbi" / "config.json") + config.user_mcp_file = str(tmp_path / "nbi" / "mcp.json") + config.env_config_file = str(tmp_path / "env-config.json") + config.env_mcp_file = str(tmp_path / "env-mcp.json") + config.deprecated_env_config_file = str(tmp_path / "dep-env.json") + config.deprecated_user_config_file = str(tmp_path / "dep-user.json") + config.user_config, config.env_config = {}, {} + config.user_mcp, config.env_mcp = {}, {} + provider, calls = self._counting_provider(["litellm-master-key-9f8e7d6c"]) + set_stored_secret_provider(provider) + + redact_env_secrets("x") + redact_env_secrets("x") + assert len(calls) == 1 + config.save() + redact_env_secrets("x") + assert len(calls) == 2 + config.load() + redact_env_secrets("x") + assert len(calls) == 3 + + def test_a_swapped_provider_is_never_served_stale_values(self, monkeypatch): + set_stored_secret_provider(lambda: ["old-provider-secret-1"]) + redact_env_secrets("x") + monkeypatch.setattr( + util_mod, "_stored_secret_provider", lambda: ["new-provider-secret-2"] + ) + assert "new-provider-secret-2" not in redact_env_secrets("new-provider-secret-2") + + def test_a_failing_provider_is_retried_not_cached(self): + state = {"fail": True} + + def provider(): + if state["fail"]: + raise RuntimeError("config unreadable") + return ["recovered-secret-value"] + + set_stored_secret_provider(provider) + assert redact_env_secrets("recovered-secret-value") == "recovered-secret-value" + state["fail"] = False + assert "recovered-secret-value" not in redact_env_secrets("recovered-secret-value") + + def test_a_newly_registered_protected_path_is_noticed(self, tmp_path, monkeypatch): + monkeypatch.setenv("HOME", str(tmp_path)) + extra = tmp_path / "elsewhere" / "extra-config.json" + extra.parent.mkdir() + extra.write_text("{}") + assert not is_protected_nbi_path(extra) + monkeypatch.setattr( + util_mod, "_registered_protected_paths", set(util_mod._registered_protected_paths) + ) + util_mod.register_protected_paths([extra]) + assert is_protected_nbi_path(extra) + + def test_a_repointed_symlink_is_noticed_after_the_ttl(self, tmp_path, monkeypatch): + monkeypatch.setenv("HOME", str(tmp_path)) + nbi = tmp_path / ".jupyter" / "nbi" + nbi.mkdir(parents=True) + shared_a = tmp_path / "shared-a.json" + shared_b = tmp_path / "shared-b.json" + shared_a.write_text("{}") + shared_b.write_text("{}") + (nbi / "config.json").symlink_to(shared_a) + monkeypatch.setattr(util_mod, "_PROTECTED_PATHS_CACHE_SECONDS", 0.0) + assert is_protected_nbi_path(shared_a) and not is_protected_nbi_path(shared_b) + (nbi / "config.json").unlink() + (nbi / "config.json").symlink_to(shared_b) + assert is_protected_nbi_path(shared_b) + + +class TestReviewFixes: + @pytest.fixture(autouse=True) + def _clear(self): + yield + set_stored_secret_provider(None) + + def test_bearer_placeholder_is_not_collected(self): + values = collect_stored_secrets( + [], {"mcpServers": {"s": {"headers": {"Authorization": "Bearer not-needed"}}}} + ) + assert values == [] + + def test_a_save_during_collection_is_not_cached(self): + calls = [] + + def provider(): + calls.append(1) + if len(calls) == 1: + # A config save lands while this walk is in progress. + util_mod.invalidate_stored_secret_cache() + return ["litellm-master-key-9f8e7d6c"] + + set_stored_secret_provider(provider) + redact_env_secrets("x") + redact_env_secrets("x") + assert len(calls) == 2 # the first result predated the save + redact_env_secrets("x") + assert len(calls) == 2 # the second was cached normally + + def test_a_hard_link_to_the_config_is_protected(self, home_root): + link = home_root / "innocent-name.json" + import os + + os.link(home_root / ".jupyter/nbi/config.json", link) + assert is_protected_nbi_path(link) + assert _run(toolsets.read_file, file_path="innocent-name.json") == ( + toolsets.PROTECTED_NBI_FILE_MESSAGE + ) + + def test_config_save_drops_the_protected_cache(self, tmp_path): + from notebook_intelligence.config import NBIConfig + + config = NBIConfig.__new__(NBIConfig) + config.nbi_user_dir = str(tmp_path / "nbi") + config.user_config_file = str(tmp_path / "nbi" / "config.json") + config.user_mcp_file = str(tmp_path / "nbi" / "mcp.json") + config.user_config, config.user_mcp = {}, {} + is_protected_nbi_path(tmp_path / "x.json") + assert util_mod._protected_paths_cache is not None + config.save() + assert util_mod._protected_paths_cache is None + + def test_a_mention_cannot_reach_the_config_through_a_symlink(self, home_root): + from notebook_intelligence.chatbook_mentions import _safe_relative_path + + (home_root / "docs").mkdir() + (home_root / "docs" / "notes.json").symlink_to( + home_root / ".jupyter/nbi/config.json" + ) + with pytest.raises(ValueError, match="unsafe mention path"): + _safe_relative_path("docs/notes.json") + assert _safe_relative_path("notes.txt").name == "notes.txt" + + +class TestClaudeChatErrorScrubbing: + def _stream(self, participant): + from notebook_intelligence.api import ChatResponse, MarkdownData + from notebook_intelligence import i18n + from tests.test_claude_client import _make_chat_request + + response = MagicMock(spec=ChatResponse, trans=i18n.english()) + asyncio.run(participant.handle_chat_request(_make_chat_request(), response)) + return [ + call.args[0].content + for call in response.stream.call_args_list + if isinstance(call.args[0], MarkdownData) + ] + + @pytest.fixture(autouse=True) + def _provider(self): + set_stored_secret_provider(lambda: ["proxy-master-key-0042"]) + yield + set_stored_secret_provider(None) + + def test_the_bail_out_string_is_scrubbed(self): + from tests.test_claude_client import _make_participant + + participant = _make_participant() + participant._client.query.return_value = "401 Received API Key = proxy-master-key-0042" + messages = self._stream(participant) + assert any("Claude agent error" in m for m in messages) + assert "proxy-master-key-0042" not in " ".join(messages) + + def test_an_exception_is_scrubbed(self): + from tests.test_claude_client import _make_participant + + participant = _make_participant() + participant._client.query.side_effect = RuntimeError( + "401 Received API Key = proxy-master-key-0042" + ) + messages = self._stream(participant) + assert any("Error" in m for m in messages) + assert "proxy-master-key-0042" not in " ".join(messages) diff --git a/tests/ts/settings-property-field.test.tsx b/tests/ts/settings-property-field.test.tsx index b76ce3f5..ee0200da 100644 --- a/tests/ts/settings-property-field.test.tsx +++ b/tests/ts/settings-property-field.test.tsx @@ -112,3 +112,86 @@ describe('ModelPropertyField', () => { ).toBeNull(); }); }); + +describe('ModelPropertyField secret', () => { + it('renders a secret property as a password field', () => { + const { container } = render( + undefined} + /> + ); + + const input = container.querySelector('input') as HTMLInputElement; + expect(input.type).toBe('password'); + expect(input.autocomplete).toBe('new-password'); + }); + + it('keeps a plain property as text', () => { + const { container } = render( + undefined} + /> + ); + + expect((container.querySelector('input') as HTMLInputElement).type).toBe( + 'text' + ); + }); + + it('selects the mask on focus so typing replaces it', () => { + const { container } = render( + undefined} + /> + ); + const input = container.querySelector('input') as HTMLInputElement; + const select = jest.spyOn(input, 'select'); + + fireEvent.focus(input); + + expect(select).toHaveBeenCalledTimes(1); + }); + + it('leaves a typed key alone on focus', () => { + const { container } = render( + undefined} + /> + ); + const input = container.querySelector('input') as HTMLInputElement; + const select = jest.spyOn(input, 'select'); + + fireEvent.focus(input); + + expect(select).not.toHaveBeenCalled(); + }); +});