fix(2.13.x): parse-limit regression, gateway LLM output and OSV advisory cap

CHANGELOG.md

@@ -1,6 +1,10 @@
 ### Unreleased maintenance fixes
 ### Fixes
 * Backport scoped `allowed-tools` grant handling from #768: recognize declarations such as `Bash(git:*)`, preserve spaces and commas inside scopes, and keep malformed grants or embedded tool names from declaring shell capability. Command restrictions remain intact in the manifest.
+* Stop backtick words in Python docstrings from marking files partially inspected (`static_parse_limit`) after the 2.13.3 shell-ownership backport. A docstring or other bare string statement is discarded, so its own body bounds a command named in it, unless the module reads docstrings back through `__doc__` or `getdoc`. The string-value guard also treats non-executing `os` calls (such as `os.makedirs` and `os.kill`) and name attributes (such as `type(error).__name__`) as inert; shell sinks, reflection and object escapes keep the whole-source fallback.
+* Ask Claude models reached through OpenAI-compatible gateways for exactly one JSON object when the schema is bound as `json_schema`. Gateways can serve that format through a tool Claude may skip, and a prose answer is rejected as `llm_structured_response_invalid`. Validation is unchanged.
+* Report which check rejected a model answer. Ledger rows and logs for rejected structured responses carry a `diagnostic` built only from fixed codes (check, channel, completion label, JSON shape, schema paths and per-attempt checks), never model output.
+* Retain up to 64 OSV advisories per locked package (was 16), so a dependency such as `filelock` with 17 advisories no longer leaves `poetry.lock` partial with `output_limit`. Per-scan detail and result budgets are unchanged.
 ---
 ### 2.13.3 (Unreleased)
 ### Security and integrity fixes

docs/SC4-osv-live-vulnerability-lookups.md

@@ -103,7 +103,7 @@ _analyze_dependencies(content, file_path)
 | **Sync vs. async** | Synchronous `httpx.Client` | The analyzer pipeline is sync. A single batch call completes in <500 ms for typical dependency files. |
 | **Caching** | In-memory dict with 1-hour TTL, keyed on `(name, version, ecosystem)` | Prevents redundant API calls when multiple skills share dependencies. Skillspector runs are short-lived CLI invocations, so memory is not a concern. |
 | **Graceful degradation** | On timeout/network error, fall back to the static list (`_FALLBACK_VULNERABLE_*`) | Ensures the tool works in air-gapped or offline environments. A warning is logged when falling back. |
-| **Finding detail level** | Batch query returns vuln IDs; detail fetched via `GET /v1/vulns/{id}` for up to 10 vulns per package | Keeps latency low — the 10-vulnerability cap limits the number of sequential detail fetches per package. The implementation selects the first 10 vulnerability IDs as returned by the OSV batch API (`_fetch_vuln_details(vuln_ids[:10])`) with no severity-based sorting or prioritisation. When more than 10 IDs are returned, the OSV client logs a warning indicating the total count and that only the first 10 will be processed, so users are alerted to the truncation. Finding messages include OSV/GHSA/CVE IDs with summaries. |
+| **Finding detail level** | Batch query returns vuln IDs; detail fetched via `GET /v1/vulns/{id}` for up to 64 vulns per package (`MAX_OSV_VULNS_PER_PACKAGE`) | The per-package cap and the aggregate per-scan budgets (64 detail requests, 256 retained results) bound latency. The implementation selects vulnerability IDs in the order the OSV batch API returns them, with no severity-based sorting or prioritisation. When a package or the scan exceeds a cap, the dependency manifest is reported as partially inspected with `output_limit`, because an omitted advisory could carry the worst severity. Finding messages include OSV/GHSA/CVE IDs with summaries. |
 | **Confidence mapping** | Map OSV severity → confidence: CRITICAL=0.9, HIGH=0.8, MEDIUM=0.7, LOW=0.6 | Replaces the per-entry hardcoded confidence values with a systematic mapping. |
 | **Timeout** | 10 s connect + read | Generous for a single POST. If exceeded, fallback activates gracefully. |
 | **Severity aggregation** | When multiple advisories affect one package, the worst severity is used for the finding | A single SC4 finding is emitted per package with a count and summary of all advisories. |

docs/scan-completeness.md

@@ -129,7 +129,7 @@ The examples below use static-only analysis, no baseline suppression, and benign
 
 Fatal execution failure takes precedence and exits 2 regardless of strict flags. Otherwise, the CLI exits 1 for requested incomplete/findings gates or a risk score above 50. Thus **default CLI exit 0 is not an installation-safety verdict**. MCP independently requires successful execution, complete analysis, zero entirely uninspected files, a score at most 50, and fulfillment of any requested LLM analysis. The table explicitly requests `use_llm=false`; default LLM requests have an additional requirement.
 
-JSON and MCP expose `analysis_completeness.status`, `is_complete`, `ledger_exceptions`, `scope_exclusions`, `analyzer_statuses`, and `limitations`, as well as `execution_successful`. Exceptions include reason, message, path, fatality, available source lines, and applicable limit metrics. Terminal and Markdown reports show these projections; SARIF uses invocation completeness and notifications. The report raises an otherwise `SAFE` recommendation to `CAUTION` for incomplete analysis while retaining the actual score and severity.
+JSON and MCP expose `analysis_completeness.status`, `is_complete`, `ledger_exceptions`, `scope_exclusions`, `analyzer_statuses`, and `limitations`, as well as `execution_successful`. Exceptions include reason, message, path, fatality, available source lines, and applicable limit metrics. An exception for a rejected model answer also carries a `diagnostic` built only from fixed codes: the check that rejected the answer (for example `not_json_object` or `invalid_field`), whether it arrived as a tool call or text, the provider's completion label, where prose or code fences surrounded a JSON object, schema field paths with their validation error types, and the check that rejected each bounded attempt, such as `check=not_json_object; channel=text; finish=stop; shape=prefix+fence; attempts=4; checks=not_json_object,not_json_object,not_json_object,not_json_object`. It never contains model output, field values, or keys the schema does not declare, which are reported as `*`. Terminal and Markdown reports show these projections; SARIF uses invocation completeness and notifications. The report raises an otherwise `SAFE` recommendation to `CAUTION` for incomplete analysis while retaining the actual score and severity.
 
 A subtlety is that component coverage can still read **100%** for AE6: analyzer work completed on each file, while a separate system event records unresolved interpretation. Consumers must use `is_complete` and the ledger, not the coverage percentage alone. No new telemetry platform is needed to explain these decisions; the existing public fields distinguish the causes.
 

src/skillspector/inspection_ledger.py

@@ -6,6 +6,7 @@
 from __future__ import annotations
 
 import logging
+import re
 from collections.abc import Callable, Iterable, Mapping
 from enum import StrEnum
 from hashlib import sha256
@@ -18,6 +19,12 @@
 MAX_INSPECTION_LEDGER_EVENTS: Final = 10_000
 MAX_FINDING_OUTPUT_RECORDS: Final = 10_000
 MAX_EFFECTIVE_FINDINGS: Final = MAX_FINDING_OUTPUT_RECORDS
+# Fixed codes, counts and schema field paths only (``check=invalid_field;
+# schema=findings.0.severity:literal_error``); anything else is dropped.
+MAX_DIAGNOSTIC_CHARACTERS: Final = 512
+_DIAGNOSTIC: Final = re.compile(
+    r"[a-z0-9_]+=[A-Za-z0-9_.:*$+,-]+(?:; [a-z0-9_]+=[A-Za-z0-9_.:*$+,-]+)*"
+)
 
 
 class LedgerOutcome(StrEnum):
@@ -314,6 +321,7 @@ class InspectionLedgerEvent(TypedDict):
     reason_code: NotRequired[LedgerReason]
     message: NotRequired[str]
     error_class: NotRequired[str]
+    diagnostic: NotRequired[str]
     stage: NotRequired[str]
     observed_characters: NotRequired[int]
     limit_characters: NotRequired[int]
@@ -355,6 +363,7 @@ class InspectionLedgerException(TypedDict):
     start_line: int | None
     end_line: int | None
     error_class: NotRequired[str]
+    diagnostic: NotRequired[str]
     analyzers: NotRequired[list[str]]
     fatal: NotRequired[bool]
     scope: NotRequired[str]
@@ -438,6 +447,19 @@ def _is_meta_phase(phase: str) -> bool:
     return phase == "meta"
 
 
+def _safe_diagnostic(diagnostic: object) -> str | None:
+    """Keep a bounded diagnostic of fixed codes; drop anything that could carry content."""
+    if (
+        isinstance(diagnostic, str)
+        and len(diagnostic) <= MAX_DIAGNOSTIC_CHARACTERS
+        and _DIAGNOSTIC.fullmatch(diagnostic)
+    ):
+        return diagnostic
+    if diagnostic is not None:
+        logger.debug("Dropping a ledger diagnostic outside the fixed-code format")
+    return None
+
+
 def ledger_event(
     *,
     outcome: LedgerOutcome,
@@ -451,6 +473,7 @@ def ledger_event(
     input_finding_ids: Iterable[str] = (),
     emitted_finding_ids: Iterable[str] = (),
     error_class: str | None = None,
+    diagnostic: str | None = None,
     stage: str | None = None,
     observed_characters: int | None = None,
     limit_characters: int | None = None,
@@ -518,6 +541,9 @@ def ledger_event(
         event["message"] = REASON_MESSAGES[reason]
     if error_class is not None:
         event["error_class"] = error_class
+    safe_diagnostic = _safe_diagnostic(diagnostic)
+    if safe_diagnostic is not None:
+        event["diagnostic"] = safe_diagnostic
     if stage is not None:
         event["stage"] = stage
     if observed_characters is not None:
@@ -650,6 +676,7 @@ def _exception(
     start_line: int | None = None,
     end_line: int | None = None,
     error_class: str | None = None,
+    diagnostic: str | None = None,
     analyzers: Iterable[str] = (),
     fatal: bool,
     scope: str | None = None,
@@ -670,6 +697,9 @@ def _exception(
         exception["analyzers"] = analyzer_ids
     if error_class:
         exception["error_class"] = error_class
+    safe_diagnostic = _safe_diagnostic(diagnostic)
+    if safe_diagnostic is not None:
+        exception["diagnostic"] = safe_diagnostic
     if scope:
         exception["scope"] = scope
     return exception
@@ -698,6 +728,7 @@ def _exception_from_event(
         start_line=event.get("start_line"),
         end_line=event.get("end_line"),
         error_class=event.get("error_class"),
+        diagnostic=event.get("diagnostic"),
         analyzers=[str(event.get("analyzer_id", ""))],
         fatal=fatal,
         scope=RULE_SET_SCOPE if _is_rule_set_record(event) else None,
@@ -719,6 +750,7 @@ def _merge_exception_projection(
             exception["start_line"],
             exception["end_line"],
             exception.get("error_class"),
+            exception.get("diagnostic"),
             # A rule-set row and a real file's row can share a path label;
             # they must never merge into one public row.
             exception.get("scope"),

src/skillspector/llm_analyzer_base.py

@@ -72,7 +72,7 @@
 from skillspector.logging_config import get_logger
 from skillspector.model_info import get_max_input_tokens
 from skillspector.models import Finding
-from skillspector.structured_response import ResponseIssue
+from skillspector.structured_response import ResponseCheck, ResponseIssue, schema_error_paths
 
 logger = get_logger(__name__)
 
@@ -123,6 +123,8 @@ class _StructuredResponseValidationError(Exception):
 
     def __init__(self, issue: ResponseIssue | None = None) -> None:
         super().__init__("model response failed validation")
+        # The check that rejected each bounded attempt for this batch, oldest first.
+        self.attempt_checks: tuple[str, ...] = ()
         self.reason = {
             ResponseIssue.REFUSED: LedgerReason.LLM_RESPONSE_REFUSED,
             ResponseIssue.TRUNCATED: LedgerReason.LLM_RESPONSE_TRUNCATED,
@@ -135,6 +137,55 @@ def __init__(self, issue: ResponseIssue | None = None) -> None:
             ResponseIssue.FORMAT_CHANGED: LedgerReason.LLM_FORMAT_CHANGED,
         }.get(issue or ResponseIssue.INVALID_JSON, LedgerReason.LLM_STRUCTURED_RESPONSE_INVALID)
 
+    @property
+    def check(self) -> str:
+        """Fixed code for the validation check that rejected this attempt."""
+        cause = self.__cause__
+        if isinstance(cause, StructuredOutputParseError):
+            return str(cause.check or cause.issue)
+        if isinstance(cause, ValidationError):
+            # A LangChain parser validated the arguments before our shared boundary.
+            missing = any(error.get("type") == "missing" for error in cause.errors())
+            return ResponseCheck.MISSING_FIELD if missing else ResponseCheck.INVALID_FIELD
+        return "unknown"
+
+    def diagnostic(self, schema: object) -> str | None:
+        """Describe the rejection with fixed codes and schema field paths only.
+
+        No response text, field values or keys the schema does not declare are
+        included, so the result is safe to retain in reports and logs.
+        """
+        try:
+            cause = self.__cause__
+            parts = [f"check={self.check}"]
+            schema_errors: tuple[str, ...] = ()
+            if isinstance(cause, StructuredOutputParseError):
+                parts.extend(
+                    f"{name}={value}"
+                    for name, value in (
+                        ("channel", cause.channel),
+                        ("finish", cause.finish),
+                        ("shape", cause.shape),
+                    )
+                    if value
+                )
+                schema_errors = cause.schema_errors
+            elif (
+                isinstance(cause, ValidationError)
+                and isinstance(schema, type)
+                and issubclass(schema, BaseModel)
+            ):
+                schema_errors = schema_error_paths(cause, schema)
+            if schema_errors:
+                parts.append("schema=" + ",".join(schema_errors))
+            if self.attempt_checks:
+                parts.append(f"attempts={len(self.attempt_checks)}")
+                parts.append("checks=" + ",".join(self.attempt_checks))
+            return "; ".join(parts)
+        except Exception:
+            # A diagnostic must never replace the validation failure it describes.
+            return None
+
 
 def _require_structured_response(response: object) -> None:
     """Reject a structured-output call that produced no result.
@@ -146,14 +197,18 @@ def _require_structured_response(response: object) -> None:
     schema object.
     """
     if response is None:
-        raise StructuredOutputParseError("model returned no structured response")
+        raise StructuredOutputParseError(
+            "model returned no structured response", check=ResponseCheck.NO_STRUCTURED_RESPONSE
+        )
     if (
         isinstance(response, BaseModel)
         and "findings" in type(response).model_fields
         and "findings" not in response.model_fields_set
     ):
         raise StructuredOutputParseError(
-            "response omits findings", issue=ResponseIssue.MISSING_FIELDS
+            "response omits findings",
+            issue=ResponseIssue.MISSING_FIELDS,
+            check=ResponseCheck.MISSING_FINDINGS,
         )
 
 
@@ -713,6 +768,8 @@ class BatchFailure:
     batch: Batch
     error_class: str
     reason: LedgerReason = LedgerReason.LLM_BATCH_FAILED
+    # Fixed validation codes for a rejected structured answer; never response text.
+    diagnostic: str | None = None
 
 
 @dataclass
@@ -813,6 +870,7 @@ def ledger_events_for_batches(
                         end_line=end_line,
                         reason=failure.reason,
                         error_class=failure.error_class,
+                        diagnostic=failure.diagnostic,
                     )
                 )
 
@@ -1173,7 +1231,9 @@ def _formatting_call(self, source: str) -> tuple[object, str]:
         schema_tokens = estimate_tokens(json.dumps(schema.model_json_schema()))
         if estimate_tokens(prompt) + schema_tokens > self._input_budget:
             raise StructuredOutputParseError(
-                "formatting input exceeds model budget", issue=ResponseIssue.LIMIT
+                "formatting input exceeds model budget",
+                issue=ResponseIssue.LIMIT,
+                check=ResponseCheck.FORMAT_INPUT_LIMIT,
             )
         llm, _ = self._model_for_call()
         return bind_structured_output(
@@ -1219,12 +1279,15 @@ def _invoke_batch_with_retries(
         structured_retries = 0
         connection_retries = 0
         format_source: str | None = None
+        attempt_checks: list[str] = []
         for attempt in range(1, LLM_BATCH_MAX_ATTEMPTS + 1):
             try:
                 if format_source is not None:
                     return self._format_batch(batch, format_source)
                 return self._invoke_batch(batch, prompt)
             except _StructuredResponseValidationError as exc:
+                attempt_checks.append(exc.check)
+                exc.attempt_checks = tuple(attempt_checks)
                 if format_source is not None:
                     # Retry transport failures, never a returned formatting result.
                     raise
@@ -1324,12 +1387,15 @@ async def _ainvoke_batch_with_retries(
         structured_retries = 0
         connection_retries = 0
         format_source: str | None = None
+        attempt_checks: list[str] = []
         for attempt in range(1, LLM_BATCH_MAX_ATTEMPTS + 1):
             try:
                 if format_source is not None:
                     return await self._aformat_batch(batch, format_source)
                 return await self._ainvoke_batch(batch, prompt)
             except _StructuredResponseValidationError as exc:
+                attempt_checks.append(exc.check)
+                exc.attempt_checks = tuple(attempt_checks)
                 if format_source is not None:
                     # Retry transport failures, never a returned formatting result.
                     raise
@@ -1424,16 +1490,19 @@ def run_batches_detailed(
                 result = self._invoke_batch_with_retries(batch, prompt, retry_outcome=retry_outcome)
                 outcome.successful.append(result)
             except _StructuredResponseValidationError as exc:
+                diagnostic = exc.diagnostic(self.response_schema)
                 logger.warning(
-                    "LLM response validation failed for %s (%s)",
+                    "LLM response validation failed for %s (%s; %s)",
                     batch.file_label,
                     exc.reason,
+                    diagnostic,
                 )
                 outcome.failures.append(
                     BatchFailure(
                         batch=batch,
                         error_class=ValidationError.__name__,
                         reason=exc.reason,
+                        diagnostic=diagnostic,
                     )
                 )
             except LLMRuntimeLimitError as exc:
@@ -1542,16 +1611,19 @@ async def _process(
         outcome = BatchExecutionResult()
         for batch, result, retry_outcome in zip(batches, results, retry_outcomes, strict=True):
             if isinstance(result, _StructuredResponseValidationError):
+                diagnostic = result.diagnostic(self.response_schema)
                 logger.warning(
-                    "LLM response validation failed for %s (%s)",
+                    "LLM response validation failed for %s (%s; %s)",
                     batch.file_label,
                     result.reason,
+                    diagnostic,
                 )
                 outcome.failures.append(
                     BatchFailure(
                         batch=batch,
                         error_class=ValidationError.__name__,
                         reason=result.reason,
+                        diagnostic=diagnostic,
                     )
                 )
                 continue