fix(supply-chain): UTF-8 BOM in requirements.txt silently drops the first dependency from SC4/OSV #962

Summary

A UTF-8 BOM (EF BB BF) at the beginning of requirements.txt causes the first dependency to be silently omitted from SkillSpector's SC4/OSV dependency analysis.

The scan can still report analysis_completeness.status=complete (or partial only because of an unrelated AE4 condition), creating a false impression that the dependency manifest was fully analyzed.

Environment

• SkillSpector version: 2.12.0

• Source commit: 54921c8c7c23008c8af2328d5ab492a904b6ece4

• Python: 3.12

• Platform: Windows

• Scan mode: --no-llm

• Test data: synthetic Skills only

Reproduction

Create a synthetic Skill whose requirements.txt begins with a UTF-8 BOM:

[EF BB BF]jinja2==2.4.1

Run the scan once with the BOM and once after removing it.

Observed results:

The difference is reproducible and is caused by the BOM on the first manifest line.

For the single-dependency jinja2 case, the observed risk score changed from 8 (BOM) to 20 (without BOM).

Root Cause

The issue appears to be in the requirements parsing path:

1. skillspector/artifacts.py — decode_text() decodes using plain UTF-8 and does not strip a UTF-8 BOM. The resulting string begins with \ufeff.

2. skillspector/nodes/build_context.py — BOM handling exists for SKILL.md frontmatter delimiter detection, but this handling does not normalize the decoded text globally.

3. _extract_packages_from_requirements_detailed() passes the first logical requirement line through _logical_requirement_lines() and _extract_python_requirement().

4. The first requirement is passed to packaging.requirements.Requirement() with the leading \ufeff, resulting in an InvalidRequirement exception.

5. The invalid first line is silently skipped instead of being reported as an analysis limitation.

This means the first dependency can be omitted without a corresponding completeness warning.

Expected Behavior

• A UTF-8 BOM at the beginning of requirements.txt should not prevent the first dependency from being analyzed.

• If a non-empty requirement line cannot be parsed, SkillSpector should report the parsing failure as an analysis limitation rather than silently skipping it.

• The analysis completeness status should not imply that all dependencies were successfully analyzed when a manifest entry was silently dropped.

Suggested Fix

Possible approaches:

• Normalize the BOM specifically when reading dependency manifests, for example by using utf-8-sig for requirements.txt.

• Alternatively, strip a leading \ufeff from the decoded manifest content before parsing.

• Ensure that non-empty requirement lines that fail parsing are surfaced as explicit analysis limitations.

The fix should preserve existing behavior for ordinary UTF-8 files without a BOM.

Related Issues

• fix: strip leading BOM before SKILL.md frontmatter detection #471 — BOM handling for SKILL.md frontmatter (a different parsing path).

• Static-pattern false positives found scanning public skills (P2 BOM, RA2 plist, EA3 OFL, plus design-level items) #644 / fix(analyzers): drop P2 BOM, RA2 plist-substring and EA3 OFL false positives #645 — BOM-related static-pattern false positives (different behavior from this dependency omission).

• Pipenv dependencies are skipped by SC4 despite complete scan reporting #764 — Pipenv dependency parsing silently skipping entries.

• package.json skips optionalDependencies and checks overridden dependencies at the wrong version #770 — package.json optional dependency handling.

These issues appear related to BOM or dependency-analysis completeness, but they do not appear to address this exact requirements.txt first-line omission.

Security Impact

A malicious or vulnerable dependency placed on the first line of a BOM-encoded requirements.txt may not be included in SC4/OSV analysis.

This can produce a false negative in a supply-chain security scan, particularly when users rely on the scan as a pre-installation review gate.

Evidence

The leading bytes of the affected file are:

EF BB BF

Removing only those three bytes makes the first dependency detectable in the same test environment.

All tests were performed using synthetic Skill samples; no third-party Skill content or production environment was involved.