diff --git a/CHANGELOG.md b/CHANGELOG.md index 132d661..9261995 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,8 +21,9 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). (exit 11) before the first zip uploads, instead of after. A path that does not exist now exits 10 (input not found) rather than 11; both are in the no-fallback range. - **Presigned symbol uploads no longer hold the whole archive in memory.** The PUT body and the - SHA-1 are streamed from disk in 64 KiB chunks (the PUT keeps an exact `Content-Length` and - re-opens the file per retry), and the ELF identity scan memory-maps libraries instead of reading + SHA-1 are streamed from disk in 64 KiB chunks (the PUT keeps an exact `Content-Length`, + re-opens the file per retry and follows 307/308 redirects itself, which reqwest cannot do for a + streamed body), and the ELF identity scan memory-maps libraries instead of reading them. Peak RSS for four 100 MB libraries dropped from 564 MB to 36 MB. Wire format unchanged. The one new constraint: a directory passed to `--type elf` must not be written to while it is scanned (a library truncated under its mapping is undefined behaviour, in practice SIGBUS), so @@ -43,6 +44,28 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). original key order (it was alphabetical), and `inject` writes `debug_id` / `debugId` last. Both are still valid maps; an invalid-UTF-8 map outside a string value now reports as invalid JSON (exit 11) rather than I/O (exit 10). +- **`sourcemaps inject` edits bundles in place.** The debug-id stub is appended (or, to re-key, our + own trailing stub is swapped) without copying the bundle: a 150 MB bundle's dirty memory went + from 317 MB to 2.4 MB. Output bytes are identical, and permissions and symlinks survive as before. + A map that cannot take the id (not a JSON object, invalid, read-only) is now refused BEFORE the + bundle is touched, so a failed run no longer leaves a stamped bundle beside an un-keyed map. + +### Fixed +- **`dsym uuid` and `debug-files upload --type dsym` no longer report a Mach-O with no `LC_UUID` + as the nil UUID** (`00000000-0000-0000-0000-000000000000`). Such a slice can never match a crash + report and every one would collide on the same all-zero key, so it is skipped; a bundle with no + usable slice is rejected (and fails an `xcode upload-dsyms` build phase, as any unreadable + bundle does). The app-binary path already treated nil as absent. +- **`--type elf` only treats real ELF files as ELF.** `symbolic` identifies Mach-O, PE and PDB just + as readily, so a macOS-built `.so` found by a directory scan would have been uploaded as an "elf" + symbol keyed by its Mach-O UUID. A file without the `\x7fELF` magic now has no build-id and is + skipped with the usual warning. +- **`build-env read-plist` (and the `xcode` flows that read `Info.plist`) no longer crash on a + deeply nested plist.** Tens of thousands of nested arrays overflowed the stack and aborted the + process with SIGABRT; it now reads as `{}` like any other unusable plist. Plists over 1 MiB are + refused up front. +- **`ios-deps` no longer reports a nameless dependency** (`library::`) for a truncated or mangled + `Podfile.lock` line such as `- (2.0)`. ## [0.7.13] - 2026-10-07 diff --git a/CLAUDE.md b/CLAUDE.md index aa49a21..49a4fb9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -114,6 +114,13 @@ rule, or entry name requires a major version bump and a coordinated rollout. ZIP uses method 93 (Z_STANDARD). `--no-zstd` is diagnostic only. - Every upload request carries the `X-Bugsee-Uploader: cli` telemetry header (set in `upload::http`). +- Bounded memory: inputs are artefacts and symbol files of hundreds of MB, so never load a + whole file onto the heap (`fs::read`, `read_to_string`, `tokio::fs::read`, or a `.clone()` of a + body per retry). Hash and copy through a 64 KiB buffer; send request bodies with + `upload::http::file_body` + an explicit `Content-Length` (a presigned S3 PUT rejects chunked); + read identity out of headers with `symbols::mapped::map_file` (the one place a file is memory-mapped, so that `unsafe` lives only there); walk + JSON with `inject::mapjson` (struson), not a `serde_json::Value` tree. `scripts/e2e_flows.py` + enforces a peak-RSS budget on ~96 MB inputs for every large-input flow, so a regression fails CI. - Return a typed `error::Error` (or `anyhow` wrapping one) so `main`'s `classify` maps it to the right exit code. A bare `anyhow::anyhow!` falls through to exit 1 (structural) — use a typed variant whenever the code matters to a caller. @@ -125,6 +132,15 @@ rule, or entry name requires a major version bump and a coordinated rollout. response contract, not just "something happened". - `tests/` holds integration tests that exec the COMPILED binary (`assert_cmd`) and pin exit codes + stdout JSON. +- `tests/file_handling.rs` pins file handling through the compiled binary: exact bytes left on + disk, large payloads on the wire, stdout kept empty, exit codes for unusable input. The streaming + JSON code is also tested differentially against the `serde_json::Value` behaviour it replaced. +- `tests/broken_files.rs` pins what the binary does with BROKEN input (corrupt JSON / ELF / + Mach-O / PDB / zip / plist / manifests): never a crash, a stable exit code, nothing sent for + input that could not be identified, the user's files untouched. A new file-reading code path + needs a truncation and a garbage case there (or a unit-level sweep next to the parser, as + `symbols/elf.rs` and `symbols/dsym.rs` do: a damaged file may fail to identify but must never + identify as something else, because the identity is the upload key). - `scripts/e2e_flows.py` is a stdlib-only end-to-end harness that drives a real binary through every upload flow against a protocol-accurate mock server. CI runs it from-source on every push (`.github/workflows/e2e.yml`) and against the diff --git a/scripts/e2e_flows.py b/scripts/e2e_flows.py index 4cd6e83..7534bc6 100644 --- a/scripts/e2e_flows.py +++ b/scripts/e2e_flows.py @@ -324,6 +324,132 @@ def run(binpath, flow, args, expect_code=0, expect_stderr=None): return ok +# --- Memory regression ------------------------------------------------------------------- +# +# The CLI handles artefacts and symbol files of hundreds of MB and must do it in +# bounded memory: stream, or map and look at headers, never hold a whole file (or +# two) on the heap. A regression to `fs::read` is invisible to every functional +# check, so run each large-input flow in a fresh process and bound its PEAK RSS. +# +# Inputs are ~96 MB; every flow must stay under MEM_LIMIT_MB. The budget is far above +# the streaming cost (~30-40 MB: runtime + zstd encoders) and far below any whole-file +# read (>= the input, doubled by a per-retry clone), so it is not flaky either way. +MEM_INPUT_MB = 96 +MEM_LIMIT_MB = 64 + +_RSS_WRAPPER = ( + "import resource, subprocess, sys\n" + "r = subprocess.run(sys.argv[1:], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n" + "print(r.returncode, resource.getrusage(resource.RUSAGE_CHILDREN).ru_maxrss)\n" +) + + +def peak_rss_mb(argv, env): + """(exit code, peak RSS in MB) of `argv`, measured in a throwaway wrapper process so + RUSAGE_CHILDREN's maximum belongs to this command alone. None where RSS is unavailable.""" + if sys.platform == "win32": + return None + r = subprocess.run([sys.executable, "-c", _RSS_WRAPPER] + argv, capture_output=True, + text=True, env=env) + try: + rc, raw = r.stdout.split() + raw = int(raw) + except ValueError: + return None + # ru_maxrss is bytes on macOS, kilobytes on Linux. + return int(rc), raw / (1024 * 1024 if sys.platform == "darwin" else 1024) + + +def _write_big(path, total_mb, block): + with open(path, "wb") as f: + for _ in range(total_mb): + f.write(block()) + + +def memory_checks(binpath, fix, results): + if sys.platform == "win32": + print(" [SKIP] memory checks (no RSS accounting on Windows)") + return + mem = os.path.join(fix, "memory") + os.makedirs(mem, exist_ok=True) + env = dict(os.environ, BUGSEE_APP_TOKEN=TOKEN) + endpoint = f"http://127.0.0.1:{STATE['port']}" + base = [binpath, "--endpoint", endpoint, "--app-token", TOKEN] + mb = 1024 * 1024 + + # Incompressible artefact (stored in the upload zip, so the PUT is ~its size). + artefact = os.path.join(mem, "app.aab") + _write_big(artefact, MEM_INPUT_MB, lambda: os.urandom(mb)) + payload = os.path.join(mem, "p.json") + with open(payload, "w") as f: + json.dump({"uuid": "mem-build", "version": "1.0"}, f) + # A text mapping (compressible, like the real thing). + pg = os.path.join(mem, "pg") + os.makedirs(pg, exist_ok=True) + line = b"com.example.pkg.ClassNameNumber -> a.b:\n int fieldName -> a\n" + _write_big(os.path.join(pg, "mapping.txt"), MEM_INPUT_MB, lambda: line * (mb // len(line))) + # Native libraries: the real ELF fixture (it has a GNU build-id) padded to ~32 MB each, + # three ABIs with distinct build-ids, in AGP's merged_native_libs layout. + fixture = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "tests", + "fixtures", "elf", "libsymbol1.so") + elf = open(fixture, "rb").read() + bid = bytes.fromhex("bca64abfec40dbb631bb8f1c37414472") + pos = elf.index(bid) + for i, abi in enumerate(["arm64-v8a", "armeabi-v7a", "x86_64"]): + d = os.path.join(mem, "libs", "lib", abi) + os.makedirs(d, exist_ok=True) + lib = bytearray(elf) + lib[pos] ^= i + 1 + with open(os.path.join(d, "libbig.so"), "wb") as f: + f.write(bytes(lib)) + for _ in range(MEM_INPUT_MB // 3): + f.write(os.urandom(mb)) + # A ~64 MB source map: a small `mappings` and a large `sourcesContent`. + maps = os.path.join(mem, "maps") + os.makedirs(maps, exist_ok=True) + with open(os.path.join(maps, "app.js.map"), "w") as f: + f.write('{"version":3,"debug_id":"44444444-4444-4444-4444-444444444444",' + '"sources":["a.ts"],"names":[],"mappings":"AAAA;","sourcesContent":["') + chunk = "x" * mb + for _ in range(MEM_INPUT_MB * 2 // 3): + f.write(chunk) + f.write('"]}') + # `sourcemaps inject` over the same map (the bundle stays small). + inj = os.path.join(mem, "inject") + os.makedirs(inj, exist_ok=True) + shutil.copy(os.path.join(maps, "app.js.map"), os.path.join(inj, "app.js.map")) + with open(os.path.join(inj, "app.js"), "w") as f: + f.write("console.log(1);\n") + + flows = [ + ("mem_artefact_put", base + ["upload", "build", "--payload-json", payload, + "--artifact", artefact]), + ("mem_proguard", base + ["debug-files", "upload", "--type", "proguard", + "--version", "1", "--build", "1", pg]), + ("mem_elf_directory", base + ["debug-files", "upload", "--type", "elf", + "--version", "1", "--build", "1", + "--uuid", "11111111-2222-3333-4444-555555555555", + os.path.join(mem, "libs")]), + ("mem_sourcemap", base + ["debug-files", "upload", "--type", "sourcemaps", + "--version", "1", "--build", "1", maps]), + ("mem_sourcemap_strip", base + ["debug-files", "upload", "--type", "sourcemaps", + "--strip-sources-content", "--version", "1", + "--build", "1", maps]), + ("mem_inject", [binpath, "sourcemaps", "inject", inj]), + ] + for flow, argv in flows: + STATE["flow"] = flow + got = peak_rss_mb(argv, env) + if got is None: + print(f" [SKIP] {flow} (no RSS available)") + continue + rc, rss = got + ok = rc == 0 and rss < MEM_LIMIT_MB + print(f" [{'PASS' if ok else 'FAIL'}] {flow}: exit {rc}, peak RSS {rss:.0f} MB " + f"(budget {MEM_LIMIT_MB} MB, input ~{MEM_INPUT_MB} MB)") + results[flow + "_bounded_memory"] = ok + + def main(): ap = argparse.ArgumentParser() ap.add_argument("--bin", required=True, help="path to bugsee-cli[.exe]") @@ -597,6 +723,9 @@ def main(): print(" [FAIL] could not verify the register-only build:", e) results["build_register_only_ships_no_bytes"] = False + print("\n=== MEMORY ===") + memory_checks(binpath, fix, results) + srv.shutdown() if not a.keep: shutil.rmtree(work, ignore_errors=True) diff --git a/src/cli/build_env.rs b/src/cli/build_env.rs index 5f108ee..62469b7 100644 --- a/src/cli/build_env.rs +++ b/src/cli/build_env.rs @@ -321,6 +321,53 @@ fn env_truthy(value: Option<&String>) -> bool { /// failure, root that isn't a Dict). The Python callers treat /// absence as "field unknown" rather than failing. pub fn read_plist_to_json(path: &std::path::Path) -> serde_json::Map { + // `plist::Value` is built (and later dropped) recursively, one stack frame per + // nesting level. A malformed or hostile plist a few tens of thousands of levels + // deep overflowed the 8 MiB main-thread stack and ABORTED the process (SIGABRT, + // no error, no `{}`). Two bounds make that impossible, whatever the file holds: + // - an Info.plist is a few KB; refuse anything over `MAX_PLIST_BYTES`, which + // caps the nesting depth (a level costs at least ~6 bytes in either format); + // - parse on a thread whose stack is far larger than that depth can use. + // As a side effect a panic inside the parser is contained and reads as `{}` too. + const MAX_PLIST_BYTES: u64 = 1024 * 1024; + const PLIST_PARSE_STACK_BYTES: usize = 256 * 1024 * 1024; + + match std::fs::metadata(path) { + Ok(m) if m.len() <= MAX_PLIST_BYTES => {} + _ => return serde_json::Map::new(), + } + let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0); + let owned = path.to_path_buf(); + match std::thread::Builder::new() + .name("plist-parse".into()) + .stack_size(PLIST_PARSE_STACK_BYTES) + .spawn(move || parse_plist_to_json(&owned)) + { + Ok(handle) => handle.join().unwrap_or_else(|_| { + tracing::warn!(path = %path.display(), "plist parser panicked; treating the plist as empty"); + serde_json::Map::new() + }), + // A 256 MiB stack is reserved up front, which a tight `ulimit -v`, cgroup or + // `vm.overcommit_memory=2` can refuse. A small plist is parsed right here instead + // (at most ~1k nesting levels: far inside even a 2 MiB worker-thread stack); a + // large one is not, because only the big stack makes its depth safe. + Err(e) => { + const INLINE_PLIST_BYTES: u64 = 16 * 1024; + if size <= INLINE_PLIST_BYTES { + tracing::warn!(error = %e, "cannot spawn the plist parser thread; parsing inline"); + parse_plist_to_json(path) + } else { + tracing::warn!( + error = %e, path = %path.display(), + "cannot spawn the plist parser thread; a {size}-byte plist is too large to parse inline" + ); + serde_json::Map::new() + } + } + } +} + +fn parse_plist_to_json(path: &std::path::Path) -> serde_json::Map { let mut out = serde_json::Map::new(); let value = match plist::Value::from_file(path) { Ok(v) => v, @@ -597,6 +644,136 @@ mod tests { } } + // ── read_plist_to_json on broken / hostile files ────────────── + + /// A binary plist whose objects form one chain of `n` nested arrays (4-byte refs). + fn nested_bplist(n: u32, width: usize) -> Vec { + // `width` (3 or 4) is both the object-ref size and the offset-table entry size. + // 3 bytes is the densest encoding that still addresses >65 535 objects: ~7 bytes + // per nesting level, i.e. the deepest chain that fits under the size cap. + let be = |v: u64| v.to_be_bytes()[8 - width..].to_vec(); + let mut body = b"bplist00".to_vec(); + let mut offsets = Vec::new(); + for i in 0..n { + offsets.push(body.len() as u64); + if i + 1 < n { + body.push(0xa1); // array of 1 + body.extend_from_slice(&be(i as u64 + 1)); + } else { + body.push(0x50); // empty string + } + } + let table = body.len() as u64; + for o in offsets { + body.extend_from_slice(&be(o)); + } + body.extend_from_slice(&[0; 6]); + body.extend_from_slice(&[width as u8, width as u8]); // offset size, object-ref size + body.extend_from_slice(&(n as u64).to_be_bytes()); + body.extend_from_slice(&0u64.to_be_bytes()); // top object + body.extend_from_slice(&table.to_be_bytes()); + body + } + + fn plist_map(bytes: &[u8]) -> serde_json::Map { + let dir = tempfile::tempdir().unwrap(); + let p = dir.path().join("Info.plist"); + std::fs::write(&p, bytes).unwrap(); + read_plist_to_json(&p) + } + + /// Nesting deep enough to overflow the main-thread stack used to ABORT the whole + /// process. It must read as an ordinary unusable plist (`{}`), in both formats and + /// right up to the size cap — debug-build frames are the largest, so this is the + /// worst case the thread's stack has to absorb. + #[test] + fn a_plist_nested_past_any_stack_reads_as_empty_instead_of_crashing() { + let xml = |n: usize| { + [ + &b""[..], + &b"".repeat(n), + &b"".repeat(n), + b"", + ] + .concat() + }; + // Largest XML nesting that still fits under the 1 MiB size cap. + assert!(xml(69_000).len() < 1024 * 1024); + assert!(plist_map(&xml(69_000)).is_empty()); + assert!(plist_map(&xml(200_000)).is_empty(), "over the cap"); + // Binary: ~9 bytes per level, so ~110k levels fit under the cap. + assert!(nested_bplist(110_000, 4).len() < 1024 * 1024); + assert!(plist_map(&nested_bplist(110_000, 4)).is_empty()); + // The true worst case under the cap: 3-byte refs, ~149k levels. + let densest = nested_bplist(149_000, 3); + assert!(densest.len() < 1024 * 1024, "{}", densest.len()); + assert!(plist_map(&densest).is_empty()); + assert!( + plist_map(&nested_bplist(300_000, 4)).is_empty(), + "over the cap" + ); + } + + #[test] + fn broken_plists_read_as_empty_and_valid_neighbours_still_read() { + let valid: &[u8] = br#"CFBundleNameAppN7"#; + let m = plist_map(valid); + assert_eq!(m["CFBundleName"], "App"); + assert_eq!(m["N"], "7"); + + let mut broken: Vec<(String, Vec)> = vec![ + ("empty".into(), Vec::new()), + ( + "garbage".into(), + (0..300u32).map(|i| (i * 37 % 251) as u8).collect(), + ), + ("bplist magic only".into(), b"bplist00".to_vec()), + ("bplist truncated".into(), b"bplist00\xd1\x01\x02".to_vec()), + ( + "array root".into(), + br#""#.to_vec(), + ), + ( + "no root".into(), + br#""#.to_vec(), + ), + ( + "not utf-8".into(), + b"\xff\xfex" + .to_vec(), + ), + ]; + // Every strict prefix of a valid XML plist is invalid: empty, never a panic + // and never a half-read dictionary. + for n in [0, 1, 20, 60, 100, valid.len() - 10, valid.len() - 1] { + broken.push((format!("xml truncated at {n}"), valid[..n].to_vec())); + } + for (label, bytes) in &broken { + let m = plist_map(bytes); + assert!(m.is_empty(), "{label}: {m:?}"); + } + } + + /// Entity declarations are not expanded (no billion-laughs blow-up, no external + /// fetch): a reference just reads as empty. + #[test] + fn xml_entities_are_not_expanded() { + let bomb = br#"]>k&c;"#; + assert_eq!(plist_map(bomb)["k"], ""); + let xxe = br#"]>k&x;"#; + assert_eq!(plist_map(xxe)["k"], ""); + } + + /// Something far larger than any Info.plist is refused up front, not parsed. + #[test] + fn an_oversized_plist_is_refused() { + let mut big = + br#"a"#.to_vec(); + big.extend(std::iter::repeat_n(b'x', 1024 * 1024)); + big.extend_from_slice(b""); + assert!(plist_map(&big).is_empty()); + } + // ── env_truthy — cross-language contract ───────────────────── #[test] diff --git a/src/cli/dsym.rs b/src/cli/dsym.rs index dfbc9bf..f7cf9ee 100644 --- a/src/cli/dsym.rs +++ b/src/cli/dsym.rs @@ -151,6 +151,10 @@ fn push_slices_from_macho(path: &Path, out: &mut Vec) { Err(_) => return, }; for obj in archive.objects().flatten() { + // No LC_UUID reads as the nil UUID: not an identity, so not reported. + if obj.debug_id().is_nil() { + continue; + } out.push(DsymSliceView { uuid: format_uuid(obj.debug_id()), arch: obj.arch().name().to_string(), @@ -180,6 +184,52 @@ mod tests { use super::*; use tempfile::TempDir; + /// A Mach-O without an `LC_UUID` reads as the nil UUID: not an identity, so it + /// is not reported (an integrator would otherwise key a build on all zeros). + #[test] + fn extract_uuids_omits_nil_uuid_slices() { + let tmp = TempDir::new().unwrap(); + let good: [u8; 16] = [7; 16]; + let fat = crate::symbols::test_macho::fat_macho(&[ + (0x0100_000c, 0, good), + (0x0100_0007, 3, [0; 16]), + ]); + let p = tmp.path().join("fat"); + std::fs::write(&p, &fat).unwrap(); + let out = extract_uuids(&p); + assert_eq!( + out, + [format_uuid(DebugId::from_uuid(uuid::Uuid::from_bytes( + good + )))] + ); + + let only_nil = tmp.path().join("nil"); + std::fs::write( + &only_nil, + crate::symbols::test_macho::thin_macho(0x0100_000c, 0, [0; 16]), + ) + .unwrap(); + assert!(extract_uuids(&only_nil).is_empty()); + } + + /// Truncated Mach-O files never crash the extractor and never report a made-up UUID. + #[test] + fn extract_uuids_on_truncated_machos_is_empty_or_the_true_uuid() { + let tmp = TempDir::new().unwrap(); + let valid = crate::symbols::test_macho::thin_macho(0x0100_000c, 0, [9; 16]); + let true_uuid = format_uuid(DebugId::from_uuid(uuid::Uuid::from_bytes([9; 16]))); + let p = tmp.path().join("m"); + for len in 0..valid.len() { + std::fs::write(&p, &valid[..len]).unwrap(); + let out = extract_uuids(&p); + assert!( + out.is_empty() || out == [true_uuid.clone()], + "len {len}: {out:?}" + ); + } + } + /// A DWARF slice over 1 GiB (large games) is identified like any other: there is no /// size cap, since only the headers of the mapping are read. The file is sparse, so /// this costs no disk and the parse touches a few pages. diff --git a/src/cli/ios_deps.rs b/src/cli/ios_deps.rs index 1deb240..27b8960 100644 --- a/src/cli/ios_deps.rs +++ b/src/cli/ios_deps.rs @@ -322,6 +322,13 @@ pub fn parse_podfile_lock(path: &Path) -> Vec { // Pod line: ` - Name (Version):` (2-space indent). if let Some(pod_body) = parse_pod_line(line) { let (name, version) = parse_name_version(pod_body); + if name.is_empty() { + // A truncated or mangled line (`- (2.0)`) names nothing; a nameless + // entry would put a bogus `library::` dependency in the graph. Its + // child lines have no owner either, so they are dropped with it. + current_pod = None; + continue; + } pods.insert( name.clone(), PodInfo { diff --git a/src/cli/xcactivitylog.rs b/src/cli/xcactivitylog.rs index 9964801..78d898d 100644 --- a/src/cli/xcactivitylog.rs +++ b/src/cli/xcactivitylog.rs @@ -1775,6 +1775,142 @@ mod tests { assert!(t.summary.is_none() && t.timeline.is_none()); } + // ── corrupted logs ───────────────────────────────────────────── + + /// A synthetic log with several sections, a blob and unicode titles. + fn sample_log() -> Vec { + build_log( + &[ + ("Ld App normal arm64", T0 + 1.0, T0 + 3.0), + ("CompileSwift normal arm64 héllo.swift", T0 + 0.5, T0 + 2.0), + ], + &[("Build App", T0, T0 + 5.0)], + &[], + true, + ) + } + + /// Cutting the stream anywhere is survivable: `None` or a parse of what is left, + /// never a panic and never a hang on a half-read length. + #[test] + fn every_truncation_of_a_log_is_survivable() { + let log = sample_log(); + assert!( + parse_stream(&log).is_some(), + "control: the whole log parses" + ); + for cut in 0..log.len() { + let r = std::panic::catch_unwind(|| parse_stream(&log[..cut])); + assert!(r.is_ok(), "parse_stream PANICKED on a {cut}-byte prefix"); + } + } + + /// Flip, replace and insert bytes all over the stream, concentrating on the + /// length-prefixed fields: a corrupted length (`99999999999`) must not turn into a + /// huge allocation or an out-of-bounds read. + #[test] + fn corrupted_bytes_never_panic_or_over_allocate() { + let log = sample_log(); + let mut state = 0xA076_1D64_78BD_642Fu64; + let mut next = || { + state ^= state << 13; + state ^= state >> 7; + state ^= state << 17; + state + }; + let started = std::time::Instant::now(); + for round in 0..2000 { + let mut b = log.clone(); + for _ in 0..1 + next() % 4 { + let at = (next() % b.len() as u64) as usize; + match next() % 4 { + 0 => b[at] = next() as u8, + 1 => b[at] = b"0123456789^#%\"@"[(next() % 14) as usize], + 2 => { + b.remove(at); + } + _ => b.insert(at, next() as u8), + } + } + let r = std::panic::catch_unwind(|| parse_stream(&b)); + assert!( + r.is_ok(), + "parse_stream PANICKED on corruption round {round}" + ); + } + assert!( + started.elapsed() < std::time::Duration::from_secs(60), + "corrupted input must not make the tokenizer crawl" + ); + } + + #[test] + fn a_length_prefix_claiming_the_moon_is_rejected_cheaply() { + // `SLF0` header, then a string token claiming 2^60 bytes: must come back at once. + let mut data = b"SLF0".to_vec(); + data.extend_from_slice(b"99999999999999999999\""); + data.extend_from_slice(b"1152921504606846976\"x"); + let r = std::panic::catch_unwind(|| parse_stream(&data)); + assert!(r.is_ok()); + } + + /// Gzip-level damage: truncated stream, flipped payload byte, trailing junk, + /// empty file. All read as "no timings" (`None`), none panics. + #[test] + fn damaged_gzip_logs_parse_as_none() { + let td = tempfile::tempdir().unwrap(); + let path = write_gzipped_log(td.path(), &sample_log()); + let raw = std::fs::read(&path).unwrap(); + assert!( + parse_xcactivitylog(&path).is_some(), + "control: the intact log parses" + ); + let mut flipped = raw.clone(); + let mid = flipped.len() / 2; + flipped[mid] ^= 0xff; + let cases: Vec<(&str, Vec)> = vec![ + ("empty file", Vec::new()), + ("gzip magic only", vec![0x1f, 0x8b]), + ("truncated in half", raw[..raw.len() / 2].to_vec()), + ("trailer chopped", raw[..raw.len() - 6].to_vec()), + ("flipped payload byte", flipped), + ("not gzip", b"SLF0 plain, never compressed".to_vec()), + ]; + for (label, bytes) in cases { + let p = td.path().join("broken.xcactivitylog"); + std::fs::write(&p, bytes).unwrap(); + let r = std::panic::catch_unwind(|| parse_xcactivitylog(&p)); + assert!(r.is_ok(), "{label}: PANICKED"); + assert!(r.unwrap().is_none(), "{label}: must read as no timings"); + } + } + + /// A log that inflates far past the cap (a decompression bomb) is cut at the cap, + /// not inflated into memory. + #[test] + fn a_decompression_bomb_is_capped() { + use flate2::write::GzEncoder; + use flate2::Compression; + use std::io::Write; + let td = tempfile::tempdir().unwrap(); + let p = td.path().join("bomb.xcactivitylog"); + let mut enc = GzEncoder::new(std::fs::File::create(&p).unwrap(), Compression::best()); + enc.write_all(b"SLF0").unwrap(); + let zeros = vec![0u8; 1024 * 1024]; + for _ in 0..(super::MAX_DECOMPRESSED / (1024 * 1024) + 30) { + enc.write_all(&zeros).unwrap(); // ~50 MB of zeros, a few KB compressed + } + enc.finish().unwrap(); + assert!( + std::fs::metadata(&p).unwrap().len() < 200_000, + "the bomb really is tiny" + ); + // Returns promptly with no timings; the proof of the cap is that this does not + // hold ~50 MB (it reads at most MAX_DECOMPRESSED + 1 bytes). + let r = std::panic::catch_unwind(|| parse_xcactivitylog(&p)); + assert!(r.is_ok() && r.unwrap().is_none()); + } + #[test] fn resolve_swallows_bad_gzip() { // A `.xcactivitylog` that is not valid gzip → parse returns None → diff --git a/src/cli/xcode_ipa.rs b/src/cli/xcode_ipa.rs index 435a5a5..c2c88a0 100644 --- a/src/cli/xcode_ipa.rs +++ b/src/cli/xcode_ipa.rs @@ -273,6 +273,89 @@ mod tests { app } + /// The app's identity comes from two files that can each be damaged: a broken + /// Info.plist or executable means "no UUID" (the caller falls back to a random one + /// so the build record still lands), never a panic and never a made-up UUID. + #[test] + fn main_executable_uuid_is_none_for_every_broken_app() { + use crate::symbols::test_macho::{thin_macho, CPU_SUBTYPE_ARM64_ALL, CPU_TYPE_ARM64}; + let plist = |exe: &str| { + format!( + r#"CFBundleExecutable{exe}"# + ) + }; + let valid = thin_macho(CPU_TYPE_ARM64, CPU_SUBTYPE_ARM64_ALL, [7; 16]); + let nil = thin_macho(CPU_TYPE_ARM64, CPU_SUBTYPE_ARM64_ALL, [0; 16]); + + let build = |plist_bytes: &[u8], exe: Option<&[u8]>| { + let td = tempfile::tempdir().unwrap(); + let app = td.path().join("A.app"); + std::fs::create_dir_all(&app).unwrap(); + std::fs::write(app.join("Info.plist"), plist_bytes).unwrap(); + if let Some(b) = exe { + std::fs::write(app.join("MyApp"), b).unwrap(); + } + (td, app) + }; + + // Control: an intact app reports the real UUID. + let (_t, app) = build(plist("MyApp").as_bytes(), Some(&valid)); + assert_eq!(main_executable_uuid(&app).unwrap(), "07".repeat(16)); + + let mut cases: Vec<(String, Vec, Option>)> = vec![ + ("garbage plist".into(), vec![0xff; 100], Some(valid.clone())), + ("empty plist".into(), Vec::new(), Some(valid.clone())), + ( + "truncated plist".into(), + plist("MyApp").as_bytes()[..60].to_vec(), + Some(valid.clone()), + ), + ( + "plist without the key".into(), + br#""#.to_vec(), + Some(valid.clone()), + ), + ( + "executable missing".into(), + plist("MyApp").into_bytes(), + None, + ), + ( + "empty executable".into(), + plist("MyApp").into_bytes(), + Some(Vec::new()), + ), + ( + "garbage executable".into(), + plist("MyApp").into_bytes(), + Some(vec![0xab; 500]), + ), + ( + "executable with no LC_UUID".into(), + plist("MyApp").into_bytes(), + Some(nil), + ), + ( + "executable named by a path escape".into(), + plist("../outside").into_bytes(), + Some(valid.clone()), + ), + ]; + for n in 0..valid.len() { + cases.push(( + format!("executable cut at {n}"), + plist("MyApp").into_bytes(), + Some(valid[..n].to_vec()), + )); + } + for (label, plist_bytes, exe) in cases { + let (_t, app) = build(&plist_bytes, exe.as_deref()); + let got = std::panic::catch_unwind(|| main_executable_uuid(&app)) + .unwrap_or_else(|_| panic!("{label}: PANICKED")); + assert_eq!(got, None, "{label}"); + } + } + #[test] fn main_executable_uuid_reads_real_macho_uuid() { // Positive symbolic path for IPA extraction: the other diff --git a/src/compress/mod.rs b/src/compress/mod.rs index 2aa5cc0..176e5a9 100644 --- a/src/compress/mod.rs +++ b/src/compress/mod.rs @@ -356,6 +356,108 @@ mod tests { } } + /// Sizes straddling the 64 KiB copy buffer (and empty): every entry must come + /// back byte-identical with a valid CRC, for each compression method. + #[test] + fn every_method_round_trips_sizes_around_the_copy_buffer() { + let dir = tempfile::tempdir().unwrap(); + for len in [0usize, 1, 65_535, 65_536, 65_537, 131_072, 300_001] { + // Patterned bytes: not a constant run (which would compress away any + // chunking bug) and not text. + let data: Vec = (0..len).map(|i| ((i * 31 + i / 251) % 256) as u8).collect(); + let src = dir.path().join("src.bin"); + std::fs::write(&src, &data).unwrap(); + for (label, entry, strategy) in [ + ( + "zstd", + ZipEntry::compressed("e.bin", &src), + Strategy::Zstd(3), + ), + ( + "deflate", + ZipEntry::compressed("e.bin", &src), + Strategy::Deflate, + ), + ("stored", ZipEntry::stored("e.bin", &src), Strategy::Zstd(3)), + ] { + let out = dir.path().join("out.zip"); + let size = pack_entries(&[entry], &out, strategy).unwrap(); + assert_eq!( + size, + std::fs::metadata(&out).unwrap().len(), + "{label} {len}" + ); + let mut archive = ZipArchive::new(File::open(&out).unwrap()).unwrap(); + let mut e = archive.by_name("e.bin").unwrap(); + let mut got = Vec::new(); + // `read_to_end` verifies the CRC-32 at EOF: corruption is an Err here. + e.read_to_end(&mut got).unwrap(); + assert_eq!(got, data, "{label} {len}"); + } + } + } + + #[test] + fn a_missing_source_is_an_error_not_an_empty_entry() { + let dir = tempfile::tempdir().unwrap(); + let out = dir.path().join("out.zip"); + let missing = dir.path().join("nope"); + let err = pack_entries( + &[ZipEntry::compressed("e", &missing)], + &out, + Strategy::default(), + ) + .unwrap_err(); + assert_eq!(err.kind(), std::io::ErrorKind::NotFound); + } + + /// The archive is built by streaming: the bytes of a source that is larger than + /// anything we would want in memory come out right, with a bounded buffer. + #[test] + fn a_large_source_packs_by_streaming() { + let dir = tempfile::tempdir().unwrap(); + let src = dir.path().join("big.bin"); + { + let mut f = BufWriter::new(File::create(&src).unwrap()); + let block: Vec = (0..1_000_003u32).map(|i| (i % 253) as u8).collect(); + for _ in 0..12 { + f.write_all(&block).unwrap(); // ~12 MB + } + } + let out = dir.path().join("big.zip"); + pack_entries( + &[ZipEntry::stored("big.bin", &src)], + &out, + Strategy::default(), + ) + .unwrap(); + let mut archive = ZipArchive::new(File::open(&out).unwrap()).unwrap(); + let mut e = archive.by_name("big.bin").unwrap(); + assert_eq!(e.size(), 12_000_036); + use sha1::Digest; + let mut hasher_in = sha1::Sha1::new(); + let mut hasher_out = sha1::Sha1::new(); + { + let mut f = File::open(&src).unwrap(); + let mut buf = [0u8; 8192]; + loop { + let n = f.read(&mut buf).unwrap(); + if n == 0 { + break; + } + hasher_in.update(&buf[..n]); + } + loop { + let n = e.read(&mut buf).unwrap(); + if n == 0 { + break; + } + hasher_out.update(&buf[..n]); + } + assert_eq!(hasher_in.finalize(), hasher_out.finalize()); + } + } + #[test] fn resolve_strategy_applies_production_floor_and_flag_rules() { // Default is zstd at the documented default level. diff --git a/src/inject/mapjson.rs b/src/inject/mapjson.rs index ca8e50a..1f38a62 100644 --- a/src/inject/mapjson.rs +++ b/src/inject/mapjson.rs @@ -348,3 +348,373 @@ mod tests { )); } } + +/// Differential tests: the streaming implementations must agree with the +/// `serde_json::Value`-tree logic they replaced, over many generated documents +/// (whitespace, escapes, surrogate pairs, exponent numbers, nested sections). +#[cfg(test)] +mod differential { + use super::*; + use serde_json::Value; + + /// Deterministic xorshift: reproducible failures without a `rand` dependency. + struct Rng(u64); + impl Rng { + fn next(&mut self) -> u64 { + self.0 ^= self.0 << 13; + self.0 ^= self.0 >> 7; + self.0 ^= self.0 << 17; + self.0 + } + fn below(&mut self, n: u64) -> u64 { + self.next() % n + } + fn pick<'a>(&mut self, xs: &[&'a str]) -> &'a str { + xs[self.below(xs.len() as u64) as usize] + } + fn ws(&mut self) -> &'static str { + ["", "", " ", "\n", "\t ", "\r\n "][self.below(6) as usize] + } + } + + const KEYS: &[&str] = &[ + "version", + "sources", + "sourcesContent", + "mappings", + "debug_id", + "debugId", + "uuid", + "sections", + "map", + "names", + "file", + "offset", + "x\\u00e9y", + "k", + ]; + const STRINGS: &[&str] = &[ + r#""""#, + r#""a""#, + r#""h\u00e9llo""#, + "\"h\u{e9}llo \u{2603}\"", + r#""esc \n \" \\ \/ \t""#, + r#""\ud83d\ude00""#, + r#""AAAA;AACA,SAAS""#, + r#""11111111-1111-1111-1111-111111111111""#, + ]; + const NUMBERS: &[&str] = &[ + "0", + "-1", + "3", + "1.50", + "2e10", + "-0.5E-3", + "100000000000000000000", + "1e100", + "-2.5E-150", + "99999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999", + ]; + + fn scalar(r: &mut Rng) -> String { + match r.below(5) { + 0 => "null".into(), + 1 => "true".into(), + 2 => "false".into(), + 3 => r.pick(NUMBERS).into(), + _ => r.pick(STRINGS).into(), + } + } + + fn value(r: &mut Rng, depth: u32) -> String { + if depth == 0 || r.below(3) == 0 { + return scalar(r); + } + if r.below(2) == 0 { + let n = r.below(4); + let items: Vec = (0..n).map(|_| value(r, depth - 1)).collect(); + let sep = format!("{},{}", r.ws(), r.ws()); + format!("[{}{}{}]", r.ws(), items.join(&sep), r.ws()) + } else { + object(r, depth - 1, false) + } + } + + /// An object over the map vocabulary. `maplike` biases towards the keys the + /// transforms care about and recurses into `sections[].map`. Keys are unique + /// within an object (duplicates have their own targeted test). + fn object(r: &mut Rng, depth: u32, maplike: bool) -> String { + let n = r.below(6); + let mut used = std::collections::HashSet::new(); + let mut members = Vec::new(); + for _ in 0..n { + let key = r.pick(KEYS); + if !used.insert(key) { + continue; + } + let v = match key { + "sections" if depth > 0 => { + let m = r.below(3); + let secs: Vec = (0..m) + .map(|_| { + let mut parts = vec![format!(r#""offset":{}"#, value(r, 1))]; + if r.below(4) != 0 { + parts.push(format!(r#""map":{}"#, object(r, depth - 1, true))); + } + if r.below(5) == 0 { + parts.push(format!(r#""url":{}"#, scalar(r))); + } + format!("{{{}}}", parts.join(",")) + }) + .collect(); + format!("[{}]", secs.join(",")) + } + "sourcesContent" if maplike || r.below(2) == 0 => { + format!(r#"["{}","src2"]"#, "x".repeat(r.below(200) as usize)) + } + _ => value(r, depth.saturating_sub(1)), + }; + members.push(format!("{}\"{}\"{}:{}{}", r.ws(), key, r.ws(), r.ws(), v)); + } + format!("{{{}{}}}", members.join(","), r.ws()) + } + + fn doc(r: &mut Rng) -> String { + format!("{}{}{}", r.ws(), object(r, 3, true), r.ws()) + } + + // --- the reference (old, tree-based) behaviour ------------------------------- + + fn ref_strip(map: &mut serde_json::Map) -> bool { + let mut removed = map.remove("sourcesContent").is_some(); + if let Some(Value::Array(sections)) = map.get_mut("sections") { + for section in sections { + if let Some(Value::Object(inner)) = section.get_mut("map") { + removed |= ref_strip(inner); + } + } + } + removed + } + + fn ref_ids(v: &Value, keys: &[&str]) -> Vec> { + keys.iter() + .map(|k| v.get(*k).and_then(Value::as_str).map(str::to_string)) + .collect() + } + + const CASES: u64 = 600; + + #[test] + fn strip_sources_content_agrees_with_the_tree_implementation() { + let mut r = Rng(0x9E37_79B9_7F4A_7C15); + let mut removed_some = 0; + for case in 0..CASES { + let text = doc(&mut r); + let mut expected: Value = serde_json::from_str(&text) + .unwrap_or_else(|e| panic!("generator produced invalid JSON ({e}): {text}")); + let expect_removed = ref_strip(expected.as_object_mut().unwrap()); + let mut out = Vec::new(); + let removed = strip_sources_content(text.as_bytes(), &mut out) + .unwrap_or_else(|e| panic!("case {case}: {e}\n{text}")); + let got: Value = serde_json::from_slice(&out).unwrap_or_else(|e| { + panic!( + "case {case}: output not JSON ({e}): {}", + String::from_utf8_lossy(&out) + ) + }); + assert_eq!(got, expected, "case {case}\ninput: {text}"); + assert_eq!(removed, expect_removed, "case {case}\ninput: {text}"); + removed_some += removed as u32; + } + assert!( + removed_some > 50, + "the generator must exercise stripping ({removed_some})" + ); + } + + #[test] + fn top_level_strings_agrees_with_the_tree_implementation() { + let keys = ["debug_id", "debugId", "uuid"]; + let mut r = Rng(0xD1B5_4A32_D192_ED03); + for case in 0..CASES { + let text = doc(&mut r); + let v: Value = serde_json::from_str(&text).unwrap(); + let got = top_level_strings(text.as_bytes(), &keys).unwrap(); + assert!(got.is_object); + assert_eq!(got.values, ref_ids(&v, &keys), "case {case}\ninput: {text}"); + } + } + + #[test] + fn set_debug_ids_agrees_with_the_tree_implementation() { + let mut r = Rng(0xA076_1D64_78BD_642F); + for case in 0..CASES { + let text = doc(&mut r); + let mut expected: Value = serde_json::from_str(&text).unwrap(); + let obj = expected.as_object_mut().unwrap(); + obj.insert("debug_id".into(), Value::String("new-id".into())); + obj.insert("debugId".into(), Value::String("new-id".into())); + let mut out = Vec::new(); + set_debug_ids(text.as_bytes(), &mut out, "new-id").unwrap(); + let got: Value = serde_json::from_slice(&out).unwrap(); + assert_eq!(got, expected, "case {case}\ninput: {text}"); + // Idempotent: applying it to its own output changes nothing semantically. + let mut again = Vec::new(); + set_debug_ids(out.as_slice(), &mut again, "new-id").unwrap(); + assert_eq!(serde_json::from_slice::(&again).unwrap(), expected); + } + } + + #[test] + fn duplicate_keys_resolve_like_a_parsed_value() { + // serde_json keeps the LAST of a duplicated key; so must the stream. + let text = r#"{"debug_id":"a","debug_id":"b","uuid":"u","uuid":7,"sourcesContent":["s"],"sourcesContent":["t"]}"#; + let v: Value = serde_json::from_str(text).unwrap(); + let keys = ["debug_id", "uuid"]; + assert_eq!( + top_level_strings(text.as_bytes(), &keys).unwrap().values, + ref_ids(&v, &keys) + ); + let mut out = Vec::new(); + assert!(strip_sources_content(text.as_bytes(), &mut out).unwrap()); + assert!(!String::from_utf8(out).unwrap().contains("sourcesContent")); + } + + #[test] + fn truncated_or_trailing_garbage_documents_are_rejected_everywhere() { + let mut r = Rng(0x1234_5678_9ABC_DEF1); + for case in 0..200 { + let text = doc(&mut r); + let text = text.trim_end(); + // Any strict prefix of a top-level object is invalid JSON. + let cut = 1 + r.below(text.len() as u64 - 1) as usize; + let mut cut = cut; + while !text.is_char_boundary(cut) { + cut -= 1; + } + let bad = &text[..cut]; + assert!( + serde_json::from_str::(bad).is_err(), + "case {case}: {bad}" + ); + assert!( + top_level_strings(bad.as_bytes(), &["uuid"]).is_err(), + "case {case}: {bad}" + ); + assert!( + strip_sources_content(bad.as_bytes(), Vec::new()).is_err(), + "case {case}: {bad}" + ); + assert!( + set_debug_ids(bad.as_bytes(), Vec::new(), "x").is_err(), + "case {case}: {bad}" + ); + // Trailing garbage after a complete document. + let junk = format!("{text} x"); + assert!( + top_level_strings(junk.as_bytes(), &["uuid"]).is_err(), + "case {case}" + ); + assert!( + strip_sources_content(junk.as_bytes(), Vec::new()).is_err(), + "case {case}" + ); + } + } + + #[test] + fn a_large_map_streams_through_with_its_big_values_intact() { + // Bigger than any internal buffer: a 3 MB mappings string must pass through + // `transfer_to` byte for byte while a 3 MB sourcesContent entry is dropped. + let mappings = "AAAA;".repeat(600_000); + let content = "z".repeat(3_000_000); + let text = format!( + r#"{{"version":3,"sourcesContent":["{content}"],"mappings":"{mappings}","debug_id":"d"}}"# + ); + let mut out = Vec::new(); + assert!(strip_sources_content(text.as_bytes(), &mut out).unwrap()); + let got: Value = serde_json::from_slice(&out).unwrap(); + assert_eq!(got["mappings"].as_str().unwrap(), mappings); + assert!(got.get("sourcesContent").is_none()); + assert_eq!(got["debug_id"], "d"); + } + + /// Numbers no double can hold, exponents beyond +-99 and very long integers are valid JSON + /// grammar. They are skipped when reading ids and COPIED VERBATIM by the rewrites: the + /// stream never parses a number, so it must not refuse one (a refusal made + /// `--strip-sources-content` give up and upload the sources it was asked to drop). + #[test] + fn out_of_range_and_huge_numbers_copy_verbatim() { + let long = "7".repeat(150); + for n in ["1e999999", "1e100", "-3E-120", long.as_str()] { + let doc = format!(r#"{{"n":{n},"sourcesContent":["x"],"debug_id":"d"}}"#); + assert_eq!( + top_level_strings(doc.as_bytes(), &["debug_id"]) + .unwrap() + .values, + [Some("d".to_string())] + ); + let mut out = Vec::new(); + assert!( + strip_sources_content(doc.as_bytes(), &mut out).unwrap(), + "{n}" + ); + assert_eq!( + String::from_utf8(out).unwrap(), + format!(r#"{{"n":{n},"debug_id":"d"}}"#) + ); + let mut ids = Vec::new(); + set_debug_ids(doc.as_bytes(), &mut ids, "x").unwrap(); + assert!( + String::from_utf8(ids) + .unwrap() + .contains(&format!(r#""n":{n}"#)), + "{n}" + ); + } + } + + /// 200 000 randomly corrupted documents (byte replaced / deleted / inserted): + /// the stream accepts exactly what `serde_json` accepts, and everything it + /// copies out re-parses — it never launders invalid JSON into an upload. + #[test] + fn random_corruption_is_accepted_exactly_when_serde_accepts_it() { + let base = br#"{"version":3,"file":"a\u00e9.js","sources":["a.ts","b\"q"],"sourcesContent":["x\ny","\ud83d\ude00"],"names":["n1"],"mappings":"AAAA;AACA","x":[1,2.5e3,-0,true,null,{"k":"v"}],"y":[1e100,-2.5E-150]}"#; + let mut r = Rng(0x1234_5678_9ABC_DEF1); + for case in 0..200_000 { + let mut b = base.to_vec(); + for _ in 0..1 + r.below(2) { + let at = r.below(b.len() as u64) as usize; + match r.below(3) { + 0 => b[at] = r.next() as u8, + 1 => { + b.remove(at); + } + _ => b.insert(at, r.next() as u8), + } + } + let serde_res = serde_json::from_slice::(&b); + // The one deliberate difference: a number no f64 holds is rejected by serde + // ("number out of range") but is valid JSON the stream copies verbatim. + let range_only = serde_res + .as_ref() + .err() + .is_some_and(|e| e.to_string().contains("number out of range")); + let mut out = Vec::new(); + let streamed_ok = strip_sources_content(b.as_slice(), &mut out).is_ok(); + assert!( + streamed_ok == serde_res.is_ok() || (streamed_ok && range_only), + "case {case} disagrees on {:?}", + String::from_utf8_lossy(&b) + ); + if streamed_ok && !range_only { + assert!( + serde_json::from_slice::(&out).is_ok(), + "case {case}: output is not valid JSON: {}", + String::from_utf8_lossy(&out) + ); + } + } + } +} diff --git a/src/inject/mod.rs b/src/inject/mod.rs index 862c233..a71ec16 100644 --- a/src/inject/mod.rs +++ b/src/inject/mod.rs @@ -331,78 +331,169 @@ fn collect_targets( /// Mirrors `inject_one`'s decision, and `would_rewrite_agrees_with_what_inject_actually_writes` /// pins the two together so they cannot drift. fn would_rewrite(js_path: &Path) -> Result { - let content = std::fs::read_to_string(js_path)?; - let map_path = paired_map(js_path, &content); - Ok(match existing_debug_id(&content) { + let mapped = crate::symbols::mapped::map_file(js_path)?; + let content = bundle_text(&mapped)?; + let map_path = paired_map(js_path, content); + Ok(match existing_debug_id(content) { // A regenerated map re-keys the bundle; a foreign id still needs our runtime registration. Some(id) => { - restamp_id(&content, &id, map_path.as_deref())?.is_some() + restamp_id(content, &id, map_path.as_deref())?.is_some() || !content.contains(&runtime_registration(&id)) } None => true, }) } +/// A bundle's bytes as text. Invalid UTF-8 is an I/O `InvalidData` error, exactly +/// as `read_to_string` reported it. +fn bundle_text(bytes: &[u8]) -> Result<&str> { + std::str::from_utf8(bytes) + .map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e).into()) +} + +/// How a bundle is to change. Bundles are only ever APPENDED to (or, to re-key, have +/// our own trailing stub swapped), so the change is expressed as a tail rather +/// than as a whole new file. +struct JsEdit { + /// Cut the file back to this many bytes first (dropping a stale stub of ours). + truncate_to: Option, + /// Then append this. + append: String, +} + +/// What `inject_one` decided for a bundle, for the stats and the log line. +enum JsOutcome { + Restamped { stale: String, fresh: Uuid }, + Registered { id: String }, + Already, + Injected { id: Uuid }, +} + +/// Apply `edit` to the bundle IN PLACE: no copy of the bundle is made or held, +/// and the file keeps its inode, permissions and any symlink (as `fs::write` +/// did, which also truncated and rewrote through the same path). +fn apply_js_edit(js_path: &Path, edit: &JsEdit) -> Result<()> { + use std::io::{Seek, SeekFrom, Write}; + let mut file = std::fs::OpenOptions::new().write(true).open(js_path)?; + if let Some(len) = edit.truncate_to { + file.set_len(len)?; + } + file.seek(SeekFrom::End(0))?; + file.write_all(edit.append.as_bytes())?; + Ok(()) +} + fn inject_one(js_path: &Path, dry_run: bool, stats: &mut InjectStats) -> Result<()> { - let content = std::fs::read_to_string(js_path)?; - let map_path = paired_map(js_path, &content); - - let (debug_id, freshly_computed) = match existing_debug_id(&content) { - Some(id) => match restamp_id(&content, &id, map_path.as_deref())? { - // Our stub, over a map regenerated with different content: re-key the - // bundle, or the upload dedups against the STALE map (webpack keeps a - // `[contenthash]` JS file it considers unchanged but re-emits its map). - Some((original, fresh)) => { - if !dry_run { - std::fs::write(js_path, format!("{original}{}", runtime_stub(&fresh)))?; - } - stats.js_restamped += 1; - tracing::info!( - path = %js_path.display(), - stale = %id, - debug_id = %fresh, - "re-stamped debug-id: the bundle's map was regenerated" - ); - (fresh.to_string(), true) - } - // Another tool's id (Rollup 4 `sourcemapDebugIds` writes its own comment): - // keep the id — its map already carries it — but add our runtime - // registration, without which the SDK cannot attach it to a crash frame. - None if !content.contains(&runtime_registration(&id)) => { - if !dry_run { - std::fs::write(js_path, format!("{content}{}", runtime_registration(&id)))?; + // Decide everything with the bundle mapped read-only (zero-copy `&str`), then + // DROP the mapping before touching the file: Windows refuses to truncate a + // file that has a mapped view open. + let (map_path, debug_id, freshly_computed, outcome, edit) = { + let mapped = crate::symbols::mapped::map_file(js_path)?; + let content = bundle_text(&mapped)?; + let map_path = paired_map(js_path, content); + + match existing_debug_id(content) { + Some(id) => match restamp_id(content, &id, map_path.as_deref())? { + // Our stub, over a map regenerated with different content: re-key the + // bundle, or the upload dedups against the STALE map (webpack keeps a + // `[contenthash]` JS file it considers unchanged but re-emits its map). + Some((original, fresh)) => ( + map_path, + fresh.to_string(), + true, + JsOutcome::Restamped { stale: id, fresh }, + Some(JsEdit { + truncate_to: Some(original.len() as u64), + append: runtime_stub(&fresh), + }), + ), + // Another tool's id (Rollup 4 `sourcemapDebugIds` writes its own comment): + // keep the id — its map already carries it — but add our runtime + // registration, without which the SDK cannot attach it to a crash frame. + None if !content.contains(&runtime_registration(&id)) => { + let edit = JsEdit { + truncate_to: None, + append: runtime_registration(&id), + }; + ( + map_path, + id.clone(), + false, + JsOutcome::Registered { id }, + Some(edit), + ) } - stats.js_registered += 1; - tracing::info!( - path = %js_path.display(), - debug_id = %id, - "registered an existing debug-id another tool wrote" - ); - (id, false) - } + None => (map_path, id, false, JsOutcome::Already, None), + }, None => { - stats.js_already += 1; - (id, false) - } - }, - None => { - let id = match map_path.as_deref() { - Some(m) => compute_debug_id_with_map_file(content.as_bytes(), m)?, - None => compute_debug_id(content.as_bytes()), - }; - if !dry_run { - std::fs::write(js_path, format!("{content}{}", runtime_stub(&id)))?; + let id = match map_path.as_deref() { + Some(m) => compute_debug_id_with_map_file(content.as_bytes(), m)?, + None => compute_debug_id(content.as_bytes()), + }; + ( + map_path, + id.to_string(), + true, + JsOutcome::Injected { id }, + Some(JsEdit { + truncate_to: None, + append: runtime_stub(&id), + }), + ) } - stats.js_injected += 1; - tracing::info!(path = %js_path.display(), debug_id = %id, "injected debug-id"); - (id.to_string(), true) } }; - if let Some(map_path) = map_path { - if write_map_debug_id(&map_path, &debug_id, freshly_computed, dry_run)? { - stats.maps_updated += 1; - tracing::debug!(path = %map_path.display(), debug_id = %debug_id, "wrote debug_id into map"); + // EVERYTHING that can fail on the map happens BEFORE the first byte of the bundle + // changes: the map is validated (a JSON object), its rewrite is staged in a scratch + // file, and the original is opened for writing (a read-only map is refused here). + // A map that cannot take the id then leaves the bundle exactly as the bundler wrote + // it, instead of a bundle stamped with an id its map never got (which a retry would + // not repair: `restamp_id` leaves our own stub alone). + let map_plan = match map_path.as_deref() { + Some(m) => Some(plan_map_rewrite(m, &debug_id, freshly_computed, dry_run)?), + None => None, + }; + + if let (Some(edit), false) = (&edit, dry_run) { + apply_js_edit(js_path, edit)?; + } + let mut map_updated = false; + if let Some(MapPlan::Rewrite(rewrite)) = map_plan { + map_updated = true; + if let Some(rewrite) = rewrite { + apply_map_rewrite(rewrite)?; + } + } + + match outcome { + JsOutcome::Restamped { stale, fresh } => { + stats.js_restamped += 1; + tracing::info!( + path = %js_path.display(), + stale = %stale, + debug_id = %fresh, + "re-stamped debug-id: the bundle's map was regenerated" + ); + } + JsOutcome::Registered { id } => { + stats.js_registered += 1; + tracing::info!( + path = %js_path.display(), + debug_id = %id, + "registered an existing debug-id another tool wrote" + ); + } + JsOutcome::Already => stats.js_already += 1, + JsOutcome::Injected { id } => { + stats.js_injected += 1; + tracing::info!(path = %js_path.display(), debug_id = %id, "injected debug-id"); + } + } + if map_updated { + stats.maps_updated += 1; + if let Some(m) = map_path.as_deref() { + tracing::debug!(path = %m.display(), debug_id = %debug_id, "wrote debug_id into map"); } } Ok(()) @@ -436,7 +527,9 @@ fn restamp_id<'a>( else { return Ok(None); }; - if top.values.iter().any(Option::is_some) { + // A map the id cannot be written into (not a JSON object) is not re-keyed either: the + // rewrite would refuse it, and the bundle must not change when it does. + if !top.is_object || top.values.iter().any(Option::is_some) { return Ok(None); } let fresh = compute_debug_id_with_map_file(original.as_bytes(), map_path)?; @@ -527,14 +620,32 @@ fn map_id_action( } } -/// Make a `.map` JSON carry `debug_id` AND `debugId` equal to the bundle's id, -/// as [`map_id_action`] decides. Returns whether the map was changed. -fn write_map_debug_id( +/// A map rewrite staged in full but not yet applied (see [`plan_map_rewrite`]). +struct MapRewrite { + /// The original map, already OPENED for writing (so a read-only map fails at plan time). + dest: std::fs::File, + /// The complete new content. + scratch: std::fs::File, +} + +/// What the paired map needs. +enum MapPlan { + /// Already carries the right id, or must be left alone (shared with another bundle). + Unchanged, + /// To be rewritten. `None` in a dry run: validated and counted, never staged. + Rewrite(Option), +} + +/// Decide, and STAGE, making a `.map` JSON carry `debug_id` AND `debugId` equal to the +/// bundle's id, as [`map_id_action`] decides. Nothing on disk changes here: all the ways this +/// can fail (not a JSON object, invalid JSON, unreadable, not writable, no scratch space) +/// surface now, so the caller can refuse before touching the bundle. +fn plan_map_rewrite( map_path: &Path, debug_id: &str, freshly_computed: bool, dry_run: bool, -) -> Result { +) -> Result { // Pass 1 (streamed, also validates the whole file): what ids does it carry? let top = map_top_level(map_path, &["debug_id", "debugId"])?; if !top.is_object { @@ -550,7 +661,7 @@ fn write_map_debug_id( debug_id, freshly_computed, ) { - MapIdAction::Keep => return Ok(false), + MapIdAction::Keep => return Ok(MapPlan::Unchanged), MapIdAction::Fill => {} MapIdAction::Replace { stale } => tracing::warn!( path = %map_path.display(), @@ -566,44 +677,52 @@ fn write_map_debug_id( "source map carries a different debug id than its bundle, which already had one; left as is \ (is this map shared by more than one bundle?)" ); - return Ok(false); + return Ok(MapPlan::Unchanged); } } - if !dry_run { - // Pass 2: stream the map into a scratch file with the ids set, then write its - // CONTENT into the original (not rename: that would swap the inode and drop the - // symlink / hard link the build gave the map; and not `fs::copy`: it also copies - // the scratch file's metadata, i.e. its 0o600 mode, onto the map). - // The scratch file is anonymous (a killed run leaves no stray `.tmpXXXX`). It lives - // beside the map, on the volume that already holds one copy of it: a tens-of-MB map - // must not need that much free space in `$TMPDIR`, which in CI containers is often a - // small tmpfs. Only when the directory refuses a file (it is read-only: editing the - // map in place never needed it writable) does it fall back to the OS temp dir. - let dir = map_path - .parent() - .filter(|p| !p.as_os_str().is_empty()) - .unwrap_or_else(|| Path::new(".")); - let mut scratch = tempfile::tempfile_in(dir).or_else(|_| tempfile::tempfile())?; - { - let input = std::io::BufReader::new(std::fs::File::open(map_path)?); - let mut out = std::io::BufWriter::new(&scratch); - mapjson::set_debug_ids(input, &mut out, debug_id).map_err(|e| match e { - mapjson::MapJsonError::Io(e) => Error::Io(e), - mapjson::MapJsonError::Json(m) => Error::InputInvalid(format!( - "failed to rewrite sourcemap JSON: {m} ({})", - map_path.display() - )), - })?; - std::io::Write::flush(&mut out)?; - } - let mut dest = std::fs::OpenOptions::new() - .write(true) - .truncate(true) - .open(map_path)?; - std::io::Seek::seek(&mut scratch, std::io::SeekFrom::Start(0))?; - std::io::copy(&mut scratch, &mut dest)?; - } - Ok(true) + if dry_run { + return Ok(MapPlan::Rewrite(None)); + } + + // The original is opened for writing (NOT truncated yet) up front: a read-only map is + // an error now, not after the bundle was already stamped. + let dest = std::fs::OpenOptions::new().write(true).open(map_path)?; + + // Pass 2: stream the map into a scratch file with the ids set. Its CONTENT will later go + // into the original (not rename: that would swap the inode and drop the symlink / hard + // link the build gave the map; and not `fs::copy`: it also copies the scratch file's + // metadata, i.e. its 0o600 mode, onto the map). + // The scratch file is anonymous (a killed run leaves no stray `.tmpXXXX`). It lives + // beside the map, on the volume that already holds one copy of it: a tens-of-MB map + // must not need that much free space in `$TMPDIR`, which in CI containers is often a + // small tmpfs. Only when the directory refuses a file (it is read-only: editing the + // map in place never needed it writable) does it fall back to the OS temp dir. + let dir = map_path + .parent() + .filter(|p| !p.as_os_str().is_empty()) + .unwrap_or_else(|| Path::new(".")); + let scratch = tempfile::tempfile_in(dir).or_else(|_| tempfile::tempfile())?; + { + let input = std::io::BufReader::new(std::fs::File::open(map_path)?); + let mut out = std::io::BufWriter::new(&scratch); + mapjson::set_debug_ids(input, &mut out, debug_id).map_err(|e| match e { + mapjson::MapJsonError::Io(e) => Error::Io(e), + mapjson::MapJsonError::Json(m) => Error::InputInvalid(format!( + "failed to rewrite sourcemap JSON: {m} ({})", + map_path.display() + )), + })?; + std::io::Write::flush(&mut out)?; + } + Ok(MapPlan::Rewrite(Some(MapRewrite { dest, scratch }))) +} + +/// Write a staged rewrite into the original map: truncate it and copy the new content in. +fn apply_map_rewrite(mut rewrite: MapRewrite) -> Result<()> { + std::io::Seek::seek(&mut rewrite.scratch, std::io::SeekFrom::Start(0))?; + rewrite.dest.set_len(0)?; + std::io::copy(&mut rewrite.scratch, &mut rewrite.dest)?; + Ok(()) } /// Read the keying id from a `.map` for upload: `debug_id` (modern) / `debugId` @@ -1810,3 +1929,369 @@ mod tests { assert_eq!(read_debug_id(&p).unwrap(), None); } } + +/// What `inject` does to files on disk: exact bytes, in-place edits, what survives +/// (permissions, symlinks), what must NOT change (dry runs, rejected input) and +/// what must not be left behind (temp files). +#[cfg(test)] +mod file_edits { + use super::*; + + fn inject_dir(dir: &Path) -> Result { + inject_paths(&[dir.to_path_buf()], &[], true, false) + } + + fn names(dir: &Path) -> Vec { + let mut v: Vec = std::fs::read_dir(dir) + .unwrap() + .map(|e| e.unwrap().file_name().to_string_lossy().into_owned()) + .collect(); + v.sort(); + v + } + + const MAP: &str = r#"{"version":3,"sources":["a.ts"],"names":["x"],"mappings":"AAAA"}"#; + + /// Whatever the bundle ends with (or doesn't), injection is EXACTLY + /// `bundle || stub(id)` — nothing re-encoded, nothing normalised — and a second + /// run is a byte-for-byte no-op. + #[test] + fn appends_exactly_the_stub_to_awkward_bundles() { + let big = format!("{}\n", "var a=1;".repeat(30_000)); + let cases: Vec<(&str, String)> = vec![ + ("empty", String::new()), + ("no trailing newline", "a()".into()), + ("trailing newline", "a()\n".into()), + ("CRLF", "a()\r\nb()\r\n".into()), + ("multibyte", "var s = 'héllo ☃ 😀';\n".into()), + ("BOM first", "\u{feff}a()\n".into()), + ("one huge line", big), + ]; + for (label, content) in cases { + let dir = tempfile::tempdir().unwrap(); + let js = dir.path().join("a.js"); + std::fs::write(&js, &content).unwrap(); + + inject_dir(dir.path()).unwrap(); + let id = compute_debug_id(content.as_bytes()); + assert_eq!( + std::fs::read(&js).unwrap(), + format!("{content}{}", runtime_stub(&id)).into_bytes(), + "{label}" + ); + + let once = std::fs::read(&js).unwrap(); + inject_dir(dir.path()).unwrap(); + assert_eq!(std::fs::read(&js).unwrap(), once, "{label}: not idempotent"); + } + } + + #[test] + fn with_a_map_the_id_covers_both_and_both_files_are_updated() { + let dir = tempfile::tempdir().unwrap(); + let js = dir.path().join("a.js"); + let map = dir.path().join("a.js.map"); + let content = "a()\n//# sourceMappingURL=a.js.map\n"; + std::fs::write(&js, content).unwrap(); + std::fs::write(&map, MAP).unwrap(); + + let stats = inject_dir(dir.path()).unwrap(); + assert_eq!((stats.js_injected, stats.maps_updated), (1, 1)); + + let id = compute_debug_id_with_map(content.as_bytes(), Some(MAP.as_bytes())); + assert_eq!( + std::fs::read_to_string(&js).unwrap(), + format!("{content}{}", runtime_stub(&id)) + ); + let got: serde_json::Value = serde_json::from_slice(&std::fs::read(&map).unwrap()).unwrap(); + let mut expected: serde_json::Value = serde_json::from_str(MAP).unwrap(); + expected["debug_id"] = id.to_string().into(); + expected["debugId"] = id.to_string().into(); + assert_eq!(got, expected, "every other key survives the rewrite"); + assert_eq!(names(dir.path()), ["a.js", "a.js.map"], "no temp file left"); + } + + #[test] + fn a_map_with_unicode_and_escapes_round_trips() { + let dir = tempfile::tempdir().unwrap(); + let map = dir.path().join("a.js.map"); + let original = r#"{"version":3,"sources":["src/héllo ☃.ts","q\"uote"],"sourcesContent":["line1\nline2\t\\ 😀"],"names":[],"mappings":"AAAA","x_google_ignoreList":[0],"n":1.50}"#; + std::fs::write(dir.path().join("a.js"), "a()\n").unwrap(); + std::fs::write(&map, original).unwrap(); + inject_dir(dir.path()).unwrap(); + + let got: serde_json::Value = serde_json::from_slice(&std::fs::read(&map).unwrap()).unwrap(); + let mut expected: serde_json::Value = serde_json::from_str(original).unwrap(); + let id = got["debug_id"].as_str().unwrap().to_string(); + expected["debug_id"] = id.clone().into(); + expected["debugId"] = id.into(); + assert_eq!(got, expected); + assert!( + std::fs::read_to_string(&map).unwrap().contains("1.50"), + "number text is kept, not round-tripped through f64" + ); + } + + /// Re-keying swaps ONLY our own trailing stub: the bundle is cut back to the + /// original bytes and the new stub appended — no remnant of the old id. + #[test] + fn restamp_cuts_back_to_the_original_bytes_before_appending() { + let dir = tempfile::tempdir().unwrap(); + let js = dir.path().join("a.js"); + let map = dir.path().join("a.js.map"); + let content = "function f(){return 1}\n//# sourceMappingURL=a.js.map\n"; + std::fs::write(&js, content).unwrap(); + std::fs::write(&map, MAP).unwrap(); + inject_dir(dir.path()).unwrap(); + let first = std::fs::read_to_string(&js).unwrap(); + let old_id = existing_debug_id(&first).unwrap(); + + // The bundler regenerates the map with different content (and no ids). + let regenerated = r#"{"version":3,"sources":["b.ts"],"names":[],"mappings":"BBBB"}"#; + std::fs::write(&map, regenerated).unwrap(); + let stats = inject_dir(dir.path()).unwrap(); + assert_eq!(stats.js_restamped, 1); + + let new_id = compute_debug_id_with_map(content.as_bytes(), Some(regenerated.as_bytes())); + assert_ne!(new_id.to_string(), old_id); + let after = std::fs::read_to_string(&js).unwrap(); + assert_eq!(after, format!("{content}{}", runtime_stub(&new_id))); + assert!(!after.contains(&old_id), "no remnant of the stale id"); + assert!( + std::fs::read_to_string(&map) + .unwrap() + .contains(&new_id.to_string()), + "the regenerated map carries the new id" + ); + } + + /// A bundle another tool already stamped keeps its bytes and only GAINS our registration. + #[test] + fn registering_a_foreign_id_only_appends() { + let dir = tempfile::tempdir().unwrap(); + let js = dir.path().join("a.js"); + let id = "11111111-1111-1111-1111-111111111111"; + let content = format!("a()\n//# debugId={id}\n//# sourceMappingURL=a.js.map\n"); + std::fs::write(&js, &content).unwrap(); + std::fs::write( + dir.path().join("a.js.map"), + format!(r#"{{"version":3,"debug_id":"{id}","debugId":"{id}","mappings":""}}"#), + ) + .unwrap(); + let stats = inject_dir(dir.path()).unwrap(); + assert_eq!(stats.js_registered, 1); + assert_eq!( + std::fs::read_to_string(&js).unwrap(), + format!("{content}{}", runtime_registration(&id)) + ); + } + + #[test] + fn a_dry_run_changes_no_byte_and_no_mtime() { + let dir = tempfile::tempdir().unwrap(); + let js = dir.path().join("a.js"); + let map = dir.path().join("a.js.map"); + std::fs::write(&js, "a()\n").unwrap(); + std::fs::write(&map, MAP).unwrap(); + let snap = |p: &Path| { + ( + std::fs::read(p).unwrap(), + std::fs::metadata(p).unwrap().modified().unwrap(), + ) + }; + let (js0, map0) = (snap(&js), snap(&map)); + let stats = inject_paths(&[dir.path().to_path_buf()], &[], true, true).unwrap(); + assert_eq!( + (stats.js_injected, stats.maps_updated), + (1, 1), + "it still reports" + ); + assert_eq!(snap(&js), js0); + assert_eq!(snap(&map), map0); + assert_eq!( + names(dir.path()), + ["a.js", "a.js.map"], + "dry run leaves no temp file" + ); + } + + #[test] + fn a_bundle_that_is_not_utf8_is_an_io_error_and_untouched() { + let dir = tempfile::tempdir().unwrap(); + let js = dir.path().join("a.js"); + std::fs::write(&js, b"a()\xff\xfe\n").unwrap(); + let err = inject_dir(dir.path()).unwrap_err(); + assert!(matches!(err, Error::Io(_)), "{err:?}"); + assert_eq!(std::fs::read(&js).unwrap(), b"a()\xff\xfe\n"); + } + + /// A map that is not a JSON object cannot take an id: the run fails and NOTHING is + /// touched. In particular the bundle must not be stamped first: it would then carry an id + /// its map never got, and a retry would not repair it (our own stub is left alone). + #[test] + fn an_unusable_map_leaves_both_files_untouched() { + for (label, map) in [ + ("not json", "{not json"), + ("an array", "[]"), + ("truncated", r#"{"version":3,"mappings":"AA"#), + ("trailing junk", r#"{"version":3} x"#), + ] { + let dir = tempfile::tempdir().unwrap(); + let js = dir.path().join("a.js"); + std::fs::write(&js, "a()\n").unwrap(); + let mp = dir.path().join("a.js.map"); + std::fs::write(&mp, map).unwrap(); + let err = inject_dir(dir.path()).unwrap_err(); + assert!(matches!(err, Error::InputInvalid(_)), "{label}: {err:?}"); + assert_eq!(std::fs::read_to_string(&mp).unwrap(), map, "{label}"); + assert_eq!( + std::fs::read_to_string(&js).unwrap(), + "a()\n", + "{label}: the bundle was stamped before the map was found unusable" + ); + assert_eq!(names(dir.path()), ["a.js", "a.js.map"], "{label}"); + } + } + + /// Same guarantee on a RE-run: a stamped bundle whose map was replaced by something + /// unusable is not re-keyed (that rewrite would be refused), so it is not changed either. + #[test] + fn an_unusable_regenerated_map_does_not_restamp_the_bundle() { + for (label, map) in [("an array", "[]"), ("not json", "{nope")] { + let dir = tempfile::tempdir().unwrap(); + let js = dir.path().join("a.js"); + std::fs::write(&js, "a()\n//# sourceMappingURL=a.js.map\n").unwrap(); + std::fs::write(dir.path().join("a.js.map"), MAP).unwrap(); + inject_dir(dir.path()).unwrap(); + let stamped = std::fs::read(&js).unwrap(); + + std::fs::write(dir.path().join("a.js.map"), map).unwrap(); + let err = inject_dir(dir.path()).unwrap_err(); + assert!(matches!(err, Error::InputInvalid(_)), "{label}: {err:?}"); + assert_eq!( + std::fs::read(&js).unwrap(), + stamped, + "{label}: bundle changed" + ); + } + } + + #[cfg(unix)] + mod unix { + use super::*; + use std::os::unix::fs::{symlink, PermissionsExt}; + + fn mode(p: &Path) -> u32 { + std::fs::metadata(p).unwrap().permissions().mode() & 0o777 + } + + #[test] + fn permissions_survive_on_both_files() { + let dir = tempfile::tempdir().unwrap(); + let js = dir.path().join("a.js"); + let map = dir.path().join("a.js.map"); + std::fs::write(&js, "a()\n").unwrap(); + std::fs::write(&map, MAP).unwrap(); + // Neither is 0o600, the mode of the sibling temp file the map rewrite goes + // through: a map pinned at 0o600 would pass even if the temp's mode leaked + // onto it (this test once did). + std::fs::set_permissions(&js, std::fs::Permissions::from_mode(0o640)).unwrap(); + std::fs::set_permissions(&map, std::fs::Permissions::from_mode(0o664)).unwrap(); + inject_dir(dir.path()).unwrap(); + assert_eq!(mode(&js), 0o640); + assert_eq!(mode(&map), 0o664, "the map rewrite must not reset its mode"); + } + + #[test] + fn a_symlinked_bundle_and_map_stay_links_and_their_targets_change() { + let dir = tempfile::tempdir().unwrap(); + let real = tempfile::tempdir().unwrap(); + std::fs::write(real.path().join("real.js"), "a()\n").unwrap(); + std::fs::write(real.path().join("real.js.map"), MAP).unwrap(); + symlink(real.path().join("real.js"), dir.path().join("a.js")).unwrap(); + symlink(real.path().join("real.js.map"), dir.path().join("a.js.map")).unwrap(); + + inject_dir(dir.path()).unwrap(); + for l in ["a.js", "a.js.map"] { + assert!( + std::fs::symlink_metadata(dir.path().join(l)) + .unwrap() + .file_type() + .is_symlink(), + "{l} must still be a symlink" + ); + } + assert!(std::fs::read_to_string(real.path().join("real.js")) + .unwrap() + .contains("//# debugId=")); + assert!(std::fs::read_to_string(real.path().join("real.js.map")) + .unwrap() + .contains("\"debug_id\"")); + assert_eq!( + names(real.path()), + ["real.js", "real.js.map"], + "no temp file" + ); + } + + /// Rewriting a map needs no write access to its DIRECTORY (editing existing files in + /// place never did): a read-only `dist/` holding writable files must still inject, and + /// nothing may be created beside them. + /// A map that cannot be written (read-only) is refused BEFORE the bundle is stamped. + #[test] + fn a_read_only_map_is_an_error_and_the_bundle_stays_unstamped() { + let dir = tempfile::tempdir().unwrap(); + let js = dir.path().join("a.js"); + let map = dir.path().join("a.js.map"); + std::fs::write(&js, "a()\n").unwrap(); + std::fs::write(&map, MAP).unwrap(); + std::fs::set_permissions(&map, std::fs::Permissions::from_mode(0o444)).unwrap(); + if std::fs::OpenOptions::new().write(true).open(&map).is_ok() { + return; // running as root: permissions are not enforced + } + assert!(matches!(inject_dir(dir.path()), Err(Error::Io(_)))); + assert_eq!( + std::fs::read(&js).unwrap(), + b"a()\n", + "the bundle was stamped" + ); + assert_eq!(std::fs::read_to_string(&map).unwrap(), MAP); + } + + #[test] + fn files_in_a_read_only_directory_are_still_rewritten() { + let dir = tempfile::tempdir().unwrap(); + std::fs::write(dir.path().join("a.js"), "a()\n").unwrap(); + std::fs::write(dir.path().join("a.js.map"), MAP).unwrap(); + std::fs::set_permissions(dir.path(), std::fs::Permissions::from_mode(0o555)).unwrap(); + let can_create = std::fs::File::create(dir.path().join("probe")).is_ok(); + let result = inject_dir(dir.path()); + let listing = names(dir.path()); + std::fs::set_permissions(dir.path(), std::fs::Permissions::from_mode(0o755)).unwrap(); + if can_create { + return; // running as root: directory permissions are not enforced + } + result.unwrap(); + assert_eq!(listing, ["a.js", "a.js.map"]); + assert!(std::fs::read_to_string(dir.path().join("a.js.map")) + .unwrap() + .contains("debug_id")); + assert!(std::fs::read_to_string(dir.path().join("a.js")) + .unwrap() + .contains("//# debugId=")); + } + + #[test] + fn a_read_only_bundle_is_an_error_and_stays_as_it_was() { + let dir = tempfile::tempdir().unwrap(); + let js = dir.path().join("a.js"); + std::fs::write(&js, "a()\n").unwrap(); + std::fs::set_permissions(&js, std::fs::Permissions::from_mode(0o444)).unwrap(); + if std::fs::OpenOptions::new().write(true).open(&js).is_ok() { + return; // running as root: permissions are not enforced + } + assert!(matches!(inject_dir(dir.path()), Err(Error::Io(_)))); + assert_eq!(std::fs::read(&js).unwrap(), b"a()\n"); + } + } +} diff --git a/src/symbols/dsym.rs b/src/symbols/dsym.rs index 3f073f3..a8694f1 100644 --- a/src/symbols/dsym.rs +++ b/src/symbols/dsym.rs @@ -73,6 +73,7 @@ pub fn identify(dsym_path: &Path) -> Result { } let mut slices = Vec::new(); + let mut saw_nil_uuid = false; for entry in std::fs::read_dir(&dwarf_dir)? { let entry = entry?; if !entry.file_type()?.is_file() { @@ -95,6 +96,19 @@ pub fn identify(dsym_path: &Path) -> Result { e, )) })?; + // A slice without an LC_UUID reads as the nil UUID. It can never match a + // crash report, and every such slice would collide on the same all-zero + // key on the server, so it is not a symbol: skip it (the same stance + // `xcode_ipa::select_preferred_uuid` takes for the app binary). + if obj.debug_id().is_nil() { + tracing::warn!( + path = %bin_path.display(), + arch = %obj.arch().name(), + "Mach-O slice has no LC_UUID (nil UUID); skipping it" + ); + saw_nil_uuid = true; + continue; + } slices.push(DsymSlice { uuid: obj.debug_id().to_string(), arch: obj.arch().name().to_string(), @@ -103,10 +117,17 @@ pub fn identify(dsym_path: &Path) -> Result { } if slices.is_empty() { - return Err(Error::InputInvalid(format!( - "dSYM bundle contains no Mach-O slices: {}", - dsym_path.display() - ))); + return Err(Error::InputInvalid(if saw_nil_uuid { + format!( + "no Mach-O slice in the dSYM bundle carries an LC_UUID: {}", + dsym_path.display() + ) + } else { + format!( + "dSYM bundle contains no Mach-O slices: {}", + dsym_path.display() + ) + })); } Ok(DsymIdentity { slices }) } @@ -163,6 +184,120 @@ mod tests { use std::fs; use std::io::Write; + // ── broken Mach-O / DWARF inputs ─────────────────────────────── + + const ARM64: u32 = 0x0100_000c; + const X86_64: u32 = 0x0100_0007; + const GOOD: [u8; 16] = [ + 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff, + 0x01, + ]; + + fn bundle_with(bytes: &[u8]) -> (tempfile::TempDir, PathBuf) { + let tmp = tempfile::tempdir().unwrap(); + let dsym = tmp.path().join("App.dSYM"); + let dwarf = dsym.join("Contents/Resources/DWARF"); + fs::create_dir_all(&dwarf).unwrap(); + fs::write(dwarf.join("App"), bytes).unwrap(); + (tmp, dsym) + } + + fn invalid(err: Error) -> String { + match err { + Error::InputInvalid(m) => m, + other => panic!("expected InputInvalid, got {other:?}"), + } + } + + /// A Mach-O with no `LC_UUID` reads as the nil UUID, which can never match a + /// crash report and would register every such dSYM under the same all-zero key. + /// It is not a symbol: the bundle is rejected, and says why. + #[test] + fn a_macho_without_a_uuid_is_rejected_not_registered_as_nil() { + let (_t, dsym) = bundle_with(&crate::symbols::test_macho::thin_macho(ARM64, 0, [0; 16])); + let msg = invalid(identify(&dsym).unwrap_err()); + assert!(msg.contains("LC_UUID"), "{msg}"); + } + + #[test] + fn a_nil_slice_in_a_fat_binary_is_skipped_and_the_real_ones_kept() { + let fat = crate::symbols::test_macho::fat_macho(&[(ARM64, 0, GOOD), (X86_64, 3, [0; 16])]); + let (_t, dsym) = bundle_with(&fat); + let id = identify(&dsym).unwrap(); + assert_eq!(id.slices.len(), 1, "{:?}", id.slices); + assert_eq!(id.slices[0].arch, "arm64"); + assert!(!id.slices[0].uuid.starts_with("00000000-0000-0000")); + } + + /// Whatever prefix of a valid Mach-O survives, `identify` returns an error or the + /// TRUE identity — never a panic and never a different UUID. + #[test] + fn every_truncation_of_a_macho_is_an_error_or_the_true_identity() { + let valid = crate::symbols::test_macho::thin_macho(ARM64, 0, GOOD); + let want = identify(&bundle_with(&valid).1).unwrap().slices; + assert_eq!(want.len(), 1); + for len in 0..valid.len() { + let (_t, dsym) = bundle_with(&valid[..len]); + if let Ok(id) = identify(&dsym) { + assert_eq!( + id.slices.iter().map(|s| &s.uuid).collect::>(), + want.iter().map(|s| &s.uuid).collect::>(), + "truncated to {len} bytes reported a different identity" + ); + } + } + } + + #[test] + fn corrupted_headers_never_panic_and_are_invalid_or_identified() { + let valid = crate::symbols::test_macho::fat_macho(&[(ARM64, 0, GOOD), (X86_64, 3, GOOD)]); + let mut cases: Vec<(String, Vec)> = vec![ + ("empty".into(), Vec::new()), + ("macho magic only".into(), vec![0xcf, 0xfa, 0xed, 0xfe]), + ("fat magic only".into(), vec![0xca, 0xfe, 0xba, 0xbe]), + ( + "fat claiming 2^32-1 slices".into(), + [ + &[0xca, 0xfe, 0xba, 0xbe, 0xff, 0xff, 0xff, 0xff][..], + &[0u8; 100], + ] + .concat(), + ), + ( + "garbage".into(), + (0..4000u32).map(|i| (i * 131 % 253) as u8).collect(), + ), + ("zeroed".into(), vec![0; valid.len()]), + ]; + // Every header byte replaced by 0x00 / 0xff / 0x7f / 0x80. + for i in 0..valid.len().min(96) { + for v in [0x00u8, 0xff, 0x7f, 0x80] { + let mut b = valid.clone(); + b[i] = v; + cases.push((format!("byte {i} = {v:#x}"), b)); + } + } + for (label, bytes) in cases { + let (_t, dsym) = bundle_with(&bytes); + // The assertion is "returns" — a panic fails the test; both outcomes are fine. + let _ = identify(&dsym) + .map(|id| id.slices.len()) + .map_err(|e| e.to_string()); + let _ = label; + } + } + + #[test] + fn a_subdirectory_or_empty_file_in_the_dwarf_folder_does_not_hide_a_good_slice() { + let (_t, dsym) = bundle_with(&crate::symbols::test_macho::thin_macho(ARM64, 0, GOOD)); + let dwarf = dsym.join("Contents/Resources/DWARF"); + fs::create_dir(dwarf.join("nested")).unwrap(); + fs::write(dwarf.join("junk"), b"").unwrap(); + // The empty file is not a Mach-O: that is an error for the bundle, loudly, + // rather than a silently partial identity. + assert!(identify(&dsym).is_err()); + } + #[test] fn rejects_plain_directory_without_dwarf_dir() { // Any NAME is accepted here (discovery owns the name policy, so a diff --git a/src/symbols/elf.rs b/src/symbols/elf.rs index f4fd182..dc20bfa 100644 --- a/src/symbols/elf.rs +++ b/src/symbols/elf.rs @@ -275,6 +275,12 @@ fn parse_elf_identity(bytes: &[u8]) -> ElfIdentity { has_debug_info: false, has_symbols: false, }; + // `symbolic` identifies Mach-O, PE and PDB just as readily, and their ids would + // read as a "build-id" here: a macOS `.so` or a stray `.pdb` named `*.so` would + // be registered as an ELF symbol under a foreign key. This uploader takes ELF. + if !bytes.starts_with(b"\x7fELF") { + return unknown(); + } let archive = match Archive::parse(bytes) { Ok(a) => a, Err(_) => return unknown(), @@ -546,6 +552,226 @@ mod tests { ); } + // ── corrupted / truncated ELF files ──────────────────────────── + + const TRUE_BUILD_ID: &str = "bca64abfec40dbb631bb8f1c37414472"; + + fn fixture() -> Vec { + std::fs::read( + Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/elf/libsymbol1.so"), + ) + .unwrap() + } + + /// Deterministic xorshift, so a failure names a reproducible byte. + fn xorshift(state: &mut u64) -> u64 { + *state ^= *state << 13; + *state ^= *state >> 7; + *state ^= *state << 17; + *state + } + + /// Run `parse_elf_identity`, turning a panic into a reportable failure. + fn parse_catching(bytes: &[u8], label: &str) -> ElfIdentity { + std::panic::catch_unwind(|| parse_elf_identity(bytes)) + .unwrap_or_else(|_| panic!("parse_elf_identity PANICKED on {label}")) + } + + /// The key of a symbol is its build-id, so the one thing a damaged file must never + /// do is report ANOTHER build-id: it would be uploaded under the wrong key and + /// look like a different library's symbols. Any prefix of the real library either + /// parses to no build-id or to the TRUE one — nothing in between. + #[test] + fn a_truncated_elf_has_no_build_id_or_the_true_one() { + let elf = fixture(); + let mut cuts: Vec = (0..=130).collect(); // every byte of the headers + let mut at = 131usize; + while at < elf.len() { + cuts.push(at); + at += at / 7 + 13; // geometric-ish sweep over the whole file + } + cuts.push(elf.len() - 1); + for cut in cuts { + let id = parse_catching(&elf[..cut], &format!("a {cut}-byte prefix")); + assert!( + id.build_id.is_none() || id.build_id.as_deref() == Some(TRUE_BUILD_ID), + "prefix of {cut} bytes reported build-id {:?}", + id.build_id + ); + } + // And the whole file still parses to the true identity (the sweep's control). + assert_eq!( + parse_catching(&elf, "the whole file").build_id.as_deref(), + Some(TRUE_BUILD_ID) + ); + } + + /// Damage to the ELF header and the section/program header tables — the + /// structures `symbolic` walks to find the build-id note and the debug sections — + /// must be survivable: no panic, whatever it then reports. + #[test] + fn corrupt_elf_headers_and_tables_never_panic() { + let elf = fixture(); + // e_shoff (0x28), e_phoff (0x20), e_shentsize/num/shstrndx (0x3a..0x40), e_phnum. + let fields: &[(usize, usize)] = &[ + (0x20, 8), + (0x28, 8), + (0x36, 2), + (0x38, 2), + (0x3a, 2), + (0x3c, 2), + (0x3e, 2), + ]; + let mut checked = 0; + for &(off, len) in fields { + for fill in [0x00u8, 0xff, 0x7f, 0x80, 0x01] { + let mut b = elf.clone(); + b[off..off + len].fill(fill); + parse_catching(&b, &format!("header field @{off:#x} = {fill:#x}")); + checked += 1; + } + } + // Every one of the first 64 header bytes flipped to a few values. + for i in 0..64 { + for v in [0x00u8, 0xff, 0x55] { + let mut b = elf.clone(); + b[i] = v; + parse_catching(&b, &format!("e_ident/header byte {i} = {v:#x}")); + checked += 1; + } + } + // The section header table is at e_shoff; flip bytes throughout it. + let shoff = u64::from_le_bytes(elf[0x28..0x30].try_into().unwrap()) as usize; + let shnum = u16::from_le_bytes(elf[0x3c..0x3e].try_into().unwrap()) as usize; + assert!(shoff > 0 && shnum > 0, "fixture has section headers"); + let table_end = (shoff + shnum * 64).min(elf.len()); + for i in (shoff..table_end).step_by(5) { + let mut b = elf.clone(); + b[i] ^= 0xff; + parse_catching(&b, &format!("section header byte {i} flipped")); + checked += 1; + } + assert!(checked > 100); + } + + #[test] + fn random_byte_flips_anywhere_never_panic() { + let elf = fixture(); + let mut state = 0x2545_F491_4F6C_DD1Du64; + for round in 0..300 { + let mut b = elf.clone(); + for _ in 0..1 + round % 8 { + let at = (xorshift(&mut state) % b.len() as u64) as usize; + b[at] = xorshift(&mut state) as u8; + } + parse_catching(&b, &format!("random corruption round {round}")); + } + } + + #[test] + fn non_elf_and_degenerate_inputs_have_no_identity() { + for (label, bytes) in [ + ("empty", Vec::new()), + ("one byte", vec![0x7f]), + ("magic only", b"\x7fELF".to_vec()), + ("zeros", vec![0; 4096]), + ("a text file", b"#!/bin/sh\necho hi\n".repeat(50)), + ("a ZIP", b"PK\x03\x04junk".repeat(40)), + ( + "a Mach-O", + crate::symbols::test_macho::thin_macho(0x0100_000c, 0, [3; 16]), + ), + ] { + let id = parse_catching(&bytes, label); + assert_eq!(id.build_id, None, "{label}"); + } + } + + /// A library that parses but carries no build-id (`-Wl,--build-id` was off) is + /// reported without one, so the uploader can warn and skip it. + #[test] + fn scan_dir_reports_damaged_libraries_without_a_build_id_instead_of_failing() { + let elf = fixture(); + let dir = tempfile::tempdir().unwrap(); + std::fs::write(dir.path().join("libgood.so"), &elf).unwrap(); + std::fs::write(dir.path().join("libhalf.so"), &elf[..elf.len() / 2]).unwrap(); + std::fs::write(dir.path().join("libzero.so"), vec![0u8; 5000]).unwrap(); + std::fs::write(dir.path().join("libtext.so"), b"not an elf").unwrap(); + let libs = scan_dir(dir.path(), &ExtraSuffixes::default()).unwrap(); + let by_name = |n: &str| libs.iter().find(|l| l.name == n).unwrap(); + assert_eq!( + by_name("libgood.so").build_id.as_deref(), + Some(TRUE_BUILD_ID) + ); + for damaged in ["libhalf.so", "libzero.so", "libtext.so"] { + assert_eq!(by_name(damaged).build_id, None, "{damaged}"); + } + let kept = keep_richest_per_build_id(libs); + assert_eq!( + kept.len(), + 4, + "keyless libs pass through for the uploader to warn about" + ); + } + + /// A ZIP entry that fails its CRC (bit rot in `native-debug-symbols.zip`) is an + /// error for the archive, never a silently-wrong library. + #[test] + fn a_zip_entry_with_a_bad_crc_is_an_error() { + let elf = fixture(); + let dir = tempfile::tempdir().unwrap(); + let zip_path = dir.path().join("n.zip"); + { + let mut zw = zip::ZipWriter::new(std::fs::File::create(&zip_path).unwrap()); + zw.start_file( + "arm64-v8a/libfoo.so", + SimpleFileOptions::default().compression_method(zip::CompressionMethod::Stored), + ) + .unwrap(); + zw.write_all(&elf).unwrap(); + zw.finish().unwrap(); + } + let mut raw = std::fs::read(&zip_path).unwrap(); + let mid = raw.len() / 2; // inside the stored payload + raw[mid] ^= 0x5a; + std::fs::write(&zip_path, &raw).unwrap(); + let out = dir.path().join("out"); + std::fs::create_dir_all(&out).unwrap(); + assert!(extract_libs(&zip_path, &out, &ExtraSuffixes::default()).is_err()); + } + + #[test] + fn truncated_or_garbage_zips_are_errors() { + let dir = tempfile::tempdir().unwrap(); + let good = dir.path().join("good.zip"); + { + let mut zw = zip::ZipWriter::new(std::fs::File::create(&good).unwrap()); + zw.start_file("a/libfoo.so", SimpleFileOptions::default()) + .unwrap(); + zw.write_all(&fixture()).unwrap(); + zw.finish().unwrap(); + } + let raw = std::fs::read(&good).unwrap(); + let out = dir.path().join("out"); + std::fs::create_dir_all(&out).unwrap(); + for (label, bytes) in [ + ("empty", Vec::new()), + ("half", raw[..raw.len() / 2].to_vec()), + ("no end-of-central-directory", raw[..raw.len() - 8].to_vec()), + ( + "garbage", + (0..5000u32).map(|i| (i * 7 % 251) as u8).collect(), + ), + ] { + let p = dir.path().join("bad.zip"); + std::fs::write(&p, bytes).unwrap(); + assert!( + extract_libs(&p, &out, &ExtraSuffixes::default()).is_err(), + "{label}" + ); + } + } + fn lib(name: &str, build_id: Option<&str>, richness: (bool, bool, u64)) -> ElfLib { ElfLib { name: name.to_string(), diff --git a/src/symbols/mapped.rs b/src/symbols/mapped.rs index 0709ed8..9b5eb00 100644 --- a/src/symbols/mapped.rs +++ b/src/symbols/mapped.rs @@ -51,4 +51,43 @@ mod tests { assert!(map_file(&p).unwrap().is_empty()); assert!(map_file(&dir.path().join("missing")).is_err()); } + + #[test] + fn a_directory_is_an_error_not_a_panic_or_empty_bytes() { + // `File::open` succeeds on a directory on Unix; mapping it must not. + let dir = tempfile::tempdir().unwrap(); + assert!(map_file(dir.path()).is_err()); + } + + #[test] + fn large_and_page_straddling_files_map_whole() { + let dir = tempfile::tempdir().unwrap(); + for len in [4095usize, 4096, 4097, 1_000_003] { + let data: Vec = (0..len).map(|i| (i % 251) as u8).collect(); + let p = dir.path().join("f"); + std::fs::write(&p, &data).unwrap(); + let m = map_file(&p).unwrap(); + assert_eq!(m.len(), len); + assert_eq!(&*m, &data[..]); + } + } + + #[cfg(unix)] + #[test] + fn an_unreadable_file_is_a_permission_error() { + use std::os::unix::fs::PermissionsExt; + let dir = tempfile::tempdir().unwrap(); + let p = dir.path().join("f"); + std::fs::write(&p, b"secret").unwrap(); + std::fs::set_permissions(&p, std::fs::Permissions::from_mode(0o000)).unwrap(); + let readable = std::fs::File::open(&p).is_ok(); // root ignores modes + let r = map_file(&p); + std::fs::set_permissions(&p, std::fs::Permissions::from_mode(0o644)).unwrap(); + if !readable { + assert_eq!( + r.err().unwrap().kind(), + std::io::ErrorKind::PermissionDenied + ); + } + } } diff --git a/src/symbols/pdb.rs b/src/symbols/pdb.rs index 3afa6f3..1953cd5 100644 --- a/src/symbols/pdb.rs +++ b/src/symbols/pdb.rs @@ -240,6 +240,82 @@ mod tests { (dir, path) } + // ── corrupted / truncated PDBs ───────────────────────────────── + + /// A damaged PDB either fails to identify or identifies as ITSELF: never a + /// panic, and never a different debug id (it is the upload key). + #[test] + fn every_truncation_of_a_pdb_is_an_error_or_the_true_identity() { + let valid = synth_pdb(guid(), 7, 1, MACHINE_AMD64); + let want = "dfb8e43a-f242-3d73-a453-aeb6a777ef75-1"; + let mut cuts: Vec = (0..=200).collect(); + let mut at = 201usize; + while at < valid.len() { + cuts.push(at); + at += at / 9 + 17; + } + cuts.push(valid.len() - 1); + for cut in cuts { + let (_d, p) = write_temp(&valid[..cut]); + match identify(&p) { + Ok(id) => assert!( + id.slices.iter().all(|s| s.uuid == want), + "{cut}-byte prefix identified as {:?}", + id.slices + ), + Err(Error::InputInvalid(_)) => {} + Err(other) => panic!("{cut}-byte prefix: unexpected {other:?}"), + } + } + } + + #[test] + fn corrupted_msf_structures_never_panic() { + let valid = synth_pdb(guid(), 7, 1, MACHINE_AMD64); + let mut state = 0x9E37_79B9_7F4A_7C15u64; + let mut next = || { + state ^= state << 13; + state ^= state >> 7; + state ^= state << 17; + state + }; + // The superblock fields (page size, page count, directory size / location) + // set to hostile values: a zero or huge page size, absurd counts. + for off in (32..56).step_by(4) { + for v in [0u32, 1, 0xffff_ffff, 0x8000_0000, 4096 * 1024] { + let mut b = valid.clone(); + b[off..off + 4].copy_from_slice(&v.to_le_bytes()); + let (_d, p) = write_temp(&b); + let _ = identify(&p); + } + } + for _ in 0..300 { + let mut b = valid.clone(); + for _ in 0..1 + next() % 6 { + let at = (next() % b.len() as u64) as usize; + b[at] = next() as u8; + } + let (_d, p) = write_temp(&b); + let _ = identify(&p); + } + } + + #[test] + fn non_pdb_files_are_rejected_as_invalid() { + for (label, bytes) in [ + ("empty", Vec::new()), + ("msf magic only", super::MSF7_MAGIC.to_vec()), + ("zeros", vec![0u8; 8192]), + ("an ELF header", b"\x7fELF\x02\x01\x01".repeat(300)), + ] { + let (_d, p) = write_temp(&bytes); + assert!( + matches!(identify(&p), Err(Error::InputInvalid(_))), + "{label}" + ); + } + } + /// The upload key. This exact string is what the worker re-derives from the /// same bytes (`symbolfiles/pdb.py`, same `symbolic` major) and what a /// crashing Windows module resolves against — changing the formatting (to diff --git a/src/upload/http.rs b/src/upload/http.rs index 807d6f6..8a269ea 100644 --- a/src/upload/http.rs +++ b/src/upload/http.rs @@ -327,6 +327,81 @@ mod tests { use wiremock::matchers::method; use wiremock::{Mock, MockServer, ResponseTemplate}; + async fn put_streamed( + server: &MockServer, + path: &std::path::Path, + declared_len: u64, + ) -> reqwest::Result { + reqwest::Client::new() + .put(format!("{}/put", server.uri())) + .header(reqwest::header::CONTENT_LENGTH, declared_len) + .body(file_body(path)) + .send() + .await + } + + /// Sizes around the 64 KiB chunk: the streamed body is the file, byte for byte, + /// with the length the caller declared and no chunked encoding. + #[tokio::test] + async fn file_body_streams_every_byte_around_the_chunk_size() { + let dir = tempfile::tempdir().unwrap(); + for len in [0usize, 1, 65_535, 65_536, 65_537, 200_000] { + let server = MockServer::start().await; + Mock::given(method("PUT")) + .respond_with(ResponseTemplate::new(200)) + .mount(&server) + .await; + let data: Vec = (0..len).map(|i| (i % 251) as u8).collect(); + let p = dir.path().join("payload"); + std::fs::write(&p, &data).unwrap(); + let declared = file_len(&p).await.unwrap(); + assert_eq!(declared, len as u64); + + put_streamed(&server, &p, declared).await.unwrap(); + let req = &server.received_requests().await.unwrap()[0]; + assert_eq!(req.body, data, "len {len}"); + assert_eq!( + req.headers.get("content-length").unwrap().to_str().unwrap(), + len.to_string() + ); + assert!(req.headers.get("transfer-encoding").is_none(), "len {len}"); + } + } + + /// A payload that vanished (or cannot be read) fails the request instead of + /// sending an empty body that the server would store as a valid, empty symbol. + #[tokio::test] + async fn file_body_of_a_missing_file_fails_the_request() { + let server = MockServer::start().await; + Mock::given(method("PUT")) + .respond_with(ResponseTemplate::new(200)) + .mount(&server) + .await; + let dir = tempfile::tempdir().unwrap(); + let result = put_streamed(&server, &dir.path().join("gone"), 10).await; + assert!(result.is_err(), "must not succeed with an empty body"); + } + + /// The file shrank after its length was read (a concurrent writer): the body + /// ends short of the declared Content-Length, which must surface as a failed + /// request, never as a "successful" upload of a truncated symbol. + #[tokio::test] + async fn file_body_shorter_than_its_declared_length_fails_the_request() { + let server = MockServer::start().await; + Mock::given(method("PUT")) + .respond_with(ResponseTemplate::new(200)) + .mount(&server) + .await; + let dir = tempfile::tempdir().unwrap(); + let p = dir.path().join("payload"); + std::fs::write(&p, vec![7u8; 1000]).unwrap(); + let result = put_streamed(&server, &p, 5000).await; + assert!( + result.is_err(), + "a truncated body must not look like success" + ); + } + // ── put_file: streaming PUT that follows 307/308 itself ────────────────── fn payload(len: usize) -> (tempfile::TempDir, std::path::PathBuf, Vec) { diff --git a/tests/broken_files.rs b/tests/broken_files.rs new file mode 100644 index 0000000..2787dbd --- /dev/null +++ b/tests/broken_files.rs @@ -0,0 +1,901 @@ +//! How the compiled binary behaves on BROKEN input files: corrupted or truncated JSON, +//! ELF, Mach-O/DWARF, PDB, zip, plist and dependency manifests. +//! +//! The contract these pin, per flow: +//! - never a crash (no panic exit 101, no signal / abort); +//! - a stable exit code an integrator can branch on, and a message naming the file; +//! - nothing is sent to the server for input that could not be identified; +//! - the user's own files are left exactly as they were; +//! - stdout stays reserved for structured output. +//! +//! Where a flow deliberately TOLERATES a broken file (it warns and carries on), the test +//! says so, so loosening or tightening that is a conscious change. + +use std::path::{Path, PathBuf}; + +use serde_json::{json, Value}; +use wiremock::matchers::{method, path as wm_path}; +use wiremock::{Mock, MockServer, ResponseTemplate}; + +mod common; + +const FIXTURE_ELF: &str = "tests/fixtures/elf/libsymbol1.so"; + +fn write(dir: &Path, rel: &str, bytes: &[u8]) -> PathBuf { + let p = dir.join(rel); + std::fs::create_dir_all(p.parent().unwrap()).unwrap(); + std::fs::write(&p, bytes).unwrap(); + p +} + +fn fixture_elf() -> Vec { + std::fs::read(Path::new(env!("CARGO_MANIFEST_DIR")).join(FIXTURE_ELF)).unwrap() +} + +/// Exit code of a finished process, failing the test on a crash (panic = 101 or a signal). +fn code_of(out: &std::process::Output) -> i32 { + let code = out.status.code().unwrap_or(-1); + assert!( + code != 101 && code >= 0, + "the binary CRASHED (status {:?}):\n{}", + out.status, + String::from_utf8_lossy(&out.stderr) + ); + code +} + +fn run(args: &[&str]) -> std::process::Output { + common::cli().args(args).output().unwrap() +} + +fn stderr(out: &std::process::Output) -> String { + String::from_utf8_lossy(&out.stderr).into_owned() +} + +// --------------------------------------------------------------------------- +// Source-map JSON: upload, --strip-sources-content and inject, one table +// --------------------------------------------------------------------------- + +const VALID_MAP: &str = r#"{"version":3,"debug_id":"22222222-2222-2222-2222-222222222222","sources":["a"],"sourcesContent":["x"],"mappings":"AAAA"}"#; + +/// (label, map bytes, upload dry-run, strip dry-run, inject) exit codes. +/// +/// 11 = input invalid. Non-object but well-formed JSON (`[]`, `3`) is NOT a syntax +/// error: upload / strip dry runs tolerate it (there is no id to read), while +/// `inject` cannot write an id into it and says so. +fn json_cases() -> Vec<(&'static str, Vec, i32, i32, i32)> { + let deep_array = format!("{}{}", "[".repeat(100_000), "]".repeat(100_000)); + let deep_object = format!("{}1{}", r#"{"a":"#.repeat(50_000), "}".repeat(50_000)); + let bom = format!("\u{feff}{VALID_MAP}"); + let trailing_doc = format!("{VALID_MAP}{VALID_MAP}"); + let truncated = VALID_MAP[..40].to_string(); + let trailing_comma = format!("{},}}", &VALID_MAP[..VALID_MAP.len() - 1]); + let c = |label, bytes: &str, a, b, c| (label, bytes.as_bytes().to_vec(), a, b, c); + vec![ + c("empty", "", 11, 11, 11), + c("whitespace only", " \n\t ", 11, 11, 11), + c("truncated", &truncated, 11, 11, 11), + c("trailing comma", &trailing_comma, 11, 11, 11), + c("byte-order mark", &bom, 11, 11, 11), + c("two documents", &trailing_doc, 11, 11, 11), + c("NaN literal", r#"{"a":NaN}"#, 11, 11, 11), + c("bad escape", r#"{"a":"\x"}"#, 11, 11, 11), + c("lone surrogate", r#"{"debug_id":"\ud800"}"#, 11, 11, 11), + c( + "raw control char in a string", + "{\"a\":\"x\ty\"}", + 11, + 11, + 11, + ), + c("NUL byte in a string", "{\"a\":\"x\0y\"}", 11, 11, 11), + c("100k-deep arrays", &deep_array, 11, 11, 11), + c("50k-deep objects", &deep_object, 11, 11, 11), + c("array at the top level", "[]", 0, 0, 11), + c("number at the top level", "3", 0, 0, 11), + c("null at the top level", "null", 0, 0, 11), + c("string at the top level", "\"s\"", 0, 0, 11), + // Numbers no double holds (or with a 3-digit exponent) are valid JSON: they are + // skipped when reading ids and copied verbatim when rewriting, never refused. + c( + "number beyond any double", + r#"{"version":3,"n":1e999999,"mappings":""}"#, + 0, + 0, + 0, + ), + c( + "three-digit exponent", + r#"{"version":3,"n":1e100,"mappings":""}"#, + 0, + 0, + 0, + ), + // Duplicate keys are legal; the last one wins, as in any JSON parser. + c( + "duplicate keys", + r#"{"debug_id":"a","debug_id":"b","mappings":""}"#, + 0, + 0, + 0, + ), + ] +} + +#[test] +fn broken_source_maps_never_crash_and_exit_with_stable_codes() { + for (label, bytes, want_upload, want_strip, want_inject) in json_cases() { + let tmp = tempfile::tempdir().unwrap(); + let map = write(tmp.path(), "m/app.js.map", &bytes); + let dir = tmp.path().join("m"); + let dir_s = dir.to_str().unwrap(); + let base = [ + "debug-files", + "upload", + "--type", + "sourcemaps", + "--version", + "1", + "--build", + "1", + "--dry-run", + ]; + + let up = common::cli().args(base).arg(&dir).output().unwrap(); + assert_eq!( + code_of(&up), + want_upload, + "upload: {label}\n{}", + stderr(&up) + ); + assert!( + up.stdout.is_empty(), + "upload: {label}: stdout must stay empty" + ); + + let strip = common::cli() + .args(base) + .arg("--strip-sources-content") + .arg(&dir) + .output() + .unwrap(); + assert_eq!( + code_of(&strip), + want_strip, + "strip: {label}\n{}", + stderr(&strip) + ); + + // Inject needs a bundle next to the map; it must not touch a map it rejects. + write(tmp.path(), "m/app.js", b"a()\n"); + let inj = run(&["sourcemaps", "inject", dir_s]); + assert_eq!( + code_of(&inj), + want_inject, + "inject: {label}\n{}", + stderr(&inj) + ); + if want_inject == 11 { + assert_eq!( + std::fs::read(&map).unwrap(), + bytes, + "inject rewrote a map it rejected: {label}" + ); + assert_eq!( + std::fs::read(dir.join("app.js")).unwrap(), + b"a()\n", + "inject stamped the bundle although it rejected the map: {label}" + ); + let left: Vec<_> = std::fs::read_dir(&dir) + .unwrap() + .map(|e| e.unwrap().file_name()) + .collect(); + assert_eq!(left.len(), 2, "{label}: stray files left behind: {left:?}"); + } + if want_upload == 11 { + assert!( + stderr(&up).contains(map.to_str().unwrap()) || stderr(&up).contains("sourcemap"), + "{label}: the message must name the problem: {}", + stderr(&up) + ); + } + } +} + +/// A real (non-dry-run) upload of a map that cannot be identified registers NOTHING. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn an_unidentifiable_source_map_sends_nothing_to_the_server() { + let server = MockServer::start().await; + Mock::given(method("POST")) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({"code": 0}))) + .expect(0) + .mount(&server) + .await; + Mock::given(method("PUT")) + .respond_with(ResponseTemplate::new(200)) + .expect(0) + .mount(&server) + .await; + let tmp = tempfile::tempdir().unwrap(); + for (i, bytes) in [&b"{ not json"[..], b"[]", b"\"s\"", b"", b"{\"a\":"] + .iter() + .enumerate() + { + write(tmp.path(), &format!("m{i}/app.js.map"), bytes); + } + let endpoint = server.uri(); + let root = tmp.path().to_path_buf(); + tokio::task::spawn_blocking(move || { + for i in 0..5 { + let out = common::cli() + .args(["--endpoint", &endpoint, "--app-token", "TKN"]) + .args([ + "debug-files", + "upload", + "--type", + "sourcemaps", + "--version", + "1", + "--build", + "1", + ]) + .arg(root.join(format!("m{i}"))) + .output() + .unwrap(); + let code = code_of(&out); + assert_ne!(code, 0, "case m{i} must fail: {}", stderr(&out)); + } + }) + .await + .unwrap(); +} + +async fn mock_symbol_server() -> MockServer { + let server = MockServer::start().await; + let put_url = format!("{}/put", server.uri()); + Mock::given(method("POST")) + .and(wm_path("/apps/TKN/symbols")) + .respond_with( + ResponseTemplate::new(200).set_body_json(json!({"code": 0, "endpoint": put_url})), + ) + .mount(&server) + .await; + Mock::given(method("PUT")) + .respond_with(ResponseTemplate::new(200)) + .mount(&server) + .await; + server +} + +/// The bytes of the single entry of the (only) uploaded ZIP. +async fn uploaded_entry(server: &MockServer) -> Vec { + let reqs = server.received_requests().await.unwrap(); + let put = reqs + .iter() + .find(|r| r.method.as_str() == "PUT") + .expect("uploaded"); + let mut archive = zip::ZipArchive::new(std::io::Cursor::new(put.body.clone())).unwrap(); + let mut body = Vec::new(); + std::io::Read::read_to_end(&mut archive.by_index(0).unwrap(), &mut body).unwrap(); + body +} + +/// `--strip-sources-content` over a map it cannot process (not a JSON object) uploads the +/// ORIGINAL (a privacy preference is not a reason to fail the upload) — and says so +/// loudly, because the sources then go out unstripped. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn strip_over_an_unprocessable_map_uploads_the_original_and_warns() { + let server = mock_symbol_server().await; + let tmp = tempfile::tempdir().unwrap(); + // Valid JSON, but an array: it has no `sourcesContent` to strip. `--uuid` supplies its key. + let odd = br#"[{"sourcesContent":["secret"]}]"#; + write(tmp.path(), "m/app.js.map", odd); + let endpoint = server.uri(); + let dir = tmp.path().join("m"); + let warned = tokio::task::spawn_blocking(move || { + let out = common::cli() + .args(["--endpoint", &endpoint, "--app-token", "TKN"]) + .args([ + "debug-files", + "upload", + "--type", + "sourcemaps", + "--strip-sources-content", + ]) + .args(["--version", "1", "--build", "1"]) + .args(["--uuid", "55555555-5555-5555-5555-555555555555"]) + .arg(&dir) + .output() + .unwrap(); + assert_eq!(code_of(&out), 0, "{}", stderr(&out)); + stderr(&out).contains("UNSTRIPPED") + }) + .await + .unwrap(); + assert!( + warned, + "the unstripped fallback must be announced, not silent" + ); + assert_eq!( + uploaded_entry(&server).await, + odd.to_vec(), + "the original bytes, untouched" + ); +} + +/// REGRESSION: numbers that are valid JSON but unusual (`1e100`, beyond any double, 150 +/// digits) used to make the strip give up and upload the UNSTRIPPED map, silently. The sources +/// must be gone from what is uploaded, and every number's text preserved. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn strip_removes_sources_from_maps_with_unusual_numbers() { + let long = "7".repeat(150); + for n in ["1e100", "-3E-120", "1e999999", long.as_str()] { + let server = mock_symbol_server().await; + let tmp = tempfile::tempdir().unwrap(); + let map = format!( + r#"{{"version":3,"debug_id":"66666666-6666-6666-6666-666666666666","n":{n},"sourcesContent":["secret source"],"mappings":"AAAA"}}"# + ); + write(tmp.path(), "m/app.js.map", map.as_bytes()); + let endpoint = server.uri(); + let dir = tmp.path().join("m"); + tokio::task::spawn_blocking(move || { + let out = common::cli() + .args(["--endpoint", &endpoint, "--app-token", "TKN"]) + .args([ + "debug-files", + "upload", + "--type", + "sourcemaps", + "--strip-sources-content", + ]) + .args(["--version", "1", "--build", "1"]) + .arg(&dir) + .output() + .unwrap(); + assert_eq!(code_of(&out), 0, "{}", stderr(&out)); + assert!(!stderr(&out).contains("UNSTRIPPED"), "{}", stderr(&out)); + }) + .await + .unwrap(); + let uploaded = String::from_utf8(uploaded_entry(&server).await).unwrap(); + assert!( + !uploaded.contains("secret source"), + "{n}: the source leaked: {uploaded}" + ); + assert!( + uploaded.contains(&format!(r#""n":{n}"#)), + "{n}: number text changed: {uploaded}" + ); + } +} + +// --------------------------------------------------------------------------- +// ELF: damaged libraries in a directory, damaged zips +// --------------------------------------------------------------------------- + +fn elf_dry_run(input: &Path) -> std::process::Output { + common::cli() + .args([ + "debug-files", + "upload", + "--type", + "elf", + "--version", + "1", + "--build", + "1", + ]) + .args([ + "--uuid", + "00000000-0000-0000-0000-000000000000", + "--dry-run", + ]) + .arg(input) + .output() + .unwrap() +} + +/// A directory with one good library and every kind of damaged one: the damaged are +/// warned about and skipped, the good one is still uploaded, and the run succeeds. +#[test] +fn damaged_libraries_are_skipped_with_a_warning_and_the_good_one_survives() { + let elf = fixture_elf(); + let tmp = tempfile::tempdir().unwrap(); + write(tmp.path(), "libs/libgood.so", &elf); + write(tmp.path(), "libs/libempty.so", b""); + write(tmp.path(), "libs/libhalf.so", &elf[..elf.len() / 2]); + write(tmp.path(), "libs/libhead.so", &elf[..64]); + write(tmp.path(), "libs/libzero.so", &vec![0u8; 8192]); + write( + tmp.path(), + "libs/libtext.so", + b"#!/bin/sh\necho not a library\n", + ); + // A VALID Mach-O (it has a real UUID) named *.so, as a macOS-built module would be: + // not an ELF, so it must not become an "elf" symbol keyed by its Mach-O UUID. + let macho = thin_macho_with_uuid([0x5a; 16]); + write(tmp.path(), "libs/libmacho.so", &macho); + + let out = elf_dry_run(&tmp.path().join("libs")); + assert_eq!(code_of(&out), 0, "{}", stderr(&out)); + assert!(out.stdout.is_empty()); + let err = stderr(&out); + assert!(err.contains("would register + upload 1 libraries"), "{err}"); + assert_eq!(err.matches("no GNU build-id").count(), 6, "{err}"); +} + +/// If NOTHING in the directory is usable, that is a run with no symbols: it succeeds +/// (the zip contract) but says so. +#[test] +fn a_directory_of_only_damaged_libraries_uploads_nothing_and_says_so() { + let tmp = tempfile::tempdir().unwrap(); + write(tmp.path(), "libs/liba.so", b"junk"); + write(tmp.path(), "libs/libb.so", &[0u8; 100]); + let out = elf_dry_run(&tmp.path().join("libs")); + assert_eq!(code_of(&out), 0); + assert!( + stderr(&out).contains("would register + upload 0 libraries"), + "{}", + stderr(&out) + ); +} + +#[test] +fn broken_native_symbol_zips_are_input_invalid() { + let elf = fixture_elf(); + let tmp = tempfile::tempdir().unwrap(); + let good = write(tmp.path(), "good.zip", &[]); + { + let mut zw = zip::ZipWriter::new(std::fs::File::create(&good).unwrap()); + zw.start_file( + "arm64-v8a/libfoo.so", + zip::write::SimpleFileOptions::default(), + ) + .unwrap(); + std::io::Write::write_all(&mut zw, &elf).unwrap(); + zw.finish().unwrap(); + } + let raw = std::fs::read(&good).unwrap(); + let mut crc = raw.clone(); + let mid = crc.len() / 2; + crc[mid] ^= 0x5a; + let cases: Vec<(&str, Vec)> = vec![ + ("empty file", Vec::new()), + ("truncated in half", raw[..raw.len() / 2].to_vec()), + ( + "end-of-central-directory missing", + raw[..raw.len() - 12].to_vec(), + ), + ( + "random bytes", + (0..4000u32).map(|i| (i * 7 % 251) as u8).collect(), + ), + ("a flipped byte in the payload", crc), + ]; + for (label, bytes) in cases { + let p = write(tmp.path(), "bad.zip", &bytes); + let out = elf_dry_run(&p); + assert_eq!(code_of(&out), 11, "{label}: {}", stderr(&out)); + assert!( + stderr(&out).contains("bad.zip"), + "{label}: names the file: {}", + stderr(&out) + ); + } +} + +// --------------------------------------------------------------------------- +// Mach-O / dSYM / PDB +// --------------------------------------------------------------------------- + +fn bundle_with(parent: &Path, name: &str, dwarf: &[u8]) -> PathBuf { + let b = parent.join(format!("{name}.dSYM")); + write(&b, &format!("Contents/Resources/DWARF/{name}"), dwarf); + b +} + +fn thin_macho_with_uuid(uuid: [u8; 16]) -> Vec { + let mut b = vec![0xcf, 0xfa, 0xed, 0xfe]; + b.extend_from_slice(&0x0100_000cu32.to_le_bytes()); // arm64 + b.extend_from_slice(&0u32.to_le_bytes()); + b.extend_from_slice(&0x0au32.to_le_bytes()); // MH_DSYM + b.extend_from_slice(&1u32.to_le_bytes()); + b.extend_from_slice(&24u32.to_le_bytes()); + b.extend_from_slice(&[0u8; 8]); + b.extend_from_slice(&0x1bu32.to_le_bytes()); // LC_UUID + b.extend_from_slice(&24u32.to_le_bytes()); + b.extend_from_slice(&uuid); + b +} + +#[test] +fn dsym_uuid_on_broken_machos_prints_an_empty_list_and_never_a_nil_uuid() { + let tmp = tempfile::tempdir().unwrap(); + let valid = thin_macho_with_uuid([0xAB; 16]); + let cases: Vec<(&str, Vec, Vec<&str>)> = vec![ + ( + "valid", + valid.clone(), + vec!["ABABABAB-ABAB-ABAB-ABAB-ABABABABABAB"], + ), + ("no LC_UUID (nil)", thin_macho_with_uuid([0; 16]), vec![]), + ("empty", Vec::new(), vec![]), + ("magic only", vec![0xcf, 0xfa, 0xed, 0xfe], vec![]), + ( + "fat header claiming 2^32 slices", + [ + &[0xca, 0xfe, 0xba, 0xbe, 0xff, 0xff, 0xff, 0xff][..], + &[0u8; 64], + ] + .concat(), + vec![], + ), + ( + "garbage", + (0..3000u32).map(|i| (i * 31 % 253) as u8).collect(), + vec![], + ), + ( + "valid, cut mid-command", + valid[..valid.len() - 5].to_vec(), + vec![], + ), + ]; + for (label, bytes, want) in cases { + for target in ["file", "bundle"] { + let bundle = bundle_with(tmp.path(), "App", &bytes); + let path = if target == "file" { + bundle.join("Contents/Resources/DWARF/App") + } else { + bundle + }; + let out = run(&["dsym", "uuid", path.to_str().unwrap()]); + assert_eq!(code_of(&out), 0, "{label}/{target}: {}", stderr(&out)); + let got: Vec = serde_json::from_slice(&out.stdout) + .unwrap_or_else(|e| panic!("{label}/{target}: stdout is not a JSON array ({e})")); + assert_eq!(got, want, "{label}/{target}"); + } + } +} + +#[test] +fn a_dsym_with_unreadable_dwarf_is_skipped_by_upload_but_fails_the_build_phase() { + let tmp = tempfile::tempdir().unwrap(); + for (label, bytes) in [ + ( + "garbage", + (0..3000u32) + .map(|i| (i * 31 % 253) as u8) + .collect::>(), + ), + ("empty", Vec::new()), + ("no LC_UUID", thin_macho_with_uuid([0; 16])), + ] { + let dsyms = tmp.path().join(label.replace(' ', "_")); + bundle_with(&dsyms, "App", &bytes); + + // `debug-files upload`: the bundle is warned about and skipped (exit 0, nothing sent). + let out = common::cli() + .args([ + "debug-files", + "upload", + "--type", + "dsym", + "--version", + "1", + "--build", + "1", + "--dry-run", + ]) + .arg(&dsyms) + .output() + .unwrap(); + assert_eq!(code_of(&out), 0, "{label}: {}", stderr(&out)); + assert!( + stderr(&out).contains("not a readable dSYM bundle"), + "{label}: {}", + stderr(&out) + ); + + // `xcode upload-dsyms`: symbols silently not uploaded is the failure it exists to + // prevent, so the same bundle FAILS the build phase (11). + let out = common::cli() + .args(["xcode", "upload-dsyms", "--no-background"]) + .env("DWARF_DSYM_FOLDER_PATH", &dsyms) + .env("BUGSEE_APP_TOKEN", "TKN") + .env("BUGSEE_ENDPOINT", "http://127.0.0.1:1") + .output() + .unwrap(); + assert_eq!(code_of(&out), 11, "{label}: {}", stderr(&out)); + assert!( + stderr(&out).contains("could not be read"), + "{label}: {}", + stderr(&out) + ); + } +} + +#[test] +fn broken_pdbs_are_not_identified_and_never_crash() { + let tmp = tempfile::tempdir().unwrap(); + let msf = b"Microsoft C/C++ MSF 7.00\r\n\x1aDS\0\0\0"; + for (label, bytes, want) in [ + ("empty", Vec::new(), 10), + ( + "random bytes", + (0..4000u32).map(|i| (i * 13 % 251) as u8).collect(), + 10, + ), + ("MSF magic only", msf.to_vec(), 11), + ( + "MSF magic + zeroed superblock", + [&msf[..], &[0u8; 64]].concat(), + 11, + ), + ( + "MSF magic + hostile superblock", + [&msf[..], &[0xffu8; 64]].concat(), + 11, + ), + ] { + let dir = tmp.path().join(label.replace([' ', '+'], "_")); + write(&dir, "a.pdb", &bytes); + let out = common::cli() + .args([ + "debug-files", + "upload", + "--type", + "pdb", + "--version", + "1", + "--build", + "1", + "--dry-run", + ]) + .arg(&dir) + .output() + .unwrap(); + assert_eq!(code_of(&out), want, "{label}: {}", stderr(&out)); + assert!(out.stdout.is_empty(), "{label}"); + } +} + +// --------------------------------------------------------------------------- +// Plists +// --------------------------------------------------------------------------- + +/// `build-env read-plist` reads `{}` for anything unusable, INCLUDING nesting deep +/// enough to overflow a stack (which used to abort the process with SIGABRT). +#[test] +fn read_plist_of_broken_files_is_an_empty_object_and_never_a_crash() { + let tmp = tempfile::tempdir().unwrap(); + let nested_xml = [ + &b""[..], + &b"".repeat(60_000), + &b"".repeat(60_000), + b"", + ] + .concat(); + for (label, bytes) in [ + ("empty", Vec::new()), + ( + "random bytes", + (0..300u32).map(|i| (i * 37 % 251) as u8).collect(), + ), + ( + "truncated XML", + b"CFBundle".to_vec(), + ), + ("bplist magic only", b"bplist00".to_vec()), + ( + "array root", + b"".to_vec(), + ), + ("60k-deep nesting", nested_xml), + ] { + let p = write(tmp.path(), "Info.plist", &bytes); + let out = run(&["build-env", "read-plist", p.to_str().unwrap()]); + assert_eq!(code_of(&out), 0, "{label}: {}", stderr(&out)); + assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), "{}", "{label}"); + } +} + +// --------------------------------------------------------------------------- +// iOS dependency manifests +// --------------------------------------------------------------------------- + +fn ios_deps(pods: &[u8], pins: &[u8]) -> Value { + let tmp = tempfile::tempdir().unwrap(); + write(tmp.path(), "Podfile.lock", pods); + write(tmp.path(), "Package.resolved", pins); + let out = common::cli() + .args(["ios-deps", "collect", "--project-root"]) + .arg(tmp.path()) + .output() + .unwrap(); + assert_eq!(code_of(&out), 0, "{}", stderr(&out)); + serde_json::from_slice(&out.stdout).expect("stdout is always the JSON document") +} + +/// Broken manifests degrade to fewer entries, never to a crash, a non-JSON stdout, or a +/// bogus nameless dependency in the graph. +#[test] +fn broken_dependency_manifests_still_yield_a_clean_json_document() { + let deep = [&b"["[..].repeat(100_000)[..], &b"]".repeat(100_000)].concat(); + let big_lock = [&b"PODS:\n"[..], &b" - Foo (1.0)\n".repeat(50_000)].concat(); + let cases: Vec<(&str, Vec, Vec)> = vec![ + ("empty", Vec::new(), Vec::new()), + ( + "random bytes", + (0..500u32).map(|i| (i * 37 % 251) as u8).collect(), + (0..500u32).map(|i| (i * 53 % 251) as u8).collect(), + ), + ( + "truncated", + b"PODS:\n - Foo (1.0\n - ".to_vec(), + br#"{"pins":[{"identity":"a","state":{"ver"#.to_vec(), + ), + ( + "not UTF-8", + b"PODS:\n - \xff\xfe (1.0)\n".to_vec(), + b"\xff".to_vec(), + ), + ( + "nameless pod lines", + b"PODS:\n - (1.0)\n - \n - Real (2.0)\n".to_vec(), + Vec::new(), + ), + ("100k-deep JSON", Vec::new(), deep), + ("50k pods", big_lock, Vec::new()), + ]; + for (label, pods, pins) in cases { + let doc = ios_deps(&pods, &pins); + let entries = doc["entries"] + .as_array() + .unwrap_or_else(|| panic!("{label}: no entries array")); + for e in entries { + let name = e["name"].as_str().unwrap_or(""); + assert!( + !name.is_empty(), + "{label}: nameless dependency in the graph: {e}" + ); + assert_ne!(e["id"], "library::", "{label}"); + } + } + // The survivable line of a mangled lockfile is still reported. + let doc = ios_deps(b"PODS:\n - (1.0)\n - \n - Real (2.0)\n", b""); + let names: Vec<&str> = doc["entries"] + .as_array() + .unwrap() + .iter() + .map(|e| e["name"].as_str().unwrap()) + .collect(); + assert_eq!(names, ["Real"]); +} + +// --------------------------------------------------------------------------- +// Build registration payloads and artefacts +// --------------------------------------------------------------------------- + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_broken_build_payload_fails_before_anything_is_sent() { + let server = MockServer::start().await; + Mock::given(method("POST")) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({"ok": true}))) + .expect(0) + .mount(&server) + .await; + Mock::given(method("PUT")) + .respond_with(ResponseTemplate::new(200)) + .expect(0) + .mount(&server) + .await; + let tmp = tempfile::tempdir().unwrap(); + let artifact = write(tmp.path(), "app.aab", b"PK\x03\x04 fake"); + let endpoint = server.uri(); + let root = tmp.path().to_path_buf(); + tokio::task::spawn_blocking(move || { + for (label, payload) in [ + ("empty", &b""[..]), + ("truncated", br#"{"uuid":"a","vers"#), + ("an array", b"[]"), + ("null", b"null"), + ("binary", &[0xff, 0xfe, 0x00, 0x01]), + ] { + let p = write(&root, "p.json", payload); + let out = common::cli() + .args([ + "--endpoint", + &endpoint, + "--app-token", + "TKN", + "upload", + "build", + ]) + .arg("--payload-json") + .arg(&p) + .arg("--artifact") + .arg(&artifact) + .output() + .unwrap(); + assert_eq!(code_of(&out), 11, "{label}: {}", stderr(&out)); + assert!( + stderr(&out).contains("--payload-json"), + "{label}: {}", + stderr(&out) + ); + } + }) + .await + .unwrap(); +} + +#[test] +fn unusable_build_artifacts_are_input_not_found() { + let tmp = tempfile::tempdir().unwrap(); + let payload = write(tmp.path(), "p.json", br#"{"uuid":"a"}"#); + let dir = tmp.path().join("a_directory"); + std::fs::create_dir_all(&dir).unwrap(); + for (label, artifact) in [ + ("a directory", dir), + ("a missing file", tmp.path().join("nope.aab")), + ] { + let out = common::cli() + .args([ + "--endpoint", + "http://127.0.0.1:1", + "--app-token", + "TKN", + "upload", + "build", + "--dry-run", + ]) + .arg("--payload-json") + .arg(&payload) + .arg("--artifact") + .arg(&artifact) + .output() + .unwrap(); + assert_eq!(code_of(&out), 10, "{label}: {}", stderr(&out)); + } +} + +// --------------------------------------------------------------------------- +// IL2CPP line maps: a documented gap (characterisation) +// --------------------------------------------------------------------------- + +/// The IL2CPP mappings JSON is packed as-is and NOT validated: a corrupt file uploads +/// "successfully". This pins that on purpose — rejecting it would be a behaviour +/// change integrators must be told about — so a future validation lands deliberately. +#[test] +fn il2cpp_linemap_does_not_validate_the_mappings_json_today() { + let tmp = tempfile::tempdir().unwrap(); + for (label, bytes) in [ + ("empty", &b""[..]), + ("truncated", br#"{"a":"#), + ("binary", &[0xff, 0xfe, 0x00]), + ] { + let dir = tmp.path().join(label); + write(&dir, "LineNumberMappings.json", bytes); + let out = common::cli() + .args([ + "debug-files", + "upload", + "--type", + "il2cpp-linemap", + "--version", + "1", + "--build", + "1", + ]) + .args([ + "--uuid", + "11111111-2222-3333-4444-555555555555", + "--dry-run", + ]) + .arg(&dir) + .output() + .unwrap(); + assert_eq!(code_of(&out), 0, "{label}: {}", stderr(&out)); + } +} diff --git a/tests/file_handling.rs b/tests/file_handling.rs new file mode 100644 index 0000000..05e1bf1 --- /dev/null +++ b/tests/file_handling.rs @@ -0,0 +1,547 @@ +//! File reading, processing and output, end to end through the COMPILED binary. +//! +//! The unit tests pin each piece; these pin what an integrator can observe: the +//! exact bytes left on disk, the exact bytes sent over the wire (including large +//! payloads that must stream), stdout staying reserved for structured output, and +//! exit codes for unusable input. + +use std::io::Read; +use std::path::{Path, PathBuf}; + +use serde_json::{json, Value}; +use sha1::{Digest, Sha1}; +use wiremock::matchers::{method, path as wm_path}; +use wiremock::{Mock, MockServer, ResponseTemplate}; + +mod common; + +const FIXTURE_ELF: &str = "tests/fixtures/elf/libsymbol1.so"; +const FIXTURE_BUILD_ID: &str = "bca64abfec40dbb631bb8f1c37414472"; + +fn write(dir: &Path, rel: &str, bytes: &[u8]) -> PathBuf { + let p = dir.join(rel); + std::fs::create_dir_all(p.parent().unwrap()).unwrap(); + std::fs::write(&p, bytes).unwrap(); + p +} + +fn sha1_hex(bytes: &[u8]) -> String { + hex::encode(Sha1::digest(bytes)) +} + +/// Patterned, non-constant bytes (a constant run would hide a chunking bug). +fn pattern(len: usize) -> Vec { + (0..len).map(|i| ((i * 31 + i / 251) % 256) as u8).collect() +} + +/// The entries of an uploaded ZIP, decompressed, in order. +fn zip_entries(zip_bytes: &[u8]) -> Vec<(String, Vec)> { + let mut archive = zip::ZipArchive::new(std::io::Cursor::new(zip_bytes.to_vec())).unwrap(); + (0..archive.len()) + .map(|i| { + let mut e = archive.by_index(i).unwrap(); + let mut data = Vec::new(); + e.read_to_end(&mut data).unwrap(); // verifies the CRC + (e.name().to_string(), data) + }) + .collect() +} + +async fn requests(server: &MockServer, verb: &str) -> Vec { + server + .received_requests() + .await + .unwrap() + .into_iter() + .filter(|r| r.method.as_str() == verb) + .collect() +} + +// --------------------------------------------------------------------------- +// `sourcemaps inject`: bytes on disk, idempotence, stdout discipline, exit codes +// --------------------------------------------------------------------------- + +const MAP: &str = r#"{"version":3,"sources":["a.ts"],"names":[],"mappings":"AAAA"}"#; + +#[test] +fn inject_stamps_files_idempotently_and_keeps_stdout_empty() { + let tmp = tempfile::tempdir().unwrap(); + let js = write(tmp.path(), "dist/app.js", b"console.log(1);\n"); + let map = write(tmp.path(), "dist/app.js.map", MAP.as_bytes()); + let dist = tmp.path().join("dist"); + + let out = common::cli() + .args(["sourcemaps", "inject"]) + .arg(&dist) + .assert() + .success() + .stdout("") // stdout is reserved for structured output + .get_output() + .clone(); + assert!( + String::from_utf8_lossy(&out.stderr).contains("inject"), + "progress goes to stderr" + ); + + let stamped_js = std::fs::read_to_string(&js).unwrap(); + assert!( + stamped_js.starts_with("console.log(1);\n"), + "original bytes are a prefix" + ); + assert!(stamped_js.contains("//# debugId=")); + let map_json: Value = serde_json::from_slice(&std::fs::read(&map).unwrap()).unwrap(); + assert_eq!(map_json["debug_id"], map_json["debugId"]); + assert_eq!(map_json["mappings"], "AAAA"); + + // Second run: no byte changes at all. + let (js1, map1) = (std::fs::read(&js).unwrap(), std::fs::read(&map).unwrap()); + common::cli() + .args(["sourcemaps", "inject"]) + .arg(&dist) + .assert() + .success(); + assert_eq!(std::fs::read(&js).unwrap(), js1); + assert_eq!(std::fs::read(&map).unwrap(), map1); +} + +#[test] +fn inject_dry_run_reports_but_writes_nothing() { + let tmp = tempfile::tempdir().unwrap(); + let js = write(tmp.path(), "app.js", b"a()\n"); + let map = write(tmp.path(), "app.js.map", MAP.as_bytes()); + common::cli() + .args(["sourcemaps", "inject", "--dry-run"]) + .arg(tmp.path()) + .assert() + .success() + .stdout(""); + assert_eq!(std::fs::read(&js).unwrap(), b"a()\n"); + assert_eq!(std::fs::read(&map).unwrap(), MAP.as_bytes()); + let mut left: Vec<_> = std::fs::read_dir(tmp.path()) + .unwrap() + .map(|e| e.unwrap().file_name().into_string().unwrap()) + .collect(); + left.sort(); + assert_eq!(left, ["app.js", "app.js.map"]); +} + +#[test] +fn inject_rejects_an_unusable_map_and_leaves_it_alone() { + use predicates::prelude::PredicateBooleanExt; + let tmp = tempfile::tempdir().unwrap(); + let js = write(tmp.path(), "app.js", b"a()\n"); + let map = write(tmp.path(), "app.js.map", b"{ not json"); + common::cli() + .args(["sourcemaps", "inject"]) + .arg(tmp.path()) + .assert() + .code(11) // InputInvalid + .stderr(predicates::str::contains("not valid JSON")) + .stderr(predicates::str::contains("injected debug-id").not()); + assert_eq!(std::fs::read(&map).unwrap(), b"{ not json"); + assert_eq!( + std::fs::read(&js).unwrap(), + b"a()\n", + "a failed inject must not leave a stamped bundle behind" + ); +} + +/// Rewriting a map must not depend on `$TMPDIR` having room (or existing): in CI containers +/// it is often a tiny tmpfs. The scratch file lives beside the map. +#[test] +fn inject_does_not_need_tmpdir() { + let tmp = tempfile::tempdir().unwrap(); + write(tmp.path(), "app.js", b"a()\n"); + let map = write(tmp.path(), "app.js.map", MAP.as_bytes()); + common::cli() + .args(["sourcemaps", "inject"]) + .arg(tmp.path()) + .env("TMPDIR", "/nonexistent/tmp") + .env("TMP", "/nonexistent/tmp") + .env("TEMP", "/nonexistent/tmp") + .assert() + .success(); + assert!(String::from_utf8(std::fs::read(&map).unwrap()) + .unwrap() + .contains("debug_id")); +} + +// --------------------------------------------------------------------------- +// stdout discipline for `debug-files upload` +// --------------------------------------------------------------------------- + +#[test] +fn upload_dry_runs_write_nothing_to_stdout_for_every_file_type() { + let tmp = tempfile::tempdir().unwrap(); + let elf = Path::new(env!("CARGO_MANIFEST_DIR")).join(FIXTURE_ELF); + write( + tmp.path(), + "native/lib/arm64-v8a/libsymbol1.so", + &std::fs::read(elf).unwrap(), + ); + write(tmp.path(), "proguard/mapping.txt", b"a.B -> c.D:\n"); + write( + tmp.path(), + "maps/app.js.map", + br#"{"version":3,"debug_id":"22222222-2222-2222-2222-222222222222","mappings":"AAAA"}"#, + ); + for (kind, dir) in [ + ("elf", "native"), + ("proguard", "proguard"), + ("sourcemaps", "maps"), + ] { + common::cli() + .args([ + "debug-files", + "upload", + "--type", + kind, + "--version", + "1", + "--build", + "1", + ]) + .args([ + "--uuid", + "00000000-0000-0000-0000-000000000000", + "--dry-run", + ]) + .arg(tmp.path().join(dir)) + .assert() + .success() + .stdout(""); + } +} + +// --------------------------------------------------------------------------- +// What goes over the wire for large payloads +// --------------------------------------------------------------------------- + +/// The artefact is the biggest thing the CLI sends: it must arrive intact, with an +/// exact Content-Length (no chunked body), whatever its size relative to the stream +/// chunk — here 5 MB, STORED inside the upload ZIP. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_large_artefact_arrives_intact_with_an_exact_content_length() { + let server = MockServer::start().await; + let tmp = tempfile::tempdir().unwrap(); + let artefact = pattern(5 * 1024 * 1024 + 17); + let artifact_path = write(tmp.path(), "app.aab", &artefact); + let payload = write(tmp.path(), "p.json", br#"{"uuid":"abc","version":"1.0"}"#); + let put_url = format!("{}/artefact-put", server.uri()); + + Mock::given(method("POST")) + .and(wm_path("/v2/apps/TKN/builds")) + .respond_with( + ResponseTemplate::new(200).set_body_json( + json!({"ok": true, "result": {"build_id": "b1", "endpoint": put_url}}), + ), + ) + .expect(1) + .mount(&server) + .await; + Mock::given(method("PUT")) + .and(wm_path("/artefact-put")) + .respond_with(ResponseTemplate::new(200)) + .expect(1) + .mount(&server) + .await; + + let endpoint = server.uri(); + tokio::task::spawn_blocking(move || { + common::cli() + .args([ + "--endpoint", + &endpoint, + "--app-token", + "TKN", + "upload", + "build", + ]) + .arg("--payload-json") + .arg(&payload) + .arg("--artifact") + .arg(&artifact_path) + .assert() + .success(); + }) + .await + .unwrap(); + + let put = &requests(&server, "PUT").await[0]; + assert_eq!( + put.headers.get("content-length").unwrap().to_str().unwrap(), + put.body.len().to_string() + ); + assert!(put.headers.get("transfer-encoding").is_none()); + let entries = zip_entries(&put.body); + assert_eq!(entries.len(), 1); + assert_eq!(entries[0].0, "app.aab"); + assert_eq!( + sha1_hex(&entries[0].1), + sha1_hex(&artefact), + "artefact bytes survive the stream" + ); +} + +/// A directory of large libraries: each upload ZIP holds exactly that library's +/// bytes, registered under that library's OWN build-id. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn directory_libraries_upload_with_their_own_bytes_and_build_ids() { + let server = MockServer::start().await; + let tmp = tempfile::tempdir().unwrap(); + let fixture = std::fs::read(Path::new(env!("CARGO_MANIFEST_DIR")).join(FIXTURE_ELF)).unwrap(); + // Two libraries with different build-ids (flip the id's first byte), each padded + // past the 64 KiB stream chunk so the PUT spans several chunks. + let id = hex::decode(FIXTURE_BUILD_ID).unwrap(); + let pos = fixture.windows(id.len()).position(|w| w == id).unwrap(); + let mut other = fixture.clone(); + other[pos] ^= 0xff; + let other_id = hex::encode(&other[pos..pos + id.len()]); + let (mut a, mut b) = (fixture.clone(), other.clone()); + a.extend(pattern(700_000)); + b.extend(pattern(900_001)); + write(tmp.path(), "libs/lib/arm64-v8a/libfoo.so", &a); + write(tmp.path(), "libs/lib/x86_64/libfoo.so", &b); + + let put_url = format!("{}/put", server.uri()); + Mock::given(method("POST")) + .and(wm_path("/apps/TKN/symbols")) + .respond_with( + ResponseTemplate::new(200).set_body_json(json!({"code": 0, "endpoint": put_url})), + ) + .expect(2) + .mount(&server) + .await; + Mock::given(method("PUT")) + .respond_with(ResponseTemplate::new(200)) + .expect(2) + .mount(&server) + .await; + + let endpoint = server.uri(); + let libs = tmp.path().join("libs"); + tokio::task::spawn_blocking(move || { + common::cli() + .args(["--endpoint", &endpoint, "--app-token", "TKN"]) + .args([ + "debug-files", + "upload", + "--type", + "elf", + "--version", + "1", + "--build", + "1", + ]) + .args(["--uuid", "00000000-0000-0000-0000-000000000000"]) + .arg(&libs) + .assert() + .success(); + }) + .await + .unwrap(); + + let posts: Vec = requests(&server, "POST") + .await + .iter() + .map(|r| serde_json::from_slice(&r.body).unwrap()) + .collect(); + let ids: Vec<&str> = posts.iter().map(|p| p["uuid"].as_str().unwrap()).collect(); + assert!( + ids.contains(&FIXTURE_BUILD_ID) && ids.contains(&other_id.as_str()), + "{ids:?}" + ); + for p in &posts { + assert_eq!(p["format"], "elf"); + assert_eq!(p["transform"], "breakpad"); + } + // Every PUT body is a zip whose single entry is one of the two libraries, whole, + // and the `hash` registered for it is the SHA-1 of the ZIP that was uploaded. + let puts = requests(&server, "PUT").await; + let mut seen = Vec::new(); + for put in &puts { + let entries = zip_entries(&put.body); + assert_eq!(entries.len(), 1); + assert_eq!(entries[0].0, "libfoo.so"); + seen.push(sha1_hex(&entries[0].1)); + assert!( + posts.iter().any(|p| p["hash"] == sha1_hex(&put.body)), + "registered hash must describe the uploaded ZIP" + ); + assert_eq!( + put.headers.get("content-length").unwrap().to_str().unwrap(), + put.body.len().to_string() + ); + } + seen.sort(); + let mut want = vec![sha1_hex(&a), sha1_hex(&b)]; + want.sort(); + assert_eq!(seen, want); +} + +/// `--strip-sources-content`: what is uploaded has no source and is still a valid +/// map; what is on disk is untouched; the registered hash is of the uploaded bytes. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn strip_sources_content_uploads_a_valid_stripped_copy_and_spares_the_original() { + let server = MockServer::start().await; + let tmp = tempfile::tempdir().unwrap(); + let original = json!({ + "version": 3, + "debug_id": "33333333-3333-3333-3333-333333333333", + "sources": ["a.ts", "b.ts"], + "sourcesContent": ["const secret = 1;\n", "x".repeat(300_000)], + "names": [], + "mappings": "AAAA;AACA", + }); + let bytes = serde_json::to_vec(&original).unwrap(); + write(tmp.path(), "maps/app.js.map", &bytes); + + let put_url = format!("{}/put", server.uri()); + Mock::given(method("POST")) + .and(wm_path("/apps/TKN/symbols")) + .respond_with( + ResponseTemplate::new(200).set_body_json(json!({"code": 0, "endpoint": put_url})), + ) + .expect(1) + .mount(&server) + .await; + Mock::given(method("PUT")) + .respond_with(ResponseTemplate::new(200)) + .expect(1) + .mount(&server) + .await; + + let endpoint = server.uri(); + let dir = tmp.path().join("maps"); + tokio::task::spawn_blocking(move || { + common::cli() + .args(["--endpoint", &endpoint, "--app-token", "TKN"]) + .args([ + "debug-files", + "upload", + "--type", + "sourcemaps", + "--strip-sources-content", + ]) + .args(["--version", "1", "--build", "1"]) + .arg(&dir) + .assert() + .success(); + }) + .await + .unwrap(); + + let entries = zip_entries(&requests(&server, "PUT").await[0].body); + assert_eq!(entries.len(), 1); + let uploaded: Value = serde_json::from_slice(&entries[0].1).unwrap(); + let mut expected = original.clone(); + expected.as_object_mut().unwrap().remove("sourcesContent"); + assert_eq!(uploaded, expected, "exactly the map minus its sources"); + + let post: Value = serde_json::from_slice(&requests(&server, "POST").await[0].body).unwrap(); + assert_eq!( + post["uuid"], "33333333-3333-3333-3333-333333333333", + "the key is carried over" + ); + assert_eq!( + post["hash"], + sha1_hex(&entries[0].1), + "hash describes what was uploaded" + ); + assert_eq!( + std::fs::read(tmp.path().join("maps/app.js.map")).unwrap(), + bytes, + "the user's own map is not rewritten" + ); +} + +/// A ProGuard mapping goes out as ONE zstd entry whose bytes are the file's — also +/// when the file is empty or not valid UTF-8 (identity is over raw bytes). +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn proguard_mappings_round_trip_raw_bytes() { + for (label, content) in [ + ("empty", Vec::new()), + ("crlf", b"a.B -> c.D:\r\n int x -> a\r\n".to_vec()), + ("non-utf8", b"a.B -> c.D:\n\xff\xfe\x00\x01\n".to_vec()), + ("large", pattern(400_003)), + ] { + let server = MockServer::start().await; + let tmp = tempfile::tempdir().unwrap(); + write(tmp.path(), "pg/mapping.txt", &content); + let put_url = format!("{}/put", server.uri()); + Mock::given(method("POST")) + .and(wm_path("/apps/TKN/symbols")) + .respond_with( + ResponseTemplate::new(200).set_body_json(json!({"code": 0, "endpoint": put_url})), + ) + .mount(&server) + .await; + Mock::given(method("PUT")) + .respond_with(ResponseTemplate::new(200)) + .mount(&server) + .await; + let endpoint = server.uri(); + let dir = tmp.path().join("pg"); + tokio::task::spawn_blocking(move || { + common::cli() + .args(["--endpoint", &endpoint, "--app-token", "TKN"]) + .args(["debug-files", "upload", "--type", "proguard"]) + .args(["--version", "1", "--build", "1"]) + .arg(&dir) + .assert() + .success(); + }) + .await + .unwrap(); + + let entries = zip_entries(&requests(&server, "PUT").await[0].body); + assert_eq!( + entries, + [("mapping.txt".to_string(), content.clone())], + "{label}" + ); + let post: Value = serde_json::from_slice(&requests(&server, "POST").await[0].body).unwrap(); + // Java `UUID.nameUUIDFromBytes`: MD5 of the raw bytes, version-3 / IETF bits. + let mut md5 = ::digest(&content); + md5[6] = (md5[6] & 0x0f) | 0x30; + md5[8] = (md5[8] & 0x3f) | 0x80; + assert_eq!( + post["uuid"].as_str().unwrap().replace('-', ""), + hex::encode(md5), + "{label}: debug-id is over the raw bytes" + ); + } +} + +/// An empty/zero-byte library inside a scanned directory must not crash the scan +/// (it cannot be memory-mapped): it is reported as having no build-id and skipped. +#[test] +fn a_zero_byte_library_in_a_directory_is_skipped_not_fatal() { + let tmp = tempfile::tempdir().unwrap(); + let fixture = std::fs::read(Path::new(env!("CARGO_MANIFEST_DIR")).join(FIXTURE_ELF)).unwrap(); + write(tmp.path(), "libs/libok.so", &fixture); + write(tmp.path(), "libs/libempty.so", b""); + common::cli() + .args([ + "debug-files", + "upload", + "--type", + "elf", + "--version", + "1", + "--build", + "1", + ]) + .args([ + "--uuid", + "00000000-0000-0000-0000-000000000000", + "--dry-run", + ]) + .arg(tmp.path().join("libs")) + .assert() + .success() + .stderr(predicates::str::contains("no GNU build-id")) + .stderr(predicates::str::contains( + "would register + upload 1 libraries", + )); +}