e2e(campaign): API, option and appearance device tests (N-02..N-09, N-11, N-13, N-14)

[krassx]

Campaign PREP-api: device tests for plan items N-02, N-03, N-04, N-05, N-06, N-07, N-08, N-09, N-11, N-13 and N-14 (beta-campaign-plan.md section 3.1). Every new test title carries its plan ID(s) in brackets, so the evidence collector maps it.

What

• scenarios/api*.ts(x): the api-* scenarios and a dispatcher; App.tsx gets additive hooks only.
• New suites: attach, relaunch, secure-rectangles-api, console-levels, options-readback, option-effects, api-arguments, lifecycle-events, sdk-breadcrumbs, report-dialog-keys, feedback-keys.
• -PbugseeE2eAutoLaunch=true (Android debug): merges com.bugsee.app-token (placeholder) + dead $$ENDPOINT meta-data for N-02's manifest auto-launch. Off by default.
• harness.ts: wait for Metro to serve the run's scenario before every Debug launch (Android and iPhone too). A Debug app reloads when e2e-scenario.json changes after launch, restarting the scenario mid-run.
• Issue URLs (bugsee-cocoa#195..#200, bugsee-android#214..#216) added to the existing it.failing pins.

Proof (smallest runs, WOD_LX1 / simulator / iPhone XS)

All suites pass or fail only on pinned, evidenced product bugs; per-item, per-target status in .superpowers/.../campaign-prep-api.md.

New product bugs, pinned it.failing and listed "to file" in sdk-issues-filed.md:

• Android 7.3.0: Report ActionBarColor never applied; Feedback ActionBarColor / EmailContinueNotActiveColor never read, InputTextHintColor ignored; no SDK-generated breadcrumbs; detect.crash=false still reports NDK crashes; report-processing-in-process=false files no report.
• iOS beta5: report close/send button colours never applied; feedback has no e-mail screen; detect.kill not implemented; body-without-type=false still records untyped bodies.
• Wrapper (to fix): iOS native-only RCTLog line recorded 3x; blank-token rejection lacks E_TOKEN.

Plan correction: both SDKs fire AfterReportShown when the report UI is shown (documented), not on dismissal; EVT-09 asserts that, no tap needed.

Not covered (reasons in the tests and the doc): EVT-18 order, FB-06 on the XS, OPT-022 notify-flush (affects the POST only), OPT-062 websocket (RN's SocketRocket not captured), OPT-098 broadcast trigger (no in-app sender).

yarn lint, yarn typecheck, yarn test (3247) pass. No product logic changed (no Stryker run).

🤖 Generated with Claude Code

[cursor]

Stale comment

Deep code review (campaign API / option / appearance e2e)

Test-only PR: N-02..N-09, N-11, N-13, N-14 device cases. No product SDK logic. Reviewed against surrounding harness (startRun / awaitMetroServes), span-lifecycle.test.ts capture shape, Jest 29 testPathPattern, and CI run 37585858557 (ios e2e (simulator) job 112675691361).

P1 High — relaunch.test.ts is executed by CI's launch-only Jest regex

Location: examples/bare/e2e/relaunch.test.ts:51 (new file). Triggered by existing .github/workflows/ci.yml (~393): yarn e2e launch.test.ts.

Problem: Jest 29 treats a positional path as a regex, not an exact filename. yarn e2e launch.test.ts becomes /launch.test.ts/i. The new file relaunch.test.ts contains that substring, so the job that comments "launch.test.ts only" now also runs this campaign suite.

Impact: The "Reach Status.Launched" gate is no longer launch-only. On this head it is red: launch.test.ts PASS, then relaunch.test.ts FAIL ×5. Log: Ran all test suites matching /launch.test.ts/i. First startRun('api-relaunch') already dies in awaitMetroServes (Metro never served a bundle carrying nonce … last: HTTP 200, 4751988 bytes without the nonce). The job no longer means what the workflow comment says, and a required check fails.

Scenario: ios e2e (simulator) runs yarn e2e launch.test.ts with maxWorkers: 1. Jest picks launch.test.ts then relaunch.test.ts. After the launch suite, Metro still serves the previous ~4.7 MB bundle; the new nonce never appears within 60s; the relaunch beforeAll throws.

Fix: Do both, or at least one that uniquely selects launch.test.ts:

1. CI: --testPathPattern='(^|/)launch\.test\.ts$' (or --runTestsByPath e2e/launch.test.ts).
2. This PR: rename to e.g. api-relaunch.test.ts so the existing pattern cannot swallow it.

P2 Medium — [API-19f] never asserts child parentage

Location: examples/bare/e2e/api-arguments.test.ts:312-320 (lookup at 314; guard at 318-319).

Problem: Two independent misses, either of which skips the parentage expect:

1. span() matches record.description === …. startTransaction(\api-txn-${nonce}`, 'api.flow')stores that string as the transaction **name**, not description.span-lifecycle.test.tsalready treatsnamevsdescriptionseparately and finds the root byparentSpanId === undefined`.
2. The guard reads OTLP snake_case (span_id / parent_span_id). Device captures in this repo are camelCase (spanId / parentSpanId) — see span-lifecycle.test.ts:26-27, 92, 99.

The remaining asserts only require the child to exist and the name string to appear somewhere in the JSON.

Impact: API-19f is a false green. A child with the wrong parent, or no parent link, still passes.

Scenario: Performance capture has name: "api-txn-<nonce>" and child { description: "child-<nonce>", parentSpanId: "…", spanId: "…" }. parent is undefined (no description match) or found but span_id is undefined. The if is skipped; the test passes.

Fix: Find the parent by name (or the root span with parentSpanId === undefined in that transaction). Assert child.parentSpanId === parent.spanId. Drop the optional snake_case guard so a missing link fails.

---

Verdict

1. Overall risk: Medium (CI gate broken; one campaign assertion is a no-op. No product-code defect.)
2. Merge recommendation: Request changes. Do not merge while ios e2e (simulator) is red for this filename collision.
3. Most important to fix: (1) stop /launch.test.ts/i matching relaunch.test.ts; (2) make API-19f assert camelCase parentage for real.
4. Positives: Honest it.failing pins; Android Debug awaitMetroServes before launch (avoids mid-run Metro reload); autolaunch overlay is a dedicated flavor, not a silent default; N-03 screenshot-before/after relaunch is a real option-effect check.

  

_{Sent by Cursor Automation: Bugsee code review}

[cursor]

Stale comment

Deep code review (re-review of c650bc9)

Test-only PR: N-02..N-09, N-11, N-13, N-14 device cases. No product SDK logic. Rechecked the two prior findings against Jest 29.7 testPathPatternToRegExp (absolute test.path), span-lifecycle.test.ts capture shape, and the surrounding harness (startRun / awaitMetroServes).

Prior findings

• P1 fully addressed. Suite is api-relaunch.test.ts. CI runs yarn e2e '/launch\.test\.ts$'. Jest 29 compiles that to new RegExp('/launch\\.test\\.ts$', 'i') and matches absolute paths, so the path must end in /launch.test.ts. .../e2e/launch.test.ts matches; .../e2e/api-relaunch.test.ts does not (relaunch is not /launch after the last slash). The previous /launch.test.ts/i collision is gone.
• P2 fully addressed. [API-19f] finds the transaction by name, takes the root of spans with no parentSpanId, and asserts child.parentSpanId === root.spanId unconditionally (camelCase, same as span-lifecycle.test.ts).

No new P0–P2. I did not invent a defect from CI lint, typecheck, unit on run 37588305266: that job is red on this SHA, but the log is not readable while sibling jobs are still running. The previous head (e86998e) passed that job. Treat a still-red lint job as a merge gate, not as a reviewed code bug.

---

Verdict

1. Overall risk: Low (prior CI/assertion defects fixed; remaining work is campaign e2e. No product-code defect.)
2. Merge recommendation: Approve the review findings. Do not merge while lint, typecheck, unit is red; confirm ios e2e (simulator) stays launch-only.
3. Most important remaining: Wait for a green lint, typecheck, unit on this SHA. Nothing else from this review.
4. Positives: Honest it.failing pins with filed SDK issues; Android Debug waits for Metro before launch (avoids mid-run reload); autolaunch overlay is a dedicated flavor, not a silent default; N-03 screenshot-before/after relaunch is a real option-effect check; API-19f now matches how the capture is actually shaped.

  

_{Sent by Cursor Automation: Bugsee code review}

[cursor]

Stale comment

Deep code review (re-review of 7d6ca55)

Test-only PR: N-02..N-09, N-11, N-13, N-14 device cases. No product SDK logic. Rechecked the full diff against the harness (startRun / awaitMetroServes), span-lifecycle.test.ts capture shape, Jest 29 testPathPattern (absolute test.path), the Android autolaunch overlay, and the campaign scenarios App.tsx dispatches.

Prior findings

• P1 fully addressed. Suite is api-relaunch.test.ts. CI runs yarn e2e '/launch\.test\.ts$'. Jest 29 compiles that to new RegExp('/launch\\.test\\.ts$', 'i') on the absolute path, so the path must end in /launch.test.ts. .../e2e/api-relaunch.test.ts does not (relaunch is not /launch after the last slash). example-wiring.test.ts now pins that command and asserts the regex selects only launch.test.ts among the e2e suites.
• P2 fully addressed. [API-19f] finds the transaction by name, takes the root of spans with no parentSpanId, and asserts child.parentSpanId === root.spanId unconditionally (camelCase, same as span-lifecycle.test.ts).

lint, typecheck, unit was red on c650bc9 (37588305266) because the wiring test still expected yarn e2e launch.test.ts. This SHA updates that pin; the same job is success on 37589560526.

No new P0–P2. Re-read attach/pre-launch ownership, option-effect controls, it.failing pins, and the autolaunch debug source-set overlay; nothing there is a false green or a product-path defect.

---

Verdict

1. Overall risk: Low (prior CI/assertion defects fixed; lint gate green; remaining work is campaign e2e. No product-code defect.)
2. Merge recommendation: Approve.
3. Most important issues to fix: None from this review.
4. Positives: Honest it.failing pins with filed SDK issues; Android Debug waits for Metro before launch (avoids mid-run reload); autolaunch overlay is a dedicated Gradle property, not a silent default; N-03 screenshot-before/after relaunch is a real option-effect check; API-19f now matches how the capture is actually shaped; the launch-only CI step is regression-tested against the e2e directory.

  

_{Sent by Cursor Automation: Bugsee code review}

[cursor]

Stale comment

Deep code review (re-review of 836ee32)

Test-only PR: N-02..N-09, N-11, N-13, N-14 device cases. No product SDK logic. Rechecked the full diff against the harness (startRun / awaitMetroServes), span-lifecycle.test.ts capture shape, Jest 29 testPathPattern (absolute test.path), the Android autolaunch overlay, App.tsx's isApiScenario dispatcher, and the option-effect / appearance suites.

Delta since 7d6ca55

836ee32 only rewrites it.failing titles and known-bug comments to cite the filed issues (bugsee-cocoa#201..#204, bugsee-android#217..#221). No assertion, harness, CI, or scenario-control flow change.

Prior findings

• P1 fully addressed. Suite is api-relaunch.test.ts. CI runs yarn e2e '/launch\.test\.ts$'. Jest 29 compiles that to new RegExp('/launch\\.test\\.ts$', 'i') on the absolute path, so the path must end in /launch.test.ts. .../e2e/api-relaunch.test.ts does not. example-wiring.test.ts still pins that command and asserts the regex selects only launch.test.ts among the e2e suites.
• P2 fully addressed. [API-19f] finds the transaction by name, takes the root of spans with no parentSpanId, and asserts child.parentSpanId === root.spanId unconditionally (camelCase, same as span-lifecycle.test.ts).

No new P0–P2. Re-read attach/pre-launch ownership, option-effect controls vs defaults, it.failing pins, and the autolaunch debug source-set overlay; nothing there is a false green or a product-path defect.

---

Verdict

1. Overall risk: Low (prior CI/assertion defects fixed; this SHA is issue-link comments only. No product-code defect.)
2. Merge recommendation: Approve.
3. Most important issues to fix: None from this review.
4. Positives: Honest it.failing pins now point at filed SDK issues; Android Debug waits for Metro before launch (avoids mid-run reload); autolaunch overlay is a dedicated Gradle property, not a silent default; N-03 screenshot-before/after relaunch is a real option-effect check; API-19f matches how the capture is actually shaped; the launch-only CI step is regression-tested against the e2e directory.

  

_{Sent by Cursor Automation: Bugsee code review}

[cursor]

Stale comment

Deep code review (re-review of 7a41147)

Test-only PR: N-02..N-09, N-11, N-13, N-14 device cases. No product SDK logic. Rechecked the full diff after the rebase onto #62 (794b780): App.tsx now dispatches both flow-* (FlowStage) and api-* (ApiStage); the names do not overlap. Also rechecked startRun / awaitMetroServes, Jest 29 testPathPattern, the Android autolaunch overlay, API-19f parentage, and the option-effect / appearance suites.

Prior findings

• P1 fully addressed. Suite is api-relaunch.test.ts. CI runs yarn e2e '/launch\.test\.ts$'. Jest 29 compiles that to new RegExp('/launch\\.test\\.ts$', 'i') on the absolute path, so the path must end in /launch.test.ts. .../e2e/api-relaunch.test.ts does not (re sits in front of launch). example-wiring.test.ts still pins that command and asserts the regex selects only launch.test.ts among the e2e suites.
• P2 fully addressed. [API-19f] finds the transaction by name, takes the root of spans with no parentSpanId, and asserts child.parentSpanId === root.spanId unconditionally (camelCase, same as span-lifecycle.test.ts).

New finding

One P2 in the OPT-025 pin: Android it.failing wraps a JS fatal and an NDK crash behind a single empty-bundle assertion, so the JS half of detect.crash=false cannot go red. OPT-081 in the same file already splits read-back (it) from the known no-bundle bug (it.failing); this case should too.

---

Verdict

1. Overall risk: Low (test-only; prior CI/assertion defects still fixed; one campaign pin cannot catch a JS-crash-detection regression on Android.)
2. Merge recommendation: Approve. Split OPT-025 before treating that row as device proof of the JS path.
3. Most important issue to fix: OPT-025 — separate the JS-fatal assertion (it on Android) from the NDK it.failing pin (bugsee-android#220).
4. Positives: Honest it.failing pins with filed SDK issues; Android/iPhone Debug waits for Metro before launch (avoids mid-run reload); autolaunch overlay is a dedicated Gradle property, not a silent default; N-03 screenshot-before/after relaunch is a real option-effect check; API-19f matches the capture shape; the launch-only CI step is still regression-tested against the e2e directory; rebase onto the flow dispatcher is clean.

  

_{Sent by Cursor Automation: Bugsee code review}

[cursor]

Stale comment

Deep code review (re-review of 80f9fb2)

Test-only PR: N-02..N-09, N-11, N-13, N-14 device cases. No product SDK logic. Rechecked the full diff after the OPT-025 split, including startRun / awaitMetroServes, Jest 29 testPathPattern, the Android autolaunch overlay, API-19f parentage, App.tsx flow-* vs api-* dispatch, attach/pre-launch ownership, and the option-effect / appearance suites.

Prior findings

• P1 fully addressed. Suite is api-relaunch.test.ts. CI runs yarn e2e '/launch\.test\.ts$'. Jest 29 compiles that to new RegExp('/launch\\.test\\.ts$', 'i') on the absolute path, so the path must end in /launch.test.ts. .../e2e/api-relaunch.test.ts does not. example-wiring.test.ts still pins that command and asserts the regex selects only launch.test.ts among the e2e suites.
• P2 fully addressed. [API-19f] finds the transaction by name, takes the root of spans with no parentSpanId, and asserts child.parentSpanId === root.spanId unconditionally (camelCase, same as span-lifecycle.test.ts).
• P2 fully addressed. [OPT-025] is two tests over the same launches. The JS fatal + idle launch is a plain it on A and X. The native crash + idle launch is it.failing on Android (bugsee-android#220) and a plain it on the XS.

New finding

One P2 of the same class as the OPT-025 pin: iOS [OPT-007] it.failing wraps the working option-on assertion with the known option-off leak (bugsee-cocoa#204), so a regression of "untyped body kept with the option on" cannot turn the test red. OPT-025/081 in this same file already split that way.

---

Verdict

1. Overall risk: Low (test-only; prior CI/assertion defects still fixed; one iOS option-effect pin cannot catch a body-without-type positive-path regression.)
2. Merge recommendation: Approve. Split OPT-007 before treating that iOS row as device proof of the option-on path.
3. Most important issue to fix: OPT-007 — separate option-on (it on A/S/X, body kept) from option-off (it.failing on iOS for bugsee-cocoa#204, it on Android).
4. Positives: OPT-025 is now a real JS-fatal gate on Android; honest it.failing pins with filed SDK issues; Android/iPhone Debug waits for Metro before launch; autolaunch overlay is a dedicated Gradle property; N-03 screenshot-before/after relaunch is a real option-effect check; API-19f matches the capture shape; rebase onto the flow dispatcher is still clean.

  

_{Sent by Cursor Automation: Bugsee code review}

[cursor]

Stale comment

Deep review (adbf04f)

Test-only campaign coverage (N-02..N-09, N-11, N-13, N-14). adbf04f splits OPT-007 the same way 80f9fb2 split OPT-025: option-on is a plain it on A/S/X; the default-off leak stays it.failing on iOS (bugsee-cocoa#204).

Prior threads (CI launch pattern, API-19f parentage, OPT-025, OPT-007) are fully_addressed.

Two remaining P2s of the same class: a working half still cannot turn its suite red.

Did not run device e2e (needs the handset / XS). Inspected the new suites against startRun, App.tsx dispatch, api-lifecycle / api-options / api-args, and the CI launch regex.

Findings

P2 FLOW-13 native RCTLog — console-levels.test.ts:78
P2 OPT-026 does not exercise detect.early-crash — option-effects.test.ts:191

Rechecked, not posted

• FLOW-15 Android it.failing still also asserts api-own-crumb; API-28 remains the addBreadcrumb gate.
• attach() treats anything other than Launched as idle; autolaunch still requires after=2.
• API-41c still asserts rolling-capture parts after delete(false), not that the uploaded bundle remains.
• notify() relay still asserts body/fields/sixth-arg, not severity/urgent.

Overall

1. Overall risk: Low
2. Merge recommendation: Approve; split FLOW-13 and isolate OPT-026 so those names are real gates.
3. Most important to fix: FLOW-13 Custom mapping as a plain it; OPT-026 needs an actually-early crash and/or an option-off control (today detect.crash recovers it).
4. Positive: OPT-007/025/081 splits are the right shape; API-19f parentage is unconditional; CI '/launch\.test\.ts$' is pinned in example-wiring.test.ts; product bugs stay it.failing with issue numbers rather than being weakened.

  

_{Sent by Cursor Automation: Bugsee code review}

[cursor]

Stale comment

Deep review (d3c59d0)

Test-only campaign coverage (N-02..N-09, N-11, N-13, N-14). d3c59d0 addresses both remaining P2s from adbf04f.

Prior findings

• FLOW-13 fully addressed. Same launch, two tests. reaches the report is a plain it on A/S/X: iOS asserts exactly one Custom (98) Warning (2) line (the bridge forward); Android asserts at least one logcat line. lands once and is filtered once stays it.failing on iOS (W1) and a plain it on Android.
• OPT-026 fully addressed. Skipped with the isolation reason; EFFECT_OPTIONS['early-crash'] removed. Read-back stays in N-06 (detect.early-crash: false). Staging S-5 for the synchronous send.
• Earlier threads (CI launch pattern, API-19f, OPT-025, OPT-007) remain fully_addressed.

No remaining P0–P3. Did not run device e2e.

Overall

1. Overall risk: Low
2. Merge recommendation: Approve
3. Most important issues to fix: None
4. Positive: The OPT-007/025/081/FLOW-13 splits are the right shape for it.failing pins; OPT-026 is an honest skip rather than a passing test that did not isolate the option.

  

_{Sent by Cursor Automation: Bugsee code review}

[cursor]

Deep review (d3c59d0)

Test-only campaign coverage (N-02..N-09, N-11, N-13, N-14). d3c59d0 addresses the two remaining P2s from adbf04f.

Prior threads (CI launch pattern, API-19f parentage, OPT-025, OPT-007, FLOW-13, OPT-026) are fully_addressed.

Did not run device e2e (needs the handset / XS). Inspected the new suites against startRun, App.tsx dispatch, api-lifecycle / api-options / api-args, Jest 29 testPathPattern, and the CI launch regex. Rechecked remaining it.failing pins for a working half that cannot go red; nothing left of that class that is not already gated elsewhere (API-28 for addBreadcrumb next to FLOW-15).

Prior findings at this SHA

P2 FLOW-13 — fully addressed. Two tests over the same launch. ‘reaches the report’ is a plain it on A, S and X (iOS: exactly one Custom (98) line at Warning (2)). ‘lands once and is filtered once’ stays it.failing on iOS (W1) and a plain it on Android. A G16 “fix” that drops RCTLog forwarding now turns the first test red.

P2 OPT-026 — fully addressed. Skipped with the isolation reason; early-crash is gone from EFFECT_OPTIONS. A JS-raised crash is after startRun saw Launched, where detect.crash recovers it with or without detect.early-crash. Read-back stays in N-06; the synchronous send is staging S-5.

No remaining P0–P3.

Rechecked, not posted

• FLOW-15 Android it.failing still also asserts api-own-crumb; API-28 remains the addBreadcrumb gate.
• attach() treats anything other than Launched as idle; autolaunch still requires after=2.
• API-41c still asserts rolling-capture parts after delete(false), not that the uploaded bundle remains.
• notify() relay still asserts body/fields/sixth-arg, not severity/urgent.
• $$ENDPOINT is the Android option key, not an AGP ${name} placeholder.

Overall

1. Overall risk: Low
2. Merge recommendation: Approve.
3. Most important to fix: None from this review.
4. Positive: FLOW-13/OPT-007/025/081 splits are the right shape; OPT-026 is an honest skip instead of a passing test that did not isolate the option; API-19f parentage is unconditional; CI '/launch\\.test\\.ts$' is pinned in example-wiring.test.ts; product bugs stay it.failing with issue numbers rather than being weakened.

  

_{Sent by Cursor Automation: Bugsee code review}