Skip to content

Close popup walls before research and stop testing them - #295

Merged
DavertMik merged 2 commits into
mainfrom
research-close-walls
Nov 6, 2026
Merged

DavertMik merged 2 commits into
mainfrom
research-close-walls

Conversation

@DavertMik

@DavertMik DavertMik commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Explorbot prefers modals as the main content source, so a cookie consent or marketing popup gets researched as the page and becomes a test target.

Change

  • Researcher: when an overlay is detected, the oracle checks whether it is a wall (cookie consent, paywall, marketing, announcement). If so, the existing cancelInUi() closes it and the page is re-captured before research starts. Regular modals stay open and are researched as before. The check and its statement live in a new WithWalls mixin (src/ai/researcher/walls.ts).
  • Planner: ignored-pages.md stops it planning tests for these popups, which also covers small banners that overlay detection ignores.

No overlay means no extra AI call.

Known limitations

  • cancelInUi() only clicks at (0, 0) and presses Escape, so a wall that needs Accept or Reject stays open. Nothing re-checks it, so research continues with the wall in place.
  • A wall with no dialog role that is already present on first load of a very different page isn't detected.
  • /plan cookie consent also has the wall closed before research, so Planner won't see it.

Testing

  • Integration: 176 passed, 1 skipped, 0 failed, including 2 new tests.
  • Unit: 8 failures (7 Provider, 1 ApibotConfigParser), the same 8 as on main without this change.
  • bun run build fails locally on chromium-bidi resolution, the same as on a clean origin/main.
  • Not yet run against a real model or a real consent popup. This needs a regression run, ideally on a site with a consent popup; approving it is left to the reviewer.

🤖 Generated with Claude Code

A cookie consent or marketing modal was treated as the page's main
content: Researcher described it and Planner targeted it first.

Researcher now asks the oracle whether a detected overlay is a wall, has
Navigator close it, and researches only once it is gone. A wall that
stays open raises ResearchBlockedError, so Captain takes over planning.
Regular modals stay open and are researched as before.

Planner no longer plans tests for such popups, and the shared popup rule
tells Tester, Navigator, Captain and Rerunner to dismiss or ignore them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the Navigator call, re-check, ResearchBlockedError and popup rule
with one step: when an overlay is detected, ask the oracle whether it is
a wall and close it with the existing cancelInUi. The check lives in a
WithWalls mixin next to its statement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown

Explorbot Self-Regression

Commit 06e19f7 · run

Scenario Result Attempts Time Previous median Change
basic (native) PASS 1/3 5m 3s 5m 34s -9%
experience: control OK — failed as expected 1/1 24s 28s -14%
experience: seeded PASS 1/3 44s 44s 0%
slow (native) PASS 1/3 1m 42s 1m 49s -6%
gated (native) PASS 1/3 2m 5s 2m 15s -7%

Time is the passing attempt (the last attempt if none passed). Change compares it with the median passing time of the last 5 runs; ±20% or more is flagged.

Attempt details

  • basic (native) attempt 1 — PASS (5m 3s): login evidence: PASS (post-login plan=true, post-login research=true); research: PASS (files=5, wellFormed=true, keywords=7/3); scenarios: PASS (tests=7/5, features=4/3); tests passed: PASS (5 passed, 0 failed (reporter: 5 passed, 0 failed))
  • experience: control attempt 1 — PASS (24s): control: OK — failed as expected (0 passed, 0 failed)
  • experience: seeded attempt 1 — PASS (44s): seeded: PASS (1 passed, 0 failed)
  • slow (native) attempt 1 — PASS (1m 42s): login evidence: PASS (post-login plan=true, post-login research=true); research: PASS (files=2, wellFormed=true, keywords=7/3); scenarios: PASS (tests=7/5, features=4/3); tests passed: PASS (1 passed, 0 failed (reporter: 1 passed, 0 failed))
  • gated (native) attempt 1 — PASS (2m 5s): login evidence: PASS (post-login plan=true, post-login research=true); research: PASS (files=4, wellFormed=true, keywords=7/3); scenarios: PASS (tests=8/5, features=4/3); tests passed: PASS (1 passed, 0 failed (reporter: 1 passed, 0 failed))

Session analysis — basic (native):

Session Analysis

Creating an issue, opening its detail view, and the empty-result search state all work in the tested flows. Keyword filtering is not confirmed: its only test had a failed click step, and the empty-title validation test also had failed steps.

Coverage

  • Pages: /issues, /issues?new=1, /issues/6
  • Features: new issue drawer from toolbar, empty-title validation, issue detail view, keyword search, empty search results

What works

  • Issue creation - ET-1 Creating a new issue from the toolbar adds it to the issues list
  • Issue detail view - ET-3 Opening an issue title from the list shows that issue's detail view
  • Empty search results - ET-5 Searching issues with a term that matches nothing shows an empty result

Execution Issues

  • ET-2 Submitting the new issue form with an empty title shows a validation error and creates no issue - passed despite failed steps; redundant submit clicks; no validation assertion.
  • ET-4 Searching issues by a keyword shows only issues matching that keyword - passed despite failed search textbox click; malformed locator.

Session analysis — slow (native):

Session Analysis

New issue creation from the New Issue drawer works. The new issue appears in the list with default status open and priority normal.

Coverage

  • Pages: /issues, /issues?new=1
  • Features: New issue drawer, issue list display

Execution Issues

  • ET-1 Create a new issue from the New Issue button and verify it appears in the issues list - First Create click used toolbar scope and failed; form-close unverified.

Session analysis — gated (native):

Session Analysis

Creating an issue from the workspace toolbar saves it, and the new issue appears in the issues table with its title. Whether the priority selection was actually applied is not confirmed by a clean run.

Coverage

  • Pages: /workspace, /issues, /issues?new=1
  • Features: toolbar issue creation, new-issue form (title, priority), issues table listing

Execution Issues

  • ET-1 Create a new issue from the toolbar and verify it appears in the issues list - Two priority steps logged FAILED with no error; expectations unchecked.

@DavertMik
DavertMik merged commit 84199c6 into main Nov 6, 2026
3 checks passed
@DavertMik
DavertMik deleted the research-close-walls branch November 6, 2026 12:24

This branch was successfully deployed

1 active deployment
regression — 94782ec8 Deployed Nov 6, 2026 by DavertMik via regression #205
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant