Skip to content

fix(crawlers): Use the HTTP client and session pool of AdaptivePlaywrightCrawler in its sub crawlers - #2451

Open
Mantisus wants to merge 1 commit into
apify:v2from
Mantisus:adaptive-shared-collaborators
Open

Mantisus wants to merge 1 commit into
apify:v2from
Mantisus:adaptive-shared-collaborators

Conversation

@Mantisus

Copy link
Copy Markdown
Collaborator

Description

  • Both sub crawlers of AdaptivePlaywrightCrawler use the HTTP client and the session pool of the adaptive crawler, instead of building defaults of their own that nothing entered or closed.
  • Static requests and robots.txt lookups go through the client that the crawler enters and closes on every run.

Issues

Testing

  • Integration test, for static and browser requests, that the sub crawlers send through the HTTP client the crawler closes.

✍️ Drafted by Claude Code

@Mantisus
Mantisus marked this pull request as draft November 15, 2026 17:15
@github-actions github-actions Bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Nov 15, 2026
@Mantisus Mantisus self-assigned this Nov 15, 2026
@Mantisus
Mantisus force-pushed the adaptive-shared-collaborators branch from ed84b3c to 176b3f8 Compare November 15, 2026 22:40
@Mantisus
Mantisus force-pushed the adaptive-shared-collaborators branch from 176b3f8 to 5880cff Compare November 18, 2026 14:53
@Mantisus
Mantisus marked this pull request as ready for review November 18, 2026 14:53
@Mantisus
Mantisus requested a review from vdusek November 18, 2026 14:53

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant