Skip to content

fix(s3vectors): restore query pagination - #352

Open
Lawliet2004 wants to merge 3 commits into
codeforstartups:developmentfrom
Lawliet2004:codex/fix-query-pagination-audit
Open

Lawliet2004 wants to merge 3 commits into
codeforstartups:developmentfrom
Lawliet2004:codex/fix-query-pagination-audit

Conversation

@Lawliet2004

@Lawliet2004 Lawliet2004 commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Description

S3 Vectors queries raise OperationNotPageableError on the boto3 and aioboto3 versions installed by make install. On SDK versions that support continuation, reading NextToken instead of the API's nextToken stops the query after the first response.

Changes

  • Call query_vectors directly in the sync and async stores and forward nextToken until the results reach top-k or the service has no more pages. This preserves stream chunking, rate limiting, and error translation.
  • Add regression coverage with real boto3 and aioboto3 stub clients for token and filter forwarding, empty intermediate pages, top-k limits, later-page errors, and retries. Update the logging fixture for direct queries.

Older SDKs without continuation fields still support a single response. The dependency constraints are unchanged. AWS QueryVectors API · boto3 query_vectors

Related issues

This is a regression follow-up to #16. The November 6 audit is tracked separately in #353.

Testing

  • make run-ci on Python 3.12.15 passed Ruff, strict mypy, and pytest with 1,125 tests passed, 2 opt-in live AWS tests skipped, and 1 snapshot passed.
  • The focused store, error, and logging test selection passed all 100 tests.
  • git diff --check passed.

Copilot AI balanced review requested due to automatic review settings November 6, 2026 14:34

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@Kaap10 Kaap10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The query_vectors pagination and nextToken casing fix is solid and passes all tests.

However, please remove the unrelated dated issue audit document from the PR (see inline comment) so this PR remains a focused bugfix.

Comment thread docs/ISSUE_AUDIT_2026-11-06.md Outdated
@@ -0,0 +1,165 @@
# Issue audit — 6 November 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this file from the PR. While this issue audit is valuable and well-researched, committing a dated, point-in-time triage report into the permanent repository documentation (docs/) will quickly become stale and pollutes user-facing documentation. Please move this analysis to a GitHub Discussion or tracking issue instead.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review, @Kaap10. I removed the audit file from this PR and moved the write-up to #353. Could you re-review the updated PR?

@Lawliet2004 Lawliet2004 changed the title fix(s3vectors): restore query pagination and audit unresolved issues fix(s3vectors): restore query pagination Nov 8, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants