Skip to content

Fix ListObjectsV2 pagination snapshot inconsistency#409

Merged
bootjp merged 2 commits intofeature/s3-implfrom
copilot/sub-pr-396
Mar 22, 2026
Merged

Fix ListObjectsV2 pagination snapshot inconsistency#409
bootjp merged 2 commits intofeature/s3-implfrom
copilot/sub-pr-396

Conversation

Copy link
Contributor

Copilot AI commented Mar 22, 2026

ListObjectsV2 loaded bucket metadata at an initial readTS, then silently replaced readTS with token.ReadTS from the continuation token — leaving meta.Generation derived from a different snapshot than the actual scan. This could cause incorrect token validation or mis-keyed scans when bucket state changed between pages.

Changes

  • adapter/s3.golistObjectsV2: Reorder initialization so the continuation token is parsed first, the final readTS is resolved (token.ReadTS if present, otherwise s.readTS()), and bucket metadata is loaded at that resolved timestamp before any token validation occurs.
// Before: meta loaded at initial readTS, then readTS replaced without reloading meta
readTS := s.readTS()
meta, _, _ := s.loadBucketMetaAt(ctx, bucket, readTS)
token, _ := decodeS3ContinuationToken(...)
if token != nil && token.ReadTS != 0 {
    readTS = token.ReadTS  // meta now inconsistent with readTS
}

// After: token parsed first, single consistent readTS used for meta load
token, _ := decodeS3ContinuationToken(...)
readTS := s.readTS()
if token != nil && token.ReadTS != 0 {
    readTS = token.ReadTS
}
meta, _, _ := s.loadBucketMetaAt(ctx, bucket, readTS)  // consistent snapshot

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Add S3 adapter Fix ListObjectsV2 pagination snapshot inconsistency Mar 22, 2026
Copilot AI requested a review from bootjp March 22, 2026 19:01
@bootjp bootjp marked this pull request as ready for review March 22, 2026 19:20
@bootjp bootjp merged commit c98b088 into feature/s3-impl Mar 22, 2026
4 checks passed
@bootjp bootjp deleted the copilot/sub-pr-396 branch March 22, 2026 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants