Skip to content

[#10683] fix(iceberg-rest-server): support Idempotency-Key header and replay for mutation endpoints#10751

Open
laserninja wants to merge 1 commit intoapache:mainfrom
laserninja:fix/10683-iceberg-idempotency-key
Open

[#10683] fix(iceberg-rest-server): support Idempotency-Key header and replay for mutation endpoints#10751
laserninja wants to merge 1 commit intoapache:mainfrom
laserninja:fix/10683-iceberg-idempotency-key

Conversation

@laserninja
Copy link
Copy Markdown
Collaborator

What changes were proposed in this pull request?

Add support for the Idempotency-Key HTTP header on all mutation (POST/DELETE) endpoints in the Iceberg REST server. When a client sends a valid UUIDv7 idempotency key with a mutation request, the server caches the successful response and replays it on subsequent retries, preventing duplicate side-effects.

Key implementation details:

  • New IcebergIdempotencyManager class with:
    • UUIDv7 format validation for the header value
    • Caffeine-based response cache with configurable TTL (default 30 min, max 10K entries)
    • Thread-safe replayOrExecute() using ConcurrentHashMap key locks with double-check pattern
    • Cache key includes idempotency key + HTTP method + URI + normalized query string
  • Added @HeaderParam("Idempotency-Key") to all mutation endpoints:
    • Tables: createTable, updateTable, dropTable
    • Namespaces: createNamespace, updateNamespace, dropNamespace, registerTable
    • Views: createView, replaceView, dropView
  • Advertises idempotency-key-lifetime (ISO-8601 duration) in GET /v1/config defaults
  • New config entry idempotency-key-lifetime-minutes (default: 30)

Why are the changes needed?

Fixes #10683

Network failures and client retries against the Iceberg REST catalog can cause duplicate table/namespace/view creation or deletion. The Iceberg REST spec recommends idempotency key support for mutation endpoints to safely handle retries.

Does this PR introduce any user-facing change?

Yes:

  • Clients can now send an Idempotency-Key: <UUIDv7> header on mutation requests for safe retry
  • GET /v1/config response now includes idempotency-key-lifetime in defaults
  • New server config: idempotency-key-lifetime-minutes (default 30)

How was this patch tested?

  • Unit tests added for all three operation classes:
    • TestIcebergTableOperations: testCreateTableWithIdempotencyKey, testCreateTableRejectsInvalidIdempotencyKey
    • TestIcebergNamespaceOperations: testRegisterTableWithIdempotencyKey
    • TestIcebergViewOperations: testCreateViewWithIdempotencyKey
  • TestIcebergConfig updated to verify idempotency-key-lifetime in config response
  • All 100 tests across 4 suites pass locally (JDK 17)
  • Tests verify replay returns cached response on second call with same key
  • Tests verify events fire exactly once (no duplicate side-effects)
  • Tests verify invalid (non-UUIDv7) keys return 400 Bad Request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement] Support Idempotency-Key header in Iceberg REST server

1 participant