Allow configuring HTTP client timeout for DynamoDB backend#31797
Open
Kyslik wants to merge 2 commits intohashicorp:mainfrom
Open
Allow configuring HTTP client timeout for DynamoDB backend#31797Kyslik wants to merge 2 commits intohashicorp:mainfrom
Kyslik wants to merge 2 commits intohashicorp:mainfrom
Conversation
|
@Kyslik is attempting to deploy a commit to the HashiCorp Team on Vercel. A member of the Team first needs to authorize it. |
Author
|
@mathew-amala could you please take a look 🤞🏻 . |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR allows operators to configure the HTTP client timeout used by the DynamoDB backend.
By default, the timeout remains zero, which preserves existing behavior. In Go's
net/httppackage, a timeout of zero means no timeout, so requests may block indefinitely.As described in issue #26580, a deadlock scenario is possible when using the DynamoDB HA backend. If the active instance experiences an intermittent network drop, it may fail to release the lock while another instance acquires it, resulting in multiple nodes acting as active. Recovery typically requires a restart; however, if an intermediate proxy or firewall properly closes the stale connection, the instance can recover and transition back to standby. Configuring a non-zero HTTP client timeout mitigates this condition by preventing indefinitely blocked requests on these two calls
vault/physical/dynamodb/dynamodb.go
Line 770 in 781ba45
vault/physical/dynamodb/dynamodb.go
Line 814 in 781ba45
The additional validation rejecting negative timeout values is defensive. Although
time.ParseDurationpermits negative durations, they are not meaningful in this context (related reading golang/go#39177).A dedicated test was not added because this change is limited to configuration parsing and assignment of
http.Client.Timeout, which relies entirely on Go's standard library (time.ParseDurationandnet/http). The logic is straightforward, contains no custom timeout behavior, and does not alter request execution semantics beyond setting the standard client field, so additional unit coverage would provide limited value.There is an alternative fix to make client sensibly timeout, using an
UpdateItemWithContextandGetItemWithContext, however context is not in theLockinterface thus would require a lot more work.🤔 , unless hardcoded
context.WithTimeoutcan be used:I think the client timeout should be configurable even though
UpdateItemWithContextmay be favorable.