Add timeout for aggregations#2026
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the deployment configuration for the ingestion-helper-service in cloudbuild.yaml to include a 60-minute timeout and adds a 30-minute timeout to the HTTP POST call in spanner-ingestion-workflow.yaml. Feedback suggests increasing the workflow timeout to 3600 seconds to match the service's deployment timeout and avoid premature failures.
| call: http.post | ||
| args: | ||
| url: ${helper_url} | ||
| timeout: 1800 |
There was a problem hiding this comment.
The timeout for the run_aggregation HTTP call is set to 1800 seconds (30 minutes), but the service is deployed with a 60-minute timeout. To avoid premature workflow failures, these timeouts should be consistent. Note that per repository rules, the import-aggregation-helper is responsible for graph data aggregation, while the spanner-ingestion-helper works on metadata tables. Ensure the correct service is being targeted and consider increasing this value to 3600 seconds.
timeout: 3600References
- The import-aggregation-helper works on graph data and does not need access to the metadata tables.
- The spanner-ingestion-workflow uses the graph database for its dataflow job, while the spanner-ingestion-helper works on the metadata tables in a separate database.
Default timeout is 5 minutes causing workflow failures. Increased to max supported.