Skip to content

bundle: report deployment history drain wait time - #7821

Draft
shreyas-goenka wants to merge 1 commit into
mainfrom
shreyas-goenka_data/shreyas-goenka/dms-drain-duration
Draft

shreyas-goenka wants to merge 1 commit into
mainfrom
shreyas-goenka_data/shreyas-goenka/dms-drain-duration

Conversation

@shreyas-goenka

Copy link
Copy Markdown
Contributor

Changes

Record the foreground wait spent draining deployment-history updates as dms.operation_buffer_drain in the existing deploy timing telemetry. Preserve zero waits and include timeout waits; uploads that overlap resource deployment are excluded.

Why

Queue counts cannot tell us how many seconds history recording adds to a deployment. The temporary performance dashboard needs this measurement to show drain latency separately from the whole deploy phase.

Tests

  • go test ./libs/dms ./bundle/phases
  • go test ./acceptance -run '^TestAccept/bundle/dms/multiple-resources' (timer emitted with a buffer; omitted without one)

This PR was written with Codex.

@github-actions github-actions Bot added the DABs DABs related issues label Nov 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Commit: 5d8d24673 | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

1 tests: regression=1 unreleased=0 coverage=0 failing=0
test branch main (de394ed) latest (v1.27.0)
TestAccept/bundle/dms/multiple-resources/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/multiple-resources/DMS=true: branch ✅ | main (de394ed) ❌ | latest (v1.27.0) ❌

main (de394ed):

acceptance_test.go:1388: Diff:
        --- bundle/dms/multiple-resources/output.txt
        +++ /tmp/TestAcceptbundledmsmultiple-resourcesDMS=true2957516246/002/output.txt
        @@ -29,8 +29,8 @@
         
         >>> cat [OUTPUT_DIR]/out.requests.txt
         {
        -  "measurement_count": 1,
        -  "duration_is_nonnegative": true
        +  "measurement_count": 0,
        +  "duration_is_nonnegative": false
         }
         
         >>> print_requests.py --dms //versions/1/operations --sort --del-body state --oneline
        
FAIL

latest (v1.27.0):

acceptance_test.go:1388: Diff:
        --- bundle/dms/multiple-resources/output.txt
        +++ /tmp/TestAcceptbundledmsmultiple-resourcesDMS=true3970286492/002/output.txt
        @@ -24,13 +24,24 @@
           "recording_attempt_success_count": 5,
           "queue_size_in_range": true
         }
        +{
        +  "staged_operation_count": null,
        +  "operation_recording_success_count": null,
        +  "operation_recording_failure_count": null,
        +  "operation_recording_pending_count": null,
        +  "queued_update_count_at_timeout": null,
        +  "recording_attempt_failure_count": null,
        +  "recording_attempt_success_count": null,
        +  "queue_size_in_range": false
        +}
         
         === Drain telemetry measures foreground wait, including zero milliseconds
         
         >>> cat [OUTPUT_DIR]/out.requests.txt
        +jq: error (at <stdin>:583): Cannot iterate over null (null)
         {
        -  "measurement_count": 1,
        -  "duration_is_nonnegative": true
        +  "measurement_count": 0,
        +  "duration_is_nonnegative": false
         }
         
         >>> print_requests.py --dms //versions/1/operations --sort --del-body state --oneline
        
FAIL

@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 5d8d246

Run: 44571209927

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 274 19 14:54
💚​ aws-windows-latest-4core-16gb 1 276 17 8:52
💚​ azure linux-2core-8gb 1 273 19 15:03
💚​ azure-windows-latest-4core-16gb 1 275 17 9:56
💚​ gcp linux-2core-8gb 1 274 19 18:32
💚​ gcp-windows-latest-4core-16gb 1 276 17 10:04
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 38 slowest tests (at least 2 minutes):
duration env testname
5:32 azure linux-2core-8gb TestFilerRecursiveDelete/workspace_files_extensions
5:14 aws-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir
4:22 aws-windows-latest-4core-16gb TestAccept
4:19 gcp linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
4:18 aws linux-2core-8gb TestImportDir
3:55 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsStat
3:54 azure-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files_extensions
3:51 gcp-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files_extensions
3:51 azure-windows-latest-4core-16gb TestAccept
3:49 gcp-windows-latest-4core-16gb TestAccept
3:37 gcp linux-2core-8gb TestImportDir
3:21 aws linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
3:18 azure-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files
3:18 aws-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsStat
3:10 azure linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
3:08 gcp linux-2core-8gb TestImportDirWithOverwriteFlag
2:59 azure-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir
2:59 aws linux-2core-8gb TestFilerWorkspaceFilesExtensionsStat
2:42 azure linux-2core-8gb TestAccept/bundle/dms/multiple-resources/DMS=true
2:39 azure-windows-latest-4core-16gb TestSyncIncrementalFileSync
2:39 gcp linux-2core-8gb TestSyncIncrementalSyncFileToPythonNotebook
2:36 azure linux-2core-8gb TestFilerWorkspaceFilesExtensionsDelete
2:32 aws linux-2core-8gb TestFilerRecursiveDelete/workspace_files
2:27 gcp linux-2core-8gb TestExportDir
2:25 gcp linux-2core-8gb TestFilerReadDir/workspace_files
2:25 gcp linux-2core-8gb TestImportDirDoesNotOverwrite
2:22 azure linux-2core-8gb TestFilerReadDir/workspace_files
2:21 azure-windows-latest-4core-16gb TestAccept/bundle/dms/multiple-resources/DMS=true
2:18 aws linux-2core-8gb TestFilerWorkspaceNotebook/rJupyterNb.ipynb
2:16 aws linux-2core-8gb TestImportDirWithOverwriteFlag
2:14 gcp linux-2core-8gb TestWorkspaceFilesExtensions_ExportFormatIsPreserved/source_scala
2:09 azure linux-2core-8gb TestImportDir
2:08 azure linux-2core-8gb TestImportDirWithOverwriteFlag
2:07 azure linux-2core-8gb TestImportDirDoesNotOverwrite
2:05 aws-windows-latest-4core-16gb TestFilerReadWrite/workspace_files
2:01 azure linux-2core-8gb TestAccept
2:01 gcp linux-2core-8gb TestAccept/bundle/dms/multiple-resources/DMS=true
2:01 azure-windows-latest-4core-16gb TestExportWithFileFlag

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants