aitools: stop polling tests from waiting on the real poll interval - #7824
Conversation
## Changes This only changes unit tests in `experimental/aitools/cmd`. Production code is untouched. - `query_test.go`: `TestExecuteAndPollWithPolling` and `TestExecuteAndPollFailsDuringPolling` now build their client with `sqlexec.WithPollInterval(time.Millisecond, time.Millisecond)`. They still go through the poll loop, but each poll waits 1ms. - `batch_test.go`, `statement_test.go`: `TestExecuteBatchPreservesInputOrder` and `TestGetStatementResultPolls` now call `t.Parallel()`. These two tests can't take a poll interval: `executeBatch` and `getStatementResult` create their own `sqlexec.Client` from the API interface. Running them in parallel means their 1s waits overlap. ## Why The `sqlexec` client polls with an additive backoff that starts at 1s and grows by 1s per poll. With mocked responses, every test that reached the poll loop slept for real, one test after another: | Test | Polls | Wait before | Wait after | | --- | --- | --- | --- | | `TestExecuteAndPollWithPolling` | 2 | 1s + 2s | ~2ms | | `TestExecuteAndPollFailsDuringPolling` | 1 | 1s | ~1ms | | `TestExecuteBatchPreservesInputOrder` | 1 | 1s | 1s (in parallel) | | `TestGetStatementResultPolls` | 1 | 1s | 1s (in parallel) | That's about 6s of sleeping before (my estimate from the backoff schedule) and about 1s after. The tests check the same things as before. ## Tests ``` go test ./experimental/aitools/cmd -count=1 -run 'TestExecuteAndPoll|TestExecuteBatch|TestGetStatementResult' -v --- PASS: TestExecuteAndPollWithPolling (0.00s) --- PASS: TestExecuteAndPollFailsDuringPolling (0.00s) --- PASS: TestGetStatementResultPolls (1.00s) --- PASS: TestExecuteBatchPreservesInputOrder (1.00s) ok github.com/databricks/cli/experimental/aitools/cmd 1.015s ``` This is a test-only change, so there's no changelog fragment. Co-authored-by: Isaac <no-reply@databricks.com>
There was a problem hiding this comment.
Automated review at 4f887ea. Each finding was verified against the code.
No blocking issues found. 1 non-blocking comment(s) below.
This test-only PR reduces polling-test wall-clock time from ~6 s to ~1 s by injecting a fastPoll option into the two tests that build their own sqlexec.Client, and by marking the other two (whose callers hard-code sqlexec.New without option forwarding) as t.Parallel(). The approach is correct and an improvement, though the parallel workaround still sleeps 1 s per test when run in isolation.
The polling tests for executeBatch and getStatementResult can now use the 1ms poll interval too, so they no longer wait on the real 1s interval or depend on running in parallel. Co-authored-by: Isaac <no-reply@databricks.com>
simonfaltum
left a comment
There was a problem hiding this comment.
🤖 Automated review from simonfaltum's review agent (pr-review skill) at 2a44125. Each finding was verified against the code; simonfaltum may follow up.
No blocking issues found, approving. 4 non-blocking comment(s) and 1 question(s) below.
This PR makes the aitools polling tests use a 1ms poll interval instead of the real 1s+ backoff. To do that, it adds a variadic sqlexec.Option parameter to executeBatch and getStatementResult. The code is correct and production behaviour is unchanged: the package's tests pass, including under -race, and take ~0s instead of ~6s. The main ask is to update the PR description, which still describes the first commit's t.Parallel() approach and will become the squash commit message. The rest are small comment and test nits, plus one optional question about passing a *sqlexec.Client instead of options.
Questions
executeBatchandgetStatementResultnow takeopts ...sqlexec.Option, but only tests pass any. Both production callers (query.go:243,statement_get.go:33) pass none, and both functions useapi/warehouseIDonly to build the client. In the same package,executeAndPollandcancelInFlightalready take a*sqlexec.Client, which is howTestExecuteAndPollWithPollinginjectsfastPoll. Would you consider having these two functions take a*sqlexec.Clientthat their callers build? That would remove the test-only parameter and match the existing seam. Keeping the options is also reasonable if you prefer them. (non-blocking)
Accepted answers
- Forward poll options so the last two polling tests skip the real 1s wait: #7824 (comment)
Comments outside the changed lines
- [nice to have]
experimental/aitools/cmd/batch.go: The PR description still describes the first commit, and this repo squash-merges with the PR body as the commit message. It says "Production code is untouched" and thatTestExecuteBatchPreservesInputOrder/TestGetStatementResultPollscallt.Parallel()because they "can't take a poll interval". At head,executeBatch(batch.go:63) andgetStatementResult(statement_get.go:63) takeopts ...sqlexec.Option, both tests passfastPoll, and there is not.Parallel(). These are also stale: the table's "1s (in parallel)" rows, the "about 1s after" estimate, and the pasted test output (1.00s each, 1.015s total). All four tests now run in ~0s. Please update Changes/Why/Tests to match the head commit. Skipping the changelog is still right (experimental/, no user-visible change), but "test-only change" no longer holds. While you're there: the commits are co-authored by Isaac, and.agents/skills/pr-checklistasks for a one-line italic agent disclosure at the end of the body.
Co-authored-by: Isaac <no-reply@databricks.com>
- Say why executeBatch and getStatementResult take poll options. - Move fastPoll to the top of query_test.go and fix its comment. - Make TestExecuteBatchPreservesInputOrder deterministic: with two slots, the slow statement's poll blocks until the 'fast2' submission starts, so it always completes last. Co-authored-by: Isaac <no-reply@databricks.com>
❌ Integration test reportCommit: 31e9487
29 interesting tests: 24 RECOVERED, 5 flaky
Top 50 slowest tests (at least 2 minutes):
|
Changes
This only changes unit tests in
experimental/aitools/cmd. Production code is untouched.query_test.go:TestExecuteAndPollWithPollingandTestExecuteAndPollFailsDuringPollingnow build their client withsqlexec.WithPollInterval(time.Millisecond, time.Millisecond). They still go through the poll loop, but each poll waits 1ms.batch_test.go,statement_test.go:TestExecuteBatchPreservesInputOrderandTestGetStatementResultPollsnow callt.Parallel(). These two tests can't take a poll interval:executeBatchandgetStatementResultcreate their ownsqlexec.Clientfrom the API interface. Running them in parallel means their 1s waits overlap.Why
The
sqlexecclient polls with an additive backoff that starts at 1s and grows by 1s per poll. With mocked responses, every test that reached the poll loop slept for real, one test after another:TestExecuteAndPollWithPollingTestExecuteAndPollFailsDuringPollingTestExecuteBatchPreservesInputOrderTestGetStatementResultPollsThat's about 6s of sleeping before (my estimate from the backoff schedule) and about 1s after. The tests check the same things as before.
Tests
This is a test-only change, so there's no changelog fragment.