From bc8fb152590f25b953620a1c620873ffcda21252 Mon Sep 17 00:00:00 2001
From: Pieter Noordhuis <pieter.noordhuis@databricks.com>
Date: Wed, 3 Jun 2026 13:29:46 +0200
Subject: [PATCH] acceptance: document LOG.* files, gron.py over jq, --keep
 gotcha, and EnvMatrix scoping
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Four idioms surfaced in PR #5414 review that weren't captured in the
acceptance-test guide:

- Limit `EnvMatrix.DATABRICKS_BUNDLE_ENGINE` when the test inspects
  engine-specific state shape (resources.json vs terraform.tfstate).
- Prefer `gron.py | grep <field>` over inline `jq` for single-value
  lookups — the gron output line includes the JSON path, so the test
  log is self-explanatory.
- Don't pass `--keep` to print_requests.py if a later call follows;
  the second call re-reads the buffer and double-prints.
- Use `LOG.<name>` for cleanup-noise stderr that should show up only
  under `go test -v` (not dropped with `2>/dev/null`).

Each rule has a GOOD / BAD example and points at existing tests that
already follow the pattern.

Co-authored-by: Isaac
---
 .agent/rules/testing.md | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/.agent/rules/testing.md b/.agent/rules/testing.md
index 67f41cd2ea2..4328c8daa16 100644
--- a/.agent/rules/testing.md
+++ b/.agent/rules/testing.md
@@ -87,6 +87,8 @@ acceptance/cmd/fs/cp/file-to-dir/
 
 If the only reason for divergence is a server-side default that one engine sets and the other doesn't, set the field explicitly in `databricks.yml` so both engines produce identical output. Don't paper over it with per-engine files.
 
+**RULE: Limit `EnvMatrix.DATABRICKS_BUNDLE_ENGINE` when the test exercises engine-specific behavior.** Direct's `PrepareState` / `resources.json` shape and terraform's drift loop / `.tfstate` shape are not shared. If a test inspects either, set the matrix to just the relevant engine and leave a one-line `test.toml` comment explaining why. Cross-engine parity of the contract (request/response shape, destroy semantics) belongs in a sibling test that exercises both.
+
 ### Reference
 
 - Tests live in `acceptance/` with a nested directory structure.
@@ -132,6 +134,38 @@ Available on `PATH` during test execution (from `acceptance/bin/`):
 - `gron.py`: flatten JSON into greppable discrete assignments (simpler than `jq` for searching JSON).
 - `jq` is also available for JSON processing.
 
+**RULE: Prefer `gron.py | grep <field>` over inline `jq` paths for single-value lookups.** The gron form prints the full assignment (e.g. `json.state["resources.foo.bar"].state.purge_on_delete = true;`), so the path is self-evident in the test output and reviewers don't have to mentally walk a jq expression.
+
+GOOD:
+
+```bash
+gron.py < .databricks/bundle/default/resources.json | grep purge_on_delete
+```
+
+BAD:
+
+```bash
+jq -r '.state["resources.postgres_projects.proj"].state.purge_on_delete' .databricks/bundle/default/resources.json
+```
+
+**RULE: Don't pass `--keep` to `print_requests.py` if you make a later `print_requests.py` call in the same test.** `--keep` retains `out.requests.txt`, so the next call re-reads the same accumulated buffer and double-prints earlier requests. Use `--keep` only when this is the last invocation AND something else (e.g. a separate `jq`) needs to read the file afterward.
+
+### Diagnostic output
+
+**RULE: For cleanup-noise or other stderr a developer might want to see under `go test -v`, append to a `LOG.<name>` file instead of dropping it with `2>/dev/null`.** Files named `LOG` or `LOG.<suffix>` are surfaced by the test runner only in verbose mode (see `acceptance/selftest/log/`), and they aren't part of the diffed output so they don't trigger spurious failures.
+
+GOOD:
+
+```bash
+$CLI postgres delete-project --purge "projects/$NAME" 2>>LOG.delete-project || true
+```
+
+BAD:
+
+```bash
+$CLI postgres delete-project --purge "projects/$NAME" 2>/dev/null || true
+```
+
 ### Update workflow
 
 **RULE: Run `./task test-update` to regenerate outputs, then `./task fmt` and `./task lint`.** If fmt or lint modify files in `acceptance/`, there's an issue in the source files. Fix the source, regenerate, and verify fmt/lint pass cleanly.