Repository navigation
Conversation
|
Commit: Regression test reportSelected 2 changed tests (limit=20, 0 not selected) 2 tests: regression=2 unreleased=0 coverage=0 failing=0
TestAccept/air/feedback: branch ✅ | main (aed6702) ❌ | latest (v1.23.0) ❌main (aed6702): latest (v1.23.0): same output as main (aed6702) TestAccept/air/help: branch ✅ | main (aed6702) ❌ | latest (v1.23.0) ❌main (aed6702): latest (v1.23.0): same output as main (aed6702) |
b40f37c to
6af851a
Compare
Integration test reportCommit: 6af851a
Top 6 slowest tests (at least 2 minutes):
|
|
This PR depends on backend changes that need to be merged and released before it merges:
Please hold this PR until those are live. |
simonfaltum
left a comment
There was a problem hiding this comment.
Requesting changes for the transient authentication failure handling noted inline. The focused AIR feedback unit and local acceptance tests passed for this head.
Two non-blocking questions:
- Could
submitFeedbackshare the one-shot sender inlibs/telemetry/logger.go:200, while keeping explicit submission semantics and its acknowledgement check? Both implementations construct the same request, routing header, timestamp, and emptyitemsworkaround. If the sender stays separate, please document why the API requires an emptyitemsarray. - Could validation reuse the existing AIR
INVALID_ARGS/PERMANENTpair? This command introducesINVALID_ARGUMENT/VALIDATIONinto the common JSON envelope.
This review was generated with GitHub MCP.
| if err == nil || errors.Is(err, root.ErrAlreadyPrinted) { | ||
| return err | ||
| } | ||
| return renderError(cmd.Context(), cmd, "AUTHENTICATION_ERROR", "AUTHENTICATION", false, err) |
There was a problem hiding this comment.
[P2][blocking] root.MustWorkspaceClient can fail on a temporary OAuth connection error before /telemetry-ext is called. This branch marks every such failure retryable: false, so an agent following the JSON response abandons a report that is safe to retry.
I reproduced this with a local OAuth endpoint that closes the token connection: air feedback returned AUTHENTICATION_ERROR/AUTHENTICATION with retryable: false and zero feedback submissions. air get reported the same failure as INTERNAL_ERROR/TRANSIENT with retryable: true.
Please route this branch through the existing authError helper and add a regression case for a transient authentication failure.
| # Explicit feedback still submits when automatic usage telemetry is disabled. | ||
| Env.DATABRICKS_CLI_DISABLE_TELEMETRY = "1" | ||
|
|
||
| [[Repls]] |
There was a problem hiding this comment.
[nit] The built-in UUID replacement runs before config.Repls is appended, so this rule never sees an original UUID and cannot produce [FEEDBACK_ID]. The golden already uses [UUID]; please remove this unused replacement block.
| if o.message == "" || !utf8.ValidString(o.message) || utf8.RuneCountInString(o.message) > feedbackMessageLimit { | ||
| return fmt.Errorf("--message must contain 1 to %d UTF-8 characters", feedbackMessageLimit) | ||
| } | ||
| if o.reporter != "human" && o.reporter != "agent" { |
There was a problem hiding this comment.
[nit] Reporter values and error code/kind pairs are repeated across validation, flag defaults, and rendering. .agents/rules/style-guide-go.md:40 requires named constants for repeated strings; please extract the shared values.
| Product: "PRODUCT_AIR", | ||
| FeedbackSource: "SOURCE_AIR_CLI", | ||
| ReporterType: "REPORTER_" + strings.ToUpper(o.reporter), | ||
| CLIVersion: build.GetInfo().Version, |
There was a problem hiding this comment.
Did you test this? the schema doesn't look aligned to the proto fast_feedback_log event. For example it has no CLIVersion field.
Typically I see this info get inlined into a synthetic user agent.
| if err == nil || errors.Is(err, root.ErrAlreadyPrinted) { | ||
| return err | ||
| } | ||
| return renderError(cmd.Context(), cmd, "AUTHENTICATION_ERROR", "AUTHENTICATION", false, err) |
There was a problem hiding this comment.
Every MustWorkspaceClient failure is reported here as AUTHENTICATION_ERROR with retryable: false, including a transient one such as a dropped OAuth token request, where nothing has been sent yet and a retry is safe. air get, logs and pools report that same failure as INTERNAL_ERROR/TRANSIENT with retryable: true, so please route this through authError like they do and add a test for a non-auth failure on this branch.
| // errors can be rendered, while still rejecting input before authentication. | ||
| cmd.PreRunE = func(cmd *cobra.Command, args []string) error { | ||
| if err := cobra.NoArgs(cmd, args); err != nil { | ||
| return renderError(cmd.Context(), cmd, "INVALID_ARGUMENT", "VALIDATION", false, err) |
There was a problem hiding this comment.
Bad input is reported as INVALID_ARGUMENT/VALIDATION here, but every other air command uses INVALID_ARGS/PERMANENT for it, so a caller that keys on the envelope's code or kind across subcommands won't recognise these. I'd reuse the existing pair before this ships, since the codes are hard to change once agents depend on them.
pardis-beikzadeh-db
left a comment
There was a problem hiding this comment.
One blocker inline (feedback.go:66), plus @simonfaltum's open thread on feedback.go:130. Two non-blocking follow-ups inline.
|
|
||
| // These bindings mirror FastFeedbackLog in universe. Feedback text is sent only | ||
| // through that explicitly submitted feedback schema, never a CLI usage event. | ||
| type fastFeedbackLog struct { |
There was a problem hiding this comment.
Please use the existing libs/telemetry/protos types instead of a second copy of FrontendLog here:
- Add
FastFeedbackLogtoprotos.FrontendLogEntry. - Use typed constants for product, source, and reporter, like
protos.AirPackagingMode. That replaces the string concatenation on line 166. - Send it through a one-shot helper exported from
libs/telemetrythat wrapsattempt. That also keeps the comment on whyItemsmust be empty.
| Do not include credentials or sensitive data. Follow-up contact is opt-in. | ||
|
|
||
| This is an explicit submission, independent of automatic usage telemetry. | ||
| Success confirms receipt by the collector; processing into review tools is |
There was a problem hiding this comment.
Non-blocking: a few phrases in this help text may be hard for CLI users to follow. Suggested wording:
- Line 103: "to Fast Feedback" could be "to Databricks", unless users already know the Fast Feedback name.
- Lines 108-109: "Success confirms receipt by the collector; processing into review tools is asynchronous." could be "A success message means Databricks received your feedback."
| trace $CLI air feedback --message "The workaround succeeded" | ||
|
|
||
| title "validation errors" | ||
| trace musterr $CLI air feedback --message " " -o json | jq -e '.error.code == "INVALID_ARGUMENT"' |
There was a problem hiding this comment.
Non-blocking: with the jq -e pipe, output.txt records only true, so the error message is never checked. acceptance/air/get/script uses errcode trace $CLI air get ... -o json. The same pattern here would record the full error for all four cases.
|
This PR has been merged FYI #7724 which adds a databricks feedback command. |
|
As Simon brought to my attention, #7724 does the same thing. Can we close this one? |
|
Superseded by #7724, which implements the shared This comment was generated with GitHub MCP. |
Changes
Add databricks air feedback with workspace authentication, human/agent attribution, optional run and agent context, contact opt-in, and a JSON receipt. Include command help, acceptance coverage, and a changelog entry.
Why
Let users and coding agents report actionable AIR product friction through their existing workspace login, with enough context for product and engineering teams to investigate.
Tests
The source is unchanged at 6af851a. GitHub CI passed Linux, macOS, and Windows tests in both deployment modes, lint, generated-file validation, and regression reporting. External integration tests passed across AWS, Azure, and GCP on Linux and Windows. The regression report confirms the new feedback/help tests pass on this branch and fail on main/latest, where the command is absent.
Earlier local unit suites passed 12,519 root tests (132 skipped) and 73 tools tests. Focused feedback/help/config-help acceptance tests passed. The broader local acceptance run encountered unrelated AppKit/bundle timeouts; no clean full local acceptance pass is claimed.
Two real staging submissions were traced from CLI receipts to raw text and existing fields. New typed metadata and the full Jira/Insights path remain pending deployment verification.
This PR was written with Codex.
Related PRs and rollout
Employee dashboard: go/air-cli-feedback.
Deployment steps, request-protection evidence, and verification handoff.
Implementation and review readiness do not imply deployment or owner approval.