Skip to content

Allow AI agents to submit feedback without confirmation - #7795

Merged
simonfaltum merged 5 commits into
mainfrom
simonfaltum/feedback-agent-submissions
Nov 20, 2026
Merged

simonfaltum merged 5 commits into
mainfrom
simonfaltum/feedback-agent-submissions

Conversation

@simonfaltum

@simonfaltum simonfaltum commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Changes

Allow AI agents to submit databricks feedback directly, including when no terminal is attached. Keep --yes / -y accepted as deprecated compatibility flags for existing callers. An explicit false value from an agent declines the submission before authentication, including when a terminal is attached.

CLI help explicitly excludes customer content, personally identifiable information (PII), credentials and other sensitive data. It tells agents to use the existing --can-be-contacted flag only with the user's explicit agreement and to notify the user with the reference ID after a successful submission. Update the telemetry documentation and remove the obsolete confirmation helpers.

Why

Mandatory confirmation interrupts agent workflows and blocks noninteractive feedback. Direct submissions let agents report CLI problems while the receipt and agent instructions keep users informed.

Tests

  • Unit tests for cmd/feedback, libs/cmdio and libs/telemetry pass. Feedback tests cover human and agent submissions, with and without a terminal, and check the receipt and agent attribution.
  • Local feedback acceptance tests cover CLI help, submissions with closed stdin, text and JSON receipts, legacy approval flags (including explicit rejection before authentication), validation, routing and disabled telemetry.
  • The affected Go packages pass the pinned linter and formatter. Whitespace and changelog validation pass.

This pull request and its description were written by Isaac.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 2be1b3f46 | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

1 tests: regression=1 unreleased=0 coverage=0 failing=0
test branch main (06e25a1) latest (v1.27.0)
TestAccept/cmd/feedback ✅ ❌ ❌
TestAccept/cmd/feedback: branch ✅ | main (06e25a1) ❌ | latest (v1.27.0) ❌

main (06e25a1):

acceptance_test.go:1400: Diff:
        --- cmd/feedback/output.txt
        +++ /tmp/TestAcceptcmdfeedback79775073/002/output.txt
        @@ -11,15 +11,14 @@
         The message is sent with the workspace and user you are authenticated as and
         the CLI user agent, which includes the CLI version, operating system and, if
         detected, the AI agent running the CLI. No logs, files or transcripts are
        -collected. Do not include customer content, personally identifiable information
        -(PII), credentials or other sensitive data. Follow-up contact is opt-in with
        ---can-be-contacted. Feedback is sent even when DATABRICKS_CLI_DISABLE_TELEMETRY
        -is set, because this command explicitly submits feedback.
        +collected. Do not include credentials or sensitive data. Follow-up contact is
        +opt-in with --can-be-contacted. Feedback is sent even when
        +DATABRICKS_CLI_DISABLE_TELEMETRY is set, because you asked to send it.
         
        -AI agents can submit feedback directly without asking the user for approval.
        -Use --can-be-contacted only if the user explicitly agrees to be contacted.
        -After a successful submission, tell the user that feedback was sent and include
        -the reference ID.
        +AI agents: submit feedback only with the user's approval. When an agent is
        +detected, the command asks for confirmation, or requires --yes when no terminal
        +is attached. Show the user the exact message and the flags you pass with it,
        +such as --can-be-contacted, and pass --yes only after the user approves it.
         
         Usage:
           databricks feedback MESSAGE [flags]
        @@ -33,6 +31,7 @@
               --command string                  CLI command the feedback is about, for example "bundle deploy"
           -h, --help[=true|false]               help for feedback
               --run-id string                   ID of the job run the feedback is about
        +  -y, --yes[=true|false]                Confirm sending feedback written by an AI agent
         
         Global Flags:
               --debug[=true|false]   enable debug logging
        @@ -147,6 +146,7 @@
               --command string                  CLI command the feedback is about, for example "bundle deploy"
           -h, --help[=true|false]               help for feedback
               --run-id string                   ID of the job run the feedback is about
        +  -y, --yes[=true|false]                Confirm sending feedback written by an AI agent
         
         Global Flags:
               --debug[=true|false]   enable debug logging
        @@ -179,9 +179,10 @@
         Error: --run-id must be a positive job run ID
         
         >>> env -u DATABRICKS_HOST -u DATABRICKS_TOKEN AGENT=test-agent [CLI] feedback --yes=false hello
        -Flag --yes has been deprecated, omit this flag when sending feedback; skip the feedback command when declining
        -Error: feedback was not sent because --yes=false
        +Error: feedback from an AI agent must be confirmed by the user; run with --yes once the user approves the message
         
        +Note for AI agents (test-agent): show the user the exact feedback message and the flags you pass with it, such as --can-be-contacted, and pass --yes only after the user approves sending it
        +
         === General CLI feedback
         
         >>> [CLI] feedback the CLI should support ...
        @@ -206,120 +206,8 @@
         === An AI agent can submit without confirmation
         
         >>> AGENT=test-agent [CLI] feedback hello
        -Thank you! Your feedback was sent. Reference: [UUID][10]
        +Error: feedback from an AI agent must be confirmed by the user; run with --yes once the user approves the message
         
        ->>> print_requests.py //telemetry-ext
        -{
        -  "frontend_log_event_id": "[UUID][10]",
        -  "entry": {
        -    "fast_feedback_log": {
        -      "reference_event_id": "[UUID][11]",
        -      "parent_ui": "DeveloperEcosystem.DatabricksCLI",
        -      "origin": "cli",
        -      "can_be_contacted": false,
        -      "ui_versions": "cli:[CLI_VERSION]",
        -      "user_agent": "cli/[CLI_VERSION] cmd/feedback agent/test-agent",
        -      "text": "hello"
        -    }
        -  }
        -}
        +Note for AI agents (test-agent): show the user the exact feedback message and the flags you pass with it, such as --can-be-contacted, and pass --yes only after the user approves sending it
         
        -=== An AI agent can receive a JSON receipt
        -
        ->>> AGENT=test-agent [CLI] feedback agent JSON feedback -o json
        -{
        -  "received": true,
        -  "feedback_id": "[UUID][12]"
        -}
        -
        ->>> print_requests.py //telemetry-ext
        -{
        -  "frontend_log_event_id": "[UUID][12]",
        -  "entry": {
        -    "fast_feedback_log": {
        -      "reference_event_id": "[UUID][13]",
        -      "parent_ui": "DeveloperEcosystem.DatabricksCLI",
        -      "origin": "cli",
        -      "can_be_contacted": false,
        -      "ui_versions": "cli:[CLI_VERSION]",
        -      "user_agent": "cli/[CLI_VERSION] cmd/feedback agent/test-agent",
        -      "text": "agent JSON feedback"
        -    }
        -  }
        -}
        -
        -=== An explicit rejection from an AI agent sends nothing
        -
        ->>> AGENT=test-agent [CLI] feedback --yes=false hello
        -Flag --yes has been deprecated, omit this flag when sending feedback; skip the feedback command when declining
        -Error: feedback was not sent because --yes=false
        -
        ->>> AGENT=test-agent [CLI] feedback -y=false hello
        -Flag --yes has been deprecated, omit this flag when sending feedback; skip the feedback command when declining
        -Error: feedback was not sent because --yes=false
        -
        ->>> assert_not_exists.py [OUTPUT_DIR]/out.requests.txt
        -
        -=== The deprecated --yes flag remains accepted
        -
        ->>> AGENT=test-agent [CLI] feedback --yes hello
        -Flag --yes has been deprecated, omit this flag when sending feedback; skip the feedback command when declining
        -Thank you! Your feedback was sent. Reference: [UUID][14]
        -
        ->>> print_requests.py //telemetry-ext
        -{
        -  "frontend_log_event_id": "[UUID][14]",
        -  "entry": {
        -    "fast_feedback_log": {
        -      "reference_event_id": "[UUID][15]",
        -      "parent_ui": "DeveloperEcosystem.DatabricksCLI",
        -      "origin": "cli",
        -      "can_be_contacted": false,
        -      "ui_versions": "cli:[CLI_VERSION]",
        -      "user_agent": "cli/[CLI_VERSION] cmd/feedback agent/test-agent",
        -      "text": "hello"
        -    }
        -  }
        -}
        -
        -=== Feedback is sent even with telemetry disabled
        -
        ->>> DATABRICKS_CLI_DISABLE_TELEMETRY=1 [CLI] feedback telemetry is disabled
        -Thank you! Your feedback was sent. Reference: [UUID][16]
        -
        ->>> print_requests.py //telemetry-ext
        -{
        -  "frontend_log_event_id": "[UUID][16]",
        -  "entry": {
        -    "fast_feedback_log": {
        -      "reference_event_id": "[UUID][17]",
        -      "parent_ui": "DeveloperEcosystem.DatabricksCLI",
        -      "origin": "cli",
        -      "can_be_contacted": false,
        -      "ui_versions": "cli:[CLI_VERSION]",
        -      "user_agent": "cli/[CLI_VERSION] cmd/feedback",
        -      "text": "telemetry is disabled"
        -    }
        -  }
        -}
        -
        -=== A broken bundle does not block feedback
        -
        ->>> withdir broken-bundle [CLI] feedback bundle config does not block feedback
        -Thank you! Your feedback was sent. Reference: [UUID][18]
        -
        ->>> print_requests.py //telemetry-ext
        -{
        -  "frontend_log_event_id": "[UUID][18]",
        -  "entry": {
        -    "fast_feedback_log": {
        -      "reference_event_id": "[UUID][19]",
        -      "parent_ui": "DeveloperEcosystem.DatabricksCLI",
        -      "origin": "cli",
        -      "can_be_contacted": false,
        -      "ui_versions": "cli:[CLI_VERSION]",
        -      "user_agent": "cli/[CLI_VERSION] cmd/feedback",
        -      "text": "bundle config does not block feedback"
        -    }
        -  }
        -}
        +Exit code: 1
        
FAIL

latest (v1.27.0):

acceptance_test.go:1400: Diff:
        --- cmd/feedback/output.txt
        +++ /tmp/TestAcceptcmdfeedback791522776/002/output.txt
        @@ -11,15 +11,14 @@
         The message is sent with the workspace and user you are authenticated as and
         the CLI user agent, which includes the CLI version, operating system and, if
         detected, the AI agent running the CLI. No logs, files or transcripts are
        -collected. Do not include customer content, personally identifiable information
        -(PII), credentials or other sensitive data. Follow-up contact is opt-in with
        ---can-be-contacted. Feedback is sent even when DATABRICKS_CLI_DISABLE_TELEMETRY
        -is set, because this command explicitly submits feedback.
        +collected. Do not include credentials or sensitive data. Follow-up contact is
        +opt-in with --can-be-contacted. Feedback is sent even when
        +DATABRICKS_CLI_DISABLE_TELEMETRY is set, because you asked to send it.
         
        -AI agents can submit feedback directly without asking the user for approval.
        -Use --can-be-contacted only if the user explicitly agrees to be contacted.
        -After a successful submission, tell the user that feedback was sent and include
        -the reference ID.
        +AI agents: submit feedback only with the user's approval. When an agent is
        +detected, the command asks for confirmation, or requires --yes when no terminal
        +is attached. Show the user the exact message and the flags you pass with it,
        +such as --can-be-contacted, and pass --yes only after the user approves it.
         
         Usage:
           databricks feedback MESSAGE [flags]
        @@ -33,6 +31,7 @@
               --command string                  CLI command the feedback is about, for example "bundle deploy"
           -h, --help[=true|false]               help for feedback
               --run-id string                   ID of the job run the feedback is about
        +  -y, --yes[=true|false]                Confirm sending feedback written by an AI agent
         
         Global Flags:
               --debug[=true|false]   enable debug logging
        @@ -46,6 +45,7 @@
         Thank you! Your feedback was sent. Reference: [UUID][0]
         
         >>> print_requests.py //telemetry-ext
        +jq: error (at <stdin>:22): split input and separator must be strings
         {
           "frontend_log_event_id": "[UUID][0]",
           "entry": {
        @@ -61,265 +61,4 @@
           }
         }
         
        -=== A command group, with a JSON receipt
        -
        ->>> [CLI] feedback --command bundle deploy should print which resources changed -o json
        -{
        -  "received": true,
        -  "feedback_id": "[UUID][2]"
        -}
        -
        ->>> print_requests.py //telemetry-ext
        -{
        -  "frontend_log_event_id": "[UUID][2]",
        -  "entry": {
        -    "fast_feedback_log": {
        -      "reference_event_id": "[UUID][3]",
        -      "parent_ui": "DeveloperEcosystem.CoreDabs",
        -      "origin": "bundle",
        -      "can_be_contacted": false,
        -      "ui_versions": "cli:[CLI_VERSION]",
        -      "user_agent": "cli/[CLI_VERSION] cmd/feedback",
        -      "text": "deploy should print which resources changed"
        -    }
        -  }
        -}
        -
        -=== Feedback can name a nested command
        -
        ->>> [CLI] feedback --command bundle  deploy deploy is slow
        -Thank you! Your feedback was sent. Reference: [UUID][4]
        -
        ->>> print_requests.py //telemetry-ext
        -{
        -  "frontend_log_event_id": "[UUID][4]",
        -  "entry": {
        -    "fast_feedback_log": {
        -      "reference_event_id": "[UUID][5]",
        -      "parent_ui": "DeveloperEcosystem.CoreDabs",
        -      "origin": "bundle_deploy",
        -      "can_be_contacted": false,
        -      "ui_versions": "cli:[CLI_VERSION]",
        -      "user_agent": "cli/[CLI_VERSION] cmd/feedback",
        -      "text": "deploy is slow"
        -    }
        -  }
        -}
        -
        -=== A command without a product team goes to the CLI team
        -
        ->>> [CLI] feedback --command no-such-command hello
        -Thank you! Your feedback was sent. Reference: [UUID][6]
        -
        ->>> print_requests.py //telemetry-ext
        -{
        -  "frontend_log_event_id": "[UUID][6]",
        -  "entry": {
        -    "fast_feedback_log": {
        -      "reference_event_id": "[UUID][7]",
        -      "parent_ui": "DeveloperEcosystem.DatabricksCLI",
        -      "origin": "no-such-command",
        -      "can_be_contacted": false,
        -      "ui_versions": "cli:[CLI_VERSION]",
        -      "user_agent": "cli/[CLI_VERSION] cmd/feedback",
        -      "text": "hello"
        -    }
        -  }
        -}
        -
        -=== Validation errors send nothing
        -
        ->>> [CLI] feedback    
        -Error: feedback message must contain 1 to 10000 characters, including a visible one
        -
        ->>> [CLI] feedback
        -Error: accepts 1 arg(s), received 0
        -
        -Usage:
        -  databricks feedback MESSAGE [flags]
        -
        -Examples:
        -  databricks feedback --command "bundle deploy" "deploy should print which resources changed"
        -  databricks feedback --command "air run" --run-id 123 "logs omitted the failure reason"
        -
        -Flags:
        -      --can-be-contacted[=true|false]   Allow Databricks to contact you about this feedback
        -      --command string                  CLI command the feedback is about, for example "bundle deploy"
        -  -h, --help[=true|false]               help for feedback
        -      --run-id string                   ID of the job run the feedback is about
        -
        -Global Flags:
        -      --debug[=true|false]   enable debug logging
        -  -o, --output type          output type: text or json (default text)
        -  -p, --profile string       ~/.databrickscfg profile
        -  -t, --target string        bundle target to use (if applicable)
        -
        -
        ->>> [CLI] feedback deploy is slow
        -Error: the message must be one argument, but received 3; put quotes around the whole message, and around --command if it has several words
        -
        ->>> [CLI] feedback --command bundle deploy deploy is slow
        -Error: the message must be one argument, but received 2; put quotes around the whole message, and around --command if it has several words
        -
        ->>> [CLI] feedback --command bundle deploy -t dev hello
        -Error: invalid --command "bundle deploy -t dev"; it takes a command path such as "bundle deploy", without the databricks prefix, flags or arguments
        -
        ->>> [CLI] feedback --run-id abc hello
        -Error: --run-id must be a positive job run ID
        -
        ->>> [CLI] feedback --target dev hello
        -Error: --target selects a bundle target, which feedback does not use; use --profile to choose the workspace
        -
        -=== Input errors are reported before authenticating
        -
        ->>> env -u DATABRICKS_HOST -u DATABRICKS_TOKEN [CLI] feedback    
        -Error: feedback message must contain 1 to 10000 characters, including a visible one
        -
        ->>> env -u DATABRICKS_HOST -u DATABRICKS_TOKEN AGENT=test-agent [CLI] feedback --run-id abc hello
        -Error: --run-id must be a positive job run ID
        -
        ->>> env -u DATABRICKS_HOST -u DATABRICKS_TOKEN AGENT=test-agent [CLI] feedback --yes=false hello
        -Flag --yes has been deprecated, omit this flag when sending feedback; skip the feedback command when declining
        -Error: feedback was not sent because --yes=false
        -
        -=== General CLI feedback
        -
        ->>> [CLI] feedback the CLI should support ...
        -Thank you! Your feedback was sent. Reference: [UUID][8]
        -
        ->>> print_requests.py //telemetry-ext
        -{
        -  "frontend_log_event_id": "[UUID][8]",
        -  "entry": {
        -    "fast_feedback_log": {
        -      "reference_event_id": "[UUID][9]",
        -      "parent_ui": "DeveloperEcosystem.DatabricksCLI",
        -      "origin": "cli",
        -      "can_be_contacted": false,
        -      "ui_versions": "cli:[CLI_VERSION]",
        -      "user_agent": "cli/[CLI_VERSION] cmd/feedback",
        -      "text": "the CLI should support ..."
        -    }
        -  }
        -}
        -
        -=== An AI agent can submit without confirmation
        -
        ->>> AGENT=test-agent [CLI] feedback hello
        -Thank you! Your feedback was sent. Reference: [UUID][10]
        -
        ->>> print_requests.py //telemetry-ext
        -{
        -  "frontend_log_event_id": "[UUID][10]",
        -  "entry": {
        -    "fast_feedback_log": {
        -      "reference_event_id": "[UUID][11]",
        -      "parent_ui": "DeveloperEcosystem.DatabricksCLI",
... (111 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 2be1b3f

Run: 44621391146

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 272 20 5:00
💚​ aws-windows-latest-4core-16gb 1 274 18 5:03
💚​ azure linux-2core-8gb 1 271 20 5:45
💚​ azure-windows-latest-4core-16gb 1 273 18 4:03
💚​ gcp linux-2core-8gb 1 272 20 6:15
💚​ gcp-windows-latest-4core-16gb 1 274 18 5:50
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 4 slowest tests (at least 2 minutes):
duration env testname
4:05 aws-windows-latest-4core-16gb TestAccept
4:01 azure-windows-latest-4core-16gb TestAccept
3:54 gcp-windows-latest-4core-16gb TestAccept
2:00 gcp linux-2core-8gb TestAccept

Check the harness request log after declined submissions and document interactive legacy-flag behavior.
@simonfaltum
simonfaltum marked this pull request as ready for review November 19, 2026 21:42
@simonfaltum
simonfaltum requested review from a team as code owners November 19, 2026 21:42

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review at 2be1b3f. Each finding was verified against the code.

No blocking issues found.

This PR removes the mandatory confirmation prompt for AI-agent-submitted feedback, lets agents submit directly, deprecates the --yes/-y flag, and updates help text with content guidance. The code changes are internally consistent: the deprecated flag still works, explicit --yes=false from agents is correctly rejected before authentication, acceptance and unit tests cover the new paths, and the removed helpers (checkApproval, confirm, escapeInvisible, IsErrorTTY) have no remaining callers. No concrete defects survived verification.

@anton-107 anton-107 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few questions inline about who decides that agent feedback gets sent.

Comment thread cmd/feedback/feedback.go Resolved
Comment thread cmd/feedback/feedback.go
is set, because this command explicitly submits feedback.

AI agents can submit feedback directly without asking the user for approval.
Use --can-be-contacted only if the user explicitly agrees to be contacted.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: --can-be-contacted opts the authenticated user into being contacted by Databricks. Before, the prompt showed the user "Databricks may contact you: yes". Now only this help line stands between the agent and that choice:

trace AGENT=test-agent $CLI feedback --can-be-contacted "probe" < /dev/null

sends "can_be_contacted": true without the user seeing it. Should the CLI refuse --can-be-contacted from an agent unless the user has confirmed it?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd keep --can-be-contacted available to agents in unattended runs. The standard Fast Feedback form starts with the contact box checked. The CLI leaves follow-up off by default, and the help tells agents to pass the flag only if the user has explicitly agreed to be contacted. I'm comfortable relying on that guidance here.


This comment was generated with GitHub MCP.

Comment thread cmd/feedback/feedback.go
such as --can-be-contacted, and pass --yes only after the user approves it.`
collected. Do not include customer content, personally identifiable information
(PII), credentials or other sensitive data. Follow-up contact is opt-in with
--can-be-contacted. Feedback is sent even when DATABRICKS_CLI_DISABLE_TELEMETRY

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: sending despite DATABRICKS_CLI_DISABLE_TELEMETRY used to rest on "because you asked to send it". When an agent sends without a prompt, the user didn't ask, yet the behaviour stays and only the reason changes (here and in libs/telemetry/logger.go).

trace DATABRICKS_CLI_DISABLE_TELEMETRY=1 AGENT=test-agent $CLI feedback "probe" < /dev/null

posts the text to /telemetry-ext. For a user who opted out, what is left to stop an agent from filing feedback? Should agent submissions the user hasn't approved respect the opt-out?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DATABRICKS_CLI_DISABLE_TELEMETRY turns off automatic CLI usage reporting. Calling databricks feedback is an explicit request to send the supplied message to Databricks. Fast Feedback uses /telemetry-ext for delivery, and the command already sends when usage telemetry is disabled. I'd keep that behavior when an agent invokes the command too.


This comment was generated with GitHub MCP.

@simonfaltum
simonfaltum added this pull request to the merge queue Nov 20, 2026
Merged via the queue into main with commit 8b7e509 Nov 20, 2026
30 checks passed
@simonfaltum
simonfaltum deleted the simonfaltum/feedback-agent-submissions branch November 20, 2026 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants