Repository navigation
feat(admin): Add read-only agent query passthrough endpoint - #8689
Draft
sentry-junior[bot] wants to merge 2 commits into
Draft
sentry-junior[bot] wants to merge 2 commits into
sentry-junior[bot] wants to merge 2 commits into
Conversation
Add POST /agent/query so automation clients such as Junior can run read-only SQL against ClickHouse system tables and the querylog cluster. It reuses the System Queries and Querylog validation and never runs in sudo or clusterless mode. System queries default to the storage's query node, and responses are capped at 1000 rows. Gate it on a new AgentQueries role. IAM policy bindings now accept serviceAccount: members, and a bound service account gets only its bound roles instead of the default roles every person gets. Co-Authored-By: Pierre Massat <336345+phacops@users.noreply.github.com>
The querylog table is querylog_dist in distributed mode, so the hardcoded querylog_local failed the test_distributed CI job.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
POST /agent/queryto snuba-admin. Automation clients such as Junior can use it to run read-only SQL against ClickHouse system tables and the querylog cluster.{"target": "system", "storage": "errors", "sql": "SELECT count() FROM system.parts"} {"target": "querylog", "sql": "SELECT count() FROM querylog_local"}Safety
run_system_query_on_host_with_sql/run_querylog_query): system queries can read onlysystem.*tables, and querylog queries can read only the querylog table.sudo_mode=Falseandclusterless_mode=False. Anysudokey in the body is ignored.host/portare optional and default to the storage's query node.truncatedflag.X-Snuba-Admin-Requesterheader that says which person the agent is acting for. That header is only logged and is never used for authorization.Auth
AgentQueriesrole that grants only the newagenttool. The tool is also added to/clickhouse_nodesand/clickhouse_querylog_schemaso a client can discover storages and the querylog schema.serviceAccount:<email>members.DEFAULT_ROLES(which includesProductTools). Service accounts with no binding keep today's behavior, so no existing caller changes.Still needed before this works end to end (not in this PR)
serviceAccount:→roles/AgentQueriesbinding to the deployed IAM policy file.getsentry/junior-prodthat calls this endpoint.Open questions
/agent/querythe shape we want, or should agents call the existing/run_clickhouse_system_queryand/clickhouse_querylog_querydirectly with the new role?DEFAULT_ROLES? This PR leaves them unchanged.Tests
tests/admin/test_agent_views.pycovers role resolution, the 403 check, request validation, the no-sudo guarantee, truncation, and real ClickHouse system and querylog queries. I ran it locally together withtest_authorization.py,test_api.py,test_system_queries.py, andtest_querylog_audit_log.py. The only failure wastest_merge_table_function_is_rejected, which also fails onmasteragainst my local ClickHouse. ruff and mypy pass.via @phacops.
--
View Session [Sentry]