Skip to content

experimental postgres query: allow comments after the terminating ';' - #7678

Merged
denik merged 4 commits into
mainfrom
denik/postgres-trailing-comment
Nov 16, 2026
Merged

denik merged 4 commits into
mainfrom
denik/postgres-trailing-comment

Conversation

@denik

@denik denik commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Changes

A ; followed only by whitespace and comments now ends the single statement in experimental postgres query.

Why

"SELECT 1; -- done", "SELECT 1; /* done */" and any --file ending with a comment line were rejected as multiple statements before connecting. Postgres treats the comment-only remainder as an empty statement.

Tests

New acceptance test acceptance/experimental/postgres/query/trailing-comment and unit cases; "SELECT 1; SELECT 2" is still rejected, and the error now points at the right argv index.

Found by analyzing test coverage gaps: the code path was not covered by any test.

This pull request and its description were written by Isaac.

denik and others added 2 commits November 15, 2026 09:17
…er ';'

Records current behavior: a single statement followed by a comment after
its terminating ';' is rejected as multiple statements.

Co-authored-by: Isaac <no-reply@databricks.com>
checkSingleStatement treated any ';' not at the very end of the input as a
statement separator, so a single statement followed by a trailing comment
("SELECT 1; -- done", or a .sql file ending in a comment line) was rejected
as multiple statements. Postgres parses the comment-only remainder as an
empty statement and accepts the input, so the pre-flight check now only
rejects a ';' followed by something other than whitespace and comments.

Co-authored-by: Isaac <no-reply@databricks.com>
@denik
denik requested a review from a team as a code owner November 16, 2026 10:28
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Commit: c279f1592 | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

1 tests: regression=1 unreleased=0 coverage=0 failing=0
test branch main (bf8caad) latest (v1.26.0)
TestAccept/experimental/postgres/query/trailing-comment ✅ ❌ ❌
TestAccept/experimental/postgres/query/trailing-comment: branch ✅ | main (bf8caad) ❌ | latest (v1.26.0) ❌

main (bf8caad):

acceptance_test.go:1374: Diff:
        --- experimental/postgres/query/trailing-comment/output.txt
        +++ /tmp/TestAcceptexperimentalpostgresquerytrailing-comment3640781852/002/output.txt
        @@ -2,22 +2,31 @@
         === Trailing line comment after ';'
         
         >>> musterr [CLI] experimental postgres query --target my-instance SELECT 1; -- done
        -Error: database instance "my-instance" is not ready for accepting connections (state: STOPPED)
        +Error: argv[1]: input contains multiple statements (a ';' separates two or more statements)
        +This command runs one statement per input. To run multiple statements:
        +  - Pass each as a separate positional:  query "SELECT 1" "SELECT 2"
        +  - Pass each in its own --file:         query --file q1.sql --file q2.sql
         
         === Trailing block comment after ';'
         
         >>> musterr [CLI] experimental postgres query --target my-instance SELECT 1; /* done */
        -Error: database instance "my-instance" is not ready for accepting connections (state: STOPPED)
        +Error: argv[1]: input contains multiple statements (a ';' separates two or more statements)
        +This command runs one statement per input. To run multiple statements:
        +  - Pass each as a separate positional:  query "SELECT 1" "SELECT 2"
        +  - Pass each in its own --file:         query --file q1.sql --file q2.sql
         
         === File ending with a comment line after the statement
         
         >>> musterr [CLI] experimental postgres query --target my-instance --file q.sql
        -Error: database instance "my-instance" is not ready for accepting connections (state: STOPPED)
        +Error: --file q.sql: input contains multiple statements (a ';' separates two or more statements)
        +This command runs one statement per input. To run multiple statements:
        +  - Pass each as a separate positional:  query "SELECT 1" "SELECT 2"
        +  - Pass each in its own --file:         query --file q1.sql --file q2.sql
         
         === Two statements are still rejected
         
         >>> musterr [CLI] experimental postgres query --target my-instance SELECT 1; -- one SELECT 1; SELECT 2
        -Error: argv[2]: input contains multiple statements (a ';' separates two or more statements)
        +Error: argv[1]: input contains multiple statements (a ';' separates two or more statements)
         This command runs one statement per input. To run multiple statements:
           - Pass each as a separate positional:  query "SELECT 1" "SELECT 2"
           - Pass each in its own --file:         query --file q1.sql --file q2.sql
        
FAIL

latest (v1.26.0): same output as main (bf8caad)

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: c279f15

Run: 43880631067

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 16 6:14
💚​ aws-windows-latest-4core-16gb 1 272 14 3:47
💚​ azure linux-2core-8gb 1 269 16 4:36
💚​ azure-windows-latest-4core-16gb 1 271 14 3:25
💚​ gcp linux-2core-8gb 1 270 16 7:17
💚​ gcp-windows-latest-4core-16gb 1 272 14 3:57
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at b7ed2df. Each finding was verified against the code; simonfaltum may follow up.

No blocking issues found, approving. 4 non-blocking comment(s) and 2 question(s) below.

This PR lets experimental postgres query accept a single statement followed by -- or /* */ comments after its terminating ;, and adds unit cases plus the first acceptance test for the command. The change is correct: Postgres drops the empty trailing statement, SELECT 1; SELECT 2 is still rejected, and the error now names the right argv index. CI, the unit tests and lint are green. One non-blocking suggestion: end line comments at \r as well as \n, so CR-only inputs can't slip a second statement past the pre-flight check. There are three small nits and two optional questions about structure, a single scan pass and the acceptance test's indirect signal.

Questions

  1. onlyWhitespaceAndComments is a second scan that repeats the main loop's -- and /* handling, and SELECT 1;;, SELECT 1; ; and SELECT 1; -- x\n; are still rejected, although Postgres drops those empty statements just as it drops the comment-only remainder. Would you consider a single pass instead, setting a terminated flag on ; and rejecting only on real content after it? That keeps one copy of the comment logic and would also make the TrimRight unnecessary. Or is the narrower change intended? (non-blocking)
  2. The acceptance test infers that the pre-flight check accepted an input from the unrelated not ready for accepting connections (state: STOPPED) targeting error, so a change to that message or to the order of checks in runQuery would churn it. Apart from the --file case, it covers the same inputs as the new unit cases. Is the indirect signal intended, or would you keep only the --file case (or drop the acceptance test)? (non-blocking)

Comments outside the changed lines

  • [nit] experimental/postgres/cmd/multistatement.go:31: onlyWhitespaceAndComments already skips trailing whitespace, so the strings.TrimRight(s, " \t\r\n") is no longer what allows a trailing ;, despite what its comment says. end := len(s) gives the same result for every input. Keeping the trim also puts the whitespace set in two places (the cutset string and the comparisons at line 112), which can drift apart. Use len(s) and drop the comment.


func scanLineComment(s string, start, end int) int {
i := start + 2
for i < end && s[i] != '\n' {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] scanLineComment stops only at \n, but the Postgres lexer ends -- comments at \r too (non_newline [^\n\r] in scan.l). Since onlyWhitespaceAndComments now reuses it, "SELECT 1; -- x\rSELECT 2" passes the pre-flight check, which rejected it before this PR. With several inputs (e.g. query "INSERT ..." --file cr_endings.sql), the earlier inputs now run, and may write, before the server rejects this one with "cannot insert multiple commands into a prepared statement". The main loop already had the same gap ("SELECT 1 -- x\r; SELECT 2"). Stopping the loop at s[i] != '\n' && s[i] != '\r' fixes both; a CR-only regression case in TestCheckSingleStatement would pin it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 71ddc47: scanLineComment stops at \r as well as \n, which closes both gaps (SELECT 1; -- x\rSELECT 2 and SELECT 1 -- x\r; SELECT 2); both are unit cases now.

Comment on lines +13 to +15
printf 'SELECT 1;\n-- end of file\n' > q.sql
trace musterr $CLI experimental postgres query --target my-instance --file q.sql
rm q.sql

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] The script creates q.sql with static content via printf and removes it afterwards. .agents/rules/testing.md asks for static inputs to be committed to the test directory instead of created in script. Commit trailing-comment/q.sql with that content and drop the printf and rm lines; output.txt does not change, because neither line is traced.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 71ddc47: trailing-comment/q.sql is committed and the printf/rm lines are gone; output.txt is unchanged.

return end
}

func onlyWhitespaceAndComments(s string, start, end int) bool {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] .agents/rules/comments.md asks for Go functions to have a doc comment that starts with the function name and ends with a period; the new onlyWhitespaceAndComments has none. A line such as // onlyWhitespaceAndComments reports whether s[start:end] holds only whitespace and non-nesting comments; an unterminated /* counts as a comment. would cover it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 71ddc47: the helpers (isSQLSpace, startsComment, scanLineComment) have doc comments; onlyWhitespaceAndComments is gone (see the single-pass change).

… pass

Address review feedback: a single pass accepts any run of whitespace,
comments and ';' after the terminating ';' (Postgres drops those empty
statements), so 'SELECT 1;;' is accepted too. Line comments end at '\r'
as well as '\n', like the Postgres lexer, so a CR can't hide a second
statement. The test's input file is committed instead of written by the
script.

Co-authored-by: Isaac <no-reply@databricks.com>
@denik

denik commented Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @simonfaltum — addressed in 71ddc47:

  • Single pass: on ; the loop sets a terminated flag and keeps scanning; after it only whitespace, comments and further ; are accepted. This removes the second scanner and the TrimRight, and SELECT 1;;, SELECT 1; ;, SELECT 1; -- a\n; are accepted now (unit cases added).
  • \r ends line comments, as in the Postgres lexer.
  • Acceptance test: I kept the inline-argument cases next to --file since they're what users type at the shell, and the STOPPED-instance error is the cheapest signal that the pre-flight check passed. Happy to trim it to the --file case if you prefer.

The scanner ended a block comment at the first "*/", while Postgres nests
them. In "SELECT 1; /* /* */--*/ SELECT 2" the "--" after the inner "*/"
then hid the second statement from the pre-flight check. Track nesting
depth, which also accepts "SELECT 1; /* /* */ */".

Co-authored-by: Isaac <no-reply@databricks.com>
@denik

denik commented Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

Follow-ups (out of scope for this PR), from an adversarial review — gaps in the pre-flight scanner that already exist on main (the server's extended protocol still rejects a second statement):

  • scanQuoted ignores backslash escapes in E'...' strings (SELECT E'\\''; SELECT 2; --').
  • A $ inside an identifier is read as a dollar-quote opener (SELECT a$b$; SELECT 2; --$b$).
  • No test pins the server-side rejection of multi-command strings (QueryExecModeExec), which every scanner gap relies on.

@denik
denik added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit fd0bad5 Nov 16, 2026
27 checks passed
@denik
denik deleted the denik/postgres-trailing-comment branch November 16, 2026 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants