A quoted free-text term in the search command, such as
`search "foo-bar" source=logs`, was emitted to query_string without its
quotes unless the value contained a space. The analyzer then split the
value into tokens and default_operator=OR matched any one of them.
The same over-match was fixed for fielded text comparisons in opensearch-project#5697, but
unfielded literals were left on the legacy whitespace check. Emit a
phrase for any user-quoted unfielded string, keeping the existing
exception for a whitespace-free value with an unescaped wildcard.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Randy Miller <randym13@gmail.com>
Description
A quoted free-text term in the
searchcommand was matched as an OR of its tokens whenever the value had no spaces.returned every document containing any one of
axya3,xya3,bahhoraaa, instead of only documents containing the whole value.Cause
#5697 made emission depend on the field's index mapping and on whether the user quoted the value, which fixed this over-match for fielded comparisons on text fields (
body="foo=bar"). Unfielded literals have no field type, soSearchLiteral.toQueryString(Function)sent them to the legacy branch, which quotes only when the value contains a space. A quoted value without spaces went out as a bare term, the analyzer split it, anddefault_operator=ORjoined the pieces.Change
SearchLiteral.toQueryString(Function)now emits a phrase for any user-quoted unfielded string. It keeps the same exception the text branch already has: a whitespace-free value with an unescaped wildcard stays unquoted, so"log_*"still works as a pattern.An unfielded term is matched against
index.query.default_field, which normally includes analyzed text, so honoring the quotes here follows the same reasoning as the text branch. On fields with a no-op analyzer a quoted phrase and a bare term resolve to the same single term, so those are unaffected.Emitted
query_stringfor unfielded literals, before and after:"foo-bar"foo\-bar"foo\-bar""foo=bar"foo=bar"foo=bar""user@example.com"user@example.com"user@example.com""error"error"error""foo bar""foo bar""foo bar""foo*"foo*foo*foo-bar(unquoted)foo\-barfoo\-barFielded comparisons and
INlists are untouched; they calltoQueryString(ExprType)directly.This also changes the v2 engine's output for quoted unfielded terms, because it reaches emission through the same method.
Testing
SearchLiteralTest: values the analyzer splits, values with whitespace, wildcard terms, escaped wildcards, unquoted terms, and a numeric literal.AstSearchExpressionTest.testSimpleSearchTerm, which expectedsearch "error"to emiterror. Its comment already read "Double quotes are preserved for phrase search".:core:test,:ppl:test,:opensearch:testand:api:testpass locally (9,515 tests), along withspotlessCheckforcoreandppl.3.9.0.0tag, replaced the plugin'scorejar in an OpenSearch 3.9.0 container, and ran the query from the issue against a text field. It returned 1 of 4 documents, where the stock 3.9.0 plugin returned 3. Quoted single words, quoted wildcards, quoted phrases with spaces, and fielded comparisons returned the same rows on both.SearchCommandITandCalciteCrossClusterSearchITcontain quoted free-text terms without spaces ("@example.com","email","user","john.doe+newsletter@company.com","Hattie"); from reading them I expect the asserted rows to be unchanged.Not addressed
The unquoted form
search foo-barstill matches either token. The docs list hyphenated values as valid unquoted terms, so this may be worth a follow-up, but without quotes there is no explicit request for a phrase and I left it alone.Related Issues
Resolves #5871
Follow-up to #5697 / #5682.
Check List
--signoffor-s.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.