Skip to content

feat(sql): support nested lambda functions#19936

Merged
bohutang merged 1 commit into
mainfrom
feat/nested-lambda
May 29, 2026
Merged

feat(sql): support nested lambda functions#19936
bohutang merged 1 commit into
mainfrom
feat/nested-lambda

Conversation

@bohutang
Copy link
Copy Markdown
Member

@bohutang bohutang commented May 29, 2026

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Remove the restriction that prevented lambda functions from being used inside other lambda functions, aligning with Snowflake's TRANSFORM behavior.

Problem: Multi-level nested array transformations (e.g. events[*].attributes[*].value.stringValue) required verbose LATERAL FLATTEN + GROUP BY workarounds because json_array_transform could not be nested inside another json_array_transform.

Root cause: A conservative guard added in the initial lambda implementation (PR #12217, 2023) blocked all lambda-in-lambda usage at the type checker level. However, the evaluator already supports recursive execution — eval_lambda_block creates a new Evaluator that naturally handles nested LambdaFunctionCall nodes.

Fix:

  1. Remove the type checker guard that rejected nested lambdas.
  2. Implement parameter shadowing via UnqualifiedWildcardInVisible — when a lambda parameter has the same name as an outer column, the outer column is hidden from unqualified lookup but remains accessible via qualified reference (e.g. table.column). This preserves backward compatibility with existing queries.

After this change:

SELECT json_array_transform(
  raw_data:events,
  e -> object_insert(e, 'attributes',
    json_array_transform(e:attributes, a ->
      CASE WHEN a:key::string = 'body'
        THEN object_insert(a, 'value',
          object_insert(a:value, 'stringValue',
            to_variant(left(a:value:stringValue::string, 50)), true), true)
        ELSE a END
    ), true)
) FROM traces_table;

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions Bot added the pr-feature this PR introduces a new feature to the codebase label May 29, 2026
@bohutang bohutang requested a review from b41sh May 29, 2026 05:31
@bohutang bohutang force-pushed the feat/nested-lambda branch from 7b4afc5 to 50e2756 Compare May 29, 2026 06:18
@bohutang bohutang force-pushed the feat/nested-lambda branch from 50e2756 to 80683e8 Compare May 29, 2026 07:37
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 80683e88d7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/query/sql/src/planner/semantic/type_check/lambda.rs Outdated
@bohutang bohutang force-pushed the feat/nested-lambda branch from 80683e8 to 7e771c6 Compare May 29, 2026 08:04
Remove the restriction that prevented lambda functions from being used
inside other lambda functions. This aligns with Snowflake's TRANSFORM
behavior and enables multi-level array transformations without requiring
LATERAL FLATTEN workarounds.

The evaluator already supports recursive lambda execution via
eval_lambda_block creating a new Evaluator. The restriction was only
in the type checker as a conservative guard from the initial lambda
implementation (PR #12217, 2023).
@bohutang bohutang force-pushed the feat/nested-lambda branch from 7e771c6 to 01d6046 Compare May 29, 2026 08:09
@bohutang bohutang added the ci-cloud Build docker image for cloud test label May 29, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Docker Image for PR

  • tag: pr-19936-f398a33-1780055850

note: this image tag is only available for internal use.

@bohutang bohutang merged commit 0f2ae5f into main May 29, 2026
116 of 117 checks passed
@bohutang bohutang deleted the feat/nested-lambda branch May 29, 2026 12:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cloud Build docker image for cloud test pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants