Skip to content

Generate the modular Go SDK's field mapping in climod with genkit's cli-mod mode - #7268

Merged
parthban-db merged 2 commits into
mainfrom
parthban/sdk-go-field-paths
Nov 19, 2026
Merged

parthban-db merged 2 commits into
mainfrom
parthban/sdk-go-field-paths

Conversation

@parthban-db

@parthban-db parthban-db commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Changes

Adds climod/, a genkit target in the new cli-mod mode, which maps every field of the modular Go SDK (github.com/databricks/sdk-go) to its JSON path. task generate-clijson now also builds the Go SDK's proto descriptor set at the pinned Universe commit and runs genkit experimental-update-sdk --dir climod, which writes two files from the same data:

  • climod/.codegen/sdk-go-field-paths.txt, the mapping for people. For each request and response message, it lists every field's Go path next to the JSON path that the SDK writes:

    postgres/v1 CreateBranchRequest
      Parent                             parent
      BranchId                           branch_id
      Branch.Spec.SourceBranch           branch.spec.source_branch
      Branch.Spec.Expiration.Ttl         branch.spec.ttl
      ...
    

    A oneof member's Go path goes through the oneof field, while its JSON path doesn't, for example Spec.Expiration.Ttl and spec.ttl. [*] is a list element and .* is a map value, the same notation as libs/structs/structpath.

  • climod/.codegen/cli.json, the same mapping for code, in the shape of .codegen/cli.json. Each message is keyed by package path and Go type, and each field by its JSON name, with the Go path, a ref to nested messages, and for a oneof member the Go type that its oneof field holds:

    "postgres/v1.BranchSpec": {
      "fields": {
        "source_branch": { "go": "SourceBranch" },
        "ttl": { "go": "Expiration.Ttl", "oneof_variant": "BranchSpec_Expiration_Ttl" }
      }
    }

This PR doesn't check in the generated files. The daily CLI update adds them: with https://github.com/databricks-eng/universe/pull/3127766, it runs the same cli-mod step whenever climod/.codegen.json exists.

Why

The modular Go SDK's public structs carry no JSON tags, so a developer can't tell from the Go type which JSON field a struct field becomes. Its names also differ from the classic SDK's names in .codegen/cli.json, for example postgres/v1.BranchSpec with the oneof Expiration, against postgres.BranchSpec with a plain ttl field. So the files come from the Go SDK's own proto view, through a genkit mode like the SDKs', rather than from the OpenAPI spec. The mode can grow to carry more of what the CLI needs about the modular SDK.

The genkit side has merged in Universe: the mode in https://github.com/databricks-eng/universe/pull/3027478, the text file in https://github.com/databricks-eng/universe/pull/2938255, and cli.json in https://github.com/databricks-eng/universe/pull/3027479. The current .codegen/_openapi_sha (e13a7b6) has the mode but not the writers, so task generate-clijson runs the new step and writes nothing under climod/ until the pin includes Universe commit 1d13828. The daily CLI update runs update-sdk on its own rather than generate-clijson, so https://github.com/databricks-eng/universe/pull/3127766 adds the same step there.

Tests

  • The genkit change has unit tests for the field mapping (oneofs, lists and maps of messages, nested, recursive, and empty messages) and for both exact files the mode writes.
  • I ran the mode from the final commit of https://github.com/databricks-eng/universe/pull/3027479 on the Go SDK view at that commit. It wrote both files, and cli.json has 3,231 messages. Every ref resolves to a schema in the file, and each of the 771 oneof_variant values names a struct in the generated Go SDK.
  • The edited Taskfile.yml parses. I didn't run task generate-clijson end to end.

This pull request and its description were written by Isaac.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: fa4a0bd

Run: 43875489473

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 15 9:05
💚​ aws-windows-latest-4core-16gb 1 272 13 6:46
💚​ azure linux-2core-8gb 1 269 15 7:48
💚​ azure-windows-latest-4core-16gb 1 271 13 4:39
💚​ gcp linux-2core-8gb 1 270 15 8:04
💚​ gcp-windows-latest-4core-16gb 1 272 13 4:23
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 1 slowest tests (at least 2 minutes):
duration env testname
2:30 aws-windows-latest-4core-16gb TestAccept

@parthban-db
parthban-db force-pushed the parthban/sdk-go-field-paths branch from eb8f623 to 9c36f74 Compare October 30, 2026 17:39
@parthban-db parthban-db changed the title Add the JSON path of every modular Go SDK field to .codegen Generate the modular Go SDK's CLI spec in climod with genkit's cli-mod mode Oct 30, 2026
…li-mod mode

The modular Go SDK's structs carry no JSON tags, and its names differ from the classic SDK names in .codegen/cli.json. task generate-clijson now also runs genkit's cli-mod mode on climod/, which writes climod/.codegen/sdk-go-field-paths.txt, with every request and response field's Go path next to its JSON path, and climod/.codegen/cli.json, with the same mapping per message.

Co-authored-by: Isaac <no-reply@databricks.com>
@parthban-db
parthban-db force-pushed the parthban/sdk-go-field-paths branch from 9c36f74 to 6917043 Compare October 30, 2026 18:05
@parthban-db parthban-db changed the title Generate the modular Go SDK's CLI spec in climod with genkit's cli-mod mode Generate the modular Go SDK's field mapping in climod with genkit's cli-mod mode Oct 30, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR has not received an update in a while. If you want to keep this PR open, please leave a comment below or push a new commit and auto-close will be canceled.

@github-actions github-actions Bot added the Stale label Nov 14, 2026
task generate-clijson writes them once .codegen/_openapi_sha points at a Universe commit with the cli-mod writers, so this PR keeps only the climod target and the Taskfile step.

Co-authored-by: Isaac <no-reply@databricks.com>
@parthban-db
parthban-db marked this pull request as ready for review November 16, 2026 13:24
@parthban-db
parthban-db requested a review from a team as a code owner November 16, 2026 13:24
@github-actions github-actions Bot removed the Stale label Nov 17, 2026
@parthban-db
parthban-db added this pull request to the merge queue Nov 19, 2026
Merged via the queue into main with commit eb16e76 Nov 19, 2026
37 checks passed
@parthban-db
parthban-db deleted the parthban/sdk-go-field-paths branch November 19, 2026 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants