Skip to content

Accept whole-number floats returned as ints by the Python mutator - #7842

Open
denik wants to merge 2 commits into
denik/acc-pydabs-numeric-variablefrom
denik/pydabs-numeric-variable-fix
Open

denik wants to merge 2 commits into
denik/acc-pydabs-numeric-variablefrom
denik/pydabs-numeric-variable-fix

Conversation

@denik

@denik denik commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Changes

When merging Python mutator output, an int equal to the original float is treated as unchanged and the original value is kept.

Why

JSON doesn't tell 1.0 from 1, so a float variable default came back from Python as an int and failed the bundle with unexpected change at "variables.id.default" (update). See #7841.

Not fixed: a whole float beyond int64 (e.g. 0012345678901234567890) still fails, because Python returns an int our YAML loader can't parse. That needs a loader change and is left as is.

Tests

Updated golden from #7841; unit tests in apply_python_output_test.go.

This pull request and its description were written by Isaac.

JSON doesn't tell 1.0 from 1, so a float variable default comes back from
Python as an int. Treat an int equal to the original float as unchanged and
keep the original value.

Co-authored-by: Isaac <no-reply@databricks.com>
@denik
denik added this pull request to stack #7843 November 20, 2026 09:39
Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions github-actions Bot added DABs DABs related issues PyDABs labels Nov 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Commit: c942a1fea | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

4 tests: regression=4 unreleased=0 coverage=0 failing=0
test branch main (f4a7fe5) latest (v1.27.0)
TestAccept/bundle/python/numeric-variable-default/DMS=/PYDAB_VERSION=0.266.0 ✅ ❌ ❌
TestAccept/bundle/python/numeric-variable-default/DMS=/PYDAB_VERSION=current ✅ ❌ ❌
TestAccept/bundle/python/numeric-variable-default/DMS=true/PYDAB_VERSION=0.266.0 ✅ ❌ ❌
TestAccept/bundle/python/numeric-variable-default/DMS=true/PYDAB_VERSION=current ✅ ❌ ❌
TestAccept/bundle/python/numeric-variable-default/DMS=/PYDAB_VERSION=0.266.0: branch ✅ | main (f4a7fe5) ❌ | latest (v1.27.0) ❌

main (f4a7fe5):

acceptance_test.go:1400: Diff:
        --- bundle/python/numeric-variable-default/output.txt
        +++ /tmp/TestAcceptbundlepythonnumeric-variable-defaultDMS=PYDAB_VER457048964/002/output.txt
        @@ -7,16 +7,28 @@
         === leading_zero
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t leading_zero --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default" (update)
        +
        +
        +Exit code: 1
         {"default":12345678901234,"value":12345678901234}
         
         === float_whole
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t float_whole --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default" (update)
        +
        +
        +Exit code: 1
         {"default":1,"value":1}
         
         === exponent
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t exponent --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default" (update)
        +
        +
        +Exit code: 1
         {"default":1000,"value":1000}
         
         === float_fraction
        @@ -27,6 +29,10 @@
         === complex
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t complex --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default.nested" (update)
        +
        +
        +Exit code: 1
         {"default":{"nested":1},"value":{"nested":1}}
         
         === huge

latest (v1.27.0): same output as main (f4a7fe5)

TestAccept/bundle/python/numeric-variable-default/DMS=/PYDAB_VERSION=current: branch ✅ | main (f4a7fe5) ❌ | latest (v1.27.0) ❌

main (f4a7fe5):

acceptance_test.go:1400: Diff:
        --- bundle/python/numeric-variable-default/output.txt
        +++ /tmp/TestAcceptbundlepythonnumeric-variable-defaultDMS=PYDAB_VER2525073509/002/output.txt
        @@ -7,16 +7,28 @@
         === leading_zero
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t leading_zero --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default" (update)
        +
        +
        +Exit code: 1
         {"default":12345678901234,"value":12345678901234}
         
         === float_whole
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t float_whole --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default" (update)
        +
        +
        +Exit code: 1
         {"default":1,"value":1}
         
         === exponent
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t exponent --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default" (update)
        +
        +
        +Exit code: 1
         {"default":1000,"value":1000}
         
         === float_fraction
        @@ -27,6 +29,10 @@
         === complex
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t complex --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default.nested" (update)
        +
        +
        +Exit code: 1
         {"default":{"nested":1},"value":{"nested":1}}
         
         === huge

latest (v1.27.0): same output as main (f4a7fe5)

TestAccept/bundle/python/numeric-variable-default/DMS=true/PYDAB_VERSION=0.266.0: branch ✅ | main (f4a7fe5) ❌ | latest (v1.27.0) ❌

main (f4a7fe5):

acceptance_test.go:1400: Diff:
        --- bundle/python/numeric-variable-default/output.txt
        +++ /tmp/TestAcceptbundlepythonnumeric-variable-defaultDMS=truePYDAB1516913527/002/output.txt
        @@ -7,16 +7,28 @@
         === leading_zero
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t leading_zero --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default" (update)
        +
        +
        +Exit code: 1
         {"default":12345678901234,"value":12345678901234}
         
         === float_whole
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t float_whole --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default" (update)
        +
        +
        +Exit code: 1
         {"default":1,"value":1}
         
         === exponent
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t exponent --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default" (update)
        +
        +
        +Exit code: 1
         {"default":1000,"value":1000}
         
         === float_fraction
        @@ -27,6 +29,10 @@
         === complex
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t complex --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default.nested" (update)
        +
        +
        +Exit code: 1
         {"default":{"nested":1},"value":{"nested":1}}
         
         === huge

latest (v1.27.0): same output as main (f4a7fe5)

TestAccept/bundle/python/numeric-variable-default/DMS=true/PYDAB_VERSION=current: branch ✅ | main (f4a7fe5) ❌ | latest (v1.27.0) ❌

main (f4a7fe5):

acceptance_test.go:1400: Diff:
        --- bundle/python/numeric-variable-default/output.txt
        +++ /tmp/TestAcceptbundlepythonnumeric-variable-defaultDMS=truePYDAB2811971768/002/output.txt
        @@ -7,16 +7,28 @@
         === leading_zero
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t leading_zero --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default" (update)
        +
        +
        +Exit code: 1
         {"default":12345678901234,"value":12345678901234}
         
         === float_whole
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t float_whole --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default" (update)
        +
        +
        +Exit code: 1
         {"default":1,"value":1}
         
         === exponent
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t exponent --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default" (update)
        +
        +
        +Exit code: 1
         {"default":1000,"value":1000}
         
         === float_fraction
        @@ -27,6 +29,10 @@
         === complex
         
         >>> uv run [UV_ARGS] -q [CLI] bundle validate -t complex --output json
        +Error: internal error when merging output of Python mutator: unexpected change at "variables.id.default.nested" (update)
        +
        +
        +Exit code: 1
         {"default":{"nested":1},"value":{"nested":1}}
         
         === huge
        
FAIL

latest (v1.27.0): same output as main (f4a7fe5)

@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: c942a1f

Run: 44712849078

Env 🔄​flaky ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 273 20 3:19
✅​ aws-windows-latest-4core-16gb 275 18 3:58
✅​ azure linux-2core-8gb 272 20 2:56
🔄​ azure-windows-latest-4core-16gb 2 272 18 8:33
✅​ gcp linux-2core-8gb 273 20 3:17
✅​ gcp-windows-latest-4core-16gb 275 18 3:53
Test Name azure-windows-latest-4core-16gb
🔄​ TestDashboardAssumptions_WorkspaceImport 🔄​f
🔄​ TestSyncIncrementalFileOverwritesFolder 🔄​f
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:51 gcp-windows-latest-4core-16gb TestAccept
3:44 aws-windows-latest-4core-16gb TestAccept
3:42 azure-windows-latest-4core-16gb TestAccept

@denik
denik marked this pull request as ready for review November 20, 2026 12:56
@denik
denik requested review from a team as code owners November 20, 2026 12:56

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review at c942a1f. Each finding was verified against the code.

Not ready yet: 1 blocking issue(s), tagged [blocking].

This PR fixes the Python mutator rejecting whole-number floats (like 1.0) that come back from Python as ints, by adding an isWholeFloatAsInt guard in VisitUpdate. The approach is reasonable and the tests cover the main cases. Two issues remain: the changelog entry will fail CI because it lacks the required trailing PR link, and the float/int comparison can silently drop a genuine change for large values beyond 2^53 due to float64 precision loss—a pattern the codebase already guards against elsewhere.

@@ -0,0 +1 @@
* Fix `bundle` commands failing with "unexpected change at variables.X.default" when a bundle with Python code has a variable default such as `1.0`, `1e3` or `0012345678901234`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] .nextchanges/README.md requires a trailing PR link on every entry and CI enforces it. Append ([#7842](https://github.com/databricks/cli/pull/7842)) after the period.

// back from Python as an int.
func isWholeFloatAsInt(left, right dyn.Value) bool {
return left.Kind() == dyn.KindFloat && right.Kind() == dyn.KindInt &&
left.MustFloat() == float64(right.MustInt())

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] float64(right.MustInt()) is lossy for any int64 whose absolute value exceeds 2^53: e.g. float64(9007199254740993) rounds to 9007199254740992.0, so if the original float is 9007199254740992.0 the comparison returns true and the real update is silently dropped. normalizeFloat in libs/dyn/convert/normalize.go:441 already guards against exactly this with a round-trip check. Adding && right.MustInt() == int64(left.MustFloat()) here would close the gap.

@Sankalp-Mittal Sankalp-Mittal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI Generated] Review by review-factory (Claude Opus 5.5 + Claude Fable 5.2, cross-verified). Only points the reviewers agreed on are posted.

// back from Python as an int.
func isWholeFloatAsInt(left, right dyn.Value) bool {
return left.Kind() == dyn.KindFloat && right.Kind() == dyn.KindInt &&
left.MustFloat() == float64(right.MustInt())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI Generated] Above 2^53 Python returns the float's shortest decimal, not its exact value: 1234567890123456789.0 comes back as 1234567890123456800. This check accepts it, but an exact round-trip would not. Can a test pin that case?

}

// isWholeFloatAsInt reports whether right is the int Python returns for the float left.
// JSON doesn't tell 1.0 from 1, so a float such as a variable default of 1.0 comes

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI Generated] nit: It's json.Marshal in writeInputFile (python_mutator.go:506) that writes 1.0 as 1; Python's json keeps 1.0 a float. Could the comment name that cause instead of JSON itself?

VisitUpdate: func(valuePath dyn.Path, _, right dyn.Value) (dyn.Value, error) {
VisitUpdate: func(valuePath dyn.Path, left, right dyn.Value) (dyn.Value, error) {
if isWholeFloatAsInt(left, right) {
return left, nil

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI Generated] nit: applyPythonOutput's doc comment lists where the result differs from Python's output (kept locations, undone empty deletes). Keeping the original float when Python returns a whole int is a third case; add a bullet there?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues PyDABs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants