Skip to content

support flag kParseFloatAndOverflowNumAsNumStr to parse float as string - #156

Merged
yangzhg merged 1 commit into
bytedance:masterfrom
howz97:parse_float_as_str
Nov 13, 2026
Merged

yangzhg merged 1 commit into
bytedance:masterfrom
howz97:parse_float_as_str

Conversation

@howz97

@howz97 howz97 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

hive JsonToMapUDF 的实现是不解析 double ,直接当成 string 进行序列化,所以得到结果 1e10 这样非标准的格式。

而之前的bolt是依赖解析 flag kParseOverflowNumAsNumStr 在语义上的 bug 所以能通过测试。看命名应该是仅把 overflow 的 数字当成 string 对待,但是 bug 导致 所有数字都被当成 string 对待,恰好跟 hive udf 的实现一致了。

现在 sonic master 修复了 kParseOverflowNumAsNumStr 的语义,1e10 还没有 overflow,所以 1e10 被解析成 double 再序列化就得到了 10000000000 。

如果要坚持使用 sonic master ,就需要增加一个 kParseNumAsNumStr 把float都当成 string 解析。

Comment thread include/sonic/dom/parser.h Resolved
Comment thread include/sonic/dom/flags.h Resolved
Comment thread tests/jsonpath_test.cpp Resolved
@howz97
howz97 force-pushed the parse_float_as_str branch from 89132cd to b1ff9a6 Compare November 12, 2026 12:30
@howz97
howz97 requested a review from zhangxffff November 12, 2026 12:42

@yangzhg yangzhg left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.43%. Comparing base (c8db640) to head (b1ff9a6).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #156      +/-   ##
==========================================
- Coverage   69.67%   69.43%   -0.24%     
==========================================
  Files          27       27              
  Lines        3706     3707       +1     
  Branches     1189     1189              
==========================================
- Hits         2582     2574       -8     
+ Misses        211      203       -8     
- Partials      913      930      +17     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@yangzhg
yangzhg merged commit 3ecbef2 into bytedance:master Nov 13, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants