Repository navigation
fix: keep a leading + sign in the number spelling before hooks see it - #160
Merged
Merged
Conversation
The JSON5 num_literal rule consumed a leading + but returned the
unsigned token alone, while - was kept, so parse_int, parse_float and
parse_constant received a spelling that differed from the document:
loads('+NaN', parse_constant=str) gave 'NaN' and
loads('+0xff', parse_int=str) gave '0xff'. Keep the + in the token,
mirror the generated parser, and extend the two sign checks in lib.py
(the hex-prefix test in _walk_ast and _parse_int) so +0xff parses as
255 instead of falling through to base 10.
Values are unchanged; only hook spellings become as-written.
Fixes dpranke#159
Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
Regenerated-parser equivalent of the json5.g change: the + branch of num_literal now returns the sign with the token, as the - branch does. Part of dpranke#159 Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
_walk_ast and the default _parse_int helper only stripped - before the hex-prefix test; a +0xff token would fall through to base-10 int() and raise. Strip either sign, so the + forms of hex literals parse the same way the - forms do. Part of dpranke#159 Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
parse_int/parse_float/parse_constant now see '+5', '+0xff', '+1.5', '+Infinity' and '+NaN' with the sign, matching the - spellings that were already asserted. Part of dpranke#159 Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
Contributor
Author
|
Thanks for merging this one too — appreciate the quick turnaround on both! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What's wrong
The JSON5 grammar accepts a leading
+on numbers (+Infinitywas added in #63), butnum_literal's+branch consumes the sign and returns the unsigned token alone — the-branch keeps its sign. So theparse_int/parse_float/parse_constanthooks receive a spelling that differs from the document, breaking the "passed the integer string as written" contract documented inloads()/parse()(and the as-writtenparse_constantsemantics established around-NaNin #152/#157):The fix
json5/json5.g: the+branch returns'+' + n, as the-branch does.json5/parser.py(generated): the same one-token change in_num_literal__c1_, mirroring_num_literal__c0_.json5/lib.py:_walk_ast's hex-prefix test and the default_parse_inthelper strip either sign (v[:1] in ('-', '+')/s.lstrip('+-')) instead of only-, so a+0xfftoken reaches the hex branch and parses as 255 rather than falling through to base-10int()and raising.tests/lib_test.py: assert the+spellings arrive at the hooks (+5,+0xffforparse_int;+1.5forparse_float;+Infinity/+NaNforparse_constant) and+0xff→ 255 as a value.Values are unchanged everywhere (
+1→ 1,+0xf→ 15,+Infinity→ inf all keep passing); only the hook spellings become as-written.Verification
System python3, no dependencies beyond the package's own sources:
python3 -m unittest discover -s tests→ 80 tests OK (including the new+assertions).+5→5,+0xff→255,+0X2a→42,+.5→0.5,+1e3→1000.0,+Infinity→inf,+NaN→nan, plus-/unsigned forms unchanged.+ 5,++5,+-5,+,+Infinit,+Nanstill reject with the same messages.parser.pyis generated code; the change mirrors the-branch's generated form exactly so it matches what a regeneration from the updated.gwould produce.Fixes #159