Repository navigation
test(conformance): tag unversioned v0.9 cases in functions.yaml and actions.yaml and add v1.0 copies - #4242
test(conformance): tag unversioned v0.9 cases in functions.yaml and actions.yaml and add v1.0 copies#4242vojtech wants to merge 3 commits into
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
There was a problem hiding this comment.
Code Review
This pull request updates the conformance tests in actions.yaml and functions.yaml to support protocol version 1.0 alongside v0.9 by explicitly marking existing tests with protocolVersion: "v0.9" and adding corresponding v10 test cases with updated expectation formats. The review feedback highlights three instances in functions.yaml where new _v10 test cases were placed after section header comments, resulting in incorrect grouping and duplicated header comments. Moving these test cases before the section headers will keep the test suite properly organized.
| # --- RegexFunction (D21) --- | ||
|
|
||
| - name: test_evaluate_function_required_object_empty_v10 | ||
| description: Verifies required returns true for empty object (D20). | ||
| protocolVersion: "1.0" | ||
| action: evaluate_function | ||
| function: required | ||
| args: | ||
| value: {} | ||
| expect: | ||
| valid: true | ||
|
|
||
| # --- RegexFunction (D21) --- |
There was a problem hiding this comment.
The new test_evaluate_function_required_object_empty_v10 test case is placed after the # --- RegexFunction (D21) --- section header comment, which incorrectly groups it under the Regex section instead of the Required section. Additionally, this results in a duplicated # --- RegexFunction (D21) --- comment. Let's move the test case before the section header to keep the tests correctly grouped.
- name: test_evaluate_function_required_object_empty_v10
description: Verifies required returns true for empty object (D20).
protocolVersion: "1.0"
action: evaluate_function
function: required
args:
value: {}
expect:
valid: true
# --- RegexFunction (D21) ---| # --- LengthFunction (D24) --- | ||
|
|
||
| - name: test_evaluate_function_email_invalid_single_char_tld_v10 | ||
| description: Verifies email returns false on single-character TLD (D22). | ||
| protocolVersion: "1.0" | ||
| action: evaluate_function | ||
| function: email | ||
| args: | ||
| value: "test@test.c" | ||
| expect: | ||
| valid: false | ||
|
|
||
| # --- LengthFunction (D24) --- |
There was a problem hiding this comment.
The new test_evaluate_function_email_invalid_single_char_tld_v10 test case is placed after the # --- LengthFunction (D24) --- section header comment, which incorrectly groups it under the Length section instead of the Email section. Additionally, this results in a duplicated # --- LengthFunction (D24) --- comment. Let's move the test case before the section header to keep the tests correctly grouped.
- name: test_evaluate_function_email_invalid_single_char_tld_v10
description: Verifies email returns false on single-character TLD (D22).
protocolVersion: "1.0"
action: evaluate_function
function: email
args:
value: "test@test.c"
expect:
valid: false
# --- LengthFunction (D24) ---| # --- AndFunction (D25, D26) --- | ||
|
|
||
| - name: test_evaluate_function_numeric_unparseable_string_v10 | ||
| description: Verifies numeric returns false for unparseable string without raising (D23). | ||
| protocolVersion: "1.0" | ||
| action: evaluate_function | ||
| function: numeric | ||
| args: | ||
| value: "not_a_number" | ||
| expect: | ||
| valid: false | ||
|
|
||
| # --- AndFunction (D25, D26) --- |
There was a problem hiding this comment.
The new test_evaluate_function_numeric_unparseable_string_v10 test case is placed after the # --- AndFunction (D25, D26) --- section header comment, which incorrectly groups it under the And section instead of the Numeric section. Additionally, this results in a duplicated # --- AndFunction (D25, D26) --- comment. Let's move the test case before the section header to keep the tests correctly grouped.
- name: test_evaluate_function_numeric_unparseable_string_v10
description: Verifies numeric returns false for unparseable string without raising (D23).
protocolVersion: "1.0"
action: evaluate_function
function: numeric
args:
value: "not_a_number"
expect:
valid: false
# --- AndFunction (D25, D26) ---functions.yaml and actions.yaml expect v0.9 behaviour5d3b171 to
f5f4932
Compare
23 cases in
conformance/core/have noprotocolVersionand expect v0.9 behaviour that the v1.0 spec changed. The Python, Dart and Swift core runners default an unversioned case to v0.9, so they pass today. That default is only in the runner code:conformance_schema.jsonandconformance/README.mddo not state it, andconformance_harness_implementation_guide.mdgives the opposite default ("1.0") for agent fixtures. A harness for a 1.0-only renderer has no documented way to know these cases are v0.9, and has to fail them.functions.yaml(21 cases): bare boolean instead ofValidationResult. The cases expecttrue/false(for exampletest_evaluate_function_required_string_empty:expect: false). The v1.0 spec says these functions return aValidationResult:result.validis extracted only where the value is used as aDynamicBoolean, not for a direct call. Theregex_v10_*cases in the same file already setprotocolVersion: "1.0"and expect{valid: …}.Affected:
requiredstring_valid, string_empty, null, array_empty, array_valid, object_empty;regexmatch, mismatch;emailvalid, invalid_no_domain, invalid_single_char_tld;lengthstring_valid, string_too_short, string_too_long, array_count_valid, array_count_too_short;numericvalid_float, valid_string, too_small, too_large, unparseable_string.actions.yaml(2 cases): barepathinstead of@path.test_action_event_with_data_bindingandtest_action_event_with_relative_scopebind event context with{path: …}. In v1.0 aDataBindingis{"@path": …}(common_types.json), andreserved_keys.yamlkeeps a barepathkey as literal data. The 1.0 cases in the same file already bind with"@path".Change (against main
3ed9601862):protocolVersion: "v0.9". This matches what the runners already do, so v0.9 coverage is unchanged.functions.yamlalready tagstest_evaluate_function_length_null_valueand…_too_shortthis way._v10copies withprotocolVersion: "1.0"that expect{valid: <bool>}(19 infunctions.yaml) or bind with"@path"(2 inactions.yaml). Before this,functions.yamlhad no direct v1.0evaluate_functioncase forrequired,email,lengthornumeric.emailcases get no_v10copy. In v1.0 a failedemailcall also returnscode,messageandseverity, andvalidation_result.yamlalready covers that result.Testing
pytest tests/test_conformance.pyinpython/a2ui_core(853 passed).swift test --filter "BasicCatalogConformanceTests|ActionsConformanceTests"(both suites pass).pytest conformance/tests/test_conformance_yaml.py test_feature_registry.py test_check_executed_cases.py(172 passed), and Prettier reports both files formatted._v10copies. The tworegexcopies were not run against it.🤖 Generated with Claude Code