Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion edgebench/tasks/schemathesis_datagen_pipeline/NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Sibling of `schemathesis_config_modernization` (same task authors, same recipe).
(hatchling 1.30.1). The editable install comes out byte-identical to v1's. Base: `bases/python`.
- `judge/structured_result.py`, `judge/write_result.py`, `judge/run.sh`: v1's eval program and
parser, shared with the sibling task.
- `instruction.md`: v1's prompt, verbatim.
- `instruction.md`: v1's prompt, verbatim in 1.0.0; 1.0.1 adds the Target 1 interface.

## Deviations from v1

Expand Down Expand Up @@ -52,6 +52,16 @@ range only from 0.3235 to 0.6718; the submissions below cover that range.

Every row is valid, matches exactly with zero spread, and has identical per-phase summaries.

## Changes

### 1.0.1

`TODO.md` does not give the module path, names or format keys that hidden Phase 1 imports, or the
literal Range forms it checks, so a module that follows the requirements with other names scores 0
of 15 on Phase 1 (15.38 of 100 points). `instruction.md` now states them. The judge
and work images are unchanged (same tags), so grading is unchanged. `TODO.md` stays as in v1
because the work image pins v1's commit.

## Known issues

Kept as in v1, so scores stay comparable. Fixing any of them changes scores and bumps `version`.
Expand Down
16 changes: 15 additions & 1 deletion edgebench/tasks/schemathesis_datagen_pipeline/instruction.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,20 @@ You are an expert software engineer implementing a roadmap task in the current r

Read `/app/TODO.md` completely before editing code. It contains the full project overview, goals, target requirements, and completion criteria.

## Target 1 Module Interface

`/app/TODO.md` does not name the following; they are part of Target 1. The new module is `schemathesis.specs.openapi.headers`, and it exposes:

- `KNOWN_HEADER_FORMATS`: dict from lowercase header name to format key. `if-match` and `if-none-match` map to `_if_match_header`, `if-modified-since` and `if-unmodified-since` to `_http_date_header`, and `range` to `_range_header`.
- `if_match_values()`, `http_date_values()`, `range_values()`, `range_slightly_invalid_values()`: each takes no arguments and returns a Hypothesis strategy of strings.
- `get_header_format_strategies(mode)`: takes a `GenerationMode` and returns a dict from those three format keys to strategies.

`if_match_values()` lists allow spaces after a comma, but none before a comma or around the whole list. `range_values()` serializes as lowercase `bytes=` followed by comma-separated range specs with no spaces between them.

`range_slightly_invalid_values()` generates four forms: inverted bounds (`bytes=5-2`), a negative first position (`bytes=-1-N`), a wrong unit starting with `invalid=` (`invalid=0-100`), and the exact empty range set `bytes=`.

In negative mode, the `_range_header` strategy mixes a valid Range value, the exact value `bytes=`, and a random value that starts with neither `bytes=` nor `invalid=`.

## Workflow

1. Read the task document before making code changes.
Expand All @@ -17,4 +31,4 @@ Read `/app/TODO.md` completely before editing code. It contains the full project

- Treat `/app/TODO.md` as the authoritative task specification.
- Do not modify hidden tests.
- Do not rely on external task text beyond the task document.
- Do not rely on external task text beyond the task document and the interface above.
2 changes: 1 addition & 1 deletion edgebench/tasks/schemathesis_datagen_pipeline/task.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ name = "Schemathesis Datagen Pipeline"
category = "Systems & Software Engineering"
language = "Python"
description = "Implement eight feature targets in the Schemathesis Python API testing framework, including structured HTTP header generation strategies, coverage-phase hooks, discriminator-aware validation and data generation, and schema-driven code generation fixes."
version = "1.0.0"
version = "1.0.1"
status = "migrated"

[environment]
Expand Down