warehouse_forklift_routing: raise the per-input limit to the stated 150 s

edgebench/tasks/warehouse_forklift_routing/NOTES.md

@@ -1,26 +1,38 @@
 # warehouse_forklift_routing
 
 CodeChef WAREHOUS, in ALE-Bench style. The agent writes `solution.py` (stdin -> stdout). The
-judge runs it on 50 fixed inputs (each with T=5 grids, 6 ≤ R, C ≤ 20), 8 at a time with a 60 s
+judge runs it on 50 fixed inputs (each with T=5 grids, 6 ≤ R, C ≤ 20), 8 at a time with a 150 s
 wall-clock limit each, scores each output with `tools/bin/tester` (the average over the 5 grids of
 `(S+2)/(R+C−1) − 2RC + 20`, S = command length, printed with one decimal), and sums the 50
 per-input scores. If any input is not `OK` the total is `inf`. Raw score = that sum (minimize).
 The rescale is the v1 one (`piecewise_log_min`).
 
+## Changes
+
+### 1.1.0
+
+`TIME_LIMIT` goes from 60 s to 150 s per input, the instruction's 30 s per test case for T=5. A
+solver taking 60 to 150 s on any input scored `inf`; it now scores. The formula and pool are
+unchanged; tested valid submissions moved at most 0.15 raw between judge means. Nominal grading time
+is at most 7 rounds of (150 s + 30 s tester) = 1260 s, excluding overhead, inside `timeout_sec`. It
+can outlast the 900 s final wait in `sforge run`: there, a 140 s deadline grade was still running at
+that wait and was lost, though an earlier valid score survives.
+
 ## Source
 
 - Workspace (READMEs, `tools/bin/{gen,tester}`, `tools/src/`) and the 50 judge inputs: the task
   authors' repository (not public), identical by sha256 and mode to the published v1 images. The
   tools are Python scripts.
 - `environment/workspace/solution.py` (starter) and `judge/eval_warehouse_manager.py`:
-  byte-identical to v1's inline scripts and to the published v1 images. `judge/grade.py` is new and
+  byte-identical to v1's inline scripts and to the published v1 images, except `TIME_LIMIT`, changed
+  in 1.1.0. `judge/grade.py` is new and
   writes `result.json` with v1's scoring.
 - `instruction.md`: v1's prompt, verbatim. Base: `bases/python310`.
 
 ## Deviations from v1
 
 1. Root only: no `agent` user and no `sudo` (SCHEMA rule 3).
-2. The eval script moved from `/tmp/` to `/judge/` (content unchanged).
+2. The eval script moved from `/tmp/` to `/judge/` (content unchanged until 1.1.0, see Changes).
 3. **`inf` becomes invalid.** When any input fails (including the empty submission, whose starter
    prints empty lines), v1 reported score `inf`, which was not rankable; the new grader returns
    `valid=false`. Same ranking effect.
@@ -51,18 +63,17 @@ Kept as in v1, so scores stay comparable. Fixing any of them changes scores and
 
 1. **The scored inputs are reproducible.** They come from the public generator with a fixed seed
    range; the instruction says to use seeds 0..10000 only, but nothing enforces it.
-2. **Time limit mismatch.** The instruction and README say 30 s per test case; the judge allows
-   60 s per input (5 test cases), and the tester gets its own 30 s.
-3. **Host-dependent score.** Inputs run 8 in parallel with no CPU pinning, so time-budgeted
+2. **Host-dependent score.** Inputs run 8 in parallel with no CPU pinning, so time-budgeted
    searches score differently with host speed and load, and a slow host can turn a near-limit
-   solution into a timeout.
-4. **One failed input makes the whole submission unrankable**, including a timeout caused by host
+   solution into a timeout. At the official 4 CPUs each input gets about half a CPU, so a solver
+   doing fixed work above a computed 75 s of CPU per input still times out.
+3. **One failed input makes the whole submission unrankable**, including a timeout caused by host
    load.
-5. **No judge resource limits.** The instruction's 1 GB memory limit is not enforced.
-6. Per-input scores are rounded to one decimal before summing.
-7. The score parser does not accept negative numbers; no real submission comes close, so this is
+4. **No judge resource limits.** The instruction's 1 GB memory limit is not enforced.
+5. Per-input scores are rounded to one decimal before summing.
+6. The score parser does not accept negative numbers; no real submission comes close, so this is
    theoretical.
-8. **The submission can tamper with grading.** Solutions run as root next to the grader and the
+7. **The submission can tamper with grading.** Solutions run as root next to the grader and the
    tester (the cross-task issue in `tasks/SCHEMA.md`).
-9. **The starter does not pass.** The instruction calls `solution.py` a baseline to improve, but it
+8. **The starter does not pass.** The instruction calls `solution.py` a baseline to improve, but it
    prints empty lines that the tester rejects.

edgebench/tasks/warehouse_forklift_routing/judge/Dockerfile

@@ -15,7 +15,7 @@
 #   9 rm -rf .git && git init && git config user.* && add && commit -m init
 #                                                                -> kept, author/dates pinned to v1's commit
 #  10 heredoc > /tmp/eval_warehouse_manager.py; chmod +x         -> COPY judge/eval_warehouse_manager.py
-#                                                                   /judge/ (byte-identical, moved out of /tmp)
+#                                                                   /judge/ (moved out of /tmp; TIME_LIMIT changed in 1.1.0)
 #   v1 eval_cmd + parser score_sum                               -> judge/grade.py (task.toml [judge].cmd)
 # (v1 judge image has no tools/src/ and no tools/README.md; neither is needed: tester is self-contained.)
 

edgebench/tasks/warehouse_forklift_routing/judge/eval_warehouse_manager.py

@@ -6,7 +6,7 @@
 WORK_DIR = "/home/workspace/warehouse_manager"
 CASES_DIR = os.path.join(WORK_DIR, "cases")
 TESTER = os.path.join(WORK_DIR, "tools", "bin", "tester")
-TIME_LIMIT = 60
+TIME_LIMIT = 150
 
 def run_case(args):
     i, case_file = args

edgebench/tasks/warehouse_forklift_routing/judge/grade.py

@@ -5,7 +5,7 @@
   v1 eval_cmd : cd /home/workspace/warehouse_manager && python3 /tmp/eval_warehouse_manager.py
   v1 parser   : score_sum  (sforge/harness/log_parsers/score_sum.py)
   v1 score    : first `TOTAL_SCORE <n>` in the eval output (sforge/harness/grading.py:extract_score)
-/judge/eval_warehouse_manager.py is byte-identical to the v1 /tmp/eval_warehouse_manager.py.
+/judge/eval_warehouse_manager.py is the v1 /tmp/eval_warehouse_manager.py, except TIME_LIMIT (NOTES.md).
 This wrapper only runs it and translates its stdout into /opt/sforge/result.json.
 stdout = logs only.
 """

edgebench/tasks/warehouse_forklift_routing/task.toml

@@ -4,7 +4,7 @@ name = "Warehouse Forklift Routing"
 category = "Combinatorial Optimization"
 language = "Python"
 description = "Route a forklift in a grid warehouse to receive goods arriving in random order, store them, and dispatch them in sequential order, minimizing total movement."
-version = "1.0.0"
+version = "1.1.0"
 status = "migrated"
 
 [environment]
@@ -26,7 +26,7 @@ timeout_sec = 3600
 cpus = 4
 memory = "8g"
 # Official leaderboard limits (v1 kept them only in the experiment YAMLs).
-# eval_warehouse_manager.py runs the 50 cases 8 at a time, each `python3 solution.py` under a 60 s
+# eval_warehouse_manager.py runs the 50 cases 8 at a time, each `python3 solution.py` under a 150 s
 # wall-clock timeout. Scores of time-limited (anytime) solutions depend on host CPU speed and
 # load; see NOTES.md.