Skip to content

Restrict LCB private test case unpickling#1241

Open
resolvicomai wants to merge 1 commit into
huggingface:mainfrom
resolvicomai:codex/restrict-lcb-private-test-unpickle
Open

Restrict LCB private test case unpickling#1241
resolvicomai wants to merge 1 commit into
huggingface:mainfrom
resolvicomai:codex/restrict-lcb-private-test-unpickle

Conversation

@resolvicomai
Copy link
Copy Markdown

Summary

  • load LCB private test case payloads with a restricted unpickler that rejects pickle globals
  • preserve the existing pickled JSON string payload format
  • add focused coverage for normal payloads and unsupported global constructors

Tests

  • PYTHONPATH=src uv run --with pytest --with numpy --with tqdm pytest tests/unit/tasks/tasks/lcb/test_codegen_metrics.py -q
  • PYTHONPATH=src uv run --with ruff ruff check src/lighteval/tasks/tasks/lcb/codegen_metrics.py tests/unit/tasks/tasks/lcb/test_codegen_metrics.py
  • PYTHONPATH=src uv run --with ruff ruff format --check src/lighteval/tasks/tasks/lcb/codegen_metrics.py tests/unit/tasks/tasks/lcb/test_codegen_metrics.py
  • PYTHONPATH=src uv run python -m py_compile src/lighteval/tasks/tasks/lcb/codegen_metrics.py tests/unit/tasks/tasks/lcb/test_codegen_metrics.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant