FEAT: paginated seed browsing

[Jackson Severino da Rocha (jacksonsr451)]

Description

Fixes #2748. This PR adds a read-only REST API to browse the seeds that are stored in memory, one page at a time. It uses the dataset selection_key from #2762 and follows the contract from Roman Lutz (@romanlutz) in #2748. This comment lists the places where it is different from that contract, and why.

Routes

• GET /api/datasets/seeds?selection_key=<key> returns one page of logical examples.
• GET /api/datasets/seeds/{example_id}?selection_key=<key> returns all seeds of one example.

A logical example is the set of seeds that have the same prompt_group_id. A seed without a group is an example by itself, and its ID is the seed ID.

Behavior

• Pagination: The API uses opaque keyset cursors. A cursor is valid only for the same selection_key and filters. A bad or mismatched cursor returns 400.
• Order: Examples are sorted by the earliest date_added of all their members, newest first, then by example ID.
• Filters: The list accepts modality, seed_type, and harm_category. Values of one filter use OR, and different filters use AND. Different members of one example can match different filters, and the API returns the complete example. Harm categories match complete values and ignore case.
• Search: search finds literal text in text prompts and objectives, and ignores case. %, _, [, and \ are not patterns. Simulated-conversation configurations are not searched; use seed_type=simulated_conversation to find them.
• List items: Each item has a preview of at most 100 characters and a preview_truncated flag. It also has the seed types, modalities, piece and objective counts, harm categories, and has_unlabeled_harm. Media members show only the file name. No list item shows an absolute path or a URL query string.
• Detail: members is a list of domain seeds (SeedPrompt, SeedObjective, or SeedSimulatedConversation), objectives first, then by sequence. These are the same objects as get_seeds() returns, with IDs, group IDs, provenance, hashes, parameters, and conditions.
• No side effects: Browsing does not render templates, run simulated conversations, read media, call targets, or write to memory. If a stored seed cannot be read, browsing skips it and logs a warning.

Code

• MemoryInterface.get_seed_examples_async and get_seed_example_async select the example IDs for one page in SQL, then get the members of only those examples. SQLite and Azure SQL have their own harm-category predicates.
• DatasetService makes the summaries and previews. The REST models are in pyrit/backend/models/datasets.py.
• get_seeds() behavior does not change. This PR does not change the schema and adds no migrations.

Tests and Documentation

Documentation: There is a new "Browse stored seeds" section in doc/code/datasets/0_dataset.md.

Tests:

• tests/unit/memory/memory_interface/test_interface_seed_examples.py: order, tied timestamps, order when a filter matches only some members, complete groups, filter matches across members, named and unnamed datasets, literal search characters, search that ignores media and simulated-conversation JSON, and skipped unreadable seeds.
• tests/unit/backend/test_seed_example_routes.py: pages and cursors, bad requests, safe previews, full text in the detail when the preview is truncated, objective conditions, domain seed members, and empty pages.
• tests/integration/memory/test_seed_examples_azure_sql_integration.py: filters, literal search, and order on Azure SQL.

Results:

• uv run --no-sync pytest tests/unit/memory/memory_interface/test_interface_seed_examples.py tests/unit/backend/test_seed_example_routes.py tests/unit/memory/test_memory_models.py tests/unit/memory/test_migration.py -q: 180 passed
• uv run --no-sync pytest tests/unit/backend -q: 1995 passed, 5 skipped, 1 failed. The failed test, test_scenario_run_routes.py::TestResumeScenarioRunRoute::test_resume_has_no_get_preflight, passes when it runs alone. It does not use the code in this PR.
• pre-commit run --all-files: passed, except enforce_alembic_revision_immutability. Locally, that hook flags the deletion of this PR's own earlier migration, which is not on main. The CI check compares against origin/main, so it does not see that file.
• The Azure SQL integration test was not run after the latest changes.
• JupyText was not run. This PR does not change notebooks.

[Jackson Severino da Rocha (jacksonsr451)]

@microsoft-github-policy-service agree

[Richard Lundeen (richlundeen)]

Jackson Severino da Rocha (@jacksonsr451) Roman Lutz (@romanlutz) I pushed a commit (343b7a7) that makes the seed browsing API smaller. Most of the PR follows Roman's contract in #2748. These are the places where it is different, and why:

1. Text search does not include simulated-conversation configurations.
The contract says that search includes stored simulated-conversation configurations. The stored value of a configuration is JSON. Its keys include adversarial_chat_system_prompt, data_type, value, num_turns, and simulated_target_system_prompt. Thus, common words such as "prompt", "system", "value", "text", or "chat" matched every configuration. Also, json.dumps escapes non-ASCII characters, quotes, and newlines, so a search for that text did not find a match.
Now search looks only in text prompts and text objectives. To find configurations, use seed_type=simulated_conversation. Search is done for each example, so a configuration in the same group as an objective is still found by the objective text. A correct search of configuration text needs JSON functions that are not portable, or a new column. We can add that later if we need it.

2. There are no is_template or is_configuration fields on the summary.
The contract says to label configuration and template results clearly. The summary already has seed_types, which shows simulated_conversation, and the preview text is [Simulated conversation configuration]. Thus, is_configuration gave the same data again. is_template was calculated from parameters, and the detail already returns parameters. Also, the YAML loader sets is_jinja_template=True for all YAML seeds, so a flag for "template" does not show reliably whether a seed is a template. This is a new API. We can add a field later without a breaking change, but if we remove a field, clients break.

3. The detail returns domain seed objects.
The detail members field is list[SeedUnion] (SeedPrompt, SeedObjective, or SeedSimulatedConversation). It is not a separate DTO. Memory returns the same objects as get_seeds(), so the API and the Python library show the same data, with all typed fields (parameters, conditions, provenance, and hashes). The downside: a change to Seed also changes this REST schema.

4. We do not have special code for legacy simulated-conversation rows.
The contract notes that the reconstruction of a legacy simulated-conversation seed can load template files. Browsing uses the usual seed reconstruction. Legacy *_path inputs are deprecated and will be removed in 1.4.0, and no current code writes them. Thus, we did not add code only for legacy rows. If a stored seed cannot be read, browsing skips it and logs a warning. An example with no readable seeds is not shown, but total counts it.

Other changes in this push (not from the contract):

• I removed the is_jinja_template column and its migration. That flag is a trust flag for rendering. Browsing does not render, so it does not need the flag.
• I removed the old routes and replaced the tests with smaller memory, route, and integration tests. The tests cover the cases that Roman listed: tied timestamps, complete groups, filter matches across members, unnamed datasets, literal search characters, and bad cursors. They also cover sort order under a filter, full detail text when the preview is truncated, objective conditions, empty pages, and the search exclusion above.

This comment was drafted with GitHub Copilot and reviewed by me.