You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
FEAT: Explore datasets and use seeds in chat and custom scenarios #2744
Is your feature request related to a problem? Please describe.
People using CoPyRIT need a way to discover seed prompts without leaving the GUI, inspect the datasets already in memory, optionally load built-in datasets, and use a selected example in a human-led attack. Selecting several examples should lead to a configurable scenario rather than a batch-chat feature.
The backend already lists dataset names from memory and registered providers. Chat already supports media previews and copying content into its input. This series should reuse that infrastructure and remain mostly GUI work plus thin service APIs.
Describe the solution you'd like
User experience
Add a Datasets section with searchable/filterable dataset cards. Cards show meaningful counts, modalities, harm categories when present, and whether a dataset is loaded or available from a provider. Clicking a card opens a paginated prompt table with expandable content and metadata. Images render, and audio/video have playback controls.
For one selected prompt or linked multimodal example, provide:
Import to new attack.
Import to current conversation within the current attack.
Import to new conversation within the current attack.
These actions populate an editable draft. They never send to a target automatically. Add a "Need inspiration? Browse seed prompts" entry point in chat, preserving the draft and originating attack/conversation while browsing.
For multiple selected examples, provide Configure scenario. Reuse Scanner configuration, execution, progress, and history. Defaults are exactly prompt sending, no converters, and a compatible configured objective-scorer preset. Let the user configure standard attack techniques, converter pipelines, target, and scoring before confirming launch.
Agreed design boundaries
Memory is the source of browsable seeds. Provider loading is explicit and writes through existing memory APIs. Catalog browsing, filtering, and estimates must not download datasets.
Use credentials already configured on the backend. Loaded datasets remain in the deployment's shared memory. Do not introduce browser token entry, a credential store, or private per-user dataset access.
Use configured instances from ScorerRegistry, not a hardcoded list of scorer class names. Prefer the compatible instance tagged default_objective_scorer. Existing composites are valid presets; raw refusal=true must not be presented as attack success.
Preserve existing objectives. If a selected example lacks an objective required by the scenario, ask for it rather than inventing one or treating a file path as an objective.
Keep grouped text/media, roles, and ordering intact. Distinguish literal prompts, objectives, templates, and simulated-conversation configurations. Unsupported import shapes remain inspectable with an explanation; do not silently flatten or execute them.
Distinguish logical examples from seed/prompt pieces and objectives. A text/image pair is one example with two pieces. Missing harm labels mean "Not labeled", not "Safe". Unknown provider counts are not zero, and provider estimates are not exact loaded counts.
Preserve chat drafts, attachments, target/converter settings, browsing filters, and return context. Confirm before replacing a draft. Respect existing target compatibility and conversation mutation restrictions.
Use bounded database-side pagination and lazy media loading. No autoplay. Keep safe media serving and clear missing-file/unsupported-format fallbacks.
No memory schema migrations, new dataset tables, new execution engine, or persistent dataset copied for each selection. Narrow memory read-query helpers are allowed when required for counts and pagination.
Delivery sequence
The main explorer sequence is released to contributors one item at a time. Item 12 (#2755, the general scorer registry backend API) ran as an independent track and is completed in #2990. The overlapping general-API issue #2970 is also completed by that PR and remains related work, not an eighteenth child.
The remaining dataset-specific scorer requirements are explicit in #2757 (exact-selection and registry-name request contract), #2760 (compatible defaults, composites/inverters, and objective-result semantics), and #2759 (configured-instance picker, tags/default/compatibility metadata, and setup guidance). The full identifier returned by #2990 does not by itself implement these behaviors.
Item 13 (#2757) is released only after both item 11 (#2756) and item 12 (#2755) are complete, followed by maintainer promotion. #2756 is still open, so completing the scorer API does not release #2757 early. Each issue includes its own focused tests and failure states; the final item does not defer that work.
Order
Work item
1
#2746 FEAT GUI: Add memory-backed dataset summary API (completed in #2762)
Other open children carry not ready yet until their prerequisites have merged and the maintainer promotes them.
On promotion, remove not ready yet and apply help wanted. Keep at most one active main-sequence item; the independent scorer API track is now complete. Do not mark the entire series help wanted at once.
Describe alternatives you've considered, if relevant
A second dataset store or copying selections into new persistent datasets would duplicate memory and is out of scope.
Automatically fetching all providers to obtain counts would make browsing expensive and can require gated access. Show available metadata without fetching.
Concatenating several unrelated prompts into one chat draft would change their meaning. Use the scenario flow for multi-select.
A new batch runner, general scorer-builder GUI, unscored scenario mode, dataset editor/deleter, or arbitrary user-supplied loader code is not part of this series. The general scorer API creation route already shipped in FEAT: expose ScorerRegistry through the backend API #2990 and should be reused, not rebuilt.
Additional context
Read doc\code\framework.md and the applicable .github\instructions files before implementing a child issue. Keep business logic in the appropriate framework component; the GUI and backend are adapters.
Useful starting points: DatasetService.list_datasets_async(), the dataset routes/models, ChatWindow and ChatInputArea, MessageList, the media route and attack media mappers, ScenarioDetail, ScenarioConfigurationResolver, ScorerRegistry, and ScorerInitializer.
Coordinate with #2735 for counts without materializing unloaded datasets and #2653 for decomposition of the Scanner configuration page. These are related work, not requests to duplicate or take over those issues.
Contributors must be able to develop and test this feature with local seed fixtures, mock providers, and media fixtures without privileged Hugging Face access or paid target calls.
Is your feature request related to a problem? Please describe.
People using CoPyRIT need a way to discover seed prompts without leaving the GUI, inspect the datasets already in memory, optionally load built-in datasets, and use a selected example in a human-led attack. Selecting several examples should lead to a configurable scenario rather than a batch-chat feature.
The backend already lists dataset names from memory and registered providers. Chat already supports media previews and copying content into its input. This series should reuse that infrastructure and remain mostly GUI work plus thin service APIs.
Describe the solution you'd like
User experience
Add a Datasets section with searchable/filterable dataset cards. Cards show meaningful counts, modalities, harm categories when present, and whether a dataset is loaded or available from a provider. Clicking a card opens a paginated prompt table with expandable content and metadata. Images render, and audio/video have playback controls.
For one selected prompt or linked multimodal example, provide:
These actions populate an editable draft. They never send to a target automatically. Add a "Need inspiration? Browse seed prompts" entry point in chat, preserving the draft and originating attack/conversation while browsing.
For multiple selected examples, provide Configure scenario. Reuse Scanner configuration, execution, progress, and history. Defaults are exactly prompt sending, no converters, and a compatible configured objective-scorer preset. Let the user configure standard attack techniques, converter pipelines, target, and scoring before confirming launch.
Agreed design boundaries
Delivery sequence
The main explorer sequence is released to contributors one item at a time. Item 12 (#2755, the general scorer registry backend API) ran as an independent track and is completed in #2990. The overlapping general-API issue #2970 is also completed by that PR and remains related work, not an eighteenth child.
The remaining dataset-specific scorer requirements are explicit in #2757 (exact-selection and registry-name request contract), #2760 (compatible defaults, composites/inverters, and objective-result semantics), and #2759 (configured-instance picker, tags/default/compatibility metadata, and setup guidance). The full identifier returned by #2990 does not by itself implement these behaviors.
Item 13 (#2757) is released only after both item 11 (#2756) and item 12 (#2755) are complete, followed by maintainer promotion. #2756 is still open, so completing the scorer API does not release #2757 early. Each issue includes its own focused tests and failure states; the final item does not defer that work.
Contributor readiness
Describe alternatives you've considered, if relevant
Additional context
Read doc\code\framework.md and the applicable .github\instructions files before implementing a child issue. Keep business logic in the appropriate framework component; the GUI and backend are adapters.
Useful starting points: DatasetService.list_datasets_async(), the dataset routes/models, ChatWindow and ChatInputArea, MessageList, the media route and attack media mappers, ScenarioDetail, ScenarioConfigurationResolver, ScorerRegistry, and ScorerInitializer.
Coordinate with #2735 for counts without materializing unloaded datasets and #2653 for decomposition of the Scanner configuration page. These are related work, not requests to duplicate or take over those issues.
Contributors must be able to develop and test this feature with local seed fixtures, mock providers, and media fixtures without privileged Hugging Face access or paid target calls.