[BREAKING] [FIX]: Evaluate attack verdicts over final traces

[Spencer Schoenberg (spencrr)]

Description

XPIA now evaluates the attack objective once over the final trace, separately from online stopping. stop_when accepts an evaluator, None, or the default StopWhen.AUTO. StopWhen.AUTO reuses the verdict evaluator as the stop condition only when detection stays true as turns are appended: ToolCalled, SideEffectOccurred, and ResponseContains with ResponseScope.ANY_TURN. StopWhen is exported from rampart.attacks and rampart.

When the stop and verdict evaluators are the same object, the latest online evaluation is reused if it already covers the final trace, so no duplicate judge call is made. Final verdict evidence is stored on Result.final_trace_evaluation, trace completion uses TraceEndReason, and online stop evidence uses EvaluationPurpose.STOP_CHECK. Final evaluation runs inside the active session and injection stack. Observability downgrades are recorded without mutating response metadata, and cleanup failures discard otherwise successful verdict evidence and return ERROR.

The list-based resolve_as_attack and per-turn evaluate_turn_async helpers are removed because no built-in strategy uses them. The trace-contract declaration records a compatible change against main.

Depends on #149.

Breaking changes

• XPIA verdicts are computed once over the final trace. Single-trigger attacks with deterministic evaluators keep their verdicts.
• With the default StopWhen.AUTO, other evaluators, including LLM judges, no longer stop early. They are called once on the final trace, and adaptive drivers can run up to max_turns. Pass the same evaluator as stop_when to restore per-turn early stopping without a duplicate final call.
• With the default, stochastic evaluators are sampled once per run instead of once per turn, so trial pass rates can shift.
• Auto-stopped attacks expose online evaluator feedback to adaptive drivers.
• Removed helpers: replace resolve_as_attack(eval_results=...) with resolve_attack_verdict(evaluation=...), and replace evaluate_turn_async with run_trace_async plus evaluate_final_trace_async.

The upgrade note in docs/attacks/xpia.md covers these changes.

Checklist

• pre-commit run --all-files passes
• Tests added or updated for changes — automatic, explicit, and disabled stopping; StopWhen values; exact evaluator call counts; stable-evaluator classification and composition; cleanup ordering; zero and max turns; observability; metadata isolation; and summaries
• Documentation updated

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.