feat: add logs-to-training pipeline prototype (PII-safe ingest, SFT/DPO export, validation, split integrity)

[theUtkarshRaj]

Summary

This PR adds an implementation-first prototype for transforming production QA + agentic logs into privacy-safe, trajectory-aware training datasets.

Included

• Canonical Langfuse/Pydantic log schema
• PII redaction with stable placeholders
• Session segmentation
• Complexity tagging
• Tool consistency validation
• LoRA-ready SFT export
• DPO candidate generation + hard negatives
• Split integrity safeguards (disjoint SFT/DPO/eval)
• Gold examples
• Validation CLI
• CI + tests
• Apache-2.0 licensing

Motivation

This aligns directly with the DMP 2026 project goal of converting production logs into safe, trajectory-aware datasets suitable for SFT (LoRA) and DPO workflows, with emphasis on data quality, privacy, and downstream training readiness.

Validation

• pytest passing
• ruff clean
• GitHub Actions CI included

Notes

This is designed as a modular prototype aligned with midpoint milestone goals. If preferred, I’m happy to split this into smaller PRs (core pipeline / validation / DPO extensions) for easier review.

[theUtkarshRaj]

Built this as a full implementation-first prototype for the complete DMP midpoint slice, but with particular emphasis beyond baseline ingest/export on downstream governance layers:

• split integrity (disjoint SFT / DPO / eval)
• validation CLI
• hard-negative DPO generation
• gold example fixtures
• CI / packaging

My intent was to complement core prototype directions with stronger training-readiness, quality safeguards, and reviewer tooling. Happy to modularize or split advanced layers if narrower review boundaries are preferred.