Skip to content

Heal Prompt Drift (code + tests) #727

@vishalramvelu

Description

@vishalramvelu

Summary
Enhance the existing pdd update repo-wide mode to be a reliable "heal all drift" command for teams using Claude Code alongside PDD.

Current State: Most of This Already Exists
pdd update with no arguments already runs in repo-wide mode (update_main.py lines 796-1009):

Scans the repo for all prompt-code pairs (find_and_resolve_all_pairs())
Detects changed code files via git diff against the base branch
Runs update_file_pair() on each changed pair — updating prompts from modified code
Saves fingerprints after success (save_fingerprint(operation="update"))
Syncs updated prompts to architecture.json (update_architecture_from_prompt())
Even does PRD sync — if architecture changed, calls an LLM to check if the PRD needs updating
Shows a progress bar and summary table with costs
This is already a working batch upward sync. The issue is about the gaps that remain.

What's Missing

  1. Update -ed docs (Critical — depends on feat: add PDD_SKIP_DUPLICATE_CHECK to skip Step 1 duplicate hard stop #732)
    The existing repo-wide mode updates prompt files but NOT the docs they reference via (README.md, API docs, schema docs). After pdd update, prompts describe the current code but their context is stale. The LLM still gets wrong information during the next pdd generate or pdd change.

This is the single most important gap. It's tracked as gltanaka/pdd#732, which was closed but may not be fully implemented. Verify and reopen if needed.

  1. Regenerate examples and upload dev unit to PDD Cloud grounding
    After pdd update heals prompts, the grounding database still has the old (prompt, code) pair. The next pdd generate on a related module pulls stale few-shot examples.

Currently, _auto_submit_example() (in sync_main.py lines 426-502) only runs after a successful full sync pipeline. It is NOT called from update_main.py. So after pdd update --all:

Prompts are current ✓
Fingerprints are saved ✓
Architecture.json is synced ✓
Example is NOT regenerated ✗
Dev unit is NOT uploaded to PDD Cloud for grounding ✗
What's needed: After each successful prompt update, the system should:

Regenerate the example file — lightweight step that shows how to use the module with its current interface. The example should reflect the current code, not the old version.
Upload the dev unit (prompt + code + example + test) to PDD Cloud — so grounding has the current (prompt, code) pair for few-shot retrieval.
This must NOT regenerate code or tests. The code is the human-edited version that works. The tests (if they exist) are valid. Only the example and the grounding entry need refreshing.

Implementation options:

Option A: Add a post-update step in update_main.py that calls _auto_submit_example() (or a variant) after each successful update_file_pair(). Requires example regeneration first.
Option B: Add a standalone pdd submit command that uploads the current dev unit without running any pipeline. Users run it after pdd update.
Option C: Make pdd sync smart enough to detect command='update' in the fingerprint and skip code/test regeneration — only regenerate the example and upload. This would make pdd update --all → pdd sync the complete workflow.
Option A is simplest. Option C is most robust but requires changes to the sync state machine.

  1. --dry-run mode
    Show what would be updated without running LLM calls:

pdd update --dry-run

Repository drift report:
Changed files: 12 of 47 pairs
Estimated cost: $6-12

Drifted modules:
1. user_service.py → user_service_python.prompt
2. api_router.py → api_router_python.prompt
3. auth_middleware.py → auth_middleware_python.prompt
...

Included docs that may need updating:
- README.md (included by 8 prompts)
- docs/api_reference.md (included by 3 prompts)
4. --budget cap
Each update_file_pair() costs ~$0.50-1.00. For large projects with many drifted files, total cost can be significant. Add a --budget flag that stops processing when the cap is reached.

  1. Dependency ordering
    Currently processes files in filesystem order. Should process in dependency order (modules with more downstream dependents first) so that if budget runs out, the most impactful modules were updated first.

  2. --all flag (UX improvement)
    The current CLI uses no-args to trigger repo-wide mode:

pdd update # repo-wide mode (existing)
pdd update mymodule.py # single-file mode (existing)
Consider adding an explicit --all flag for clarity:

pdd update --all # explicit repo-wide mode
pdd update --all --dry-run
This is a UX improvement, not a blocker. The no-args mode already works.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    In progress

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions