[jasper-yl]

Which issue or RFC does this PR close?

Implements RFC #2617 (Source plugins: collection runners and targets). Builds on #2638 (RFC #2579), which builds on #2616 (RFC #2549): this branch contains their commits, so review the commits after e1107039 and merge after #2638.

Rationale for this change

The previous Source plugin framework kept collection targets on each device behind a management service with a reverse channel. RFC #2617 keeps targets on the central Server, lets runners pull them, and makes Source plugins ordinary directory plugins next to Artifact plugins.

What changes are included in this PR?

• Declaration. SourcePlugin(version, min_compatible_version, definitions, collect, target, scopes, setup, api, pages, upgrade) loads from POWERCONTEXT_SERVER_PLUGINS_DIR like Artifact plugins, including reloading and upgrade. A plugin with only definitions extends the Source write API; target declares a parameter model and identity parameters with fixed normalizers. Artifact and Source plugin declarations share PluginSpec.
• Persistence. Revision v1_5_0_006 (SQLite and MySQL) adds pc_source_runners and pc_source_collect_targets and drops pc_source_write_plugins; its domain step refuses the upgrade while the old registry still lists plugins. Target checkpoints reuse pc_connector_checkpoints. Portable bundles add source_runner (name and bound subject) and source_collect_target records; runner reports and target status restart empty.
• API and MCP. GET /v1/source-runners (pc_source_runners), POST /v1/source-runners/{runner}/sync, DELETE /v1/source-runners/{runner}, and /v1/scopes/{scope_id}/source-targets (pc_source_add, pc_source_status, pc_source_target_set_enabled, pc_source_target_delete). Targets of the server runner need server.admin; a runner name is bound to the subject of its first sync; identity parameters cannot change. POST /v1/source-observations and POST /v1/connector-checkpoints/commit accept the runner's plugin name and version (and the commit a target status); the Server checks the version against its plugin's compatible range (plugin_version_incompatible), and its plugin resolves and projects the observation. Device calls to plugin routes carry X-PowerContext-Plugin-Version.
• Server runner. Each collecting Source plugin gets a Supervisor binding; a Worker collects one Scope's enabled server targets and commits the Sources, checkpoints, and target status in one transaction. CollectResult.fail fails one target while the others commit. New targets are collected right away.
• Device runner. powercontext source-agent --server ... --plugins ... syncs every 30 seconds, collects on each plugin's schedule and new targets right away, submits Sources and then each checkpoint with its status, and retries on the next sync after plugin_version_incompatible. ctx.center.call_plugin reaches the plugin's own routes on the Server.
• Removed. /v1/source-management/*, powercontext source load, the SOURCE_PLUGINS_DIR and SOURCE_MANAGEMENT_* settings, and powercontext.source_management. The tickets, ticket-chat, and images examples are rewritten as Source plugins; integration tool lists and the DSH secret scan follow the new MCP tools.
• Docs (en and zh): the Source plugin guide is rewritten for runners and targets, with a migration section; the plugin writing guide and UDS deployment notes point to it.

Are there any user-facing changes?

Yes, breaking for the previous Source plugin framework: its API, CLI command, settings, and MCP tools (pc_source_capabilities, pc_source_instances, pc_source_instance_set_enabled) are removed, and the upgrade is refused while write plugins are still loaded. New API and MCP tools as listed above; the Desktop contract is regenerated and qualified. release/upgrade-support.json records the schema review.

How was this change tested?

• Full suite on a Linux VM (Ubuntu 24.04, 4 cores, Node installed) at c7fb4b8c: uv run pytest -n 4 — 10378 passed, 984 skipped, 10 failed. All 10 are fixed in d057191d and pass on the VM (44 passed with the plugin e2e tests): the CLI imported server dependencies through source-agent, the OceanBase restore layer docs lacked the new tables, and two upgrade-history tests listed the domain steps without source_collection.
• make docs-test: verified 1106 public pages and their internal links. make js-test, make opencode-test, and make cindy-test pass, with committed bundles current.
• Hooks (uv run prek run) and uv run ty check pass locally; tests/test_upgrade_release_gate.py passes with the recorded schema review.
• Desktop qualification sqlite-source-plugins-a80648a8-v1: anonymous loopback, static Bearer, HTTPS with CA/base path, and enforced Provider modes passed against the wheel built from a80648a8; bundled_product_qualification_covers_every_desktop_operation passed.
• tests/e2e/test_source_plugins.py runs a real Server with an identity map and a real powercontext source-agent subprocess: only server.admin adds server targets; duplicate and conflicting targets; invalid parameters; collection on add; a failing target keeps its checkpoint while another target in the call commits; identity parameters are immutable; the device runner submits Sources and checkpoints and resubmits nothing next time; another credential cannot sync as the runner; a newer device plugin is rejected with plugin_version_incompatible; disabled targets are not synced; deleting keeps collected Sources.
• Migration: v1_5_0_006 is refused while pc_source_write_plugins holds a row and otherwise adds the new tables; registered SQLite fingerprints of earlier revisions are unchanged; the MySQL layout resources pass the shared layout validation in the suite but were not run against a native OceanBase or seekDB.
• tests/builtin/portability/test_bundle.py round-trips targets and their runner without status.

AI usage statement

Implemented with Claude Code (Claude Opus 5.5) under the author's direction; the author made the design decisions in the RFC review. Claude Code ran the tests and checks listed above.

🤖 Generated with Claude Code

[hidb4ai]

Reviewed the Source runner/target changes after e110703 at d057191. The runner-ownership bypass, an accepted target configuration that prevents Scope export, and the split checkpoint/status transaction need to be fixed before merging. Details and reproductions are inline.

[hidb4ai]

[P1] Enforce target ownership independently of optional plugin metadata

Omitting both plugin and status skips _runner_target and sends the same target binding through the generic commit path. Against a real Server in enforced access mode, a different Scope contributor gets 403 with plugin present but 200 without it; the owning runner then syncs the injected checkpoint. This also lets that contributor change a reserved server target's checkpoint, although PATCHing that target correctly returns 403. A caller can therefore skip collection or poison its binding without runner/admin authority. Resolve collection-target bindings server-side and enforce their ownership/version checks regardless of which optional fields the caller supplies.

[hidb4ai]

[P2] Keep accepted unsynced-runner targets exportable

create_source_target accepts a device runner that has never synced (the new e2e test does this too), so a persisted target can legitimately have no pc_source_runners row. This dependency check then rejects the entire Scope export. Reproduced by creating a ticket-chat target for never-synced through HTTP (201), then calling PortableBundleService.export: it raises BundleFormatError: collection target references a missing runner. A newly configured/offline device thus prevents backing up unrelated data in the same Scope. Make target admission and bundle dependencies agree so every accepted configuration can be exported and restored.

[hidb4ai]

[P2] Commit the checkpoint and target status in one transaction

application.ingestion.commit durably commits the checkpoint before this separate status transaction starts, violating RFC #2617's atomic checkpoint/status contract. With a real SQLite Server and a trigger that rejects only the status UPDATE, the request returns 500 but the next sync already contains the new checkpoint ({"cursor":42}), while all target status fields remain untouched. Removing the trigger and retrying with the original expected checkpoint returns 409, so the failed request cannot recover its status write. Keep the checkpoint CAS and target status update in the same transaction so they commit or roll back together.