Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
8439875
feat(memory): persist versioned source evidence
MaoMengww Oct 14, 2026
5dfa2ad
feat(memory): verify historical evidence during projection rebuild
MaoMengww Oct 14, 2026
034e0cc
fix(memory): enforce evidence schema safety and backend conformance
MaoMengww Oct 14, 2026
4d1bf17
fix(memory): preserve unrelated database inspection
MaoMengww Oct 14, 2026
cf8068a
test(memory): consolidate source evidence E2E acceptance
MaoMengww Oct 14, 2026
dd1d128
merge: integrate upstream artifact handles with evidence migration
MaoMengww Oct 14, 2026
0d70cba
fix(memory): verify evidence against retained source material
MaoMengww Oct 15, 2026
3305b6b
fix(sources): align remote payload aliases with manifests
MaoMengww Oct 15, 2026
c9ee82d
fix(sources): preserve envelope keys with alias generators
MaoMengww Oct 16, 2026
dc9b6a1
fix(sources): align envelope validation with serialization
MaoMengww Oct 17, 2026
8ee3aba
fix(memory): preserve source identity and shared evidence
MaoMengww Oct 18, 2026
6a836b1
fix(sqlite): preserve startup catalog compatibility
MaoMengww Oct 19, 2026
b5a4c1e
fix(sources): keep unverifiable evidence neutral
MaoMengww Oct 19, 2026
40d6c80
Merge upstream master into source evidence declarations
MaoMengww Oct 19, 2026
2bce54e
fix(sources): reject omitted identity before capture
MaoMengww Oct 19, 2026
bb36c93
fix(sources): reject transformed identity before capture
MaoMengww Oct 19, 2026
88cd8f2
Merge upstream master into source evidence declarations
MaoMengww Oct 19, 2026
73ec5aa
fix(sources): separate worker and historical identity decoding
MaoMengww Oct 20, 2026
25d9e7a
fix(sources): disambiguate canonical history from business aliases
MaoMengww Oct 20, 2026
5ceed57
fix(sources): normalize recursive transport root schemas
MaoMengww Oct 20, 2026
f4ca7f8
merge: integrate upstream retrieval lifecycle validation
MaoMengww Oct 20, 2026
2b95df6
fix(sources): preserve versioned alias compatibility
MaoMengww Oct 21, 2026
13517fd
fix(sources): keep upgraded alias definitions compatible
MaoMengww Oct 21, 2026
3c5c0ec
merge: sync upstream master and refresh the atomic maintenance baseline
MaoMengww Oct 21, 2026
3a8b38b
fix(sources): preserve stable remote manifest fingerprints
MaoMengww Oct 22, 2026
4a7f41f
merge: sync upstream master and preserve nullable nested manifests
MaoMengww Oct 23, 2026
457e3f5
Merge remote-tracking branch 'upstream/master' into feature/1975-sour…
MaoMengww Oct 25, 2026
95c1536
fix(portability): preserve accepted evidence in portable archives
MaoMengww Oct 26, 2026
fa71650
merge: sync upstream master and register Source evidence tables
MaoMengww Oct 27, 2026
6b20c38
merge: sync upstream and preserve migration revision checksums
MaoMengww Oct 28, 2026
d215c98
feat(memory): add rebuildable source evidence projection
MaoMengww Oct 28, 2026
34c5276
fix(memory): persist versioned Source evidence and close review findings
MaoMengww Oct 28, 2026
93ca5df
merge: sync upstream master and preserve source evidence upgrade decl…
MaoMengww Oct 29, 2026
08a4386
ci(e2e): budget the six Server recovery scenarios
MaoMengww Oct 30, 2026
5da4f75
merge: sync upstream master and preserve evidence migrations
MaoMengww Oct 30, 2026
8444f08
fix(persistence,sources): keep accepted manifests and MySQL-family ev…
MaoMengww Oct 30, 2026
6d5ca48
merge: sync upstream master and chain evidence after source write plu…
MaoMengww Oct 31, 2026
9e1a47b
merge: sync upstream master and chain evidence after artifact storage
MaoMengww Oct 31, 2026
4e9dd0f
merge: sync upstream validation consolidation and keep evidence review
MaoMengww Oct 31, 2026
ff88ee0
merge: sync upstream master and keep evidence-aware migration gates
MaoMengww Nov 3, 2026
c9cede7
fix(migrations): align evidence gates with upstream CI
MaoMengww Nov 4, 2026
ed62759
merge: sync upstream ownership migration and preserve evidence review
MaoMengww Nov 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .github/workflows/e2e-harness.yml
Original file line number Diff line number Diff line change
Expand Up @@ -255,12 +255,14 @@ jobs:

# Each group keeps a hard deadline. The six independent recovery scenarios
# must not consume a single shared deadline before client-exit recovery runs.
# One scenario reloads the baseline, upgrades the full Server chain, recovers
# it, and verifies history, so the group budget covers 6 x ~110s plus collection.
- name: Recover committed Server DDL and domain checkpoints on real OceanBase without skips
env:
POWERCONTEXT_TEST_MIGRATION_OCEANBASE_URL: >-
mysql+aoceanbase://root%40test:powercontext-e2e@127.0.0.1:2881/pc_migration_probe_server_recovery?charset=utf8mb4
run: |
timeout --signal=TERM --kill-after=10s 600s uv run --locked pytest -vv -s -rs \
timeout --signal=TERM --kill-after=10s 900s uv run --locked pytest -vv -s -rs \
tests/builtin/persistence/test_mysql_migration_runner.py::test_mysql_full_server_interruption_preserves_history_and_original_maintenance \
-o faulthandler_timeout=180 --durations=0 \
-k oceanbase --junitxml=.powercontext/recall-effort-ci/full-server-checkpoints.xml
Expand Down Expand Up @@ -299,8 +301,8 @@ jobs:
Path(".powercontext/recall-effort-ci/full-server-maintenance.xml"),
)
cases = [case for report in reports for case in ElementTree.parse(report).iter("testcase")]
if len(cases) != 11 or any(case.find(tag) is not None for case in cases for tag in ("skipped", "failure", "error")):
raise SystemExit("All eleven native full Server recovery and maintenance scenarios must pass without skips.")
if len(cases) != 13 or any(case.find(tag) is not None for case in cases for tag in ("skipped", "failure", "error")):
raise SystemExit("All thirteen native full Server recovery and maintenance scenarios must pass without skips.")
PY

- name: Collect the database log
Expand Down
34 changes: 27 additions & 7 deletions docs/en/docs/operate/database-migrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ The design follows [RFC #1771](https://github.com/oceanbase/powercontext/pull/17
The full Server framework uses two normalization checkpoints before its shared migration chain for the PowerContext
1.5.0 target (stored compatibility IDs shown below):

- **1.3.0:** `s1300 → s1400 → s1401 → s1402 → v1_4_1_004 → v1_4_1_005 → v1_4_1_006 → v1_5_0_001`.
- **1.4.0:** `r1400 → s1400 → s1401 → s1402 → v1_4_1_004 → v1_4_1_005 → v1_4_1_006 → v1_5_0_001` through a separate frozen source bridge.
- **1.3.0:** `s1300 → s1400 → s1401 → s1402 → v1_4_1_004 → v1_4_1_005 → v1_4_1_006 → v1_5_0_001 → v1_5_0_002 → v1_5_0_003`.
- **1.4.0:** `r1400 → s1400 → s1401 → s1402 → v1_4_1_004 → v1_4_1_005 → v1_4_1_006 → v1_5_0_001 → v1_5_0_002 → v1_5_0_003` through a separate frozen source bridge.

Application release numbers and schema revisions are distinct. The 1.4.0 release is not the `s1400` checkpoint:
its existing Candidate schema is current, but its Topic attempt storage still needs the registered bridge.
Expand Down Expand Up @@ -98,6 +98,15 @@ earlier revision resources and checksums remain unchanged. Native vector tables
are admitted to later maintenance only after exact layout validation. These registrations do not expand the
accepted source versions or establish installed-release upgrade acceptance.

Revision `v1_5_0_002` adds the Source evidence bindings, exact Atomic Memory evidence snapshots, and evidence schema marker. Its own domain step validates retained Source material and historical evidence for both lexical and vector profiles. Earlier projection validators and their recorded revision checksums remain unchanged; evidence validation is required only from this revision onward. Existing Sources and Memory revisions without accepted declarations stay neutral (`untrusted` / `unknown`); upgrading never backfills trust. Current projection repair and rebuild also require evidence validation.

Revision `v1_5_0_003` registers `pc_atomic_memory_evidence`, the disposable exact-revision evidence projection. It is
derived from retained history, can be discarded and rebuilt, and never becomes the authority for a trust decision.

The runtime bundle stamps `pc_atomic_memory_evidence_schema.schema_version` with `p0005`, its own revision for the same
tables; the Server chain registers that shape as `v1_5_0_002`. Operator messages name both so a runtime deployment and a
managed Server deployment can each apply the revision they own.

An interrupted [portable restore](portable-archive.md) keeps normal startup and workers blocked until its projections
are complete. Retry `powercontext archive restore <bundle> --yes` with the same validated bundle. Only this explicit
recovery path can open an incomplete projection set, after validating the registered layout and the configured vector
Expand Down Expand Up @@ -144,16 +153,18 @@ Sequence `000` identifies a registered historical baseline; upgrade steps start
| `v1_4_1_001` | 1.4.1 | 1 | `s1400` | Complete Server storage transition |
| `v1_4_1_002` | 1.4.1 | 2 | `s1401` | Offline Memory history archive |
| `v1_4_1_003` | 1.4.1 | 3 | `s1402` | Portable restore receipts and timestamp adaptation |
| `v1_4_1_004` | 1.4.1 | 4 | `v1_4_1_004` | Durable owner recovery intents |
| `v1_4_1_004` | 1.4.1 | 4 | `v1_4_1_004` | Durable external-owner recovery intents |
| `v1_4_1_005` | 1.4.1 | 5 | `v1_4_1_005` | Nullable Atomic Memory description projection |
| `v1_4_1_006` | 1.4.1 | 6 | `v1_4_1_006` | Source write plugin registrations |
| `v1_5_0_001` | 1.5.0 | 1 | `v1_5_0_001` | Artifact plugin storage and search registrations |
| `v1_5_0_002` | 1.5.0 | 2 | `v1_5_0_002` | Retained Source evidence and historical validation |
| `v1_5_0_003` | 1.5.0 | 3 | `v1_5_0_003` | Disposable exact-revision evidence projection |

Both registered source paths reach `v1_5_0_001`; only 1.3.0 is selected for CI upgrade acceptance:
Both registered source paths reach `v1_5_0_003`; only 1.3.0 is selected for CI upgrade acceptance:

```text
1.3.0: v1_3_0_000 -> v1_4_1_001 -> v1_4_1_002 -> v1_4_1_003 -> v1_4_1_004 -> v1_4_1_005 -> v1_4_1_006 -> v1_5_0_001
1.4.0: v1_4_0_000 -> v1_4_1_001 -> v1_4_1_002 -> v1_4_1_003 -> v1_4_1_004 -> v1_4_1_005 -> v1_4_1_006 -> v1_5_0_001
1.3.0: v1_3_0_000 -> v1_4_1_001 -> v1_4_1_002 -> v1_4_1_003 -> v1_4_1_004 -> v1_4_1_005 -> v1_4_1_006 -> v1_5_0_001 -> v1_5_0_002 -> v1_5_0_003
1.4.0: v1_4_0_000 -> v1_4_1_001 -> v1_4_1_002 -> v1_4_1_003 -> v1_4_1_004 -> v1_4_1_005 -> v1_4_1_006 -> v1_5_0_001 -> v1_5_0_002 -> v1_5_0_003
```

`status` and `plan` show `source_migration`, `target_migration` and `pending_migrations`
Expand All @@ -175,7 +186,7 @@ existing plans and original backups usable. Partial prototype bundles keep their

For a new Server migration, use the release-based name directly as its Alembic `revision` and
register its schema, domain steps and recovery checkpoints as usual. `down_revision` must reference
the actual preceding ID (`v1_5_0_001` when appending directly to the current head), even if that ID has
the actual preceding ID (`v1_5_0_003` when appending directly to the current head), even if that ID has
a display alias. The framework follows this registered chain, not lexical name ordering. Already
registered names and release assignments stay immutable; a version bump must not rename history.

Expand Down Expand Up @@ -270,6 +281,11 @@ bound row buffering; this is an offline whole-projection operation. It does not
embedded seekDB Code configurations. The compatibility command `server atomic-memory-rebuild-projection` delegates managed
databases to this bundle and supports `--action plan/apply/verify` with the same confirmation and backup options.

Both maintenance entry points run from the migration bundle and therefore admit evidence against the builtin Source
registry; they do not receive a host's code-supplied `SourceDefinitionRegistry`. The runtime evidence read path
(`AtomicMemory.evidence`) uses the configured registry, so a deployment with custom native Definitions should rebuild
through the Python API with `sources=<configured repository>` when it needs an identical verdict.

## Topic Memory recovery bundle

Select `--bundle topic-memory-recovery` for the versioned `powercontext.topic-memory-recovery.v1` conversion. It recognizes
Expand Down Expand Up @@ -723,6 +739,10 @@ SQL literals reach the backend unchanged.

Repositories run after schema checks. Both full Server source chains register exact per-statement schema transitions and recovery prefixes through `mysql_state_layouts`. Domain steps attach to the shared revision and use its connection, lock, and recovery point; verification precedes version advancement. Read-only live checks are separate from migration-copy validation. Vector configurations still require separate registration.

During `v1_5_0_002`, an interrupted MySQL-compatible upgrade can resume a registered CREATE-table prefix before the
final `p0005` evidence marker is inserted. Full evidence validation starts once that marker exists; a completed
database with missing or incompatible evidence storage still fails admission.

Online readiness reads Atomic Memory authority and projections from the same snapshot without locking business head rows. Offline projection repair retains its locked state comparison and write-side concurrency protection.

The 1.4.0 bridge is assembled from immutable shared resources and its `server_sources/1.4.0` overlay. Source
Expand Down
8 changes: 8 additions & 0 deletions docs/en/docs/operate/observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,14 @@ including code stages. Code-specific diagnostic fields are included only for `co
| `duration_ms`, `error_code`, `error_category` | Duration in milliseconds, diagnostic code, and bounded operational error classification |
| `stage`, `processing_stage`, `failure_reason`, `input_item_count`, `estimated_input_tokens`, `token_estimator_id`, `effective_budget`, `provider_request_id`, `finish_reason`, `failure_location`, `cause_exception_type` | Processing failure stage/reason and safe accounting fields; `stage` names the emitting boundary or domain location (Worker/supervisor stage, domain block name, or `worker` fallback), `processing_stage` the bounded domain stage. See [Processing failure diagnostics](../../development/processing-failures.md) |

`reason` is a bounded cause for a retained-evidence downgrade. The `atomic_memory.evidence.neutral` event reports that
accepted evidence could not be attested and therefore resolved to `untrusted` / `unknown`; it never carries material,
declaration bodies, or requested content. Its `unit` is the retained fact (`source`, `declaration`, `manifest`, or
`revision`), `outcome` is `neutral`, and `reason` is one of `schema_mismatch`, `digest_mismatch`, `no_manifest`,
`decode_failed`, `legacy_unverifiable`, `invalid_declaration`, `manifest_conflict`, `definition_missing`, or
`evidence_mismatch`. Integrity causes are emitted at `WARNING`; expected historical gaps stay at `INFO` or `DEBUG`.
Integrity causes such as `manifest_conflict` are alertable and distinct from an absent declaration.

Server roles are `all` (API and supervisor), `api`, and `background`; child processes use `worker`. Explicit logging
configuration without a deployment role uses `server`. JSON identity fields correspond to trace resource attributes
`service.name`, `service.version`, `service.instance.id`, and `powercontext.service.role`. `service_version` is absent
Expand Down
1 change: 1 addition & 0 deletions docs/en/docs/operate/portable-archive.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ Format version 1 carries the portable relational representation of these support
| --- | --- |
| Scope identity, hierarchy, context/external references, and creation identity | Scope access bindings and host-local default selection |
| Source journal heads, Source records, and referenced remote Source Definition manifests | Search projections and indexes |
| Accepted Memory evidence: per-Source declaration bindings, the Definition manifests they reference, and exact-revision evidence snapshots | Today's registry declarations, which never replace a saved snapshot |
| Artifact Revisions, their timestamps/actors/rollback metadata, stable handles, lineage, cross-Scope publication provenance, and heads | Source cursors and scheduler state |
| Atomic Memory merge relationships and marked merge-input lineage | External Skill registrations and host-local installation state |
| Candidate versions, decision heads, and their evidence references | Audit events, usage facts, evaluation receipts, and restore receipts |
Expand Down
50 changes: 50 additions & 0 deletions docs/en/docs/workflows/sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,54 @@ content under the same identity as if it were a new observation.
- Subject evidence for a Scope profile: follow [Scope profiles](use-profiles.md).
- Application or evaluation events: use the application's adapter; [Bub capture](../integrations/evaluation.md) is opt-in.

## Declare source authority

A trusted adapter can declare how its material should be trusted. The optional `memory_evidence` field on a Source
Definition carries a versioned `MemoryEvidenceDeclaration` with `authority` (`untrusted`, `user_asserted`,
`repository_attested`, or `system_attested`) and `verification` (`unknown`, `verified`, or `not_verified`). Only
adapter or deployment code supplies it; Source text, references, model output, and retrieval rank cannot.

```python
from powercontext.sources import AdapterSourceDefinition, MemoryEvidenceDeclaration

definition = AdapterSourceDefinition(
existing_adapter,
memory_evidence=MemoryEvidenceDeclaration(
authority="repository_attested",
verification="not_verified",
),
)
```

Remote Source observations configure the same declaration in trusted deployment code, keyed by the exact Definition
name, version, and manifest fingerprint. A missing, invalid, or unavailable declaration resolves to
`untrusted`/`unknown`; it does not block capture, processing, or existing retrieval.

Observation manifests describe the serialized payload for every field the payload must carry. Validation aliases do
not require workers to rename serialized keys, including when a model forbids extra fields.
For unions, an alias-renamed branch may retain the validation manifest when another branch has an equivalent flat
primitive-property schema covering its serialized output. Unproven branch coverage uses the serialization schema, except
for a field whose default is `null`: it can reach the payload as `null`, so a renamed shape below it never replaces an
accepted manifest. Populating such a field with a renamed key requires the nested model to serialize by alias, which
keeps the validation keys authoritative. Changing what a registered Definition serializes requires a new Definition
version.

Each Memory revision records the declaration and exact Source materialization accepted with it. That retained evidence
is authoritative and versioned. The index projection beside it is derived: maintenance can discard and rebuild it
without consulting today's registry. Declarations add no ranking weight, score, or access right.

Read the retained facts back for one exact revision:

```python
from powercontext.builtin.runtime import AtomicMemoryEvidenceAnnotation

memory = runtime.atomic_memory.for_scope(scope_id)
annotation: AtomicMemoryEvidenceAnnotation = await memory.evidence(artifact_id, revision=3)
for source in annotation.sources:
print(source.source_ref.source_type, source.declaration.authority, source.declaration.verification)
```

A declaration that can no longer be attested resolves to `untrusted`/`unknown`. A record that no longer matches its
exact Source material raises `AtomicMemoryEvidenceError`, which callers catch through the `PowerContextError` family.

Source writes and reads are subject to the selected [Scope and access policy](scopes-and-access.md).
18 changes: 18 additions & 0 deletions docs/en/rfcs/1652_memory_quality_and_lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,24 @@ candidate pool or delivered Context Pack occupied by a lower-authority class, bu
legitimate answer-bearing evidence. This is an opt-in experiment, not a stage-1 default or a substitute for the Source
declaration contract.

## Current implementation amendment: source evidence declaration

The Source evidence prerequisite is implemented as specified above. This section supersedes the "provisionally named"
wording and the statement that `SourceDefinition` carries no authority or verification declaration.

- `SourceDefinition` and `AdapterSourceDefinition` expose an optional `memory_evidence` declaration with `authority`,
`verification`, and `declaration_version="1"`.
- Trusted adapter code, or deployment configuration keyed by an exact `(name, version, fingerprint)` Definition
manifest, supplies the declaration. Source text, `SourceRef`, model output, and retrieval rank cannot.
- Each Atomic Memory revision retains the accepted declaration with its exact Source materialization and manifest
fingerprint. A separate index-owned projection holds the same facts for that revision and can be discarded and
rebuilt from retained evidence without consulting the current registry.
- Missing, invalid, or unverifiable declarations remain `untrusted`/`unknown` for reads and rebuilds. There is no
trust backfill, and existing write authority, generation eligibility, body text, content hashes, citations, and
search contracts are unchanged.
- Stage-1 quality ranking, lifecycle actions, and receipt-linked usage feedback remain unimplemented. This slice is
their prerequisite, not a substitute for them.

## Independent quality dimensions

The policy keeps these dimensions separate:
Expand Down
Loading
Loading