Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
98ca20d
docs(reporting): specify official finality policies
rachitm022 Nov 18, 2026
4f551fb
docs(reporting): plan official finality support
rachitm022 Nov 18, 2026
3c330bf
feat(reporting): retain selected finality policies
rachitm022 Nov 18, 2026
e488156
feat(reporting): validate official finality admission
rachitm022 Nov 18, 2026
083044b
fix(reporting): validate dual-finality routes and isolate legacy reco…
rachitm022 Nov 18, 2026
5ced54e
fix(reporting): preserve verification key identity without finality p…
rachitm022 Nov 18, 2026
f196ceb
feat(reporting): expose retained observation history
rachitm022 Nov 18, 2026
c3ca8e6
fix(reporting): require concrete observation history for stabilization
rachitm022 Nov 18, 2026
8a53137
feat(reporting): prove official source and cutoff finality
rachitm022 Nov 18, 2026
ae1ab9c
fix(reporting): bind official proof to verified source acquisitions
rachitm022 Nov 18, 2026
1779114
fix(reporting): recover ineligible cutoff reads and bind verified pub…
rachitm022 Nov 18, 2026
d63fd84
fix(reporting): freeze source authority and serialize observation work
rachitm022 Nov 18, 2026
d569ed6
fix(reporting): bind source authority and isolate manual restatements
rachitm022 Nov 18, 2026
cefd387
fix(reporting): preserve poll cadence and verify replay evidence
rachitm022 Nov 18, 2026
4a0856f
feat(reporting): prove stabilized official finality
rachitm022 Nov 18, 2026
c62a274
fix(reporting): recover and verify stabilized snapshot evidence
rachitm022 Nov 18, 2026
b0f3379
fix(reporting): fence stabilized reservations and verify snapshot ori…
rachitm022 Nov 18, 2026
88835ed
fix(reporting): preserve stabilized recovery and terminal publication
rachitm022 Nov 18, 2026
16c4bf1
fix(reporting): retain scheduled cadence and verify snapshot reuse
rachitm022 Nov 18, 2026
5354345
fix(reporting): verify recovered schedules and tracked snapshot origins
rachitm022 Nov 18, 2026
ee49fec
fix(reporting): enforce explicit official finality on admission and r…
rachitm022 Nov 18, 2026
05763b9
fix(reporting): recover frozen close routes and verify replay origins
rachitm022 Nov 18, 2026
f2b48f3
fix(reporting): validate historical snapshots and recover frozen clos…
rachitm022 Nov 18, 2026
c6e07df
fix(reporting): verify pending and completed close routes before disp…
rachitm022 Nov 18, 2026
c83f172
fix(reporting): restore first Core snapshots and detect retained offi…
rachitm022 Nov 18, 2026
9ceabe0
fix(reporting): serialize snapshot roots and recover interrupted Core…
rachitm022 Nov 18, 2026
2371c44
fix(reporting): resume failed first Core snapshots on their frozen id…
rachitm022 Nov 18, 2026
12426ec
style(reporting): align finality examples and fixtures with formatter…
rachitm022 Nov 18, 2026
92708c3
refactor(reporting): keep verified snapshot rows bound to their revision
rachitm022 Nov 18, 2026
fe591d1
fix(reporting): preserve historical PostgreSQL checkpoint contracts
rachitm022 Nov 18, 2026
ff1c882
docs(reporting): describe stabilization checkpoint persistence
rachitm022 Nov 18, 2026
324a20d
test(reporting): exercise readiness after removing checkpoint storage
rachitm022 Nov 18, 2026
6be29d1
test(reporting): bind late-account producers to their declared finali…
rachitm022 Nov 18, 2026
88f43c4
test(reporting): isolate finality policy tests from frozen SDK controls
rachitm022 Nov 18, 2026
87a318f
fix(reporting): resolve provisional storage against the selected ledger
rachitm022 Nov 18, 2026
6b32a6c
fix(reporting): revalidate official finality against active snapshot …
rachitm022 Nov 18, 2026
9f0bcac
fix(reporting): satisfy finality code quality checks
rachitm022 Nov 18, 2026
46fe1bb
fix(test): isolate historical reporting fixtures
rachitm022 Nov 18, 2026
af67b23
fix(reporting): keep period closure source independent
rachitm022 Nov 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 68 additions & 9 deletions docs/reporting-provisional-observations.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,24 +34,83 @@ reuse, installed-package checks and independent integration review. Follow the
- Snapshot-only reporting retains one final inclusive read at the boundary,
including after downtime. Valid new source evidence may extend that boundary.
Expiry does not create an official revision.
- Existing explicitly configured official-close behavior remains in force.
`official_close_lag` still requires an explicitly declared `restatement_window`
and an official offering. The SDK fallback does not opt an adapter into
automatic official publication.
- Setting `official_close_lag` opts a snapshot offering into an explicit
snapshot-to-official route. This requires a declared `restatement_window`
and a selected definition policy. For `source_final` and
`contractual_cutoff`, the lag can delay the source attempt. For
`stabilized`, the SDK promotes the snapshot when its scheduled observations
satisfy the selected policy, independently of the lag value.

The durable acquisition freezes the source request, predecessor revision and
observation ordinal. A retry renews only `deadline_at`, the execution budget for
that attempt. It retains the execution key, run ID, cutoff, scope and remaining
request fields even after the original deadline expires. Once reserved, a
successful retry completes that observation before a later acquisition starts.

## Selecting official finality

For a new generation with `required_finality: official`, the adopter copies
one policy from the pinned report definition into
`ReportingDefinitionBinding.finality_policy`. Use the same selection when
configuring an explicit snapshot-to-official close. To migrate unfinished
official work with no policy, configure a new generation with the selected
policy. Retained revisions and exact replays keep their original evidence.

During a rolling deployment, upgrade every process that reads configuration
or obligation definitions before any writer persists a definition containing
`finality_policy`. Earlier readers reject the additional definition field.

- A `source_final` policy uses the authoritative offering. Its
`provider_declared` or `provider_job_terminal` evidence must carry the exact
selected signal and the upstream finalization time.
- A `contractual_cutoff` policy uses the snapshot offering. Its full, verified
observation must reach the declared period-end cutoff.
- A `stabilized` policy uses full-period batch, constituent and metric
watermarks from durable scheduled snapshots. A poll may arrive after its
due time within the same cadence slot. Missing the entire slot restarts the
unchanged interval. A manual publication that changes the active snapshot
also restarts the interval at the next scheduled observation. Only scheduled
observations can establish the interval.

`stable_since` is a source observation and `official_close_lag` is scheduling
metadata. Snapshot-only generations need no selection and remain snapshots
regardless of elapsed time. After an official close the producer accepts exact
replay. Later corrections use adjustments.

Custom Core ledgers that publish official revisions must implement
`ReportingSourcePublicationStore.source_publication(account_id)` with a lock
shared across all publishers for that account. Stabilization additionally
requires durable reservations, bounded scheduled history, checkpoints, exact
snapshot origins and generation schedule recovery. An ordinary snapshot route
can keep using its snapshot publisher when the definition lists an unused
finality policy. Each scheduled read checks the winner of its own atomic reservation.

When a definition carries an unused finality policy, a Core producer without
observation or checkpoint support can call `close_elapsed_periods` and then
`acquire_obligation` for the first snapshot of the period it just created.
If a manual or scheduled first snapshot encounters a retryable fetch failure,
the same producer can resume the frozen request with a fresh deadline. It
checks the recorded slice retry and any shared backoff before dispatch. An
interrupted staged read, caller cancellation, or an unconfirmed inline storage
operation also preserves the execution identity so the source can replay an
existing seal. Only a scheduled read for that obligation can claim the retry.
The ledger rejects a competing reservation or a
second snapshot root during the atomic revision commit. With a selected policy
and no account publication lock, additional reads and later periods need durable
evidence ruling out a competing close. Recovery checks immutable official
revisions as well as retained observations and reservations when the stabilization
lookup returns no schedule.

## Persistence and compatibility

`PgReportingLedgerStore.create_schema()` installs two additive private tables:
`reporting_provisional_acquisitions` and `reporting_provisional_observations`.
They retain immutable reservation and observation metadata. The existing
restatement checkpoint remains the progress cursor. The in-memory store has the
same atomic operations. Custom producer stores must implement
`PgReportingLedgerStore.create_schema()` installs
`reporting_provisional_acquisitions` and `reporting_provisional_observations`
for immutable reservations and observations. The existing restatement checkpoint
tracks progress. Its stabilization policy and next due time live in
`adcp_reporting_stabilization_checkpoints`, keyed to the checkpoint's obligation,
ordinal and timestamp. An older writer can advance the checkpoint without
inheriting the previous policy or deadline. The in-memory store has the same
atomic operations. Custom producer stores must implement
`ProvisionalObservationStore` from `adcp.reporting.ledger.provisional` together
with `RestatementCheckpointStore`.

Expand Down
42 changes: 42 additions & 0 deletions docs/reporting-source-adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -266,3 +266,45 @@ and the [conformance tests](../tests/conformance/reporting/test_inline_cell_avai
Run `run_reporting_source_replay_conformance` against your adapter and staging
store to verify that the same execution key returns the original sealed
evidence even if source measurements or availability later change.

## Official source evidence

For a new generation with `required_finality: official`, copy one policy from
the pinned report definition into `ReportingDefinitionBinding.finality_policy`.
The adopter selects the policy for the generation. An explicit snapshot-to-official
offering also needs a selection. Snapshot-only offerings need no policy. Their
final scheduled read still produces a snapshot, even after prolonged downtime.

| Selected basis | Source offering | Required evidence |
| --- | --- | --- |
| `source_final` | Authoritative | Return `provider_declared` or `provider_job_terminal` evidence with the exact selected `source_signal` in `evidence_ref`. The provider's timestamp becomes `finalized_at`. |
| `contractual_cutoff` | Snapshot | Read and verify complete staged content at or after period end plus `duration_after_period_end`. The source observation and read cutoff must reach that boundary and the available data must cover the period end. |
| `stabilized` | Snapshot | The SDK checks verified content from consecutive scheduled observations at the frozen cadence against `minimum_age` and `unchanged_for`. |

For example, if the definition declares the signal `warehouse.close.completed`,
an adapter can return the timestamp of the completed upstream job.

```python
from adcp.reporting.inline_source import InlineFetchResult
from adcp.reporting.source import ReportingFinalityEvidenceV1

return InlineFetchResult(
rows=rows,
data_through=period_end,
finality_evidence=ReportingFinalityEvidenceV1(
basis="provider_job_terminal",
evidence_ref="warehouse.close.completed",
observed_at=job.completed_at,
),
)
```

`stable_since` records source evidence. Setting `official_close_lag` opts a
snapshot offering into an explicit close. For `source_final` and
`contractual_cutoff`, the lag can delay the source attempt. For `stabilized`,
the SDK promotes the snapshot when the scheduled observations satisfy
`minimum_age` and `unchanged_for`, independently of the lag value. A custom Core
ledger that admits official work must implement
`ReportingSourcePublicationStore.source_publication(account_id)` and serialize
publication for that account across workers. Exact reads and replays retain
the original policy fields of previously published official revisions.
Loading
Loading