Repository navigation
Description
Summary
ReportingProducer.run_worker() publishes at most one revision per obligation, forever. A source that keeps revising a closed period -- which is most ad servers -- has no supported path to correct what was published, and no path to ever declare a period final.
The restate=True switch exists on acquire_obligation, but nothing in the SDK ever sets it and no policy decides when it should be set. In practice every adopter with a restating source must build a second scheduler beside run_worker.
Current behavior
adcp/reporting/ledger/producer.py:
satisfied = any(item.readable for item in revisions)
if satisfied and not restate:
return Nonerun_worker → _acquire_pending → acquire_obligation(...) uses the default restate=False, so a period is read exactly once and then skipped on every subsequent turn.
$ grep -rn "restate=True" adcp/reporting/
$ # no results
The skip itself is correct and the docstring justifies it well -- re-reading a settled period every turn would burn upstream quota to republish identical bytes. The gap is that there is no way to express when a period stops being settled.
Why this matters
Google Ad Manager continues to adjust a closed day for several days: invalid-traffic filtering, late viewability, delayed conversions. The same is true of FreeWheel and most server-side sources.
The current shape produces the worst of both outcomes:
- the revision is labelled
snapshot, so it is permanently marked "may change"; - nothing ever changes it, so it never settles either.
A buyer verifies the digest, reconciles, and books a number the seller's own ad server has since revised. The seller has no way to tell them, and no way to ever say "this one is final."
The protocol already models the two states -- snapshot supersedes, official is terminal ("An official close is terminal. A later source correction is an adjustment, never another acquisition"). What is missing is the policy that moves a period from the first to the second.
Proposed surface
Declare settling behaviour on the offering, since it is a property of the source:
ProvisionalSnapshotOfferingV1(
...,
# How long after period.end the source may still revise this period.
restatement_window="P3D",
# Optional re-read cadence inside that window; defaults to the schedule.
restatement_cadence="P1D",
# Optional: promote to official once the window closes.
official_close_lag="P3D",
)revision_semantics is currently a fixed Literal["provisional_replaceable"], which states that revisions may be replaced but carries no window, so it cannot drive a worker.
Then run_worker would, without any new caller code:
- re-acquire a satisfied
snapshotobligation whilenow < period.end + restatement_window, no more often thanrestatement_cadence; - commit a superseding revision only when the observed content actually differs, so an unchanged re-read costs one upstream call and no new bytes;
- promote to
officialonce the window closes, where the offering supports official finality; - leave today's behaviour unchanged when no window is declared.
A source that learns per-slice that a reading is still moving should also be able to say so directly -- for example a provisional_until on the fetch result -- so an adapter with real evidence can override the declared default.
Acceptance criteria
- An offering declaring a restatement window causes
run_workerto re-acquire satisfied snapshot obligations inside that window, with no caller changes. - Re-reads honour the declared cadence and do not re-read on every turn.
- An unchanged re-read commits no revision; a changed one commits a superseding revision and the prior bytes stay readable.
- Once the window closes, the period is promoted to
officialwhere the offering supports it, and is never re-acquired afterwards. - An offering with no declared window behaves exactly as today.
- A per-slice provisional signal from the source overrides the declared window.
-
get_reporting_statusdistinguishes "settled and final" from "published but still inside its restatement window," so a consumer knows whether it is safe to bill. - Tests cover an unchanged re-read, a changed re-read, window expiry with promotion, and a source that has no window.
Adopter impact
Embedded Sales Agent is GAM-backed. Today it would publish a day at period end + 1 hour and freeze that reading permanently, while GAM continues adjusting it for days. We can declare the window -- that is our fact to know -- but we cannot act on it without either this policy or a second scheduler duplicating run_worker's leasing and identity logic.
Related: #1172 (adapter-first service), #1167 (post-official adjustments as the terminal-state counterpart).
Activity
bokelley commentedon Sep 16, 2026
Two implementation notes from wiring the adopter side against this.
1. For a multi-tenant seller the per-slice signal is the primary mechanism, not the override.
The issue frames the offering-level
restatement_windowas the main input and the per-slice signal as an override. That inverts for us. ESA runs many tenants behind one offering, and each tenant's ad server settles differently — GAM, FreeWheel, and AdsWizz have nothing in common here. The window is a property of the tenant's ad server, so a single static value on the offering cannot express it.What we can do is report it per slice, from the account's resolved adapter:
For that to drive
run_worker,provisional_untilhas to be persisted on the committed revision, because the worker decides whether to re-acquire before it calls the source. The rule becomes "re-acquire whilenow < last_revision.provisional_until," which needs no offering field at all and degrades correctly: a source that reports nothing is never re-read, exactly as today.The offering-level window is still worth having as a default for single-source sellers. It just should not be the only way in.
2.
ProvisionalSnapshotOfferingV1cannot express official close today.It is a fixed literal with no window, so it states that revisions may be replaced while carrying nothing actionable. Combined with the offering being snapshot-only, there is no shape on this class for "promote to official once the window closes" — acceptance criterion 4 has nowhere to land. Promotion would need either a widened
revision_semantics, or a separate offering type that supportsofficialfinality with this one naming it as its successor.Worth settling that before implementation, because it decides whether this issue is a producer-only change or also a source-contract change.
3. Refusing to guess should be a supported state.
We deliberately leave an uncharacterised adapter with no window, which keeps its periods provisional forever. That is the honest default — closing a period officially on a guessed settling time hands a buyer a final figure the ad server later contradicts, which is worse than never closing it. It would help if
get_reporting_statuscould distinguish "still inside a known window" from "this source has no characterised settling behaviour," since those mean different things to a consumer deciding whether to bill.bokelley commentedon Sep 16, 2026
Triage
Classification: Feature request
Bucket(s): handlers, client
Status: clarify
What the experts said:
restatement_window/restatement_cadenceare clean SDK-policy fields that don't touch the wire;official_close_lagpromotion requires either a pairedAuthoritativeOfferingV1or a newrevision_semanticsvariant (both are protocol-level changes, not SDK-only); the "content-differs-skip" rule needs a correction becausepublication_content_fingerprint_v1deliberately excludes acquisition timing sodata_throughcan advance on a quiet campaign — suppressing the commit stops watermarks from advancing.My take: The restatement-window and cadence fields are implementable now and non-breaking. The
official_close_lagpath and the "content differs" rule each hit a real protocol constraint that needs a design decision before the implementation brief can be written cleanly.Clarifying questions:
official_close_lagpromotion route. TheReportingRevisionRecord.__post_init__enforcesfinality_basis,finality_policy_id, andfinalized_atfor any official revision, andSourceBatchManifestV1ties publication class torevision_kind. When the window expires, should the worker issue a new slice request against a pairedAuthoritativeOfferingV1(usingfinality_basis="elapsed_settlement_window"from the existingReportingFinalityEvidenceV1.basisenum), or do you intend a direct ledger-side promotion without a new manifest? The first is more work but preserves the immutable-revision invariant; the second is a new code path that bypasses manifest validation."Changed re-read" skip rule. The acceptance criterion says "an unchanged re-read commits no revision."
publication_content_fingerprint_v1deliberately excludes acquisition timing sodata_throughcan advance independently of row bytes — suppressing a commit whenobject_set_sha256matches would freezedata_throughfor a quiet campaign, making it look stale to watermark consumers. Should the rule instead be: always commit a new revision, but reuse the same staged object generation (i.e. skip the upload, not the ledger write) when row content is identical?feat(reporting): complete seller-side Managed Delivery and Reconciled Billing #1167 boundary. Issue feat(reporting): complete seller-side Managed Delivery and Reconciled Billing #1167's accepted-criteria item "Adjustments and their receipts preserve official-revision/finality rules" and
correction_policycurrently live onAuthoritativeOfferingV1. If feat(reporting): complete seller-side Managed Delivery and Reconciled Billing #1167 extendscorrection_policyto cover the provisional path — covering the same window period asofficial_close_lag— the two overlap. Isofficial_close_lagdesigned to be the SDK's own policy that assumes no paired authoritative offering, or does it require one?Triaged by Claude Code. Session: https://claude.ai/code/session_015xT7RAu7hePz1fxa7UfoAa
Generated by Claude Code
bokelley commentedon Sep 16, 2026
Triage
Classification: Feature request
Bucket(s): client, handlers (neither label found in repo — gap noted)
Status: ready-for-human
What the experts said:
provisional_untilon committed revisions is architecturally sound, but_checksum_binds_the_declarationrecomputescapabilities_sha256over the full declaration — adding any field toProvisionalSnapshotOfferingV1(even optional) invalidates stored checksums for deployed adapters, making this a deployment-breaking change regardless of Pydantic additive-ness; official-close promotion can be kept as silent producer-side policy (avoids widening theLiteral) but the choice must be explicit.run_workerextension requires no public API break (existingrestate=True+required_finalitydispatch covers it); wideningrevision_semanticsfrom its fixedLiteralbreaks exhaustive-match consumers and requires alias-export coordination; AC7 (get_reporting_statusdistinction) gates on an upstream spec-schema addition outside this repo.My take: @bokelley's three design notes are confirmed by experts and materially change the scope — the
capabilities_sha256checksum invariant alone means this is not a non-breaking additive change. Three decisions need explicit answers before an implementation brief can be written.For @bokelley — three design questions:
capabilities_sha256bump (requiring adapter re-registration) fit a minor release, or should new settling fields land on a successor offering type to avoid breaking deployed adapters?revision_semanticsto advertise the behaviour to buyers? The latter is a source-contract break.get_reporting_status"inside known window vs. no window declared" distinction requires an upstream spec-schema addition — in scope here or a follow-on?Triaged by Claude Code. Session: https://claude.ai/code/session_01EeyD2H9y9zXGX6gqBg1QZ2
Generated by Claude Code
bokelley commentedon Sep 16, 2026
Triage addendum — code-reviewer completed after the initial comment.
Two additional findings worth noting before implementation:
Dedup comparator (implementation note, not a blocker on the design questions above): The acceptance criterion "an unchanged re-read commits no revision" cannot use
revision_content_sha256for the comparison. That field's JCS input includesreporting_revision_id, which is derived frommanifest.publication_id— two source reads of identical rows produce differentrevision_content_sha256values because the publication IDs differ. The correct comparator for "same rows as the prior revision" issource_manifest_sha256(stored onReportingRevisionRecord). This should be reflected in the implementation spec.official_close_lagpromotion needs its own code path:commit_revision_from_manifestreadsmanifest.finality_evidence.observed_atforfinalized_at— a clock-driven promotion doesn't come from a source manifest. A separate path (or apromote_snapshot_to_official()method) is needed, wherefinality_basis="contractual_cutoff"andfinalized_at=period.end + official_close_lag. The existingReportingRevisionRecord.__post_init__accepts these values; it's the call site that needs design. This is complementary to question 1 above (whether a pairedAuthoritativeOfferingV1is required).capabilities_sha256is not a concern:reporting_source_capabilities_sha256_v1usesmodel_dump(exclude_none=True), so optional-None fields are excluded from the hash. Existing declarations that don't set the new fields will produce identical hashes. Not breaking.Scope recommendation: The restatement re-acquisition + cadence throttle is self-contained and can land first. The
official_close_lagpromotion path andget_reporting_statuschanges to expose settling state are a natural second PR.Triaged by Claude Code. Session: https://claude.ai/code/session_015xT7RAu7hePz1fxa7UfoAa
Generated by Claude Code
bokelley commentedon Sep 16, 2026
4. "Keep re-reading" and "may close officially" need to be separate signals.
Implementing the adopter side surfaced a conflation in my own proposal. A single
provisional_untilis being asked to carry two independent decisions:Those want opposite defaults. Re-reading should default on, because a change after the window is only ever surfaced by a re-poll: declining to re-poll does not make the published number safer, it guarantees the seller never learns it was wrong. Official close should default off, because closing on a guessed settling time hands a buyer a final figure the source later contradicts.
We initially modelled an unmeasured adapter as "no window," which conservatively meant never closing it — but it also meant never re-reading it, which is the worse of the two failures and the one that actually bites. We now apply a 3-day re-read floor to every adapter and let measured windows override it in either direction, while relying on the offering being snapshot-only to prevent any official close.
Concretely, a source should be able to say "re-read me until T" without thereby asserting "and I am final at T." If the producer derives both from one field, every adopter has to pick which failure they prefer.
This interacts with note 2 above: because
ProvisionalSnapshotOfferingV1cannot express official close at all today, the re-read half is the only half that is actionable — which is a good argument for landing it first and independently.bokelley commentedon Sep 16, 2026
Triage
Classification: Feature request
Bucket(s): client, handlers
Status: ready-to-implement (re-read scope only)
What the experts said:
ProvisionalSnapshotOfferingV1structurally cannot produce official close (ReportingRevisionRecord.__post_init__hard-refusesfinality_basison snapshot revisions); re-read-only is a complete non-misleading unit; 3-day floor belongs as SDK policy inrun_worker, not an offering declaration;provisional_until: datetime | None = NoneonReportingRevisionRecordis safe but must be frozen at commit time from the lockeddelivery_config_version, not re-derived at re-read time.My take: Note 4 resolves the two remaining design questions from the prior triage. Official-close and AC7 defer cleanly with no half-open door — the offering type structurally prevents accidental promotion, so re-read-only ships as a complete unit.
Implementation brief:
adcp/reporting/ledger/models.py: Addprovisional_until: datetime | None = NonetoReportingRevisionRecord. Field is excluded from both content hashes — no checksum breakage.adcp/reporting/ledger/producer.py: In_acquire_pending, computereread_until = max(period.end + timedelta(days=3), last_revision.provisional_until or epoch); passrestate=Truetoacquire_obligationwhennow < reread_untilfor snapshot obligations. Freezeprovisional_untilat commit time from the offering'sdelivery_config_version. Usesource_manifest_sha256(notrevision_content_sha256) as the "unchanged rows" comparator — skip the ledger write on identical rows, but always advancedata_through.pytest adcp/reporting/,mypy src/,ruff check ./black --check .. Non-breaking: new optional field (None default), no offering surface change, existingrestate=Falsepath unchanged outside the window.Triaged by Claude Code. Session: https://claude.ai/code/session_01LdpbxVrbCSHWQ3Cak6emmb
Generated by Claude Code