Goal
Implement the diff-sync subcommand of cmd/ingestion, currently stubbed to return an error. After a one-time full-import, diff-sync brings the registry up to date with incremental upstream changes — without re-reading the entire .osm.pbf.
Tracking under #10.
Approach options
- OSM diff files: OpenStreetMap publishes minutely / hourly / daily diff files (
.osc.gz). Pull a window of diffs, apply them. Standard approach; library support exists.
- Overpass periodic re-query: re-query a bounded region via Overpass on a schedule. Simpler but expensive and harder to keep precisely consistent.
Recommend OSM diff files for correctness; Overpass only as a fallback for spot-checking.
Open questions
- Diff cadence: minutely / hourly / daily? Daily is plenty for accessibility data.
- State tracking: where do we persist "last applied sequence number"? A small
ingestion_state table?
- Failure recovery: if a diff application fails mid-batch, how do we resume cleanly?
- Deletions: OSM diffs include node deletions. Do we hard-delete from
places, soft-delete via place_status, or skip deletions entirely until the precedence rules issue is settled?
Out of scope
- Diff-sync for non-OSM sources (not on the roadmap yet).
- Backfill of historical diffs older than the
full-import cutoff.
Acceptance
cmd/ingestion diff-sync applies pending OSM diff files and updates places accordingly.
- Last-applied-sequence is persisted across runs.
- Idempotent: re-running with no new diffs is a no-op.
- Has at least one integration test covering a small diff fixture.
Goal
Implement the
diff-syncsubcommand ofcmd/ingestion, currently stubbed to return an error. After a one-timefull-import,diff-syncbrings the registry up to date with incremental upstream changes — without re-reading the entire.osm.pbf.Tracking under #10.
Approach options
.osc.gz). Pull a window of diffs, apply them. Standard approach; library support exists.Recommend OSM diff files for correctness; Overpass only as a fallback for spot-checking.
Open questions
ingestion_statetable?places, soft-delete viaplace_status, or skip deletions entirely until the precedence rules issue is settled?Out of scope
full-importcutoff.Acceptance
cmd/ingestion diff-syncapplies pending OSM diff files and updatesplacesaccordingly.