Skip to content

feat(backend): add PostgresDocumentStore and test it on PostgreSQL 16-18 in CI - #180

Open
raederan wants to merge 8 commits into
mainfrom
ci-test-backends
Open

raederan wants to merge 8 commits into
mainfrom
ci-test-backends

Conversation

@raederan

@raederan raederan commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Add PostgresDocumentStore (JSONB, one row per IRI, upsert, batch writes in one transaction), following the SqliteDocumentStore pattern. It sits behind the new oold[postgres] extra (psycopg 3), so core dependencies are unchanged.
  • Add a tests-postgres CI job: GitHub service containers, PostgreSQL 16, 17 and 18, Python 3.12. It fails if any Postgres test is skipped, and the PG 17 leg uploads coverage to Codecov.
  • Tests: a postgres_store fixture (own table per test, dropped afterwards), store specific tests, and a benchmark test reusing the existing roundtrip procedure.
  • Docs: backend how-to section, local test recipe, README and architecture tables. Fixes the SQLite example (file_path is actually db_path).

Notes

  • Postgres tests skip without OOLD_TEST_POSTGRES_URL, so make test and the OS/Python matrix are unchanged. Run them locally with OOLD_TEST_POSTGRES_URL=postgresql://... make test.
  • Not done: optimized query generation for nested JSON paths. query() still raises NotImplementedError.

Refs #35 (3 of 4 acceptance criteria; nested-path query generation remains)

@raederan
raederan requested a review from simontaurus November 16, 2026 15:17
@raederan raederan self-assigned this Nov 16, 2026
@raederan raederan added documentation Improvements or additions to documentation enhancement New feature or request labels Nov 16, 2026
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Release preview

Merging this PR would release v2.4.0 (current: v2.3.2).

Changelog preview (truncated)
## v2.4.0 (2026-11-16)

### Continuous Integration

- Test against postgres 16, 17 and 18 via service containers
  ([`373457d`](https://github.com/OO-LD/oold-python/commit/373457dbca15192f0d057ea627b6513542b6b827))

- Upload postgres job coverage to codecov
  ([`259a3e9`](https://github.com/OO-LD/oold-python/commit/259a3e9bd95076551213306f29760858332715b0))

### Documentation

- **backend**: Document PostgresDocumentStore and fix sqlite db_path
  ([`20040e9`](https://github.com/OO-LD/oold-python/commit/20040e91a426867d8a50becbb878065238a4d4b6))

### Features

- **backend**: Add PostgresDocumentStore behind oold[postgres]
  ([`bf87158`](https://github.com/OO-LD/oold-python/commit/bf87158dc0522e43576a1d5187226d85552d3117))

### Testing

- **backend**: Add gated postgres tests with per-test table
  ([`9e1d783`](https://github.com/OO-LD/oold-python/commit/9e1d78301038d82462c8a1b95b87648f483e95fd))

- **backend**: Cover postgres store close and missing driver
  ([`5508b55`](https://github.com/OO-LD/oold-python/commit/5508b55d6ca13f689416d70e6565a1cf618ebcf1))

- **backend**: Keep benchmark test names stable
  ([`dbe1a14`](https://github.com/OO-LD/oold-python/commit/dbe1a14b3efc08f9b34917ec40e2073572687a7f))

- **backend**: Run one contract suite across all stores
  ([`36ad667`](https://github.com/OO-LD/oold-python/commit/36ad66735322908a8b39886d5d0b189f525198ed))

Preview via python-semantic-release and conventional commits.

@github-actions

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

Click to see benchmark comparison
📊 Benchmark Comparison (threshold: 1.3x)
============================================================

⚠️  New benchmark: test_store_roundtrip[dict]
⚠️  New benchmark: test_store_roundtrip[sqlite]
⚠️  New benchmark: test_store_roundtrip[sparql]
⚠️  Removed benchmark: test_simple_dict_document_store
⚠️  Removed benchmark: test_sqlite_document_store
⚠️  Removed benchmark: test_local_sparql_store
➖ Unchanged (within threshold):
  ➖ test_oneof_subschema: 0.0500s → 0.0467s (-6.5%)
  ➖ test_enum_docstrings: 0.0354s → 0.0378s (+6.8%)
  ➖ test_subclass_inheritance: 0.0380s → 0.0446s (+17.4%)
  ➖ test_class_hierarchy: 0.0415s → 0.0383s (-7.8%)
  ➖ test_core[v1]: 0.0306s → 0.0276s (-10.0%)
  ➖ test_core[v2]: 0.0311s → 0.0305s (-1.8%)
  ➖ test_schema_generation[v1]: 0.0012s → 0.0011s (-3.5%)
  ➖ test_schema_generation[v2]: 0.0025s → 0.0024s (-0.5%)
  ➖ test_simple_json: 0.0004s → 0.0004s (-0.3%)
  ➖ test_complex_graph: 0.0011s → 0.0011s (-1.3%)

============================================================
Summary: 0 regressions, 0 improvements, 10 unchanged
============================================================

✅ No significant performance regressions

Threshold: 1.3x (30% slower triggers a regression warning)

Note: Benchmarks are informational only and won't fail the build.

💡 Tip: Download the benchmark-results artifact for detailed JSON data

@codecov

codecov Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

Click to see benchmark comparison
📊 Benchmark Comparison (threshold: 1.3x)
============================================================

➖ Unchanged (within threshold):
  ➖ test_simple_dict_document_store: 0.0008s → 0.0007s (-3.1%)
  ➖ test_sqlite_document_store: 0.0008s → 0.0009s (+8.4%)
  ➖ test_local_sparql_store: 0.0204s → 0.0212s (+3.7%)
  ➖ test_oneof_subschema: 0.0459s → 0.0367s (-20.0%)
  ➖ test_enum_docstrings: 0.0351s → 0.0343s (-2.1%)
  ➖ test_subclass_inheritance: 0.0370s → 0.0379s (+2.4%)
  ➖ test_class_hierarchy: 0.0342s → 0.0325s (-4.9%)
  ➖ test_core[v1]: 0.0269s → 0.0241s (-10.4%)
  ➖ test_core[v2]: 0.0306s → 0.0290s (-5.5%)
  ➖ test_schema_generation[v1]: 0.0008s → 0.0009s (+13.7%)
  ➖ test_schema_generation[v2]: 0.0017s → 0.0017s (-1.7%)
  ➖ test_simple_json: 0.0003s → 0.0003s (-1.9%)
  ➖ test_complex_graph: 0.0008s → 0.0008s (-1.0%)

============================================================
Summary: 0 regressions, 0 improvements, 13 unchanged
============================================================

✅ No significant performance regressions

Threshold: 1.3x (30% slower triggers a regression warning)

Note: Benchmarks are informational only and won't fail the build.

💡 Tip: Download the benchmark-results artifact for detailed JSON data

@github-actions

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

Click to see benchmark comparison
📊 Benchmark Comparison (threshold: 1.3x)
============================================================

➖ Unchanged (within threshold):
  ➖ test_simple_dict_document_store: 0.0015s → 0.0015s (-0.7%)
  ➖ test_sqlite_document_store: 0.0016s → 0.0016s (+0.2%)
  ➖ test_local_sparql_store: 0.0357s → 0.0371s (+3.8%)
  ➖ test_oneof_subschema: 0.0535s → 0.0541s (+1.2%)
  ➖ test_enum_docstrings: 0.0443s → 0.0450s (+1.6%)
  ➖ test_subclass_inheritance: 0.0500s → 0.0480s (-4.1%)
  ➖ test_class_hierarchy: 0.0458s → 0.0454s (-1.0%)
  ➖ test_core[v1]: 0.0330s → 0.0341s (+3.4%)
  ➖ test_core[v2]: 0.0379s → 0.0385s (+1.7%)
  ➖ test_schema_generation[v1]: 0.0015s → 0.0015s (+1.5%)
  ➖ test_schema_generation[v2]: 0.0032s → 0.0032s (-0.8%)
  ➖ test_simple_json: 0.0006s → 0.0006s (+0.7%)
  ➖ test_complex_graph: 0.0014s → 0.0015s (+6.2%)

============================================================
Summary: 0 regressions, 0 improvements, 13 unchanged
============================================================

✅ No significant performance regressions

Threshold: 1.3x (30% slower triggers a regression warning)

Note: Benchmarks are informational only and won't fail the build.

💡 Tip: Download the benchmark-results artifact for detailed JSON data

@github-actions

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

Click to see benchmark comparison
📊 Benchmark Comparison (threshold: 1.3x)
============================================================

➖ Unchanged (within threshold):
  ➖ test_simple_dict_document_store: 0.0012s → 0.0012s (+4.2%)
  ➖ test_sqlite_document_store: 0.0013s → 0.0013s (-0.5%)
  ➖ test_local_sparql_store: 0.0278s → 0.0273s (-1.7%)
  ➖ test_oneof_subschema: 0.0440s → 0.0451s (+2.7%)
  ➖ test_enum_docstrings: 0.0372s → 0.0369s (-0.8%)
  ➖ test_subclass_inheritance: 0.0398s → 0.0406s (+2.0%)
  ➖ test_class_hierarchy: 0.0383s → 0.0373s (-2.8%)
  ➖ test_core[v1]: 0.0269s → 0.0264s (-2.0%)
  ➖ test_core[v2]: 0.0332s → 0.0304s (-8.6%)
  ➖ test_schema_generation[v1]: 0.0012s → 0.0012s (+0.3%)
  ➖ test_schema_generation[v2]: 0.0025s → 0.0025s (+1.9%)
  ➖ test_simple_json: 0.0004s → 0.0004s (-2.6%)
  ➖ test_complex_graph: 0.0011s → 0.0011s (+1.8%)

============================================================
Summary: 0 regressions, 0 improvements, 13 unchanged
============================================================

✅ No significant performance regressions

Threshold: 1.3x (30% slower triggers a regression warning)

Note: Benchmarks are informational only and won't fail the build.

💡 Tip: Download the benchmark-results artifact for detailed JSON data

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant