Skip to content

feat(backend): query nested JSON paths with a GIN index in PostgresDocumentStore - #181

Open
raederan wants to merge 6 commits into
ci-test-backendsfrom
feat-postgres-query
Open

raederan wants to merge 6 commits into
ci-test-backendsfrom
feat-postgres-query

Conversation

@raederan

@raederan raederan commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Summary

Covers the last acceptance criterion of #35: optimized query generation for nested JSON paths. Stacked on #180, so this merges into ci-test-backends, not main.

  • DSL: the | operator, and nested paths by attribute chaining. Person.address.city == "Berlin" builds the dotted condition address.city. Chaining lives in the shared descriptor FieldProxy (pydantic v2 and v1). Links and lists of models raise a clear error, and the legacy bindings raise NotImplementedError.
  • SimpleDictDocumentStore: nested paths and or.
  • PostgresDocumentStore.query(): equality is jsonb containment, served by a new GIN jsonb_path_ops index (index=False opts out). Ranges only compare values of the same JSON type. One round trip, ordered by IRI, with an optional query_limit.
  • SPARQL resolvers: nested paths now raise. With a @vocab in the context they used to expand into a made-up predicate and silently match nothing.
  • Docs: a querying section, the embedded versus link rule, and the differences between the stores.

Notes

Refs #35, #48

@raederan raederan mentioned this pull request Nov 16, 2026
4 tasks
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Release preview

Merging this PR would release v2.4.0 (current: v2.3.2).

Changelog preview (truncated)
## v2.4.0 (2026-11-16)

### Bug Fixes

- **backend**: Reject nested paths in the SPARQL resolver with a clear error
  ([`582c192`](https://github.com/OO-LD/oold-python/commit/582c192cb527782d927754091d250ca4fee5fc55))

### Continuous Integration

- Test against postgres 16, 17 and 18 via service containers
  ([`373457d`](https://github.com/OO-LD/oold-python/commit/373457dbca15192f0d057ea627b6513542b6b827))

- Upload postgres job coverage to codecov
  ([`259a3e9`](https://github.com/OO-LD/oold-python/commit/259a3e9bd95076551213306f29760858332715b0))

### Documentation

- **backend**: Document PostgresDocumentStore and fix sqlite db_path
  ([`20040e9`](https://github.com/OO-LD/oold-python/commit/20040e91a426867d8a50becbb878065238a4d4b6))

- **backend**: Document querying and nested paths
  ([`01de6dc`](https://github.com/OO-LD/oold-python/commit/01de6dc09f290eeb1222ceec10e174574ea8022e))

### Features

- **backend**: Add PostgresDocumentStore behind oold[postgres]
  ([`bf87158`](https://github.com/OO-LD/oold-python/commit/bf87158dc0522e43576a1d5187226d85552d3117))

- **backend**: Add query support to PostgresDocumentStore with a GIN index
  ([`f1c614f`](https://github.com/OO-LD/oold-python/commit/f1c614faa91d6065cbdb9b1239625fc2035b0fc8))

- **backend**: Match nested paths and or in SimpleDictDocumentStore
  ([`347fcf0`](https://github.com/OO-LD/oold-python/commit/347fcf099bebc7cdb97104ff43182c5478e9bb43))

- **model**: Build nested field conditions with attribute chaining
  ([`7e67f27`](https://github.com/OO-LD/oold-python/commit/7e67f2717edc6604d5ec13d7c38308ae9e074a45))

- **query**: Add the or operator and dotted field paths to the DSL
  ([`5300e3d`](https://github.com/OO-LD/oold-python/commit/5300e3d1cb6d63a4c39bdb6d83b0e0e8a112bbda))

### Testing

- **backend**: Add gated postgres tests with per-test table
  ([`9e1d783`](https://github.com/OO-LD/oold-python/commit/9e1d78301038d82462c8a1b95b87648f483e95fd))

- **backend**: Cover postgres store close and missing driver
  ([`5508b55`](https://github.com/OO-LD/oold-python/commit/5508b55d6ca13f689416d70e6565a1cf618ebcf1))

- **backend**: Keep benchmark test names stable
  ([`dbe1a14`](https://github.com/OO-LD/oold-python/commit/dbe1a14b3efc08f9b34917ec40e2073572687a7f))

- **backend**: Run one contract suite across all stores
  ([`36ad667`](https://github.com/OO-LD/oold-python/commit/36ad66735322908a8b39886d5d0b189f525198ed))

Preview via python-semantic-release and conventional commits.

@raederan
raederan requested a review from simontaurus November 16, 2026 16:28
@raederan raederan self-assigned this Nov 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

Click to see benchmark comparison
📊 Benchmark Comparison (threshold: 1.3x)
============================================================

➖ Unchanged (within threshold):
  ➖ test_simple_dict_document_store: 0.0010s → 0.0011s (+0.5%)
  ➖ test_sqlite_document_store: 0.0011s → 0.0011s (+0.5%)
  ➖ test_local_sparql_store: 0.0222s → 0.0222s (+0.1%)
  ➖ test_oneof_subschema: 0.0350s → 0.0351s (+0.4%)
  ➖ test_enum_docstrings: 0.0299s → 0.0298s (-0.1%)
  ➖ test_subclass_inheritance: 0.0319s → 0.0317s (-0.6%)
  ➖ test_class_hierarchy: 0.0298s → 0.0302s (+1.1%)
  ➖ test_core[v1]: 0.0209s → 0.0214s (+2.2%)
  ➖ test_core[v2]: 0.0248s → 0.0248s (+0.0%)
  ➖ test_schema_generation[v1]: 0.0011s → 0.0011s (-0.8%)
  ➖ test_schema_generation[v2]: 0.0022s → 0.0023s (+3.3%)
  ➖ test_simple_json: 0.0004s → 0.0004s (+0.8%)
  ➖ test_complex_graph: 0.0010s → 0.0010s (+0.3%)

============================================================
Summary: 0 regressions, 0 improvements, 13 unchanged
============================================================

✅ No significant performance regressions

Threshold: 1.3x (30% slower triggers a regression warning)

Note: Benchmarks are informational only and won't fail the build.

💡 Tip: Download the benchmark-results artifact for detailed JSON data

@codecov

codecov Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@raederan raederan added documentation Improvements or additions to documentation enhancement New feature or request labels Nov 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

Click to see benchmark comparison
📊 Benchmark Comparison (threshold: 1.3x)
============================================================

➖ Unchanged (within threshold):
  ➖ test_simple_dict_document_store: 0.0016s → 0.0016s (+0.3%)
  ➖ test_sqlite_document_store: 0.0017s → 0.0017s (+0.3%)
  ➖ test_local_sparql_store: 0.0372s → 0.0382s (+2.7%)
  ➖ test_oneof_subschema: 0.0577s → 0.0581s (+0.8%)
  ➖ test_enum_docstrings: 0.0493s → 0.0482s (-2.2%)
  ➖ test_subclass_inheritance: 0.0531s → 0.0516s (-2.8%)
  ➖ test_class_hierarchy: 0.0480s → 0.0485s (+1.0%)
  ➖ test_core[v1]: 0.0342s → 0.0353s (+3.1%)
  ➖ test_core[v2]: 0.0411s → 0.0411s (+0.1%)
  ➖ test_schema_generation[v1]: 0.0016s → 0.0017s (+2.3%)
  ➖ test_schema_generation[v2]: 0.0033s → 0.0034s (+3.4%)
  ➖ test_simple_json: 0.0007s → 0.0007s (+0.7%)
  ➖ test_complex_graph: 0.0016s → 0.0016s (-0.3%)

============================================================
Summary: 0 regressions, 0 improvements, 13 unchanged
============================================================

✅ No significant performance regressions

Threshold: 1.3x (30% slower triggers a regression warning)

Note: Benchmarks are informational only and won't fail the build.

💡 Tip: Download the benchmark-results artifact for detailed JSON data

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant