Skip to content

perf(backend): benchmark RDF import and export and fix quadratic local SPARQL import - #182

Open
raederan wants to merge 10 commits into
feat-postgres-queryfrom
rdf-import-export-performance
Open

raederan wants to merge 10 commits into
feat-postgres-queryfrom
rdf-import-export-performance

Conversation

@raederan

Copy link
Copy Markdown
Member

Closes #29 (issue).

Stacked on #181. Retarget to main once #180 and #181 have merged.

Summary

  • Adds a benchmark harness in tests/perf/ for RDF export, store and import at 1k to 100k triples: time, CPU, peak memory, GC, per-call latency, SPARQL call count and a scaling exponent. Each run uses its own subprocess. Run it with make benchmark-rdf.
  • Profiling found two bottlenecks in LocalSparqlBackend. Import was quadratic, because a FILTER scanned the whole graph once per IRI. Store spent most of its time parsing a SPARQL DELETE and building a graph per IRI.
  • Both now use the graph index directly. At 100k triples, import goes from about 22,000 s (predicted) to 6.1 s, and store from 24.1 s to 1.9 s. All scenarios are linear.
  • Missing IRIs now resolve to None, and storing None deletes the subject. Before, these raised IndexError and UnboundLocalError.
  • Adds a Benchmark RDF workflow that runs the full sizes, and psutil as a dev dependency.

Full results, bottleneck profile and the before and after matrix are in the issue comments.

@raederan
raederan requested a review from simontaurus November 17, 2026 16:08
@raederan raederan self-assigned this Nov 17, 2026
@raederan raederan added documentation Improvements or additions to documentation enhancement New feature or request labels Nov 17, 2026
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Release preview

Merging this PR would release v2.4.0 (current: v2.3.2).

Changelog preview (truncated)
## v2.4.0 (2026-11-17)

### Bug Fixes

- **backend**: Reject nested paths in the SPARQL resolver with a clear error
  ([`582c192`](https://github.com/OO-LD/oold-python/commit/582c192cb527782d927754091d250ca4fee5fc55))

- **backend**: Resolve missing local sparql nodes to none and delete on none
  ([`1a118ae`](https://github.com/OO-LD/oold-python/commit/1a118aee87070d8d84b744981a3fd6b1e10f5aaf))

### Continuous Integration

- Run rdf benchmark with the main workflow triggers
  ([`12e8735`](https://github.com/OO-LD/oold-python/commit/12e8735f22415b38f1541dd0d8da0f4c067f6663))

- Skip 100k triples in the rdf benchmark workflow
  ([`95bdbcd`](https://github.com/OO-LD/oold-python/commit/95bdbcde4a58aacceeea47bec4856546e7f73cb9))

- Test against postgres 16, 17 and 18 via service containers
  ([`373457d`](https://github.com/OO-LD/oold-python/commit/373457dbca15192f0d057ea627b6513542b6b827))

- Upload postgres job coverage to codecov
  ([`259a3e9`](https://github.com/OO-LD/oold-python/commit/259a3e9bd95076551213306f29760858332715b0))

### Documentation

- **backend**: Document PostgresDocumentStore and fix sqlite db_path
  ([`20040e9`](https://github.com/OO-LD/oold-python/commit/20040e91a426867d8a50becbb878065238a4d4b6))

- **backend**: Document querying and nested paths
  ([`01de6dc`](https://github.com/OO-LD/oold-python/commit/01de6dc09f290eeb1222ceec10e174574ea8022e))

- **perf**: Add rdf benchmark how-to and bottleneck report
  ([`0748329`](https://github.com/OO-LD/oold-python/commit/0748329548edc0e2580e331f88a9ffa2c53d4bb6))

### Features

- **backend**: Add PostgresDocumentStore behind oold[postgres]
  ([`bf87158`](https://github.com/OO-LD/oold-python/commit/bf87158dc0522e43576a1d5187226d85552d3117))

- **backend**: Add query support to PostgresDocumentStore with a GIN index
  ([`f1c614f`](https://github.com/OO-LD/oold-python/commit/f1c614faa91d6065cbdb9b1239625fc2035b0fc8))

- **backend**: Match nested paths and or in SimpleDictDocumentStore
  ([`347fcf0`](https://github.com/OO-LD/oold-python/commit/347fcf099bebc7cdb97104ff43182c5478e9bb43))

- **model**: Build nested field conditions with attribute chaining
  ([`7e67f27`](https://github.com/OO-LD/oold-python/commit/7e67f2717edc6604d5ec13d7c38308ae9e074a45))

- **query**: Add the or operator and dotted field paths to the DSL
  ([`5300e3d`](https://github.com/OO-LD/oold-python/commit/5300e3d1cb6d63a4c39bdb6d83b0e0e8a112bbda))

### Performance Improvements

- **backend**: Look up local sparql nodes without scanning the graph
  ([`4aabd2f`](https://github.com/OO-LD/oold-python/commit/4aabd2fdca21a21ff678f2f0c98aba2331719800))

### Testing

- **backend**: Add gated postgres tests with per-test table
  ([`9e1d783`](https://github.com/OO-LD/oold-python/commit/9e1d78301038d82462c8a1b95b87648f483e95fd))

- **backend**: Cover postgres store close and missing driver
  ([`5508b55`](https://github.com/OO-LD/oold-python/commit/5508b55d6ca13f689416d70e6565a1cf618ebcf1))

- **backend**: Keep benchmark test names stable
  ([`dbe1a14`](https://github.com/OO-LD/oold-python/commit/dbe1a14b3efc08f9b34917ec40e2073572687a7f))

- **backend**: Run one contract suite across all stores
  ([`36ad667`](https://github.com/OO-LD/oold-python/commit/36ad66735322908a8b39886d5d0b189f525198ed))

- **perf**: Add deterministic triple dataset generator
  ([`6b8909c`](https://github.com/OO-LD/oold-python/commit/6b8909cb89950958b1573be2b0bc70ffa41cfd55))

- **perf**: Add rdf export and import scenarios
  ([`5d89c19`](https://github.com/OO-LD/oold-python/commit/5d89c19bb3d0c3bd019775da5d8d6b0d174222cd))

- **perf**: Add resource metrics harness
  ([`884df69`](https://github.com/OO-LD/oold-python/commit/884df69096a1ad30f7536c2bdfb5d56d71ea40dc))

Preview via python-semantic-release and conventional commits.

@github-actions

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

Click to see benchmark comparison
📊 Benchmark Comparison (threshold: 1.3x)
============================================================

⚠️  Performance Regressions:
  ❌ test_oneof_subschema: 0.0444s → 0.0837s (+88.6%, ratio: 1.89x)
  ❌ test_enum_docstrings: 0.0384s → 0.0597s (+55.4%, ratio: 1.55x)
  ❌ test_subclass_inheritance: 0.0422s → 0.0597s (+41.4%, ratio: 1.41x)
  ❌ test_core[v1]: 0.0279s → 0.0385s (+37.9%, ratio: 1.38x)

✅ Performance Improvements:
  ✅ test_local_sparql_store: 0.0279s → 0.0032s (-88.6%, ratio: 0.11x)

➖ Unchanged (within threshold):
  ➖ test_simple_dict_document_store: 0.0012s → 0.0012s (-0.2%)
  ➖ test_sqlite_document_store: 0.0013s → 0.0013s (-0.7%)
  ➖ test_class_hierarchy: 0.0376s → 0.0481s (+27.9%)
  ➖ test_core[v2]: 0.0332s → 0.0371s (+11.6%)
  ➖ test_schema_generation[v1]: 0.0012s → 0.0012s (+1.1%)
  ➖ test_schema_generation[v2]: 0.0025s → 0.0024s (-2.1%)
  ➖ test_simple_json: 0.0004s → 0.0004s (-1.1%)
  ➖ test_complex_graph: 0.0011s → 0.0012s (+5.1%)

============================================================
Summary: 4 regressions, 1 improvements, 8 unchanged
============================================================

⚠️  Regressions detected but not failing build (informational only)

Threshold: 1.3x (30% slower triggers a regression warning)

Note: Benchmarks are informational only and won't fail the build.

💡 Tip: Download the benchmark-results artifact for detailed JSON data

@codecov

codecov Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

Click to see benchmark comparison
📊 Benchmark Comparison (threshold: 1.3x)
============================================================

✅ Performance Improvements:
  ✅ test_local_sparql_store: 0.0325s → 0.0039s (-87.9%, ratio: 0.12x)

➖ Unchanged (within threshold):
  ➖ test_simple_dict_document_store: 0.0014s → 0.0015s (+0.8%)
  ➖ test_sqlite_document_store: 0.0016s → 0.0016s (+0.7%)
  ➖ test_oneof_subschema: 0.0512s → 0.0523s (+2.1%)
  ➖ test_enum_docstrings: 0.0433s → 0.0443s (+2.4%)
  ➖ test_subclass_inheritance: 0.0467s → 0.0465s (-0.3%)
  ➖ test_class_hierarchy: 0.0449s → 0.0451s (+0.4%)
  ➖ test_core[v1]: 0.0312s → 0.0310s (-0.7%)
  ➖ test_core[v2]: 0.0362s → 0.0368s (+1.5%)
  ➖ test_schema_generation[v1]: 0.0015s → 0.0015s (+3.0%)
  ➖ test_schema_generation[v2]: 0.0030s → 0.0031s (+2.7%)
  ➖ test_simple_json: 0.0006s → 0.0006s (+0.5%)
  ➖ test_complex_graph: 0.0013s → 0.0013s (+0.4%)

============================================================
Summary: 0 regressions, 1 improvements, 12 unchanged
============================================================

✅ No significant performance regressions

Threshold: 1.3x (30% slower triggers a regression warning)

Note: Benchmarks are informational only and won't fail the build.

💡 Tip: Download the benchmark-results artifact for detailed JSON data

@raederan

Copy link
Copy Markdown
Member Author

ci skippes 100k (only local)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant