MDEV-39834: Add TidesDB 9(v9.3.3) storage engine by guycipher · Pull Request #5166 · MariaDB/server

guycipher · 2026-06-02T21:21:45Z

The libtidesdb C library is vendored under storage/tidesdb/libtidesdb/ and compiled into a static archive that is statically linked into the loadable plugin module, mirroring storage/rocksdb, the engine is self-contained and needs no system-installed libtidesdb. include/providers is removed from the
engine's include dirs so the bundled zstd/lz4/snappy link directly instead of routing through MariaDB's compression-provider plugins.

For cross-version portability, spatial- and fulltext-index detection uses the HA_SPATIAL / HA_FULLTEXT key flags (with a HA_SPATIAL/HA_FULLTEXT -> *_legacy rename shim) rather than KEY::algorithm, which older servers leave
unset; my_global.h is included before handler.h so the server typedefs it needs are defined.

Adds the tidesdb mysql-test suite covering CRUD, MVCC and pessimistic locking, transactions/savepoints, isolation levels, compression, encryption, TTL, full-text, spatial, partitioning, object-store offload and crash/durability. The vector test auto-skips where the VECTOR type is unavailable (before 11.7).

Builds and full mysql-test suite shows green on MariaDB 11.4 (LTS) through to 13.0.2

The libtidesdb C library is vendored under storage/tidesdb/libtidesdb/ and compiled into a static archive that is statically linked into the loadable plugin module, mirroring storage/rocksdb, the engine is self-contained and needs no system-installed libtidesdb. include/providers is removed from the engine's include dirs so the bundled zstd/lz4/snappy link directly instead of routing through MariaDB's compression-provider plugins. For cross-version portability, spatial- and fulltext-index detection uses the HA_SPATIAL / HA_FULLTEXT key flags (with a HA_SPATIAL/HA_FULLTEXT -> *_legacy rename shim) rather than KEY::algorithm, which older servers leave unset; my_global.h is included before handler.h so the server typedefs it needs are defined. Adds the tidesdb mysql-test suite covering CRUD, MVCC and pessimistic locking, transactions/savepoints, isolation levels, compression, encryption, TTL, full-text, spatial, partitioning, object-store offload and crash/durability. The vector test auto-skips where the VECTOR type is unavailable (before 11.7). Builds and full mysql-test suite shows green on MariaDB 11.4 (LTS) through to 13.0.2

CLAassistant · 2026-06-02T21:21:52Z

All committers have signed the CLA.

gemini-code-assist

Code Review

This pull request integrates the TidesDB storage engine into MariaDB, adding the core handler implementation in ha_tidesdb.h, build configuration in CMakeLists.txt, vendored libtidesdb source files, and an extensive test suite. The reviewer provided several constructive suggestions, including moving the public header copy operation in CMakeLists.txt from configure time to build time to prevent stale headers during incremental builds. Additionally, the reviewer recommended using C++11 in-class member initializers for various member variables in ha_tidesdb.h to avoid uninitialized memory issues, and explicitly marking the ha_tidesdb destructor with override.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

guycipher · 2026-06-02T21:24:43Z

Base (11.4) >> 11.5 >> 12.0 >> … >> 12.x >> 13.0 >> main

Can propagate forward to every newer branch automatically.

guycipher · 2026-06-02T21:29:09Z

Full TideSQL reference: https://tidesdb.com/reference/tidesql/

guycipher · 2026-06-02T21:39:09Z

I see

/home/buildbot/storage/tidesdb/ha_tidesdb.cc: In member function 'virtual int ha_tidesdb::write_row(const uchar*)':
/home/buildbot/storage/tidesdb/ha_tidesdb.cc:6694:28: error: structured bindings only available with '-std=c++17' or '-std=gnu++17' [-Werror=c++17-extensions]
 6694 |                 for (auto &[term, tf] : tf_map)
      |                            ^
/home/buildbot/storage/tidesdb/ha_tidesdb.cc: In member function 'virtual int ha_tidesdb::update_row(const uchar*, const uchar*)':
/home/buildbot/storage/tidesdb/ha_tidesdb.cc:7898:32: error: structured bindings only available with '-std=c++17' or '-std=gnu++17' [-Werror=c++17-extensions]
 7898 |                     for (auto &[term, tf] : old_tf)
      |                                ^
/home/buildbot/storage/tidesdb/ha_tidesdb.cc:7906:32: error: structured bindings only available with '-std=c++17' or '-std=gnu++17' [-Werror=c++17-extensions]
 7906 |                     for (auto &[term, tf] : new_tf)
      |                                ^
/home/buildbot/storage/tidesdb/ha_tidesdb.cc:7926:32: error: structured bindings only available with '-std=c++17' or '-std=gnu++17' [-Werror=c++17-extensions]
 7926 |                     for (auto &[term, old_cnt] : old_tf)
      |                                ^
/home/buildbot/storage/tidesdb/ha_tidesdb.cc:7936:32: error: structured bindings only available with '-std=c++17' or '-std=gnu++17' [-Werror=c++17-extensions]
 7936 |                     for (auto &[term, new_cnt] : new_tf)
      |                                ^
/home/buildbot/storage/tidesdb/ha_tidesdb.cc: In member function 'virtual int ha_tidesdb::delete_row(const uchar*)':
/home/buildbot/storage/tidesdb/ha_tidesdb.cc:8206:28: error: structured bindings only available with '-std=c++17' or '-std=gnu++17' [-Werror=c++17-extensions]
 8206 |                 for (auto &[term, tf] : tf_map)
      |                            ^
/home/buildbot/storage/tidesdb/ha_tidesdb.cc: In member function 'virtual ha_rows ha_tidesdb::multi_range_read_info_const(uint, RANGE_SEQ_IF*, void*, uint, uint*, uint*, ha_rows, Cost_estimate*)':
/home/buildbot/storage/tidesdb/ha_tidesdb.cc:8609:25: error: 'struct TABLE' has no member named 'part_info'
 8609 |     if (table && table->part_info) return rows;
      |                         ^~~~~~~~~
/home/buildbot/storage/tidesdb/ha_tidesdb.cc: In member function 'virtual FT_INFO* ha_tidesdb::ft_init_ext(uint, uint, String*)':
/home/buildbot/storage/tidesdb/ha_tidesdb.cc:9850:16: error: structured bindings only available with '-std=c++17' or '-std=gnu++17' [-Werror=c++17-extensions]
 9850 |     for (auto &[pk_str, score] : doc_scores)
      |                ^
cc1plus: all warnings being treated as errors
make[2]: *** [storage/tidesdb/CMakeFiles/tidesdb.dir/build.make:79: storage/tidesdb/CMakeFiles/tidesdb.dir/ha_tidesdb.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:10332: storage/tidesdb/CMakeFiles/tidesdb.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

Having a look as I'm gonna guess we need to pass that wf.

guycipher · 2026-06-02T21:42:37Z

C++11 is base I am going to assume, 13 seems to be C++17. I'll keep the engine at bay with C++11 as the library aligns with C11.

Build fixes for MariaDB 11.4-11.7 (C++11; buildbot builds with -Werror): - Replace C++17 structured bindings with C++11 iterator access in the FTS write_row/update_row/delete_row and ft_init_ext paths. - Guard TABLE::part_info with WITH_PARTITION_STORAGE_ENGINE, which only defines that member when partitioning is compiled in. Initialization hardening (PR review feedback): - Value-initialize TidesDB_share encryption members (encrypted, encryption_key_id, encryption_key_version) in the constructor. - Mark ~ha_tidesdb() override.

guycipher · 2026-06-02T22:41:16Z

Something to have a look at: https://buildbot.mariadb.org/#/builders/554/builds/22266 and https://buildbot.mariadb.org/#/builders/554/builds/22266/steps/3/logs/stdio not sure if CI or something I need to touch on at plugin level.

guycipher · 2026-06-02T23:48:54Z

Caught it, had to create mariadb-plugin-tidesdb.install

guycipher · 2026-06-03T01:21:42Z

https://buildbot.mariadb.org/#/builders/534/builds/39281/steps/6/logs/stdio buildbot/amd64-ubuntu-2204-debug-ps does not seem to be due to TidesDB implementation. v9.3.3 of TidesDB and v4.5.4 of TideSQL plugin implemented. Ready for review.

dr-m · 2026-06-03T06:25:13Z

https://buildbot.mariadb.org/#/builders/534/builds/39281/steps/6/logs/stdio buildbot/amd64-ubuntu-2204-debug-ps does not seem to be due to TidesDB implementation.

Right, there is an open ticket MDEV-38195 about failures of the test events.events_restart.

gkodinov

Thank you for your contribution! This is a preliminary review.

I'm approving it as it is, as there're no major violations. Or, if there are any, they're in the competency of the final reviewer. Nevertheless, I'll mention what I see that can be improved.

First of all: please consider merging some of the commits. Ideally there should be one commit per distinct "feature". I'd guess that makes 1 in your case.

Next: this technically is a new feature. As such, I'd suggest considering rebasing to the main branch. Unless of course, there is some special consideration why the targed should be as is.

Finally, there should be a Jira for house-keeping. I'll start one for you, using the PR explanation. But this needs to be improved significantly I'd guess, until it evolves into a proper design specification that would help with the final review.

There's also one small thing I've noticed below.

Please stand by for the final review.

gkodinov · 2026-06-03T08:53:27Z

I do not think you need this file.

Thank you @gkodinov understood for 1. For 2, to be able to propagate forward to newer branches selectively or not and main with a floor of support for 11.4. For 3 understood, I appreciate the help with that and I can work off it from there.

I do not think you need this file.

You're right it's a fossil.

Thank you @gkodinov understood for 1. For 2, to be able to propagate forward to newer branches selectively or not and main with a floor of support for 11.4. For 3 understood, I appreciate the help with that and I can work off it from there.

Have you tried just plaining atop of the main branch and re-compiling. You're not changing anything in the server itself. So it might work just as well. You could maybe need to alter some of the calls back to the server, but I do not think it has changed that much. Then it's a matter of running the regression tests successfully (and debugging any failures).
I (and the final reviewer) can guide you through that.

All conditionals are handled from 11.4 to 13.02, though verified sparingly there is a full list. Main sounds ok, if condenses is main, we can do that.

…YSQL_ADD_PLUGIN)

gemini-code-assist Bot reviewed Jun 2, 2026

View reviewed changes

Comment thread storage/tidesdb/CMakeLists.txt

Comment thread storage/tidesdb/ha_tidesdb.h

Comment thread storage/tidesdb/ha_tidesdb.h

Comment thread storage/tidesdb/ha_tidesdb.h

Comment thread storage/tidesdb/ha_tidesdb.h

Comment thread storage/tidesdb/ha_tidesdb.h Outdated

Debian package the TidesDB plugin (mariadb-plugin-tidesdb)

44923d5

gkodinov added the External Contribution All PRs from entities outside of MariaDB Foundation, Corporation, Codership agreements. label Jun 3, 2026

gkodinov self-assigned this Jun 3, 2026

gkodinov approved these changes Jun 3, 2026

View reviewed changes

gkodinov changed the title ~~Adds TidesDB 9(v9.3.3) storage engine~~ MDEV-39834: Add TidesDB 9(v9.3.3) storage engine Jun 3, 2026

gkodinov requested a review from sanja-byelkin June 3, 2026 09:08

gkodinov assigned sanja-byelkin and unassigned gkodinov Jun 3, 2026

for tidesdb engine drop legacy plug.in (unused; superseded by CMAKE M…

5208b0c

…YSQL_ADD_PLUGIN)

Uh oh!

Conversation

guycipher commented Jun 2, 2026

Uh oh!

CLAassistant commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

guycipher commented Jun 2, 2026

Uh oh!

guycipher commented Jun 2, 2026

Uh oh!

guycipher commented Jun 2, 2026

Uh oh!

guycipher commented Jun 2, 2026

Uh oh!

guycipher commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

guycipher commented Jun 2, 2026

Uh oh!

guycipher commented Jun 3, 2026

Uh oh!

dr-m commented Jun 3, 2026

Uh oh!

gkodinov left a comment

Choose a reason for hiding this comment

Uh oh!

gkodinov Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

guycipher Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

guycipher Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

gkodinov Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

guycipher Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

5 participants

CLAassistant commented Jun 2, 2026 •

edited

Loading

guycipher commented Jun 2, 2026 •

edited

Loading

guycipher Jun 4, 2026 •

edited

Loading