Python wheel should export DuckDB C API symbols (or provide a path to them)

[nelson2005]

What happens?

Prior to 1.4.1, the DuckDB Python wheel exported all C API symbols on every platform. Starting with 1.4.1, duckdb-python#81 restricted exports to only PyInit__duckdb and duckdb_adbc_init via linker flags in CMakeLists.txt L83-L110.

On macOS, -exported_symbol is a hard allowlist — all C API symbols are gone. On Linux, --export-dynamic-symbol is additive, so all symbols remain visible as a side effect. This means the C API works on Linux today but is broken on macOS — and could break on Linux too if a version script is added in the future.

Version	macOS C API symbols	Linux C API symbols
1.3.x	~6,300 ✅	~6,300 ✅
1.4.0	~6,300 ✅	~5,900 ✅
1.4.1+	0 ❌	~5,900 ✅ (by accident)
1.5.x	0 ❌	~6,300 ✅ (by accident)

This is a breaking change for projects that load the C API from the Python wheel.

Use case

numbduck wraps 160 DuckDB C API functions for use inside numba @njit compiled code. It loads the shared library from the installed duckdb Python package and resolves C API symbols at runtime. This worked on all platforms through duckdb 1.4.0, but breaks on macOS starting with 1.4.1.

The standalone libduckdb downloads have all symbols, but requiring a separate install (brew install duckdb) alongside pip install duckdb is a friction point — users expect the Python package to be self-contained.

Possible solutions

1. Export all duckdb_* C API symbols from the Python wheel on all platforms. The symbols are already compiled into the binary — they're just hidden by the linker flags.
2. Add a ctypes-friendly entry point — e.g. a function that returns a pointer to a struct of C API function pointers (similar to how duckdb_adbc_init works for ADBC).
3. Bundle libduckdb.dylib/.so alongside the Python extension in the wheel, so ctypes.CDLL can load it directly.
4. Document the limitation — if the C API is intentionally not part of the Python wheel's public interface, document that C API consumers should use the standalone libduckdb downloads.

Option 1 is the smallest change — just remove the linker flags. The PR #81 commit message says these are "the only two [symbols] that are ever needed," but that assumes all consumers go through Python or ADBC.

Environment

• duckdb 1.4.1 through 1.5.1 (macOS broken; Linux works by accident)
• macOS ARM64 and x86_64
• Verified with nm -gD / nm -g on installed wheel .so files

[nelson2005]

Update: we shipped a workaround in Goykhman/numbduck#17 — find_duckdb_shared_lib() now falls back to a standalone libduckdb.dylib (Homebrew or manual download) on macOS when the Python wheel lacks C API symbols. This gets macOS users unblocked but adds an extra install step (brew install duckdb) that isn't needed on any other platform.

The underlying issue remains: the C API symbols are compiled into the wheel binary on all platforms — they're just hidden by linker flags on macOS. Any project loading DuckDB's C API from the Python package hits this on macOS, and Linux is one version script away from the same breakage.

[evertlammerts]

Hey @nelson2005, thanks for the heads up. I tried to clean up the exported symbols last year because from the two symbols that we knew had to be exported, only one actually was.

I'll put this on my list and try to get this in before 1.5.3.

[nelson2005]

Thanks, much appreciated!

[nelson2005]

@evertlammerts do you think there'd be any interest in an article about jitted duckdb? It's a new capability, me and goykhman worked on it because we do a lot of numerical work using numba and there was no way to call duckdb from jitted code.
It's a little specific, but this technique permits calling to/from duckdb without involving the Python interpreter and the GIL, creating some significant benefits in various dimensions over plain-python and pyarrow alternative implementations. I hadn't seen anyone implementing this technique for duckdb, though it's quite possible I missed it