Skip to content

BLD: add option to build future-compatible abi3 wheels - #1600

Draft
neutrinoceros wants to merge 2 commits into
pyproj4:mainfrom
neutrinoceros:bld/abi3
Draft

neutrinoceros wants to merge 2 commits into
pyproj4:mainfrom
neutrinoceros:bld/abi3

Conversation

@neutrinoceros

Copy link
Copy Markdown
Contributor

This allows targeting CPython's LimitedAPI (and stable ABI) by setting PYPROJ_LIMITED_API=1 at build time. Resulting wheels are future compatible with yet-unreleased Python versions.
The trade-off is that runtime performance might be impacted, though I suggest measuring it against real load-bearing applications, as I found that in many numpy-based extensions the impact to be negligible or even non-measurable.
I could successfully build the package and run the test suite with this locally.

I suggest a couple possible approaches to leverage this:

  • append a cp314-abi3 target to the existing build matrix. Version-specific wheels will still be prefered by installers when available but will default to these otherwise, instead of having to, and often failing to build pyproj from source.
  • replace version-specific wheels with a single cp311-abi3 wheel per platform. If there are no visible performance regressions this is definitely the way to go.

Comment thread setup.py

# restrict LIMITED_API usage:
# - require an env var PYPROJ_LIMITED_API=1
# - LIMITED_API is not compatible with free-threading (as of CPython 3.14)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note this comment is going to age like milk because there's a new version of the Limited API coming up in Python 3.15 that's compatible with both GIL-enabled and free-threaded builds. However there are a couple other blockers right now (Cython and setuptools both have work-in-progress patches to support this). Note that I'd be happy to revisit this later this year to add cp315-abi3.abi3t as a target

Comment thread setup.py Outdated
Comment thread setup.py
@snowman2 snowman2 added this to the 4.0.0 milestone Oct 10, 2026
@codecov

codecov Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.81%. Comparing base (5900c5a) to head (0482753).
⚠️ Report is 13 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1600      +/-   ##
==========================================
- Coverage   95.84%   95.81%   -0.04%     
==========================================
  Files          20       20              
  Lines        1879     1886       +7     
==========================================
+ Hits         1801     1807       +6     
- Misses         78       79       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@snowman2

Copy link
Copy Markdown
Member

Mind enabling this in the wheel builds to test?

@neutrinoceros

Copy link
Copy Markdown
Contributor Author

enabling this in CI revealed that it is currently blocked by cython

pyproj/_geod.c:1412:6: This module uses CPython specific internals of 'array.array', which are not available in PyPy or the limited API

and indeed, compilation then predictibly fails because the generated module isn't limited-api-compliant

@snowman2

Copy link
Copy Markdown
Member

You could test it by adding a flag to use the slower alternative:

IF CTE_PYTHON_IMPLEMENTATION == "CPython":

@neutrinoceros
neutrinoceros force-pushed the bld/abi3 branch 3 times, most recently from 8721db0 to 0ef9dbf Compare October 13, 2026 19:47
@neutrinoceros

Copy link
Copy Markdown
Contributor Author

thanks for the pointer. I now expect the current CI run to succeed. Should I drop the second commit before undrafting ?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants