Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
492a4f6
Add OWN_GIL mode for true parallel Python execution
benoitc Mar 14, 2026
0851392
Add OWN_GIL features test suite with reactor dispatch support
benoitc Mar 15, 2026
f6853a6
Register py_event_loop module in OWN_GIL subinterpreters
benoitc Mar 15, 2026
68edf93
Add process-local environments for OWN_GIL contexts
benoitc Mar 15, 2026
f57f30b
Add OWN_GIL internals documentation
benoitc Mar 15, 2026
81c871a
Document reactor/event loop integration with OWN_GIL
benoitc Mar 15, 2026
fd2008e
Add owngil to context_create type spec
benoitc Mar 15, 2026
a79b522
Fix async_callback for subinterpreter compatibility
benoitc Mar 15, 2026
fba1ae8
Enable asyncio compat tests for subinterpreters
benoitc Mar 15, 2026
d65b301
Fix asyncio compat tests for Python 3.12+
benoitc Mar 15, 2026
1394a35
Fix event loop thread-local context in process_ready_tasks
benoitc Mar 15, 2026
971225c
Add thread-local event loop context test
benoitc Mar 15, 2026
06c986d
Add per-process namespace for event loop API
benoitc Mar 15, 2026
854444c
Add thread-local namespace for reentrant calls
benoitc Mar 15, 2026
25a820f
Fix test_task_factory for Python 3.11 compatibility
benoitc Mar 15, 2026
6650811
Fix eager task execution in Python 3.12+
benoitc Mar 15, 2026
f6e4362
Fix test_task_factory for Python 3.11 compatibility
benoitc Mar 15, 2026
6fa98db
Fix deprecation warning: use erlang.run() instead of erlang.install()
benoitc Mar 15, 2026
cd082db
Fix thread-safety race condition in handle pool
benoitc Mar 15, 2026
8f69641
Update macOS CI to version 15 (Sequoia)
benoitc Mar 15, 2026
c17197f
Use rm -rf _build for clean CI builds
benoitc Mar 15, 2026
f52aba5
Use verbose test output for debugging CI failures
benoitc Mar 15, 2026
8f5a15c
Add verbose output for failing asyncio test
benoitc Mar 15, 2026
3b2fa26
Increase OWN_GIL init timeout to 5s and add error logging
benoitc Mar 15, 2026
380228f
Add stderr output for test failures to improve CI visibility
benoitc Mar 15, 2026
d8fb49a
Add init_error flag for fast OWN_GIL init failure detection
benoitc Mar 15, 2026
7cfd82c
Fix race conditions, deadlocks, and add debug logging
benoitc Mar 15, 2026
1a98fa7
Fix flaky test_time assertion for CI timing variance
benoitc Mar 15, 2026
a3ad3e5
Add Py_NewRef compatibility for Python < 3.10
benoitc Mar 15, 2026
9c5435c
Increase timing tolerance in test_sleep_async_concurrent for CI
benoitc Mar 15, 2026
e6126ee
Fix OWN_GIL safety issues: mutex leak, ABBA deadlock, dangling env
benoitc Mar 15, 2026
7537c69
Document OWN_GIL safety mechanisms and lock ordering
benoitc Mar 15, 2026
7921bbe
Document OWN_GIL mode features and usage
benoitc Mar 15, 2026
3dcec6f
Document OWN_GIL and event loop per-process environments
benoitc Mar 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,10 @@ jobs:
otp: "27.0"
python: "3.13"
# macOS
- os: macos-14
- os: macos-15
otp: "27"
python: "3.12"
- os: macos-14
- os: macos-15
otp: "27"
python: "3.13"

Expand Down Expand Up @@ -69,7 +69,7 @@ jobs:

- name: Clean and compile
run: |
rebar3 clean
rm -rf _build
rebar3 compile

- name: Run tests
Expand Down Expand Up @@ -162,7 +162,7 @@ jobs:
env:
PYTHON_GIL: "0"
run: |
rebar3 clean
rm -rf _build
rebar3 compile

- name: Run tests
Expand Down Expand Up @@ -223,7 +223,7 @@ jobs:

- name: Clean and compile with ASan
run: |
rm -rf _build/cmake
rm -rf _build
mkdir -p _build/cmake
cd _build/cmake
cmake ../../c_src -DENABLE_ASAN=ON -DENABLE_UBSAN=ON
Expand Down
53 changes: 53 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,21 @@

## 2.2.0 (unreleased)

### Fixed

- **OWN_GIL Safety Fixes** - Critical fixes for OWN_GIL subinterpreter mode
- **Mutex leak in erlang module** - `async_futures_mutex` now always destroyed in
`erlang_module_free()` regardless of `pipe_initialized` flag
- **ABBA deadlock prevention** - Fixed lock ordering in `event_loop_down()` and
`event_loop_destructor()` to acquire GIL before `namespaces_mutex`, matching the
normal execution path and preventing deadlocks
- **Dangling env pointer detection** - Added `interp_id` validation in
`owngil_execute_*_with_env()` functions to detect and reject env resources
created by a different interpreter, returning `{error, env_wrong_interpreter}`
- **OWN_GIL callback documentation** - Documented that `erlang.call()` from OWN_GIL
contexts uses `thread_worker_call()` rather than suspension/resume protocol;
re-entrant calls to the same OWN_GIL context are not supported

### Added

- **PyBuffer API** - Zero-copy WSGI input buffer for streaming HTTP bodies
Expand Down Expand Up @@ -52,6 +67,44 @@
- `examples/bench_async_task.erl` - Erlang benchmark runner
- `priv/test_async_task.py` - Python async task implementation

- **OWN_GIL Context Mode** - True parallel Python execution (Python 3.12+)
- `py_context:start_link(Id, owngil)` - Create context with dedicated pthread and GIL
- Each OWN_GIL context runs in its own thread with independent Python GIL
- Enables true CPU parallelism across multiple Python contexts
- Full feature support: channels, buffers, callbacks, PIDs, reactor, async tasks
- `py_context:get_nif_ref/1` - Get NIF reference for low-level operations
- New benchmark: `examples/bench_owngil.erl` comparing SHARED_GIL vs OWN_GIL
- See [OWN_GIL Internals](docs/owngil_internals.md) for architecture details

- **Process-Local Environments for OWN_GIL** - Namespace isolation within shared contexts
- `py_context:create_local_env/1` - Create isolated Python namespace for calling process
- `py_nif:context_exec(Ref, Code, Env)` - Execute with process-local environment
- `py_nif:context_eval(Ref, Expr, Locals, Env)` - Evaluate with process-local environment
- `py_nif:context_call(Ref, Mod, Func, Args, Kwargs, Env)` - Call with process-local environment
- Multiple Erlang processes can share an OWN_GIL context with isolated namespaces
- Interpreter ID validation prevents cross-interpreter env usage

- **Per-Process Event Loop Namespaces** - Process isolation for event loop API
- `py_nif:event_loop_exec/2` - Execute code in calling process's namespace
- `py_nif:event_loop_eval/2` - Evaluate expression in calling process's namespace
- Functions defined via exec callable via `create_task` with `__main__` module
- Automatic cleanup when Erlang process exits

- **OWN_GIL Test Suites** - Feature verification
- `py_context_owngil_SUITE` - Core OWN_GIL functionality (15 tests)
- `py_owngil_features_SUITE` - Feature integration (44 tests covering channels,
buffers, callbacks, PIDs, reactor, async tasks, asyncio, local envs)

### Changed

- **Event Loop Lock Ordering** - GIL acquired before `namespaces_mutex` in cleanup paths
to prevent ABBA deadlocks with normal execution path

- **Asyncio Compatibility** - Fixed for Python 3.12+ with subinterpreters
- Thread-local event loop context in `process_ready_tasks`
- Eager task execution handling for Python 3.12+
- Deprecation warning fix: use `erlang.run()` instead of `erlang.install()`

## 2.1.0 (2026-03-12)

### Added
Expand Down
Loading
Loading