Skip to content

Fix IEEE 754 compliance for real number operations#29

Merged
polvalente merged 1 commit intoelixir-nx:mainfrom
blasphemetheus:fix/ieee754-compliance
Mar 24, 2026
Merged

Fix IEEE 754 compliance for real number operations#29
polvalente merged 1 commit intoelixir-nx:mainfrom
blasphemetheus:fix/ieee754-compliance

Conversation

@blasphemetheus
Copy link
Contributor

Erlang's :math module raises ArithmeticError for overflow and domain errors instead of returning IEEE 754 special values. This causes downstream libraries (like Nx: elixir-nx/nx#1707) to crash on valid inputs.

Fixes:

  • exp(large): returns :infinity instead of crashing
  • sinh(large): returns :infinity/:neg_infinity based on sign
  • cosh(large): returns :infinity
  • asin/acos outside [-1,1]: returns :nan instead of crashing
  • acosh below 1: returns :nan
  • atanh outside (-1,1): returns :nan
  • atanh(1): returns :infinity, atanh(-1): returns :neg_infinity
  • sqrt(negative): returns :nan instead of crashing
  • pow(0, negative): returns :infinity instead of crashing
  • divide(x, 0.0): returns :infinity/:neg_infinity/:nan per IEEE 754
  • divide(x, -0.0): respects negative zero sign (OTP 27+ matching)
  • cot(0): returns :infinity instead of crashing (1/tan(0) = 1/0)
  • acot(0): returns pi/2 instead of crashing
  • acsc(0): returns :infinity instead of crashing
  • acsc(values < 1): returns :nan for domain errors

34 new tests covering all IEEE 754 edge cases.
All 169 tests pass (99 doctests + 70 tests).

Erlang's :math module raises ArithmeticError for overflow and domain
errors instead of returning IEEE 754 special values. This causes
downstream libraries (like Nx) to crash on valid inputs.

Fixes:

- exp(large): returns :infinity instead of crashing
- sinh(large): returns :infinity/:neg_infinity based on sign
- cosh(large): returns :infinity
- asin/acos outside [-1,1]: returns :nan instead of crashing
- acosh below 1: returns :nan
- atanh outside (-1,1): returns :nan
- atanh(1): returns :infinity, atanh(-1): returns :neg_infinity
- sqrt(negative): returns :nan instead of crashing
- pow(0, negative): returns :infinity instead of crashing
- divide(x, 0.0): returns :infinity/:neg_infinity/:nan per IEEE 754
- divide(x, -0.0): respects negative zero sign (OTP 27+ matching)
- cot(0): returns :infinity instead of crashing (1/tan(0) = 1/0)
- acot(0): returns pi/2 instead of crashing
- acsc(0): returns :infinity instead of crashing
- acsc(values < 1): returns :nan for domain errors

34 new tests covering all IEEE 754 edge cases.
All 169 tests pass (99 doctests + 70 tests).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
blasphemetheus added a commit to blasphemetheus/nx that referenced this pull request Mar 19, 2026
…ter f64

Four independent fixes:

1. window_scatter_max/min on f64: binary size mismatch. Fixed by
   casting scatter result to output type before to_binary.

2. Nx.slice on scalar tensor: bin_slice crashed calling hd([]) on
   empty strides list. Added scalar guard clause.

3. Nx.linspace with n=1: divided by zero computing step size.
   Special-cased to return start value directly.

4. Nx.gather with scalar indices: gave unhelpful Erlang error.
   Moved shape check before indexed_axes call.

IEEE 754 overflow/domain/divzero tests are skipped pending upstream
fix in the Complex library (elixir-nx/complex#29).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@polvalente polvalente merged commit ab2149e into elixir-nx:main Mar 24, 2026
2 checks passed
@polvalente
Copy link
Contributor

@blasphemetheus we can point Nx at the github version of complex for now, and then we can release both when next Nx release is ready to go out

@blasphemetheus blasphemetheus deleted the fix/ieee754-compliance branch March 24, 2026 16:13
blasphemetheus added a commit to blasphemetheus/nx that referenced this pull request Mar 24, 2026
Not for merging — just for verifying that the Complex IEEE 754 fixes
(elixir-nx/complex#29) resolve all the ArithmeticError crashes in Nx
(pinv, least_squares, etc.).

Full Nx test suite: 0 failures.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants