I used spark 3.3.0, which includes netlib v2.2.1.
When I ran spark on jdk17, i got a warning from yarn logs WARN InstanceBuilder$JavaBLAS: Failed to load implementation from:dev.ludovic.netlib.blas.VectorBLAS.
I debug and found that the throwable message is java.lang.NoClassDefFoundError: jdk/incubator/vector/Vector, after i added jvm options--add-modules=jdk.incubator.vector to spark, this warning disappered.
|
log.warning("Failed to load implementation from:" + VectorBLAS.class.getName()); |
I think this warning messages can be written in more detail.
I used spark 3.3.0, which includes netlib v2.2.1.
When I ran spark on jdk17, i got a warning from yarn logs
WARN InstanceBuilder$JavaBLAS: Failed to load implementation from:dev.ludovic.netlib.blas.VectorBLAS.I debug and found that the throwable message is
java.lang.NoClassDefFoundError: jdk/incubator/vector/Vector, after i added jvm options--add-modules=jdk.incubator.vectorto spark, this warning disappered.netlib/blas/src/main/java/dev/ludovic/netlib/blas/InstanceBuilder.java
Line 72 in c63517f
I think this warning messages can be written in more detail.