Repository navigation
Intel Math Kernel Library (MKL) #116
Description
Activity
Intels MKL library implement the LAPACK interfaces - so it would be logical to think about a layer with different LAPACK providers - in other words - a strategy pattern. Let users decide which LAPACK provider they want to use.
It will be not much work since LAPACK usual means --> all interfaces are the same. So the PInvoke class would be the same - just the native lib is different.
As fdncred already said. :)
Sounds great.
@fdncred sorry yes i know but I want to mention it here at this place ;)
@dotChris90 Please add MKL as a new provider when you have a chance.
yes. but after seeing Intel website I think we go strategy that mkl provider expects the lib installed. on their site they said u have to registry etc..... So.... I don't wanna quarrel with them. in case we simple copy their DLLs and publish them with NumSharp.... think Intel will be not happy.
so installation must be done manually by user.
At moment not sure what is most comfortable way to change providers.
My suggestion is that static numpy class has a static property lapack provider which is an enum.
The static lapack class will check this enum and will call the specific static lapack provider classes.
The user can change provider global. I think most comfortable.
@dotChris90, I'll research this a bit because other people, namely microsoft, point to dlls such as this https://docs.microsoft.com/en-us/cognitive-toolkit/setup-mkl-on-windows.
Now that I look closer, this one ls mklml and is opensource by intel. Perhaps that is different.
Check this out. Intel allows one to redistribute MKL. It's in the license.
https://software.intel.com/en-us/mkl/license-faqAnd here is the full license.
https://software.intel.com/en-us/license/intel-simplified-software-licenseHere's a few Github projects that wrap MKL in C#. May be easier just to use something that already exists.
https://github.com/DNRY/CSIntelPerfLibs
https://github.com/Rafka86/SharpMKL
https://github.com/Proxem/BlasNet@fdncred much thanks for investigation. This makes the situation much better. 👍
24 remaining items
@fdncred definitely. I am just highly confused about the mkl libraries out there.
is anaconda mkl the same like normal mkl and why there is no package for windows... the apt get installer also just download the file and put at specific locations...
drives me crazy....
@dotChris90 it appears to me that anaconda's MKL is just the regular MKL. It also looks to me like there is a windows package. See the links below.
https://anaconda.org/anaconda/mkl
https://docs.anaconda.com/accelerate/mkl-overview/
https://repo.anaconda.com/pkgs/main/win-64/
For my two cents, I think Proxem's BlasNet is the best wrapper for MKL. It looks pretty mature.
https://github.com/Proxem/BlasNetReacted by TechGuy@fdncred Thanks for the update. ;)
From the URL I agree with you. 👍
I already downloaded the package via pip and checked it but the package brings multiple MKL dlls. In anaconda location you will find binaries with mkl_corem mkl_xyz, …. about 6 or 7 files. But I though the mkl thing is one single DLL …. but anaconda shows multiple .... this is extrem strange behaviour...I agree Proxems BlasNet looks fine but it does not solve our main problem. Proxem just brings Wrapper code in C# - please do not misunderstand me but I am not very impressed by this …. because there are a lot of C# Code-Generators who do the same job - e.g. ClangSharp, T4, SWIG and if I check the Mono team I am 100% sure I can find more generators. xD Such automatic wrapper always simple take a C header file and generate Code …. I really do not see any benefit to use it BlasNet. I saw the Code and as far as I can see the main methods we can use could be also generated automaticly by ClangSharp (or T4) so not sure why add package for something we can auto generate xD.
Our (at least what makes me feel uncomfortable) problem is : Proxem does not bring a native MKL Lib package. I have to install it manually. In Linux its easy - "sudo apt-get install" but on windows it is impossible to do this. This makes CI Testing impossible. How do we want to do unit testing or automatic benchmark testing on Appveyor? Unfortunately -.- we can not do this - except we package this MKL DLL into a nuget package. damn windows has no apt get....
Probably not impossible for CI. Have you looked at using chocolatey to install MKL? It looks like one of microsoft's packages includes it https://chocolatey.org/packages/microsoft-r-open. It may not be perfect but it may work.
I've had a lot of experience with ClangSharp and NClang as well as other generators to take a C/C++ library to C# and I'm quite sure that they are not easy. I've found with one for months trying to get the library ported and it's very time consuming. You can get close pretty easily but you can't get complete easily. It'll take work for sure.
Reacted by Haymo Kutschbachdamn choco package management didn't think on it yes. can try. thanks.
If u say it is hard maybe I was too easy thinking. the only complex thing we could have with code generation is.... lapack 100 % uses pointer. so generator can not know what's array and what pass by reference.
hm...yes could be trickier. but as long as we just using some few functions from lapack.... writing own is easier. but yes maybe later need some more professional things.
Yes, IntPtr from C/C++ can be tricky and requires hand massaging for every call, from my experience. Marshalling arrays isn't too complex if you know the count of the array and the size of the array. Ref, In, Out params can be tricky, especially if you have an [In, Out] parameter in C/C++, where you pass in out value in a parameter and you return a different value out of the same parameter. If you get stuck on something reach out to me and I may be able to help.
@fdncred can you help with
SciSharp/TensorFlow.NET#86?@Oceania2018, yes, i'll look at it. I'm not promising i can fix it but I'll try. ;)
Anyway I’d appreciate. Try to make it work. Enjoy tensorflow.net.
for traceability. Intel just offer you help if u have customer service.. yeah every company need money I know....
But post in forum and got answer direct from Intel.
https://software.intel.com/en-us/comment/1932183#comment-1932183
as we can read there. don't worry take the mkl lib.
answers was (2019 Jan 10th, 7:09 Berlin time)
please refer here : license FAQ
you won't have a problem redistributing it.
Reacted by TechGuyHi folks,
I saw on the Intel forum that you were asking about distributing MKL via nuget.
It is available now in nuget channel as well. I was just wondering, if this may be useful to you: https://www.nuget.org/packages?q=intelmklBest regards,
Maria@Oceania2018, we shipped this as OpenBLAS rather than MKL - the pluggable provider you described. Referencing the package assigns
TensorEngine.Blasfrom a[ModuleInitializer], so the reference is the whole opt-in (0.70.0, #628):// dotnet add package NumSharp.Interop.OpenBLAS OpenBlasEngine.Enable(); Console.WriteLine(OpenBlasEngine.Enabled); // True // products / factorisations now route through OpenBLAS, byte-identical to NumPy var c = np.matmul(a, b);
Requires NumSharp.Interop.OpenBLAS (
dotnet add package NumSharp.Interop.OpenBLAS). An MKL backend could implement the sameIBlasBackendseam if anyone wants it. Closing.Keeping this open for a day we will extend to NumSharp.Interop.IntelMKL
Performance-sensitive algorithms can be swapped with alternative implementations by the concept of providers like Intel MKL.
https://software.intel.com/en-us/performance-libraries