The Intel MKL pkg-config files did not work, because they expect that
the MKLROOT environment variable is set. This change replaces
occurences by the actual path of MKL in the Nix store.
Since the pkg-config files seem to break quite frequently after
upgrades, add a post-install check to validate the pkg-config files.
This is based on previous work for switching between BLAS and LAPACK
implementation in Debian[1] and Gentoo[2]. The goal is to have one way
to depend on the BLAS/LAPACK libraries that all packages must use. The
attrs “blas” and “lapack” are used to represent a wrapped BLAS/LAPACK
provider. Derivations that don’t care how BLAS and LAPACK are
implemented can just use blas and lapack directly. If you do care what
you get (perhaps for some CPP), you should verify that blas and lapack
match what you expect with an assertion.
The “blas” package collides with the old “blas” reference
implementation. This has been renamed to “blas-reference”. In
addition, “lapack-reference” is also included, corresponding to
“liblapack” from Netlib.org.
Currently, there are 3 providers of the BLAS and LAPACK interfaces:
- lapack-reference: the BLAS/LAPACK implementation maintained by netlib.org
- OpenBLAS: an optimized version of BLAS and LAPACK
- MKL: Intel’s unfree but highly optimized BLAS/LAPACK implementation
By default, the above implementations all use the “LP64” BLAS and
LAPACK ABI. This corresponds to “openblasCompat” and is the safest way
to use BLAS/LAPACK. You may received some benefits from “ILP64” or
8-byte integer BLAS at the expense of breaking compatibility with some
packages.
This can be switched at build time with an override like:
import <nixpkgs> {
config.allowUnfree = true;
overlays = [(self: super: {
lapack = super.lapack.override {
lapackProvider = super.lapack-reference;
};
blas = super.blas.override {
blasProvider = super.lapack-reference;
};
})];
}
or, switched at runtime via LD_LIBRARY_PATH like:
$ LD_LIBRARY_PATH=$(nix-build -E '(with import <nixpkgs> {}).lapack.override { lapackProvider = pkgs.mkl; is64bit = true; })')/lib:$(nix-build -E '(with import <nixpkgs> {}).blas.override { blasProvider = pkgs.mkl; is64bit = true; })')/lib ./your-blas-linked-binary
By default, we use OpenBLAS LP64 also known in Nixpkgs as
openblasCompat.
[1]: https://wiki.debian.org/DebianScience/LinearAlgebraLibraries
[2]: https://wiki.gentoo.org/wiki/Blas-lapack-switch
Since Intel's default openmp implementation is available in the same src
tarball, we can just include it in the package. This means that `mkl` now "just
works" without any environment variables, fragile setup-hooks, or forced
propagation.
Since the openmp implementation is only needed at runtime (and for test cases),
users can substitute a different one if they prefer by exporting it with
`LD_PRELOAD`, which is how Intel recommends handling this. If they do not do so,
`libiomp.so` lives next to `libmkl_rt.so` and thus will be in the RPATH as a
sane default.
Since this still comes from the same src tarball, we can ship it without losing
the fixed-output derivation; likewise, since Hydra is not building or caching
these, shipping these proprietary packages costs no bandwidth for the nix
community.
This packags the Intel Math Kernel library on x86-64 platforms, which is a
dependency for many data science and machine learning packages.
Upstream, Intel provides proprietary binary RPMs with a permissive
redistribution license. These have been repackaged in both Debian and Anaconda,
so we are not the first distribution to redistribute.
