This is based on previous work for switching between BLAS and LAPACK
implementation in Debian[1] and Gentoo[2]. The goal is to have one way
to depend on the BLAS/LAPACK libraries that all packages must use. The
attrs “blas” and “lapack” are used to represent a wrapped BLAS/LAPACK
provider. Derivations that don’t care how BLAS and LAPACK are
implemented can just use blas and lapack directly. If you do care what
you get (perhaps for some CPP), you should verify that blas and lapack
match what you expect with an assertion.
The “blas” package collides with the old “blas” reference
implementation. This has been renamed to “blas-reference”. In
addition, “lapack-reference” is also included, corresponding to
“liblapack” from Netlib.org.
Currently, there are 3 providers of the BLAS and LAPACK interfaces:
- lapack-reference: the BLAS/LAPACK implementation maintained by netlib.org
- OpenBLAS: an optimized version of BLAS and LAPACK
- MKL: Intel’s unfree but highly optimized BLAS/LAPACK implementation
By default, the above implementations all use the “LP64” BLAS and
LAPACK ABI. This corresponds to “openblasCompat” and is the safest way
to use BLAS/LAPACK. You may received some benefits from “ILP64” or
8-byte integer BLAS at the expense of breaking compatibility with some
packages.
This can be switched at build time with an override like:
import <nixpkgs> {
config.allowUnfree = true;
overlays = [(self: super: {
lapack = super.lapack.override {
lapackProvider = super.lapack-reference;
};
blas = super.blas.override {
blasProvider = super.lapack-reference;
};
})];
}
or, switched at runtime via LD_LIBRARY_PATH like:
$ LD_LIBRARY_PATH=$(nix-build -E '(with import <nixpkgs> {}).lapack.override { lapackProvider = pkgs.mkl; is64bit = true; })')/lib:$(nix-build -E '(with import <nixpkgs> {}).blas.override { blasProvider = pkgs.mkl; is64bit = true; })')/lib ./your-blas-linked-binary
By default, we use OpenBLAS LP64 also known in Nixpkgs as
openblasCompat.
[1]: https://wiki.debian.org/DebianScience/LinearAlgebraLibraries
[2]: https://wiki.gentoo.org/wiki/Blas-lapack-switch
My earlier change mistakenly expected `toString false` to produce '0'
instead of the empty string, leading to unexpected config changes.
Intended to address issue mentioned here and in following discussion:
https://github.com/NixOS/nixpkgs/pull/53972#issuecomment-459981602
Sorry, folks!
(special-case handling of bools here makes this "cleanup" a bit
less of an obvious win but hopefully still preferable overall :))
-----------
makeFlags in resulting derivation, according to this one-liner:
$ nix show-derivation -f . openblas|jq ".[].env.makeFlags"
before:
"BINARY=64 CC=cc CROSS= DYNAMIC_ARCH=1 FC=gfortran HOSTCC=cc INTERFACE64=1 NO_BINARY_MODE= NO_STATIC=1 NUM_THREADS=64 PREFIX=/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9 TARGET=ATHLON USE_OPENMP=1"
after:
"BINARY=64 CC=cc CROSS=0 DYNAMIC_ARCH=1 FC=gfortran HOSTCC=cc INTERFACE64=1 NO_BINARY_MODE=0 NO_STATIC=1 NUM_THREADS=64 PREFIX=/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9 TARGET=ATHLON USE_OPENMP=1"
Without knowing how `placeholder` works, it seems interesting if
entirely unrelated that the `PREFIX` is same for both! :). TIL.
This reverse changes made cfc4313 which broke builds on linux. Still
not sure why this broke Linux, but we can investigate it later.
“buildPackages.stdenv.cc” shouldn’t be put in nativeBuildInputs in
general. Either way, this restores hashes to before that commit.
thanks to @jethrokuan
The openblas script relies on CC=clang for succesfully compiling on
Darwin systems. This seems like bad behavior, but we get things like
https://hydra.nixos.org/build/87657058 otherwise.
Fixes#55536.
Comments on conflicts:
- llvm: d6f401e1 vs. 469ecc70 - docs for 6 and 7 say the default is
to build all targets, so we should be fine
- some pypi hashes: they were equivalent, just base16 vs. base32
* openblas: simplify a bit, fix doCheck so tests are enabled non-cross.
* doCheck should be 'true' in (at least) the non-cross case,
this looks like an inverted check that's largely benign
* doCheck will be set to 'false' in the cross case anyway,
makeDerivation does this IIRC
* targetPrefix can be used without checking, probably by design
Derivation hash does change but no "real" functionality change intended.
* openblas: nix types for config attrs (hash-preserving)
* openblas: more nix-ification, merge in cross attrs, prefer to always set
(but set appropriately for cross and non-cross cases both)
* I'm not sure what NO_BINARY_MODE does,
this change now sets explicitly false in the non-cross scenario
(previously unset unless cross).
* Drop musl NO_AFFINITY case, will be removed in upgrade shortly
* openblas: 0.3.4 -> 0.3.5
Since the exposure of the version attribute done in
892a0e8ff4, the OpenBLAS build fails for
i686-linux:
https://nix-cache.s3.amazonaws.com/log/wi79zyfmwdpwx7bm29dzqh4vglx3x550-openblas-0.3.0.drv
According to @edolstra the build slaves of Hydra updated to a new
kernel, which seems to be the real cause for this issue. The latter is
already tracked upstream[1] and a fix[2] is already included in version
0.3.1.
This very update cases 4795 rebuilds across all architectures we
support, so it's still not significant enough to go through staging. In
addition the number of rebuilds doesn't include the amount of builds
that are currently failing.
My original idea was to add a patch just for fixing this on i686-linux
and do the real update via staging, but the amount of rebuilds still is
in an acceptable range IMO and @edolstra agreed on that on IRC.
[1]: https://github.com/xianyi/OpenBLAS/issues/1575
[2]: https://github.com/xianyi/OpenBLAS/pull/1583
Signed-off-by: aszlig <aszlig@nix.build>
Cc: @ttuegel
