This is a better name since we have multiple 64-bit things that could
be referred to.
LP64 : integer=32, long=64, pointer=64
ILP64 : integer=64, long=64, pointer=64
This makes packages use lapack and blas, which can wrap different
BLAS/LAPACK implementations.
treewide: cleanup from blas/lapack changes
A few issues in the original treewide:
- can’t assume blas64 is a bool
- unused commented code
This is based on previous work for switching between BLAS and LAPACK
implementation in Debian[1] and Gentoo[2]. The goal is to have one way
to depend on the BLAS/LAPACK libraries that all packages must use. The
attrs “blas” and “lapack” are used to represent a wrapped BLAS/LAPACK
provider. Derivations that don’t care how BLAS and LAPACK are
implemented can just use blas and lapack directly. If you do care what
you get (perhaps for some CPP), you should verify that blas and lapack
match what you expect with an assertion.
The “blas” package collides with the old “blas” reference
implementation. This has been renamed to “blas-reference”. In
addition, “lapack-reference” is also included, corresponding to
“liblapack” from Netlib.org.
Currently, there are 3 providers of the BLAS and LAPACK interfaces:
- lapack-reference: the BLAS/LAPACK implementation maintained by netlib.org
- OpenBLAS: an optimized version of BLAS and LAPACK
- MKL: Intel’s unfree but highly optimized BLAS/LAPACK implementation
By default, the above implementations all use the “LP64” BLAS and
LAPACK ABI. This corresponds to “openblasCompat” and is the safest way
to use BLAS/LAPACK. You may received some benefits from “ILP64” or
8-byte integer BLAS at the expense of breaking compatibility with some
packages.
This can be switched at build time with an override like:
import <nixpkgs> {
config.allowUnfree = true;
overlays = [(self: super: {
lapack = super.lapack.override {
lapackProvider = super.lapack-reference;
};
blas = super.blas.override {
blasProvider = super.lapack-reference;
};
})];
}
or, switched at runtime via LD_LIBRARY_PATH like:
$ LD_LIBRARY_PATH=$(nix-build -E '(with import <nixpkgs> {}).lapack.override { lapackProvider = pkgs.mkl; is64bit = true; })')/lib:$(nix-build -E '(with import <nixpkgs> {}).blas.override { blasProvider = pkgs.mkl; is64bit = true; })')/lib ./your-blas-linked-binary
By default, we use OpenBLAS LP64 also known in Nixpkgs as
openblasCompat.
[1]: https://wiki.debian.org/DebianScience/LinearAlgebraLibraries
[2]: https://wiki.gentoo.org/wiki/Blas-lapack-switch
* Switch to default buildPhase & installPhase
* In preConfigure
* Do not add -DNPARTITION to CHOLMOD_CONFIG. That would disable the use of Metis but we already have that.
* Do not remove -lrt on Darwin, Darwin compiler can handle that and the code no longer exists anyway.
* With CUDA enabled
* Do not replace CUDA_ROOT. It does not exist any more. Instead we are setting CUDA_PATH in makeFlags.
* Do not replace GPU_BLAS_PATH, it defaults to CUDA_PATH so it will end up with the same value.
* Do not add -DCHOLMOD_OMP_NUM_THREADS to GPU_CONFIG. Why would be having the library use the same number of threads as the builder a good idea?
* Do not replace CUDA_PATH, we are setting it in makeFlags now.
* Do not replace CUDART_LIB and CUBLAS_LIB. They were being replaced incorrectly (cuda libs are located in lib directory, not lib64). Instead set the correct paths in makeFlags.
* Do not replace CUDA_INC_PATH. Its default looks like it will end up with the same value.
* Do not replace NV20, NV30, NV35 – not used any more.
* Do not replace NVCC, defaults to the same.
* Do not replace NVCCFLAGS, we just used the default from SourceSparse 4.4.7 with -gencode=arch=compute_60,code=compute_60 tacked on top. Current upstream default looks much better.
* Stop adding -DNTIMER to CFLAGS on Darwin – clock_gettime is supported by macOS 10.12 SDK.
* In buildPhase
* Move the make arguments to makeFlags and library to buildFlags, allowing us to drop the manual make call. I did not verify all of these are still needed.
* Remove the creation of libsuitesparse.so. As far as I could tell it is some kind of remnant of our old expression – perhaps due to past deficiencies of the build scripts, we created the individual libraries as symlinks to libsuitesparse.so: e36b3ec0a5 But since the build script can now build individual .so libraries, there should be no need for this abomination. No other distros do this either.
* In installPhase
* No need to copy things manually, there is an install target. We just need to pass INSTALL=$out flag to make to let it know where to install the files.
* I do not have means of verifying the darwin dylib name fix but it looks like it might be fixed in an upcoming release.
* I dropped the rpath fixup as it does not seem to be needed any more (ldd does not report any unresolved libraries).
* Split to multiple outputs
* Fetch source from GitLab (the only source for future releases)
* Re-order attrset to be more idiomatic
* Format with nixpkgs-fmt
Naive concatenation of $LD_LIBRARY_PATH can result in an empty
colon-delimited segment; this tells glibc to load libraries from the
current directory, which is definitely wrong, and may be a security
vulnerability if the current directory is untrusted. (See #67234, for
example.) Fix this throughout the tree.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
I haven't been doing any maintenance for a long time now and not only
do I get notified, it also creates a fake impression that all these
packages had at least one maintainer when in practice they had none.
Quoting from the splitString docstring:
NOTE: this function is not performant and should never be used.
This replaces trivial uses of splitString for splitting version
strings with the (potentially builtin) splitVersion.
Sage now by default expects the lcalc library to be named Lfunction
(instead of libLfunction). This could be changed by an environment
variable (https://trac.sagemath.org/ticket/28224), but various distros
seem to agree on this standard
(https://groups.google.com/forum/#!topic/sage-packaging/xvh55IxHTZg) so
it's best just to follow it. The old standard was set by sage anyway and
sage is the only consumer of lcalc in nixpkgs.
There ver very many conflicts, basically all due to
name -> pname+version. Fortunately, almost everything was auto-resolved
by kdiff3, and for now I just fixed up a couple evaluation problems,
as verified by the tarball job. There might be some fallback to these
conflicts, but I believe it should be minimal.
Hydra nixpkgs: ?compare=1538299
Maintenance taken over by debian package maintainer jgmbenoit:
https://trac.sagemath.org/ticket/3360#comment:17
This moves sympow to his fork, since there is no offical
version-controlled source repository from the original author and they
do not seem to maintain sympow anymore. We had already accumulated quite
some patches from debian, who have effectively maintained sympow for a
while now.
This avoids the potential conflict between autoconf flags and the waf
flags. There is some overlap between the two but waf errors when it
doesn’t recognize the flag.
My earlier change mistakenly expected `toString false` to produce '0'
instead of the empty string, leading to unexpected config changes.
Intended to address issue mentioned here and in following discussion:
https://github.com/NixOS/nixpkgs/pull/53972#issuecomment-459981602
Sorry, folks!
(special-case handling of bools here makes this "cleanup" a bit
less of an obvious win but hopefully still preferable overall :))
-----------
makeFlags in resulting derivation, according to this one-liner:
$ nix show-derivation -f . openblas|jq ".[].env.makeFlags"
before:
"BINARY=64 CC=cc CROSS= DYNAMIC_ARCH=1 FC=gfortran HOSTCC=cc INTERFACE64=1 NO_BINARY_MODE= NO_STATIC=1 NUM_THREADS=64 PREFIX=/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9 TARGET=ATHLON USE_OPENMP=1"
after:
"BINARY=64 CC=cc CROSS=0 DYNAMIC_ARCH=1 FC=gfortran HOSTCC=cc INTERFACE64=1 NO_BINARY_MODE=0 NO_STATIC=1 NUM_THREADS=64 PREFIX=/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9 TARGET=ATHLON USE_OPENMP=1"
Without knowing how `placeholder` works, it seems interesting if
entirely unrelated that the `PREFIX` is same for both! :). TIL.
This reverse changes made cfc4313 which broke builds on linux. Still
not sure why this broke Linux, but we can investigate it later.
“buildPackages.stdenv.cc” shouldn’t be put in nativeBuildInputs in
general. Either way, this restores hashes to before that commit.
thanks to @jethrokuan
The openblas script relies on CC=clang for succesfully compiling on
Darwin systems. This seems like bad behavior, but we get things like
https://hydra.nixos.org/build/87657058 otherwise.
Fixes#55536.
Comments on conflicts:
- llvm: d6f401e1 vs. 469ecc70 - docs for 6 and 7 say the default is
to build all targets, so we should be fine
- some pypi hashes: they were equivalent, just base16 vs. base32
* openblas: simplify a bit, fix doCheck so tests are enabled non-cross.
* doCheck should be 'true' in (at least) the non-cross case,
this looks like an inverted check that's largely benign
* doCheck will be set to 'false' in the cross case anyway,
makeDerivation does this IIRC
* targetPrefix can be used without checking, probably by design
Derivation hash does change but no "real" functionality change intended.
* openblas: nix types for config attrs (hash-preserving)
* openblas: more nix-ification, merge in cross attrs, prefer to always set
(but set appropriately for cross and non-cross cases both)
* I'm not sure what NO_BINARY_MODE does,
this change now sets explicitly false in the non-cross scenario
(previously unset unless cross).
* Drop musl NO_AFFINITY case, will be removed in upgrade shortly
* openblas: 0.3.4 -> 0.3.5
Since Intel's default openmp implementation is available in the same src
tarball, we can just include it in the package. This means that `mkl` now "just
works" without any environment variables, fragile setup-hooks, or forced
propagation.
Since the openmp implementation is only needed at runtime (and for test cases),
users can substitute a different one if they prefer by exporting it with
`LD_PRELOAD`, which is how Intel recommends handling this. If they do not do so,
`libiomp.so` lives next to `libmkl_rt.so` and thus will be in the RPATH as a
sane default.
Since this still comes from the same src tarball, we can ship it without losing
the fixed-output derivation; likewise, since Hydra is not building or caching
these, shipping these proprietary packages costs no bandwidth for the nix
community.
Suitesparse libraries would end up containing references to the build
directory. fixDarwinDylibNames appears to not fix this problem. We
manually use intall_name_tool to set the library paths correctly.
