Commit Graph

3882 Commits

Author SHA1 Message Date
Vladimír Čunát
677e3960b5
Merge #82342: rustPlatform: increase build-speed of checkPhase
...for rust-packages (into staging)
2020-06-05 09:12:30 +02:00
Frederik Rietdijk
08900c0554 Merge master into staging-next 2020-06-04 15:25:54 +02:00
Benjamin Hipple
4af635feff
Merge pull request #87866 from danieldk/validate-pkgconfig-hook
validatePkgConfig: init
2020-06-02 23:27:27 -04:00
DwarfMaster
a9eab6d2b9 Fix building of emacsWithPackages not findind subdirs.el 2020-06-02 11:52:39 +02:00
Florian Klink
8a388c8296
Merge pull request #82258 from erikarvstedt/fix-xchg-caching
fix inconsistent caching of VM xchg dirs
2020-06-01 22:34:48 +02:00
Erik Arvstedt
d85dc4f690
qemu-vm: fix inconsistent caching of xchg dirs
xchg is advertised as a bidirectional exchange dir, but file content
transfer from host to VM fails due to caching:
If a file is read in the VM and then modified on the host, subsequent
re-reads in the VM can yield old, cached data.
This is caused by the use of 9p's cache=loose mode that is explicitly
meant for read-only mounts.

9p doesn't provide any suitable cache modes, so fix this by disabling
caching.

Also, remove a now unnecessary sync in the test driver.
2020-06-01 21:55:33 +02:00
Alex Rice
e215c3bcac
agda: install literate files 2020-06-01 13:59:20 +01:00
Daniël de Kok
13083b1a01 validatePkgConfig: init
This adds the `validatePkgConfig` hook, which can be used to validate
pkg-config files in the output(s). Currently, this will just run
`pkg-config --validate` on all `.pc` files, capturing errors such as
the issue that was fixed in #87789.

The hook could be extended in the future with more fine-grained
checks.
2020-05-31 20:45:03 +02:00
Pavol Rusnak
9444756222
appimage: replace radare2+jq with readelf+awk+sha256sum
radare2 does not play nicely with filenames containing spaces
https://github.com/radareorg/radare2/issues/16958
2020-05-29 11:14:16 +02:00
Frederik Rietdijk
362d88c2b1 Merge staging-next into staging 2020-05-27 15:27:28 +02:00
Michael Howell
c21cbf22d0
buildRustCrate: Replace hyphen with underscore in env variables (#88054)
* Add test case for include dir
* buildRustCrate: replace hyphen with underscore in env

This fixes a bug that prevents encoding_c from building.
2020-05-26 20:52:18 +02:00
adisbladis
d8b6e82126
Merge pull request #88856 from zowoq/prefetch-docker
dockerTools: fix skopeo commands
2020-05-25 09:52:09 +02:00
rnhmjoj
aee614c996
treewide: replace bazaar with breezy 2020-05-25 09:22:54 +02:00
zowoq
365d07cea0 dockerTools: fix skopeo commands 2020-05-25 14:29:54 +10:00
Florian Klink
9b7a4694a0 fetchgitlab: escape rev to support "+" in it 2020-05-24 22:24:50 +02:00
Shea Levy
fffa6e8c18
Merge pull request #88762 from alexbiehl/alex/docker-tools
dockerTools: Properly add /nix/ and /nix/store/ first to layer.tar
2020-05-24 08:34:42 -04:00
Alex Biehl
886c92332c dockerTools: Properly add /nix/ and /nix/store/ first to layer.tar
In https://github.com/NixOS/nixpkgs/pull/58431 the authors ensured that
the resulting layer.tar would always list

  /nix/
  /nix/store/

first to fully comply to the tar spec. Various refactorings later it is only
ensured to create /nix/ but NOT /nix/store anymore. Instead tar transformed
them to /nix/nix and /nix/nix/store.
2020-05-24 14:28:35 +02:00
Frederik Rietdijk
d578248611 Merge staging-next into staging 2020-05-24 10:10:06 +02:00
Matthew Bauer
c74a9cd3e2
Merge pull request #69144 from SFrijters/gdc
gdc: init at 9.3.0
2020-05-23 16:30:47 -05:00
Orivej Desh
16d7f7edae Merge branch 'master' into staging 2020-05-22 09:13:23 +00:00
John Ericson
eae4f22176
Merge pull request #88518 from Ericson2314/fix-pkg-config
pkg-config, pkgconf: Misc fixes
2020-05-21 16:05:42 -04:00
Profpatsch
8b270a96fa skawarePackages.cleanPackaging: copy docFiles before rm noise 2020-05-21 19:46:38 +02:00
John Ericson
7e9d807f2c pkg-config-wrapper: Move comments outside of strings
This is much better because then we can freely keep the comments up to
date without causing mass rebuilds.

Someday, somebody should make the same change with `cc-wrapper` and
`bintools-wrapper`.
2020-05-21 11:35:46 -04:00
John Ericson
95080ea90b pkgconf: Wrap with pkg-config-wrapper 2020-05-21 11:35:46 -04:00
John Ericson
4b2ab665fb pkg-config-wrapper: Provide autoconf macros
This fixes libarchive, and probably numerous other builds.
2020-05-21 09:16:34 -04:00
John Ericson
f29919da61 pkg-config: Multiple outputs
This is good practice, and will help with tests. Wrapper adjust to match
bintools-wrapper re these new outputs.
2020-05-21 08:58:34 -04:00
Stefan Frijters
fc9b93d2fc gdc: init at 9.3.0 2020-05-19 23:06:52 +02:00
lewo
a498da343a
Merge pull request #87154 from utdemir/buildimage-optimizations
Some performance optimizations to dockerTools.build{,Layered}Image
2020-05-19 15:39:25 +02:00
Jan Tojnar
f8a9c6efac
Merge branch 'staging-next' into staging 2020-05-18 21:09:48 +02:00
John Ericson
c71ab32a67 pkg-config-wrapper: Init
This fixes longstanding build issues
2020-05-16 00:21:21 +00:00
John Ericson
ee8cfe4bf3 treewide: pkg-config has targetPrefix 2020-05-16 00:21:21 +00:00
zowoq
42232493a3 dockerTools: pass insecure-policy and tmpdir to skopeo 2020-05-16 08:46:13 +10:00
lewo
df928fafd4
Merge pull request #87725 from nlewo/fix-dockertools
dockerTools.examples: fix the nginx image
2020-05-15 16:54:09 +02:00
lewo
98a723e6b3
Merge pull request #87253 from utdemir/dockertools-preserve-env
Preserve environment variables from the parent image on dockerTools.buildImage
2020-05-15 09:59:00 +02:00
Dmitry Kalinkin
9943fd1a1d
Merge pull request #76653 from alexarice/agda-rework
Agda rework
2020-05-14 18:02:50 -04:00
Alex Rice
d30e2468e0
agda: rework builder 2020-05-14 20:54:11 +01:00
Frederik Rietdijk
404fe35d65 Merge staging-next into staging 2020-05-14 09:37:03 +02:00
Matthew Bauer
d38bad590e
Merge pull request #83180 from matthewbauer/only-add-ldflags-for-macos
bintools: only add macos flags when targeting macOS
2020-05-13 10:34:40 -05:00
Antoine Eiche
9ac444d80e dockerTools.examples: fix the nginx image
Nginx now requires directories `/var/log/nginx` and `/var/cache/nginx`
to be existing.

The objective is to fix the test `nixosTests.docker-tools`.
2020-05-13 10:29:53 +02:00
Maximilian Bosch
6b23cfe689
rustPlatform: add buildAndTestSubdir-argument
There are several tarballs (such as the `rust-lang/rust`-source) with a
`Cargo.toml` at root and several sub-packages (with their own Cargo.toml)
without using workspaces[1].

In such a case it's needed to move into a subdir to only build the
specified sub-package (e.g. `rustfmt` or `rsl`), however the artifacts
are at `/target` in the root-dir of the build environment. This breaks
the build since `buildRustPackage` searches for executables in `target`
(which is at the build-env's root) at the end of the `buildPhase`.

With the optional `buildAndTestSubdir`-argument, the builder moves into
the specified subdir using `pushd`/`popd` during `buildPhase` and
`checkPhase`.

Also moved the logic to find executables and libs to the end of the `buildPhase`
from a custom `postBuild`-hook to fix packages with custom `build`/`install`-procedures
such as `uutils-coreutils`.

[1] https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html
2020-05-13 01:47:17 +02:00
Maximilian Bosch
736462d995
rustPlatform: make it possible to override the profile for cargo test 2020-05-13 01:39:44 +02:00
John Ericson
1ac5398589 *-wrapper; Switch from infixSalt to suffixSalt
I hate the thing too even though I made it, and rather just get rid of
it. But we can't do that yet. In the meantime, this brings us more
inline with autoconf and will make it slightly easier for me to write a
pkg-config wrapper, which we need.
2020-05-12 00:44:44 -04:00
Matthew Bauer
43545032af
Merge pull request #87314 from matthewbauer/bazel-flat
build-bazel-package: switch hash mode to “flat”
2020-05-11 15:27:48 -05:00
Frederik Rietdijk
d20d734bcf Merge master into staging-next 2020-05-11 09:19:26 +02:00
Vladimír Čunát
5eaabaf089
Merge branch 'staging-next' into PR 82342
Hydra nixpkgs: ?compare=1586582
2020-05-11 08:22:59 +02:00
talyz
9f09253e52
php.buildPecl: Allow PECLs to depend on other PECLs
Some PECLs depend on other PECLs and, like internal PHP extension
dependencies, need to be loaded in the correct order. This makes this
possible by adding the argument "peclDeps" to buildPecl, which adds
the extension to buildInputs and is treated the same way as
internalDeps when the extension config is generated.
2020-05-09 23:38:21 +02:00
Matthew Bauer
1e19cc37bd build-bazel-package: pass proxy env vars 2020-05-09 13:35:47 -05:00
Matthew Bauer
2a8d05627b build-bazel-package: switch hash mode to “flat”
flat hashes can be substituted through hashed-mirrors, while recursive
hashes can’t. This is especially important for Bazel since the bazel
fetch dependencies can come from multiple different methods (git,
http, ftp, etc.). To do this, we create tar archives from the
output/external directory, which is then extracted to build. All of
the Bazel hashes are all updated.
2020-05-09 13:33:26 -05:00
Maximilian Bosch
f236714d65
rustPlatform: fix log 2020-05-08 22:59:50 +02:00
Maximilian Bosch
e32c005772
rustPlatform: don't install test executables
This is done by gathering all binaries to install before running the
checkPhase.
2020-05-08 22:59:50 +02:00
Utku Demir
f5a90a7aab
dockerTools.buildImage: Preserve environment variables from the parent image 2020-05-08 21:49:16 +12:00
Frederik Rietdijk
bc48203b98 Merge staging-next into staging 2020-05-08 06:46:14 +02:00
Benjamin Hipple
7f845578af rustPlatform: fix bug with ambiguous diff tool
If a user provides `nativeBuildInputs = [ llvmPackages.bintools ]` or any other
package containing a `${prefix}/bin/diff`, the builder could use it instead
of the standard unix `diff`, causing a build failure.

This updates the call to specify an abspath to `diff` and avoid reliance on `PATH`.

Resolves #87081
2020-05-06 21:03:41 -04:00
Utku Demir
f12346d493
dockerTools: Calculate tarsum's on the fly
Calculating the tarsum after creating a layer is inefficient, since
we have to read the tarball we've just written from the disk.

This commit simultaneously calculates the tarsum while creating the
tarball.
2020-05-07 11:50:07 +12:00
Utku Demir
69f6294724
dockerTools.buildLayeredImage: Avoid appending to tarballs when building layers
Appending to an existing tar archive repeatedly seems to be a quadratic
operation, since tar seems to traverse the existing archive even using
the `-r, --append` flag. This commit avoids that by passing the list of
files to a single tar invocation.
2020-05-07 11:50:00 +12:00
Maximilian Bosch
04248f606f
rustPlatform: increase build-speed of checkPhase for rust-packages
When running `cargo test --release`, the artifacts from `buildPhase`
will be reused here. Previously, most of the stuff had to be recompiled
without optimizations.
2020-05-06 23:47:31 +02:00
Frederik Rietdijk
9875bbae75 Merge master into staging-next 2020-05-05 19:51:09 +02:00
Benjamin Hipple
ba160ab741
Merge pull request #85410 from matthewbauer/skip-post-fetch-in-hashed-mirrors
fetchurl: don’t run ‘postFetch’ on hashed-mirrors
2020-05-04 22:45:19 -04:00
Jörg Thalheim
2fb5dac372 nix-prefetch-git: also add store path to json output 2020-05-04 17:43:29 +05:30
Pavol Rusnak
7b0167204d treewide: use https for nixos.org and hydra.nixos.org
tarballs.nixos.org is omitted from the change because urls from there
are always hashed and checked
2020-05-03 22:14:21 -07:00
adisbladis
fafb127947
dockertools: Add a buildLayeredImageWithNixDb function
This is analogous to buildImageWithNixDb but instead uses
buildLayeredImage under the hood.
2020-05-02 15:59:39 +01:00
Frederik Rietdijk
afb1041148 Merge master into staging-next 2020-05-02 09:39:00 +02:00
Jörg Thalheim
d67098ec2e
appimage: fix r2 commands with newer radare2 version 2020-05-01 12:36:18 +01:00
Frederik Rietdijk
484ee79050 Merge staging-next into staging 2020-05-01 08:57:10 +02:00
Frederik Rietdijk
2da19f9483
Merge pull request #85653 from veprbl/pr/darwin_binutils_add_man
darwin.binutils: propagate man pages from darwin.cctools
2020-05-01 08:49:56 +02:00
Mario Rodas
f7ac956a2b buildDunePackage: add support for parallel build
dune allows to specify the number of jobs to build the package.
2020-04-29 16:08:52 +02:00
Simonas Kazlauskas
dde3e8edf4 nix-prefetch-docker: Add jq into path
Otheriwse running the tool fails with

    line 41: jq: command not found
2020-04-27 22:21:55 +03:00
zimbatm
1bf7f5f240
Merge pull request #85750 from zimbatm/srcOnly-docs
srcOnly: fix invocation and document
2020-04-27 14:49:55 +02:00
zimbatm
31ead7d030
srcOnly: rename buildInputs to nativeBuildInputs
The only reason to pass build inputs is to extend the unpackPhase with
custom unpack commands. Eg: add "unrar" to unpack rar sources. And those
should really be passed as native build inputs. Why? Because
nativeBuildInputs is for dependencies that are used at build time but
will not propagate as runtime dependencies. And also, cross-compilation.
2020-04-27 14:31:40 +02:00
Dmitry Kalinkin
f0da3b5e94
Merge pull request #85952 from LnL7/darwin-lapack
lapack: fix darwin library identifiers
2020-04-26 00:39:26 -04:00
Silvan Mosberger
eeb4e522b5
Merge pull request #85255 from Infinisil/sanitize-run-command-name 2020-04-25 00:11:41 +02:00
Daiderd Jordan
5ae928e18f
lapack: fix darwin library identifiers
The build system already sets these properly to the absolute path so no
need to patch the libraries on darwin.

    $ otool -D result/lib/liblapacke.dylib
    result/lib/liblapacke.dylib:
    /nix/store/k88gy5s765yn3dc5ws3jbykyvklm7z96-openblas-0.3.8/lib/libopenblasp-r0.3.8.dylib

Fixes #85713
2020-04-24 20:00:56 +02:00
Frederik Rietdijk
cff0669a48 Merge master into staging-next 2020-04-23 08:11:16 +02:00
Benjamin Hipple
21be830522
Merge pull request #85799 from matthewbauer/blas-lapack-expand-docs
BLAS/LAPACK fixup and expand docs
2020-04-22 21:56:20 -04:00
Matthew Bauer
5a500ff0ba blas,lapack: use correct name for library
To match the soname, we need to use libblas.so.3, liblapack.so.3.
2020-04-22 12:37:04 -05:00
Frederik Rietdijk
8374a2a0ee Merge master into staging-next 2020-04-22 17:20:20 +02:00
zimbatm
bbbc419186
srcOnly: fix invocation and document
Previously, callPackage would try and fill the arguments such as `name`
and `src` which would cause problems if those existed as top-level
attributes. This also makes it clearer what part is the function
signature.

Then document the derivation inline in the code to explain the ellipsis
and various use-cases.
2020-04-22 14:06:28 +02:00
edef
e71990711f
Merge pull request #85680 from NixOS/fix-srconly
srcOnly: ignore additional arguments
2020-04-21 22:45:34 +00:00
Frederik Rietdijk
23be4a8b4d Merge master into staging-next 2020-04-21 19:59:56 +02:00
edef
11e13acc46 srcOnly: ignore additional arguments
This reverts commit b32a057425,
which breaks even the most straightforward uses of srcOnly:

    nix-repl> srcOnly guile
    error: anonymous function at /home/src/nixpkgs/pkgs/build-support/src-only/default.nix:1:1 called with unexpected argument 'drvPath', at /home/src/nixpkgs/lib/customisation.nix:69:16

    nix-repl> srcOnly hello
    error: anonymous function at /home/src/nixpkgs/pkgs/build-support/src-only/default.nix:1:1 called with unexpected argument 'drvPath', at /home/src/nixpkgs/lib/customisation.nix:69:16

Link: https://github.com/NixOS/nixpkgs/pull/80903#issuecomment-617172927
2020-04-21 13:20:17 +00:00
Frederik Rietdijk
803b3d296c Merge staging-next into staging 2020-04-21 08:29:51 +02:00
Matthew Bauer
8f58fa044d blas,lapack: symlink headers when using mkl
Some of these are necessary for things like python.pkgs.numexpr
2020-04-21 00:30:44 -05:00
Matthew Bauer
0caa375b4c blas,lapack: provide symlink for libmkl_rt.so
This is needed for numpy to detect mkl correctly.
2020-04-20 23:47:10 -05:00
Dmitry Kalinkin
c450617f99
bintools-wrapper: only propagate .info output if it exists in the original 2020-04-20 23:49:02 -04:00
Matthew Bauer
ff2f2644f8 blas,lapack: use isILP64 instead of is64bit
This is a better name since we have multiple 64-bit things that could
be referred to.

LP64  : integer=32, long=64, pointer=64
ILP64 : integer=64, long=64, pointer=64
2020-04-20 16:02:43 -05:00
Matthew Bauer
3c9c894f83 blas,lapack: don’t patchelf
We have some unused RPATHs we don’t want shrunk. These are used in MKL
to dlopen based on architecture.
2020-04-20 15:50:28 -05:00
Lars Jellema
8e79583510 gnat: init at 9.3.0 2020-04-20 14:41:22 +02:00
Sébastien Maret
712bf551d8 blas: fix build on darwin 2020-04-19 23:07:10 +02:00
John Ericson
1ea80c2cc3 Merge remote-tracking branch 'upstream/master' into staging 2020-04-18 15:40:49 -04:00
John Ericson
eb4acb6ca9
Merge pull request #85468 from Ericson2314/no-dumpmachine
treewide: Get rid of -dumpmachine in favor of static info
2020-04-18 00:35:19 -04:00
John Ericson
e3d50e5cb0 Merge branch 'master' of github.com:NixOS/nixpkgs into staging 2020-04-18 00:10:08 -04:00
Milan Pässler
fc5090b315 blas: fix build 2020-04-18 03:06:47 +02:00
John Ericson
33c2a76c5e Merge remote-tracking branch 'upstream/master' into staging 2020-04-17 18:40:51 -04:00
Matthew Bauer
d83e9c9573 setup-hooks/audit-blas.sh: init
Add a simple hook that makes sure blas and lapack are linked to the
right place.
2020-04-17 16:24:25 -05:00
Matthew Bauer
43873351ff blas/lapack: add wrapper for “alternative”s of BLAS/LAPACK provider
This is based on previous work for switching between BLAS and LAPACK
implementation in Debian[1] and Gentoo[2]. The goal is to have one way
to depend on the BLAS/LAPACK libraries that all packages must use. The
attrs “blas” and “lapack” are used to represent a wrapped BLAS/LAPACK
provider. Derivations that don’t care how BLAS and LAPACK are
implemented can just use blas and lapack directly. If you do care what
you get (perhaps for some CPP), you should verify that blas and lapack
match what you expect with an assertion.

The “blas” package collides with the old “blas” reference
implementation. This has been renamed to “blas-reference”. In
addition, “lapack-reference” is also included, corresponding to
“liblapack” from Netlib.org.

Currently, there are 3 providers of the BLAS and LAPACK interfaces:

- lapack-reference: the BLAS/LAPACK implementation maintained by netlib.org
- OpenBLAS: an optimized version of BLAS and LAPACK
- MKL: Intel’s unfree but highly optimized BLAS/LAPACK implementation

By default, the above implementations all use the “LP64” BLAS and
LAPACK ABI. This corresponds to “openblasCompat” and is the safest way
to use BLAS/LAPACK. You may received some benefits from “ILP64” or
8-byte integer BLAS at the expense of breaking compatibility with some
packages.

This can be switched at build time with an override like:

    import <nixpkgs> {
        config.allowUnfree = true;
        overlays = [(self: super: {
          lapack = super.lapack.override {
            lapackProvider = super.lapack-reference;
          };
          blas = super.blas.override {
            blasProvider = super.lapack-reference;
          };
        })];
      }

or, switched at runtime via LD_LIBRARY_PATH like:

    $ LD_LIBRARY_PATH=$(nix-build -E '(with import <nixpkgs> {}).lapack.override { lapackProvider = pkgs.mkl; is64bit = true; })')/lib:$(nix-build -E '(with import <nixpkgs> {}).blas.override { blasProvider = pkgs.mkl; is64bit = true; })')/lib ./your-blas-linked-binary

By default, we use OpenBLAS LP64 also known in Nixpkgs as
openblasCompat.

[1]: https://wiki.debian.org/DebianScience/LinearAlgebraLibraries
[2]: https://wiki.gentoo.org/wiki/Blas-lapack-switch
2020-04-17 16:23:55 -05:00
Matthew Bauer
6367a55312 fetchurl: don’t run ‘postFetch’ on hashed-mirrors
hashed-mirrors are content addressed. So if $outputHash is in the
hashed-mirror, changes from ‘postFetch’ would already be made. So,
running postFetch will end up applying the change /again/, which we
don’t want.
2020-04-16 16:36:56 -05:00
Silvan Mosberger
c60e559e13
trivial-builders: Sanitize derivation name
This then supports using functions like writeShellScriptBin with script names
that would be invalid as derivation names
2020-04-15 00:31:57 +02:00
John Ericson
c8a6ea5161 Merge remote-tracking branch 'upstream/master' into staging 2020-04-13 22:17:15 -04:00
John Ericson
204d710405 treewide: Get rid of -dumpmachine in favor of static info 2020-04-13 19:26:19 -04:00
Andreas Rammhold
84b91899c3
Merge pull request #85172 from andir/buildRustCrate-proc-macro
buildRustCrate: support proc-macro in default prelude
2020-04-13 23:35:19 +02:00