Commit Graph

95 Commits

Author SHA1 Message Date
volth
f3282c8d1e treewide: remove unused variables (#63177)
* treewide: remove unused variables

* making ofborg happy
2019-06-16 19:59:05 +00:00
Matthew Bauer
7f23d9fc5f
Merge pull request #59755 from matthewbauer/patches-in-make-derivation
make-derivation: put patches in all derivations
2019-04-23 18:33:04 -04:00
volth
b0193379e5
make-derivation.nix: minor
it seems to change nothing (.platform is always there), just to be consisted with the style of other checks
2019-04-21 20:05:13 +00:00
volth
b1c3581b46
make-derivation.nix: @matthewbauer's review 2019-04-21 19:16:28 +00:00
volth
5d87bc2650 fix bootstrap when platform.gcc.arch=="skylake" 2019-04-21 16:37:49 +00:00
Matthew Bauer
c7ccb9f197 make-derivation: put patches in all derivations
This puts patches in all derivations even if it unspecified by the
derivation. By default it will be an empty list. This simplifies
overrides, as we can now assume that patches is a valid name so that
this works:

self: super: {
  mypkg = super.pkg.overrideAttrs (o: {
    patches = o.patches ++ [ ./my-very-own.patch ];
  });
}

That is, you don’t need to provide a default "or []", make-derivation
provides one for you.

Unfortunately, this is a mass rebuild.
2019-04-16 22:51:26 -04:00
Graham Christensen
817c933878
check-env: don't execute check-meta.nix 15,000 times
Generated from https://github.com/NixOS/nix/pull/2761:

```
                                                                                          ns     calls ns/call
- /home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:22:5 591200 15026 39.3451
+ /home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:22:5 8744   308   28.3896
```

more, generated by:

```
$ NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 nix-instantiate ./pkgs/top-level/release.nix -A unstable > before 2>&1
$ jq -r '.functions | map((.name + ":" + .file + ":" + (.line|tostring) + ":" + (.column|tostring) + " " + (.count|tostring))) | .[]' before | sort  > before.list
```

applying this patch, then:

```
$ NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 nix-instantiate ./pkgs/top-level/release.nix -A unstable > after 2>&1
$ jq -r '.functions | map((.name + ":" + .file + ":" + (.line|tostring) + ":" + (.column|tostring) + " " + (.count|tostring))) | .[]' after | sort  > after.list
```

and then diffing before.list and after.list to get:

```
                                                                                                        calls
- :/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:4:1               7513
+ :/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:4:1               154

- mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:23                7513
+ mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:23                154

- mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:26                7513
+ mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:26                154

- onlyLicenses:/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:21:18 15026
+ onlyLicenses:/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:21:18 308
```

The following information is from `NIX_SHOW_STATS=1 GC_INITIAL_HEAP_SIZE=4g nix-env -f ./outpaths.nix -qaP --no-name --out-path --arg checkMeta true`:

| stat                       | before         | after          | Δ               | Δ%      |
|:---------------------------|---------------:|---------------:|:----------------|--------:|
| **cpuTime**                |        179.915 |        145.543 | 🡖 34.372        | -19.10% |
| **envs-bytes**             |  3,900,878,824 |  3,599,483,208 | 🡖 301,395,616   |  -7.73% |
| **envs-elements**          |    214,426,071 |    185,881,709 | 🡖 28,544,362    | -13.31% |
| **envs-number**            |    136,591,891 |    132,026,846 | 🡖 4,565,045     |  -3.34% |
| **gc-heapSize**            | 11,400,048,640 | 12,314,890,240 | 🡕 914,841,600   |   8.02% |
| **gc-totalBytes**          | 25,976,902,560 | 24,510,740,176 | 🡖 1,466,162,384 |  -5.64% |
| **list-bytes**             |  1,665,290,080 |  1,665,290,080 | 0               |         |
| **list-concats**           |      7,264,417 |      7,264,417 | 0               |         |
| **list-elements**          |    208,161,260 |    208,161,260 | 0               |         |
| **nrAvoided**              |    191,359,386 |    179,693,661 | 🡖 11,665,725    |  -6.10% |
| **nrFunctionCalls**        |    119,665,062 |    116,348,547 | 🡖 3,316,515     |  -2.77% |
| **nrLookups**              |     80,996,257 |     76,069,825 | 🡖 4,926,432     |  -6.08% |
| **nrOpUpdateValuesCopied** |    213,930,649 |    213,930,649 | 0               |         |
| **nrOpUpdates**            |     12,025,937 |     12,025,937 | 0               |         |
| **nrPrimOpCalls**          |     88,105,604 |     86,451,598 | 🡖 1,654,006     |  -1.88% |
| **nrThunks**               |    196,842,044 |    175,126,701 | 🡖 21,715,343    | -11.03% |
| **sets-bytes**             |  7,678,425,776 |  7,285,767,928 | 🡖 392,657,848   |  -5.11% |
| **sets-elements**          |    310,241,340 |    294,373,227 | 🡖 15,868,113    |  -5.11% |
| **sets-number**            |     29,079,202 |     27,601,310 | 🡖 1,477,892     |  -5.08% |
| **sizes-Attr**             |             24 |             24 | 0               |         |
| **sizes-Bindings**         |              8 |              8 | 0               |         |
| **sizes-Env**              |             16 |             16 | 0               |         |
| **sizes-Value**            |             24 |             24 | 0               |         |
| **symbols-bytes**          |     16,474,666 |     16,474,676 | 🡕 10            |   0.00% |
| **symbols-number**         |        376,426 |        376,427 | 🡕 1             |   0.00% |
| **values-bytes**           |  6,856,506,288 |  6,316,585,560 | 🡖 539,920,728   |  -7.87% |
| **values-number**          |    285,687,762 |    263,191,065 | 🡖 22,496,697    |  -7.87% |

The following information is from `NIX_SHOW_STATS=1 GC_INITIAL_HEAP_SIZE=4g nix-instantiate ./nixos/release-combined.nix -A tested`:

| stat                       | before         | after          | Δ               | Δ%     |
|:---------------------------|---------------:|---------------:|:----------------|-------:|
| **cpuTime**                |        256.071 |        237.531 | 🡖 18.54         | -7.24% |
| **envs-bytes**             |  7,111,004,192 |  7,041,478,520 | 🡖 69,525,672    | -0.98% |
| **envs-elements**          |    346,236,940 |    339,588,487 | 🡖 6,648,453     | -1.92% |
| **envs-number**            |    271,319,292 |    270,298,164 | 🡖 1,021,128     | -0.38% |
| **gc-heapSize**            |  8,995,291,136 | 10,110,009,344 | 🡕 1,114,718,208 | 12.39% |
| **gc-totalBytes**          | 37,172,737,408 | 36,878,391,888 | 🡖 294,345,520   | -0.79% |
| **list-bytes**             |  1,886,162,656 |  1,886,163,472 | 🡕 816           |  0.00% |
| **list-concats**           |      6,898,114 |      6,898,114 | 0               |        |
| **list-elements**          |    235,770,332 |    235,770,434 | 🡕 102           |  0.00% |
| **nrAvoided**              |    328,829,821 |    326,618,157 | 🡖 2,211,664     | -0.67% |
| **nrFunctionCalls**        |    240,850,845 |    239,998,495 | 🡖 852,350       | -0.35% |
| **nrLookups**              |    144,849,632 |    142,126,339 | 🡖 2,723,293     | -1.88% |
| **nrOpUpdateValuesCopied** |    251,032,504 |    251,032,504 | 0               |        |
| **nrOpUpdates**            |     17,903,110 |     17,903,110 | 0               |        |
| **nrPrimOpCalls**          |    140,674,913 |    139,485,975 | 🡖 1,188,938     | -0.85% |
| **nrThunks**               |    294,643,131 |    288,678,022 | 🡖 5,965,109     | -2.02% |
| **sets-bytes**             |  9,464,322,192 |  9,456,172,048 | 🡖 8,150,144     | -0.09% |
| **sets-elements**          |    377,474,889 |    377,134,877 | 🡖 340,012       | -0.09% |
| **sets-number**            |     50,615,607 |     50,616,875 | 🡕 1,268         |  0.00% |
| **sizes-Attr**             |             24 |             24 | 0               |        |
| **sizes-Bindings**         |              8 |              8 | 0               |        |
| **sizes-Env**              |             16 |             16 | 0               |        |
| **sizes-Value**            |             24 |             24 | 0               |        |
| **symbols-bytes**          |      3,147,102 |      3,147,064 | 🡖 38            | -0.00% |
| **symbols-number**         |         82,819 |         82,819 | 0               |        |
| **values-bytes**           | 11,147,448,768 | 10,996,111,512 | 🡖 151,337,256   | -1.36% |
| **values-number**          |    464,477,032 |    458,171,313 | 🡖 6,305,719     | -1.36% |
2019-04-11 19:21:38 -04:00
Frederik Rietdijk
2fcb11a244 Merge staging-next into master 2019-03-01 09:06:20 +01:00
Matthew Bauer
b86e62d30d llvm: support cross compilation with useLLVM flag
You can build (partially) with LLVM toolchain using the useLLVM flag.
This works like so:

  nix-build -A hello --arg crossSystem '{ system =
    "aarch64-unknown-linux-musl"; useLLVM = true }'

also don’t separate debug info in lldClang

It doesn’t work currently with that setup hook. Missing build-id?
2019-02-26 19:45:35 -05:00
Daiderd Jordan
7ec53a932f
stdenv: only set __darwinAllowLocalNetworking on darwin
This is a darwin only nix attribute for sandbox builds, it can be
ignored on other platforms to avoid unnecessary rebuilds.
2019-02-14 21:38:08 +01:00
Matthew Bauer
5c09d977c7 Merge remote-tracking branch 'origin/master' into staging 2019-02-09 12:14:06 -05:00
Vladimír Čunát
ff101946a8
Merge branch 'master' into staging-next
Hydra nixpkgs: ?compare=1503899
2019-02-06 14:24:46 +01:00
Matthew Bauer
c6f8f8d98d make-derivation: only modify name when name is given
This preserves Nix’s native error handling of missing name:

  error: derivation name missing
2019-02-04 09:15:03 -05:00
Vincent Laporte
0d17ecce2c
mkDerivation: cleaner handling of the name argument 2019-02-01 16:02:42 +00:00
Vladimír Čunát
5effa4e0f9
Merge branch 'master' into staging-next
Comments on conflicts:
- llvm: d6f401e1 vs. 469ecc70 - docs for 6 and 7 say the default is
  to build all targets, so we should be fine
- some pypi hashes: they were equivalent, just base16 vs. base32
2019-02-01 09:22:29 +01:00
Matthew Bauer
8020bd6869
Merge pull request #35884 from dtzWill/fix/man-in-outputsToInstall
default to including "man" in outputsToInstall
2019-01-31 14:55:12 -05:00
Matthew Bauer
e2fe4c2d49 make-derivation: fix ordering of conditionals
cross should have higher precedence
2019-01-28 11:38:30 -05:00
Matthew Bauer
9fd1c170cc make-derivation: try to fix stdenv.cc == null conditional
sometimes this gets an infinite recursion error
2019-01-28 10:41:19 -05:00
Matthew Bauer
7e589e5594 make-derivation: fix position in trace
For a long time now, tracing has been broken in Nixpkgs. So when you
have an eval error you would get something like this:

  error: while evaluating the attribute 'buildInputs' of the derivation 'hello-2.10' at /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:185:11:
  while evaluating 'chooseDevOutputs' at /home/mbauer/nixpkgs/lib/attrsets.nix:474:22, called from undefined position:
  while evaluating 'optionals' at /home/mbauer/nixpkgs/lib/lists.nix:257:5, called from /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:132:17:

This is coming from how Nix handles string context and how
make-derivation messes with the "name" attribute. This commit should
restore the old behavior so you get a nice line number like:

  error: while evaluating the attribute 'buildInputs' of the derivation 'hello-2.10' at /home/mbauer/nixpkgs/pkgs/applications/misc/hello/default.nix:4:3:
  while evaluating 'chooseDevOutputs' at /home/mbauer/nixpkgs/lib/attrsets.nix:474:22, called from undefined position:
  while evaluating 'optionals' at /home/mbauer/nixpkgs/lib/lists.nix:257:5, called from /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:132:17:

NOTE: This will still be broken for cross compilation due to the
prefixes we are adding to name.
2019-01-28 09:25:47 -05:00
Matthew Bauer
ae16dd1a15
stdenv/make-derivation: don't hide broken packages
This behavior ended up breaking the handleEvalIssue functionality by hiding those packages. So something like this:

$ nix-env -iA nixpkgs.zoom-us

would silently fail, without telling the user how to fix it! Regardless, this "bug" should be handled in Nix - not Nixpkgs.

Fixes #38952.
2019-01-27 17:02:17 -05:00
Matthew Bauer
8babcc3d44
Merge branch 'master' into fix/man-in-outputsToInstall 2019-01-26 10:51:12 -05:00
Frederik Rietdijk
a1a5ea5943 stdenv: make checkInputs native
We can't run the checkPhase when build != host, so we may as well make
the checkInputs native.

This signicantly improves the situation of Python packages when enabling
strictDeps.
2019-01-13 14:43:18 +01:00
Jan Tojnar
c5881ec2c9
Merge remote-tracking branch 'upstream/master' into staging 2018-11-30 20:09:45 +01:00
Matthew Bauer
28e2277305 make-derivation: remove selfConsistent check
version is set in lots of places but might not need to be in a name.

Alternative to #50364.
2018-11-30 17:58:33 +01:00
Frederik Rietdijk
1828a5c5ba Merge master into staging-next 2018-11-30 17:46:21 +01:00
Matthew Bauer
e6834171b9
Merge pull request #51183 from matthewbauer/make-derivation-no-assertions
Fix breaking changes to make-derivation
2018-11-29 19:35:55 -06:00
Vaibhav Sagar
b5504e26b0 pkgs/stdenv/generic/make-derivation.nix: s/targetPlatform/hostPlatform/ 2018-11-29 17:54:59 -05:00
Matthew Bauer
a56fe056ec make-derivation: don’t add host suffix if there is no c compiler
Some trivial builders use the name attr to choose the exec name
produced. For example nixos-install,

  nixos-install = makeProg {
    name = "nixos-install";
    src = ./nixos-install.sh;
    nix = config.nix.package.out;
    path = makeBinPath [ nixos-enter ];
  };

When cross compiling, this puts the prog in,

  /bin/nixos-install-powerpc64le-unknown-linux-gnu
2018-11-28 12:44:10 -06:00
Matthew Bauer
d64f288154 make-derivation: don’t assert on separateDebugInfo
When separateDebugInfo = true & !stdenv.hostPlatform.isLinux, we
always gave an error. There is no reason to do this. Instead, just
don’t add separate debug info when we aren’t on Linux.
2018-11-28 12:29:00 -06:00
Matthew Bauer
13e45aeed3 make-derivation: remove selfConsistent check
version is set in lots of places but might not need to be in a name.

Alternative to #50364.
2018-11-28 12:26:21 -06:00
Vladimír Čunát
a5de78b7d7
Merge branch 'master' into staging-next 2018-11-26 10:28:00 +01:00
Matthew Bauer
7eeb02d47b Revert "make-derivation: add disallowedReferences in strictDeps"
This reverts commit 8dbfb61e46.
Also reverts commit fc99c337ed.

Fixes #50915
2018-11-25 15:53:28 -06:00
Frederik Rietdijk
c31cb577ae Merge master into staging-next 2018-11-22 09:57:08 +01:00
Matthew Bauer
fc99c337ed make-derivation: don’t disallow propagated native inputs
propagateNativeBuildInputs will end up going in the output derivation.
This case is allowed to end up in references because of that. Sorry
for the disruption!

Fixes #50865
2018-11-21 09:39:47 -06:00
Frederik Rietdijk
1d3bff25db Merge staging-next into staging 2018-11-11 14:28:08 +01:00
Frederik Rietdijk
53d00c3351 Merge master into staging-next 2018-11-10 11:08:54 +01:00
Matthew Bauer
6d531f3541 make-derivation: enable pie hardening with musl
Fixes #49071

On ld.gold, we produce broken executables when linking with the Musl
libc. This appears to be a known bug when using ld.gold and Musl. This
thread describes the workaround as enabling PIE when using ld.gold and
Musl:

https://www.openwall.com/lists/musl/2015/05/01/5

By default we don’t enable PIE to avoid breaking things. But in the
Musl case we are breaking things by not enabling PIE. So this adds a
special case for defaultHardeningFlags which keeps the pie hardening
for everything. Any packages that break with PIE can add the pie flag
to disableHardeningFlags array (a no-op for now on anything but Musl).
2018-11-07 15:24:51 +01:00
Patrick Hilhorst
320c9c10de
make-derivation: use pname-version as default name if both are present 2018-11-06 00:04:21 +01:00
Frederik Rietdijk
322f87137c Merge master into staging-next 2018-11-04 11:33:17 +01:00
Frederik Rietdijk
abea6f461a Revert "Merge pull request #49398 from Synthetica9/implement-rfc0035" to fix eval
This reverts commit 3fc7d5eb83, reversing
changes made to 1fddf2b689.

The idea is good, however, before enforcing, make sure all occurences
are fixed.
2018-11-04 11:02:43 +01:00
Frederik Rietdijk
cb4ff927a1 Merge master into staging-next 2018-11-04 08:49:24 +01:00
Matthew Bauer
8dbfb61e46 make-derivation: add disallowedReferences in strictDeps
When strictDeps = true, we don’t want native build inputs to end up in
the output. For instance gcc is a builtin native build input and
should only show up in an output if it is also listed in buildInputs.

/cc @ericson2314
2018-11-02 19:31:51 -05:00
Patrick Hilhorst
1f7fc09176
make-derivation: use a more descriptive assert message
As suggested by @Profpatsch
2018-10-30 14:33:14 +01:00
Patrick Hilhorst
c7e026bec4
make-derivation: use lib.assertMsg
As suggested by @Profpatsch
2018-10-29 18:25:59 +01:00
Patrick Hilhorst
5be927db14
make-derivation: use ? instead of builtins.hasAttr
As suggested by @edolstra
2018-10-29 15:17:13 +01:00
Patrick Hilhorst
2962f94fec
make-derivation: add check that the name is consistent with pname and version 2018-10-29 14:58:12 +01:00
Patrick Hilhorst
149a55eca7
make-derivation: get position info from version 2018-10-29 14:51:22 +01:00
Patrick Hilhorst
efca8b4b97
make-derivation: use pname-version as default name if both are present 2018-10-29 14:51:08 +01:00
Matthew Bauer
b3041b4455 make-derivation: set CMAKE_SYSTEM_* when cross compiling
Uses uname data to find what to set these variables:

- CMAKE_SYSTEM_NAME
- CMAKE_SYSTEM_PROCESSOR
- CMAKE_SYSTEM_VERSION
- CMAKE_HOST_SYSTEM_NAME
- CMAKE_HOST_SYSTEM_PROCESSOR
- CMAKE_HOST_SYSTEM_VERSION
2018-10-16 21:50:37 -05:00
John Ericson
7319013ea1 Merge remote-tracking branch 'upstream/master' into staging 2018-09-18 16:55:42 -04:00