Commit Graph

1488 Commits

Author SHA1 Message Date
John Ericson
97a48835b7 mkDerivation, cc-wrapper: Check hardening flag validity in Nix
This becomes necessary if more wrappers besides cc-wrapper start
supporting hardening flags. Also good to make the warning into an
error.

Also ensure interface is being used right: Not as a string, not just in
bash.
2017-08-30 17:53:42 +02:00
Daiderd Jordan
92652b4d79
darwin-CF: use @rpath for library id and add an rpath entry for CF based on NIX_COREFOUNDATION_RPATH 2017-08-28 23:24:58 +02:00
John Ericson
a470be5a16 expand-response-params: Build more normally 2017-08-25 15:10:02 -04:00
John Ericson
2e7a390212 Merge pull request #28057 from obsidiansystems/stdenv-set-u
stdenv-setup: use `set -u`
2017-08-25 11:19:58 -04:00
Vladimír Čunát
8137a8cb73
gawk: refactor
- Don't build with libsigsegv by default.  The build apparently attempted
  to link against it, but it never retained the reference anyway...
- Side effect: stdenv bootstrapping needs no libsigsegv anymore.
- Run checks, but only in the interactive gawk by default on Linux,
  so that stdenv bootstrap isn't slowed down (by glibc locales, etc.).
- xz should be no longer needed in inputs, as we have it in stdenvs now.

The whole change was triggered by some used kernel versions still
breaking libsigsegv tests #28464.
2017-08-24 11:06:53 +02:00
Tuomas Tynkkynen
0c0fad6141 treewide: Consistently call ARM 'arm'
No need for silly differences.
2017-08-24 01:17:01 +03:00
John Ericson
81194eef45 stdenv-setup: Use set -u as much as possible
Older bash version, like those in the bootstrap tools and on macOS,
currently confuse variables defined as an empty array with undefined
variables. `${foo+"${foo[@]}"}` will prevent `set -u` problems with
empty arrays and older without making a single '' in the empty case.

Care is taken to `set +u` when running hooks so as to not break existing
packages.
2017-08-23 15:57:56 -04:00
John Ericson
0135e61b4c Merge remote-tracking branch 'upstream/master' into staging
That way the tarball job succeeds
2017-08-21 18:42:57 -04:00
John Ericson
c035711072 cc-wrapper: Remove unused params
Ensured hashes unchanged and eval succeeds in tarball job
2017-08-21 18:40:41 -04:00
Frederik Rietdijk
6bbc3a0b24 Merge commit '3b29468313bc8604fe8f85c8d9316fd276d3985c' into HEAD 2017-08-21 04:44:40 +02:00
Frederik Rietdijk
04cd1db2b7 Merge remote-tracking branch 'upstream/master' into HEAD 2017-08-21 01:28:27 +02:00
John Ericson
fbab1d485b stdenvs: Distinguish between extraBuildInputs and extraNativeBuildInputs
This version continues to use bash + stdenv/setup for the default
inputs.
2017-08-18 12:02:13 -04:00
Tuomas Tynkkynen
7320fa9d45 Revert "stdenvs: Distinguish between extraBuildInputs and extraNativeBuildInputs"
This reverts commit eeabf85780.

This change suddenly makes tons of stdenv internals visible in
nativeBuildInputs of every derivation, which doesn't seem desirable.
E.g:

````
nix-repl> hello.nativeBuildInputs
[ «derivation /nix/store/bcfkyf6bhssxd2vzwgzmsbn7b5b9rpxc-patchelf-0.9.drv»
  «derivation /nix/store/4wnshnz9wwanpfzcrdd76rri7pyqn9sk-paxctl-0.9.drv»
  << snip 10+ lines >>
  «derivation /nix/store/d35pgh1lcg5nm0x28d899pxj30b8c9b2-gcc-wrapper-6.4.0.drv»
]
````
2017-08-18 13:21:56 +03:00
Daiderd Jordan
6a870a59f8 Merge pull request #28174 from matthewbauer/darwin-in-release
enable hydra jobs for packages x86_64-linux does not support
2017-08-17 23:29:00 +02:00
Frederik Rietdijk
8f2ea38f8f Merge remote-tracking branch 'upstream/master' into HEAD 2017-08-17 18:34:17 +02:00
John Ericson
61d241f405 darwin stdenv: Float persistentN bindings into per-stage lets 2017-08-16 16:36:21 -04:00
John Ericson
eeabf85780 stdenvs: Distinguish between extraBuildInputs and extraNativeBuildInputs
Additionally, instead of pulling them from `setup.sh`, route them via
Nix. This gets us one step closer to making stdenv be a plain attribute
set instead of a derivation.
2017-08-15 18:24:54 -04:00
John Ericson
a71cf06b16 mkDerivation: Simply Nix
No hashes were changed by this
2017-08-15 16:13:30 -04:00
Vladimír Čunát
a1bb61312c
stdenv: fixup allowedRequisites on aarch64-linux 2017-08-15 08:09:26 +02:00
Will Dietz
3e8d68e514 make-bootstrap-tools: Fix config opts for ash builtins after upgrade 2017-08-14 22:40:11 +03:00
Vladimír Čunát
505e94256e
stdenv: resurrect the allowedRequisites check
Discovered in #28091.  I'm sorry I forgot to re-check my TODOs, long ago.
2017-08-13 11:44:36 +02:00
Tuomas Tynkkynen
3e9f76774a
nixpkgs release: Fix Darwin-only jobs
Currently the logic of generating nixpkgs Hydra jobs is to walk through
the pkgs evaluated for system = "x86_64-linux", collect any derivations
and their meta.platforms values. However, that doesn't work for
packages whose meta.platforms doesn't include x86_64-linux, as just
evaluating their meta attribute raises an error so they get skipped
completely.

As a less-intrusive fix (i.e. anything than rewriting the current package
enumeration logic), allow passing `config.allowUnsupportedSystem = true`
to permit evaluating packages regardless of their platform and use that
in the package listing phase.

Fixes #25200
2017-08-12 20:38:27 -07:00
Linus Heckemann
17753fa005 stdenv: fix typo in setup.sh 2017-08-09 17:33:02 +01:00
John Ericson
42f35503b5 cc-wrapper: Make hygienic
See the added comments for what exactly has been done.
2017-08-07 03:05:50 -04:00
John Ericson
8ba6012dfd Merge PR #27536 2017-08-03 17:25:30 -04:00
John Ericson
0c37778c2c cc-wrapper: WIP linking hack for mac OS
Probably best to override Haskell packages set, or anything else
linking a lot of libraries, with this.
2017-07-31 17:02:56 -04:00
John Ericson
9be40841ea Merge remote-tracking branch 'upstream/master' into staging-base
Conflicts:
	pkgs/build-support/cc-wrapper/default.nix
	pkgs/build-support/gcc-wrapper-old/builder.sh
	pkgs/build-support/trivial-builders.nix
	pkgs/desktops/kde-4.14/kde-package/default.nix
	pkgs/development/compilers/openjdk-darwin/8.nix
	pkgs/development/compilers/openjdk-darwin/default.nix
	pkgs/development/compilers/openjdk/7.nix
	pkgs/development/compilers/openjdk/8.nix
	pkgs/development/compilers/oraclejdk/jdk-linux-base.nix
	pkgs/development/compilers/zulu/default.nix
	pkgs/development/haskell-modules/generic-builder.nix
	pkgs/misc/misc.nix
	pkgs/stdenv/generic/builder.sh
	pkgs/stdenv/generic/setup.sh
2017-07-26 13:46:04 -04:00
John Ericson
820e4021d3 stdenv-setup: Remove any declare -g
This is invalid before bash-4.2, affecting bash used impurely in
nix-shell on MacOS.
2017-07-26 09:11:18 -04:00
John Ericson
ea7d13cf1a stdenv-setup and misc hooks: Work with bash-3.4 for MacOS nix-shell
This is a temporary measure until this impurity is removed from Nix.
2017-07-26 09:08:01 -04:00
John Ericson
f6f40e3fe5 stdenv-setup and misc pkgs: Revert to space-deliminated propagated-* files
We cannot switch to line-delimited yet, because certain Nix commands do
not read in the entire file, but just the first line.
2017-07-26 09:07:55 -04:00
John Ericson
34c0ba498c stdenv-setup: Add quotes that don't do anything for consistency.
@vcunat and others rightly point out that it's easier to quote always,
than learn Bash's idiosyncrasies enough to know when it doesn't make a
difference.

This reverts commit 2743078f66, which
removes quotes that don't do anything, and then goes further adding
even more quotes.
2017-07-25 14:36:00 -04:00
John Ericson
98cff3f446 darwin stdenv: Ensure libSystem reexports the right libraries
The logic was made pure for the normal libSystem, but this change never
made it to the bootstrap tools. Deduplication the logic as the comment
suggests would have prevented this, but here's a stop-gap until we do
so.
2017-07-25 14:35:43 -04:00
Eelco Dolstra
ca8aa5dc87
Use GCC 7 by default 2017-07-25 13:56:57 +02:00
Dan Peebles
2829ea57cb stdenv/setup.sh: undo local -n change
It's better than the eval solution this is adding back, but until we can
rely on a particular version of bash in nix-shell, this just breaks too
much stuff.

See c94f3d5575
and https://github.com/NixOS/nix/pull/1483 for the better long-term
solution.
2017-07-24 13:09:32 -04:00
Eelco Dolstra
6669a3b477
stdenv: Remove log nesting
Nix/Hydra no longer support pretty printing of logs, so this is no
longer useful.
2017-07-24 14:30:46 +02:00
John Ericson
b087618ac0 Revert "stdenv: Store one package per line in nix-support/propagated-*"
As @oxij points out in [1], this breakage is especially serious because
it changes the contents of built environments without a corresonding
change in their hashes. Also, the revert is easier than I thought.

This reverts commit 3cb745d5a6.

[1]: https://github.com/NixOS/nixpkgs/pull/27427#issuecomment-317293040
2017-07-24 01:05:30 -04:00
Daniel Peebles
bd2e91e3a2 Merge pull request #27318 from copumpkin/darwin-high-sierra
Support High Sierra on Darwin
2017-07-18 17:06:06 -04:00
John Ericson
aaaa470ff8 mkDerivation: Fix errors from #27365
`nix-build pkgs/top-level/release.nix -A tarball` now succeeds.

`configureFlags = null` lead to a type error, and one overrideDrv
needed to be converted to to append a configureFlags list instead of
string due to the normalization.

Thanks @vcunat for alerting me to the issues---sorry I did not catch
them before merging my own PR.
2017-07-15 13:47:21 -04:00
John Ericson
e826a6a247 stdenv: Move some logic from cross adapter to stdenv proper
Eventually the adapter will be removed.  Moved is

 - Name suffix from hostPlatform

 - configurePlatforms

   To not cause more breakage, the default is currently [], but
   eventually it will be [ "build" "host" ], as the cross adapter makes
   it today.
2017-07-13 19:05:36 -04:00
John Ericson
30a1420414 stdenv-setup: Pull out and explain 3-part printing of commands
@Dezgeg made the good point that the reasons for doing this were not at
all intuitive.
2017-07-13 16:31:39 -04:00
John Ericson
2743078f66 stdenv-setup: Remove useless quotes
foo=$1 surprisingly doesn't need quotes in Bash. Word splits are only
syntactic in string variable (not array var!) assignments.
2017-07-13 14:59:53 -04:00
John Ericson
273a4c1c78 stdenv-setup: Combine [[ .. ]] && [[ .. ]] into one [[ .. && .. ]]
Also remove useless quotes on same line
2017-07-13 14:57:16 -04:00
John Ericson
5d693c84d2 stdenv-setup: Clean up 'substitute()' for style and error handling
It now blows up on null byte in file (rather than silently truncating),
and invalid arguments (rather than silently skipping).
2017-07-12 17:47:20 -04:00
John Ericson
5d4efb2c81 stdenv-setup: Misc improvements as directed by ShellCheck
I took some liberties with the flags-echoing code to make it more
concise and correct. Also, a few warnings in findInputs and friends I
skipped because I am going to rewrite those anyways.

Thanks @grahamc for telling me about this great linter!
2017-07-12 15:31:10 -04:00
John Ericson
8d76effc17 stdenv-setup: Make the package accumulators associative arrays instead of strings
This is generally cleaner: less eval, less worrying about separators,
and probably also faster. I got the idea from that python wrapper
script.
2017-07-12 15:30:56 -04:00
Dan Peebles
0419452113 Fix Darwin stdenv to work on 10.13
The main changes are in libSystem, which lost the coretls component in 10.13
and some hardening changes that quietly crash any program that uses %n in
a non-constant format string, so we've needed to patch a lot of programs that
use gnulib.
2017-07-11 21:56:38 -04:00
John Ericson
3cb745d5a6 stdenv: Store one package per line in nix-support/propagated-*
This makes those files a bit easier to read. Also, for what it's worth,
it brings us one baby step closer to handling spaces in store paths.

Also, I optimized handling of many transitive deps with read. Probably,
not very beneficial, but nice to enforce the pkg-per-line structure.
Doing so let me find much dubious code and fix it.

Two misc notes:

 - `propagated-user-env-packages` also needed to be adjusted as
   sometimes it is copied to/from the propagated input files.

 - `local fd` should ensure that file descriptors aren't clobbered
   during recursion.
2017-07-10 13:32:13 -04:00
John Ericson
5896d84dbb stdenv: Stop reversing the list of sandbox stuff
We're breaking hashes anyways
2017-07-10 11:25:51 -04:00
John Ericson
2f198956c7 stdenv: Make separate-debug-info.sh a nativeBuildInput 2017-07-10 11:25:51 -04:00
Vladimír Čunát
bfb7ef86f3
Merge branch 'master' into staging
Mass rebuilds incoming.  The mass-rebuild situation got really messy
this weekend.
2017-07-09 18:07:52 +02:00
Daiderd Jordan
980346592c
Merge branch 'staging' into master 2017-07-08 22:22:17 +02:00
John Ericson
a302d7360f top-level: {build,host,target}Platform are defined in the stdenv instead
See #27069 for a discussion of this
2017-07-07 12:55:02 -04:00
John Ericson
f5364122b5 Merge pull request #27215 from obsidiansystems/stdenv-harden
Harden stdenv in two misc ways
2017-07-07 12:48:51 -04:00
John Ericson
afc2023993 stdenv: Have mkDerivation pull the "extra" arguments from stdenv instead
Something more elaborate is needed for the "*Platform" arguments.
2017-07-07 12:16:51 -04:00
John Ericson
4cf4d7180d stdenv: Conservatively move mkDerivation into it's own file 2017-07-07 12:16:51 -04:00
Vladimír Čunát
e8e57452f4 stdenv: separate all meta-checking code (~200 lines)
Only cosmetic changes are done otherwise.
Real refactoring is left for later.

There's a small slow-down on my machine:
$ time nix-env -qa -P >/dev/null
gets from ~2.8 to ~3.5 seconds (negligible change in RAM).
That's most likely caused by sharing less computation between different
mkDerivation calls, and I plan to improve that soon.
2017-07-07 12:16:26 -04:00
Vladimír Čunát
dfc004e69c lib.lists.mutuallyExclusive: add function 2017-07-07 12:02:29 -04:00
Vladimír Čunát
5afcdc88fa stdenv: simple refactor to get rid of pos'
Suggested by Ericson2314.
2017-07-07 12:02:29 -04:00
Vladimír Čunát
7fdf18e892 stdenv: refactor (no change in semantics)
This just moves some expressions around in preparation to further changes.
2017-07-07 12:02:29 -04:00
John Ericson
e57a220f81 stdenv, swift: Use local fd in is* bash functions for hygiene 2017-07-07 11:40:07 -04:00
John Ericson
a14cf06182 stdenv: Harden hook runners
Instead of eval, use a "nameref" to get the name of the array and
iterate with that. Also, make the for-loop parameter a local variable,
too.
2017-07-07 11:35:09 -04:00
Ryan Trinkle
7004641566 Merge pull request #26974 from obsidiansystems/response-file-parsing-speed
cc-wrapper: improve response file parsing speed
2017-07-05 16:18:22 -04:00
Vladimír Čunát
5328aac7be
Merge branch 'staging'
Comparison looks OK; I'll try some fixes on master directly.
http://hydra.nixos.org/eval/1372577?compare=1372497
2017-07-05 08:55:26 +02:00
Tim Steinbach
d788244423
stdenv / booter: imap -> imap1
Fix the usage of imap as a follow-up to #25543
2017-07-04 21:04:13 -04:00
Orivej Desh
2bc7b4e134 cc-wrapper: simplify expandResponseParams parser
Import from b2446902fe
2017-07-03 21:51:23 +00:00
Ryan Trinkle
d07f30f628 cc-wrapper: improve response file parsing speed 2017-06-30 15:20:53 -04:00
John Ericson
d61c22341b Merge accepted cross compilation PRs into staging 2017-06-29 18:07:13 -04:00
John Ericson
ad8d8fb2f5 stdenv: Simplify dependency code
This is a bit simpler now, but more importantly it scales better when I
double the number of sorts of dependencies as part of my cross
compilation work.
2017-06-29 17:45:08 -04:00
Daiderd Jordan
d76bc8ed15 Merge pull request #26818 from LnL7/llvm-manpages
llvm-packages: get rid of extra build depedencies for manpages
2017-06-26 21:40:55 +02:00
Daiderd Jordan
f65b5365bb
darwin-stdenv: remove llvm/clang manpage overrides from 2017-06-26 20:24:17 +02:00
David McFarland
f81117e324 stdenv: remove leftover 'system' reference
the rest were removed in 1dc6f15de9
2017-06-26 09:33:28 -03:00
David McFarland
cdc5cf52c1 cygwin: rebase fixes
- use fixupOutputsHook to find libs in all outputs
- don't rebase symlinks
2017-06-26 09:26:10 -03:00
John Ericson
e0492d58a6 cross stdenv adaptor: Disable checkPhase by default
Before gcc-cross-wrapper did this unconditionally
2017-06-22 17:52:28 -04:00
John Ericson
594d264205 cross stdenv adaptor: Support --host --build --target across the board
Packages get --host and --target by default, but can explicitly request
any subset to be passed as needed. See docs for more info.

rustc: Avoid hash breakage by using the old (ignored)
dontSetConfigureCross when not cross building
2017-06-22 17:52:28 -04:00
John Ericson
9bfd03eff7 cross stdenv adaptor: Remove ccCross binutils attrs and binutils extra buildDepends
It now has the correct wrapped tools and nothing else is needed.
2017-06-22 17:52:28 -04:00
John Ericson
459f1c60f5 cc-wrapper: Learn about target prefixes
This is first step towards getting rid of gcc-wrapper-cross
2017-06-22 17:52:27 -04:00
John Ericson
c4ba2e3ef6 cc-wrapper: Remove stdenv.is* for targetPlatform.is*
Modify bootstrapping stdenvs to make sure `targetPlatform` also passed.
2017-06-22 17:52:27 -04:00
Daiderd Jordan
9e2aba3572
darwin-stdenv: remove python-sphinx and a bunch of other dependencies from the stdenv 2017-06-01 23:23:56 +02:00
Vladimír Čunát
538aa0f808
Merge branch 'staging' (early part)
The comparison looks nice on Hydra.
2017-05-30 16:08:21 +02:00
John Ericson
20e756a093 lib: Consolidate platform configurations (used for crossSystem)
This is good for maintenance and education.
2017-05-29 18:56:03 -04:00
Daiderd Jordan
db4ff1d305 Merge pull request #26091 from LnL7/darwin-cf-10.10
CoreFounation: 10.9 -> 10.10
2017-05-28 20:38:16 +02:00
Daiderd Jordan
779ec14402 Merge pull request #25921 from dtzWill/feature/llvm-manpages
llvm-4/clang-4: Build and install man pages
2017-05-28 20:29:21 +02:00
Daiderd Jordan
077840e528
darwin-stdenv: use darwin.ICU instread of icu 2017-05-27 23:19:49 +02:00
Vladimír Čunát
8004e79415
Merge branch 'master' into staging 2017-05-24 03:24:06 +02:00
John Ericson
eaa509f33a stdenv: Rename isGNU to isHurd as GNU is a userland
Elsewhere, things called GNU indeed includes GNU/Linux or GNU/Hurd, but this
predicate was defined excluding Linux regardless of userland.
2017-05-22 13:55:26 -04:00
Frederik Rietdijk
99cfab07b9 Merge remote-tracking branch 'upstream/master' into HEAD 2017-05-22 09:21:40 +02:00
John Ericson
1dc6f15de9 stdenv: define is* predicates with hostPlatform.is*
This is a saner default until stdenv's are removed altogether
2017-05-22 00:25:02 -04:00
John Ericson
c5c6606048 lib: Infer libc field of platform if not specified
This is especially useful when not cross compiling. It means we can
remove the `stdenv.isGlibc` predicate too.

Additionally, use this to simplify the logic to choose the
appropriate libiconv derivation.
2017-05-22 00:25:02 -04:00
Daiderd Jordan
1e23b49f46
darwin-stdenv: allow clang-unwrapped.man 2017-05-21 12:40:07 +02:00
John Ericson
7e096024d7 glibc: Fix for cross 2017-05-19 18:44:24 -04:00
Vladimír Čunát
e9aeb55f3b
Merge branch 'master' into staging 2017-05-18 11:24:18 +02:00
John Ericson
bfa8f30499 linux cross stdenv: Use the cross stdenv and nativeBuildInputs
This is a cross derivation---it's built on one platform to run on
another---so let's structure it like all the other cross derivations.
2017-05-17 15:33:05 -04:00
John Ericson
5870795cc6 linux cross stdenv: Fix eval errors
`pkgsNoParams` was removed by me, but then #25035 was merged using it,
leading to an unbound identifier.

It would be nice to get travis to do build release-cross.nix or
something to catch these things.
2017-05-17 13:42:51 -04:00
Vladimír Čunát
fa5d543770
Merge branch 'master' into staging 2017-05-06 13:24:28 +02:00
Vladimír Čunát
ef5844be6c
stdenv: disable audit-tmpdir on non-Linux for now
Without changing any hashes.
2017-05-06 13:19:07 +02:00
Dan Peebles
c217f59344 darwin.make-bootstrap-tools.test: fix build breakage
In the extremely unlikely case that our store hash path ends in several
digits (as is the case right now), the Darwin ld will try to interpret
those digits as a version number and barf. To avoid that, we pass in the
SDK version explicitly to stop it from trying to figure it out from iffy
context.
2017-05-05 21:34:28 -04:00
Eelco Dolstra
94d164dd7f
Add a setup hook for detecting $TMPDIR references in RPATHs and wrapper scripts 2017-05-04 20:23:57 +02:00
Domen Kožar
e057e5927e Merge pull request #25427 from aneeshusa/fix-meta-priority-types
Fix meta priority types
2017-05-02 09:38:32 +02:00
Aneesh Agrawal
d3acf9891c stdenv: More useful error message on bad meta attrs
This helps in debugging meta attribute type errors,
which are now enforced as of commit
90b9719f4f.
2017-05-02 01:45:30 -04:00
Dan Peebles
f3a05a0fb3 stdenv: disable checkMeta by default until issues resolved
https://github.com/NixOS/nixpkgs/pull/25304#issuecomment-298385426
2017-05-01 13:51:12 -04:00
Michael Raskin
bcbafdefc4 Merge pull request #25035 from elitak/cross-staging
Add some ARM platforms
2017-05-01 13:30:32 +02:00
Eric Sagnes
7004919243 stdenv-generic: add meta attributes checks 2017-04-29 17:07:01 +09:00
Dan Peebles
90b9719f4f treewide: fix the remaining issues with meta attributes 2017-04-29 04:24:34 +00:00
Dan Peebles
1a4ca220e1 treewide: fix assorted issues revealed by the meta checker
Turns out a couple of the licenses were wrong, as well as being strings.
2017-04-28 23:07:42 -04:00
Dan Peebles
32ae4bfc20 stdenv-generic: add meta attribute checking
This is turned off by default but I think we should fix all packages to
respect it and then turn it on by default
2017-04-28 18:12:18 -04:00
John Ericson
761af14778 Merge pull request #25227 from obsidiansystems/cross-purge-binutilsCross
Purge binutilsCross
2017-04-26 09:09:06 -04:00
John Ericson
295315cc87 binutilsCross: Remove and use binutils instead always
See previous commit for what was done to `binutils` to make this
possible.

There were some uses of `forcedNativePackages` added. The
combination of overrides with that attribute is highly spooky: it's
often important that if an overridden package comes from it, the
replaced arguments for that package come from it. Long term this
package set and all the spookiness should be gone and irrelevant:

  "Move along, nothing to see here!"

No hashes should be changed with this commit
2017-04-25 21:36:19 -04:00
John Ericson
85b4d30c0b binutils: Respect the targetPlatform
Use `buildPackages.binutils` to get build = host != target binutils,
i.e. the old `binutilsCross`, and use
`buildPackages.buildPackages.binutils` to get build = host = target
binutils, i.e. the old `binutils`.

`buildPackages` chains like this are supposed to remove the need for
all such `*Cross` derivations. We start with binutils because it's
comparatively easy.

No hashes of cross-tests should be changed
2017-04-25 21:31:50 -04:00
John Ericson
6cb0f0bcd9 Merge pull request #25225 from Ericson2314/linux-cross-stdenv-eval
linux cross stdenv: Pull platforms from lib to cut eval time
2017-04-25 20:01:46 -04:00
John Ericson
78bb5f5f37 linux cross stdenv: Pull platforms from lib to cut eval time 2017-04-25 19:57:05 -04:00
John Ericson
75441dd64a Merge pull request #25194 from obsidiansystems/host-target-unconfuse
stdenv.cross is a silly attribute that needs to go leaving the well-defined hostPlatform and targetPlatform. This PR doesn't remove it, but changes its definition: before it tracked the target platform which is sometimes more useful for compilers, and now it tracks the host platform which is more useful for everything else. Most usages are libraries, falling in the "everything else" category, so changing the definition makes sense to appease the majority. The few compiler (gcc in particular) uses that exist I remove to use targetPlatform --- preserving correctness and becoming more explicit in the process.

I would also update the documentation aside mentioning stdenv.cross as deprecated, but the definition given actually erroneously assumes this PR is already merged!
2017-04-24 19:07:48 -04:00
John Ericson
af6e4c5b0d Merge pull request #25190 from obsidiansystems/ios-stdenv-tiny-refactor
crossStdenv on iphone: Just get info from `targetPlatform`
2017-04-24 16:57:54 -04:00
John Ericson
49c99b70cf cross-stdenv: Only prune most overrides in the final stage
Before all overrides were also pruned in the previous stage, now
only gcc and binutils are, because they alone care about about the
target platform. The rest of the overrides don't, so it's better to
preserve them in order to avoid spurious rebuilds.
2017-04-24 16:31:53 -04:00
John Ericson
db5a921945 crossStdenv on iphone: Just get info from targetPlatform 2017-04-24 16:12:26 -04:00
Tuomas Tynkkynen
84982c28de Merge remote-tracking branch 'upstream/master' into staging 2017-04-24 15:04:43 +03:00
John Ericson
863d79b364 top-level: Introduce targetPackages and a "double link fold"
Each bootstrapping stage ought to just depend on the previous stage, but
poorly-written compilers break this elegence. This provides an easy-enough
way to depend on the next stage: targetPackages. PLEASE DO NOT USE IT
UNLESS YOU MUST!

I'm hoping someday in a pleasant future I can revert this commit :)
2017-04-23 14:01:12 -04:00
Eric Litak
c3eca1f8dc platforms: add pogoplug4 (armv5tel softfloat) 2017-04-19 14:09:56 -07:00
Eric Litak
3b7395683c platforms: add scaleway-c1 (armv7 sans NEON) 2017-04-19 14:09:56 -07:00
Frederik Rietdijk
e0abe74baf Merge remote-tracking branch 'upstream/master' into HEAD 2017-04-18 11:25:43 +02:00
Tuomas Tynkkynen
28f87e4141 stdenv: ARM bootstrap: Update bootstrap tarballs to latest hydra-built ones
Fixes sandboxed build of glibc. Discussion about a similar failure on aarch64 at:
8bfa9f528c.

Picked from the following cross-trunk evaluation:
http://hydra.nixos.org/eval/1349278 based on nixpkgs
commit 1f32d4b4eb.

armv5tel job: http://hydra.nixos.org/build/51569718
armv6l job: http://hydra.nixos.org/build/51569717
armv7l job: http://hydra.nixos.org/build/51569713
2017-04-15 01:52:33 +03:00
Vladimír Čunát
f7a4f146c9
Merge branch 'master' into staging
This includes a fix for a bad merge.
2017-04-14 19:22:02 +02:00
Vladimír Čunát
f3ceb764e4
Merge #23374: mkDerivation: simplify non-cross builds
Don't pass buildInputs to stdenv builder in nativeBuildInputs.
2017-04-14 11:01:10 +02:00
Tuomas Tynkkynen
1f32d4b4eb make-bootstrap-tools.nix: Fix bzip2
Apparently our native bzip2 builds switched to using dynamic libraries at some point.
2017-04-13 17:22:55 +03:00
Dan Peebles
72d9016b8b darwin.make-bootstrap-tools: fix to use LLVM 4
This should now roughly match the bootstrap tools we're using on Darwin
2017-04-08 16:38:48 -04:00
Dan Peebles
b9e558597d stdenv-darwin: bump to use LLVM 4.0 & new bootstrap tools 2017-04-07 14:36:21 -04:00
Tuomas Tynkkynen
20d9edff17 stdenv: aarch64: Update bootstrap tarballs
Hopefully fixes sandboxed build of glibc on aarch64, as discussed on
8bfa9f528c.

Picked from the following cross-trunk evaluation:
http://hydra.nixos.org/eval/1341395 based on nixpkgs
commit bb3ef8a95c.

build job: http://hydra.nixos.org/build/50125932

(busybox's hash not changing is not a bug!)
2017-03-15 19:17:52 +02:00
Tuomas Tynkkynen
ce56c99edc mkDerivation: Don't pass buildInputs to stdenv builder in nativeBuildInputs
When not cross compiling, nativeBuildInputs and buildInputs have
identical behaviour. Currently that is implemented by having
mkDerivation do a concatenation of those variables in Nix code and pass
that to the builder via the nativeBuildInputs attribute.

However, that has some annoying side effects, like `foo.buildInputs`
evaluating to `[ ]` even if buildInputs were specified in the nix
expression for foo.

Instead, pass buildInputs and nativeBuildInputs in separate variables as
usual, and move the logic of cross compilation vs. native compilation to
the stdenv builder script. This is probably a tiny bit uglier but
fixes the previous problem.

Issue #4855.
2017-03-02 03:26:48 +02:00
Graham Christensen
a9c875fc2e
nixpkgs: allow packages to be marked insecure
If a package's meta has `knownVulnerabilities`, like so:

    stdenv.mkDerivation {
      name = "foobar-1.2.3";

      ...

      meta.knownVulnerabilities = [
        "CVE-0000-00000: remote code execution"
        "CVE-0000-00001: local privilege escalation"
      ];
    }

and a user attempts to install the package, they will be greeted with
a warning indicating that maybe they don't want to install it:

    error: Package ‘foobar-1.2.3’ in ‘...default.nix:20’ is marked as insecure, refusing to evaluate.

    Known issues:

     - CVE-0000-00000: remote code execution
     - CVE-0000-00001: local privilege escalation

    You can install it anyway by whitelisting this package, using the
    following methods:

    a) for `nixos-rebuild` you can add ‘foobar-1.2.3’ to
       `nixpkgs.config.permittedInsecurePackages` in the configuration.nix,
       like so:

         {
           nixpkgs.config.permittedInsecurePackages = [
             "foobar-1.2.3"
           ];
         }

    b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
    ‘foobar-1.2.3’ to `permittedInsecurePackages` in
    ~/.config/nixpkgs/config.nix, like so:

         {
           permittedInsecurePackages = [
             "foobar-1.2.3"
           ];
         }

Adding either of these configurations will permit this specific
version to be installed. A third option also exists:

  NIXPKGS_ALLOW_INSECURE=1 nix-build ...

though I specifically avoided having a global file-based toggle to
disable this check. This way, users don't disable it once in order to
get a single package, and then don't realize future packages are
insecure.
2017-02-24 07:41:05 -05:00
Graham Christensen
59d61ef34a Revert "nixpkgs: allow packages to be marked insecure" 2017-02-23 09:41:42 -05:00
Graham Christensen
38771badd3
nixpkgs: allow packages to be marked insecure
If a package's meta has `knownVulnerabilities`, like so:

    stdenv.mkDerivation {
      name = "foobar-1.2.3";

      ...

      meta.knownVulnerabilities = [
        "CVE-0000-00000: remote code execution"
        "CVE-0000-00001: local privilege escalation"
      ];
    }

and a user attempts to install the package, they will be greeted with
a warning indicating that maybe they don't want to install it:

    error: Package ‘foobar-1.2.3’ in ‘...default.nix:20’ is marked as insecure, refusing to evaluate.

    Known issues:

     - CVE-0000-00000: remote code execution
     - CVE-0000-00001: local privilege escalation

    You can install it anyway by whitelisting this package, using the
    following methods:

    a) for `nixos-rebuild` you can add ‘foobar-1.2.3’ to
       `nixpkgs.config.permittedInsecurePackages` in the configuration.nix,
       like so:

         {
           nixpkgs.config.permittedInsecurePackages = [
             "foobar-1.2.3"
           ];
         }

    b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
    ‘foobar-1.2.3’ to `permittedInsecurePackages` in
    ~/.config/nixpkgs/config.nix, like so:

         {
           permittedInsecurePackages = [
             "foobar-1.2.3"
           ];
         }

Adding either of these configurations will permit this specific
version to be installed. A third option also exists:

  NIXPKGS_ALLOW_INSECURE=1 nix-build ...

though I specifically avoided having a global file-based toggle to
disable this check. This way, users don't disable it once in order to
get a single package, and then don't realize future packages are
insecure.
2017-02-17 20:49:49 -05:00
John Ericson
f6ef6b56fe Merge pull request #22387 from Ericson2314/cross-3-platforms
cross stdenv: let build package's build deps resolve to native packages
2017-02-05 17:41:31 -05:00
John Ericson
5eaea6cee0 cross stdenv: let build package's build deps resolve to native packages
This fixes the "sliding window" principle:
  0. Run packages:       build = native;  host = foreign; target = foreign;
  1. Build packages:     build = native;  host = native;  target = foreign;
  2. Vanilla packages:   build = native;  host = native;  target = native;
  3. Vanilla packages:   build = native;  host = native;  target = native;
  n+3. ...

Each stage's build dependencies are resolved against the previous stage,
and the "foreigns" are shifted accordingly. Vanilla packages alone are
built against themsevles, since there are no more "foreign"s to shift away.

Before, build packages' build dependencies were resolved against
themselves:
  0. Run packages:       build = native;  host = foreign; target = foreign;
  1. Build packages:     build = native;  host = native;  target = foreign;
  2. Build packages:     build = native;  host = native;  target = foreign;
  n+2. ...

This is wrong because that principle is violated by the target
platform staying foreign.

This will change the hashes of many build packages and run packages, but
that is OK. This is an unavoidable cost of fixing cross compiling.

The cross compilation docs have been updated to reflect this fix.
2017-02-05 12:01:53 -05:00
Tuomas Tynkkynen
b2343099ab make-bootstrap-tools{,-cross}.nix: Fix build after binutils changes
Broken after commit 17a344a ("binutils: Add lib output").
2017-02-04 23:39:25 +02:00
Eelco Dolstra
9d6a55aefd
~/.nixpkgs -> ~/.config/nixpkgs
The former is still respected as a fallback for config.nix for
backwards compatibility (but not for overlays because they're a new
feature).
2017-02-01 16:07:55 +01:00
Tuomas Tynkkynen
18599495c4 stdenv: make is64bit evaluate true on aarch64
This should fix the NSS build.
2017-01-29 20:28:14 +02:00
Tuomas Tynkkynen
d1d8ed21b6 stdenv: Aarch64 bootstrap: Update bootstrap tarballs to hydra-built ones
Picked from the following cross-trunk evaluation:
http://hydra.nixos.org/eval/1326772#tabs-still-succeed based on nixpkgs
commit 264d42b9cf.

dist job: http://hydra.nixos.org/build/47094514
2017-01-29 01:34:02 +02:00
Tuomas Tynkkynen
e2a2f6d595 Merge pull request #22117 from dezgeg/aarch64-for-merge
Aarch64 (ARM64) support
2017-01-26 17:52:28 +02:00
John Ericson
c869fe022e top-level: no more need to expose splicedPackages
This was just done temporarily on the last cross-overhauling PR for
testing purposes.
2017-01-25 09:24:55 -05:00
Tuomas Tynkkynen
5c0a385e1c stdenv: Add aarch64 bootstrap files
These are temporary and will be switched to Hydra-build ones once all
the aarch4 changs are merged.
2017-01-25 00:01:53 +02:00
Tuomas Tynkkynen
de3cac0ece make-bootstrap-tools.nix test: Use busybox from store
Our bootstrap tools are actually broken right now due to busybox not
working when invoked directly from a store path. (It says e.g.
"0qqqw19y4gmknajw8vg4fvhx9gxdqlhz-busybox: applet not found").
Make this test actually fail in such case, the next commit will fix the
problem with busybox.
2017-01-25 00:01:52 +02:00
Tuomas Tynkkynen
c909f1b18e stdenv: Add updateAutoconfGnuConfigScriptsHook for aarch64
This is required for Aarch64 since a lot of source tarballs ship with
outdated configure scripts that don't recognize aarch64. Simply
replacing the config.guess and config.sub with new versions from
upstream makes them build again.

This same approach is used by at least Buildroot and Fedora. In
principle this could be enabled for all architectures but
conditionalizing this on aarch64 avoids a mass rebuild on x86.
2017-01-25 00:01:52 +02:00
Tuomas Tynkkynen
7c8a060c09 stdenv: Bringup aarch64 architecture support 2017-01-25 00:01:51 +02:00
John Ericson
bfb147b6a8 top-level: Only splice as needed for performance 2017-01-24 11:37:56 -05:00
John Ericson
a1a798f017 top-level: crossSystem is no longer exposed to packages. Use *Platform. 2017-01-24 11:37:56 -05:00
John Ericson
1c0365bd88 cross-stdenv: Inline useless bindings and reindent
Semantics should be unchanged
2017-01-24 11:37:56 -05:00
John Ericson
92edcb7ebb top-level: Lay the groundwork for {build,host,target}Platform
The long term goal is a big replace:
  { inherit system platform; } => buildPlatform
  crossSystem => hostPlatform
  stdenv.cross => targetPlatform
And additionally making sure each is defined even when not cross compiling.

This commit refactors the bootstrapping code along that vision, but leaves
the old identifiers with their null semantics in place so packages can be
modernized incrementally.
2017-01-24 11:37:56 -05:00
John Ericson
bf17d6dacf top-level: Introduce buildPackages for resolving build-time deps
[N.B., this package also applies to the commits that follow it in the same
PR.]

In most cases, buildPackages = pkgs so things work just as before. For
cross compiling, however, buildPackages is resolved as the previous
bootstrapping stage. This allows us to avoid the mkDerivation hacks cross
compiling currently uses today.

To avoid a massive refactor, callPackage will splice together both package
sets. Again to avoid churn, it uses the old `nativeDrv` vs `crossDrv` to do
so. So now, whether cross compiling or not, packages with get a `nativeDrv`
and `crossDrv`---in the non-cross-compiling case they are simply the same
derivation. This is good because it reduces the divergence between the
cross and non-cross dataflow. See `pkgs/top-level/splice.nix` for a comment
along the lines of the preceding paragraph, and the code that does this
splicing.

Also, `forceNativeDrv` is replaced with `forceNativePackages`. The latter
resolves `pkgs` unless the host platform is different from the build
platform, in which case it resolves to `buildPackages`. Note that the
target platform is not important here---it will not prevent
`forcedNativePackages` from resolving to `pkgs`.

--------

Temporarily, we make preserve some dubious decisions in the name of preserving
hashes:

Most importantly, we don't distinguish between "host" and "target" in the
autoconf sense. This leads to the proliferation of *Cross derivations
currently used. What we ought to is resolve native deps of the cross "build
packages" (build = host != target) package set against the "vanilla
packages" (build = host = target) package set. Instead, "build packages"
uses itself, with (informally) target != build in all cases.

This is wrong because it violates the "sliding window" principle of
bootstrapping stages that shifting the platform triple of one stage to the
left coincides with the next stage's platform triple. Only because we don't
explicitly distinguish between "host" and "target" does it appear that the
"sliding window" principle is preserved--indeed it is over the reductionary
"platform double" of just "build" and "host/target".

Additionally, we build libc, libgcc, etc in the same stage as the compilers
themselves, which is wrong because they are used at runtime, not build
time. Fixing this is somewhat subtle, and the solution and problem will be
better explained in the commit that does fix it.

Commits after this will solve both these issues, at the expense of breaking
cross hashes. Native hashes won't be broken, thankfully.

--------

Did the temporary ugliness pan out? Of the packages that currently build in
`release-cross.nix`, the only ones that have their hash changed are
`*.gcc.crossDrv` and `bootstrapTools.*.coreutilsMinimal`. In both cases I
think it doesn't matter.

 1. GCC when doing a `build = host = target = foreign` build (maximally
    cross), still defines environment variables like `CPATH`[1] with
    packages.  This seems assuredly wrong because whether gcc dynamically
    links those, or the programs built by gcc dynamically link those---I
    have no idea which case is reality---they should be foreign. Therefore,
    in all likelihood, I just made the gcc less broken.

 2. Coreutils (ab)used the old cross-compiling infrastructure to depend on
    a native version of itself. When coreutils was overwritten to be built
    with fewer features, the native version it used would also be
    overwritten because the binding was tight. Now it uses the much looser
    `BuildPackages.coreutils` which is just fine as a richer build dep
    doesn't cause any problems and avoids a rebuild.

So, in conclusion I'd say the conservatism payed off. Onward to actually
raking the muck in the next PR!

[1]: https://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html
2017-01-24 11:37:56 -05:00
Nicolas B. Pierron
da8cf2662a Fix missing overlays argument in stdenv/linux/default.nix 2017-01-16 01:17:33 +01:00
Nicolas B. Pierron
f5dfe78a1e Add overlays mechanism to Nixpkgs.
This patch add a new argument to Nixpkgs default expression named "overlays".

By default, the value of the argument is either taken from the environment variable `NIXPKGS_OVERLAYS`,
or from the directory `~/.nixpkgs/overlays/`.  If the environment variable does not name a valid directory
then this mechanism would fallback on the home directory.  If the home directory does not exists it will
fallback on an empty list of overlays.

The overlays directory should contain the list of extra Nixpkgs stages which would be used to extend the
content of Nixpkgs, with additional set of packages.  The overlays, i-e directory, files, symbolic links
are used in alphabetical order.

The simplest overlay which extends Nixpkgs with nothing looks like:

```nix
self: super: {
}
```

More refined overlays can use `super` as the basis for building new packages, and `self` as a way to query
the final result of the fix-point.

An example of overlay which extends Nixpkgs with a small set of packages can be found at:
  https://github.com/nbp/nixpkgs-mozilla/blob/nixpkgs-overlay/moz-overlay.nix

To use this file, checkout the repository and add a symbolic link to
the `moz-overlay.nix` file in `~/.nixpkgs/overlays` directory.
2017-01-16 01:17:33 +01:00
John Ericson
abaf790ea9 stdenv/booter.nix: Add longer note explaining indexing 2017-01-13 13:47:17 -05:00
John Ericson
ff35560460 linux stdenv: Inline stage funs to conform to new convention
Code is just moved around
2017-01-13 13:23:26 -05:00
John Ericson
b10f415c21 linux stdenv: Remove stray use of stage0 to bootstrap more elegantly 2017-01-13 13:23:26 -05:00
John Ericson
6a45e911c4 linux stdenv: Utilize overrides and prevStage better
`gcc-unwrapped` basically replaces `gccPlain`. It may seem like an ugly
polution to stick it in all-packages, but a future PR will enshrine this
`*-unwrapped` pattern. In any event, the long term goal is stdenvs might
need to tweak how compilers are booted and wrapped, but the code to build
the unwrapped compilers themselves should be generic.
2017-01-13 13:23:25 -05:00
David Grayson
0f33b9f7f1 top-level: Do stdenvOverrides in stage.nix even if crossSystem exists.
Instead, the cross stdenv will patch up the override field -- the complexity
is now confined to the one place it matters.
2017-01-13 13:23:25 -05:00
John Ericson
3e197f7d81 top-level: Normalize stdenv booting
Introduce new abstraction, `stdenv/booter.nix` for composing bootstraping
stages, and use it everywhere for consistency. See that file for more doc.

Stdenvs besides Linux and Darwin are completely refactored to utilize this.
Those two, due to their size and complexity, are minimally edited for
easier reviewing.

No hashes should be changed.
2017-01-13 13:23:23 -05:00
John Ericson
0ef8b69d12 top-level: Modernize stdenv.overrides giving it self and super
Document breaking change in 17.03 release notes
2017-01-13 10:36:11 -05:00
Tuomas Tynkkynen
bde8632106 coreutils: Build with libattr to support xattrs
Fixes #21649
2017-01-05 00:55:35 +02:00
John Ericson
19fbe80c3d top-level: avoid another assert false while we're at it 2016-12-15 17:09:46 -05:00
John Ericson
7960a1b1b8 linux stdenv: Avoid assert false
On one hand, don't want to pass garbage that affects hash, on the other
hand footguns are bad.

Now, factored out the derivation so only need to pass in what is used.
2016-12-15 17:09:29 -05:00
John Ericson
670256dc39 linux stdenv: Rename the bootstrap directory to bootstrap-files 2016-12-15 17:05:39 -05:00
John Ericson
a94af71da7 linux stdenv: Fix assert that broke tests...and hydra 2016-12-15 16:45:08 -05:00
John Ericson
5c6234a7d3 top-level: Allow manually specifying a stdenv, and fix stdenv tests
- The darwin test can now force the use of the freshly-booted darwin stdenv
 - The linux test now passes enough dummy arguments

This may make debugging harder, if so, check out #20889
2016-12-03 17:21:07 -08:00
John Ericson
4751d9e5ad top-level: turn the screw
- Non-cross stdenvs are honest and assert that `crossSystem` is null

 - `crossSystem` is a mandatory argument to top-level/stage.nix, just like
   `system` and `platform`

 - Broken default arguments on stdenvs for testing are gone.

 - All stdenvs (but little-used stdenvNix) take the same arguments for easy
   testing.
2016-12-01 11:24:33 -05:00
John Ericson
39753f5360 top-level: Close over fewer arguments for stdenv stages
This makes the flow of data easier to understand. There's little downside
because the args in question are already inspected by the stdenvs.

cross-compiling in particular is simpler because we don't need to worry
about overriding the config closed over by `allPackages`.
2016-11-30 19:11:03 -05:00
John Ericson
d240a0da1a top-level: Remove cycles: stdenv calls in top-level but not vice versa
This commit changes the dependencies of stdenv, and clean-up the stdenv
story by removing the `defaultStdenv` attribute as well as the `bootStdenv`
parameter.

Before, the final bootstrapping stage's stdenv was provided by
all-packages, which was iterating multiple times over the
top-level/default.nix expression, and non-final bootstrapping stages'
stdenvs were explicitly specified with the `bootStdenv` parameter.

Now, all stages' stdenvs are specified with the `stdenv` parameter.
For non-final bootstrapping stages, this is a small change---basically just
rename the parameter.
For the final stage, top-level/default.nix takes the chosen stdenv and
makes the final stage with it.

`allPackages` is used to make all bootstrapping stages, final and
non-final alike. It's basically the expression of `stage.nix` (along with a
few partially-applied default arguments)

Note, the make-bootstrap-tools scripts are temporarily broken
2016-11-30 19:10:59 -05:00
John Ericson
2df990967b Move up inherit binding for consistency 2016-11-30 19:03:22 -05:00
John Ericson
ddeb0d2d6b top-level: Stop exposing all stdenvs 2016-11-30 19:03:01 -05:00
Domen Kožar
45f579b9e7 allowUnfree: mention the solution that works for nix-shell as well 2016-11-16 15:14:19 +01:00
Shea Levy
7df3d7446f Add initial basic support for cross-compiling to iOS 2016-11-15 16:31:55 -05:00
John Ericson
f68e16f023 top-level: Make cross compiling slightly saner
Removes the weird stdenv cycle used to match the old infrastructure.
It turns out that matching it so precisely is not needed.
2016-11-12 20:20:44 +01:00
John Ericson
6bfe04277f top-level: Make config-overriden stdenv bootstrap more normally 2016-11-06 21:28:38 -08:00
John Ericson
e22346c35e top-level: Make stdenvCross which appears at first glance normal...
...but actually is weird just like the original
2016-11-06 21:27:38 -08:00
Domen Kožar
62edf873aa Merge pull request #18660 from aneeshusa/add-override-attrs
mkDerivation: add overrideAttrs function
2016-10-30 11:32:15 +01:00
John Ericson
d9bc6eb7f0 top-level: Make stdenv/default.nix more concise with inherit (expr) id; syntax 2016-10-27 21:47:46 -07:00
Frederik Rietdijk
7077a270bf Merge remote-tracking branch 'upstream/master' into HEAD 2016-10-26 13:06:43 +02:00
Joachim F
3d5630fac9 Merge pull request #19769 from groxxda/license
stdenv.hasLicense: ? supports nested lookup
2016-10-24 15:19:12 +02:00
Alexander Ried
a0ac2ae35e stdenv: throwEvalHelp performance (#19779) 2016-10-22 20:24:56 +02:00
Frederik Rietdijk
e56832d730 Merge remote-tracking branch 'upstream/master' into HEAD 2016-10-22 17:23:24 +02:00
Alexander Ried
43ce115ca9 stdenv.hasLicense: ? supports nested lookup
this avoids one copy of the attrset
2016-10-22 02:43:13 +02:00
Dan Peebles
e4dba74e8a darwin.stdenv: update MACOSX_DEPLOYMENT_TARGET
No point in claiming we're compatible with a version we don't try to
support, and this will probably help with a few other things elsewhere.
2016-10-16 20:47:43 -04:00
Dan Peebles
6a380c20e0 darwin.make-bootstrap-tools: Fix test target
I'll probably get rid of this eventually since the test-pkgs stuff is
more realistic, but this gives quick feedback for now.
2016-10-15 21:36:51 -04:00
Vladimír Čunát
027efec879 Merge staging without python splitting for now
The split needs more time to finish rebuilding,
but the rest seems OK and there are security fixes.
2016-10-14 09:24:21 +02:00
Vladimír Čunát
e0210d7cbf tested job: fixup evaluation after 5dadc3a
I believe hardcoding system to builtins.currentSystem isn't very good,
as e.g. Hydra surely evaluates on Linux only...
2016-10-14 09:21:38 +02:00
Dan Peebles
5dadc3a30c darwin.make-bootstrap-tools: fix for new stdenv setup 2016-10-13 22:32:19 -04:00
Vladimír Čunát
727fc259d6 Merge branch 'master' into staging 2016-10-13 09:53:16 +02:00
Vladimír Čunát
af38c05587 stdenv stripHash(): fixup after #19324 2016-10-12 23:45:30 +02:00
Dan Peebles
d8a0307a5d [darwin.stdenv] Fix to work on Sierra
This reinstates the libSystem selective symbol export machinery we used
to have, but locks it to the symbols that were present in 10.11 and skips
the actual compiled code we put into that library in favor of the system
initialization code. That should make it more stable and less likely to
do weird stuff than the last time we did this.
2016-10-12 00:08:13 -04:00
Profpatsch
bef6bef0d2
stdenv/stripHash: print to stdout, not to variable
`stripHash` documentation states that it prints out the stripped name to
the stdout, but the function stored the value in `strippedName`
instead.

Basically all usages did something like
`$(stripHash $foo | echo $strippedName)` which is just braindamaged.
Fixed the implementation and all invocations.
2016-10-11 18:34:36 +02:00
Aneesh Agrawal
39b64b52ed mkDerivation: add overrideAttrs function
This is similar to `overrideDerivation`, but overrides the arguments to
`mkDerivation` instead of the underlying `derivation` call.

Also update `makeOverridable` so that uses of `overrideAttrs` can be
followed by `override` and `overrideDerivation`, i.e. they can be
mix-and-matched.
2016-10-02 11:08:34 -04:00
Eelco Dolstra
7a4209c356 Merge remote-tracking branch 'origin/master' into staging 2016-09-20 17:46:09 +02:00
Chris Martin
10f2befa58 stdenv.mkDerivation: add comments w/ manual links (#18707) 2016-09-18 11:20:53 +02:00
Dan Peebles
ff945c5c83 stdenv-darwin: downgrade default LLVM back to 3.7
The 3.8 upgrade was premature and appears to be breaking a load of stuff
that I can't look into right now. Will take it back to 3.8 and test more
thoroughly before pushing more broadly. I should learn to change fewer
variables at a time :)
2016-09-14 01:02:01 -04:00
Dan Peebles
7b9d3f8605 stdenv-darwin: upgrade a few more things
It's a long build and generally painful to split into smaller commits,
so I apologize for lumping many changes into one commit but this is far
easier.

There are still several outdated parts of the darwin stdenv but these
changes should bring us closer to the goal.

Fixes #18461
2016-09-12 01:46:44 -04:00
Nikolay Amiantov
adaee7352b stdenv: leave SSL_CERT_FILE in shells (#15571) 2016-09-01 20:50:08 +02:00
Tuomas Tynkkynen
d3dc3d4130 Merge remote-tracking branch 'dezgeg/shuffle-outputs' into staging
https://github.com/NixOS/nixpkgs/pull/14766
2016-08-30 12:43:37 +03:00
obadz
6389f9b349 Revert "make-bootstrap-tools.darwin: upgrade to llvm 3.8"
This reverts commit bddf4e2180.

Trying to fix stdenvBootstrapTools.x86_64-darwin.stdenv

cc @copumpkin @domenkozar
2016-08-29 16:19:36 +01:00
Tuomas Tynkkynen
5326e85f3d stdenv.mkDerivation: Use chooseDevOutputs 2016-08-29 14:49:51 +03:00
Robin Gloster
e17bc25943
Merge remote-tracking branch 'upstream/master' into staging 2016-08-29 00:24:47 +00:00
Tuomas Tynkkynen
ff9491917f stdenv: Add platforms 2016-08-28 18:04:09 +03:00
obadz
3de6e5be50 Merge branch 'master' into staging
Conflicts:
      pkgs/applications/misc/navit/default.nix
      pkgs/applications/networking/mailreaders/alpine/default.nix
      pkgs/applications/networking/mailreaders/realpine/default.nix
      pkgs/development/compilers/ghc/head.nix
      pkgs/development/libraries/openssl/default.nix
      pkgs/games/liquidwar/default.nix
      pkgs/games/spring/springlobby.nix
      pkgs/os-specific/linux/kernel/perf.nix
      pkgs/servers/sip/freeswitch/default.nix
      pkgs/tools/archivers/cromfs/default.nix
      pkgs/tools/graphics/plotutils/default.nix
2016-08-27 23:54:54 +01:00
Tuomas Tynkkynen
1409bc003b make-bootstrap-tools-cross.nix: Fix gcc build-time options for armv5tel
I broke this in the cleanups I did in 171c7f0, the gcc inside the bootstrap
tarball is not getting built with the correct --with-fpu, --with-float
etc. options.
2016-08-28 00:30:48 +03:00
Eelco Dolstra
8a84fc0217 Tweak error message 2016-08-26 18:58:49 +02:00
Robin Gloster
b6c204f088
stdenv substitute: fail on non-existant input file
fixes #9744
2016-08-26 16:27:36 +00:00
Shea Levy
8b9b9fad31 Revert "Revert "Merge branch 'modprobe-fix' of git://github.com/abbradar/nixpkgs""
Revert a revert of a merge that shouldn't have been in master but was intentionally in staging.

Next time I'll do this right after the revert instead of so far down the line...

This reverts commit 9adad8612b.
2016-08-24 07:35:30 -04:00
obadz
2a552a0888 Partially revert "make-bootstrap-tools.darwin: upgrade to llvm 3.8"
This partially reverts commit bddf4e2180.

cc @copumpkin

Was breaking nix-build <nixpkgs/nixos/release-combined.nix> -A nixpkgs.tarball
See http://hydra.nixos.org/build/38886695/nixlog/2
2016-08-21 15:26:51 +01:00
Dan Peebles
bddf4e2180 make-bootstrap-tools.darwin: upgrade to llvm 3.8
This will break part of the bootstrap tools tests because the new tools
need some changes in the stdenv, but if I change them all at once, the
stdenv breaks with the old bootstrap tools. So I'm doing this first, then
will make changes to the stdenv once this bundle is built and I can use
it.

I also added some functionality to let me test one set of bootstrap tools
on another nixpkgs tree, which makes testing a lot more pleasant.
2016-08-19 23:22:22 -04:00
Shea Levy
9adad8612b Revert "Merge branch 'modprobe-fix' of git://github.com/abbradar/nixpkgs"
Was meant to go into staging, sorry

This reverts commit 57b2d1e9b0, reversing
changes made to 760b2b9048.
2016-08-15 19:05:52 -04:00
Dan Peebles
8f4bbdce99 make-bootstrap-tools: fix on darwin
A couple of things broke after my stdenv change.
2016-08-14 16:20:33 -04:00
Tuomas Tynkkynen
61faa4415e stdenv: ARM bootstrap: Update bootstrap tarballs to hydra-built ones
Picked from the following cross-trunk evaluation:
http://hydra.nixos.org/eval/1283982#tabs-inputs based on nixpkgs
commit 33a1d8080b.

armv5tel job: http://hydra.nixos.org/build/37908177
armv6l job: http://hydra.nixos.org/build/37908176
armv7l job: http://hydra.nixos.org/build/37908175
2016-07-20 18:51:09 +03:00
Tuomas Tynkkynen
3d50989651 make-bootstrap-tools{,-cross}.nix: Create deterministic tars 2016-07-20 02:38:10 +03:00
Tuomas Tynkkynen
53c039b96a make-bootstrap-tools-cross.nix: Support 'dist' target 2016-07-20 02:38:10 +03:00
Tuomas Tynkkynen
5cbb6ac151 make-bootstrap-tools-cross.nix: More syncs from the non-cross version 2016-07-20 02:38:10 +03:00
Tuomas Tynkkynen
954e54f563 make-bootstrap-tools-cross.nix: Reference correct outputs 2016-07-20 02:38:10 +03:00
Tuomas Tynkkynen
0968641eda make-bootstrap-tools-cross.nix: Use busybox with musl
Do this because I cannot figure out how to refer to `glibc.static`
inside busybox when cross building in a manner that works.
2016-07-20 02:38:10 +03:00
Tuomas Tynkkynen
171c7f0e63 make-bootstrap-tools-cross.nix: Have more consistency with platforms.nix
E.g. we had `arch = "arm"` in the former and `arch = "armv6"` in the
latter. Try to have some more consistency.
2016-07-20 02:38:10 +03:00
Tuomas Tynkkynen
9f0dc294da make-bootstrap-tools-cross.nix: Add/fix some .so dependencies
- cloog, ppl, cloogppl aren't used by recent GCCs. Kill references to them.
- Use correct versions of isl, as the current GCC depends
  on non-default versions of them.
- Also clarify isl dynamic libraries are needed in cross
  builds, but not in native builds
- Since aeb3d8c (bzip2: fix cross build on mingw by using autoconf patch),
  it seems that the bzip2 binary depends on libbz2 when cross compiling.
  So copy libbz2 into the bootstrap tarball as well.
- Curl isn't used in the bootstrap tools since e6f61b4cf3.
2016-07-20 02:38:10 +03:00
Tuomas Tynkkynen
8d62f62a78 stdenv: Remove unpack-bootstrap-tools-arm.sh
This file was using the 'double patchelf' hack, which hasn't been needed
for a while, after the original patchelf bug was fixed:

65a4dc6aa9
2016-07-20 02:38:10 +03:00
Tuomas Tynkkynen
0cdf892769 darwin make-bootstrap-tools: Document reason for special coreutils
Discussion:
https://github.com/NixOS/nixpkgs/commit/ac4958efc33322c9
2016-07-09 17:23:43 +03:00
Tuomas Tynkkynen
644d4e9ac9 Revert "Revert "make-bootstrap-tools: try again to fix on Darwin""
This reverts commit debd401b0f.

We must not use a single-binary build for the bootstrap since the common
binary gains a dynamic linkage to gmp (due to 'factor' and 'expr'
handling arbitrary-precision arithmetic).
2016-07-09 17:19:18 +03:00
Vladimír Čunát
1e2d0205a8 make-bootstrap-tools: try again to fix on Darwin
For discussion see:
https://github.com/NixOS/nixpkgs/commit/ac4958efc33322c9
2016-07-03 16:34:47 +02:00
Vladimír Čunát
debd401b0f Revert "make-bootstrap-tools: try again to fix on Darwin"
This reverts commit ac4958efc3.
2016-07-03 16:34:46 +02:00
Vladimír Čunát
ac4958efc3 make-bootstrap-tools: try again to fix on Darwin
I've got no way to test, so just trying a likely safer way blindly.
2016-07-03 14:21:55 +02:00
Vladimír Čunát
f4792cdc0c make-bootstrap-tools*: fixup after #16406
Our coreutils now uses single-binary-build mode where, by default,
simple shebang scripts are used for all the binaries. That doesn't work
e.g. with the Linux unpacker which only handles standard binaries and
symlinks. Let's use the symlinked mode instead for boostrapping.
This does NOT change any stdenv hashes.

I only tested the case most important to me:
$ nix-build pkgs/top-level/release.nix -A stdenvBootstrapTools.x86_64-linux.test
2016-06-28 09:54:22 +02:00
Vladimír Čunát
6b27ceb006 Merge 'master' into staging and re-revert merge
... from staging to master, reverted temporarily in aa9a04883e.
2016-06-23 12:09:03 +02:00
Vladimír Čunát
aa9a04883e Revert "Merge branch 'staging'" due to glibc
The main output started to retain dependency on bootstrap-tools; see
https://github.com/NixOS/nixpkgs/pull/15867#issuecomment-227949096

This reverts commit c05d829598, reversing
changes made to f073df60d6.
2016-06-23 09:25:10 +02:00
Vladimír Čunát
c05d829598 Merge branch 'staging' 2016-06-22 10:49:56 +02:00
John Ericson
5b7bdb2861 Remove more dead code: stdenvDarwinNaked no longer exists 2016-06-20 23:53:09 -07:00
John Ericson
47d95a7571 Remove dead code in stdenvCross adaptor 2016-06-20 23:53:09 -07:00
Tuomas Tynkkynen
2d3d105b02 make-bootstrap-tools-cross.nix: Strip extra whitespace 2016-06-10 16:17:30 +03:00
Domen Kožar
7a5b85cdda pkgs.runCommand: passAsFile (buildCommand can be very long)
Close #15803. This avoids the error:

while setting up the build environment: executing
‘/nix/store/7sb42axk5lrxqz45nldrb2pchlys14s1-bash-4.3-p42/bin/bash’:
Argument list too long

Note: I wanted to make it optional based on buildCommand length,
but that seems pointless as I'm sure it's less performant.

Amended by vcunat:
https://github.com/NixOS/nixpkgs/pull/15803#issuecomment-224841225
2016-06-10 10:49:26 +02:00
Joachim Fasting
6648b04381
stdenv: fix paxmark
On Linux, paxctl's setup hook should overwrite the paxmark stub, but the
stub is defined after the setup hooks are sourced, so the stub ends up
overwriting the real function.  The result is that paxmark fails to do
anything.  The fix is to define the stub before any setup hooks are
sourced.  Thanks to @vcunat for figuring this out.

Closes #15492
2016-05-27 18:57:59 +02:00
Lluís Batlle i Rossell
f6f5c185ca Allow not rebasing in cygwin stdenv.
This changes cygwin stdenv, but I don't think it will hurt much people.

This allows mkDerivation to get "dontRebase=true" to skip the usual cygwin
rebase. This is required, if we are using this stdenv to build DLLs for win32
inside x86_64-cygwin, because /bin/rebase crashes at finding an arch mismatch.
Additionally, we don't need any rebase for libraries built by visual studio and
meant for visual studio (my use case).

I'm using nix in x86_64-cygwin to build libraries with visual studio, both for
x86_64 and x86.
2016-05-20 17:17:10 +02:00
Vladimír Čunát
81df035429 stdenv setup.sh: revert most of changes around #14907
I'm giving this up. Feel free to find some reasonable variant that works
at least on Linux and Darwin. Problems encountered:
- During bootstrap of Darwin stdenv `env -0` and some bash features
  don't work.
- Without `env -0` the contents of some multi-line phases is taken as
  variable declarations, which wouldn't typically matter, but the PR
  wanted to refuse bash-invalid names which would be occasionally
  triggered. This commit dowgrades that to a warning with explanation.
2016-05-12 04:53:37 +02:00
Vladimír Čunát
62fc8859c1 stdenv substituteAll: use yet another implementation
It turned out that process substitution fed into a while-cycle
isn't recognized during darwin bootstrap:
http://hydra.nixos.org/build/35382446/nixlog/1/raw

Also fix broken NIX_DEBUG output, noticed by abbradar.
2016-05-08 19:41:50 +02:00
Vladimír Čunát
9e0d0423fe stdenv substituteAll: use more robust code
The set/env fix in #14907 wasn't very good, so let's use a null-delimited
approach. Suggested by Aszlig.
In particular, this should fix a mass-breakage on Darwin, though I was
unable to test that.
2016-05-07 11:23:30 +02:00
Vladimír Čunát
1dc36904d8 Merge #14920: windows improvements, mainly mingw 2016-05-05 08:30:19 +02:00
Nikolay Amiantov
62616ec5e2 Merge commit 'refs/pull/14907/head' of git://github.com/NixOS/nixpkgs into staging 2016-04-25 18:02:47 +03:00
Nikolay Amiantov
d4794c3630 stdenv: clarify how outputsToInstall is chosen
See https://github.com/NixOS/nixpkgs/pull/14694/files#r60013871
2016-04-25 13:24:39 +03:00
Nikolay Amiantov
5ff40ddedf add get* helper functions and mass-replace manual outputs search with them 2016-04-25 13:24:39 +03:00
Profpatsch
a2d38bc7fc doc/stdenv.xml document substitution env variables
The filtering of environment variables that start with an uppercase
letter is documented in the manual.
2016-04-23 21:41:35 +02:00
Profpatsch
77fa336849 setup.hs: substitute uses only valid bash names
bash variable names may only contain alphanumeric ASCII-symbols and _,
and must not start with a number. Nix expression attribute names however
might contain nearly every character (in particular spaces and dashes).

Previously, a substitution that was not a valid bash name would be
expanded to an empty string. This commit introduce a check that throws
a (hopefully) helpful error when a wrong name is used in a substitution.
2016-04-23 17:54:32 +02:00
Vladimír Čunát
8b292a1b35 stdenv on mingw: fix 64-bin DLL detection in some cases
In particular, this makes 64-bit libpng create DLL now,
and thus depending packages won't fail anymore (e.g. freetype).
2016-04-23 10:52:01 +02:00
Vladimír Čunát
6e7787e666 stdenv for windows: auto-link dependency DLLs
For every *.{exe,dll} in $output/bin/ we try to find all (potential)
transitive dependencies and symlink those DLLs into $output/bin
so they are found on invocation.
(DLLs are first searched in the directory of the running exe file.)

The links are relative, so relocating whole /nix/store won't break them.
The hook is activated on cygwin and when cross-compiling to mingw.
2016-04-23 10:52:00 +02:00
Alexander Ried
c6952ece24 bootstrap-tools-linux: Use tar without acl 2016-04-18 00:02:44 +02:00
Alexander Ried
28780a4dad bootstrap-tools-linux: Qualify all multi-output packages 2016-04-18 00:01:44 +02:00
Dan Peebles
d61a0e5c6f stdenv-darwin: fix build
Fixes #14704
2016-04-16 18:21:32 -04:00
Vladimír Čunát
30f14243c3 Merge branch 'master' into closure-size
Comparison to master evaluations on Hydra:
  - 1255515 for nixos
  - 1255502 for nixpkgs
2016-04-10 11:17:52 +02:00
Vladimír Čunát
710573ce6d Merge #12653: rework default outputs 2016-04-07 16:00:09 +02:00
Vladimír Čunát
2995439003 buildEnv: respect meta.outputsToInstall
As a result `systemPackages` now also respect it.
Only nix-env remains and that has a PR filed:
    https://github.com/NixOS/nix/pull/815
2016-04-07 15:59:44 +02:00
Vladimír Čunát
3342f717da stdenv: set meta.outputsToInstall unless overridden 2016-04-07 15:59:43 +02:00