Commit Graph

2479 Commits

Author SHA1 Message Date
Michael Weiss
018a5ae2f4 fetchRepoProject: Fetch into $out and make it deterministic
Fetch into $out and remove all version control files to make it
deterministic (.repo and all .git subdirectories - e.g. the .git/index
files change every time).

Additionally I've changed the default of "useArchive" to false because
fetching with "--archive" will fail for some projects (e.g.
"platform/external/iosched" from the AOSP).

Now, this function should hopefully work for every tag of the AOSP.
2017-09-17 23:16:33 +02:00
Vladimír Čunát
4ca45f229b
set-source-date-epoch-to-latest.sh: shut up a warning
> bash: warning: command substitution: ignored null byte in input
/cc #28227.  Also break the overlong line.
2017-09-17 10:35:44 +02:00
Antoine Eiche
01174c5f4d dockerTools.pullImage: use skopeo to pull the image
Before this patch, a VM was used to spawn docker that pulled the
VM. Now, the tool Skopeo does this job well so we can simplify our
dockerTools since we doesn't need Docker anymore:)

This also fixe the regression described in
https://github.com/NixOS/nixpkgs/issues/29271 : cntlm proxy doesn't
work in 17.09 while it worked in 17.03.

Note Skopeo doesn't produce the same output than docker pull so, we
have to update sha.
2017-09-17 08:26:02 +01:00
Michael Weiss
337380ea1d gitRepo: Fix an error due to missing TLS certificates
This was a problem when run inside a sandbox, e.g. via
"fetchRepoProject". The error message from repo seems unrelated:

fatal: Cannot get https://gerrit.googlesource.com/git-repo/clone.bundle
fatal: error no host given

But the exception is actually thrown due to missing certificates
(/etc/ssl/certs). It should be possible to provide another location via
environment variables (e.g. SSL_CERT_FILE, REQUESTS_CA_BUNDLE or
CURL_CA_BUNDLE) but apparently that doesn't actually work for some
reason (would have to study our Python packaging).

Now "fetchRepoProject" works without the "--no-clone-bundle" option.
2017-09-16 22:13:56 +02:00
Michael Weiss
62b9d78b18 fetchRepoProject: Fix the GnuPG verification
The verification was failing with the following error:
gpg: keyblock resource '/tmp/nix-build-XYZ.drv-0/.repo/repo/./.repoconfig/gnupg/pubring.kbx': No such file or directory

Using an absolute path for $HOME fixes this.

And since 175ecbab91 the dependencies on
"git" and "gnupg" aren't required anymore as "gitRepo" already covers
them.
2017-09-16 17:57:14 +02:00
Michael Weiss
50ce8abccf fetchRepoProject: Refactor the code
Should hopefully make it a bit more readable and less redundant.
2017-09-16 17:25:25 +02:00
Frederik Rietdijk
03fa6965ad Merge remote-tracking branch 'upstream/master' into HEAD 2017-09-16 12:34:32 +02:00
John Ericson
12452178e8 Merge pull request #29381 from obsidiansystems/autoreconf-hook
autoreconfHook: Simplify by avoiding `findInputs`
2017-09-14 13:16:53 -04:00
John Ericson
d242978671 autoreconfHook: Simplify by avoiding findInputs
`findInputs` is a stdenv/setup helper we should strive not to call
elsewhere. Using normal deps is more idiomatic anyways.
2017-09-14 13:16:12 -04:00
Frederik Rietdijk
87f757e834 Merge pull request #27780 from tilpner/overridable-buildenv
Make buildEnv overridable
2017-09-14 17:26:33 +02:00
Domen Kožar
f49b7d3c88
vm: remove trusty-updates hash as it changes too often 2017-09-14 11:29:10 +02:00
Jörg Thalheim
9e7db9a9f9 Merge pull request #28963 from nlewo/docker-readiness
dockerTools.pullImage: change the docker deamon readiness mechanism
2017-09-13 10:39:16 +01:00
Frederik Rietdijk
628b6c0e9d Merge remote-tracking branch 'upstream/master' into HEAD 2017-09-11 22:52:53 +02:00
Domen Kožar
c7a152e5d0
vm: bump trusty-updates hash
In future we might just not use the updates if hash changes too
often.
2017-09-11 13:02:19 +02:00
Eelco Dolstra
ec8d41f08c
Revert "Merge pull request #28557 from obsidiansystems/binutils-wrapper"
This reverts commit 0a944b345e, reversing
changes made to 61733ed6cc.

I dislike these massive stdenv changes with unclear motivation,
especially when they involve gratuitous mass renames like NIX_CC ->
NIX_BINUTILS. The previous such rename (NIX_GCC -> NIX_CC) caused
months of pain, so let's not do that again.
2017-09-07 12:51:21 +02:00
John Ericson
eb326c9cb7 macos-sierra-shared: Fix, after binutils-wrapper broke it
cctool's as needs to be told use to use gnu as, or else we'd need a
dependency cycle between cctools and clang for this case.

In general, this is not a problem because clang uses its own integrated
assembler where possible, and gnu as otherwise.
2017-09-06 14:28:14 -04:00
John Ericson
3601a97e3c binutils-wrapper: Accidentally deleted macOS Sierra reexport hack
Did this when spliting off binutils-wrapper from cc-wrapper in
40e9b2a7e6: I deleted the file instead of
moving it.
2017-09-04 11:26:41 -04:00
Antoine Eiche
132e790735 dockerTools.pullImage: change the docker deamon readiness mechanism
To wait for the docker deamon, curl requests are sent. However, if a
http proxy is set, it will respond instead of the docker daemon.
To avoid this, we send docker ps command instead of curl command.
2017-09-04 10:52:16 +02:00
John Ericson
0a944b345e Merge pull request #28557 from obsidiansystems/binutils-wrapper
Binutils-wrapper: Init by refactoring out of cc-wrapper
2017-09-03 10:37:27 -04:00
Vladimír Čunát
51d6d27e90
Merge #28227: set-source-date-epoch-to-latest.sh: ignore generated files 2017-09-02 21:36:34 +02:00
Frederik Rietdijk
d0dab8a330 Merge remote-tracking branch 'upstream/master' into HEAD 2017-09-02 11:10:52 +02:00
John Ericson
dbf6d20d64 binutils-wrapper: Import separately from cc-wrapper 2017-09-01 11:44:56 -04:00
John Ericson
40e9b2a7e6 binutils-wrapper: Init
Factor a binutils wrapper out of cc-wrapper. While only LD is wrapped,
the setup hook defines environment variables on behalf of other
utilites.
2017-09-01 11:44:55 -04:00
John Ericson
fbb7d335db cc-wrapper: Use separate mangler for "bool" variables
This avoids any `NIX_FOOBAR=1 1` not triggering conditions.
2017-09-01 11:44:54 -04:00
John Ericson
1f5807d760 cc-wrapper: Pull variable mangler into utils.sh
In preparation for splitting out binutils-wrapper
2017-09-01 11:44:54 -04:00
John Ericson
94c0267fc1 cc-wrapper: Clean up dynamic linking with x86 multilib
It's better layering to do everything in ld-wrapper.
2017-09-01 11:44:54 -04:00
John Ericson
3d3a6e0fac cc-wrapper: Remove support for NIX_LDFLAGS_HARDEN
It has long been deprecated
2017-08-31 13:54:32 -04:00
John Ericson
46fd4bcb14 cc-wrapper: Remove {START,EXEC}_HOOK
These are no longer used by anything
2017-08-31 13:54:07 -04:00
Frederik Rietdijk
6d4bd78fad Merge commit '2858c41' into HEAD 2017-08-30 21:07:07 +02:00
John Ericson
97a48835b7 mkDerivation, cc-wrapper: Check hardening flag validity in Nix
This becomes necessary if more wrappers besides cc-wrapper start
supporting hardening flags. Also good to make the warning into an
error.

Also ensure interface is being used right: Not as a string, not just in
bash.
2017-08-30 17:53:42 +02:00
John Ericson
822a8d0148 cc-wrapper: Remove redundant hardening
GCC just passes `-z ...` flags to ld unaltered, and they are already
passed to LD anyways. On the other hand, `-pie` affects gcc behavior
too.
2017-08-30 17:53:42 +02:00
Antoine Eiche
d4b0883ad2 vm: Add trusty-updates to the packages list 2017-08-30 02:18:56 +02:00
Daiderd Jordan
5a28fd660a
darwin-frameworks: move fixup setup-hook 2017-08-28 23:25:11 +02:00
Daiderd Jordan
92652b4d79
darwin-CF: use @rpath for library id and add an rpath entry for CF based on NIX_COREFOUNDATION_RPATH 2017-08-28 23:24:58 +02:00
John Ericson
42e639066b Merge pull request #28556 from obsidiansystems/cc-wrapper-nix-cleanup
cc-wrapper: Cleanup of Nix
2017-08-26 17:48:34 -04:00
John Ericson
df7c305c4c cc-wrapper: Leverage the setup script instead of buildCommand 2017-08-25 15:10:05 -04:00
John Ericson
dc47e763c1 cc-wrapper: Use set -u for better maintainability 2017-08-25 15:10:05 -04:00
John Ericson
a470be5a16 expand-response-params: Build more normally 2017-08-25 15:10:02 -04:00
Frederik Rietdijk
656e14f64f Merge remote-tracking branch 'upstream/python-wip' into HEAD 2017-08-25 19:37:57 +02:00
John Ericson
287fce6402 expand-response-params: Pull out of cc-wrapper
No hashes were changed
2017-08-25 11:21:09 -04:00
Daniel Peebles
4f7f48fb1a Merge pull request #28521 from obsidiansystems/cc-wrapper-dead-code
cc-wrapper: Remove dead code
2017-08-24 16:05:51 -04:00
volth
0d80fabbf9 mirrors: update apache and cpan mirrors
removed dead mirrors, added new
2017-08-24 04:22:07 +00:00
Tuomas Tynkkynen
0c0fad6141 treewide: Consistently call ARM 'arm'
No need for silly differences.
2017-08-24 01:17:01 +03:00
John Ericson
08f3a60e7b cc-wrapper: Remove dead code
ccPath is only defined below, so this condition would never be true.

Worse, that's not quite true: what if somebody happend to have `/clang`
and no sandboxing. Boy, wouldn't that be annoying to debug!
2017-08-23 17:09:47 -04:00
John Ericson
0135e61b4c Merge remote-tracking branch 'upstream/master' into staging
That way the tarball job succeeds
2017-08-21 18:42:57 -04:00
John Ericson
c035711072 cc-wrapper: Remove unused params
Ensured hashes unchanged and eval succeeds in tarball job
2017-08-21 18:40:41 -04:00
Bojan Nikolic
ad8d5649a2 set +u workaround for empty arrays
libDirs can be empty, which in combination with "set -u" of
9f1e009975 will cause a variable unbound
error on old bash versions
2017-08-21 12:27:26 +01:00
Daiderd Jordan
5d3527c148
cc-wrapper: don't export CPP
CPP defaults to $(CC) -E and setting it explicitly seems to cause
issues in a number of packages like gcc and gdb.

https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
2017-08-16 21:38:32 +02:00
John Ericson
3c9cf282bb cc-wrapper: Improve set -u compliance 2017-08-14 14:44:32 -04:00
Eelco Dolstra
9eb901b70d
Merge remote-tracking branch 'origin/gcc-6' into staging 2017-08-14 11:38:00 +02:00
Frederik Rietdijk
62dac1bdd9 Merge remote-tracking branch 'upstream/master' into HEAD 2017-08-14 09:34:10 +02:00
Frederik Rietdijk
13bbaee21d Merge pull request #27881 from mimadrid/fix/http-https
Update homepage attributes: http -> https
2017-08-13 21:53:20 +02:00
volth
4ebeb7d782 use \0 instead of \n 2017-08-13 14:03:05 +00:00
volth
b971f8de8e set-source-date-epoch-to-latest.sh: ignore generated files
set-source-date-epoch-to-latest.sh to ignore files newer than "$NIX_BUILD_TOP/.." (unlike "$NIX_BUILD_TOP" it is root-owned and cannot be touched by nixbld1).
2017-08-13 09:33:09 +00:00
Vladimír Čunát
6899c7fdb9
Merge branch 'master' into gcc-6 2017-08-12 10:09:41 +02:00
Tuomas Tynkkynen
58834f727f makeWrapper: use consistent style
All the other plain variable references (before e1d46c0c4) are without
braces.
2017-08-11 21:32:55 +03:00
Tuomas Tynkkynen
aeeb23853a stdenv: Don't move info pages to the 'man' output
Because if you were to disable man pages via 'programs.man.enable',
the "man" output wouldn't be installed and the info pages would go
missing.
2017-08-11 21:32:55 +03:00
Tuomas Tynkkynen
e4876d6e43 stdenv: Don't move man or info pages to $doc
Because .doc is normally not installed to user environments, these
manpages are unaccessible by default otherwise.
2017-08-11 21:32:54 +03:00
Domen Kožar
486e1c3c16 Merge pull request #27998 from davidak/macOS
replace "Mac OS X" and "OS X" with "macOS"
2017-08-11 13:01:36 +02:00
Daiderd Jordan
bf8c125170
cc-wrapper: fix LD_DYLD_PATH on darwin
Having multiple compilers in the build environment would result in an
invalid LD_DYLD_PATH like /usr/lib/dyld/usr/lib/dyld.
Since the path is hardcoded in XNU it can't be anything but
/usr/lib/dyld anyway.
2017-08-10 00:22:58 +02:00
Orivej Desh
771f28bac8 cc-wrapper: Fix support for qtbase-setup-hook
Revert https://github.com/NixOS/nixpkgs/pull/27657#issuecomment-318161946
due to https://github.com/NixOS/nixpkgs/pull/28021#issuecomment-321045542
2017-08-09 18:17:46 +02:00
Frederik Rietdijk
0ff782e141 makeWrapper: fix regression introduced in #24944
In some cases wrappers could not be made. See e.g.
https://github.com/NixOS/nixpkgs/pull/24944#issuecomment-321175692
2017-08-09 12:18:47 +02:00
Peter Simons
67f37b70e8 Merge pull request #24944 from ahmedtd/make-makewrapper-picky
makeWrapper: Fail loudly when misused
2017-08-08 17:26:38 +02:00
Orivej Desh
a6e7dffd1f cc-wrapper: Fix standalone ld 2017-08-08 11:03:51 +02:00
Orivej Desh
c8e9dcc8a4 cc-wrapper: Fix standalone gcc
This ensures that all salted variables are defined even if the wrapped program
is invoked outside nix-build environment.
2017-08-08 11:03:51 +02:00
Orivej Desh
c8f7f18e69 cc-wrapper: Fix adding directories to rpath
This fixes a bug introduced in #27831: `for path in "$dir"/lib*.so` assumed that
all libs match `lib*.so`, but 07674788d6 started
adding libs that match `*.so` and `*.so.*`.
2017-08-08 11:03:51 +02:00
Taahir Ahmed
e1d46c0c4d makeWrapper: Only wrap normal executable files
`makeWrapper` and `wrapProgram` are being invoked on all kinds of
wacky things (usually with the help of bash globs or other machine
assistance).

So far, I have come across `wrapProgram` being invoked on a directory,
as well as on the empty string.

As far as I can tell, it's only valid to invoke these utilities on a
normal (non-directory, non-device) executable file.  This commit
enforces that precondition.
2017-08-08 03:46:47 -05:00
Robin Gloster
3e981b9e33
Revert "cc-wrapper: fix set -u errors"
This reverts commit 67a41eafe9.

see #28021 for discussion
2017-08-08 10:20:00 +02:00
Robin Gloster
67a41eafe9
cc-wrapper: fix set -u errors
cc @Ericson2314
2017-08-08 10:15:34 +02:00
Peter Simons
b196230e71 Merge pull request #25185 from ahmedtd/fix-makewrapper-unsupported-args
Fix makewrapper unsupported args
2017-08-08 08:33:56 +02:00
Taahir Ahmed
ece5387b09 makeWrapper: Die on unsupported arguments
Previously, makeWrapper would accept arguments it didn't recognize,
potentially allowing argument misspellings or broken callers.

Now, makeWrapper dies with a backtrace if it is called incorrectly.

Also changes `wrapProgram` so that it doesn't pass through the first
argument twice --- this was tripping up the argument checking.
2017-08-08 00:18:12 -05:00
Taahir Ahmed
12354b8eb5 dieHook: Add die utility function
Calling `die "Error message"` causes the current script to exit with
an error, printing a backtrace
2017-08-08 00:17:56 -05:00
davidak
3270aa896b replace "Mac OS X" and "OS X" with "macOS"
as it is the official name since 2016

https://en.wikipedia.org/wiki/Macintosh_operating_systems#Desktop

exception are parts refering to older versions of macOS like

"GUI support for Mac OS X 10.6 - 10.12. Note that Emacs 23 and later [...]"
2017-08-07 21:41:30 +02:00
John Ericson
810fb0c968 cc-wrapper: Don't slurp NIX_DONT_SET_RPATH twice
Otherwise we end with a definition like `1 1`, which doesn't trigger the right
conditional.
2017-08-07 11:09:25 -04:00
John Ericson
2493454e13 cc-wrapper: Use set -u throughout
Now is an opportune time to do this, as the infixSalt conversion in
`add-flags.sh` ensures that all the relevant `NIX_*` vars will be
defined even if empty.
2017-08-07 03:05:51 -04:00
John Ericson
42f35503b5 cc-wrapper: Make hygienic
See the added comments for what exactly has been done.
2017-08-07 03:05:50 -04:00
John Ericson
9f1e009975 cc-wrapper: Unconditionally use @infixSalt@ accross the board
This is basically a sed job, in preparation of the next commit. The
rules are more or less:

  - s"NIX_(.._WRAPPER_)?([a-zA-Z0-9@]*)"NIX_\1@infixSalt@_\2"g

  - except for non-cc-wrapper-specific vars like `NIX_DEBUG`
2017-08-07 03:05:50 -04:00
John Ericson
5ba3972add cc-wrapper: Stop preWrapping
This is an ugly temp hack for cross compilation, but now we have something better on the way.

Bind `infixSalt` as an environment variable as it will be used in it.
2017-08-07 03:05:50 -04:00
Robin Gloster
145be4e340
Merge commit 'ad83979e5986fa3bb0d254c2be9482c12a8743b9' from staging 2017-08-06 11:21:53 +02:00
Vladimír Čunát
f779df87b1
Merge branch 'master' into staging
Let's drop i686-linux here as well.
2017-08-06 10:03:25 +02:00
Frederik Rietdijk
8fcb070363 Merge pull request #27320 from jtojnar/fix/unbreak-polari
polari: unbreak
2017-08-05 17:49:47 +02:00
Jan Tojnar
0a142d3112
wrap-gapps-hook: also wrap symlinks to executables
Some programs store the executable in a different place and link it
from the `bin` directory. For example, Polari links `$out/bin/polari`
to `$out/share/polari/org.gnome.Polari`. `wrapGAppsHook` did not follow
symlinks so it was not able to wrap Polari, making it unable to access
GObject introspection definitions required for running the program.

I made the wrapping script follow symlinks to fix this corner case.
2017-08-05 13:23:05 +02:00
John Ericson
fdd07f62e8 Merge pull request #27879 from obsidiansystems/cc-wrapper-shellcheck
cc-wrapper: Pass shellcheck and other cleanups
2017-08-04 14:19:56 -04:00
John Ericson
6463fd3d7e cc-wrapper: Pass shellcheck and other cleanups
In many cases, this involved taking @orivej's and @edolstra's recent
ld-wrapper improvements, and applying then elsewhere.
2017-08-04 12:47:29 -04:00
Michael Alan Dorman
ef5ba4d5b2 Fix emacs wrapper expression to honor bash array use
In 8d76eff, @Ericson2314 changed the representation of the value that
`findInputs` generated from a whitespace-separated bunch strings to an
actual array of strings.

Expressions that *consume* that value, however, also needed to be
changed to iterate over all the contents of the array, else they would
only select the first value, which turns out to be somewhat limiting.

Fixes #27873
2017-08-04 07:52:18 -04:00
Robin Gloster
2d64d1e601
fetchurlBoot: add name attr 2017-08-04 08:22:04 +02:00
John Ericson
8ba6012dfd Merge PR #27536 2017-08-03 17:25:30 -04:00
John Ericson
cd6c452ade cc-wrapper: Don't hardcode MacOS min version for delegates 2017-08-03 16:37:27 -04:00
John Ericson
2e61d8b71c cc-wrapper: Handle more -l.... gotchas 2017-08-03 16:37:27 -04:00
John Ericson
f1d48ca6ba Merge pull request #27859 from adisbladis/revert_27427
Revert "buildenv: read propagated-user-env-packages line-by-line"
2017-08-03 11:17:34 -04:00
mimadrid
09e0cc7cc7
Update homepage attributes: http -> https
Homepage link "http://.../" is a permanent redirect to "https://.../" and should be updated
https://repology.org/repository/nix_stable/problems
2017-08-03 11:56:15 +02:00
Mathias Schreck
86d9b09c9b dockerTools: fix image json and manifest
The image json is not exactly the same as the layer json, therefore I
changed the implementation to use the `baseJson` which doesn’t include
layer specific details like `id`, `size` or the checksum of the layer.

Also the `history` entry was missing in the image json. I’m not totally
sure if this field is required, but a I got an error from a docker
registry when I’ve tried to receive the distribution manifest of an
image without those `history` entry:

GET: `http://<registry-host>/v2/<imageName>/manifests/<imageTag>`

```json
{
  "errors": [
    {
      "code": "MANIFEST_INVALID",
      "message": "manifest invalid",
      "detail": {}
    }
  ]
}
```

I’ve also used a while loop to iterate over all layers which should make
sure that the order of the layers is correct. Previously `find` was
used and I’m not sure if the order was always correct.
2017-08-03 11:52:03 +02:00
tilpner
77f1c7d5bc
buildenv: Make buildEnv overridable
callPackage already calls makeOverridable, but that just
makes the function that evaluates to buildEnv overridable,
not buildEnv itself.
If no overridable version of buildEnv is used during construction,
users can't override e.g. `paths` at all
2017-08-02 11:56:37 +02:00
Sergey Mironov
f49c2fbf7a
trivial-builders.nix: add writeShellScriptBin builder 2017-08-01 20:23:08 -04:00
Frederik Rietdijk
f74759450b Merge remote-tracking branch 'upstream/master' into HEAD 2017-08-01 16:28:48 +02:00
Orivej Desh
07674788d6 Respect NIX_DONT_SET_RPATH when .so appears in the command line
Unified processing of command line arguments in ld-wrapper broke support for
`NIX_DONT_SET_RPATH` and revealed that ld-wrapper adds the directory of its
`-plugin` argument to runpath. This pull request fixes that. It treats
`dir/libname.so` as `-L dir -l name`, because this is how `ld.so` interprets
resulting binary: with `dir` in `RUNPATH` and the bare `libname.so` (without
`dir`) in `NEEDED`, it looks for `libname.so` in each `RUNPATH` and chooses the
first, even when the linker was invoked with an absolute path to `.so`.
2017-08-01 11:57:33 +02:00
adisbladis
283364a2fe
Revert "buildenv: read propagated-user-env-packages line-by-line"
This reverts commit dce958ac39.
2017-08-01 15:41:06 +08:00
John Ericson
1cd74ac70a cc-wrapper: Make sure symbol tables of child delegators is not empty
As described in https://github.com/NixOS/nixpkgs/issues/18461, MacOS no
longer accepts dylibs which only reexport other dylibs, because their
symbol tables are empty. To get around this, we define an object file
with a single "private extern" symbol, which hopefully won't clobber
anything.
2017-07-31 17:02:56 -04:00
John Ericson
9242c8b331 cc-wrapper: Use two scripts instead of patch 2017-07-31 17:02:56 -04:00
John Ericson
0c37778c2c cc-wrapper: WIP linking hack for mac OS
Probably best to override Haskell packages set, or anything else
linking a lot of libraries, with this.
2017-07-31 17:02:56 -04:00