Commit Graph

3619 Commits

Author SHA1 Message Date
Silvan Mosberger
0a351c3f65
dockerTools.*: Assertion against building for Darwin (#77952)
dockerTools.*: Assertion against building for Darwin
2020-01-31 21:17:40 +01:00
lewo
86f8732194
Merge pull request #78834 from nlewo/fix-two-layers-image
Fix dockerTools.buildLayerImage with 2 layers
2020-01-30 21:10:52 +01:00
Antoine Eiche
01a68479cc dockerTools.buildLayeredImage: assert maxLayers > 1
Since a layer is reserved for "customization", the image can not
contains less than 2 layers.

The user gets the following message at evaluation:

    nix-instantiate nixos/tests/docker-tools.nix
    trace: the maxLayers argument of dockerTools.buildLayeredImage function must be greather than 1 (current value: 1)
2020-01-30 21:09:52 +01:00
Antoine Eiche
283bcc1003 dockerTools.buildLayeredImage: fix image with only 2 layers
A test is also added to ensure an image with 2 layers can be built.
2020-01-30 21:09:44 +01:00
Andreas Rammhold
a57d0fe0bb
buildRustCrate: fix #78412
`build.rs` files might create files. Those files are supposed to go into
`OUT_DIR` (envirionment variable) and not be overlayed onto the source
tree.
2020-01-28 14:07:58 +01:00
Andreas Rammhold
19698d15ce
buildRustCrateTests: add regression test for #74071 2020-01-28 14:07:58 +01:00
Frederik Rietdijk
7562c2112d
Merge pull request #77739 from NixOS/staging-next
Staging next
2020-01-28 14:03:29 +01:00
Andreas Rammhold
cd27940df0
Merge pull request #78188 from andir/buildRustCrate-support-rlib
buildRustCrate: support `rlib` dependency type
2020-01-28 13:47:57 +01:00
Frederik Rietdijk
dce0ca29d9 Merge master into staging-next 2020-01-28 10:46:13 +01:00
Mario Rodas
c3ccac15eb
libredirect: fix build on darwin
Fix build failure on darwin due to absence of `O_TMPFILE`.
2020-01-23 20:20:20 -05:00
Andreas Rammhold
78faab1be0 buildRustCrateTests: add test case for rlib linking 2020-01-21 17:46:32 +01:00
Andreas Rammhold
406e0c9d51
buildRustCrateTests: fix some formatting issues 2020-01-21 17:32:48 +01:00
Andreas Rammhold
d6a8b55fb0
buildRustCrate: treat rlib crates just like lib crates
Both version provide `rlib` files to link against. Previously we would
try to find a matching shared library in the `lib` output.
2020-01-21 17:22:59 +01:00
Andreas Rammhold
60fea726de
Merge pull request #77822 from andir/buildRustCrate-remove-custom-lib-path
buildRustCrate: remove custom lib path heuristic
2020-01-21 17:21:55 +01:00
Silvan Mosberger
b6a9211bf4
dockerTools.*: Assertion against building for darwin
Building a docker image with darwin binaries just yields a confusing
error when ran:

  standard_init_linux.go:211: exec user process caused "exec format error"

This change prevents people from building such images in the first place
2020-01-20 19:14:12 +01:00
Jonas Schürmann
61870727de mirrors.nix: Use HTTPS in maven mirrors
Since 2020-01-15, the maven central repository does no longer support HTTP.
See https://blog.sonatype.com/central-repository-moving-to-https for details.
2020-01-20 09:54:14 +01:00
Mario Rodas
21d0b2067f
Merge pull request #76831 from mbrgm/docker-slim
docker-slim: init at 1.26.1
2020-01-18 07:51:03 -05:00
Andreas Rammhold
69c96adc53
buildRustCrateTests: use releaseTools.aggregate
Previously I did use `runCommand` to do the same. Using
releaseTools.aggregate seems a lot saner and we might get nicer hydra
output of the tests that are failing.
2020-01-16 13:24:15 +01:00
Andreas Rammhold
29a8575e3d
buildRustCrate: remove one of the odd library filename cases
It used to be the case (ref missing) that cargo did treat
`src/$libName.rs` as an alternative to `src/lib.rs` when the latter
wasn't present. Recently I failed to reproduce that with vanilla cargo
and it started to cause pain with some crates of the form:

some_crate/
 `- src
   `- main.rs
   `- some_crate.rs

We would build `src/some_crate.rs` and thing it is a library while that
might not be the actual case. This crate is a valid `bin` crate not a
`lib` crate as far as I can tell from the samples I took.

I removed support for the previously required heuristic and commented
out the test cases in case we will need them again. We could crawl in
the Git history but chances are that the next person looking into this
doesn't know about the history.
2020-01-16 13:24:13 +01:00
Anders Kaseorg
3cd8ce3bce treewide: Fix unsafe concatenation of $LD_LIBRARY_PATH
Naive concatenation of $LD_LIBRARY_PATH can result in an empty
colon-delimited segment; this tells glibc to load libraries from the
current directory, which is definitely wrong, and may be a security
vulnerability if the current directory is untrusted.  (See #67234, for
example.)  Fix this throughout the tree.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2020-01-15 09:47:03 +01:00
Andreas Rammhold
cb007e69a1 stdenv: make symlinks that refer to the same output relative
While looking at the graph of all the outputs in my personal binary
cache it became obvious that we have a lot of self references within the
package set. That isn't an isuse by itself. However it increases the
size of the binary cache for every (reproducible) build of a package
that carries references to itself. You can no longer deduplicate the
outputs since they are all unique. One of the ways to get rid of (a few)
references is to rewrite all the symlinks that are currently used to be
relative symlinks. Two build of something that didn't really change but
carries a self-reference can the be store as the same NAR file again.

I quickly hacked together this change to see if that would yield and
success. My bash scripting skills are probably not great but so far it
seem to somewhat work.
2020-01-15 09:26:40 +01:00
Benjamin Hipple
6e8c377562 rustPlatform.buildRustPackage: cleaner output on verifyCargoDeps (#77567)
When this fails, the user may want to copy-paste the path to the "bad"
Cargo.lock file to inspect. The trailing `.` on `$cargoDeps.` gets caught in
most terminal copy-pastes. Since half the lines already don't have it, this
removes it from all of them for consistent output.
2020-01-12 17:19:17 +00:00
Antoine Eiche
da261e3631 dockerTools.buildLayeredImage: fix typo in comments 2020-01-11 09:02:30 +01:00
lewo
0d983f9f99
Merge pull request #75911 from Simspace/parallel-docker-buildlayeredimage
when building a layered docker image, ignore it if tar encounters cha…
2020-01-11 08:55:34 +01:00
Andreas Rammhold
3e61906e1c
buildRustCrate: slight "rewording" and reformatting
There is no point in reinventinb builtins through `filterAttrs` or the
like. Lets just stick to what we already have in our toolbelt.
2020-01-07 11:57:34 +01:00
Andreas Rammhold
a3a51763f9
buildRustCrate: add buildTests flag to tell rustc to build tests instead of binaries
This helps us instruct rustc to build tests instead of binaries. The
actual build will then ONLY produce test executables. This is a first
step towards having rust crate tests within nixpkgs.

We default back to only a single output in test cases since that is the
only reasonable thing to do here.

Producing libraries or binaries in addition to tests would theoretically
be feasible but usually generates different dependency trees. It is very
common to have some libraries in `[dev-depdendencies]` within Cargo.toml
just for your tests. To not start mixing things up going with a
dedicated derivation for the test build sounds like the best choice for
now.

To use this you must provide a proper test dependency chain to
`buildRustCrate` (as you would usually do with your non-test inputs).
And then set the `buildTests` attribute to `true`. The derivation will
then contain all tests that were built in `$out/tests`. All common test
patterns and directories should be supported and tested by this change.

Below is an example how you would run a single test from the derivation.
This commit contains some more examples in the `buildRustCrateTests`
attribute set that might be helpful.

```
let
  drv = buildRustCrate {
     …
     buildTests true;
  };
in runCommand "test-my-crate" {} ''
  touch $out
  exec ${drv}/tests/my-test
''
```
2020-01-07 11:57:34 +01:00
Andreas Rammhold
6383b42dae
buildRustCrate: fixup usage of builtins.filterSource
While unifying most of the lib function calls I accidentially changed
the filterSource functions as well. Since there were no tests I ended
up forgetting about this case (even thought I ran into it…).
2020-01-07 00:49:48 +01:00
Marius Bergmann
d2394a4789 docker-slim: init at 1.26.1
Co-authored-by: Mario Rodas <marsam@users.noreply.github.com>
Co-authored-by: Oleksii Filonenko <brightone@protonmail.com>
2020-01-03 11:18:00 +01:00
Andreas Rammhold
9f03cb8562
Merge pull request #75563 from andir/cleanup-buildRustCrate
Cleanup buildRustCrate expression
2020-01-02 13:42:33 +01:00
John Ericson
cfd013813e
Merge pull request #74090 from obsidiansystems/ghcjs-cross-without-cc
stdenv, haskell: bonafied GHCJS cross compilation without stdenv.cc
2019-12-30 16:40:43 -08:00
Richard Wallace
3be767593b dockerTools.buildLayeredImage: fix building layered images in parallel
when tar'ing store paths into layered archives when building layered
images, don't use the absolute nix store path so that tar won't complain
if something new is added to the nix store

when building the final docker image, ignore any file changes tar
detects in the layers. they are all immutable and the only thing that
might change is the number of hard links due to store optimization
2019-12-30 14:47:11 -07:00
Robin Gloster
ac8eaa8507
treewide: fix *Flags 2019-12-30 04:50:37 +01:00
Alyssa Ross
b9d274b89d rustPlatform.fetchcargo: expose 2019-12-23 18:27:56 +00:00
Alyssa Ross
fdfbb4671e rustPlatform: forward unpackPhase to fetchcargo
If a custom unpackPhase is used for the package, it needs to also be
used for fetchcargo so the same source is available for vendoring.
2019-12-23 18:27:56 +00:00
Alyssa Ross
839c9e9344 rustPlatform: forward fetchcargo args to stdenv
Most stdenv wrappers already work like this -- it allows greater
customisation.  We just have to be careful to remove arguments we're
using that shouldn't be passed to stdenv.  I've been conservative
here, because fetchcargo checksums shouldn't change lightly.
2019-12-23 18:27:56 +00:00
Frederik Rietdijk
f79e10061f Merge master into staging-next 2019-12-19 09:19:50 +01:00
Graham Christensen
64453c8dbd
Merge pull request #75781 from grahamc/dockertools/remove-implementation-detail-layers
dockertools.buildLayeredImage: remove implementation detail layers
2019-12-17 16:03:11 -05:00
Frederik Rietdijk
bf2877b9ab Merge master into staging-next 2019-12-17 17:16:52 +01:00
Michael Raskin
26df2f4e9e
Merge pull request #74862 from alyssais/fetchgittiles
fetchFromGitiles: init
2019-12-17 15:23:08 +00:00
Frederik Rietdijk
7aedd744d8 Merge master into staging-next 2019-12-16 20:28:10 +01:00
Graham Christensen
75b8f3820d
Merge pull request #75779 from grahamc/dockertools/fixup-max-layers
dockerTools.buildLayeredImage: update maxlayers from 24 to 100 to match documentation
2019-12-16 14:11:36 -05:00
Graham Christensen
9c02760855
dockerTools.buildLayeredImage: update maxlayers from 24 to 100 to match documentation
mkManyPureLayers already was changed, and this function was not updated.
2019-12-16 13:14:21 -05:00
Graham Christensen
77452740c6
docker examples: Drop unneeded contents list 2019-12-16 13:11:27 -05:00
Graham Christensen
12e2416380
dockerTools.buildLayeredImage: Exclude top level implementation detail layers 2019-12-16 13:03:15 -05:00
Graham Christensen
700f4c5388
dockerTools.buildLayeredImage: prepare to exclude some paths
Without changing behavior, since this code is fiddly, make it possible
to add a filtering step before packaging individual paths.
2019-12-16 12:57:04 -05:00
Graham Christensen
aec80dddc0
dockerTools.buildLayeredImage: pass a list of closures to mkManyPureLayers so it can exclude the top-most level
Before, every docker image had three extra layers:

1. A `closure` layer which is an internal implementation detail of
   calculating the closure of the container
2. a `name-config.json` layer which is the images' run-time
   configuration, and has no business being *in* the image as a layer.
3. a "bulk-layers" layer which is again and implementation detail
   around collecting the image's closure.

None of these layers need to be in the final product.
2019-12-16 12:48:05 -05:00
Graham Christensen
f6d75f550e
dockerTools.buildLayeredImage: tweak formatting on contentsEnv 2019-12-16 12:36:45 -05:00
Alyssa Ross
a50653295d buildSkawarePackage: pass through extra args
This allows things like hooks other than postInstall to be passed
through to mkDerivation, which is very useful when customising or
debugging a package.
2019-12-16 13:27:10 +00:00
Jan Tojnar
aa3cb8b00e
Merge branch 'staging-next' into staging 2019-12-14 23:09:23 +01:00
Jan Tojnar
429561978b
Merge branch 'master' into staging-next 2019-12-14 23:09:06 +01:00