Commit Graph

129 Commits

Author SHA1 Message Date
Silvan Mosberger
0a351c3f65
dockerTools.*: Assertion against building for Darwin (#77952)
dockerTools.*: Assertion against building for Darwin
2020-01-31 21:17:40 +01:00
Antoine Eiche
01a68479cc dockerTools.buildLayeredImage: assert maxLayers > 1
Since a layer is reserved for "customization", the image can not
contains less than 2 layers.

The user gets the following message at evaluation:

    nix-instantiate nixos/tests/docker-tools.nix
    trace: the maxLayers argument of dockerTools.buildLayeredImage function must be greather than 1 (current value: 1)
2020-01-30 21:09:52 +01:00
Antoine Eiche
283bcc1003 dockerTools.buildLayeredImage: fix image with only 2 layers
A test is also added to ensure an image with 2 layers can be built.
2020-01-30 21:09:44 +01:00
Silvan Mosberger
b6a9211bf4
dockerTools.*: Assertion against building for darwin
Building a docker image with darwin binaries just yields a confusing
error when ran:

  standard_init_linux.go:211: exec user process caused "exec format error"

This change prevents people from building such images in the first place
2020-01-20 19:14:12 +01:00
Antoine Eiche
da261e3631 dockerTools.buildLayeredImage: fix typo in comments 2020-01-11 09:02:30 +01:00
Richard Wallace
3be767593b dockerTools.buildLayeredImage: fix building layered images in parallel
when tar'ing store paths into layered archives when building layered
images, don't use the absolute nix store path so that tar won't complain
if something new is added to the nix store

when building the final docker image, ignore any file changes tar
detects in the layers. they are all immutable and the only thing that
might change is the number of hard links due to store optimization
2019-12-30 14:47:11 -07:00
Graham Christensen
64453c8dbd
Merge pull request #75781 from grahamc/dockertools/remove-implementation-detail-layers
dockertools.buildLayeredImage: remove implementation detail layers
2019-12-17 16:03:11 -05:00
Graham Christensen
75b8f3820d
Merge pull request #75779 from grahamc/dockertools/fixup-max-layers
dockerTools.buildLayeredImage: update maxlayers from 24 to 100 to match documentation
2019-12-16 14:11:36 -05:00
Graham Christensen
9c02760855
dockerTools.buildLayeredImage: update maxlayers from 24 to 100 to match documentation
mkManyPureLayers already was changed, and this function was not updated.
2019-12-16 13:14:21 -05:00
Graham Christensen
77452740c6
docker examples: Drop unneeded contents list 2019-12-16 13:11:27 -05:00
Graham Christensen
12e2416380
dockerTools.buildLayeredImage: Exclude top level implementation detail layers 2019-12-16 13:03:15 -05:00
Graham Christensen
700f4c5388
dockerTools.buildLayeredImage: prepare to exclude some paths
Without changing behavior, since this code is fiddly, make it possible
to add a filtering step before packaging individual paths.
2019-12-16 12:57:04 -05:00
Graham Christensen
aec80dddc0
dockerTools.buildLayeredImage: pass a list of closures to mkManyPureLayers so it can exclude the top-most level
Before, every docker image had three extra layers:

1. A `closure` layer which is an internal implementation detail of
   calculating the closure of the container
2. a `name-config.json` layer which is the images' run-time
   configuration, and has no business being *in* the image as a layer.
3. a "bulk-layers" layer which is again and implementation detail
   around collecting the image's closure.

None of these layers need to be in the final product.
2019-12-16 12:48:05 -05:00
Graham Christensen
f6d75f550e
dockerTools.buildLayeredImage: tweak formatting on contentsEnv 2019-12-16 12:36:45 -05:00
tomberek
81b0a20dfa buildImageWithNixDb: export USER (#74959)
dockerTools.buildImageWithNixDb: export USER

Changes to Nix user detection (./src/nix-channel/nix-channel.cc#L-166)
cause this function to error. Exporting USER fixes this.
2019-12-07 10:06:42 +01:00
Antoine Eiche
09a669478f dockerTools.examples.nix: set USER in the container config
A USER is required by Nix.
See 9348f9291e/src/libutil/util.cc (L478).
2019-12-05 09:45:51 +01:00
Nick Spinale
8166bc934b build-support/docker: set default image arch to host arch
The architecture of an image should default to the architecture for
which that image is being composed or pulled. buildPackages.go.GOARCH is
an easy way to compute that architecture with the correct terminology.
2019-09-16 14:22:30 +00:00
Graham Christensen
8adaae2df2
dockertools.buildLayeredImage: default layer count up to 100 2019-08-12 19:59:40 -04:00
Sarah Brofeldt
19abdb765b
Merge pull request #63865 from dingxiangfei2009/dockertools-cross-compilation
dockerTools: use skopeo on the right platform
2019-07-31 20:11:18 +02:00
Ding Xiang Fei
ceab72e21b dockerTools: use skopeo on the right platform 2019-07-02 14:53:24 +08:00
worldofpeace
3f4a353737 treewide: use dontUnpack 2019-07-01 04:23:51 -04:00
volth
f3282c8d1e treewide: remove unused variables (#63177)
* treewide: remove unused variables

* making ofborg happy
2019-06-16 19:59:05 +00:00
Matthew Bauer
4e5c9b8cf4
Merge pull request #54921 from grabango/master
Use nativeBuildInputs for building Docker images
2019-06-10 21:38:47 -04:00
Daniël de Kok
3b0d5b5ed3 nixos/tests/docker-tools: check layer sharing with buildLayeredImage
Adapted from grahamc's blog post on layered Docker images in Nix:

https://grahamc.com/blog/nix-and-layered-docker-images
2019-06-06 15:03:08 +02:00
Daniël de Kok
d7f3186b48 dockerTools.buildLayeredImage: restore layer sharing
PR #58431 added /nix/store to each layer.tar. However, the timestamp was
not explicitly set while adding /nix and /nix/store to the archive. This
resulted in different SHA256 hashes of layer.tar between image builds.

This change sets time and owner when tar'ing /nix/store.
2019-06-06 07:40:20 +02:00
Antoine Eiche
a5a5820048 dockerTools: Fix the layer order
The layer order was not correct when a parent image was used: parent
image layers were above the new created layer.

This commits simplifies the code related to layer ordering. In
particular, layers in `layer-list` are ordered from bottom-most to
top-most. This is also the order of layers in the `rootfs.diff_ids`
attribute of the image configuration.
2019-05-07 16:52:25 +02:00
Antoine Eiche
5ef1223f30 nixos/tests/docker-tools: verify order of layers in stacked images 2019-05-07 16:52:13 +02:00
rnhmjoj
5ea4160b33
treewide: fixup bcf54ce
fix the executable bit for scripts installed with substituteAll
and some remaining shebangs.
2019-04-12 09:09:22 +02:00
lewo
a8beff987d
Merge pull request #58350 from xtruder/pkgs/dockerTools/storePathToLayer/runtimeShell
dockerTools: storePathToLayer use runtimeShell in script
2019-04-06 19:01:19 +02:00
Jaka Hudoklin
c087b608e8
Merge pull request #58360 from xtruder/pkgs/dockerTools/nix-prefetch-docker
dockerTools: add nix-prefetch-docker script
2019-04-06 14:44:19 +02:00
Jaka Hudoklin
c1f0d19785
dockerTools: add nix-prefetch-docker script 2019-04-06 14:16:40 +02:00
lewo
dc3ed336df
Merge pull request #58345 from xtruder/pkgs/dockerTools/pullImage/finalImageName
dockerTools: add finalImageName parameter for pullImage
2019-03-28 16:25:01 +01:00
Alex Biehl
1b1e23024b Strip leading ./ in customization layer 2019-03-27 12:08:50 +01:00
Alex Biehl
096e3cbdec Add /nix/store to each layer.tar
To be totally consistent with the way Docker builds images we need to
include the /nix/store in the layer tarballs first.
2019-03-27 11:35:27 +01:00
Jaka Hudoklin
468df177c4
dockerTools: add finalImageName parameter for pullImage 2019-03-26 19:35:14 +01:00
Jaka Hudoklin
5d429f6822
dockerTools: storePathToLayer use runtimeShell in script 2019-03-26 11:07:24 +01:00
Antoine Eiche
fe6860800b dockerTools.buildImage.runAsRoot: preserve layers ordering at image unpacking
This patch preserves the ordering of layers of a parent image when the
image is unpacked.

Fixes #55290
2019-03-12 12:04:22 +01:00
Antoine Eiche
af23d1e2e7 nixos/test/docker-tools: fix Nix image digest 2019-03-06 09:26:33 +01:00
Wael M. Nasreddine
9b22a51712
build-support/docker: fix the build of tarsum with Go 1.12 2019-03-05 10:53:21 -08:00
Jörg Thalheim
dadc7eb329
treewide: use runtimeShell instead of stdenv.shell whenever possible
Whenever we create scripts that are installed to $out, we must use runtimeShell
in order to get the shell that can be executed on the machine we create the
package for. This is relevant for cross-compiling. The only use case for
stdenv.shell are scripts that are executed as part of the build system.
Usages in checkPhase are borderline however to decrease the likelyhood
of people copying the wrong examples, I decided to use runtimeShell as well.
2019-02-26 14:10:49 +00:00
Austin Seipp
c36c048c0e dockerTools: mark store-path-to-layer.sh as executable
bcf54ce5bb introduced a treewide change to
use ${stdenv.shell} where-ever possible. However, this broke a script
used by dockerTools, store-path-to-layer.sh, as it did not preserve the
+x mode bit. This meant the file got put into the store as mode 0444,
resulting in a build-time error later on that looked like:

    xargs: /nix/store/jixivxhh3c8sncp9xlkc4ls3y5f2mmxh-store-path-to-layer.sh: Permission denied

However, in a twist of fate, bcf54ce5bb
not only introduced this regression but, in this particular instance,
didn't even fix the original bug: the store-path-to-layer.sh script
*still* uses /bin/sh as its shebang line, rather than an absolute path
to stdenv. (Fixing this can be done in a separate commit.)

Signed-off-by: Austin Seipp <aseipp@pobox.com>
2019-02-21 08:30:47 -06:00
Vladimír Čunát
024407bf9a
Merge branch 'master' into staging-next
Hydra nixpkgs: ?compare=1505754
2019-02-19 12:11:04 +01:00
Antoine Eiche
da7cd82ab1 dockerTools.buildImage: preserve layers ordering at image repacking
This patch preserves the ordering of layers of a parent image when the
new image is packed.

It is currently not the case: layers are stacked in the reverse order.

Fixes #55290
2019-02-16 00:50:23 +01:00
Kevin Rauwolf
2890a38652 Use nativeBuildInputs for building Docker images 2019-01-29 20:29:18 -08:00
Vladimír Čunát
3fe32b675e
Merge branch 'master' into staging 2019-01-22 15:59:42 +01:00
lewo
105ffa445d
Merge pull request #54270 from xtruder/build-support/docker/layered_image_tag_passthru
dockerTools: buildLayeredImage passthru imageTag
2019-01-20 12:21:20 +01:00
Jaka Hudoklin
1787afb861
dockerTools: buildLayeredImage passthru imageTag 2019-01-18 13:52:07 +01:00
rnhmjoj
bcf54ce5bb
treewide: use ${stdenv.shell} instead of /bin/sh where possible 2019-01-16 20:37:15 +01:00
lewo
7612a6add4
Merge pull request #52870 from xtruder/pkgs/dockerTools/buildLayeredImage/extraCommands
dockerTools: allow to pass extraCommands, uid and gid to buildLayered image
2019-01-10 19:00:19 +01:00
Jaka Hudoklin
954cda5c9d
dockerTools: allow to pass extraCommands, uid and gid to buildLayeredImage 2019-01-10 16:02:23 +01:00