Commit Graph

42 Commits

Author SHA1 Message Date
Florian Klink
f7ee2706c2 dockerTools.fakeNss: init
This provides a /etc/passwd and /etc/group that contain root and nobody.

Useful when packaging binaries that insist on using nss to look up
username/groups (like nginx).

The current nginx example used the `runAsRoot` parameter to setup
/etc/group and /etc/passwd (which also doesn't exist in
buildLayeredImage), so we can now just use fakeNss there and use
buildLayeredImage.
2020-12-02 14:56:07 +01:00
adisbladis
b7b22c5814
dockerTools: Always cross compile for another arch in the cross example
The example fails to build on aarch64, so lets cross build for gnu64.
2020-11-20 12:57:58 +01:00
adisbladis
5357abf49a
dockerTools: Add cross example 2020-11-19 18:13:21 +01:00
Utku Demir
ae82f81bfa
dockerTools.streamLayeredImage: Store the customisation layer as a tarball
This fixes as issue described here[1], where permissions set by 'extraCommands'
were ignored by Nix.

[1] https://github.com/NixOS/nixpkgs/pull/91084#issuecomment-669834938
2020-09-04 16:53:23 +12:00
Johan Thomsen
f5db415e2f nixos/tests/dockerTools: add test for running non-root containers with buildLayeredImage
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2020-07-31 10:14:07 +02:00
Robert Hensing
8c0459f611 dockerTools: Always set imageTag attribute
The image tag can be specified or generated from the output hash.
Previously, a generated tag could be recovered from the evaluated
image with some string operations.

However, with the introduction of streamLayeredImage, it's not
feasible to compute the generated tag yourself.

With this change, the imageTag attribute is set unconditionally,
for the buildImage, buildLayeredImage, streamLayeredImage functions.
2020-07-11 16:58:25 +02:00
Jos van Bakel
66737168a4
dockerTools.buildLayeredImage: fix created=now 2020-07-09 09:34:18 +02:00
Utku Demir
06db331922
dockerTools: Verify nix-store contents on buildLayeredImage test 2020-07-06 16:59:58 +12:00
Utku Demir
cc46362929
dockerTools: Support files directly under /nix/store
Also makes sure that the files inside a layer added in a sorted order
to make the results more deterministic.
2020-07-04 22:00:57 +12:00
Robert Hensing
caf47063b4 dockerTools: test that tar keeps nix binary symlinks intact 2020-06-19 10:08:34 +02:00
lewo
df928fafd4
Merge pull request #87725 from nlewo/fix-dockertools
dockerTools.examples: fix the nginx image
2020-05-15 16:54:09 +02:00
Antoine Eiche
9ac444d80e dockerTools.examples: fix the nginx image
Nginx now requires directories `/var/log/nginx` and `/var/cache/nginx`
to be existing.

The objective is to fix the test `nixosTests.docker-tools`.
2020-05-13 10:29:53 +02:00
Utku Demir
f5a90a7aab
dockerTools.buildImage: Preserve environment variables from the parent image 2020-05-08 21:49:16 +12:00
Robert Hensing
6dab1b50a6 buildLayeredImage: Allow empty store, no paths to add
This is useful when buildLayeredImage is called in a generic way
that should allow simple (base) images to be built, which may not
reference any store paths.
2020-02-28 14:59:04 +01:00
Antoine Eiche
baa78de594 nixosTests.docker-tools: add bulk-layer test
A regression test for https://github.com/NixOS/nixpkgs/issues/78744.
2020-02-14 09:26:26 +01:00
Richard Wallace
3b65b3f6d6 dockerTools.buildLayeredImage: store all paths passed in final layer
Fixes #78744

My previous change broke when there are more packages than the maximum
number of layers. I had assumed that the `store-path-to-layer.sh` was
only ever passed a single store path, but that is not the case if
there are multiple packages going into the final layer. To fix this, we
loop through the paths going into the final layer, appending them to the
tar file and making sure they end up at the right path.
2020-02-14 09:26:26 +01:00
Antoine Eiche
283bcc1003 dockerTools.buildLayeredImage: fix image with only 2 layers
A test is also added to ensure an image with 2 layers can be built.
2020-01-30 21:09:44 +01:00
Graham Christensen
77452740c6
docker examples: Drop unneeded contents list 2019-12-16 13:11:27 -05:00
Antoine Eiche
09a669478f dockerTools.examples.nix: set USER in the container config
A USER is required by Nix.
See 9348f9291e/src/libutil/util.cc (L478).
2019-12-05 09:45:51 +01:00
Daniël de Kok
3b0d5b5ed3 nixos/tests/docker-tools: check layer sharing with buildLayeredImage
Adapted from grahamc's blog post on layered Docker images in Nix:

https://grahamc.com/blog/nix-and-layered-docker-images
2019-06-06 15:03:08 +02:00
Antoine Eiche
5ef1223f30 nixos/tests/docker-tools: verify order of layers in stacked images 2019-05-07 16:52:13 +02:00
Jaka Hudoklin
468df177c4
dockerTools: add finalImageName parameter for pullImage 2019-03-26 19:35:14 +01:00
Antoine Eiche
af23d1e2e7 nixos/test/docker-tools: fix Nix image digest 2019-03-06 09:26:33 +01:00
lewo
7612a6add4
Merge pull request #52870 from xtruder/pkgs/dockerTools/buildLayeredImage/extraCommands
dockerTools: allow to pass extraCommands, uid and gid to buildLayered image
2019-01-10 19:00:19 +01:00
Jaka Hudoklin
954cda5c9d
dockerTools: allow to pass extraCommands, uid and gid to buildLayeredImage 2019-01-10 16:02:23 +01:00
Antoine Eiche
43cbba0274 dockerTools.examples.runAsRootParentImage: init
Example of running something as root on top of a parent image.
This is a regression test related to the PR #52109.
2018-12-27 11:15:35 +01:00
Graham Christensen
c88337c9ac
dockerTools.buildImage: support using a layered image in fromImage
Docker images used to be, essentially, a linked list of layers. Each
layer would have a tarball and a json document pointing to its parent,
and the image pointed to the top layer:

    imageA  ----> layerA
                    |
                    v
                  layerB
                    |
                    v
                  layerC

The current image spec changed this format to where the Image defined
the order and set of layers:

    imageA  ---> layerA
            |--> layerB
            `--> layerC

For backwards compatibility, docker produces images which follow both
specs: layers point to parents, and images also point to the entire
list:

    imageA  ---> layerA
            |      |
            |      v
            |--> layerB
            |      |
            |      v
            `--> layerC

This is nice for tooling which supported the older version and never
updated to support the newer format.

Our `buildImage` code only supported the old version, so in order for
`buildImage` to properly generate an image based on another image
with `fromImage`, the parent image's layers must fully support the old
mechanism.

This is not a problem in general, but is a problem with
`buildLayeredImage`.

`buildLayeredImage` creates images with newer image spec, because
individual store paths don't have a guaranteed parent layer. Including
a specific parent ID in the layer's json makes the output less likely
to cache hit when published or pulled.

This means until now, `buildLayeredImage` could not be the input to
`buildImage`.

The changes in this PR change `buildImage` to only use the layer's
manifest when locating parent IDs. This does break buildImage on
extremely old Docker images, though I do wonder how many of these
exist.

This work has been sponsored by Target.
2018-12-05 14:25:54 -05:00
Graham Christensen
fb2d153dac
dockerTools: test buildLayeredImage 2018-09-27 14:19:43 -04:00
Graham Christensen
aedc651903
dockerTools.buildImage: test that created=now makes an unstable date 2018-09-20 13:06:14 -04:00
Antoine Eiche
d44b778d10 dockerTools.examples: explicitly set image tag to fix docker-tools tests
docker-tools tests load images without specifying any tag
value. Docker then uses the image with tag "latest" which doesn't
exist anymore since commit 39e678e24e.
2018-07-06 16:38:42 +02:00
Samuel Dionne-Riel
902b0593be tests/docker-tools: Adds regression test for #34779 2018-05-24 12:23:51 -04:00
Antoine Eiche
736848723e dockerTools.pullImage: Skopeo pulls images by digest
Skopeo is used to pull images from a Docker registry (instead of a
Docker deamon in a VM).

An image reference is specified with its name and its digest which is
an immutable image identifier (unlike image name and tag).

Skopeo can be used to get the digest of an image, for instance:
$ skopeo inspect docker://docker.io/nixos/nix:1.11 | jq -r '.Digest'
2018-05-02 21:32:20 +02:00
Antoine Eiche
e8f452f110 dockerTools: add an onTopOfPulledImage example
This allows to test if a pulled image can be updated by using our
Docker tools.
2018-03-13 11:59:22 +01:00
Wei-Ming Yang
70e9b60b33
dockerTools.examples: correct a typo in comments
This commit is for correcting a typo in comments.
2018-01-01 16:13:40 +08:00
Robin Gloster
20677fca59
dockerTools: fix hash to accomodate the pullImage revert 2017-09-28 14:09:49 +02:00
Robin Gloster
5c6dc717a6
Revert "dockerTools.pullImage: use skopeo to pull the image"
This reverts commit 01174c5f4d.

See https://github.com/NixOS/nixpkgs/pull/29302#issuecomment-332809092
for more information. This broke image format compatibility and
therefore amongst others mesos.
2017-09-28 14:09:49 +02:00
Antoine Eiche
ff4d7f0fd2 dockerTools.examples.nix: set NIX_PAGER=cat environment variable 2017-09-25 09:39:15 +02:00
Antoine Eiche
df589a438e dockerTools.buildImageWithNixDb: populate the Nix Db of the image Nix store
Currently, the contents closure is copied to the layer but there is no
nix database initialization. If pkgs.nix is added in the contents,
nix-store doesn't work because there is no nix database.

From the contents of the layer, this commit generates and loads the
database in the nix store of the container. This only works if there
is no parent layer that already have a nix store (to support several
nix layers, we would have to merge nix databases of parent layers).

We also add an example to play with the nix store inside the
container. Note it seems `more` is a missing dependency of the nix
package!
2017-09-20 20:14:24 +02:00
Antoine Eiche
01174c5f4d dockerTools.pullImage: use skopeo to pull the image
Before this patch, a VM was used to spawn docker that pulled the
VM. Now, the tool Skopeo does this job well so we can simplify our
dockerTools since we doesn't need Docker anymore:)

This also fixe the regression described in
https://github.com/NixOS/nixpkgs/issues/29271 : cntlm proxy doesn't
work in 17.09 while it worked in 17.03.

Note Skopeo doesn't produce the same output than docker pull so, we
have to update sha.
2017-09-17 08:26:02 +01:00
Matej Cotman
8d3cb0af9e docker tools: fix pull image function 2017-05-26 18:48:16 +02:00
Allen Nelson
940aafcbcc use rsync, comment out pulls 2016-10-03 15:12:29 -05:00
Allen Nelson
b9007c8d83 add examples file 2016-10-03 12:07:33 -05:00