Commit Graph

88 Commits

Author SHA1 Message Date
Antoine Eiche
132e790735 dockerTools.pullImage: change the docker deamon readiness mechanism
To wait for the docker deamon, curl requests are sent. However, if a
http proxy is set, it will respond instead of the docker daemon.
To avoid this, we send docker ps command instead of curl command.
2017-09-04 10:52:16 +02:00
Mathias Schreck
86d9b09c9b dockerTools: fix image json and manifest
The image json is not exactly the same as the layer json, therefore I
changed the implementation to use the `baseJson` which doesn’t include
layer specific details like `id`, `size` or the checksum of the layer.

Also the `history` entry was missing in the image json. I’m not totally
sure if this field is required, but a I got an error from a docker
registry when I’ve tried to receive the distribution manifest of an
image without those `history` entry:

GET: `http://<registry-host>/v2/<imageName>/manifests/<imageTag>`

```json
{
  "errors": [
    {
      "code": "MANIFEST_INVALID",
      "message": "manifest invalid",
      "detail": {}
    }
  ]
}
```

I’ve also used a while loop to iterate over all layers which should make
sure that the order of the layers is correct. Previously `find` was
used and I’m not sure if the order was always correct.
2017-08-03 11:52:03 +02:00
Antoine Eiche
9082b66096 dockerTools: fix permissions on base image
If the base image has been built with nixpkgs.dockerTools, the image
configuration and manifest are readonly so we first need to change
their permissions before removing them.

Fix #27632.
2017-07-31 13:52:35 +02:00
Antoine Eiche
9ee7e8b67e docker: generate the image configuration and manifest
This is required to push images to the Docker registry v2.
2017-07-26 23:05:29 +02:00
Antoine Eiche
8a431e13b5 docker: Remove ./ pattern when packing an image
Elements in images tar.gz generated by docker don't start by './'.
2017-07-26 23:05:29 +02:00
Antoine Eiche
0a4c43065c docker: do not import configuration and manifest from the base image
Fix #27632.
2017-07-26 20:01:41 +02:00
Antoine Eiche
e28d817c9a docker: lowercase image name and tag
The docker loading (docker 1.12.6) of an image with uppercase in the
name fails with the following message:
invalid reference format: repository name must be lowercase
2017-07-25 10:47:51 +02:00
Daiderd Jordan
90ff6b1d03 Merge pull request #27017 from LnL7/docker-pure-layer
docker-tools: set user/group when creating a pure layer
2017-07-15 12:22:53 +02:00
Sarah Brofeldt
9e61958399 tarsum: Fix source path in docker.src 2017-07-13 23:09:04 +02:00
Daiderd Jordan
83fbc0f035
docker-tools: fixup permssions for extraCommands 2017-07-08 13:57:04 +02:00
Daiderd Jordan
dde5865140
docker-tools: set group/owner when creating a pure layer 2017-07-01 17:08:48 +02:00
Michael Fellinger
54419f3016 using inherited instead 2017-06-27 22:22:08 +02:00
Michael Fellinger
704e04b108 dockerTools.buildImage: configurable timestamp
This way not all images have to be from 47 years ago, making it much easier to find the one you're looking for.
2017-06-27 22:11:17 +02:00
Matej Cotman
8d3cb0af9e docker tools: fix pull image function 2017-05-26 18:48:16 +02:00
rht
5628cebcf0 /bin/sh -> ${stdenv.shell} 2017-04-30 17:01:07 +02:00
Ryan Trinkle
dce2c258ac dockerTools: optionally preserve directory symlinks
In some cases, this seems to save a lot (>40%) of space.
2017-04-23 09:45:21 -04:00
Tim Stewart
5ca1646bb0 docker: improve reproducibility of layers
This patch fixes file modification times to $SOURCE_DATE_EPOCH, and
ensures that files originating from the store are owned by root:root.
Both changes improve reproducibility, and the latter allows proper
building on a host where the store is owned by a non-root user.
2017-04-16 22:46:24 -04:00
Lazar Bodor
186ed3bf8f dockerTools: rsync contents with -keep-dirlinks 2016-11-24 12:48:26 +11:00
Daiderd Jordan
f84e2efcfd Merge pull request #20018 from adnelson/use_basename_docker
docker.buildUtils: use baseNameOf to allow for slashes in name
2016-11-01 12:29:28 +01:00
Test
ca5fda39dc docker.buildUtils: use baseNameOf to allow for slashes in name 2016-10-31 10:11:48 -05:00
Tom Boettcher
30b09b8620 dockerTools: Fix loop typo.
This typo was causing only a single layer to be de-duplicated,
rather than all parent layers.
2016-10-24 14:30:00 -05:00
Allen Nelson
940aafcbcc use rsync, comment out pulls 2016-10-03 15:12:29 -05:00
Allen Nelson
b9007c8d83 add examples file 2016-10-03 12:07:33 -05:00
Allen Nelson
4abe579250 add docs to docker build functions
bring back ls_tar

replace goPackages with go

don't hardcode /nix/store in vmTools

more docs
2016-09-29 12:52:57 -05:00
Tom Boettcher
1e8b69c35e dockerTools: Fix layer redundancy.
When building an image with multiple layers, files
already included in an underlying layer are supposed to
be excluded from the current layer. However, some subtleties
in the way filepaths are compared seem to be blocking this.

Specifically:
* tar generates relative filepaths with directories ending in '/'
* find generates absolute filepaths with no trailing slashes on directories

That is, paths extracted from the underlying tarball look like:
    nix/store/.../foobar/
whereas the layer being generated uses paths like:
    /nix/store/.../foobar

This patch modifies the output of "tar -t" to match the latter format.
2016-09-23 16:40:59 -05:00
Profpatsch
61462c94e6 lib/fetchers.nix: factor out impure proxy vars (#18702)
Apparently everyone just copied those variables, instead of creating a
library constant for them. Some even removed the comment. -.-
2016-09-17 21:50:01 +02:00
Nathan Zadoks
b9284e445b dockerTools: disable compression in tarsum.go
Previously, tarsum would compress the (discarded) tarball produced.
That's a waste of CPU, and a waste of time.
2016-07-02 02:22:36 -04:00
Nathan Zadoks
23e9e33975 dockerTools: format tarsum.go with gofmt
Nearly all Go code on this earth is formatted with gofmt, and it's
somewhat surprising to find a Go file that isn't formatted accordingly.
2016-07-02 02:22:36 -04:00
Kamil Chmielewski
7eb671ebcd no more goPackages 2016-06-09 13:08:00 +02:00
Brian McKenna
0167b61ef4 dockerTools: only add "/nix" if it exists
The /nix path in 4d200538 of the layer tar didn't exist for some
packages, such as cacert. This is because cacert just creates an /etc
directory and doesn't depend on any other /nix paths. If we tried
putting this directory in the tar and using overlayfs with it, we'd get
"Invalid argument" when trying to remove the directory.

We now check whether the closure is non-empty before telling tar to
store the /nix directory.

Fixes #14710.
2016-04-16 01:16:49 +10:00
Brian McKenna
bc2f314f73 dockerTools: make tars deterministic
There were two sources of non-determinisim coming into the images. The
first was tar mtimes, the second was pigz/gzip times.

An example image now passes with the --check flag.
2016-04-15 09:29:15 +10:00
Luca Bruno
44d651485a dockerTools: fix difference between base files and layer files 2016-04-14 12:23:49 +02:00
Luca Bruno
4d200538c2 dockerTools: fix /nix/store permissions 2016-04-14 12:23:48 +02:00
Brian McKenna
d150fe8915 dockerTools: use pigz for final image tar
Saves a few seconds on large images.
2016-04-11 16:32:47 +10:00
Brian McKenna
ebb911cc0b dockerTools: remove tarballs functionality
I think the intention of this functionality was to provide a simple
alternative to the "runAsRoot" and "contents" attributes.

The implementation caused very slow builds of Docker images. Almost all
of the build time was spent in IO for tar, due to tarballs being
created, immediately extracted, then recreated. I had 30 minute builds
on some of my images which are now down to less than 2 minutes. A couple
of other users on #nix IRC have observed similar improvements.

The implementation also mutated the produced Docker layers without
changing their hashes. Using non-empty tarballs would produce images
which got cached incorrectly in Docker.

I have a commit which just fixes the performance problem but I opted to
completely remove the tarball feature after I found out that it didn't
correctly implement the Docker Image Specification due to the broken
hashing.
2016-04-11 16:32:43 +10:00
Asko Soukka
584427c694 dockerTools: Fix issue where image name with repository prefix was not supported 2016-02-18 18:16:58 +02:00
Arthur Noel
903129f770 dockerTools: private registry support
* authorization token is optional
* registry url is taken from X-Docker-Endpoints header
* pull.sh correctly resumes partial layer downloads
* detjson.py does not fail on missing keys
2016-01-27 21:13:07 +00:00
Luca Bruno
4a4561ce24 dockerTools: nix functions for manipulating docker images 2016-01-13 10:27:37 +01:00