Multi-threaded builds exacerbate the non-determinism in ghc package ids, which
is a serious problem for libraries. Packages that define only executables,
however, should be safe to build with parallelism enabled.
Packages that don't have a Setup.hs file get to use a default version that
lives in the Nix store. By default ghc tries to put the Setup.o and Setup.hi
files in the same directory as the source file, which isn't writable. This
leads to build errors [1]. Thus, we re-direct those paths to a build-local
writable location: $TMPDIR.
Arguably, we could also use "." or copy the /nix/store/deadbeef-Setup.hs file
into the local source directory before compiling, which would work fine, too.
[1] https://github.com/NixOS/nixpkgs/issues/4851
Without this, the generated pack files are non-deterministic.
I didn't notice this issue in my earlier testing, because my test repo
had too few commits for the thread scheduling to take effect. (Test repo
had about 10 commits.)
Add more files to the delete list:
* .git/FETCH_HEAD
* .git/ORIG_HEAD
* .git/refs/remotes/origin/HEAD
* .git/config
Further, remove all remote branches, remove tags not reachable from the
given 'rev', do a full repack and then garbage collect unreferenced
objects.
According to my testing, the result is fully deterministic. As in "any
change done to the upstream repo, ahead of 'rev', will not affect the
hash of the resulting 'clone'". Even changing the clone URL will not
change the output hash, because .git/config is removed.
A new version of git can of course change store format, but that's
unavoidable.
For big repositories, the repack operation may be a bit heavy. But as
far as I can see there is no cheaper way to determinism.
Hydra generates a GHC closure for Darwin that for no apparent reason
contains an ancient, broken Haddock binary -- probably because of an
impurity in the build system. That bug makes those GHC binaries
unusable: <https://github.com/NixOS/nixpkgs/issues/2689>.
This likely exacerbates the non-determinism in ghc package ids, so until
that is fixed let's live with the slow builds.
This reverts commit 817c0e4144.
