JakeHillion/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #57611 from Ericson2314/stage-braid-not-chain

Browse Source 
top-level: Create `pkgs{Build,Host,Target}{Build,Host,Target}`
This commit is contained in:
John Ericson 2019-03-25 21:56:59 -04:00 committed by GitHub
commit aa0cf64422
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
17 changed files with 432 additions and 198 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

374
doc/cross-compilation.xml
View File

@ -12,11 +12,12 @@
computing power and memory to compile their own programs. One might think computing power and memory to compile their own programs. One might think
that cross-compilation is a fairly niche concern. However, there are that cross-compilation is a fairly niche concern. However, there are
significant advantages to rigorously distinguishing between build-time and significant advantages to rigorously distinguishing between build-time and
run-time environments! This applies even when one is developing and run-time environments! Significant, because the benefits apply even when one
deploying on the same machine. Nixpkgs is increasingly adopting the opinion is developing and deploying on the same machine. Nixpkgs is increasingly
that packages should be written with cross-compilation in mind, and nixpkgs adopting the opinion that packages should be written with cross-compilation
should evaluate in a similar way (by minimizing cross-compilation-specific in mind, and nixpkgs should evaluate in a similar way (by minimizing
special cases) whether or not one is cross-compiling. cross-compilation-specific special cases) whether or not one is
cross-compiling.
</para> </para>
<para> <para>
@ -30,7 +31,7 @@
<section xml:id="sec-cross-packaging"> <section xml:id="sec-cross-packaging">
<title>Packaging in a cross-friendly manner</title> <title>Packaging in a cross-friendly manner</title>
<section xml:id="sec-cross-platform-parameters"> <section xml:id="ssec-cross-platform-parameters">
<title>Platform parameters</title> <title>Platform parameters</title>
<para> <para>
@ -218,8 +219,20 @@
</variablelist> </variablelist>
</section> </section>
<section xml:id="sec-cross-specifying-dependencies"> <section xml:id="ssec-cross-dependency-categorization">
<title>Specifying Dependencies</title> <title>Theory of dependency categorization</title>
<note>
<para>
This is a rather philosophical description that isn't very
Nixpkgs-specific. For an overview of all the relevant attributes given to
<varname>mkDerivation</varname>, see
<xref
linkend="ssec-stdenv-dependencies"/>. For a description of how
everything is implemented, see
<xref linkend="ssec-cross-dependency-implementation" />.
</para>
</note>
<para> <para>
In this section we explore the relationship between both runtime and In this section we explore the relationship between both runtime and
@ -227,84 +240,98 @@
</para> </para>
<para> <para>
A runtime dependency between 2 packages implies that between them both the A run time dependency between two packages requires that their host
host and target platforms match. This is directly implied by the meaning of platforms match. This is directly implied by the meaning of "host platform"
"host platform" and "runtime dependency": The package dependency exists and "runtime dependency": The package dependency exists while both packages
while both packages are running on a single host platform. are running on a single host platform.
</para> </para>
<para> <para>
A build time dependency, however, implies a shift in platforms between the A build time dependency, however, has a shift in platforms between the
depending package and the depended-on package. The meaning of a build time depending package and the depended-on package. "build time dependency"
dependency is that to build the depending package we need to be able to run means that to build the depending package we need to be able to run the
the depended-on's package. The depending package's build platform is depended-on's package. The depending package's build platform is therefore
therefore equal to the depended-on package's host platform. Analogously, equal to the depended-on package's host platform.
the depending package's host platform is equal to the depended-on package's
target platform.
</para> </para>
<para> <para>
In this manner, given the 3 platforms for one package, we can determine the If both the dependency and depending packages aren't compilers or other
three platforms for all its transitive dependencies. This is the most machine-code-producing tools, we're done. And indeed
important guiding principle behind cross-compilation with Nixpkgs, and will <varname>buildInputs</varname> and <varname>nativeBuildInputs</varname>
be called the <wordasword>sliding window principle</wordasword>. have covered these simpler build-time and run-time (respectively) changes
for many years. But if the dependency does produce machine code, we might
need to worry about its target platform too. In principle, that target
platform might be any of the depending package's build, host, or target
platforms, but we prohibit dependencies from a "later" platform to an
earlier platform to limit confusion because we've never seen a legitimate
use for them.
</para> </para>
<para> <para>
Some examples will make this clearer. If a package is being built with a Finally, if the depending package is a compiler or other
<literal>(build, host, target)</literal> platform triple of <literal>(foo, machine-code-producing tool, it might need dependencies that run at "emit
bar, bar)</literal>, then its build-time dependencies would have a triple time". This is for compilers that (regrettably) insist on being built
of <literal>(foo, foo, bar)</literal>, and <emphasis>those together with their source langauges' standard libraries. Assuming build !=
packages'</emphasis> build-time dependencies would have a triple of host != target, a run-time dependency of the standard library cannot be run
<literal>(foo, foo, foo)</literal>. In other words, it should take two at the compiler's build time or run time, but only at the run time of code
"rounds" of following build-time dependency edges before one reaches a emitted by the compiler.
fixed point where, by the sliding window principle, the platform triple no
longer changes. Indeed, this happens with cross-compilation, where only
rounds of native dependencies starting with the second necessarily coincide
with native packages.
</para> </para>
<note>
<para>
The depending package's target platform is unconstrained by the sliding
window principle, which makes sense in that one can in principle build
cross compilers targeting arbitrary platforms.
</para>
</note>
<para> <para>
How does this work in practice? Nixpkgs is now structured so that Putting this all together, that means we have dependencies in the form
build-time dependencies are taken from <varname>buildPackages</varname>, "host → target", in at most the following six combinations:
whereas run-time dependencies are taken from the top level attribute set. <table>
For example, <varname>buildPackages.gcc</varname> should be used at <caption>Possible dependency types</caption>
build-time, while <varname>gcc</varname> should be used at run-time. Now, <thead>
for most of Nixpkgs's history, there was no <tr>
<varname>buildPackages</varname>, and most packages have not been <th>Dependency's host platform</th>
refactored to use it explicitly. Instead, one can use the six <th>Dependency's target platform</th>
(<emphasis>gasp</emphasis>) attributes used for specifying dependencies as </tr>
documented in <xref linkend="ssec-stdenv-dependencies"/>. We "splice" </thead>
together the run-time and build-time package sets with <tbody>
<varname>callPackage</varname>, and then <varname>mkDerivation</varname> <tr>
for each of four attributes pulls the right derivation out. This splicing <td>build</td>
can be skipped when not cross-compiling as the package sets are the same, <td>build</td>
but is a bit slow for cross-compiling. Because of this, a </tr>
best-of-both-worlds solution is in the works with no splicing or explicit <tr>
access of <varname>buildPackages</varname> needed. For now, feel free to <td>build</td>
use either method. <td>host</td>
</tr>
<tr>
<td>build</td>
<td>target</td>
</tr>
<tr>
<td>host</td>
<td>host</td>
</tr>
<tr>
<td>host</td>
<td>target</td>
</tr>
<tr>
<td>target</td>
<td>target</td>
</tr>
</tbody>
</table>
</para> </para>
<note> <para>
<para> Some examples will make this table clearer. Suppose there's some package
There is also a "backlink" <varname>targetPackages</varname>, yielding a that is being built with a <literal>(build, host, target)</literal>
package set whose <varname>buildPackages</varname> is the current package platform triple of <literal>(foo, bar, baz)</literal>. If it has a
set. This is a hack, though, to accommodate compilers with lousy build build-time library dependency, that would be a "host → build" dependency
systems. Please do not use this unless you are absolutely sure you are with a triple of <literal>(foo, foo, *)</literal> (the target platform is
packaging such a compiler and there is no other way. irrelevant). If it needs a compiler to be built, that would be a "build →
</para> host" dependency with a triple of <literal>(foo, foo, *)</literal> (the
</note> target platform is irrelevant). That compiler, would be built with another
compiler, also "build → host" dependency, with a triple of <literal>(foo,
foo, foo)</literal>.
</para>
</section> </section>
<section xml:id="sec-cross-cookbook"> <section xml:id="ssec-cross-cookbook">
<title>Cross packaging cookbook</title> <title>Cross packaging cookbook</title>
<para> <para>
@ -450,21 +477,202 @@ nix-build &lt;nixpkgs&gt; --arg crossSystem '{ config = "&lt;arch&gt;-&lt;os&gt;
<section xml:id="sec-cross-infra"> <section xml:id="sec-cross-infra">
<title>Cross-compilation infrastructure</title> <title>Cross-compilation infrastructure</title>
<para> <section xml:id="ssec-cross-dependency-implementation">
To be written. <title>Implementation of dependencies</title>
</para>
<note>
<para> <para>
If one explores Nixpkgs, they will see derivations with names like The categorizes of dependencies developed in
<literal>gccCross</literal>. Such <literal>*Cross</literal> derivations is <xref
a holdover from before we properly distinguished between the host and linkend="ssec-cross-dependency-categorization"/> are specified as
target platforms—the derivation with "Cross" in the name covered the lists of derivations given to <varname>mkDerivation</varname>, as
<literal>build = host != target</literal> case, while the other covered the documented in <xref linkend="ssec-stdenv-dependencies"/>. In short,
<literal>host = target</literal>, with build platform the same or not based each list of dependencies for "host → target" of "foo → bar" is called
on whether one was using its <literal>.nativeDrv</literal> or <varname>depsFooBar</varname>, with exceptions for backwards
<literal>.crossDrv</literal>. This ugliness will disappear soon. compatibility that <varname>depsBuildHost</varname> is instead called
<varname>nativeBuildInputs</varname> and <varname>depsHostTarget</varname>
is instead called <varname>buildInputs</varname>. Nixpkgs is now structured
so that each <varname>depsFooBar</varname> is automatically taken from
<varname>pkgsFooBar</varname>. (These <varname>pkgsFooBar</varname>s are
quite new, so there is no special case for
<varname>nativeBuildInputs</varname> and <varname>buildInputs</varname>.)
For example, <varname>pkgsBuildHost.gcc</varname> should be used at
build-time, while <varname>pkgsHostTarget.gcc</varname> should be used at
run-time.
</para> </para>
</note>
<para>
Now, for most of Nixpkgs's history, there were no
<varname>pkgsFooBar</varname> attributes, and most packages have not been
refactored to use it explicitly. Prior to those, there were just
<varname>buildPackages</varname>, <varname>pkgs</varname>, and
<varname>targetPackages</varname>. Those are now redefined as aliases to
<varname>pkgsBuildHost</varname>, <varname>pkgsHostTarget</varname>, and
<varname>pkgsTargetTarget</varname>. It is acceptable, even
recommended, to use them for libraries to show that the host platform is
irrelevant.
</para>
<para>
But before that, there was just <varname>pkgs</varname>, even though both
<varname>buildInputs</varname> and <varname>nativeBuildInputs</varname>
existed. [Cross barely worked, and those were implemented with some hacks
on <varname>mkDerivation</varname> to override dependencies.] What this
means is the vast majority of packages do not use any explicit package set
to populate their dependencies, just using whatever
<varname>callPackage</varname> gives them even if they do correctly sort
their dependencies into the multiple lists described above. And indeed,
asking that users both sort their dependencies, <emphasis>and</emphasis>
take them from the right attribute set, is both too onerous and redundant,
so the recommended approach (for now) is to continue just categorizing by
list and not using an explicit package set.
</para>
<para>
To make this work, we "splice" together the six
<varname>pkgsFooBar</varname> package sets and have
<varname>callPackage</varname> actually take its arguments from that. This
is currently implemented in <filename>pkgs/top-level/splice.nix</filename>.
<varname>mkDerivation</varname> then, for each dependency attribute, pulls
the right derivation out from the splice. This splicing can be skipped when
not cross-compiling as the package sets are the same, but still is a bit
slow for cross-compiling. We'd like to do something better, but haven't
come up with anything yet.
</para>
</section>
<section xml:id="ssec-bootstrapping">
<title>Bootstrapping</title>
<para>
Each of the package sets described above come from a single bootstrapping
stage. While <filename>pkgs/top-level/default.nix</filename>, coordinates
the composition of stages at a high level,
<filename>pkgs/top-level/stage.nix</filename> "ties the knot" (creates the
fixed point) of each stage. The package sets are defined per-stage however,
so they can be thought of as edges between stages (the nodes) in a graph.
Compositions like <literal>pkgsBuildTarget.targetPackages</literal> can be
thought of as paths to this graph.
</para>
<para>
While there are many package sets, and thus many edges, the stages can also
be arranged in a linear chain. In other words, many of the edges are
redundant as far as connectivity is concerned. This hinges on the type of
bootstrapping we do. Currently for cross it is:
<orderedlist>
<listitem>
<para>
<literal>(native, native, native)</literal>
</para>
</listitem>
<listitem>
<para>
<literal>(native, native, foreign)</literal>
</para>
</listitem>
<listitem>
<para>
<literal>(native, foreign, foreign)</literal>
</para>
</listitem>
</orderedlist>
In each stage, <varname>pkgsBuildHost</varname> refers the the previous
stage, <varname>pkgsBuildBuild</varname> refers to the one before that, and
<varname>pkgsHostTarget</varname> refers to the current one, and
<varname>pkgsTargetTarget</varname> refers to the next one. When there is
no previous or next stage, they instead refer to the current stage. Note
how all the invariants regarding the mapping between dependency and depending
packages' build host and target platforms are preserved.
<varname>pkgsBuildTarget</varname> and <varname>pkgsHostHost</varname> are
more complex in that the stage fitting the requirements isn't always a
fixed chain of "prevs" and "nexts" away (modulo the "saturating"
self-references at the ends). We just special case each instead. All the primary
edges are implemented is in <filename>pkgs/stdenv/booter.nix</filename>,
and secondarily aliases in <filename>pkgs/top-level/stage.nix</filename>.
</para>
<note>
<para>
Note the native stages are bootstrapped in legacy ways that predate the
current cross implementation. This is why the the bootstrapping stages
leading up to the final stages are ignored inthe previous paragraph.
</para>
</note>
<para>
If one looks at the 3 platform triples, one can see that they overlap such
that one could put them together into a chain like:
<programlisting>
(native, native, native, foreign, foreign)
</programlisting>
If one imagines the saturating self references at the end being replaced
with infinite stages, and then overlays those platform triples, one ends up
with the infinite tuple:
<programlisting>
(native..., native, native, native, foreign, foreign, foreign...)
</programlisting>
On can then imagine any sequence of platforms such that there are bootstrap
stages with their 3 platforms determined by "sliding a window" that is the
3 tuple through the sequence. This was the original model for
bootstrapping. Without a target platform (assume a better world where all
compilers are multi-target and all standard libraries are built in their
own derivation), this is sufficient. Conversely if one wishes to cross
compile "faster", with a "Canadian Cross" bootstraping stage where
<literal>build != host != target</literal>, more bootstrapping stages are
needed since no sliding window providess the pesky
<varname>pkgsBuildTarget</varname> package set since it skips the Canadian
cross stage's "host".
</para>
<note>
<para>
It is much better to refer to <varname>buildPackages</varname> than
<varname>targetPackages</varname>, or more broadly package sets that do
not mention "target". There are three reasons for this.
</para>
<para>
First, it is because bootstrapping stages do not have a unique
<varname>targetPackages</varname>. For example a <literal>(x86-linux,
x86-linux, arm-linux)</literal> and <literal>(x86-linux, x86-linux,
x86-windows)</literal> package set both have a <literal>(x86-linux,
x86-linux, x86-linux)</literal> package set. Because there is no canonical
<varname>targetPackages</varname> for such a native (<literal>build ==
host == target</literal>) package set, we set their
<varname>targetPackages</varname>
</para>
<para>
Second, it is because this is a frequent source of hard-to-follow
"infinite recursions" / cycles. When only package sets that don't mention
target are used, the package set forms a directed acyclic graph. This
means that all cycles that exist are confined to one stage. This means
they are a lot smaller, and easier to follow in the code or a backtrace. It
also means they are present in native and cross builds alike, and so more
likely to be caught by CI and other users.
</para>
<para>
Thirdly, it is because everything target-mentioning only exists to
accommodate compilers with lousy build systems that insist on the compiler
itself and standard library being built together. Of course that is bad
because bigger derivations means longer rebuilds. It is also problematic because
it tends to make the standard libraries less like other libraries than
they could be, complicating code and build systems alike. Because of the
other problems, and because of these innate disadvantages, compilers ought
to be packaged another way where possible.
</para>
</note>
<note>
<para>
If one explores Nixpkgs, they will see derivations with names like
<literal>gccCross</literal>. Such <literal>*Cross</literal> derivations is
a holdover from before we properly distinguished between the host and
target platforms—the derivation with "Cross" in the name covered the
<literal>build = host != target</literal> case, while the other covered
the <literal>host = target</literal>, with build platform the same or not
based on whether one was using its <literal>.nativeDrv</literal> or
<literal>.crossDrv</literal>. This ugliness will disappear soon.
</para>
</note>
</section>
</section> </section>
</chapter> </chapter>

3
doc/reviewing-contributions.xml
View File

@ -189,7 +189,8 @@ $ git rebase --onto nixos-unstable BASEBRANCH FETCH_HEAD <co
</listitem> </listitem>
<listitem> <listitem>
<para> <para>
The <link xlink:href="https://github.com/Mic92/nix-review">nix-review</link> The
<link xlink:href="https://github.com/Mic92/nix-review">nix-review</link>
tool can be used to review a pull request content in a single command. tool can be used to review a pull request content in a single command.
<varname>PRNUMBER</varname> should be replaced by the number at the end <varname>PRNUMBER</varname> should be replaced by the number at the end
of the pull request title. You can also provide the full github pull of the pull request title. You can also provide the full github pull

41
doc/stdenv.xml
View File

@ -222,9 +222,10 @@ genericBuild
</footnote> </footnote>
But even if one is not cross compiling, the platforms imply whether or not But even if one is not cross compiling, the platforms imply whether or not
the dependency is needed at run-time or build-time, a concept that makes the dependency is needed at run-time or build-time, a concept that makes
perfect sense outside of cross compilation. For now, the run-time/build-time perfect sense outside of cross compilation. By default, the
distinction is just a hint for mental clarity, but in the future it perhaps run-time/build-time distinction is just a hint for mental clarity, but with
could be enforced. <varname>strictDeps</varname> set it is mostly enforced even in the native
case.
</para> </para>
<para> <para>
@ -348,7 +349,10 @@ let f(h, h + 1, i) = i + h
<para> <para>
Overall, the unifying theme here is that propagation shouldn't be Overall, the unifying theme here is that propagation shouldn't be
introducing transitive dependencies involving platforms the depending introducing transitive dependencies involving platforms the depending
package is unaware of. The offset bounds checking and definition of package is unaware of. [One can imagine the dependending package asking for
dependencies with the platforms it knows about; other platforms it doesn't
know how to ask for. The platform description in that scenario is a kind of
unforagable capability.] The offset bounds checking and definition of
<function>mapOffset</function> together ensure that this is the case. <function>mapOffset</function> together ensure that this is the case.
Discovering a new offset is discovering a new platform, and since those Discovering a new offset is discovering a new platform, and since those
platforms weren't in the derivation "spec" of the needing package, they platforms weren't in the derivation "spec" of the needing package, they
@ -2633,21 +2637,20 @@ addEnvHooks "$hostOffset" myBashFunction
happens. It prevents nix from cleaning up the build environment happens. It prevents nix from cleaning up the build environment
immediately and allows the user to attach to a build environment using immediately and allows the user to attach to a build environment using
the <command>cntr</command> command. Upon build error it will print the <command>cntr</command> command. Upon build error it will print
instructions on how to use <command>cntr</command>, which can be used instructions on how to use <command>cntr</command>, which can be used to
to enter the environment for debugging. Installing cntr and enter the environment for debugging. Installing cntr and running the
running the command will provide shell access to the build sandbox of command will provide shell access to the build sandbox of failed build.
failed build. At <filename>/var/lib/cntr</filename> the sandboxed At <filename>/var/lib/cntr</filename> the sandboxed filesystem is
filesystem is mounted. All commands and files of the system are still mounted. All commands and files of the system are still accessible
accessible within the shell. To execute commands from the sandbox use within the shell. To execute commands from the sandbox use the cntr exec
the cntr exec subcommand. Note that <command>cntr</command> also needs subcommand. Note that <command>cntr</command> also needs to be executed
to be executed on the machine that is doing the build, which might not on the machine that is doing the build, which might not be the case when
be the case when remote builders are enabled. <command>cntr</command> is remote builders are enabled. <command>cntr</command> is only supported
only supported on Linux-based platforms. To use it first add on Linux-based platforms. To use it first add <literal>cntr</literal> to
<literal>cntr</literal> to your your <literal>environment.systemPackages</literal> on NixOS or
<literal>environment.systemPackages</literal> on NixOS or alternatively alternatively to the root user on non-NixOS systems. Then in the package
to the root user on non-NixOS systems. Then in the package that is that is supposed to be inspected, add <literal>breakpointHook</literal>
supposed to be inspected, add <literal>breakpointHook</literal> to to <literal>nativeBuildInputs</literal>.
<literal>nativeBuildInputs</literal>.
<programlisting> <programlisting>
nativeBuildInputs = [ breakpointHook ]; nativeBuildInputs = [ breakpointHook ];
</programlisting> </programlisting>

19
doc/submitting-changes.xml
View File

@ -354,23 +354,22 @@ Additional information.
<title>Tested compilation of all pkgs that depend on this change using <command>nix-review</command></title> <title>Tested compilation of all pkgs that depend on this change using <command>nix-review</command></title>
<para> <para>
If you are updating a package's version, you can use nix-review to make sure all If you are updating a package's version, you can use nix-review to make
packages that depend on the updated package still compile correctly. sure all packages that depend on the updated package still compile
The <command>nix-review</command> utility can look for and build all dependencies correctly. The <command>nix-review</command> utility can look for and build
either based on uncommited changes with the <literal>wip</literal> option or all dependencies either based on uncommited changes with the
specifying a github pull request number. <literal>wip</literal> option or specifying a github pull request number.
</para> </para>
<para> <para>
review changes from pull request number 12345: review changes from pull request number 12345:
<screen>nix-shell -p nix-review --run "nix-review pr 12345"</screen> <screen>nix-shell -p nix-review --run "nix-review pr 12345"</screen>
</para> </para>
<para> <para>
review uncommitted changes: review uncommitted changes:
<screen>nix-shell -p nix-review --run "nix-review wip"</screen> <screen>nix-shell -p nix-review --run "nix-review wip"</screen>
</para> </para>
</section> </section>
<section xml:id="submitting-changes-tested-execution"> <section xml:id="submitting-changes-tested-execution">

10
pkgs/development/compilers/ghc/8.2.2.nix
View File

@ -1,4 +1,4 @@
{ stdenv, targetPackages { stdenv, pkgsBuildTarget, targetPackages
# build-tools # build-tools
, bootPkgs , bootPkgs
@ -70,11 +70,9 @@ let
++ stdenv.lib.optional (!enableIntegerSimple) gmp ++ stdenv.lib.optional (!enableIntegerSimple) gmp
++ stdenv.lib.optional (platform.libc != "glibc") libiconv; ++ stdenv.lib.optional (platform.libc != "glibc") libiconv;
toolsForTarget = toolsForTarget = [
if hostPlatform == buildPlatform then pkgsBuildTarget.targetPackages.stdenv.cc
[ targetPackages.stdenv.cc ] ++ stdenv.lib.optional useLLVM llvmPackages.llvm ] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
else assert targetPlatform == hostPlatform; # build != host == target
[ stdenv.cc ] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
targetCC = builtins.head toolsForTarget; targetCC = builtins.head toolsForTarget;

10
pkgs/development/compilers/ghc/8.4.4.nix
View File

@ -1,4 +1,4 @@
{ stdenv, targetPackages { stdenv, pkgsBuildTarget, targetPackages
# build-tools # build-tools
, bootPkgs , bootPkgs
@ -72,11 +72,9 @@ let
++ stdenv.lib.optional (!enableIntegerSimple) gmp ++ stdenv.lib.optional (!enableIntegerSimple) gmp
++ stdenv.lib.optional (platform.libc != "glibc" && !targetPlatform.isWindows) libiconv; ++ stdenv.lib.optional (platform.libc != "glibc" && !targetPlatform.isWindows) libiconv;
toolsForTarget = toolsForTarget = [
if hostPlatform == buildPlatform then pkgsBuildTarget.targetPackages.stdenv.cc
[ targetPackages.stdenv.cc ] ++ stdenv.lib.optional useLLVM llvmPackages.llvm ] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
else assert targetPlatform == hostPlatform; # build != host == target
[ stdenv.cc ] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
targetCC = builtins.head toolsForTarget; targetCC = builtins.head toolsForTarget;

10
pkgs/development/compilers/ghc/8.6.4.nix
View File

@ -1,4 +1,4 @@
{ stdenv, targetPackages { stdenv, pkgsBuildTarget, targetPackages
# build-tools # build-tools
, bootPkgs , bootPkgs
@ -72,11 +72,9 @@ let
++ stdenv.lib.optional (!enableIntegerSimple) gmp ++ stdenv.lib.optional (!enableIntegerSimple) gmp
++ stdenv.lib.optional (platform.libc != "glibc" && !targetPlatform.isWindows) libiconv; ++ stdenv.lib.optional (platform.libc != "glibc" && !targetPlatform.isWindows) libiconv;
toolsForTarget = toolsForTarget = [
if hostPlatform == buildPlatform then pkgsBuildTarget.targetPackages.stdenv.cc
[ targetPackages.stdenv.cc ] ++ stdenv.lib.optional useLLVM llvmPackages.llvm ] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
else assert targetPlatform == hostPlatform; # build != host == target
[ stdenv.cc ] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
targetCC = builtins.head toolsForTarget; targetCC = builtins.head toolsForTarget;

10
pkgs/development/compilers/ghc/head.nix
View File

@ -1,4 +1,4 @@
{ stdenv, targetPackages { stdenv, pkgsBuildTarget, targetPackages
# build-tools # build-tools
, bootPkgs , bootPkgs
@ -69,11 +69,9 @@ let
++ stdenv.lib.optional (!enableIntegerSimple) gmp ++ stdenv.lib.optional (!enableIntegerSimple) gmp
++ stdenv.lib.optional (platform.libc != "glibc" && !targetPlatform.isWindows) libiconv; ++ stdenv.lib.optional (platform.libc != "glibc" && !targetPlatform.isWindows) libiconv;
toolsForTarget = toolsForTarget = [
if hostPlatform == buildPlatform then pkgsBuildTarget.targetPackages.stdenv.cc
[ targetPackages.stdenv.cc ] ++ stdenv.lib.optional useLLVM llvmPackages.llvm ] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
else assert targetPlatform == hostPlatform; # build != host == target
[ stdenv.cc ] ++ stdenv.lib.optional useLLVM buildLlvmPackages.llvm;
targetCC = builtins.head toolsForTarget; targetCC = builtins.head toolsForTarget;

15
pkgs/development/compilers/go/1.11.nix
View File

@ -1,7 +1,8 @@
{ stdenv, fetchFromGitHub, tzdata, iana-etc, go_bootstrap, runCommand, writeScriptBin { stdenv, fetchFromGitHub, tzdata, iana-etc, go_bootstrap, runCommand, writeScriptBin
, perl, which, pkgconfig, patch, procps, pcre, cacert, llvm, Security, Foundation , perl, which, pkgconfig, patch, procps, pcre, cacert, llvm, Security, Foundation
, mailcap, runtimeShell , mailcap, runtimeShell
, buildPackages, targetPackages }: , buildPackages, pkgsTargetTarget
}:
let let
@ -152,16 +153,12 @@ stdenv.mkDerivation rec {
# {CC,CXX}_FOR_TARGET must be only set for cross compilation case as go expect those # {CC,CXX}_FOR_TARGET must be only set for cross compilation case as go expect those
# to be different from CC/CXX # to be different from CC/CXX
CC_FOR_TARGET = if (stdenv.hostPlatform != stdenv.targetPlatform) then CC_FOR_TARGET = if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${targetPackages.stdenv.cc}/bin/${targetPackages.stdenv.cc.targetPrefix}cc" "${pkgsTargetTarget.stdenv.cc}/bin/${pkgsTargetTarget.stdenv.cc.targetPrefix}cc"
else if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${stdenv.cc.targetPrefix}cc"
else else
null; null;
CXX_FOR_TARGET = if (stdenv.hostPlatform != stdenv.targetPlatform) then CXX_FOR_TARGET = if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${targetPackages.stdenv.cc}/bin/${targetPackages.stdenv.cc.targetPrefix}c++" "${pkgsTargetTarget.stdenv.cc}/bin/${pkgsTargetTarget.stdenv.cc.targetPrefix}c++"
else if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${stdenv.cc.targetPrefix}c++"
else else
null; null;

15
pkgs/development/compilers/go/1.12.nix
View File

@ -1,7 +1,8 @@
{ stdenv, fetchurl, tzdata, iana-etc, go_bootstrap, runCommand, writeScriptBin { stdenv, fetchurl, tzdata, iana-etc, go_bootstrap, runCommand, writeScriptBin
, perl, which, pkgconfig, patch, procps, pcre, cacert, llvm, Security, Foundation , perl, which, pkgconfig, patch, procps, pcre, cacert, llvm, Security, Foundation
, mailcap, runtimeShell , mailcap, runtimeShell
, buildPackages, targetPackages }: , buildPackages, pkgsTargetTarget
}:
let let
@ -154,16 +155,12 @@ stdenv.mkDerivation rec {
# {CC,CXX}_FOR_TARGET must be only set for cross compilation case as go expect those # {CC,CXX}_FOR_TARGET must be only set for cross compilation case as go expect those
# to be different from CC/CXX # to be different from CC/CXX
CC_FOR_TARGET = if (stdenv.hostPlatform != stdenv.targetPlatform) then CC_FOR_TARGET = if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${targetPackages.stdenv.cc}/bin/${targetPackages.stdenv.cc.targetPrefix}cc" "${pkgsTargetTarget.stdenv.cc}/bin/${pkgsTargetTarget.stdenv.cc.targetPrefix}cc"
else if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${stdenv.cc.targetPrefix}cc"
else else
null; null;
CXX_FOR_TARGET = if (stdenv.hostPlatform != stdenv.targetPlatform) then CXX_FOR_TARGET = if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${targetPackages.stdenv.cc}/bin/${targetPackages.stdenv.cc.targetPrefix}c++" "${pkgsTargetTarget.stdenv.cc}/bin/${pkgsTargetTarget.stdenv.cc.targetPrefix}c++"
else if (stdenv.buildPlatform != stdenv.targetPlatform) then
"${stdenv.cc.targetPrefix}c++"
else else
null; null;

4
pkgs/development/interpreters/guile/1.8.nix
View File

@ -1,4 +1,4 @@
{ stdenv, buildPackages { stdenv, pkgsBuildBuild, buildPackages
, fetchurl, makeWrapper, gawk, pkgconfig , fetchurl, makeWrapper, gawk, pkgconfig
, libtool, readline, gmp , libtool, readline, gmp
}: }:
@ -23,7 +23,7 @@ stdenv.mkDerivation rec {
depsBuildBuild = [ buildPackages.stdenv.cc ] depsBuildBuild = [ buildPackages.stdenv.cc ]
++ stdenv.lib.optional (stdenv.hostPlatform != stdenv.buildPlatform) ++ stdenv.lib.optional (stdenv.hostPlatform != stdenv.buildPlatform)
buildPackages.buildPackages.guile_1_8; pkgsBuildBuild.guile_1_8;
nativeBuildInputs = [ makeWrapper gawk pkgconfig ]; nativeBuildInputs = [ makeWrapper gawk pkgconfig ];
buildInputs = [ readline libtool ]; buildInputs = [ readline libtool ];

4
pkgs/development/interpreters/guile/2.0.nix
View File

@ -1,4 +1,4 @@
{ stdenv, buildPackages { stdenv, pkgsBuildBuild, buildPackages
, fetchpatch, fetchurl, makeWrapper, gawk, pkgconfig , fetchpatch, fetchurl, makeWrapper, gawk, pkgconfig
, libffi, libtool, readline, gmp, boehmgc, libunistring , libffi, libtool, readline, gmp, boehmgc, libunistring
, coverageAnalysis ? null , coverageAnalysis ? null
@ -22,7 +22,7 @@
depsBuildBuild = [ buildPackages.stdenv.cc ] depsBuildBuild = [ buildPackages.stdenv.cc ]
++ stdenv.lib.optional (stdenv.hostPlatform != stdenv.buildPlatform) ++ stdenv.lib.optional (stdenv.hostPlatform != stdenv.buildPlatform)
buildPackages.buildPackages.guile_2_0; pkgsBuildBuild.guile_2_0;
nativeBuildInputs = [ makeWrapper gawk pkgconfig ]; nativeBuildInputs = [ makeWrapper gawk pkgconfig ];
buildInputs = [ readline libtool libunistring libffi ]; buildInputs = [ readline libtool libunistring libffi ];

4
pkgs/development/interpreters/guile/default.nix
View File

@ -1,4 +1,4 @@
{ stdenv, buildPackages { stdenv, pkgsBuildBuild, buildPackages
, fetchurl, makeWrapper, gawk, pkgconfig , fetchurl, makeWrapper, gawk, pkgconfig
, libffi, libtool, readline, gmp, boehmgc, libunistring , libffi, libtool, readline, gmp, boehmgc, libunistring
, coverageAnalysis ? null , coverageAnalysis ? null
@ -23,7 +23,7 @@
depsBuildBuild = [ buildPackages.stdenv.cc ] depsBuildBuild = [ buildPackages.stdenv.cc ]
++ stdenv.lib.optional (stdenv.hostPlatform != stdenv.buildPlatform) ++ stdenv.lib.optional (stdenv.hostPlatform != stdenv.buildPlatform)
buildPackages.buildPackages.guile; pkgsBuildBuild.guile;
nativeBuildInputs = [ makeWrapper gawk pkgconfig ]; nativeBuildInputs = [ makeWrapper gawk pkgconfig ];
buildInputs = [ readline libtool libunistring libffi ]; buildInputs = [ readline libtool libunistring libffi ];

26
pkgs/stdenv/booter.nix
View File

@ -95,13 +95,25 @@ stageFuns: let
__hatPackages = nextStage; __hatPackages = nextStage;
}; };
}; };
in thisStage =
if args.__raw or false if args.__raw or false
then args' then args'
else allPackages ((builtins.removeAttrs args' ["selfBuild"]) // { else allPackages ((builtins.removeAttrs args' ["selfBuild"]) // {
buildPackages = if args.selfBuild or true then null else prevStage; adjacentPackages = if args.selfBuild or true then null else rec {
targetPackages = if args.selfBuild or true then null else nextStage; pkgsBuildBuild = prevStage.buildPackages;
}); pkgsBuildHost = prevStage;
pkgsBuildTarget =
if args.stdenv.targetPlatform == args.stdenv.hostPlatform
then pkgsBuildHost
else assert args.stdenv.hostPlatform == args.stdenv.buildPlatform; thisStage;
pkgsHostHost =
if args.stdenv.hostPlatform == args.stdenv.targetPlatform
then thisStage
else assert args.stdenv.buildPlatform == args.stdenv.hostPlatform; pkgsBuildHost;
pkgsTargetTarget = nextStage;
};
});
in thisStage;
# This is a hack for resolving cross-compiled compilers' run-time # This is a hack for resolving cross-compiled compilers' run-time
# deps. (That is, compilers that are themselves cross-compiled, as # deps. (That is, compilers that are themselves cross-compiled, as

3
pkgs/top-level/all-packages.nix
View File

@ -25,9 +25,6 @@ let
in in
{ {
# Allow callPackage to fill in the pkgs argument
inherit pkgs;
# A stdenv capable of building 32-bit binaries. On x86_64-linux, # A stdenv capable of building 32-bit binaries. On x86_64-linux,
# it uses GCC compiled with multilib support; on i686-linux, it's # it uses GCC compiled with multilib support; on i686-linux, it's
# just the plain stdenv. # just the plain stdenv.

23
pkgs/top-level/splice.nix
View File

@ -96,19 +96,20 @@ let
} @ args: } @ args:
if actuallySplice then spliceReal args else pkgsHostTarget; if actuallySplice then spliceReal args else pkgsHostTarget;
splicedPackages = splicePackages rec { splicedPackages = splicePackages {
pkgsBuildBuild = pkgs.buildPackages.buildPackages; inherit (pkgs)
pkgsBuildHost = pkgs.buildPackages; pkgsBuildBuild pkgsBuildHost pkgsBuildTarget
pkgsBuildTarget = pkgsHostHost pkgsHostTarget
if pkgs.stdenv.targetPlatform == pkgs.stdenv.hostPlatform pkgsTargetTarget
then pkgsBuildHost ;
else assert pkgs.stdenv.hostPlatform == pkgs.stdenv.buildPlatform; pkgsHostTarget;
pkgsHostHost = {}; # unimplemented
pkgsHostTarget = pkgs;
pkgsTargetTarget = pkgs.targetPackages;
} // { } // {
# These should never be spliced under any circumstances # These should never be spliced under any circumstances
inherit (pkgs) pkgs buildPackages targetPackages; inherit (pkgs)
pkgsBuildBuild pkgsBuildHost pkgsBuildTarget
pkgsHostHost pkgsHostTarget
pkgsTargetTarget
buildPackages pkgs targetPackages
;
inherit (pkgs.stdenv) buildPlatform targetPlatform hostPlatform; inherit (pkgs.stdenv) buildPlatform targetPlatform hostPlatform;
}; };

59
pkgs/top-level/stage.nix
View File

@ -21,18 +21,23 @@
## Other parameters ## Other parameters
## ##
, # The package set used at build-time. If null, `buildPackages` will , # Either null or an object in the form:
# be defined internally as the final produced package set itself. This allows
# us to avoid expensive splicing.
buildPackages
, # The package set used in the next stage. If null, `targetPackages` will be
# defined internally as the final produced package set itself, just like with
# `buildPackages` and for the same reasons.
# #
# THIS IS A HACK for compilers that don't think critically about cross- # {
# compilation. Please do *not* use unless you really know what you are doing. # pkgsBuildBuild = ...;
targetPackages # pkgsBuildHost = ...;
# pkgsBuildTarget = ...;
# pkgsHostHost = ...;
# # pkgsHostTarget skipped on purpose.
# pkgsTargetTarget ...;
# }
#
# These are references to adjacent bootstrapping stages. The more familiar
# `buildPackages` and `targetPackages` are defined in terms of them. If null,
# they are instead defined internally as the current stage. This allows us to
# avoid expensive splicing. `pkgsHostTarget` is skipped because it is always
# defined as the current stage.
adjacentPackages
, # The standard environment to use for building packages. , # The standard environment to use for building packages.
stdenv stdenv
@ -70,11 +75,33 @@ let
inherit (self) runtimeShell; inherit (self) runtimeShell;
}; };
stdenvBootstappingAndPlatforms = self: super: { stdenvBootstappingAndPlatforms = self: super: let
buildPackages = (if buildPackages == null then self else buildPackages) withFallback = thisPkgs:
// { recurseForDerivations = false; }; (if adjacentPackages == null then self else thisPkgs)
targetPackages = (if targetPackages == null then self else targetPackages)
// { recurseForDerivations = false; }; // { recurseForDerivations = false; };
in {
# Here are package sets of from related stages. They are all in the form
# `pkgs{theirHost}{theirTarget}`. For example, `pkgsBuildHost` means their
# host platform is our build platform, and their target platform is our host
# platform. We only care about their host/target platforms, not their build
# platform, because the the former two alone affect the interface of the
# final package; the build platform is just an implementation detail that
# should not leak.
pkgsBuildBuild = withFallback adjacentPackages.pkgsBuildBuild;
pkgsBuildHost = withFallback adjacentPackages.pkgsBuildHost;
pkgsBuildTarget = withFallback adjacentPackages.pkgsBuildTarget;
pkgsHostHost = withFallback adjacentPackages.pkgsHostHost;
pkgsHostTarget = self // { recurseForDerivations = false; }; # always `self`
pkgsTargetTarget = withFallback adjacentPackages.pkgsTargetTarget;
# Older names for package sets. Use these when only the host platform of the
# package set matter (i.e. use `buildPackages` where any of `pkgsBuild*`
# would do, and `targetPackages` when any of `pkgsTarget*` would do (if we
# had more than just `pkgsTargetTarget`).)
buildPackages = self.pkgsBuildHost;
pkgs = self.pkgsHostTarget;
targetPackages = self.pkgsTargetTarget;
inherit stdenv; inherit stdenv;
}; };
@ -87,7 +114,7 @@ let
inherit (hostPlatform) system; inherit (hostPlatform) system;
}; };
splice = self: super: import ./splice.nix lib self (buildPackages != null); splice = self: super: import ./splice.nix lib self (adjacentPackages != null);
allPackages = self: super: allPackages = self: super:
let res = import ./all-packages.nix let res = import ./all-packages.nix