Commit 986f361946 started to use
<nix/fetchurl.nix> to "download" the bootstrap binaries from the
Nixpkgs tree, using the file:/// scheme. This has really bad
consequences:
* It makes any derivation depend on the path of the Nixpkgs tree. So
evaluating a package will produce a different .drv file when run
from different locations. No wonder Hydra evaluation has been so
slow lately: for every Nixpkgs evaluation, it had to create tens of
thousands of .drv files, even if nothing had changed.
* It requires the builder to have file system access to the Nixpkgs
tree. So if your tree is in your home directory, the stdenv
bootstrap would probably fail.
So now the binaries are downloaded from tarballs.nixos.org.
Also dropped PowerPC "support".
Stdenv adapters are kinda weird and un-idiomatic (especially when they
don't actually change stdenv). It's more idiomatic to say
buildInputs = [ makeCoverageAnalysisReport ];
This removes the need for hacks like stdenv.regenerate. It also
ensures that overrideGCC is now stackable (so ‘stdenv = useGoldLinker
clangStdenv’ works).
setup.sh uses the anti-pattern `for f in $(find ...); do` in several
places. `find` returns one path per line, but `for` splits its arguments
by words, so paths which contain spaces are incorrectly split! The
correct way is `find ... | while read f; do`
Binutils nowadays contains ld.gold, which depends on libstdc++. So it
needs to be built with the new GCC rather than the one from
bootstrap-tools.
Issue #1469.
Treating fixupPhase specially is really ugly. Also, it collides with
the work in the multiple-outputs branch (which already has support for
fixing up all outputs).
Partial revert of 0a44a09121.
Some programs, e.g. guile-config, has a shebang that ends in '\':
#!/usr/bin/guile-1.8 \
-e main -s
!#
;;;; guile-config --- utility for linking programs with Guile
;;;; Jim Blandy <jim@red-bean.com> --- September 1997
This currently breaks patchShebangs:
$ read oldPath arg0 args <<< 'shebang \'; echo $?
1
$ echo $oldPath
shebang
$ echo $arg0
$ echo $args
(And setup.sh/patchShebangs is run with 'set -e' so any command that
return non-zero aborts the build.)
Fix by telling 'read' to not interpret backslashes (with the -r flag):
$ read -r oldPath arg0 args <<< 'shebang \'; echo $?
0
$ echo $oldPath
shebang
$ echo $arg0
\
$ echo $args
Also needed: escape the escape characters so that sed doesn't interpret
them.
patchShebangs has a bug that shows itself on files that have the
executable bit set but have no shebang (i.e. a blank/empty first line).
The shell would then evaluate this:
if [ != '#!' ]; then
# not evaluated
fi
With proper quoting we get the correct behaviour:
if [ "" != '#!' ]; then
# this will be evaluated
fi
When building e.g. perl for the first time there is no perl in PATH yet,
so command -v perl will fail.
This brings back the previous behaviour of silently not patching
shebangs for which there is no available command in PATH.
Currently "/usr/bin/env python" is rewritten to "/nix/store/.../env
python". That doesn't really improve anything because the interpreter
still have to be located in $PATH at runtime. The result is that many
nix package expressions do .../bin/env fixup themselves.
Instead of everyone having to do this patching locally, add the
functionality to the standard environment patchShebangs function so that
everyone can benefit.
The function ‘mkDerivation’ now checks whether the current platform
type is included in a package's meta.platform field. If not, it
throws an exception:
$ nix-build -A linux --argstr system x86_64-darwin
error: user-thrown exception: the package ‘linux-3.10.15’ is not supported on ‘x86_64-darwin’
These packages also no longer show up in ‘nix-env -qa’ output. This
means, for instance, that the number of packages shown on
x86_64-freebsd has dropped from 9268 to 4764.
Since meta.platforms was also used to prevent Hydra from building some
packages, there now is a new attribute meta.hydraPlatforms listing the
platforms on which Hydra should build the package (which defaults to
meta.platforms).
meta.license is can be a string or a list of strings. But there is one
unhandled case where "unfree" (or "unfree-redistributable") is a part of
a list. It will currently not be detected as an "unfree" package and
Hydra will attempt to build it. This should fix it.
Example: http://hydra.nixos.org/build/6553461
set CMAKE_LIBRARY_PATH, CMAKE_INCLUDE_PATH based on NIX_CFLAGS_COMPILE and
NIX_LDFLAGS so that cmake's find_library like functions find all the libraries
gcc knows about thanks to the gcc wrapper
This is particular useful with myEnvFun which then also sets those CMAKE_* env
variables.`
Because setup.sh has to change this causes many rebuilds - thus it should be
included in a stdenv-update like branch
Also cmake builds in parallel perfectly fine
update cmake to latest minor number, I didn't change the patches,
just reapplied them manually recordin a new patch
Since nix-1.4, nix's corepkgs contain a fetchurl suitable for
downloading the bootstrap binaries. Doing this will allow us to have a
nixpkgs with no in-tree binaries without breaking the purity of the
bootstrap (though for now, they are fetched in-tree until the binaries
are added to nixos.org somewhere). As an additional small benefit, the
in-tree binaries do not have to be hashed on every instantiation as they
do now.
The fetchurl in nix-1.2 and 1.3 isn't able to make binaries executable,
so it can't be used for this case. In that case, attempting to build the
bootstrap will show a message asking the user to manually download each
file and add it to the store with nix-store --add (but the hash is
ultimately the same, of course).
Signed-off-by: Shea Levy <shea@shealevy.com>
Conflict in kerberos, which was updated both in master and in
stdenv-updates. Kept the stdenv-updates version, except pulled in the
enableParallelBuilding change from master.
Signed-off-by: Shea Levy <shea@shealevy.com>
Conflicts:
pkgs/development/libraries/kerberos/krb5.nix
Before this, the passthru attributes were only merged in with the
derivation attribute set, and there was no way to distinguish after the
fact which attributes were part of the derivation and which came from
passthru. Now passthru can be looked at separately as well.
Signed-off-by: Shea Levy <shea@shealevy.com>
Conflicts:
pkgs/development/libraries/libxslt/default.nix
Commit 1764ea2b0a introduced changes to libxslt
in an awkward way to avoid re-builds on Linux. This patch has been simplified
during this merge.
With multiple outputs, adding attributes to a derivation without
changing the {drv,out}Path is no longer as trivial as simply using the
`//' operator, as we usually want to add the attribute to _each_ output,
and even if we only care about one that one output can be reached via
multiple paths.
For stdenv.mkDerivation, we already had code in place to add passthru
and meta attributes to derivations. This commit simply factors part of
that code out into a lib function addPassthru, which takes a derivation
and an attribute set and appends the attribute set to each output of the
derivation.
Signed-off-by: Shea Levy <shea@shealevy.com>
Before, only the first output (and not even that when accessed through 'all' or its corresponding attribtue) had meta information and the relevant passthru attributes.
This doesn't change stdenv's hash and the tarball still builds, I'm pretty sure this is safe for master.
I'm not sure whether this was by intention, but so far postPatch hooks were
silently skipped whenever the patches list was empty. This change could possibly
change the build results of the following packages:
* gcc
* cmake (264)
* systemtap
* quemu-kvm
These packages all have in common that they have a postPatch hook and the
patches list can be empty when certain conditions are met.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Due to xz being override in the last stdenv and also in the previous, the
nixpkgs xz ended up being built by bootstrap-tools, and thus depending on it
through libgcc_so.so.1. That ends up making 'nix' with a runtime
dependency on bootstrap-tools.
If set to false, mkDerivation will throw an exception if a package has
an unfree license. ‘release-lib.nix’ uses this to enforce that we
don't build unfree packages as part of the Nixpkgs channel. Since
this is set through Nixpkgs' ‘config’ argument, it's more finegrained
than $HYDRA_DISALLOW_UNFREE.
Although patching it made some programs run (configure tests), some others
crashed with segfault. So I don't think there is any win patching it. The
proper way to solve the bootstrap in the raspberry pi is, as far as I've been
testing, use glibc 2.17 libs in bootstrap-tools with the same ld.so name as the
bootstrapped glibc.
This is a problem inherent in our way to bootstrap, that first replaces
the glibc of a given gcc+glibc (bootstrap-tools) with gcc-wrapper tricks, and
then builds a new gcc. A nicer way would be to build a gcc without glibc,
then the glibc, then the final gcc, as we do with cross-tools.
Some comments about this problem in
https://github.com/NixOS/nixpkgs/issues/234#issuecomment-11764352
This solves the problem of the change of name in ld.so between
glibc 2.13 and 2.17 (at least for armhf). Some comments about it in
https://github.com/NixOS/nixpkgs/issues/234#issuecomment-11764352
As there says, without this, gmp (after glibc built) tests crashed
- segfault.
Now that we have a way to alter /bin/sh in chroots on non-NixOS Linux
platforms, it may be useful to have access to stdenv.shell package and to
static bash contained in bootstrap tools. So make them accessible via stdenv
attribute set.
libstdc++ and libmudflapth link to the dynamic loader; if the
bootstrap uses another dynamic loader name, and the rpath of these
libs isn't changed, they will fail to load.
Conflicts:
pkgs/development/compilers/gcc/4.6/default.nix
pkgs/development/compilers/gcc/4.7/default.nix
The 4.7 had some weird parameters added in crossAttrs; I've removed
them, but I don't understand where they come from.
This is for consistency with terminology in stdenv (and the terms
"hostDrv" and "buildDrv" are not very intuitive, even if they're
consistent with GNU terminology).
This allows various applications. It allows users to set global
optimisation flags, e.g.
stdenv.userHook = ''NIX_CFLAGS_COMPILE+=" -funroll-loops"'';
But the impetus is as an alternative to issue #229, allowing impure
stdenv setup for people who want to use distcc:
stdenv.userHook = "source /my/impure/setup-script.sh";
This is probably a bad idea, but at least now it's a bad idea in
people's configuration and not in Nixpkgs. :-)
This adapter causes the resulting binaries to have debug info and no
optimisations. Example use (in all-packages.nix):
foo = callPackage ./foo.nix {
stdenv = keepDebugInfo stdenv;
};
If the environment variable HYDRA_DISALLOW_UNFREE is set to "1", then
evaluation of a package with license "unfree" will throw an error.
Thus such packages or any packages that depend on them will fail to
evaluate.
what the new nix thinks the fuloong is.
Anyone having the old nix should use a nixpkgs previous to this change to build
the new nix. And then, with the new nix, he can use any newer nixpkgs revision.
svn path=/nixpkgs/trunk/; revision=31751
bash's pattern replacement feature. "replace-literal" is an
uncommon command so it was a headache during the bootstrap.
svn path=/nixpkgs/branches/stdenv-updates/; revision=31681
some redundant builds (e.g., GMP was built three times).
* Updated GMP to 5.0.2.
* Updated PPL to 0.11.2.
* Remove ad hoc flags to build GCC's dependencies statically.
Instead, use the ‘makeStaticLibraries’ stdenv adapter.
* Build GMP with C++ support by default.
svn path=/nixpkgs/branches/stdenv-updates/; revision=30891
These markers follow the format of those of `nix-store --print-build-trace',
which allows extraction of, say, phase durations in a similar way.
svn path=/nixpkgs/branches/stdenv-updates/; revision=26115
does not override NIX_CFLAGS_COMPILE anymore in the mkDerivation parameter
attributes. This way, apacheHttpd can be built properly with coverage
information.
An indication of this problem came from the nixos tests.subversion failure.
svn path=/nixpkgs/branches/stdenv-updates/; revision=24938
bootstrap-tools, because zlib was there and binutils were not having an
explicit buildInput for zlib. Due to that, we ended up with stdenv
(gcc-wrapper) depending on bootstrap-tools as runtime dependency.
svn path=/nixpkgs/branches/stdenv-updates/; revision=24790
newest I uploaded yesterday. These have gcc 4.5.1, that fulfills the glibc
2.12.1 requirement of having a compiler equal or greater to gcc 4.4 to build
proper x86 32-bit code (http://sourceware.org/bugzilla/show_bug.cgi?id=12123)
svn path=/nixpkgs/branches/stdenv-updates/; revision=24523
had a little svn mess in the working directory that ended up in a not working
stdenvLinux.
This time I even chose better names for the attributes, so they match better
the comments.
svn path=/nixpkgs/branches/stdenv-updates/; revision=23428
I had done an attempt recently, unsuccesful, which ended in a recent revert.
This change works.
I even updated the comments in the file.
svn path=/nixpkgs/branches/stdenv-updates/; revision=23425
What I did results in segfaults in built binaries.
svn merge -c -23370 ^/nixpkgs/branches/stdenv-updates
svn path=/nixpkgs/branches/stdenv-updates/; revision=23376
This should allow keeping the i686-linux boostrap-tools.
I updated some days ago the x86_64-linux bootstrap-tools, but that update is
not needed anymore. We can revert that boostrap-tools update if anyone wants.
Pro:
- new gcc building gcc and glibc.
Contra:
- maybe some old systems (patched red had kernels come to mind) break with that update?
svn path=/nixpkgs/branches/stdenv-updates/; revision=23370
Allowing 'curl' build without some dependencies, if it is told not to build
with them.
Updating the make-bootstrap-tools for the latest gcc/glibc, removing the
dependency on klibc, and updating unpack-boostrap-tools for the latest
boostrap-tools to work.
svn path=/nixpkgs/branches/stdenv-updates/; revision=23305
If a build expressions has set "enableParallelBuilding = true", then the
generic builder may utilize more than one CPU core to build that particular
expression. This feature works out of the box for GNU Make. Expressions that
use other build drivers like Boost.Jam or SCons have to specify appropriate
flags such as "-j${NIX_BUILD_CORES}" themselves.
svn path=/nixpkgs/trunk/; revision=23042
Instead, use the generic package override mechanism to use packages
from earlier bootstrap phases.
* Don't rely on the existence of attributes such as
`stdenv.coreutils'.
svn path=/nixpkgs/trunk/; revision=22991
If a build expressions has set "enableParallelBuilding = true", then the
generic builder may utilize more than one CPU core to build that particular
expression. This feature works out of the box for GNU Make. Expressions that
use other build drivers like Boost.Jam or SCons have to specify appropriate
flags such as "-j${NIX_BUILD_CORES}" themselves.
svn path=/nixpkgs/branches/stdenv-updates/; revision=22399
"meta" and "passthru", and these attributes will be appended to the usual
mkDerivation attributes only if the package is cross built.
This allows putting some of the cross-building logic in the mkDerivation
nix parameters, and not only in the final builder script, as it was until now.
svn path=/nixpkgs/trunk/; revision=20272
* libpng updated to 1.4.0.
* For libjpegStatic, use a stdenv adapter to build a static library.
svn path=/nixpkgs/branches/stdenv-updates/; revision=19605
I fixed conflicts regarding the renaming 'kernel' -> 'linux' in all-packages.
Also a small conflict in all-packages about making openssl overridable.
And I some linux 2.6.31-zen kernel files also marked in conflict.
svn path=/nixpkgs/branches/stdenv-updates/; revision=19438
we can't build easily with them gcc-4.4 natively, because of a bug in the ld of the
previous bootstrap-tools.
I updated the unpack script to include the new gcc-4.4 libraries *ppl*.so in patchelfing.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18835
(dynamic libraries of ppl and cloogppl) into the package.
We need newer binutils in the bootstrap-tools for the armv5tel in order to
be able to build gcc 4.4 from them.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18747
renaming.
I think directory renaming breaks the usual merges... because it leaves the
'to be removed' directory in the working directory still. A manual 'rm' of the
'to be removed' directory fixed the commit.
svn merge ^/nixpkgs/trunk
svn path=/nixpkgs/branches/stdenv-updates/; revision=18661
this because 'ld' wants to know the path of every library involved in a dynamic
linking. I imagine that ld does not need that in native builds because it can
call the loader for it to resolve the library rpaths, but this is not the case
for cross-building.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18577
inputs did not get in)
Updating the xorg builder script to support cross building (in fact, support
for propagating the required build inputs).
svn path=/nixpkgs/branches/stdenv-updates/; revision=18569
- Disabling guile test, because one fails. I commented on that in the source.
On cross builds:
- Adding stripping
- Updating the glibc-2.11 expression to match the parameters of glibc-2.9,
which I was updating more.
- Renaming from selfNativeBuildInput to selfBuildNativeInput, so this matches
better the pattern buildNativeInputs.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18550
- Before this changes, cflags and ldflags for the native and the cross compiler
got mixed. Not all the gcc-wrapper/gcc-cross-wrapper variables are
independant now, but enough, I think.
- Fixed the generic stdenv expression, which did a big mess on buildInputs and
buildNativeInputs. Now it distinguishes when there is a stdenvCross or not.
Maybe we should have a single stdenv and forget about the stdenvCross
adapter - this could end in a stdenv a bit complex, but simpler than the
generic stdenv + adapter.
- Added basic support in pkgconfig for cross-builds: a single PKG_CONFIG_PATH
now works for both the cross and the native compilers, but I think this
should work well for most cases I can think of.
- I tried to fix the guile expression to cross-biuld; guile is built, but not
its manual, so the derivation still fails. Guile requires patching to
cross-build, as far as I understnad.
- Made the glibcCross build to be done through the usage of a
gcc-cross-wrapper over the gcc-cross-stage-static, instead of using it
directly.
- Trying to make physfs (a neverball dependency) cross build.
- Updated the gcc expression to support building a cross compiler without getting
derivation variables mixed with those of the stdenvCross.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18534
on a derivation with a meta.license attribute which does not satisfy the
license predicate.
With this adapter you can abort any install which depends on software
which are not free by default. You can try it with MPlayer, because
MPlayer depends of win32codecs flagged as "unfree".
svn path=/nixpkgs/trunk/; revision=18530
its dependencies. To make it works, you need to change the default
stdenv as documented in the error message.
./maintainers/scripts/dep-licenses.sh <attribute name>
svn path=/nixpkgs/trunk/; revision=18508
statement of the gcc-wrapper coreutils dependency on r17867.
I don't have i686-darwin to try this.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18490
true/false, which tells whether the derivation needs itself as
buildNativeInput.
For example, in order to build cross ncurses, we need the a native build
ncurses.
(As libtool does not work in stdenv, I have not tested this change, to check
whether finally ncurses cross-build)
svn path=/nixpkgs/branches/stdenv-updates/; revision=18489
derivation, the "buildInputs" in every stdenv mkDerivation don't map now
directly to the environment
variable "buildInputs" in the builder, but "buildNativeInputs". So, the inputs
build by the native compiler.
When cross compiling, they will map to the environment variable "buildInputs"
(yes, now the same name), which means does to be built with the cross compiler.
I think I improved the naming of variables a bit. There was a big mess,
specially in the stdenv adapter for cross building, and also in the default
builder script.
I also tried to add proper manager of propagatedInputBuilds, these being
propagated considering the host or build origin of that input build (so, at the
end, being those propagatedInputBuilds being propagated properly to the native
or the cross compiler.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18477
the cross compilation functionality.
- I renamed some expected stdenv.mkDerivation parameter attributes so we can
keep this branch properly updated from trunk. We agreed with Nicolas Pierron
doing a massive renaming, so all current buildInputs become hostInputs (input
as build for the host machine, in autotools terminology) , and
then buildInputs would mean "input as for the build machine".
By now, the specific "input as for the build machine" is specified through
buildNativeInputs. We should fix this in the merge to trunk.
- I made the generic stdenv understand the buildNativeInputs, otherwise if
we start changing nixpkgs expressions so they distinguish the current
buildInputs into buildInputs and buildNativeInputs, we could break even more
nixpkgs for other platforms.
- I changed the default result of mkDerivation so it becomes the derivation for
to be run in the build machine. This allows, without any special rewriting,
"fetchurl" derivations to be always results for the build machine to use
them.
- The change above implies that, for anyone wanting to cross-compile, has to
build the hostDrv of the wanted derivation. For example, after this commit,
the usual test of "nix-build -A bison.hostDrv arm.nix" works. I described
the contents of this arm.nix in r18398.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18471
I made the stdenvCross adapter simpler, according to Nicolas Pierron comments,
and I commented it a bit.
There are still jobs to do. At least:
- Plan for the general renaming from buildInputs to hostInputs
- We should not break merges from trunk.
- Make the generic stdenv understand about host/buildInputs, at least for
native builds, because it is used in the always-native building of
stdenvLinux. This should allow us to remove all duplications of "*NoCross" in
nixpkgs.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18449
needing to keep a new tree of expressions apart for the expressions to get
cross-compiled.
I changed the whole way of using cross compilation with nixpkgs, which before
was done through a simple adapter.
Now the adapter became complex, and I've tried to avoid the most obvious
recursivities. For example, the fetchurl expression should
never be cross-compiled, as the gmp, mpfr, and some others, like
some ncurses, perl, ... I made overrided copies of those necessary as
perlNoCross, ncursesNoCross, as stdenvNoCross, keeping in mind that
the stdenv (capable of cross compilation) is built upon stdenvNoCross using
an adapter.
So, to cross compile, instead of building using "nixpkgs/default.nix",
you should build with your
own "myarchiteture.nix", which should have contents like these, for example:
import /etc/nixos/nixpkgs/default.nix
{
crossSystem = {
config = "armv5tel-unknown-linux-gnueabi";
bigEndian = false;
arch = "arm";
float = "soft";
};
}
svn path=/nixpkgs/branches/stdenv-updates/; revision=18398
It still does not work, but I think I already get glibc cross compiled.
Next: gcc and g++, and set some setup script hooks on stdenvCross.
It took quite enough hours for this commit.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18351
My idea is to provide special stdenv expressions that will contain in the path
additional cross compilers. As most expressions for programs accept a stdenv parameter,
we could substitute this parameter with the special stdenv, which will have a
generic builder that attempts the usual "--target=..." and can additionally
have an env variable like "cross" with the target architecture set.
So, finally we could have additional expressions like this:
bashRealArm = makeOverridable (import ../shells/bash) {
inherit fetchurl bison;
stdenv = stdenvCross "armv5tel-unknown-linux-gnueabi";
};
Meanwhile it does not work - I still cannot get the cross-gcc to build.
I think it does not fill the previous expressions with a lot of noise, so I
think it may be a good path to follow.
I only touched some files of the current stdenv: gcc-4.3, kernel headers
2.6.28, glibc 2.9, ...
I tried to use the gcc-cross-wrapper, that may be very outdated. Maybe I will
update it, or update the gcc-wrapper expression to make it fit the cross tools,
but meanwhile I even cannot build gcc, so I have not tested the wrapper.
This new idea on cross compiling is not similar to that of the
nixpkgs/branches/cross-compilation, which mostly added bare new expressions for
anything to be cross compiled, if I understood it correctly.
I cared not to break anything of the usual stdenv in all this work.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18343
This comes from:
svn diff ^/nixpkgs/trunk/@18255 ^/nixpkgs/branches/stdenv-updates/ > diff
patch -p0 < diff
and then adding into svn all files new from the patch.
trunk@18255 comes from the last time I updated stdenv-updates from trunk.
svn path=/nixpkgs/stdenv-updates2/; revision=18272
