In preparation for the deprecation of `stdenv.isX`.
These shorthands are not conducive to cross-compilation because they
hide the platforms.
Darwin might get cross-compilation for which the continued usage of `stdenv.isDarwin` will get in the way
One example of why this is bad and especially affects compiler packages
https://www.github.com/NixOS/nixpkgs/pull/343059
There are too many files to go through manually but a treewide should
get users thinking when they see a `hostPlatform.isX` in a place where it
doesn't make sense.
```
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv.is" "stdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv'.is" "stdenv'.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "clangStdenv.is" "clangStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "gccStdenv.is" "gccStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenvNoCC.is" "stdenvNoCC.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "inherit (stdenv) is" "inherit (stdenv.hostPlatform) is"
fd --type f "\.nix" | xargs sd --fixed-strings "buildStdenv.is" "buildStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "effectiveStdenv.is" "effectiveStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "originalStdenv.is" "originalStdenv.hostPlatform.is"
```
For a long time, we've had `crossLibcStdenv`, `*Cross` libc attributes,
and `*bsdCross` pre-libc package sets. This was always bad because
having "cross" things is "not declarative": the naming doesn't reflect
what packages *need* but rather how we *provide* something. This is
ugly, and creates needless friction between cross and native building.
Now, almost all of these `*Cross` attributes are gone: just these are
kept:
- Glibc's and Musl's are kept, because those packages are widely used
and I didn't want to risk changing the native builds of those at this
time.
- generic `libcCross`, `theadsCross`, and friends, because these relate
to the convolulted GCC bootstrap which still needs to be redone.
The BSD and obscure Linux or freestnanding libcs have conversely all
been made to use a new `stdenvNoLibc`, which is like the old
`crossLibcStdenv` except:
1. It usable for native and cross alike
2. It named according to what it *is* ("a standard environment without
libc but with a C compiler"), rather than some non-compositional
jargon ("the stdenv used for building libc when cross compiling",
yuck).
I should have done this change long ago, but I was stymied because of
"infinite recursions". The problem was that in too many cases we are
overriding `stdenv` to *remove* things we don't need, and this risks
cyles since those more minimal stdenvs are used to build things in the
more maximal stdenvs.
The solution is to pass `stage.nix` `stdenvNoCC`, so we can override to
*build up* rather than *tear down*. For now, the full `stdenv` is also
passed, so I don't need to change the native bootstraps, but I can see
this changing as we make things more uniform and clean those up.
(adapted from commit 51f1ecaa59)
(adapted from commit 1743662e55)
Most Linux distributions are enabling this these days and it does
protect against real world vulnerabilities as demonstrated by
CVE-2018-16864 and CVE-2018-16865.
Fix#53753.
Information on llvm version support gleaned from
6609892a2d68e07da3e5092507a730
Information on gcc version support a lot harder to gather,
but both 32bit and 64bit arm do appear to be supported
based on the test suite.
To build the security wrappers[1] the pkgsStatic stdenv is used, so the
binaries are static. However, the hostPlatform may have gcc attributes
that are *required* to build binaries so they can run on the host
platform. In particular, this is the case when using gcc.arch, which
ends up injecting -march=... in the gcc wrapper. Those attributes are
not contained in hostPlatform.parsed.
This change sets the same gcc attributes found in the hostPlatform for
the pkgsStatic cross system, so it can build binaries with the same gcc
flags.
[1]: nixos/modules/security/wrappers/default.nix
* Extend libc
Include non-libc core libraries in the libc package. Many of these
mirror libraries present in glibc on linux, such as libgcc, libraries
used for iconv, and libraries used for reading kernel info (libkvm,
libprocstat, libmemstat).
Without this many packages outside the freebsd tree would need to be
modified to include standard dependencies which would already be on
the system for other packages.
* Mark FreeBSD as using LLVM
* Update default LLVM version FreeBSD
* Use patch monolith
The patchesRoot system combined with the fact that each derivation
will Request specific names of patches makes it very annoying to use
other FreeBSD source trees with nixpkgs. This new system allows
providing one Or more entire trees of patches whose contents will be
dynamically Parsed and only the relevant patches will be applied for
any one Derivation.
With this commit, the following knobs are available for specifying the
FreeBSD source:
- overriding `freebsd.versionInfo`, for picking another official
supported FreeBSD release.
- overriding `freebsd.source` for specifying a specific unpatched
FreeBSD source tree.
- overriding `freebsd.patches`, for specifying the patches to apply.
Co-Authored-by: Audrey Dutcher <audrey@rhelmot.io>
Co-Authored-by: John Ericson <John.Ericson@Obsidian.Systems>
Replace writeReferencesToFile with writeClosure.
Make writeClosure accept a list of paths instead of a path.
Re-implement with JSON-based exportReferencesGraph interface provided by
__structuredAttrs = true.
Reword the documentation.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Someone Serge <sergei.kozlukov@aalto.fi>
Closes#193336Closes#261694
Related to #108984
The goal here was to get the following flake to build and run on
`aarch64-darwin`:
```nix
{ inputs.nixpkgs.url = <this branch>;
outputs = { nixpkgs, ... }: {
checks.aarch64-darwin.default =
nixpkgs.legacyPackages.aarch64-darwin.nixosTest {
name = "test";
nodes.machine = { };
testScript = "";
};
};
}
```
… and after this change it does. There's no longer a need for the
user to set `nodes.*.nixpkgs.pkgs` or
`nodes.*.virtualisation.host.pkgs` as the correct values are inferred
from the host system.
this equates to -ftrivial-auto-var-init=pattern
clang has removed support for -ftrivial-auto-var-init=zero and
are unlikely to re-add it, so use -ftrivial-auto-var-init=pattern
on both compilers if only to make behaviour more consistent
between the two.
add to pkgsExtraHardening's defaultHardeningFlags.
This introduces the `pkgs/by-name` directory as proposed by RFC 140.
Included are:
- The implementation to add packages defined in that directory to the
top-level package scope
- Contributer documentation on how to add packages to it
- A GitHub Actions workflow to check the structure of it on all PRs
While searching for something different I wondered why there is a
trivial-builders.nix file next to the trivial-builders directory where
only tests live. Lets fix that.
Many packages don't work on aarch64-darwin yet or are x86-only binary packages.
This gives aarch64-darwin users easy access to x86_64-darwin packages to run via
Rosetta 2.
"x86" was used as a short-hand instead of x86_64 because x86_64-darwin is the
only x86 Darwin package set.
Commit 82ff1f5db1 in #182807 removed two lines
from stage.nix which were responsible for making sure the `gcc` for `pkgsStatic`
on powerpc64 was built with the `--with-abi=elfv2` flag.
Unfortunately this causes build failures for `pkgsCross.ppc64.pkgsStatic`, as
reported here:
https://github.com/NixOS/nixpkgs/pull/182807#issuecomment-1247268226
This commit reverts the deletion.
Unfortunately ugly kludges like this are necessary because nixpkgs'
`lib/systems/` doesn't understand the difference between a libc and an abi. So
we have no clean way to tell nixpkgs "musl on big-endian powerpc64 always uses
the ELFv2 ABI" -- it thinks that musl is an ABI. Until that gets fixed there is
no better way to add the flag.
This simplifies usages and makes the default value consistent.
In a few cases, the default value was interpreted to be `false`,
but this is useless, because virtually nobody will explicitly
set `allowAliases = true;`.
MIPS has a large space of {architecture,abi,endianness}; this commit
adds all of them to lib/systems/platforms.nix so we can be done with
it.
Currently lib/systems/inspect.nix has a single "isMips" predicate,
which is a bit ambiguous now that we will have both mips32 and mips64
support, with the latter having two ABIs. Let's add four new
predicates (isMips32, isMips64, isMips64n32, and isMips64n64) and
treat the now-ambiguous isMips as deprecated in favor of the
more-specific predicates. These predicates are used mainly for
enabling/disabling target-specific workarounds, and it is extremely
rare that a platform-specific workaround is needed, and both mips32
and mips64 need exactly the same workaround.
The separate predicates (isMips64n32 and isMips64n64) for ABI
distinctions are, unfortunately, useful. Boost's user-scheduled
threading (used by nix) does does not currently supports mips64n32,
which is a very desirable ABI on routers since they rarely have
more than 2**32 bytes of DRAM.
The comment
a dirty hack that should be removed
has led me to believe that nixpkgsFun isn't the right solution,
but bypassing it is worse, because it creates a second, inner
overriding mechanism that doesn't pass its changes to the old,
outer overriding mechanism.
Before this change:
nix-repl> ((import <nixpkgs> {}).appendOverlays([(f: s: { foobarbaz = "ok"; })])).foobarbaz
"ok"
nix-repl> ((import <nixpkgs> {}).appendOverlays([(f: s: { foobarbaz = "ok"; })])).pkgsCross.aarch64-multiplatform.foobarbaz
error: attribute 'foobarbaz' missing
at «string»:1:1:
1| ((import <nixpkgs> {}).appendOverlays([(f: s: { foobarbaz = "ok"; })])).pkgsCross.aarch64-multiplatform.foobarbaz
| ^
2|
After this change:
nix-repl> ((import ./. {}).appendOverlays([(f: s: { foobarbaz = "ok"; })])).pkgsCross.aarch64-multiplatform.foobarbaz
"ok"
Thanks to samueldr for discovering this problem.
The old stdenv adapters were subtly wrong in two ways:
- `overrideAttrs` leaked the original, unoverridden `mkDerivation`.
- `stdenv.override` would throw away any manually-set `mkDerivation`
from a stdenv reverting to the original.
Now, `mkDerivation` is controlled (nearly directly) via an argument, and
always correctly closes over the final ("self") stdenv. This means the
adapters can work entirely via `.override` without any manual `stdenv //
...`, and both those issues are fixed.
Note hashes are changed, because stdenvs no previously overridden like
`stdenvNoCC` and `crossLibcStdenv` now are. I had to add some
`dontDisableStatic = true` accordingly. The flip side however is that
since the overrides compose, we no longer need to override anything but
the default `stdenv` from which all the others are created.
Fixes pkgsCross.muslpi.nix (depends on
pkgsCross.muslpi.busybox-sandbox-shell, which depends on
pkgsCross.muslpi.pkgsStatic.stdenv)
Currently, the ABI adapter in pkgsStatic does not
recognize musleabi/musleabihf and falls back to "musl".
```
> nix eval -f . pkgsCross.muslpi.stdenv.hostPlatform.config
"armv6l-unknown-linux-musleabihf"
> nix eval -f . pkgsCross.muslpi.pkgsStatic.stdenv.hostPlatform.config
"armv6l-unknown-linux-musl"
```
This results in an invalid config passed to GCC configuration, which
fails with `Configuration armv6l-unknown-linux-musl not supported`.
PR #122778 allowed the linker being chosen independently from useLLVM
which also affected pkgsLLVM where we were relying on this behavior.
For platform sets assembled from scratch useLLVM still implies
linker == "lld", however in the case of pkgsLLVM we update the current
platform via the set update operator which means that `linker` won't
be re-evaluated. Using ld.bfd with pkgsLLVM is okay to a certain extent,
but with C++ things begin to break.
We fix this by setting linker explicitly.
Provide a version of nixpkgs which is built using the LLVM toolchain.
This is achieved by reusing the useLLVM attribute usable in the cross
toolchain bootstrapping which is a) perfectly acceptable and b) the only
way to bootstrap a LLVM stdenv across platforms without unreasonable
amounts of work.
This is mostly working now since we fixed some issues with
llvmPackages_7.lldClang in #119717 and compile glibc with gcc at all
times as of #119786 (since clang can't compile our glibc at the moment).
There are seemingly some issues left to be ironed out, for example the
C++ library of ncurses fails to build with a linking issue. I have
attempted to alleviate that problem with #120790, but to no avail so
far.
Nonetheless, having a working pkgsLLVM.hello is a good start.
A currently present issue is that there is no way to change the libc
since there is some issue with lldClang actually picking up on the set
libc. This will need investigation and fixing.
The staging logic reconstructs the target platform, discarding
powerpc64's custom gcc.abi = elfv2 setup. Musl requires ELFv2 ABI so
this should be set unconditionally here.
