The Darwin stdenv rework conditionally sets `NIX_CC_USE_RESPONSE_FILE`
depending on the `ARG_MAX` of the build system. If it is at least 1 MiB,
the stdenv passes the arguments on the command-line (like Linux).
Otherwise, it falls back to the response file. This was done to prevent
intermitent failures with clang 16 being unable to read the response
file. Unfortunately, this breaks `gccStdenv` on older Darwin platforms.
Note: While the stdenv logic will also be reverted, this change is
needed for compatibility with clang 16.
GCC is capable of using a response file, but it does not work correctly
when the response file is a file descriptor. This can be reproduced
using the following sequence of commands:
$ nix shell nixpkgs#gcc; NIX_CC_USE_RESPONSE_FILE=1 gcc
# Linux
/nix/store/9n9gjvzci75gp2sh1c4rh626dhizqynl-binutils-2.39/bin/ld: unrecognized option '-B/nix/store/vnwdak3n1w2jjil119j65k8mw1z23p84-glibc-2.35-224/lib/'
/nix/store/9n9gjvzci75gp2sh1c4rh626dhizqynl-binutils-2.39/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
# Darwin
ld: unknown option: -mmacosx-version-min=11.0
collect2: error: ld returned 1 exit status
Instead of using process substitution, create a temporary file and
remove it in a trap. This should also prevent the intermitent build
failures with clang 16 on older Darwin systems.
Fixes#245167
When a response file is in use, "$*" contains the response file and not
the parameters; both the linker and compiler wrappers are updated to use
the response-expanded params.
The compiler driver likes to pass parameters to the linker via a
response file, including -shared.
LLD rejects the combination of (-shared -pie), whereas other linkers
silently ignore the contradiction:
```
ld.lld: error: -shared and -pie may not be used together
```
This breaks certain configurations using LLD as a linker.
Changing `add-hardening.sh` results in a full rebuild. To avoid the
rebuild, here is a quick test case which shows the new hardening script
allows the link to succeed:
```
{ pkgs ? import <nixpkgs> {} }:
let
# gcc silently accepts -shared -pie together, lld does not.
linker = pkgs.wrapBintoolsWith { bintools = pkgs.llvmPackages.lld; };
patchWrapper = prev: prev.overrideAttrs (final: prev: let
prevScript = builtins.match (".*(/nix/store/[a-z0-9]+-add-hardening.sh).*") prev.postFixup;
in {
postFixup = (builtins.replaceStrings prevScript ["${./new-add-hardening.sh}"] prev.postFixup);
});
in
pkgs.stdenv.mkDerivation {
name = "nixpkgs-hardening-bug";
src = pkgs.writeText "src.c" "int main(int argc, char* argv[]) { return 0; }";
NIX_HARDENING_ENABLE = "pie";
unpackPhase = ":";
buildPhase = ''
$CC -c -o src.o $src
bash -x ${patchWrapper linker}/bin/ld.lld -o $out @${pkgs.writeText "responsefile" "-shared"} src.o
'';
}
```
Fixes: #178162
Signed-off-by: Peter Waller <p@pwaller.net>
Our gcc_multi and glibc_multi expressions merge together a
32-bit-targeted and 64-bit-targeted gcc. However they do not thread
through the passthru.libgcc from these merged gccs.
This commit corrects that.
It also extends passthru.libgcc to allow a *list* rather than just a
single outpath.
Resolves part of #221891 (at least getting it back to the error
message it gave before).
According to <https://gcc.gnu.org/legacy-ml/gcc-patches/2015-08/msg00836.html>,
all code is position-independent on Windows. Some compilers
apparently warn for -fPIC on Windows, and clang errors:
> clang-15: error: unsupported option '-fPIC' for target 'x86_64-pc-windows-msvc'
I'm guessing the check was hostPlatform instead of targetPlatform by mistake.
In https://github.com/NixOS/nixpkgs/pull/209870 I tried to unify the
treatment of clang and gcc in cc-wrapper as much as possible.
However it appears that I went too far.
Clang requires -isystem flags in order to be able to find gcc's
libstdc++. Gcc does not need these flags. If they are added,
gfortran will get confused:
https://github.com/NixOS/nixpkgs/pull/209870#issuecomment-1500550903
This commit deunifies the chunk of code that adds the -isystem
flags, and explains why this chunk applies only to clang.
When wrapping `clang` and using a `gccForLibs` whose `libgcc` is in
its own output (rather than the `lib` output), this commit will adds
`-L${gccForLibs.libgcc}/lib` to `cc-ldflags`.
If that flag is not added, `firefox` will fail to compile because it
invokes `clang-wrapper` with `-fuse-ld=lld` and passes `-lgcc_s` to
`lld`, but does not tell `lld` where to find `libgcc_s.so`. In that
situation, firefox will fail to link.
Without this change $target-cpp is used unwrapped and is missing
standard header search paths among other things).
Example failure:
$ nix build -f. -L pkgsStatic.netbsd.compat
...
> checking how to run the C preprocessor... x86_64-unknown-linux-musl-cpp
> configure: error: in `/build/cvs-export/tools/compat':
> configure: error: C preprocessor "x86_64-unknown-linux-musl-cpp" fails sanity check
> See `config.log' for more details
cc-wrapper has essentially two separate codepaths: the `gccForLibs`
codepath, used only by non-gcc (i.e. clang) compilers, and the
"other" codepath.
This PR allows non-clang compilers to opt-in to the `gccForLibs`
codepath (off by default). To allow this, a new parameter
`ccForLibs` is exposed, since it would be extremely confusing for
gcc to be able to use `gccForLibs` but not do so by default.
This reverts commit 8c80bd08b7
("build-support/cc-wrapper: pass in non-existent --sysroot= to untangle
from libc").
This change was good in spirit: we caught a few genuine problems with
`scons` based packages (`godot`, `fluxus`) and unexpected `-idirafter`
includes in various boot loadres (`ipxe`, wimboot`):
https://github.com/NixOS/nixpkgs/pull/210004#issuecomment-1407162693
Unfortunately `--sysroot=` also has a negative impact on libary search
order for DT_NEEDED libraries and RUNPATHs of linked libraries. This
unexpectedly broke `dmd`, `d-seams`, `llvmPackages_rocm.compiler-rt`).
An interesting case of unexpected breakage is `usbmuxd2` where the bug
exposed incomplete library move on `libstdc++fs` in `gcc`.
The library breakage is very non-intuitive (on top of already unusual
layout of `cc-wrapper` driver). Let's revert this change for now.
Once it lands we can undo `--sysroot=/` workarounds merged for
`staging-next`.
After https://github.com/NixOS/nixpkgs/pull/210004 `usbmuxd2` started
failing to build as:
usbmuxd2-unstable> .../ld: cannot find -lstdc++fs: No such file or directory
usbmuxd2-unstable> clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
This started happening because #210004 exposed a long-standing bug in
`gcc` derivation: `cc.lib` is missing `libstdc++fs` library:
$ find $(nix-build --no-link -A stdenv.cc.cc.lib) | fgrep libstdc | unnix
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++fs.la
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.la
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.so.6.0.29
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.so
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.so.6
It was not moved from `cc.out` output:
$ find $(nix-build --no-link -A stdenv.cc.cc) | fgrep libstdc | unnix
/<<NIX>>/gcc-11.3.0/lib/libstdc++.a
/<<NIX>>/gcc-11.3.0/lib/libstdc++fs.a
This change adds `cc` library lookup path back to `staging-next` until
`gcc` is fixed.`
FreeBSD doesn't use LLVM's cxxabi implementation, for backwards
compatibility reasons. Software expects the libcxxrt API when
building on FreeBSD. This fixes the build of
pkgsCross.x86_64-freebsd.boost.
Gcc does not allow `-march=` on PowerPC:
https://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/RS_002f6000-and-PowerPC-Options.html#RS_002f6000-and-PowerPC-Options
Instead, `-mcpu=` should be used to set the minimum instruction set
and `-mtune=` is used to optimize instruction scheduling for a
specific processor. Both flags take the same set of valid values,
which includes `native`.
This commit causes `isGccArchSupported` to return `false` for PowerPC
targets so we never pass an `-march=` flag, since that will always be
rejected by gcc.
I would like to add an extra `gcc` build step during linux bootstrap
(https://github.com/NixOS/nixpkgs/issues/208412). This makes it early
bootstrap compiler linked and targeted against `bootstrapTools` `glibc`
including it's headers.
Without this change `gcc`'s spec files always prefer `bootstrapTools` `glibc`
for header search path (passed in as --with-native-system-header-dir=). We'can't
override it with:
- `-I` option as it gets stacked before gcc-specific headers, we need to keep
glibc headers after gcc as gcc cleans namespace up for C standard by using
#include_next and by undefining system macros.
- `-idirafter` option as it gets appended after existing `glibc`-includes
This `--sysroot=/nix/store/does/not/exist` hack allows us to remove existing
`glibc` headers and add new ones with `-idirafter`.
We use `cc-cflags-before` instead of `libc-cflags` to allow user to define
their own `--sysroot=` (like `firefox` does).
To keep it working prerequisite cross-symlink in gcc.libs is required:
https://github.com/NixOS/nixpkgs/pull/209153
Switches that gnatmake needs to pass to gcc must be given as
"-cargs <gcc_switches>" after at least the files to compile (see the
gnatmake docs for all the subtleties). This commit makes that happen,
in a way that triggers rebuilds only for things that actually depend on
GNAT, and not the other compilers contained in GCC.
On darwin clang driver always sets -D_FORTIFY_SOURCE=0 under asan.
This causes -Werror to trip over macro redefinition:
<command line>:1:9: error: '_FORTIFY_SOURCE' macro redefined [-Werror,-Wmacro-redefined]
#define _FORTIFY_SOURCE 2
^
To avoid it let's always explicitly undefine it first before redefining.
Currently, clang++ statically links against libstdc++ instead of
dynamically links, because the -L path included in the cc-wrapper is
incorrect. The gccForLibs.lib output only contains the architecture
subdirectory if the target and host platform are not the same. (See
targetConfig set in gcc/<version>/default.nix and the gcc/builder.nix)
This fixes the incorrect linking by using the correct path for both the
cross and native cases. This also matches the cc_solib set above in
cc-wrapper/default.nix
Tested by compiling a simple cpp binary and noting that it now correctly
dynamically links against libstdc++, natively on x86_64 and arm64, as
well as x86_64 -> arm64 cross compilation.
Co-Authored-By: Sebastian Ullrich <sebasti@nullri.ch>
There is context here that I needed when resolving an issue in which
libc was added to NIX_CFLAGS_COMPILE before the C++ stdlib that took
me awhile to understand.
It was suggested to me that this context be included as a comment,
since it is not obvious and could help others in the future.
POSIX sh (and `bash`) impose a restriction on environment variable name
format and disallow hypheps in the names. Normally it's not a problem
as nothing usually tries to refer nyphenated names.
One exception is `nix develop` (https://github.com/NixOS/nix/issues/6848):
$ nix develop -f. gcc -L
gcc-wrapper> ...-get-env.sh: line 70: expand-response-params: bad substitution
Note that bash usually uses explicitly created `expandResponseParams`
variant of the same variable.
To work the problem around let's avoid environment variable export and move
it to `passthru` for `cc` (used ina few places) and remove it completely for
`binutils` (does not seem to be used at all).
Noticed this bug when was trying to bootstrap m4 on darwin. That fixes
line 163: no such file or directory error
That does not solve all problems staging has on darwin.
This enables users to make use of clang's multi-platform/target support
without having to go through full cross system setup. This is especially useful
for generating bpf object files, I'm not even usre what would a no-userland
cross compile system tuple even look like to even try going that route.
Fixes#176128
Otherwise, these warnings are emitted:
command-line option '-Wformat=1' is valid for C/C++/ObjC/ObjC++ but not for Fortran
command-line option '-Wformat-security' is valid for C/C++/ObjC/ObjC++ but not for Fortran
'-Werror=' argument '-Werror=format-security' is not valid for Fortran
Fixes part of #27218
