Unified processing of command line arguments in ld-wrapper broke support for
`NIX_DONT_SET_RPATH` and revealed that ld-wrapper adds the directory of its
`-plugin` argument to runpath. This pull request fixes that. It treats
`dir/libname.so` as `-L dir -l name`, because this is how `ld.so` interprets
resulting binary: with `dir` in `RUNPATH` and the bare `libname.so` (without
`dir`) in `NEEDED`, it looks for `libname.so` in each `RUNPATH` and chooses the
first, even when the linker was invoked with an absolute path to `.so`.
As described in https://github.com/NixOS/nixpkgs/issues/18461, MacOS no
longer accepts dylibs which only reexport other dylibs, because their
symbol tables are empty. To get around this, we define an object file
with a single "private extern" symbol, which hopefully won't clobber
anything.
The time to expand rpath was proportional to the number of -L flags times the
number of -l flags. Now it is proportional to their sum (assuming constant
number of files in each directory in an -L flag).
Issue reported by @nh2 at https://github.com/NixOS/nixpkgs/issues/27609#issuecomment-317916623
As @oxij points out in [1], this breakage is especially serious because
it changes the contents of built environments without a corresonding
change in their hashes. Also, the revert is easier than I thought.
This reverts commit 3cb745d5a6.
[1]: https://github.com/NixOS/nixpkgs/pull/27427#issuecomment-317293040
Besides deduplicating overlapping logic, clear warning messages were
added for:
- No glob/path for dynamic linker provided (use default glob)
- Glob did not expand to anything (don't append flag)
- glob expanded to multiple things (take first, like before)
This makes those files a bit easier to read. Also, for what it's worth,
it brings us one baby step closer to handling spaces in store paths.
Also, I optimized handling of many transitive deps with read. Probably,
not very beneficial, but nice to enforce the pkg-per-line structure.
Doing so let me find much dubious code and fix it.
Two misc notes:
- `propagated-user-env-packages` also needed to be adjusted as
sometimes it is copied to/from the propagated input files.
- `local fd` should ensure that file descriptors aren't clobbered
during recursion.
I think it's ok to export things which aren't wrapped. The cc-wrapper
can be thought of as responsible for all of binutils and the c
compiler, only wrapping those binaries which are necessary to
interposition---as opposed to all binaries it thinks are relevaant.
Conversely, adding the setup hook to the unwrapped compilers would be
unforunate as hooks are ugly hacks and the compilers themselves take
a long time to rebuild. Better to wholely separate "pure packages" from
hacks.
Eventually we should avoid this "pre-wrapping" and just update those
files in nixpkgs. This mass-rebuild change is best done along with
those needed to reduce the disparity between native and cross (i.e.
making native the "identity cross").
We now (on cross) require per-target flag interposition by putting the
triple in the names of the relevant environment variables, e.g:
export NIX_arm_unknown_linux_gnu_CFLAGS_COMPILE=...
The wrapper also has a `infixSalt` attribute (and "_" prefixed and
suffixed variants) to assist downstream packages.
Note how that the dashes are replaced to keep the identifier valid.
Using names like this allows us to keep the settings for different
compilers seperate.
I think it might be even better to use names like `NIX_{BUILD,HOST}...`
using the platform's role rather than the platform itself, but this
would be more work as the previous stages' tools would have to be re-
wrapped to take on their new role. I therefore didn't do this for now,
but that route should be thoroughly explored in the future.
This fixes the Stack Clash issue rediscovered by Qualys. See
https://www.qualys.com/2017/06/19/stack-clash/stack-clash.txt
for more information on the topic, specifically section III.
We don't have the kernel mitigation available because it is a Grsecurity
feature which we don't support anymore. Other distributions like Gentoo
Hardened and Arch already have `-fstack-check` enabled by default.
See the Gentoo page on Stack Clash for more information on this solution:
https://wiki.gentoo.org/wiki/Hardened/Gentoo_Hardened_and_Stack_Clash
This unfortunately doesn't apply to clang because `-fstack-check` is a
noop there. Note that the GCC implementation also has problems that could
be exploited to circumvent these checks but it is still better than
keeping it disabled.
This value is require to get c++ std include path for libclang based tools (vim plugins in my case).
I currently extract it this with this rather command:
```
eval echo $(nix-instantiate --eval --expr 'with (import <nixpkgs>) {}; clang.default_cxx_stdlib_compile')
```
it did not trigger any recompilation on my system.
This makes the response file handling more consistent with GCC.
For example, a reponse file may contain:
"-Wl,$ORIGIN"
GCC will treat this as a double quoted string and not expand the
variable reference. Previously, cc-wrapper would expand the variable
in the same was as if the string was provided on the command line.
The following parameters are now available:
* hardeningDisable
To disable specific hardening flags
* hardeningEnable
To enable specific hardening flags
Only the cc-wrapper supports this right now, but these may be reused by
other wrappers, builders or setup hooks.
cc-wrapper supports the following flags:
* fortify
* stackprotector
* pie (disabled by default)
* pic
* strictoverflow
* format
* relro
* bindnow
The importance of glibc makes it worthwhile to provide debug
symbols. However, this revealed an issue with separateDebugInfo: it
was indiscriminately adding --build-id to all ld invocations, while in
fact it should only do that for final links. Glibc also uses non-final
("relocatable") links, leading to subsequent failure to apply a build
ID ("Cannot create .note.gnu.build-id section, --build-id
ignored"). So now ld-wrapper.sh only passes --build-id for final
links.
Otherwise, when building glibc and other packages, the "strip" from
bootstrapTools is used, which doesn't recognise some tags produced by
the newer "ld" from binutils.
The ld-wrapper.sh script calls `readlink` in some circumstances. We need
to ensure that this is the `readlink` from the `coreutils` package so
that flag support is as expected.
This is accomplished by explicitly setting PATH at the top of each shell
script.
Without doing this, the following happens with a trivial `main.c`:
```
nix-env -f "<nixpkgs>" -iA pkgs.clang
$ clang main.c -L /nix/../nix/store/2ankvagznq062x1gifpxwkk7fp3xwy63-xnu-2422.115.4/Library -o a.out
readlink: illegal option -- f
usage: readlink [-n] [file ...]
```
The key element is the `..` in the path supplied to the linker via a
`-L` flag. With this patch, the above invocation works correctly on
darwin, whose native `/usr/bin/readlink` does not support the `-f` flag.
The explicit path also ensures that the `grep` called by `cc-wrapper.sh`
is the one from Nix.
Fixes#6447
... because cc-wrapper is meant to propagate man pages into user envs,
and info pages are rather large.
Also replace the duplicate g++ and gcc man1 pages by a symlink.
Note: -B argument seems more like for gcc's main output,
though it's used in a bit strange way here.
(Upstream default is /usr/lib/gcc/ which we don't move.)
This reverts d927da8dae. Having a copy
of gcc-wrapper/setup-hook.sh is bad for maintainability - it had
already started to diverge. Also, gccStdInc gave a nix-env conflict
with the standard gcc. And it wasn't actually used in Nixpkgs.
Instead, if you really need to change "-isystem" to "-I", you can now
set ccIncludeFlag to "-I".
Now development stuff is propagated from the first output,
and userEnvPkgs from the one with binaries.
Also don't move *.la files (yet). It causes problems, and they're small.
- there were many easy merge conflicts
- cc-wrapper needed nontrivial changes
Many other problems might've been created by interaction of the branches,
but stdenv and a few other packages build fine now.
- Add a conditional flag for the c++ std lib
- Build binaries that get linked by our own dyld (someday)
- Automatically add framework directories in the setup hook
