Add this hook to checkPhase to allow for running MPI application in
the sandbox. It detects the MPI implementations and sets the respective
environment variables.
Before the change the following command did not work:
$ nix develop -i --impure --expr 'with import <nixpkgs> { system = "i686-linux"; }; (buildFHSUserEnv { name = "t"; targetPkgs = ps: with ps; [ libmpc stdenv.cc ]; }).env'
$ ld -lmpc -o a
ld: cannot find -lmpc: No such file or directory
It is expected to work as `NIX_LDFLAGS` does contain valid values:
$ echo $NIX_LDFLAGS
-L/usr/lib -L/usr/lib32
Note that for `gcc` it does work:
$ printf "int main(){}" | gcc -x c - -lmpc -o a
It happens because `HOST` role is enabled for `cc`:
$ echo $NIX_CC_WRAPPER_TARGET_HOST_i686_unknown_linux_gnu
1
But not for `BINTOOLS`:
$ echo $NIX_BINTOOLS_WRAPPER_TARGET_HOST_i686_unknown_linux_gnu
<empty>
The change adds BINTOOLS role and fixes linking:
$ nix develop -i --impure --expr 'with import ~/nm { system = "i686-linux"; }; (buildFHSUserEnv { name = "t"; targetPkgs = ps: with ps; [ libmpc stdenv.cc ]; }).env'
$ printf "int main(){}" | gcc -x c - -lmpc -o a
$ ld -lmpc -o a
ld: warning: cannot find entry symbol _start; not setting start address
Previously `writeDarwinBundle` used a handcrafted shell wrapper, however
this causes issues on Apple Silicon Macs as script-only application
bundles are always run under Rosetta[0][1].
Replacing the handcrafted shell wrapper with a binary wrapper allows
apps to run natively instead of requiring Rosetta. However, this means
we can no longer use `$1` and `$@`.
After checking nearly every current usage of `desktopToDarwinBundle`,
there were no apps that used `%[fFuU]` before the last argument, meaning
removing them naively is good enough for the current apps.
[0]: https://developer.apple.com/documentation/apple-silicon/building-a-universal-macos-binary
[1]: https://stackoverflow.com/a/68208374
The `name` & `version` attributes only existed in a previous form of
the API before it was switched over to using `fetchzip` &
`applyPatches`[1]. The attributes existed to be able to throw an
evaluation error with upgrade instructions when this was used. However,
this was before 22.11, so this shouldn't be in use anymore (and if
somebody is doing a migration from a very old NixOS, this is still
documented in the 22.11 release-notes[2]).
Anyways, this simplifies the code a little bit and also having both
`appName`/`appVersion` and `name`/`version` in there is quite confusing. But
still, I figured it's less confusing to not re-use attributes that were
deprecated in the past, hence the alternative naming.
[1] 3ca9b9a8ad
[2] d41b381310
This change makes sure that each Nextcloud plugin installed provides a
`meta` section with proper license information.
Unfortunately, the metadata from the appstore is useless for this
purpose since it doesn't differentiate between e.g. AGPL 3.x and AGPL
3.x or any later version. In fact, this isn't consistent between their
software, e.g. `bookmarks` has `agpl3Plus` according to the files'
headers[1] whereas `twofactor_nextcloud_notification` is AGPL 3 only[2].
I don't think there's any trivial and reasonable way to retrieve this
information programatically, so I decided to change the format of
`nextcloud-apps.json`[3] to also contain the license in the form of the
license attribute we have in `lib/licenses.nix`, i.e. GNU AGPL 3 or
later is `agpl3Plus`.
I retrieved the information using the following approach:
* go to the source code of $app at the revision we currently have
packaged
* check for a license identifier (does it specify the license only or
the license "or any later version")?
* first in `src/main.js` because most apps from Nextcloud itself used
actual spdx identifiers in the frontend's source-code.
* then in `lib/AppInfo/Application.php` which each Nextcloud app has.
License changes should be updated accordingly when updating the apps. As
with any other package in nixpkgs as well, this currently needs to be
done manually (or as part of the review process)[4].
Also, I decided to change the `name` of the `applyPatches` derivation
from `source-patched` to `nextcloud-app-${appName}-${appVersion}`. When
deploying a lot of apps (and probably displaying the diff using
`nix store diff-closures` on deploy), the current output isn't very
helpful. This is purely optional because I didn't want to break the
interface of `fetchNextcloudApp` again.
[1] https://github.com/nextcloud/bookmarks/blob/v13.1.0/lib/AppInfo/Application.php#L6
[2] https://github.com/nextcloud/twofactor_nextcloud_notification/blob/v3.7.0/lib/AppInfo/Application.php
[3] This isn't really well-defined since it's preprocessed with `jq(1)`
before passing the apps to nc4nix.
[4] Though notable license changes (e.g. agpl -> gpl) would also pop up
in the diff of <nextcloudversion>.json, so this is pretty easy to
catch.
The command `fakechroot` errored with buffer overflows. The `proot`
command doesn't seem to suffer from the same problem. The tar command
creating the layer errors with "permission denied" on a bunch of paths
in /proc but the layer seems to get built anyway.
this splits hardeningCFlags into hardeningCFlagsAfter and
hardeningCFlagsBefore (where most flags still remain) to allow
us to *append* `-D_FORTIFY_SOURCE=` values to the command-line,
forcing our choice of fortify level and avoiding potential
redefinition warnings/errors through use of `-U_FORTIFY_SOURCE`
While using libredirect in conjunction with geckodriver, I stumbled on
odd segfaults that happened when running the wrapped statx() call from
libredirect:
0x00007ffff7ddd541 in __strncmp_avx2 () from .../lib/libc.so.6
0x00007ffff7f6fe57 in statx () from .../lib/libredirect.so
0x00005555558d35bd in std::sys::unix::fs::try_statx::h2045d39b0c66d4e8 ()
0x00005555558d2230 in std::sys::unix::fs::stat::ha063998dfb361520 ()
0x0000555555714019 in mozversion::firefox_version::hdc3b57eb04947426 ()
0x00005555556a603c in geckodriver::capabilities::FirefoxCapabilities::version::h58e289917bd3c721 ()
0x00005555556a77f5 in <geckodriver::capabilities::FirefoxCapabilities as webdriver::capabilities::BrowserCapabilities>::validate_custom::h62d23cf9fd63b719 ()
0x000055555562a7c8 in webdriver::capabilities::SpecNewSessionParameters::validate::h60da250d33f0989f ()
0x00005555556d7a13 in <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold::h9427a360a3d0bf8f ()
0x0000555555669d85 in <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter::hd274d536ea29bb33 ()
0x00005555555c05ef in core::iter::adapters::try_process::hdf96a01ec1f9b8bd ()
0x000055555561768d in <webdriver::capabilities::SpecNewSessionParameters as webdriver::capabilities::CapabilitiesMatching>::match_browser::hfbd8c38f6db17e9f ()
0x00005555555ca6ef in <geckodriver::marionette::MarionetteHandler as webdriver::server::WebDriverHandler<geckodriver::command::GeckoExtensionRoute>>::handle_command::h13b98b9cb87a69d6 ()
0x00005555555e859e in webdriver::server::Dispatcher<T,U>::run::h746a8bf2f0bc24fd ()
0x000055555569ff0f in std::sys_common::backtrace::__rust_begin_short_backtrace::h3b920773bd467d2a ()
0x00005555555dbc99 in core::ops::function::FnOnce::call_once{{vtable.shim}}::h81ba7228877515f7 ()
0x00005555558d31a3 in std::sys::unix:🧵:Thread:🆕:thread_start::h4514580219a899c5 ()
0x00007ffff7d0ce24 in start_thread () from .../lib/libc.so.6
0x00007ffff7d8e9b0 in clone3 () from .../lib/libc.so.6
The reason why I found this odd was because it happens in the following
piece of code (shortened a bit):
1 static const char * rewrite(const char * path, char * buf)
2 {
3 if (path == NULL) return path;
4 for (int n = 0; n < nrRedirects; ++n) {
5 int len = strlen(from[n]);
6 if (strncmp(path, from[n], len) != 0) continue;
7 if (snprintf(buf, PATH_MAX, "%s%s", to[n], path + len) >= PATH_MAX)
8 abort();
9 return buf;
10 }
11 return path;
12 }
When inspecting the assembly, I found that the check for the null
pointer in line 3 was completely missing and the code was directly
entering the loop and then eventually segfault when running strncmp()
with a null pointer as its first argument.
I confirmed that indeed that check was missing by compiling libredirect
with "-O0" and comparing the generated assembly with the optimized one.
The one compiled with "-O0" had that check while the optimized one did
not and indeed when running geckodriver with the unoptimized version it
worked fine.
Digging in the Git history, I found 5677ce2008,
which actually introduced the null pointer check. Going back to that
commit however, the check actually was still in the generated assembly.
So I bisected between that commit and the most recent one and ended up
with commit ca8aa5dc87, which moved
everything to use GCC 7.
I haven't found out why *exactly* GCC was optimizing the check away, but
playing around on Godbolt with various other compilers seems that other
compilers such as Clang are doing it as well. Additionally, given that
passing NULL to stat() is UB, my guess is that compilers tend to assume
that such an argument can't be NULL. My assumption is based on the fact
that GCC warns with "argument 1 null where non-null expected" when
passing NULL to eg. stat().
To address this for now, I marked the path argument of the rewrite()
volatile and also added a test that should cause a segfault in case this
would regress again as it already did.
Signed-off-by: aszlig <aszlig@nix.build>
Since coreutils v9.2 the `--no-clobber` flag results in a non-zero exit
code when the destination files exist. Using `--update=none` will now
reproduce the old behavior of `--no-clobber`.
However, the `--update=none` flag was introduced in coreutils v9.3 and
thus `mergeImages` will fail if you have an older version than v9.3 in
stdenv after applying this commit.
[coreutils v9.3 changelog](f386722dc0/NEWS (L48))
Derivations built with `writeShellScriptBin`
should always be runnable with `nix run`. At present,
the derivation is missing both `meta.mainProgram`
and `pname`– this means that `nix run` falls back
to inferring the bin path from `name`. This is
unreliable and depends on faulty heuristics.
For context, reference the following snippet from
`nix run --help`:
If installable evaluates to a derivation, it will try to execute the
program <out>/bin/<name>, where out is the primary output store path
of the derivation, and name is the first of the following that exists:
· The meta.mainProgram attribute of the derivation.
· The pname attribute of the derivation.
· The name part of the value of the name attribute of the derivation.
This is very useful in conjunction with meta.pkgConfigModules, as the
new tester can use the list provided by this meta attribute as a default
value for moduleNames, making its usage in passthru.tests very
convenient.
For backwards compatibility, a shim under the old name is maintained
with a warning.
vcunat said
> This invocation of mktemp creates the file in the current directory, which is bad practice. We should add "--tmpdir=$TMPDIR" or make the template absolute.
> I noticed because one package did cd $src during installing, which is a read-only path...
The Darwin stdenv rework conditionally sets `NIX_CC_USE_RESPONSE_FILE`
depending on the `ARG_MAX` of the build system. If it is at least 1 MiB,
the stdenv passes the arguments on the command-line (like Linux).
Otherwise, it falls back to the response file. This was done to prevent
intermitent failures with clang 16 being unable to read the response
file. Unfortunately, this breaks `gccStdenv` on older Darwin platforms.
Note: While the stdenv logic will also be reverted, this change is
needed for compatibility with clang 16.
GCC is capable of using a response file, but it does not work correctly
when the response file is a file descriptor. This can be reproduced
using the following sequence of commands:
$ nix shell nixpkgs#gcc; NIX_CC_USE_RESPONSE_FILE=1 gcc
# Linux
/nix/store/9n9gjvzci75gp2sh1c4rh626dhizqynl-binutils-2.39/bin/ld: unrecognized option '-B/nix/store/vnwdak3n1w2jjil119j65k8mw1z23p84-glibc-2.35-224/lib/'
/nix/store/9n9gjvzci75gp2sh1c4rh626dhizqynl-binutils-2.39/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
# Darwin
ld: unknown option: -mmacosx-version-min=11.0
collect2: error: ld returned 1 exit status
Instead of using process substitution, create a temporary file and
remove it in a trap. This should also prevent the intermitent build
failures with clang 16 on older Darwin systems.
Fixes#245167
Before the change the hook had a chance to run `strip` against the same
file using multiple link paths. In case of `gcc` `libgcc.a` was stripped
multiple times in parallel and produces corrupted archive.
The change runs inputs via `realpath | uniq` to make sure we don't
attempt to strip the same files multiple times.
* Preferentially use the stdenv clang if it is new enough to produce
arm64e binaries; and
* Fix incompatible function pointer conversions (results in an error
with clang 16).
In 787af0f79f
I had to change ${go-modules} to $goModules to allow overrideAttrs to work;
However, env vars cannot contain -, so i had to change go-modules too.
This in turn broke nix-update because it uses the go-modules attr.
Instead of making nix-update more complicated, make go-modules naming match cargoDeps.
`fd --type f | xargs sd '\bgo-modules\b' 'goModules'`
and revert change to pkgs/applications/misc/dstask/default.nix
and pkgs/servers/http/dave/default.nix
and pkgs/os-specific/darwin/plistwatch/default.nix
release note added
While trying to exclude a generated `/share/info/dir` in a package
from symlinking into a profile, we also ended up excluding the Emacs
info manual for `dired-x`. This change excludes only the dir file.
Most FHSEnv-wrapped packages in Nixpkgs wrap a x86_64-linux binary; making
multiArch unnecessary bloat closure size. Saves about 1G in anki-bin.
This makes multiArch FHSEnvs the exception rather than the rule.
; Conflicts:
; pkgs/development/libraries/nss/generic.nix
- bb53634671 removed the conditionals since firefox-esr-91 and nss <3.69 has been dropped a while ago.
- cb3762857d updated the conditionals to always partition the tests based on the nss_latest version. Chosen that since it will remain future proof.
When lib overrides were used, before this commit, they would not be made
available in the configuration evaluation of nixosTest's nodes.
Sample code:
``` nix
let
pkgs = import ./. {
overlays = [
(new: old: {
lib = old.lib.extend (self: super: {
sorry_dave = builtins.trace "There are no pod bay doors" "sorry dave";
});
})
];
};
in
pkgs.testers.runNixOSTest {
name = "demo lib overlay";
nodes = {
machine = { lib, ... }: {
environment.etc."got-lib-overlay".text = lib.sorry_dave;
};
};
testScript = { nodes }:
''
start_all()
machine.succeed('grep dave /etc/got-lib-overlay')
'';
}
```
The flag iterates through the lockfile entries, rewrites `resolved` URLs
to those that will be in the cache (like `fixup_yarn_lock` from
yarn2nix), removes `integrity` for git deps whose hash won't match the
reproducible repacking that the fetcher does, writes the amended
lockfile, and exits.
When a response file is in use, "$*" contains the response file and not
the parameters; both the linker and compiler wrappers are updated to use
the response-expanded params.
The compiler driver likes to pass parameters to the linker via a
response file, including -shared.
LLD rejects the combination of (-shared -pie), whereas other linkers
silently ignore the contradiction:
```
ld.lld: error: -shared and -pie may not be used together
```
This breaks certain configurations using LLD as a linker.
Changing `add-hardening.sh` results in a full rebuild. To avoid the
rebuild, here is a quick test case which shows the new hardening script
allows the link to succeed:
```
{ pkgs ? import <nixpkgs> {} }:
let
# gcc silently accepts -shared -pie together, lld does not.
linker = pkgs.wrapBintoolsWith { bintools = pkgs.llvmPackages.lld; };
patchWrapper = prev: prev.overrideAttrs (final: prev: let
prevScript = builtins.match (".*(/nix/store/[a-z0-9]+-add-hardening.sh).*") prev.postFixup;
in {
postFixup = (builtins.replaceStrings prevScript ["${./new-add-hardening.sh}"] prev.postFixup);
});
in
pkgs.stdenv.mkDerivation {
name = "nixpkgs-hardening-bug";
src = pkgs.writeText "src.c" "int main(int argc, char* argv[]) { return 0; }";
NIX_HARDENING_ENABLE = "pie";
unpackPhase = ":";
buildPhase = ''
$CC -c -o src.o $src
bash -x ${patchWrapper linker}/bin/ld.lld -o $out @${pkgs.writeText "responsefile" "-shared"} src.o
'';
}
```
Fixes: #178162
Signed-off-by: Peter Waller <p@pwaller.net>
This change adds a flag to update shebang paths that point to the Nix
store. This is particularly useful when a cross-compiled package uses
same script at compile-time and run-time, but the interpreter must be
changed since hostPlatform != buildPlatform.
