PR #58431 added /nix/store to each layer.tar. However, the timestamp was
not explicitly set while adding /nix and /nix/store to the archive. This
resulted in different SHA256 hashes of layer.tar between image builds.
This change sets time and owner when tar'ing /nix/store.
To avoid symlink loops to /host in nested chrootenvs we need to remove
one level of indirection. This is also what's generally expected of
/host contents.
* Remove unused argument from pivot_root;
* Factor out tmpdir creation into a separate function;
* Remove unused fstype from bind mount;
* Use unlink instead of a treewalk to remove empty temporary directory.
The problem with stacking chrootenv before was that CLONE_NEWUSER cannot
be used when a child uses chroot. So instead of that we use pivot_root
which replaces root in the whole namespace. This requires our new root
to be an actual fs so we mount tmpfs.
not all valid file names are valid derivation names. This can cause
troubles when, for example, trying to place systemd template unit
files, which contain an '@' in their name, in an initrd.
Fixes#53987
glibc 2.27 (and possibly other versions) can't handle an `nopenfd` value larger than 2^19 in `ntfw`, which is problematic if you've set the maximum number of fds per process to a value higher than that.
Commit "patchShebangs: Allow for multiple arguments" 4a1e51f957
removed the check. We don't want to break existing usages so this
introduces it again with a successful exit code.
It's tempting to think patchShebangs supports multiple arguments.
Without this patch it just silently ignores all but the first. Now it
patches the shebangs in all of its arguments.
Fixes: #57695
This hook allows to add NixOS driver libraries path to given ELF
objects' RUNPATH. We use it instead of settings RUNPATH manually
everywhere. It must be invoked in postFixup so that RUNPATH stripping
does not remove the path.
It puts the path first instead of last so that system-wide drivers
are always preferred.
The layer order was not correct when a parent image was used: parent
image layers were above the new created layer.
This commits simplifies the code related to layer ordering. In
particular, layers in `layer-list` are ordered from bottom-most to
top-most. This is also the order of layers in the `rootfs.diff_ids`
attribute of the image configuration.
While it might be useful in some cases, there are too many caveats to be worth it.
When libredirect intercepts dlopen call and calls the original function, the dynamic
loader will use libredirect.so's DT_RUNPATH entry instead of the one from the ELF file
the dlopen call originated from. That means that when program tries to dlopen a library
that it expects to find on its RPATH, the call will fail.
This broke Sublime Text for just that reason.
Especially as a new user it is a much better experience to receive a
proper help response to `-h`. Currently passing `-h` will cause some
runtime error with the `git remote` error help being shown. Not very
helpful.
It doesn't hurt to be a bit more user friendly in this case.
This introduces extra escaping for $NIX_STORE that gets interpolated
into a PCRE. The escaping is performed using a standard Perl function
"quotemeta" (see "perldoc -f quotemeta" for reference). The same value
is also used in sed regex which uses POSIX basic regular expressions
instead of PCRE, so it needs fewer characters to be escaped. It should
not cause much problem to not change sed invocation, but I replace it
with equivalent Perl expression (actually the behavior is changed to not
output a newline character after the matched output).
We want to make sure this value is explicitly set. Infering it for
every arch leads to annoying failures like:
https://hydra.nixos.org/build/92583832/
Perhaps we can enable it in the future with some smarter handling of
cc-wrapper.sh.
Creating the timestamp in the patched script's directory has a few
drawbacks:
* if "foo.timestamp" already exists, it will be overwritten
* it requires the directory to be writable
Adds pkgsCross.wasm32 and pkgsCross.wasm64. Use it to build Nixpkgs
with a WebAssembly toolchain.
stdenv/cross: use static overlay on isWasm
isWasm doesn’t make sense dynamically linked.
It is useful to make these dynamic and not bake them into gcc. This
means we don’t have to rebuild gcc to change these values. Instead, we
will pass cflags to gcc based on platform values. This was already
done hackily for android gcc (which is multi-target), but not for our
own gccs which are single target.
To accomplish this, we need to add a few things:
- add ‘arch’ to cpu
- add NIX_CFLAGS_COMPILE_BEFORE flag (goes before args)
- set -march everywhere
- set mcpu, mfpu, mmode, and mtune based on targetPlatform.gcc flags
cc-wrapper: only set -march when it is in the cpu type
Some architectures don’t have a good mapping of -march. For instance
POWER architecture doesn’t support the -march flag at all!
https://gcc.gnu.org/onlinedocs/gcc/RS_002f6000-and-PowerPC-Options.html#RS_002f6000-and-PowerPC-Options
gobject-introspection uses glib’s g_module_open function, which in turn relies
on dlopen. I also implemented openat, since I initially thought this function
was used but turns out dlopen uses the openat signal directly. We might as
well keep it, even thought I do not need it at the moment.
this adds libc++ to the LLVM cross, giving us access to the full
Nixpkgs set. This requires 4 stages of wrapped compilers:
- Clang with no libraries
- Clang with just compiler-rt
- Clang with Libc, and compiler-rt
- Clang with Libc++, Libc, and compiler-rt
This iteration was long, about five weeks (2fcb11a2), I think.
Darwin: it's missing a few thousand binaries and there's a make-netbsd
regression, but I suppose these aren't merge blockers.
buildRustCrate has a handy `include` helper, that only imports those whitelisted
files and folders to the store.
However, the function's matching logic is broken and includes all files,
regardless of whether or not they're whitelisted, as long as the whitelist
contains at least one name (regardless of whether that name exists). This is
because it doesn't take into account that
`lib.strings.removePrefix "foo" "bar" == "bar"` (that is, paths that don't match
the prefix are passed straight through).
This rare sitation was caught when building zoom-us package:
```
automatically fixing dependencies for ELF files
/nix/store/71d65fplq44y9yn2fvkpn2d3hrszracd-auto-patchelf-hook/nix-support/setup-hook: line 213: echo: write error: Broken pipe
/nix/store/71d65fplq44y9yn2fvkpn2d3hrszracd-auto-patchelf-hook/nix-support/setup-hook: line 210: echo: write error: Broken pipe
```
The worst is that derivation continued and resulted into broken package:
https://github.com/NixOS/nixpkgs/pull/55566#issuecomment-470065690
I hope, replacing `grep -q` with `grep` will remove this race condition.
When a package provides both executables and gio modules, it is quite
probable the executables will need those modules. wrapGAppsHook wraps
executables with GIO_EXTRA_MODULES picked up from dependencies
but forgets about the package being built. Let’s add to consideration.
Closes: https://github.com/NixOS/nixpkgs/issues/50254
On very large graphs (14k+ paths), we'd end up with a massive in
memory tree of mostly duplication.
We can safely cache trees and point back to them later, saving
memory.
While it is not obvious from the source, cargo sets CARGO_MANIFEST_DIR to an absolute directory. This let to a build problem with the popular "tera" crate using the "pest" crate.
## Cargo details
The variable is set here:
f7c91ba622/src/cargo/core/compiler/compilation.rs (L229)
and computed from the `manifest_path`:
f7c91ba622/src/cargo/core/package.rs (L163)
The manifest path is also exported via `cargo metadata` where you can see that it is absolute.
Whenever we create scripts that are installed to $out, we must use runtimeShell
in order to get the shell that can be executed on the machine we create the
package for. This is relevant for cross-compiling. The only use case for
stdenv.shell are scripts that are executed as part of the build system.
Usages in checkPhase are borderline however to decrease the likelyhood
of people copying the wrong examples, I decided to use runtimeShell as well.
The execlineb program is the launcher (and lexer) of execline scripts.
So it makes a lot of sense to have all the small tools in scope by
default.
We append to the end of PATH so that they can be easily overwritten by
the user.
Co-authored-by: Alyssa Ross <hi@alyssa.is>
The appimageTools attrset contains utilities to prevent
the usage of appimage-run to package AppImages, like done/attempted
in #49370 and #53156.
This has the advantage of allowing for per-package environment changes,
and extracts into the store instead of the users home directory.
The package list was extracted into appimageTools to prevent
duplication.
bcf54ce5bb introduced a treewide change to
use ${stdenv.shell} where-ever possible. However, this broke a script
used by dockerTools, store-path-to-layer.sh, as it did not preserve the
+x mode bit. This meant the file got put into the store as mode 0444,
resulting in a build-time error later on that looked like:
xargs: /nix/store/jixivxhh3c8sncp9xlkc4ls3y5f2mmxh-store-path-to-layer.sh: Permission denied
However, in a twist of fate, bcf54ce5bb
not only introduced this regression but, in this particular instance,
didn't even fix the original bug: the store-path-to-layer.sh script
*still* uses /bin/sh as its shebang line, rather than an absolute path
to stdenv. (Fixing this can be done in a separate commit.)
Signed-off-by: Austin Seipp <aseipp@pobox.com>
The original change in #55372 was supposed to fix the case where a store
path which is a file should be placed into `buildEnv` which broke with a
fairly misleading Perl error.
Unfortunately this introduced a regression, `findFiles` can have targets
that are files if the file isn't a store path. Rather than adding more
obscure checks with probably further regressions, I figured that it's
better to replicate the behavior of `lib.isStorePath` and explicitly
check if the store path is a file and break in this case only.
This should also fix recent staging issues.
