The docker loading (docker 1.12.6) of an image with uppercase in the
name fails with the following message:
invalid reference format: repository name must be lowercase
As @oxij points out in [1], this breakage is especially serious because
it changes the contents of built environments without a corresonding
change in their hashes. Also, the revert is easier than I thought.
This reverts commit 3cb745d5a6.
[1]: https://github.com/NixOS/nixpkgs/pull/27427#issuecomment-317293040
Fixes#27406.
Commit 5d4efb2c81 added an assertion to `stopNest'
which requires it be correctly paired with `startNest'. `fetchurl' calls
`stopNest', but never calls `startNest'; the former calls are removed.
The `DISPLAY` environment variable is propagated into chroots built with
`buildFHSUserEnv`, but currently the `XAUTHORITY` variable is not. When
the latter is set, its value is usually necessary in order to connect to
the X server identified by the former.
This matters for users running gdm3, for example, who have `XAUTHORITY`
set to something like `/run/user/1000/gdm/Xauthority` instead of the X
default of `~/.Xauthority`, which doesn't exist in that setup.
Fixes#21532.
Since 3cb745d5a6, the format of
propagated-user-env-packages has changed and propagated packages have not been
included by buildenv, including in the system environment.
The buildenv builder is modified to read propagated-user-env-packages
line-by-line, instead of expecting all packages on one line.
Besides deduplicating overlapping logic, clear warning messages were
added for:
- No glob/path for dynamic linker provided (use default glob)
- Glob did not expand to anything (don't append flag)
- glob expanded to multiple things (take first, like before)
This makes those files a bit easier to read. Also, for what it's worth,
it brings us one baby step closer to handling spaces in store paths.
Also, I optimized handling of many transitive deps with read. Probably,
not very beneficial, but nice to enforce the pkg-per-line structure.
Doing so let me find much dubious code and fix it.
Two misc notes:
- `propagated-user-env-packages` also needed to be adjusted as
sometimes it is copied to/from the propagated input files.
- `local fd` should ensure that file descriptors aren't clobbered
during recursion.
They're additional commits from #26877.
Changing names of the fetched stuff was changing very many hashes,
and I think it's better to avoid that for the moment to reduce work
needed by nixpkgs users. The fetchers are expected to be commonly
used even outside nixpkgs, and the current naming wasn't that bad
usually.
(commit analogical to d10c3cc5eedf58e80e2; I haven't noticed the part of
the PR has already got to master)
The freedesktop specification says that the autostart folder is located at $XDG_CONFIG_DIRS/autostart.
Nixos defines XDG_CONFIG_DIRS as /etc/xdg. So we need to copy the desktop files into the right folder
with make-startupitem.
This will override the existing winsymlinks setting. nativestrict
will cause ln to fail if it's unable to create a native symlink.
Native symlinks are required for the windows dll loader to find the
libraries.
This script is also used for cross-mingw, but setting CYGWIN
shouldn't cause a problem.
This allows users to specify a custom registry src,
because currently every packager would need to create
an outdated Cargo.lock just to be compatible with the
probably outdated rustRegistry in nixpkgs.
Currently there is no easy way to convince cargo to
do that, so this makes that workaround unnecessary.
I think it's ok to export things which aren't wrapped. The cc-wrapper
can be thought of as responsible for all of binutils and the c
compiler, only wrapping those binaries which are necessary to
interposition---as opposed to all binaries it thinks are relevaant.
Conversely, adding the setup hook to the unwrapped compilers would be
unforunate as hooks are ugly hacks and the compilers themselves take
a long time to rebuild. Better to wholely separate "pure packages" from
hacks.
Eventually we should avoid this "pre-wrapping" and just update those
files in nixpkgs. This mass-rebuild change is best done along with
those needed to reduce the disparity between native and cross (i.e.
making native the "identity cross").
We now (on cross) require per-target flag interposition by putting the
triple in the names of the relevant environment variables, e.g:
export NIX_arm_unknown_linux_gnu_CFLAGS_COMPILE=...
The wrapper also has a `infixSalt` attribute (and "_" prefixed and
suffixed variants) to assist downstream packages.
Note how that the dashes are replaced to keep the identifier valid.
Using names like this allows us to keep the settings for different
compilers seperate.
I think it might be even better to use names like `NIX_{BUILD,HOST}...`
using the platform's role rather than the platform itself, but this
would be more work as the previous stages' tools would have to be re-
wrapped to take on their new role. I therefore didn't do this for now,
but that route should be thoroughly explored in the future.
This fixes the Stack Clash issue rediscovered by Qualys. See
https://www.qualys.com/2017/06/19/stack-clash/stack-clash.txt
for more information on the topic, specifically section III.
We don't have the kernel mitigation available because it is a Grsecurity
feature which we don't support anymore. Other distributions like Gentoo
Hardened and Arch already have `-fstack-check` enabled by default.
See the Gentoo page on Stack Clash for more information on this solution:
https://wiki.gentoo.org/wiki/Hardened/Gentoo_Hardened_and_Stack_Clash
This unfortunately doesn't apply to clang because `-fstack-check` is a
noop there. Note that the GCC implementation also has problems that could
be exploited to circumvent these checks but it is still better than
keeping it disabled.
* ultrastardx-beta: init at 1.3.5
* libbass, libbass_fx: init at 24
* ultrastar-creator: init at 2017-04-12
* buildSupport/plugins.nix: add diffPlugins
Helper function to compare expected plugin lists to the found plugins.
* ultrastar-manager: init at 2017-05-24
The plugins are built in their own derivations, speeding up (re-)compilation.
The `diffPlugins` function from `beets` is reused to test for changes in the
plugin list on updates.
* beets: switch to diffPlugins
The function is basically just extracted for better reusability.
This value is require to get c++ std include path for libclang based tools (vim plugins in my case).
I currently extract it this with this rather command:
```
eval echo $(nix-instantiate --eval --expr 'with (import <nixpkgs>) {}; clang.default_cxx_stdlib_compile')
```
it did not trigger any recompilation on my system.
`SSL_CERT_FILE` has been replaced with `NIX_SSL_CERT_FILE`.
Before this change using `fetchdarcs` resulted in an error message like:
```
Identifying repository http://hub.darcs.net/scravy/easyplot inventory
darcs failed: Not a repository: http://hub.darcs.net/scravy/easyplot (Peer certificate cannot be authenticated with given CA certificates)
HINT: Do you have the right URI for the repository?
builder for ‘/nix/store/imyvcs6lvb5yva66krc5wk39931sam8v-fetchdarcs.drv’ failed with exit code 2
```
I was getting the following error building tide from Melpa:
nix-build -E '(import <nixpkgs> {}).emacs25WithPackages (p: [p.melpaPackages.tide])'
File tide-20170509.1134.tar is large (10.2M), really open? (y or n) Error reading from stdin
builder for ‘/nix/store/gs9ik7yf8iilsikkfing74i70m0diax3-emacs-tide-20170509.1134.drv’ failed with exit code 255
cannot build derivation ‘/nix/store/m3p080aani4rw82llp8nqk93cw2nvirk-emacs-with-packages-25.2.drv’: 1 dependencies couldn't be built
Solution was to disable the large file warning threshold when
installing packages.
This still causes some uncached rebuilds, but master(!) and staging
move too fast forward rebuild-wise, so Hydra might never catch up.
(There are also other occasional problems.)
Therefore I merge at this point where the rebuild isn't that bad.
We need to make sure that `$revs` ends with a space, since files must always
end with newlines. The previous code ignored the last entry in `$revs`, because
read already returns non-zero exit code for the last entry, as it does not end
with a space.
This change fixes several defects in the way `wrapGAppsHook` selected
the executable to wrap.
Previously, it would wrap any top-level files in the target `/bin` and
`/libexec` directories, including directories and non-executable
files. In addition, it failed to wrap files in subdirectories.
Now, it uses `find` to iterate over these directory hierarchies,
selecting only executable files for wrapping.
This change fixes several defects in the way `wrapGAppsHook` selected
the executable to wrap.
Previously, it would wrap any top-level files in the target `/bin` and
`/libexec` directories, including directories and non-executable
files. In addition, it failed to wrap files in subdirectories.
Now, it uses `find` to iterate over these directory hierarchies,
selecting only executable files for wrapping.
When not using sandboxing, /usr/share/git-core/templates may leak into the
nix build through the libgit2 hardcoded default template search path. We now
explictly set the templatedir to avoid this problem.
See https://github.com/bennofs/nix-index/issues/2#issuecomment-296268983 for
an example case of nondeterminism.
This typo was likely introduced by copy-pasting the error message from elsewhere and forgetting to change the text, during the MD5 deprecation process (#4491).
This patch fixes file modification times to $SOURCE_DATE_EPOCH, and
ensures that files originating from the store are owned by root:root.
Both changes improve reproducibility, and the latter allows proper
building on a host where the store is owned by a non-root user.
Every Rust derivation used to emit a warning like the following:
```
setting SOURCE_DATE_EPOCH to timestamp 1490877042 of file cargo-6e0c18c/Cargo.lock
warning: file cargo-6e0c18c/Cargo.lock may be generated; SOURCE_DATE_EPOCH may be non-deterministic
```
The reason is that the dependencies are copied without preserving
timestamps. Changing the build script to timestamp-preserving copy
removes the warning.
This is a flag that disables subversion keyword substitution.
Keyword substitution inserts metadata into the files being checked
out, and is therefore somewhat at odds with build reproducibility.
In particular, it can become a problem if you're trying to switch
between svn and a git export of the same thing (keyword substitutions
are normally not exported into git).
This allows for a less blanket approach than nuke-refs, targetting specific
references that we know we don't want rather than all references that we don't
know we want.
Because of bash 4.4 the semantics GLOBIGNORE changed.
This resulted in already compressed manpages to be compressed twice.
Also be careful about symlinks to fix#21777, e.g. the ledger example.
See
8214bb953d
for the cargo commit which deprecated the registry.index key, and
implements this as a replacement. This gets rid of the error message
warning: custom registry support via the `registry.index` configuration is being removed, this functionality will not work in the future
This script is not needed anymore since "nix-prefetch-url --unpack
<url>" and "nix-prefetch-url -A foo.src" (where "foo.src" is a
fetchzip / fetchFromGitHub call) work fine.
This reverts commit 3d9017602b.
This didn't quite work as I had expected ... While it seemed okay at
first, it fails to propagate all the attributes it used to (notably
features). I'll revisit this later but reverting for now.
This adds a `dhallToNix` utility which compiles expression from the Dhall
configuration language to Nix using Nix's support for "import from derivation".
The main motivation of this compiler is to allow users to carve out small typed
subsets of Nix projects. Everything in the Dhall language (except `Double`s)
can be translated to Nix in this way, including functions.
This is required for Aarch64 since a lot of source tarballs ship with
outdated configure scripts that don't recognize aarch64. Simply
replacing the config.guess and config.sub with new versions from
upstream makes them build again.
This same approach is used by at least Buildroot and Fedora. In
principle this could be enabled for all architectures but
conditionalizing this on aarch64 avoids a mass rebuild on x86.
To achieve reproducible results, `cpio` archive members are added in
sorted order and inodes renumbered.
The `cpio-clean.pl` script is made obsolete by setting mtimes via
`touch` & using `cpio --reproducible`. Suggested by @dezgeg in
https://github.com/NixOS/nixpkgs/pull/21273#issuecomment-268116605.
Note that using `--reproducible` means that initial ramdisk creation now
requires at least `cpio` version 2.12 (released in 2015).
Deprecation warnings should not be used in Nixpkgs because they spam
innocent "nix-env -qa" users with (in this case) dozens of messages
that they can't do anything about.
This also reverts commit 2ca8833383.
I believe this reduces surprises and is actually simpler semantically.
This is important e.g. for relative symlinks when moving both source
and target - now the order of moving won't matter.
Fixes#20723 (a particular instance of the surprise).
This makes the response file handling more consistent with GCC.
For example, a reponse file may contain:
"-Wl,$ORIGIN"
GCC will treat this as a double quoted string and not expand the
variable reference. Previously, cc-wrapper would expand the variable
in the same was as if the string was provided on the command line.
This commit extends fetchFromGitHub with ability to fetch GitHub
repositories with submodules, so we can use the function consistently
with all GitHub repositories.
Note it doesn't change the previous behavior.
Include files are typically in the libc package's `dev` output but previously `-isystem` was looking in `out`, resulting in my cross-compilation environment not finding its include files.
The motivation for this change is the following: As gnu-netcat,
e. g. does not support ipv6, it is not suitable as default netcat.
This commit also fixes all obvious build issues caused by this change.
This is a temporary work-around to fix using grsecurity on NixOS with
the new kernelPackages/kernelPatches machinery.
For whatever reason, when `security.grsecurity.enable = true`, the grsec
patch ends up being applied twice, causing the kernel build to fail.
Until the root cause of this is identified, we hack around it by simply
pruning duplicate patches in the grsec kernel builder.
Closes#19698
`stripHash` documentation states that it prints out the stripped name to
the stdout, but the function stored the value in `strippedName`
instead.
Basically all usages did something like
`$(stripHash $foo | echo $strippedName)` which is just braindamaged.
Fixed the implementation and all invocations.
extraFlagsArray should not be exposed outside of `makeWrapper`, it
should only be possible to set it inside a script supplied via the
`--run` argument.
This ensures that most "trivial" derivations used to build NixOS
configurations no longer depend on GCC. For commands that do invoke
gcc, there is runCommandCC.
This is a standard environment that doesn't contain a C/C++
compiler. This is mostly to prevent trivial builders like runCommand
and substituteAll from pulling in gcc for simple configuration changes
on NixOS.
When building an image with multiple layers, files
already included in an underlying layer are supposed to
be excluded from the current layer. However, some subtleties
in the way filepaths are compared seem to be blocking this.
Specifically:
* tar generates relative filepaths with directories ending in '/'
* find generates absolute filepaths with no trailing slashes on directories
That is, paths extracted from the underlying tarball look like:
nix/store/.../foobar/
whereas the layer being generated uses paths like:
/nix/store/.../foobar
This patch modifies the output of "tar -t" to match the latter format.
They now go to devman, devdoc, or $outputMan, in that order. This is
to prevent cases such as the man-pages package quietly losing its
section 3 pages.
This builds elisp to setup an emacs buffer with the packages given
available. See shlevy/nix-buffer for more information.
Currently only modifies $PATH.
This one was already merged into release-16.09, so let's not have the
stable branch is ahead of master and confuse things. In addition to
that, currently we have an odd situation that master has less things
actually finished building than in staging.
Conflicts:
pkgs/data/documentation/man-pages/default.nix
This was one of the ways to build packages, we are trying
hard to minimize different ways so it's easier for newcomers
to learn only one way.
This also:
- removes texLive (old), fixes#14807
- removed upstream-updater, if that code is still used it should be in
separate repo
- changes a few packages like gitit/mit-scheme to use new texlive
Previously, features.grsecurity wasn't actually set due to a bug in the
grsec builder. We now rely on the generic kernel builder to set features
from kernelPatches.
This reverts commit da68127737, reversing
changes made to 2ba494b728.
These changes are reverted because they cause an incompatibility which
they should not.
"emacsWrapper" replaces emacsWithPackages. In addition to "packagesFun",
emacsWrapper has an optional variable called "execStart". execStart can
be used to append elisp to the default site-start.el script. This is
useful for providing a way to load a user's .emacs.d/init.el
file. "emacsWithPackages" is implemented with emacsWrapper for
convenience and compatability.
Sometimes patches start without a leading prefix. We default to strip
one prefix or path component from patches (-p1) in the patchPhase in
stdenv.
As all patches should therefore be in this format, fetchpatch should
have an option to normalize patch paths. This commit introduces a new
argument to fetchpatch called addPrefixes that adds one patch prefix to
the old and new paths in a patch before putting it into the store.
fetchMavenArtifact downloads a Maven artifact given a group id, an artifact id,
and a version.
Example usage:
org_apache_httpcomponents_httpclient_4_5_2 = fetchMavenArtifact {
groupId = "org.apache.httpcomponents";
artifactId = "httpclient";
version = "4.5.2";
sha256 = "0ms00zc28pwqk83nwwbafhq6p8zci9mrjzbqalpn6v0d80hwdzqd";
# Optionally: repos = [ ... urls to some Maven repos to use ... ];
# Optionally: url, urls - pointing directly to a specific jar url.
};
Now `org_apache_httpcomponents_httpclient_4_5_2.jar` points to the downloaded
JAR file, while `org_apache_httpcomponents_httpclient_4_5_2` refers to a
derivation that when used used in `buildInputs` will be automatically added to
the Java classpath.
downloads.sourceforge.net is the official way to download tarballs from
SourceForge. However, it is reported as unreliable due to SF's weird
load balancing system.
This commit gives the official mirror utmost priority, and will use
other configured mirrors (which may be temporary) as a fallback only
when the official one can't be reached/download fails/hangs.
References: NixOs/nixpkgs#16900
This patch fixes#16614 and #16741.
The first issue was caused by the fact that both `/share` and
`/share/fish/vendor_completions.d` end in the `pathsToLink`. The
`pkgs/build-support/buildenv/builder.pl` creates `/share`, then links
`/share/fish` under `/share` and then tries to create the directory
`/share/fish/vendor_completions.d` and fails because it already exists.
The simplest way to reproduce the issue is to build the next Nix
expression:
```nix
let pkgs = import <nixpkgs> { };
in pkgs.buildEnv {
name = "buildenv-issue";
paths = [
pkgs.fish
pkgs.vim
];
pathsToLink = [
"/share"
"/share/fish/vendor_completions.d"
];
}
```
The second issue is more critical and was caused by the fact findFiles
doesn't recurse deep enough. It stops at first unique directory for the
package (e.g., "/share" or even "/") and later the scripts decides it
shouldn't link it as it doesn't match pathsToLink (e.g., "/share/fish"),
so the result is empty.
The test:
```nix
let pkgs = import <nixpkgs> { };
in pkgs.buildEnv {
name = "buildenv-issue";
paths = [
pkgs.fish
pkgs.vim
];
pathsToLink = [
"/share/fish/functions"
];
}
```
or
```nix
let pkgs = import <nixpkgs> { };
in pkgs.buildEnv {
name = "buildenv-issue";
paths = [
pkgs.vim
];
pathsToLink = [
"/share"
];
}
```
Prior to this commit, trailing whitespace would be introduced when
modifying '#!' lines with no arguments. For example (whitespace added):
/nix/store/.../foo: interpreter directive changed
from "/bin/bash"
to "/nix/store/...-bash-4.3-p42/bin/bash "
/nix/store/.../bar: interpreter directive changed
from "/bin/baz wef"
to "/nix/store/...-baz wef "
We add a sed command to strip trailing whitespace, so the above commands
would drop the two spaces after "bash", or the one space after "baz wef".
abbradar: fixed commit title
Closes#16785.
This fixes a regression caused by commit f56ab9e
("nix-prefetch-git: Include the date in the machine-readable [...]")
where a couple of directory paths printed by pushd/popd appeared before
the JSON output on stdout (thus breaking it). Fix it by redirecting the
extraneous output to /dev/null.
Reported by Michael Alan Dorman <mdorman@ironicdesign.com>.
stdout, in strict ISO 8601 format.
This will be helpful for automatically updating fetchgit expressions
and the dates in version numbers associated with them.
stripHash uses a global variable to communicate it's computation
results, but it's not necessary. You can just pipe to stdout in a
subshell. A function mostly behaves like just another command.
baseHash() also introduces a suffix-stripping capability since it's
something the users of the function tend to use.
The tests need to expand passed variable and very carefully.
I could see no other easy way than to change single-quoting in
makeWrapper to double-quoting.
The tests now fail with the same problem as on master...
This patch replaces the old grsecurity kernels with a single NixOS
specific grsecurity kernel. This kernel is intended as a general
purpose kernel, tuned for casual desktop use.
Providing only a single kernel may seem like a regression compared to
offering a multitude of flavors. It is impossible, however, to
effectively test and support that many options. This is amplified by
the reality that very few seem to actually use grsecurity on NixOS,
meaning that bugs go unnoticed for long periods of time, simply because
those code paths end up never being exercised. More generally, it is
hopeless to anticipate imagined needs. It is better to start from a
solid foundation and possibly add more flavours on demand.
While the generic kernel is intended to cover a wide range of use cases,
it cannot cover everything. For some, the configuration will be either
too restrictive or too lenient. In those cases, the recommended
solution is to build a custom kernel --- this is *strongly* recommended
for security sensitive deployments.
Building a custom grsec kernel should be as simple as
```nix
linux_grsec_nixos.override {
extraConfig = ''
GRKERNSEC y
PAX y
# and so on ...
'';
}
```
The generic kernel should be usable both as a KVM guest and host. When
running as a host, the kernel assumes hardware virtualisation support.
Virtualisation systems other than KVM are *unsupported*: users of
non-KVM systems are better served by compiling a custom kernel.
Unlike previous Grsecurity kernels, this configuration disables `/proc`
restrictions in favor of `security.hideProcessInformation`.
Known incompatibilities:
- ZFS: can't load spl and zfs kernel modules; claims incompatibility
with KERNEXEC method `or` and RAP; changing to `bts` does not fix the
problem, which implies we'd have to disable RAP as well for ZFS to
work
- `kexec()`: likely incompatible with KERNEXEC (unverified)
- Xen: likely incompatible with KERNEXEC and UDEREF (unverified)
- Virtualbox: likely incompatible with UDEREF (unverified)
Close#15803. This avoids the error:
while setting up the build environment: executing
‘/nix/store/7sb42axk5lrxqz45nldrb2pchlys14s1-bash-4.3-p42/bin/bash’:
Argument list too long
Note: I wanted to make it optional based on buildCommand length,
but that seems pointless as I'm sure it's less performant.
Amended by vcunat:
https://github.com/NixOS/nixpkgs/pull/15803#issuecomment-224841225
This takes another approach at binding FHS directory structure. We
now bind-mount all the root filesystem to directory "/host" in the target tree.
From that we symlink all the directories into the tree if they do not already
exist in FHS structure.
This probably makes `CHROOTENV_EXTRA_BINDS` unnecessary -- its main usecase was
to add bound directories from the host to the sandbox, and we not just symlink
all of them. I plan to get some feedback on its usage and maybe deprecate it.
This also drops old `buildFHSChrootEnv` infrastructure. The main problem with it
is it's very difficult to unmount a recursive-bound directory when mount is not
sandboxed. This problem is a bug even without these changes -- if
you have for example `/home/alice` mounted to somewhere, you wouldn't see
it in `buildFHSChrootEnv` now. With the new directory structure, it's
impossible to use regular bind at all. After some tackling with this I realized
that the fix would be brittle and dangerous (if you don't unmount everything
clearly and proceed to removing the temporary directory, bye-bye fs!). It also
probably doesn't worth it because I haven't heard that someone actually uses it
for a long time, and `buildFHSUserEnv` should cover most cases while being much
more maintainable and safe for the end-user.
1. When multiple versions of the same package are required
$revs is an array.
2. When cargo fetch is run it usually doesn't need a network
connection. But when it does SSL_CERT_FILE isn't set.
