Due the recent inclusion of broadcom-bt-firmware in enableAllFirmware,
it was required to set `nixpkgs.config.allowUnfree` to obtain the full
list. To make this dependency more explicit an assertion is added and an
alternative option `enableRedistributalFirmware` is provided to only
obtain firmware with an license allowing redistribution.
This changes much of the make-disk-image.nix logic (and thus most NixOS
image building) to use LKL to set up the target directory structure rather
than a Linux VM. The only work we still do in a VM is less IO-heavy stuff
that while still time-consuming, is less of the overall load. The goal is
to kill more of that stuff, but that will require deeper changes to NixOS
activation scripts and switch-to-configuration.pl, and I don't want to
bite off too much at once.
The key distinction I'm drawing is that there's a component that deals
with the store of the machine being built, and another component for
the store building it. The inner part of it assumes nothing from the
builder (doesn't need chroot or root powers) so it can run comfortably
inside a Nix build, as well as nixos-rebuild. I have some upcoming work
that will use that to significantly speed up and streamline image builds
for NixOS, especially on virtualized hosts like EC2, but it's also a
reasonable speedup on native hosts.
Currently, if you try to build a NixOS config including channel.nix, e.g.:
nix-build -I nixpkgs=. -I nixos-config=nixos/modules/installer/cd-dvd/installation-cd-minimal.nix nixos -A config.system.build.isoImage
twice in a row, you end up with two different build results. This is
caused by the 'result' symlink of the first build affecting the channel
contents of the second build.
If we use filterSource with a predicate that ignores the 'result'
symlinks, the problem is gone. Do the same thing for VIM/Emacs
swap/backup files to avoid even more 'spurious' rebuilds.
Additionally, filter out the '.git' directory at the same time, as we
'rm -rf' it from the result anyway. This avoids a considerable amount of
unnecessary file I/O copying and deleting the .git directory.
Turns out all variants of start.elf and fixup.dat are needed (depending
on what's in config.txt). I was under the mistaken impression that you
were supposed to rename one of the variants to switch using them, but
nope.
* cpu-freq: Try powersave if ondemand is not available
* Revert "cpu-freq: Try powersave if ondemand is not available"
This reverts commit 4dc56db37e32dcfecd667ebbf88263e47b296097.
Consult available scaling governors; for freshly generated configs, this provides a better experience than relying on a default that might not work everywhere.
The profile minimal has several drawbacks: no man pages, unusual 'dbus'
lib that makes many X11 pieces to rebuild, etc.
With xz compression in the squashfs, despite these additions, the iso is
smaller than what it was in 16.09.
This needs to be included for VirtualBox to detect that it needs to start the video driver. "modesetting" is also set in virtualbox-image.nix but this line seems to take precedence over that one (even though the virtualbox-image.nix has a higher override?) This should fix the problems that I and a few others have been having with the .ova files built for nixos.org.
Fixes#20007.
`startAt = ""` as in `startAt = optionalString false ...` results
in an invalid timer unit (due to "" being promoted to a singleton
list and not filtered out).
Ref: c9941c4b5e
Since some changes to the setuid wrappers, there is a symlink involved
and it doesn't resolve correctly inside the chroot. Do the check inside
the chroot to make it work again.
This partially reverts commit 0aa7520670.
Fine for rsync to be in system path but we still need the explicit path
in nixos-install in case it is invoked from non-NixOS systems and also
to fix OVA test failure
See also 0aa7520670
cc @edolstra
This reverts commit 582313bafe.
Removing rsync is actually pointless because nixos-install depends on
it. So if it's part of the system closure, we may as well provide it
to users.
Probably with the next Nix release we can drop the use of rsync and
use "nix copy" instead.
Before this commit updating /var/setuid-wrappers/ folder introduced
a small window where NixOS activation scripts could be terminated
and resulted into empty /var/setuid-wrappers/ folder.
That's very unfortunate because one might lose sudo binary.
Instead we use two atomic operations mv and ln (as described in
https://axialcorps.com/2013/07/03/atomically-replacing-files-and-directories/)
to achieve atomicity.
Since /var/setuid-wrappers is not a directory anymore, tmpfs mountpoints
were removed in installation scripts and in boot process.
Tested:
- upgrade /var/setuid-wrappers/ from folder to a symlink
- make sure /run/setuid-wrappers-dirs/ legacy symlink is really deleted
- Replace hand-rolled version of nixos-install in make-disk-image by an
actual call to nixos-install
- Required a few cleanups of nixos-install
- nixos-install invokes an activation script which the hand-rolled version
in make-disk-image did not do. We remove /etc/machine-id as that's
a host-specific, impure, output of the activation script
Testing:
nix-build '<nixpkgs/nixos/release.nix>' -A tests.installer.simple passes
Also tried generating an image with:
nix-build -E 'let
pkgs = import <nixpkgs> {};
lib = pkgs.lib;
nixos = import <nixpkgs/nixos> {
configuration = {
fileSystems."/".device = "/dev/disk/by-label/nixos";
boot.loader.grub.devices = [ "/dev/sda" ];
boot.loader.grub.extraEntries = '"''"'
menuentry "Ubuntu" {
insmod ext2
search --set=root --label ubuntu
configfile /boot/grub/grub.cfg
}
'"''"';
};
};
in import <nixpkgs/nixos/lib/make-disk-image.nix> {
inherit pkgs lib;
config = nixos.config;
diskSize = 2000;
partitioned = false;
installBootLoader = false;
}'
Then installed the image:
$ sudo df if=./result/nixos.img of=/dev/sdaX bs=1M
$ sudo resize2fs /dev/disk/by-label/nixos
$ sudo mount /dev/disk/by-label/nixos /mnt
$ sudo mount --rbind /proc /mnt/proc
$ sudo mount --rbind /dev /mnt/dev
$ sudo chroot /mnt /nix/var/nix/profiles/system/bin/switch-to-configuration boot
[ … optionally do something about passwords … ]
and successfully rebooted to that image.
Was doing all this from inside a Ubuntu VM with a single user nix install.
- Fix --no-bootloader which didn't do what it advertised
- Hardcode nixbld GID so that systems which do not have a nixbld user
can still run nixos-install (only with --closure since they can't
build anything)
- Cleanup: get rid of NIX_CONF_DIR(=/tmp)/nix.conf and pass arguments instead
- Cleanup: don't assume that the target system has '<nixpkgs/nixos>' or
'<nixos-config>' to see if config.users.mutableUsers. Instead check if
/var/setuid-wrappers/passwd is there
Installing NixOS now works from a Ubuntu host (using --closure).
nix-build -A tests.installer.simple '<nixpkgs/nixos/release.nix>' succeeds ✓
This reverts commit 1e534e234b.
We already should have a .git directory if it is managed via Git,
otherwise there is no way to get the Git revision if neither
.git-revision or .git is present.
But having .git-revision _and_ .git present seems very much redundant to
me.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Cc: @bennofs, @Profpatsch
Issue: #17218
- RPi3 successfully gets to U-Boot, but then fails to boot the kernel
due to a missing device tree file. This should get added to the 4.8
kernel release once this patch is merged: https://lkml.org/lkml/2016/6/1/841
- RPi2 is not tested, but it should successfully boot the NixOS image.
The Nix store squashfs is stored inside the initrd instead of separately
(cherry picked from commit 976fd407796877b538c470d3a5253ad3e1f7bc68)
Signed-off-by: Domen Kožar <domen@dev.si>
This command was useful when NixOS was spread across multiple
repositories, but now it's pretty pointless (and obfuscates what
happens, i.e. "git clone git://github.com/NixOS/nixpkgs.git").
- Enforce that an option declaration has a "defaultText" if and only if the
type of the option derives from "package", "packageSet" or "nixpkgsConfig"
and if a "default" attribute is defined.
- Enforce that the value of the "example" attribute is wrapped with "literalExample"
if the type of the option derives from "package", "packageSet" or "nixpkgsConfig".
- Warn if a "defaultText" is defined in an option declaration if the type of
the option does not derive from "package", "packageSet" or "nixpkgsConfig".
- Warn if no "type" is defined in an option declaration.
Allow usage of list of strings instead of a comma-separated string
for filesystem options. Deprecate the comma-separated string style
with a warning message; convert this to a hard error after 16.09.
15.09 was just released, so this provides a deprecation period during
the 16.03 release.
closes#10518
Signed-off-by: Robin Gloster <mail@glob.in>
Reportedly some ARM boards need some boot code at the start of a SD card
that could be larger than a megabyte. Change it to 8M, and while at it
reduce the /boot size such that the root partition should now start on a
128M boundary (the flash on SD cards really don't like non-aligned
writes these days).
Building config.system.build.isoImage would fail with the following
error using the channel:
ln: failed to create symbolic link
'/nix/store/zz0hzi5imrg4927v6f8mv281qs6v6pbq-nixos-16.03pre69762.e916273/nixos/nixpkgs/.': File exists
The fix skips symlink as it already exists if the channel
nixpkgs copy is used.
Fixes#10367
Setting nixosVersion to something custom is useful for meaningful GRUB
menus and /nix/store paths, but actuallly changing it rebulids the
whole system path (because of `nixos-version` script and manual
pages). Also, changing it is not a particularly good idea because you
can then be differentitated from other NixOS users by a lot of
programs that read /etc/os-release.
This patch introduces an alternative option that does all you want
from nixosVersion, but rebuilds only the very top system level and
/etc while using your label in the names of system /nix/store paths,
GRUB and other boot loaders' menus, getty greetings and so on.
This commit adds the options --build-host and --target-host to nixos-rebuild.
--build-host instructs nixos-rebuild to perform all nix builds on the
specified host (via ssh). Build results are then copied back to the
local machine and used when activating the system.
--build-target instructs nixos-rebuild to activate the configuration
not on the local machine but on the specified remote host. Build
results are copied to the target machine and then activated there (via ssh).
It is possible to combine the usage of --build-host and --target-host,
in which case you can perform the build on one remote machine and deploy
the configuration to another remote machine. The only requirement is that
the build host has a working ssh connection to the target host (if the
target is not local), and that the local machine can connect to both
the target and the build host. Also, your user must be allowed to copy
nix closures between the local machine and the target and host machines.
At no point in time are the configuration sources (the nix files) copied
anywhere. Instead, nix evaluation always happens locally
(with nix-instantiate). The drv-file is then copied and realised remotely
(with nix-store).
As a convenience, if only --target-host is specified, --build-host is
implicitly set to that host too. So if you want to build locally and deploy
remotely you have to explicitly set "--build-host localhost".
To activate (test, boot or switch) you need to have root access to the
target host. You can specify this by "--target-host root@myhost".
I have tested the obvious scenarios and they are working. Some of the
combinations of --build-host and --target-host and the various actions might
not make much sense, and should maybe be forbidden (like setting a remote
target host when building a VM), and some combinations might not work at all.
We seem to be in an unfortunate situation: booting without 'nomodeset'
causes hangs when booting on some NVIDIA cards (6948c3ab80), but on the
other hand adding 'nomodeset' prevents X from starting on other hardware
(e.g. issue #10381 and my Thinkpad X250 with an integrated Broadwell GPU).
Attempt to remedy this situation a bit by adding a separate entry in the
ISOLINUX menu (with the non-'nomodeset' being the default).
The most complex problems were from dealing with switches reverted in
the meantime (gcc5, gmp6, ncurses6).
It's likely that darwin is (still) broken nontrivially.
Since commits 89e9837 and 5b8dae8 the manual no longer depends on
evaluation of any packages from nixpkgs, so all errors of the form
"Package 'foo' is not supported on 'armv7l-linux'" are gone.
mmc_block and sdhci_acpi are both necessary for a Bay Trail Chromebook with an
internal eMMC drive. The sdhci_acpi module is detectable but I can not figure
out a way to check whether the mmc_block module is needed by just looking at
/sys/
I needed to add sdhci_acpi and mmc_block to my initrd modules in order to boot
my Chromebook. Looking under /sys/class/mmc_host/*/device/driver/module will
give us the sdhci_acpi dependency.
This makes the firmware available (or would, if someone switched off
enableAllFirmware). Corresponding kernel module should get auto-loaded.
See #9948. Close#9971.
If nixos-install is run on a machine with `nix.distributedBuilds = true`
the installation will fail at some point like this:
Died at /nix/store/4frhrl31cl7iahlz6vyvysy5dmr6xnh3-nix-1.10/libexec/nix/build-remote.pl line 115, <STDIN> line 1.
This is due to `nix.distributedBuilds` setting
NIX_BUILD_HOOK=/nix/store/.../build-remote.pl in the global environment,
which then gets confused in the minimal chroot created by nixos-install.
To avoid these kinds of issues with build hooks, just disable them in
the chroot.
Avoids this warning when running `nixos-rebuild switch`:
````
building Nix...
building the system configuration...
trace: Obsolete option `services.virtualboxGuest.enable' is used. It was renamed to `virtualisation.virtualbox.guest.enable'.
````
This option requests compatibility with older NixOS releases with
respect to stateful data, in cases where new releases have defaults
that might be incompatible with system state of existing NixOS
deployments. For instance, if we change the default version of
PostgreSQL, existing deployments will break if the new version can't
read databases created by the old version.
So for example, setting
system.stateVersion = "15.07";
requests that options like services.postgresql.package use defaults
corresponding to the 15.07 release branch. Note that
nixos-generate-config emits this option. (In the future, NixOps may
set system.stateVersion to the NixOS release in use when the machine
was created.)
See also #7939 for another motivating example.
The resulting image can be copied to a SD card with `dd` and is directly
bootable by a suitably configured U-Boot. Though depending on the board, some
extra steps are required for copying U-Boot itself to the SD card.
Inside the image is a partition table, with a FAT32 /boot and a normal
writable EXT4 rootfs. It's possible to directly reuse the SD image's
partition layout and "install" NixOS on the same SD card by replacing
the default configuration.nix and nixos-rebuild, and actually is the
preferred way to use these images. To assist in this installation
method, the boot scripts on the image automatically resize the rootfs
partition to fit the SD card on the first boot.
The SD images come in two flavors; one for the ARMv6 Raspberry Pi,
and one multiplatform image for all the boards supported by the
mainline kernel's multi_v7_defconfig config target. At the moment, these
have been tested on:
- Raspberry Pi Model B (512MB model)
- NVIDIA Jetson TK1
- Linksprite pcDuino3 Nano
To build, run:
nix-build '<nixpkgs/nixos>' -A config.system.build.sdImage \
-I nixos-config='<nixpkgs/nixos/modules/installer/cd-dvd/sd-image-armv7l-multiplatform.nix>'
It comes in handy to alter the menu label if you're not building a NixOS
installer image but for example if you want to build a live system and
still want to re-use the iso-image.nix module.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Should fix at least nixos.tests.installer.simple.x86_64-linux
http://hydra.nixos.org/build/23001712:
machine# error: cannot download Encode-Locale-1.03.tar.gz from any mirror
machine# builder for ‘/nix/store/y8gbx2d2fdcvvjy1z53xksfgq66ydlx0-Encode-Locale-1.03.tar.gz.drv’ failed with exit code 1
machine# cannot build derivation ‘/nix/store/y1knci7rix3asnh2b4kfv8jhl2j99xih-perl-Encode-Locale-1.03.drv’: 1 dependencies couldn't be built
machine# cannot build derivation ‘/nix/store/7xspjwh48kg16drv1jjg5cffaqbxbp8p-perl-libwww-perl-6.05.drv’: 1 dependencies couldn't be built
machine# cannot build derivation ‘/nix/store/8qsmz3bbk1jwhh50c3i9700bkmn8ns5c-nss-cacert-3.19.1.drv’: 1 dependencies couldn't be built
machine# cannot build derivation ‘/nix/store/0rgf2l3mdszs4a989ympwc9gk2k8wq6z-nixos-artwork-e71b684.drv’: 1 dependencies couldn't be built
...
Commit 159fed47bc (nixos/grub: Fix video display on efi) changed BIOS
systems to start in non-text mode as well. Enable FB_VESA to get a
framebuffer console on BIOS systems. Change FRAMEBUFFER_CONSOLE to 'y'
instead of the default 'm' to so the user doesn't need to manually load
the fbcon module anymore.
Other distros have similar defaults, at least on Arch:
CONFIG_FB_VESA=y
CONFIG_FRAMEBUFFER_CONSOLE=y
and on Ubuntu (12.04):
CONFIG_FB_VESA=m
CONFIG_FRAMEBUFFER_CONSOLE=y
Fixes#8139
Passing the chroot flag to nixos-install without arguments should now give you a
Bash shell as intended rather than try an empty path.
This was masked by the user's shell (usually /bin/bash) being defaulted to by
chroot, and being found since their paths used NixOS conventions.
When bootstrapping from other distributions, nixos-install is unable to find
various tools in the chroot since their paths aren't aware of NixOS conventions.
This makes a small change to existing code by specifying nixpkgs/nixos instead
of just nixos when running nix-instantiate in the chroot. I haven't tested this
outside of bootstrapping, but the same specification is used elsewhere in the
code so I don't see why it wouldn't work.
This partially reverts commit 3a4fd0bfc6.
Addresses another concern by @edolstra that users might not want to
update *all* channels. We're now reverting to the old behaviour but
after updating the "nixos" channel, we just check whether the channel
ships with a file called ".update-on-nixos-rebuild" and if it exists, we
update that channel as well.
Other channels than these are not touched anymore.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Should make it even easier to use custom channels, because whenever the
user does a "nixos-rebuild --upgrade", it will also upgrade possibly
used ("used" as in referenced in configuration.nix) channels besides
"nixos". And if you also ship a channel tied to a particular version of
nixpkgs or even remove the "nixos" channels, you won't run into
unexpected situations where the system is not updating your custom
channels.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Fixes#6795.
This was co-authored with @bobvanderlinden.
(cherry picked from commit e19ac248ae59fd327c32b1ae3e37792c22a7c7ac)
Signed-off-by: Domen Kožar <domen@dev.si>
Conflicts:
nixos/modules/installer/cd-dvd/iso-image.nix
There are a number of hidden restrictions on the syslinux
configuration file that come into play when UNetbootin
compatiblity is desired. With this commit these are documented.
This changes the bootloader for iso generation from Grub to
syslinux. In addition this adds USB booting support, so that
"dd" can be used to burn the generated ISO to USB thumbdrives
instead of needing applications like UnetBootin.
The nixbld group doesn't need read permission, it only needs write and
execute permission.
(cherry picked from commit 066758758e7c0768ff8da51d208cdae0f33b368c)
‘nixos-rebuild dry-activate’ builds the new configuration and then
prints what systemd services would be stopped, restarted etc. if the
configuration were actually activated. This could be extended later to
show other activation actions (like uids being deleted).
To prevent confusion, ‘nixos-rebuild dry-run’ has been renamed to
‘nixos-rebuild dry-build’.
Since we're using HTTPS for the binary cache (introduced in faf0797) by
default, the binary cache should also be available during installation.
The file that is defined in SSL_CERT_FILE outside of the chroot is
copied over to /tmp/ca-cert.crt inside the chroot, so we have an
absolute path we can reference during nixos-install. However, this might
end up with the file not being cleaned up properly from outside of the
store, but neither would be /tmp/root so the cleanup issue needs to be
solved in another place (or commit to be more exact).
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Especially new users could be confused by this, so we're now marking
services.virtualbox.enable as obsolete and defaulting to
services.virtualboxGuest.enable instead. I believe this now makes it
clear, that this option is for guest additions only.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
The NixOS manual says modules have the following signature:
{ config, lib, pkgs, ... }:
But our generated configuration.nix file lacks the 'lib' part. Add it.
The host id value gets generated by reading a 32-bit value from
/dev/urandom.
This makes programs that rely on a correct host id more reliable.
It also makes using ZFS more seamless, as you don't need to configure
the hostId manually; instead, it becomes part of your config from the
moment you install NixOS.
The old boot.spl.hostid option was not working correctly due to an
upstream bug.
Instead, now we will create the /etc/hostid file so that all applications
(including the ZFS kernel modules, ZFS user-space applications and other
unrelated programs) pick-up the same system-wide host id. Note that glibc
(and by extension, the `hostid` program) also respect the host id configured in
/etc/hostid, if it exists.
The hostid option is now mandatory when using ZFS because otherwise, ZFS will
require you to force-import your ZFS pools if you want to use them, which is
undesirable because it disables some of the checks that ZFS does to make sure it
is safe to import a ZFS pool.
The /etc/hostid file must also exist when booting the initrd, before the SPL
kernel module is loaded, so that ZFS picks up the hostid correctly.
The complexity in creating the /etc/hostid file is due to having to
write the host ID as a 32-bit binary value, taking into account the
endianness of the machine, while using only shell commands and/or simple
utilities (to avoid exploding the size of the initrd).
This changes the bootloader for iso generation from Grub to
syslinux. In addition this adds USB booting support, so that
"dd" can be used to burn the generated ISO to USB thumbdrives
instead of needing applications like UnetBootin.
