Using a simple algorithm, convert the references to a path in to a
sorted list of dependent paths based on how often they're referenced
and how deep in the tree they live. Equally-"popular" paths are then
sorted by name.
The existing writeReferencesToFile prints the paths in a simple
ascii-based sorting of the paths.
Sorting the paths by graph improves the chances that the difference
between two builds appear near the end of the list, instead of near
the beginning. This makes a difference for Nix builds which export a
closure for another program to consume, if that program implements its
own level of binary diffing.
For an example, Docker Images. If each store path is a separate layer
then Docker Images can be very efficiently transfered between systems,
and we get very good cache reuse between images built with the same
version of Nixpkgs. However, since Docker only reliably supports a
small number of layers (42) it is important to pick the individual
layers carefully. By storing very popular store paths in the first 40
layers, we improve the chances that the next Docker image will share
many of those layers.*
Given the dependency tree:
A - B - C - D -\
\ \ \ \
\ \ \ \
\ \ - E ---- F
\- G
Nodes which have multiple references are duplicated:
A - B - C - D - F
\ \ \
\ \ \- E - F
\ \
\ \- E - F
\
\- G
Each leaf node is now replaced by a counter defaulted to 1:
A - B - C - D - (F:1)
\ \ \
\ \ \- E - (F:1)
\ \
\ \- E - (F:1)
\
\- (G:1)
Then each leaf counter is merged with its parent node, replacing the
parent node with a counter of 1, and each existing counter being
incremented by 1. That is to say `- D - (F:1)` becomes `- (D:1, F:2)`:
A - B - C - (D:1, F:2)
\ \ \
\ \ \- (E:1, F:2)
\ \
\ \- (E:1, F:2)
\
\- (G:1)
Then each leaf counter is merged with its parent node again, merging
any counters, then incrementing each:
A - B - (C:1, D:2, E:2, F:5)
\ \
\ \- (E:1, F:2)
\
\- (G:1)
And again:
A - (B:1, C:2, D:3, E:4, F:8)
\
\- (G:1)
And again:
(A:1, B:2, C:3, D:4, E:5, F:9, G:2)
and then paths have the following "popularity":
A 1
B 2
C 3
D 4
E 5
F 9
G 2
and the popularity contest would result in the paths being printed as:
F
E
D
C
B
G
A
* Note: People who have used a Dockerfile before assume Docker's
Layers are inherently ordered. However, this is not true -- Docker
layers are content-addressable and are not explicitly layered until
they are composed in to an Image.
Thanks to @symphorien for this work, which apart from the update itself
includes a few more fixes and cleanups.
I've tested building and running the upgraded Paperwork and while I
haven't done extensive testing on every little feature it seems to work
so far.
The changes also include an addition to fetchFromGitLab, which allows to
specify a group.
Merges: #46487
* needs at least qtgraphicaleffects not mentioned in docs
* doesn't want to use our libqmatrixclient, so let it use bundled
* took a few runs (or perhaps just patience with one) to populate,
when my profile icon appeared it seemed ready-- then click it
to see list of rooms and such.
- New dependency on 'getconf' binary for
3aa619e9ef/src/vm_memory_monitor.erl (L448)
- New dependency on 'socat' for systemd notifications
4a3ee3a336/src/rabbit.erl (L361)
- elixir_1_6 for a new 'rabbitmqctl' tool
- Replace patching with providing custom PATH, as we already have some
other things here
- Renamed package in all-packages.nix from a legacy spelling
This includes the initialy commit was done by @Mic92 plus a few fixes
from my side. So essentially this avoids patching statically linked
executables and also speeds up searching for ELF files altogether.
I've tested this by comparing the outputs of all the derivations which
make use of this hook using the following Nix expression:
let
getPackagesForRev = rev: with import (builtins.fetchGit {
url = ./.;
inherit rev;
}) { config.allowUnfree = true; }; [
cups-kyodialog3 elasticsearch franz gurobi javacard-devkit
masterpdfeditor maxx oracle-instantclient powershell reaper
teamviewer unixODBCDrivers.msodbcsql17 virtlyst wavebox zoom-us
];
pkgs = import <nixpkgs> {};
baseRev = "ef764eb0d8314b81a012dae04642b4766199956d";
in pkgs.runCommand "diff-contents" {
chset = pkgs.lib.zipListsWith (old: new: pkgs.runCommand "diff" {
inherit old new;
nativeBuildInputs = [ pkgs.nukeReferences ];
} ''
mkdir -p "''${NIX_STORE#/}"
cp --no-preserve=all -r "$old" "''${NIX_STORE#/}"
cp --no-preserve=all -r "$new" "''${NIX_STORE#/}"
find "''${old#/}" "''${new#/}" \
\( -type f -exec nuke-refs {} + \) -o \( -type l -delete \)
mkdir "$out"
echo "$old" > "$out/old-path"
echo "$new" > "$out/new-path"
diff -Nur "''${old#/}" "''${new#/}" > "$out/diff" || :
'') (getPackagesForRev baseRev) (getPackagesForRev "");
} ''
err=0
for c in $chset; do
if [ -s "$c/diff" ]; then
echo "$(< "$c/old-path") -> $(< "$c/new-path")" \
"differs, report: $c/diff" >&2
err=1
fi
done
[ $err -eq 0 ] && touch "$out"
''
With these changes there is only one derivation which has altered
contents, which is "franz". However the reason why it has differing
contents is not directly because of the autoPatchelfHook changes, but
because the "env-vars" file from the builder is in
"$out/opt/franz/env-vars" (Cc: @gnidorah) and we now have different
contents for NIX_CFLAGS_COMPILE and other environment variables.
I also tested this against a random static binary and the hook no longer
tries to patch it.
Merges: #47222
The compilers themselves can pull them from `bootPkgs`, where they
should always come from anyways. This enforces that, simplifies that
code, and allows use to avoid more `rec { ... }` too.
2a6e4ae49a and
e51f736076 reverted a bit too much, and I
initially missed this when reviewing. The release notes already still
mention this change, too.
This isn't necessary and causes build failures for certain versions.
CC libavfilter/libmpcodecs/vf_eq2.o
CC libavfilter/libmpcodecs/vf_fil.o
CC libavfilter/libmpcodecs/vf_fspp.o
libavfilter/libmpcodecs/vf_fspp.c:939:18: error: unknown token in expression
movq %mm2, 0*8+(%rsp)
/cc ZHF #45961
Add package libratbag and service module ratbagd
Libratbag contains ratbagd daemon and ratbagctl cli to configure
buttons, dpi, leds, etc. of gaming mice.
Add mvnetbiz to maintainers.
This package providesa completion input method for faster typing.
See https://mike-fabian.github.io/ibus-typing-booster
Detailed instructions how to activate this IBus engine on your desktop
can be found in the upstream docs: https://mike-fabian.github.io/ibus-typing-booster/documentation.html
A simple VM with the Gnome3 desktop and activated `ibus' looks like
this:
```nix
{
emojipicker = { pkgs, ... }: {
services.xserver = {
enable = true;
desktopManager.gnome3.enable = true;
desktopManager.xterm.enable = false;
};
users.extraUsers.vm = {
password = "vm";
isNormalUser = true;
};
i18n.inputMethod.ibus.engines = [
pkgs.ibus-engines.typing-booster
];
i18n.inputMethod.enabled = "ibus";
virtualisation.memorySize = 2048;
};
}
```
Fixes#38721
This is only a minor bugfix release and updates the fallback CA root
certificates. For NixOS this is usually not required as the probe paths
will match there, but for non-NixOS users it might be helpful.
Signed-off-by: aszlig <aszlig@nix.build>
Qt 5.11 was downgraded to 5.9 because of two issues:
- spawns errors like
```
qrc:/qml/SignInWaiting.qml:20:9: QML BusyIndicator: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:26:9: QML Text: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:20:9: QML BusyIndicator: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:26:9: QML Text: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:20:9: QML BusyIndicator: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:26:9: QML Text: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:20:9: QML BusyIndicator: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:26:9: QML Text: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:20:9: QML BusyIndicator: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:26:9: QML Text: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
```
- Google login doesn't work. It just doesn't start embedded webbrowser
This also updates the bootstrap tool builder to LLVM 5, but not the ones
we actually use for bootstrap. I'll make that change in a subsequent commit
so as to provide traceable provenance of the bootstrap tools.
A little shim derivation to get this header for Darwin, where it is
needed for cross compilation.
There's no real reason to do glibc and musl like that, but as I'm
maintaining it I suppose I can go overboard like that.
morituri has been dead for a while now and uses gst-python which is
no longer supported wth Python 2. whipper is a maintained fork,
packaged, for example, in Arch.
Looks CoreFoundation related.
Undefined symbols for architecture x86_64:
"_CFURLResourceIsReachable", referenced from:
fsevent_sys::core_foundation::str_path_to_cfstring_ref::h0ea4bd94e2c613f2 in libfsevent_sys-ef30b6879660a6c1.rlib(fsevent_sys-ef30b6879660a6c1.fsevent_sys7-49ce33334334dd3a5c7883bf4070f954.rs.rcgu.o)
ld: symbol(s) not found for architecture x86_64
/cc ZHF #45961
The build expression got quiet large over time and to make it a bit
easier to grasp the different scripts involved in the build are now
separated from the nix file.
While building the tests LD gets called with -mmacosx-version-min=10.10
which is a CC flag, causing the build to fail with LD=ld. This is
pretty common with perl packages.
/cc ZHF #45961
See https://hydra.nixos.org/build/80828287
Moves `mahotas` out of `python-packages.nix` into its own file and fixes
broken test cases by skipping them using nosetest's `@nottest`
annotation.
These tests broke from time to time in a sandbox and are therefore
considered impure.
Addresses #45960
The examples fail with an opengl related issue:
Undefined symbols for architecture x86_64:
"SimpleOpenGL3App::SimpleOpenGL3App(char const*, int, int, bool)", referenced from:
_main in main_opengl_single_example.o
"_useShadowMap", referenced from:
GL_ShapeDrawer::drawScene(btDiscreteDynamicsWorld const*, bool, int) in GL_ShapeDrawer.o
ld: symbol(s) not found for architecture x86_64
And the tests need an extra dependencly, possibley related to
https://github.com/bulletphysics/bullet3/issues/819
ld: library not found for -lBussIK
/cc ZHF #45961
Create the top-level packages attribute
'hylafaxplus' that builds HylaFAX+ .
Note:
The nobody uid and the nogroup gid
are hardcoded in the package.
The package build recipe file
contains options to modify these ids.
See https://hydra.nixos.org/build/80998335.
Upstream doesn't support QT 5.11 ATM which broke compilation:
```
src/dialogs/savedialog.cpp: In constructor ‘SaveDialog::SaveDialog(QWidget*, Qt::WindowFlags)’:
src/dialogs/savedialog.cpp:37:34: error: invalid use of incomplete type ‘class QButtonGroup’
group = new QButtonGroup(this);
```
The Arch community recommends to use an older QT version to fix
this (https://aur.archlinux.org/packages/chessx/).
Furthermore the `QT_PLUGIN_PATH` wasn't set properly which broke the
runtime since QT coudln't find the `xcb` plugin:
```
qt.qpa.plugin: Could not find the Qt platform plugin "xcb" in ""
This application failed to start because no Qt platform plugin could be initialized.
Reinstalling the application may fix this problem.
```
Finally, some minor style fixes were made for consistent indentation.
Addresses #45960
- our version is from 2015
- it doesn't build
- upstream project is dead, last release 2012, last commit Oct 2016.
- used by only 1 nixpkgs package: `boo`, marked broken since 2016.
Fixes the build for `python3Packages.trio' for the next ZHF iteration.
Please refer to the Hydra build for further reference: https://hydra.nixos.org/build/80617356
`python3Packages.sniffio` is needed for the build, otherwise the build
aborts with an error like this:
```
Could not find a version that satisfies the requirement sniffio (from trio==0.6.0) (from versions: )
No matching distribution found for sniffio (from trio==0.6.0)
```
See #45960
This aims to make the `weechat` package even more configurable. It
allows to specify scripts and commands using the `configure` function
inside a `weechat.override` expression.
The package can be configured like this:
```
with import <nixpkgs> { };
weechat.override {
plugins = { availablePlugins, ... }: {
plugins = builtins.attrValues availablePlugins;
init = ''
/set foo bar
/server add freenode chat.freenode.org
'';
scripts = [ "/path/to/script.py" ];
};
}
```
All commands are passed to `weechat --run-command "/set foo bar;/server ..."`.
The `plugins' attribute is not necessarily required anymore, if it's
sufficient to add `init' commands, the `plugins' will be
`builtins.attrValues availablePlugins' by default.
Additionally the result contains `weechat` and `weechat-headless`
(introduced in WeeChat 2.1) now.
Intuitively, one cares mainly about the host platform: Platforms differ
in meaningful ways but compilation is morally a pure process and
probably doesn't care, or those difference are already abstracted away.
@Dezgeg also empirically confirmed that > 95% of checks are indeed of
the host platform.
Yet these attributes in the old cross infrastructure were defined to be
the build platform, for expediency. And this was never before changed.
(For native builds build and host coincide, so it isn't clear what the
intention was.)
Fixing this doesn't affect native builds, since again they coincide. It
also doesn't affect cross builds of anything in Nixpkgs, as these are no
longer used. It could affect external cross builds, but I deem that
unlikely as anyone thinking about cross would use more explicit
attributes for clarity, all the more so because the rarity of inspecting
the build platform.
Introduce a `skawarePackages.buildPackage` function that contains the
common setup, removing a lot of duplication.
In particular, we require that the build directory has to be empty
after the `fixupPhase`, to make sure every relevant file is moved to
the outputs.
A next step would be to deduplicate the `configureFlags` attributes
and only require a `skawareInputs` field.
The github repository was downloaded instead of the pypi repository
for testing (needed `conftest.py`). Major work was done on the
underlying dependencies to make distributed work on python 2.7,
3+. Note that the test **do** take a significant amount of time (10-15
minutes).
- moved to `python-modules`
- compatible with 2.7, 3+
- all tests pass (previously tests were not run)
