JakeHillion/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/master' into gcc-cross

Browse Source
This commit is contained in:
Matthew Bauer 2020-08-26 16:10:02 -05:00
commit 872b23d214
15353 changed files with 298405 additions and 154788 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

96
.editorconfig
View File

@ -1,5 +1,5 @@
# EditorConfig configuration for nixpkgs
# http://EditorConfig.org
# https://EditorConfig.org
# Top-most EditorConfig file
root = true
@ -11,18 +11,96 @@ insert_final_newline = true
trim_trailing_whitespace = true
charset = utf-8
# Ignore diffs/patches
[*.{diff,patch}]
end_of_line = unset
insert_final_newline = unset
trim_trailing_whitespace = unset
# see https://nixos.org/nixpkgs/manual/#chap-conventions
# Match nix/ruby/docbook files, set indent to spaces with width of two
[*.{nix,rb,xml}]
# Match json/lockfiles/markdown/nix/perl/python/ruby/shell/docbook files, set indent to spaces
[*.{json,lock,md,nix,pl,pm,py,rb,sh,xml}]
indent_style = space
# Match docbook files, set indent width of one
[*.xml]
indent_size = 1
# Match json/lockfiles/markdown/nix/ruby files, set indent width of two
[*.{json,lock,md,nix,rb}]
indent_size = 2
# Match shell/python/perl scripts, set indent to spaces with width of four
[*.{sh,py,pl}]
indent_style = space
# Match perl/python/shell scripts, set indent width of four
[*.{pl,pm,py,sh}]
indent_size = 4
# Match diffs, avoid to trim trailing whitespace
[*.{diff,patch}]
trim_trailing_whitespace = false
# Match gemfiles, set indent to spaces with width of two
[Gemfile]
indent_size = 2
indent_style = space
# Disable file types or individual files
# some of these files may be auto-generated and/or require significant changes
[*.{c,h}]
insert_final_newline = unset
trim_trailing_whitespace = unset
[*.{key,ovpn}]
insert_final_newline = unset
end_of_line = unset
[*.lock]
indent_size = unset
[deps.nix]
insert_final_newline = unset
[gemset.nix]
insert_final_newline = unset
[node-packages.nix]
insert_final_newline = unset
[nixos/modules/services/networking/ircd-hybrid/*.{conf,in}]
trim_trailing_whitespace = unset
[nixos/tests/systemd-networkd-vrf.nix]
trim_trailing_whitespace = unset
[pkgs/applications/editors/emacs-modes/recipes-archive-melpa.json]
indent_size = unset
[pkgs/build-support/dotnetenv/Wrapper/**]
end_of_line = unset
insert_final_newline = unset
trim_trailing_whitespace = unset
[pkgs/build-support/upstream-updater/**]
trim_trailing_whitespace = unset
[pkgs/development/compilers/elm/registry.dat]
end_of_line = unset
insert_final_newline = unset
[pkgs/development/lisp-modules/quicklisp-to-nix.nix]
indent_size = unset
[pkgs/development/haskell-modules/hackage-packages.nix]
indent_style = unset
indent_size = unset
trim_trailing_whitespace = unset
[pkgs/development/mobile/androidenv/generated/{addons,packages}.nix]
trim_trailing_whitespace = unset
[pkgs/development/node-packages/node-packages.nix]
insert_final_newline = unset
[pkgs/servers/dict/wordnet_structures.py]
indent_size = unset
trim_trailing_whitespace = unset
[pkgs/top-level/perl-packages.nix]
indent_size = unset

45
.github/CODEOWNERS vendored
View File

@ -10,6 +10,12 @@
# This file
/.github/CODEOWNERS @edolstra
# GitHub actions
/.github/workflows @Mic92 @zowoq
# EditorConfig
/.editorconfig @Mic92 @zowoq
# Libraries
/lib @edolstra @nbp @infinisil
/lib/systems @nbp @ericson2314 @matthewbauer
@ -55,8 +61,12 @@
# NixOS integration test driver
/nixos/lib/test-driver @tfc
# New NixOS modules
/nixos/modules/module-list.nix @Infinisil
# Updaters
## update.nix
/maintainers/scripts/update.nix @jtojnar
/maintainers/scripts/update.py @jtojnar
## common-updater-scripts
/pkgs/common-updater/scripts/update-source-version @jtojnar
# Python-related code and docs
/maintainers/scripts/update-python-libraries @FRidh
@ -67,7 +77,7 @@
# Haskell
/pkgs/development/compilers/ghc @cdepillabout
/pkgs/development/haskell-modules @cdepillabout @infinisil
/pkgs/development/haskell-modules @cdepillabout
/pkgs/development/haskell-modules/default.nix @cdepillabout
/pkgs/development/haskell-modules/generic-builder.nix @cdepillabout
/pkgs/development/haskell-modules/hoogle.nix @cdepillabout
@ -82,8 +92,8 @@
/pkgs/development/r-modules @peti
# Ruby
/pkgs/development/interpreters/ruby @alyssais @zimbatm
/pkgs/development/ruby-modules @alyssais @zimbatm
/pkgs/development/interpreters/ruby @alyssais
/pkgs/development/ruby-modules @alyssais
# Rust
/pkgs/development/compilers/rust @Mic92 @LnL7
@ -177,7 +187,24 @@
/nixos/modules/services/monitoring/prometheus/exporters.xml @WilliButz
/nixos/tests/prometheus-exporters.nix @WilliButz
# PHP
/pkgs/development/interpreters/php @etu
/pkgs/top-level/php-packages.nix @etu
/pkgs/build-support/build-pecl.nix @etu
# PHP interpreter, packages, extensions, tests and documentation
/doc/languages-frameworks/php.section.md @NixOS/php
/nixos/tests/php @NixOS/php
/pkgs/build-support/build-pecl.nix @NixOS/php
/pkgs/development/interpreters/php @NixOS/php
/pkgs/top-level/php-packages.nix @NixOS/php
# Podman, CRI-O modules and related
/nixos/modules/virtualisation/containers.nix @NixOS/podman @zowoq
/nixos/modules/virtualisation/cri-o.nix @NixOS/podman @zowoq
/nixos/modules/virtualisation/podman.nix @NixOS/podman @zowoq
/nixos/tests/cri-o.nix @NixOS/podman @zowoq
/nixos/tests/podman.nix @NixOS/podman @zowoq
# Blockchains
/pkgs/applications/blockchains @mmahut
# Go
/pkgs/development/compilers/go @kalbasit @Mic92 @zowoq
/pkgs/development/go-modules @kalbasit @Mic92 @zowoq
/pkgs/development/go-packages @kalbasit @Mic92 @zowoq

11
.github/CONTRIBUTING.md vendored
View File

@ -50,12 +50,13 @@ For package version upgrades and such a one-line commit message is usually suffi
## Backporting changes
To [backport a change into a release branch](https://nixos.org/nixpkgs/manual/#submitting-changes-stable-release-branches):
Follow these steps to backport a change into a release branch in compliance with the [commit policy](https://nixos.org/nixpkgs/manual/#submitting-changes-stable-release-branches).
1. Take note of the commit in which the change was introduced into `master`.
2. Check out the target _release branch_, e.g. `release-19.09`. Do not use a _channel branch_ like `nixos-19.09` or `nixpkgs-19.09`.
3. Use `git cherry-pick -x <original commit>`.
4. Open your backport PR. Make sure to select the release branch (e.g. `release-19.09`) as the target branch of the PR, and link to the PR in which the original change was made to `master`.
1. Take note of the commits in which the change was introduced into `master` branch.
2. Check out the target _release branch_, e.g. `release-20.03`. Do not use a _channel branch_ like `nixos-20.03` or `nixpkgs-20.03`.
3. Create a branch for your change, e.g. `git checkout -b backport`.
4. When the reason to backport is not obvious from the original commit message, use `git cherry-pick -xe <original commit>` and add a reason. Otherwise use `git cherry-pick -x <original commit>`. That's fine for minor version updates that only include security and bug fixes, commits that fixes an otherwise broken package or similar.
5. Push to GitHub and open a backport pull request. Make sure to select the release branch (e.g. `release-20.03`) as the target branch of the pull request, and link to the pull request in which the original change was comitted to `master`. The pull request title should be the commit title with the release version as prefix, e.g. `[20.03]`.
## Reviewing contributions

6
.github/ISSUE_TEMPLATE/bug_report.md vendored
View File

@ -25,6 +25,12 @@ If applicable, add screenshots to help explain your problem.
**Additional context**
Add any other context about the problem here.
**Notify maintainers**
<!--
Please @ people who are in the `meta.maintainers` list of the offending package or module.
If in doubt, check `git blame` for whoever last touched something.
-->
**Metadata**
Please run `nix-shell -p nix-info --run "nix-info -m"` and paste the result.

13
.github/PULL_REQUEST_TEMPLATE.md vendored
View File

@ -1,4 +1,13 @@
<!-- Nixpkgs has a lot of new incoming Pull Requests, but not enough people to review this constant stream. Even if you aren't a committer, we would appreciate reviews of other PRs, especially simple ones like package updates. Just testing the relevant package/service and leaving a comment saying what you tested, how you tested it and whether it worked would be great. List of open PRs: <https://github.com/NixOS/nixpkgs/pulls>, for more about reviewing contributions: <https://hydra.nixos.org/job/nixpkgs/trunk/manual/latest/download/1/nixpkgs/manual.html#chap-reviewing-contributions>. Reviewing isn't mandatory, but it would help out a lot and reduce the average time-to-merge for all of us. Thanks a lot if you do! -->
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://hydra.nixos.org/job/nixpkgs/trunk/manual/latest/download/1/nixpkgs/manual.html#chap-reviewing-contributions
-->
###### Motivation for this change
@ -6,7 +15,7 @@
<!-- Please check what applies. Note that these are not hard requirements but merely serve as information for reviewers. -->
- [ ] Tested using sandboxing ([nix.useSandbox](http://nixos.org/nixos/manual/options.html#opt-nix.useSandbox) on NixOS, or option `sandbox` in [`nix.conf`](http://nixos.org/nix/manual/#sec-conf-file) on non-NixOS linux)
- [ ] Tested using sandboxing ([nix.useSandbox](https://nixos.org/nixos/manual/options.html#opt-nix.useSandbox) on NixOS, or option `sandbox` in [`nix.conf`](https://nixos.org/nix/manual/#sec-conf-file) on non-NixOS linux)
- Built on platform(s)
- [ ] NixOS
- [ ] macOS

43
.github/stale.yml vendored
View File

@ -1,32 +1,41 @@
# Configuration for probot-stale - https://github.com/probot/stale
# Number of days of inactivity before an issue becomes stale
daysUntilStale: 180
# Number of days of inactivity before a stale issue is closed
daysUntilClose: false
# Issues with these labels will never be considered stale
exemptLabels:
- 1.severity: security
- "1.severity: security"
# Label to use when marking an issue as stale
staleLabel: 2.status: stale
staleLabel: "2.status: stale"
# Comment to post when marking an issue as stale. Set to `false` to disable
markComment: >
Thank you for your contributions.
pulls:
markComment: |
Hello, I'm a bot and I thank you in the name of the community for your contributions.
This has been automatically marked as stale because it has had no
activity for 180 days.
Nixpkgs is a busy repository, and unfortunately sometimes PRs get left behind for too long. Nevertheless, we'd like to help committers reach the PRs that are still important. This PR has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.
If this is still important to you, we ask that you leave a
comment below. Your comment can be as simple as "still important
to me". This lets people see that at least one person still cares
about this. Someone will have to do this at most twice a year if
there is no other activity.
If this is still important to you and you'd like to remove the stale label, we ask that you leave a comment. Your comment can be as simple as "still important to me". But there's a bit more you can do:
Here are suggestions that might help resolve this more quickly:
If you received an approval by an unprivileged maintainer and you are just waiting for a merge, you can @ mention someone with merge permissions and ask them to help. You might be able to find someone relevant by using [Git blame](https://git-scm.com/docs/git-blame) on the relevant files, or via [GitHub's web interface](https://docs.github.com/en/github/managing-files-in-a-repository/tracking-changes-in-a-file). You can see if someone's a member of the [nixpkgs-committers](https://github.com/orgs/NixOS/teams/nixpkgs-committers) team, by hovering with the mouse over their username on the web interface, or by searching them directly on [the list](https://github.com/orgs/NixOS/teams/nixpkgs-committers).
1. Search for maintainers and people that previously touched the
related code and @ mention them in a comment.
2. Ask on the [NixOS Discourse](https://discourse.nixos.org/).
3. Ask on the [#nixos channel](irc://irc.freenode.net/#nixos) on
[irc.freenode.net](https://freenode.net).
If your PR wasn't reviewed at all, it might help to find someone who's perhaps a user of the package or module you are changing, or alternatively, ask once more for a review by the maintainer of the package/module this is about. If you don't know any, you can use [Git blame](https://git-scm.com/docs/git-blame) on the relevant files, or [GitHub's web interface](https://docs.github.com/en/github/managing-files-in-a-repository/tracking-changes-in-a-file) to find someone who touched the relevant files in the past.
If your PR has had reviews and nevertheless got stale, make sure you've responded to all of the reviewer's requests / questions. Usually when PR authors show responsibility and dedication, reviewers (privileged or not) show dedication as well. If you've pushed a change, it's possible the reviewer wasn't notified about your push via email, so you can always [officially request them for a review](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/requesting-a-pull-request-review), or just @ mention them and say you've addressed their comments.
Lastly, you can always ask for help at [our Discourse Forum](https://discourse.nixos.org/), or more specifically, [at this thread](https://discourse.nixos.org/t/prs-in-distress/3604) or at [#nixos' IRC channel](https://webchat.freenode.net/#nixos).
issues:
markComment: |
Hello, I'm a bot and I thank you in the name of the community for opening this issue.
To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.
The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it.
If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use [Git blame](https://git-scm.com/docs/git-blame) or [GitHub's web interface](https://docs.github.com/en/github/managing-files-in-a-repository/tracking-changes-in-a-file) on the relevant files to find them.
Lastly, you can always ask for help at [our Discourse Forum](https://discourse.nixos.org/) or at [#nixos' IRC channel](https://webchat.freenode.net/#nixos).
# Comment to post when closing a stale issue. Set to `false` to disable
closeComment: false

21
.github/workflows/pending-clear.yml vendored Normal file
View File

@ -0,0 +1,21 @@
name: "clear pending status"
on:
check_suite:
types: [ completed ]
jobs:
action:
runs-on: ubuntu-latest
steps:
- name: clear pending status
if: github.repository_owner == 'NixOS' && github.event.check_suite.app.name == 'OfBorg'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
curl \
-X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: token $GITHUB_TOKEN" \
-d '{"state": "success", "target_url": " ", "description": " ", "context": "Wait for ofborg"}' \
"https://api.github.com/repos/NixOS/nixpkgs/statuses/${{ github.event.check_suite.head_sha }}"

20
.github/workflows/pending-set.yml vendored Normal file
View File

@ -0,0 +1,20 @@
name: "set pending status"
on:
pull_request_target:
jobs:
action:
runs-on: ubuntu-latest
steps:
- name: set pending status
if: github.repository_owner == 'NixOS'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
curl \
-X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: token $GITHUB_TOKEN" \
-d '{"state": "failure", "target_url": " ", "description": "This failed status will be cleared when ofborg finishes eval.", "context": "Wait for ofborg"}' \
"https://api.github.com/repos/NixOS/nixpkgs/statuses/${{ github.event.pull_request.head.sha }}"

5
.gitignore vendored
View File

@ -11,7 +11,12 @@ result-*
.version-suffix
.DS_Store
.mypy_cache
__pycache__
/pkgs/development/libraries/qt-5/*/tmp/
/pkgs/desktops/kde-5/*/tmp/
/pkgs/development/mobile/androidenv/xml/*
# generated by pkgs/common-updater/update-script.nix
update-git-commits.txt

5
README.md
View File

@ -34,6 +34,7 @@ the main ones:
* [Nix](https://github.com/NixOS/nix) - the purely functional package manager
* [NixOps](https://github.com/NixOS/nixops) - the tool to remotely deploy NixOS machines
* [nixos-hardware](https://github.com/NixOS/nixos-hardware) - NixOS profiles to optimize settings for different hardware
* [Nix RFCs](https://github.com/NixOS/rfcs) - the formal process for making substantial changes to the community
* [NixOS homepage](https://github.com/NixOS/nixos-homepage) - the [NixOS.org](https://nixos.org) website
* [hydra](https://github.com/NixOS/hydra) - our continuous integration system
@ -45,9 +46,9 @@ Nixpkgs and NixOS are built and tested by our continuous integration
system, [Hydra](https://hydra.nixos.org/).
* [Continuous package builds for unstable/master](https://hydra.nixos.org/jobset/nixos/trunk-combined)
* [Continuous package builds for the NixOS 19.09 release](https://hydra.nixos.org/jobset/nixos/release-19.09)
* [Continuous package builds for the NixOS 20.03 release](https://hydra.nixos.org/jobset/nixos/release-20.03)
* [Tests for unstable/master](https://hydra.nixos.org/job/nixos/trunk-combined/tested#tabs-constituents)
* [Tests for the NixOS 19.09 release](https://hydra.nixos.org/job/nixos/release-19.09/tested#tabs-constituents)
* [Tests for the NixOS 20.03 release](https://hydra.nixos.org/job/nixos/release-20.03/tested#tabs-constituents)
Artifacts successfully built with Hydra are published to cache at
https://cache.nixos.org/. When successful build and test criteria are

2
doc/Makefile
View File

@ -1,4 +1,4 @@
MD_TARGETS=$(addsuffix .xml, $(basename $(wildcard ./*.md ./**/*.md)))
MD_TARGETS=$(addsuffix .xml, $(basename $(shell find . -type f -regex '.*\.md$$')))
.PHONY: all
all: validate format out/html/index.html out/epub/manual.epub

2
doc/builders/images/appimagetools.xml
View File

@ -63,7 +63,7 @@ type2.AppImage: ELF 64-bit LSB executable, x86-64, version 1 (SYSV) (Lepton 3.x)
appimageTools.wrapType2 { # or wrapType1
name = "patchwork"; <co xml:id='ex-appimageTools-wrapping-1' />
src = fetchurl { <co xml:id='ex-appimageTools-wrapping-2' />
url = https://github.com/ssbc/patchwork/releases/download/v3.11.4/Patchwork-3.11.4-linux-x86_64.AppImage;
url = "https://github.com/ssbc/patchwork/releases/download/v3.11.4/Patchwork-3.11.4-linux-x86_64.AppImage";
sha256 = "1blsprpkvm0ws9b96gb36f0rbf8f5jgmw4x6dsb1kswr4ysf591s";
};
extraPkgs = pkgs: with pkgs; [ ]; <co xml:id='ex-appimageTools-wrapping-3' />

23
doc/builders/images/dockertools.xml
View File

@ -166,7 +166,7 @@ hello latest de2bf4786de6 About a minute ago 25.2MB
<title>buildLayeredImage</title>
<para>
Create a Docker image with many of the store paths being on their own layer to improve sharing between images.
Create a Docker image with many of the store paths being on their own layer to improve sharing between images. The image is realized into the Nix store as a gzipped tarball. Depending on the intended usage, many users might prefer to use <function>streamLayeredImage</function> instead, which this function uses internally.
</para>
<variablelist>
@ -327,6 +327,27 @@ pkgs.dockerTools.buildLayeredImage {
</section>
</section>
<section xml:id="ssec-pkgs-dockerTools-streamLayeredImage">
<title>streamLayeredImage</title>
<para>
Builds a script which, when run, will stream an uncompressed tarball of a Docker image to stdout. The arguments to this function are as for <function>buildLayeredImage</function>. This method of constructing an image does not realize the image into the Nix store, so it saves on IO and disk/cache space, particularly with large images.
</para>
<para>
The image produced by running the output script can be piped directly into <command>docker load</command>, to load it into the local docker daemon:
<screen><![CDATA[
$(nix-build) | docker load
]]></screen>
</para>
<para>
Alternatively, the image be piped via <command>gzip</command> into <command>skopeo</command>, e.g. to copy it into a registry:
<screen><![CDATA[
$(nix-build) | gzip --fast | skopeo copy docker-archive:/dev/stdin docker://some_docker_registry/myimage:tag
]]></screen>
</para>
</section>
<section xml:id="ssec-pkgs-dockerTools-fetchFromRegistry">
<title>pullImage</title>

94
doc/builders/packages/cataclysm-dda.section.md Normal file
View File

@ -0,0 +1,94 @@
# Cataclysm: Dark Days Ahead
## How to install Cataclysm DDA
To install the latest stable release of Cataclysm DDA to your profile, execute
`nix-env -f "<nixpkgs>" -iA cataclysm-dda`. For the curses build (build
without tiles), install `cataclysmDDA.stable.curses`. Note: `cataclysm-dda` is
an alias to `cataclysmDDA.stable.tiles`.
If you like access to a development build of your favorite git revision,
override `cataclysm-dda-git` (or `cataclysmDDA.git.curses` if you like curses
build):
```nix
cataclysm-dda-git.override {
version = "YYYY-MM-DD";
rev = "YOUR_FAVORITE_REVISION";
sha256 = "CHECKSUM_OF_THE_REVISION";
}
```
The sha256 checksum can be obtained by
```sh
nix-prefetch-url --unpack "https://github.com/CleverRaven/Cataclysm-DDA/archive/${YOUR_FAVORITE_REVISION}.tar.gz"
```
The default configuration directory is `~/.cataclysm-dda`. If you prefer
`$XDG_CONFIG_HOME/cataclysm-dda`, override the derivation:
```nix
cataclysm-dda.override {
useXdgDir = true;
}
```
## Customizing with mods
To install Cataclysm DDA with mods of your choice, you can use `withMods`
attribute:
```nix
cataclysm-dda.withMods (mods: with mods; [
tileset.UndeadPeople
])
```
All mods, soundpacks, and tilesets available in nixpkgs are found in
`cataclysmDDA.pkgs`.
Here is an example to modify existing mods and/or add more mods not available
in nixpkgs:
```nix
let
customMods = self: super: lib.recursiveUpdate super {
# Modify existing mod
tileset.UndeadPeople = super.tileset.UndeadPeople.overrideAttrs (old: {
# If you like to apply a patch to the tileset for example
patches = [ ./path/to/your.patch ];
});
# Add another mod
mod.Awesome = cataclysmDDA.buildMod {
modName = "Awesome";
version = "0.x";
src = fetchFromGitHub {
owner = "Someone";
repo = "AwesomeMod";
rev = "...";
sha256 = "...";
};
# Path to be installed in the unpacked source (default: ".")
modRoot = "contents/under/this/path/will/be/installed";
};
# Add another soundpack
soundpack.Fantastic = cataclysmDDA.buildSoundPack {
# ditto
};
# Add another tileset
tileset.SuperDuper = cataclysmDDA.buildTileSet {
# ditto
};
};
in
cataclysm-dda.withMods (mods: with mods.extend customMods; [
tileset.UndeadPeople
mod.Awesome
soundpack.Fantastic
tileset.SuperDuper
])
```

30
doc/builders/packages/citrix.xml
View File

@ -4,34 +4,36 @@
<title>Citrix Workspace</title>
<para>
<note>
<para>
Please note that the <literal>citrix_receiver</literal> package has been deprecated since its development was <link xlink:href="https://docs.citrix.com/en-us/citrix-workspace-app.html">discontinued by upstream</link> and has been replaced by <link xlink:href="https://www.citrix.com/products/workspace-app/">the citrix workspace app</link>.
</para>
</note>
<link xlink:href="https://www.citrix.com/products/receiver/">Citrix Receiver</link> and <link xlink:href="https://www.citrix.com/products/workspace-app/">Citrix Workspace App</link> are a remote desktop viewers which provide access to <link xlink:href="https://www.citrix.com/products/xenapp-xendesktop/">XenDesktop</link> installations.
The <link xlink:href="https://www.citrix.com/products/workspace-app/">Citrix Workspace App</link> is a remote desktop viewer which provides access to <link xlink:href="https://www.citrix.com/products/xenapp-xendesktop/">XenDesktop</link> installations.
</para>
<section xml:id="sec-citrix-base">
<title>Basic usage</title>
<para>
The tarball archive needs to be downloaded manually as the license agreements of the vendor for <link xlink:href="https://www.citrix.com/downloads/citrix-receiver/">Citrix Receiver</link> or <link xlink:href="https://www.citrix.de/downloads/workspace-app/linux/workspace-app-for-linux-latest.html">Citrix Workspace</link> need to be accepted first. Then run <command>nix-prefetch-url file://$PWD/linuxx64-$version.tar.gz</command>. With the archive available in the store the package can be built and installed with Nix.
The tarball archive needs to be downloaded manually as the license agreements of the vendor for <link xlink:href="https://www.citrix.de/downloads/workspace-app/linux/workspace-app-for-linux-latest.html">Citrix Workspace</link> needs to be accepted first. Then run <command>nix-prefetch-url file://$PWD/linuxx64-$version.tar.gz</command>. With the archive available in the store the package can be built and installed with Nix.
</para>
</section>
<warning>
<title>Caution with <command>nix-shell</command> installs</title>
<para>
It's recommended to install <literal>Citrix Receiver</literal> and/or <literal>Citrix Workspace</literal> using <literal>nix-env -i</literal> or globally to ensure that the <literal>.desktop</literal> files are installed properly into <literal>$XDG_CONFIG_DIRS</literal>. Otherwise it won't be possible to open <literal>.ica</literal> files automatically from the browser to start a Citrix connection.
</para>
</warning>
<section xml:id="sec-citrix-selfservice">
<title>Citrix Selfservice</title>
<para>
The <link xlink:href="https://support.citrix.com/article/CTX200337">selfservice</link> is an application managing Citrix desktops and applications. Please note that this feature only works with at least <package>citrix_workspace_20_06_0</package> and later versions.
</para>
<para>
In order to set this up, you first have to <link xlink:href="https://its.uiowa.edu/support/article/102186">download the <literal>.cr</literal> file from the Netscaler Gateway</link>. After that you can configure the <command>selfservice</command> like this:
<screen>
<prompt>$ </prompt>storebrowse -C ~/Downloads/receiverconfig.cr
<prompt>$ </prompt>selfservice
</screen>
</para>
</section>
<section xml:id="sec-citrix-custom-certs">
<title>Custom certificates</title>
<para>
The <literal>Citrix Workspace App</literal> in <literal>nixpkgs</literal> trust several certificates <link xlink:href="https://curl.haxx.se/docs/caextract.html">from the Mozilla database</link> by default. However several companies using Citrix might require their own corporate certificate. On distros with imperative packaging these certs can be stored easily in <link xlink:href="https://developer-docs.citrix.com/projects/receiver-for-linux-command-reference/en/13.7/"><literal>$ICAROOT</literal></link>, however this directory is a store path in <literal>nixpkgs</literal>. In order to work around this issue the package provides a simple mechanism to add custom certificates without rebuilding the entire package using <literal>symlinkJoin</literal>:
The <literal>Citrix Workspace App</literal> in <literal>nixpkgs</literal> trusts several certificates <link xlink:href="https://curl.haxx.se/docs/caextract.html">from the Mozilla database</link> by default. However several companies using Citrix might require their own corporate certificate. On distros with imperative packaging these certs can be stored easily in <link xlink:href="https://developer-docs.citrix.com/projects/receiver-for-linux-command-reference/en/13.7/"><literal>$ICAROOT</literal></link>, however this directory is a store path in <literal>nixpkgs</literal>. In order to work around this issue the package provides a simple mechanism to add custom certificates without rebuilding the entire package using <literal>symlinkJoin</literal>:
<programlisting>
<![CDATA[with import <nixpkgs> { config.allowUnfree = true; };
let extraCerts = [ ./custom-cert-1.pem ./custom-cert-2.pem /* ... */ ]; in

1
doc/builders/packages/index.xml
View File

@ -18,6 +18,7 @@
<xi:include href="opengl.xml" />
<xi:include href="shell-helpers.xml" />
<xi:include href="steam.xml" />
<xi:include href="cataclysm-dda.section.xml" />
<xi:include href="urxvt.xml" />
<xi:include href="weechat.xml" />
<xi:include href="xorg.xml" />

8
doc/builders/packages/steam.xml
View File

@ -45,13 +45,7 @@
<title>How to play</title>
<para>
For 64-bit systems it's important to have
<programlisting>hardware.opengl.driSupport32Bit = true;</programlisting>
in your <filename>/etc/nixos/configuration.nix</filename>. You'll also need
<programlisting>hardware.pulseaudio.support32Bit = true;</programlisting>
if you are using PulseAudio - this will enable 32bit ALSA apps integration. To use the Steam controller or other Steam supported controllers such as the DualShock 4 or Nintendo Switch Pro, you need to add
<programlisting>hardware.steam-hardware.enable = true;</programlisting>
to your configuration.
Use <programlisting>programs.steam.enable = true;</programlisting> if you want to add steam to systemPackages and also enable a few workarrounds aswell as Steam controller support or other Steam supported controllers such as the DualShock 4 or Nintendo Switch Pr.
</para>
</section>

2
doc/builders/packages/urxvt.xml
View File

@ -50,7 +50,7 @@ map (p: p.name) pkgs.rxvt-unicode.plugins
In addition to <literal>plugins</literal> the options
<literal>extraDeps</literal> and <literal>perlDeps</literal> can be used
to install extra packages.
<literal>extraDeps</literal> can be used, for example, to provide
<literal>extraDeps</literal> can be used, for example, to provide
<literal>xsel</literal> (a clipboard manager) to the clipboard plugin,
without installing it globally:
<programlisting>rxvt-unicode.override { configure = { availablePlugins, ... }: {

2
doc/contributing/quick-start.xml
View File

@ -111,7 +111,7 @@
</para>
<para>
The exact syntax and semantics of the Nix expression language, including the built-in function, are described in the Nix manual in the <link
xlink:href="http://hydra.nixos.org/job/nix/trunk/tarball/latest/download-by-type/doc/manual/#chap-writing-nix-expressions">chapter on writing Nix expressions</link>.
xlink:href="https://hydra.nixos.org/job/nix/trunk/tarball/latest/download-by-type/doc/manual/#chap-writing-nix-expressions">chapter on writing Nix expressions</link>.
</para>
</listitem>
<listitem>

48
doc/contributing/submitting-changes.xml
View File

@ -407,23 +407,47 @@ Additional information.
<section xml:id="submitting-changes-stable-release-branches">
<title>Stable release branches</title>
<itemizedlist>
<para>
For cherry-picking a commit to a stable release branch (<quote>backporting</quote>), use <literal>git cherry-pick -x &lt;original commit&gt;</literal> so that the original commit id is included in the commit.
</para>
<para>
Add a reason for the backport by using <literal>git cherry-pick -xe &lt;original commit&gt;</literal> instead when it is not obvious from the original commit message. It is not needed when it’s a minor version update that includes security and bug fixes but don’t add new features or when the commit fixes an otherwise broken package.
</para>
<para>
Here is an example of a cherry-picked commit message with good reason description:
</para>
<screen>
zfs: Keep trying root import until it works
Works around #11003.
(cherry picked from commit 98b213a11041af39b39473906b595290e2a4e2f9)
Reason: several people cannot boot with ZFS on NVMe
</screen>
<para>
Other examples of reasons are:
</para>
<itemizedlist spacing="compact">
<listitem>
<para>
If you're cherry-picking a commit to a stable release branch (“backporting”), always use <command>git cherry-pick -xe</command> and ensure the message contains a clear description about why this needs to be included in the stable branch.
Previously the build would fail due to, e.g., <literal>getaddrinfo</literal> not being defined
</para>
</listitem>
<listitem>
<para>
An example of a cherry-picked commit would look like this:
The previous download links were all broken
</para>
</listitem>
<listitem>
<para>
Crash when starting on some X11 systems
</para>
<screen>
nixos: Refactor the world.
The original commit message describing the reason why the world was torn apart.
(cherry picked from commit abcdef)
Reason: I just had a gut feeling that this would also be wanted by people from
the stone age.
</screen>
</listitem>
</itemizedlist>
</section>

106
doc/languages-frameworks/agda.section.md Normal file
View File

@ -0,0 +1,106 @@
---
title: Agda
author: Alex Rice (alexarice)
date: 2020-01-06
---
# Agda
## How to use Agda
Agda can be installed from `agda`:
```
$ nix-env -iA agda
```
To use agda with libraries, the `agda.withPackages` function can be used. This function either takes:
+ A list of packages,
+ or a function which returns a list of packages when given the `agdaPackages` attribute set,
+ or an attribute set containing a list of packages and a GHC derivation for compilation (see below).
For example, suppose we wanted a version of agda which has access to the standard library. This can be obtained with the expressions:
```
agda.withPackages [ agdaPackages.standard-library ]
```
or
```
agda.withPackages (p: [ p.standard-library ])
```
or can be called as in the [Compiling Agda](#compiling-agda) section.
If you want to use a library in your home directory (for instance if it is a development version) then typecheck it manually (using `agda.withPackages` if necessary) and then override the `src` attribute of the package to point to your local repository.
Agda will not by default use these libraries. To tell agda to use the library we have some options:
- Call `agda` with the library flag:
```
$ agda -l standard-library -i . MyFile.agda
```
- Write a `my-library.agda-lib` file for the project you are working on which may look like:
```
name: my-library
include: .
depend: standard-library
```
- Create the file `~/.agda/defaults` and add any libraries you want to use by default.
More information can be found in the [official Agda documentation on library management](https://agda.readthedocs.io/en/v2.6.1/tools/package-system.html).
## Compiling Agda
Agda modules can be compiled with the `--compile` flag. A version of `ghc` with `ieee` is made available to the Agda program via the `--with-compiler` flag.
This can be overridden by a different version of `ghc` as follows:
```
agda.withPackages {
pkgs = [ ... ];
ghc = haskell.compiler.ghcHEAD;
}
```
## Writing Agda packages
To write a nix derivation for an agda library, first check that the library has a `*.agda-lib` file.
A derivation can then be written using `agdaPackages.mkDerivation`. This has similar arguments to `stdenv.mkDerivation` with the following additions:
+ `everythingFile` can be used to specify the location of the `Everything.agda` file, defaulting to `./Everything.agda`. If this file does not exist then either it should be patched in or the `buildPhase` should be overridden (see below).
+ `libraryName` should be the name that appears in the `*.agda-lib` file, defaulting to `pname`.
+ `libraryFile` should be the file name of the `*.agda-lib` file, defaulting to `${libraryName}.agda-lib`.
### Building Agda packages
The default build phase for `agdaPackages.mkDerivation` simply runs `agda` on the `Everything.agda` file.
If something else is needed to build the package (e.g. `make`) then the `buildPhase` should be overridden.
Additionally, a `preBuild` or `configurePhase` can be used if there are steps that need to be done prior to checking the `Everything.agda` file.
`agda` and the Agda libraries contained in `buildInputs` are made available during the build phase.
### Installing Agda packages
The default install phase copies agda source files, agda interface files (`*.agdai`) and `*.agda-lib` files to the output directory.
This can be overridden.
By default, agda sources are files ending on `.agda`, or literate agda files ending on `.lagda`, `.lagda.tex`, `.lagda.org`, `.lagda.md`, `.lagda.rst`. The list of recognised agda source extensions can be extended by setting the `extraExtensions` config variable.
To add an agda package to `nixpkgs`, the derivation should be written to `pkgs/development/libraries/agda/${library-name}/` and an entry should be added to `pkgs/top-level/agda-packages.nix`. Here it is called in a scope with access to all other agda libraries, so the top line of the `default.nix` can look like:
```
{ mkDerivation, standard-library, fetchFromGitHub }:
```
and `mkDerivation` should be called instead of `agdaPackages.mkDerivation`. Here is an example skeleton derivation for iowa-stdlib:
```
mkDerivation {
version = "1.5.0";
pname = "iowa-stdlib";
src = ...
libraryFile = "";
libraryName = "IAL-1.3";
buildPhase = ''
patchShebangs find-deps.sh
make
'';
}
```
This library has a file called `.agda-lib`, and so we give an empty string to `libraryFile` as nothing precedes `.agda-lib` in the filename. This file contains `name: IAL-1.3`, and so we let `libraryName = "IAL-1.3"`. This library does not use an `Everything.agda` file and instead has a Makefile, so there is no need to set `everythingFile` and we set a custom `buildPhase`.
When writing an agda package it is essential to make sure that no `.agda-lib` file gets added to the store as a single file (for example by using `writeText`). This causes agda to think that the nix store is a agda library and it will attempt to write to it whenever it typechecks something. See [https://github.com/agda/agda/issues/4613](https://github.com/agda/agda/issues/4613).

6
doc/languages-frameworks/android.section.md
View File

@ -167,7 +167,7 @@ parameters that the SDK composition function (the function shown in the
previous section) supports.
This build function is particularly useful when it is desired to use
[Hydra](http://nixos.org/hydra): the Nix-based continuous integration solution
[Hydra](https://nixos.org/hydra): the Nix-based continuous integration solution
to build Android apps. An Android APK gets exposed as a build product and can be
installed on any Android device with a web browser by navigating to the build
result page.
@ -186,11 +186,13 @@ with import <nixpkgs> {};
androidenv.emulateApp {
name = "emulate-MyAndroidApp";
platformVersion = "28";
abiVersion = "x86_64"; # armeabi-v7a, mips, x86
abiVersion = "x86"; # armeabi-v7a, mips, x86_64
systemImageType = "google_apis_playstore";
}
```
Additional flags may be applied to the Android SDK's emulator through the runtime environment variable `$NIX_ANDROID_EMULATOR_FLAGS`.
It is also possible to specify an APK to deploy inside the emulator
and the package and activity names to launch it:

2
doc/languages-frameworks/beam.xml
View File

@ -68,7 +68,7 @@
<title>How to Install BEAM Packages</title>
<para>
BEAM builders are not registered at the top level, simply because they are not relevant to the vast majority of Nix users.
BEAM builders are not registered at the top level, simply because they are not relevant to the vast majority of Nix users.
To install any of those builders into your profile, refer to them by their attribute path <literal>beamPackages.rebar3</literal>:
</para>

17
doc/languages-frameworks/emscripten.section.md
View File

@ -21,11 +21,11 @@ Modes of use of `emscripten`:
* **Declarative usage**:
This mode is far more power full since this makes use of `nix` for dependency management of emscripten libraries and targets by using the `mkDerivation` which is implemented by `pkgs.emscriptenStdenv` and `pkgs.buildEmscriptenPackage`. The source for the packages is in `pkgs/top-level/emscripten-packages.nix` and the abstraction behind it in `pkgs/development/em-modules/generic/default.nix`.
* build and install all packages:
* `nix-env -iA emscriptenPackages`
* dev-shell for zlib implementation hacking:
* `nix-shell -A emscriptenPackages.zlib`
* build and install all packages:
* `nix-env -iA emscriptenPackages`
* dev-shell for zlib implementation hacking:
* `nix-shell -A emscriptenPackages.zlib`
## Imperative usage
@ -90,7 +90,7 @@ See the `zlib` example:
libz.so.${old.version} -I . -o example.js
echo "Using node to execute the test"
${pkgs.nodejs}/bin/node ./example.js
${pkgs.nodejs}/bin/node ./example.js
set +x
if [ $? -ne 0 ]; then
@ -112,7 +112,7 @@ See the `zlib` example:
### Usage 2: pkgs.buildEmscriptenPackage
This `xmlmirror` example features a emscriptenPackage which is defined completely from this context and no `pkgs.zlib.override` is used.
This `xmlmirror` example features a emscriptenPackage which is defined completely from this context and no `pkgs.zlib.override` is used.
xmlmirror = pkgs.buildEmscriptenPackage rec {
name = "xmlmirror";
@ -163,7 +163,7 @@ This `xmlmirror` example features a emscriptenPackage which is defined completel
checkPhase = ''
'';
};
};
### Declarative debugging
@ -182,4 +182,3 @@ Use `nix-shell -I nixpkgs=/some/dir/nixpkgs -A emscriptenPackages.libz` and from
Using this toolchain makes it easy to leverage `nix` from NixOS, MacOSX or even Windows (WSL+ubuntu+nix). This toolchain is reproducible, behaves like the rest of the packages from nixpkgs and contains a set of well working examples to learn and adapt from.
If in trouble, ask the maintainers.

17
doc/languages-frameworks/gnome.xml
View File

@ -40,6 +40,23 @@
</para>
</section>
<section xml:id="ssec-icon-theme-packaging">
<title>Packaging icon themes</title>
<para>
Icon themes may inherit from other icon themes. The inheritance is specified using the <literal>Inherits</literal> key in the <filename>index.theme</filename> file distributed with the icon theme. According to the <link xlink:href="https://specifications.freedesktop.org/icon-theme-spec/icon-theme-spec-latest.html">icon theme specification</link>, icons not provided by the theme are looked for in its parent icon themes. Therefore the parent themes should be installed as dependencies for a more complete experience regarding the icon sets used.
</para>
<para>
The package <package>hicolor-icon-theme</package> provides a setup hook which makes symbolic links for the parent themes into the directory <filename>share/icons</filename> of the current theme directory in the nix store, making sure they can be found at runtime. For that to work the packages providing parent icon themes should be listed as propagated build dependencies, together with <package>hicolor-icon-theme</package>.
</para>
<para>
Also make sure that <filename>icon-theme.cache</filename> is installed for each theme provided by the package, and set <code>dontDropIconThemeCache</code> to <code>true</code> so that the cache file is not removed by the <package>gtk3</package> setup hook.
</para>
</section>
<section xml:id="ssec-gnome-themes">
<title>GTK Themes</title>

40
doc/languages-frameworks/go.xml
View File

@ -36,10 +36,14 @@ pet = buildGoModule rec {
sha256 = "0m2fzpqxk7hrbxsgqplkg7h2p7gv6s1miymv3gvw0cz039skag0s";
};
modSha256 = "1879j77k96684wi554rkjxydrj8g3hpp0kvxz03sd8dmwr3lh83j"; <co xml:id='ex-buildGoModule-1' />
vendorSha256 = "1879j77k96684wi554rkjxydrj8g3hpp0kvxz03sd8dmwr3lh83j"; <co xml:id='ex-buildGoModule-1' />
subPackages = [ "." ]; <co xml:id='ex-buildGoModule-2' />
deleteVendor = true; <co xml:id='ex-buildGoModule-3' />
runVend = true; <co xml:id='ex-buildGoModule-4' />
meta = with lib; {
description = "Simple command-line snippet manager, written in Go";
homepage = "https://github.com/knqyf263/pet";
@ -56,7 +60,7 @@ pet = buildGoModule rec {
<calloutlist>
<callout arearefs='ex-buildGoModule-1'>
<para>
<varname>modSha256</varname> is the hash of the output of the intermediate fetcher derivation.
<varname>vendorSha256</varname> is the hash of the output of the intermediate fetcher derivation.
</para>
</callout>
<callout arearefs='ex-buildGoModule-2'>
@ -64,16 +68,26 @@ pet = buildGoModule rec {
<varname>subPackages</varname> limits the builder from building child packages that have not been listed. If <varname>subPackages</varname> is not specified, all child packages will be built.
</para>
</callout>
<callout arearefs='ex-buildGoModule-3'>
<para>
<varname>deleteVendor</varname> removes the pre-existing vendor directory and fetches the dependencies. This should only be used if the dependencies included in the vendor folder are broken or incomplete.
</para>
</callout>
<callout arearefs='ex-buildGoModule-4'>
<para>
<varname>runVend</varname> runs the vend command to generate the vendor directory. This is useful if your code depends on c code and go mod tidy does not include the needed sources to build.
</para>
</callout>
</calloutlist>
</para>
<para>
<varname>modSha256</varname> can also take <varname>null</varname> as an input.
<varname>vendorSha256</varname> can also take <varname>null</varname> as an input.
When `null` is used as a value, the derivation won't be a
fixed-output derivation but disable the build sandbox instead. This can be useful outside
of nixpkgs where re-generating the modSha256 on each mod.sum changes is cumbersome,
but will fail to build by Hydra, as builds with a disabled sandbox are discouraged.
When `null` is used as a value, rather than fetching the dependencies
and vendoring them, we use the vendoring included within the source repo.
If you'd like to not have to update this field on dependency changes,
run `go mod vendor` in your source repo and set 'vendorSha256 = null;'
</para>
</section>
@ -191,18 +205,6 @@ deis = buildGoPackage rec {
To extract dependency information from a Go package in automated way use <link xlink:href="https://github.com/kamilchm/go2nix">go2nix</link>. It can produce complete derivation and <varname>goDeps</varname> file for Go programs.
</para>
<para>
<varname>buildGoPackage</varname> produces <xref linkend='chap-multiple-output' xrefstyle="select: title" /> where <varname>bin</varname> includes program binaries. You can test build a Go binary as follows:
<screen>
<prompt>$ </prompt>nix-build -A deis.bin
</screen>
or build all outputs with:
<screen>
<prompt>$ </prompt>nix-build -A deis.all
</screen>
<varname>bin</varname> output will be installed by default with <varname>nix-env -i</varname> or <varname>systemPackages</varname>.
</para>
<para>
You may use Go packages installed into the active Nix profiles by adding the following to your ~/.bashrc:
<screen>

12
doc/languages-frameworks/haskell.section.md
View File

@ -101,17 +101,15 @@ to compile your Haskell packages with any GHC version you please. The following
command displays the complete list of available compilers:
```
$ nix-env -f "<nixpkgs>" -qaP -A haskell.compiler
haskell.compiler.ghc8101 ghc-8.10.0.20191210
haskell.compiler.integer-simple.ghc8101 ghc-8.10.0.20191210
haskell.compiler.ghcHEAD ghc-8.10.20191119
haskell.compiler.integer-simple.ghcHEAD ghc-8.10.20191119
haskell.compiler.ghc8101 ghc-8.10.1
haskell.compiler.integer-simple.ghc8101 ghc-8.10.1
haskell.compiler.ghcHEAD ghc-8.11.20200505
haskell.compiler.integer-simple.ghcHEAD ghc-8.11.20200505
haskell.compiler.ghc822Binary ghc-8.2.2-binary
haskell.compiler.ghc844 ghc-8.4.4
haskell.compiler.ghc863Binary ghc-8.6.3-binary
haskell.compiler.ghc865 ghc-8.6.5
haskell.compiler.integer-simple.ghc865 ghc-8.6.5
haskell.compiler.ghc881 ghc-8.8.1
haskell.compiler.integer-simple.ghc881 ghc-8.8.1
haskell.compiler.ghc882 ghc-8.8.2
haskell.compiler.integer-simple.ghc882 ghc-8.8.2
haskell.compiler.ghc883 ghc-8.8.3
@ -369,7 +367,7 @@ automatically select the right version of GHC and other build tools to build,
test and execute apps in an existing project downloaded from somewhere on the
Internet. Pass the `--nix` flag to any `stack` command to do so, e.g.
```shell
git clone --recursive https://github.com/yesodweb/wai
git clone --recurse-submodules https://github.com/yesodweb/wai.git
cd wai
stack --nix build
```

4
doc/languages-frameworks/index.xml
View File

@ -5,6 +5,7 @@
<para>
The <link linkend="chap-stdenv">standard build environment</link> makes it easy to build typical Autotools-based packages with very little code. Any other kind of package can be accomodated by overriding the appropriate phases of <literal>stdenv</literal>. However, there are specialised functions in Nixpkgs to easily build packages for other programming languages, such as Perl or Haskell. These are described in this chapter.
</para>
<xi:include href="agda.section.xml" />
<xi:include href="android.section.xml" />
<xi:include href="beam.xml" />
<xi:include href="bower.xml" />
@ -17,10 +18,11 @@
<xi:include href="idris.section.xml" />
<xi:include href="ios.section.xml" />
<xi:include href="java.xml" />
<xi:include href="lua.xml" />
<xi:include href="lua.section.xml" />
<xi:include href="node.section.xml" />
<xi:include href="ocaml.xml" />
<xi:include href="perl.xml" />
<xi:include href="php.section.xml" />
<xi:include href="python.section.xml" />
<xi:include href="qt.xml" />
<xi:include href="r.section.xml" />

2
doc/languages-frameworks/ios.section.md
View File

@ -18,7 +18,7 @@ The primary objective of this project is to use the Nix expression language to
specify how iOS apps can be built from source code, and to automatically spawn
iOS simulator instances for testing.
This component also makes it possible to use [Hydra](http://nixos.org/hydra),
This component also makes it possible to use [Hydra](https://nixos.org/hydra),
the Nix-based continuous integration server to regularly build iOS apps and to
do wireless ad-hoc installations of enterprise IPAs on iOS devices through
Hydra.

252
doc/languages-frameworks/lua.section.md Normal file
View File

@ -0,0 +1,252 @@
---
title: Lua
author: Matthieu Coudron
date: 2019-02-05
---
# User's Guide to Lua Infrastructure
## Using Lua
### Overview of Lua
Several versions of the Lua interpreter are available: luajit, lua 5.1, 5.2, 5.3.
The attribute `lua` refers to the default interpreter, it is also possible to refer to specific versions, e.g. `lua5_2` refers to Lua 5.2.
Lua libraries are in separate sets, with one set per interpreter version.
The interpreters have several common attributes. One of these attributes is
`pkgs`, which is a package set of Lua libraries for this specific
interpreter. E.g., the `busted` package corresponding to the default interpreter
is `lua.pkgs.busted`, and the lua 5.2 version is `lua5_2.pkgs.busted`.
The main package set contains aliases to these package sets, e.g.
`luaPackages` refers to `lua5_1.pkgs` and `lua52Packages` to
`lua5_2.pkgs`.
### Installing Lua and packages
#### Lua environment defined in separate `.nix` file
Create a file, e.g. `build.nix`, with the following expression
```nix
with import <nixpkgs> {};
lua5_2.withPackages (ps: with ps; [ busted luafilesystem ])
```
and install it in your profile with
```shell
nix-env -if build.nix
```
Now you can use the Lua interpreter, as well as the extra packages (`busted`,
`luafilesystem`) that you added to the environment.
#### Lua environment defined in `~/.config/nixpkgs/config.nix`
If you prefer to, you could also add the environment as a package override to the Nixpkgs set, e.g.
using `config.nix`,
```nix
{ # ...
packageOverrides = pkgs: with pkgs; {
myLuaEnv = lua5_2.withPackages (ps: with ps; [ busted luafilesystem ]);
};
}
```
and install it in your profile with
```shell
nix-env -iA nixpkgs.myLuaEnv
```
The environment is is installed by referring to the attribute, and considering
the `nixpkgs` channel was used.
#### Lua environment defined in `/etc/nixos/configuration.nix`
For the sake of completeness, here's another example how to install the environment system-wide.
```nix
{ # ...
environment.systemPackages = with pkgs; [
(lua.withPackages(ps: with ps; [ busted luafilesystem ]))
];
}
```
### How to override a Lua package using overlays?
Use the following overlay template:
```nix
final: prev:
{
lua = prev.lua.override {
packageOverrides = luaself: luaprev: {
luarocks-nix = luaprev.luarocks-nix.overrideAttrs(oa: {
pname = "luarocks-nix";
src = /home/my_luarocks/repository;
});
};
luaPackages = lua.pkgs;
}
```
### Temporary Lua environment with `nix-shell`
There are two methods for loading a shell with Lua packages. The first and recommended method
is to create an environment with `lua.buildEnv` or `lua.withPackages` and load that. E.g.
```sh
$ nix-shell -p 'lua.withPackages(ps: with ps; [ busted luafilesystem ])'
```
opens a shell from which you can launch the interpreter
```sh
[nix-shell:~] lua
```
The other method, which is not recommended, does not create an environment and requires you to list the packages directly,
```sh
$ nix-shell -p lua.pkgs.busted lua.pkgs.luafilesystem
```
Again, it is possible to launch the interpreter from the shell.
The Lua interpreter has the attribute `pkgs` which contains all Lua libraries for that specific interpreter.
## Developing with Lua
Now that you know how to get a working Lua environment with Nix, it is time
to go forward and start actually developing with Lua. There are two ways to
package lua software, either it is on luarocks and most of it can be taken care
of by the luarocks2nix converter or the packaging has to be done manually.
Let's present the luarocks way first and the manual one in a second time.
### Packaging a library on luarocks
[Luarocks.org](www.luarocks.org) is the main repository of lua packages.
The site proposes two types of packages, the rockspec and the src.rock
(equivalent of a [rockspec](https://github.com/luarocks/luarocks/wiki/Rockspec-format) but with the source).
These packages can have different build types such as `cmake`, `builtin` etc .
Luarocks-based packages are generated in pkgs/development/lua-modules/generated-packages.nix from
the whitelist maintainers/scripts/luarocks-packages.csv and updated by running maintainers/scripts/update-luarocks-packages.
[luarocks2nix](https://github.com/nix-community/luarocks) is a tool capable of generating nix derivations from both rockspec and src.rock (and favors the src.rock).
The automation only goes so far though and some packages need to be customized.
These customizations go in `pkgs/development/lua-modules/overrides.nix`.
For instance if the rockspec defines `external_dependencies`, these need to be manually added in in its rockspec file then it won't work.
You can try converting luarocks packages to nix packages with the command `nix-shell -p luarocks-nix` and then `luarocks nix PKG_NAME`.
Nix rely on luarocks to install lua packages, basically it runs:
`luarocks make --deps-mode=none --tree $out`
#### Packaging a library manually
You can develop your package as you usually would, just don't forget to wrap it
within a `toLuaModule` call, for instance
```nix
mynewlib = toLuaModule ( stdenv.mkDerivation { ... });
```
There is also the `buildLuaPackage` function that can be used when lua modules
are not packaged for luarocks. You can see a few examples at `pkgs/top-level/lua-packages.nix`.
## Lua Reference
### Lua interpreters
Versions 5.1, 5.2 and 5.3 of the lua interpreter are available as
respectively `lua5_1`, `lua5_2` and `lua5_3`. Luajit is available too.
The Nix expressions for the interpreters can be found in `pkgs/development/interpreters/lua-5`.
#### Attributes on lua interpreters packages
Each interpreter has the following attributes:
- `interpreter`. Alias for `${pkgs.lua}/bin/lua`.
- `buildEnv`. Function to build lua interpreter environments with extra packages bundled together. See section *lua.buildEnv function* for usage and documentation.
- `withPackages`. Simpler interface to `buildEnv`.
- `pkgs`. Set of Lua packages for that specific interpreter. The package set can be modified by overriding the interpreter and passing `packageOverrides`.
#### `buildLuarocksPackage` function
The `buildLuarocksPackage` function is implemented in `pkgs/development/interpreters/lua-5/build-lua-package.nix`
The following is an example:
```nix
luaposix = buildLuarocksPackage {
pname = "luaposix";
version = "34.0.4-1";
src = fetchurl {
url = "https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/luaposix-34.0.4-1.src.rock";
sha256 = "0yrm5cn2iyd0zjd4liyj27srphvy0gjrjx572swar6zqr4dwjqp2";
};
disabled = (luaOlder "5.1") || (luaAtLeast "5.4");
propagatedBuildInputs = [ bit32 lua std_normalize ];
meta = with stdenv.lib; {
homepage = "https://github.com/luaposix/luaposix/";
description = "Lua bindings for POSIX";
maintainers = with maintainers; [ vyp lblasc ];
license.fullName = "MIT/X11";
};
};
```
The `buildLuarocksPackage` delegates most tasks to luarocks:
* it adds `luarocks` as an unpacker for `src.rock` files (zip files really).
* configurePhase` writes a temporary luarocks configuration file which location
is exported via the environment variable `LUAROCKS_CONFIG`.
* the `buildPhase` does nothing.
* `installPhase` calls `luarocks make --deps-mode=none --tree $out` to build and
install the package
* In the `postFixup` phase, the `wrapLuaPrograms` bash function is called to
wrap all programs in the `$out/bin/*` directory to include `$PATH`
environment variable and add dependent libraries to script's `LUA_PATH` and
`LUA_CPATH`.
By default `meta.platforms` is set to the same value as the interpreter unless overridden otherwise.
#### `buildLuaApplication` function
The `buildLuaApplication` function is practically the same as `buildLuaPackage`.
The difference is that `buildLuaPackage` by default prefixes the names of the packages with the version of the interpreter.
Because with an application we're not interested in multiple version the prefix is dropped.
#### lua.withPackages function
The `lua.withPackages` takes a function as an argument that is passed the set of lua packages and returns the list of packages to be included in the environment.
Using the `withPackages` function, the previous example for the luafilesystem environment can be written like this:
```nix
with import <nixpkgs> {};
lua.withPackages (ps: [ps.luafilesystem])
```
`withPackages` passes the correct package set for the specific interpreter version as an argument to the function. In the above example, `ps` equals `luaPackages`.
But you can also easily switch to using `lua5_2`:
```nix
with import <nixpkgs> {};
lua5_2.withPackages (ps: [ps.lua])
```
Now, `ps` is set to `lua52Packages`, matching the version of the interpreter.
### Possible Todos
* export/use version specific variables such as `LUA_PATH_5_2`/`LUAROCKS_CONFIG_5_2`
* let luarocks check for dependencies via exporting the different rocktrees in temporary config
### Lua Contributing guidelines
Following rules should be respected:
* Make sure libraries build for all Lua interpreters.
* Commit names of Lua libraries should reflect that they are Lua libraries, so write for example `luaPackages.luafilesystem: 1.11 -> 1.12`.

36
doc/languages-frameworks/lua.xml
View File

@ -1,36 +0,0 @@
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xml:id="sec-language-lua">
<title>Lua</title>
<para>
Lua packages are built by the <varname>buildLuaPackage</varname> function. This function is implemented in <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/lua-modules/generic/default.nix"> <filename>pkgs/development/lua-modules/generic/default.nix</filename></link> and works similarly to <varname>buildPerlPackage</varname>. (See <xref linkend="sec-language-perl"/> for details.)
</para>
<para>
Lua packages are defined in <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/lua-packages.nix"><filename>pkgs/top-level/lua-packages.nix</filename></link>. Most of them are simple. For example:
<programlisting>
fileSystem = buildLuaPackage {
name = "filesystem-1.6.2";
src = fetchurl {
url = "https://github.com/keplerproject/luafilesystem/archive/v1_6_2.tar.gz";
sha256 = "1n8qdwa20ypbrny99vhkmx8q04zd2jjycdb5196xdhgvqzk10abz";
};
meta = {
homepage = "https://github.com/keplerproject/luafilesystem";
hydraPlatforms = stdenv.lib.platforms.linux;
maintainers = with maintainers; [ flosse ];
};
};
</programlisting>
</para>
<para>
Though, more complicated package should be placed in a seperate file in <link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/lua-modules"><filename>pkgs/development/lua-modules</filename></link>.
</para>
<para>
Lua packages accept additional parameter <varname>disabled</varname>, which defines the condition of disabling package from luaPackages. For example, if package has <varname>disabled</varname> assigned to <literal>lua.luaversion != "5.1"</literal>, it will not be included in any luaPackages except lua51Packages, making it only be built for lua 5.1.
</para>
</section>

19
doc/languages-frameworks/node.section.md
View File

@ -12,10 +12,9 @@ When it is desired to use NPM libraries in a development project, use the
`node2nix` generator directly on the `package.json` configuration file of the
project.
The package set also provides support for multiple Node.js versions. The policy
is that a new package should be added to the collection for the latest stable LTS
release (which is currently 10.x), unless there is an explicit reason to support
a different release.
The package set provides support for the official stable Node.js versions.
The latest stable LTS release in `nodePackages`, as well as the latest stable
Current release in `nodePackages_latest`.
If your package uses native addons, you need to examine what kind of native
build system it uses. Here are some examples:
@ -26,7 +25,7 @@ build system it uses. Here are some examples:
After you have identified the correct system, you need to override your package
expression while adding in build system as a build input. For example, `dat`
requires `node-gyp-build`, so we override its expression in `default-v10.nix`:
requires `node-gyp-build`, so we override its expression in `default.nix`:
```nix
dat = nodePackages.dat.override (oldAttrs: {
@ -36,14 +35,14 @@ dat = nodePackages.dat.override (oldAttrs: {
To add a package from NPM to nixpkgs:
1. Modify `pkgs/development/node-packages/node-packages-v10.json` to add, update
or remove package entries. (Or `pkgs/development/node-packages/node-packages-v8.json`
for packages depending on Node.js 8.x)
1. Modify `pkgs/development/node-packages/node-packages.json` to add, update
or remove package entries to have it included in `nodePackages` and
`nodePackages_latest`.
2. Run the script: `(cd pkgs/development/node-packages && ./generate.sh)`.
3. Build your new package to test your changes:
`cd /path/to/nixpkgs && nix-build -A nodePackages.<new-or-updated-package>`.
To build against a specific Node.js version (e.g. 10.x):
`nix-build -A nodePackages_10_x.<new-or-updated-package>`
To build against the latest stable Current Node.js version (e.g. 14.x):
`nix-build -A nodePackages_latest.<new-or-updated-package>`
4. Add and commit all modified and generated files.
For more information about the generation process, consult the

318
doc/languages-frameworks/perl.xml
View File

@ -3,159 +3,193 @@
xml:id="sec-language-perl">
<title>Perl</title>
<para>
Nixpkgs provides a function <varname>buildPerlPackage</varname>, a generic package builder function for any Perl package that has a standard <varname>Makefile.PL</varname>. It’s implemented in <link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/perl-modules/generic"><filename>pkgs/development/perl-modules/generic</filename></link>.
</para>
<para>
Perl packages from CPAN are defined in <link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/perl-packages.nix"><filename>pkgs/top-level/perl-packages.nix</filename></link>, rather than <filename>pkgs/all-packages.nix</filename>. Most Perl packages are so straight-forward to build that they are defined here directly, rather than having a separate function for each package called from <filename>perl-packages.nix</filename>. However, more complicated packages should be put in a separate file, typically in <filename>pkgs/development/perl-modules</filename>. Here is an example of the former:
<programlisting>
ClassC3 = buildPerlPackage rec {
name = "Class-C3-0.21";
src = fetchurl {
url = "mirror://cpan/authors/id/F/FL/FLORA/${name}.tar.gz";
sha256 = "1bl8z095y4js66pwxnm7s853pi9czala4sqc743fdlnk27kq94gz";
};
};
</programlisting>
Note the use of <literal>mirror://cpan/</literal>, and the <literal>${name}</literal> in the URL definition to ensure that the name attribute is consistent with the source that we’re actually downloading. Perl packages are made available in <filename>all-packages.nix</filename> through the variable <varname>perlPackages</varname>. For instance, if you have a package that needs <varname>ClassC3</varname>, you would typically write
<programlisting>
foo = import ../path/to/foo.nix {
inherit stdenv fetchurl ...;
inherit (perlPackages) ClassC3;
};
</programlisting>
in <filename>all-packages.nix</filename>. You can test building a Perl package as follows:
<screen>
<prompt>$ </prompt>nix-build -A perlPackages.ClassC3
</screen>
<varname>buildPerlPackage</varname> adds <literal>perl-</literal> to the start of the name attribute, so the package above is actually called <literal>perl-Class-C3-0.21</literal>. So to install it, you can say:
<screen>
<prompt>$ </prompt>nix-env -i perl-Class-C3
</screen>
(Of course you can also install using the attribute name: <literal>nix-env -i -A perlPackages.ClassC3</literal>.)
</para>
<para>
So what does <varname>buildPerlPackage</varname> do? It does the following:
<orderedlist>
<listitem>
<para>
In the configure phase, it calls <literal>perl Makefile.PL</literal> to generate a Makefile. You can set the variable <varname>makeMakerFlags</varname> to pass flags to <filename>Makefile.PL</filename>
</para>
</listitem>
<listitem>
<para>
It adds the contents of the <envar>PERL5LIB</envar> environment variable to <literal>#! .../bin/perl</literal> line of Perl scripts as <literal>-I<replaceable>dir</replaceable></literal> flags. This ensures that a script can find its dependencies. (This can cause this shebang line to become too long for Darwin to handle; see the note below.)
</para>
</listitem>
<listitem>
<para>
In the fixup phase, it writes the propagated build inputs (<varname>propagatedBuildInputs</varname>) to the file <filename>$out/nix-support/propagated-user-env-packages</filename>. <command>nix-env</command> recursively installs all packages listed in this file when you install a package that has it. This ensures that a Perl package can find its dependencies.
</para>
</listitem>
</orderedlist>
</para>
<para>
<varname>buildPerlPackage</varname> is built on top of <varname>stdenv</varname>, so everything can be customised in the usual way. For instance, the <literal>BerkeleyDB</literal> module has a <varname>preConfigure</varname> hook to generate a configuration file used by <filename>Makefile.PL</filename>:
<programlisting>
{ buildPerlPackage, fetchurl, db }:
buildPerlPackage rec {
name = "BerkeleyDB-0.36";
src = fetchurl {
url = "mirror://cpan/authors/id/P/PM/PMQS/${name}.tar.gz";
sha256 = "07xf50riarb60l1h6m2dqmql8q5dij619712fsgw7ach04d8g3z1";
};
preConfigure = ''
echo "LIB = ${db.out}/lib" > config.in
echo "INCLUDE = ${db.dev}/include" >> config.in
'';
}
</programlisting>
</para>
<para>
Dependencies on other Perl packages can be specified in the <varname>buildInputs</varname> and <varname>propagatedBuildInputs</varname> attributes. If something is exclusively a build-time dependency, use <varname>buildInputs</varname>; if it’s (also) a runtime dependency, use <varname>propagatedBuildInputs</varname>. For instance, this builds a Perl module that has runtime dependencies on a bunch of other modules:
<programlisting>
ClassC3Componentised = buildPerlPackage rec {
name = "Class-C3-Componentised-1.0004";
src = fetchurl {
url = "mirror://cpan/authors/id/A/AS/ASH/${name}.tar.gz";
sha256 = "0xql73jkcdbq4q9m0b0rnca6nrlvf5hyzy8is0crdk65bynvs8q1";
};
propagatedBuildInputs = [
ClassC3 ClassInspector TestException MROCompat
];
};
</programlisting>
</para>
<para>
On Darwin, if a script has too many <literal>-I<replaceable>dir</replaceable></literal> flags in its first line (its “shebang line”), it will not run. This can be worked around by calling the <literal>shortenPerlShebang</literal> function from the <literal>postInstall</literal> phase:
<programlisting>
{ stdenv, buildPerlPackage, fetchurl, shortenPerlShebang }:
ImageExifTool = buildPerlPackage {
pname = "Image-ExifTool";
version = "11.50";
src = fetchurl {
url = "https://www.sno.phy.queensu.ca/~phil/exiftool/Image-ExifTool-11.50.tar.gz";
sha256 = "0d8v48y94z8maxkmw1rv7v9m0jg2dc8xbp581njb6yhr7abwqdv3";
};
buildInputs = stdenv.lib.optional stdenv.isDarwin shortenPerlShebang;
postInstall = stdenv.lib.optional stdenv.isDarwin ''
shortenPerlShebang $out/bin/exiftool
'';
};
</programlisting>
This will remove the <literal>-I</literal> flags from the shebang line, rewrite them in the <literal>use lib</literal> form, and put them on the next line instead. This function can be given any number of Perl scripts as arguments; it will modify them in-place.
</para>
<section xml:id="ssec-generation-from-CPAN">
<title>Generation from CPAN</title>
<section xml:id="ssec-perl-running">
<title>Running perl programs on the shell</title>
<para>
Nix expressions for Perl packages can be generated (almost) automatically from CPAN. This is done by the program <command>nix-generate-from-cpan</command>, which can be installed as follows:
When executing a Perl script, it is possible you get an error such as <literal>./myscript.pl: bad interpreter: /usr/bin/perl: no such file or directory</literal>. This happens when the script expects Perl to be installed at <filename>/usr/bin/perl</filename>, which is not the case when using Perl from nixpkgs. You can fix the script by changing the first line to:
<programlisting>
#!/usr/bin/env perl
</programlisting>
to take the Perl installation from the <literal>PATH</literal> environment variable, or invoke Perl directly with:
<screen>
<prompt>$ </prompt>perl ./myscript.pl
</screen>
</para>
<screen>
<prompt>$ </prompt>nix-env -i nix-generate-from-cpan
</screen>
<para>
When the script is using a Perl library that is not installed globally, you might get an error such as <literal>Can't locate DB_File.pm in @INC (you may need to install the DB_File module)</literal>. In that case, you can use <command>nix-shell</command> to start an ad-hoc shell with that library installed, for instance:
<screen>
<prompt>$ </prompt>nix-shell -p perl perlPackages.DBFile --run ./myscript.pl
</screen>
</para>
<para>
This program takes a Perl module name, looks it up on CPAN, fetches and unpacks the corresponding package, and prints a Nix expression on standard output. For example:
<screen>
<prompt>$ </prompt>nix-generate-from-cpan XML::Simple
XMLSimple = buildPerlPackage rec {
name = "XML-Simple-2.22";
src = fetchurl {
url = "mirror://cpan/authors/id/G/GR/GRANTM/${name}.tar.gz";
sha256 = "b9450ef22ea9644ae5d6ada086dc4300fa105be050a2030ebd4efd28c198eb49";
};
propagatedBuildInputs = [ XMLNamespaceSupport XMLSAX XMLSAXExpat ];
meta = {
description = "An API for simple XML files";
license = with stdenv.lib.licenses; [ artistic1 gpl1Plus ];
};
};
</screen>
The output can be pasted into <filename>pkgs/top-level/perl-packages.nix</filename> or wherever else you need it.
If you are always using the script in places where <command>nix-shell</command> is available, you can embed the <command>nix-shell</command> invocation in the shebang like this:
<programlisting>
#!/usr/bin/env nix-shell
#! nix-shell -i perl -p perl perlPackages.DBFile
</programlisting>
</para>
</section>
<section xml:id="ssec-perl-cross-compilation">
<title>Cross-compiling modules</title>
<section xml:id="ssec-perl-packaging">
<title>Packaging Perl programs</title>
<para>
Nixpkgs has experimental support for cross-compiling Perl modules. In many cases, it will just work out of the box, even for modules with native extensions. Sometimes, however, the Makefile.PL for a module may (indirectly) import a native module. In that case, you will need to make a stub for that module that will satisfy the Makefile.PL and install it into <filename>lib/perl5/site_perl/cross_perl/${perl.version}</filename>. See the <varname>postInstall</varname> for <varname>DBI</varname> for an example.
Nixpkgs provides a function <varname>buildPerlPackage</varname>, a generic package builder function for any Perl package that has a standard <varname>Makefile.PL</varname>. It’s implemented in <link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/perl-modules/generic"><filename>pkgs/development/perl-modules/generic</filename></link>.
</para>
<para>
Perl packages from CPAN are defined in <link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/perl-packages.nix"><filename>pkgs/top-level/perl-packages.nix</filename></link>, rather than <filename>pkgs/all-packages.nix</filename>. Most Perl packages are so straight-forward to build that they are defined here directly, rather than having a separate function for each package called from <filename>perl-packages.nix</filename>. However, more complicated packages should be put in a separate file, typically in <filename>pkgs/development/perl-modules</filename>. Here is an example of the former:
<programlisting>
ClassC3 = buildPerlPackage rec {
name = "Class-C3-0.21";
src = fetchurl {
url = "mirror://cpan/authors/id/F/FL/FLORA/${name}.tar.gz";
sha256 = "1bl8z095y4js66pwxnm7s853pi9czala4sqc743fdlnk27kq94gz";
};
};
</programlisting>
Note the use of <literal>mirror://cpan/</literal>, and the <literal>${name}</literal> in the URL definition to ensure that the name attribute is consistent with the source that we’re actually downloading. Perl packages are made available in <filename>all-packages.nix</filename> through the variable <varname>perlPackages</varname>. For instance, if you have a package that needs <varname>ClassC3</varname>, you would typically write
<programlisting>
foo = import ../path/to/foo.nix {
inherit stdenv fetchurl ...;
inherit (perlPackages) ClassC3;
};
</programlisting>
in <filename>all-packages.nix</filename>. You can test building a Perl package as follows:
<screen>
<prompt>$ </prompt>nix-build -A perlPackages.ClassC3
</screen>
<varname>buildPerlPackage</varname> adds <literal>perl-</literal> to the start of the name attribute, so the package above is actually called <literal>perl-Class-C3-0.21</literal>. So to install it, you can say:
<screen>
<prompt>$ </prompt>nix-env -i perl-Class-C3
</screen>
(Of course you can also install using the attribute name: <literal>nix-env -i -A perlPackages.ClassC3</literal>.)
</para>
<para>
So what does <varname>buildPerlPackage</varname> do? It does the following:
<orderedlist>
<listitem>
<para>
In the configure phase, it calls <literal>perl Makefile.PL</literal> to generate a Makefile. You can set the variable <varname>makeMakerFlags</varname> to pass flags to <filename>Makefile.PL</filename>
</para>
</listitem>
<listitem>
<para>
It adds the contents of the <envar>PERL5LIB</envar> environment variable to <literal>#! .../bin/perl</literal> line of Perl scripts as <literal>-I<replaceable>dir</replaceable></literal> flags. This ensures that a script can find its dependencies. (This can cause this shebang line to become too long for Darwin to handle; see the note below.)
</para>
</listitem>
<listitem>
<para>
In the fixup phase, it writes the propagated build inputs (<varname>propagatedBuildInputs</varname>) to the file <filename>$out/nix-support/propagated-user-env-packages</filename>. <command>nix-env</command> recursively installs all packages listed in this file when you install a package that has it. This ensures that a Perl package can find its dependencies.
</para>
</listitem>
</orderedlist>
</para>
<para>
<varname>buildPerlPackage</varname> is built on top of <varname>stdenv</varname>, so everything can be customised in the usual way. For instance, the <literal>BerkeleyDB</literal> module has a <varname>preConfigure</varname> hook to generate a configuration file used by <filename>Makefile.PL</filename>:
<programlisting>
{ buildPerlPackage, fetchurl, db }:
buildPerlPackage rec {
name = "BerkeleyDB-0.36";
src = fetchurl {
url = "mirror://cpan/authors/id/P/PM/PMQS/${name}.tar.gz";
sha256 = "07xf50riarb60l1h6m2dqmql8q5dij619712fsgw7ach04d8g3z1";
};
preConfigure = ''
echo "LIB = ${db.out}/lib" > config.in
echo "INCLUDE = ${db.dev}/include" >> config.in
'';
}
</programlisting>
</para>
<para>
Dependencies on other Perl packages can be specified in the <varname>buildInputs</varname> and <varname>propagatedBuildInputs</varname> attributes. If something is exclusively a build-time dependency, use <varname>buildInputs</varname>; if it’s (also) a runtime dependency, use <varname>propagatedBuildInputs</varname>. For instance, this builds a Perl module that has runtime dependencies on a bunch of other modules:
<programlisting>
ClassC3Componentised = buildPerlPackage rec {
name = "Class-C3-Componentised-1.0004";
src = fetchurl {
url = "mirror://cpan/authors/id/A/AS/ASH/${name}.tar.gz";
sha256 = "0xql73jkcdbq4q9m0b0rnca6nrlvf5hyzy8is0crdk65bynvs8q1";
};
propagatedBuildInputs = [
ClassC3 ClassInspector TestException MROCompat
];
};
</programlisting>
</para>
<para>
On Darwin, if a script has too many <literal>-I<replaceable>dir</replaceable></literal> flags in its first line (its “shebang line”), it will not run. This can be worked around by calling the <literal>shortenPerlShebang</literal> function from the <literal>postInstall</literal> phase:
<programlisting>
{ stdenv, buildPerlPackage, fetchurl, shortenPerlShebang }:
ImageExifTool = buildPerlPackage {
pname = "Image-ExifTool";
version = "11.50";
src = fetchurl {
url = "https://www.sno.phy.queensu.ca/~phil/exiftool/Image-ExifTool-11.50.tar.gz";
sha256 = "0d8v48y94z8maxkmw1rv7v9m0jg2dc8xbp581njb6yhr7abwqdv3";
};
buildInputs = stdenv.lib.optional stdenv.isDarwin shortenPerlShebang;
postInstall = stdenv.lib.optional stdenv.isDarwin ''
shortenPerlShebang $out/bin/exiftool
'';
};
</programlisting>
This will remove the <literal>-I</literal> flags from the shebang line, rewrite them in the <literal>use lib</literal> form, and put them on the next line instead. This function can be given any number of Perl scripts as arguments; it will modify them in-place.
</para>
<section xml:id="ssec-generation-from-CPAN">
<title>Generation from CPAN</title>
<para>
Nix expressions for Perl packages can be generated (almost) automatically from CPAN. This is done by the program <command>nix-generate-from-cpan</command>, which can be installed as follows:
</para>
<screen>
<prompt>$ </prompt>nix-env -i nix-generate-from-cpan
</screen>
<para>
This program takes a Perl module name, looks it up on CPAN, fetches and unpacks the corresponding package, and prints a Nix expression on standard output. For example:
<screen>
<prompt>$ </prompt>nix-generate-from-cpan XML::Simple
XMLSimple = buildPerlPackage rec {
name = "XML-Simple-2.22";
src = fetchurl {
url = "mirror://cpan/authors/id/G/GR/GRANTM/${name}.tar.gz";
sha256 = "b9450ef22ea9644ae5d6ada086dc4300fa105be050a2030ebd4efd28c198eb49";
};
propagatedBuildInputs = [ XMLNamespaceSupport XMLSAX XMLSAXExpat ];
meta = {
description = "An API for simple XML files";
license = with stdenv.lib.licenses; [ artistic1 gpl1Plus ];
};
};
</screen>
The output can be pasted into <filename>pkgs/top-level/perl-packages.nix</filename> or wherever else you need it.
</para>
</section>
<section xml:id="ssec-perl-cross-compilation">
<title>Cross-compiling modules</title>
<para>
Nixpkgs has experimental support for cross-compiling Perl modules. In many cases, it will just work out of the box, even for modules with native extensions. Sometimes, however, the Makefile.PL for a module may (indirectly) import a native module. In that case, you will need to make a stub for that module that will satisfy the Makefile.PL and install it into <filename>lib/perl5/site_perl/cross_perl/${perl.version}</filename>. See the <varname>postInstall</varname> for <varname>DBI</varname> for an example.
</para>
</section>
</section>
</section>

137
doc/languages-frameworks/php.section.md Normal file
View File

@ -0,0 +1,137 @@
# PHP {#sec-php}
## User Guide {#ssec-php-user-guide}
### Overview {#ssec-php-user-guide-overview}
Several versions of PHP are available on Nix, each of which having a
wide variety of extensions and libraries available.
The different versions of PHP that nixpkgs provides are located under
attributes named based on major and minor version number; e.g.,
`php74` is PHP 7.4.
Only versions of PHP that are supported by upstream for the entirety
of a given NixOS release will be included in that release of
NixOS. See [PHP Supported
Versions](https://www.php.net/supported-versions.php).
The attribute `php` refers to the version of PHP considered most
stable and thoroughly tested in nixpkgs for any given release of
NixOS - not necessarily the latest major release from upstream.
All available PHP attributes are wrappers around their respective
binary PHP package and provide commonly used extensions this way. The
real PHP 7.4 package, i.e. the unwrapped one, is available as
`php74.unwrapped`; see the next section for more details.
Interactive tools built on PHP are put in `php.packages`; composer is
for example available at `php.packages.composer`.
Most extensions that come with PHP, as well as some popular
third-party ones, are available in `php.extensions`; for example, the
opcache extension shipped with PHP is available at
`php.extensions.opcache` and the third-party ImageMagick extension at
`php.extensions.imagick`.
### Installing PHP with extensions {#ssec-php-user-guide-installing-with-extensions}
A PHP package with specific extensions enabled can be built using
`php.withExtensions`. This is a function which accepts an anonymous
function as its only argument; the function should accept two named
parameters: `enabled` - a list of currently enabled extensions and
`all` - the set of all extensions, and return a list of wanted
extensions. For example, a PHP package with all default extensions and
ImageMagick enabled:
```nix
php.withExtensions ({ enabled, all }:
enabled ++ [ all.imagick ])
```
To exclude some, but not all, of the default extensions, you can
filter the `enabled` list like this:
```nix
php.withExtensions ({ enabled, all }:
(lib.filter (e: e != php.extensions.opcache) enabled)
++ [ all.imagick ])
```
To build your list of extensions from the ground up, you can simply
ignore `enabled`:
```nix
php.withExtensions ({ all, ... }: with all; [ imagick opcache ])
```
`php.withExtensions` provides extensions by wrapping a minimal php
base package, providing a `php.ini` file listing all extensions to be
loaded. You can access this package through the `php.unwrapped`
attribute; useful if you, for example, need access to the `dev`
output. The generated `php.ini` file can be accessed through the
`php.phpIni` attribute.
If you want a PHP build with extra configuration in the `php.ini`
file, you can use `php.buildEnv`. This function takes two named and
optional parameters: `extensions` and `extraConfig`. `extensions`
takes an extension specification equivalent to that of
`php.withExtensions`, `extraConfig` a string of additional `php.ini`
configuration parameters. For example, a PHP package with the opcache
and ImageMagick extensions enabled, and `memory_limit` set to `256M`:
```nix
php.buildEnv {
extensions = { all, ... }: with all; [ imagick opcache ];
extraConfig = "memory_limit=256M";
}
```
#### Example setup for `phpfpm` {#ssec-php-user-guide-installing-with-extensions-phpfpm}
You can use the previous examples in a `phpfpm` pool called `foo` as
follows:
```nix
let
myPhp = php.withExtensions ({ all, ... }: with all; [ imagick opcache ]);
in {
services.phpfpm.pools."foo".phpPackage = myPhp;
};
```
```nix
let
myPhp = php.buildEnv {
extensions = { all, ... }: with all; [ imagick opcache ];
extraConfig = "memory_limit=256M";
};
in {
services.phpfpm.pools."foo".phpPackage = myPhp;
};
```
#### Example usage with `nix-shell` {#ssec-php-user-guide-installing-with-extensions-nix-shell}
This brings up a temporary environment that contains a PHP interpreter
with the extensions `imagick` and `opcache` enabled:
```sh
nix-shell -p 'php.withExtensions ({ all, ... }: with all; [ imagick opcache ])'
```
### Installing PHP packages with extensions {#ssec-php-user-guide-installing-packages-with-extensions}
All interactive tools use the PHP package you get them from, so all
packages at `php.packages.*` use the `php` package with its default
extensions. Sometimes this default set of extensions isn't enough and
you may want to extend it. A common case of this is the `composer`
package: a project may depend on certain extensions and `composer`
won't work with that project unless those extensions are loaded.
Example of building `composer` with additional extensions:
```nix
(php.withExtensions ({ all, enabled }:
enabled ++ (with all; [ imagick redis ]))
).packages.composer
```

637
doc/languages-frameworks/python.section.md
View File

@ -9,7 +9,7 @@
Several versions of the Python interpreter are available on Nix, as well as a
high amount of packages. The attribute `python` refers to the default
interpreter, which is currently CPython 2.7. It is also possible to refer to
specific versions, e.g. `python35` refers to CPython 3.5, and `pypy` refers to
specific versions, e.g. `python38` refers to CPython 3.8, and `pypy` refers to
the default PyPy interpreter.
Python is used a lot, and in different ways. This affects also how it is
@ -25,10 +25,10 @@ however, are in separate sets, with one set per interpreter version.
The interpreters have several common attributes. One of these attributes is
`pkgs`, which is a package set of Python libraries for this specific
interpreter. E.g., the `toolz` package corresponding to the default interpreter
is `python.pkgs.toolz`, and the CPython 3.5 version is `python35.pkgs.toolz`.
is `python.pkgs.toolz`, and the CPython 3.8 version is `python38.pkgs.toolz`.
The main package set contains aliases to these package sets, e.g.
`pythonPackages` refers to `python.pkgs` and `python35Packages` to
`python35.pkgs`.
`pythonPackages` refers to `python.pkgs` and `python38Packages` to
`python38.pkgs`.
#### Installing Python and packages
@ -36,121 +36,191 @@ The Nix and NixOS manuals explain how packages are generally installed. In the
case of Python and Nix, it is important to make a distinction between whether the
package is considered an application or a library.
Applications on Nix are typically installed into your user
profile imperatively using `nix-env -i`, and on NixOS declaratively by adding the
package name to `environment.systemPackages` in `/etc/nixos/configuration.nix`.
Dependencies such as libraries are automatically installed and should not be
installed explicitly.
Applications on Nix are typically installed into your user profile imperatively
using `nix-env -i`, and on NixOS declaratively by adding the package name to
`environment.systemPackages` in `/etc/nixos/configuration.nix`. Dependencies
such as libraries are automatically installed and should not be installed
explicitly.
The same goes for Python applications and libraries. Python applications can be
installed in your profile. But Python libraries you would like to use for
development cannot be installed, at least not individually, because they won't
be able to find each other resulting in import errors. Instead, it is possible
to create an environment with `python.buildEnv` or `python.withPackages` where
the interpreter and other executables are able to find each other and all of the
modules.
The same goes for Python applications. Python applications can be installed in
your profile, and will be wrapped to find their exact library dependencies,
without impacting other applications or polluting your user environment.
In the following examples we create an environment with Python 3.5, `numpy` and
`toolz`. As you may imagine, there is one limitation here, and that's that
you can install only one environment at a time. You will notice the complaints
about collisions when you try to install a second environment.
But Python libraries you would like to use for development cannot be installed,
at least not individually, because they won't be able to find each other
resulting in import errors. Instead, it is possible to create an environment
with `python.buildEnv` or `python.withPackages` where the interpreter and other
executables are wrapped to be able to find each other and all of the modules.
##### Environment defined in separate `.nix` file
In the following examples we will start by creating a simple, ad-hoc environment
with a nix-shell that has `numpy` and `toolz` in Python 3.8; then we will create
a re-usable environment in a single-file Python script; then we will create a
full Python environment for development with this same environment.
Create a file, e.g. `build.nix`, with the following expression
```nix
with import <nixpkgs> {};
Philosphically, this should be familiar to users who are used to a `venv` style
of development: individual projects create their own Python environments without
impacting the global environment or each other.
python35.withPackages (ps: with ps; [ numpy toolz ])
```
and install it in your profile with
```shell
nix-env -if build.nix
```
Now you can use the Python interpreter, as well as the extra packages (`numpy`,
`toolz`) that you added to the environment.
#### Ad-hoc temporary Python environment with `nix-shell`
##### Environment defined in `~/.config/nixpkgs/config.nix`
The simplest way to start playing with the way nix wraps and sets up Python
environments is with `nix-shell` at the cmdline. These environments create a
temporary shell session with a Python and a *precise* list of packages (plus
their runtime dependencies), with no other Python packages in the Python
interpreter's scope.
If you prefer you could also add the environment as a package override to the
Nixpkgs set, e.g. using `config.nix`,
```nix
{ # ...
packageOverrides = pkgs: with pkgs; {
myEnv = python35.withPackages (ps: with ps; [ numpy toolz ]);
};
}
```
and install it in your profile with
```shell
nix-env -iA nixpkgs.myEnv
```
The environment is is installed by referring to the attribute, and considering
the `nixpkgs` channel was used.
##### Environment defined in `/etc/nixos/configuration.nix`
For the sake of completeness, here's another example how to install the
environment system-wide.
```nix
{ # ...
environment.systemPackages = with pkgs; [
(python35.withPackages(ps: with ps; [ numpy toolz ]))
];
}
```
#### Temporary Python environment with `nix-shell`
The examples in the previous section showed how to install a Python environment
into a profile. For development you may need to use multiple environments.
`nix-shell` gives the possibility to temporarily load another environment, akin
to `virtualenv`.
There are two methods for loading a shell with Python packages. The first and
recommended method is to create an environment with `python.buildEnv` or
`python.withPackages` and load that. E.g.
To create a Python 3.8 session with `numpy` and `toolz` available, run:
```sh
$ nix-shell -p 'python35.withPackages(ps: with ps; [ numpy toolz ])'
$ nix-shell -p 'python38.withPackages(ps: with ps; [ numpy toolz ])'
```
opens a shell from which you can launch the interpreter
By default `nix-shell` will start a `bash` session with this interpreter in our
`PATH`, so if we then run:
```
[nix-shell:~/src/nixpkgs]$ python3
Python 3.8.1 (default, Dec 18 2019, 19:06:26)
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy; import toolz
```
Note that no other modules are in scope, even if they were imperatively
installed into our user environment as a dependency of a Python application:
```
>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'requests'
```
We can add as many additional modules onto the `nix-shell` as we need, and we
will still get 1 wrapped Python interpreter. We can start the interpreter
directly like so:
```sh
[nix-shell:~] python3
$ nix-shell -p 'python38.withPackages(ps: with ps; [ numpy toolz requests ])' --run python3
these derivations will be built:
/nix/store/xbdsrqrsfa1yva5s7pzsra8k08gxlbz1-python3-3.8.1-env.drv
building '/nix/store/xbdsrqrsfa1yva5s7pzsra8k08gxlbz1-python3-3.8.1-env.drv'...
created 277 symlinks in user environment
Python 3.8.1 (default, Dec 18 2019, 19:06:26)
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>>
```
The other method, which is not recommended, does not create an environment and
requires you to list the packages directly,
Notice that this time it built a new Python environment, which now includes
`requests`. Building an environment just creates wrapper scripts that expose the
selected dependencies to the interpreter while re-using the actual modules. This
means if any other env has installed `requests` or `numpy` in a different
context, we don't need to recompile them -- we just recompile the wrapper script
that sets up an interpreter pointing to them. This matters much more for "big"
modules like `pytorch` or `tensorflow`.
Module names usually match their names on [pypi.org](https://pypi.org/), but
you can use the [Nixpkgs search website](https://nixos.org/nixos/packages.html)
to find them as well (along with non-python packages).
At this point we can create throwaway experimental Python environments with
arbitrary dependencies. This is a good way to get a feel for how the Python
interpreter and dependencies work in Nix and NixOS, but to do some actual
development, we'll want to make it a bit more persistent.
##### Running Python scripts and using `nix-shell` as shebang
Sometimes, we have a script whose header looks like this:
```python
#!/usr/bin/env python3
import numpy as np
a = np.array([1,2])
b = np.array([3,4])
print(f"The dot product of {a} and {b} is: {np.dot(a, b)}")
```
Executing this script requires a `python3` that has `numpy`. Using what we learned
in the previous section, we could startup a shell and just run it like so:
```
nix-shell -p 'python38.withPackages(ps: with ps; [ numpy ])' --run 'python3 foo.py'
The dot product of [1 2] and [3 4] is: 11
```
But if we maintain the script ourselves, and if there are more dependencies, it
may be nice to encode those dependencies in source to make the script re-usable
without that bit of knowledge. That can be done by using `nix-shell` as a
[shebang](https://en.wikipedia.org/wiki/Shebang_(Unix), like so:
```python
#!/usr/bin/env nix-shell
#!nix-shell -i python3 -p "python3.withPackages(ps: [ ps.numpy ])"
import numpy as np
a = np.array([1,2])
b = np.array([3,4])
print(f"The dot product of {a} and {b} is: {np.dot(a, b)}")
```
Then we simply execute it, without requiring any environment setup at all!
```sh
$ nix-shell -p python35.pkgs.numpy python35.pkgs.toolz
$ ./foo.py
The dot product of [1 2] and [3 4] is: 11
```
Again, it is possible to launch the interpreter from the shell. The Python
interpreter has the attribute `pkgs` which contains all Python libraries for
that specific interpreter.
If the dependencies are not available on the host where `foo.py` is executed, it
will build or download them from a Nix binary cache prior to starting up, prior
that it is executed on a machine with a multi-user nix installation.
This provides a way to ship a self bootstrapping Python script, akin to a
statically linked binary, where it can be run on any machine (provided nix is
installed) without having to assume that `numpy` is installed globally on the
system.
By default it is pulling the import checkout of Nixpkgs itself from our nix
channel, which is nice as it cache aligns with our other package builds, but we
can make it fully reproducible by pinning the `nixpkgs` import:
```python
#!/usr/bin/env nix-shell
#!nix-shell -i python3 -p "python3.withPackages(ps: [ ps.numpy ])"
#!nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/d373d80b1207d52621961b16aa4a3438e4f98167.tar.gz
import numpy as np
a = np.array([1,2])
b = np.array([3,4])
print(f"The dot product of {a} and {b} is: {np.dot(a, b)}")
```
This will execute with the exact same versions of Python 3.8, numpy, and system
dependencies a year from now as it does today, because it will always use
exactly git commit `d373d80b1207d52621961b16aa4a3438e4f98167` of Nixpkgs for all
of the package versions.
This is also a great way to ensure the script executes identically on different
servers.
##### Load environment from `.nix` expression
As explained in the Nix manual, `nix-shell` can also load an
expression from a `.nix` file. Say we want to have Python 3.5, `numpy`
and `toolz`, like before, in an environment. Consider a `shell.nix` file
with
We've now seen how to create an ad-hoc temporary shell session, and how to
create a single script with Python dependencies, but in the course of normal
development we're usually working in an entire package repository.
As explained in the Nix manual, `nix-shell` can also load an expression from a
`.nix` file. Say we want to have Python 3.8, `numpy` and `toolz`, like before,
in an environment. We can add a `shell.nix` file describing our dependencies:
```nix
with import <nixpkgs> {};
(python35.withPackages (ps: [ps.numpy ps.toolz])).env
(python38.withPackages (ps: [ps.numpy ps.toolz])).env
```
Executing `nix-shell` gives you again a Nix shell from which you can run Python.
And then at the command line, just typing `nix-shell` produces the same
environment as before. In a normal project, we'll likely have many more
dependencies; this can provide a way for developers to share the environments
with each other and with CI builders.
What's happening here?
@ -158,9 +228,9 @@ What's happening here?
imports the `<nixpkgs>` function, `{}` calls it and the `with` statement
brings all attributes of `nixpkgs` in the local scope. These attributes form
the main package set.
2. Then we create a Python 3.5 environment with the `withPackages` function.
2. Then we create a Python 3.8 environment with the `withPackages` function, as before.
3. The `withPackages` function expects us to provide a function as an argument
that takes the set of all python packages and returns a list of packages to
that takes the set of all Python packages and returns a list of packages to
include in the environment. Here, we select the packages `numpy` and `toolz`
from the package set.
@ -168,59 +238,106 @@ To combine this with `mkShell` you can:
```nix
with import <nixpkgs> {};
let
pythonEnv = python35.withPackages (ps: [
pythonEnv = python38.withPackages (ps: [
ps.numpy
ps.toolz
]);
in mkShell {
buildInputs = [
pythonEnv
hello
black
mypy
libffi
openssl
];
}
```
##### Execute command with `--run`
A convenient option with `nix-shell` is the `--run`
option, with which you can execute a command in the `nix-shell`. We can
e.g. directly open a Python shell
This will create a unified environment that has not just our Python interpreter
and its Python dependencies, but also tools like `black` or `mypy` and libraries
like `libffi` the `openssl` in scope. This is generic and can span any number of
tools or languages across the Nixpkgs ecosystem.
```sh
$ nix-shell -p python35Packages.numpy python35Packages.toolz --run "python3"
##### Installing environments globally on the system
Up to now, we've been creating environments scoped to an ad-hoc shell session,
or a single script, or a single project. This is generally advisable, as it
avoids pollution across contexts.
However, sometimes we know we will often want a Python with some basic packages,
and want this available without having to enter into a shell or build context.
This can be useful to have things like vim/emacs editors and plugins or shell
tools "just work" without having to set them up, or when running other software
that expects packages to be installed globally.
To create your own custom environment, create a file in `~/.config/nixpkgs/overlays/`
that looks like this:
```nix
# ~/.config/nixpkgs/overlays/myEnv.nix
self: super: {
myEnv = super.buildEnv {
name = "myEnv";
paths = [
# A Python 3 interpreter with some packages
(self.python3.withPackages (
ps: with ps; [
pyflakes
pytest
python-language-server
]
))
# Some other packages we'd like as part of this env
self.mypy
self.black
self.ripgrep
self.tmux
];
};
}
```
or run a script
You can then build and install this to your profile with:
```sh
$ nix-shell -p python35Packages.numpy python35Packages.toolz --run "python3 myscript.py"
nix-env -iA myEnv
```
##### `nix-shell` as shebang
In fact, for the second use case, there is a more convenient method. You can add
a [shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) to your script
specifying which dependencies `nix-shell` needs. With the following shebang, you
can just execute `./myscript.py`, and it will make available all dependencies
and run the script in the `python3` shell.
One limitation of this is that you can only have 1 Python env installed
globally, since they conflict on the `python` to load out of your `PATH`.
```py
#! /usr/bin/env nix-shell
#! nix-shell -i python3 -p "python3.withPackages(ps: [ps.numpy])"
If you get a conflict or prefer to keep the setup clean, you can have `nix-env`
atomically *uninstall* all other imperatively installed packages and replace
your profile with just `myEnv` by using the `--replace` flag.
import numpy
##### Environment defined in `/etc/nixos/configuration.nix`
print(numpy.__version__)
For the sake of completeness, here's how to install the environment system-wide
on NixOS.
```nix
{ # ...
environment.systemPackages = with pkgs; [
(python38.withPackages(ps: with ps; [ numpy toolz ]))
];
}
```
### Developing with Python
Now that you know how to get a working Python environment with Nix, it is time
to go forward and start actually developing with Python. We will first have a
look at how Python packages are packaged on Nix. Then, we will look at how you
can use development mode with your code.
Above, we were mostly just focused on use cases and what to do to get started
creating working Python environments in nix.
#### Packaging a library
Now that you know the basics to be up and running, it is time to take a step
back and take a deeper look at at how Python packages are packaged on Nix. Then,
we will look at how you can use development mode with your code.
#### Python library packages in Nixpkgs
With Nix all packages are built by functions. The main function in Nix for
building Python libraries is `buildPythonPackage`. Let's see how we can build the
@ -231,11 +348,11 @@ building Python libraries is `buildPythonPackage`. Let's see how we can build th
buildPythonPackage rec {
pname = "toolz";
version = "0.7.4";
version = "0.10.0";
src = fetchPypi {
inherit pname version;
sha256 = "43c2c9e5e7a16b6c88ba3088a9bfc82f7db8e13378be7c78d6c14a5f8ed05afd";
sha256 = "08fdd5ef7c96480ad11c12d472de21acd32359996f69a5259299b540feba4560";
};
doCheck = false;
@ -260,8 +377,9 @@ information. The output of the function is a derivation.
An expression for `toolz` can be found in the Nixpkgs repository. As explained
in the introduction of this Python section, a derivation of `toolz` is available
for each interpreter version, e.g. `python35.pkgs.toolz` refers to the `toolz`
derivation corresponding to the CPython 3.5 interpreter.
for each interpreter version, e.g. `python38.pkgs.toolz` refers to the `toolz`
derivation corresponding to the CPython 3.8 interpreter.
The above example works when you're directly working on
`pkgs/top-level/python-packages.nix` in the Nixpkgs repository. Often though,
you will want to test a Nix expression outside of the Nixpkgs tree.
@ -273,13 +391,13 @@ and adds it along with a `numpy` package to a Python environment.
with import <nixpkgs> {};
( let
my_toolz = python35.pkgs.buildPythonPackage rec {
my_toolz = python38.pkgs.buildPythonPackage rec {
pname = "toolz";
version = "0.7.4";
version = "0.10.0";
src = python35.pkgs.fetchPypi {
src = python38.pkgs.fetchPypi {
inherit pname version;
sha256 = "43c2c9e5e7a16b6c88ba3088a9bfc82f7db8e13378be7c78d6c14a5f8ed05afd";
sha256 = "08fdd5ef7c96480ad11c12d472de21acd32359996f69a5259299b540feba4560";
};
doCheck = false;
@ -290,12 +408,12 @@ with import <nixpkgs> {};
};
};
in python35.withPackages (ps: [ps.numpy my_toolz])
in python38.withPackages (ps: [ps.numpy my_toolz])
).env
```
Executing `nix-shell` will result in an environment in which you can use
Python 3.5 and the `toolz` package. As you can see we had to explicitly mention
Python 3.8 and the `toolz` package. As you can see we had to explicitly mention
for which Python version we want to build a package.
So, what did we do here? Well, we took the Nix expression that we used earlier
@ -312,7 +430,7 @@ Our example, `toolz`, does not have any dependencies on other Python packages or
system libraries. According to the manual, `buildPythonPackage` uses the
arguments `buildInputs` and `propagatedBuildInputs` to specify dependencies. If
something is exclusively a build-time dependency, then the dependency should be
included as a `buildInput`, but if it is (also) a runtime dependency, then it
included in `buildInputs`, but if it is (also) a runtime dependency, then it
should be added to `propagatedBuildInputs`. Test dependencies are considered
build-time dependencies and passed to `checkInputs`.
@ -412,7 +530,7 @@ buildPythonPackage rec {
meta = with lib; {
description = "A pythonic wrapper around FFTW, the FFT library, presenting a unified interface for all the supported transforms";
homepage = http://hgomersall.github.com/pyFFTW;
homepage = "http://hgomersall.github.com/pyFFTW";
license = with licenses; [ bsd2 bsd3 ];
maintainers = with maintainers; [ fridh ];
};
@ -420,13 +538,129 @@ buildPythonPackage rec {
```
Note also the line `doCheck = false;`, we explicitly disabled running the test-suite.
#### Testing Python Packages
#### Develop local package
It is highly encouraged to have testing as part of the package build. This
helps to avoid situations where the package was able to build and install,
but is not usable at runtime. Currently, all packages will use the `test`
command provided by the setup.py (i.e. `python setup.py test`). However,
this is currently deprecated https://github.com/pypa/setuptools/pull/1878
and your package should provide its own checkPhase.
As a Python developer you're likely aware of [development mode](http://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode) (`python setup.py develop`);
instead of installing the package this command creates a special link to the project code.
That way, you can run updated code without having to reinstall after each and every change you make.
Development mode is also available. Let's see how you can use it.
*NOTE:* The `checkPhase` for python maps to the `installCheckPhase` on a
normal derivation. This is due to many python packages not behaving well
to the pre-installed version of the package. Version info, and natively
compiled extensions generally only exist in the install directory, and
thus can cause issues when a test suite asserts on that behavior.
*NOTE:* Tests should only be disabled if they don't agree with nix
(e.g. external dependencies, network access, flakey tests), however,
as many tests should be enabled as possible. Failing tests can still be
a good indication that the package is not in a valid state.
#### Using pytest
Pytest is the most common test runner for python repositories. A trivial
test run would be:
```
checkInputs = [ pytest ];
checkPhase = "pytest";
```
However, many repositories' test suites do not translate well to nix's build
sandbox, and will generally need many tests to be disabled.
To filter tests using pytest, one can do the following:
```
checkInputs = [ pytest ];
# avoid tests which need additional data or touch network
checkPhase = ''
pytest tests/ --ignore=tests/integration -k 'not download and not update'
'';
```
`--ignore` will tell pytest to ignore that file or directory from being
collected as part of a test run. This is useful is a file uses a package
which is not available in nixpkgs, thus skipping that test file is much
easier than having to create a new package.
`-k` is used to define a predicate for test names. In this example, we are
filtering out tests which contain `download` or `update` in their test case name.
Only one `-k` argument is allows, and thus a long predicate should be concatenated
with "\" and wrapped to the next line.
*NOTE:* In pytest==6.0.1, the use of "\" to continue a line (e.g. `-k 'not download \'`) has
been removed, in this case, it's recommended to use `pytestCheckHook`.
#### Using pytestCheckHook
`pytestCheckHook` is a convenient hook which will substitute the setuptools
`test` command for a checkPhase which runs `pytest`. This is also beneficial
when a package may need many items disabled to run the test suite.
Using the example above, the analagous pytestCheckHook usage would be:
```
checkInputs = [ pytestCheckHook ];
# requires additional data
pytestFlagsArray = [ "tests/" "--ignore=tests/integration" ];
disabledTests = [
# touches network
"download"
"update"
];
```
This is expecially useful when tests need to be conditionallydisabled,
for example:
```
disabledTests = [
# touches network
"download"
"update"
] ++ lib.optionals (pythonAtLeast "3.8") [
# broken due to python3.8 async changes
"async"
] ++ lib.optionals stdenv.isDarwin [
# can fail when building with other packages
"socket"
];
```
Trying to concatenate the related strings to disable tests in a regular checkPhase
would be much harder to read. This also enables us to comment on why specific tests
are disabled.
#### Using pythonImportsCheck
Although unit tests are highly prefered to valid correctness of a package. Not
all packages have test suites that can be ran easily, and some have none at all.
To help ensure the package still works, `pythonImportsCheck` can attempt to import
the listed modules.
```
pythonImportsCheck = [ "requests" "urllib" ];
```
roughly translates to:
```
postCheck = ''
PYTHONPATH=$out/${python.sitePackages}:$PYTHONPATH
python -c "import requests; import urllib"
'';
```
However, this is done in it's own phase, and not dependent on whether `doCheck = true;`
This can also be useful in verifying that the package doesn't assume commonly
present packages (e.g. `setuptools`)
### Develop local package
As a Python developer you're likely aware of [development mode](http://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode)
(`python setup.py develop`); instead of installing the package this command
creates a special link to the project code. That way, you can run updated code
without having to reinstall after each and every change you make. Development
mode is also available. Let's see how you can use it.
In the previous Nix expression the source was fetched from an url. We can also
refer to a local source instead using `src = ./path/to/source/tree;`
@ -435,7 +669,7 @@ If we create a `shell.nix` file which calls `buildPythonPackage`, and if `src`
is a local source, and if the local source has a `setup.py`, then development
mode is activated.
In the following example we create a simple environment that has a Python 3.5
In the following example we create a simple environment that has a Python 3.8
version of our package in it, as well as its dependencies and other packages we
like to have in the environment, all specified with `propagatedBuildInputs`.
Indeed, we can just add any package we like to have in our environment to
@ -443,7 +677,7 @@ Indeed, we can just add any package we like to have in our environment to
```nix
with import <nixpkgs> {};
with python35Packages;
with python38Packages;
buildPythonPackage rec {
name = "mypackage";
@ -455,7 +689,6 @@ buildPythonPackage rec {
It is important to note that due to how development mode is implemented on Nix
it is not possible to have multiple packages simultaneously in development mode.
### Organising your packages
So far we discussed how you can use Python on Nix, and how you can develop with
@ -481,11 +714,11 @@ We first create a function that builds `toolz` in `~/path/to/toolz/release.nix`
buildPythonPackage rec {
pname = "toolz";
version = "0.7.4";
version = "0.10.0";
src = fetchPypi {
inherit pname version;
sha256 = "43c2c9e5e7a16b6c88ba3088a9bfc82f7db8e13378be7c78d6c14a5f8ed05afd";
sha256 = "08fdd5ef7c96480ad11c12d472de21acd32359996f69a5259299b540feba4560";
};
meta = with lib; {
@ -497,17 +730,17 @@ buildPythonPackage rec {
}
```
It takes an argument `buildPythonPackage`.
We now call this function using `callPackage` in the definition of our environment
It takes an argument `buildPythonPackage`. We now call this function using
`callPackage` in the definition of our environment
```nix
with import <nixpkgs> {};
( let
toolz = callPackage /path/to/toolz/release.nix {
buildPythonPackage = python35Packages.buildPythonPackage;
buildPythonPackage = python38Packages.buildPythonPackage;
};
in python35.withPackages (ps: [ ps.numpy toolz ])
in python38.withPackages (ps: [ ps.numpy toolz ])
).env
```
@ -515,17 +748,17 @@ Important to remember is that the Python version for which the package is made
depends on the `python` derivation that is passed to `buildPythonPackage`. Nix
tries to automatically pass arguments when possible, which is why generally you
don't explicitly define which `python` derivation should be used. In the above
example we use `buildPythonPackage` that is part of the set `python35Packages`,
and in this case the `python35` interpreter is automatically used.
example we use `buildPythonPackage` that is part of the set `python38Packages`,
and in this case the `python38` interpreter is automatically used.
## Reference
### Interpreters
Versions 2.7, 3.5, 3.6, 3.7 and 3.8 of the CPython interpreter are available as
respectively `python27`, `python35`, `python36`, `python37` and `python38`. The
Versions 2.7, 3.6, 3.7 and 3.8 of the CPython interpreter are available as
respectively `python27`, `python36`, `python37` and `python38`. The
aliases `python2` and `python3` correspond to respectively `python27` and
`python37`. The default interpreter, `python`, maps to `python2`. The PyPy
`python38`. The default interpreter, `python`, maps to `python2`. The PyPy
interpreters compatible with Python 2.7 and 3 are available as `pypy27` and
`pypy3`, with aliases `pypy2` mapping to `pypy27` and `pypy` mapping to `pypy2`.
The Nix expressions for the interpreters can be found in
@ -548,7 +781,7 @@ Each interpreter has the following attributes:
- `buildEnv`. Function to build python interpreter environments with extra packages bundled together. See section *python.buildEnv function* for usage and documentation.
- `withPackages`. Simpler interface to `buildEnv`. See section *python.withPackages function* for usage and documentation.
- `sitePackages`. Alias for `lib/${libPrefix}/site-packages`.
- `executable`. Name of the interpreter executable, e.g. `python3.7`.
- `executable`. Name of the interpreter executable, e.g. `python3.8`.
- `pkgs`. Set of Python packages for that specific interpreter. The package set can be modified by overriding the interpreter and passing `packageOverrides`.
### Building packages and applications
@ -571,15 +804,16 @@ attribute set is created for each available Python interpreter. The available
sets are
* `pkgs.python27Packages`
* `pkgs.python35Packages`
* `pkgs.python36Packages`
* `pkgs.python37Packages`
* `pkgs.python38Packages`
* `pkgs.python39Packages`
* `pkgs.pypyPackages`
and the aliases
* `pkgs.python2Packages` pointing to `pkgs.python27Packages`
* `pkgs.python3Packages` pointing to `pkgs.python37Packages`
* `pkgs.python3Packages` pointing to `pkgs.python38Packages`
* `pkgs.pythonPackages` pointing to `pkgs.python2Packages`
#### `buildPythonPackage` function
@ -643,12 +877,9 @@ following are specific to `buildPythonPackage`:
appears more than once in dependency tree. Default is `true`.
* `disabled` ? false: If `true`, package is not built for the particular Python
interpreter version.
* `dontWrapPythonPrograms ? false`: Skip wrapping of python programs.
* `dontWrapPythonPrograms ? false`: Skip wrapping of Python programs.
* `permitUserSite ? false`: Skip setting the `PYTHONNOUSERSITE` environment
variable in wrapped programs.
* `installFlags ? []`: A list of strings. Arguments to be passed to `pip
install`. To pass options to `python setup.py install`, use
`--install-option`. E.g., `installFlags=["--install-option='--cpp_implementation'"]`.
* `format ? "setuptools"`: Format of the source. Valid options are
`"setuptools"`, `"pyproject"`, `"flit"`, `"wheel"`, and `"other"`.
`"setuptools"` is for when the source has a `setup.py` and `setuptools` is
@ -662,8 +893,11 @@ following are specific to `buildPythonPackage`:
variables which will be available when the binary is run. For example,
`makeWrapperArgs = ["--set FOO BAR" "--set BAZ QUX"]`.
* `namePrefix`: Prepends text to `${name}` parameter. In case of libraries, this
defaults to `"python3.5-"` for Python 3.5, etc., and in case of applications
defaults to `"python3.8-"` for Python 3.8, etc., and in case of applications
to `""`.
* `pipInstallFlags ? []`: A list of strings. Arguments to be passed to `pip
install`. To pass options to `python setup.py install`, use
`--install-option`. E.g., `pipInstallFlags=["--install-option='--cpp_implementation'"]`.
* `pythonPath ? []`: List of packages to be added into `$PYTHONPATH`. Packages
in `pythonPath` are not propagated (contrary to `propagatedBuildInputs`).
* `preShellHook`: Hook to execute commands before `shellHook`.
@ -730,7 +964,7 @@ Another difference is that `buildPythonPackage` by default prefixes the names of
the packages with the version of the interpreter. Because this is irrelevant for
applications, the prefix is omitted.
When packaging a python application with `buildPythonApplication`, it should be
When packaging a Python application with `buildPythonApplication`, it should be
called with `callPackage` and passed `python` or `pythonPackages` (possibly
specifying an interpreter version), like this:
@ -761,7 +995,7 @@ luigi = callPackage ../applications/networking/cluster/luigi { };
```
Since the package is an application, a consumer doesn't need to care about
python versions or modules, which is why they don't go in `pythonPackages`.
Python versions or modules, which is why they don't go in `pythonPackages`.
#### `toPythonApplication` function
@ -875,7 +1109,7 @@ thus be also written like this:
```nix
with import <nixpkgs> {};
(python36.withPackages (ps: [ps.numpy ps.requests])).env
(python38.withPackages (ps: [ps.numpy ps.requests])).env
```
In contrast to `python.buildEnv`, `python.withPackages` does not support the
@ -898,14 +1132,15 @@ are used in `buildPythonPackage`.
- `pipBuildHook` to build a wheel using `pip` and PEP 517. Note a build system
(e.g. `setuptools` or `flit`) should still be added as `nativeBuildInput`.
- `pipInstallHook` to install wheels.
- `pytestCheckHook` to run tests with `pytest`.
- `pytestCheckHook` to run tests with `pytest`. See [example usage](#using-pytestcheckhook).
- `pythonCatchConflictsHook` to check whether a Python package is not already existing.
- `pythonImportsCheckHook` to check whether importing the listed modules works.
- `pythonRemoveBinBytecode` to remove bytecode from the `/bin` folder.
- `setuptoolsBuildHook` to build a wheel using `setuptools`.
- `setuptoolsCheckHook` to run tests with `python setup.py test`.
- `venvShellHook` to source a Python 3 `venv` at the `venvDir` location. A
`venv` is created if it does not yet exist.
`venv` is created if it does not yet exist. `postVenvCreation` can be used to
to run commands only after venv is first created.
- `wheelUnpackHook` to move a wheel to the correct folder so it can be installed
with the `pipInstallHook`.
@ -932,7 +1167,7 @@ pythonPackages.buildPythonPackage {
Running `nix-shell` with no arguments should give you the environment in which
the package would be built with `nix-build`.
Shortcut to setup environments with C headers/libraries and python packages:
Shortcut to setup environments with C headers/libraries and Python packages:
```shell
nix-shell -p pythonPackages.pyramid zlib libjpeg git
@ -960,10 +1195,9 @@ has security implications and is relevant for those using Python in a
When the environment variable `DETERMINISTIC_BUILD` is set, all bytecode will
have timestamp 1. The `buildPythonPackage` function sets `DETERMINISTIC_BUILD=1`
and [PYTHONHASHSEED=0](https://docs.python.org/3.5/using/cmdline.html#envvar-PYTHONHASHSEED).
and [PYTHONHASHSEED=0](https://docs.python.org/3.8/using/cmdline.html#envvar-PYTHONHASHSEED).
Both are also exported in `nix-shell`.
### Automatic tests
It is recommended to test packages as part of the build process.
@ -976,7 +1210,7 @@ example of such a situation is when `py.test` is used.
#### Common issues
* Non-working tests can often be deselected. By default `buildPythonPackage`
runs `python setup.py test`. Most python modules follows the standard test
runs `python setup.py test`. Most Python modules follows the standard test
protocol where the pytest runner can be used instead. `py.test` supports a
`-k` parameter to ignore test methods or classes:
@ -1014,7 +1248,7 @@ with import <nixpkgs> {};
packageOverrides = self: super: {
pandas = super.pandas.overridePythonAttrs(old: {name="foo";});
};
in pkgs.python35.override {inherit packageOverrides;};
in pkgs.python38.override {inherit packageOverrides;};
in python.withPackages(ps: [ps.pandas])).env
```
@ -1036,7 +1270,7 @@ with import <nixpkgs> {};
packageOverrides = self: super: {
scipy = super.scipy_0_17;
};
in (pkgs.python35.override {inherit packageOverrides;}).withPackages (ps: [ps.blaze])
in (pkgs.python38.override {inherit packageOverrides;}).withPackages (ps: [ps.blaze])
).env
```
@ -1049,12 +1283,12 @@ If you want the whole of Nixpkgs to use your modifications, then you can use
```nix
let
pkgs = import <nixpkgs> {};
newpkgs = import pkgs.path { overlays = [ (pkgsself: pkgssuper: {
python27 = let
packageOverrides = self: super: {
numpy = super.numpy_1_10;
newpkgs = import pkgs.path { overlays = [ (self: super: {
python38 = let
packageOverrides = python-self: python-super: {
numpy = python-super.numpy_1_18;
};
in pkgssuper.python27.override {inherit packageOverrides;};
in super.python38.override {inherit packageOverrides;};
} ) ]; };
in newpkgs.inkscape
```
@ -1127,14 +1361,14 @@ If you want to create a Python environment for development, then the recommended
method is to use `nix-shell`, either with or without the `python.buildEnv`
function.
### How to consume python modules using pip in a virtual environment like I am used to on other Operating Systems?
### How to consume Python modules using pip in a virtual environment like I am used to on other Operating Systems?
While this approach is not very idiomatic from Nix perspective, it can still be
useful when dealing with pre-existing projects or in situations where it's not
feasible or desired to write derivations for all required dependencies.
This is an example of a `default.nix` for a `nix-shell`, which allows to consume
a virtual environment created by `venv`, and install python modules through
a virtual environment created by `venv`, and install Python modules through
`pip` the traditional way.
Create this `default.nix` file, together with a `requirements.txt` and simply
@ -1149,7 +1383,7 @@ in pkgs.mkShell rec {
name = "impurePythonEnv";
venvDir = "./.venv";
buildInputs = [
# A python interpreter including the 'venv' module is required to bootstrap
# A Python interpreter including the 'venv' module is required to bootstrap
# the environment.
pythonPackages.python
@ -1163,7 +1397,7 @@ in pkgs.mkShell rec {
pythonPackages.requests
# In this particular example, in order to compile any binary extensions they may
# require, the python modules listed in the hypothetical requirements.txt need
# require, the Python modules listed in the hypothetical requirements.txt need
# the following packages to be installed locally:
taglib
openssl
@ -1174,16 +1408,23 @@ in pkgs.mkShell rec {
zlib
];
# Run this command, only after creating the virtual environment
postVenvCreation = ''
unset SOURCE_DATE_EPOCH
pip install -r requirements.txt
'';
# Now we can execute any commands within the virtual environment.
# This is optional and can be left out to run pip manually.
postShellHook = ''
pip install -r requirements.txt
# allow pip to install wheels
unset SOURCE_DATE_EPOCH
'';
}
```
In case the supplied venvShellHook is insufficient, or when python 2 support is
In case the supplied venvShellHook is insufficient, or when Python 2 support is
needed, you can define your own shell hook and adapt to your needs like in the
following example:
@ -1229,7 +1470,7 @@ in pkgs.mkShell rec {
```
Note that the `pip install` is an imperative action. So every time `nix-shell`
is executed it will attempt to download the python modules listed in
is executed it will attempt to download the Python modules listed in
requirements.txt. However these will be cached locally within the `virtualenv`
folder and not downloaded again.
@ -1290,32 +1531,8 @@ self: super: {
### How to use Intel's MKL with numpy and scipy?
A `site.cfg` is created that configures BLAS based on the `blas` parameter of
the `numpy` derivation. By passing in `mkl`, `numpy` and packages depending on
`numpy` will be built with `mkl`.
The following is an overlay that configures `numpy` to use `mkl`:
```nix
self: super: {
python37 = super.python37.override {
packageOverrides = python-self: python-super: {
numpy = python-super.numpy.override {
blas = super.pkgs.mkl;
};
};
};
}
```
`mkl` requires an `openmp` implementation when running with multiple processors.
By default, `mkl` will use Intel's `iomp` implementation if no other is
specified, but this is a runtime-only dependency and binary compatible with the
LLVM implementation. To use that one instead, Intel recommends users set it with
`LD_PRELOAD`.
Note that `mkl` is only available on `x86_64-{linux,darwin}` platforms;
moreover, Hydra is not building and distributing pre-compiled binaries using it.
MKL can be configured using an overlay. See the section "[Using overlays to
configure alternatives](#sec-overlays-alternatives-blas-lapack)".
### What inputs do `setup_requires`, `install_requires` and `tests_require` map to?

4
doc/languages-frameworks/ruby.xml
View File

@ -32,7 +32,7 @@ bundlerEnv rec {
meta = with lib; {
description = "A monitoring framework that aims to be simple, malleable, and scalable";
homepage = http://sensuapp.org/;
homepage = "http://sensuapp.org/";
license = with licenses; mit;
maintainers = with maintainers; [ theuni ];
platforms = platforms.unix;
@ -69,7 +69,7 @@ bundlerApp {
meta = with lib; {
description = "Tool and libraries for maintaining Ruby gems.";
homepage = https://github.com/nyarly/corundum;
homepage = "https://github.com/nyarly/corundum";
license = licenses.mit;
maintainers = [ maintainers.nyarly ];
platforms = platforms.unix;

85
doc/languages-frameworks/rust.section.md
View File

@ -43,7 +43,6 @@ rustPlatform.buildRustPackage rec {
homepage = "https://github.com/BurntSushi/ripgrep";
license = licenses.unlicense;
maintainers = [ maintainers.tailhook ];
platforms = platforms.all;
};
}
```
@ -75,6 +74,90 @@ pkgs.rustPlatform.buildRustPackage {
}
```
### Running package tests
When using `buildRustPackage`, the `checkPhase` is enabled by default and runs
`cargo test` on the package to build. To make sure that we don't compile the
sources twice and to actually test the artifacts that will be used at runtime,
the tests will be ran in the `release` mode by default.
However, in some cases the test-suite of a package doesn't work properly in the
`release` mode. For these situations, the mode for `checkPhase` can be changed like
so:
```nix
rustPlatform.buildRustPackage {
/* ... */
checkType = "debug";
}
```
Please note that the code will be compiled twice here: once in `release` mode
for the `buildPhase`, and again in `debug` mode for the `checkPhase`.
#### Tests relying on the structure of the `target/` directory
Some tests may rely on the structure of the `target/` directory. Those tests
are likely to fail because we use `cargo --target` during the build. This means that
the artifacts
[are stored in `target/<architecture>/release/`](https://doc.rust-lang.org/cargo/guide/build-cache.html),
rather than in `target/release/`.
This can only be worked around by patching the affected tests accordingly.
#### Disabling package-tests
In some instances, it may be necessary to disable testing altogether (with `doCheck = false;`):
* If no tests exist -- the `checkPhase` should be explicitly disabled to skip
unnecessary build steps to speed up the build.
* If tests are highly impure (e.g. due to network usage).
There will obviously be some corner-cases not listed above where it's sensible to disable tests.
The above are just guidelines, and exceptions may be granted on a case-by-case basis.
However, please check if it's possible to disable a problematic subset of the
test suite and leave a comment explaining your reasoning.
### Building a package in `debug` mode
By default, `buildRustPackage` will use `release` mode for builds. If a package
should be built in `debug` mode, it can be configured like so:
```nix
rustPlatform.buildRustPackage {
/* ... */
buildType = "debug";
}
```
In this scenario, the `checkPhase` will be ran in `debug` mode as well.
### Custom `build`/`install`-procedures
Some packages may use custom scripts for building/installing, e.g. with a `Makefile`.
In these cases, it's recommended to override the `buildPhase`/`installPhase`/`checkPhase`.
Otherwise, some steps may fail because of the modified directory structure of `target/`.
### Building a crate with an absent or out-of-date Cargo.lock file
`buildRustPackage` needs a `Cargo.lock` file to get all dependencies in the
source code in a reproducible way. If it is missing or out-of-date one can use
the `cargoPatches` attribute to update or add it.
```
{ lib, rustPlatform, fetchFromGitHub }:
rustPlatform.buildRustPackage rec {
(...)
cargoPatches = [
# a patch file to add/update Cargo.lock in the source code
./add-Cargo.lock.patch
];
}
```
## Compiling Rust crates using Nix instead of Cargo
### Simple operation

27
doc/languages-frameworks/texlive.xml
View File

@ -149,31 +149,4 @@ EOF
]]></programlisting>
</para>
</section>
<section xml:id="sec-language-texlive-known-problems">
<title>Known problems</title>
<itemizedlist>
<listitem>
<para>
Some tools are still missing, e.g. luajittex;
</para>
</listitem>
<listitem>
<para>
some apps aren't packaged/tested yet (asymptote, biber, etc.);
</para>
</listitem>
<listitem>
<para>
feature/bug: when a package is rejected by <varname>pkgFilter</varname>, its dependencies are still propagated;
</para>
</listitem>
<listitem>
<para>
in case of any bugs or feature requests, file a github issue or better a pull request and /cc @vcunat.
</para>
</listitem>
</itemizedlist>
</section>
</section>

7
doc/languages-frameworks/vim.section.md
View File

@ -261,12 +261,7 @@ deoplete-fish = super.deoplete-fish.overrideAttrs(old: {
Sometimes plugins require an override that must be changed when the plugin is updated. This can cause issues when Vim plugins are auto-updated but the associated override isn't updated. For these plugins, the override should be written so that it specifies all information required to install the plugin, and running `./update.py` doesn't change the derivation for the plugin. Manually updating the override is required to update these types of plugins. An example of such a plugin is `LanguageClient-neovim`.
To add a new plugin:
1. run `./update.py` and create a commit named "vimPlugins: Update",
2. add the new plugin to [vim-plugin-names](/pkgs/misc/vim-plugins/vim-plugin-names) and add overrides if required to [overrides.nix](/pkgs/misc/vim-plugins/overrides.nix),
3. run `./update.py` again and create a commit named "vimPlugins.[name]: init at [version]" (where `name` and `version` can be found in [generated.nix](/pkgs/misc/vim-plugins/generated.nix)), and
4. create a pull request.
To add a new plugin, run `./update.py --add "[owner]/[name]"`. **NOTE**: This script automatically commits to your git repository. Be sure to check out a fresh branch before running.
## Important repositories

8
doc/old/cross.txt
View File

@ -60,7 +60,7 @@ stdenv.mkDerivation {
name = "binutils-2.16.1-arm";
builder = ./builder.sh;
src = fetchurl {
url = http://ftp.nluug.nl/gnu/binutils/binutils-2.16.1.tar.bz2;
url = "http://ftp.nluug.nl/gnu/binutils/binutils-2.16.1.tar.bz2";
sha256 = "1ian3kwh2vg6hr3ymrv48s04gijs539vzrq62xr76bxbhbwnz2np";
};
inherit noSysDirs;
@ -84,7 +84,7 @@ stdenv.mkDerivation {
name = "linux-headers-2.6.13.1-arm";
builder = ./builder.sh;
src = fetchurl {
url = http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.13.1.tar.bz2;
url = "http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.13.1.tar.bz2";
sha256 = "12qxmc827fjhaz53kjy7vyrzsaqcg78amiqsb3qm20z26w705lma";
};
}
@ -220,7 +220,7 @@ fi
preConfigure=preConfigure
preConfigure() {
# Determine the frontends to build.
langs="c"
if test -n "$langCC"; then
@ -262,7 +262,7 @@ postInstall() {
#if test -z "$profiledCompiler"; then
#makeFlags="bootstrap"
#else
#else
#makeFlags="profiledbootstrap"
#fi

4
doc/preface.chapter.md
View File

@ -37,12 +37,12 @@ security updates. More up to date packages and modules are available via the
Both `nixos-unstable` and `nixpkgs` follow the `master` branch of the Nixpkgs
repository, although both do lag the `master` branch by generally
[a couple of days](https://howoldis.herokuapp.com/). Updates to a channel are
[a couple of days](https://status.nixos.org/). Updates to a channel are
distributed as soon as all tests for that channel pass, e.g.
[this table](https://hydra.nixos.org/job/nixpkgs/trunk/unstable#tabs-constituents)
shows the status of tests for the `nixpkgs` channel.
The tests are conducted by a cluster called [Hydra](http://nixos.org/hydra/),
The tests are conducted by a cluster called [Hydra](https://nixos.org/hydra/),
which also builds binary packages from the Nix expressions in Nixpkgs for
`x86_64-linux`, `i686-linux` and `x86_64-darwin`.
The binaries are made available via a [binary cache](https://cache.nixos.org).

6
doc/release-notes.xml
View File

@ -190,7 +190,7 @@ preConfigure = "configureFlagsArray=(\"CFLAGS=-O0 -g\")";</programlisting>
The function <function>fetchurl</function> now has support for two different kinds of mirroring of files. First, it has support for <emphasis>content-addressable mirrors</emphasis>. For example, given the <function>fetchurl</function> call
<programlisting>
fetchurl {
url = http://releases.mozilla.org/<replaceable>...</replaceable>/firefox-2.0.0.6-source.tar.bz2;
url = "http://releases.mozilla.org/<replaceable>...</replaceable>/firefox-2.0.0.6-source.tar.bz2";
sha1 = "eb72f55e4a8bf08e8c6ef227c0ade3d068ba1082";
}</programlisting>
<function>fetchurl</function> will first try to download this file from <link
@ -286,7 +286,7 @@ export NIX_MIRRORS_sourceforge=http://osdn.dl.sourceforge.net/sourceforge/</prog
<note>
<para>
This release of Nixpkgs requires <link
xlink:href='http://nixos.org/releases/nix/nix-0.10/'>Nix 0.10</link> or higher.
xlink:href='https://nixos.org/releases/nix/nix-0.10/'>Nix 0.10</link> or higher.
</para>
</note>
@ -436,7 +436,7 @@ stdenv.mkDerivation {
<listitem>
<para>
Distribution files have been moved to <link
xlink:href="http://nixos.org/" />.
xlink:href="https://nixos.org/" />.
</para>
</listitem>
<listitem>

6
doc/stdenv/meta.xml
View File

@ -155,17 +155,17 @@ hello-2.3 A program that produces a familiar, friendly greeting
<itemizedlist>
<listitem>
<para>
Single license referenced by attribute (preferred) <literal>stdenv.lib.licenses.gpl3</literal>.
Single license referenced by attribute (preferred) <literal>stdenv.lib.licenses.gpl3Only</literal>.
</para>
</listitem>
<listitem>
<para>
Single license referenced by its attribute shortName (frowned upon) <literal>"gpl3"</literal>.
Single license referenced by its attribute shortName (frowned upon) <literal>"gpl3Only"</literal>.
</para>
</listitem>
<listitem>
<para>
Single license referenced by its attribute spdxId (frowned upon) <literal>"GPL-3.0"</literal>.
Single license referenced by its attribute spdxId (frowned upon) <literal>"GPL-3.0-only"</literal>.
</para>
</listitem>
<listitem>

2
doc/stdenv/multiple-output.xml
View File

@ -106,7 +106,7 @@
</para>
<para>
The reason for why <literal>glibc</literal> deviates from the convention is because referencing a library provided by <literal>glibc</literal> is a very common operation among Nix packages. For instance, third-party executables packaged by Nix are typically patched and relinked with the relevant version of <literal>glibc</literal> libraries from Nix packages (please see the documentation on <link xlink:href="https://nixos.org/patchelf.html">patchelf</link> for more details).
The reason for why <literal>glibc</literal> deviates from the convention is because referencing a library provided by <literal>glibc</literal> is a very common operation among Nix packages. For instance, third-party executables packaged by Nix are typically patched and relinked with the relevant version of <literal>glibc</literal> libraries from Nix packages (please see the documentation on <link xlink:href="https://github.com/NixOS/patchelf/blob/master/README">patchelf</link> for more details).
</para>
</section>

229
doc/stdenv/stdenv.xml
View File

@ -14,7 +14,7 @@
stdenv.mkDerivation {
name = "libfoo-1.2.3";
src = fetchurl {
url = http://example.org/libfoo-1.2.3.tar.bz2;
url = "http://example.org/libfoo-1.2.3.tar.bz2";
sha256 = "0x2g1jqygyr5wiwg4ma1nd7w4ydpy82z9gkcv8vh2v8dn3y58v5m";
};
}</programlisting>
@ -145,7 +145,7 @@ genericBuild
</listitem>
<listitem>
<para>
GNU Make. It has been patched to provide <quote>nested</quote> output that can be fed into the <command>nix-log2xml</command> command and <command>log2html</command> stylesheet to create a structured, readable output of the build steps performed by Make.
GNU Make.
</para>
</listitem>
<listitem>
@ -254,7 +254,7 @@ let f(h, h + 1, i) = i + h
<variablelist>
<title>Variables specifying dependencies</title>
<varlistentry>
<varlistentry xml:id="var-stdenv-depsBuildBuild">
<term>
<varname>depsBuildBuild</varname>
</term>
@ -267,7 +267,7 @@ let f(h, h + 1, i) = i + h
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-nativeBuildInputs">
<term>
<varname>nativeBuildInputs</varname>
</term>
@ -280,7 +280,7 @@ let f(h, h + 1, i) = i + h
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-depsBuildTarget">
<term>
<varname>depsBuildTarget</varname>
</term>
@ -296,7 +296,7 @@ let f(h, h + 1, i) = i + h
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-depsHostHost">
<term>
<varname>depsHostHost</varname>
</term>
@ -306,7 +306,7 @@ let f(h, h + 1, i) = i + h
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-buildInputs">
<term>
<varname>buildInputs</varname>
</term>
@ -319,7 +319,7 @@ let f(h, h + 1, i) = i + h
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-depsTargetTarget">
<term>
<varname>depsTargetTarget</varname>
</term>
@ -329,7 +329,7 @@ let f(h, h + 1, i) = i + h
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-depsBuildBuildPropagated">
<term>
<varname>depsBuildBuildPropagated</varname>
</term>
@ -339,7 +339,7 @@ let f(h, h + 1, i) = i + h
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-propagatedNativeBuildInputs">
<term>
<varname>propagatedNativeBuildInputs</varname>
</term>
@ -349,7 +349,7 @@ let f(h, h + 1, i) = i + h
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-depsBuildTargetPropagated">
<term>
<varname>depsBuildTargetPropagated</varname>
</term>
@ -359,7 +359,7 @@ let f(h, h + 1, i) = i + h
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-depsHostHostPropagated">
<term>
<varname>depsHostHostPropagated</varname>
</term>
@ -369,7 +369,7 @@ let f(h, h + 1, i) = i + h
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-propagatedBuildInputs">
<term>
<varname>propagatedBuildInputs</varname>
</term>
@ -379,7 +379,7 @@ let f(h, h + 1, i) = i + h
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-depsTargetTargetPropagated">
<term>
<varname>depsTargetTargetPropagated</varname>
</term>
@ -396,7 +396,7 @@ let f(h, h + 1, i) = i + h
<variablelist>
<title>Variables affecting <literal>stdenv</literal> initialisation</title>
<varlistentry>
<varlistentry xml:id="var-stdenv-NIX_DEBUG">
<term>
<varname>NIX_DEBUG</varname>
</term>
@ -410,7 +410,7 @@ let f(h, h + 1, i) = i + h
<variablelist>
<title>Attributes affecting build properties</title>
<varlistentry>
<varlistentry xml:id="var-stdenv-enableParallelBuilding">
<term>
<varname>enableParallelBuilding</varname>
</term>
@ -427,7 +427,7 @@ let f(h, h + 1, i) = i + h
<variablelist>
<title>Special variables</title>
<varlistentry>
<varlistentry xml:id="var-stdenv-passthru">
<term>
<varname>passthru</varname>
</term>
@ -504,7 +504,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
There are a number of variables that control what phases are executed and in what order:
<variablelist>
<title>Variables affecting phase control</title>
<varlistentry>
<varlistentry xml:id="var-stdenv-phases">
<term>
<varname>phases</varname>
</term>
@ -517,7 +517,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-prePhases">
<term>
<varname>prePhases</varname>
</term>
@ -527,7 +527,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-preConfigurePhases">
<term>
<varname>preConfigurePhases</varname>
</term>
@ -537,7 +537,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-preBuildPhases">
<term>
<varname>preBuildPhases</varname>
</term>
@ -547,7 +547,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-preInstallPhases">
<term>
<varname>preInstallPhases</varname>
</term>
@ -557,7 +557,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-preFixupPhases">
<term>
<varname>preFixupPhases</varname>
</term>
@ -567,7 +567,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-preDistPhases">
<term>
<varname>preDistPhases</varname>
</term>
@ -577,7 +577,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-postPhases">
<term>
<varname>postPhases</varname>
</term>
@ -635,7 +635,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
<variablelist>
<title>Variables controlling the unpack phase</title>
<varlistentry>
<varlistentry xml:id="var-stdenv-src">
<term>
<varname>srcs</varname> / <varname>src</varname>
</term>
@ -645,7 +645,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-sourceRoot">
<term>
<varname>sourceRoot</varname>
</term>
@ -655,7 +655,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-setSourceRoot">
<term>
<varname>setSourceRoot</varname>
</term>
@ -665,7 +665,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-preUnpack">
<term>
<varname>preUnpack</varname>
</term>
@ -675,7 +675,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-postUnpack">
<term>
<varname>postUnpack</varname>
</term>
@ -685,7 +685,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontUnpack">
<term>
<varname>dontUnpack</varname>
</term>
@ -695,7 +695,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontMakeSourcesWritable">
<term>
<varname>dontMakeSourcesWritable</varname>
</term>
@ -705,7 +705,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-unpackCmd">
<term>
<varname>unpackCmd</varname>
</term>
@ -727,7 +727,17 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
<variablelist>
<title>Variables controlling the patch phase</title>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontPatch">
<term>
<varname>dontPatch</varname>
</term>
<listitem>
<para>
Set to true to skip the patch phase.
</para>
</listitem>
</varlistentry>
<varlistentry xml:id="var-stdenv-patches">
<term>
<varname>patches</varname>
</term>
@ -737,7 +747,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-patchFlags">
<term>
<varname>patchFlags</varname>
</term>
@ -747,7 +757,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-prePatch">
<term>
<varname>prePatch</varname>
</term>
@ -757,7 +767,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-postPatch">
<term>
<varname>postPatch</varname>
</term>
@ -779,7 +789,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
<variablelist>
<title>Variables controlling the configure phase</title>
<varlistentry>
<varlistentry xml:id="var-stdenv-configureScript">
<term>
<varname>configureScript</varname>
</term>
@ -789,7 +799,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-configureFlags">
<term>
<varname>configureFlags</varname>
</term>
@ -799,7 +809,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontConfigure">
<term>
<varname>dontConfigure</varname>
</term>
@ -809,7 +819,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-configureFlagsArray">
<term>
<varname>configureFlagsArray</varname>
</term>
@ -819,7 +829,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontAddPrefix">
<term>
<varname>dontAddPrefix</varname>
</term>
@ -829,7 +839,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-prefix">
<term>
<varname>prefix</varname>
</term>
@ -839,7 +849,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-prefixKey">
<term>
<varname>prefixKey</varname>
</term>
@ -849,7 +859,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontAddDisableDepTrack">
<term>
<varname>dontAddDisableDepTrack</varname>
</term>
@ -859,7 +869,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontFixLibtool">
<term>
<varname>dontFixLibtool</varname>
</term>
@ -875,7 +885,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontDisableStatic">
<term>
<varname>dontDisableStatic</varname>
</term>
@ -888,7 +898,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-configurePlatforms">
<term>
<varname>configurePlatforms</varname>
</term>
@ -903,7 +913,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-preConfigure">
<term>
<varname>preConfigure</varname>
</term>
@ -913,7 +923,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-postConfigure">
<term>
<varname>postConfigure</varname>
</term>
@ -935,7 +945,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
<variablelist>
<title>Variables controlling the build phase</title>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontBuild">
<term>
<varname>dontBuild</varname>
</term>
@ -945,7 +955,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-makefile">
<term>
<varname>makefile</varname>
</term>
@ -955,7 +965,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-makeFlags">
<term>
<varname>makeFlags</varname>
</term>
@ -973,7 +983,7 @@ makeFlags = [ "PREFIX=$(out)" ];
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-makeFlagsArray">
<term>
<varname>makeFlagsArray</varname>
</term>
@ -989,7 +999,7 @@ preBuild = ''
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-buildFlags">
<term>
<varname>buildFlags</varname> / <varname>buildFlagsArray</varname>
</term>
@ -999,7 +1009,7 @@ preBuild = ''
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-preBuild">
<term>
<varname>preBuild</varname>
</term>
@ -1009,7 +1019,7 @@ preBuild = ''
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-postBuild">
<term>
<varname>postBuild</varname>
</term>
@ -1039,7 +1049,7 @@ preBuild = ''
<variablelist>
<title>Variables controlling the check phase</title>
<varlistentry>
<varlistentry xml:id="var-stdenv-doCheck">
<term>
<varname>doCheck</varname>
</term>
@ -1057,11 +1067,11 @@ preBuild = ''
</term>
<listitem>
<para>
See the build phase for details.
See the <link xlink:href="#var-stdenv-makeFlags">build phase</link> for details.
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-checkTarget">
<term>
<varname>checkTarget</varname>
</term>
@ -1071,7 +1081,7 @@ preBuild = ''
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-checkFlags">
<term>
<varname>checkFlags</varname> / <varname>checkFlagsArray</varname>
</term>
@ -1081,7 +1091,7 @@ preBuild = ''
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-checkInputs">
<term>
<varname>checkInputs</varname>
</term>
@ -1091,7 +1101,7 @@ preBuild = ''
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-preCheck">
<term>
<varname>preCheck</varname>
</term>
@ -1101,7 +1111,7 @@ preBuild = ''
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-postCheck">
<term>
<varname>postCheck</varname>
</term>
@ -1123,7 +1133,7 @@ preBuild = ''
<variablelist>
<title>Variables controlling the install phase</title>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontInstall">
<term>
<varname>dontInstall</varname>
</term>
@ -1139,11 +1149,11 @@ preBuild = ''
</term>
<listitem>
<para>
See the build phase for details.
See the <link xlink:href="#var-stdenv-makeFlags">build phase</link> for details.
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-installTargets">
<term>
<varname>installTargets</varname>
</term>
@ -1155,7 +1165,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-installFlags">
<term>
<varname>installFlags</varname> / <varname>installFlagsArray</varname>
</term>
@ -1165,7 +1175,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-preInstall">
<term>
<varname>preInstall</varname>
</term>
@ -1175,7 +1185,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-postInstall">
<term>
<varname>postInstall</varname>
</term>
@ -1219,7 +1229,7 @@ installTargets = "install-bin install-doc";</programlisting>
<variablelist>
<title>Variables controlling the fixup phase</title>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontFixup">
<term>
<varname>dontFixup</varname>
</term>
@ -1229,7 +1239,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontStrip">
<term>
<varname>dontStrip</varname>
</term>
@ -1239,7 +1249,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontStripHost">
<term>
<varname>dontStripHost</varname>
</term>
@ -1249,7 +1259,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontStripTarget">
<term>
<varname>dontStripTarget</varname>
</term>
@ -1259,7 +1269,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontMoveSbin">
<term>
<varname>dontMoveSbin</varname>
</term>
@ -1269,7 +1279,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-stripAllList">
<term>
<varname>stripAllList</varname>
</term>
@ -1279,7 +1289,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-stripAllFlags">
<term>
<varname>stripAllFlags</varname>
</term>
@ -1289,17 +1299,17 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-stripDebugList">
<term>
<varname>stripDebugList</varname>
</term>
<listitem>
<para>
List of directories to search for libraries and executables from which only debugging-related symbols should be stripped. It defaults to <literal>lib bin sbin</literal>.
List of directories to search for libraries and executables from which only debugging-related symbols should be stripped. It defaults to <literal>lib lib32 lib64 libexec bin sbin</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-stripDebugFlags">
<term>
<varname>stripDebugFlags</varname>
</term>
@ -1309,7 +1319,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontPatchELF">
<term>
<varname>dontPatchELF</varname>
</term>
@ -1319,7 +1329,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontPatchShebangs">
<term>
<varname>dontPatchShebangs</varname>
</term>
@ -1329,7 +1339,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontPruneLibtoolFiles">
<term>
<varname>dontPruneLibtoolFiles</varname>
</term>
@ -1339,7 +1349,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-forceShare">
<term>
<varname>forceShare</varname>
</term>
@ -1349,7 +1359,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-setupHook">
<term>
<varname>setupHook</varname>
</term>
@ -1360,7 +1370,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-preFixup">
<term>
<varname>preFixup</varname>
</term>
@ -1370,7 +1380,7 @@ installTargets = "install-bin install-doc";</programlisting>
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-postFixup">
<term>
<varname>postFixup</varname>
</term>
@ -1409,7 +1419,7 @@ set debug-file-directory ~/.nix-profile/lib/debug
<variablelist>
<title>Variables controlling the installCheck phase</title>
<varlistentry>
<varlistentry xml:id="var-stdenv-doInstallCheck">
<term>
<varname>doInstallCheck</varname>
</term>
@ -1421,7 +1431,7 @@ set debug-file-directory ~/.nix-profile/lib/debug
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-installCheckTarget">
<term>
<varname>installCheckTarget</varname>
</term>
@ -1431,7 +1441,7 @@ set debug-file-directory ~/.nix-profile/lib/debug
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-installCheckFlags">
<term>
<varname>installCheckFlags</varname> / <varname>installCheckFlagsArray</varname>
</term>
@ -1441,7 +1451,7 @@ set debug-file-directory ~/.nix-profile/lib/debug
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-installCheckInputs">
<term>
<varname>installCheckInputs</varname>
</term>
@ -1451,7 +1461,7 @@ set debug-file-directory ~/.nix-profile/lib/debug
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-preInstallCheck">
<term>
<varname>preInstallCheck</varname>
</term>
@ -1461,7 +1471,7 @@ set debug-file-directory ~/.nix-profile/lib/debug
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-postInstallCheck">
<term>
<varname>postInstallCheck</varname>
</term>
@ -1483,7 +1493,7 @@ set debug-file-directory ~/.nix-profile/lib/debug
<variablelist>
<title>Variables controlling the distribution phase</title>
<varlistentry>
<varlistentry xml:id="var-stdenv-distTarget">
<term>
<varname>distTarget</varname>
</term>
@ -1493,7 +1503,7 @@ set debug-file-directory ~/.nix-profile/lib/debug
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-distFlags">
<term>
<varname>distFlags</varname> / <varname>distFlagsArray</varname>
</term>
@ -1503,7 +1513,7 @@ set debug-file-directory ~/.nix-profile/lib/debug
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-tarballs">
<term>
<varname>tarballs</varname>
</term>
@ -1513,7 +1523,7 @@ set debug-file-directory ~/.nix-profile/lib/debug
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-dontCopyDist">
<term>
<varname>dontCopyDist</varname>
</term>
@ -1523,7 +1533,7 @@ set debug-file-directory ~/.nix-profile/lib/debug
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-preDist">
<term>
<varname>preDist</varname>
</term>
@ -1533,7 +1543,7 @@ set debug-file-directory ~/.nix-profile/lib/debug
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="var-stdenv-postDist">
<term>
<varname>postDist</varname>
</term>
@ -1991,7 +2001,7 @@ addEnvHooks "$hostOffset" myBashFunction
</para>
</listitem>
</varlistentry>
<varlistentry>
<varlistentry xml:id="setup-hook-autopatchelfhook">
<term>
autoPatchelfHook
</term>
@ -2000,15 +2010,16 @@ addEnvHooks "$hostOffset" myBashFunction
This is a special setup hook which helps in packaging proprietary software in that it automatically tries to find missing shared library dependencies of ELF files based on the given <varname>buildInputs</varname> and <varname>nativeBuildInputs</varname>.
</para>
<para>
You can also specify a <envar>runtimeDependencies</envar> environment variable which lists dependencies that are unconditionally added to all executables.
</para>
<para>
You can also specify a <varname>runtimeDependencies</varname> variable which lists dependencies to be unconditionally added to <glossterm>rpath</glossterm> of all executables.
This is useful for programs that use <citerefentry>
<refentrytitle>dlopen</refentrytitle>
<manvolnum>3</manvolnum> </citerefentry> to load libraries at runtime.
</para>
<para>
In certain situations you may want to run the main command (<command>autoPatchelf</command>) of the setup hook on a file or a set of directories instead of unconditionally patching all outputs. This can be done by setting the <envar>dontAutoPatchelf</envar> environment variable to a non-empty value.
In certain situations you may want to run the main command (<command>autoPatchelf</command>) of the setup hook on a file or a set of directories instead of unconditionally patching all outputs. This can be done by setting the <varname>dontAutoPatchelf</varname> environment variable to a non-empty value.
</para>
<para>
By default <command>autoPatchelf</command> will fail as soon as any ELF file requires a dependency which cannot be resolved via the given build inputs. In some situations you might prefer to just leave missing dependencies unpatched and continue to patch the rest. This can be achieved by setting the <envar>autoPatchelfIgnoreMissingDeps</envar> environment variable to a non-empty value.
</para>
<para>
The <command>autoPatchelf</command> command also recognizes a <parameter class="command">--no-recurse</parameter> command line flag, which prevents it from recursing into subdirectories.
@ -2073,6 +2084,16 @@ postInstall = ''
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
validatePkgConfig
</term>
<listitem>
<para>
The <literal>validatePkgConfig</literal> hook validates all pkg-config (<filename>.pc</filename>) files in a package. This helps catching some common errors in pkg-config files, such as undefined variables.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
cmake

12
doc/using/configuration.xml
View File

@ -85,19 +85,19 @@
<title>Installing packages on unsupported systems</title>
<para>
There are also two ways to try compiling a package which has been marked as unsuported for the given system.
There are also two ways to try compiling a package which has been marked as unsupported for the given system.
</para>
<itemizedlist>
<listitem>
<para>
For allowing the build of a broken package once, you can use an environment variable for a single invocation of the nix tools:
For allowing the build of an unsupported package once, you can use an environment variable for a single invocation of the nix tools:
<programlisting>$ export NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1</programlisting>
</para>
</listitem>
<listitem>
<para>
For permanently allowing broken packages to be built, you may add <literal>allowUnsupportedSystem = true;</literal> to your user's configuration file, like this:
For permanently allowing unsupported packages to be built, you may add <literal>allowUnsupportedSystem = true;</literal> to your user's configuration file, like this:
<programlisting>
{
allowUnsupportedSystem = true;
@ -162,10 +162,10 @@
</programlisting>
</para>
<para>
The following example configuration blacklists the <literal>gpl3</literal> and <literal>agpl3</literal> licenses:
The following example configuration blacklists the <literal>gpl3Only</literal> and <literal>agpl3Only</literal> licenses:
<programlisting>
{
blacklistedLicenses = with stdenv.lib.licenses; [ agpl3 gpl3 ];
blacklistedLicenses = with stdenv.lib.licenses; [ agpl3Only gpl3Only ];
}
</programlisting>
</para>
@ -387,7 +387,7 @@ fi
</screen>
<para>
Now just run <literal>source $HOME/.profile</literal> and you can starting loading man pages from your environent.
Now just run <literal>source $HOME/.profile</literal> and you can starting loading man pages from your environment.
</para>
</section>

140
doc/using/overlays.xml
View File

@ -137,4 +137,144 @@ self: super:
Overlays are similar to other methods for customizing Nixpkgs, in particular the <literal>packageOverrides</literal> attribute described in <xref linkend="sec-modify-via-packageOverrides"/>. Indeed, <literal>packageOverrides</literal> acts as an overlay with only the <varname>super</varname> argument. It is therefore appropriate for basic use, but overlays are more powerful and easier to distribute.
</para>
</section>
<section xml:id="sec-overlays-alternatives">
<title>Using overlays to configure alternatives</title>
<para>
Certain software packages have different implementations of the
same interface. Other distributions have functionality to switch
between these. For example, Debian provides <link
xlink:href="https://wiki.debian.org/DebianAlternatives">DebianAlternatives</link>.
Nixpkgs has what we call <literal>alternatives</literal>, which
are configured through overlays.
</para>
<section xml:id="sec-overlays-alternatives-blas-lapack">
<title>BLAS/LAPACK</title>
<para>
In Nixpkgs, we have multiple implementations of the BLAS/LAPACK
numerical linear algebra interfaces. They are:
</para>
<itemizedlist>
<listitem>
<para>
<link xlink:href="https://www.openblas.net/">OpenBLAS</link>
</para>
<para>
The Nixpkgs attribute is <literal>openblas</literal> for
ILP64 (integer width = 64 bits) and
<literal>openblasCompat</literal> for LP64 (integer width =
32 bits). <literal>openblasCompat</literal> is the default.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="http://www.netlib.org/lapack/">LAPACK
reference</link> (also provides BLAS)
</para>
<para>
The Nixpkgs attribute is <literal>lapack-reference</literal>.
</para>
</listitem>
<listitem>
<para>
<link
xlink:href="https://software.intel.com/en-us/mkl">Intel
MKL</link> (only works on the x86_64 architecture, unfree)
</para>
<para>
The Nixpkgs attribute is <literal>mkl</literal>.
</para>
</listitem>
<listitem>
<para>
<link
xlink:href="https://developer.amd.com/amd-aocl/blas-library/">AMD
BLIS/LIBFLAME</link> (optimized for modern AMD x86_64 CPUs)
</para>
<para>
The AMD BLIS library, with attribute <literal>amd-blis</literal>,
provides a BLAS implementation. The complementary AMD LIBFLAME
library, with attribute <literal>amd-libflame</literal>, provides
a LAPACK implementation.
</para>
</listitem>
</itemizedlist>
<para>
Introduced in <link
xlink:href="https://github.com/NixOS/nixpkgs/pull/83888">PR
#83888</link>, we are able to override the <literal>blas</literal>
and <literal>lapack</literal> packages to use different implementations,
through the <literal>blasProvider</literal> and
<literal>lapackProvider</literal> argument. This can be used
to select a different provider. BLAS providers will have
symlinks in <literal>$out/lib/libblas.so.3</literal> and
<literal>$out/lib/libcblas.so.3</literal> to their respective
BLAS libraries. Likewise, LAPACK providers will have symlinks
in <literal>$out/lib/liblapack.so.3</literal> and
<literal>$out/lib/liblapacke.so.3</literal> to their respective
LAPACK libraries. For example, Intel MKL is both a BLAS and
LAPACK provider. An overlay can be created to use Intel MKL
that looks like:
</para>
<programlisting>
self: super:
{
blas = super.blas.override {
blasProvider = self.mkl;
}
lapack = super.lapack.override {
lapackProvider = self.mkl;
}
}
</programlisting>
<para>
This overlay uses Intel’s MKL library for both BLAS and LAPACK
interfaces. Note that the same can be accomplished at runtime
using <literal>LD_LIBRARY_PATH</literal> of
<literal>libblas.so.3</literal> and
<literal>liblapack.so.3</literal>. For instance:
</para>
<programlisting>
$ LD_LIBRARY_PATH=$(nix-build -A mkl)/lib:$LD_LIBRARY_PATH nix-shell -p octave --run octave
</programlisting>
<para>
Intel MKL requires an <literal>openmp</literal> implementation
when running with multiple processors. By default,
<literal>mkl</literal> will use Intel’s <literal>iomp</literal>
implementation if no other is specified, but this is a
runtime-only dependency and binary compatible with the LLVM
implementation. To use that one instead, Intel recommends users
set it with <literal>LD_PRELOAD</literal>. Note that
<literal>mkl</literal> is only available on
<literal>x86_64-linux</literal> and
<literal>x86_64-darwin</literal>. Moreover, Hydra is not
building and distributing pre-compiled binaries using it.
</para>
<para>
For BLAS/LAPACK switching to work correctly, all packages must
depend on <literal>blas</literal> or <literal>lapack</literal>.
This ensures that only one BLAS/LAPACK library is used at one
time. There are two versions versions of BLAS/LAPACK currently
in the wild, <literal>LP64</literal> (integer size = 32 bits)
and <literal>ILP64</literal> (integer size = 64 bits). Some
software needs special flags or patches to work with
<literal>ILP64</literal>. You can check if
<literal>ILP64</literal> is used in Nixpkgs with
<varname>blas.isILP64</varname> and
<varname>lapack.isILP64</varname>. Some software does NOT work
with <literal>ILP64</literal>, and derivations need to specify
an assertion to prevent this. You can prevent
<literal>ILP64</literal> from being used with the following:
</para>
<programlisting>
{ stdenv, blas, lapack, ... }:
assert (!blas.isILP64) &amp;&amp; (!lapack.isILP64);
stdenv.mkDerivation {
...
}
</programlisting>
</section>
</section>
</chapter>

19
flake.nix
View File

@ -1,8 +1,6 @@
# Experimental flake interface to Nixpkgs.
# See https://github.com/NixOS/rfcs/pull/49 for details.
{
edition = 201909;
description = "A collection of packages for the Nix package manager";
outputs = { self }:
@ -14,23 +12,30 @@
lib = import ./lib;
systems = [ "x86_64-linux" "i686-linux" "x86_64-darwin" "aarch64-linux" ];
systems = [
"x86_64-linux"
"i686-linux"
"x86_64-darwin"
"aarch64-linux"
"armv6l-linux"
"armv7l-linux"
];
forAllSystems = f: lib.genAttrs systems (system: f system);
in
{
lib = lib // {
lib = lib.extend (final: prev: {
nixosSystem = { modules, ... } @ args:
import ./nixos/lib/eval-config.nix (args // {
modules = modules ++
[ { system.nixos.versionSuffix =
".${lib.substring 0 8 self.lastModified}.${self.shortRev or "dirty"}";
system.nixos.revision = lib.mkIf (self ? rev) self.rev;
".${final.substring 0 8 (self.lastModifiedDate or self.lastModified)}.${self.shortRev or "dirty"}";
system.nixos.revision = final.mkIf (self ? rev) self.rev;
}
];
});
};
});
checks.x86_64-linux.tarball = jobs.tarball;

7
lib/attrsets.nix
View File

@ -4,7 +4,7 @@
let
inherit (builtins) head tail length;
inherit (lib.trivial) and;
inherit (lib.strings) concatStringsSep;
inherit (lib.strings) concatStringsSep sanitizeDerivationName;
inherit (lib.lists) fold concatMap concatLists;
in
@ -253,7 +253,7 @@ rec {
/* Like `mapAttrsRecursive', but it takes an additional predicate
function that tells it whether to recursive into an attribute
set. If it returns false, `mapAttrsRecursiveCond' does not
recurse, but does apply the map function. It is returns true, it
recurse, but does apply the map function. If it returns true, it
does recurse, and does not apply the map function.
Type:
@ -310,7 +310,7 @@ rec {
path' = builtins.storePath path;
res =
{ type = "derivation";
name = builtins.unsafeDiscardStringContext (builtins.substring 33 (-1) (baseNameOf path'));
name = sanitizeDerivationName (builtins.substring 33 (-1) (baseNameOf path'));
outPath = path';
outputs = [ "out" ];
out = res;
@ -469,6 +469,7 @@ rec {
getBin = getOutput "bin";
getLib = getOutput "lib";
getDev = getOutput "dev";
getMan = getOutput "man";
/* Pick the outputs of packages to place in buildInputs */
chooseDevOutputs = drvs: builtins.map getDev drvs;

6
lib/default.nix
View File

@ -67,7 +67,7 @@ let
inherit (trivial) id const pipe concat or and bitAnd bitOr bitXor
bitNot boolToString mergeAttrs flip mapNullable inNixShell min max
importJSON warn info showWarnings nixpkgsVersion version mod compare
splitByAndCompare functionArgs setFunctionArgs isFunction;
splitByAndCompare functionArgs setFunctionArgs isFunction toHexString toBaseDigits;
inherit (fixedPoints) fix fix' converge extends composeExtensions
makeExtensible makeExtensibleWithCustomName;
inherit (attrsets) attrByPath hasAttrByPath setAttrByPath
@ -77,7 +77,7 @@ let
genAttrs isDerivation toDerivation optionalAttrs
zipAttrsWithNames zipAttrsWith zipAttrs recursiveUpdateUntil
recursiveUpdate matchAttrs overrideExisting getOutput getBin
getLib getDev chooseDevOutputs zipWithNames zip
getLib getDev getMan chooseDevOutputs zipWithNames zip
recurseIntoAttrs dontRecurseIntoAttrs;
inherit (lists) singleton forEach foldr fold foldl foldl' imap0 imap1
concatMap flatten remove findSingle findFirst any all count
@ -141,7 +141,7 @@ let
mergeAttrsWithFunc mergeAttrsConcatenateValues
mergeAttrsNoOverride mergeAttrByFunc mergeAttrsByFuncDefaults
mergeAttrsByFuncDefaultsClean mergeAttrBy
fakeSha256 fakeSha512
fakeHash fakeSha256 fakeSha512
nixType imap;
inherit (versions)
splitVersion;

1
lib/deprecated.nix
View File

@ -272,6 +272,7 @@ rec {
imap = imap1;
# Fake hashes. Can be used as hash placeholders, when computing hash ahead isn't trivial
fakeHash = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
fakeSha256 = "0000000000000000000000000000000000000000000000000000000000000000";
fakeSha512 = "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000";
}

59
lib/generators.nix
View File

@ -48,8 +48,10 @@ rec {
else if isAttrs v then err "attrsets" v
# functions can’t be printed of course
else if isFunction v then err "functions" v
# let’s not talk about floats. There is no sensible `toString` for them.
else if isFloat v then err "floats" v
# Floats currently can't be converted to precise strings,
# condition warning on nix version once this isn't a problem anymore
# See https://github.com/NixOS/nix/pull/3480
else if isFloat v then libStr.floatToString v
else err "this value is" (toString v);
@ -126,6 +128,59 @@ rec {
# map input to ini sections
mapAttrsToStringsSep "\n" mkSection attrsOfAttrs;
/* Generate a git-config file from an attrset.
*
* It has two major differences from the regular INI format:
*
* 1. values are indented with tabs
* 2. sections can have sub-sections
*
* generators.toGitINI {
* url."ssh://git@github.com/".insteadOf = "https://github.com";
* user.name = "edolstra";
* }
*
*> [url "ssh://git@github.com/"]
*> insteadOf = https://github.com/
*>
*> [user]
*> name = edolstra
*/
toGitINI = attrs:
with builtins;
let
mkSectionName = name:
let
containsQuote = libStr.hasInfix ''"'' name;
sections = libStr.splitString "." name;
section = head sections;
subsections = tail sections;
subsection = concatStringsSep "." subsections;
in if containsQuote || subsections == [ ] then
name
else
''${section} "${subsection}"'';
# generation for multiple ini values
mkKeyValue = k: v:
let mkKeyValue = mkKeyValueDefault { } " = " k;
in concatStringsSep "\n" (map (kv: "\t" + mkKeyValue kv) (lib.toList v));
# converts { a.b.c = 5; } to { "a.b".c = 5; } for toINI
gitFlattenAttrs = let
recurse = path: value:
if isAttrs value then
lib.mapAttrsToList (name: value: recurse ([ name ] ++ path) value) value
else if length path > 1 then {
${concatStringsSep "." (lib.reverseList (tail path))}.${head path} = value;
} else {
${head path} = value;
};
in attrs: lib.foldl lib.recursiveUpdate { } (lib.flatten (recurse [ ] attrs));
toINI_ = toINI { inherit mkKeyValue mkSectionName; };
in
toINI_ (gitFlattenAttrs attrs);
/* Generates JSON from an arbitrary (non-function) value.
* For more information see the documentation of the builtin.

10
lib/kernel.nix
View File

@ -8,13 +8,13 @@ with lib;
option = x:
x // { optional = true; };
yes = { tristate = "y"; };
no = { tristate = "n"; };
module = { tristate = "m"; };
freeform = x: { freeform = x; };
yes = { tristate = "y"; optional = false; };
no = { tristate = "n"; optional = false; };
module = { tristate = "m"; optional = false; };
freeform = x: { freeform = x; optional = false; };
/*
Common patterns/legacy used in common-config/hardened-config.nix
Common patterns/legacy used in common-config/hardened/config.nix
*/
whenHelpers = version: {
whenAtLeast = ver: mkIf (versionAtLeast version ver);

147
lib/licenses.nix
View File

@ -2,7 +2,7 @@
let
spdx = lic: lic // {
url = "http://spdx.org/licenses/${lic.spdxId}.html";
url = "https://spdx.org/licenses/${lic.spdxId}.html";
};
in
@ -28,7 +28,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
fullName = "Academic Free License v3.0";
};
agpl3 = spdx {
agpl3Only = spdx {
spdxId = "AGPL-3.0-only";
fullName = "GNU Affero General Public License v3.0 only";
};
@ -40,13 +40,13 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
amazonsl = {
fullName = "Amazon Software License";
url = http://aws.amazon.com/asl/;
url = "https://aws.amazon.com/asl/";
free = false;
};
amd = {
fullName = "AMD License Agreement";
url = http://developer.amd.com/amd-license-agreement/;
url = "https://developer.amd.com/amd-license-agreement/";
free = false;
};
@ -57,7 +57,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
arphicpl = {
fullName = "Arphic Public License";
url = https://www.freedesktop.org/wiki/Arphic_Public_License/;
url = "https://www.freedesktop.org/wiki/Arphic_Public_License/";
};
artistic1 = spdx {
@ -85,6 +85,11 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
fullName = ''Beerware License'';
};
blueOak100 = spdx {
spdxId = "BlueOak-1.0.0";
fullName = "Blue Oak Model License 1.0.0";
};
bsd0 = spdx {
spdxId = "0BSD";
fullName = "BSD Zero Clause License";
@ -95,6 +100,11 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
fullName = ''BSD 2-clause "Simplified" License'';
};
bsd2Patent = spdx {
spdxId = "BSD-2-Clause-Patent";
fullName = ''BSD-2-Clause Plus Patent License'';
};
bsd3 = spdx {
spdxId = "BSD-3-Clause";
fullName = ''BSD 3-clause "New" or "Revised" License'';
@ -105,9 +115,14 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
fullName = ''BSD 4-clause "Original" or "Old" License'';
};
bsdProtection = spdx {
spdxId = "BSD-Protection";
fullName = "BSD Protection License";
};
bsl11 = {
fullName = "Business Source License 1.1";
url = https://mariadb.com/bsl11;
url = "https://mariadb.com/bsl11";
free = false;
};
@ -230,7 +245,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
eapl = {
fullName = "EPSON AVASYS PUBLIC LICENSE";
url = http://avasys.jp/hp/menu000000700/hpg000000603.htm;
url = "https://avasys.jp/hp/menu000000700/hpg000000603.htm";
free = false;
};
@ -246,7 +261,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
elastic = {
fullName = "ELASTIC LICENSE";
url = https://github.com/elastic/elasticsearch/blob/master/licenses/ELASTIC-LICENSE.txt;
url = "https://github.com/elastic/elasticsearch/blob/master/licenses/ELASTIC-LICENSE.txt";
free = false;
};
@ -262,7 +277,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
epson = {
fullName = "Seiko Epson Corporation Software License Agreement for Linux";
url = https://download.ebz.epson.net/dsc/du/02/eula/global/LINUX_EN.html;
url = "https://download.ebz.epson.net/dsc/du/02/eula/global/LINUX_EN.html";
free = false;
};
@ -271,7 +286,17 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
fullName = "European Union Public License 1.1";
};
fdl12 = spdx {
eupl12 = spdx {
spdxId = "EUPL-1.2";
fullName = "European Union Public License 1.2";
};
fdl11Only = spdx {
spdxId = "GFDL-1.1-only";
fullName = "GNU Free Documentation License v1.1 only";
};
fdl12Only = spdx {
spdxId = "GFDL-1.2-only";
fullName = "GNU Free Documentation License v1.2 only";
};
@ -281,7 +306,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
fullName = "GNU Free Documentation License v1.2 or later";
};
fdl13 = spdx {
fdl13Only = spdx {
spdxId = "GFDL-1.3-only";
fullName = "GNU Free Documentation License v1.3 only";
};
@ -293,7 +318,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
ffsl = {
fullName = "Floodgap Free Software License";
url = http://www.floodgap.com/software/ffsl/license.html;
url = "https://www.floodgap.com/software/ffsl/license.html";
free = false;
};
@ -303,16 +328,16 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
g4sl = {
fullName = "Geant4 Software License";
url = https://geant4.web.cern.ch/geant4/license/LICENSE.html;
url = "https://geant4.web.cern.ch/geant4/license/LICENSE.html";
};
geogebra = {
fullName = "GeoGebra Non-Commercial License Agreement";
url = https://www.geogebra.org/license;
url = "https://www.geogebra.org/license";
free = false;
};
gpl1 = spdx {
gpl1Only = spdx {
spdxId = "GPL-1.0-only";
fullName = "GNU General Public License v1.0 only";
};
@ -322,7 +347,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
fullName = "GNU General Public License v1.0 or later";
};
gpl2 = spdx {
gpl2Only = spdx {
spdxId = "GPL-2.0-only";
fullName = "GNU General Public License v2.0 only";
};
@ -334,12 +359,12 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
gpl2ClasspathPlus = {
fullName = "GNU General Public License v2.0 or later (with Classpath exception)";
url = https://fedoraproject.org/wiki/Licensing/GPL_Classpath_Exception;
url = "https://fedoraproject.org/wiki/Licensing/GPL_Classpath_Exception";
};
gpl2Oss = {
fullName = "GNU General Public License version 2 only (with OSI approved licenses linking exception)";
url = https://www.mysql.com/about/legal/licensing/foss-exception;
url = "https://www.mysql.com/about/legal/licensing/foss-exception";
};
gpl2Plus = spdx {
@ -347,7 +372,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
fullName = "GNU General Public License v2.0 or later";
};
gpl3 = spdx {
gpl3Only = spdx {
spdxId = "GPL-3.0-only";
fullName = "GNU General Public License v3.0 only";
};
@ -359,7 +384,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
gpl3ClasspathPlus = {
fullName = "GNU General Public License v3.0 or later (with Classpath exception)";
url = https://fedoraproject.org/wiki/Licensing/GPL_Classpath_Exception;
url = "https://fedoraproject.org/wiki/Licensing/GPL_Classpath_Exception";
};
hpnd = spdx {
@ -370,7 +395,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
# Intel's license, seems free
iasl = {
fullName = "iASL";
url = http://www.calculate-linux.org/packages/licenses/iASL;
url = "https://old.calculate-linux.org/packages/licenses/iASL";
};
ijg = spdx {
@ -385,13 +410,13 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
inria-compcert = {
fullName = "INRIA Non-Commercial License Agreement for the CompCert verified compiler";
url = "http://compcert.inria.fr/doc/LICENSE";
url = "http://compcert.inria.fr/doc/LICENSE"; # https is broken
free = false;
};
inria-icesl = {
fullName = "INRIA Non-Commercial License Agreement for IceSL";
url = "http://shapeforge.loria.fr/icesl/EULA_IceSL_binary.pdf";
url = "http://shapeforge.loria.fr/icesl/EULA_IceSL_binary.pdf"; # https is broken
free = false;
};
@ -413,7 +438,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
# Proprietary binaries; free to redistribute without modification.
issl = {
fullName = "Intel Simplified Software License";
url = https://software.intel.com/en-us/license/intel-simplified-software-license;
url = "https://software.intel.com/en-us/license/intel-simplified-software-license";
free = false;
};
@ -422,7 +447,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
fullName = "JasPer License";
};
lgpl2 = spdx {
lgpl2Only = spdx {
spdxId = "LGPL-2.0-only";
fullName = "GNU Library General Public License v2 only";
};
@ -432,7 +457,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
fullName = "GNU Library General Public License v2 or later";
};
lgpl21 = spdx {
lgpl21Only = spdx {
spdxId = "LGPL-2.1-only";
fullName = "GNU Lesser General Public License v2.1 only";
};
@ -442,7 +467,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
fullName = "GNU Lesser General Public License v2.1 or later";
};
lgpl3 = spdx {
lgpl3Only = spdx {
spdxId = "LGPL-3.0-only";
fullName = "GNU Lesser General Public License v3.0 only";
};
@ -452,6 +477,11 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
fullName = "GNU Lesser General Public License v3.0 or later";
};
lgpllr = spdx {
spdxId = "LGPLLR";
fullName = "Lesser General Public License For Linguistic Resources";
};
libpng = spdx {
spdxId = "Libpng";
fullName = "libpng License";
@ -469,7 +499,12 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
llgpl21 = {
fullName = "Lisp LGPL; GNU Lesser General Public License version 2.1 with Franz Inc. preamble for clarification of LGPL terms in context of Lisp";
url = http://opensource.franz.com/preamble.html;
url = "https://opensource.franz.com/preamble.html";
};
llvm-exception = spdx {
spdxId = "LLVM-exception";
fullName = "LLVM Exception"; # LLVM exceptions to the Apache 2.0 License
};
lppl12 = spdx {
@ -489,11 +524,11 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
miros = {
fullName = "MirOS License";
url = https://opensource.org/licenses/MirOS;
url = "https://opensource.org/licenses/MirOS";
};
# spdx.org does not (yet) differentiate between the X11 and Expat versions
# for details see http://en.wikipedia.org/wiki/MIT_License#Various_versions
# for details see https://en.wikipedia.org/wiki/MIT_License#Various_versions
mit = spdx {
spdxId = "MIT";
fullName = "MIT License";
@ -519,12 +554,6 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
fullName = "Microsoft Public License";
};
msrla = {
fullName = "Microsoft Research License Agreement";
url = "http://research.microsoft.com/en-us/projects/pex/msr-la.txt";
free = false;
};
nasa13 = spdx {
spdxId = "NASA-1.3";
fullName = "NASA Open Source Agreement 1.3";
@ -541,6 +570,12 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
fullName = "Non-Profit Open Software License 3.0";
};
obsidian = {
fullName = "Obsidian End User Agreement";
url = "https://obsidian.md/eula";
free = false;
};
ocamlpro_nc = {
fullName = "OCamlPro Non Commercial license version 1";
url = "https://alt-ergo.ocamlpro.com/http/alt-ergo-2.2.0/OCamlPro-Non-Commercial-License.pdf";
@ -589,14 +624,14 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
postman = {
fullName = "Postman EULA";
url = https://www.getpostman.com/licenses/postman_base_app;
url = "https://www.getpostman.com/licenses/postman_base_app";
free = false;
};
psfl = spdx {
spdxId = "Python-2.0";
fullName = "Python Software Foundation License version 2";
#url = http://docs.python.org/license.html;
url = "https://docs.python.org/license.html";
};
publicDomain = {
@ -605,7 +640,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
purdueBsd = {
fullName = " Purdue BSD-Style License"; # also know as lsof license
url = https://enterprise.dejacode.com/licenses/public/purdue-bsd;
url = "https://enterprise.dejacode.com/licenses/public/purdue-bsd";
};
qhull = spdx {
@ -620,7 +655,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
qwt = {
fullName = "Qwt License, Version 1.0";
url = http://qwt.sourceforge.net/qwtlicense.html;
url = "https://qwt.sourceforge.io/qwtlicense.html";
};
ruby = spdx {
@ -646,7 +681,14 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
smail = {
shortName = "smail";
fullName = "SMAIL General Public License";
url = http://metadata.ftp-master.debian.org/changelogs/main/d/debianutils/debianutils_4.8.1_copyright;
url = "https://sources.debian.org/copyright/license/debianutils/4.9.1/";
};
sspl = {
shortName = "SSPL";
fullName = "Server Side Public License";
url = "https://www.mongodb.com/licensing/server-side-public-license";
free = false;
};
tcltk = spdx {
@ -656,7 +698,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
ufl = {
fullName = "Ubuntu Font License 1.0";
url = http://font.ubuntu.com/ufl/ubuntu-font-licence-1.0.txt;
url = "https://ubuntu.com/legal/font-licence";
};
unfree = {
@ -675,6 +717,11 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
# channel and NixOS images.
};
unicode-dfs-2016 = spdx {
spdxId = "Unicode-DFS-2016";
fullName = "Unicode License Agreement - Data Files and Software (2016)";
};
unlicense = spdx {
spdxId = "Unlicense";
fullName = "The Unlicense";
@ -713,7 +760,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
wadalab = {
fullName = "Wadalab Font License";
url = https://fedoraproject.org/wiki/Licensing:Wadalab?rd=Licensing/Wadalab;
url = "https://fedoraproject.org/wiki/Licensing:Wadalab?rd=Licensing/Wadalab";
};
wtfpl = spdx {
@ -728,7 +775,7 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
xfig = {
fullName = "xfig";
url = "http://mcj.sourceforge.net/authors.html#xfig";
url = "http://mcj.sourceforge.net/authors.html#xfig"; # https is broken
};
zlib = spdx {
@ -745,4 +792,16 @@ lib.mapAttrs (n: v: v // { shortName = n; }) {
spdxId = "ZPL-2.1";
fullName = "Zope Public License 2.1";
};
} // {
# TODO: remove legacy aliases
agpl3 = lib.licenses.agpl3Only;
fdl11 = lib.licenses.fdl11Only;
fdl12 = lib.licenses.fdl12Only;
fdl13 = lib.licenses.fdl13Only;
gpl1 = lib.licenses.gpl1Only;
gpl2 = lib.licenses.gpl2Only;
gpl3 = lib.licenses.gpl3Only;
lgpl2 = lib.licenses.lgpl2Only;
lgpl21 = lib.licenses.lgpl21Only;
lgpl3 = lib.licenses.lgpl3Only;
}

4
lib/lists.nix
View File

@ -73,8 +73,8 @@ rec {
lconcat [ "a" "b" "c" ]
=> "zabc"
# different types
lstrange = foldl (str: int: str + toString (int + 1)) ""
strange [ 1 2 3 4 ]
lstrange = foldl (str: int: str + toString (int + 1)) "a"
lstrange [ 1 2 3 4 ]
=> "a2345"
*/
foldl = op: nul: list:

2
lib/minver.nix
View File

@ -1,2 +1,2 @@
# Expose the minimum required version for evaluating Nixpkgs
"2.0"
"2.2"

160
lib/modules.nix
View File

@ -58,6 +58,23 @@ rec {
default = check;
description = "Whether to check whether all option definitions have matching declarations.";
};
_module.freeformType = mkOption {
# Disallow merging for now, but could be implemented nicely with a `types.optionType`
type = types.nullOr (types.uniq types.attrs);
internal = true;
default = null;
description = ''
If set, merge all definitions that don't have an associated option
together using this type. The result then gets combined with the
values of all declared options to produce the final <literal>
config</literal> value.
If this is <literal>null</literal>, definitions without an option
will throw an error unless <option>_module.check</option> is
turned off.
'';
};
};
config = {
@ -65,35 +82,55 @@ rec {
};
};
collected = collectModules
(specialArgs.modulesPath or "")
(modules ++ [ internalModule ])
({ inherit config options lib; } // specialArgs);
merged =
let collected = collectModules
(specialArgs.modulesPath or "")
(modules ++ [ internalModule ])
({ inherit lib options config; } // specialArgs);
in mergeModules prefix (reverseList collected);
options = mergeModules prefix (reverseList collected);
options = merged.matchedOptions;
# Traverse options and extract the option values into the final
# config set. At the same time, check whether all option
# definitions have matching declarations.
# !!! _module.check's value can't depend on any other config values
# without an infinite recursion. One way around this is to make the
# 'config' passed around to the modules be unconditionally unchecked,
# and only do the check in 'result'.
config = yieldConfig prefix options;
yieldConfig = prefix: set:
let res = removeAttrs (mapAttrs (n: v:
if isOption v then v.value
else yieldConfig (prefix ++ [n]) v) set) ["_definedNames"];
in
if options._module.check.value && set ? _definedNames then
foldl' (res: m:
foldl' (res: name:
if set ? ${name} then res else throw "The option `${showOption (prefix ++ [name])}' defined in `${m.file}' does not exist.")
res m.names)
res set._definedNames
else
res;
result = {
config =
let
# For definitions that have an associated option
declaredConfig = mapAttrsRecursiveCond (v: ! isOption v) (_: v: v.value) options;
# If freeformType is set, this is for definitions that don't have an associated option
freeformConfig =
let
defs = map (def: {
file = def.file;
value = setAttrByPath def.prefix def.value;
}) merged.unmatchedDefns;
in if defs == [] then {}
else declaredConfig._module.freeformType.merge prefix defs;
in if declaredConfig._module.freeformType == null then declaredConfig
# Because all definitions that had an associated option ended in
# declaredConfig, freeformConfig can only contain the non-option
# paths, meaning recursiveUpdate will never override any value
else recursiveUpdate freeformConfig declaredConfig;
checkUnmatched =
if config._module.check && config._module.freeformType == null && merged.unmatchedDefns != [] then
let
firstDef = head merged.unmatchedDefns;
baseMsg = "The option `${showOption (prefix ++ firstDef.prefix)}' defined in `${firstDef.file}' does not exist.";
in
if attrNames options == [ "_module" ]
then throw ''
${baseMsg}
However there are no options defined in `${showOption prefix}'. Are you sure you've
declared your options properly? This can happen if you e.g. declared your options in `types.submodule'
under `config' rather than `options'.
''
else throw baseMsg
else null;
result = builtins.seq checkUnmatched {
inherit options;
config = removeAttrs config [ "_module" ];
inherit (config) _module;
@ -174,12 +211,16 @@ rec {
/* Massage a module into canonical form, that is, a set consisting
of ‘options’, ‘config’ and ‘imports’ attributes. */
unifyModuleSyntax = file: key: m:
let addMeta = config: if m ? meta
then mkMerge [ config { meta = m.meta; } ]
else config;
let
addMeta = config: if m ? meta
then mkMerge [ config { meta = m.meta; } ]
else config;
addFreeformType = config: if m ? freeformType
then mkMerge [ config { _module.freeformType = m.freeformType; } ]
else config;
in
if m ? config || m ? options then
let badAttrs = removeAttrs m ["_file" "key" "disabledModules" "imports" "options" "config" "meta"]; in
let badAttrs = removeAttrs m ["_file" "key" "disabledModules" "imports" "options" "config" "meta" "freeformType"]; in
if badAttrs != {} then
throw "Module `${key}' has an unsupported attribute `${head (attrNames badAttrs)}'. This is caused by introducing a top-level `config' or `options' attribute. Add configuration attributes immediately on the top level instead, or move all of them (namely: ${toString (attrNames badAttrs)}) into the explicit `config' attribute."
else
@ -188,7 +229,7 @@ rec {
disabledModules = m.disabledModules or [];
imports = m.imports or [];
options = m.options or {};
config = addMeta (m.config or {});
config = addFreeformType (addMeta (m.config or {}));
}
else
{ _file = m._file or file;
@ -196,7 +237,7 @@ rec {
disabledModules = m.disabledModules or [];
imports = m.require or [] ++ m.imports or [];
options = {};
config = addMeta (removeAttrs m ["_file" "key" "disabledModules" "require" "imports"]);
config = addFreeformType (addMeta (removeAttrs m ["_file" "key" "disabledModules" "require" "imports" "freeformType"]));
};
applyIfFunction = key: f: args@{ config, options, lib, ... }: if isFunction f then
@ -233,7 +274,23 @@ rec {
declarations in all modules, combining them into a single set.
At the same time, for each option declaration, it will merge the
corresponding option definitions in all machines, returning them
in the ‘value’ attribute of each option. */
in the ‘value’ attribute of each option.
This returns a set like
{
# A recursive set of options along with their final values
matchedOptions = {
foo = { _type = "option"; value = "option value of foo"; ... };
bar.baz = { _type = "option"; value = "option value of bar.baz"; ... };
...
};
# A list of definitions that weren't matched by any option
unmatchedDefns = [
{ file = "file.nix"; prefix = [ "qux" ]; value = "qux"; }
...
];
}
*/
mergeModules = prefix: modules:
mergeModules' prefix modules
(concatMap (m: map (config: { file = m._file; inherit config; }) (pushDownProperties m.config)) modules);
@ -280,9 +337,9 @@ rec {
defnsByName' = byName "config" (module: value:
[{ inherit (module) file; inherit value; }]
) configs;
in
(flip mapAttrs declsByName (name: decls:
# We're descending into attribute ‘name’.
resultsByName = flip mapAttrs declsByName (name: decls:
# We're descending into attribute ‘name’.
let
loc = prefix ++ [name];
defns = defnsByName.${name} or [];
@ -291,7 +348,10 @@ rec {
in
if nrOptions == length decls then
let opt = fixupOptionType loc (mergeOptionDecls loc decls);
in evalOptionValue loc opt defns'
in {
matchedOptions = evalOptionValue loc opt defns';
unmatchedDefns = [];
}
else if nrOptions != 0 then
let
firstOption = findFirst (m: isOption m.options) "" decls;
@ -299,9 +359,27 @@ rec {
in
throw "The option `${showOption loc}' in `${firstOption._file}' is a prefix of options in `${firstNonOption._file}'."
else
mergeModules' loc decls defns
))
// { _definedNames = map (m: { inherit (m) file; names = attrNames m.config; }) configs; };
mergeModules' loc decls defns);
matchedOptions = mapAttrs (n: v: v.matchedOptions) resultsByName;
# an attrset 'name' => list of unmatched definitions for 'name'
unmatchedDefnsByName =
# Propagate all unmatched definitions from nested option sets
mapAttrs (n: v: v.unmatchedDefns) resultsByName
# Plus the definitions for the current prefix that don't have a matching option
// removeAttrs defnsByName' (attrNames matchedOptions);
in {
inherit matchedOptions;
# Transforms unmatchedDefnsByName into a list of definitions
unmatchedDefns = concatLists (mapAttrsToList (name: defs:
map (def: def // {
# Set this so we know when the definition first left unmatched territory
prefix = [name] ++ (def.prefix or []);
}) defs
) unmatchedDefnsByName);
};
/* Merge multiple option declarations into a single declaration. In
general, there should be only one declaration of each option.

9
lib/options.nix
View File

@ -191,7 +191,14 @@ rec {
Example:
(showOption ["foo" "bar" "baz"]) == "foo.bar.baz"
(showOption ["foo" "bar.baz" "tux"]) == "foo.\"bar.baz\".tux"
(showOption ["foo" "bar.baz" "tux"]) == "foo.bar.baz.tux"
Placeholders will not be quoted as they are not actual values:
(showOption ["foo" "*" "bar"]) == "foo.*.bar"
(showOption ["foo" "<name>" "bar"]) == "foo.<name>.bar"
Unlike attributes, options can also start with numbers:
(showOption ["windowManager" "2bwm" "enable"]) == "windowManager.2bwm.enable"
*/
showOption = parts: let
escapeOptionPart = part:

17
lib/sources.nix
View File

@ -63,17 +63,14 @@ rec {
# https://nixos.org/nix/manual/#builtin-filterSource
#
# name: Optional name to use as part of the store path.
# This defaults `src.name` or otherwise `baseNameOf src`.
# We recommend setting `name` whenever `src` is syntactically `./.`.
# Otherwise, you depend on `./.`'s name in the parent directory,
# which can cause inconsistent names, defeating caching.
# This defaults to `src.name` or otherwise `"source"`.
#
cleanSourceWith = { filter ? _path: _type: true, src, name ? null }:
let
isFiltered = src ? _isLibCleanSourceWith;
origSrc = if isFiltered then src.origSrc else src;
filter' = if isFiltered then name: type: filter name type && src.filter name type else filter;
name' = if name != null then name else if isFiltered then src.name else baseNameOf src;
name' = if name != null then name else if isFiltered then src.name else "source";
in {
inherit origSrc;
filter = filter';
@ -148,10 +145,14 @@ rec {
# packed-refs file, so we have to grep through it:
then
let fileContent = readFile packedRefsName;
matchRef = match (".*\n([^\n ]*) " + file + "\n.*") fileContent;
in if matchRef == null
matchRef = builtins.match "([a-z0-9]+) ${file}";
isRef = s: builtins.isString s && (matchRef s) != null;
# there is a bug in libstdc++ leading to stackoverflow for long strings:
# https://github.com/NixOS/nix/issues/2147#issuecomment-659868795
refs = builtins.filter isRef (builtins.split "\n" fileContent);
in if refs == []
then throw ("Could not find " + file + " in " + packedRefsName)
else lib.head matchRef
else lib.head (matchRef (lib.head refs))
else throw ("Not a .git directory: " + path);
in readCommitFromFile "HEAD";

63
lib/strings.nix
View File

@ -315,6 +315,21 @@ rec {
*/
escapeNixString = s: escape ["$"] (builtins.toJSON s);
/* Quotes a string if it can't be used as an identifier directly.
Type: string -> string
Example:
escapeNixIdentifier "hello"
=> "hello"
escapeNixIdentifier "0abc"
=> "\"0abc\""
*/
escapeNixIdentifier = s:
# Regex from https://github.com/NixOS/nix/blob/d048577909e383439c2549e849c5c2f2016c997e/src/libexpr/lexer.l#L91
if builtins.match "[a-zA-Z_][a-zA-Z0-9_'-]*" s != null
then s else escapeNixString s;
# Obsolete - use replaceStrings instead.
replaceChars = builtins.replaceStrings or (
del: new: s:
@ -597,6 +612,22 @@ rec {
*/
fixedWidthNumber = width: n: fixedWidthString width "0" (toString n);
/* Convert a float to a string, but emit a warning when precision is lost
during the conversion
Example:
floatToString 0.000001
=> "0.000001"
floatToString 0.0000001
=> trace: warning: Imprecise conversion from float to string 0.000000
"0.000000"
*/
floatToString = float: let
result = toString float;
precise = float == builtins.fromJSON result;
in if precise then result
else lib.warn "Imprecise conversion from float to string ${result}" result;
/* Check whether a value can be coerced to a string */
isCoercibleToString = x:
builtins.elem (builtins.typeOf x) [ "path" "string" "null" "int" "float" "bool" ] ||
@ -678,4 +709,36 @@ rec {
=> "1.0"
*/
fileContents = file: removeSuffix "\n" (builtins.readFile file);
/* Creates a valid derivation name from a potentially invalid one.
Type: sanitizeDerivationName :: String -> String
Example:
sanitizeDerivationName "../hello.bar # foo"
=> "-hello.bar-foo"
sanitizeDerivationName ""
=> "unknown"
sanitizeDerivationName pkgs.hello
=> "-nix-store-2g75chlbpxlrqn15zlby2dfh8hr9qwbk-hello-2.10"
*/
sanitizeDerivationName = string: lib.pipe string [
# Get rid of string context. This is safe under the assumption that the
# resulting string is only used as a derivation name
builtins.unsafeDiscardStringContext
# Strip all leading "."
(x: builtins.elemAt (builtins.match "\\.*(.*)" x) 0)
# Split out all invalid characters
# https://github.com/NixOS/nix/blob/2.3.2/src/libstore/store-api.cc#L85-L112
# https://github.com/NixOS/nix/blob/2242be83c61788b9c0736a92bb0b5c7bbfc40803/nix-rust/src/store/path.rs#L100-L125
(builtins.split "[^[:alnum:]+._?=-]+")
# Replace invalid character ranges with a "-"
(concatMapStrings (s: if lib.isList s then "-" else s))
# Limit to 211 characters (minus 4 chars for ".drv")
(x: substring (lib.max (stringLength x - 207) 0) (-1) x)
# If the result is empty, replace it with "unknown"
(x: if stringLength x == 0 then "unknown" else x)
];
}

3
lib/systems/default.nix
View File

@ -32,6 +32,7 @@ rec {
/**/ if final.isDarwin then "libSystem"
else if final.isMinGW then "msvcrt"
else if final.isWasi then "wasilibc"
else if final.isRedox then "relibc"
else if final.isMusl then "musl"
else if final.isUClibc then "uclibc"
else if final.isAndroid then "bionic"
@ -65,6 +66,8 @@ rec {
freebsd = "FreeBSD";
openbsd = "OpenBSD";
wasi = "Wasi";
redox = "Redox";
genode = "Genode";
}.${final.parsed.kernel.name} or null;
# uname -p

6
lib/systems/doubles.nix
View File

@ -22,6 +22,8 @@ let
"wasm64-wasi" "wasm32-wasi"
"x86_64-redox"
"powerpc64le-linux"
"riscv32-linux" "riscv64-linux"
@ -35,6 +37,8 @@ let
"vc4-none"
"js-ghcjs"
"aarch64-genode" "i686-genode" "x86_64-genode"
];
allParsed = map parse.mkSystemFromString all;
@ -67,7 +71,9 @@ in {
openbsd = filterDoubles predicates.isOpenBSD;
unix = filterDoubles predicates.isUnix;
wasi = filterDoubles predicates.isWasi;
redox = filterDoubles predicates.isRedox;
windows = filterDoubles predicates.isWindows;
genode = filterDoubles predicates.isGenode;
embedded = filterDoubles predicates.isNone;

9
lib/systems/examples.nix
View File

@ -163,6 +163,15 @@ rec {
libc = "newlib";
};
#
# Redox
#
x86_64-unknown-redox = {
config = "x86_64-unknown-redox";
libc = "relibc";
};
#
# Darwin
#

4
lib/systems/inspect.nix
View File

@ -33,7 +33,7 @@ rec {
isBSD = { kernel = { families = { inherit (kernelFamilies) bsd; }; }; };
isDarwin = { kernel = { families = { inherit (kernelFamilies) darwin; }; }; };
isUnix = [ isBSD isDarwin isLinux isSunOS isCygwin ];
isUnix = [ isBSD isDarwin isLinux isSunOS isCygwin isRedox ];
isMacOS = { kernel = kernels.macos; };
isiOS = { kernel = kernels.ios; };
@ -46,7 +46,9 @@ rec {
isCygwin = { kernel = kernels.windows; abi = abis.cygnus; };
isMinGW = { kernel = kernels.windows; abi = abis.gnu; };
isWasi = { kernel = kernels.wasi; };
isRedox = { kernel = kernels.redox; };
isGhcjs = { kernel = kernels.ghcjs; };
isGenode = { kernel = kernels.genode; };
isNone = { kernel = kernels.none; };
isAndroid = [ { abi = abis.android; } { abi = abis.androideabi; } ];

6
lib/systems/parse.nix
View File

@ -277,8 +277,10 @@ rec {
openbsd = { execFormat = elf; families = { inherit bsd; }; };
solaris = { execFormat = elf; families = { }; };
wasi = { execFormat = wasm; families = { }; };
redox = { execFormat = elf; families = { }; };
windows = { execFormat = pe; families = { }; };
ghcjs = { execFormat = unknown; families = { }; };
genode = { execFormat = elf; families = { }; };
} // { # aliases
# 'darwin' is the kernel for all of them. We choose macOS by default.
darwin = kernels.macos;
@ -389,12 +391,16 @@ rec {
then { cpu = elemAt l 0; vendor = elemAt l 1; kernel = "windows"; }
else if (elemAt l 2 == "wasi")
then { cpu = elemAt l 0; vendor = elemAt l 1; kernel = "wasi"; }
else if (elemAt l 2 == "redox")
then { cpu = elemAt l 0; vendor = elemAt l 1; kernel = "redox"; }
else if hasPrefix "netbsd" (elemAt l 2)
then { cpu = elemAt l 0; vendor = elemAt l 1; kernel = elemAt l 2; }
else if (elem (elemAt l 2) ["eabi" "eabihf" "elf"])
then { cpu = elemAt l 0; vendor = "unknown"; kernel = elemAt l 1; abi = elemAt l 2; }
else if (elemAt l 2 == "ghcjs")
then { cpu = elemAt l 0; vendor = "unknown"; kernel = elemAt l 2; }
else if hasPrefix "genode" (elemAt l 2)
then { cpu = elemAt l 0; vendor = elemAt l 1; kernel = elemAt l 2; }
else throw "Target specification with 3 components is ambiguous";
"4" = { cpu = elemAt l 0; vendor = elemAt l 1; kernel = elemAt l 2; abi = elemAt l 3; };
}.${toString (length l)}

76
lib/tests/maintainers.nix Normal file
View File

@ -0,0 +1,76 @@
# to run these tests (and the others)
# nix-build nixpkgs/lib/tests/release.nix
{ # The pkgs used for dependencies for the testing itself
pkgs
, lib
}:
let
inherit (lib) types;
maintainerModule = { config, ... }: {
options = {
name = lib.mkOption {
type = types.str;
};
email = lib.mkOption {
type = types.str;
};
github = lib.mkOption {
type = types.nullOr types.str;
default = null;
};
githubId = lib.mkOption {
type = types.nullOr types.ints.unsigned;
default = null;
};
keys = lib.mkOption {
type = types.listOf (types.submodule {
options.longkeyid = lib.mkOption { type = types.str; };
options.fingerprint = lib.mkOption { type = types.str; };
});
default = [];
};
};
};
checkMaintainer = handle: uncheckedAttrs:
let
prefix = [ "lib" "maintainers" handle ];
checkedAttrs = (lib.modules.evalModules {
inherit prefix;
modules = [
maintainerModule
{
_file = toString ../../maintainers/maintainer-list.nix;
config = uncheckedAttrs;
}
];
}).config;
checkGithubId = lib.optional (checkedAttrs.github != null && checkedAttrs.githubId == null) ''
echo ${lib.escapeShellArg (lib.showOption prefix)}': If `github` is specified, `githubId` must be too.'
# Calling this too often would hit non-authenticated API limits, but this
# shouldn't happen since such errors will get fixed rather quickly
info=$(curl -sS https://api.github.com/users/${checkedAttrs.github})
id=$(jq -r '.id' <<< "$info")
echo "The GitHub ID for GitHub user ${checkedAttrs.github} is $id:"
echo -e " githubId = $id;\n"
'';
in lib.deepSeq checkedAttrs checkGithubId;
missingGithubIds = lib.concatLists (lib.mapAttrsToList checkMaintainer lib.maintainers);
success = pkgs.runCommandNoCC "checked-maintainers-success" {} ">$out";
failure = pkgs.runCommandNoCC "checked-maintainers-failure" {
nativeBuildInputs = [ pkgs.curl pkgs.jq ];
outputHash = "sha256:${lib.fakeSha256}";
outputHAlgo = "sha256";
outputHashMode = "flat";
SSL_CERT_FILE = "${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt";
} ''
${lib.concatStringsSep "\n" missingGithubIds}
exit 1
'';
in if missingGithubIds == [] then success else failure

52
lib/tests/misc.nix
View File

@ -3,6 +3,23 @@
# if the resulting list is empty, all tests passed
with import ../default.nix;
let
testSanitizeDerivationName = { name, expected }:
let
drv = derivation {
name = strings.sanitizeDerivationName name;
builder = "x";
system = "x";
};
in {
# Evaluate the derivation so an invalid name would be caught
expr = builtins.seq drv.drvPath drv.name;
inherit expected;
};
in
runTests {
@ -85,6 +102,16 @@ runTests {
expected = 9;
};
testToHexString = {
expr = toHexString 250;
expected = "FA";
};
testToBaseDigits = {
expr = toBaseDigits 2 6;
expected = [ 1 1 0 ];
};
# STRINGS
testConcatMapStrings = {
@ -490,4 +517,29 @@ runTests {
expected = "'-X' 'PUT' '--data' '{\"id\":0}' '--retry' '3' '--url' 'https://example.com/foo' '--url' 'https://example.com/bar' '--verbose'";
};
testSanitizeDerivationNameLeadingDots = testSanitizeDerivationName {
name = "..foo";
expected = "foo";
};
testSanitizeDerivationNameAscii = testSanitizeDerivationName {
name = " !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~";
expected = "-+--.-0123456789-=-?-ABCDEFGHIJKLMNOPQRSTUVWXYZ-_-abcdefghijklmnopqrstuvwxyz-";
};
testSanitizeDerivationNameTooLong = testSanitizeDerivationName {
name = "This string is loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong";
expected = "loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong";
};
testSanitizeDerivationNameTooLongWithInvalid = testSanitizeDerivationName {
name = "Hello there aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa &&&&&&&&";
expected = "there-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-";
};
testSanitizeDerivationNameEmpty = testSanitizeDerivationName {
name = "";
expected = "unknown";
};
}

28
lib/tests/modules.sh
View File

@ -3,7 +3,10 @@
# This script is used to test that the module system is working as expected.
# By default it test the version of nixpkgs which is defined in the NIX_PATH.
cd ./modules
# https://stackoverflow.com/a/246128/6605742
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
cd "$DIR"/modules
pass=0
fail=0
@ -207,6 +210,29 @@ checkConfigOutput "empty" config.value.foo ./declare-lazyAttrsOf.nix ./attrsOf-c
checkConfigError 'The option value .* in .* is not of type .*' \
config.value ./declare-int-unsigned-value.nix ./define-value-list.nix ./define-value-int-positive.nix
## Freeform modules
# Assigning without a declared option should work
checkConfigOutput 24 config.value ./freeform-attrsOf.nix ./define-value-string.nix
# No freeform assigments shouldn't make it error
checkConfigOutput '{ }' config ./freeform-attrsOf.nix
# but only if the type matches
checkConfigError 'The option value .* in .* is not of type .*' config.value ./freeform-attrsOf.nix ./define-value-list.nix
# and properties should be applied
checkConfigOutput yes config.value ./freeform-attrsOf.nix ./define-value-string-properties.nix
# Options should still be declarable, and be able to have a type that doesn't match the freeform type
checkConfigOutput false config.enable ./freeform-attrsOf.nix ./define-value-string.nix ./declare-enable.nix
checkConfigOutput 24 config.value ./freeform-attrsOf.nix ./define-value-string.nix ./declare-enable.nix
# and this should work too with nested values
checkConfigOutput false config.nest.foo ./freeform-attrsOf.nix ./freeform-nested.nix
checkConfigOutput bar config.nest.bar ./freeform-attrsOf.nix ./freeform-nested.nix
# Check whether a declared option can depend on an freeform-typed one
checkConfigOutput null config.foo ./freeform-attrsOf.nix ./freeform-str-dep-unstr.nix
checkConfigOutput 24 config.foo ./freeform-attrsOf.nix ./freeform-str-dep-unstr.nix ./define-value-string.nix
# Check whether an freeform-typed value can depend on a declared option, this can only work with lazyAttrsOf
checkConfigError 'infinite recursion encountered' config.foo ./freeform-attrsOf.nix ./freeform-unstr-dep-str.nix
checkConfigError 'The option .* is used but not defined' config.foo ./freeform-lazyAttrsOf.nix ./freeform-unstr-dep-str.nix
checkConfigOutput 24 config.foo ./freeform-lazyAttrsOf.nix ./freeform-unstr-dep-str.nix ./define-value-string.nix
cat <<EOF
====== module tests ======
$pass Pass

2
lib/tests/modules/define-option-dependently.nix
View File

@ -7,7 +7,7 @@
# Always defined, but the value depends on the presence of an option.
config = {
value = if options ? enable then 360 else 7;
}
}
# Only define if possible.
// lib.optionalAttrs (options ? enable) {
enable = true;

12
lib/tests/modules/define-value-string-properties.nix Normal file
View File

@ -0,0 +1,12 @@
{ lib, ... }: {
imports = [{
value = lib.mkDefault "def";
}];
value = lib.mkMerge [
(lib.mkIf false "nope")
"yes"
];
}

3
lib/tests/modules/freeform-attrsOf.nix Normal file
View File

@ -0,0 +1,3 @@
{ lib, ... }: {
freeformType = with lib.types; attrsOf (either str (attrsOf str));
}

3
lib/tests/modules/freeform-lazyAttrsOf.nix Normal file
View File

@ -0,0 +1,3 @@
{ lib, ... }: {
freeformType = with lib.types; lazyAttrsOf (either str (lazyAttrsOf str));
}

7
lib/tests/modules/freeform-nested.nix Normal file
View File

@ -0,0 +1,7 @@
{ lib, ... }: {
options.nest.foo = lib.mkOption {
type = lib.types.bool;
default = false;
};
config.nest.bar = "bar";
}

8
lib/tests/modules/freeform-str-dep-unstr.nix Normal file
View File

@ -0,0 +1,8 @@
{ lib, config, ... }: {
options.foo = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
};
config.foo = lib.mkIf (config ? value) config.value;
}

8
lib/tests/modules/freeform-unstr-dep-str.nix Normal file
View File

@ -0,0 +1,8 @@
{ lib, config, ... }: {
options.value = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
};
config.foo = lib.mkIf (config.value != null) config.value;
}

20
lib/tests/release.nix
View File

@ -1,14 +1,22 @@
{ pkgs ? import ((import ../.).cleanSource ../..) {} }:
{ # The pkgs used for dependencies for the testing itself
# Don't test properties of pkgs.lib, but rather the lib in the parent directory
pkgs ? import ../.. {} // { lib = throw "pkgs.lib accessed, but the lib tests should use nixpkgs' lib path directly!"; }
}:
pkgs.runCommandNoCC "nixpkgs-lib-tests" {
buildInputs = [ pkgs.nix (import ./check-eval.nix) ];
NIX_PATH = "nixpkgs=${toString pkgs.path}";
buildInputs = [
pkgs.nix
(import ./check-eval.nix)
(import ./maintainers.nix {
inherit pkgs;
lib = import ../.;
})
];
} ''
datadir="${pkgs.nix}/share"
export TEST_ROOT=$(pwd)/test-tmp
export NIX_BUILD_HOOK=
export NIX_CONF_DIR=$TEST_ROOT/etc
export NIX_DB_DIR=$TEST_ROOT/db
export NIX_LOCALSTATE_DIR=$TEST_ROOT/var
export NIX_LOG_DIR=$TEST_ROOT/var/log/nix
export NIX_STATE_DIR=$TEST_ROOT/var/nix
@ -17,8 +25,8 @@ pkgs.runCommandNoCC "nixpkgs-lib-tests" {
cacheDir=$TEST_ROOT/binary-cache
nix-store --init
cd ${pkgs.path}/lib/tests
bash ./modules.sh
cp -r ${../.} lib
bash lib/tests/modules.sh
touch $out
''

10
lib/tests/systems.nix
View File

@ -12,21 +12,23 @@ let
expected = lib.sort lib.lessThan y;
};
in with lib.systems.doubles; lib.runTests {
testall = mseteq all (linux ++ darwin ++ freebsd ++ openbsd ++ netbsd ++ illumos ++ wasi ++ windows ++ embedded ++ js);
testall = mseteq all (linux ++ darwin ++ freebsd ++ openbsd ++ netbsd ++ illumos ++ wasi ++ windows ++ embedded ++ js ++ genode ++ redox);
testarm = mseteq arm [ "armv5tel-linux" "armv6l-linux" "armv6l-none" "armv7a-linux" "armv7l-linux" "arm-none" "armv7a-darwin" ];
testi686 = mseteq i686 [ "i686-linux" "i686-freebsd" "i686-netbsd" "i686-openbsd" "i686-cygwin" "i686-windows" "i686-none" "i686-darwin" ];
testi686 = mseteq i686 [ "i686-linux" "i686-freebsd" "i686-genode" "i686-netbsd" "i686-openbsd" "i686-cygwin" "i686-windows" "i686-none" "i686-darwin" ];
testmips = mseteq mips [ "mipsel-linux" ];
testx86_64 = mseteq x86_64 [ "x86_64-linux" "x86_64-darwin" "x86_64-freebsd" "x86_64-openbsd" "x86_64-netbsd" "x86_64-cygwin" "x86_64-solaris" "x86_64-windows" "x86_64-none" ];
testx86_64 = mseteq x86_64 [ "x86_64-linux" "x86_64-darwin" "x86_64-freebsd" "x86_64-genode" "x86_64-redox" "x86_64-openbsd" "x86_64-netbsd" "x86_64-cygwin" "x86_64-solaris" "x86_64-windows" "x86_64-none" ];
testcygwin = mseteq cygwin [ "i686-cygwin" "x86_64-cygwin" ];
testdarwin = mseteq darwin [ "x86_64-darwin" "i686-darwin" "aarch64-darwin" "armv7a-darwin" ];
testfreebsd = mseteq freebsd [ "i686-freebsd" "x86_64-freebsd" ];
testgenode = mseteq genode [ "aarch64-genode" "i686-genode" "x86_64-genode" ];
testredox = mseteq redox [ "x86_64-redox" ];
testgnu = mseteq gnu (linux /* ++ kfreebsd ++ ... */);
testillumos = mseteq illumos [ "x86_64-solaris" ];
testlinux = mseteq linux [ "aarch64-linux" "armv5tel-linux" "armv6l-linux" "armv7a-linux" "armv7l-linux" "i686-linux" "mipsel-linux" "riscv32-linux" "riscv64-linux" "x86_64-linux" "powerpc64le-linux" ];
testnetbsd = mseteq netbsd [ "i686-netbsd" "x86_64-netbsd" ];
testopenbsd = mseteq openbsd [ "i686-openbsd" "x86_64-openbsd" ];
testwindows = mseteq windows [ "i686-cygwin" "x86_64-cygwin" "i686-windows" "x86_64-windows" ];
testunix = mseteq unix (linux ++ darwin ++ freebsd ++ openbsd ++ netbsd ++ illumos ++ cygwin);
testunix = mseteq unix (linux ++ darwin ++ freebsd ++ openbsd ++ netbsd ++ illumos ++ cygwin ++ redox);
}

51
lib/trivial.nix
View File

@ -332,4 +332,55 @@ rec {
*/
isFunction = f: builtins.isFunction f ||
(f ? __functor && isFunction (f.__functor f));
/* Convert the given positive integer to a string of its hexadecimal
representation. For example:
toHexString 0 => "0"
toHexString 16 => "10"
toHexString 250 => "FA"
*/
toHexString = i:
let
toHexDigit = d:
if d < 10
then toString d
else
{
"10" = "A";
"11" = "B";
"12" = "C";
"13" = "D";
"14" = "E";
"15" = "F";
}.${toString d};
in
lib.concatMapStrings toHexDigit (toBaseDigits 16 i);
/* `toBaseDigits base i` converts the positive integer i to a list of its
digits in the given base. For example:
toBaseDigits 10 123 => [ 1 2 3 ]
toBaseDigits 2 6 => [ 1 1 0 ]
toBaseDigits 16 250 => [ 15 10 ]
*/
toBaseDigits = base: i:
let
go = i:
if i < base
then [i]
else
let
r = i - ((i / base) * base);
q = (i - r) / base;
in
[r] ++ go q;
in
assert (base >= 2);
assert (i >= 0);
lib.reverseList (go i);
}

6
lib/types.nix
View File

@ -486,9 +486,15 @@ rec {
else value
) defs;
freeformType = (evalModules {
inherit modules specialArgs;
args.name = "‹name›";
})._module.freeformType;
in
mkOptionType rec {
name = "submodule";
description = freeformType.description or name;
check = x: isAttrs x || isFunction x || path.check x;
merge = loc: defs:
(evalModules {

1330
maintainers/maintainer-list.nix
View File

File diff suppressed because it is too large Load Diff

43
maintainers/scripts/build.nix Normal file
View File

@ -0,0 +1,43 @@
{ maintainer }:
# based on update.nix
# nix-build build.nix --argstr maintainer <yourname>
let
pkgs = import ./../../default.nix {};
maintainer_ = pkgs.lib.maintainers.${maintainer};
packagesWith = cond: return: set:
(pkgs.lib.flatten
(pkgs.lib.mapAttrsToList
(name: pkg:
let
result = builtins.tryEval
(
if pkgs.lib.isDerivation pkg && cond name pkg
then [ (return name pkg) ]
else if pkg.recurseForDerivations or false || pkg.recurseForRelease or false
then packagesWith cond return pkg
else [ ]
);
in
if result.success then result.value
else [ ]
)
set
)
);
in
packagesWith
(name: pkg:
(
if builtins.hasAttr "maintainers" pkg.meta
then (
if builtins.isList pkg.meta.maintainers
then builtins.elem maintainer_ pkg.meta.maintainers
else maintainer_ == pkg.meta.maintainers
)
else false
)
)
(name: pkg: pkg)
pkgs

2
maintainers/scripts/debian-patches.sh
View File

@ -2,7 +2,7 @@
# Download patches from debian project
# Usage $0 debian-patches.txt debian-patches.nix
# An example input and output files can be found in applications/graphics/xara/
# An example input and output files can be found in tools/graphics/plotutils
DEB_URL=https://sources.debian.org/data/main
declare -a deb_patches

2
maintainers/scripts/hydra-eval-failures.py
View File

@ -79,7 +79,7 @@ def cli(jobset):
and print a summary of failed builds
"""
url = "http://hydra.nixos.org/jobset/{}".format(jobset)
url = "https://hydra.nixos.org/jobset/{}".format(jobset)
# get the last evaluation
click.echo(click.style(

10
maintainers/scripts/luarocks-packages.csv
View File

@ -12,12 +12,13 @@ compat53,,,,,vcunat
cosmo,,,,,marsam
coxpcall,,,1.17.0-1,,
cqueues,,,,,vcunat
cyrussasl,,,,,vcunat
cyrussasl,,,,,
digestif,,,,lua5_3,
dkjson,,,,,
fifo,,,,,
http,,,,,vcunat
inspect,,,,,
ldbus,,http://luarocks.org/dev,,,
ldoc,,,,,
lgi,,,,,
linenoise,,,,,
@ -50,9 +51,9 @@ luaepnf,,,,,
luaevent,,,,,
luaexpat,,,1.3.0-1,,arobyn flosse
luaffi,,http://luarocks.org/dev,,,
luafilesystem,,,1.7.0-2,,flosse vcunat
luafilesystem,,,1.7.0-2,,flosse
lualogging,,,,,
luaossl,,,,lua5_1,vcunat
luaossl,,,,lua5_1,
luaposix,,,,,vyp lblasc
luarepl,,,,,
luasec,,,,,flosse
@ -65,6 +66,7 @@ luazip,,,,,
lua-yajl,,,,,pstn
luuid,,,,,
luv,,,,,
lyaml,,,,,lblasc
markdown,,,,,
mediator_lua,,,,,
mpack,,,,,
@ -72,9 +74,9 @@ moonscript,,,,,arobyn
nvim-client,,,,,
penlight,,,,,
rapidjson,,,,,
readline,,,,,
say,,,,,
std__debug,std._debug,,,,
std_normalize,std.normalize,,,,
stdlib,,,,,vyp
pulseaudio,,,,,doronbehar
vstruct,,,,,

1 # nix name luarocks name server version luaversion maintainers
12 cosmo marsam
13 coxpcall 1.17.0-1
14 cqueues vcunat
15 cyrussasl vcunat
16 digestif lua5_3
17 dkjson
18 fifo
19 http vcunat
20 inspect
21 ldbus http://luarocks.org/dev
22 ldoc
23 lgi
24 linenoise
51 luaevent
52 luaexpat 1.3.0-1 arobyn flosse
53 luaffi http://luarocks.org/dev
54 luafilesystem 1.7.0-2 flosse vcunat flosse
55 lualogging
56 luaossl lua5_1 vcunat
57 luaposix vyp lblasc
58 luarepl
59 luasec flosse
66 lua-yajl pstn
67 luuid
68 luv
69 lyaml lblasc
70 markdown
71 mediator_lua
72 mpack
74 nvim-client
75 penlight
76 rapidjson
77 readline
78 say
79 std__debug std._debug
80 std_normalize std.normalize
81 stdlib vyp
pulseaudio doronbehar
82 vstruct

3
maintainers/scripts/rebuild-amount.sh
View File

@ -60,7 +60,7 @@ nixexpr() {
"darwin-tested" "unstable" "stdenvBootstrapTools"
"moduleSystem" "lib-tests" # these just confuse the output
];
in
tweak (builtins.removeAttrs hydraJobs blacklist)
EONIX
@ -124,4 +124,3 @@ if [ -n "$optPrint" ]; then
echo
cat "$newlist"
fi

85
maintainers/scripts/update.nix
View File

@ -9,6 +9,16 @@
# TODO: add assert statements
let
pkgs = import ./../../default.nix (
if include-overlays == false then
{ overlays = []; }
else if include-overlays == true then
{ } # Let Nixpkgs include overlays impurely.
else { overlays = include-overlays; }
);
inherit (pkgs) lib;
/* Remove duplicate elements from the list based on some extracted value. O(n^2) complexity.
*/
nubOn = f: list:
@ -16,43 +26,44 @@ let
[]
else
let
x = pkgs.lib.head list;
xs = pkgs.lib.filter (p: f x != f p) (pkgs.lib.drop 1 list);
x = lib.head list;
xs = lib.filter (p: f x != f p) (lib.drop 1 list);
in
[x] ++ nubOn f xs;
pkgs = import ./../../default.nix (if include-overlays then { } else { overlays = []; });
packagesWithPath = relativePath: cond: return: pathContent:
let
result = builtins.tryEval pathContent;
packagesWith = cond: return: set:
nubOn (pkg: pkg.updateScript)
(pkgs.lib.flatten
(pkgs.lib.mapAttrsToList
(name: pkg:
let
result = builtins.tryEval (
if pkgs.lib.isDerivation pkg && cond name pkg
then [(return name pkg)]
else if pkg.recurseForDerivations or false || pkg.recurseForRelease or false
then packagesWith cond return pkg
else []
);
in
if result.success then result.value
else []
)
set
)
);
dedupResults = lst: nubOn (pkg: pkg.updateScript) (lib.concatLists lst);
in
if result.success then
let
pathContent = result.value;
in
if lib.isDerivation pathContent then
lib.optional (cond relativePath pathContent) (return relativePath pathContent)
else if lib.isAttrs pathContent then
# If user explicitly points to an attrSet or it is marked for recursion, we recur.
if relativePath == [] || pathContent.recurseForDerivations or false || pathContent.recurseForRelease or false then
dedupResults (lib.mapAttrsToList (name: elem: packagesWithPath (relativePath ++ [name]) cond return elem) pathContent)
else []
else if lib.isList pathContent then
dedupResults (lib.imap0 (i: elem: packagesWithPath (relativePath ++ [i]) cond return elem) pathContent)
else []
else [];
packagesWith = packagesWithPath [];
packagesWithUpdateScriptAndMaintainer = maintainer':
let
maintainer =
if ! builtins.hasAttr maintainer' pkgs.lib.maintainers then
if ! builtins.hasAttr maintainer' lib.maintainers then
builtins.throw "Maintainer with name `${maintainer'} does not exist in `maintainers/maintainer-list.nix`."
else
builtins.getAttr maintainer' pkgs.lib.maintainers;
builtins.getAttr maintainer' lib.maintainers;
in
packagesWith (name: pkg: builtins.hasAttr "updateScript" pkg &&
packagesWith (relativePath: pkg: builtins.hasAttr "updateScript" pkg &&
(if builtins.hasAttr "maintainers" pkg.meta
then (if builtins.isList pkg.meta.maintainers
then builtins.elem maintainer pkg.meta.maintainers
@ -61,23 +72,23 @@ let
else false
)
)
(name: pkg: pkg)
(relativePath: pkg: pkg)
pkgs;
packagesWithUpdateScript = path:
let
attrSet = pkgs.lib.attrByPath (pkgs.lib.splitString "." path) null pkgs;
pathContent = lib.attrByPath (lib.splitString "." path) null pkgs;
in
if attrSet == null then
if pathContent == null then
builtins.throw "Attribute path `${path}` does not exists."
else
packagesWith (name: pkg: builtins.hasAttr "updateScript" pkg)
(name: pkg: pkg)
attrSet;
packagesWith (relativePath: pkg: builtins.hasAttr "updateScript" pkg)
(relativePath: pkg: pkg)
pathContent;
packageByName = name:
let
package = pkgs.lib.attrByPath (pkgs.lib.splitString "." name) null pkgs;
package = lib.attrByPath (lib.splitString "." name) null pkgs;
in
if package == null then
builtins.throw "Package with an attribute name `${name}` does not exists."
@ -125,15 +136,15 @@ let
packageData = package: {
name = package.name;
pname = pkgs.lib.getName package;
updateScript = map builtins.toString (pkgs.lib.toList package.updateScript);
pname = lib.getName package;
updateScript = map builtins.toString (lib.toList package.updateScript);
};
packagesJson = pkgs.writeText "packages.json" (builtins.toJSON (map packageData packages));
optionalArgs =
pkgs.lib.optional (max-workers != null) "--max-workers=${max-workers}"
++ pkgs.lib.optional (keep-going == "true") "--keep-going";
lib.optional (max-workers != null) "--max-workers=${max-workers}"
++ lib.optional (keep-going == "true") "--keep-going";
args = [ packagesJson ] ++ optionalArgs;

71
maintainers/team-list.nix
View File

@ -17,17 +17,88 @@
{ lib }:
with lib.maintainers; {
acme = {
members = [
aanderse
andrew-d
arianvp
emily
flokli
m1cr0man
];
scope = "Maintain ACME-related packages and modules.";
};
freedesktop = {
members = [ jtojnar worldofpeace ];
scope = "Maintain Freedesktop.org packages for graphical desktop.";
};
golang = {
members = [
c00w
cstrahan
Frostman
kalbasit
mdlayher
mic92
orivej
rvolosatovs
zowoq
];
scope = "Maintain Golang compilers.";
};
gnome = {
members = [
hedning
jtojnar
worldofpeace
dasj19
];
scope = "Maintain GNOME desktop environment and platform.";
};
jitsi = {
members = [
mmilata
petabyteboy
prusnak
ryantm
];
scope = "Maintain Jitsi.";
};
matrix = {
members = [
ma27
pacien
fadenb
mguentner
ekleog
ralith
];
scope = "Maintain the ecosystem around Matrix, a decentralized messenger.";
};
php = {
members = [
aanderse
etu
globin
ma27
talyz
];
scope = "Maintain PHP related packages and extensions.";
};
podman = {
members = [
adisbladis
saschagrunert
vdemeester
zowoq
];
scope = "Maintain Podman and CRI-O related packages and modules.";
};
}

2
nixos/README
View File

@ -2,4 +2,4 @@
NixOS is a Linux distribution based on the purely functional package
management system Nix. More information can be found at
http://nixos.org/nixos and in the manual in doc/manual.
https://nixos.org/nixos and in the manual in doc/manual.

42
nixos/doc/manual/administration/boot-problems.xml
View File

@ -19,9 +19,9 @@
</term>
<listitem>
<para>
Start a root shell if something goes wrong in stage 1 of the boot process
(the initial ramdisk). This is disabled by default because there is no
authentication for the root shell.
Allows the user to start a root shell if something goes wrong in stage 1
of the boot process (the initial ramdisk). This is disabled by default
because there is no authentication for the root shell.
</para>
</listitem>
</varlistentry>
@ -38,6 +38,33 @@
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>boot.debug1devices</literal>
</term>
<listitem>
<para>
Like <literal>boot.debug1</literal>, but runs stage1 until kernel modules are loaded and device nodes are created.
This may help with e.g. making the keyboard work.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>boot.debug1mounts</literal>
</term>
<listitem>
<para>
Like <literal>boot.debug1</literal> or
<literal>boot.debug1devices</literal>, but runs stage1 until all
filesystems that are mounted during initrd are mounted (see
<option><link linkend="opt-fileSystems._name__.neededForBoot">neededForBoot</link></option>
). As a motivating example, this could be useful if you've forgotten to set
<option><link linkend="opt-fileSystems._name__.neededForBoot">neededForBoot</link></option>
on a file system.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>boot.trace</literal>
@ -79,6 +106,15 @@
<manvolnum>1</manvolnum></citerefentry>.
</para>
<para>
Notice that for <literal>boot.shell_on_fail</literal>,
<literal>boot.debug1</literal>, <literal>boot.debug1devices</literal>, and
<literal>boot.debug1mounts</literal>, if you did <emphasis>not</emphasis>
select "start the new shell as pid 1", and you <literal>exit</literal> from
the new shell, boot will proceed normally from the point where it failed, as
if you'd chosen "ignore the error and continue".
</para>
<para>
If no login prompts or X11 login screens appear (e.g. due to hanging
dependencies), you can press Alt+ArrowUp. If you’re lucky, this will start

2
nixos/doc/manual/configuration/adding-custom-packages.xml
View File

@ -11,7 +11,7 @@
the package to your clone, and (optionally) submit a patch or pull request to
have it accepted into the main Nixpkgs repository. This is described in
detail in the <link
xlink:href="http://nixos.org/nixpkgs/manual">Nixpkgs
xlink:href="https://nixos.org/nixpkgs/manual">Nixpkgs
manual</link>. In short, you clone Nixpkgs:
<screen>
<prompt>$ </prompt>git clone https://github.com/NixOS/nixpkgs

2
nixos/doc/manual/configuration/config-syntax.xml
View File

@ -14,7 +14,7 @@
when managing complex systems. The syntax and semantics of the Nix language
are fully described in the
<link
xlink:href="http://nixos.org/nix/manual/#chap-writing-nix-expressions">Nix
xlink:href="https://nixos.org/nix/manual/#chap-writing-nix-expressions">Nix
manual</link>, but here we give a short overview of the most important
constructs useful in NixOS configuration files.
</para>

1
nixos/doc/manual/configuration/configuration.xml
View File

@ -18,6 +18,7 @@
<xi:include href="user-mgmt.xml" />
<xi:include href="file-systems.xml" />
<xi:include href="x-windows.xml" />
<xi:include href="gpu-accel.xml" />
<xi:include href="xfce.xml" />
<xi:include href="networking.xml" />
<xi:include href="linux-kernel.xml" />

11
nixos/doc/manual/configuration/file-systems.xml
View File

@ -16,6 +16,17 @@
fsType = "ext4";
};
</programlisting>
This will create an entry in <filename>/etc/fstab</filename>, which will
generate a corresponding
<link xlink:href="https://www.freedesktop.org/software/systemd/man/systemd.mount.html">systemd.mount</link>
unit via
<link xlink:href="https://www.freedesktop.org/software/systemd/man/systemd-fstab-generator.html">systemd-fstab-generator</link>.
The filesystem will be mounted automatically unless
<literal>"noauto"</literal> is present in <link
linkend="opt-fileSystems._name__.options">options</link>.
<literal>"noauto"</literal> filesystems can be mounted explicitly using
<command>systemctl</command> e.g. <command>systemctl start
data.mount</command>.
Mount points are created automatically if they don’t already exist. For
<option><link linkend="opt-fileSystems._name__.device">device</link></option>,
it’s best to use the topology-independent device aliases in

258
nixos/doc/manual/configuration/gpu-accel.xml Normal file
View File

@ -0,0 +1,258 @@
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xi="http://www.w3.org/2001/XInclude"
version="5.0"
xml:id="sec-gpu-accel">
<title>GPU acceleration</title>
<para>
NixOS provides various APIs that benefit from GPU hardware
acceleration, such as VA-API and VDPAU for video playback; OpenGL and
Vulkan for 3D graphics; and OpenCL for general-purpose computing.
This chapter describes how to set up GPU hardware acceleration (as far
as this is not done automatically) and how to verify that hardware
acceleration is indeed used.
</para>
<para>
Most of the aforementioned APIs are agnostic with regards to which
display server is used. Consequently, these instructions should apply
both to the X Window System and Wayland compositors.
</para>
<section xml:id="sec-gpu-accel-opencl">
<title>OpenCL</title>
<para>
<link xlink:href="https://en.wikipedia.org/wiki/OpenCL">OpenCL</link> is a
general compute API. It is used by various applications such as
Blender and Darktable to accelerate certain operations.
</para>
<para>
OpenCL applications load drivers through the <emphasis>Installable Client
Driver</emphasis> (ICD) mechanism. In this mechanism, an ICD file
specifies the path to the OpenCL driver for a particular GPU family.
In NixOS, there are two ways to make ICD files visible to the ICD
loader. The first is through the <varname>OCL_ICD_VENDORS</varname>
environment variable. This variable can contain a directory which
is scanned by the ICL loader for ICD files. For example:
<screen><prompt>$</prompt> export \
OCL_ICD_VENDORS=`nix-build '&lt;nixpkgs&gt;' --no-out-link -A rocm-opencl-icd`/etc/OpenCL/vendors/</screen>
</para>
<para>
The second mechanism is to add the OpenCL driver package to
<xref linkend="opt-hardware.opengl.extraPackages"/>. This links the
ICD file under <filename>/run/opengl-driver</filename>, where it will
be visible to the ICD loader.
</para>
<para>
The proper installation of OpenCL drivers can be verified through
the <command>clinfo</command> command of the <package>clinfo</package>
package. This command will report the number of hardware devices
that is found and give detailed information for each device:
</para>
<screen><prompt>$</prompt> clinfo | head -n3
Number of platforms 1
Platform Name AMD Accelerated Parallel Processing
Platform Vendor Advanced Micro Devices, Inc.</screen>
<section xml:id="sec-gpu-accel-opencl-amd">
<title>AMD</title>
<para>
Modern AMD <link
xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics
Core Next</link> (GCN) GPUs are supported through the
<package>rocm-opencl-icd</package> package. Adding this package to
<xref linkend="opt-hardware.opengl.extraPackages"/> enables OpenCL
support:
<programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [
rocm-opencl-icd
];</programlisting>
</para>
</section>
<section xml:id="sec-gpu-accel-opencl-intel">
<title>Intel</title>
<para>
<link
xlink:href="https://en.wikipedia.org/wiki/List_of_Intel_graphics_processing_units#Gen8">Intel
Gen8 and later GPUs</link> are supported by the Intel NEO OpenCL
runtime that is provided by the
<package>intel-compute-runtime</package> package. For Gen7 GPUs,
the deprecated Beignet runtime can be used, which is provided
by the <package>beignet</package> package. The proprietary Intel
OpenCL runtime, in the <package>intel-ocl</package> package, is
an alternative for Gen7 GPUs.
</para>
<para>
The <package>intel-compute-runtime</package>, <package>beignet</package>,
or <package>intel-ocl</package> package can be added to
<xref linkend="opt-hardware.opengl.extraPackages"/> to enable OpenCL
support. For example, for Gen8 and later GPUs, the following
configuration can be used:
<programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [
intel-compute-runtime
];</programlisting>
</para>
</section>
</section>
<section xml:id="sec-gpu-accel-vulkan">
<title>Vulkan</title>
<para>
<link xlink:href="https://en.wikipedia.org/wiki/Vulkan_(API)">Vulkan</link> is a
graphics and compute API for GPUs. It is used directly by games or indirectly though
compatibility layers like <link xlink:href="https://github.com/doitsujin/dxvk/wiki">DXVK</link>.
</para>
<para>
By default, if <xref linkend="opt-hardware.opengl.driSupport"/> is enabled,
<package>mesa</package> is installed and provides Vulkan for supported hardware.
</para>
<para>
Similar to OpenCL, Vulkan drivers are loaded through the <emphasis>Installable Client
Driver</emphasis> (ICD) mechanism. ICD files for Vulkan are JSON files that specify
the path to the driver library and the supported Vulkan version. All successfully
loaded drivers are exposed to the application as different GPUs.
In NixOS, there are two ways to make ICD files visible to Vulkan applications: an
environment variable and a module option.
</para>
<para>
The first option is through the <varname>VK_ICD_FILENAMES</varname>
environment variable. This variable can contain multiple JSON files, separated by
<literal>:</literal>. For example:
<screen><prompt>$</prompt> export \
VK_ICD_FILENAMES=`nix-build '&lt;nixpkgs&gt;' --no-out-link -A amdvlk`/share/vulkan/icd.d/amd_icd64.json</screen>
</para>
<para>
The second mechanism is to add the Vulkan driver package to
<xref linkend="opt-hardware.opengl.extraPackages"/>. This links the
ICD file under <filename>/run/opengl-driver</filename>, where it will
be visible to the ICD loader.
</para>
<para>
The proper installation of Vulkan drivers can be verified through
the <command>vulkaninfo</command> command of the <package>vulkan-tools</package>
package. This command will report the hardware devices and drivers found,
in this example output amdvlk and radv:
</para>
<screen><prompt>$</prompt> vulkaninfo | grep GPU
GPU id : 0 (Unknown AMD GPU)
GPU id : 1 (AMD RADV NAVI10 (LLVM 9.0.1))
...
GPU0:
deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
deviceName = Unknown AMD GPU
GPU1:
deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU</screen>
<para>
A simple graphical application that uses Vulkan is <command>vkcube</command>
from the <package>vulkan-tools</package> package.
</para>
<section xml:id="sec-gpu-accel-vulkan-amd">
<title>AMD</title>
<para>
Modern AMD <link
xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics
Core Next</link> (GCN) GPUs are supported through either radv, which is
part of <package>mesa</package>, or the <package>amdvlk</package> package.
Adding the <package>amdvlk</package> package to
<xref linkend="opt-hardware.opengl.extraPackages"/> makes both drivers
available for applications and lets them choose. A specific driver can
be forced as follows:
<programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [
<package>amdvlk</package>
];
# For amdvlk
<xref linkend="opt-environment.variables"/>.VK_ICD_FILENAMES =
"/run/opengl-driver/share/vulkan/icd.d/amd_icd64.json";
# For radv
<xref linkend="opt-environment.variables"/>.VK_ICD_FILENAMES =
"/run/opengl-driver/share/vulkan/icd.d/radeon_icd.x86_64.json";
</programlisting>
</para>
</section>
</section>
<section xml:id="sec-gpu-accel-common-issues">
<title>Common issues</title>
<section xml:id="sec-gpu-accel-common-issues-permissions">
<title>User permissions</title>
<para>
Except where noted explicitly, it should not be necessary to
adjust user permissions to use these acceleration APIs. In the default
configuration, GPU devices have world-read/write permissions
(<filename>/dev/dri/renderD*</filename>) or are tagged as
<code>uaccess</code> (<filename>/dev/dri/card*</filename>). The
access control lists of devices with the <varname>uaccess</varname>
tag will be updated automatically when a user logs in through
<command>systemd-logind</command>. For example, if the user
<emphasis>jane</emphasis> is logged in, the access control list
should look as follows:
<screen><prompt>$</prompt> getfacl /dev/dri/card0
# file: dev/dri/card0
# owner: root
# group: video
user::rw-
user:jane:rw-
group::rw-
mask::rw-
other::---</screen>
If you disabled (this functionality of) <command>systemd-logind</command>,
you may need to add the user to the <code>video</code> group and
log in again.
</para>
</section>
<section xml:id="sec-gpu-accel-common-issues-mixing-nixpkgs">
<title>Mixing different versions of nixpkgs</title>
<para>
The <emphasis>Installable Client Driver</emphasis> (ICD)
mechanism used by OpenCL and Vulkan loads runtimes into its address
space using <code>dlopen</code>. Mixing an ICD loader mechanism and
runtimes from different version of nixpkgs may not work. For example,
if the ICD loader uses an older version of <package>glibc</package>
than the runtime, the runtime may not be loadable due to
missing symbols. Unfortunately, the loader will generally be quiet
about such issues.
</para>
<para>
If you suspect that you are running into library version mismatches
between an ICL loader and a runtime, you could run an application with
the <code>LD_DEBUG</code> variable set to get more diagnostic
information. For example, OpenCL can be tested with
<code>LD_DEBUG=files clinfo</code>, which should report missing
symbols.
</para>
</section>
</section>
</chapter>

3
nixos/doc/manual/configuration/profiles/demo.xml
View File

@ -9,7 +9,6 @@
This profile just enables a <systemitem class="username">demo</systemitem>
user, with password <literal>demo</literal>, uid <literal>1000</literal>,
<systemitem class="groupname">wheel</systemitem> group and
<link linkend="opt-services.xserver.displayManager.sddm.autoLogin"> autologin
in the SDDM display manager</link>.
<link linkend="opt-services.xserver.displayManager.autoLogin">autologin in the SDDM display manager</link>.
</para>
</section>

2
nixos/doc/manual/configuration/summary.xml
View File

@ -10,7 +10,7 @@
expression language. It’s not complete. In particular, there are many other
built-in functions. See the
<link
xlink:href="http://nixos.org/nix/manual/#chap-writing-nix-expressions">Nix
xlink:href="https://nixos.org/nix/manual/#chap-writing-nix-expressions">Nix
manual</link> for the rest.
</para>

47
nixos/doc/manual/configuration/x-windows.xml
View File

@ -31,6 +31,7 @@
<xref linkend="opt-services.xserver.windowManager.twm.enable"/> = true;
<xref linkend="opt-services.xserver.windowManager.icewm.enable"/> = true;
<xref linkend="opt-services.xserver.windowManager.i3.enable"/> = true;
<xref linkend="opt-services.xserver.windowManager.herbstluftwm.enable"/> = true;
</programlisting>
</para>
<para>
@ -89,10 +90,50 @@
using lightdm for a user <literal>alice</literal>:
<programlisting>
<xref linkend="opt-services.xserver.displayManager.lightdm.enable"/> = true;
<xref linkend="opt-services.xserver.displayManager.lightdm.autoLogin.enable"/> = true;
<xref linkend="opt-services.xserver.displayManager.lightdm.autoLogin.user"/> = "alice";
<xref linkend="opt-services.xserver.displayManager.autoLogin.enable"/> = true;
<xref linkend="opt-services.xserver.displayManager.autoLogin.user"/> = "alice";
</programlisting>
The options are named identically for all other display managers.
</para>
</simplesect>
<simplesect xml:id="sec-x11--graphics-cards-intel">
<title>Intel Graphics drivers</title>
<para>
There are two choices for Intel Graphics drivers in X.org:
<literal>modesetting</literal> (included in the <package>xorg-server</package> itself)
and <literal>intel</literal> (provided by the package <package>xf86-video-intel</package>).
</para>
<para>
The default and recommended is <literal>modesetting</literal>.
It is a generic driver which uses the kernel
<link xlink:href="https://en.wikipedia.org/wiki/Mode_setting">mode setting</link>
(KMS) mechanism. It supports Glamor (2D graphics acceleration via OpenGL)
and is actively maintained but may perform worse in some cases (like in old chipsets).
</para>
<para>
The second driver, <literal>intel</literal>, is specific to Intel GPUs,
but not recommended by most distributions: it lacks several modern features
(for example, it doesn't support Glamor) and the package hasn't been officially
updated since 2015.
</para>
<para>
The results vary depending on the hardware, so you may have to try both drivers.
Use the option <xref linkend="opt-services.xserver.videoDrivers"/> to set one.
The recommended configuration for modern systems is:
<programlisting>
<xref linkend="opt-services.xserver.videoDrivers"/> = [ "modesetting" ];
<xref linkend="opt-services.xserver.useGlamor"/> = true;
</programlisting>
If you experience screen tearing no matter what, this configuration was
reported to resolve the issue:
<programlisting>
<xref linkend="opt-services.xserver.videoDrivers"/> = [ "intel" ];
<xref linkend="opt-services.xserver.deviceSection"/> = ''
Option "DRI" "2"
Option "TearFree" "true"
'';
</programlisting>
Note that this will likely downgrade the performance compared to
<literal>modesetting</literal> or <literal>intel</literal> with DRI 3 (default).
</para>
</simplesect>
<simplesect xml:id="sec-x11-graphics-cards-nvidia">

10
nixos/doc/manual/configuration/xfce.xml
View File

@ -16,11 +16,11 @@
effects, some example settings:
<programlisting>
<link linkend="opt-services.picom.enable">services.picom</link> = {
<link linkend="opt-services.picom.enable">enable</link> = true;
<link linkend="opt-services.picom.fade">fade</link> = true;
<link linkend="opt-services.picom.inactiveOpacity">inactiveOpacity</link> = "0.9";
<link linkend="opt-services.picom.shadow">shadow</link> = true;
<link linkend="opt-services.picom.fadeDelta">fadeDelta</link> = 4;
<link linkend="opt-services.picom.enable">enable</link> = true;
<link linkend="opt-services.picom.fade">fade</link> = true;
<link linkend="opt-services.picom.inactiveOpacity">inactiveOpacity</link> = 0.9;
<link linkend="opt-services.picom.shadow">shadow</link> = true;
<link linkend="opt-services.picom.fadeDelta">fadeDelta</link> = 4;
};
</programlisting>
</para>

68
nixos/doc/manual/development/freeform-modules.xml Normal file
View File

@ -0,0 +1,68 @@
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xi="http://www.w3.org/2001/XInclude"
version="5.0"
xml:id="sec-freeform-modules">
<title>Freeform modules</title>
<para>
Freeform modules allow you to define values for option paths that have not been declared explicitly. This can be used to add attribute-specific types to what would otherwise have to be <literal>attrsOf</literal> options in order to accept all attribute names.
</para>
<para>
This feature can be enabled by using the attribute <literal>freeformType</literal> to define a freeform type. By doing this, all assignments without an associated option will be merged using the freeform type and combined into the resulting <literal>config</literal> set. Since this feature nullifies name checking for entire option trees, it is only recommended for use in submodules.
</para>
<example xml:id="ex-freeform-module">
<title>Freeform submodule</title>
<para>
The following shows a submodule assigning a freeform type that allows arbitrary attributes with <literal>str</literal> values below <literal>settings</literal>, but also declares an option for the <literal>settings.port</literal> attribute to have it type-checked and assign a default value. See <xref linkend="ex-settings-typed-attrs"/> for a more complete example.
</para>
<programlisting>
{ lib, config, ... }: {
options.settings = lib.mkOption {
type = lib.types.submodule {
freeformType = with lib.types; attrsOf str;
# We want this attribute to be checked for the correct type
options.port = lib.mkOption {
type = lib.types.port;
# Declaring the option also allows defining a default value
default = 8080;
};
};
};
}
</programlisting>
<para>
And the following shows what such a module then allows
</para>
<programlisting>
{
# Not a declared option, but the freeform type allows this
settings.logLevel = "debug";
# Not allowed because the the freeform type only allows strings
# settings.enable = true;
# Allowed because there is a port option declared
settings.port = 80;
# Not allowed because the port option doesn't allow strings
# settings.port = "443";
}
</programlisting>
</example>
<note>
<para>
Freeform attributes cannot depend on other attributes of the same set without infinite recursion:
<programlisting>
{
# This throws infinite recursion encountered
settings.logLevel = lib.mkIf (config.settings.port == 80) "debug";
}
</programlisting>
To prevent this, declare options for all attributes that need to depend on others. For above example this means to declare <literal>logLevel</literal> to be an option.
</para>
</note>
</section>

2
nixos/doc/manual/development/releases.xml
View File

@ -57,7 +57,7 @@
<listitem>
<para>
<link xlink:href="https://github.com/NixOS/nixos-org-configurations/pull/18">
Make sure a channel is created at http://nixos.org/channels/. </link>
Make sure a channel is created at https://nixos.org/channels/. </link>
</para>
</listitem>
<listitem>

Some files were not shown because too many files have changed in this diff Show More