Commit Graph

694 Commits

Author SHA1 Message Date
Robert Schütz
243053e521 python310Packages.mistune: 0.8.4 -> 2.0.4
Remove mistune_0_8 because it's insecure.
2022-08-15 06:53:01 +00:00
pennae
af98bacbe0 Revert "nixos/docs: cache mergeJSON md conversion on baseOptionsJSON"
This reverts commit 52b0ad17e3.

we only needed this because mergeJSON was slow, but in the interim we
found a better solution to the slowness.
2022-08-05 17:13:47 +02:00
pennae
645cfa59ac nixos/make-option-docs: add xref support to markdown conversion 2022-08-03 22:01:14 +02:00
pennae
7a091b2686 nixos/make-options-doc: reuse markdown instance in mergeJSON
this doesn't construct a new (expensive) parser for every option, making
rendering about 30x faster.
2022-08-03 22:00:24 +02:00
pennae
6b13dd0e9e
Merge pull request #183491 from pennae/automatic-md-conversions
treewide: automatically md-convert option descriptions
2022-08-02 02:15:30 +02:00
Silvan Mosberger
fe2c9ae814
Merge pull request #177012 from hercules-ci/nixos-modular-system
`flake.lib.nixosSystem`: Allow `nixpkgs.system` to be set modularly; improve error message
2022-08-01 15:05:16 +02:00
Alyssa Ross
5330c0a1af
treewide: use isAarch where appropriate 2022-07-30 16:18:27 +00:00
pennae
52b0ad17e3 nixos/docs: cache mergeJSON md conversion on baseOptionsJSON
with ever more options being markdown rather than docbook the conversion
time is starting to become a significant factor of doc build time.
luckily we can pre-convert all nixos option docs to MD and cache the
result of this conversion, then merge the already-converted json file
with user option docs. we leave options.json unconverted to keep it as
close to the actual nix code as possible.
2022-07-28 23:20:02 +02:00
pennae
18be724a58 nixos/make-options-doc: give MD conversion error locations
during docs conversion it can be very useful to know exactly *where* the
error the script complained about is. the name of the option should be
sufficient since option merging is rather rare, and won't merge doc
attributes anyway.
2022-07-28 20:03:02 +02:00
Philippe Schaaf
5ae6580474 add hub mode comment
Signed-off-by: Philippe Schaaf <philippe.schaaf@secunet.com>
2022-07-21 16:34:09 +02:00
Philippe Schaaf
f6a290932e use vde switch in hubmode by default
Within a dual VM test-setup a strange behaviour was observed.
The two VMs are connected via one vde_switch instance
(instancevirtualisation.vlans = [ 1 ]; IMO a bad attribute name for
switch instances, has nothing to do with VLANs in sense of 802.1Q).

A ping on the base interface (eth1) works, but not on VLAN
subinterfaces (vlan1@eth1). A tcpdump of eth1 includes the ARP requests
tagged with the subinterfaces VLAN ID, but responses seems not to pass
the vde_switch. This works fine if performed on the base interface.

Putting the vde_switch in hub mode results in flooding
traffic to all vde_switch ports. This results in a expected behaviour
and a ping on a VLAN subinterface works as expected.

Signed-off-by: Philippe Schaaf <philippe.schaaf@secunet.com>
2022-07-21 11:38:03 +02:00
Robert Hensing
711e653a65 nixos/eval-config: Allow system to be set modularly when arg is null 2022-07-10 13:35:54 +02:00
Martin Weinelt
c0d1de14b3
Merge pull request #179291 from dotlambda/utillinux-alias
treewide: don't use utillinux alias
2022-07-02 00:05:54 +02:00
Robert Hensing
7c81905344 nixos/make-options-doc: Support Nix-provided declaration locations
Feature was introduced in https://github.com/NixOS/nixpkgs/pull/174460,
but wasn't supported in `mergeJSON.py` yet.
2022-06-27 22:07:20 +02:00
Robert Hensing
8bff3fef40 nixos/make-options-doc: Support block quotes
Our tooling would trip without the inner <para>, despite the docbook
docs suggesting that <para> occurs in <blockquote> and vice versa.
2022-06-27 17:41:32 +02:00
Robert Hensing
e04aa1bcd9 nixos/make-options-doc: Escape inline code and code blocks 2022-06-27 17:41:32 +02:00
Robert Hensing
f900ed1749 nixos/make-options-doc: Support newline md node
This occurs in the ast generated for blockquotes.
2022-06-27 17:41:32 +02:00
Robert Hensing
bccc3e747b nixos/make-options-doc: Fix exception handler for arity /= 1 methods 2022-06-27 17:41:32 +02:00
Robert Hensing
aff2dbbc82 make-options-doc: Make variablelist id configurable
I've tried XInclude set-xml-id first, but our tooling did not pick up on it.
2022-06-27 17:41:32 +02:00
Robert Schütz
40990c3586 treewide: don't use utillinux alias 2022-06-27 04:55:23 +00:00
Robert Hensing
fa9c83ca7f
Merge pull request #174460 from hercules-ci/module-docs-Nix-driven-location-links
make-options-doc: Support Nix-provided declaration links
2022-06-22 15:48:46 +02:00
Lassulus
773cbfbd28
Merge pull request #151633 from wucke13/sd-image-experiment
honor `sdImage.compressImage` also in the build process
2022-06-22 14:48:21 +02:00
Robert Hensing
e2c261f2c0
Merge pull request #176146 from pennae/module-docs-markdown
treewide: markdown option docs
2022-06-21 13:16:02 +02:00
Jörg Thalheim
f5ca531054
Merge pull request #158602 from hercules-ci/systemd-unit-mixed-list-error
nixos/systemd: Custom error when mixing list/non-list defs
2022-06-19 07:05:04 +01:00
Robert Hensing
2c8bbf33fd nixos/test-driver: Support mypy through regular mechanisms 2022-06-17 12:22:57 +02:00
Robert Hensing
cee66a8cd5 make-options-doc: Support Nix-provided declaration links
Previously, the location logic was hardcoded, supporting only
Nixpkgs and NixOps properly, leaving other uses of the module
system without good location support.
2022-06-15 00:45:05 +02:00
pennae
320aa2a791 treewide: attempt at markdown option docs 2022-06-12 12:44:38 +02:00
Robert Hensing
9970c0b874
Merge pull request #174470 from hercules-ci/module-docs-visibility-override
make-options-doc: Allow transformOptions to override visibility
2022-06-10 14:27:58 +02:00
Robert Hensing
bad676c7ed
Merge pull request #174441 from tljuniper/172325-nixostest-override-python-pkgs-additional-param
nixos/test-driver: additional Python packages in test driver
2022-06-05 20:46:01 +02:00
Bobby Rong
9d2a8e3190
nixos/test-driver: fix type hint for send_chars 2022-06-04 12:05:47 +08:00
Robert Hensing
793180cf55
Merge branch 'master' into 172325-nixostest-override-python-pkgs-additional-param 2022-06-03 14:22:13 +02:00
Michael Schneider
fd4ebd8990 add opt out flag for type checking 2022-06-03 13:37:04 +02:00
Michael Schneider
814027378b nixos/test-driver: Typecheck TestScript 2022-06-02 10:05:45 +02:00
Robert Hensing
53326189ba
Merge pull request #174829 from blaggacao/fix/cleanup-nixpkgs-initial-system
nixos: remove effect-less nixpgks.initialSystem
2022-05-31 16:26:25 +02:00
tljuniper
a99736e399 nixos/test-driver: add option to add extra python packages to test code 2022-05-29 11:22:23 +02:00
Alyssa Ross
1e396f9802
nixos/testing-python.nix: fix typo in warning
"pyton" -> "python"
2022-05-27 13:00:25 +00:00
David Arnold
646e214e11
nixos: remove effect-less nixpgks.initialSystem
- initialSystem was keeping track of the evaluating system
- it had been used by `nesting.children`
- since, 20.09, `nesting.children` has been replaced with named
  specializations

It appears that this option was left over and not cleand up properly.
2022-05-26 20:00:05 -05:00
Robert Hensing
75bc6da237 make-options-doc: Filter options after transformOptions
This allows the user-supplied function to change the visibility
of options.
2022-05-25 12:50:07 +02:00
Robert Hensing
90344e9f40
Merge pull request #171163 from hercules-ci/nixpkgs-config-doc
Add generated `nixpkgs.config` doc to Nixpkgs manual
2022-05-19 20:51:38 +02:00
Robert Hensing
0b02135d3b nixosOptionsDoc: refactor
Thanks to Infinisil for pointing this out.
2022-05-16 22:53:23 +02:00
Robert Hensing
062bc5e74a lib.types.functionTo: Add pseudo-attr to generated docs 2022-05-13 09:01:05 +02:00
Robert Hensing
ae172a2bb4 treewide: nixosTest -> testers.nixosTest 2022-05-09 14:16:06 +02:00
Robert Hensing
28f99aad31 nixos/testing-python.nix: Set meta.mainProgram 2022-05-05 12:33:57 +02:00
Robert Hensing
bb2c5a3684 nixosOptionsDoc: Make appendix tag optional 2022-05-01 21:51:19 +02:00
Will Fancher
69d8047516 nixos: Fix up systemd shutdown ramfs 2022-04-28 14:12:14 -04:00
Anna Gillert
dbc95f15b8 nixos/test-driver: Avoid shell injection in machine.execute() 2022-04-21 10:54:00 +02:00
Anna Gillert
f7e89a59da nixos/test-driver: fix missing shellopts in execute
Without this fix, setting the shellopts in `machine.execute` is
inconsitent.  When no timeout is used, shellopts `set -euo pipefail` are
applied to the command as expected. When a timeout is specified, the
shellopts are not applied to the command itself (which is called inside
a `sh -c` that doesn't inherit the shellopts) but rather to the
`timeout` command, leading to the following full command:

```bash
(set -euo pipefail; timeout 900 sh -c 'cmd') | (base64 --wrap 0; echo)\n
```

With this fix, this is the command we get:

```bash
timeout 900 sh -c 'set -euo pipefail; false | true') | (base64 --wrap 0; echo)\n
```
2022-04-21 10:54:00 +02:00
Graham Christensen
d65eff4fb6
Merge pull request #167902 from DeterminateSystems/openstack-zfs-cleanup
openstack-image: init, make-single-disk-zfs-image: init
2022-04-16 19:44:53 -04:00
Jacek Galowicz
6b414c5b66
Merge pull request #168609 from rnhmjoj/pr-test-driver
nixos/test-driver: highlight driver log lines
2022-04-14 13:09:36 +02:00
rnhmjoj
ca1b1f6dc0
nixos/test-driver: highlight driver log lines
There is a whole lot of noise in a NixOS test log due to journal,
commands, and test driver messages all being mixing together.
With this commit the test driver messages are highlighted so you don't
have to squint too much to see where a subtest starts and ends or what
was the last command being run.

Here's an excerpt from a highlighted log:

    machine # [    6.647826] dhcpcd[668]: eth0: adding default route via 10.0.2.2
    machine # I've had a great day!
    (finished: must succeed: curl -sfL https://example.com/how-are-you? | tee /dev/stderr, in 0.19 seconds)
    (finished: subtest: Privoxy can filter https requests, in 0.22 seconds)
    subtest: Blocks are working
    machine: waiting for TCP port 443
    machine # Connection to localhost (::1) 443 port [tcp/https] succeeded!
    (finished: waiting for TCP port 443, in 0.03 seconds)
    machine: must fail: curl -f https://example.com/ads 1>&2
    machine # [    6.730608] nscd[742]: 742 monitored file `/etc/resolv.conf` was written to
    machine # [    6.733071] privoxy[814]: 2022-04-14 09:41:01.695 7fa995c39640 Actions: +block{Fake ads} ...
    machine # curl: (22) The requested URL returned error: 403
    machine # [    6.753098] systemd[1]: Stopping Name Service Cache Daemon...
    machine # [    6.755632] systemd[1]: nscd.service: Deactivated successfully.
    machine # [    6.757069] systemd[1]: Stopped Name Service Cache Daemon.
    (finished: must fail: curl -f https://example.com/ads 1>&2, in 0.07 seconds)
    machine: must succeed: curl -f https://example.com/PRIVOXY-FORCE/ads 1>&2
    machine # [    6.765099] systemd[1]: Starting Name Service Cache Daemon...
    machine # [    6.781519] nscd[925]: 925 monitoring file `/etc/passwd` (1)
    machine # [    6.844105] systemd[1]: Reached target Network is Online.
    machine # [    6.848860] privoxy[814]: 2022-04-14 09:41:01.805 7fa995c39640 Actions: +block{Fake ads} ...
    machine # [    6.856068] nscd[925]: 925 monitoring file `/etc/nsswitch.conf` (8)
    machine # [    6.858229] systemd[1]: Reached target Multi-User System.
    machine # [    6.858934] nscd[925]: 925 monitoring directory `/etc` (2)
    machine # [    6.861315] systemd[1]: Startup finished in 2.374s (kernel) + 4.451s (userspace) = 6.826s.
    machine # Hot Nixpkgs PRs in your area. Click here!
    machine # [    6.873055] nscd[925]: 925 monitoring directory `/etc` (2)
    machine # [    6.874489] nscd[925]: 925 monitoring file `/etc/nsswitch.conf` (8)
    machine # [    6.875599] nscd[925]: 925 monitoring directory `/etc` (2)
    (finished: must succeed: curl -f https://example.com/PRIVOXY-FORCE/ads 1>&2, in 0.12 seconds)
    (finished: subtest: Blocks are working, in 0.22 seconds)
    subtest: Temporary certificates are cleaned
    machine: must succeed: test $(ls /run/privoxy/certs | wc -l) -gt 0
    (finished: must succeed: test $(ls /run/privoxy/certs | wc -l) -gt 0, in 0.02 seconds)
    machine: must succeed: date -s "$(date --date '12 days')"
    (finished: must succeed: date -s "$(date --date '12 days')", in 0.02 seconds)
    machine # [    6.959589] systemd[1]: Started Logrotate Service.
    machine # [    6.966685] systemd[1]: Starting Cleanup of Temporary Directories...
    machine # [    6.974783] systemd[1]: logrotate.service: Deactivated successfully.
    machine # [    7.004493] systemd[1]: systemd-tmpfiles-clean.service: Deactivated successfully.
    machine # [    7.005457] systemd[1]: Finished Cleanup of Temporary Directories.
    machine: must succeed: test $(ls /run/privoxy/certs | wc -l) -eq 0
    (finished: must succeed: test $(ls /run/privoxy/certs | wc -l) -eq 0, in 0.01 seconds)
    (finished: subtest: Temporary certificates are cleaned, in 0.15 seconds)
    (finished: run the VM test script, in 7.44 seconds)
    test script finished in 7.48s
    cleanup
    kill machine (pid 8)
2022-04-14 11:54:21 +02:00