Robert Hensing
bad676c7ed
Merge pull request #174441 from tljuniper/172325-nixostest-override-python-pkgs-additional-param
...
nixos/test-driver: additional Python packages in test driver
2022-06-05 20:46:01 +02:00
Bobby Rong
9d2a8e3190
nixos/test-driver: fix type hint for send_chars
2022-06-04 12:05:47 +08:00
Robert Hensing
793180cf55
Merge branch 'master' into 172325-nixostest-override-python-pkgs-additional-param
2022-06-03 14:22:13 +02:00
Michael Schneider
814027378b
nixos/test-driver: Typecheck TestScript
2022-06-02 10:05:45 +02:00
tljuniper
a99736e399
nixos/test-driver: add option to add extra python packages to test code
2022-05-29 11:22:23 +02:00
Anna Gillert
dbc95f15b8
nixos/test-driver: Avoid shell injection in machine.execute()
2022-04-21 10:54:00 +02:00
Anna Gillert
f7e89a59da
nixos/test-driver: fix missing shellopts in execute
...
Without this fix, setting the shellopts in `machine.execute` is
inconsitent. When no timeout is used, shellopts `set -euo pipefail` are
applied to the command as expected. When a timeout is specified, the
shellopts are not applied to the command itself (which is called inside
a `sh -c` that doesn't inherit the shellopts) but rather to the
`timeout` command, leading to the following full command:
```bash
(set -euo pipefail; timeout 900 sh -c 'cmd') | (base64 --wrap 0; echo)\n
```
With this fix, this is the command we get:
```bash
timeout 900 sh -c 'set -euo pipefail; false | true') | (base64 --wrap 0; echo)\n
```
2022-04-21 10:54:00 +02:00
rnhmjoj
ca1b1f6dc0
nixos/test-driver: highlight driver log lines
...
There is a whole lot of noise in a NixOS test log due to journal,
commands, and test driver messages all being mixing together.
With this commit the test driver messages are highlighted so you don't
have to squint too much to see where a subtest starts and ends or what
was the last command being run.
Here's an excerpt from a highlighted log:
[2mmachine # [ 6.647826] dhcpcd[668]: eth0: adding default route via 10.0.2.2[0m
[2mmachine # I've had a great day![0m
(finished: must succeed: curl -sfL https://example.com/how-are-you ? | tee /dev/stderr, in 0.19 seconds)
(finished: subtest: Privoxy can filter https requests, in 0.22 seconds)
[1m[32msubtest: Blocks are working[0m
machine: [1m[32mwaiting for TCP port 443[0m
[2mmachine # Connection to localhost (::1) 443 port [tcp/https] succeeded![0m
(finished: waiting for TCP port 443, in 0.03 seconds)
machine: [1m[32mmust fail: curl -f https://example.com/ads 1>&2[0m
[2mmachine # [ 6.730608] nscd[742]: 742 monitored file `/etc/resolv.conf` was written to[0m
[2mmachine # [ 6.733071] privoxy[814]: 2022-04-14 09:41:01.695 7fa995c39640 Actions: +block{Fake ads} ...
[2mmachine # curl: (22) The requested URL returned error: 403[0m
[2mmachine # [ 6.753098] systemd[1]: Stopping Name Service Cache Daemon...[0m
[2mmachine # [ 6.755632] systemd[1]: nscd.service: Deactivated successfully.[0m
[2mmachine # [ 6.757069] systemd[1]: Stopped Name Service Cache Daemon.[0m
(finished: must fail: curl -f https://example.com/ads 1>&2, in 0.07 seconds)
machine: [1m[32mmust succeed: curl -f https://example.com/PRIVOXY-FORCE/ads 1>&2[0m
[2mmachine # [ 6.765099] systemd[1]: Starting Name Service Cache Daemon...[0m
[2mmachine # [ 6.781519] nscd[925]: 925 monitoring file `/etc/passwd` (1)[0m
[2mmachine # [ 6.844105] systemd[1]: Reached target Network is Online.[0m
[2mmachine # [ 6.848860] privoxy[814]: 2022-04-14 09:41:01.805 7fa995c39640 Actions: +block{Fake ads} ...
[2mmachine # [ 6.856068] nscd[925]: 925 monitoring file `/etc/nsswitch.conf` (8)[0m
[2mmachine # [ 6.858229] systemd[1]: Reached target Multi-User System.[0m
[2mmachine # [ 6.858934] nscd[925]: 925 monitoring directory `/etc` (2)[0m
[2mmachine # [ 6.861315] systemd[1]: Startup finished in 2.374s (kernel) + 4.451s (userspace) = 6.826s.[0m
[2mmachine # Hot Nixpkgs PRs in your area. Click here![0m
[2mmachine # [ 6.873055] nscd[925]: 925 monitoring directory `/etc` (2)[0m
[2mmachine # [ 6.874489] nscd[925]: 925 monitoring file `/etc/nsswitch.conf` (8)[0m
[2mmachine # [ 6.875599] nscd[925]: 925 monitoring directory `/etc` (2)[0m
(finished: must succeed: curl -f https://example.com/PRIVOXY-FORCE/ads 1>&2, in 0.12 seconds)
(finished: subtest: Blocks are working, in 0.22 seconds)
[1m[32msubtest: Temporary certificates are cleaned[0m
machine: [1m[32mmust succeed: test $(ls /run/privoxy/certs | wc -l) -gt 0[0m
(finished: must succeed: test $(ls /run/privoxy/certs | wc -l) -gt 0, in 0.02 seconds)
machine: [1m[32mmust succeed: date -s "$(date --date '12 days')"[0m
(finished: must succeed: date -s "$(date --date '12 days')", in 0.02 seconds)
[2mmachine # [ 6.959589] systemd[1]: Started Logrotate Service.[0m
[2mmachine # [ 6.966685] systemd[1]: Starting Cleanup of Temporary Directories...[0m
[2mmachine # [ 6.974783] systemd[1]: logrotate.service: Deactivated successfully.[0m
[2mmachine # [ 7.004493] systemd[1]: systemd-tmpfiles-clean.service: Deactivated successfully.[0m
[2mmachine # [ 7.005457] systemd[1]: Finished Cleanup of Temporary Directories.[0m
machine: [1m[32mmust succeed: test $(ls /run/privoxy/certs | wc -l) -eq 0[0m
(finished: must succeed: test $(ls /run/privoxy/certs | wc -l) -eq 0, in 0.01 seconds)
(finished: subtest: Temporary certificates are cleaned, in 0.15 seconds)
(finished: run the VM test script, in 7.44 seconds)
test script finished in 7.48s
[1m[32mcleanup[0m
kill machine (pid 8)
2022-04-14 11:54:21 +02:00
Janne Heß
7d0e21c77e
nixos/test-runner: Allow writing to qemu stdin
2022-03-25 12:08:21 +01:00
Viktor Kleen
6d766ae8b7
nixos/test-driver: deduplicate VLANs
...
Naively deduplicate VLANs in the python driver for NixOS tests. The
current implementation accidentally works, since the VLan class mutates
the environment. On construction it sets QEMU_VDE_SOCKET_${id} and this
environment variable gets overwritten once a second VLAN with the same
id is constructed. Because the NIC flags passed to qemu just use the
QEMU_VDE_SOCKET_${id} environment variable, this implicitly chooses a
single vde_switch process for each VLAN.
However, this leads to unusable vde_switch processes being spawned in
each test run and as a side effect makes it impossible to access the
correct VLan objects in the interactive test driver. It also makes it
remarkably hard to understand why the current implementation ever
worked.
2022-03-22 13:16:28 +00:00
Daniel Fullmer
24fb872079
nixos/test-driver: fix regression when using legacy start command
...
The aarch64-linux versions of the boot.uefiUsb and boot.uefiCdrom tests
were broken by b0fc9da879
.
That commit was a refactor which omitted the qemuBinary option, which was
previously available in the legacy start command. This restores that
option and fixes the tests previously mentioned.
2022-02-17 18:12:24 -08:00
Lassulus
b80607af67
Merge pull request #154804 from Mic92/tests-prompt
...
nixos/tests: add prompt to shell_interact()
2022-02-02 18:50:46 +01:00
Marijan Petričević
411bb3c35f
add --no-interactive flag
2022-01-26 11:27:01 +01:00
Marijan Petričević
446c21fdc7
factor out tmp-dir checks
2022-01-25 11:31:25 +01:00
Marijan Petričević
03c90974a7
add tmp_dir access check
2022-01-25 11:14:57 +01:00
Marijan Petričević
e17fcbc966
introduce writeable_dir argparse type
2022-01-25 10:21:33 +01:00
Marijan Petričević
7765670c8a
make output_directory an optional parameter
2022-01-25 01:18:52 +01:00
Marijan Petričević
f9b5f9dba7
nixos/test-driver: use an argument instead of the out env-var
2022-01-25 01:17:29 +01:00
Jörg Thalheim
f472e52aa6
nixos/tests: add prompt to shell_interact()
...
Example session:
>>> start_all()
>>> machines[0].shell_interact()
client: waiting for the VM to finish booting
client: connected to guest root shell
client: (connecting took 0.00 seconds)
(finished: waiting for the VM to finish booting, in 0.05 seconds)
client: Terminal is ready (there is no initial prompt):
$ ls -la
total 39
drwxrwxrwt 11 root root 4096 Jan 15 06:06 .
drwxr-xr-x 17 root root 4096 Jan 15 06:06 ..
drwxrwxrwt 2 root root 4096 Jan 15 06:06 .font-unix
drwxrwxrwt 2 root root 4096 Jan 15 06:06 .ICE-unix
drwx------ 2 root root 2 Jan 12 20:19 shared
drwx------ 3 root root 4096 Jan 15 06:06 systemd-private-b8f21699ea684491926859758de41975-nscd.service-txgYVZ
drwx------ 3 root root 4096 Jan 15 06:06 systemd-private-b8f21699ea684491926859758de41975-systemd-logind.service-U3GmlL
drwxrwxrwt 2 root root 4096 Jan 15 06:06 .Test-unix
drwxrwxrwt 2 root root 4096 Jan 15 06:06 .X11-unix
drwxr-xr-x 2 root root 2 Jan 15 06:06 xchg
drwxrwxrwt 2 root root 4096 Jan 15 06:06 .XIM-unix
2022-01-15 07:07:06 +01:00
Patrick Hilhorst
793a2f50f1
nixos/test-driver: remove unused imports, add pylint unused-import check
2022-01-02 23:12:21 +01:00
Patrick Hilhorst
a2f5092867
nixos/test-driver: simplify logic, reduce interaction surface
2022-01-02 22:52:17 +01:00
Patrick Hilhorst
7830f000c5
nixos/test-driver: simplify coopmulti
2022-01-02 22:20:04 +01:00
Patrick Hilhorst
ac6c06c549
nixos/test-driver: bump version
2022-01-01 23:17:37 +01:00
Patrick Hilhorst
4e1556ed4d
nixos/test-driver: add polling_condition
2022-01-01 23:17:32 +01:00
Marijan Petričević
ab693de868
nixos/test-driver: make the test-driver a python package
2021-12-10 12:27:45 +00:00
Artturin
60422ba2ea
nixos/test-driver: add 10ms delay to send_key
...
attempt to fix https://github.com/NixOS/nixpkgs/issues/147294
2021-12-03 20:04:56 +02:00
Jacek Galowicz
58371472fe
Merge pull request #146512 from DeterminateSystems/better-visibility
...
nixos/test-driver: more context when step finishes, give more functions nested labels
2021-12-02 09:26:21 +00:00
Jacek Galowicz
57dc1085dc
Merge pull request #146271 from DeterminateSystems/better-timeouts
...
nixos/test-driver: add execute_with_timeout, add (functional) timeouts to more functions
2021-12-01 19:01:05 +00:00
Cole Helbling
af765f3abd
nixos/test-driver: give more functions nested labels
...
This will make it easier to trace through the test execution without
having to scroll through the entire kernel output.
2021-12-01 10:29:56 -08:00
Cole Helbling
c6ee63259a
nixos/test-driver: more context when step finishes
...
When displaying the amount of time some step took, with no other
context, it becomes nigh impossible (especially in longer tests) to see
when specific steps finished.
2021-12-01 10:29:56 -08:00
Patrick Hilhorst
1334a62539
test-driver.py: directly import pathlib.Path
2021-11-20 01:37:08 +01:00
Patrick Hilhorst
9e71014ede
test-driver.py: always export single machine as 'machine'
2021-11-19 23:02:30 +01:00
Patrick Hilhorst
a8f693ed48
test-driver.py: fix weird non-pythonism
2021-11-19 22:28:32 +01:00
Cole Helbling
e62b8020f3
nixos/test-driver: add (functional) timeouts to more functions
...
A retry timeout doesn't really help if the thing it's retrying may block
forever.
2021-11-17 16:11:10 -08:00
Cole Helbling
363d7f3ae8
nixos/test-driver: add timeout parameter to execute
2021-11-17 16:11:03 -08:00
Janne Heß
1640359f33
nixos/test-runner: Fix execute() flakiness
...
Instead of using the magic string, we now just base64-encode everything
and check for a newline.
2021-10-28 11:51:20 +02:00
Janne Heß
1ed5606d8a
nixos/test-runner: Fix thread cleanup
2021-10-24 15:11:46 +02:00
K900
9611041ef5
nixos/lib/test-driver: set defaults in __init__ instead of class level
...
Just a minor code style nit to avoid potential footguns
2021-10-19 16:29:19 +03:00
K900
a874235dff
nixos/lib/test-driver: clean up threads correctly
...
The current implementation just forks off a thread to read
QEMU's stdout and lets it exist forever. This, however,
makes the interpreter shutdown racy, as the thread could
still be running and writing out buffered stdout when the
main thread exits (and since it's using the low level API,
the worker thread does not get cleaned up by the atexit hooks
installed by `threading`, either). So, instead of doing that,
let's create a real `threading.Thread` object, and also
explicitly `join` it along with the other stuff when cleaning up.
2021-10-19 16:23:51 +03:00
Janne Heß
540dc908ca
nixos/test-runner: Print exceptions that happen
2021-10-14 00:15:53 +02:00
Jacek Galowicz
5c666cdf62
Re-RAII-ify the NixOS integration test driver's VLAN class.
...
We have no usecase for manually/selectively starting or stopping VLANs
in integration tests.
By starting and stopping the VLANs with the constructor and destructor
of VLAN objects, we remove the obligation and complexity to maintain
network lifetime separately.
2021-10-05 14:38:48 -05:00
Maximilian Bosch
3f63e3ce65
nixos/test-driver: fix graphics for VM
...
`-nographic` should only be set if no display is available. Otherwise,
tools such as `nixos-build-vms(8)` will never open up a graphical
display.
2021-10-05 14:38:48 -05:00
David Arnold
b0fc9da879
nixos/test/test-driver: Class-ify the test driver
...
This commit encapsulates the involved domain into classes and
defines explicit and typed arguments where untyped dicts where used.
It preserves backwards compatibility through legacy wrappers.
2021-10-05 14:38:48 -05:00
David Arnold
2832896c5b
nixos/test: some test fixes in succession of #125992
2021-08-20 13:38:22 -05:00
David Arnold
db614e11d6
nixos/tests/test-driver: better control test env symbols
...
Previous to this commit, the entire test driver environment was shared
with the actual python test environment.
This is a hefty api surface. This commit selectively exposes only those
symbols to the test environment that are actually meant to be used by
tests.
2021-08-19 23:55:26 -05:00
Maximilian Bosch
e1ec5acd31
nixos/test-driver: start interactive mode if testScript
is empty
...
This is relevant for `nixos-build-vms(8)` which doesn't have a
test-script. In that case it's more intuitive to directly go into the
interactive mode which is IMHO more intuitive.
2021-08-12 23:01:03 +02:00
David Arnold
926fb93968
nixos/tests/test-driver: normalise test driver entrypoint(s)
...
Previously the driver was configured exclusively through convoluted
environment variables.
Now the driver's defaults are configured through env variables.
Some additional concerns are in the github comments of this PR.
2021-08-05 19:07:11 -05:00
Ingo Blechschmidt
5143ab9f74
tigervnc, tightvnc: add basic tests
...
Co-Authored-By: Ingo Blechschmidt <iblech@web.de>
2021-07-20 15:22:31 +02:00
misuzu
702a1c23a4
nixos/test-driver: allow overriding qemu binary in create_startcommand
2021-07-14 12:47:12 +03:00
Jörg Thalheim
48aec70ea1
nixos/test-driver: replace termlib with socat
...
telnetlib does not handle unicode, which is annoying when using systemctl.
Also this gives us a nice readline with history.
2021-06-22 09:08:57 +02:00