JakeHillion/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
622eb5f699
nixpkgs/nixos/tests/chromium.nix

179 lines
5.0 KiB
Nix
Raw Normal View History

nixos/tests/chromium: Allow to override packages. Of course, this could be done via packageOverrides, but this is more explicit and makes it possible to run the tests with various Chromium overrides. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-19 14:48:14 +00:00
import ./make-test.nix (
{ pkgs
, channelMap ? {
stable = pkgs.chromium;
beta = pkgs.chromiumBeta;
dev = pkgs.chromiumDev;
}
, ...
}: rec {
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 17:41:56 +00:00
name = "chromium";
all tests: added meta.maintainers section
2015-07-12 11:09:40 +01:00
meta = with pkgs.stdenv.lib.maintainers; {
maintainers = [ aszlig ];
};
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 17:41:56 +00:00
nixos/tests/chromium: Detect popup using OCR. This will make the test a lot more reliable, because we no longer need to press ESC multiple times hoping that it will close the popup. Unfortunately in order to run this test I needed to locally revert the gyp update from a305e6855dd8723683c77635f45ae28411c8f36c. With the old gyp version however the test runs fine and it's able to properly detect the popup. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-22 10:23:09 +01:00
enableOCR = true;
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 17:41:56 +00:00
machine.imports = [ ./common/x11.nix ];
Grmbl http://hydra.nixos.org/build/24983009
2015-08-25 10:26:32 +01:00
machine.virtualisation.memorySize = 2047;
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 17:41:56 +00:00
startupHTML = pkgs.writeText "chromium-startup.html" ''
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Chromium startup notifier</title>
</head>
<body onload="javascript:document.title='startup done'">
<img src="file://${pkgs.fetchurl {
tests-chromium: fix link to svg file closes #11208
2015-11-29 21:43:28 +00:00
url = "http://nixos.org/logo/nixos-hex.svg";
sha256 = "0wxpp65npdw2cg8m0cxc9qff1sb3b478cxpg1741d8951g948rg8";
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 17:41:56 +00:00
}}" />
</body>
</html>
'';
testScript = let
xdo = name: text: let
xdoScript = pkgs.writeText "${name}.xdo" text;
in "${pkgs.xdotool}/bin/xdotool '${xdoScript}'";
in ''
sub createNewWin {
$machine->nest("creating a new Chromium window", sub {
$machine->execute("${xdo "new-window" ''
search --onlyvisible --name "startup done"
windowfocus --sync
windowactivate --sync
xdotool: 2.20110530.1 -> 3.20150503.1
2015-09-12 11:12:57 +01:00
''}");
$machine->execute("${xdo "new-window" ''
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 17:41:56 +00:00
key Ctrl+n
''}");
});
}
sub closeWin {
Machine::retry sub {
$machine->execute("${xdo "close-window" ''
search --onlyvisible --name "new tab"
windowfocus --sync
windowactivate --sync
xdotool: 2.20110530.1 -> 3.20150503.1
2015-09-12 11:12:57 +01:00
''}");
$machine->execute("${xdo "close-window" ''
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 17:41:56 +00:00
key Ctrl+w
''}");
for (1..20) {
my ($status, $out) = $machine->execute("${xdo "wait-for-close" ''
search --onlyvisible --name "new tab"
''}");
return 1 if $status != 0;
$machine->sleep(1);
}
}
}
sub waitForNewWin {
my $ret = 0;
$machine->nest("waiting for new Chromium window to appear", sub {
for (1..20) {
my ($status, $out) = $machine->execute("${xdo "wait-for-window" ''
search --onlyvisible --name "new tab"
windowfocus --sync
windowactivate --sync
''}");
if ($status == 0) {
$ret = 1;
last;
}
$machine->sleep(1);
}
});
return $ret;
}
sub createAndWaitForNewWin {
for (1..3) {
createNewWin;
return 1 if waitForNewWin;
}
die "new window didn't appear within 60 seconds";
}
sub testNewWin {
my ($desc, $code) = @_;
createAndWaitForNewWin;
subtest($desc, $code);
closeWin;
}
sub chromiumTest {
my ($channel, $pkg, $code) = @_;
$machine->waitForX;
my $url = "file://${startupHTML}";
my $args = "--user-data-dir=/tmp/chromium-$channel";
$machine->execute(
"ulimit -c unlimited; ".
"$pkg/bin/chromium $args \"$url\" & disown"
);
nixos/tests/chromium: Detect popup using OCR. This will make the test a lot more reliable, because we no longer need to press ESC multiple times hoping that it will close the popup. Unfortunately in order to run this test I needed to locally revert the gyp update from a305e6855dd8723683c77635f45ae28411c8f36c. With the old gyp version however the test runs fine and it's able to properly detect the popup. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-22 10:23:09 +01:00
$machine->waitForText(qr/Type to search or enter a URL to navigate/);
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 17:41:56 +00:00
$machine->waitUntilSucceeds("${xdo "check-startup" ''
search --sync --onlyvisible --name "startup done"
# close first start help popup
tests/chromium: Work around popup close flakiness. It's not nice to send the escape key over and over again just to ensure the popup is closed, because even *if* it fails to close the popup 4 times in a row it's just very unlikely that it will be closed. But in order to make really sure, we might need to do a screenshot and detect visual changes. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-04-02 13:08:23 +01:00
key -delay 1000 Escape
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 17:41:56 +00:00
windowfocus --sync
windowactivate --sync
''}");
createAndWaitForNewWin;
$machine->screenshot($channel."_emptywin");
closeWin;
$machine->screenshot($channel."_startup_done");
subtest("Chromium $channel", $code);
$machine->shutdown;
}
nixos/tests/chromium: Allow to override packages. Of course, this could be done via packageOverrides, but this is more explicit and makes it possible to run the tests with various Chromium overrides. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-19 14:48:14 +00:00
for (${let
mkArray = name: pkg: "[\"${name}\", \"${pkg}\"]";
chanArrays = pkgs.lib.mapAttrsToList mkArray channelMap;
in pkgs.lib.concatStringsSep ", " chanArrays}) {
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 17:41:56 +00:00
my ($channel, $pkg) = @$_;
chromiumTest $channel, $pkg, sub {
testNewWin "check sandbox", sub {
$machine->succeed("${xdo "type-url" ''
search --sync --onlyvisible --name "new tab"
windowfocus --sync
type --delay 1000 "chrome://sandbox"
''}");
$machine->succeed("${xdo "submit-url" ''
search --sync --onlyvisible --name "new tab"
windowfocus --sync
key --delay 1000 Return
''}");
$machine->screenshot($channel."_sandbox");
$machine->succeed("${xdo "submit-url" ''
search --sync --onlyvisible --name "sandbox status"
windowfocus --sync
xdotool: 2.20110530.1 -> 3.20150503.1
2015-09-12 11:12:57 +01:00
''}");
$machine->succeed("${xdo "submit-url" ''
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 17:41:56 +00:00
key --delay 1000 Ctrl+a Ctrl+c
''}");
my $clipboard = $machine->succeed("${pkgs.xclip}/bin/xclip -o");
die "sandbox not working properly: $clipboard"
nixos/tests/chromium: Improve sandbox checking. We no longer need have "SUID sandbox" enabled in the chrome://sandbox status page and we now also check for "You are adequately sandboxed." to be absolutely sure that we're running with proper sandboxing. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-07-02 09:51:52 +01:00
unless $clipboard =~ /namespace sandbox.*yes/mi
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 17:41:56 +00:00
&& $clipboard =~ /pid namespaces.*yes/mi
&& $clipboard =~ /network namespaces.*yes/mi
nixos/tests/chromium: Improve sandbox checking. We no longer need have "SUID sandbox" enabled in the chrome://sandbox status page and we now also check for "You are adequately sandboxed." to be absolutely sure that we're running with proper sandboxing. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-07-02 09:51:52 +01:00
&& $clipboard =~ /seccomp.*sandbox.*yes/mi
&& $clipboard =~ /you are adequately sandboxed/mi;
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 17:41:56 +00:00
};
};
}
'';
})
Reference in New Issue Copy Permalink