Commit Graph

43 Commits

Author SHA1 Message Date
Egon Elbre
947cdf0fb4 ci: fixes for new Go version
We need to make exceptions for older uplink versions, because it does
not compile with newer Go versions due to quic dependency.

Change-Id: I3e073694f0942029c56740f0689088058ee068c3
2022-11-02 15:17:46 +02:00
JT Olio
098ee71e03 test-sim-versions: write what bash step is executing
Change-Id: If1231f69601c1774bc6ce116f45105cc1b2f4e22
2022-06-10 19:33:03 +00:00
Jeff Wendling
ccb847a329 scripts/tests: write old uplink config yaml directly
sometimes these scripts want to have an access imported
after it has been potentially modified by having the
satellite address and node id added. it used to use the
uplink command to do this, but the cli api for that
has changed. rather than try to have the script detect
which uplink version is in use and call the right thing
it can always write out a valid yaml file and depend on
the new cli migrating it.

Change-Id: Ib82819699333f5f29e00117b99bfb10640033b94
2022-02-22 15:26:06 -05:00
Jeff Wendling
863b01bf09 testversions: skip some downloads for older uplinks
uplink command versions >= 1.48.0 always do multipart
uploads which cannot be downloaded by any of the
versions in the test < v1.27.6. so skip those tests.

Change-Id: I9644afbd14bfce9facfd87644d132f7d66367d62
2022-02-22 17:24:07 +00:00
Jeff Wendling
0a860a2e89 scripts: fix builds for new uplink command
the new uplink command expects to be able to migrate
and so we need to specify the --legacy-config-dir
flag sometimes as well. but unfortunately, the old
uplink will error if it gets a flag it doesn't
understand, so we have to set it as an env var.
but we can't use the env var for the --config-dir
flag all the time because the old uplink doesn't
look for that env var.

Change-Id: I019315192c0e6c348814527794342d823a5f9ec3
2022-02-16 21:40:06 +00:00
Artur M. Wolff
785eb93cd7 ci: install gateway@latest instead of @main
Gateway-ST frequent release cycle has been resurrected, which means it's
safer to use the latest release tag in the storj repository's CI now.

Change-Id: I9df1c789a9b9418ba7cceaec9cfec3cc6c448284
2021-11-01 12:16:38 +00:00
Qweder93
50baefa10e satellite/metabase: limit maximum number of parts and size
Multipart upload limits added. Last part has no size limit.
Max number of parts: 10000, min part size: 5 MiB

Change-Id: Ic2262ce25f989b34d92f662bde720d4c4d0dc93d
2021-10-22 10:16:37 +00:00
igor gaidaienko
5b4a9070b0 scripts/testversions: reduced the number of version to test 2021-10-12 13:28:46 +03:00
Artur M. Wolff
cc9b845a83 ci: switch back to pulling gateway@main
Switch back to pulling gateway@main for continuous integration because
https://review.dev.storj.io/c/storj/gateway/+/5834 was merged. This
change is a partial revert of
https://review.dev.storj.io/c/storj/storj/+/5832.

Change-Id: I0a87f6f18a9a863fe92003b76c830335f9253ced
2021-09-28 15:01:04 +00:00
Egon Elbre
8ef03b0967 ci: cleanup ws before build, fix gateway install
Jenkins uses the same folder for different PR-s. Sometimes other runs do
not cleanup after themselves (e.g. timeout), hence add a cleanWs step to
ensure we delete files in the workspace.

gateway-st introduced a replace directive in go.mod, which does not work
with go install. Hardcode to the last version without the directive.
Using this fix to unblock ci builds.

Change-Id: I5e5d75bf47e30a5a8b6d835867c0c9176f25e08a
2021-09-23 15:57:47 +03:00
Yaroslav Vorobiov
b426248936 scripts/tests: fix missing multinode binary
Change-Id: I288d06c9eaa814ddca1e4aaacd86c5277aba86cc
2021-05-14 22:47:05 +00:00
Michał Niewrzał
a5bc38b197 build: replace 'multipart-upload' branch occurences
We removed `multipart-upload` branches but we have few leftovers.

Change-Id: I74f637748a248cb286cd190a3a8f895ef85f3940
2021-03-31 11:06:47 +02:00
Moby von Briesen
2a68e0767e scripts/tests/testversions: Fix testversions Episode 2
The previous fix did not remove the comment at the front of the config
values that were modified.

Change-Id: I05729c30a5398d049c6254afec01df00230d8deb
2021-03-25 18:06:13 -04:00
Moby von Briesen
63cd807c9f scripts/tests/testversions: Increase bandwidth/storage limits for testversions
A recent change made the default usage/storage limits for projects 50gb
rather than 500gb. This increases the default limit for testversions.

Change-Id: Ibea05c0d0760662e447b6455d560a2a640801c6c
2021-03-25 14:43:28 -04:00
Michał Niewrzał
ad3e3a38c5 Merge 'main' branch
Change-Id: Ia0db1b1f9ef3e0671d3f2208881b0abc3064e200
2021-01-04 12:13:45 +01:00
Stefan Benten
7f1871b8f1 all: switch from master to main 2020-12-28 22:59:06 +01:00
Michal Niewrzal
218bbeaffa Merge 'master' branch
Change-Id: Ica5c25607a951076dd9f77e35e308062f71ce3f0
2020-12-07 15:05:52 +01:00
Yingrong Zhao
746315672f scripts/tests/testversions: fix indentation
Change-Id: Iaa5aec27f0ad78e1d8bf1a68aa5a62762c8ab537
2020-12-04 21:54:55 +00:00
Yingrong Zhao
0faf7d5293 scripts/tests/testversions: fix race in install_sim
Change-Id: I0792686d99a222d5977fd913425e2b94d100c40e
2020-12-04 18:18:14 +00:00
Kaloyan Raev
cc9e9ee1f5 storj-sim: use gateway from multipart-upload branch
Change-Id: I0886d277b3b757c8b00975a3e95c2d0d1228488b
2020-12-04 15:15:47 +02:00
Yingrong Zhao
13555f3983 scripts/tests/testversions: enable concurrent installation for each
version

The test-versions test currently takes 1h 40min to run each time. By
running each installation concurrently, hopefully, it will reduce the execution
time for the whole test.

Change-Id: I680c7d9945e982894b11825c9075c167f754e087
2020-12-03 15:01:37 +00:00
Michal Niewrzal
7c384c8293 Merge 'master' branch
Change-Id: I1eefd5a56449e577820977d61fa4a22bdd4fc230
2020-11-16 10:02:54 +01:00
Yingrong Zhao
54c5d564a1 scripts/tests/testversions: fix older uplink setup
This PR does follwing changes:
    1. Change oldest uplink version in the test to v0.35.3
        When the test is first created, we decided to support uplink
        version starting from v0.17.1, however with many API changes,
        older uplinks are not usable with latest version of the network
        anymore. One of the reasons being older uplinks uses deprecated
        endpoint. Therefore, we will change the oldest uplink version to
        the one that's using only new endpoints.
    2. Disable tls certificate verification in uplink
    3. Use storj-sim version control server instead of production one
    4. Skip uplink version v1.3.x due to bug in that release

Change-Id: I926a6bb9829cb7181ee752437cdcb67e59197fe0
2020-11-11 17:00:01 -05:00
Yingrong Zhao
8fd841b910 scripts/tests/testversions: fix installation during setup
This PR fixes below issues:
1. remove concurrent installation for various versions
    We were doing this to decrease the amount of execution time the versions test.
    However, it's returning incorrect exit code when there's an
    installation failure.
    Right now, we are only installing two versions of `storj-sim` and
    the rest are only doing uplink cli installation. The performance of
    this test should be hugely impacted by the setup step now.
2. only remove release settings instead of deleting the entire file
    uplink CLI is referrencing `private/version` package. Therefore, we
    cannot delete it
3. add back `GATEWAY_0_API_KEY` in storj-sim
    In order to set up older version of uplink cli, we need access to
    the gate way api key.

Change-Id: Ia3c37c197bd007b6e1f7c2bd71adde42181d46f0
2020-11-10 20:38:49 +00:00
Ethan Adams
66f5368807
satellite/testing: Change testing to use PG 12.3 (#3913)
Co-authored-by: Stefan Benten <mail@stefan-benten.de>
2020-06-25 20:17:39 +03:00
Yingrong Zhao
c78f88beb6 scripts/tests: use the same version for storj-sim as satellites
Change-Id: If85b14a0a5af334b9547d73bc4dbe65cdd856984
2020-06-05 13:17:17 -04:00
Yingrong Zhao
4ab553cf3f scripts: only latest releases have version file
Change-Id: Ice212ac39b011c18b1d3d48ea9e6f580b4fd7f0c
2020-04-04 18:47:33 +00:00
Yingrong Zhao
b3939bc1ba scripts/tests: fix test-version and rolling-upgrade test installation
Change-Id: I2c1262aa2e4fd1f5aa34bffe9c76f272fe076d49
2020-04-02 22:07:06 +00:00
Michal Niewrzal
d444fbadea scripts: cleanup rolling upgrade test
* add script for easy rolling upgrade test local execution
* remove unneeded binaries building for rolling upgrade and versions
tests
* unify build process for Jenkins and local execution for rolling
upgrade and versions tests

Change-Id: Ic11211b83f3f447494bbd5827d2af77ea4b20dfe
2020-04-01 12:30:08 +00:00
Michal Niewrzal
0374e30678 scripts: fix storj-sim installation function for rolling upgrade tests
Change-Id: Ifb749e0a41c57b90cc03293c2588b2ceb6a0a15a
2020-03-31 09:11:00 +00:00
Yingrong Zhao
831668478a scripts/tests: fix gateway installation
Change-Id: I1629644f543505c27d4edb8f7bbe97d037cdc8a8
2020-03-29 19:24:05 +00:00
Michal Niewrzal
3a3648c80b scripts: add script for running versions tests locally
Change-Id: Iaa2a2d085b7edd46f63a0d79b4e731ea72412953
2020-03-24 15:44:00 +00:00
JT Olio
3b66ba6f02 scripts/tests: uplink no longer respects --client.segment-size
this is going to make all the tests slower but it is what it is

test-sim-aws.sh is removed because it was moved to storj/gateway repo.

Change-Id: I10727e747a4c3740b1c9054ce7d17313b4fa310b
2020-03-22 17:48:57 +00:00
Yingrong Zhao
1a875baa1d scripts/tests: fix arguments orders passed to test-versions.sh
Change-Id: I2e3b501477823413137f6a43fcdc4347af27e43c
2020-03-11 13:50:18 +00:00
Yingrong Zhao
46b04a38cc scripts/tests/: fix uplink access to contain satellite id
Change-Id: I7dfb6bc2da3bf84a81d75a286ee9488e32ea4f01
2020-03-10 21:15:39 +00:00
Yingrong Zhao
ac34485f5d scripts/tests: install correct version of gateway
1. only run release tags that don't contain 'rc'
2. install gateway version that's the same as satellite
3. update gateway access to contain satellite id

Change-Id: I8ca1418302c3aafdf0c4eaaf8361422a1eec2bd4
2020-02-26 13:12:31 +00:00
Egon Elbre
e30f7b35b6 cmd/gateway: use a separate repository
Change-Id: Idbb0b2b6cf0e60c6d5d91218c24524d72285cf26
2020-02-24 10:03:03 +02:00
Yingrong Zhao
5de4f66553 scripts/tests: change multisegment file to be 128kb
To cover a special case: an object that has 2 remote segments
and 1 inline segment.

Change-Id: Ia8d82bb67fc6cf76af9c7f44cd738cab6df591e9
2020-01-22 17:12:11 +00:00
Moby von Briesen
d32626fe8c scripts/tests: update uplink config migration for test versions
Updates config migration to occur for any v0.30.x release rather than
specifically 30.4

Also updates the config for the rolling upgrade test to use 64 kib
segments, and use smaller files for the final upload of rolling upgrade.

Change-Id: I941f77fe2b9011b45f28a5f3a2430e882d2ae6b3
2020-01-21 11:50:13 -05:00
Moby von Briesen
0def7a9d2a scripts/tests/testversions;scripts/tests/rollingupgrade: update test versions script
Fix uplink setup step for uplink versions that requires an access field.

Update how script selects uplink versions to test.

Use significantly smaller remote files for test (performance).

Change-Id: If590b8798767e2a0621fb84cd3b8852d02f6d1da
2020-01-20 11:46:11 -05:00
littleskunk
b6f1a91c67 scripts/testversions,rollingupgrade: remove encrytion key
Change-Id: I6fd35fa4b29707f53e988bd00d6523b934767ecc
2020-01-16 16:01:34 +00:00
littleskunk
0c365d157f
scripts/testversions: replace apikey with access
Change-Id: I4f899cc49b63b2f04f31a6df478475d7bdbab30d
2020-01-16 14:37:31 +01:00
Yingrong Zhao
71c5c2213f scripts/tests/testversions: make binary installation and upload/download running in parallel
Change-Id: I16d87f7e16e2daf30e4d7ee5490b76c175b06930
2020-01-04 16:39:45 +00:00