Commit Graph

50 Commits

Author SHA1 Message Date
Jeff Wendling
ccb847a329 scripts/tests: write old uplink config yaml directly
sometimes these scripts want to have an access imported
after it has been potentially modified by having the
satellite address and node id added. it used to use the
uplink command to do this, but the cli api for that
has changed. rather than try to have the script detect
which uplink version is in use and call the right thing
it can always write out a valid yaml file and depend on
the new cli migrating it.

Change-Id: Ib82819699333f5f29e00117b99bfb10640033b94
2022-02-22 15:26:06 -05:00
Jeff Wendling
0a860a2e89 scripts: fix builds for new uplink command
the new uplink command expects to be able to migrate
and so we need to specify the --legacy-config-dir
flag sometimes as well. but unfortunately, the old
uplink will error if it gets a flag it doesn't
understand, so we have to set it as an env var.
but we can't use the env var for the --config-dir
flag all the time because the old uplink doesn't
look for that env var.

Change-Id: I019315192c0e6c348814527794342d823a5f9ec3
2022-02-16 21:40:06 +00:00
Artur M. Wolff
785eb93cd7 ci: install gateway@latest instead of @main
Gateway-ST frequent release cycle has been resurrected, which means it's
safer to use the latest release tag in the storj repository's CI now.

Change-Id: I9df1c789a9b9418ba7cceaec9cfec3cc6c448284
2021-11-01 12:16:38 +00:00
Artur M. Wolff
cc9b845a83 ci: switch back to pulling gateway@main
Switch back to pulling gateway@main for continuous integration because
https://review.dev.storj.io/c/storj/gateway/+/5834 was merged. This
change is a partial revert of
https://review.dev.storj.io/c/storj/storj/+/5832.

Change-Id: I0a87f6f18a9a863fe92003b76c830335f9253ced
2021-09-28 15:01:04 +00:00
Egon Elbre
8ef03b0967 ci: cleanup ws before build, fix gateway install
Jenkins uses the same folder for different PR-s. Sometimes other runs do
not cleanup after themselves (e.g. timeout), hence add a cleanWs step to
ensure we delete files in the workspace.

gateway-st introduced a replace directive in go.mod, which does not work
with go install. Hardcode to the last version without the directive.
Using this fix to unblock ci builds.

Change-Id: I5e5d75bf47e30a5a8b6d835867c0c9176f25e08a
2021-09-23 15:57:47 +03:00
Yaroslav Vorobiov
b426248936 scripts/tests: fix missing multinode binary
Change-Id: I288d06c9eaa814ddca1e4aaacd86c5277aba86cc
2021-05-14 22:47:05 +00:00
Kaloyan Raev
a264a4422b cmd: remove metainfo-migrator and metainfo-verifier
We already merged the multipart-upload branch to main. These two tools
make only sense if we are migrating a satellite from Pointer DB to
Metabase. There is one remaining satellite to migrate, but these tools
should be used from the respective release branch instead of from main.

Removing these tools from main will:
1) Avoid the mistake to use them from the main branch instead of from
the respective release branch.
2) Allow to finally remove any code related to the old Pointer DB.

Change-Id: Ied66098c5d0b8fefeb5d6e92b5e0ef5c6603df5d
2021-04-08 14:50:29 +00:00
Michał Niewrzał
a5bc38b197 build: replace 'multipart-upload' branch occurences
We removed `multipart-upload` branches but we have few leftovers.

Change-Id: I74f637748a248cb286cd190a3a8f895ef85f3940
2021-03-31 11:06:47 +02:00
Michał Niewrzał
9bc2a9a4b7 rollingupgrade: change how metainfo-migration tool is installed
Change-Id: Iad1285a770f76c9cf09a872a4bf1cd39969ee117
2021-03-26 15:03:39 +00:00
Michał Niewrzał
0ef0a3f967 rollingupgrade: add migration to metabase
For time of transition from pointerdb to metabase we need add migration step to rollingupgrade tests and comment few cases.

Change-Id: Ib12ae6aa14be35f9bf4ff3efb55cfc6957d4ceba
2021-03-25 10:52:35 +00:00
Michał Niewrzał
908a96ae30 Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: I075aaff42ca3f5dc538356cedfccd5939c75e791
2021-02-11 11:48:23 +01:00
Moby von Briesen
520da5b1e4 scripts/tests/rollingupgrade: fix rollingupgrade test
Fix logic that determines whether satellite has the marketing web server
based on version number.

Change-Id: I34df745d16acf1ce1478fe992448b18565e6d363
2021-02-08 20:06:00 -05:00
Yingrong Zhao
babe01f229 scripts/tests/rollingupgrade: fix rollingupgrade test backwards
compatibility

Change-Id: I1551f993bc8ca5e6fac07da7710dd46dbb031e47
2021-02-08 21:37:24 +00:00
Michał Niewrzał
ad3e3a38c5 Merge 'main' branch
Change-Id: Ia0db1b1f9ef3e0671d3f2208881b0abc3064e200
2021-01-04 12:13:45 +01:00
Stefan Benten
7f1871b8f1 all: switch from master to main 2020-12-28 22:59:06 +01:00
Kaloyan Raev
cc9e9ee1f5 storj-sim: use gateway from multipart-upload branch
Change-Id: I0886d277b3b757c8b00975a3e95c2d0d1228488b
2020-12-04 15:15:47 +02:00
Egon Elbre
e19fabc880 scripts/tests/rollingupgrade: fix typo in flag
Change-Id: Ia3e8d076741a30fb2a42af9b2621796a814c75ae
2020-11-18 19:15:42 +00:00
Egon Elbre
8da5e6a554 scripts/tests/rollingupgrade: use wait-for instead of sleep
Change-Id: Ie879e061d3b312705726375953767d420e922073
2020-11-18 12:00:16 +00:00
littleskunk
9ab824d3e6
jenkins/rollingupgrade: sleep 5 seconds between old api startup and database migration (#3971) 2020-11-16 21:25:11 +01:00
Jessica Grebenschikov
99c88efbbf scripts/tests: fix gateway tests
Change-Id: I9a23ef08794043ad615066ae5929df9ff3a02d69
2020-10-27 08:21:28 -07:00
Yingrong Zhao
746cbfc659 scripts/tests/rollingupgrade: test current release version on master
branch

Currently, we are testing previous release version upgrading to latest
master on each master build
However, this behavior is only desired when the test is running on a
release branch.

Change-Id: Iaeb66f44951c9e4934ca3c8316d1e490d7958239
2020-10-22 11:45:54 -04:00
Kaloyan Raev
bd177bff03 cmd/storj-sim: cleanup gateway setup
Remove usage of --non-interactive flag. It is not provided (and
necessary) by the multitenant S3 gateway anymore.

ACCESS_KEY and SECRET_KEY env vars are not provided anymore as they are
not generated by the multitenant S3 gateway.

Change-Id: I3ecfb92110e31ae13977de3899dad273daae6c1e
2020-10-06 14:22:47 +00:00
Ethan Adams
66f5368807
satellite/testing: Change testing to use PG 12.3 (#3913)
Co-authored-by: Stefan Benten <mail@stefan-benten.de>
2020-06-25 20:17:39 +03:00
Stefan Benten
db57e5cf7b
scripts/tests/rollingupgrade: continue for v1.6.4 (#3912) 2020-06-25 13:57:39 +02:00
Yingrong Zhao
c78f88beb6 scripts/tests: use the same version for storj-sim as satellites
Change-Id: If85b14a0a5af334b9547d73bc4dbe65cdd856984
2020-06-05 13:17:17 -04:00
Yingrong Zhao
175f048aa7 scripts;cmd/storj-sim: include satellite node id in satellite address
Expose satellite ID from storj-sim so we can have access to it when
changing satellite address.

Change-Id: Ife816f4d35eae2d0bc9f9ad592fe75d73d93d9ff
2020-06-05 09:44:19 -04:00
Yingrong Zhao
9a04ca0527 scripts/tests/rollingupgrade: skip v1.6.3
We removed the ability to change satellite address in for uplink cli v1.6.3. Therefore we will skip that version for now

Change-Id: I0f78cfe27b51fd6cad571636ba38266f5f672d58
2020-06-05 09:00:36 +00:00
Michal Niewrzal
056ae7ffa8 scripts/test/rollingupgrade: set correct satellite address for imported
access

Addition: use always latest gateway release

Change-Id: I5e5231e6da4b6f7900cb71bb6e227901474270ea
2020-06-04 15:04:10 +00:00
Yingrong Zhao
4ab553cf3f scripts: only latest releases have version file
Change-Id: Ice212ac39b011c18b1d3d48ea9e6f580b4fd7f0c
2020-04-04 18:47:33 +00:00
Yingrong Zhao
b3939bc1ba scripts/tests: fix test-version and rolling-upgrade test installation
Change-Id: I2c1262aa2e4fd1f5aa34bffe9c76f272fe076d49
2020-04-02 22:07:06 +00:00
Michal Niewrzal
d444fbadea scripts: cleanup rolling upgrade test
* add script for easy rolling upgrade test local execution
* remove unneeded binaries building for rolling upgrade and versions
tests
* unify build process for Jenkins and local execution for rolling
upgrade and versions tests

Change-Id: Ic11211b83f3f447494bbd5827d2af77ea4b20dfe
2020-04-01 12:30:08 +00:00
Michal Niewrzal
0374e30678 scripts: fix storj-sim installation function for rolling upgrade tests
Change-Id: Ifb749e0a41c57b90cc03293c2588b2ceb6a0a15a
2020-03-31 09:11:00 +00:00
Yingrong Zhao
831668478a scripts/tests: fix gateway installation
Change-Id: I1629644f543505c27d4edb8f7bbe97d037cdc8a8
2020-03-29 19:24:05 +00:00
JT Olio
3b66ba6f02 scripts/tests: uplink no longer respects --client.segment-size
this is going to make all the tests slower but it is what it is

test-sim-aws.sh is removed because it was moved to storj/gateway repo.

Change-Id: I10727e747a4c3740b1c9054ce7d17313b4fa310b
2020-03-22 17:48:57 +00:00
littleskunk
b10b69d9ce
test/rollingupgrade: fix stage 1 release version (#3810) 2020-03-17 22:30:07 +01:00
Yingrong Zhao
1a875baa1d scripts/tests: fix arguments orders passed to test-versions.sh
Change-Id: I2e3b501477823413137f6a43fcdc4347af27e43c
2020-03-11 13:50:18 +00:00
Yingrong Zhao
46b04a38cc scripts/tests/: fix uplink access to contain satellite id
Change-Id: I7dfb6bc2da3bf84a81d75a286ee9488e32ea4f01
2020-03-10 21:15:39 +00:00
littleskunk
842c8d8ed9
scripts/tests/rollingupgrade: fix installation for current commit 2020-03-06 17:19:55 -05:00
littleskunk
8fa8178f04
release/rollingupgrade: on release tags run rolling upgrade against previous release (#3792)
Co-authored-by: Stefan Benten <mail@stefan-benten.de>
2020-03-03 23:56:32 +01:00
Yingrong Zhao
ac34485f5d scripts/tests: install correct version of gateway
1. only run release tags that don't contain 'rc'
2. install gateway version that's the same as satellite
3. update gateway access to contain satellite id

Change-Id: I8ca1418302c3aafdf0c4eaaf8361422a1eec2bd4
2020-02-26 13:12:31 +00:00
Egon Elbre
e30f7b35b6 cmd/gateway: use a separate repository
Change-Id: Idbb0b2b6cf0e60c6d5d91218c24524d72285cf26
2020-02-24 10:03:03 +02:00
Moby von Briesen
8c19855871 scripts/tests/rollingupgrade: explicitly set debug port for old
satellite api during rolling upgrade test

The old api is using the same config file as the new satellite in the
rolling upgrade test, so we need to set it to something different so
that there is no conflict when we spin up a new storj-sim instance while
the old api is running concurrently.

Change-Id: Ia4ec2db4953f36f43275495710992831ad3916a2
2020-01-29 18:32:03 -05:00
Yingrong Zhao
5de4f66553 scripts/tests: change multisegment file to be 128kb
To cover a special case: an object that has 2 remote segments
and 1 inline segment.

Change-Id: Ia8d82bb67fc6cf76af9c7f44cd738cab6df591e9
2020-01-22 17:12:11 +00:00
Moby von Briesen
d32626fe8c scripts/tests: update uplink config migration for test versions
Updates config migration to occur for any v0.30.x release rather than
specifically 30.4

Also updates the config for the rolling upgrade test to use 64 kib
segments, and use smaller files for the final upload of rolling upgrade.

Change-Id: I941f77fe2b9011b45f28a5f3a2430e882d2ae6b3
2020-01-21 11:50:13 -05:00
littleskunk
b6f1a91c67 scripts/testversions,rollingupgrade: remove encrytion key
Change-Id: I6fd35fa4b29707f53e988bd00d6523b934767ecc
2020-01-16 16:01:34 +00:00
littleskunk
1e77cb88e7
scripts/rollingupgrade: replace apikey with access
Change-Id: I356587cdc417dabc6f15769592d70269d25051dc
2020-01-16 14:00:33 +01:00
Moby von Briesen
ea84af578b scripts/tests/rollingupgrade: create new test files for final upload
stage

The test-versions script no longer uses the `testfiles` directory, which
the final upload for the rolling-upgrade script depended on. This change
creates and populates a `testfiles` diirectory during the final upload
stage of the rolling upgrade test.

Change-Id: Iabeccbadc55a8c85a1febbd5eb4e7d889a57a8dc
2020-01-06 12:31:12 -05:00
Yingrong Zhao
07a1702f41 scripts/tests/rollingupgrade: fix test-versions.sh path referrence
Change-Id: I5c696e5d38c087c50f025796e2f48876883d0f4a
2020-01-04 19:42:15 -05:00
Moby von Briesen
e34ac3ef3a ci,scripts/tests/rolling-upgrade: run rolling upgrade test on private jenkins
Change-Id: Ic1c9f7539ee0ac371bcb856bdbcac2ff6c0ccc65
2020-01-02 16:27:41 -05:00
Moby von Briesen
aecea820fc scripts: add rolling upgrade test script
Change-Id: Ibf79c8e40da54520ce17e2e1f66124c117b32b53
2020-01-02 13:38:56 -05:00