Commit Graph

91 Commits

Author SHA1 Message Date
Egon Elbre
7802ab714f pkg/,private/: merge with private package
Initially there were pkg and private packages, however for all practical
purposes there's no significant difference between them. It's clearer to
have a single private package - and when we do get a specific
abstraction that needs to be reused, we can move it to storj.io/common
or storj.io/private.

Change-Id: Ibc2036e67f312f5d63cb4a97f5a92e38ae413aa5
2021-04-23 16:37:28 +03:00
Michał Niewrzał
a5bc38b197 build: replace 'multipart-upload' branch occurences
We removed `multipart-upload` branches but we have few leftovers.

Change-Id: I74f637748a248cb286cd190a3a8f895ef85f3940
2021-03-31 11:06:47 +02:00
Egon Elbre
74b243d1ca certificate: move protobuf to storj/storj repository
Change-Id: I4689318304b54121a65e28860430d39d94ac5bd6
2021-03-29 12:59:49 +00:00
Michał Niewrzał
237782813b Merge remote-tracking branch 'origin/multipart-upload'
Change-Id: If6c5a450b238adab55d1e0dea67d01e5f5768a9f
2021-03-23 09:44:49 +01:00
Ivan Fraixedes
536990dcf6 Jenkins: Run integration test Redis unavailability
Make Jenkins to run the integration tests that verifies that the
satellite can operates when Redis is unavailable.

Change-Id: Idace3ffb84c788f2af2c6e24eec1d63fb40c263a
2021-03-22 09:56:30 +00:00
Michał Niewrzał
d995fb497f Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: I367da03351ab80f7343332420490dde9282aa47a
2021-02-23 12:31:31 +01:00
Egon Elbre
a34da8531b ci: bump test timeout
Currently testing with coverage exceeds the timeout very easily.

Change-Id: I4e77aa98d28bebe373416ac05ee39b4c70c3fed2
2021-02-22 17:31:21 +02:00
Egon Elbre
21a6dccd36 ci: set specific line coverage targets
Currently the auto-updating keeps moving the target upwards without
allowing for variations in the pipeline.

Let's try these settings:

* >=70% lines covered, to pass
* <=60% lines covered, to mark as unhealthy
* <=50% lines covered, to mark as unstable

Change-Id: Idc75e682345f6d62af34379362c75d321e564d2c
2021-02-22 15:13:18 +02:00
Egon Elbre
bc68898846 ci: enable coverage for multipart-upload branch
Change-Id: I4797d16bbc90a3fc7fc9627386b63124755f4730
2021-02-22 13:43:32 +02:00
Michał Niewrzał
341a4b7f59 metainfo-migration: extend test case
Change-Id: I9d1ac41a18bb08200ef20f7ff2f8df5531140f99
2021-02-11 19:28:41 +01:00
Michał Niewrzał
5a23d9c167 metainfo-migrator: improve performance
Change-Id: I258605737c37dfdd8f0070dbadf0e3a9c6358cc3
2021-02-11 17:33:22 +01:00
Egon Elbre
2848bf488f satellite/metainfo/metabase: use alias pieces in segments table
This makes all tables automatically convert between aliases and piece
ID-s.

Change-Id: I27fa42c82bbb09e05e3327f85e13a000b48faffd
2021-02-11 16:40:37 +02:00
Kaloyan Raev
d0612199f0 Merge remote-tracking branch 'origin/main' into multipart-upload
Conflicts:
	go.mod
	go.sum
	satellite/metainfo/config.go
	satellite/metainfo/metainfo_test.go

Change-Id: I95cf3c1d020a7918795b5eec63f36112fdb86749
2021-02-01 14:32:12 +02:00
Egon Elbre
a700a1bdab satellite/metainfo/metabase: add benchmark
Change-Id: I4289aac252f9c951548444aa5a82e007e9f47806
2021-01-25 14:59:24 +00:00
Jessica Grebenschikov
b7d8dee5e9 satellite/console/wasm: add js tests
Change-Id: I8b1b0e81500836e0408e0517edb6c696698ab5f7
2021-01-21 20:18:03 +00:00
Jessica Grebenschikov
1f1d9fce58 satellite/console/wasm: add test to confirm wasm size isnt growing
Change-Id: I975a9f8ac3f6b98cc213140fdd7a99557efe14c8
2021-01-21 15:48:49 +00:00
Michał Niewrzał
ac058e5ecc metainfo-migration: basic pointerdb->metabase migrator
Change-Id: If183b9c99ce816521fcae6e7189a6f85ddd4eb48
2021-01-12 12:59:53 +00:00
Michał Niewrzał
ad3e3a38c5 Merge 'main' branch
Change-Id: Ia0db1b1f9ef3e0671d3f2208881b0abc3064e200
2021-01-04 12:13:45 +01:00
Stefan Benten
02d7638eda
Jenkinsfile.public: specify main branch for docker build
Docker build defaults to the master branch, when specifying a git repo.
With this change, it should be smart enough to use the proper main branch.
2020-12-29 16:56:36 +02:00
Stefan Benten
7f1871b8f1 all: switch from master to main 2020-12-28 22:59:06 +01:00
Kaloyan Raev
bafc6af992 ci: remove workaround for failing tests
Change-Id: I3eb673fae6c81bee17d7437cb870d5f5ba6978d5
2020-12-21 18:07:40 +02:00
Michal Niewrzal
bc3be9c416 Jenkinsfile: enable build failure on integration tests failure
Change-Id: Ifabcd9e72cd51de1300b5f8a4453c87ca292c60e
2020-12-07 15:31:46 +00:00
Kaloyan Raev
402a581b86 ci: fail build if metabase tests fail
Change-Id: Ic263ad2dcf777d1fb7c03bbddc9edd09a2165c83
2020-11-17 13:30:15 +00:00
Michal Niewrzal
7c384c8293 Merge 'master' branch
Change-Id: I1eefd5a56449e577820977d61fa4a22bdd4fc230
2020-11-16 10:02:54 +01:00
Egon Elbre
c0b5e7ce3e ci: ensure cockroach doesn't pollute repo
Cockroach 20.2 started automatically profiling, this is a workaround to
disable it and ensure it doesn't create any folders and files in the
repository.

Change-Id: Ib65de01ea1fc619160d710c01602ced3a3a3492e
2020-11-13 16:07:01 +00:00
Egon Elbre
2ff7925e65 ci: set GOTRACEBACK=all
Currently when there's a timeout or panic, the culprit goroutines might
not be printed. Set traceback to all, which prints all user created
goroutines.

Change-Id: I29f87812d2a60f671b3eb172499e24cf70d990b5
2020-11-11 13:57:45 +02:00
Michal Niewrzal
dbf9f121ed ci: mark failures as unstable
Change-Id: I3cfa07a35cb789f409d95d4778c07ca3355c0f3c
2020-11-09 16:48:38 +01:00
Egon Elbre
754838f5d0 ci: convert test failures to unstable
Change-Id: Ib0252c666c086379bd6e2a3a47011479fe3988ed
2020-11-09 11:18:12 +00:00
Egon Elbre
0a77deec81 satellite/metainfo/metabase: basic migrate
Change-Id: Ia34f38e763af9ba88f75038b8ecfc9eb6550eae0
2020-11-05 16:31:02 +02:00
Michal Niewrzal
809eb14ac2 satellite/metainfo/metabase: move metainfo PoC into storj repo
Change-Id: I39356d8bc7305b4a8ea0c1fb5603010ad72a68b9
2020-10-29 09:44:55 +01:00
Egon Elbre
d508c4c985 scripts: remove duplicate check config lock
We have configlock_test for checking changes so the separate script and
makefile target are not needed.

Change-Id: I2bbc1c21ad849c9b7ec8bba43c0e11e94e04f6a6
2020-09-29 10:03:34 +00:00
Egon Elbre
2d27bc8787 satellite/satellitedb: separate cockroach for migration tests
Currently Cockroach migration test is the most heavy with regards to
schema changes. This causes other tests to time out. This adds an
alternate cockroach instance that is used for migration tests.

Change-Id: I01fe9313527ff002f0bb0914dd52c3645b8eaf6d
2020-09-29 09:31:33 +00:00
Egon Elbre
9fd97fa973 ci: add check-monkit
Change-Id: I124d013ffb434fdea8ec315face090de4e572ebf
2020-09-08 17:20:27 +03:00
Egon Elbre
9dc9cd8a17 tests: allow STORJ_TEST_POSTGRES
STORJ_POSTGRES_TEST naming was not consistent with STORJ_SIM_POSTGRES.

This allows to use STORJ_TEST_POSTGRES for clarity, it still has a
fallback to STORJ_POSTGRES_TEST.

Change-Id: I6f294c66c80fcfd6750fea2a89795f3b7f5dd691
2020-07-10 16:43:49 +03:00
Egon Elbre
5bdcd86fa7 ci: test benchmarks
This runs each benchmark for one iteration to ensure that they are
valid. Unfortunately, it does not give any useful metrics as output.

Change-Id: I68940398c8dd849aed656bd12656f48d5df10128
2020-07-10 13:26:49 +00:00
Bill Thorp
3aa3732b0d jenkins: Enable Cobertura failing for code coverage
Measure code coverage "by line" (Go actually measures by block, but whatever)
The '100, 0, 0' means anything above 100 auto-passes, anything below 0 auto-fails,
and anything below 0 is reported as unstable. autoupdateHealth, if I understand
correctly, will raise the functional auto-fail limit each time, so that a drop
in coverage will result in a failed test.  Finally failUnhealty is the trigger
which actually causes the build to fail, rather than simply reporting an error.

I have tested these changes using replay, but I was unable to verify 100% that
they work as designed.  The risks of having to revert this code seems less than
the cost of spending more time playing around with Jenkins locally..

Change-Id: Ica855b36aef7e1c3023fa80611229ca691017021
2020-06-02 16:00:08 +00:00
Egon Elbre
ff7c726e99 ci: fix coverage flags
Change-Id: If9a87ac383eed3b8c7acf03938f44696af6c6f9c
2020-05-29 15:46:25 +03:00
Egon Elbre
d9fec04f50 lib: remove the old implementation
The stable uplink library is located at storj.io/uplink.
The stable uplink C library is located at storj.io/uplink-c.

Change-Id: Ia69115d6384b668490efe7e8ab674d7e8104b4f4
2020-05-29 11:36:43 +00:00
Egon Elbre
85c45cd56f private/dbutil/pgtest: support multiple databases for testing
Currently Cockroach isn't performant for concurrent database setup and
tear-down. Instead of a single instance allow setting multiple potential
connection strings and let the tests pick one connection string
randomly.

This improves test duration by ~10 minutes.

While we are at significantly changing how pgtest works, introduce
helper PickPostgres and PickCockroach for selecting the database to
reduce code duplications in multiple places.

Change-Id: I8ad171d5c4c8a4fc081ec2ae9bdd0cc948a80619
2020-04-28 21:55:49 +03:00
Jeff Wendling
bde302fdb8 jenkins: disable npm audit as its failing everything
Change-Id: I399e73b86144588758e50bdebbd0e65e8052c8b6
2020-02-10 13:32:40 -07:00
Egon Elbre
91a480f5a0 Jenkins: add storagenode npm checks
Change-Id: I93e3cc009c628e3c97a24541e7b01c75a342bda6
2020-02-03 20:35:33 +02:00
Jessica Grebenschikov
c353e8b108 run backwards compat tests with cockroachdb
Change-Id: I1de9a6e6e0681825a5d021e168f3a116591d28c2
2020-01-21 07:42:49 -08:00
Egon Elbre
76a6b28b33 ci: use tmpfs for running tests
Change-Id: I6278899bd158759c6fc0bceafd6a6c7244860f4a
2020-01-17 20:28:15 +00:00
Egon Elbre
082ec81714
uplink: move to storj.io/uplink (#3746) 2020-01-08 15:40:19 +02:00
Egon Elbre
fb4b11d13e
scripts: remove old scripts (#3742) 2020-01-07 13:28:41 +02:00
Egon Elbre
91947311f5 ci: always try to pull latest image
Change-Id: Ic1422a96705f8e66876f5c724060d7c389c5da3d
2020-01-06 15:28:04 +00:00
Egon Elbre
3e873158f4 ci: increase cockroach max-memory to reduce flakiness
Change-Id: I88343c8668ee57abfdc30f2d8d4772c61e636b78
2020-01-02 11:07:00 +00:00
Egon Elbre
6615ecc9b6 common: separate repository
Change-Id: Ibb89c42060450e3839481a7e495bbe3ad940610a
2019-12-27 14:11:15 +02:00
Egon Elbre
acb4435a67 satellite/satellitedb: improve Cockroach migrate test
Load schemas in parallel instead of one-by-one.

Optimizes from 2m30s to 1m15s.

Change-Id: I0bf6381a0ae99b44271fe55d4ee658683064c097
2019-12-21 10:58:43 +00:00
Egon Elbre
ef8bc88328 ci: use external repository
Change-Id: If26a005df45f6067240511d603fb4dd613f92b79
2019-12-19 12:05:49 +00:00