Commit Graph

258 Commits

Author SHA1 Message Date
Moby von Briesen
e34ac3ef3a ci,scripts/tests/rolling-upgrade: run rolling upgrade test on private jenkins
Change-Id: Ic1c9f7539ee0ac371bcb856bdbcac2ff6c0ccc65
2020-01-02 16:27:41 -05:00
Moby von Briesen
aecea820fc scripts: add rolling upgrade test script
Change-Id: Ibf79c8e40da54520ce17e2e1f66124c117b32b53
2020-01-02 13:38:56 -05:00
Natalie Ventura Villasana
aa3e183c2e
satellite/gracefulexit: add ge eligibility check
Adds check to see if storage nodes are eligible to initiate
graceful exit, by checking their CreatedAt date and seeing if
their "age" is greater than the new config value:
NodeMinAgeInMonths
The default for this value is 6 months for now.

https://storjlabs.atlassian.net/browse/V3-3357

Change-Id: Ib807ab8987ddb5a38a27a83886490f73fe8c5816
2019-12-31 09:31:58 -05:00
Egon Elbre
6615ecc9b6 common: separate repository
Change-Id: Ibb89c42060450e3839481a7e495bbe3ad940610a
2019-12-27 14:11:15 +02:00
Egon Elbre
ea455b6df0 all: remove code to default to grpc
We have moved to drpc so we don't need to have code for building
with grpc only.

Change-Id: I55732314dca0d5b4ce1132b68de4186a15d91b21
2019-12-20 20:12:04 +02:00
Yingrong Zhao
c6854accdf scripts: add test-versions stage to private Jenkins
test-sim-versions.sh tests upgrading the satellite, storagenodes, and uplinks from the most recent release to master, and ensures that compatibility across all uplink versions since v0.15 is maintained.

Change-Id: I80a54236d0eb2d681716caf4b825a883bdc25ef1
2019-12-20 15:52:54 +00:00
Egon Elbre
ef8bc88328 ci: use external repository
Change-Id: If26a005df45f6067240511d603fb4dd613f92b79
2019-12-19 12:05:49 +00:00
littleskunk
d5c5b57fac satellite/db: enable DeleteTallies
Change-Id: I1e2a6873b3e6398260e053592d676993272b960d
2019-12-18 13:16:06 +00:00
Yingrong Zhao
1a625887ed scripts: Add script to automate testing against all highest release
points from major releases starting from v0.15.4 for uplink

Change-Id: I7a3a300466691a47b0324ee5440d70cac42df641
2019-12-17 17:47:17 +00:00
Simon Guindon
a47d7ac89b scripts: Add script that filters postgres plaintext backup to cockroachdb compat.
Change-Id: I457e8f0566186fc76b7ae61db77c01153c3e1079
2019-12-16 22:11:16 +00:00
Simon Guindon
8242eecea6 Adding benchmarking script that reports response times.
Change-Id: Ide6f439849ec51cd41f491eb3ff00a7ad0f8a560
2019-12-16 16:34:01 -05:00
Andrew Harding
cb89496569 storagenode/trust: wire up list into pool
- also updated ping chore to pick up trust changes
- fixed small typo in blueprint
- fixed flags for storj-sim
- wired up changes to testplanet

Change-Id: I02982f3a63a1b4150b82a009ee126b25ed51917d
2019-12-13 20:32:50 +00:00
Jess G
4f282921c4
jenkins: run storj-sim integration tests with cockraochdb (#3723)
* add integration tests to jenksin

* have jenkins run storj-sim integration tests w/crdb

Change-Id: I696d55c5894aaf630dcd7a566e1dd705ee88486b

* rm crdb integration tests to see if postgres passes

Change-Id: I1727a027ff802acbff5fc55961a0d605faefcf2d

* comment out aws tests to see if that is the error

Change-Id: I456c3d36f6a4ce7760ea0b6c402b6ea16cfe77e3

* add aws profile to integration tests

Change-Id: Ic01185dbc7b84ac48dfb846f8f272b34b50379b6

* add tmp path for aws profile and creds

Change-Id: I7b82ee5a99937edd3f66ae01bfb5cb21028a62cf

* change linux KiB syntax to bytes to support osx

Change-Id: Ia1f1027ba8da64a6ba537062deb9b3519973621f
2019-12-10 11:18:02 -08:00
littleskunk
71b58edb2c satellite/repair: decrease repair interval
Change-Id: Id9efdbfaa82521c35dc41e7a52b700522c197e77
2019-12-10 00:36:00 +00:00
littleskunk
6ab72a6e79 satellite/gracefulexit: enable graceful exit in production
Change-Id: I526ce4a4de9c318f1333b793e3167f5f86d65adc
2019-12-09 17:32:34 +00:00
Malcolm Bouzi
18a5e614d9 satellite/web: add segmentio plugin (#3405) 2019-11-27 11:57:59 -05:00
Yingrong Zhao
63e51df9a6
private/testplanet: add a mock referral manager server into testplanet (#3631) 2019-11-21 17:34:49 -05:00
Matt Robinson
976881f72b
satellite/console: Add security headers (#3615)
* satellite/console: Add X-Frame-Options and Referrer-Policy security headers

* Update to use CSP instead of XFO and include tardigrade.io

* Make FrameAncestors a config option

* Update satellite-config lock

* Make help text for FrameAncestors better
2019-11-21 11:15:22 -05:00
Matt Robinson
b5707d1e5d
scripts: make update-tools.sh more verbose (#3572)
* Make update-tools.sh more verbose

* Was checking the wrong filehandle
2019-11-20 09:41:06 -05:00
littleskunk
c52c7275ad
satellite/repair: reduce upload timeout (#3597) 2019-11-18 18:52:56 +01:00
Nikolai Siedov
3fe518d547
satellite: added ability to inject stripe public key post build (#3560) 2019-11-18 13:38:43 +02:00
Jeff Wendling
ecd2ef4a21 all: build release fully dprc and test in mixed mode
Change-Id: I3bded3edf25a0b113601c8b12ecf1337f596649b
2019-11-15 10:03:18 -07:00
Yaroslav Vorobiov
53c6741ba6
satellite/payments: add API for retrieving conversion ratio, convert tokens to USD before applying to balance (#3530) 2019-11-15 16:59:39 +02:00
Yehor Butko
a8e4e9cb03
satellite/payments: project usage charges (#3512) 2019-11-15 16:27:44 +02:00
Egon Elbre
ee6c1cac8a
private: rename internal to private (#3573) 2019-11-14 21:46:15 +02:00
Matt Robinson
b2a7a9f4c4 scripts: add script to update tools (#3570) 2019-11-14 16:53:32 +02:00
Natalie Villasana
1a9757a7f2 satellite/gracefulexit: add count for order limits sent from satellite to exiting node (#3544) 2019-11-13 09:54:50 -05:00
Egon Elbre
994a69cfdc jenkins: use lower segment size for back comp test (#3097) 2019-11-06 05:53:38 -08:00
Egon Elbre
3b18c864dc
test/backwards-compatibility: fix port change (#3509) 2019-11-06 13:46:54 +02:00
Yaroslav Vorobiov
0b32690d0a satellite/peer: add payments config (#3488)
* satellite/peer: add payments config

* remove stripe-key from console config

* update config lock

* fix imports

* fix config-lock
2019-11-05 21:26:19 +01:00
littleskunk
def3dcbaa9
satellite/audit: increase timeout to 5 minutes (#3480)
* satellite/audit: increase timeout to 5 minutes

* fix lint error
2019-11-05 11:21:25 +01:00
Jess G
5abb91afcf
satellite: change the Peer name to Core (#3472)
* change satellite.Peer name to Core

* change to Core in testplanet

* missed a few places

* keep shared stuff in peer.go to stay consistent with storj/docs
2019-11-04 11:01:02 -08:00
Jennifer Li Johnson
76b64b79ba
cmd/identity: allow using redis for RevocationDB (#3259) 2019-11-01 13:27:47 -04:00
Maximillian von Briesen
590312970d satellite/gracefulexit: add flag for enabling/disabling graceful exit on the satellite (#3437) 2019-11-01 16:21:24 +02:00
Maximillian von Briesen
d9bb25b4b9 satellite/metainfo: support a wider range of values for RS.Total in satellite metainfo validation (#3431)
change uplink RS default configuration from 130 to 95
2019-10-31 15:04:33 -04:00
Michal Niewrzal
acc7b116aa
scripts: use postgres script with all tests (#3404) 2019-10-31 07:03:54 -07:00
Yingrong Zhao
bfa6699e2c
satellite/repair: add timeout for repair download from a single node(#3418) 2019-10-30 16:31:08 -04:00
Jess G
e96d615013
satellite: remove satellite API code from peer (#3414)
* rm dup api code from sa peer, update storj-sim

* fix for backwards compat tests

* use env var instead of localhost

* changes per CR

* fix env var name

* skip peer for setup
2019-10-30 12:23:09 -07:00
Natalie Villasana
4878135068
satellite/gracefulexit, storagenode/gracefulexit: add timeouts (#3407) 2019-10-30 13:40:57 -04:00
Michal Niewrzal
da2eaa7085
scripts: dev script to start postgres before tests (#3344) 2019-10-27 12:02:10 -07:00
Yingrong Zhao
fa1ac24e19
satellite/gracefulexit: add failure threshold check (#3329)
* add overall failure percentage check and inactive time frame check before sending a response to sno

* update comment

* delete node from transfer queue if it has been inactive for too long

* fix linting error

* add test config value

* fix nil pointer

* add config value into testplanet

* add unit test for overall failure threshold

* move timeframe threshold to chore

* update protolock

* add chore test

* add per peiece failure count logic

* change config name from EndpointMaxFailures to MaxFailuresPerPiece

* address comments

* fix linting error

* add error handling for no row returned from progress table

* fix test for graceful exit chore on storagenode

* fix typo InActive -> Inactive

* improve readability for failure threshold calculation

* update config lock

* change error handling for GetProgress in graceful exit endpoint on the satellite side

* return proper rpc error in endpoint

* add check in chore test for checking finish timestamp and queue
2019-10-24 12:24:42 -04:00
Egon Elbre
3c438f31bd
satellite/satellitedb: remove sqlite support (#3296) 2019-10-19 00:27:57 +03:00
littleskunk
2a5526fcc4
satellite/repair: reduce timeout (#3302) 2019-10-18 13:43:24 +02:00
Natalie Villasana
855fca003d satellite/metrics: create a metrics chore (#3263)
* add metrics counter and chore

* updates metrics observer interval release default and dev default to 15min

* add more specific check for remote pointers

* add Counter field to metrics chore, add counter tests

* rm redundant ObjectCount suffix

* make pointer check easier to read

* change metrics.Config.Interval to ChoreInterval

* rm unneeded var

* fix comment

* update satellite config lock
2019-10-16 14:08:33 -04:00
Cameron
76ad83f12c
satellite/accounting: add redis support to live accounting (#3213)
* set up redis support in live accounting

* move live.Service interface into accounting package and rename to Cache, pass into satellite

* refactor Cache to store one int64 total, add IncrBy method to redis client implementation

* add monkit tracing to live accounting
2019-10-16 12:50:29 -04:00
Natalie Villasana
cf430d2d73
scripts: add check-monitoring script to detect changes to monkit calls (#3114) 2019-10-15 13:00:14 -04:00
Jennifer Li Johnson
b185dbbee2
satellite/discovery: remove discovery related code (#3175) 2019-10-14 10:57:01 -04:00
littleskunk
96aeedcdee
OrderLimit/GracePeriod: Increase time window from 1h to 24h (#3255)
* OrderLimit/GracePeriod: Increase time window from 1h to 24h

* update satellite config lock
2019-10-13 17:40:24 +02:00
Ethan Adams
a1275746b4
satellite/gracefulexit: Implement the 'process' endpoint on the satellite (#3223) 2019-10-11 17:18:05 -04:00
Isaac Hess
9256399872
CI: test drpc and grpc (#3163)
* wip: test drpc

* Add parallel intregration test

* Add jenkinsfile.drpc

* Remove unnecessary jenkinsfile items

* testing: GOFLAGS=-drpc (#3236)

* Use GOFLAGS

* add debug

* revert tags

* revert changes

* move goflags to the correct place

* add sanity check
2019-10-11 08:30:06 -06:00