JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
3,847 Commits 5 Branches 0 Tags 94 MiB
533a65a299
Commit Graph

353 Commits

Author SHA1 Message Date
Bill Thorp
341aecfe0f satellite/console: add rate limiter to login, register, password recovery 
Added a per IP rate limiter to the console web.
Cleaned up password check to leak less bcyrpt info.

Change-Id: I3c882978bd8de3ee9428cb6434a41ab2fc405fb2
 2020-04-24 17:15:49 +00:00
Jess G
825226c98e
satellite/overlay: use node selection cache for uploads (#3859) 
* satellite/overlay: use node selection cache for uploads

Change-Id: Ibd16cccee979d0544f2f4a01749af9f36f02a6ad

* fix config lock

Change-Id: Idd307e4dee8ab92749f1ec3f996419ea0af829fd

* start fixing tests

Change-Id: I207d373a3b2a2d9312c9e72fe9bd0b01e06ad6cf

* fix test, add some more

Change-Id: I82b99c2004fca2510965f9b389f87dd4474bc722

* change config name

Change-Id: I0c0f7fc726b2565dc3828cb723f5459a940f2a0b

* add benchmarks

Change-Id: I05fa25bff8d5b65f94d918556855b95163d002e9

* revert bench to put in different PR

Change-Id: I0f6942296895594768f19614bd7b2e3b9b106ade

* add staleness to benchmark

Change-Id: Ia80a310623d5a342afa6d835402170b531b0f870

* add cache config to testplanet

Change-Id: I39abdab8cc442694da543115a9e470b2a8a25dff

* have repair select old way

Change-Id: I25a938457d7d1bcf89fd15130cb6b0ac19585252

* lower testplante config time

Change-Id: Ib56a2ed086c06bc6061388d15a10a2526a663af7

* fix test

Change-Id: I3868e9cacde2dfbf9c407afab04dc5fc2f286f69
 2020-04-24 09:11:04 -07:00
Moby von Briesen
72b93f3120 satellite/satellitedb: disqualify suspended nodes when the grace period passes 
If a node is suspended and receives an unknown or failing audit,
disqualify them if the grace period (default 1w in production) has
passed.

Migrate the nodes table so any node that is currently suspended gets
unsuspended when the satellite starts up.

Change-Id: I7b81c68026f823417faa0bf5e5cb5e67c7156b82
 2020-04-22 15:45:00 -04:00
Yingrong Zhao
0bdcf123cf bump monkit, monkit-jaeger, and private to latest 
Also bump storj.io/common and sync repo

Change-Id: If8e60db6bdf0af8077b7befcb1da304c3c4dcae4
 2020-04-22 12:30:37 -04:00
Moby von Briesen
178aa8b5e0 satellite/{metainfo,repair}: Delete expired segments from metainfo 
* Delete expired segments in expired segments service using metainfo
loop
* Add test to verify expired segments service deletes expired segments
* Ignore expired segments in checker observer
* Modify checker tests to verify that expired segments are ignored
* Ignore expired segments in segment repairer and drop from repair queue
* Add repair test to verify that a segment that expires after being
added to the repair queue is ignored and dropped from the repair queue

Change-Id: Ib2b0934db525fef58325583d2a7ca859b88ea60d
 2020-04-22 13:02:31 +00:00
Yingrong Zhao
8375a09c89 cmd: remove InitTracing from satellite and storagenode main.go file 
Change-Id: I4addbe7d0645f66abfb3e98d74d17035e9624e69
 2020-04-20 14:06:26 -04:00
Michal Niewrzal
884638a294 scripts: reduce segment size for integration tests 
This will speedup integration tests.

Change-Id: Ifb1fadd01692390457971bfe7f9ba73a0796abb2
 2020-04-20 15:13:05 +02:00
littleskunk
52cd33a633
scripts/deploy-storagenode: tag docker image as latest (#3858) 
Co-authored-by: Ivan Fraixedes <ivan@fraixed.es>
 2020-04-18 11:59:56 +02:00
VitaliiShpital
2dce4c232c web/satellite: redirect to verification page on sign up if inside iframe 
Change-Id: I606b63fd27bef46597697b491970523e8a3a0cae
 2020-04-16 13:35:49 +00:00
VitaliiShpital
158013a866 satellite/console: redirect on account activation 
Change-Id: I2506ce0fd3832bf46fbcdcc5a42bb83dc926e99a
 2020-04-15 11:49:50 +00:00
Moby von Briesen
d7794a4851 satellite/overlay: hardcode default values for audit alpha/beta 
Alpha=1 and beta=0 are the expected first values for any alpha/beta
reputation system we are using in the codebase. So we are removing the
configurability of these values.

Change-Id: Ic61861b8ea5047fa1438ea6609b1d0048bf0abc3
 2020-04-14 19:12:40 +00:00
JT Olio
e2d5b403e6 cmd/uplink: support --force (like awscli) for rb 
Change-Id: If835c6dd08ee95e7c66ba7e4c7451cb3f0f95442
 2020-04-14 18:10:54 +00:00
Cameron Ayer
3ee6c14f54 satellite/downtime: add concurrency to downtime estimation 
We want to increase our throughput for downtime estimation. This commit
adds the ability to reach out to multiple nodes concurrently for downtime
estimation. The number of concurrent routines is determined by a new config
flag, EstimationConcurrencyLimit. It also increases the default
EstimationBatchSize to 1000.

Change-Id: I800ce7ec1035885afa194c3c3f64eedd4f6f61eb
 2020-04-14 14:39:13 +00:00
Qweder93
3a9422cc9a satellite/nodestats: add pricing model to endpoint 
Change-Id: Iddace8e437216a343458f440b543cee61164f233
 2020-04-08 14:29:51 +00:00
Yingrong Zhao
96e58d21b4 cmd;pkg/server: init tracing collector in all processes 
Add tracing handler in drpc server.
Initializing tracing collector in admin, satellite api, garbage
collection, satellite core, repaier, storagenode.
Change-Id: Ie98420e35dfc6913836ebd82b517d9d12877aefc

Change-Id: I91057b6265a4ac8bde033dfde692b8a28acca99f
 2020-04-07 17:20:59 -04:00
Yingrong Zhao
4ab553cf3f scripts: only latest releases have version file 
Change-Id: Ice212ac39b011c18b1d3d48ea9e6f580b4fd7f0c
 2020-04-04 18:47:33 +00:00
Cameron Ayer
42be4bdc0f satellite/contact: add timeout to PingBack method 
Change-Id: I2ec2f82e2e10d8be16f82e9de13ce42358e47c98
 2020-04-04 18:26:30 +00:00
Yingrong Zhao
b3939bc1ba scripts/tests: fix test-version and rolling-upgrade test installation 
Change-Id: I2c1262aa2e4fd1f5aa34bffe9c76f272fe076d49
 2020-04-02 22:07:06 +00:00
Michal Niewrzal
c178a08cb8 satellite/metainfo: add max segment size and max inline size to 
BeginObject response

We want to control inline segment size and segment size on satellite
side. We need to return such information to uplink like with redundancy
scheme.

Change-Id: If04b0a45a2757a01c0cc046432c115f475e9323c
 2020-04-02 12:41:28 +00:00
Egon Elbre
90319dbec1 scripts: fix test-sim-backwards 
Change-Id: I9d6644d4b493f5f6f60a960cacfaac2b5b828a5f
 2020-04-02 00:13:42 +02:00
Egon Elbre
644df8dcdc private/version: minimal fix for tag-release.sh 
Previous split to a storj.io/private repository broke tag-release.sh
script. This is the minimal temporary fix to make things work.

This links the build information to specified variables and sets them
inline. This approach, of course, is very fragile.

Change-Id: I73db2305e6c304146e5a14b13f1d917881a7455c
 2020-04-01 13:46:45 +00:00
Michal Niewrzal
d444fbadea scripts: cleanup rolling upgrade test 
* add script for easy rolling upgrade test local execution
* remove unneeded binaries building for rolling upgrade and versions
tests
* unify build process for Jenkins and local execution for rolling
upgrade and versions tests

Change-Id: Ic11211b83f3f447494bbd5827d2af77ea4b20dfe
 2020-04-01 12:30:08 +00:00
Michal Niewrzal
0374e30678 scripts: fix storj-sim installation function for rolling upgrade tests 
Change-Id: Ifb749e0a41c57b90cc03293c2588b2ceb6a0a15a
 2020-03-31 09:11:00 +00:00
Jeff Wendling
e2ff2ce672 satellite: compensation package and commands 
Change-Id: I7fd6399837e45ff48e5f3d47a95192a01d58e125
 2020-03-30 14:08:14 -06:00
Yingrong Zhao
831668478a scripts/tests: fix gateway installation 
Change-Id: I1629644f543505c27d4edb8f7bbe97d037cdc8a8
 2020-03-29 19:24:05 +00:00
JT Olio
f28100b73f bump storj.io/private 
Change-Id: I4ddd5c34521602967b89bd18e2a71a6f1e29f436
 2020-03-27 21:57:35 +00:00
Moby von Briesen
a933bcc99a satellite/repair/repairer/ec.go: add option for downloading pieces onto disk instead of in memory during repair 
Add flag to satellite repairer, "InMemoryRepair" that allows the
satellite to decide whether to download the entire segment being
repaired into memory (this is what the satellite already does), or to
download it into temporary files on disk that will be read from in the
upload phase of repair.

This should help with handling high repair traffic on satellites that
cannot afford to spend 64mb of memory per repair worker.

Updates tests to test repair for both in memory and to disk.

Change-Id: Iddf591e165621497c98533d45bfea3c28b08a194
 2020-03-27 16:41:00 +00:00
Natalie Villasana
8e0ca0e6f5
satellite/gc: update release default for gc to run separately (#3830) 2020-03-26 14:44:18 -04:00
Michal Niewrzal
3a3648c80b scripts: add script for running versions tests locally 
Change-Id: Iaa2a2d085b7edd46f63a0d79b4e731ea72412953
 2020-03-24 15:44:00 +00:00
Michal Niewrzal
fdf40a7526 storj: remove storj/private/version package which was moved to 
`storj/private` repo

Change-Id: I81c3f5b9d5e4fe7bca760999eb045ee9734e5e2e
 2020-03-24 14:31:33 +00:00
Egon Elbre
6a7571f73e cmd/s3-benchmark: move to storj.io/benchmark 
Change-Id: Idca2b836bdf876ca28eb5cabc9bfae1d576e4a3e
 2020-03-23 19:09:42 +02:00
JT Olio
3b66ba6f02 scripts/tests: uplink no longer respects --client.segment-size 
this is going to make all the tests slower but it is what it is

test-sim-aws.sh is removed because it was moved to storj/gateway repo.

Change-Id: I10727e747a4c3740b1c9054ce7d17313b4fa310b
 2020-03-22 17:48:57 +00:00
Jennifer Johnson
699b635e5d satellite/overlay: rename newNodePercentage to newNodeFraction 
Change-Id: Ie66de91f88183b44de0773589e83e4ade9aa997a
 2020-03-19 20:09:32 +00:00
Jessica Grebenschikov
5142874144 satellite/gc: move garbage collection to its own process 
Change-Id: I7235aa83f7c641e31c62ba9d42192b2232dca4a5
 2020-03-18 16:44:01 +00:00
Egon Elbre
09e0f3de63 satellite/metainfo/piecedeletion: add Service 
Change-Id: Id7e32ed569701fa0be66f9527c43a67052994570
 2020-03-18 14:50:08 +00:00
Matt Robinson
bd4982e249
test/backwards-compatibility: Exclude rc tags from testing (#3787) 
Change-Id: Id486709306c5c75a262ea17410ae641be53df8ed

Co-authored-by: Ivan Fraixedes <ivan@fraixed.es>
Co-authored-by: littleskunk <jens.heimbuerge@googlemail.com>
 2020-03-18 00:12:05 +01:00
littleskunk
b10b69d9ce
test/rollingupgrade: fix stage 1 release version (#3810) 2020-03-17 22:30:07 +01:00
littleskunk
80acf33abc
script/release: fix error in regex (#3809) 
Co-authored-by: Stefan Benten <mail@stefan-benten.de>
 2020-03-17 17:36:17 +01:00
Stefan Benten
49a30ce4a7
satellite/payments: Set proper defaults for the release (#3806) 
* Slight adjustments to the migration

Change-Id: I68ae81c010c3414fde2845df16ab124f8d17834b

* Change Coupon Value

Change-Id: I0f241d09e5f716f1d1b3f0688643ba7f614d83c4

* Change AlphaUsage to 5GB

Change-Id: I5d25c6b5750684510cda8b14a27f38d5b2b07408

* change config lock

Change-Id: Ib7c7a54555ba2387c9aa8dd60a0501b0ee6491dd

* Use Scan properly

Change-Id: Ie39cf4644e3ddd703a254e2f5e616763dd805235

* Fix Config Lock

Change-Id: I558ecc1c1becfaaefc7aea5ad2fe83fd6bf6b561
 2020-03-16 22:53:12 +01:00
Stefan Benten
52590197c2
satellite/payments: More Cleanup and Satellite command to ensure we have stripe customers (#3805) 2020-03-16 20:34:15 +01:00
Kaloyan Raev
4f0bf3fe1d
build: cleanup more gateway targets from Makefile (#3802) 
Change-Id: Ia95caa2187b3e9e056a83cbea4230788ed4e8abd

Co-authored-by: Michal Niewrzal <michal@storj.io>
 2020-03-16 15:07:52 +01:00
Stefan Benten
bd603c0751
satellite/payments: Improve Invoice Generation (#3800) 2020-03-13 17:07:39 +01:00
Yingrong Zhao
3cf05b24d8 scripts: add benchmark test for delete operation 
Change-Id: I448cc50375c1c712d704d8cf93f5b8481372b0c8
 2020-03-12 17:03:19 +00:00
JT Olio
051569c69f
satellite: enable open registration (and add flag that disables it) SM-441 
Change-Id: I47bfedb312089f6d2bfbab013bd74ad4b8aa5f5e
 2020-03-11 03:53:34 +01:00
Yingrong Zhao
1a875baa1d scripts/tests: fix arguments orders passed to test-versions.sh 
Change-Id: I2e3b501477823413137f6a43fcdc4347af27e43c
 2020-03-11 13:50:18 +00:00
Yingrong Zhao
46b04a38cc scripts/tests/: fix uplink access to contain satellite id 
Change-Id: I7dfb6bc2da3bf84a81d75a286ee9488e32ea4f01
 2020-03-10 21:15:39 +00:00
Michal Niewrzal
c20cf25f35 cmd: migrate uplink CLI to new API 
Change-Id: I8f8fcc8dd9a68aac18fd79c4071696fb54853a60
 2020-03-09 13:26:29 +00:00
littleskunk
842c8d8ed9
scripts/tests/rollingupgrade: fix installation for current commit 2020-03-06 17:19:55 -05:00
littleskunk
8fa8178f04
release/rollingupgrade: on release tags run rolling upgrade against previous release (#3792) 
Co-authored-by: Stefan Benten <mail@stefan-benten.de>
 2020-03-03 23:56:32 +01:00
Michal Niewrzal
fb2711d05e scripts: update postgres helper script to set password 
Latest postgres docker image requires non empty password.

Change-Id: I03017e1b7ff4803fefc24c39087d9ccd4042373b
 2020-02-27 10:33:37 +00:00