Commit Graph

650 Commits

Author SHA1 Message Date
Yaroslav Vorobiov
35edc2bcc3 satellite/payments: invoice creation (#3468) 2019-11-05 15:16:02 +02:00
Egon Elbre
9c59efd33d
satellite/rewards: ensure that partner information is asked from a service (#3275) 2019-11-05 14:58:09 +02:00
Nikolai Siedov
1a0757ebf7
web/satellite: auth error messages fixed (#3426) 2019-11-05 13:55:26 +02:00
littleskunk
def3dcbaa9
satellite/audit: increase timeout to 5 minutes (#3480)
* satellite/audit: increase timeout to 5 minutes

* fix lint error
2019-11-05 11:21:25 +01:00
Michal Niewrzal
e0c2dfcb8d satellite/metainfo: don't allow uplink to commit same piece multiple times (#3460) 2019-11-04 17:26:19 -05:00
Jess G
5abb91afcf
satellite: change the Peer name to Core (#3472)
* change satellite.Peer name to Core

* change to Core in testplanet

* missed a few places

* keep shared stuff in peer.go to stay consistent with storj/docs
2019-11-04 11:01:02 -08:00
Yehor Butko
0c2e498f09
satellite/satellitedb: console tables archview comments updated (#3465) 2019-11-04 16:37:39 +02:00
Yehor Butko
761cec5ea3
satellite/payments: archview comments updated (#3464) 2019-11-04 14:30:07 +02:00
Yehor Butko
5cb46d2ce3
satellite/payments: mock payment service created, api calls from frontend returned (#3448) 2019-11-04 12:54:25 +02:00
Jess G
8d92c288e2
satellitedb: separate migration into subcommand (#3436)
* separate sadb migration, add version check

* update checkversion to do same validation as migration

* changes per CR

* add sa migration to storj-sim

* add different debug port in storj-sim for migration

* add wait for exit for storj-sim migration

* update sa docker entrypoint to support migration

* storj-sim satellite parts all wait for migration

* upgrade golang-migrate/migrate to v4 because bug

* fix go mod tidy
2019-11-02 13:09:07 -07:00
Maximillian von Briesen
f9df0ea591
satellite/gracefulexit: check for unknown error in graceful exit disable test
Allow error in graceful exit disable test to be rpcstatus.Unimplemented (grpc) or rpcstatus.Unknown (drpc)
2019-11-01 17:21:30 -04:00
Egon Elbre
aa761700af satellite/satellitedb: update nodes in sorted order (#3446) 2019-11-01 18:07:23 +01:00
Maximillian von Briesen
590312970d satellite/gracefulexit: add flag for enabling/disabling graceful exit on the satellite (#3437) 2019-11-01 16:21:24 +02:00
Maximillian von Briesen
d9bb25b4b9 satellite/metainfo: support a wider range of values for RS.Total in satellite metainfo validation (#3431)
change uplink RS default configuration from 130 to 95
2019-10-31 15:04:33 -04:00
Nikolai Siedov
8ce09700af web/satellite: internal server error template added, errors separated (#3430) 2019-10-31 20:42:28 +02:00
Yehor Butko
ecde507e4e
satellite/payments: list invoices (#3389) 2019-10-31 18:56:54 +02:00
Ethan Adams
43103ae13f
lower storage node counts in tests (#3427) 2019-10-31 10:57:54 -04:00
Yingrong Zhao
bfa6699e2c
satellite/repair: add timeout for repair download from a single node(#3418) 2019-10-30 16:31:08 -04:00
Jess G
e96d615013
satellite: remove satellite API code from peer (#3414)
* rm dup api code from sa peer, update storj-sim

* fix for backwards compat tests

* use env var instead of localhost

* changes per CR

* fix env var name

* skip peer for setup
2019-10-30 12:23:09 -07:00
Jess G
4d85b11574
satellite/contact: improve errors in contact endpoints (#3356)
* improve errors in satellite contact endpoints

* add changes per CR comments

* update pingback method so it still updates node table

* fix err and returns

* fix zap logging to be better
2019-10-30 11:57:21 -07:00
Natalie Villasana
4878135068
satellite/gracefulexit, storagenode/gracefulexit: add timeouts (#3407) 2019-10-30 13:40:57 -04:00
Natalie Villasana
5453886231 satellite/repair, uplink/ecclient: remove unused expiration arg from ec.Repair and ec.putPiece (#3416) 2019-10-30 11:35:00 -04:00
Egon Elbre
65a8e0bcbc
{satellite,storagenode}/gracefulexit: clearer log messages (#3413) 2019-10-30 10:21:27 +02:00
Maximillian von Briesen
54594e79c3 satellite/gracefulexit: add metrics on satellite for graceful exit (#3355) 2019-10-29 16:22:20 -04:00
Maximillian von Briesen
cd0940724c satellite/gracefulexit: use sync2 cycle inside satellite graceful exit endpoint (#3394) 2019-10-29 14:40:42 -04:00
Maximillian von Briesen
cd3d3850f9 satellite/gracefulexit: only allow one connection per node to graceful exit endpoint (#3357) 2019-10-29 13:23:17 -04:00
Yaroslav Vorobiov
30a3205745
satellite/payments: update account balance (#3379) 2019-10-29 18:04:34 +02:00
Cameron
b2ff13f1fa
{cmd/satellite, storj/satellite}: create command to run repair process in isolation (#3341)
* set up satellite repair run command

* add separated repair process to storj-sim

* add repairer peer to satellite in testplanet

* move api run cmd into api.go

* add satellite run repair to entrypoint
2019-10-29 10:55:57 -04:00
Nikolai Siedov
6354b38849
web/satellite: auth graphql api replaced with REST API (#3396) 2019-10-29 16:24:16 +02:00
Egon Elbre
2e44a9fa6b
satellite/satellitedb/dbx: remove sqlite.sql file (#3409) 2019-10-29 12:35:27 +02:00
Yingrong Zhao
ec690929df
satellite/metainfo: fix index out of range error for validate pointer (#3398)
fix index out of range error in metainfo when validate a pointer
2019-10-28 17:09:57 -04:00
Ethan Adams
7c2daa4dd9
Use original pointer when calling UpdatePieces (#3397) 2019-10-28 14:43:46 -04:00
Ethan Adams
9905f2c61e add piece num to transfer queue PK (#3390) 2019-10-28 11:08:33 -04:00
Yingrong Zhao
292e64ee2f
satellite/gracefulexit: check duplicate node id before update pointer (#3380)
* check duplicate node id before update pointer

* add test for transfer failure when pointer already contain the receiving node id

* check exiting and receiving nod are still in the pointer

* check node id only exists once in a pointer

* return error if the existing node doesn't match with the piece info in the pointer

* try to recreate the issue on jenkins

* should not remove exiting node piece in test

* Update satellite/gracefulexit/endpoint.go

Co-Authored-By: Maximillian von Briesen <mobyvb@gmail.com>

* Update satellite/gracefulexit/endpoint.go

Co-Authored-By: Maximillian von Briesen <mobyvb@gmail.com>
2019-10-27 14:20:22 -04:00
Maximillian von Briesen
a4e618fd1f handle context cancelled in satellite graceful exit endpoint (#3388) 2019-10-27 10:14:25 -04:00
Cameron
279cd56253 change uuid conversion (#3384) 2019-10-26 17:51:39 -04:00
Ethan Adams
e54d290d2e satellite/gracefulexit: Add signatures for success/failed exit finished messages. (#3368)
* add signatures, fix process loop bug, move delete to on success

* added tests for signatures

* PR comment updates

* fixed setting reason by default.

* updates for PR comments

* added signed failure when verificationi fails

* moved to sign_test

* fix panic

* removed testplanet from test
2019-10-25 16:36:26 -04:00
Maximillian von Briesen
6df4d7bc73
storagenode/gracefulexit + satellite/gracefulexit: add storagenode-side transfer validation (#3371)
* Make the exiting node check piece hashes, piece IDs, and piece hash signatures before relaying successful transfer data to the satellite.
* Enable immediate graceful exit failure for "successful" transfers that fail satellite-side validation.
* Move transfer piece logic in storagenode worker to separate function (to make the worker easier to understand)
2019-10-25 13:16:20 -04:00
Ethan Adams
f0caa6ce5e
satellite:gracefulexit: Update pointer after success (#3369) 2019-10-25 11:14:22 -04:00
Nikolai Siedov
f4f142bf3e
satellite/console: get, update auth api endpoints added (#3375) 2019-10-25 15:07:17 +03:00
Michal Niewrzal
7f6893ea86
satellite/metainfo: fix broken object listing (#3348) 2019-10-24 14:05:08 -07:00
Natalie Villasana
696c567e89
satellite/gracefulexit: add piece hash validation for successful transfer (#3313) 2019-10-24 15:38:40 -04:00
Yingrong Zhao
fa1ac24e19
satellite/gracefulexit: add failure threshold check (#3329)
* add overall failure percentage check and inactive time frame check before sending a response to sno

* update comment

* delete node from transfer queue if it has been inactive for too long

* fix linting error

* add test config value

* fix nil pointer

* add config value into testplanet

* add unit test for overall failure threshold

* move timeframe threshold to chore

* update protolock

* add chore test

* add per peiece failure count logic

* change config name from EndpointMaxFailures to MaxFailuresPerPiece

* address comments

* fix linting error

* add error handling for no row returned from progress table

* fix test for graceful exit chore on storagenode

* fix typo InActive -> Inactive

* improve readability for failure threshold calculation

* update config lock

* change error handling for GetProgress in graceful exit endpoint on the satellite side

* return proper rpc error in endpoint

* add check in chore test for checking finish timestamp and queue
2019-10-24 12:24:42 -04:00
Yehor Butko
e82245e10e
satellite/payments: credit card selection (#3304) 2019-10-23 20:33:24 +03:00
Nikolai Siedov
655e2b3422
sattelite/console: sattellite name API endpoint added (#3349) 2019-10-23 18:36:58 +03:00
Yaroslav Vorobiov
1a304f5ef9
satellite/payments: add payments loop, update pending transactions (#3318) 2019-10-23 15:04:54 +03:00
Michal Niewrzal
521c39bda0
uplink/metainfo: cleanup method names (#3315) 2019-10-22 23:59:56 -07:00
Maximillian von Briesen
abb567f6ae
cmd/satellite: add graceful exit reports command to satellite CLI (#3300)
* update lock file and add comment

* add created at and bytes transferred

* cleanup

* rename db func to GetGracefulExitNodesByTimeFrame

* fix flag

* split into two overlay functions

* := to =

* fix test

* add node not found error class

* fix overlay test

* suggested test changes

* review suggestions

* get exit status from overlay.Get()

* check rows.Err

* fix panic when ExitFinishedAt is nil

* fix comments in cmdGracefulExit
2019-10-22 21:06:01 -04:00
JT Olio
2c6fa3c5f8
pkg/rpc: remove read/write deadlines as a mechanism for request timeouts (#3335)
libuplink was incorrectly setting timeouts to 10 seconds still, but
should have been at least 10 minutes. the order sender was setting them
to 1 hour. we don't want timeouts in uplink-side logic as it establishes
a minimum rate on tcp streams.

instead of all of this, just use tcp keep alive. tcp keep alive packets are
sent every 15 seconds and if the peer stops responding the connection
dies. this is enabled by default with go. this will kill tcp connections
when they stop working.

Change-Id: I3d7ad49f71950b3eb43044eedf4b17993116045b
2019-10-22 17:57:24 -06:00
Ethan Adams
3e0d12354a
storagenode/gracefulexit: Implement storage node graceful exit worker - part 1 (#3322) 2019-10-22 16:42:21 -04:00