Commit Graph

2722 Commits

Author SHA1 Message Date
Maximillian von Briesen
abb567f6ae
cmd/satellite: add graceful exit reports command to satellite CLI (#3300)
* update lock file and add comment

* add created at and bytes transferred

* cleanup

* rename db func to GetGracefulExitNodesByTimeFrame

* fix flag

* split into two overlay functions

* := to =

* fix test

* add node not found error class

* fix overlay test

* suggested test changes

* review suggestions

* get exit status from overlay.Get()

* check rows.Err

* fix panic when ExitFinishedAt is nil

* fix comments in cmdGracefulExit
2019-10-22 21:06:01 -04:00
Jeff Wendling
51d5d8656a pkg/rpc: drpc connection pooling
keep a pool of connections open when dialing for drpc. this
makes it so that long lived clients (like lib/uplink's Project)
don't continue to use a bad connection forever. it also allows
for concurrent rpcs.

Change-Id: If649b286050e4f09c413fadc3e1ce88f5fc6e600
2019-10-22 18:15:24 -06:00
JT Olio
2c6fa3c5f8
pkg/rpc: remove read/write deadlines as a mechanism for request timeouts (#3335)
libuplink was incorrectly setting timeouts to 10 seconds still, but
should have been at least 10 minutes. the order sender was setting them
to 1 hour. we don't want timeouts in uplink-side logic as it establishes
a minimum rate on tcp streams.

instead of all of this, just use tcp keep alive. tcp keep alive packets are
sent every 15 seconds and if the peer stops responding the connection
dies. this is enabled by default with go. this will kill tcp connections
when they stop working.

Change-Id: I3d7ad49f71950b3eb43044eedf4b17993116045b
2019-10-22 17:57:24 -06:00
Ethan Adams
3e0d12354a
storagenode/gracefulexit: Implement storage node graceful exit worker - part 1 (#3322) 2019-10-22 16:42:21 -04:00
Nikolai Siedov
3eec4e9070
satellite/console: add REST delete API endpoint (#3337) 2019-10-22 19:17:09 +03:00
Fadila
9d84e63470
installer/windows: custom welcome dialog (#3228) 2019-10-22 17:25:28 +02:00
Vitalii Shpital
4489c86539
web/storagenode: node version bug fixed (#3339) 2019-10-22 17:46:07 +03:00
Kaloyan Raev
97018d6d64
Expose range download in gomobile (#3336) 2019-10-22 14:51:01 +03:00
Vitalii Shpital
867771787f
web/satellite: project owner status added on team page (#3160) 2019-10-22 14:12:49 +03:00
Michal Niewrzal
04c2454c71
satellite/metainfo: pass streamID/segmentID between Batch request/response (#3311) 2019-10-22 03:23:22 -07:00
paul cannon
5e78f4000b storagenode/pieces: remove old comment (#3334)
the reservedSpace member it's talking about was removed quite a while
ago.

Change-Id: I28433b2a44467376a408453d875c389656347cab
2019-10-22 12:51:51 +03:00
paul cannon
78000c0191 Makefile: handle empty/default $GOPATH (#3333)
Use `$(CURDIR)` in place of `$(shell pwd)`. This saves a
fork+exec, is more readable, and is more idiomatic. The only case where
those should have different values is when someone's configured `$SHELL`
is badly broken, and I'd argue in that case `$(CURDIR)` is preferable.

Change-Id: I6a3abac1e2e197f5332825ffad2bbd3da1fc4700
2019-10-22 12:44:13 +03:00
Bryan White
94cec7e3b0 add config dir arg to windows installer (#3328) 2019-10-21 11:22:54 -07:00
Nikolai Siedov
6dd478e43c
satellite/console: forgot password, resend email endpoints added, default http route replaced with gorilla mux (#3327) 2019-10-21 19:42:49 +03:00
Jess G
1014d5a7d1
cmd/satellite: add API run command to satellite dockerfile entrypoint (#3319)
* update satellite entrypoint to add API run command

* make true a string for clarity
2019-10-21 09:16:59 -07:00
Kaloyan Raev
244fea774b
gomobile Reader.Read should take offset and length params (#3326) 2019-10-21 18:17:31 +03:00
Nikolai Siedov
1814fbfa89
satellite/console: new passwordChange API endpoint (#3308) 2019-10-21 15:48:29 +03:00
Bryan White
f468816f13
{internal/version,versioncontrol,cmd/storagenode-updater}: add rollout to storagenode updater (#3276) 2019-10-21 12:50:59 +02:00
Ivan Fraixedes
91872fcbd3 README: Replace reference Rocket chat to forum (#3324) 2019-10-21 11:48:32 +03:00
Bryan White
243ba1cb17
{versioncontrol,internal/version,cmd/*}: refactor version control (#3253) 2019-10-20 09:56:23 +02:00
Charles Clinton Pustejovsky III
f65801309c Updating Support Link in README (#3314)
Replace Rocket chat by our community forum link
2019-10-19 18:15:13 +02:00
Egon Elbre
3c438f31bd
satellite/satellitedb: remove sqlite support (#3296) 2019-10-19 00:27:57 +03:00
Egon Elbre
89ed997706
satellite/satellitedb: switch to postgres only (#3320) 2019-10-18 22:03:10 +03:00
Matt Robinson
c0ffd59887 Go 1.13.3 (#3321) 2019-10-18 20:14:40 +02:00
Egon Elbre
44bf98ee9c
cmd/storj-sim: make postgres default to STORJ_SIM_POSTGRES (#3317) 2019-10-18 19:12:30 +03:00
littleskunk
2a5526fcc4
satellite/repair: reduce timeout (#3302) 2019-10-18 13:43:24 +02:00
Stefan Benten
02b9303420 Adding cpustejovsky to the CLA (#3316) 2019-10-18 13:26:23 +02:00
Egon Elbre
f929310add pkg/rpc/rpcstatus: fix drpc grpc compatibilty (#3306)
When code is compiled without -tags=drpc the statuses for drpc server
weren't handled, which meant an uplink using -tags=drpc didn't get the
correct status code.
2019-10-17 15:21:20 -04:00
Ivan Fraixedes
071d1c4313
upload: Add more info to returned error response & to logs (#3218)
* uplink/storage/segments: return error no optimal threshold
  Return an error if the store get less uploaded pieces than the indicated
  by the optimal threshold.

* satellite/metainfo: Fix gRPC status error & add reason
  This commit fix the CommitSegment endpoint method to return an
  "Invalid Argument" status code when uplink submits invalid data which is
  detected when filtering invalid pieces by filterInvalidPieces endpoint
  method.

  Because filterInvalidPieces is also used by CommitSegmentOld, such
  method part has been changed accordingly.

  * An initial check in CommitSegment to detect earlier if uplink sends an
    invalid number of upload pieces.
  * Add more information to some log messages.
  * Return more information to uplink when it sends a number of invalid
    pieces which make impossible to finish the operation successfully.

* satellite/metainfo: Swap some "sugar" loggers to normal ones
  Swap "sugar" loggers to normal ones because they impact the performance
  in production systems and they should only be used under specific
  circumstances which were none of the ones changed.
2019-10-17 20:01:40 +02:00
Nikolai Siedov
f4162bd33f
satellite/console: add REST registration API endpoint (#3303) 2019-10-17 19:34:27 +03:00
Michal Niewrzal
ee55d14f35
mobile: create release script (#3197) 2019-10-17 08:39:34 -07:00
Natalie Villasana
45c35d7c3f
satellite/satellitedb: add exit_status column to nodes table (#3301) 2019-10-17 11:01:39 -04:00
Yehor Butko
26cc625dc6
satellite/console: payments api (#3297) 2019-10-17 17:42:18 +03:00
Natalie Villasana
22d0f89941
satellite/gracefulexit: use zap.Stringer instead of zap.String (#3299) 2019-10-17 10:29:35 -04:00
Yaroslav Vorobiov
24e72f35d3
satellite/payments: token deposit (#3283) 2019-10-17 17:04:50 +03:00
Nikolay Yurchenko
afdd340586
name removed from header (#3209) 2019-10-17 15:25:18 +03:00
Vitalii Shpital
774758c656
web/satellite: bucket name bug fixed (#3240) 2019-10-17 13:57:20 +03:00
Vitalii Shpital
41dc5ef7ff
web/satellite: styling bugs fixed (#3277) 2019-10-17 13:34:45 +03:00
Vitalii Shpital
25dbf46bb2
web/satellite: validation added on delete account popup (#3290) 2019-10-17 13:19:21 +03:00
Egon Elbre
edf8318c0e
jenkins: update to golangci-lint 1.21.0 (#3289)
* jenkins: update to golint-1.21.0
* don't use underscores in var
* make handles private to satisfy linter

lib\uplinkc\universe.go:32:39: exported method Add returns unexported type storj.io/storj/lib/uplinkc._Ctype_long, which can be annoying to use (golint)
   func (m *Universe) Add(x interface{}) Handle {

* disable wsl explicitly
2019-10-17 10:42:22 +03:00
JT Olio
863e4cacbe
examples/scope: small scope exploder command line tool (#3266)
this provides a small tool that breaks a scope into
its constituent parts

Change-Id: If011dd457a175eeb6e013a17d1aee4f3edfb8b0c
2019-10-16 22:40:41 -06:00
Jess G
34764e5c97
cmd/satellite: create API subcommand (#3280)
* create sa api subcommand

* add api to storj-sim

* require postgres

* pass live accounting cache into satellite.NewAPI
2019-10-16 13:34:25 -07:00
Natalie Villasana
855fca003d satellite/metrics: create a metrics chore (#3263)
* add metrics counter and chore

* updates metrics observer interval release default and dev default to 15min

* add more specific check for remote pointers

* add Counter field to metrics chore, add counter tests

* rm redundant ObjectCount suffix

* make pointer check easier to read

* change metrics.Config.Interval to ChoreInterval

* rm unneeded var

* fix comment

* update satellite config lock
2019-10-16 14:08:33 -04:00
Yingrong Zhao
e5099f31f3
add context.Clean and correct rpc error code (#3295) 2019-10-16 13:50:01 -04:00
Cameron
76ad83f12c
satellite/accounting: add redis support to live accounting (#3213)
* set up redis support in live accounting

* move live.Service interface into accounting package and rename to Cache, pass into satellite

* refactor Cache to store one int64 total, add IncrBy method to redis client implementation

* add monkit tracing to live accounting
2019-10-16 12:50:29 -04:00
Ivan Fraixedes
21c6737b10
uplink/ecclient: clarify defer logic in putPiece (#3247)
Refactoring the 'defer' function logic to just only have what's important to not forget before returning but simplifying its logic for making easy to understand the overall function logic.
2019-10-16 18:05:22 +02:00
littleskunk
6e7607239c
satellite/repair: improve logging (#3287)
* satellite/repair: improve logging

* use Stringer wherever possible
2019-10-16 17:28:56 +02:00
Nikolai Siedov
c1033fc1ed
satellite/console: move token to separate endpoint (#3292) 2019-10-16 18:01:15 +03:00
JT Olio
bbe89ec9d1
add siedov mykola to clabot (#3293)
Change-Id: I10a498233d181706a8b3149b18c015d20a101b8d
2019-10-16 08:12:07 -06:00
littleskunk
2301a8287f Satellite/PieceHashValidation: Increase time window from 2h to 24h to avoid timezone issues (#3291) 2019-10-16 06:47:08 -06:00