Commit Graph

554 Commits

Author SHA1 Message Date
Jess G
1014d5a7d1
cmd/satellite: add API run command to satellite dockerfile entrypoint (#3319)
* update satellite entrypoint to add API run command

* make true a string for clarity
2019-10-21 09:16:59 -07:00
Bryan White
f468816f13
{internal/version,versioncontrol,cmd/storagenode-updater}: add rollout to storagenode updater (#3276) 2019-10-21 12:50:59 +02:00
Bryan White
243ba1cb17
{versioncontrol,internal/version,cmd/*}: refactor version control (#3253) 2019-10-20 09:56:23 +02:00
Egon Elbre
89ed997706
satellite/satellitedb: switch to postgres only (#3320) 2019-10-18 22:03:10 +03:00
Egon Elbre
44bf98ee9c
cmd/storj-sim: make postgres default to STORJ_SIM_POSTGRES (#3317) 2019-10-18 19:12:30 +03:00
Yehor Butko
26cc625dc6
satellite/console: payments api (#3297) 2019-10-17 17:42:18 +03:00
Jess G
34764e5c97
cmd/satellite: create API subcommand (#3280)
* create sa api subcommand

* add api to storj-sim

* require postgres

* pass live accounting cache into satellite.NewAPI
2019-10-16 13:34:25 -07:00
Cameron
76ad83f12c
satellite/accounting: add redis support to live accounting (#3213)
* set up redis support in live accounting

* move live.Service interface into accounting package and rename to Cache, pass into satellite

* refactor Cache to store one int64 total, add IncrBy method to redis client implementation

* add monkit tracing to live accounting
2019-10-16 12:50:29 -04:00
Yingrong Zhao
87e3764390
storagenode/cmd: add exit-status command for graceful exit (#3264)
* add exit-status command

* remove todo and fix format

* fix status display

* change startExit to exit progress

* fix linting error

* add successful column in exit progress

* fix test

* remove extra new line

* fix TYPOS

* format the percentage better
2019-10-15 18:07:32 -04:00
Jennifer Li Johnson
b185dbbee2
satellite/discovery: remove discovery related code (#3175) 2019-10-14 10:57:01 -04:00
Yingrong Zhao
743a0fc38b storagenode/cmd: create start graceful exit CLI (#3202) 2019-10-11 09:58:12 -04:00
Egon Elbre
e9c36d560f
satellite: make PointerDB an argument to satellite.New (#3233) 2019-10-10 21:06:26 +03:00
Bill Thorp
9cbb0c4374
cmd/storj-sim: expose GATEWAY_X_API_KEY (#3225)
* expose API_KEY
2019-10-09 14:33:53 -04:00
Cameron
10b364a2da
cmd/storagenode: enable migration of configs of different types (#3189)
* move deprecated flags code to deprecated.go, refactor to allow migration of other flags
* hide deprecated flags
2019-10-08 14:26:53 -04:00
JT Olio
37491d0d32 storagenode: embed the console into the binary and makefile (#3164)
* web/storagenode: add package-lock.json
* storagenode: compile console into binary
2019-10-08 10:52:19 +02:00
Jennifer Li Johnson
7ceaabb18e
Delete Bootstrap and Kademlia (#2974) 2019-10-04 16:48:41 -04:00
Michal Niewrzal
0911b7d1fb
uplink/gateway: wizard message update (#3150) 2019-10-04 07:43:26 -07:00
littleskunk
b2e328f118 storagenode/dashboard: update online status (#3168) 2019-10-03 20:31:39 +02:00
Jennifer Li Johnson
755cbd4dce
storagenode/main: map aliases for kademlia config values (#3118) 2019-09-30 19:33:00 -04:00
Bryan White
c8aa821ccb
pkg/certificates: move certificate package to root (#3107) 2019-09-26 09:11:05 -07:00
Jeff Wendling
098cbc9c67 all: use pkg/rpc instead of pkg/transport
all of the packages and tests work with both grpc and
drpc. we'll probably need to do some jenkins pipelines
to run the tests with drpc as well.

most of the changes are really due to a bit of cleanup
of the pkg/transport.Client api into an rpc.Dialer in
the spirit of a net.Dialer. now that we don't need
observers, we can pass around stateless configuration
to everything rather than stateful things that issue
observations. it also adds a DialAddressID for the
case where we don't have a pb.Node, but we do have an
address and want to assert some ID. this happened
pretty frequently, and now there's no more weird
contortions creating custom tls options, etc.

a lot of the other changes are being consistent/using
the abstractions in the rpc package to do rpc style
things like finding peer information, or checking
status codes.

Change-Id: Ief62875e21d80a21b3c56a5a37f45887679f9412
2019-09-25 15:37:06 -06:00
Egon Elbre
e0d5cbbbd7
jenkins: update golangci to v1.19.1 (#3119) 2019-09-25 19:35:56 +03:00
Michal Niewrzal
366a0be6aa
storagenode: avoid starting command on service stop (#3105) 2019-09-24 12:17:22 -07:00
Kaloyan Raev
8d16349b4c
storagenode/winsvc: avoid the 15s sleep when stopping the service (#3098) 2019-09-24 13:01:36 -06:00
Bryan White
515799267f fix certificates auth export command (#3110)
* fix certificates auth export command

* actually fix command config logic

* fix test-certificates.sh

* simplify
2019-09-24 10:38:18 -06:00
Isaac Hess
580e511b4c
storagenode/storagenodedb: Migrate to separate dbs (#3081)
* storagenode/storagenodedb: Migrate to separate dbs

* storagenode/storagenodedb: Add migration to drop versions tables

* Put drop table statements into a transaction.

* Fix CI errors.

* Fix CI errors.

* Changes requested from PR feedback.

* storagenode/storagenodedb: fix tx commit
2019-09-23 12:36:46 -07:00
Jennifer Li Johnson
d2502bb51b Adds tests for kad replacement and restores kad operator configs (#3094)
* test that all nodes can check in with all satellites

* keep kademlia config

* add untrusted satellite test

* use getversion

* remove kademlia config changes in test-sim-backwards.sh

* add kademlia flags back to storj-sim storagenode

* reset kademlia flags in storagenode entrypoint
2019-09-20 16:02:23 -04:00
Michal Niewrzal
68d281db44 autoupdater: use blang/semver (#3063) 2019-09-20 17:22:40 +03:00
Jennifer Li Johnson
724bb44723
Remove Kademlia dependencies from Satellite and Storagenode (#2966)
What:

cmd/inspector/main.go: removes kad commands
internal/testplanet/planet.go: Waits for contact chore to finish
satellite/contact/nodesservice.go: creates an empty nodes service implementation
satellite/contact/service.go: implements Local and FetchInfo methods & adds external address config value
satellite/discovery/service.go: replaces kad.FetchInfo with contact.FetchInfo in Refresh() & removes Discover()
satellite/peer.go: sets up contact service and endpoints
storagenode/console/service.go: replaces nodeID with contact.Local()
storagenode/contact/chore.go: replaces routing table with contact service
storagenode/contact/nodesservice.go: creates empty implementation for ping and request info nodes service & implements RequestInfo method
storagenode/contact/service.go: creates a service to return the local node and update its own capacity
storagenode/monitor/monitor.go: uses contact service in place of routing table
storagenode/operator.go: moves operatorconfig from kad into its own setup
storagenode/peer.go: sets up contact service, chore, pingstats and endpoints
satellite/overlay/config.go: changes NodeSelectionConfig.OnlineWindow default to 4hr to allow for accurate repair selection
Removes kademlia setups in:

cmd/storagenode/main.go
cmd/storj-sim/network.go
internal/testplane/planet.go
internal/testplanet/satellite.go
internal/testplanet/storagenode.go
satellite/peer.go
scripts/test-sim-backwards.sh
scripts/testdata/satellite-config.yaml.lock
storagenode/inspector/inspector.go
storagenode/peer.go
storagenode/storagenodedb/database.go
Why: Replacing Kademlia

Please describe the tests:
• internal/testplanet/planet_test.go:

TestBasic: assert that the storagenode can check in with the satellite without any errors
TestContact: test that all nodes get inserted into both satellites' overlay cache during testplanet setup
• satellite/contact/contact_test.go:

TestFetchInfo: Tests that the FetchInfo method returns the correct info
• storagenode/contact/contact_test.go:

TestNodeInfoUpdated: tests that the contact chore updates the node information
TestRequestInfoEndpoint: tests that the Request info endpoint returns the correct info
Please describe the performance impact: Node discovery should be at least slightly more performant since each node connects directly to each satellite and no longer needs to wait for bootstrapping. It probably won't be faster in real time on start up since each node waits a random amount of time (less than 1 hr) to initialize its first connection (jitter).
2019-09-19 15:56:34 -04:00
Kaloyan Raev
45df0c5340
storagenode/process: respond to Windows Service events (#3025) 2019-09-19 19:37:40 +03:00
Michal Niewrzal
d860353603
storagenode-updater: respond to Windows Service events (#3077) 2019-09-19 04:00:26 -07:00
Michal Niewrzal
43846f2074
cmd: rename auto-updater to storagenode-updater (#3089) 2019-09-19 02:33:56 -07:00
Michal Niewrzal
d65386f69e
auto-updater: unpack/check binary, restart service (#2968) 2019-09-17 00:19:56 -07:00
Matt Robinson
0c2ae7786e storagenode/dashboard: Enable storagenode dashboard in docker images (#3024) 2019-09-16 15:09:59 +02:00
Andrew Harding
f550ab5d1c
Uplink "import" command (#2981)
* uplink import cmd

* pkg/process: fix import order

* fix golangci-lint failures

* remove "help" from the satellite config lock file
2019-09-13 12:33:30 -06:00
littleskunk
1d8cd526e0 storj-sim: correct storagenode dashboard config (#3010) 2019-09-12 15:20:52 +03:00
paul cannon
c139ed8ea1 storagenode/console: remove kademlia (#2942)
this is a trivial operation for storagenode/console, as it doesn't
really need or use kademlia in the first place.

What:

Removes kademlia from storagenode/console

Why:

We are in the process of getting rid of kademlia, and this is one place where it's particularly easy.

Please describe the tests:

Existing tests exercise storagenode/console behavior; if they continue to work, everything here should be tested satisfactorily.
Please describe the performance impact:

None
2019-09-11 16:41:43 -04:00
Michal Niewrzal
40ff56f6c7
versioncontrol: add new version schema (#2991) 2019-09-11 07:01:36 -07:00
Bryan White
6c80f01bf0
pkg/certificates: add authorization endpoint and refactor (#2971) 2019-09-11 10:36:44 +02:00
paul cannon
7cf5650560 pkg/certificate: properly close certificateclient.Client (#2986) 2019-09-10 19:24:41 +03:00
Egon Elbre
0d4fd6ceeb
cmd/uplink: fix progressbar data race (#2987) 2019-09-10 11:51:49 +03:00
Egon Elbre
7589ca796f
cmd/storj-sim: allow overriding executables (#2976)
The backward compatibility test upgrades only half of the storage nodes and
tests with both the release and new uplink.
2019-09-09 22:13:38 +03:00
Michal Niewrzal
587be8f206
auto-updater: download versions and archive with binary (#2922) 2019-09-05 23:10:05 +02:00
Bryan White
1fc0c63a1d
{cmd,pkg}/certificates: service refactor (#2938) 2019-09-05 17:11:21 +02:00
Bryan White
b222936d19
cmd/certificates: separate subcommand configs (#2932) 2019-09-05 16:56:57 +02:00
Bryan White
62df8ddb0c cmd/certificates: auth export improvements & certificates test script (#2897) 2019-09-04 19:58:38 +02:00
Yingrong Zhao
10a896bf73
web/marketing: static asset path (#2872)
* use relative path instead of absolute path

* add template func baseURL

* add a method

* update storj-sim

* add comment
2019-08-30 18:43:53 -04:00
Michal Niewrzal
c680970e03
Storage node auto-updater skeleton (#2902) 2019-08-30 11:02:36 +02:00
littleskunk
7e37452abb open port for storage node dashboard API (#2899) 2019-08-29 11:47:43 -04:00
Stefan Benten
d0ab3c03ec cmd/*: Change loglevel from error to warn (#2876) 2019-08-27 11:24:47 +02:00