Commit Graph

693 Commits

Author SHA1 Message Date
Bryan White
cb827cbe31 cmd/{uplink,gateway}: update cli copy "http" -> "https"
Change-Id: I47f46221ab3eb6065d382acfe75865abf09b1ce5
(cherry picked from commit 580a00561ae35b4830acc1f4dd990329df4d3c7b)
2020-01-21 11:35:15 +01:00
Yaroslav
c636b06191 satellite/console: use cookie based auth scheme
Change-Id: I143b56f49fa9028ec172db8c29fd93577c3e7878
2020-01-20 21:35:23 +02:00
Bryan White
3b55b50eac cmd/uplink: Add ability to generate named accesses
Change-Id: I2cf58c1c41bfffa800949dc441a488c16a448375
2020-01-20 10:02:13 +01:00
Moby von Briesen
273eb66fae cmd/storagenode,storagenode/preflight: add config flag to disable
storagenode database preflight check.

Disable preflight database check by default, and have the option to
enable it. This will allow us to enable it once it is definitely
working.

Also change the name of the config flag for preflight  time sync.

Change-Id: Ie2e20f9e25dcb38794eafa7e1505e7c6ff287c99
2020-01-17 17:53:17 +00:00
Michal Niewrzal
22af78b62e cmd/uplink: fix 'must specify access' for old uplink configurations
Change-Id: I73b0483b1907a854048cdb1080a013844fb550f4
2020-01-17 12:02:31 +00:00
Moby von Briesen
e115bc1903 cmd/storagenode;storagenode/storagenodedb: add preflight database check
for storagenode

Ensure that database schema matches latest test migration schema before
allowing the node to start up.

Ensure minimal read/write functionality for each storagenode database
before allowing the node to start up.

This will eliminate many unhandled audit errors we are seeing.

Change-Id: Ic0e628b04a9c35b7a8243f6a81d4683918170ba9
2020-01-16 18:44:46 +00:00
Jeff Wendling
78c6d5bb32 satellite/satellitedb: reported_serials table for processing orders
this commit introduces the reported_serials table. its purpose is
to allow for blind writes into it as nodes report in so that we have
minimal contention. in order to continue to accurately account for
used bandwidth, though, we cannot immediately add the settled amount.
if we did, we would have to give up on blind writes.

the table's primary key is structured precisely so that we can quickly
find expired orders and so that we maximally benefit from rocksdb
path prefix compression. we do this by rounding the expires at time
forward to the next day, effectively giving us storagenode petnames
for free. and since there's no secondary index or foreign key
constraints, this design should use significantly less space than
the current used_serials table while also reducing contention.

after inserting the orders into the table, we have a chore that
periodically consumes all of the expired orders in it and inserts
them into the existing rollups tables. this is as if we changed
the nodes to report as the order expired rather than as soon as
possible, so the belief in correctness of the refactor is higher.

since we are able to process large batches of orders (typically
a day's worth), we can use the code to maximally batch inserts into
the rollup tables to make inserts as friendly as possible to
cockroach.

Change-Id: I25d609ca2679b8331979184f16c6d46d4f74c1a6
2020-01-15 19:21:21 -07:00
Ivan Fraixedes
f9b2af934b cmd/gateway: Fix entrypoint setup command
Fix the setup command of the entrypoint script used by the gateway
docker container.

Issue: https://github.com/storj/storj/issues/3748#issuecomment-574257152
Change-Id: I94c4d7aa148761645dd065f86b7d8e2d72bb727c
2020-01-15 11:29:27 +00:00
crawter
41d5e86306 satellite/payments: coupon addition removed
Change-Id: I92781d9133603fdefd58b19a6f0ac6b1c6df3ac6
2020-01-14 16:24:48 +00:00
Michal Niewrzal
6c4e3b64df cmd/uplink: 'uplink access inspect' for displaying access fields
https://storjlabs.atlassian.net/browse/V3-3492

Change-Id: Id75aad869f8d06444f970f1da50e784179993894
2020-01-14 15:29:38 +00:00
Michal Niewrzal
c8ccd26e04 cmd/uplink: import imports 'access' into existing configuration
https://storjlabs.atlassian.net/browse/V3-3491

Change-Id: I9c5f649ded314bb3a2235588c746913a3ec2d203
2020-01-14 13:18:48 +00:00
crawter
a57ce18f58 satellite/payments: coupons, coupons usage, invoice generation with pricing model applied
Change-Id: Ic5d5a2fc116388647efe46896cfccc2038c77537
2020-01-14 12:45:00 +00:00
Kaloyan Raev
172918596b cmd/uplink: output cleanup of uplink share command
JIRA: https://storjlabs.atlassian.net/browse/V3-3499

The `uplink share` command does not print the restricted API key and the
restricted encryption access anymore.

Change-Id: Ie4ebe0b27067ee00af97c775f4e06f558b894fe2
2020-01-13 16:53:36 +00:00
Egon Elbre
ff267168c5 private/migrate: add ctx argument
Change-Id: I3d65912d89261386413c494c7ed1576fed4dcaf4
2020-01-13 15:52:26 +02:00
Egon Elbre
24958bd7d3 satellite: add ctx to DB.CreateTables
Change-Id: I9ecad624cf5a7fc9c86bb91c68f96a3a4efd2e92
2020-01-13 15:31:09 +02:00
Michal Niewrzal
36db00b2bf cmd/uplink: don't require setup or import if --access is set
We want to make using uplink as easy as possible. That's why we wan't to
avoid requiring setup or import command before normal usage if user
specified --access flag. If this flag is set then rest flags should be
set as defaults.

https://storjlabs.atlassian.net/browse/V3-3490

Change-Id: I95a7bd77a3f00b8d9981fee513e9e77aef298bca
2020-01-11 07:47:53 +00:00
Michal Niewrzal
b579c260ab cmd: rename "scope" flag to "access"
We decided that better name for "scope" will be "access". This change
refactors cmd part of code but don't touch libuplink. For backward
compatibility old configs with "scope" field will be loaded without any
issue. Old flag "scope" won't be supported directly from command line.

https://storjlabs.atlassian.net/browse/V3-3488

Change-Id: I349d6971c798380d147937c91e887edb5e9ae4aa
2020-01-10 15:27:53 +00:00
Natalie Ventura Villasana
131c4d94f0 {cmd/uplink, lib/uplink}: change RS total to 110
Bump the uplink's RS Total value from 95 to 110.

Change-Id: I208beed17b5f53fe359847e0469a29fec80a4f7e
2020-01-10 14:27:32 +00:00
Jeff Wendling
77fd41a02e satellite: add an expiring lru cache around api keys
Change-Id: I995429c66affd33da59b091f28f09ca122070b5e
2020-01-09 22:13:41 -07:00
Egon Elbre
082ec81714
uplink: move to storj.io/uplink () 2020-01-08 15:40:19 +02:00
Egon Elbre
00c0c51b1c cmd/uplink: fix TestSetGetMeta flakiness
testrand.Path was also returning folders which has different behavior
for cp.

Change-Id: Ia53a2709bf3e768b3b7063a6137ec474c2622cb2
2020-01-08 12:25:25 +00:00
Egon Elbre
f41d440944 all: reduce number of log messages
Remove starting up messages from peers. We expect all of them to start,
if they don't, then they should return an error why they don't start.
The only informative message is when a service is disabled.

When doing initial database setup then each migration step isn't
informative, hence print only a single line with the final version.

Also use shorter log scopes.

Change-Id: Ic8b61411df2eeae2a36d600a0c2fbc97a84a5b93
2020-01-06 19:03:46 +00:00
Simon Guindon
e1e7cebe49 satellite/metainfo: added rate limiting support to the metainfo loop.
As per discussed we decided to rate limit how fast we iterate through
the metainfo database in the metainfo loop. This puts in place a
mechanism for rate limiting and burst limiting if need be in the future.

The default for this rate limiting is still no limits so it stays the
same as our previous functionality.

Change-Id: I950f7192962b0e49f082d2c4284e2d52b0a925c7
2020-01-03 15:00:29 -05:00
Egon Elbre
e03d3fb577 uplink: move configs to cmd/uplink/cmd
Change-Id: Ifc1d3440dcef429c2a6142c16f3e991abf49f1d2
2020-01-02 09:40:57 +00:00
Egon Elbre
6615ecc9b6 common: separate repository
Change-Id: Ibb89c42060450e3839481a7e495bbe3ad940610a
2019-12-27 14:11:15 +02:00
Kaloyan Raev
7df3c9efc3 cmd/uplink: use arguments in share command as allowed path prefixes
Fixes Least Authority Issue F:
https://storjlabs.atlassian.net/browse/V3-3409

If the --allowed-path-prefix flag is not set to the `share` command, any
command arguments will be used as allowed path prefixes.

This patch also improves the output of the `share` command to print the
state of all restrictions, so users can confirm they match their
intention.

Change-Id: Id1b4df20b182d3fe04cb2196feea090975fce8b4
2019-12-27 10:02:33 +00:00
Egon Elbre
d55288cf68 pkg/rpc: replace methods with direct calls to pb
Change-Id: I8bd015d8d316a2c12c1daceca1d9fd257f6f57bc
2019-12-22 17:12:43 +02:00
Egon Elbre
006baa9ca6 pkg/rpc: remove drpc aliases
We need to split up pb package, which means we cannot have a core package
that depends on them.

Change-Id: I7f4f6fd82f89a51a9b2ad08bf2b1207253b8a215
2019-12-22 16:58:08 +02:00
ccase
6f1eaef8d4 cmd/uplink: Pass -- in tests to avoid treating generated arg strings as flags.
Change-Id: I41c50b9f645b57ddc8832b0fc92f1c6bfaf2de8d
2019-12-21 15:15:15 +00:00
Egon Elbre
98243360d6 cmd/storagenode-updater: faster update test
flate compression with default settings plays very poorly
together with race, causing test to take a significant amount
of time.

Use pass-through compression to avoid the issue.

Improves test from 2m45s to 17s.

Change-Id: Iadf1381c538736d48e018164697bdfd3356e24b8
2019-12-21 02:59:31 +02:00
Andrew Harding
62c58f4a9a satellite: consistent report range arguments
This change updates the three satellite report commands that accept date
ranges to parse and treat those dates uniformly.

- End dates are now uniformly exclusive. Exclusive end dates helps
operators avoid one-off errors on month boundaries, as in the operator
does not have to remember how many days are in that month and can just
run the report from the 1st (inclusive) through the 1st (exclusive).
- Fixed the date range validity check which only failed if the start
date came after the end date (it should have failed dates that were
equal since the check happened after adjusting for inclusivity).

Change-Id: Ib2ee1c71ddb916c6e1906834d5ff0dc47d1a5801
2019-12-20 17:17:09 +00:00
JT Olio
389d1821ea uplink/paths/encryption: support commandline argument to override path cipher to be urlsafe base64 for lists and deletes () 2019-12-19 12:29:00 +01:00
Egon Elbre
2daf24a1ea private/testcontext: remove version dependency
Change-Id: Ibabf5ec774dcdb1e4fc2f200368281c69b62e6c2
2019-12-18 15:24:44 +00:00
Vitalii Shpital
53d9bc4530
storagenode/notifications: db created () 2019-12-16 19:59:01 +02:00
Andrew Harding
cb89496569 storagenode/trust: wire up list into pool
- also updated ping chore to pick up trust changes
- fixed small typo in blueprint
- fixed flags for storj-sim
- wired up changes to testplanet

Change-Id: I02982f3a63a1b4150b82a009ee126b25ed51917d
2019-12-13 20:32:50 +00:00
Ivan Fraixedes
9ac2c4d815 cmd/segment-reaper: Add test cases from/to processSegment
Add randomized test cases for testing the observer processSegment method
when the observer has a time range (from and to) set and there are
objects with segments whose creation date is outside of this range.

Change-Id: Ieac82a21f278f0850b95275bfdd2e8a812cc57a5
2019-12-13 17:23:32 +00:00
Michal Niewrzal
e1d99522b9 cmd/segment-reaper: test processSegment for single project
This is a test for `observer.processSegment` method from `segment-reaper
detect` command.

Change-Id: I14b7245848766b7aa01e5edffcf94c61a06a4c0a
2019-12-12 10:26:05 +00:00
Jeff Wendling
6ce22be744 cmd/storj-sim: make initial provisioning nicer
the old code to eventually create an api key on a satellite had
some issues. namely, there were some ignored errors, swallowed
errors, incorrect returns of nil errors when there should have
been an error, and it did not handle working against an already
existing database.

this commit fixes the above issues and organizes the code into
a set of methods performing individual steps rather than one big
function. it adds retries and attempts to get existing values
instead of creating them when possible, which means that it will
work if the values already exist. additionally, it removes the
3 second sleep in favor of a bounded retry loop with a small sleep
which improves startup times.

Change-Id: I4de04659e5a62dd3f675fbf3c76f3311c410a03e
2019-12-11 19:41:57 +00:00
Michal Niewrzal
b294569840 segment-reaper: fix for not analyzing last project in detect command
This change fixes issue where last project in DB was not analyzed to
find zombie segments

Change-Id: I9efd8a39a65f72d85c7aae325a0e4e3cc1cb9afb
2019-12-11 13:18:13 +00:00
Kaloyan Raev
958772467a cmd/storagenode-updater, pkg/process: Fix logging timestamp
After changing how we execute the storagenode-updater process we lost
timestamps in the log.

The fix is to start using zap logging.

The Windows Installer is changed to register the storagenode-updater
service in a way that the Windows Service Manager passes the
--log.output flag instead of the old --log.

The old --log flag is deprecated, but not removed. We will support it
for backward compatibility. This is required as the storagenode-updater
can auto-updated itself, but the Windows Service Manager of this old
installtion will continue passing the old --log flag when starting it.

Change-Id: I690dff27e01335e617aa314032ecbadc4ea8cbd5
Signed-off-by: Kaloyan Raev <kaloyan@storj.io>
2019-12-11 10:05:48 +00:00
Michal Niewrzal
011bb5d551 segment-reaper: test from/to detect parameters
Unit test for testing from/to parameters of `segment-reaper detect`
command.

Change-Id: I85c8839b676c61e7f046cb5ccc92d988ca51d6e3
2019-12-10 12:18:44 +01:00
Ivan Fraixedes
e961b1840c
cmd/segment-reaper: Implement unit test for observer ()
Implement some unit test cases for the observer.processSegment method and fix a bug found by these tests.

A production snapshot is more certain but it's huge for having in the repository and run the test with it by the CI.
We want to have tests for the different cases to detect zombie segments and relaying on production data cannot guarantee to have all of them.

NOTE the test has been implemented with random values for not having always the same combination of segments list and the same values avoiding that the implementation gets stale due to the test. The issue about this is that it's harder to understand and we could get only sometimes failures in the CI in case that the implementation has some bug.

The random tests allow to eventually check cases that a static test may now cover because it is not expressed or because it is needed to implement a large number of cases.

Because we are worried that the test implementation is complex and we could have bugs on it despite that the same bugs should exist in both, the implementation and the test, moreover that we have to consider that the implementation and the tests have been written by different people. Because of that, we may replace entirely these random tests by a list of static ones.
2019-12-10 11:01:58 +01:00
Jeff Wendling
48da8baab5 storj-sim: work with cockroach:// urls for satellite databases
for storj-sim to work, we need to avoid schemas in cockroach urls
so we have storj-sim create namespaced databases instead of schemas
and we have the migrate command create the database in the same way
that it would create a schema for postgres. then it works!

a follow up commit will move the creation of the database/schemas
into storj-sim's setup step so that we can avoid doing these icky
creations during normal migration calls. it will also make the
pointerdb have an explicit call to migrate instead of just doing
it every time it's opened.

Change-Id: If69ef5cb96b6866b0438c761bd445afb3597ae5f
2019-12-09 23:44:00 +00:00
Michal Niewrzal
27462f68e9
cmd/segment-reaper: add detecting and printing zombie segments () 2019-12-09 03:30:52 -08:00
Michal Niewrzal
b17d40ffd4
segment-reaper: delete command logic () 2019-12-09 00:25:39 -08:00
Ethan Adams
9420fa9fc5 satellite/gracefulexit: Add graceful exit completed/failed receipt verification to satellite CLI () 2019-12-03 17:09:39 -05:00
Ivan Fraixedes
0bf7ac5b20 cmd/segment-reaper: Use bitmask type in observer ()
bitmask type was created for making easy to understand the observer bit
logic.
This commit makes the changes in observer for using the bitmask type.
2019-11-27 13:11:07 +01:00
Ivan Fraixedes
7288d01781
cmd/segment-reaper: several refactorings ()
* Move the observer implementation and the type definitions related with
  it and helper functions to its own file.
* Cluster struct type is used as a key for a ObjectsMap type.
  Observer struct type has a field of ObjectsMap.
  Cluster has a field for project ID.
  Observer processSegment method uses its ObjectMap field for tracking
  objects.

  However Observer processSegment clears the map once the projectID
  diverges from the one kept in the Observer lastProjectID field, meaning
  that it isn't needed to keep the projectID as part of the ObjectMap key.

  For this reason, ObjectMap can use as a key just only the bucket name
  and Cluster struct isn't needed.

  Because of such change, the ObjectMap type has been renamed to a more
  descriptive name.
* Make the types defined for this specific package not being exported.
* Create a constructor function for observer to encapsulate the map
  allocation.
* Don't throw away the entirely buckets objects map when an empty one
  is used to reuse part of the allocations.

  Encapsulate the clearing up logic into a method.
* Make the analyzeProject function to be a method of observer.
2019-11-27 10:28:43 +01:00
Yingrong Zhao
66f1a1680f
add completion receipt to exit-status cli command on storage node () 2019-11-26 12:32:26 -05:00
Michal Niewrzal
1e33995a89
cmd/segment-reaper: skeleton command for zombie deletion () 2019-11-26 05:01:31 -08:00
JT Olio
01895d8bdd lib/uplink: explain safe versions ()
Change-Id: Idf7391474ab10b0dab6ce97c773cabee59d5b664
2019-11-25 16:55:54 +01:00
JT Olio
031ba86de5
argon2: choose a steady parallelism value ()
* argon2: choose a steady parallelism value

Change-Id: I6006da7d7980cda88f5f08ee759612df23a8132d

* whoops, not cruft

Change-Id: Ied9039f9a9be1d0f6ff3c7d5c4839a83fc7b4b1f

* fix broken test file

Change-Id: I07288cd6cef32ba387f2f003febff5c297e50997

* fix linting error

Change-Id: Icdbda8b709cc100a86f3859303c40edb8dff1e0f
2019-11-22 14:00:04 -07:00
Nikolai Siedov
ded7f6e2eb
web/satellite: registration/welcome message fixed, usage-report url fixed, storj-sim fixed () 2019-11-22 19:03:15 +02:00
Kaloyan Raev
bb16c07dc2
storagenode/updater: read identity location from storagenode's config.yaml () 2019-11-22 15:57:37 +02:00
Ivan Fraixedes
77d4add101
cmd/segment-reaper: Implement bitmask type ()
Define a bitmask type for implementing its logic to detect if an index
is set or not.
2019-11-22 11:53:48 +01:00
Michal Niewrzal
8ea09d5556
cmd/segment-reaper: add iteration over DB () 2019-11-18 09:13:20 -08:00
Kaloyan Raev
f4a626bbc6 storagenode-updater: re-enable auto-update storagenode-updater ()
storagenode-updater: re-enable auto-update for storagenode-updater
2019-11-18 08:26:50 +01:00
Jeff Wendling
ecd2ef4a21 all: build release fully dprc and test in mixed mode
Change-Id: I3bded3edf25a0b113601c8b12ecf1337f596649b
2019-11-15 10:03:18 -07:00
Michal Niewrzal
ead8af3c16
storagenode/updater: recover command in Service Manager () 2019-11-15 06:37:29 -08:00
Egon Elbre
ee6c1cac8a
private: rename internal to private () 2019-11-14 21:46:15 +02:00
Michal Niewrzal
4912922631
cmd: skeleton for segment-reaper detect command () 2019-11-14 07:57:27 -08:00
Bryan White
b56cc21710 cmd/storagenode-updater: make updater test windows compatible () 2019-11-14 16:03:49 +02:00
Egon Elbre
1e64006e32 lint: add staticcheck as a separate step () 2019-11-14 10:31:30 +02:00
Bryan White
4c822b630d {certificates,pkg/rpcstatus}: improve error logging () 2019-11-13 11:07:21 +01:00
Bryan White
7cc4217fef
cmd/storagenode-updater: simplify and reorder update sqeuence () 2019-11-12 13:31:57 +01:00
Kaloyan Raev
20623fdc96
Increase min required difficulty to 36 in signing service () 2019-11-11 12:27:09 +02:00
Bryan White
53a060cdfa
cmd/storagenode-updater: TODO cleanup () 2019-11-07 09:42:07 +01:00
Caleb Case
3b78addb2d
Metadata Access from Uplink CLI () 2019-11-06 11:04:12 -05:00
Michal Niewrzal
2f13c0adf3
storj-sim: add rollout seed to avoid warnings () 2019-11-06 06:06:10 -08:00
littleskunk
7eb6724c92
logging: unify logging around satellite ID, node ID and piece ID ()
* logging: unify logging around satellite ID, node ID and piece ID

* unify segment index
2019-11-05 22:04:07 +01:00
Yaroslav Vorobiov
35edc2bcc3 satellite/payments: invoice creation () 2019-11-05 15:16:02 +02:00
Jess G
5abb91afcf
satellite: change the Peer name to Core ()
* change satellite.Peer name to Core

* change to Core in testplanet

* missed a few places

* keep shared stuff in peer.go to stay consistent with storj/docs
2019-11-04 11:01:02 -08:00
Jennifer Li Johnson
0621830f05 cmd/storj-sim: allow user to designate which redis db they want to start at () 2019-11-04 14:55:02 +02:00
Kaloyan Raev
84fea5820f
Fix error when storagenode-updater service is stopped () 2019-11-04 11:22:08 +02:00
Kaloyan Raev
857c7f3cd7
storagenode/updater: disable self-autoupdate () 2019-11-04 09:33:39 +02:00
Jess G
8d92c288e2
satellitedb: separate migration into subcommand ()
* separate sadb migration, add version check

* update checkversion to do same validation as migration

* changes per CR

* add sa migration to storj-sim

* add different debug port in storj-sim for migration

* add wait for exit for storj-sim migration

* update sa docker entrypoint to support migration

* storj-sim satellite parts all wait for migration

* upgrade golang-migrate/migrate to v4 because bug

* fix go mod tidy
2019-11-02 13:09:07 -07:00
Egon Elbre
be2dd1ca72 cmd/storj-sim: add --redis flag () 2019-11-01 20:48:37 +01:00
Jennifer Li Johnson
76b64b79ba
cmd/identity: allow using redis for RevocationDB () 2019-11-01 13:27:47 -04:00
Kaloyan Raev
3c35339f02 Increase default difficulty in identity CLI to 36 () 2019-10-31 16:49:24 +01:00
Michal Niewrzal
015350e230
storagenode-updater: add autoupdating () 2019-10-31 05:27:53 -07:00
Jess G
e96d615013
satellite: remove satellite API code from peer ()
* rm dup api code from sa peer, update storj-sim

* fix for backwards compat tests

* use env var instead of localhost

* changes per CR

* fix env var name

* skip peer for setup
2019-10-30 12:23:09 -07:00
Egon Elbre
95cd3fc750 cmd/storj-sim: use less ports () 2019-10-29 11:25:25 -04:00
Cameron
b2ff13f1fa
{cmd/satellite, storj/satellite}: create command to run repair process in isolation ()
* set up satellite repair run command

* add separated repair process to storj-sim

* add repairer peer to satellite in testplanet

* move api run cmd into api.go

* add satellite run repair to entrypoint
2019-10-29 10:55:57 -04:00
Egon Elbre
93353df4d6
internal/sync2: make Fence accept context () 2019-10-28 16:04:31 +02:00
Bryan White
f17b34d02e
cmd/storagenode-updater: add "...up to date" log line if shoudn't update () 2019-10-28 11:19:19 +01:00
Kaloyan Raev
b10d444560
Rename --log-path back to --log for storagenode-updater () 2019-10-28 10:30:27 +02:00
paul cannon
1469f7f41f
storagenode/contact: wait for UpdateSelf before start ()
When the contact chore starts running before the monitor service has
provided any useful capacity data, the first outgoing contact has
not-very-helpful data for the satellite. This change causes the contact
chore to wait until capacity data is available. The wait should be quite
short in all reasonable cases: even when a node starts with a lot of
stored pieces and no cached spaceUsedDB data, new data will have been
calculated and cached by the call to
`peer.Storage2.CacheService.Init(ctx)` in `storagenode.cmdRun()` before
`peer.Run(ctx)`.

Change-Id: Ibc26d5c1fc10a23006c00bc3f13ff6cf71f8bf1d
2019-10-26 12:16:25 -05:00
Maximillian von Briesen
abb567f6ae
cmd/satellite: add graceful exit reports command to satellite CLI ()
* update lock file and add comment

* add created at and bytes transferred

* cleanup

* rename db func to GetGracefulExitNodesByTimeFrame

* fix flag

* split into two overlay functions

* := to =

* fix test

* add node not found error class

* fix overlay test

* suggested test changes

* review suggestions

* get exit status from overlay.Get()

* check rows.Err

* fix panic when ExitFinishedAt is nil

* fix comments in cmdGracefulExit
2019-10-22 21:06:01 -04:00
JT Olio
2c6fa3c5f8
pkg/rpc: remove read/write deadlines as a mechanism for request timeouts ()
libuplink was incorrectly setting timeouts to 10 seconds still, but
should have been at least 10 minutes. the order sender was setting them
to 1 hour. we don't want timeouts in uplink-side logic as it establishes
a minimum rate on tcp streams.

instead of all of this, just use tcp keep alive. tcp keep alive packets are
sent every 15 seconds and if the peer stops responding the connection
dies. this is enabled by default with go. this will kill tcp connections
when they stop working.

Change-Id: I3d7ad49f71950b3eb43044eedf4b17993116045b
2019-10-22 17:57:24 -06:00
Jess G
1014d5a7d1
cmd/satellite: add API run command to satellite dockerfile entrypoint ()
* update satellite entrypoint to add API run command

* make true a string for clarity
2019-10-21 09:16:59 -07:00
Bryan White
f468816f13
{internal/version,versioncontrol,cmd/storagenode-updater}: add rollout to storagenode updater () 2019-10-21 12:50:59 +02:00
Bryan White
243ba1cb17
{versioncontrol,internal/version,cmd/*}: refactor version control () 2019-10-20 09:56:23 +02:00
Egon Elbre
89ed997706
satellite/satellitedb: switch to postgres only () 2019-10-18 22:03:10 +03:00
Egon Elbre
44bf98ee9c
cmd/storj-sim: make postgres default to STORJ_SIM_POSTGRES () 2019-10-18 19:12:30 +03:00
Yehor Butko
26cc625dc6
satellite/console: payments api () 2019-10-17 17:42:18 +03:00
Jess G
34764e5c97
cmd/satellite: create API subcommand ()
* create sa api subcommand

* add api to storj-sim

* require postgres

* pass live accounting cache into satellite.NewAPI
2019-10-16 13:34:25 -07:00
Cameron
76ad83f12c
satellite/accounting: add redis support to live accounting ()
* set up redis support in live accounting

* move live.Service interface into accounting package and rename to Cache, pass into satellite

* refactor Cache to store one int64 total, add IncrBy method to redis client implementation

* add monkit tracing to live accounting
2019-10-16 12:50:29 -04:00
Yingrong Zhao
87e3764390
storagenode/cmd: add exit-status command for graceful exit ()
* add exit-status command

* remove todo and fix format

* fix status display

* change startExit to exit progress

* fix linting error

* add successful column in exit progress

* fix test

* remove extra new line

* fix TYPOS

* format the percentage better
2019-10-15 18:07:32 -04:00
Jennifer Li Johnson
b185dbbee2
satellite/discovery: remove discovery related code () 2019-10-14 10:57:01 -04:00
Yingrong Zhao
743a0fc38b storagenode/cmd: create start graceful exit CLI () 2019-10-11 09:58:12 -04:00