Commit Graph

4202 Commits

Author SHA1 Message Date
Qweder93
49ad90dcd8 storagenode/reputation: unknown_score (unknown_alpha / unknown_alpha+unknown_beta) added to reputation stats, https://storjlabs.atlassian.net/browse/SG-326
Change-Id: I0b29ad736f7a11c7e57a846b6891f4b40755aa48
2020-05-20 11:25:14 +00:00
Egon Elbre
5d016425f1 satellite/{contact,downtime,overlay}: use NodeURL
Change-Id: I555a479a89e0ddbf0499898bdbc8574282cd6846
2020-05-20 11:09:05 +00:00
Egon Elbre
941d10cbc3 private/testplanet: remove Peer.Local()
Currently storagenode depends on overlay.NodeDossier, this is the first
step in removing it.

Change-Id: I034a3f1601835f8349bd41752455022e19bcc707
2020-05-20 11:05:34 +00:00
Egon Elbre
94b2b315f7 storagenode/trust: refactor GetAddress to GetNodeURL
Most places now need the NodeURL rather than the ID and Address
separately. This simplifies code in multiple places.

Change-Id: I52621d8ca52296a8b5bf7afbc1001cf8bfb44239
2020-05-20 11:05:15 +00:00
Egon Elbre
ed627144ed all: use DialNodeURL throughout the codebase
Change-Id: Iaf9ae3aeef7305c937f2660c929744db2d88776c
2020-05-20 10:36:30 +00:00
Ivan Fraixedes
963db8c738 cmd/segment-reaper: Remove unneeded test cases
Remove unneeded observer's tests cases after the changes that enhanced
the segment-reaper not to be limited to objects up to 65 segments.

Also remove some tests cases that were already covered by the
TestObserver_processSegment_from_to test function.

Change-Id: I2385b406b50f7810ed22d3297b903a993f83fcfe
2020-05-20 10:10:51 +00:00
Stefan Benten
1a4ab3df48
satellite/admin: add user update endpoint (#3896) 2020-05-20 12:20:04 +03:00
littleskunk
8ec64f3daf
satellite/overlay: enable node selection cache on all satellites (#3895) 2020-05-19 19:25:53 +02:00
Natalie Villasana
c7cbdcbccf cmd/internal: request lowercase access name
Viper has a feature/bug where all YAML config keys are cast
to lowercase (see https://github.com/spf13/viper/issues/260
and https://github.com/spf13/viper/issues/411).
Since we use Viper to load our config values, it doesn't seem
like there's an easy way to preserve case sensitivity for the
access names chosen by users right now. Adding this prompt
should help user experience by clarifying that all access
names must be lowercase.

Change-Id: I47e8344bb0ca7e78458405496f20e78e3c9f9a88
2020-05-19 16:04:35 +00:00
Jess G
6fac1ed130
satellite/contact: rm unhelpful verbose logging (#3894)
* unhelpful rm verbose logging

Change-Id: I74dc29f18969250fa6838fed9bec8ba7ea160b77

* change to debug

Change-Id: I4ba880e68b40bc3e0f29d9b1aa9154e63124c904
2020-05-19 08:09:00 -07:00
Michal Niewrzal
705e82ea99 private/testplanet: add AddUser and AddProject to satellite
functionality

We want to start adding more complex test cases for billing/invoices and
we need more handy tooling to be able do this easily.

Change-Id: Ib22ac6b4ba9ee77cc91c88b0cfd2d2efc15657df
2020-05-19 13:02:04 +00:00
Stefan Benten
0a26c4af9a
satellite/admin: add coupon deletion (#3893) 2020-05-19 15:49:44 +03:00
Stefan Benten
671aca56b0
satellite/admin: add coupon creation and listing (#3891) 2020-05-19 12:36:13 +02:00
Ivan Fraixedes
cbdd3bfd7b cmd/segment-reaper: Rename test name to what it tests
Rename a test function to indicate the function that it actually tests
rather than indicating the function that calls the function under test.

Change-Id: I2c1fe165e3acf952993a9a65dfbfbdcda0d1cccc
2020-05-19 10:06:56 +00:00
Michal Niewrzal
ac375d37bc satellite/payments: remove mockpayments and add Stripe client mock
instead

Change-Id: If3496f6abc16da90d2b43fa0c5be356847a39507
2020-05-19 09:35:37 +02:00
Egon Elbre
16abf02b35 satellite/{nodeselection,overlay}: use the new package
Change-Id: I034fdbe578dec2e5c906aca82231cd3e56f26aeb
2020-05-18 21:38:43 +00:00
Stefan Benten
17d3cb1551
satellite/admin: add user creation endpoint (#3889) 2020-05-18 22:37:18 +02:00
Stefan Benten
1de813d22e
cmd/storj-admin: initial commit of new tool (#3890) 2020-05-18 22:16:15 +02:00
Ivan Fraixedes
5ea82dfb2e cmd/segment-reaper: Report zombie segment's size
Makes the detect command of the segments reaper to report the size of
each zombie segment.

Change-Id: Ibded5f4d1b53f6d2805dac96fd306452af7f2524
2020-05-18 18:25:14 +00:00
Egon Elbre
08692aef90 satellite/nodeselection: node selection with proper bias
Currently node selection cache is biased towards the same subnet. This
implements static node selection for distinct such that it selects with
equal probability subnets rather than id-s.

This is mostly a copy paste + modifications from previous node selection
state.

Change-Id: Ia5c0aaf68e7feca78fbbd7352ad369fcb77c3a05
2020-05-18 18:09:15 +00:00
littleskunk
1b06e2a928
satellite/metainfo: remove deprecated error validation (#3876) 2020-05-18 19:58:42 +02:00
Stefan Benten
70502658c8
satellite/admin: add delete project endpoint (#3888) 2020-05-18 19:36:09 +02:00
Jessica Grebenschikov
1ecbfc453f satellite/overlay: add random test for node selection from cache
Change-Id: I57b57db919b5afb3fc81176e078ac1e3a873565a
2020-05-18 16:59:52 +00:00
Ivan Fraixedes
d27cb60f81 cmd/segment-reaper: Remove unneeded test logic
After the segment-reaper was enhanced to support objects with any number
of segments, the test helper function to generate test data doesn't have
to support to generate objects with a number of segments up to or over
the previous limit.

The commit remove that unnecessary test logic.

Change-Id: I2897c5a96fb133f61de2cdb0ef7d13ee5e0151e8
2020-05-18 16:39:48 +00:00
Kaloyan Raev
aac1e3c45f satellite/payments: move inspector commands to satellite cli
This allows to seeing logs in the output of the invoice commands.

Existing ensure-stripe-customer commands is moved from the 'reports' to
the new 'billing' root command.

Change-Id: I752c7ab6ca59bfac8e0f174a45d2ab45fc18e467
2020-05-18 16:12:45 +00:00
VitaliiShpital
47def02094 satellite: account balance divided into Free Credits and Coins
Change-Id: I2ae29701bab9f0b48f236ff94b366eb534b81c6b
2020-05-18 14:25:26 +00:00
Michal Niewrzal
9fb423ac19 satellite/payments/stripecoinpayments: fix listing issue in
InvoiceApplyProjectRecords

ListUnapplied method in listing loop  was using next offset as starting
point but applyProjectRecords method was changing project record state
to applied and we were missing some records to apply.

Test will be added as a separate change.

Change-Id: Id1ca33eeb66ec7f6ff1f05b45615a8935185568e
2020-05-18 13:11:30 +00:00
VitaliiShpital
4544f2dffb web/satellite: navigation side bar reworked
Change-Id: I9be2d23a7a4af219d0b8888b7296fa7b0e57fc5a
2020-05-18 11:04:45 +00:00
Natalie Villasana
8d87a6efc9 cockroachutil/driver: handle retryable errors returned from Next
This will only work if retryable errors are returned on the first
call to Next. Otherwise if they're returned later, we will need
deeper changes at the application code level throughout the
codebase 😬👎

Change-Id: I46d795a13670f66b7f085605ba1b779f69c339c3
2020-05-15 14:49:43 -04:00
littleskunk
ef2671927d
storagenode/piecestore: move queue size defaults (#3881) 2020-05-15 19:10:26 +02:00
NickolaiYurchenko
ffbaaff179 web/storagenode: node module store tests added
Change-Id: I582fc1405263965e870a233def3f2d026490dd70
2020-05-15 14:41:48 +00:00
Stefan Benten
26f36d590e
cmd/satellite: ensure we only create stripe customers for activated users (#3884) 2020-05-15 16:00:46 +02:00
Egon Elbre
f6301612ac satellite/orders: use serial SerialNumber
By ensuring that they have less randomness it means that they can be
compressed better. Using a timestamp should be a good improvement here.

Change-Id: Ic4dabb53335a744ff1c332dd279f37ae2cd79357
2020-05-15 11:33:53 +00:00
Michal Niewrzal
2eb2c25e51 satellite/payments/stripecoinpayments: add StripeClient interface to be
able to cover more testing scenarios

Currently, its hard to implement test suite for payments because
mockpayments is on to high level and we cannot emulate many things e.g.
adding credit card. This change is first to be able to add mock for
Stripe client and do more granular tests.

Change-Id: Ied85d4bd0642debdffe1161657c1e475202e9d23
2020-05-15 10:52:44 +02:00
VitaliiShpital
1ec5eb06bd web/satellite: account dropdown reworked
Change-Id: I415f77c7c13546f467da62fc13feb2774dd54290
2020-05-14 17:08:02 +00:00
VitaliiShpital
a3459ec6b8 web/satellite: project dashboard page UI reworked
Change-Id: I134d70b3a351d2a104612727f7c24e21122a6730
2020-05-14 16:52:11 +00:00
Jeff Wendling
f03b23d2dc storage/postgreskv: monitor calls to sql.Next
Change-Id: I32c0c92b347a6d1cadfeb69117de58f4d9b41ad2
2020-05-14 16:35:25 +00:00
littleskunk
28a886f399
satellite/admin: obvious project name in documentation 2020-05-14 15:50:58 +03:00
Kaloyan Raev
49571f1a23 satellite/payments: all invoice commands require period
To avoid including multiple months in a single invoice, we need all
inspector's invoice commands to run in for specific period.

See https://storjlabs.atlassian.net/browse/USR-725

Change-Id: I3637dc189234f02350daca8d897c21765762ea55
2020-05-14 11:50:19 +00:00
Qweder93
ef87192120 storagenode/notifications: tests fixed: added time interval between inserts so created_at fields are different when running tests on Windows
Change-Id: I26ba059ab58d0216122ab2f49ae85f7ce7cfced4
2020-05-14 05:26:18 +00:00
Moby von Briesen
46df8c1977 satellite/gracefulexit: add log message when node fails validation for piece transfer
Change-Id: Ic5a53404ceb35003793aebc63637e7f8a58ef259
2020-05-13 16:58:50 +00:00
Jeff Wendling
6352d46100 satellite/satellitedb: do better ::date conversions
There is a subtle problem when one does a cast with `::date`. Observe:

    teststorj=# set timezone = 'US/Eastern';
    SET
    teststorj=# select (timestamp with time zone '2020-02-01 00:00:00+00')::date;
        date
    ------------
     2020-01-31
    (1 row)

    teststorj=# set timezone = 'UTC';
    SET
    teststorj=# select (timestamp with time zone '2020-02-01 00:00:00+00')::date;
        date
    ------------
     2020-02-01
    (1 row)

In order to correctly determine the date a timestamp is in, one has to
explicitly pick the time zone that the date truncation should use
otherwise postgres will use whatever setting the client has. These
tests were failing for me locally, because I run my postgres in
the US/Eastern time zone to try to tickle these bugs out. So it
should be `(x at time zone 'UTC')::date` instead of just `x::date`.

Change-Id: I4e9e32d4b53abc6165a4d0474f4702f8b9f801c7
2020-05-13 15:58:07 +00:00
igor gaidaienko
75ddecdf54 satellite/metainfo: Add test to verify payment report for attributed bucket
For this automated test I want to make sure the data in payment report is correct

Change-Id: I48a3b0e1dc9b2324435b763c87c363f1e310ff4e
2020-05-13 17:22:04 +03:00
Bill Thorp
5a7a4d2e98 satellite: add Go test version of satellite-config-lock tests
The current satellite config lock code relies on bash scripts and
gnu diff, it must be run as root and hence it typically requires
docker.  The old version will be removed at a later date..

I tried for several hours to run directly against cmdSetup() in
cmd/satellite/main.go, to avoid the ctx.Compile() call. I had no
luck.

Change-Id: I0a4888421e743b436d32b6af69d04759d7816751
2020-05-13 08:14:24 +00:00
NickolaiYurchenko
ff68a3c2bb web/storagenode: payout page scroll area extended
Change-Id: I937a5e7344a0c3507fe3aa2443458ee86c8f9af7
2020-05-12 21:27:14 +00:00
Ethan
159df8b2e4 Add logging listener for retrieving and setting log levels
See https://storjlabs.atlassian.net/browse/SM-752

These changes allow us to change the log level at runtime through a handler off of the debug endpoint.

Examples of changing the log level on storj-sim

To get the current level for the satellite api process:
curl -XGET 'http://127.0.0.1:10009/logging' --header 'Content-Type: text/plain'

To change the log level:
curl -XPUT 'http://127.0.0.1:10009/logging' --header 'Content-Type: text/plain' --data-raw '{"level":"error"}'

Change-Id: I05d164b290929fa06b6d78c01075ee41f8238044
2020-05-12 16:38:06 -04:00
NickolaiYurchenko
4b4222c2f9 web/storagenode: disposed held added to total held calculations
Change-Id: I593c9ce70c17b3b7a5ff0485a375fe2050ca8451
2020-05-12 17:51:17 +00:00
igor gaidaienko
1eab5e2980 satellite/console: Increase default webUI rate limit to 5
Previous limit is annoying for normal users

Change-Id: I7cb783e0b2515f415b2a055d5e811efab3810654
2020-05-12 16:12:17 +00:00
Egon Elbre
0e3be60b79 satellite/satellitedb: simplify migrate step
Change-Id: Ie4574144fb6ddd057d5fca740702c59fbdb2c5e4
2020-05-12 18:27:07 +03:00
Michal Niewrzal
0e263f0099 satellite/admin: add API endpoint for adding projects
Change-Id: I7970aa8bec810ee36727446d632a950c836f0597
2020-05-12 14:55:41 +00:00