Commit Graph

4011 Commits

Author SHA1 Message Date
Michal Niewrzal
83cc80ef06 satellite/payments/stripecoinpayments: fix listing issue for coupon
usages and credit spendings

Test will be added as a separate change.

Change-Id: I9341d2ecb736fdbd45beb96f778f04448df68bd7
2020-05-21 07:48:04 +00:00
Egon Elbre
bef84a5f9d storagenode: remove dependency to overlay.NodeDossier
This is the last dependency from storage node to satellite.

Change-Id: I12f7abb91e84f823ba5af126c6e2979519838612
2020-05-21 08:37:13 +03:00
Egon Elbre
b42778c42e private/testplanet: remove some additional Local-s
Change-Id: I49701c41efb92efca27cc18d0a3f6d6b44d3cf8b
2020-05-21 08:37:13 +03:00
Egon Elbre
7f323754a4 metainfo/piecedeletion: use NodeURL-s
Change-Id: I247dbfe03e7864e940e4cd1d0f343f38e84099e0
2020-05-21 08:37:13 +03:00
Jennifer Johnson
03e5f922c3 satellite/overlay: updates node with a vetted_at timestamp if they meet the vetting criteria
What: As soon as a node passes the vetting criteria (total_audit_count and total_uptime_count
are greater than the configured thresholds), we set vetted_at to the current timestamp.

Why: We may want to use this timestamp in future development to select new vs vetted nodes.
It also allows flexibility in node vetting experiments and allows for better metrics around
vetting times.

Please describe the tests: satellitedb_test: TestUpdateStats and TestBatchUpdateStats make sure vetted_at is set appropriately
Please describe the performance impact: This change does add extra logic to BatchUpdateStats and UpdateStats and
commits another variable to the db (vetted_at), but this should be negligible.

Change-Id: I3de804549b5f1bc359da4935bc859758ceac261d
2020-05-20 16:30:26 -04:00
Natalie Villasana
2514d6328d dbutil/cockroachutil: add monkit to QueryContext
This will help us keep track of crdb errors in influx.

Change-Id: I997596aa4eb9a2b9b81305d123c3452ecdf5dde5
2020-05-20 14:56:25 -04:00
Bill Thorp
f43cb1688d private/tagsql: verify SQL connection with ping
Use ping to make sure the database connection is valid

Change-Id: I5217e28e186f487266c8f4a1d39cce0070dc1465
2020-05-20 13:12:16 +00:00
Michal Niewrzal
fe6a6f063f private/testplanet: cleanup predefined data generation
Use Console service to create user and project instead direct DB
modifications.

Change-Id: Ib0074b38313b3dc43b7d8d63ab2775d29028fb7b
2020-05-20 12:38:43 +00:00
crawter
2c9afe7f17 storagenode/console/api/helamount: periods with heldamount data endpoint added
Change-Id: Ie893f56f02c7a76bcfc21c32c10bd1f1d05660e7
2020-05-20 11:45:06 +00:00
Qweder93
49ad90dcd8 storagenode/reputation: unknown_score (unknown_alpha / unknown_alpha+unknown_beta) added to reputation stats, https://storjlabs.atlassian.net/browse/SG-326
Change-Id: I0b29ad736f7a11c7e57a846b6891f4b40755aa48
2020-05-20 11:25:14 +00:00
Egon Elbre
5d016425f1 satellite/{contact,downtime,overlay}: use NodeURL
Change-Id: I555a479a89e0ddbf0499898bdbc8574282cd6846
2020-05-20 11:09:05 +00:00
Egon Elbre
941d10cbc3 private/testplanet: remove Peer.Local()
Currently storagenode depends on overlay.NodeDossier, this is the first
step in removing it.

Change-Id: I034a3f1601835f8349bd41752455022e19bcc707
2020-05-20 11:05:34 +00:00
Egon Elbre
94b2b315f7 storagenode/trust: refactor GetAddress to GetNodeURL
Most places now need the NodeURL rather than the ID and Address
separately. This simplifies code in multiple places.

Change-Id: I52621d8ca52296a8b5bf7afbc1001cf8bfb44239
2020-05-20 11:05:15 +00:00
Egon Elbre
ed627144ed all: use DialNodeURL throughout the codebase
Change-Id: Iaf9ae3aeef7305c937f2660c929744db2d88776c
2020-05-20 10:36:30 +00:00
Ivan Fraixedes
963db8c738 cmd/segment-reaper: Remove unneeded test cases
Remove unneeded observer's tests cases after the changes that enhanced
the segment-reaper not to be limited to objects up to 65 segments.

Also remove some tests cases that were already covered by the
TestObserver_processSegment_from_to test function.

Change-Id: I2385b406b50f7810ed22d3297b903a993f83fcfe
2020-05-20 10:10:51 +00:00
Stefan Benten
1a4ab3df48
satellite/admin: add user update endpoint (#3896) 2020-05-20 12:20:04 +03:00
littleskunk
8ec64f3daf
satellite/overlay: enable node selection cache on all satellites (#3895) 2020-05-19 19:25:53 +02:00
Natalie Villasana
c7cbdcbccf cmd/internal: request lowercase access name
Viper has a feature/bug where all YAML config keys are cast
to lowercase (see https://github.com/spf13/viper/issues/260
and https://github.com/spf13/viper/issues/411).
Since we use Viper to load our config values, it doesn't seem
like there's an easy way to preserve case sensitivity for the
access names chosen by users right now. Adding this prompt
should help user experience by clarifying that all access
names must be lowercase.

Change-Id: I47e8344bb0ca7e78458405496f20e78e3c9f9a88
2020-05-19 16:04:35 +00:00
Jess G
6fac1ed130
satellite/contact: rm unhelpful verbose logging (#3894)
* unhelpful rm verbose logging

Change-Id: I74dc29f18969250fa6838fed9bec8ba7ea160b77

* change to debug

Change-Id: I4ba880e68b40bc3e0f29d9b1aa9154e63124c904
2020-05-19 08:09:00 -07:00
Michal Niewrzal
705e82ea99 private/testplanet: add AddUser and AddProject to satellite
functionality

We want to start adding more complex test cases for billing/invoices and
we need more handy tooling to be able do this easily.

Change-Id: Ib22ac6b4ba9ee77cc91c88b0cfd2d2efc15657df
2020-05-19 13:02:04 +00:00
Stefan Benten
0a26c4af9a
satellite/admin: add coupon deletion (#3893) 2020-05-19 15:49:44 +03:00
Stefan Benten
671aca56b0
satellite/admin: add coupon creation and listing (#3891) 2020-05-19 12:36:13 +02:00
Ivan Fraixedes
cbdd3bfd7b cmd/segment-reaper: Rename test name to what it tests
Rename a test function to indicate the function that it actually tests
rather than indicating the function that calls the function under test.

Change-Id: I2c1fe165e3acf952993a9a65dfbfbdcda0d1cccc
2020-05-19 10:06:56 +00:00
Michal Niewrzal
ac375d37bc satellite/payments: remove mockpayments and add Stripe client mock
instead

Change-Id: If3496f6abc16da90d2b43fa0c5be356847a39507
2020-05-19 09:35:37 +02:00
Egon Elbre
16abf02b35 satellite/{nodeselection,overlay}: use the new package
Change-Id: I034fdbe578dec2e5c906aca82231cd3e56f26aeb
2020-05-18 21:38:43 +00:00
Stefan Benten
17d3cb1551
satellite/admin: add user creation endpoint (#3889) 2020-05-18 22:37:18 +02:00
Stefan Benten
1de813d22e
cmd/storj-admin: initial commit of new tool (#3890) 2020-05-18 22:16:15 +02:00
Ivan Fraixedes
5ea82dfb2e cmd/segment-reaper: Report zombie segment's size
Makes the detect command of the segments reaper to report the size of
each zombie segment.

Change-Id: Ibded5f4d1b53f6d2805dac96fd306452af7f2524
2020-05-18 18:25:14 +00:00
Egon Elbre
08692aef90 satellite/nodeselection: node selection with proper bias
Currently node selection cache is biased towards the same subnet. This
implements static node selection for distinct such that it selects with
equal probability subnets rather than id-s.

This is mostly a copy paste + modifications from previous node selection
state.

Change-Id: Ia5c0aaf68e7feca78fbbd7352ad369fcb77c3a05
2020-05-18 18:09:15 +00:00
littleskunk
1b06e2a928
satellite/metainfo: remove deprecated error validation (#3876) 2020-05-18 19:58:42 +02:00
Stefan Benten
70502658c8
satellite/admin: add delete project endpoint (#3888) 2020-05-18 19:36:09 +02:00
Jessica Grebenschikov
1ecbfc453f satellite/overlay: add random test for node selection from cache
Change-Id: I57b57db919b5afb3fc81176e078ac1e3a873565a
2020-05-18 16:59:52 +00:00
Ivan Fraixedes
d27cb60f81 cmd/segment-reaper: Remove unneeded test logic
After the segment-reaper was enhanced to support objects with any number
of segments, the test helper function to generate test data doesn't have
to support to generate objects with a number of segments up to or over
the previous limit.

The commit remove that unnecessary test logic.

Change-Id: I2897c5a96fb133f61de2cdb0ef7d13ee5e0151e8
2020-05-18 16:39:48 +00:00
Kaloyan Raev
aac1e3c45f satellite/payments: move inspector commands to satellite cli
This allows to seeing logs in the output of the invoice commands.

Existing ensure-stripe-customer commands is moved from the 'reports' to
the new 'billing' root command.

Change-Id: I752c7ab6ca59bfac8e0f174a45d2ab45fc18e467
2020-05-18 16:12:45 +00:00
VitaliiShpital
47def02094 satellite: account balance divided into Free Credits and Coins
Change-Id: I2ae29701bab9f0b48f236ff94b366eb534b81c6b
2020-05-18 14:25:26 +00:00
Michal Niewrzal
9fb423ac19 satellite/payments/stripecoinpayments: fix listing issue in
InvoiceApplyProjectRecords

ListUnapplied method in listing loop  was using next offset as starting
point but applyProjectRecords method was changing project record state
to applied and we were missing some records to apply.

Test will be added as a separate change.

Change-Id: Id1ca33eeb66ec7f6ff1f05b45615a8935185568e
2020-05-18 13:11:30 +00:00
VitaliiShpital
4544f2dffb web/satellite: navigation side bar reworked
Change-Id: I9be2d23a7a4af219d0b8888b7296fa7b0e57fc5a
2020-05-18 11:04:45 +00:00
Natalie Villasana
8d87a6efc9 cockroachutil/driver: handle retryable errors returned from Next
This will only work if retryable errors are returned on the first
call to Next. Otherwise if they're returned later, we will need
deeper changes at the application code level throughout the
codebase 😬👎

Change-Id: I46d795a13670f66b7f085605ba1b779f69c339c3
2020-05-15 14:49:43 -04:00
littleskunk
ef2671927d
storagenode/piecestore: move queue size defaults (#3881) 2020-05-15 19:10:26 +02:00
NickolaiYurchenko
ffbaaff179 web/storagenode: node module store tests added
Change-Id: I582fc1405263965e870a233def3f2d026490dd70
2020-05-15 14:41:48 +00:00
Stefan Benten
26f36d590e
cmd/satellite: ensure we only create stripe customers for activated users (#3884) 2020-05-15 16:00:46 +02:00
Egon Elbre
f6301612ac satellite/orders: use serial SerialNumber
By ensuring that they have less randomness it means that they can be
compressed better. Using a timestamp should be a good improvement here.

Change-Id: Ic4dabb53335a744ff1c332dd279f37ae2cd79357
2020-05-15 11:33:53 +00:00
Michal Niewrzal
2eb2c25e51 satellite/payments/stripecoinpayments: add StripeClient interface to be
able to cover more testing scenarios

Currently, its hard to implement test suite for payments because
mockpayments is on to high level and we cannot emulate many things e.g.
adding credit card. This change is first to be able to add mock for
Stripe client and do more granular tests.

Change-Id: Ied85d4bd0642debdffe1161657c1e475202e9d23
2020-05-15 10:52:44 +02:00
VitaliiShpital
1ec5eb06bd web/satellite: account dropdown reworked
Change-Id: I415f77c7c13546f467da62fc13feb2774dd54290
2020-05-14 17:08:02 +00:00
VitaliiShpital
a3459ec6b8 web/satellite: project dashboard page UI reworked
Change-Id: I134d70b3a351d2a104612727f7c24e21122a6730
2020-05-14 16:52:11 +00:00
Jeff Wendling
f03b23d2dc storage/postgreskv: monitor calls to sql.Next
Change-Id: I32c0c92b347a6d1cadfeb69117de58f4d9b41ad2
2020-05-14 16:35:25 +00:00
littleskunk
28a886f399
satellite/admin: obvious project name in documentation 2020-05-14 15:50:58 +03:00
Kaloyan Raev
49571f1a23 satellite/payments: all invoice commands require period
To avoid including multiple months in a single invoice, we need all
inspector's invoice commands to run in for specific period.

See https://storjlabs.atlassian.net/browse/USR-725

Change-Id: I3637dc189234f02350daca8d897c21765762ea55
2020-05-14 11:50:19 +00:00
Qweder93
ef87192120 storagenode/notifications: tests fixed: added time interval between inserts so created_at fields are different when running tests on Windows
Change-Id: I26ba059ab58d0216122ab2f49ae85f7ce7cfced4
2020-05-14 05:26:18 +00:00
Moby von Briesen
46df8c1977 satellite/gracefulexit: add log message when node fails validation for piece transfer
Change-Id: Ic5a53404ceb35003793aebc63637e7f8a58ef259
2020-05-13 16:58:50 +00:00