Commit Graph

6774 Commits

Author SHA1 Message Date
Yaroslav Vorobiov
0036ec99cf testsuite/storjscan: add chore test
Adds test for storjcan chore which creates one tx for claimed
wallet, triggers chore to query new payments from storjscan and
uses service to retrieve payment.

Change-Id: Ic774213420f1b01d8adfeeb5ca8054d9c918e7b1
2022-08-05 18:37:52 +00:00
Yaroslav Vorobiov
379711cfd8 testsuite/storjscan: add client payments request test
Adds integration test for storjscan API client get payments call
using testsuite/storjscan.

Change-Id: Ief16b5ffc6257dcdba3d57bb402b0c3903085a40
2022-08-05 17:58:36 +00:00
Yaroslav Vorobiov
a6ad86dc53 testsuite/storjscan: add client claim wallet test
Adds integration test for storjscan client claim eth wallet method.

Change-Id: I69d1e4f9417c1225e97d5e5c6f0f394eaf68fbef
2022-08-05 17:26:14 +00:00
Yaroslav Vorobiov
53db7a897f testsuite/storjscan: add testsuite to test storjscan integration
Create Run function which spins up storjscan, eth test network and
testplanet configured with storjscan endpoint and credentials.

Change-Id: I5bafd07b6b2e4f39228faf76af7dd7135bed5f77
2022-08-05 16:49:22 +00:00
cl-mitch
0f3da7f895
web/satellite: Added token transaction history to new payment methods flow (#4977)
Added code for SortingHeader2.vue, VerticalArrows.vue, and TokenTransactionItem.vue. This code is for some of the stubbed components on PaymentMethodsTab.vue.
2022-08-05 10:03:26 -05:00
nerdatwork
81327f669a
web/satellite: Fix documentation URL (#5049)
Keeping all documentation links from same domain `docs.storj.io`

Co-authored-by: Clement Sam <clementsam75@gmail.com>
2022-08-05 11:38:49 +00:00
Vitalii
e1a85fccf4 web/satellite: added notification when storage limit is exceeded in file browser
Added error notification when user tries to upload object after storage limit is exceeded.
Slightly changed error handling on failed preview/map request.

Issue: https://github.com/storj/storj/issues/5003

Change-Id: Iee91c1037b5c6de7b9718adcdfcea850fd3056ca
2022-08-05 07:54:50 +00:00
Vitalii
27f6fbdeda web/satellite: refactored modals to use common VModal component
Refactored MFA, project members, profile and change password modals to use common VModal component.
Didn't touch access grant modals.

Change-Id: I9af5277496a99ec907fbee57a84127064ca9c02b
2022-08-04 21:46:30 +00:00
Márton Elek
1be5277c2a satellite/consoleweb: fix flaky TestAuth tests
We had a lot of flaky test failures from TestAuth. The error message (WHICH IS NOT VISIBLE IN JEKNINS, only in tests.json):

```
FAIL: TestAuth_Register_NameSpecialChars/Postgres (1.04s)
panic: runtime error: index out of range [0] with length 0 [recovered]
	panic: runtime error: index out of range [0] with length 0

goroutine 3473 [running]:

testing.tRunner.func1.2({0x235fe40, 0xc000fe6a08})
	/usr/local/go/src/testing/testing.go:1209 +0x36c
testing.tRunner.func1()
	/usr/local/go/src/testing/testing.go:1212 +0x3b6
panic({0x235fe40, 0xc000fe6a08})
	/usr/local/go/src/runtime/panic.go:1047 +0x266
storj.io/storj/satellite/console/consoleweb/consoleapi_test.TestAuth_Register_NameSpecialChars.func1(0xc001a281a0, 0x289d650, 0xc001a30000)
	/var/lib/jenkins/workspace/storj-gerrit-verify/satellite/console/consoleweb/consoleapi/auth_test.go:773 +0x785
storj.io/storj/private/testplanet.Run.func1.1({0x289c770, 0xc0001b8008})
	/var/lib/jenkins/workspace/storj-gerrit-verify/private/testplanet/run.go:67 +0x732
storj.io/storj/private/testmonkit.RunWith({0x289c770, 0xc0001b8008}, {0x28d89b0, 0xc001a281a0}, {0x1, {0x0, 0x0}, {0x0, 0x0, 0x0}}, ...)
```

The root cause:

testplanet uses a simulated mail sender which clicks to all the registration links by default (async).

These tests creat links and check the unverified users, but without enough luck the mail sender may already clicks to the link which makes the user verified.

Change-Id: I17cd6bf4ae3e7adc223ec693976bb609370f0c44
2022-08-04 19:06:07 +00:00
Vitalii
d76acda27e satellite/{web, console}: removed account locked statuses
Removed all the account locked messages.
Removed toast notification for login screen.
Updated wrong login credentials message.

Issues:
https://github.com/storj/storj/issues/4910
https://github.com/storj/storj/issues/4953

Change-Id: I1ac0ce16d3c1317204c28a509c21ebf3686a145e
2022-08-04 16:19:23 +00:00
Vitalii
30727b9036 web/satellite: add data delay info for new project dashboard
Added label for new project dashboard which tells user that there might be a delay before actual data is shown.

Issue: https://github.com/storj/storj/issues/5002

Change-Id: Ife317f68c4bd3c5eadc72e35ed4e77c97b1c038e
2022-08-04 14:29:04 +00:00
Vitalii
c13a4702b3 web/satellite: update dashboard chart data on project switch
Refetch chart daily data on project switch.

Issue: https://github.com/storj/storj/issues/5008

Change-Id: I883d51dace3a57f84f8044b13902e9ac965eb9bc
2022-08-04 13:27:06 +00:00
wilfredasomani
e0c863ed51 web/satellite: update "deposit storj" message to use support link rather than email
This change replaces the support email with a link in the "deposit storj token" message.

see: https://github.com/storj/storj/issues/5007

Change-Id: Icff1171e563a6b9184eb0e2828cd61e3c95b45e4
2022-08-04 11:44:12 +00:00
Vitalii
ec72adb2a6 satellite/console: send email when user's account gets locked
We send an email when user's account gets locked.

Issue: https://github.com/storj/storj/issues/4967

Change-Id: I68beceda0ac09128755c0333dfa014bd5a186317
2022-08-04 10:39:20 +00:00
Márton Elek
5a2e348b06 build: bump timeout for premerge/public
As we started to run integration tests one by one (instead of parallel). We need to bump the timeout as well.

(Later we can decrease it to a better value after understanding what is reasonable)

Change-Id: I4a7e18973311fe98b0ffb3d9ed496584b1697980
2022-08-04 09:33:48 +00:00
Egon Elbre
e9692c5681 storagenode/gracefulexit: remove unused interface
Change-Id: Ie6c3d69f5177872d8f4308ac476bc87655da9e4b
2022-08-04 11:26:14 +03:00
Egon Elbre
cf92220c20 {satellite,storagenode}/gracefulexit: simplify limiter usage
Change-Id: Ied7091fe5355b96d327e3f893c5bdd4946a9e6af
2022-08-04 08:18:15 +00:00
Vitalii
f562277344 satellite/{web,console}: added string length limits for signup partner and promo params
Added string length limits for registration partner and promo params.
Limitation added both on client and server sides.

Issue: https://github.com/storj/storj-private/issues/44

Change-Id: Ifae04caad1775e0a8ca72ae7f9abcf0ea5fb564b
2022-08-04 07:40:33 +00:00
NickolaiYurchenko
af6f5568c3 web/satellite: navigation mobile adaptation
added basic adaptations for root dashboard container
added mobile navigation component

Change-Id: Ic650ff8788e615d88b2dbb2fa14f281d0d53a693
2022-08-04 00:15:44 +00:00
Vitalii
ad37ea4518 satellite/{web, console}: login captcha implemented
Implemented Recaptcha and Hcaptcha for login screen.
Slightly refactored registration page implementation.
Made 2 different login/registration captcha configs on server side to easily swap between captchas independently.

Issue: https://github.com/storj/storj/issues/4982

Change-Id: I362bd5db2d59010e90a22301893bc3e1d860293a
2022-08-03 23:02:27 +00:00
paul cannon
799b159bba satellite/reputation: offset write times by random, not by satelliteID
In an effort to distribute load on the reputation database, the
reputation write cache scheduled nodes to be written at a time offset by
the local nodeID. The idea was that no two repair workers would have the
same nodeID, so they would not tend to write to the same row at the same
time.

Instead, since all satellite processes share the same satellite ID
(duh), this caused _all_ workers to try and write to the same row at the
same time _always_. This was not ideal.

This change uses a random number instead of the satellite ID. The random
number is sourced from the number of nanoseconds since the Unix epoch.
As long as workers are not started at the exact same nanosecond, they
ought to get well-distributed offsets.

Change-Id: I149bdaa6ca1ee6043cfedcf1489dd9d3e3c7a163
2022-08-03 21:14:06 +00:00
dlamarmorgan
92be1d878f satellite/payments/stripecoinpayments: storjscan invoice generation
Add line item with unclaimed Storjscan wallet balance during invoice generation.

Change-Id: I018bfa01abfcf7bfdffba0c5a1350a69188f63d5
2022-08-03 13:24:26 -07:00
Jeremy Wharton
5f6892f95c satellite/console/consoleweb: early rejection in auth middleware
This change makes the authentication middleware reject any requests
that are not properly authenticated to prevent them from being
passed into endpoint-specific handlers.

Change-Id: I1f6b74f68fc7354e47fb825a128bad968129f420
2022-08-03 18:52:51 +00:00
dlamarmorgan
c8b72c8f71 satellite/{payments/billing,satellitedb}: refactor billing DB
Update the billing table to use generated IDs, to include the source and status fields, and to add metadata jsonb field that can be used for fields specific to a particular type of billing transaction. Additionally a new table was added to keep track of user balances

Change-Id: Ieb3a63aafd8fe21fc3386bafd43d52081b7d2838

satellite/{payments/billing,satellitedb}: refactor billing DB

Update the billing table to use generated IDs, to include the source and status fields, and to add metadata jsonb field that can be used for fields specific to a particular type of billing transaction. Additionally a new table was added to keep track of user balances

Change-Id: Ieb3a63aafd8fe21fc3386bafd43d52081b7d2838
2022-08-03 17:52:32 +00:00
Cameron
2a38740efa satellite/satellitedb: don't select public_id in ListByOwnerID because it will be NULL for old projects
Change-Id: I062fa617b622a55d9482206d3a2938cf8a080f2d
2022-08-03 11:37:12 -04:00
Michal Niewrzal
5458cbdad8 satellite/metabase/metabasetest: use interface for tests and benchmarks
metabasetest package utils can be used by both tests and benchmarks
if we will use interface TestingT from require package. This change
adjusts metabasettest.CreateObject method

Change-Id: I3c138e2ef9873b804ab5b3402804efa409397a9f
2022-08-03 13:08:06 +00:00
Márton Elek
bed2af3e87 web: use fixed version from eslint-storj
eslint-plugin-storj is uploaded to the main npm registry.

While we have no clear proof that it causes the random freeze of npm install, it can be fater anyway as fixed version can be cached safely.

Change-Id: Ie74d93c7c4863e46121a9ab6c4e6b787fa3899b3
2022-08-03 12:32:01 +00:00
Márton Elek
2a3c821bd2 build: turn off parallel Verification for premerge/public
Our tests depend on timing (timeouts / sleeps). Therefore we must not overcommit the CPU with highly parallel execution as builds become flaky.

go test can do parallel execution (that's fine as it doesn't overcommit CPU, if only one job is running), but we shouldn't execute different type of
integration tests at the same time.

This commit changes only public (PR + main build) and pre-merge (one build per patch), which are less build time sensitive.

Change-Id: Ib9aaabd301cfa9cd3515a8aa4dd8ae82f58eb811
2022-08-03 11:34:58 +00:00
Egon Elbre
bc9ab8ee5e satellite/audit,storagenode/gracefulexit: fixes to limiter
Ensure we don't rely on limiter to wait multiple times.

Change-Id: I75d48420236216d4c2fc6fa99293f51f80cd9c33
2022-08-03 10:24:16 +03:00
Egon Elbre
1cb3cbaecf cmd/storj-sim: ignore context canceled and flush output
It's possible that content was not being flushed from processes.

For now, ignore other process failures under storj-sim network test.
Once we get other processes stable, we can repropagate the error.

Change-Id: I01ed572d7c779ab6451124f1e24e3d1168b3ea79
2022-08-02 20:48:46 +03:00
Qweder93
98b8c7be06 storagenode/console: consoleAPI EstimatedPayout flacky test fixed
In rare cases time frame between creating time.Now() variable and calling
service method that receives it (after API method call) was big enough to distort
current month expectations and make test fail with last digit difference in 1

Change-Id: Ib811492d62f6598a5c40a09de6a87bffeaa0a78e
2022-08-01 15:13:15 +00:00
Egon Elbre
d2f4ea1f24 satellite/payments/stripecoinpayments: fix mock data-race
`len(m.attached)` was used without locking.

Also, use gofumpt to make whitespace usage more consistent.

Change-Id: Ifa9deedc8451f0c54e84d6ac3c2bdc1807688989
2022-08-01 14:30:39 +00:00
Moby von Briesen
78fa0c11a3 satellite/analytics: Add user agent to TrackCreateUser
Send UserAgent (as "signup_partner") to Segment and Hubspot.

Change-Id: I3313eb902e87b429e7fa8e18afd397a63a3ed07d
2022-07-29 17:27:24 -04:00
Cameron
b4ea1bac42 satellite/console/consoleweb: send email when account already exists
When a someone tries to create an account with an email that is already
associated with a verified account, send them an email with options to
sign in, create an account on another satellite, or reset password.

Change-Id: I844144d88b7356bd7064c4840c9441347a5368b0
2022-07-28 15:29:16 +00:00
wilfredasomani
0eb2cc4511 web/satellite: update table component for project member page
This changes the project member page's table to match the new table design,
And updates the common table component fit this page's use case.

For: https://github.com/storj/storj/issues/4993

Change-Id: Ic7e2c4ceade9bbc98793760927a9243914f454d0
2022-07-28 12:57:05 +00:00
Yaroslav Vorobiov
2ba06aca87 satellitedb: storjscandb return payment with UTC time
Return UTC cached payment timestamp on converting from dbx
payment.

Change-Id: I693802d5ac3b786f9973e8299a95215a2e975982
2022-07-27 20:05:37 +00:00
cl-mitch
24b6a801c2
web/satellite: Add Token card to new Payment Methods flow (#4975)
Added the Add Token and Token Balance cards to the new Payment Methods flow.

Link to associated issue: https://github.com/storj/storj/issues/4632
2022-07-27 14:38:15 -05:00
Clement Sam
d1d15cbf4f satellite: return interval_end_time in DailyStorageUsage endpoint
Updates https://github.com/storj/storj/issues/4178

Change-Id: If0c6f66c58a258300bbf563eef56caba4f1dc626
2022-07-27 18:24:27 +00:00
Clement Sam
f2f046af7a satellite/satellitedb: add interval_end_time column to accounting_rollups table
Adding an interval_end_time column to the accounting_rollups table
to keep the last interval_end_time for each daily storage tallies.

Updates https://github.com/storj/storj/issues/4178

Change-Id: If7a8210c5e9fe2fc9df84b137a8b6e3db2471c58
2022-07-27 17:53:27 +00:00
wilfredasomani
477f28fdab web/satellite: update upgrade account modal with support info
Coinpayments is not working properly; This change replaces the storj deposit form
on the upgrade account modal with a message to contact support.

Related to 1bd609c00

Change-Id: I02eeb6dd721320099a3c523c538dac597c4e8914
2022-07-27 17:05:15 +00:00
Jeff Wendling
f343b856dd go.mod: bump common/private/monkit/monkit-jaeger
this includes all of the fixes identified for end to end
tracing to finally work.

Change-Id: Ia433a2cf9130dc43b6810379af32f5c10f9d3888
2022-07-27 16:17:00 +00:00
Qweder93
2b24632610 {satellite/metainfo, satellite/accounting}: remove ValidateSegmentLimit config value and check
removed segment limit validation and checks in metainfo endpoint and accounting/projectusage
since feature is live and has always has segment limitation now

Resolves: https://github.com/storj/storj/issues/4470

Change-Id: I8cf87cbbc40ac61262f9f05e52573d3ae6410611
2022-07-27 15:36:53 +00:00
Yaroslav Vorobiov
7194973b56 satellite/payments/storjscan: add usd value to cached payments
Add USD value to storjscan cached payment struct to store and
retrieve payments from payments DB.

Change-Id: Ic0b2b4a198e4ea8230bc9a01865b429343f52767
2022-07-27 14:55:45 +00:00
Jeff Wendling
75f19cb100 cmd/uplink: run the collector goroutine
without this goroutine, spans will not be collected or sent
from the uplink.

Change-Id: Ibbf2ab79dee7447a1d8ab62005e102e670114bf2
2022-07-27 14:38:30 +00:00
Michał Niewrzał
7390f389cd satellite/metabase: non recursive listing optimization
With pointerdb listing objects operation was optimized to skip
objects from prefixes for non recursive listing. This change it
adopting this optimiaztion from old code.

Main change is to drop current page results if we detect a prefix
that needs to be skipped and jump with next listing query after this
prefix by setting cursor to "some-prefix(byte('/')+1) which is
effectively "some-prefix0".

Benchmark:
name                                                 old time/op  new time/op  delta
NonRecursiveListing/Postgres/listing_no_prefix-8      960µs ±11%   257µs ±12%  -73.19%  (p=0.008 n=5+5)
NonRecursiveListing/Postgres/listing_with_prefix-8    945µs ±11%   671µs ±12%  -28.97%  (p=0.008 n=5+5)
NonRecursiveListing/Cockroach/listing_no_prefix-8    4.31ms ± 8%  1.19ms ± 7%  -72.44%  (p=0.008 n=5+5)
NonRecursiveListing/Cockroach/listing_with_prefix-8  4.97ms ± 8%  3.35ms ±15%  -32.67%  (p=0.008 n=5+5)

Fixes https://github.com/storj/team-metainfo/issues/115

Change-Id: Iafdf3600d058abbaf441f792d32a7fc17cc08696
2022-07-27 11:26:35 +00:00
wilfredasomani
1bd609c00b web/satellite: update storj deposit form with support info
Coinpayments is not working properly; This change replaces the storj deposit form
with a message to contact support.

see: https://github.com/storj/storj-private/issues/43

Change-Id: If3792d05282c102c1d5ca0664bd8c70ad103f61c
2022-07-26 18:26:49 +00:00
JT Olio
452c243d57 cmd/uplink: support sending traces
Change-Id: I10ef49a24a5b6821b17eb5d206aacfe7e0f54e9e
2022-07-26 11:39:30 -04:00
Jeff Wendling
567986e388 cmd/uplink: minimal tracing support
this allows one to specify a trace id and cause any remote
spans to be sent up to wherever. it doesn't collect any
local traces.

Change-Id: Ia87e294bb276d966f9f3dbfbaf6e7916b1ec7af9
2022-07-26 11:39:30 -04:00
Erik van Velzen
b5fc04af89 satellite/metainfo: usage limits for copy
Previously there was no realtime administration of the storage usage
during copies. Now there is.

Closes https://github.com/storj/storj/issues/4719

Change-Id: I0d536bf551d16208116c3aceac89ed590ec473bf
2022-07-25 12:35:27 +02:00
cl-mitch
6df867bb3d
web/satellite: Added credit cards to new payment methods flow (#4978)
* tabs and routing implemented
* added new method card
* token payment card complete - need to discuss responsiveness on this
* began table implementation
* latest stripe integration changes

Co-authored-by: hovex023 <97616907+hovex023@users.noreply.github.com>
Co-authored-by: Andrew Hove <andrew.hove@concordusa.com>
Co-authored-by: Maximillian von Briesen <mobyvb@gmail.com>
2022-07-23 07:31:09 -05:00