Commit Graph

7913 Commits

Author SHA1 Message Date
Márton Elek
8d64330dfe Makefile: bump-dependencies should update storjscan, too
Change-Id: Ibde26535567df38a19e01b1a5a3819263c31d0f8
2023-03-27 15:56:05 +00:00
Vitalii
736f97b1e9 web/satellite: migrate AG ConfirmDeletePopup to use SFC composition api
Migrated component to use SFC composition api.

Change-Id: Ic4a5dbf733b4a831f5c5508ed17a0a5c746e3cae
2023-03-27 15:07:19 +00:00
Vitalii
3deddb548d web/satellite: migrate AccessGrants components to use SFC composition api
Migrated component to use SFC composition api.

Change-Id: I5cb3563626f480c05ad9ab8aabf1c02eeebc5767
2023-03-27 15:06:09 +00:00
Márton Elek
5e6d1cc00a
go.mod: bump storj.io/{common,private,uplink}
Change-Id: I7af875b6ac73af3a9d77928f69a05b80a5321696
2023-03-27 16:13:52 +02:00
Jeremy Wharton
ef5ebc9f64 satellite/satellitedb: fix panic when calculating project usage
Previously, our code produced a Go panic when attempting to index an
empty slice of bucket user agent entries. A length check has been
added to ensure that the slice is not accessed beyond its bounds.

Change-Id: I904ca275b934be1b05393a45d99ff415dcafc926
2023-03-27 07:44:50 +00:00
Wilfred Asomani
41bfbbe772 satellite/console: add endpoint to get user settings
This change adds an endpoint that gets a user's settings. It will
create a new settings entry if no settings exists. There's also a new
endpoint to change a user's onboarding status.

Issue: https://github.com/storj/storj/issues/5661

Change-Id: I9941bb9d61994af46244003f3ef4fcfe7d36918e
2023-03-24 22:19:08 +00:00
Jeremy Wharton
be567021d9 web/satellite: fetch frontend config from the satellite
This change updates the satellite frontend to retrieve its
configuration from the satellite.

References #5494

Change-Id: I1eb961f8a32f7d88173006117d41eddb57624692
2023-03-24 02:10:12 -05:00
Vitalii
fd28a665c7 web/satellite: introduce access grants pinia module
Added new access grants pinia module.

Change-Id: Icb04d6e77ecc19797e250279cbe07e333596d816
2023-03-24 00:09:57 +00:00
Wilfred Asomani
7c68b51c82 satellite/{console,db}: add onboardings columns
This change adds onboarding_start, onboarding_end and onboarding_step
columns to the user_settings table. the first two are used to determine
if a user should go through onboarding, the last will be used to as the
step a user got to before exiting onboarding without finishing.

Issue: https://github.com/storj/storj/issues/5660

Change-Id: I8070c880d0d2fc22086f24087c962f57c695cc50
2023-03-23 17:50:31 +00:00
Vitalii
2922eeed07 web/satellite: migrated components to use pinia ab testing module instead of Vuex
Migrated components to use new pinia ab testing module instead of old Vuex module.

Change-Id: Ib71ff1ad8d08829d939dee2ae87f4349bd4c33cf
2023-03-23 14:51:56 +02:00
Vitalii
a588f96b90 web/satellite: migrated UpgradeNotification component to use SFC composition api
Migrated component to use SFC composition api.

Change-Id: Ic2f11d1c77eaa5e66ed80c1c2fd6acbfe25f9052
2023-03-23 10:22:18 +00:00
Vitalii
6a99e3df31 web/satellite: migrated MobileNavigation component to use SFC composition api
Migrated component to use SFC composition api.
Fixed some router code usage in DashboardArea.
Fixed billing Detailed history router usage.
Fixed VBanner console error.

Change-Id: I372f870185a00334895c62d504e2aed44f8f5b9d
2023-03-22 21:23:01 +00:00
Jeremy Wharton
01f0e602b4 satellite/payments/stripecoinpayments: retry API calls with exp backoff
A Stripe backend implementation has been added that uses an exponential
backoff strategy to retry failed API calls. This behavior can be
configured in the satellite config.

References #5156

Change-Id: I16ff21a39775ea331c442457f976be0c95a7b695
2023-03-22 07:20:27 -05:00
Vitalii
2dee4c9afc web/satellite: migrated navigation AccountArea to use SFC composition api
Migrated component to use SFC composition api.
Deleted unused PaidTierBar component.

Change-Id: Ie7938fbcd1b65ef0da9e9aefec879d605892253b
2023-03-22 11:54:01 +02:00
Vitalii
9feee869f0 web/satellite: add ab testing pinia module
Added new ab testing pinia module.

This is a part of migration from Vuex to pinia.

Change-Id: I5e6c325d979b0641ed745de4ca17039b812dd842
2023-03-22 11:49:24 +02:00
Kaloyan Raev
98b82486bd satellite/overlay: update country code on every node check-in
We have a specific issue that a user uploaded a file to a bucket
geo-fenced to EU and one of the pieces appeared to be on a node in the
US. The country code of this node is set to SE (Sweden) in the satellite
DB. It turns out that some time ago MaxMind changed the country code of
this node's IP from Sweden to US, but this change hasn't been reflected
in the satellite's database.

So far the satellite updates the country code of a node only if its IP
changes. It was assumed that if the IP does not change, its country code
shouldn't change too. This turned to be a wrong assumption.

With this change, the satellite will look up the MaxMindDB on every
check-in to see if the country code of the node's IP has changed.

Change-Id: Icdf659b09be9fc6ad14601902032b35ba5ea78c4
2023-03-22 08:38:51 +00:00
JT Olio
58f465c8d8 storagenode/piecestore: improve cancelation detection logic
if a drpc.ClosedError was returned, it would always take the
first (failure) branch, despite the second branch's existence.

Change-Id: Ife3b27869c4e9d37ca2914e2d1d1a2c60d326309
2023-03-21 17:50:49 +00:00
JT Olio
0b4b04900a private/server: debounce noise and tls connections
to support TCP_FAST_OPEN, we're considering just using
two TCP connections in parallel per request, one with
and one without. this allows us to safely fire both
concurrently without stressing out the node too much.

see https://review.dev.storj.io/c/storj/storj/+/9933

Change-Id: I9aa8a0252350db5ace04ee125bfe469203e980ec
2023-03-21 16:51:31 +00:00
JT Olio
0208eb7693 docs/blueprints: update tcp_fastopen
Change-Id: I8d9121c38853103b07514d8e55c3bf06049283fd
2023-03-21 14:06:16 +00:00
Vitalii
fb12717256 web/satellite: fix passphrase switching outside object browser
There is a bug when switching passphrase outside object browser.
We don't generate edge credentials when switching passphrase so navigating to object browser shows old objects (for old passphrase).
By this fix we clear edge credentials every time user switches passphrase.

Change-Id: Ia09d559f90ae7f02ae5c778844beb7ec268356e7
2023-03-20 17:05:26 +00:00
JT Olio
2a63225b98 satellite/{contact,satellitedb}: preserve node message debounce support
Change-Id: I453ad35fda4e61d068db0c476dd86b50d7f2d705
2023-03-20 16:13:06 +00:00
Kaloyan Raev
2c28a0e599 cmd/uplink: ability to opt-out from object key encryption
Option added to `uplink access setup` and `uplink access create`
commands to disable object key encryption.

Related to https://github.com/storj/storj/issues/5678

Change-Id: I4789a94143742ff4b232fd60decc029ad2883c2a
2023-03-20 13:56:15 +00:00
Egon Elbre
7fa1a633aa ci: move go-licenses check to premerge
License check is unlikely to fail, hence it's better to do it as part of
premerge rather than verify.

Also, fix indentation in Jenkinsfile.

Change-Id: Ifb37b92a88eb743a28847d84ed63c0e2e784dbe2
2023-03-20 09:20:28 +00:00
Wilfred Asomani
bf45e53169 satellite/{web,payments}: display correct stripe balance
This correctly displays the stripe balance in dollars, which was
previously displayed in cents.

Change-Id: Ifb14a63a90d3701201c616780cee466525c8be18
2023-03-17 11:54:08 -05:00
Jeremy Wharton
e0c3f66040 web/satellite: display correct prices in account upgrade modal
The account upgrade modal has been updated to display prices according
to the the user's price model. Previously, the modal displayed only the
default prices which were incorrect for users with price overrides.

Resolves storj/storj-private#187

Change-Id: I58206cc8ea7e7742a37f759a84dbb24ca40dd8eb
2023-03-17 10:15:28 -05:00
Jeremy Wharton
6942da3ddb web/satellite: migrate account upgrade modal to composition API
This change updates the account upgrade modal to use the composition
API.

Change-Id: If835da7bb627f07aec88bff59127b3ba459c9a0f
2023-03-17 10:14:23 -05:00
Cameron
355f800916 web/satellite: create default project during onboarding
When a user logs in, we check if they have zero projects at the
dashboard. If so, the default project is created and they are sent to
the onboarding flow. However, this allows a user to skip onboarding if
they close out and log back in, since the dashboard is loaded first.
This change moves the default project creation to the beginning of the
overview step of onboarding to make sure any important preliminary
onboarding steps are not skippable.

Change-Id: I4fd1efc6ccd26b972fe57425efe268f8ba135c26
2023-03-16 18:19:32 -04:00
Moby von Briesen
0980a2f9ce satellitedb: Add index to users table
Add a combined index on normalized_email,status to improve performance of
common "get user" query used for the satellite UI.

Change-Id: I24a20d7826e0a68a68c2f95b5847eb819921e7c0
2023-03-16 09:08:49 -04:00
Vitalii
132a688bec web/satellite: rework session timeout reset with ongoing upload
Using eventbus was a terrible approach because if session would expire in less than 3 mins then we would do an explicit API request to reset session for each upload progress ping.
With this change we simply check if there is at least one ongoing upload and refresh session with one single API request and we don't show inactivity modal.

This is a possible fix for this issue (at least I can't reproduce it):
https://github.com/storj/storj/issues/5618

How to test:
setup storj-up with InactivityTimerDuration config value set to 120 seconds.
go to object browser and start upload of some heavy file, for example 1GB.
check if inactivity timer would be shown in 2 mins, additionaly you may check DevTools network tab.
in my case upload is successfully done in about 3-4 mins without any problems.

Change-Id: I09142a7affac08db1d02992ca2d2f40c6267324f
2023-03-15 16:00:53 +00:00
Vitalii
7942f3774f web/satellite: do not show server side encryption banner on relogin
Do not reset showing server side encryption banner for buckets page on relogin.

Issue:
https://github.com/storj/storj/issues/5555

Change-Id: I30d9d8e8370aa1fd1f60069720ecd07d8450c8ee
2023-03-15 15:23:59 +00:00
Vitalii
37d7c0efbf web/satellite: migrate VChart component to use SFC composition API
Upgraded chart.js dependency to V4.
Removed vue-chart and types/chart.js dependencies.
Refactored VChart component to rely only on chart.js dep without any additional wrappers.
Refactored VChart component to use SFC composition API.

Change-Id: Ic5e0131bff413f3205d4449db930644d113fe36d
2023-03-15 14:31:52 +00:00
Vitalii
6c49cc883e satellite/console: enable new access grant flow by default
Flip feature flag to enable new acces grant flow by default

Change-Id: I66a963ecd368d1b9889b7ee2d88ee9806366efac
2023-03-15 15:36:21 +02:00
Wilfred Asomani
61af5b3191 web/satellite: move register page to composition api
This change updates RegisterArea.vue to use the composition API.

Issue: https://github.com/storj/storj/issues/5059

Change-Id: Ieea65731cd76a9e98282f23aed90e625c6d09387
2023-03-14 21:26:27 +00:00
Jeremy Wharton
b95ef36541 satellite/payments/stripecoinpayments: add ctx to Stripe method calls
This commit ensures that all invocations of Stripe library methods
include a context. This allows us to control the timeout and
cancellation of the underlying HTTP requests made by the Stripe
library.

References #5156

Change-Id: I8ddb317f3f2cbb06cfab869fbebdaf2ad78b7999
2023-03-14 17:52:17 +00:00
Clement Sam
c3d5965ef2 storagenode/monitor: add timeout to storage dir verification
Resolves https://github.com/storj/storj/issues/4567

Change-Id: Ia071c476bcd1f5c99a9874801c94db86d1e105c6
2023-03-14 13:43:14 +00:00
Michal Niewrzal
e0577eedf5 satellite/metainfo: add missing metrics for metainfo requests
Change-Id: Icba9aa1ee552c7c180aa781de5ef9b87c4ec2f2d
2023-03-14 13:07:58 +00:00
Michal Niewrzal
ee720040c9 satellite/metainfo: use project limit cache with limiter
Metainfo needs to know rate and burst limit to be able to limit users
requests. We made cache for per project limiter but to make single
instance we need to know about limits. So far we were doing direct DB
call to get rate/burst limit for project but it's generating lots of
DB requests and can be easily cached as we even have project limit cache.

This change extends project limit cache with rate/burst limit and starts
using this change while creating project limiter instance for metainfo.

Because data size kept in project limit cache is quite small this change
also bumps a bit default capacity of the cache.

Fixes https://github.com/storj/storj/issues/5663

Change-Id: Icb42ec1632bfa0c9f74857b559083dcbd054d071
2023-03-14 08:11:11 +00:00
Wilfred Asomani
59ebb0ef27 web/satellite: move login to composition api
This change updates LoginArea.vue to use the composition API.

Issue: https://github.com/storj/storj/issues/5060

Change-Id: I34d748da1224805ed179bd2cf37e61d496d0811c
2023-03-14 00:34:07 +00:00
Michal Niewrzal
e181f4b90e satellite/accounting: test that project limit cache is really used
We just fixed case were project limit cache was not used properly. This
is test case to cover that fix.

Change-Id: Iee467f0a46836860a14ab6238a9842ffbf54ed4c
2023-03-13 17:40:55 +00:00
Michal Niewrzal
38c7671acb satellite: bump capacity for API keys cache
We have lots of direct DB requests to get API keys. It should be handled
by cache but default value is very low at the moment.

Fixes https://github.com/storj/storj/issues/5665

Change-Id: I214ebebd6e397cacff80b2f36dc4a2eea388f93d
2023-03-13 16:32:50 +00:00
paul cannon
97e20bc579 scripts/tests: fix rollingupgrade test even more
This might be pretty awful, but at least it is a complete and non-flaky
solution.

**Only when using the rollingupgrade test** (which implies a throwaway
satellite and also a PostgreSQL backend), create a trigger on the nodes
table which forces last_net to be equal to last_ip_port always.

Change-Id: I8448cf131e46576d96a414d06780270c7b2b1892
2023-03-13 15:49:07 +00:00
Michal Niewrzal
06b51258be satellite/metabase: use table stats if are up to date
Currently, to get number of entries in segments table we are doing
heavy SELECT count(*) operation. For biggest satellite it's taking
25min now. We are using this method to get stat before and after
segments loop so it adds almost 1h to overall loop time.

With current version of crdb we are using this additional code won't be
used because global configuration for stats refresh rate is inaccurate
for such large table like `segments`. Soon we should be able to upgrade
crdb and be able to adjust refresh rate per table and configure it to
satisfy defined threshold.

https://github.com/storj/storj/issues/5544

Change-Id: I05cfd9154f08894d2bc56bf716b436d1b03b87f1
2023-03-13 14:54:13 +00:00
Andrew Harding
5c744d7ed4 storagenode/pieces: close reader after use
Change-Id: Icd9df821edb668c5521732396b7d6be3b8e75c7a
2023-03-13 14:06:10 +00:00
Márton Elek
0b66d22be4 cmd/uplink: support --trace-tags for custom tags
We do regular testing with executing uplink. But sometimes the recorded execution time showed spikes.

Would be nice to know the reason of the spikes (just internet blip, or something what we should be worried about).

We can collect distributed traces, but it's not easy to find the right trace in Jaeger.

 * We can provide a random trace-id, but it should be persisted / processed
 * We can also save standard output and use `--trace-verbose` which prints out the used trace id, but it's also complicated to collect all of them in a DB

Would be nice to attach additional metadata to traces to make sure that we can filter all traces of one specific kind of test.

This patch provides this feature:

  * It always adds hostname to the trace (if you opt-in to distributed tracing, which is turned off by default)
  * Additional tags can be defined with CLI flag

Tags can be used to find the right trace in Jaeger (or in Elastic search backend of Jaeger).

Change-Id: I08f10023bbebd783f812cfca95ac6237360ac2b0
2023-03-13 12:03:39 +00:00
Jeremy Wharton
c24341bcab web/satellite: update Coupons area in Billing page
The Coupons area of the Billing page has been updated to match our
designs.

Resolves storj/storj-private#172

Change-Id: I0f3d7f3818f47a1bbc6a2dba930ab429f83f92c2
2023-03-13 08:09:06 +00:00
Jeremy Wharton
73ff35f160 satellite/payments: add partnered field to coupon struct
A field has been added to the coupon struct indicating whether it is
associated with a partner's pricing package. This is required to
alter the appearance of partner coupons in the satellite frontend.

References storj/storj-private#172

Change-Id: Ie48ae3902aaa108abf9a399242a0cd98cb53d1c3
2023-03-13 02:54:26 -05:00
Wilfred Asomani
0177ef79da satellite/{console,analytics,accountfreeze}: send unfreeze event
This change sends an event to segment for when a user is unfrozen.
It also moves freeze and warning event triggers from the autofreeze
chore to the account freeze service.

Change-Id: I5c0522b921b7baf52d6db5eb7ef841c08644a461
2023-03-10 23:21:20 +00:00
Jeremy Wharton
c4f8695e8a web/satellite: generate TypeScript for frontend config
A package has been written to generate a TypeScript file containing
classes corresponding to the frontend config. These classes will be
used across the frontend once we no longer reference meta tag values
for configuration.

References #5494

Change-Id: If425035892773167ac6d9fbfae8140cab79fbb70
2023-03-10 22:33:24 +00:00
Jeremy Wharton
4d823e8166 private/apigen: isolate TypeScript class generation
The code responsible for generating TypeScript classes has been
separated from the rest of the TypeScript generation code so that other
packages may take advantage of this functionality.

References #5494

Change-Id: I97eabd430bd6a5f748eafaf8b1d783977e75e660
2023-03-10 21:52:30 +00:00
Cameron
4e94e6188c satellite/admin: restrict api access based on user groups
This change reworks the allowedAuthorization function to check what
groups the user is a part of to determine if authorization should be
granted. By wrapping each handler with withAuth, we can specify the
allowed groups for each api method individually.

github issue: https://github.com/storj/storj/issues/5565

Change-Id: I1804dda04d5b16d19e93bd7199fb3fc89fca1294
2023-03-10 16:16:04 -05:00