Commit Graph

8119 Commits

Author SHA1 Message Date
Lizzy Thomson
f87772a0d3 satellite/console: bug fix for using a API key to retrieve a session
Change-Id: I0202f75fb78f42af9e7cb00bb3ac73839340c977
2023-05-16 14:06:32 -06:00
paul cannon
75d10fe4fa satellite/overlay: use UploadSelectionCache for GetNodesNetworkInOrder
The query for GetNodesNetworkInOrder is causing far too much load on the
database. Since it is not critical that the repair checker have
perfectly up-to-date node network information, we can use a cache
instead.

Change-Id: I07ad45bfdeb46529da093941a06c2da8a00ce878
2023-05-16 17:32:09 +00:00
Wilfred Asomani
e9fc4e9d0f web/satellite: fix billing page nav on all dashboard
This change fixes an issue where clicking "Billing" on all projects
dashboard won't navigate to the billing page.
This also makes sure an error notification for projectID is not shown
after the user is back on the all projects dashboard from team page.

related: https://github.com/storj/storj/issues/5875

Change-Id: Ie6517c9006d1a780f307f6282f83c749d369adaf
2023-05-16 15:14:18 +00:00
Márton Elek
f1b9f6b672 satellite/satellitedb: paginating on ordered invoice
87d0789691 replaces offset usage with cursors.

But to continue the interation from a specific cursor, we need to iterate over ordered records.

(at least this is what I understood based on the failing tests)

87d0789691

Change-Id: Ic4da3a7c5f03386dd4c373c05102f05871900a3a
2023-05-16 14:05:34 +00:00
Michal Niewrzal
4bdbb25d83 satellite/metabase/rangedloop: move Segment definition
We will remove segments loop soon so we need first to move
Segment definition to rangedloop package.

https://github.com/storj/storj/issues/5237

Change-Id: Ibe6aad316ffb7073cc4de166f1f17b87aac07363
2023-05-16 12:37:17 +00:00
Lizzy Thomson
7e69b22dd4 satellite/console: retrieve a web UI session using a API key
retrieve a web UI session using a API key and add new test

Issue https://github.com/storj/storj/issues/5688

Change-Id: I0ad9c783f0573b87b212b8fb065cdbf7074b782c
2023-05-15 10:24:58 -06:00
Wilfred Asomani
81b2b067e6 satellite/satellitedb/dbx/gen: update to work with updated dbx
This change updates the replacer in satellite/satellitedb/dbx/gen/main.go
to work with an updated dbx.

Change-Id: I08e89d6d27e6f1d435416105fe5f622009add7ad
2023-05-15 14:53:06 +00:00
JT Olio
e90612c285 docs/blueprint: partial rollout automation
Change-Id: I355d7071da7e501948665ee153ed641722a3d264
2023-05-15 13:49:47 +00:00
Michal Niewrzal
2592aaef9c satellite/gc/bloomfilter: remove segments loop parts
We are switching completely to ranged loop.

https://github.com/storj/storj/issues/5368

Change-Id: I1a22ac4b242998e287b2b7d8167b64e850b61a0f
2023-05-15 11:46:26 +00:00
Wilfred Asomani
5fede0ce95 web/satellite: fix no project id provided error
This change fixes an error when all projects dashboard is enabled where
there will be no project selected and an error "projectid not provided"
will be thrown.
It navigates back to the all projects dashoard so the user can select
a project.

Issue: https://github.com/storj/storj/issues/5875

Change-Id: I144c4bbd3782dcdf40b2ed2be445c7f100c36b32
2023-05-15 10:23:32 +00:00
Wilfred Asomani
331a3348a0 web/satellite: mod project options menu for all projects dashboard
The project options has been modified to account for whether the all
projects dashboard is enabled or not. The "Manage projects" item is
hidden, and an "All projects" item is added if it is enabled.

Issue: https://github.com/storj/storj/issues/5866

Change-Id: Ic2e8337d2e5577d1c27231287ab3284145fa5b9b
2023-05-12 20:10:33 +00:00
Wilfred Asomani
b11ba50b94 web/satellite: hide sidebar on onboarding
The sidebar previously was showing on the onboarding pages when all
projects dashboard is enabled. This change hides it and also hides the
"session timeout" info banner which was also showing on onboarding.

Issue: https://github.com/storj/storj/issues/5874

Change-Id: Ie418a6f286262b59dce41aba98a5c9d0015cb984
2023-05-12 18:57:02 +00:00
Vitalii
277a4c6aa3 web/satellite: updated project dashboard data cards
Updated cards to show access grants count, buckets count, team size and billing status for current project

Issue:
https://github.com/storj/storj/issues/5865

Change-Id: I7e8d3aa3e05548d593576c3a0633e9eb51f38cff
2023-05-12 14:43:09 +03:00
Vitalii
083b3d6fc1 web/satellite: added modal to update single project limit
Added new modal where user can update storage or bandwidth limit (if Pro tier).

Issue:
https://github.com/storj/storj/issues/5827

Change-Id: Ic3ae6b7a73d055b4eac93d4e2faad7bf26cb9d73
2023-05-12 11:38:39 +00:00
Vitalii
b91f72f08a web/satellite: added limit cards on project dashboard
Added storage, bandwidth, segment, free tier and coupon cards to project dashboard.

Issue:
https://github.com/storj/storj/issues/5827

Change-Id: Iaf974273a34c8ed3faf1974876fa685074d5ae61
2023-05-12 14:36:53 +03:00
paul cannon
c2710cc78d satellite/audit: improve error handling
* Don't use rpcstatus.Unknown as an indicator of dial failure; instead,
  GetShare now indicates with a per-share field where a failure happened
  (DialFailure, RequestFailure, NoFailure). Use that information in
  Verify() to determine how to treat the source node.

* Add a test that replaces a storage node with a black hole, so that
  connections there will always time out. Make sure we handle that case
  correctly.

Refs: https://github.com/storj/storj/issues/5632
Change-Id: I513a53520fb48b7187d4c4d7e14e01e2cfc0a721
2023-05-11 22:55:26 +00:00
igor gaidaienko
bc30deee11 satellite/repair: update repair test to use blake3 algo
Update repair test to also use blake3 hashing algorithm
https://github.com/storj/storj/issues/5649

Change-Id: Id8299576f8be4cfd84ddf9a6b852e653628ada72
2023-05-11 08:45:48 +00:00
Cameron
52ff7a66a0 web/satellite: use color palette instead of hex colors for button
Change-Id: I1da9c1927c2768e86f3887477250ae833e29439e
2023-05-10 17:54:26 +00:00
Michal Niewrzal
87d0789691 satellite/payments/stripe: avoid full table scan while listing records
Stripe invoice project records while listing are causing full table scan
because of OFFSET caluse. This change is refactoring query to list using
cursor.

Change-Id: I6b73b9b2815173d7ef02cf615408778476eb3b7b
2023-05-09 14:16:55 +00:00
Clement Sam
c64f3f3132 {storagenode/console,web/storagenode}: fetch pricing model from storagenode API
Instead of the hardcoded payout rates that is assumed for all satellites,
this change adds a new endpoint for fetching the pricing model for
each satellite.
The pricing model is then displayed on the Info & Estimation table
on the dashboard

Updates https://github.com/storj/storj-private/issues/245

Change-Id: Iac7669e3e6eb690bbaad6e64bbbe42dfd775f078
2023-05-09 13:25:44 +00:00
Clement Sam
d80d674863 web/storagenode: get current month earnings from estimated payout
The current month estimated earnings on the main dashboard
was calculated on the frontend using hardcoded payout rates for
all satellites.
Instead, we could just use the calculate the current month earnings
from the estimated payout info.

Updates https://github.com/storj/storj-private/issues/245

Change-Id: Ie8ec3e4f2b86455c68f664690ef28bb352107bca
2023-05-09 12:54:20 +00:00
Clement Sam
1003d8213c storagenode/blobstore/filestore: add debug log to WalkNamespace when dir does not exist
This is particularly useful for monitoring the lazyfilewalker to
make sure it is not checking the wrong directory.

Updates https://github.com/storj/storj/issues/5349

Change-Id: I7e5fcfd4545ec4157d33a9225cd1bce607ccd154
2023-05-09 10:27:13 +00:00
Márton Elek
291b541960 earthfile: remove storage folder
Content of storage folder is moved to the storagenode with c1f1aacff (as it's used only from there).
We don't need to include it manually for storagenode builds.

This Earthfile is used for nightly deployment.

Change-Id: I2d255804ce5d87ab5a9b713c2d5ae0f3b3ccc9ca
2023-05-09 09:46:43 +00:00
Michal Niewrzal
fa16d947c9 satellite/satellitedb: add index on owner_id column for projects table
We have method which is getting projects owned by specific user but it's
causing full table scan because we don't have index on owner_id column.

Change-Id: Icb71c9ac5b73104a52241ed8ba126c995c10811f
2023-05-09 08:57:32 +00:00
Wilfred Asomani
5145be1eeb satellite/console: proper constraint error check for duplicate project members
The string check previously used to check for constraint errors is now
replaced with dbx.IsConstraintError check.

Change-Id: I553ccd69e3c02b6b54441bd9f929b85a155eaf00
2023-05-08 22:23:00 +00:00
Clement Sam
018b6eeeaf storagenode: add tests for lazyfilewalker
Updates https://github.com/storj/storj/issues/5349

Change-Id: I9544c14ba2acacd5b304f151ab29c70ff61adc5b
2023-05-08 21:50:40 +00:00
dlamarmorgan
cf5d2d792c satellite/payments/stripe/service.go: fix payment for multiple invoices
Fix an error that can occur when processing multiple invoices for the same user in a single invoice cycle when the user is paying with Storj tokens.

Change-Id: I54af8c7dde1965d994f687fdfc4e4b5ef4deeb2d
2023-05-08 21:17:14 +00:00
Egon Elbre
8beb78ec3f {satellite,multinode}/console: fix WriteHeader
w.Header().Set needs to be called before WriteHeader,
because WriteHeader sends all the headers and calls to
Set won't have any effect afterwards.

Change-Id: Ia6b1c5e2cd54201a6c3980d63de04a0095b2db9a
2023-05-08 18:18:30 +00:00
Jeremy Wharton
9c753163c2 satellite/console/dbcleanup: make chore clean up project invites
The console DB cleanup chore has been extended to remove old project
member invitation records.

Resolves #5816

Change-Id: Id0a748e40f5acf03b9b903265c653b072846ba19
2023-05-08 16:36:16 +00:00
Clement Sam
cf7ce81d09 cmd/storagenode: refactor lazyfilewalker commands to satisfy the execwrapper.Command interface
Follow-up change for https://review.dev.storj.io/c/storj/storj/+/10335

Updates https://github.com/storj/storj/issues/5349

Change-Id: Iadf55bae84ebc0803a0766830e596c396dfb332b
2023-05-08 15:09:53 +00:00
Márton Elek
56dbe7738d cmd/storagenode: use the right subcommand for restarting windows services
Commit 3cf89633e9 is changed how the cobra subcommands are created for storagenode (with prefering local variables instead of package level variables).

However, there is a bug which makes it impossible to restart Storagenode services on Windows: the refactored code creates the rootCmd/runCmd twice: therefore the ctx of the running process is not exactly the same as the ctx which supposed to be stopped / cancelled.

This patch fixes this problem with re-using exising, initialized command instead of creating a new one for cancellation.

Fixes: https://github.com/storj/storj/issues/5845

Change-Id: Ib8a4b80d4574e448f65c8558e927c0908c9c5eed
2023-05-08 14:09:37 +00:00
Clement Sam
5977357a60 web/storagenode: hide payout rates for historical payout data
This is a temporal fix to buy us some time while we fix the issues.

We may revert later or maybe override with a later commit.

Updates https://github.com/storj/storj-private/issues/245

Change-Id: I27b2b6811d24d5948161553bf889fe45af013925
2023-05-08 12:53:37 +00:00
Michal Niewrzal
36e046375c satellite/repair/checker: remove segments loop parts
We are switching completely to ranged loop.

https://github.com/storj/storj/issues/5368

Change-Id: I8583549973cd36aa0e0c482c20d7a75cb7568ab3
2023-05-08 12:19:13 +00:00
Cameron
e8e6dd056a web/satellite: remove project member modal
Create modal for removing project members

issue: https://github.com/storj/storj/issues/5751

Change-Id: I621b8e1de54270faebb7577b8431903c62a6e45c
2023-05-08 11:36:56 +00:00
Wilfred Asomani
fb1a0cc784 web/satellite: redesign and allow enter passphrase skip
This change allows users to skip the enter passphrase modal and opt in
for that choice to be remembered. The enter passphrase modal is also
redesigned.

Issue: https://github.com/storj/storj/issues/5818
Issue: https://github.com/storj/storj/issues/5616

Change-Id: Ie1cf6602682af3e9625656455fa3c096a24b95d3
2023-05-08 10:52:49 +00:00
Clement Sam
291e639ac2 storagenode/pieces/lazyfilewalker: add execwrapper package
The execwrapper package wraps the exec.Cmd and has a Command
interface that mimics the behaviour of the exec.Cmd.
This is useful for testing the lazyfilewalker subprocesses
by stubbing instead of spawning a real subprocess.

Updates https://github.com/storj/storj/issues/5349

Change-Id: I14084139c76a531f2b6d7163f9aa35c3f5e192d7
2023-05-06 02:02:23 +00:00
Clement Sam
ec8bfe6b94 storagenode/pieces: capture logger time Key in zapwrapper
Updates https://github.com/storj/storj/issues/5349

Change-Id: I426f38c0ae0f93d498317e3f66ba4f5724620758
2023-05-06 02:02:23 +00:00
Jeff Wendling
6ac72e42cd all: bump storj.io/drpc
this includes a fix to cancellation when soft cancel
is enabled.

Change-Id: I1b9ab071f623153a2bd715888f97ca20277a76d7
2023-05-06 01:02:50 +00:00
Moby von Briesen
432b12d1c4 satellite/satellitedb: make value_attributions.partner_id nullable
There is still a reference to partner_id in a query, which we cannot
move until dropping the "not null" constraint for it. This change adds
that migration.

Related to https://github.com/storj/storj/issues/5432

Change-Id: I98802a6e1bd59f3d9214de3db6688d9daf664a70
2023-05-05 23:42:50 +00:00
Wilfred Asomani
e3573f44ea web/satellite: update project member invite modal
The project member invite modal has been updated to match the new figma
design.

Issue: https://github.com/storj/storj/issues/5744

Change-Id: I9ed7ad4ede0fb6f7f42f56b2d8386e12a5a5b6c1
2023-05-05 19:46:28 +00:00
Jeremy Wharton
f61230a670 satellite/console/dbcleanup: create console DB cleanup chore
A chore responsible for purging data from the console DB has been
implemented. Currently, it removes old records for unverified user
accounts. We plan to extend this functionality to include expired
project member invitations in the future.

Resolves #5790
References #5816

Change-Id: I1f3ef62fc96c10a42a383804b3b1d2846d7813f7
2023-05-05 19:11:53 +00:00
Cameron
2bb636684e web/satellite: update create bucket modal
issue: https://github.com/storj/storj/issues/5817

Change-Id: Ia86e0bad8de180fb4e6dde3ed194c9ce6f4f12f1
2023-05-05 17:50:58 +00:00
Wilfred Asomani
4ab433ba89 satellite/{web,console}: make duplicate member error readable
This change makes the error thrown when adding an existing member to a
project readable.

Issue: https://github.com/storj/storj/issues/5840

Change-Id: I4269495f9b7b09c77fbf1af1fc605e5c95bd7cbf
2023-05-05 16:46:12 +00:00
Egon Elbre
8b82dba602 storagenode/blobstore/filestore: add tracking of blobs
We've had issues with forgetting to close readers and writers.
Add leak tracking to find those pesky issues.

Change-Id: If6b0ad6e9958318a7e0affee9c6d0a1ece412b6d
2023-05-05 15:40:15 +03:00
Clement Sam
569c639cd5 web/storagenode: hide payout rates when no specific satellite is selected
We should not display payout rates when "All Satellites" option is
selected since some of the satellites may have different payout
rates.

Updates https://github.com/storj/storj-private/issues/245

Change-Id: I66cadf85aa8b57681ffbeac75c4ad97f8b057676
2023-05-05 01:00:41 +00:00
Wilfred Asomani
bf05040dd6 satellite/{console,db}: allow passphrasepromt toggling
This change adds the user's passphrase prompt setting to the
/account/settings endpoints.

Issue: https://github.com/storj/storj/issues/5616

Change-Id: I48d470d49e82096fd090b74da323b279e342546e
2023-05-04 19:26:54 +00:00
Paul Willoughby
d8081169cd satellite/metainfo: increase default MaxEncryptedObjectKeyLength
Allow a longer encrypted key length to reduce 'key length is too big'
errors in gateway-mt.  Gateway is enforcing an unencrypted key length
of 1024 bytes but when encrypted some keys are exceeding the current
limit.

updates https://github.com/storj/gateway-mt/issues/335

Change-Id: I861a2313e558c9f7d39569d21c7a3d429c83575c
2023-05-04 12:37:13 -06:00
Vitalii
a8ba7aa263 web/satellite: small styling updates for access grant flow
Fixed Copy and Download buttons.
Fixed wording styling.
Also fixed VButton's hover state if there is an icon.

Issue:
https://github.com/storj/storj/issues/5774

Change-Id: I9fbac12fe433817f4b5fc1d885f50e6f27e7f6a0
2023-05-04 17:33:06 +00:00
Vitalii
73898e0d86 web/satellite: make graph line more smooth
Made lines to be more smooth and also updates points styling

Issue:
https://github.com/storj/storj/issues/5826

Change-Id: I34229e772ceee5e48330f5662a584cd9dcd8c63d
2023-05-04 15:20:31 +00:00
Egon Elbre
511d74a76d satellite/metainfo: hardcode the prefix
This avoids a potential bug with regards to not returning the prefix
that the uplink expects.

Change-Id: I64683db57c5b22989db91b4f8e08530bf3b64ff1
2023-05-04 17:06:51 +03:00