Commit Graph

8001 Commits

Author SHA1 Message Date
Michal Niewrzal
c48bd81e5f satellite/satellitedb: update SelectAllStorageNodes* to set country code
Methods SelectAllStorageNodesUpload and SelectAllStorageNodesDownload
are not returning full info with overlay.SelectedNode because its
missing CountryCode.

Change-Id: Ie3cb396bf28d7ec4c6ab8927e5bb560236036aa6
2023-05-26 11:02:29 +00:00
Sean Harvey
403f5eff81
satellite/admin: add /api/apikeys/{apikey} GET endpoint
This allows scripted automation to get more details of the
API key such as project ID, and paid tier status.

Updates https://github.com/storj/gateway-mt/issues/321

Change-Id: I8a835752d4fd67382aca804b8c93e63de6c9a846
2023-05-26 17:18:05 +12:00
Jeremy Wharton
c3d72a269e satellite/{accounting,satellitedb}: create tallies for empty buckets
Tallies are now created for buckets with no objects. Previously, the
bucket tally collector skipped empty buckets since it created tallies
only using information from the objects table. Methods that used
bucket tallies when calculating usage costs would return incorrect
results because of this.

Change-Id: I0b37fe7159a11cc02a51562000dad9258555d9f9
2023-05-25 19:48:59 +00:00
Jeff Wendling
ccfe5cae49 cmd/uplink: pass in a maximum concurrent segments value
also change the config creation to be more robust to
changes that add defaults in the future by not fully
reconstructing the config value passed in to the
project.

Change-Id: I673e8b54ce0b951ae735bf4658525c477c26ac5a
2023-05-25 10:53:50 -04:00
Jeff Wendling
a0fbc87b31 cmd/uplink: add upload log file flag
Change-Id: I740eaf431cfe5bb3dcb746e4128dea5efc935257
2023-05-25 14:14:00 +00:00
Márton Elek
d6615cb698
makefile, docker: purge deleted (satellite) inspector tool from build files
the inspector tool of satellite is removed recently. We need to remove it from the build/docker files.

Change-Id: Icd75474ea64fe5f9dd4ed76d8597982108f93536
2023-05-25 11:04:32 +02:00
Lizzy Thomson
6f2d6a97a6 web/satellite: update modals for create projects
Update to use NewCreateProjectModal to match figma designs and update
CreateProjectPromptModal to handle free tier case and paid tier increase
project limits case

Issue https://github.com/storj/storj/issues/5814

Change-Id: I8c5f818d3867d0484b213d1bfc3ce51e904c0afa
2023-05-24 17:47:01 +00:00
Lizzy Thomson
7d745ac578 web/satellite: update wording from decentralized to distributed cloud
Issue https://github.com/storj/storj/issues/5873

Change-Id: I0cb1032a6513289482e56dabefff934a8d722134
2023-05-24 17:12:52 +00:00
Wilfred Asomani
715b9f0299 web/satellite: fix table cells styling
This fixes an issue where the "primary" class will be applied to the
wrong table cell. Also, truncation for long file names is fixed.

Change-Id: I6303f4b7d3a7a064f8dc79fac90b0bc2d5eb5024
2023-05-24 16:24:28 +00:00
Clement Sam
c6f67d4799 storagenode: fix lazyfilewalker failing with SIGPIPE
Lazyfilewalker was failing with SIGPIPE which was quite
misleading. The command was failing because the
the value of the --lower-io-priority flag was assumed
to be an arguement since it was passed as
"--lower-io-priority true" instead "--lower-io-priority=true"

Resolves https://github.com/storj/storj/issues/5900

Change-Id: Icf79fcce76dafee21659d76ee0ce19d8520c8f1d
2023-05-24 15:19:31 +00:00
Moby von Briesen
919278dbf9 satellite/satellitedb: drop partnerID columns
The last code referencing these columns was removed as of satellite
release v1.79, so it is safe to remove them now.

Resolves https://github.com/storj/storj/issues/5432

Change-Id: I2e9d641b2511a61e0b9482ef0f4955a73c290709
2023-05-24 14:38:57 +00:00
Moby von Briesen
714c9f40c1 satellite/console: Turn all projects dashboard on by default
Related issue: https://github.com/storj/storj/issues/5872

Change-Id: Idd78f78a63534e15f99ebf3015431bbe52ba18ef
2023-05-24 13:45:22 +00:00
Clement Sam
bf8072d3f3 cmd/storagenode: fix identity flag for graceful exit commands
Resolves https://github.com/storj/customer-issues/issues/780
Resolves https://github.com/storj/storj/issues/5807

Change-Id: I0af2b14f5b12ddc1e7a09afb2cf5ccf025f53502
2023-05-24 07:58:34 +00:00
Vitalii
98591f34f9 satellite/{web, console}: implemented new objects uploading hover component
Added new feature flag.
Implemented new objects uploading hover component.

Issue:
https://github.com/storj/storj/issues/5825

Change-Id: If43ade5db33aecba6c089ce727503518c59f6ffc
2023-05-23 15:52:41 +00:00
Jeremy Wharton
d8f64326f5 satellite/accounting/tally: remove objects loop
This change removes the use of the objects loop for calculating bucket
tallies. It has been superseded by a custom query.

Change-Id: I9ea4633006c9af3ea14d7de40871639e7b687c22
2023-05-22 15:18:03 -05:00
Michal Niewrzal
eabd9dd994 satellite/orders: remove unsed argument
Change-Id: I6c5221fc19f97ae6db5627d7239795ff663289e0
2023-05-22 14:35:08 +00:00
Cameron
09ec5f107d satellite/payments: Remove expired package credits
During billing, before invoice creation, check if users are part of a
package plan. If so, and if the package plan is expired, remove unused
credit from the user's balance. If the user has credit in addition to
the package credit, send an analytics event to notify someone to handle
the credit removal manually.

Change-Id: Iad71d791f67c9733f9d9e42f962c64b2780264cc
2023-05-22 11:58:37 +00:00
Wilfred Asomani
250704493d web/satellite: fix frozen status banners not showing
This change moves the call to fetch user's frozen status into the
getUser store action. It fixes an issue where the froze status banners
will not show.

Change-Id: I1a1853d5a89502e294848ceb6a8566120ecd4c10
2023-05-19 17:06:28 +00:00
JT Olio
b502310fe5 cmd/inspector: remove
this is a very old tool built in the very early days
of v3, when we didn't know how the network would be
used. this tool anticipated being able to query remote
nodes for internal state. we don't do that. i don't
think anyone uses this.

Change-Id: Ie1ded3ecbedb09313f2d6fc721039e0f15e4ee85
2023-05-19 09:11:29 +00:00
paul cannon
3dc01bd25d satellite/repair: change how we log clumped pieces
rather than only logging the last_nets we see in clumpedPieces, this
will run through all the last_nets and log any that have more than one
node. This should have the same outcome, except the counts will be 1
higher (because FindClumpedPieces won't include the first node found in
a clumped network, and this will).

This should be quite a bit faster.

Change-Id: I6a7b2fd387e98963d5295c9ecfde80f2e1ee3b7a
2023-05-19 10:38:50 +02:00
paul cannon
c856d45cc0 satellite/overlay: fix GetNodesNetworkInOrder
We were using the UploadSelectionCache previously, which does _not_ have
all nodes, or even all online nodes, in it. So all nodes with less than
MinimumVersion, or with less than MinimumDiskSpace, or nodes suspended
for unknown audit errors, or nodes that have started graceful exit, were
all missing, and ended up having empty last_nets. Even with all that,
I'm kind of surprised how many nodes this involved, but using the upload
selection cache was definitely wrong.

This change uses the download selection cache instead, which excludes
nodes only when they are disqualified, gracefully exited (completely),
or offline.

Change-Id: Iaa07c988aa29c1eb05796ac48a6f19d69f5826c1
2023-05-19 08:08:08 +00:00
Michal Niewrzal
9b3488276d satellite/gracefulexit: use node alias instead id with observer
Using node alias helps using less cpu and memory.

Fixes https://github.com/storj/storj/issues/5654

Change-Id: If3a5c7810732cbb1bff4dcb78706c81aca56b71b
2023-05-18 22:37:46 +00:00
paul cannon
de737bdee9 satellite/repair: add flag for de-clumping behavior
It seems that the "what pieces are clumped" code does not work right, so
this logic is causing repair overload or other repair failures.

Hide it behind a flag while we figure out what is going on, so that
repair can still work in the meantime.

Change-Id: If83ef7895cba870353a67ab13573193d92fff80b
2023-05-18 21:02:36 +00:00
Michal Niewrzal
c0e7f463fe satellite/metabase: remove segmentsloop package
Last change to remove segments loop from codebase.

https://github.com/storj/storj/issues/5237

Change-Id: I77b12911b6b4e390a7385e6e8057c7587e74b70a
2023-05-18 19:08:29 +00:00
Wilfred Asomani
09311b22ca web/satellite: add role column to project members table
A "Role" column has been added to the project members table to indicate
the roles of the members in the project.

Issue: https://github.com/storj/storj/issues/5743

Change-Id: I991b01e3d2cd792c21bd661feb7a225300d9a86b
2023-05-18 16:16:00 +00:00
Michal Niewrzal
cf5ff537e3 cmd/tools/metabase-verify: switch to ranged loop
We would like to remove segments loop so we need to refactor
our tools to use ranged loop.

To simplify change ranged loop is used with single range only.

https://github.com/storj/storj/issues/5237

Change-Id: I94d96d54f9d0e37b06def4f4fc16b71c5b79baba
2023-05-18 14:27:38 +00:00
Moby von Briesen
19cb08b025 web/satellite: add "public linksharing URL"
* Update defaults for gateway credentials URL and linksharing URL to use
  storjsatelliteshare.io instead of storjshare.io
* Add new config for "public linksharing URL" and set it to
  link.storjshare.io
* Use "private" linksharing URL for actions within the object browser
* Use "public" linksharing URL for sharing files externally

Resolves https://github.com/storj/storj/issues/5805

Change-Id: I2c8fbd04141755b4751dcf4d054253a7ff8d6cf3
2023-05-18 07:02:26 +00:00
Wilfred Asomani
a2a603aee8 web/satellite: rightly navigate to pricing plan page
A navigation is prevented to the onboarding if no project is selected.
However, an exception was not made for the pricing plan page and caused
the pricing plan page not to show even under passing conditions. This
change makes that exception for the pricing plan page.

Change-Id: I72ddb942a0059d748a13da0b6038b99b9b5064af
2023-05-17 19:35:37 +00:00
Vitalii
ace0ef89a2 web/satellite: migrated Vue 2 to Vue 3
Finally migrated web/satellite UI to use Vue 3.

Issue:
https://github.com/storj/storj/issues/5862

Change-Id: I609e2332142b9092d6efeb2d4b931e34cce846cc
2023-05-17 17:31:03 +00:00
paul cannon
958d8676d0 satellite/overlay: remove unnecessary test helper
Change-Id: I8439eec4ed440f60353fc620ca906a917a03613c
2023-05-17 17:04:54 +00:00
paul cannon
1f4f79b6b3 satellite/repair: don't mark clumped segments as irreparable
Clumped segments (segments with multiple pieces on the same subnet) may
need repair, but the clumped pieces are considered retrievable and we
don't need to call such segments irreparable.

We do want to know where they're coming from, though, if we can, because
we are seeing more than expected.

Change-Id: I41863b243f4bb007ef8929191a3fde1562565ef9
2023-05-17 16:24:15 +00:00
Jeremy Wharton
607b120116 satellite/satellitedb: add inviter ID column to project invites table
The project member invitations table has been modified to contain a
column for the ID of the user who sent the invitation. This ID is
required for us to return information about the inviter to the
satellite frontend.

References #5855

Change-Id: I928d987a8db2340f731ca65ce30173d4f90a9837
2023-05-17 15:46:40 +00:00
Lizzy Thomson
f87772a0d3 satellite/console: bug fix for using a API key to retrieve a session
Change-Id: I0202f75fb78f42af9e7cb00bb3ac73839340c977
2023-05-16 14:06:32 -06:00
paul cannon
75d10fe4fa satellite/overlay: use UploadSelectionCache for GetNodesNetworkInOrder
The query for GetNodesNetworkInOrder is causing far too much load on the
database. Since it is not critical that the repair checker have
perfectly up-to-date node network information, we can use a cache
instead.

Change-Id: I07ad45bfdeb46529da093941a06c2da8a00ce878
2023-05-16 17:32:09 +00:00
Wilfred Asomani
e9fc4e9d0f web/satellite: fix billing page nav on all dashboard
This change fixes an issue where clicking "Billing" on all projects
dashboard won't navigate to the billing page.
This also makes sure an error notification for projectID is not shown
after the user is back on the all projects dashboard from team page.

related: https://github.com/storj/storj/issues/5875

Change-Id: Ie6517c9006d1a780f307f6282f83c749d369adaf
2023-05-16 15:14:18 +00:00
Márton Elek
f1b9f6b672 satellite/satellitedb: paginating on ordered invoice
87d0789691 replaces offset usage with cursors.

But to continue the interation from a specific cursor, we need to iterate over ordered records.

(at least this is what I understood based on the failing tests)

87d0789691

Change-Id: Ic4da3a7c5f03386dd4c373c05102f05871900a3a
2023-05-16 14:05:34 +00:00
Michal Niewrzal
4bdbb25d83 satellite/metabase/rangedloop: move Segment definition
We will remove segments loop soon so we need first to move
Segment definition to rangedloop package.

https://github.com/storj/storj/issues/5237

Change-Id: Ibe6aad316ffb7073cc4de166f1f17b87aac07363
2023-05-16 12:37:17 +00:00
Lizzy Thomson
7e69b22dd4 satellite/console: retrieve a web UI session using a API key
retrieve a web UI session using a API key and add new test

Issue https://github.com/storj/storj/issues/5688

Change-Id: I0ad9c783f0573b87b212b8fb065cdbf7074b782c
2023-05-15 10:24:58 -06:00
Wilfred Asomani
81b2b067e6 satellite/satellitedb/dbx/gen: update to work with updated dbx
This change updates the replacer in satellite/satellitedb/dbx/gen/main.go
to work with an updated dbx.

Change-Id: I08e89d6d27e6f1d435416105fe5f622009add7ad
2023-05-15 14:53:06 +00:00
JT Olio
e90612c285 docs/blueprint: partial rollout automation
Change-Id: I355d7071da7e501948665ee153ed641722a3d264
2023-05-15 13:49:47 +00:00
Michal Niewrzal
2592aaef9c satellite/gc/bloomfilter: remove segments loop parts
We are switching completely to ranged loop.

https://github.com/storj/storj/issues/5368

Change-Id: I1a22ac4b242998e287b2b7d8167b64e850b61a0f
2023-05-15 11:46:26 +00:00
Wilfred Asomani
5fede0ce95 web/satellite: fix no project id provided error
This change fixes an error when all projects dashboard is enabled where
there will be no project selected and an error "projectid not provided"
will be thrown.
It navigates back to the all projects dashoard so the user can select
a project.

Issue: https://github.com/storj/storj/issues/5875

Change-Id: I144c4bbd3782dcdf40b2ed2be445c7f100c36b32
2023-05-15 10:23:32 +00:00
Wilfred Asomani
331a3348a0 web/satellite: mod project options menu for all projects dashboard
The project options has been modified to account for whether the all
projects dashboard is enabled or not. The "Manage projects" item is
hidden, and an "All projects" item is added if it is enabled.

Issue: https://github.com/storj/storj/issues/5866

Change-Id: Ic2e8337d2e5577d1c27231287ab3284145fa5b9b
2023-05-12 20:10:33 +00:00
Wilfred Asomani
b11ba50b94 web/satellite: hide sidebar on onboarding
The sidebar previously was showing on the onboarding pages when all
projects dashboard is enabled. This change hides it and also hides the
"session timeout" info banner which was also showing on onboarding.

Issue: https://github.com/storj/storj/issues/5874

Change-Id: Ie418a6f286262b59dce41aba98a5c9d0015cb984
2023-05-12 18:57:02 +00:00
Vitalii
277a4c6aa3 web/satellite: updated project dashboard data cards
Updated cards to show access grants count, buckets count, team size and billing status for current project

Issue:
https://github.com/storj/storj/issues/5865

Change-Id: I7e8d3aa3e05548d593576c3a0633e9eb51f38cff
2023-05-12 14:43:09 +03:00
Vitalii
083b3d6fc1 web/satellite: added modal to update single project limit
Added new modal where user can update storage or bandwidth limit (if Pro tier).

Issue:
https://github.com/storj/storj/issues/5827

Change-Id: Ic3ae6b7a73d055b4eac93d4e2faad7bf26cb9d73
2023-05-12 11:38:39 +00:00
Vitalii
b91f72f08a web/satellite: added limit cards on project dashboard
Added storage, bandwidth, segment, free tier and coupon cards to project dashboard.

Issue:
https://github.com/storj/storj/issues/5827

Change-Id: Iaf974273a34c8ed3faf1974876fa685074d5ae61
2023-05-12 14:36:53 +03:00
paul cannon
c2710cc78d satellite/audit: improve error handling
* Don't use rpcstatus.Unknown as an indicator of dial failure; instead,
  GetShare now indicates with a per-share field where a failure happened
  (DialFailure, RequestFailure, NoFailure). Use that information in
  Verify() to determine how to treat the source node.

* Add a test that replaces a storage node with a black hole, so that
  connections there will always time out. Make sure we handle that case
  correctly.

Refs: https://github.com/storj/storj/issues/5632
Change-Id: I513a53520fb48b7187d4c4d7e14e01e2cfc0a721
2023-05-11 22:55:26 +00:00
igor gaidaienko
bc30deee11 satellite/repair: update repair test to use blake3 algo
Update repair test to also use blake3 hashing algorithm
https://github.com/storj/storj/issues/5649

Change-Id: Id8299576f8be4cfd84ddf9a6b852e653628ada72
2023-05-11 08:45:48 +00:00
Cameron
52ff7a66a0 web/satellite: use color palette instead of hex colors for button
Change-Id: I1da9c1927c2768e86f3887477250ae833e29439e
2023-05-10 17:54:26 +00:00