Commit Graph

8288 Commits

Author SHA1 Message Date
Michal Niewrzal
9550b5f4a5 satellite/metabase: drop object deletion code for copy with references
With this change we are removing code responsible for deleting objects
and supporting server side copies created with references. In practice
we are restoring delete queries that we had before server side copy
implementation (with small exception, see bellow).

From deletion queries we are also removing parts with segment metadata
as result because we are not longer sending explicit delete requests to
storage nodes.

https://github.com/storj/storj/issues/5891

Change-Id: Iee4e7a9688cff27a60fb95d60dd233d996f41c85
2023-08-08 11:31:12 +00:00
Michal Niewrzal
03f8ad323d satellite/metabase: remove segment_copies support from ListSegments
We don't need to support segment copies with references anymore.
We migrated to copies where all metadata are copied from original
segment to copy.

https://github.com/storj/storj/issues/5891

Change-Id: Ic91dc21b0386ddf5c51aea45530024cd463e8ba9
2023-08-08 11:21:08 +00:00
Vitalii
6c3300c522 satellite/console: add delete api keys http endpoint
This change adds an endpoint to delete API keys, similar to GraphQL mutation.

Issue:
https://github.com/storj/storj/issues/6140

Change-Id: Ia4a808222a057a199d803d8ea6b944c411a4dc8d
2023-08-08 10:46:54 +00:00
Vitalii
5345eadffa web/satellite: use create API key http endpoint
This change uses the new /create endpoint in place of the GraphQL query.

Issue:
https://github.com/storj/storj/issues/6139

Change-Id: Ic7db9407377b742ab87c887a74a37eaec6a8dbb6
2023-08-08 10:46:25 +00:00
Michal Niewrzal
7f249ab7ca cmd/tools: remove migrate-segment-copies tool
Migration was done. We can remove tool now.

https://github.com/storj/storj/issues/5891

Change-Id: I5d56bad1ac680cd77dabfcf271788e100a6a435b
2023-08-08 10:00:40 +00:00
Michal Niewrzal
a5cbec7b3b satellite/metabase: drop bucket deletion code for copy with references
With this change we are removing code responsible to handle deleting
bucket and supporting server side copies created with references. In practice we are restoring delete queries that we had before server side copy implementation (with small exception, see bellow).

From deletion queries we are also removing parts with segment metadata
as result because we are not longer sending explicit delete requests to
storage nodes.

https://github.com/storj/storj/issues/5891

Change-Id: If866d9f3a1b01e9ebd9b49c4740a6425ba06dd43
2023-08-08 09:07:44 +00:00
Wilfred Asomani
00531fe2b0 web/satellite/vuetify: add disable MFA dialog
Added disable MFA dialog and functionality.

Issue: https://github.com/storj/storj/issues/6091

Change-Id: I24cf863b2a04cc147894ab0640e1e1491acc430d
2023-08-08 07:57:01 +00:00
Vitalii
185ebe3dcf web/satellite/vuetify: added notifications
Added notifications to Vuetify app.
Populated existing functionality with notifications.

Issue:
https://github.com/storj/storj/issues/6087

Change-Id: I8339c372bb32fbf1e0ea136c92383494c129b4b6
2023-08-08 07:14:52 +00:00
Vitalii
f57bc81ce7 satellite/console: add create api key http endpoint
This change adds an endpoint to create new API key, similar to GraphQL mutation.

Issue:
https://github.com/storj/storj/issues/6139

Change-Id: I2b35d680fa8e019666c811ad3bdf16201e3b8946
2023-08-07 23:58:38 +00:00
Vitalii
fa4f5a6ae9 web/satellite: use get project api keys http endpoint
This change uses the new /list-paged endpoint in place of the GraphQL query.

Issue:
https://github.com/storj/storj/issues/6138

Change-Id: I68c35437f3f2e18898783e2fd523cfdfb3f10de3
2023-08-07 18:59:30 +00:00
Vitalii
256bd18120 satellite/console: add get project api keys http endpoint
This change adds an endpoint to get paged project API keys, similar to GraphQL query.

Issue:
https://github.com/storj/storj/issues/6138

Change-Id: I5dea9e4ac61e798cc8a2e56a2755d722c1b66bfa
2023-08-07 18:26:31 +00:00
Vitalii
a4f7f0634d web/satellite/vuetify: added analytics events for currently implemented functionality
Populated already implemented functionality with analytics events.

Issue:
https://github.com/storj/storj/issues/6122

Change-Id: I1e250ca02debc9bedbf78024f74e0dbfa9dfbcb9
2023-08-07 17:52:39 +00:00
Vitalii
e2e437dd95 web/satellite: use analytics pinia module instead of direct API requests
Start using analytics pinia module instead of making direct API requests from components.
We do this to not mix transport layer logic with view logic.

Issue:
https://github.com/storj/storj/issues/6122

Change-Id: Idb1902aec0635df4f0e682ba50bcc97b240ac4a9
2023-08-07 17:18:10 +00:00
Wilfred Asomani
e5bcb8b209 web/satellite: use get user projects http endpoint
This change uses the new /projects endpoint in place of the GraphQL
MyProjects query.

Issue: https://github.com/storj/storj/issues/6132

Change-Id: Ie4ae69dd6def75c4b8c627aaf55c31914cf69ce5
2023-08-07 16:42:41 +00:00
Wilfred Asomani
70f6b60d91 satellite/console: add get user projects http endpoint
This change adds an endpoint to get a user's projects, similar to
the MyProjects GraphQL query.

Issue: https://github.com/storj/storj/issues/6132

Change-Id: I91feb5a1ee8c1231a8a5e6de9b8dc5b256f857c5
2023-08-07 16:05:09 +00:00
Vitalii
d8d3bb5033 web/satellite/vuetify: added update session timeout dialog
Added update session timeout dialog for vuetify app and wired it up with the backend

Issue:
https://github.com/storj/storj/issues/6090

Change-Id: Icfd981c8d24a87b199e5e1eeb599f6369e6ee5c4
2023-08-07 15:31:13 +00:00
paul cannon
6e46a926bb satellite/nodeselection: expand SelectedNode
In the repair subsystem, it is necessary to acquire several extra
properties of nodes that are holding pieces of things or may be
selected to hold pieces. We need to know if a node is 'online' (the
definition of "online" may change somewhat depending on the situation),
if a node is in the process of graceful exit, and whether a node is
suspended. We can't just filter out nodes with all of these properties,
because sometimes we need to know properties about nodes even when the
nodes are suspended or gracefully exiting.

I thought the best way to do this was to add fields to SelectedNode,
and (to avoid any confusion) arrange for the added fields to be
populated wherever SelectedNode is returned, whether or not the new
fields are necessarily going to be used.

If people would rather I use a separate type from SelectedNode, I can do
that instead.

Change-Id: I7804a0e0a15cfe34c8ff47a227175ea5862a4ebc
2023-08-07 12:44:49 +00:00
Márton Elek
0b02a48a10
satellite/nodeselection: SelectBySubnet should use placement filters for all nodes
Current node selection logic (in case of using SelectBySubnet):

 1. selects one subnet randomly
 2. selects one node randomly from the subnet
 3. applies the placement NodeFilters to the node and ignore it, if doesn't match

This logic is wrong:

 1. Imagine that we have a subnet with two DE and one GB nodes.
 2. We would like to select DE nodes
 2. In case of GB node is selected (randomly) in step2, step3 will ignore the subnet, even if there are good (DE) nodes in there.

Change-Id: I7673f52c89b46e0cc7b20a9b74137dc689d6c17e
2023-08-04 10:48:15 +02:00
Michal Niewrzal
03c52f184e satellite/metabase: adjust CommitObject to use pending_objects table
Change is adjusting CommitObject to use `pending_objects` table to
commit object.

Satellite stream id is used to determine if we need to use
`pending_objects` or `objects` table during commit.

General goal is to support both tables until `objects` table will be
free from pending objects.

Part of https://github.com/storj/storj/issues/6046

Change-Id: I2ebe0cd6b446727c98c8e210d4d00504dd0dacb6
2023-08-03 10:21:22 +00:00
Márton Elek
2ed08922d9 satellite/api: configuration option to set additional node tag authorities
Change-Id: Iba387e37cb586ce1378668c99beb3b4db8a9064d
2023-08-03 08:34:02 +00:00
Wilfred Asomani
034542db8f web/satellite/vuetify: add enable MFA dialog
Added enable MFA dialog and functionality.

Issue: https://github.com/storj/storj/issues/6091

Change-Id: Idf2f27937549b9bb709ddcc70ffa4611beea7b78
2023-08-02 17:31:02 +00:00
Michal Niewrzal
f40805763e satellite/metabase: adjust segment commit to use pending_objects table
Change is adjusting CommitSegment to check pending object existence in
`pending_objects` or `objects` table.

Satellite stream id is used to determine if we need to use
`pending_objects` or `objects` table.

General goal is to support both tables until `objects` table will be
free from pending objects. Whenever it will be needed code will be
supporting both tables at once.

Part of https://github.com/storj/storj/issues/6046

Change-Id: I954444a53b4733ae6fc909420573242b02746787
2023-08-02 16:56:25 +00:00
Cameron
fac522d8dd web/satellite: implement vuetify poc project dashboard
issue: https://github.com/storj/storj/issues/6060

Change-Id: I40497a39afacd4787468e1259225ddd75e78295b
2023-08-02 16:22:22 +00:00
Michal Niewrzal
7b2006a883 satellite/metabase: adjust BeginSegment to use pending_objects table
Change is adjusting BeginSegment to check pending object existence in
`pending_objects` or `objects` table.

Satellite stream id is used to determine if we need to use
`pending_objects` or `objects` table.

General goal is to support both tables until `objects` table will be
free from pending objects. Whenever it will be needed code will be
supporting both tables at once.

Part of https://github.com/storj/storj/issues/6046

Change-Id: I08aaa605c23d82695fde352fdbd0a7fd11f46bb5
2023-08-02 15:30:23 +00:00
Michal Niewrzal
cebf255d64 satellite/metabase: adjust BeginObjectNextVersion to use pending_objects
Change is adjusting BeginObjectNextVersion to create pending object in
`pending_objects` or `objects` table depends on configuration. This is
first change to move pending objects from objects table.

General goal is to support both tables until `objects` table will be
free from pending objects. Whenever it will be needed code will be
supporting both tables at once.

To be able to decide if we need to use `pending_objects` table or
`objects` table we extend satellite stream id to keep that information
for later use.

BeginObjectExactVersion will be not adjusted because at the moment it's
used only in tests.

Part of https://github.com/storj/storj/issues/6046

Change-Id: Ibf21965f63cca5e1775469994a29f1fd1261af4e
2023-08-02 14:42:26 +00:00
Jeremy Wharton
391a63f9fa web/satellite: bump Vuetify to v3.3.10
This change updates the Vuetify dependency version so that we may use
new components.

Change-Id: Ib4ae4db3b50c267179503ff866077379d52b6d07
2023-08-02 13:46:20 +00:00
Vitalii
2d2f1b858e web/satellite: show pending transactions during token upgrade flow
Made UI updates to reflect pending token payments during upgrade account flow.
So pending transactions with confirmations count are displayed on Add STORJ Tokens step of upgrade flow.
While this modal is open we make a request to storjscan once in 20 seconds to get recent confirmations count.
When all transactions become confirmed we show success view where a sum of STORJ tokens received is displayed.

Issue:
https://github.com/storj/storj/issues/5978

Change-Id: Icfdc1e5080ed58cea1822cb7d85551ba8064c636
2023-08-02 15:39:25 +03:00
Márton Elek
63c8cfe4c3 satellite/nodeselection: remove CountryCodeExclude
We don't need it any more, as CountryCode uses location.Set, which supports exclusion (`Without`).

Change-Id: Ie311ae19fefa0bc9a0161496af1233ef4a6607df
2023-08-02 09:48:36 +00:00
Márton Elek
f7b39aaed4 satellite/nodeselection: remove stats/size from nodeselection state
stats/size/count is not used by any production code, and it's not required, as we can assert the state with other checks.

real motivation: next commits will make the Selector of the State configurable, therefore we won't have one single Stat, it depends on the request parameters.

(we plan to support both network and id based randomization)

Change-Id: I631828fc0046d2fef5b7a674fc0268a0446e9655
2023-08-01 18:29:41 +00:00
Wilfred Asomani
7c65c0cea5 satellite/{db,console,payments}: unfreeze user with no failed invoices
This change extends the autofreeze chore to go through users who have
been warned/frozen to check if they have no failed invoices. If they do
not, this extension unwarns/unfreezes them.

Issue: https://github.com/storj/storj/issues/6077

Change-Id: I570b1d4b2e29574bd8b9ae37eb2d4fb41d178336
2023-08-01 17:54:39 +00:00
JT Olio
e78658d174 go.mod: use latest maxmind-golang library
Change-Id: I6f06785a096d7d05b175d468482db658b7436a87
2023-08-01 15:22:35 +00:00
Michal Niewrzal
65aa9c11bc satellite/gc/sender: adjust some defaults
Change-Id: I94f5a18bc2d4218e834259b2ba22f3578745c975
2023-08-01 14:27:44 +00:00
Vitalii
59034ca094 web/satellite: upgraded aws-sdk dependencies to resolve vulnerabilities
Upgraded package-lock version to correspond to node v18+.
Upgraded aws-sdk dependencies to resolve vulnerabilities.
Also, fixed typing errors in object browser pinia module.

Change-Id: I35e6e219e66f98ca167ccb4ac57ad07ac99efff1
2023-08-01 13:41:15 +00:00
Vitalii
d40091a3cb web/satellite/vuetify: change name dialog
Added change name dialog for vuetify app and wired it up with backend.

Issue:
https://github.com/storj/storj/issues/6088

Change-Id: I37af595c9e48034d59ead051918bbb4ec5e6ff31
2023-08-01 14:26:16 +03:00
Wilfred Asomani
af69e20cc8 web/satellite: fix errant pagination
This change fixes an issue where changing the size of a page will also
change the current page. It uses a more accurate calculation to
determine if the new size and current page will result in a page index
error.

Issue: https://github.com/storj/storj/issues/6094

Change-Id: Ie62f79191eb34ccd75ccd42b923b8866fe4e4d7f
2023-07-31 22:43:52 +00:00
Vitalii
ceef4b8362 web/satellite/vuetify: added change password dialog
Added change password dialog for vuetify app and wired it up with backend.

Issue:
https://github.com/storj/storj/issues/6089

Change-Id: Ib1dffa947b65c299b278a48ed1491a623895a0bd
2023-07-31 17:03:26 +03:00
Márton Elek
6f002f4220
satellite/overlay: NR placement should exclude nodes without geofencing information
https://github.com/storj/storj-private/issues/378

Change-Id: If2af02083496e5a8eefe27beabb406388ee50644
2023-07-31 09:55:54 +02:00
Vitalii
ae5e742a12 web/satellite: update wording for upgrade account banner
Updated wording because we don't want to dissuade customers from upgrading and using Storj by implying that there is a ceiling to what they can store and egress with Storj.

Issue:
https://github.com/storj/storj/issues/6040

Change-Id: I8a08bea47b698e66e023d1bfcaa5ef09dae79192
2023-07-28 09:46:49 +00:00
Vitalii
62f52c829a web/satellite/vuetify: add charts to project dashboard
Imported chart components from main project to vuetify app project dashboard with slight updates.

Issue:
https://github.com/storj/storj/issues/6071

Change-Id: I51254ea60aab59c7a841784b5dac7d41446e1df4
2023-07-27 23:16:40 +00:00
Vitalii
fcbb37fb66 web/satellite: fix build-watch
Build-watch is broken again since we don't use NODE_ENV variable anymore.
Adjusted code to work with 'mode' setting.

Change-Id: I08b19ef59b39e6b14ce05d7340f032ee8377c49c
2023-07-27 21:36:18 +00:00
Wilfred Asomani
c934974652 satellite/{console,web}: update error handling
This change updates how API errors are handled and sent.

Change-Id: Ia4c71eeb6f2d009a47b59ce77a23f70b8b10f6dc
2023-07-27 21:01:01 +00:00
Vitalii
2c56599ca0 web/satellite: revert uploads queue size
Revert upload parallelism queue size back to 4 segments at a time.
Increasing this number causes some memory issues for some users.

Issue:
https://github.com/storj/storj/issues/6120

Change-Id: Ic5bcd1f13ee625fdc2613ee23d6945c905f03142
2023-07-27 19:57:11 +03:00
Clement Sam
cc12a48c24 satellite/admin: extend admin API to allow setting and deleting geofence for projects
Issue: https://github.com/storj/storj-private/issues/357
Change-Id: Ib59319581641f1f5da71c629143e12f11eb04925
2023-07-27 11:40:26 +00:00
Michal Niewrzal
4cc167a6bd satellite/metabase: use better label for ignoring FTS queries
For some test queries we are using workaround to filter them out from
full table scan detection. To avoid confustion what is this all about
we are changing label to be more descriptive.

Change-Id: I41a744e8faf400e3e8de7e416d8f4242f9093fce
2023-07-26 20:01:38 +00:00
Michal Niewrzal
3d9c217627 satellite/metabase: add pending_objects table
This change adds only schema definition of pending_objects table and
small amount of supporting code which will be useful for testing later.

With this table we would like to achieve two major things:
* simplify `objects` table, before we will start working on object
versioning
* gain performance by removing need to filter `objects` results with `status` column, which is not indexed and we would like to avoid that

https://github.com/storj/storj/issues/6045

Change-Id: I6097ce1c644a8a3dad13185915fe01989ad41d90
2023-07-26 20:00:58 +00:00
Moby von Briesen
28737f5c62 web/satellite: Implement "sign out" in vuetify poc
Implements the "sign out" button in the vuetify app to clear all app
data and navigate to login.

For now, there is a hard refresh after the router navigates to login,
becuase the vuetify poc does not have its own login page to route to.
The hard refresh allows the user to be presented with the main app's
login page after signing out.

Change-Id: Idb1c1e6af8511ea5db6533e46b96b9c15693379f
2023-07-26 13:49:30 -04:00
Moby von Briesen
7487809476 web/satellite: unmock list buckets in vuetify app
This change implements the table on the buckets view of the vuetify app.
Searching and pagination are implemented, but sorting functionality
needs to be added in a separate change - we need to extend the
bucketsStore/backend functionality in order to support sorting.

Resolves https://github.com/storj/storj/issues/6062

Change-Id: I4e92e6facbd7b21f4ec081e41f7e568ddb3cea29
2023-07-26 13:49:30 -04:00
Michal Niewrzal
16c5b3c340 mod: bump uplink to v1.11.0
Change-Id: I88922421f250a261b78227a409f4879ae5221140
2023-07-26 15:01:18 +00:00
Vitalii
6d94d6a681 satellite/{console, accounting}: fix project bandwidth calculation which is shown on project dashboard
Another try to fix calculation of used bandwidth which is displayed on Project Dashboard.
This change sums up allocated-dead traffic for the last 3 days and settled traffic for the period which is earlier than 3 days ago.

Issue:
https://github.com/storj/storj-private/issues/293

Change-Id: I91e652eba69f81bd21e0d053ac170e2b926b3cb4
2023-07-26 06:14:48 +00:00
Vitalii
5db0fd8846 web/satellite: wire up vuetify access page to backend
Added logic to list, sort, search and paginate access grants table for Vuetify POC

Change-Id: I215a9ad4f894b6ac985cb6a3059fdcf007e3d914
2023-07-24 23:17:19 +00:00