Commit Graph

8454 Commits

Author SHA1 Message Date
Michal Niewrzal
929dc80091 satellite/metabase: add IteratePendingObjects method
New metabase method IteratePendingObjects to iterate over entries in
pending_objects table. Implementation and tests are mostly copy of
code that we are using to iterate over objects table. Main difference
is that pending_objects table have StreamID column part of primary
key instead Version. Also structure of pending object is smaller
than the one from object table but it's a detail.

Method will be used to support new table in metainfo.ListObjects
request.

Next step will be to port rest of iterator implementation to support
pending_objects table in metainfo.ListPendingObjectStreams.

Part of https://github.com/storj/storj/issues/6047

Change-Id: Ia578182f88840539f3668d4a242953e061eace02
2023-08-21 08:07:16 +00:00
Michal Niewrzal
5a8ef89824 satellite/{metainfo,metabase}: delete from pending_objects table
We are deleting pending objects while aborting multipart upload. We are
using metainfo BeginDeleteObject to do that. This change starts using
DeletePendingObjectNew to delete entry from pending_objects table when
request indicates that object is in this table.

Part of https://github.com/storj/storj/issues/6048

Change-Id: I4478a9c13c8e3db48dc5de3087ef03d1b4c47a5c
2023-08-21 08:06:23 +00:00
Wilfred Asomani
33c0a82fb7 satellite/console: add create project http endpoint
This change adds an HTTP endpoint for creating projects, to be used in
place of the GraphQL version.

Issue: https://github.com/storj/storj/issues/6195

Change-Id: I0377353418df7c152db6a935e99a3ea7ab4ce625
2023-08-21 06:58:03 +00:00
Cameron
6f47e178e2 web/satellite: add "do not share link" to emails
Change-Id: I94da487b622f083d5cf893d6dd90ad30076f6466
2023-08-18 23:16:13 +00:00
Vitalii
d0c3a59f44 web/satellite: differentiate password and passphrase autocomplete inputs
Added unique autocomplete values for password and passphrase inputs to make browser native autocomplete feature work correctly.

Issue:
https://github.com/storj/storj/issues/6151

Change-Id: I38e1bfed46beb47b3be1a620ce2a4996359feafc
2023-08-18 17:29:20 +00:00
Wilfred Asomani
f46280c93b web/satellite/vuetify-poc: add open bucket passphrase
This change requires a passphrase before opening a bucket and after
opening the objects page without going through the buckets page. It also shows if there are
objects stored with different passphrase.

Issues: https://github.com/storj/storj/issues/6100
https://github.com/storj/storj/issues/6110

Change-Id: Ica509fd6db968ef9c232a05b3ee39e0ab5f746e1
2023-08-18 10:54:05 +00:00
Jeremy Wharton
2b278bb05f web/satellite/vuetify-poc: unmock total cost, token balance, coupon
This change replaces mock data for the total cost, STORJ token balance,
and coupon information in the Vuetify project's billing overview with
real data.

Resolves #6095

Change-Id: Iad1c5141065f8761d7d13140d42871018fa9aca4
2023-08-18 10:14:57 +00:00
Márton Elek
f0afe0d2ea satelite/repairer: ignore declumping when subnet filtering is turned off with filter annotation
+ restoring the functionality of repairer.doPlacementCheck

Change-Id: I75521f2da280758345face07eeea661765717318
2023-08-18 09:35:38 +00:00
Márton Elek
5522423871 satellite/nodeselection: remove AutoExcludeSubnet filter
It's statefull, therefore it can hit naive users. (NodeFilters couldn't be reused for more than one iterations).

But looks like we don't need it, as `SelectBySubnet` doest the same job.

Change-Id: Ie85b7f9c2bd9a47293f4e3b359f8b619215c7649
2023-08-18 08:31:00 +00:00
Jeremy Wharton
db13e3ef15 web/satellite: warn when indenting with tabs
This change adds a rule to the ESLint config to warn when tabs are used
instead of spaces for indentation.

Change-Id: I068d559211e0f9e40abaea86de839a409b405674
2023-08-17 20:45:47 +00:00
Jeremy Wharton
c706df5218 web/satellite/vuetify-poc: make project invite titles unclickable
Users with a pending invitation for a project could be redirected to
that project's dashboard if its title in the projects table was
clicked. This type of redirection is only intended for projects that
the user is a member of, as navigating into any other project will send
the user back to the all projects dashboard. This change fixes this
issue, making invitation titles in the projects table unclickable.

Change-Id: I6ef6a72d92e8eb576d49d6268373af340116ee2f
2023-08-17 20:07:16 +00:00
Wilfred Asomani
ea38b9f78e web/satellite/vuetify-poc: fix vuetify form reloads
This change fixes an issue where the vuetify app will reload when
"enter" is pressed in a form. It also adds a success notification on
successfully enabling 2FA.

Change-Id: I23a66122dce4f1bd2ff21a5cd6ae0f1b22be9395
2023-08-17 19:28:11 +00:00
Jeremy Wharton
957d8d6ca0 web/satellite/vuetify-poc: add bucket deletion option to buckets table
This change allows buckets to be deleted from the Buckets page through
a dropdown menu in the buckets table. Before deletion, users are
prompted to confirm their decision in a dialog box.

Revoles #6114

Change-Id: Ie3a8bfbf94a29c4bc67b4dacbace6abe7e1bc2eb
2023-08-17 18:48:54 +00:00
Jeremy Wharton
f2e607c70f web/satellite/vuetify-poc: fix project navigation sidebar paths
This change resolves an issue where the buttons in the navigation
sidebar would not navigate to the correct page when clicked if the
current path was not a sibling of the intended page's path.

Change-Id: I400aa05954f532eabbad98f9fa36f594f08a6c10
2023-08-17 16:52:31 +00:00
Márton Elek
b218002752 satellite/overlay/placement: improve placement configurability with &&, placement, region and ! support in country
This patch makes it easier to configure existing placement rules only with string.

 1. placement(n) rule can be used to reuse earlier definitions
 2 .&& can be used in addition to all(n1,n2)
 3. country(c) accepts exclusions (like '!RU'), regions ('EU','EEA'), all and none

See the 'full example' unit test, which uses all of these, in a realistic example.

https://github.com/storj/storj/issues/6126

Change-Id: Ica76f016ebd002eb7ea8103d4258bacd6a6d77bf
2023-08-17 16:12:53 +00:00
Egon Elbre
b5e1e5a9e2 private/apigen: use typescript in docs markdown
Fixes https://github.com/storj/storj/issues/6187

Change-Id: I34016513f29a9538343f0909951ab1a4605bb585
2023-08-17 15:33:21 +00:00
Vitalii
5abed63f53 web/satellite: update estimated charges info message
Updated message to be more clear, and changed "estimated charges" to
"estimated usage". This should make this card in the billing UI less
confusing.
Updated styling.

Issue:
https://github.com/storj/storj/issues/6150

Change-Id: I5de4d72dfd437d4f6bda882f5b779b2217e84f98
2023-08-16 15:11:59 -04:00
Márton Elek
9ddc8b4ca3 satellite/repair: piecescheck.OutOfPlacementPiecesSet should not contain offline nodes
When we check the availability of the pieces, we do:

```
result.NumUnhealthyRetrievable = len(result.ClumpedPiecesSet) + len(result.OutOfPlacementPiecesSet)
// + some magic if there are overlaps between them
numHealthy := len(pieces) - len(piecesCheck.MissingPiecesSet) - piecesCheck.NumUnhealthyRetrievable
```

This works only if OutOfPlacementPieceSet doesn't contain the offline nodes (which are already included in MissingPieceSet).

But `result.OutOfPlacementPieces.Set` should include all the nodes (even offline), as in case of lucky conditions, we are able to remove those pieces from DB.

The solution is to remove all offline nodes from `NumUnhealthyRetrievable`.

Change-Id: I90baa0396352dd040e1e1516314b3271f8712034
2023-08-16 17:35:10 +00:00
Jeremy Wharton
80186ecc67 web/satellite/vuetify-poc: add project settings page
This change implements the Project Settings page in the Vuetify
project. It allows users to view and change a project's name,
description, storage limit, and bandwidth limit.

Resolves #6176

Change-Id: Ibcc56d2bb7c71e818390e69eec197508e3551803
2023-08-16 11:55:01 -05:00
Márton Elek
de7aabc8c9 satellite/{repair,rangedloop,overlay}: fix node tag placement selection for repair
This patch fixes the node tag based placement of rangedloop/repairchecker + repair process.

The main change is just adding the node tags for Reliable and KnownReliabel database calls + adding new tests to prove, it works.

https://github.com/storj/storj/issues/6126

Change-Id: I245d654a18c1d61b2c72df49afa0718d0de76da1
2023-08-16 15:45:41 +00:00
Egon Elbre
a3067b7b3b storagenode/monitor: ignore shutdown errors
This fixes some flakyness in tests.

Change-Id: I535232c5d80827d6d72c73d61134f3b2806b5db9
2023-08-16 11:53:58 +00:00
Jeremy Wharton
93ce4a0e49 web/satellite/vuetify-poc: fix offscreen notifications
This change fixes an issue where notifications would display above the
viewport if the page was scrolled down. Now, notifications are fixed to
the top-right corner of the viewport.

Change-Id: I4d55b7d149b889e7b41e2f245ff8547e998877fc
2023-08-16 10:11:28 +00:00
Wilfred Asomani
d0f5f06159 web/satellite/vuetify-poc: add create bucket dialog
This change adds the ability to create a bucket to the vuetify poc.

Issue: https://github.com/storj/storj/issues/6112

Change-Id: Ib56432e2ad09c6673c9903a49537199882fd1cf3
2023-08-16 09:25:33 +00:00
Márton Elek
da08117fcd satellite/~placement: do not ignore placement check for placement=0
There are cases when we would like to override the default placement=0 rule.

For example when we would like to exclude tagged nodes from the selection (by default).

Therefore we couldn't use a shortcut any more, we should always check the placement rules, even if we use placement=0.

TODO: we need to update common, and rename `EveryCountry` to `DefaultPlacement`, just to avoid confusion.

https://github.com/storj/storj/issues/6126

Change-Id: Iba6c655bd623e04351ea7ff91fd741785dc193e4
2023-08-16 07:06:56 +00:00
Jeremy Wharton
baef654197 web/satellite/vuetify-poc: fix visibility of input details
This change resolves an issue that 37a027a introduced wherein each
input component's details area was indiscriminately hidden. Now,
details are only hidden when there is no content to be displayed.

References #6170

Change-Id: I8ccbef3f64a54c400a2a2e9222618eb5015c8dd8
2023-08-15 19:45:53 +00:00
Márton Elek
c08792f066 satellite/overlay: implement an exclude filter for placement configuration
https://github.com/storj/storj/issues/6126

Change-Id: I05215b5d46bec958001cc020edf1fa97b00d3299
2023-08-15 17:29:29 +00:00
Márton Elek
0e17b1018c satellite/{nodeselection,overlay}: support annotations on node filters
Change-Id: I844d8a25042750aae189175842113e2f052d5b17
2023-08-15 16:49:57 +00:00
Jeff Wendling
b70fb2f87f cmd/uplink: fix progress bar crash
the progress bar was being set to inconsistent lengths
multiple times, causing a crash. this fixes that by
only setting the progress bar length once to the length
of the full object. it avoids a round trip by doing so
only after it has gotten the first read handle from the
source, so the length information is cached.

Change-Id: I112d7c79016e54ba3794e96c6174cc01b8baedb4
2023-08-15 13:10:03 +00:00
Egon Elbre
d701f9f081 Makefile: use storjlabs/ci:slim
We moved the slim build to the same image name and used the tag
for distinguishing the different builds.

Change-Id: I82e235ba4db6a0b76b1edccab3b13d006c6c9328
2023-08-15 09:49:27 +00:00
Jeff Wendling
1cbad0fcab cmd/uplink: add back parallelism
for very large machines (>10Gbit) it is still useful
to have parallelism for uploads because we're actually
bound by getting new pieces from the satellite, so doing
that in parallel provides a big win.

this change adds back that flag to exist for uploads, and
removes the backwards compatibility code for the flag with
the maximum-concurrent-pieces as they are now independent.

the upload code parallelism story is now this:

    - each object is a transfer
    - each transfer happens in N parts (size dynamically
      chosen to avoid having >10000 parts)
    - each part can happen in parallel up to the limit
      specified
    - each parallel part can have up to the limit of
      max concurrent pieces and segments

this change also changes some defaults to be better.

    - the connection pool capacity now takes into acount
      transfers, parallelism and max concurrent pieces
    - the default smallest part size is 1GiB to allow the
      new upload code path to upload multiple segments

Change-Id: Iff6709ae73425fbc2858ed360faa2d3ece297c2d
2023-08-14 20:28:58 -04:00
Jeremy Wharton
03690daa35 web/satellite/vuetify-poc: add session timeout and refresh
This change allows sessions within the Vuetify project to be refreshed.
In addition, dialogs appear to inform the user when a session is about
to expire and when it has expired.

Resolves #6147

Change-Id: I53d5508825aa9992e4fed8ce7b957d949ff28e2d
2023-08-14 16:16:51 +00:00
littleskunk
792bb113bc
satellite/console: Enable gallery view and limits area by default (#6177) 2023-08-14 17:05:19 +02:00
Moby von Briesen
b9206b1844 satellite/console: support hosting Vuetify POC on subdomain
This change allows you to host the vuetify app on <x>.example.com where
the main app is hosted on example.com. A configuration is added to
specify an exact subdomain for cookies. For example, if my production
app is hosted on us1.storj.io and my vuetify app is hosted on
vuetify.us1.storj.io, the cookie domain should be set to ".us1.storj.io"
so that any authentication cookie is accessible to lower-level
subdomains.

Since the vuetify app does not currently support login/signup on its
own, it is still required to first login to the main satellite UI, then
navigate to the Vuetify app after the session cookie is set.

If the "vuetifypoc" prefix is not desirable when using subdomain hosting
for vuetify, the VITE_VUETIFY_PREFIX variable can be modified in
web/satellite/.env before running `npm run build-vuetify`. For now, we
should keep this prefix because it makes developing on the vuetify app
significantly easier if subdomains are not being used.

Issue: https://github.com/storj/storj/issues/6144

Change-Id: Iba1a5737892c8ee8f38148a17b94e3222f8798e6
2023-08-14 13:15:41 +00:00
Márton Elek
ceb7b7375c satellite/geoip: exclude nodes with represented_contry from geofencing
Change-Id: I80b8e5d98f46559b158a26c47fff0586b97aff79
2023-08-14 12:36:33 +00:00
Wilfred Asomani
0fd7f2958f web/satellite/vuetify-poc: add functionality to add credit cards
This implements functionality to add cards to users' accounts, to the
vuetify POC.

Issue: https://github.com/storj/storj/issues/6097

Change-Id: Ie56e85e74fd44de6839e6a985877843b730a3f5f
2023-08-14 09:45:53 +00:00
Cameron
df60380793 web/satellite: use project members delete http endpoint
issue: https://github.com/storj/storj/issues/6136

Change-Id: I717755721dc29714ad161fd311f7226a69acc163
2023-08-11 13:48:35 -04:00
Cameron
969599b60b satellite/console: delete project members endpoint
This commit adds a new endpoint on the console api to delete project
members and invitations.

issue: https://github.com/storj/storj/issues/6136

Change-Id: I980bb97afd1ed2ed8f0f27cc2e8dc1d80d7eef05
2023-08-11 13:48:33 -04:00
Vitalii
d7d196fe61 web/satellite: small vuetify fixes
Removed selection/checkboxes on team table.
Made search bars on searchable tables clearable.
Navbar toggle works on small screen sizes.
Removed notifications section from the settings page.

Issue:
https://github.com/storj/storj/issues/6148

Change-Id: I531088cf0db04fd2979d39f92bb4342e0d9551a6
2023-08-11 16:14:39 +00:00
Jeremy Wharton
03b45162d9 web/satellite/vuetify-poc: update alert styling
The custom styling for the VAlert component as of
storj/vuetify-storj@71c509a has been pulled into the Vuetify project.

References #6170

Change-Id: Ia36fd936ddc33c7687bcf7b2cdd6f789d0bc0a92
2023-08-11 10:17:16 -05:00
Moby von Briesen
dbc7443c9d web/satellite: Set part size back to 64mb
Reverts this commit to make part size dynamic:
a9d979e4d7

There are issues with client-side memory usage using the dynamic sizing.

Change-Id: Iba18af1e424622b7b3bf57413322332d8e1aa63a
2023-08-11 12:53:10 +00:00
Vitalii
8b0d25cde1 web/satellite/vuetify: speed up build commands
Fixed the way we use vuetify settings. Seems like the problem was related to SASS variables.
Partially used this approach https://github.com/vuetifyjs/vuetify/issues/8169#issuecomment-1495643012

Also, bumped vite and vuetify deps.

New build time for me:
real    0m28.629s
user    0m41.237s
sys     0m3.753s

Issue:
https://github.com/storj/storj/issues/6143

Change-Id: I94f8ee77057b85faf4d1a10bed7b43ba8ea929f1
2023-08-11 12:13:08 +00:00
Jeremy Wharton
75f2152ae3 web/satellite: modularize session timeout code
This change adds a Vue composable and wrapper component for reusing
session timeout code. In the future, the composable will be used to
implement session timeout in the Vuetify project.

References #6147

Change-Id: Ibd049a8a8041007319798ac4187a6ed6487b591f
2023-08-10 20:55:57 -05:00
Vitalii
6c035a70af web/satellite: modify 'add STORJ token' behavior for free-tier users
When a user is in the free tier and attempts to add STORJ from billing screen, open the "upgrade" modal instead of the simpler "add STORJ tokens" modal.
The simpler modal is still used for Pro users.

Issue:
https://github.com/storj/storj/issues/6165

Change-Id: I261d67e1c4d569f7058da828bcb145a64136c231
2023-08-10 19:11:05 +00:00
Egon Elbre
0335697664 satellite/accounting/live: ensure we don't panic when we get nil
Change-Id: I99b6b94d37a9856e8a705679d117b42d16326e81
2023-08-10 19:41:57 +03:00
Egon Elbre
1f261bcc70 go.mod: bump lang to 1.19 and common
Change-Id: I8d91f97d786456da29ebe89a78412c50efbb8ccc
2023-08-10 18:41:15 +03:00
Jess Stingray
6f01a81648
Downgrade "context canceled" errors to Info instead of Error (#6166) 2023-08-10 09:33:49 +02:00
Vitalii
e226606ce8 web/satellite: updates for large file upload
Universal "large file notification" removed.
Threshold for user-triggered large file notification is 5gb instead of 1gb.
Large file notification appears as "info" instead of "warning".
Display large file notification on failed uploads >1gb.

Issue:
https://github.com/storj/storj/issues/6149

Change-Id: I6ae6ba50e45c65f058cbb91a5f0fbda79a49a812
2023-08-09 23:39:15 +00:00
Wilfred Asomani
6b2fb9dfc4 web/satellite: use force delete bucket
This change modifies delete bucket requests to optin for force delete
using the x-minio-force-delete header. This solves an issue where
buckets deletes will be slow for buckets that have many objects. This
change is to make the UI emulate uplink rb --force since it has better
performance.

Issue: https://github.com/storj/customer-issues/issues/930

Change-Id: I0a74c1a201e74b07eb30b917adf78ef865ce2003
2023-08-09 23:05:50 +00:00
Vitalii
7d149dca0f web/satellite: use delete API keys http endpoint
This change uses the new /delete-by-ids endpoint in place of the GraphQL query.

Issue:
https://github.com/storj/storj/issues/6140

Change-Id: Ia78016e68fca296367e650b7a919130e662ee8f6
2023-08-09 22:32:47 +00:00
Wilfred Asomani
a88711319c web/satellite: use update project http endpoint
This change uses the update project HTTP endpoint in place of the
GraphQL alternative.

Issue: https://github.com/storj/storj/issues/6134

Change-Id: I7d8203c20d8c0c1dcdddb7d1aaf472fc66568a3d
2023-08-09 16:41:47 +00:00