Commit Graph

8475 Commits

Author SHA1 Message Date
Andrew Harding
e4d6829971 private/server: Windows TCP fast-open support detection
Implements tryEnableFastOpen by creating a localhost socket and enabling
the TCP_FASTOPEN socket option. On builds where TCP_FASTOPEN isn't
available, setting the socket option fails and tryEnableFastOpen returns
false.

This was verified on a supporting build (latest Windows 10) and an
unsupporting build (Windows 8.1, couldn't find an ISO for an older
Windows 10 build).

Change-Id: I497117dc2f04acdd2b0cc836e20d12d69076b939
2023-09-18 14:15:30 +00:00
Vitalii
ecc527ad3b web/satellite: low STORJ balance notification
Added a new banner to the dashboard of all projects. This banner is displayed when a user meets the following conditions: they have no credit cards on file, they have a payment history with tokens, and their estimated charges are higher than their current balance.

Issue:
https://github.com/storj/storj/issues/6234

Change-Id: I1f90ae81032d459111b111d23ce2e1d8119e649d
2023-09-18 16:30:59 +03:00
Egon Elbre
bc517cae2f go.mod: bump common
This brings in common/grant that doesn't depend on protobuf anymore.
This ends up causing the console wasm bundle from ~11MB to ~4.7MB.

Change-Id: I145dcb9239952a7a9e352c8793c111acb61ff0cc
2023-09-15 17:43:44 +00:00
Jeremy Wharton
8a0d1e8b55 web/satellite/vuetify-poc: show region in My Account dropdown
This change displays the satellite's region in the My Account dropdown
menu of the Vuetify project.

Resolves #6279

Change-Id: Ib7d739c0104fb20ad7ee23234c66813b7f37b3a5
2023-09-15 16:31:21 +00:00
Wilfred Asomani
1efc0ceaa5 web/satellite: load usage and charges separately
This change separates the loading of usage and charges from other API
calls since that may take long. This will allow the other data to be
displayed while usage and charges are waited on.

Issue: https://github.com/storj/storj/issues/6259

Change-Id: I4a8d8f911baf432d6f1e9eee49176480197ae3ca
2023-09-15 15:50:02 +00:00
Jeremy Wharton
789b37c21f web/satellite/vuetify-poc: remove notification on close button click
This change fixes an issue where a notification would not be completely
removed after clicking the close button, causing it to invisibly prevent
elements beneath it from being clicked. Also, the file preview dialog no
longer closes when a notification is clicked.

Resolves #6280

Change-Id: I135aa2e77ddc1ec845101209f26dde5e48f10bd6
2023-09-15 15:10:48 +00:00
Vitalii
bd36749f7d web/satellite: improve bucket list on small screens
Bucket list in vuetify never requires horizontal scrolling to view the entire row.
Removed non-essential columns from the bucket list when the screen size is small.
Bucket name is the only "essential" piece of information that must always be shown.

Issue:
https://github.com/storj/storj/issues/6232

Change-Id: Id4bee3100f6d4ca112670d2f68bd63ff2dc266e9
2023-09-15 17:28:40 +03:00
Michal Niewrzal
975d953cb8 satellite/metabase: custom error for commit object
We stoped returning lots of errors as is to avoid leaking our internals
but some errors were meanigful for client. Example of such error is
"exceeded maximum number of parts". With this change we are wrapping
some important commit object errors with new ErrFailedPrecondition
error to be able to return it easily to uplink.

Change-Id: Id834b78362ed1920f0c3f6f1c7d9587bfd27e36a
2023-09-15 09:52:42 +00:00
Márton Elek
f4fe983b1e satellite/{placement,nodeselection}: introduce empty() and notEmpty() for tag value selection
It helps to implement rules like `tag("nodeid","select",notEmpty())

Change-Id: If7a4532eacc0e4e670ffe81d504aab9d5b34302f
2023-09-14 19:30:29 +00:00
Vitalii
92a69c7de4 web/satellite: add loader to object browser table
This is a fix based on early feedback from QA team.
Added loader to object browser table so that user can't change pages while request is still in progress because it breaks pagination.

Change-Id: I5cc2ff057955478b3c745c169d520e1a639eff92
2023-09-14 18:50:51 +00:00
Wilfred Asomani
bd48a5cbe6 web/satellite: allow limit increase request
This change implements a modal to request limit increases.

Issue: https://github.com/storj/storj/issues/6233

Change-Id: I40ace89a79c65751547d804e8d190e866217d379
2023-09-14 18:08:56 +00:00
Jeremy Wharton
0f3ff66485 web/satellite: show error for rate limited MFA logins
This change fixes an issue where errors were not displayed for login
attempts that failed due to rate limiting.

Change-Id: Ia3c7fccf434ad62bb252f4215676b1f32903ac53
2023-09-14 15:20:32 +00:00
Michal Niewrzal
881137539c satellite/metabase: commit object is not respecting expiration time
With pending_objects table support enabled we missed passing correctly
expiration time from pending object to committed object. This change
updates commit query to take into account expiration time.

Change-Id: I67146d5b2f7f0bda02925d16275fbc59acb705bd
2023-09-14 16:25:28 +02:00
Vitalii
8f27425284 satellite/{projectaccounting, web}: merge settled and allocated lines for bandwidth graph
Merged bandwidth graph lines to show only allocated-dead for last 3 days and settled for other days.

Issue:
https://github.com/storj/storj/issues/6072

Change-Id: Ic7f03d22ccd82d27ae6e6a85e73e144c9852e33b
2023-09-14 15:39:56 +03:00
Jeremy Wharton
ec8f3b4528 web/satellite/vuetify-poc: allow for sharing files and folders
This change allows linksharing URLs to be generated for files and
folders within the Vuetify project's file browser.

Resolves #6111

Change-Id: I8cbe81b33cb5e35de0c34bba8ccc9175c727bd94
2023-09-14 11:23:08 +00:00
Michal Niewrzal
8f1682941e mod: bump storj/uplink to v1.12.0
Change-Id: Ia58de16d4ac43c7b888000ad8ca85a979496a2cb
2023-09-14 09:20:38 +02:00
Wilfred Asomani
ccb9b7ae8e satellite/web/vuetify-poc: make tables consistent
This change makes tables in the vuetify app more consistent. Also
clearing search has been fixed for tables whose data would not populate
after search has been cleared.

Issue: https://github.com/storj/storj/issues/6267

Change-Id: I053d9e5f23662774c60d67a29f814a2c1c3067ed
2023-09-13 23:15:28 +00:00
Clement Sam
f14fabc90a cmd/storagenode: add forget-satellite subcommand
This change adds a new forget-satellite sub-command to
the storagenode CLI which cleans up untrusted satellite
data.

Issue: https://github.com/storj/storj/issues/6068
Change-Id: Iafa109fdc98afdba7582f568a61c22222da65f02
2023-09-13 19:06:55 +00:00
Wilfred Asomani
dcf3f25f93 satellite/admin: update README
add descriptions for the endpoint that removes a user from the waning
state.

Issue: https://github.com/storj/storj/issues/6118

Change-Id: I211cd3c41c7fefa295d0db1b9f43f53e33b984e6
2023-09-13 17:50:28 +00:00
Jeremy Wharton
7d8b231aaf cmd/satellite: build Vuetify project in Dockerfile
This change updates the satellite Dockerfile to build the Vuetify
project.

References #6251

Change-Id: I699360c0f7eb7a8abdd0bc523ee74910a04fecd3
2023-09-13 12:43:58 +00:00
Michal Niewrzal
fb04a22088 satellite/metainfo: flag to rollout pending_objects table support
To avoid enabling feature for every project at once we would like to
do this partially and control percentage of projects that will have
feature enabled.

https://github.com/storj/storj/issues/6258

Change-Id: Iaac7c42d39da76ed2ecc439847c3b210462befa5
2023-09-13 10:33:23 +00:00
Egon Elbre
d91ee440ba satellite/metabase: use constant for pending version
Currently it wasn't quite clear what was a stub version and an actual
version. Use a PendingVersion constant to make this distinction clear.

Also use PendingVersion = NextVersion = 0, that way it's clearer that
the version hasn't been yet determined. DefaultVersion = 1 might imply
that the object will get that version once commited, however that will
entirely depend on whether use-pending-objects is used or versioning is
enabled or not.

Change-Id: I21398141f97035c48c778f23b542266b834c44f1
2023-09-12 18:01:12 +00:00
Jeremy Wharton
4f8697568d web/satellite: use edge service URL overrides
This change makes the satellite frontend use edge service URL overrides
if they have been configured for a project.

Resolves #6188
Resolves #6190

Change-Id: I4c8fb3f5f00f450fb8cd139383972ab622234fb0
2023-09-12 12:19:20 -05:00
Jeremy Wharton
c8f4f5210d satellite/console: return edge URL overrides in project info responses
API responses containing project information now contain the edge
service URL overrides configured for that project. The overrides are
based on the project's default placement.

References #6188

Change-Id: Ifc3dc74e75c0f5daf0419ac3be184415c65b202e
2023-09-12 12:10:18 -05:00
Clement Sam
89d682f49f satellite/metainfo: prevent internal DB errors in Public API
Resolves https://github.com/storj/storj/issues/6081

Change-Id: I0e530db39947138dcafc1b6bd1710ff1ca96b8c5
2023-09-12 15:12:44 +00:00
Michal Niewrzal
e21978f11a satellite/metainfo: support getting specific object version
Protobuf definition is read to support getting specific version of
object so we just need to wire requested version into metainfo.GetObject
endpoint.

https://github.com/storj/storj/issues/6221

Change-Id: If4568b82119a6c893788a0a86e598b05ff5951cf
2023-09-12 14:27:40 +00:00
Vitalii
4e4da7be6d web/satellite/vuetify-poc: always show details container under validatable inputs
Inputs that have some custom validation messaging attached should not use hide-details=auto to prevent 'jumpy' visual experience.

Issue:
https://github.com/storj/storj/issues/6230

Change-Id: Ia90e122516eb853a3908c0f57634971243fb38b3
2023-09-12 13:47:22 +00:00
Jeremy Wharton
088496efdf web/satellite/vuetify-poc: allow for deleting files and folders
This change allows files and folders to be deleted from within the
Vuetify project's file browser.

Resolves #6106

Change-Id: I0d7b0528b08333aeec29917c4ebef6ea966ac1fa
2023-09-12 13:08:09 +00:00
Egon Elbre
28ee6f024c satellite: don't use fmt.Print in tests
Change-Id: Ia10450240ad075c9d78614adff9164f292fb1fa0
2023-09-12 15:11:02 +03:00
Márton Elek
afa5c54a35 satellite/satellitedb: add placement column to repair_queue
It makes it possible to run dedicated repair worker for different placement definitions.

https://github.com/storj/storj-private/issues/400

Change-Id: I376da867da5dbb4ab392d5f86c766f7543c32ee6
2023-09-12 11:29:18 +00:00
Vitalii
2cf233ac23 web/satellite/vuetify-poc: hide nav sidebar by default for screens less than 1280px
On screen sizes where the sidebar overlaps with the content of the page, the sidebar should be collapsed by default.

Issue:
https://github.com/storj/storj/issues/6229

Change-Id: Ia0a91acd95519de27f9ff8f1ee90c6b8e7932266
2023-09-12 13:48:53 +03:00
Cameron
f9ab2f0de7 satellite/console: prefix oidc paths with /api/v0/
add /api/v0/ prefix to oidc paths to route requests to the api
pod with split UI infrastructure.

Change-Id: I1e7e691487ab1d4e84434d204d10b7f944ae8873
2023-09-11 18:12:39 +00:00
Egon Elbre
487f64e164 satellite/satellitedb,multinode/multinodedb: update to latest dbx
Change-Id: I500df6d0541706c3960d4560721c3783d0d049ff
2023-09-11 17:21:02 +00:00
Egon Elbre
8edb9c5f98 multinode/multinodedb: don't generate rx
Change-Id: Ica1a2bb1ab42e881daa951ab61b1dd99e877aff5
2023-09-11 16:40:52 +00:00
Egon Elbre
87bfb3b02b satellite/satellitedb: don't generate rx
I'm not sure what Rx is, however, we aren't using it --
so let's remove it for now.

Change-Id: I9caacbc150479f93945477101528a4fd60ea865f
2023-09-11 16:00:15 +00:00
Egon Elbre
3e73d414d1 satellite/console/consoleweb: initialize mime lazily
Change-Id: I80b78edcf057acef9b5a599cb77308baddc07692
2023-09-11 15:19:56 +00:00
Michal Niewrzal
0a3ee6ff8a satellite/metabase: remove old object segments on overwrite
While adding support for pending_objects table one case was missed.
When we are uploading object to location where old objects exists
we are not removing old object segments at all. Old object is
overwritten with new object metadata but segments remains without
corresponding object. This fix removes all existing committed objects
(with it's segments) before committing new object.

https://github.com/storj/storj/issues/6255

Change-Id: Id657840edf763fd6aec8191788d819191b074fb7
2023-09-11 14:16:47 +00:00
Vitalii
c31fb9c1cf satellite/payments: restrict addition of duplicate credit cards
By this change we don't allow users to add credit cards that are already bind to their account.
We still allow the same CC number but with a different expiration date.

Issue:
https://github.com/storj/storj/issues/5597

Change-Id: Ifeb0cc5ae0c2f0f7596af4dead70ae7d20d30613
2023-09-11 13:24:43 +03:00
Kaloyan Raev
3119b614ae cmd/uplink: --max-object-ttl flag for share and access restrict commands
Context: https://github.com/storj/storj/issues/6249

Change-Id: Ic65a1d8aef61f1a88752a7b12a23fb854dac8f6d
2023-09-11 08:14:55 +00:00
Vitalii
9254dd2208 web/satellite/vuetify-poc: enable object browser pagination
Enabled object browser pagination for vuetify app.
Also fixed some small bug when returning to first page.

Issue:
https://github.com/storj/storj/issues/5595

Change-Id: I8b5e90a4cd7d7a79a8beeb292b7374db3f93d700
2023-09-09 00:36:55 +00:00
Wilfred Asomani
2bf4113821 web/satellite/veutify-poc: create folder
This change adds the ability to create folders in the file browser.

Issue: https://github.com/storj/storj/issues/6105

Change-Id: I0dae0f9874b571cfd0ae79b2b994b58149d70aa3
2023-09-08 23:57:13 +00:00
Wilfred Asomani
54379fc0ee satellite/{console,analytics}: allow limit increase request
This change adds a  new endpoint that submits limit increase requests
to segment.

Issue: https://github.com/storj/storj/issues/6233

Change-Id: Ie4f70aef31079acbe2f24771b3ea359d5769eb95
2023-09-08 23:17:38 +00:00
Wilfred Asomani
cd7d9cf079 satellite/{console,payments}: unfreeze user on token payment
This change unfreezes/unwarns users in either state after successful
payment with tokens.

Change-Id: I7494d6a33cf9383576f42af695df522d8f409e03
2023-09-08 12:51:01 +00:00
Cameron
c52554a2b9 web/satellite: vuetify upload file and folder
add functionality to upload files and folders in object browser

issue: https://github.com/storj/storj/issues/6101
issue: https://github.com/storj/storj/issues/6102

Change-Id: I4aa86b89adc051b91b0d7fde69dd7375b2a3f370
2023-09-08 11:47:01 +00:00
Kaloyan Raev
4e499fb9bf satellite/metainfo: respect MaxObjectTTL in BeginObject
If MaxObjectTTL is set in the API key, BeginObject will use it for the
object expiration time, unless an explicit ExpireAt is available in the
request.

Context: https://github.com/storj/storj/issues/6249

Change-Id: I2adf57d979a9c68eec3a787f3739d2f1dbec1f7e
2023-09-08 09:30:48 +00:00
Moby von Briesen
8d1a765fd6 satellite/console: Partially revert change to remove graphql
This partially reverts commit 516241e406.

Endpoints are added to the backend, as there are some customers who may
use these endpoints, even though they are no longer necessary for the
satellite UI.

Change-Id: I52a99912d9eacf269fbb2ddca603e53c4af6d6bf
2023-09-07 20:50:24 +00:00
Wilfred Asomani
754bf5f8af Revert "satellite/db: optimize project usage query"
This reverts commit 31ec421299.

This change made the usages endpoint slower for accounts with large
number of projects.

Change-Id: I95870e95c2bf3bc3050087532fd0d20cbb50748b
2023-09-07 19:27:08 +00:00
Michal Niewrzal
df037564d7 satellite/zombiedeletion: remove inactive uploads from pending_objects
With zombie deletion chore we are removing inactive pending objects from
objects table but new we need also to do this for pending_objects table.

https://github.com/storj/storj/issues/6050

Change-Id: Ia29116c103673a1d9e10c2f16654022572210a8a
2023-09-07 18:47:29 +00:00
Márton Elek
ad87d1de74
satellite/satellitedb/overlaycache: fill node tags with join for limited number of nodes
The easiest way to get node information WITH node tags is executing two queries:

 1. select all nodes
 2. select all tags

And we can pair them with a loop, using the in-memory data structures.

But this approach does work only, if we select all nodes, which is true when we use cache (upload, download, repair checker).

But repair process selects only the required nodes, where this approach is suboptimal. (full table scan for all tags, even if we need only tags for a few dozens nodes).

Possible solutions:

 1. We can introduce a cache for repair (similar to upload cache)
 2. Or we can select both node and tag information with one query (join).

This patch implements the second approach.

Note: repair itself is quite slow (10-20 seconds per segements to repair). With 15 seconds execution time and 3 minutes cache staleness, we would use the cache only 12 times per worker. Probably we don't need cache for now.

https://github.com/storj/storj/issues/6198

Change-Id: I0364d94306e9815a1c280b71e843b8f504e3d870
2023-09-07 19:27:53 +02:00
Kaloyan Raev
82b108de69 satellite/console/consolewasm: no direct cast to grant.Permission
A new field is introduced to grant.Permission in storj.io/common. Having
a direct cast here leads to compilation problems when bumping
storj.io/uplink to the latest storj.io/common. Avoiding the direct cast
resolves the issue.

Context: https://github.com/storj/storj/issues/6249

Change-Id: I3b9bc14ebcce8e192e218c621b996300753b8de4
2023-09-07 13:27:53 +03:00