Commit Graph

8088 Commits

Author SHA1 Message Date
Wilfred Asomani
676178299f satellite/db: fix long loadtime for charges endpoint
This change addresses an issue where the /charges endpoint will take a
while to respond due to a project having a large number of buckets.
The queries responsible for this have been merged into a single query to
get all data needed at a go and potentially improve performance.
Benchmarks indicate that this new way is more performant than the
previous.

name                                 old ms/op  new ms/op
Postgres/sum_all_partner_usages          7.534      0.622
Postgres/individual_partner_usages       6.104      0.588
Cockroach/sum_all_partner_usages        14.813      3.057
Cockroach/individual_partner_usages     16.046      2.852

Issue: https://github.com/storj/storj-private/issues/277

Change-Id: Ibb7f867ab6610b3cb1ba203961f7d6aef6bfda4a
2023-06-15 14:49:30 +00:00
Vitalii
2a4c1fa41e web/satellite: add delimiters to large numbers on project dashboard
Added delimiters to large numbers on project dashboard based on user location.
So in US and EU there should be different strings.

See: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/toLocaleString

Issue:
https://github.com/storj/storj/issues/5928

Change-Id: I94a0a703c2b301de422c5103e71becfbf5aa7e79
2023-06-15 12:49:01 +00:00
Moby von Briesen
3bc300ef55 satellite/console: optionally serve vuetify POC from satellite
This change slightly modifies the logic for serving the vuetify
frontend. After this change, if the config `console.use-vuetify-project`
is set to `true`, the Vuetify UI will be served at the `/vuetifypoc`
prefix. The POC is only project dashboard right now; the existing
login/registration pages must be used, but once the cookie is set, the
POC will work correctly.

Change-Id: I7725f23a0d2b04f274bab36d8be3370116687d1b
2023-06-15 12:16:31 +00:00
Egon Elbre
edbea5efe1 go.mod: bump to pgx/v5
Change-Id: I31cf3bec1d7db94f0f612f6ed04b782f8b04d876
2023-06-14 18:32:54 +03:00
Márton Elek
981dd4cbdc go.mod: bump dependencies (storj.io/private, storj.io/uplink)
Change-Id: Ie5a6a14a39786dd745116c3b6e4fdbc8e890d2c3
2023-06-14 16:36:45 +03:00
Michal Niewrzal
2e0b687581 satellite/metainfo: drop piecedeletion package
We are not using this package anymore.

Change-Id: If32315d43d73c8deb096e93cb43c03881bd9aad1
2023-06-14 11:19:21 +00:00
Michal Niewrzal
6c08d5024e satellite/metainfo: stop sending delete requests to SN
We decided that we will stop sending explicit delete requests to nodes
and we will cleanup deleted with GC instead.

https://github.com/storj/storj/issues/5888

Change-Id: I65a308cca6fb17e97e3ba85eb3212584c96a32cd
2023-06-14 10:46:02 +00:00
Wilfred Asomani
4a49bc4b65 satellite/payments: fix account freeze chore race condition
This change fixes an issue where a formerly warned/frozen user will be
warned again even though they have made payment for the invoice that got
them frozen in the first place. A payment status check is now made
right before a warn/freeze event to make sure the invoice hasn't been
paid already.

Issue: https://github.com/storj/storj/issues/5931

Change-Id: I3f6ac1e224f40107d58dc8f7bdbce58bbbea0196
2023-06-14 10:10:56 +00:00
Jeremy Wharton
6b65b7e7d0 satellite/{console,satellitedb}: clean up obsolete project member code
This change removes the obsolete project member paging code.
Previously, we implemented functionality for including project
invitations in pages of project members. However, the satellite
frontend still expected API responses to use the old paging style, so
the related code could not be removed right away. Now that the frontend
has been updated, this code is no longer necessary.

References #5855

Change-Id: I12fdaaeb869977c4d87a0d50b9a7b11c68552c82
2023-06-13 22:13:04 +00:00
Jeremy Wharton
8ee7d104a0 web/satellite: show project invitations in the Team page
The Team page now displays project member invitations alongside project
members. These invitations can be removed in the same manner that
members can.

References #5855

Change-Id: Iade690757d4430deee3378066d7dc19766f53936
2023-06-13 19:00:33 +00:00
Jeremy Wharton
ed37d72e3f web/satellite: change order of projects and invitations in dashboard
This change places project invitations ahead of projects in the cards
view of the All Projects Dashboard in order to be consistent with our
designs.

References #5855

Change-Id: I31268b97a9af2279bf90a171d0958b5a4bcbf13b
2023-06-13 11:34:55 -05:00
Wilfred Asomani
62c29ee9de web/satellite: use new invite functionality
This change uses the new project invite endpoint in place of the former
that adds invited users directly to a project's members. LoginArea
is updated to make the region/email from an invite email link uneditable.
VInput.vue's composition api code has also been updated to match other
components.

Issue: https://github.com/storj/storj/issues/5741

Change-Id: Ia3f82f5675fba442bb079dc8659b5396a25b9f03
2023-06-13 16:00:39 +00:00
Wilfred Asomani
09a7d23003 satellite/console: add endpoint to invite users to project
This change adds a new endpoint that uses the new project invite flow's
 functionality instead of directly adding users to a project's members.

Issue: https://github.com/storj/storj/issues/5741

Change-Id: I6734f7e95be07086387fb133d6bdfd95e47cf4d9
2023-06-13 15:22:25 +00:00
Vitalii
782811c634 web/satellite: check if AG name is free to use at the beginning of creation flow
By this change we check if provided access grant name is free to use at the very beginning of the creation flow.

Issue:
https://github.com/storj/storj/issues/5693

Change-Id: I06583bf458cea977cb0a920d55df50f2d19e1599
2023-06-13 14:24:16 +00:00
Vitalii
cc085553c2 satellite/{console, db}: endpoint to get all API key names by project ID
Added new endpoint, service method and DB query to get all API key names by provided project ID.

Issue:
https://github.com/storj/storj/issues/5693

Change-Id: I62e4e8ae660bd81234b75aa159a472a5aa9d5a48
2023-06-13 13:52:30 +00:00
Egon Elbre
00d233bca5 satellite/satellitedb: remove duplicate NullUUID
Change-Id: I0fa583140ab14448c26c602838aa035d8372b965
2023-06-13 15:34:41 +03:00
Jeremy Wharton
d39424993b satellite/console: only return unexpired project invitations to user
This change causes users to be served only project member invitations
that have not expired. If expired invitations are encountered during
processing of an invitation listing request, they will be removed.

References #5855

Change-Id: I6f621305f4f0a993953eb40a4dbd2375493f02e3
2023-06-13 09:42:03 +00:00
Moby von Briesen
8acb1ee5bf private/apigen: Support basic doc generation
Add some code to generate a basic markdown file documenting a generated
API. Generate this document for the API in
satellite/console/consoleweb/consoleapi/gen.

The documentation is not completely correct, as it may include some
values in the request body that are not actually usable by the
requester. This can be fixed by making sure all types used within the
generated API are properly annotated with `json` tags.

Issue: https://github.com/storj/storj-private/issues/244

Change-Id: I57b259967fb0db8f548b6598a10c825da15ba723
2023-06-13 08:48:06 +00:00
Jeremy Wharton
5b1c22a1e7 satellite/console/consoleweb/consoleql: include invites in member pages
Project member invitations may now be requested through GraphQL
queries. This is necessary for the satellite frontend to display
invitations in the Team page.

References #5855

Change-Id: Ibc8526ba768fd82c1b1890201004ef0f066df2fc
2023-06-13 07:48:07 +00:00
Jeremy Wharton
0cbb0ee12e satellite/console: delete project invitations with project members
The console service method responsible for removing members from a
project has been extended to remove project member invitations as well.
This will allow invitations to be deleted through the satellite
frontend.

References #5855

Change-Id: I90ca042cc6fb9a75fcd9b391e317caabb1c828f2
2023-06-13 07:15:43 +00:00
Jeremy Wharton
2ab6e00966 satellite/satellitedb: add method to return project members and invites
A method has been implemented that allows for paged searching through
project members and project member invitations. In the future, this
will be used to display invitations in the Team page of the satellite
frontend.

References #5855

Change-Id: I0937c425f60f1318e55202bf30b44a33ff695414
2023-06-12 17:28:50 -05:00
Wilfred Asomani
3d1e429156 web/satellite: properly style all projects dashboard
This change matches the all projects dashboard styling to the figma as
closely as currently possible.

Issue: https://github.com/storj/storj/issues/5934

Change-Id: I93edd898fbcee954265737052a1967d318370924
2023-06-12 18:38:30 +00:00
Wilfred Asomani
8cdc5bd107 web/satellite: add invites to all projects dashboard table
This change adds projects invites to the projects table.

Related: https://github.com/storj/storj/issues/5925

Change-Id: I82fbf1e47ea383377369005956a9eec3c8ac6416
2023-06-12 18:07:37 +00:00
Moby von Briesen
771ec3237a satellite/console: Update CSP to include storjsatelliteshare.io
Change-Id: I8238f6e2644a0f4ee727d0b8da56e59733dd3252
2023-06-12 13:31:00 -04:00
Jeremy Wharton
c8f31f1649 web/satellite: send analytics events for project invitation responses
This change causes the satellite frontend to send analytics events for
project invitation responses. In addition, it more properly sources
reported project invitation response errors.

References #5855

Change-Id: I374e064c39920d3af829cb8467790d864a417115
2023-06-12 16:40:05 +00:00
Jeremy Wharton
acdc9f3608 satellite/analytics: add events for project member invitation responses
This change adds analytics events for accepting and rejecting project
member invitations.

References #5855

Change-Id: I2101eb711312a683ecdb4b7d0c6d7cc3ae09c440
2023-06-12 16:07:32 +00:00
Moby von Briesen
62e3b2cfe6 web/satellite: enable customers to purchase packages after onboarding
This change allows users who are eligible to purchase a package plan to
do so, even if they missed the opportunity during onboarding.

Now, if the user is eligible, an opportunity to select a package for purchase
is presented as part of the upgrade modal.

Issue: https://github.com/storj/storj/issues/5932

Change-Id: I45575274839701bf7b80815330a4ae86a1d32093
2023-06-12 15:35:41 +00:00
Jeremy Wharton
bb620e746b satellite/satellitedb: add index for project_members project_id column
An index has been added on the project_id column of the project_members
satellite database table so that we can retrieve members of a project
without performing a full table scan.

References #5855

Change-Id: I1cc30686f836c8fd1aa319247ce857a2392e7a52
2023-06-11 18:12:33 -05:00
Michal Niewrzal
7c33521ace satellite/repair/repairer: use placement to select nodes for repair upload
We missed to set placement as a part of selection request. It can case
uploading repaired data out of specified placement.

I will provide test as a separate change.

Change-Id: I4efe67f2d5f545a1d70e831e5d297f0977a4eed1
2023-06-10 20:55:39 +02:00
JT Olio
3fff61f04a payments: don't redefine compensation rates twice
Change-Id: Ic00abe3795a000d4f0284c99f270180123a2f663
2023-06-10 12:40:43 +00:00
Vitalii
e83e98e46a web/satellite: vuetify POC
Built side Vuetify subproject inside web/satellite with limited functinality.
For now it has navigation side bar, simple project dashboard and team page (where you can list/add team members).

Issue:
https://github.com/storj/storj/issues/5854

Change-Id: I9ff3e80b8ace1dc31de6a788174c5ffc19f050f8
2023-06-10 00:59:40 +00:00
Jeff Wendling
f3c58174c4 cmd/uplink: only use new code path for uploads
downloads still need the old copy code because they aren't
parallel in the same way uploads are. revert all the code
that removed the parallel copy, only use the non-parallel
copy for uploads, and add back the parallelism and chunk
size flags and have them set the maximum concurrent pieces
flags to values based on each other when only one is set
for backwards compatibility.

mostly reverts 54ef1c8ca2

Change-Id: I8b5f62bf18a6548fa60865c6c61b5f34fbcec14c
2023-06-09 23:45:30 +00:00
Egon Elbre
f9c076abcf Jenkinsfile: fix building with musl
Change-Id: I6318d19646cdd377de1aca0e56584313c2e5b73c
2023-06-09 15:57:52 +00:00
Wilfred Asomani
8f7c59de84 satellite/web: hide project details link for project members
This change hides the project details link, introduced for the issue
linked below, if the user is only a member and not the owner of the
project.

Related: https://github.com/storj/storj/issues/5930

Change-Id: If6b0817dfa32f2f661629cc158eb4fb44dfe9ffc
2023-06-09 15:25:11 +00:00
Moby von Briesen
8a01bcfb2b web/satellite: Move pricing plan files
Move "common" pricing plan files from components/onboardingTour to
components/account/billing.

This way, the file imports make more sense when pricing plan info is
used outside of the onboarding flow.

Related to  https://github.com/storj/storj/issues/5932

Change-Id: I94f5f561c88c3a8f0046065dfa1a708c062faa93
2023-06-09 14:53:03 +00:00
Vitalii
8c1f07b73e web/satellite: fix download in object browser
Fixed download objects in object browser.

Context:
aws-sdk v3 getSignedURL method now returns a Promise<string> instead of regular string. So we have to await for promise to resolve

Issue:
https://github.com/storj/storj/issues/5956

Change-Id: I2431095e7e8cd1bbc4e866c6958a2c03898c4b4d
2023-06-09 13:54:04 +00:00
Michal Niewrzal
61933bc6f0 satellite/{metainfo,metabase}: optimize expired/zombie objects deletion
* optimize SQL for zombie objects deletion query by reducing some direct
selects to segments table
* set AOST for expired/zombie object deletion (was 0 since now)

https://github.com/storj/storj/issues/5881

Change-Id: I50482151d056a86fe0e31678a463f413d410759d
2023-06-09 11:22:46 +00:00
Michal Niewrzal
aea3baf6a9 satellite/accounting/tally: calculate pending object count
Change-Id: I4ee6072f4c60fafd809e8184ada9c1abf7edd8aa
2023-06-09 10:15:27 +00:00
Jeff Wendling
c559df6a1b all: bump storj.io/uplink
this fixes a memory leak in the upload codepath

Change-Id: I6bf6214d331cbe4e8e8912942035ef01a285e3b4
2023-06-08 19:52:24 +00:00
Wilfred Asomani
6ca9f6d2c6 web/satellite: add projects table to all projects dashboard
This change adds a tableview to all projects dashboard. It also
addresses some minor UI issues.

Issues: https://github.com/storj/storj/issues/5925
https://github.com/storj/storj/issues/5924
https://github.com/storj/storj/issues/5923

Change-Id: Ic18566b192005bb1720e0388352bc82a3739a723
2023-06-08 17:14:05 +00:00
Wilfred Asomani
7422fe393b satellite/db: add memberCount to project by userID
This change modifies the projects.GetByUserID function to also fetch
each project's member count.

Change-Id: I5ef4aa3726566920ee9360f5a4f09a8cb490c67e
2023-06-08 17:05:28 +00:00
Márton Elek
3e3ec2358d web/storagenode: zksync era compatible links wallet/transactions
Change-Id: I9079ca20e4d95abedbcf04c0d8529a0f2c6ac938
2023-06-08 13:47:02 +00:00
Egon Elbre
df53914faa satellite/satellitedb: use utilities for conversions
This avoids some potential typos.

Change-Id: Icc5262e1f96fe220dd07212c00acacf6960ee909
2023-06-08 16:13:47 +03:00
Márton Elek
974b4f938a go.mod: bump storj.io/private
Change-Id: I2d130cf2e794c8e5e0a978433cb3d504e6747af0
2023-06-08 12:01:55 +00:00
Márton Elek
e4b8febda7 satellite/metainfo: use default_placement of project during bucket creation
https://github.com/storj/storj/issues/5879

Change-Id: I100e834b3b625b3e0ff00cdcb7675ec6bb570992
2023-06-08 11:21:42 +00:00
Márton Elek
2ea6ca9c18
satellite/console: fill default placement of project based on user info
https://github.com/storj/storj/issues/5879

Change-Id: I5aacbe57a7aab65e11915dd8bf0c68f89da32fb4
2023-06-08 12:47:49 +02:00
Márton Elek
5c26661cd3 dbx/satellitedb: introduce new default_placement fields for users/projects
https://github.com/storj/storj/issues/5879

Change-Id: Ie6d50fd94c09f43c12f895444ed2a86f0820e01c
2023-06-08 10:02:10 +00:00
Wilfred Asomani
3980fcc39b web/satellite: add project details link to dashboard nav area
This change adds a link to the project dashboard nav area to navigate
to the selected project's detail page.

Issue: https://github.com/storj/storj/issues/5930

Change-Id: Ief26505feeb62e88b8e6096d4be70004dc068b64
2023-06-08 09:29:36 +00:00
Michal Niewrzal
f7c7851519 satellite/metainfo: filter metainfo.GetObjectIPs by bucket/object placement
For now we will use bucket placement to determine if we should exclude
some node IPs from metainfo.GetObjectIPs results. Bucket placement is
retrieved directly from DB in parallel to metabase
GetStreamPieceCountByNodeID request.

GetObjectIPs is not heavily used so additional request to DB shouldn't
be a problem for now.

https://github.com/storj/storj/issues/5950

Change-Id: Idf58b1cfbcd1afff5f23868ba2f71ce239f42439
2023-06-07 16:52:02 +00:00
Márton Elek
a463dd0c71 build: use go 1.19 for nighlty containers
Change-Id: I56d23cf797a893685df9014a12affa84687330a9
2023-06-07 16:08:05 +00:00