Commit Graph

8775 Commits

Author SHA1 Message Date
Wilfred Asomani
c9421d11e7 satellite/admin: add endpoints to violation freeze/unfreeze users
This change adds two new admin endpoints to freeze users for ToS
violation and to remove them from that state,

Issue: https://github.com/storj/storj-private/issues/386

Change-Id: I49c922377c9cdb315ce2777fcd35dcad432b0539
2023-10-13 10:43:44 +00:00
Wilfred Asomani
594e63f13a satellite/accountfreeze: mark billing frozen users for deletion
This change modifies the billing freeze chore to set pending deletion
status to users who are still frozen after a grace period.
It also modifies the chore to skip already deleted users.

Issue: https://github.com/storj/storj/issues/6303
https://github.com/storj/storj-private/issues/453

Change-Id: I4d0e7dd904463e99424372dc9ac81b71c0bc6e28
2023-10-12 20:25:23 +00:00
Wilfred Asomani
ee2b6e66de web/satellite/vuetify-poc: improve team page
This change makes minor improvements to the vuetify team page.

Issue: https://github.com/storj/storj/issues/6376

Change-Id: Iedd0e1f47d8075f13b424da65d719057eade14ed
2023-10-12 17:49:55 +00:00
Márton Elek
db3578d9ba satellite: durability rangeloop observer for monitoring risks
Change-Id: I92805fcc6e7c1bbe0f42bbf849d22f9908fedadb
2023-10-12 16:32:30 +00:00
Tome Boshevski
863c96b771
web/satellite: update signup pages plus logo (#6409)
* web/satellite: update signup pages plus logo

Update designs of the signup, login, forgot password, reset password, confirm email pages. Make Business as default on signup page per marketing request. Plus update the logo to remove DCS.
2023-10-12 18:05:44 +02:00
Vitalii
f41e117918 web/satellite/vuetify-poc: fix object browser pagination inside folders
Reflects the same changes made for main app.
https://review.dev.storj.io/c/storj/storj/+/11313

Change-Id: I8e0a2a64b1adfdf411c3044890e85928d592259d
2023-10-12 14:35:09 +00:00
Clement Sam
db7c6d38e5 storagenode/orders: archive unsent order for untrusted satellite
The order service still tries to settle orders at all instances even
when the satellite is marked as untrusted by the trust service, which
will always fail because the trust cache no longer has record of the
URL of the satellite, and it will keep retrying.

This leaves a lot of "satellite is untrusted" errors in the logs.
There has been several complaints on the forum because this was
happening a lot for the stefanlite and I expect it will be the
same issue for the decommisioned satellites US2 and EUN-1 once
the forget-satellite command is run to clean up the satellite.

This change allows the order service to archive unsent orders available
for any untrusted satellite, and will not attempt to settle the order.

https://github.com/storj/storj/issues/6262

Change-Id: If0f7f1783587cd18fab8917d45948f22df5b1dcf
2023-10-12 13:48:07 +00:00
Jeremy Wharton
4e0f062cb5 web/satellite/vuetify-poc: support PDF previewing
This change allows supported platforms to preview PDFs in the object
browser.

Resolves #6397

Change-Id: I6078914f0ddf5f620514e27e51d24ba8c11a2786
2023-10-12 12:00:59 +00:00
Vitalii
7038ddef7f web/satellite/vuetify-poc: fixed styling of stripe input
Made stripe input look more fancy on Payment Methods tab of Billing screen.

Issue:
https://github.com/storj/storj/issues/6399

Change-Id: I69497191b5c7a93905cdd4660ef733f17b4a2854
2023-10-12 13:44:15 +03:00
Michal Niewrzal
e3e303754b satellite/repair/checker: optimize processing, part 2
Optimizing collecting monkit metrics:
* initialize metrics once at the begining
* avoid using string in map for getting stats structs per redundancy

Benchmark results (compared against part 1 optimization change):
name                                       old time/op    new time/op    delta
RemoteSegment/Cockroach/healthy_segment-8    31.4µs ± 6%    21.7µs ± 8%  -30.73%  (p=0.008 n=5+5)

name                                       old alloc/op   new alloc/op   delta
RemoteSegment/healthy_segment-8    10.2kB ± 0%     7.4kB ± 0%  -27.03%  (p=0.008 n=5+5)

name                                       old allocs/op  new allocs/op  delta
RemoteSegment/healthy_segment-8       250 ± 0%       150 ± 0%  -40.00%  (p=0.008 n=5+5)

Change-Id: Ie09476eb469a4d6c09e52550c8ba92b3b4b34271
2023-10-12 10:02:53 +02:00
Michal Niewrzal
de4559d862 satellite/repair/checker: optimize processing, part 1
Optimization by reusing more slices.

Benchmark result:
name                                       old time/op    new time/op    delta
RemoteSegment/healthy_segment-8    33.2µs ± 1%    31.4µs ± 6%   -5.49%  (p=0.032 n=4+5)

name                                       old alloc/op   new alloc/op   delta
RemoteSegment/healthy_segment-8    15.9kB ± 0%    10.2kB ± 0%  -35.92%  (p=0.008 n=5+5)

name                                       old allocs/op  new allocs/op  delta
RemoteSegment/healthy_segment-8       280 ± 0%       250 ± 0%  -10.71%  (p=0.008 n=5+5)

Change-Id: I60462169285462dee6cd16d4f4ce1f30fb6cdfdf
2023-10-11 15:50:29 +00:00
Artur M. Wolff
3019471514 {Jenkinsfile, Makefile}: update Go to 1.21.3
Change-Id: Ia014ff4fa622903431789d0dc7e93044015ecaa9
2023-10-11 10:47:04 +02:00
Márton Elek
9186365507 storagenode/pieces: more granular io and hashing statistic
This patch adds two new monkit metric:
 * piece_writer_io: the sum of the time, which is spent with io.Write during a piece upload (excluding the fs sync of the commit)
 * piece_writer_hash: the sum of the time, which is spent with hashing

The second is especially important. My storagenode (hosted on a cloud server) spend ~30 ms on hasing data, piece_write_io time is usually 5ms for me.

These metrics can help us to identify the reason of slownes on storagenode sides.

Both of these depend on the size of the piece. To make it more meaningfull without exploding the cardinality, I created a few size categories and classified the pieces based on these. Measurements shows that it can provide usefull results (>2MB uploads are usually 23-28 ms).

Change-Id: Ifa1c205a490046655bcc34891003e7b43ed9c0bc
2023-10-11 07:33:04 +00:00
Wilfred Asomani
6308da2cc0 satellite/{payment,console,analytics} extend freeze functionality for violation freeze
This change extends the account freeze functionality account for
violation freezes as well.
Also, debug level logs in the freeze chore have been changed to info.
It adds an analytics event for when an invoice is found that belongs to
a user frozen for violation.
And finally adds whether a user is frozen for violation to the
/account/freezestatus response.

Issue: https://github.com/storj/storj-private/issues/386

Change-Id: Id8e40282dc8fd8f242da52791ab8ddbbef3da2bc
2023-10-10 18:39:29 +00:00
Tome Boshevski
211659b9b0
web/satellite: update and optimize error page (#6391)
Use 3x smaller size background image and update styles. Update button.
2023-10-10 18:02:27 +02:00
Ivan Fraixedes
386a978310 private/apigen: Validate param types
The Go generator only supports certain types as query and path
parameters and it panics on any an unsupported type.

The Document and TypeScript generator don't have any validation for
them. TypeScript generator generates code that compiles, however, it
won't work properly with all the types not supported by the Go
generator.

Because it doesn't make sense that some types may work on the TypeScript
generator, while the Go generator doesn't, doing the validation in the
Param constructor is better because it reports the issue without having
to run the Go generator and it gives a more understanding panic message.

TypeScript generator generates code that works properly with all the
types supported by the Go generator, hence, there isn't any change int
he TypeScript generator in this commit.

Change-Id: I03085283942a98341726a1560f511a46540df9f5
2023-10-10 12:37:20 +00:00
Vitalii
0db898b0a8 web/satellite/vuetify-poc: tiny changes for project dashboard
Download card: added 'per month' to Limit value.
Coupon card: replaced 'Limit' label with 'Included free usage'.
Also, fixed chart tooltip positioning.

Issue:
https://github.com/storj/storj/issues/6378

Change-Id: I6a4eb544cecaeabab6856acf6e272784f2a1c802
2023-10-10 11:41:28 +00:00
Jeremy Wharton
b6b9cccb72 web/satellite/vuetify-poc: object browser improvements
- Object preview can be opened through the object row actions menu.
- Previewed objects no longer shift vertically when transitioning.
- Previewed objects no longer display the next object's preview when
  transitioning.
- The download notification text has been changed from "Success" to
  "Download Started".
- Clicking object browser breadcrumbs no longer redirects to the
  all projects dashboard.
- Upload progress snackbar:
  - The expanded and collapsed widths are now the same.
  - Clicking an item opens the object preview for it.
  - The "Uploading" tooltip position has been moved to the left so that
    it doesn't block the cancel button.

Resolves #6379

Change-Id: Ic1f5cc7948ffa62dc0bce488b61f6d5e121c77b9
2023-10-10 08:32:27 +00:00
Jeremy Wharton
99ba88ae2f web/satellite/vuetify-poc: allow adding coupons
This change allows users to apply coupon codes via the billing
page's overview section.

Resolves #6393

Change-Id: I6f973f5293362efa8cd76149dc1b43468b49b2c9
2023-10-09 22:53:10 +00:00
Jeremy Wharton
dd3779a623 web/satellite/vuetify-poc: invite through all projects dashboard
This change allows project owners to invite members through the all
projects dashboard.

Resolves #6394

Change-Id: Id36d21432ab7b18532679e900d3e00c52fa21fc9
2023-10-09 20:32:54 +00:00
Cameron
0cc04208db satellite/console: add limit increase errors to getUserErrorMessage
Change-Id: Icc84b8d09d3c1edc3ae216b71812442d67e3b5c9
2023-10-09 17:44:48 +00:00
Vitalii
d60e0f0036 web/satellite: fix locked files calculation when pagination is enabled
The problem is that we always have to refetch objects with List command without Delimiter provided to count in objects inside folders.
This change doesn't fix the main problem for recently deleted objects.

Change-Id: Ia64579745999301c285358869e283dff09399f41
2023-10-09 13:02:16 +00:00
Sean Harvey
1c0b23a8d0
web/satellite: ensure "s/" prefix in path for linksharing urls
currently we are generated some linksharing urls with and without
the "s/" in the path. Make sure we're using "s/" for both cases
to avoid a redirect with the backwards compatible style.

Change-Id: If04a41325ebbece23d4e7f2c0431bd4905596bce
2023-10-09 13:00:52 +13:00
Vitalii
013ddbc57f web/satellite: fixed balance inconsistency for billing tabs
Made overview tab always fetch account balance on mount.
Made payment methods tab always fetch wallet (which includes wallet balance) on mount.
Account and Wallet balances are queried from transactionsDB so there is no inconsistency on backend side.

Issue:
https://github.com/storj/storj/issues/6093

Change-Id: Ifb63d20040acf03d60ad08eac51b0f20b5b7365d
2023-10-06 19:31:10 +00:00
Moby von Briesen
3216674c19 satellite/payments/accountfreeze: Increase exclusion threshold
Account freeze excludes accounts who have invoices which are unpaid and
are above a configured value. This change updates that configured value
from $100 to $1000.

Change-Id: Iff381deeb73ef2d8fccaf7a1612a33e8aeb08698
2023-10-06 14:39:20 +00:00
Jeremy Wharton
3ca626a878 web/satellite: send more analytics events for onboarding selection
This change causes analytics events to be sent when the user has chosen
to continue onboarding in the browser or to skip onboarding.
Previously, only the the selection to onboard in the CLI flow was
tracked.

Change-Id: I44bdac6b8704a67382d8655767cdaea691191cbe
2023-10-06 13:26:19 +00:00
Vitalii
60bf9531af web/satellite/vuetify-poc: add client-side sorting for buckets table
Sorting works only if buckets fit on single page (can be changed by modifing limit value).
We use custom sorting approach to sort by correct value type and not by string.
We don't use custom-key-sort prop of VDataTableServer component because it doesn't work (probably expects it to be done on backend side because VDataTable component's prop works as expected).

Issue:
https://github.com/storj/storj/issues/6342

Change-Id: I0cfbdf432e255f530457c89253a7f29b8e1cbc30
2023-10-06 12:45:42 +00:00
dlamarmorgan
7f02b73b5d cmd/satellite/main: complete/fail pending payments cmd
Add billing commands to manually complete or fail storjscan invoice
payments that are stuck in a pending state.

Change-Id: Ia19f0a2597201d9d17aad0889eaedff095d706b9
2023-10-06 01:23:34 +00:00
Wilfred Asomani
65f14ec3f5 web/satellite: prevent row action for project owner
This change fixes an issue where users might be able to remove project
owners from a project buy clicking the "invisible" row action button on
the project owner row.

Issue: https://github.com/storj/storj/issues/6345

Change-Id: Ib6f689d65f23539cc3660f72878d6d50fc502e2c
2023-10-05 20:13:21 +00:00
Clement Sam
d432a7197a storagenode/piecestore: notify if download piece was restored from trash
Updates https://github.com/storj/storj/issues/6146

Change-Id: Iece285eb5ecb6898b29096416ab10e43338480b0
2023-10-05 18:50:01 +00:00
Cameron
e072b37a86 satellite/console: add endpoint to request project limit increase
create endpoint to allow pro users to request project limit increase.

github issue: https://github.com/storj/storj/issues/6298

Change-Id: I96c3dff8bf0906904d199fc2c7ee738f3e6b04a3
2023-10-05 17:21:32 +00:00
Jeremy Wharton
22ad017f12 web/satellite/vuetify-poc: implement browser onboarding flow
This change implements the browser onboarding flow in the Vuetify
project.

Resolves #6334

Change-Id: I68ff214730cb0a39ea5f9af47d9ecbe3051f2005
2023-10-05 16:34:41 +00:00
Clement Sam
cfbb5dac14 go.mod: update storj.io/common
Change-Id: Ibdfd962b8be8407a687753d0648149a5f98a44b2
2023-10-05 15:52:06 +00:00
Clement Sam
05901aa303 cmd/tools/tag-signer: fail for comma separated tags
This change allows tag-signer to fail when key-value pairs
provided as arguments are comma-separated.
However, for cases where a value is expected to contain a comma,
we validate the value only if --confirm flag is specified

Resolves https://github.com/storj/storj/issues/6336

Change-Id: Ib6a100ee3adf529f44c8b3ca620a3c0b4f953a17
2023-10-05 13:57:53 +00:00
Jeremy Wharton
5d286399f3 web/satellite/vuetify-poc: improve the share dialog
This change makes several minor improvements to the Share dialog:
- The "Copy" button has been renamed to "Copy Link".
- A copy icon button has been added to the linksharing URL text field.
- An alert has been added notifying that anyone with the link will be
  able to see the shared data.

Resolves #6381

Change-Id: Idf4c2b2963d7174173c1fa479c90a4fb6c1bf24c
2023-10-05 12:30:21 +00:00
Egon Elbre
27a13efb17 satellite/console/consoleweb/consoleapi: fix defer in loop
Change-Id: Ifdedad8a5556be47b5f99dc8d8a8ff67904e2688
2023-10-05 12:55:36 +03:00
Egon Elbre
800bba7a0e go.mod: bump uplink
This brings in 32bit arm bug fix.

Change-Id: I61b72abf2366643f58b1e670d2e02f4072aee0b5
2023-10-05 08:51:11 +00:00
Ivan Fraixedes
99c4359062 private/apigen: Avoid clashes of types names
The TypeScript generator create types of anonymous structs. When those
anonymous structs are in endpoints with the same name, but in different
endpoint groups, the generator panic because it create different types
with the same names.

This commit fixes that problem through using the endpoint group prefix
to create the types with different names.

Change-Id: Ibe87532609ce824b80951326f9777ed5b0cc2f7a
2023-10-05 07:30:09 +00:00
Ivan Fraixedes
3193ff9155 private/apigen: Don't force casing for API group name/prefix
The API generators rely on the Name and Prefix fields of the
EndpointGroup type to generate code.

Conventional naming code requires using upper or lower case for types,
functions, etc, however requiring the user to set this fields with the
correct casing seems cumbersome for them because they can be adjusted
depending where those values are used on the generated code.

This commit lifts the restriction for the user and adjust the casing of
them according to where they are used.

Change-Id: I700a879d13b4789b4d6ba0519b4d7508061eac73
2023-10-05 06:48:50 +00:00
Sean Harvey
7b50ece931
satellite/admin: add missing test assert to project geofence tests
the existing test was missing a case where deleting the geofencing
should set the placement back to default.

this also changes EveryCountry to DefaultPlacement, as the former is
deprecated.

Change-Id: I29f4f1c3ae1d05e8f5ecaa03a70ac4f17574a475
2023-10-05 18:21:48 +13:00
Vitalii
95958a3c5c web/satellite/vuetify-poc: emit success notification on access grant delete
Show success notification after access grant deletion.

Issue:
https://github.com/storj/storj/issues/6364

Change-Id: Idebc4210ff5efbaab0c71ad519d34de4e1803420
2023-10-04 20:21:26 +00:00
Ethan Adams
6961d2e19d satellite/email: Add support for unauthenticated, cleartext SMTP connections
Change-Id: I11b4852122764c1ede188ca40d5edb14f2c4ee72
2023-10-04 16:55:55 +00:00
paul cannon
02899dfae2 satellite/{admin,console,satellitedb}: fix geofence removal
deleteGeofenceForProject wasn't able to work correctly, because
Console().Projects().Update() declines to update default_placement when
the input value is 0.

This introduces a Console().Projects().UpdateDefaultPlacement() method,
congruent to the method of the same name on Console().Users().
deleteGeofenceForProject now uses this new method, so that specifying a
new placement of 0 will work correctly.

Change-Id: I4589b36707f7e4f1cfdc66543520b0d4205c1a84
2023-10-04 16:14:29 +00:00
Moby von Briesen
a2acf359ad satellite/satellitedb: Remove dbx references to partner_id
This column is no longer used, but it is referenced in dbx. This change
removes those references and adds an exception to the migration test to
disregard this column in schema comparison.

After this change is deployed, we will need to follow up with a
migration to remove the column in a later release, and remove the
exception in the migration test.

Related to https://github.com/storj/storj/issues/5432

Change-Id: I168fb57244b347901d1ed9c7813c1338554ee644
2023-10-04 15:33:44 +00:00
paul cannon
c33475f63e storagenode/gracefulexit: fix flaky test (hopefully)
If the storagenode chore is left running and it has a chance to check in
again after we move time forward (line 139), then the satellite will
mark it as having finished GE before we check which nodes are still in
GE (line 149).

Change-Id: I350e1ef2e943f758d44132aaddd05fe248b30f3e
2023-10-04 09:53:51 -05:00
Ivan Fraixedes
47f344927d
private/apigen: Fix TS generator to apply naming conventions
Fix the TypeScript generator to generate code using the common
TypeScript conventions.

Closes https://github.com/storj/storj/issues/6360

Change-Id: I244896feea389670eca0df95d3ac58e25d600e14
2023-10-04 13:47:33 +02:00
Ivan Fraixedes
71c9547de5
private/apigen: Don't allow certain types for Request/Response
We cannot map certain types to TypeScript, hence we verify them in the
Endpoint.Validate method.

Even some types could be somehow mapped, we don't want to add more
complexity or allow types that don't make sense to be for a request or
response.

Change-Id: I51ecee286e637b1160e967d77f9ce6c7403ddfdc
2023-10-04 13:46:54 +02:00
Ivan Fraixedes
956109a097
private/apigen: Rename Endpoint fields
Rename the Endpoint fields MethodName and RequestName because they were
confusing for what they are used.

This commit also adds some validations for these fields values and other
validations for Endpoint and EndpointGroup to avoid generating invalid
code.

It also include some tests for these new validations.

Closes https://github.com/storj/storj/issues/6333

Change-Id: Iaabfc33935517889e3729c8b37be51a55eea366c
2023-10-04 13:46:54 +02:00
Ivan Fraixedes
ac86eb397a
private/apigen: Fix bug TypeScript optional class members
The TypesScript generator generates classes and checks for Go struct
field if the struct type is of a nullable type, however, it should check
if the field types is nullable because it doesn't make sense to check
that struct field for each field nor to assign to the field if it's
nullable or not depending on it.

Change-Id: Ia22a609a17752f520233c006cba17685fe142b32
2023-10-04 13:46:08 +02:00
Sean Harvey
bc7f621073
satellite/satellitedb: fix DefaultPlacement overwritten on user
this fixes cases where it's possible to update a user and the
DefaultPlacement field gets overwritten to the zero value.

it also adds UpdateDefaultPlacement which can be used to set
DefaultPlacement directly. This is needed for the geofencing
endpoints in satellite admin to set the DefaultPlacement back
to zero to delete geofencing for a user.

Change-Id: If2c798dabfa6773ed6023fb8257bf00ec7bc2e68
2023-10-04 15:17:35 +13:00