Commit Graph

8904 Commits

Author SHA1 Message Date
Márton Elek
9414e3270d
satellite/durability: remove unclassified measurements + reduce memory
Small adjustments in durability observer:
 * use local map for class names instead of struct level var (not used anywhere else, earlier GC)
 * continue / skip when the class is "unclassified" / 0.
 * `classes["unclassified"] = 0`: doesn't really matter, but this is more true

Change-Id: Ib927bab52d5502ad67ecc8570fda218fbfb6a95b
2023-12-07 16:57:10 +01:00
Moby von Briesen
c87f380e2e satellite/repair: Fix flaky test
Don't use global monkit.Default in Queue Stat test, or it can fail due
to concurrently-executing tests.

Change-Id: I061d626f26220705c8dd0de17ac7e14c81831d7f
2023-12-05 16:19:30 -05:00
Vitalii
5094100e21 web/satellite: fix indentation for TS logic
Added new eslint rule to fix indentation for typescript logic.

Change-Id: I08ad94411c579ac6c4687217a29c3b7dc61b0d76
2023-12-05 20:17:10 +00:00
Vitalii
61a9d63d76 web/{satellite, vuetify}: rework locked objects logic
When a user enters the encryption passphrase when trying to enter a bucket, only display a "you may have files uploaded with a different passphrase" message when: 0 objects can be listed with the provided passphrase, but there are >0 objects in the bucket.

When a user enters the encryption passphrase when trying to enter a bucket, and >0 objects can be listed with that passphrase, proceed immediately with opening the bucket and displaying the files: do not show any indication in the file list view that there might be "locked files".

Issue:
https://github.com/storj/storj-private/issues/516

Change-Id: I2ae3809971867e5b69b804192a380a6919ed0108
2023-12-05 19:27:04 +00:00
Michal Niewrzal
45cfaa8743 satellite/metainfo: set internalpb.StreamID Versioned field
To have the same UploadID returned from BeginUpload, ListObjects
and ListPendingObjectStreams we need to set StreamID.Versioned field
everywhere.

Change-Id: I1328d9c476767559b8feb7c5bcd5afb154f7cee3
2023-12-05 14:46:56 +00:00
Vitalii
c5e44072a8 web/satellite/vuetify-poc: prevent page reload on folder create
Remove default page reload behavior on form submit during folder creation.

Issue:
https://github.com/storj/storj/issues/6559

Change-Id: I6d5c433adb4046238f958761b4d1f27696cfc459
2023-12-05 10:22:43 +00:00
Vitalii
b31e417546 satellite/payments: use stripe idempotency on create/update invoice items
Add idempotency key for every create/update invoice item request.
Key consists of public project ID, line item description and formatted invoicing period.
This should eliminate the possibility of making duplicate create/update actions.
Also, with this change, we mark a project record as consumed only after the appropriate invoice items are created or the record itself is skipped.

Issue:
https://github.com/storj/storj/issues/6501

Change-Id: I7506a8f3886d7f575bcc0facc3f107513352a312
2023-12-05 11:16:53 +02:00
Márton Elek
504c72f29d satellite/repair: run chore which updates repair_queue stat
Change-Id: I18c9e79e700ac690c3aa78ee0df0cf6089acd2ac
2023-12-05 04:33:46 +00:00
Wilfred Asomani
0590eadc17 satellite/{db,console,billing}: fix legal/violation freeze
This change fixes the behavior of legal freeze; it now does not allow
lists and deletes for users in that freeze. It also fixes an issue
where users who have been legal/violation frozen will have their limits
upgraded, partially exiting freeze status when they pay with STORJ
tokens.

Issue: storj/storj-private/#517
storj/storj-private/#515

Change-Id: I6fa2b6353159984883c60a2888617da1ee51ce0a
2023-12-04 21:19:44 +00:00
Ivan Fraixedes
e1c12674c5 satellite/admin: Fix API key delete by name
We couldn't delete API keys by name whose name contained slashes because
Gorilla Mux router interpreted the as path separator and didn't resolve
to the right endpoint.

To fix the issue the name is sent as a query parameter rather than as a
path parameter.

Change-Id: Ica67d6b9f047d7c33a5350457afc822cb8d4c7a1
2023-12-04 20:03:30 +00:00
Clement Sam
8ec1a8de7d satellite/satellitedb: drop nodes.type column
This is the final change to completely drop the type
column from the nodes table.

All code references are removed and deployed, dbx changes
have been merged and deployed.

Updates https://github.com/storj/storj/issues/5426

Change-Id: I1b76fe92f25e350c1c44730c008f0e9107852509
2023-12-04 17:31:33 +00:00
Michal Niewrzal
a3f84bcb0c satellite/gc/bloomfilter: reduce number of filters in single zip
Current value is too big and can cause memory issues. With 500 bloom
filters where single filter can go up to 2MB we can end up with 1GB zip
which is more problematic if we don't want to download it before
processing.

Change-Id: I4b983afbcac39c718afbbeb764a30d32c4e80c08
2023-12-04 16:09:24 +00:00
Wilfred Asomani
5d775e61b1 satellite/{console/web}: add v2 signup flow flag
This change adds a new flag to toggle whether to use the v2 app signup
flow. If disabled, the v2 signup page will redirect to the current
app's signup page.

Issue: #6552

Change-Id: I0d48ca713dc77cb2879630613f92123baced3ca3
2023-12-04 15:18:37 +00:00
Márton Elek
9e6c421cc5 satellite/durability: report bus_factor with the new reports
New instance based satellites are reporting durability alerts like `class=last_net, value=192.168.0.0`.

Bus factor will be reported as `class=bus_factor value=last_net`.

Change-Id: I0130866fa8e50b3d2876f580a78383bb1c609159
2023-12-04 14:06:20 +00:00
Cameron
5c79f59b79 web/satellite: remove VChart tooltip elements on unmounted
We use a custom plugin for the chart tooltips. If a user navigates away
from the page while a tooltip is rendered, the element doesn't get
removed. To fix it, when the chart is unmounted check for the tooltip
elements and remove them if they exist.

issue: https://github.com/storj/storj/issues/6344

Change-Id: I42122ef5fad3a1ee7d16ed554881f8c3f79f2b26
2023-12-04 12:42:21 +00:00
Vitalii
7b3770ed17 web/satellite/vuetify-poc: hide 'Add coupon' button when coupon features are disabled
Hide button if feature flag is set to false.

Issue:
https://github.com/storj/storj/issues/6547

Change-Id: I0e13bacfe6a8651ebc573abc3f23a749821d63a5
2023-12-04 10:38:13 +00:00
Vitalii
775ee860a5 web/satellite/vuetify-poc: remove billing information tab
Removed billing information tab because it's non-functional.

Issue:
https://github.com/storj/storj/issues/6546

Change-Id: Ieb9a691dcc5015863108e11895bb75536fb310b2
2023-12-04 10:36:59 +00:00
Vitalii
207dad4362 cmd/satellite: bind new billing command to config
This is a fix for invoice generation flow (method 2).
New command wasn't bind to config.
Method 1 still works as expected.

Change-Id: I481a0646f3f8f6f877ee8efab3622dd525d8c22a
2023-12-01 23:49:20 +00:00
Vitalii
0c913ab548 satellite/{console, web}: show banner for bot accounts
Display a banner on the dashboard of all projects for bot accounts, with a prompt to submit a support ticket.

Issue:
https://github.com/storj/storj-private/issues/503

Change-Id: Id5f2aae0394b4fdca968a4de93bda19d6b0eb37a
2023-12-01 23:01:11 +00:00
JT Olio
591971b4dc satellite/metainfo: reduce database hits for segment creation
Change-Id: I48a748b48daefa95f1dfbf6a0d75e65a568ee36a
2023-12-01 20:28:48 +00:00
Ivan Fraixedes
5d492a9e01 satellite/admin/back-office: Don't send project ID
The project ID must never be sent out from the server because old
projects used that ID as salt for creating access grants.

We must always expose the public ID and the server must retrieve the ID
when the public ID is sent in to the server and needs it for performing
some DB queries.

Change-Id: I4a9d94049ef84f578b62907827d6c09dfd1db4db
2023-12-01 19:07:42 +00:00
Wilfred Asomani
94bbda81dc web/satellite/vuetify-poc: add delete action to file preview
This change adds a menu to the file preview dialog with an action to
delete files.

Issue: #6553

Change-Id: I3b282696077b884df4171a85dee6a62dc9fb0513
2023-12-01 18:18:57 +00:00
Wilfred Asomani
4b660e58fa web/satellite/vuetify-poc: add partner branding to signup
This change adds partner branding to the signup page.

Issue: #6528

Change-Id: Idbed1818b30e80a818311ee4a73d6660a3aadc6d
2023-12-01 17:11:25 +00:00
JT Olio
8a42cad6a1 cmd/tools/convert-node-id: show node id for an id.cert
Change-Id: Iae392398dac3d4f45ef829b25b27a2a7a69b0932
2023-12-01 15:24:39 +00:00
Vitalii
b776382f18 satellite/admin: functionality to disable account bot restriction
Added new admin functionality to disable account bot restriction by activating it.
This must be used for accounts with status PendingBotVerification.

Note: PendingBotVerification status is automatically applied for accounts with high captcha score right after activation.

Issue:
https://github.com/storj/storj-private/issues/503

Change-Id: I4c9ee834075a7abaf221ac054a455a8d27debf40
2023-12-01 13:09:49 +00:00
Vitalii
594b0933f1 satellite/console: PendingBotVerification status for users with high captcha score
Added new captcha score cutoff threshold config value (default is 0.8).
Added new user status PendingBotVerification which is applied right after account activation if signup captcha score is above threshold.
Restricted project creation/joining if user's status is PendingBotVerification.

Issue:
https://github.com/storj/storj-private/issues/503

Change-Id: I9fa9932ffad48ea4f5ce8235178bd4af45a1bc48
2023-12-01 15:09:01 +02:00
Ivan Fraixedes
fb31761bad satellite/admin/back-office: Add auth middleware
Create an API generator middleware for being able to hook the new
satellite admin authorization in the endpoints.

The commit fixes a bug found in the API generator that caused that
fields of types of the same package of the generated code where wrongly
added. Concretely:

- The package matching was missing in the function middlewareFields,
  hence it was generating code that referenced types with the package
  name.
- middlewareFields function was not adding the pointer symbol (*) when
  the type was from the same package where the generated code is
  written.

There is also an accidental enhancement in the API generator because I
thought that the bug commented above corresponded to it, rather than
removing it, I though that was worthwhile to keep it because it was
already implemented. This enhancement allows to use fields in the
middleware with packages whose last path part contains `-` or `.`, using
a package rename in the import statement.

Change-Id: Ie98b303226a8e8845e494f25054867f95a283aa0
2023-12-01 00:29:49 +00:00
JT Olio
0467903b52 cmd/uplink: don't show throughput during copy, since it's wrong
we can bring it back once we understand this issue:
https://github.com/vbauerster/mpb/issues/135

Change-Id: I2cbdcfc260786e1ab08d1badc78770a792785650
2023-11-30 21:20:50 +00:00
Wilfred Asomani
8568e49578 satellite/{web,console}: remove upload modal flag
This change removes the config flag for the new upload modal.

Issue: #6515

Change-Id: I69612c6730f933d6320cb3c05995900eac98ac91
2023-11-30 19:43:24 +00:00
JT Olio
e03091dd51 go.mod: bump storj.io/{common,uplink}
Change-Id: Ifb7c44e57643e1e3cfab5f9f70f1bfa5224c5025
2023-11-30 11:24:19 -05:00
Egon Elbre
a44ed9457e satellite/metabase: allow creating delete markers for missing objects
Change-Id: Ic8f6f80c234478b3e7b3bc4e88fbb5df9a783456
2023-11-30 16:33:42 +02:00
Wilfred Asomani
50c9f4c85a web/satellite/vuetify-poc: add account setup dialog
This change copies the account setup dialog from the static v2 repo and adds functionality to it

Issue: #6481

Change-Id: I0cacec3edf054c1945df7593b73e3c0f0f96677c
2023-11-30 13:15:31 +00:00
Ivan Fraixedes
ea022ede46 satellite/admin/back-office: Add method for auth middleware
Add a new method to the Authorizer to use it with the API midleware that
we are going to implement for injecting it into the handler generated by
the API generator.

This new method will reduce the lines of code to generate and avoiding
errors that are more difficult to test in generated code.

The commit deletes the Middleware method because we won't used due to
the API generator doesn't support "standard" middlewares and allows
their customization via code generated and injected inside the handler
base logic generated by the API generator.

Change-Id: Ie427eb2eea94797913e2c357cf097ecf1e2e63ef
2023-11-30 11:47:34 +00:00
Wilfred Asomani
a3a7df91e3 web/satellite/vuetify-poc: add signup functionality
This change implements functionality for signup and account activation.

Issue: #6481

Change-Id: Ide8f743dc3996c8b2a23a494f8d8500e1af658bc
2023-11-30 10:13:04 +00:00
Cameron
26b6ab8a3a web/satellite: only fill charts with zero if day is complete
Before this change we would fill all days on the chart with zero if no
data existed. This led to concerns when users saw their stored data
value drop to zero. Instead, we will leave incomplete days blank if
there is no usage data. This includes today and future days.

issue: https://github.com/storj/customer-issues/issues/1157

Change-Id: I4b9def392b89684fe63f1503274e868a240eb353
2023-11-30 08:28:10 +00:00
Moby von Briesen
dcc3245954 satellite/console: Update project members endpoint
We don't need to send all the information that the endpoint currently
provides - name and email is sufficient.

Change-Id: I088d61edfc022a4a40c96e1faebd48808af5b00f
2023-11-29 17:34:35 -05:00
Wilfred Asomani
f749b8ff51 satellite/{db,console}: support v2 app account set up
This change modifies the register endpoint handler to not require name
for signups from the v2 app and adds a new endpoint for completing
account information (e.g. name). This is to support the new signup and
account setup flow of the v2 app.

Issue: #6470

Change-Id: I256e1c804fcdbc8ce05aa82d6bc4b0263f55daa5
2023-11-29 20:59:55 +00:00
Márton Elek
1ba314428f go.mod: make bump-dependencies (uplink, common, ...)
It requires more work at this time, as https://review.dev.storj.io/c/storj/private/+/11066 modified the way how we configure the debug port.

Change-Id: I4a7cc999e13fd3514064a515b21885bb4d39ff16
2023-11-29 16:55:41 +00:00
dlamarmorgan
fd13cf1058 dbx/satellitedb: add default_versioning field for projects
Change-Id: I294a85c244976493b490a58f17536f06f6c01306
2023-11-29 15:29:14 +00:00
Wilfred Asomani
ab65572af0 web/satellite: implement account activation with code
This change implements account activation using OTP code. Based on
whether this activation method is activated, signup will require
    the user to fill the activation code to activate their account.

Issue: #6428

Change-Id: Ifdd3e34ddb8ab299dd24b183f9c484613b2e8888
2023-11-29 13:29:50 +00:00
Wilfred Asomani
116d8cbea1 satellite/console: implement account activation with code
This change implements account activation using OTP code. Based on
whether this activation method is activated, signup will generate a
6-digit code and send it via email. A new endpoint is added to validate
this code and activate the user's account and log them in.

Issue: #6428

Change-Id: Ia78bb123258021bce78ab9e98dce2c900328057a
2023-11-29 13:06:37 +00:00
Vitalii
1546732afc web/satellite/vuetify-poc: set input value length limits
Ensure that text inputs have length limits, like in production and more.

Issue:
https://github.com/storj/storj/issues/6536

Change-Id: I2f216cc57d6bcb49b88c34ce53bed58327b0cdc6
2023-11-29 11:57:05 +00:00
Márton Elek
20a3045e1a satellite/durability: use fixed number of pieces in integration test
Test was flaky because we asserted if we have 15 classes:

6 email (for each used (!!!) nodes)
6 last_net (for each used (!!!) nodes)
1 wallet
1 country ("HU")
1 empty value

But there was a very low chance to use only 5 nodes, out of the 6 (RS.Success=5, RS.Total=6).

In that specific case, we had only 12 classes, as we didn't see all the used emails, as we iterated over the used nodes only (and one node was not used).

https://github.com/storj/storj/issues/6549

Change-Id: I66882d5fa9b0d5f5b2397ea856494037972d4b81
2023-11-29 11:04:20 +00:00
Moby von Briesen
03a8e7c81a satellite/analytics: improve error logging
This inserts additional information provided in error messages from
Hubspot into the Go error that is eventually logged out.
Before, we would see the a generic "sending event failed" log.
With this change, we will see more detailed information in the log, such
as a list of required fields that were not submitted.

Change-Id: I24da0646bca62f459377abe6281403020fb54c49
2023-11-29 09:49:25 +00:00
Jeremy Wharton
4822b18472 satellite/admin/back-office/ui: implement view user functionality
This change adds a search field to the new admin UI through which user
email addresses may be submitted. If the email belongs to a verified
user, the client will be redirected to the Account Details page which
is populated with the user's information.

Resolves #6469
Resolves #6475

Change-Id: Icbf3cb3f8374f2764e73a523f111c5ecf3d06569
2023-11-28 16:31:37 +00:00
Egon Elbre
7186525d5c satellite/metabase: test zombie deletion with versioning
Change-Id: I495745f84ea0a7c36763d2889170c4068347dcb7
2023-11-28 12:12:26 +00:00
Ivan Fraixedes
6885ca27fb private/apigen: Inject middleware code after param parsing
Inject the middleware code after the query and path parameters parsing
because in the case that a middleware needs to execute a more expensive
logic, it isn't executed if there are malformed or missing parameters.

Change-Id: I883ade8ee42d25accb153c78d9f583d883730d6a
2023-11-27 23:02:44 +00:00
Vitalii
5e3cab29a2 satellite/{cmd, payments}: add optional command flag to toggle invoice items aggregation
Added new optional '--aggregate' flag for billing.generate-invoices and billing.prepare-invoice-records commands to toggle invoice items aggregation.
Added new explicit command billing.create-aggregated-project-invoice-items which should be used after preparing aggregated invoice records (in cases when invoice generation happens step-by-step).

Change-Id: I04fc0110be5263edb959306d5314a4a1a8eec3ba
2023-11-27 22:11:15 +00:00
Jeremy Wharton
d2819522c6 satellite/admin/back-office: add endpoint to get users by email
This change adds an endpoint to the back office API that returns user
info based on email address.

References #6503

Change-Id: Ib48d30b0b6c6862887b3f8114f50538b3deca57b
2023-11-27 21:05:12 +00:00
Michal Niewrzal
220920edb9 satellite/metainfo: return NotFound status for (Get|Set)BucketVersioning
We need to return NotFound rpc status when bucket doesn't exist.

Change-Id: I3abdd588e2af288618b05513884bc0c545bcbacf
2023-11-27 19:45:45 +00:00