Commit Graph

6829 Commits

Author SHA1 Message Date
JT Olio
e0d3e48b66 satellite/consoleweb: monitor visits without blowing up influx namespace
Change-Id: I43de5409e66b9ef34b6c35f211b9a252d3cc3caa
2022-08-18 17:48:28 +00:00
Cameron
fa23e55654 {private,satellite}: add Typescript generation to REST API generation
github issue: https://github.com/storj/storj/issues/4984

Change-Id: Id3473a6642f7a4af80edda25a6242559efaf69e9
2022-08-18 16:37:48 +00:00
cl-mitch
c255d6949f web/satellite: Minor access grant updates
This change is a copy of the stale Github PR at https://github.com/storj/storj/pull/4972
It fixes some of the remaining minor issues outlined in https://github.com/storj/storj/issues/4966

Change-Id: Ide973b1b23440bf37a19af8aa6ead39b470ceb25
2022-08-18 15:23:44 +00:00
Malcolm Bouzi
cf4bfbcf2d web/satellite: make login and registration satellite dropdowns accessible through tab usage.
Change-Id: I279ac065eb00e6e9412c78ff8c3eb9f2e4a308ec
2022-08-18 06:22:07 +00:00
Wilfred Asomani
874e28e283 web/satellite: fix inactive buttons on billing page
This change fixes makes the "See Payment" and "Edit payment method" on the billing page
navigate to the correct corresponding pages.

Change-Id: Ia2045394643d11171cc8ad5ded6654c4293875d6
2022-08-17 20:32:27 +00:00
Wilfred Asomani
c59d0d415c satellitedb: add signup_captcha column to users table
The signup_captcha column will hold the captcha scores of new users.

see: https://github.com/storj/storj/issues/5067

Change-Id: Ia322af29a3b5b019b417843272506a3dbd1397e4
2022-08-17 14:31:52 -05:00
paul cannon
0dcc0a9ee0 satellite/reputation: reconfigure lambda and alpha
This is in response to community feedback that our existing reputation
calculation is too likely to disqualify storage nodes unfairly with
extreme swings up and down.

For details and analysis, please see the data_loss_vs_dq_chance_sim.py
tool, the "tuning reputation further.ipynb" Jupyter notebook in the
storj/datascience repository, and the discussion at

    https://forum.storj.io/t/tuning-audit-scoring/14084

In brief: changing the lambda and initial-alpha parameters in this way
causes the swings in reputation to be smaller and less likely to put a
node past the disqualification threshold unfairly.

Note: this change will cause a one-time reset of all (non-disqualified)
node reputations, because the new initial alpha value of 1000 is
dramatically different, and the disqualification threshold is going to
be much higher.

Change-Id: Id6dc4ba8fde1be3db4255b72282207bab5491ca3
2022-08-17 18:52:53 +00:00
Wilfred Asomani
99dee2daa3 web/satellite: fix bucket page guide positioning
This change fixes an issue where the bucket guide goes off screen

see: https://github.com/storj/storj/issues/5086

Change-Id: Idf0718df2ea49b195b649445f60568c35ca856d9
2022-08-17 17:57:47 +00:00
Vitalii
418372621c web/satellite: improve access grants empty state
Improved access grants empty state to show correct label depending on search query.

Issue: https://github.com/storj/storj/issues/5074

Change-Id: Ibc5e8857ed213bcea28774151472e8750400beee
2022-08-17 17:06:12 +00:00
Vitalii
9dcef77639 web/satellite: fix for billing screen
Fixed old billing layout to show new billing layout.
There was a routing issue when navigating from account dropdown.

Change-Id: I65d91681b2891ec347a6729ae3db4cd0e6073161
2022-08-17 16:28:30 +00:00
Wilfred Asomani
286f68a0bd web/satellite: fix page content bottom padding
This change removes the errant bottom padding on all page contents.

Change-Id: Ife4d86853fffda06ad34b08999d5c52a3b5754d7
2022-08-17 15:42:00 +00:00
Cameron
c82424c21d satellite/admin: add Burst field to Update Project Limits params
Since 6 fields were sent, but only 5 fields were used as input (no burst),
the Segments UI input field was being set to the burst field in the
object, leaving segments empty.

Change-Id: I4e58ed8a36e2022d1d0844e9f9001562db3dcbb8
2022-08-17 10:40:28 -04:00
Wilfred Asomani
eb547740f4 web/satellite: fix mobile signup page
This change fixes an issue where description texts will not show on the partner signup page on small screens.

Change-Id: I59868b5c598499e3fc6dba7991453ec5c4e4d998
2022-08-17 14:00:19 +00:00
Jeremy Wharton
ffd35ecfdc web/satellite: move billing nav item into My Account dropdown
This change moves the top-level Billing navigation item into the
My Account dropdown menu. A notification has been added that
informs users of this change.

Change-Id: I641b0e5ad0a6de0206f6cde9d3022033c419215a
2022-08-17 06:42:00 +00:00
dlamarmorgan
8eec61d315 satellitedb: add timestamp index to billing table
Adding an index to the timestamp field of the billing transaction table to improve query performance. This should prevent having to do a full table scan when we query for the last billing transaction of a particular source and/or type.

Change-Id: I581f09494cc8662a12efba4302022a07121ba309
2022-08-16 22:50:02 +00:00
dlamarmorgan
f596f72f44 satellite/payments/storjscan: add billing update to chore
Update chore to add confirmed transactions to the billing table.

Change-Id: I3c9a98c2ddc96f8a8905250376a1f5490d810277
2022-08-16 11:18:15 -07:00
Ivan Fraixedes
7299e9f92e
satellite/console: Add missing monkit calls
Add monkit monitoring calls to a couple of the service's methods because
they were missed.

Change-Id: I327a79634b7cdcaeb0e2701a737bb714a6478c4a
2022-08-16 13:20:18 +02:00
dlamarmorgan
ac241501ca satellitedb: add wallet address index to storjscan_wallet table
Adding an index to the wallet address field of the storjscan wallet table to improve query performance. This should prevent having to do a full table scan when we query for one or more wallet addresses for a given user in our queries.

Change-Id: Ic1b5d06c2258489e5464d186fed5270172f8cba5
2022-08-15 18:05:06 +00:00
Jeremy Wharton
1f0638719e private/apigen,cmd/apigentest: add tests for generated API code
This change implements a unit test for ensuring proper
processing of requests and responses by generated API code.
Additionally, this change requires API handlers to explicitly receive
Monkit scopes rather than assuming that `mon` will always exist in the
generated API code's namespace.

Change-Id: Iea56f139f9dad0050b7d09ea765189280c3466f2
2022-08-15 16:48:41 +00:00
wilfredasomani
26bed35f33 web/satellite: update access grants table
This change updates the access grants table to use the new table component.

see: https://github.com/storj/storj/issues/4994

Change-Id: I6fa8c3feb2023c2dacc7ea5536ba9e16652f8c26
2022-08-15 11:55:53 +00:00
Vitalii
0ec28ca623 web/satellite: add token funds modal
Created new modal which shows user their native STORJ token wallet address.
There are QR and copy buttons.
It will be used only in new billing screen.

Change-Id: Icef3c8668c548b779c07fe2b85eb5761cd1221a3
2022-08-15 14:17:30 +03:00
cl-mitch
d910074097
web/satellite Billing screen bug fixes (#5050)
Changed the Billing History tab to only display Stripe payments, updated the coupons date to change based on the duration, and addressed the following tickets:
https://github.com/storj/storj/issues/5028
https://github.com/storj/storj/issues/5027
https://github.com/storj/storj/issues/5026
https://github.com/storj/storj/issues/5021
2022-08-12 14:01:13 -05:00
Qweder93
2abe709b6e satellite/metabase: add metabase.GetObjectLatestVersion back
Restored GetObjectLatestVersion and renamed it to GetObjectLastCommitted
Add test cases to cover server-side copy

Closes https://github.com/storj/storj/issues/4866

Change-Id: I343b339a60152b8fb92fda97baf80bd8fe60d631
2022-08-11 22:14:38 +00:00
Vitalii
c552c343f9 web/satellite: make modals responsive
Made next modals reponsive:
AddTeamMemberModal
ChangePasswordModal
CreateProjectModal
CreateProjectPromptModal
DisableMFAModal
EditProfileModal
EnableMFAModal
MFARecoveryCodesModal

Issue:
https://github.com/storj/storj/issues/5031

Change-Id: I9c80137febd1d16678f3de523634efc5010c8a9d
2022-08-11 20:00:12 +00:00
hovex023
4be219c320
Web/satellite: Add responsiveness to onboarding flow (#5065)
* added media queries for onboarding flow

* responsive changes to create bucket steps

Change-Id: I2799a75db4e5029fa8a1ce11afac04333cca09c7

Co-authored-by: Moby von Briesen <mobyvb@gmail.com>
2022-08-11 13:45:11 -05:00
kimbotsao
c3e7b1409d satellitedb: add salt to projects table
Change-Id: I53691849efc169c495566648c12ae3d639614c3d
2022-08-11 12:16:53 -04:00
wilfredasomani
8b338fb655 web/satellite: show guide on bucket page
This change shows a new "no bucket" page and a
tooltip to guide users to open a bucket for upload.

see: https://github.com/storj/storj/issues/4969

Change-Id: I99fd855fe39ba7ad308b1c90f79703eeee1b1627
2022-08-11 15:36:51 +00:00
Márton Elek
96b88e286e build: fix conditional run of web builds
Jenkins doesn't do a very good job with identifying what has been changed.

While it has a syntax to defined patterns, it compares the current build with the previous build (in case of git-verify it can be a totally different branch) instead of checking the HEAD commit.

This patch introduces shell scripts to do this better:
 * It doesn't depend on Jenkins any more
 * It can be executed locally
 * It can detect web changes properly (see the relation change as an example).

Change-Id: I9d37775e3818c08c4aa96ffb78f84d57f28a2c95
2022-08-11 14:34:44 +00:00
wilfredasomani
cff8158054 satellite/console: add audit logs for failed logins
This change adds login failure logs for specific userID/email.

see: https://github.com/storj/storj/issues/4987

Change-Id: I58529145d7bd65abe47e002f34ec88018f641268
2022-08-11 12:11:21 +00:00
Márton Elek
f507de67f9 satellite: in-memory 'top'-like counter for project/partner
As a reminder
 * This counters are for data with high-cardinality
 * We have strong upper bound for memory limits
 * They can be accessed from /top monitoring interface

Example:

```
curl 172.20.0.10:11111/top
since ~ 2022-08-09T07:45:58Z
auth_request_count project=9094cff8-104e-4956-a367-97ea134b7e06 11.000000
auth_request_buckets  1.000000
auth_request_discarded  0.000000
auth_request_count partner=00000000-0000-0000-0000-000000000000 11.000000
auth_request_buckets  1.000000
auth_request_discarded  0.000000
```

Note: discarded 0 --> we didn't hit the memory limit.

Change-Id: I8db09b4aa61bade55cb324b84b7fbcb8f068c179
2022-08-11 10:21:54 +00:00
Márton Elek
0698b1ef88 go.mod: bump storj/private version
Change-Id: I72ff0a6d8b8d929f0753988e447bf2b41552199c
2022-08-11 09:08:32 +00:00
hovex023
4418216b4c
Web/satellite: Create Access Flow Modal Refactor (#5015)
Extrapolated each stage in the Access Grant flow into its own component and replaced the code on CreateAccessModal with the new components.
2022-08-10 17:36:41 -05:00
wilfredasomani
d8b010f7bd satellite/{web,satellitedb}: fix project daily usage error
The new dashboard currently gets stuck on loading and displays an error when
it fails to get usage data. Failure happens on satelliteDb due to a cockroach transaction error
caused by reading data before using AS OF SYSTEM TIME in the same transaction.
This change reverses the order of daily usage queries to avoid this error.
And hides the loaders on the dashboard if/when an error occurs.

see: https://github.com/storj/storj/issues/5012

Change-Id: I06b6ee434f72242f9b7d21dec7aaf39d1d622f1e
2022-08-10 19:18:28 +00:00
paul cannon
37a4edbaff all: reformat comments as required by gofmt 1.19
I don't know why the go people thought this was a good idea, because
this automatic reformatting is bound to do the wrong thing sometimes,
which is very annoying. But I don't see a way to turn it off, so best to
get this change out of the way.

Change-Id: Ib5dbbca6a6f6fc944d76c9b511b8c904f796e4f3
2022-08-10 18:24:55 +00:00
Lizzy Thomson
0550731598 web/satellite: add S3 credential creation to quickstart dropdown
updated the CreateAccessModal to support direct deep linking
added a new option in the quickstart dropdown that navigates to the
S3 CreateAccessModal and updated the quickstart Create Access Grant
action to use the new modal

Fixes https://github.com/storj/storj/issues/4961

Change-Id: I49388ebbfcfee74ae481c2f5b7afb7e6168190b5
2022-08-10 17:08:11 +00:00
Lizzy Thomson
b7e9d04af7 web/satellite: common table component applied to file browser
Fixes https://github.com/storj/storj/issues/4991

Change-Id: I7731c0425e16aa6ef75d06be83fc74aaafd1f2ab
2022-08-10 10:17:24 -06:00
NickolaiYurchenko
a4166548a7 web/satellite: dashboard banners fixed
call to action buttons were unreachable on some screens.
banners made scrollable since they occupies a lot of screen space when more than 1 appears.

Change-Id: Iaf0eb7170f432d260133f7d4c31ee6a4dae5d439
2022-08-10 14:58:56 +00:00
Michal Niewrzal
6cc2052f47 satellite: fix segment loop observers metrics
We made optimization for segment loop observers to avoid
heavy monkit initialization on each call. It was applied to very
often executed methods. Unfortunately we used wrong monkit
method to track function times. Instead mon.Task we used
mon.Func().

https://github.com/spacemonkeygo/monkit#how-it-works

Change-Id: I9ca454dbd828c6b43ba09ca75c341991d2fd73a8
2022-08-10 14:13:16 +00:00
Michal Niewrzal
917925bc11 satellite/metabase: delete migrated zombie objects
We noticed that in the system we have undeleted very old pending
objects. General rule is to delete them after some inactivity. Turns
out that all those objects are objects migrated to metabase from
previous DB schema. During this migration we didn't set
zombie_deletion_deadline to any value.

This change takes into account pending objects with zombie deletion
deadline set to nil during zombie deletion process.

I also checked accross all production satellites and youngest pending
objects with nil zombie_deletion_deadline are from 2021 so it is safe
to delete them.

Change-Id: Ie2b6a4b4e203c1750cf8408ee281c0631b263082
2022-08-10 10:16:47 +02:00
Clement Sam
25f8f678ab storagenode/nodestats: retrieve storage usage starting from last day of previous month
For the storagenode usage graph currently,
1. the graph is still likely to contain spikes on the first day of
the month for a newly setup storagenode because there's no previous
interval_end_time to deduct from.

2. if a node goes offline for too long, say the last usage report
in the storageusage cache has an interval_end_time of
2020-07-30 00:00:00+00:00 and later, it comes back online a few
days later, it requests for the storage usage from the satellite
starting from the current month, say 2021-01-01 00:00:00+00:00,
the calculated hours for the first day would be 48 hours and it
could be wrong because the cache is missing one day usage report.

This PR addresses second issue on the storagenode side by requesting
storage usage data, instead of just a month boundary, request for an
interval starting from the last day of the previous month to the
current day of the current month.
The first one will be a tradeoff and wouldn't really matter since
it will just be an issue on the first day the storagenode joined
the satellite.

Updates https://github.com/storj/storj/issues/4178

Change-Id: I041c56c412030ce013dd77dce11b0b5d6550927b
2022-08-10 01:37:02 +00:00
Clement Sam
9a539c4830 storagenode/storageusage: add interval_end_time, rename interval_start to timestamp
The satellite now returns the last interval_end_time for each
daily storage usage.
We need to store the interval_end_time in the storage usage cache.
Also, renamed interval_start to timestamp to avoid ambiguity since
the interval_start only stores just the date/day returned by the
satellite.

Updates https://github.com/storj/storj/issues/4178

Change-Id: I94138ba8a506eeedd6703787ee03ab3e072efa32
2022-08-10 01:03:00 +00:00
Qweder93
04c0e5d135 cmd/uplink: fix access setup flags setup
Added Setup of access maker call into cmd_access_setup to use flags during cmd call

Closes https://github.com/storj/storj/issues/4766

Change-Id: I0c75f224414099573b021b18b87d9e17192cecc5
2022-08-09 16:05:44 +00:00
Ivan Fraixedes
b398035053 satellite/console/consoleweb: Log trace reqs with INFO
Change from DEBUG level to INFO level the logs that the trace request
middleware logs because it looks that we don't log in DEBUG level in
production Satellite API pods.

For making that assumption I searched in the last 7 days logs collected
by Google Logging service for all the Satellite API pods in US1 and it
didn't show any line.

Change-Id: I620009d70d59df46d524c8cee93851bd13eceeee
2022-08-09 14:01:00 +00:00
Yaroslav Vorobiov
eca1689f17 satellite/console: add wallet payments API
Extends satelling console billing API with method to
retrieve payments for a particular wallet.

Change-Id: I2e82793c715db9353427e0a287baac57bf6a0f43
2022-08-09 14:53:54 +02:00
dlamarmorgan
174d2ec181 satellite/{payments/billing,satellitedb}: Add retry logic to insert
During an update to the billing DB, there is a special case failure that can occur if multiple updates to the table happen concurrently. In this case, the update would normally fail silently due to the balance constraint during update, and the subsequent insert for a new record fails because the user already exists in the table. The solution for this case, is to simply retry the insert with some limit to prevent infinite loops.

Change-Id: Ibe70fec2c386c25bd2484fe91f49a6a962357706
2022-08-08 14:42:01 -07:00
prerna-parashar
e8b716f97b
satellite/analytics: Added track calls for user behavioral events (#5009)
satellite/analytics: Added Access Grant page analytics events
2022-08-08 11:17:35 -07:00
Jeremy Wharton
731fecd96f private/apigen: rework request parameter handling
- Previously unused struct Endpoint.Request now defines the form
	of the request body.
- Path parameters (e.g. "id" in "/delete/{id}") are defined in
	the Endpoint.PathParams field.
- Endpoint.Params has been renamed to Endpoint.QueryParams to
	eliminate confusion.

Change-Id: Ifef51ca2f362c33086f0e43e936d50b0fdd18aa1
2022-08-08 15:41:24 +00:00
kimbotsao
3afd7bcc8b satellite/console: Reset passwords invalidates all current user sessions.
Logs out all current user sessions when a password is changed through both the
forgot password and change password methods.

Change-Id: Iaf9b4969aa45441591524906af326b9dec17939f
2022-08-08 14:49:51 +00:00
Malcolm Bouzi
4c5466aacf web/satellite: update projects table component
This changes project management page to match the new table designs.

Change-Id: I7d24f130d4703b7d273cfc0336cf679c2e6f3fd6
2022-08-08 13:28:37 +00:00
Erik van Velzen
9e64a87f7e satellite/metabase: measure delete object batch
Split out the function to delete a batch of objects from a bucket, so
that we get metrics which give a rough indication how long this operation
takes.

Part of https://github.com/storj/storj/issues/4957

Change-Id: I20a4ed5894217f4cd0b2f25aee297f0ecda57ab5
2022-08-08 12:24:26 +00:00