Commit Graph

5698 Commits

Author SHA1 Message Date
Yingrong Zhao
c8f4ff4135 satellite/satellitedb: migrate existing data from overlaycache and
audit_histories to reputations table

Change-Id: Ia47e5264b37d2b7585c8486a4e022f4559233bf0
2021-07-28 15:16:52 -04:00
Yingrong Zhao
646ce5b8cc satellite/overlay: remove reputation logic from overlay
Change-Id: I3492860e4537c7a8e4e824ec4c9c8d179134a0c0
2021-07-28 15:15:28 -04:00
Yingrong Zhao
f8914ccce0 satellite/{repair, overlay}: use reputation store in repair
Change-Id: I48db9e68f48239d48621ccc77d33618ecb83ce1a
2021-07-28 13:22:05 -04:00
Yingrong Zhao
58238d850c satellite/{audit, accounting}: use reputation store in tests
Change-Id: I86a8ccf5dcee8d108196a9f67a476fe0ccbd8257
2021-07-28 13:21:55 -04:00
Yingrong Zhao
e91574cee1 satellite/{reputation, gracefulexit}: use reputation store in
gracefulexit

With the effort to move audit related data into reputation store, this
PR updates gracefulexit endpoint to use reputation service to get a
node's audit score

Change-Id: Iad93ea689ad67ff9c57c7be16687e21e715fab7a
2021-07-28 13:21:41 -04:00
Yingrong Zhao
55a77d04bc satellite/satellitedb,private: add initial value on testplanet startup
Currently, reputation table is only populated when a node has been
audited. This is ok in production, however a lot of our tests doesn't
upload any data or trigger audits.
This PR adds an initialization step in testplanet to populate reputation
table with zero value for nodes reputation.

Change-Id: I11b381236669db346dc68a48a6d4a27334a0a8b8
2021-07-28 13:20:32 -04:00
Yingrong Zhao
6c7bf357cd satellite/{reputation,audit,overlay}: replace overlay with reputation
package in audit

This PR implements reputation store and replace overlay in audit service
to use such store for storing node's audit stats.

In order to keep the changeset smaller, most of the changes in this PR is for copying audit logic in overlay to
reputation package. In a following PR, the duplicating code will be
removed from overlay.

Change-Id: I16c12494a0970f44c422b26cf603c1dc489e5bc1
2021-07-28 13:10:48 -04:00
Jeremy Wharton
6bbb5b5cc2 web/satellite: Static error page maintenance fixes
Unifies the stylesheet for 404 and 500 errors. References the
Storj logo rather than embedding it in the HTML. Replaces references
to Storj with Storj DCS. Removes reference to nonexistent font.

Change-Id: If224f30eb0b21171fc8f575de6c4cef98a48c6f5
2021-07-28 16:18:27 +00:00
Jeremy Wharton
49fbb5010f web/satellite: Merge and update 'Forgot Password' page files
Merges the 'Forgot Password' page files into one Vue file and updates
it to be consistent with deployed code. This change is part of an
ongoing effort to remove code from the branding repo.

Change-Id: Ib18b95dd5a3269f26f4f547fc46ac6278c468a54
2021-07-28 14:40:59 +00:00
Moby von Briesen
8f072bdeee cmd/satellite: Skip non-existing users in paid tier conversion
There are some users in our QA satellite which are no longer in Stripe,
and there are some users in Stripe which are not on our QA satellite.
This change allows us to test the paid tier conversion script in QA
despite these problems.

Change-Id: If94c9e882327841d1fd294d75fd302e6a7feee41
2021-07-27 12:53:58 -04:00
Clement Sam
d73b9fff9a satellite/orders: set the expirationDate in CreatePutRepairOrderLimits
In the past ExpirationDate was available inside CreatePutRepairOrderLimits
but this was removed since the metabase segment was missing the ExpiresAt field.
Now ExpiresAt field is available in the metabase segment
and can be set correctly while executing NewSignerRepairPut.

Change-Id: I068c07492ab27bde2c44477bbd32c5872edd024a
2021-07-27 12:44:40 +00:00
Michał Niewrzał
a2b30e42a3 satellite/metainfo: log internal errors
We are missing some logging around errors returned as
Internal.

Change-Id: Ifc41634816648172ff944808d23d1e6a344e41f1
2021-07-27 09:57:59 +00:00
Jeremy Wharton
3720afb81b web/satellite: Update emails to be consistent with deployed code
Update the forgot password, invite, and welcome emails to be
cosistent with deployed code. This change is part of an ongoing
effort to remove code from the branding repo.

Change-Id: I6edb6811904552c1cf7761b96bc9aa879c2eb154
2021-07-26 22:49:28 +00:00
Jeremy Wharton
276ebcd4b6 satellite: Remove duplicate logo in error pages
Remove duplicate instance of the Storj logo in the
40X and 50X error pages.

Change-Id: I74152560d39f34fb93f5d559b9b390c3e83a61ec
2021-07-26 20:30:16 +00:00
Moby von Briesen
149f6f2626 satellite/payments: Implement coupon codes
Full path: satellite/{payments,console},web/satellite

* Adds the ability to apply coupon codes from the billing page in the
satellite UI.
* Flag for coupon code UI is split into two flags - one for the billing
page and one for the signup page. This commit implements the first, but
not the second.
* Update the Stripe dependency to v72, which is necessary to
use Stripe's promo code functionality.

Change-Id: I19d9815c48205932bef68d87d5cb0b000498fa70
2021-07-26 17:15:55 +00:00
Jeremy Wharton
dae6ed7d03 satellite/console: Implement MFA backend
Added MFA passcode and recovery code field for token requests.
Added endpoints for MFA-related activity: enabling MFA,
disabling MFA, generating a new MFA secret key, and
generating new MFA recovery codes.

Change-Id: Ia1443f05d3a2fecaa7f170f56d73c7a4e9b69ad5
2021-07-26 16:37:05 +00:00
Michał Niewrzał
420d2f6275 metabase-orphaned-segments: avoid processing recently created elements
To be sure we are comparing the same set of objects
and segments lets ignore objects and segments created
after processing was started.
Segmets without objects cannot be created in normal
way so we will have them only if we broke something in
the past.

Change-Id: I96c07caf9e5091775d4dc8dfc0fef2b08b87957c
2021-07-26 15:50:20 +00:00
StobR
42b113d92b
web/storagenode: fix ingress chart tooltip
Fixes z-index issue for tooltips when ingress graph is mouseover.
2021-07-26 15:19:40 +03:00
StobR
267a962c3b
web/storagenode: fix egress chart tooltip 2021-07-26 14:51:40 +03:00
Michał Niewrzał
a883d7f582 satellite/repair/checker: fix remote_files_checked metric
While metaloop refactoring we missed metric for all
objects processed by repair checker.

Change-Id: I100f10a36c52e2651923ecaa377261752877d673
2021-07-22 14:48:08 +00:00
Michał Niewrzał
b12d29935a satellite/metabase: remove metaloop package
We moved everything to segment loop so we can now
remove metaloop from code.

Change-Id: I9bd8d2349e5638d7cdad50f2f313f9bd89a8165c
2021-07-22 13:00:45 +00:00
Jeremy Wharton
1535bbe673 satellite/console: Forward friendly registration errors to client
Provides the means to serve an error to the user with a user-friendly
error message (serveCustomJSONError). Auth API uses this when
processing registration attempts.

Previously, the error message was inferred by the client based on
the status code of the response received from the server. However,
if multiple distinct errors fit a certain status code, it was impossible
to correctly interpret the error.

Change-Id: I2f91e9c81ba1a4d14ba67e0b4b531a48800d4799
2021-07-22 11:31:12 +00:00
Michał Niewrzał
237360e765 metabase-orphaned-segments: fix args binding for delete command
As an additon I clarify log message a little bit.

Change-Id: Id2487cc218e3dd62ad5a05b56e8b1d5d070676a7
2021-07-22 11:00:37 +02:00
Fadila Khadar
6d60d412f0 satellite/gracefulexit: use segment loop
Join the segment loop instead of the metainfo loop, to iterate only over segments.

Change-Id: I06259d363b98d4e191f2bf2d82c9b47255ee484a
2021-07-21 15:12:25 +00:00
Fadila Khadar
c4202b9451 satellite/gracefulexit: use graceful_exit_segment_transfer_queue
For being able to use the segment metainfo loop, graceful exit transfers have to include the segment stream_id/position instead of the path. For this, we created a new table graceful_exit_segment_transfer_queue that will replace the graceful_exit_transfer_queue. The table has been created in a previous migration and made accessible through graceful exit db in another one.
This changes makes graceful exit enqueue transfer items for new exiting nodes in the new table.

Change-Id: I7bd00de13e749be521d63ef3b80c168df66b9433
2021-07-21 14:02:20 +00:00
Cameron Ayer
449c873681 satellite/repair/repairer: attempt repair GETs using nodes' last IP and port first
Sometimes we see timeouts from DNS lookups when trying to do
repair GETs. Solution: try using node's last IP and port first.
If we can't connect, retry with DNS lookup.

Change-Id: I59e223aebb436118779fb18378f6e09d072f12be
2021-07-21 13:13:06 +00:00
Fadila Khadar
b0d98b1c1a satellite/gracefulexit: allow use of graceful_exit_segment_transfer_queue
For being able to use the segment metainfo loop, graceful exit transfers have to include the segment stream_id/position instead of the path. For this, we created a new table graceful_exit_segment_transfer_queue that will replace the graceful_exit_transfer_queue. The table has been created in a previous migration.
This change gives access to this table.
Graceful Exit doesn't use the table yet, this will be done in a next change.

Change-Id: I6c09cff4cc45f0529813a8898ddb2d14aadb2cb8
2021-07-21 12:34:44 +00:00
Qweder93
73cdefbc41 munltinode/console: node online statuses extended with StatusUnauthorized and StatusStorageNodeInternal
Change-Id: I12c90169f8959e5eafe2a64bf8b412f7afc48c1c
2021-07-21 15:00:41 +03:00
Michał Niewrzał
f73fcee892 cmd: add tool for orphaned segments
This tool has two commands to execute. One is to 'report'
orphaned segments. Second to 'delete' orphaned
segments.

To find such segments tool is first finding all unique
segment stream ids. As a next step its removing from this
list stream ids of existing objects. What is left is a list of
orphaned segments.

Change-Id: I4a0ae3ad0b10a8d16572bfd22ac92cfa15ca19b3
2021-07-21 10:10:58 +00:00
Cameron Ayer
373ba8fd27 satellite/repair/repairer: metrics for repair bytes uploaded and downloaded
Change-Id: Icb0850692ecc155f6c8169edf1b045b2b546ff48
2021-07-21 09:23:19 +00:00
Cameron Ayer
adc0fbddfa satellite/audit: don't fail nodes for audit if not enough pieces downloaded
In most situations where we would not get enough shares to complete
an audit, something has probably gone wrong like a forgotten delete,
and nodes should not be failed. We have an alert when this occurs.
Check the logs to see what happened. If we decide the nodes should
get audit failures, we can do it manually.

Change-Id: Ib6e408082048d31197c37ebfd7f9031135fc938f
2021-07-20 20:28:18 +00:00
Michał Niewrzał
27a714e8b0 satellite/accounting/tally: use objects iterator instead metaloop
Bucket tally calculation will be removed from metaloop and will
use metabase objects iterator directly.

At the moment only bucket tally needs objects so it make no sense
to implement separate objects loop.

Change-Id: Iee60059fc8b9a1bf64d01cafe9659b69b0e27eb1
2021-07-20 15:52:18 +00:00
Vitalii Shpital
4c7384ca2f web/satellite: removed redundant files splitting
Since we removed login/registration explicit templates and styles from branding repo we don't need file splitting in main repo anymore.
So moved html and css code into components

Change-Id: Ib297e2576f16a7fddd17cc62f6c655adf96a59b6
2021-07-20 12:44:02 +03:00
Vitalii Shpital
2489b78d22 web/satellite: move user MFA secret generation to server-side
It was decided to move user MFA secret generation to server-side and be fetched by http request

Change-Id: I5e31d35fb78d1d9f3280518fe3eb543bb8ed4377
2021-07-19 13:48:09 +00:00
Egon Elbre
5a56021bca satellite/metabase: add intLimitRange
We have quite a bit code duplication in ensuring valid int ranges.

Change-Id: Ib31db8c4b2be6a677625cab5bcdddefb60401fe3
2021-07-19 07:51:16 +00:00
Moby von Briesen
bc2f81c2fa web/satellite: Add beta checkbox to registration page
Only if is-beta-satellite is set to true.
Also cleans up some styles.

Change-Id: I2273936ae2b41a3409fb985b13cc987224a3ba85
2021-07-16 15:55:11 +00:00
Moby von Briesen
7624bdd090 web/satellite: Return server-side error message during registration
It turns out, there are multiple different 400 errors that might be
returned from the server during registration. Rather than display
hardcoded text on the client ("Validation of reCAPTCHA was
unsuccessful"), this change simply displays the error sent by the
server.

As mentioned in the comment, we should eventually do this for all
errors, but that will be a more in-depth change. The purpose of this
commit is to unblock a point release.

Change-Id: Ideca107cc4039a0dabfa0fb02c943da920f7ff4f
2021-07-16 09:43:11 -04:00
Jeremy Wharton
1e0a1b15d4 web/satellite: Repair reCAPTCHA resetting upon registration error
Previously, the reCAPTCHA would only reset if a Bad Request
error was received. Resets should initiate regardless of error
type to prevent the same reCAPTCHA response from being submitted
more then once.

Additionally, registration errors could trigger a reset for a
nonexistent reCAPTCHA element.

Change-Id: Ib405707f1803cf41e5de192f31d75153136e6c67
2021-07-15 15:42:52 -05:00
Qweder93
4a98dd40e2 storagenode/payouts: historical payouts use satellitesDB instead of trustPool
Change-Id: I39f4215f4ebf91bd1b38fbcb5c58e6ba53ceff1b
2021-07-15 16:19:18 +03:00
Bill Thorp
18c3252025 cmd/uplink: recommend notAfter expiration caveat be set
The auth service has no way to remove access grant registrations that
lack expiration dates.  We want to encourage people to set them, so as
to slow the rate at which the auth service DB fills up.

Change-Id: I1ccf629cd995dc184d2d90333166eab34d34ae07
2021-07-14 17:05:28 +00:00
Vitalii Shpital
22e88c8f0a web/satellite: disable MFA functionality
Added disable user MFA functionality to account settings.
Disable MFA popup where user will have to enter MFA passcode first.
2 buttons (enable, disable)) are visible for now until backend is ready.

Change-Id: Iff8b497a370fc1c6c08c5ccaf01ce1f2dc317126
2021-07-14 17:38:57 +03:00
Vitalii Shpital
e463eb17ac web/satellite: added enabling user MFA functionality to account settings
Added feature flagged functionality for enabling user MFA.
Added new Popup where user will scan qr code and confirm enabling
by entering passcode from MFA app. Also recovery codes will be visible afterwords

Change-Id: Ie8d1bc83c941a08fd8701442601a2d20126c8892
2021-07-14 15:13:59 +03:00
Jeremy Wharton
0d8010e353 satellite/satellitedb: Implement multi-factor authentication db
Columns for MFA status, secret key, and JSON-encoded array of
recovery codes are added to the users table.

Change-Id: Ifed7e50ec9767c1670d9682df1575678984daa60
2021-07-13 18:18:36 -05:00
Moby von Briesen
4c912c7479 web/satellite: Update paid tier banner copy
Also, show paid tier banner if a user has not added a credit card,
rather than if they are flagged as "paid_tier" in the database. This
addresses issues where we might display the banner for users who have
already added a credit card before the paid tier commit: e36001b7cf

Change-Id: I0352d48ae9f97ceab81ae065ccb97be3e5516857
2021-07-13 13:34:47 +00:00
Yaroslav Vorobiov
6db6b76b27 multinode/console: embed web assets
Embed web static files in multinode binary to be able to
release multinode as single binary.
Add make commands to build multinode binary with embeded web
assets.

Change-Id: I348aff7a7d847fae5c021cbf59abc7f892c0df80
2021-07-12 18:51:24 +03:00
Moby von Briesen
30cd7d3da3 satellite/console: Update CSP for satellite UI
* Add recaptcha to Content Security Policy
* Remove segment and customer.io from Content Security Policy

Change-Id: Ie6d767d8c023f7e69f475c37f1535e4db90953fc
2021-07-12 10:32:15 -04:00
Moby von Briesen
5870502589 cmd/satellite: Add billing command for converting customers to paid tier
We have implemented the paid tier, but it currently only handles new
users entering paid tier. It does not convert users who have already
added a credit card previously. We still want to convert these users'
project limits. This billing command can be run once to convert all old
customers with a credti card. Afterwards, we should be able to safely
remove it.

Change-Id: Ia496580b8e72ef436375b74f590fe57cca704fa8
2021-07-12 13:37:37 +00:00
Vitalii Shpital
8855c0dff7 web/satellite: added MFA feature flag, updated client-side api and Vuex store module
Added feature flag for MFA
Added new client-side api call to enable MFA returning secret
Updated users Vuex module to include new API call

Change-Id: Ia9e10f68c4a7da39b4f7c1073e657c2de98fb0db
2021-07-12 10:12:59 +00:00
Brandon Iglesias
ee107fe8cd
satellite/rewards: adding partners to list (#4159) 2021-07-09 22:47:25 +02:00
Michał Niewrzał
76c8d5d2cb satellite/metrics: move metrics to segment loop
Change-Id: Iccc0e4b6d531590c9cc57c7c74a6ce4c46b7d6a2
2021-07-08 20:29:10 +00:00