Commit Graph

324 Commits

Author SHA1 Message Date
Cameron
0633aca607 satellite/console: create new consoleauth service
We want to send email verification reminders to users from the satellite
core, but some of the functionality required to do so exists in the
satellite console service. We could simply import the console service
into the core to achieve this, but the service requires a lot of
dependencies that would go unused just to be able to send these emails.

Instead, we break out the needed functionality into a new service which
can be imported separately by the console service and the future email
chore.

The consoleauth service creates, signs, and checks the expiration of auth
tokens.

Change-Id: I2ad794b7fd256f8af24c1a8d73a203d508069078
2022-05-13 16:27:07 +00:00
Moby von Briesen
763bfc0913 satellite/console,web/satellite: Implement hCaptcha
Adds a new configuration for hcaptcha enabled, secretkey, and sitekey.
If both reCAPTCHA and hCaptcha are configured as "enabled", reCAPTCHA
will be used.

Change-Id: I73cc6e133d8da3555e0ed8b2b377cf9eb263e6dc
2022-05-13 14:57:45 +00:00
Erik van Velzen
db1cc8ca95 satellite/repair/checker: buffer repair queue
Integrate previous changes. Speed up the segment loop by batch inserting
into repair queue.

Change-Id: Ib9f4962d91960d21bad298f7771345b0dd270276
2022-05-12 16:28:05 +00:00
Vitalii
dedccbd2e4 satellite/console, web/satellite: limit failed login attempts
Added account locking on 3 or more login attempts.
Includes both password and MFA failed attempts on login.
Unlock account on successful password reset.

Change-Id: If4899b40ab4a77d531c1f18bfe22cee2cffa72e0
2022-05-11 14:49:11 +00:00
Qweder93
384118ee1e satellite/metabase: use pgutil.NullTimestampTZArray
This code cleanup to use helper DB method.

Change-Id: Idc438234cb4f7da23d6ade2d69f5d889f15e9009
2022-05-03 19:30:21 +00:00
hovex023
58f957bd1d
web/satellite: Add new access grants flow (#4765)
* Added new feature Flag for new Access Grant Flow. 

* Added 3 cards to access grant view for S3, CLI and Access grant to replace old header

* Added new formatting, text and Icon for Access Grant Delete Popup modal
2022-04-29 10:31:52 -05:00
nadimhq
dd6aac9b39
satellite/metainfo: enable server-side copy in production (#4708)
these changes enable server-side copy in production
2022-04-26 09:58:16 -04:00
Moby von Briesen
ed5ebb2527 satellite: Rename "acct mgmt api" to "rest api"
"REST API" is a more accurate descriptor of the generated API in the
console package than "account management API". The generated API is very
flexible and will allow us to implement many more endpoints outside the
scope of "account management", and "account management" is not very well
defined to begin with.

Change-Id: Ie87faeaa3c743ef4371eaf0edd2826303d592da7
2022-04-25 18:51:46 +00:00
Michał Niewrzał
6188c4c0dd satellite/metainfo: enable segment limit by default
Change-Id: I26cc0bbd97d6cc2ffc96213788f261efb749d028
2022-04-14 06:12:09 +00:00
Cameron
84b522bc06 satellite/console: create account management api keys service
We are in the process of creating an api to allow users to manage their
accounts programmatically. We would like to use api keys for
authorization. We were originally going to create an entirely new table
for these api keys, but seeing as we already have 2 other tables for
keys/tokens, api_keys and oauth_tokens, we thought it might be better to
use one of these. We're using oauth_tokens.

We create a new oidc.OAuthTokenKind for account management api keys:
KindAccountManagementTokenV0. We made the key versioned because we
likely want to improve the implementation in the future, but we want to
get something functional out the door ASAP because the account management
api feature is highly desired.

Add a new method to oidc.OAuthTokens interface for revoking v0 account
management api keys, RevokeAccountManagementTokenV0. Add update method
to dbx implementation to allow updating the expiration. We will revoke
these keys by setting the expiration to 0 so they are expired.

Change-Id: Ideb8ae04b23aa55d5825b064b5e43e32eadc1fba
2022-03-23 17:02:20 +00:00
littleskunk
6fbc56bd06
satellite/payment: enable segment fee (#4628) 2022-03-18 14:25:14 +01:00
Mya
4a110b266e satellite/console: added oidc endpoints
This change adds endpoints for supporting OpenID Connect (OIDC) and
OAuth requests. This allows application developers to easily
develop apps with Storj using common mechanisms for authentication
and authorization.

Change-Id: I2a76d48bd1241367aa2d1e3309f6f65d6d6ea4dc
2022-03-16 12:01:26 +00:00
dlamarmorgan
715077d04a satellite/{console,satellitedb}: increase project limit on paid tier upgrade
Update the user to the default paid tier project limit, which is currently 3 projects, when the user upgrades to a paid account.

Change-Id: I95b19d62cebc7d878b716355f2ebcaf0b51ca3f7
2022-03-14 16:10:23 +00:00
Michał Niewrzał
39ac90835f satellite/satellitedb: change default segment limit to 100M for paying users
We decided that we want to have segment limit for paying users high
enough to not have to change it too often.

Fixes https://github.com/storj/storj/issues/4590

Change-Id: Ic1c38bf3e2fcc000548ff4c7e7004647b39fbecf
2022-03-04 13:18:14 +00:00
Moby von Briesen
b2d342aa9b satellite/overlay: Add ability to exclude country codes on upload
Create global config to specify a list of country codes that should be
excluded from node selection during uploads.

This exclusion is not implemented when the upload selection cache is
disabled.

Change-Id: Ic41e8b4f18857a11045668eac23107da99668a72
2022-03-03 16:58:48 +00:00
NickolaiYurchenko
64176aaca4 web/satellite: registration success redirect from config
This change allows us to send newly registered users to a configured URL
to help us track user conversions for marketing campaigns.
Brave conversions continue to be tracked using the /signup-success page
within the satellite app.

Change-Id: I9b451947ce0f39d3c99b233cb4b806d361151823
2022-03-03 10:13:21 -05:00
Fadila Khadar
e776c65172 satellite/checker: pieces in excluded countries are not healthy
Add a RepairExcludedCountryCodes config flag for overlay for providing a list of country codes to exclude nodes from target repair selection.

Mark segments with less than repairThreshold pieces in countries not in the RepairExcludedCountryCodes as not healthy.
With this change, the repair process is not affected. The segment will be removed from the repair queue by the repairer.

Another change will handle the logic at the repairer level.

Fixes https://github.com/storj/team-metainfo/issues/95

Change-Id: I9231b32de117a116488de055a3e94efcabb46e81
2022-03-02 09:59:09 +00:00
Vitalii Shpital
ba6956db0f console/server, apigen: feature flag for new generated console api
Added a feture flag which will be used to indicate if new generated console api is used.
Fixed some comments from previous PR.

Change-Id: Ice31c998b0b347028a491c971a648fd1269bfd49
2022-02-28 23:00:12 +00:00
Michał Niewrzał
fbe2680500 satellite/metainfo: add feature flag for server side copy code
We would like to disable in production those parts of code
which are now mixed with new server-side copy logic.

Change-Id: Iff50682bc9545207330f58dd19b5eee53d404d7f
2022-02-24 10:43:49 +00:00
NickolaiYurchenko
70c8ccb435 web/satellite: inactivity timer to dashboard added
added InactivityTimerEnabled flag to enable/disable feature
added InactivityTimerDelay to configure delay time in seconds
default timer set up to 10 minutes
reset dom events: keypress, mouseover, mousedown, touchmove

Change-Id: Idb66067c2902b2cdbe1a972225319c8abff97927
2022-02-16 07:16:00 +00:00
Vitalii Shpital
b3e1be37ff satellite/projectaccounting: query to get daily project usage by date range
Finished implementing queries for both bandwidth and storage using pgx.Batch.
Fixed CSP styling issue.

Change-Id: I5f9e10abe8096be3115b4e1f6ed3b13f1e7232df
2022-02-04 22:32:37 +00:00
Stefan Benten
e7e39fe628
satellite/metainfo: reduce default user rate limit to 100
Currently the rate limit has kept per satellite api endpoint.
Since we run 9+ api endpoints in production, we do not need
a limit of 1000, since the intention was to allow 1000 total.
This change reduces the effective limit given 9 instances
down to 900, which should be close enough.

Change-Id: Ia579149ccc3a12e8febe0cfd5586b8a39de40f55
2022-01-21 23:37:59 +01:00
Michał Niewrzał
270c6e24f2 satellite/console: fix default limit for free-tier segment usage limit
Free-tier segment usage limit was defined as 150k, not 140k. This change
is correcting that.

https://github.com/storj/team-metainfo/issues/8

Change-Id: I71ec0961930b19fd09b2b996e01acd406a8dcf8f
2022-01-19 09:21:13 +00:00
Qweder93
8be655c487 satellite/metainfo: add segment limit validation, caching
We want to be able to limit the number of segments per project for users.
To limit this we need to check limit value associated with project
and value of used segments already in BeginMoveObject, BeginMoveSegment
and increment cache segments usage after each CommitSegment call.

Resolves https://github.com/storj/team-metainfo/issues/1

Change-Id: I6290e67c095a174b9d101c4521802d9bfe0453b8
2022-01-10 14:33:26 +02:00
Egon Elbre
04d04c0306 web/satellite: remove old object browser
Change-Id: Ief16a3887444df093c644831d86a9b3e865e5867
2022-01-05 16:29:19 +02:00
Malcolm Bouzi
6a059b7f9d web/satellite: remove all references to the old onboarding flow
Change-Id: I1f42fabdb407a7eb9c34f0341d332e66c3ce288d
2022-01-04 18:40:08 +00:00
Michał Niewrzał
87a9292127 satellite/metainfo/zombiedeletion: enable chore by default
We enabled this chore on different satellites
for testing. Works fine so we can enable this
by default for all configurations.

Change-Id: I987639685d8de5c7e5798adca30fe26bdac9e1d1
2021-12-21 13:26:18 +01:00
Michał Niewrzał
3f0babffe8 satellite/{console,satellitedb}: add project segment limit to user
All limits we have for projects have also parent limits stored
with user data. New created project is first taking limits from
owner (user) limits.
This change is extending users table with project_segment_limit
column and adds functionality to get and set value for this
column.

Change-Id: Iff5e36c62b517652390b649fc05992475916ecff
2021-12-16 08:46:01 +00:00
Jeremy Wharton
9d13c649a2 satellite/{console,satellitedb}: Forbid creating users with used email
This change disallows creation of users possessing the same email.
If a user attempts to create an account with an email address
that's already used - whether it belongs to an active account or not -
he will be notified of unsuccessful account creation. If he attempts to
log in using an email address belonging to an inactive account,
he will be presented with a link allowing him to re-send the
verification email. Attempting to register with an email address
belonging to an existing account triggers a password reset email.

Change-Id: Iefd8c3bef00ecb1dd9e8504594607aa0dca7d82e
2021-12-15 17:24:35 +00:00
prerna-parashar
a41758bba5
Track user events through HubSpot events API (#4300)
Track user events through HubSpot events API
2021-12-09 12:52:51 -08:00
Igor
4daaa9503e
satellite/console: enable object flow by default (#4298) 2021-11-29 16:43:37 +02:00
Vitalii Shpital
3de7f8d5af satellite/console, web/satellite: feature flag for new project dashboard
Added a feature flag to switch between old and new project dashboard

Change-Id: Ibb1b04ccbddb43e3f41116e3513f917deba95699
2021-11-17 16:02:47 +02:00
andriikotko
09568b3e2b
satellite/console: change default feature flag (#4274)
* satellite/console: change default feature flag
2021-11-16 11:38:42 +02:00
dlamarmorgan
44b1ca6b97 satellite/{console,satellitedb}: move project limits from config file to DB to keep limits on a per user basis
To allow for changing limits for new users, while leaving existing users limits as they are, we must store the project limits for each user. We currently store the limit for the number of projects a user can create in the user DB table. This change would also store the project bandwidth and storage limits in the same table.

Change-Id: If8d79b39de020b969f3445ef2fcc370e51d706c6
2021-11-11 15:10:00 +00:00
Ivan Fraixedes
4e67ea007c satellite/admin: Serve static UI assets
Change the satellite Admin HTTP server for:

* Embedding the UI assets into the Go binary.
* Serve the UI assets from the embedded file system or from a specific
  directory path through a configuration flag, without requiring
  authentication but keeping the authentication verification for the API
  endpoints.
* Add tests to verify that the UI assets are served without
  authentication.

Change-Id: I9003ac96f1ec585a189b67fc1cb315905403d557
2021-11-10 18:47:46 +02:00
Mya
bf51c286d9
satellite/geoip: update node check-in to associate a country code
Resolves https://github.com/storj/storj/issues/4247

Change-Id: Idfd71bf1795d48ca3c686066bbdb95b9c6594f00
2021-11-10 16:44:41 +01:00
Vitalii Shpital
3c683998f5 satellite/console, web/satellite: feature flag for new objects flow
Added a feature flag to switch between old and new objects flow

Change-Id: Id1802f9fb1230928445f16e4d0a62aecdc91c038
2021-11-08 11:45:22 +00:00
Egon Elbre
d043b9facd satellite/console: load index.html from disk when developing
This eases development by not needing to restart satellite after each
change.

Change-Id: I1063b7f431f675fb21e82dcfc2420310a9806422
2021-11-03 18:19:40 +02:00
Egon Elbre
edb8d656de satellite/metainfo: adjust piecedeletion timeouts
Currently slower storagenodes can slow down deletion queue.
To make piece deletion faster reduce the maximum time spent in
either dialing or piece deletion requests.

With this change:
* dial timeout is 3s
* request timeout is 15s
* fail threshold is set to 10min

Similarly, we'll mark storage node as failed when the timeout occurs.
The timeout usually indicates that the storagenode is overwhelmed.
Garbage collection will ensure that the pieces get deleted eventually.

Change-Id: Iec5de699f5917905f5807140e2c3252088c6399b
2021-10-28 13:37:01 +03:00
Yingrong Zhao
d441c8da15 satellite: use segment count for billing
Change-Id: Iafaf41cb6cde4cb2abdac3f75ce0fead838cb758
2021-10-27 19:21:50 +00:00
dlamarmorgan
01cfbde56b satellite/console: increase free tier project bandwidth and storage from 50 GB to 150 GB and reduce free tier max projects from 3 to 1
To make our free tier limits more clear, we will reduce the number of projects allowed from 3 to 1, and increase the storage and bandwidth limit of the free tier from 50 Gb to 150 GB. The total allotments across all projects for a given user are unchanged, just reduced to a single project.

Change-Id: Ic8dddb135f2b83a3f36e2b9fdcb477e351ec137b
2021-10-25 17:34:19 +00:00
Qweder93
50baefa10e satellite/metabase: limit maximum number of parts and size
Multipart upload limits added. Last part has no size limit.
Max number of parts: 10000, min part size: 5 MiB

Change-Id: Ic2262ce25f989b34d92f662bde720d4c4d0dc93d
2021-10-22 10:16:37 +00:00
Monty Anderson
3989107031 web/satellite: add feature flag for new browser
Change-Id: If3773b9df7f79d4c5e094140448d60ec060cdaeb
2021-10-22 03:49:08 +00:00
Vitalii Shpital
0009447773 satellite/console, web/satellite: feature flag for new navigation structure
Change-Id: I6f1fd43ea8154714187c7547ae93de0efe3482e6
2021-10-20 13:51:25 +00:00
littleskunk
016ac75052
satellite/console: Enable new onboarding workflow (#4222) 2021-10-15 21:07:35 +02:00
Jeremy Wharton
3b751a35c5 satellite/{payments,satellitedb}: Remove custom coupon implementation
Removes database tables and functionality related to our custom
coupon implementation because it has been superseded by the Stripe
coupon and promo code system. Requires implementations of the
payments Invoices interface to return coupon usages along with
invoices.

Change-Id: Iac52d2ff64afca8cc4dbb2d1f20e6ad4b39ddfde
2021-10-11 19:47:00 +00:00
Vitalii Shpital
8b91c55ec4 web/satellite, satellite/console: return old onboarding flow with feature flag
Added new feature flag for switching between different flows.
Got back old onboarding flow.

Change-Id: Idbf23ba8ff36900a47e3b4ebeb34b78351202e29
2021-09-21 11:37:30 +00:00
Egon Elbre
6e660cecdd Jenkinsfile: test cross-compile and bump deps
Change-Id: I47091de2e80bd96fcada616f75e1db07a59fb0c1
2021-09-16 18:59:31 +03:00
crawter
e1a4195a35 satellite/metainfo/endpoint: MaxObjectKey length validation for BeginObject and CommitObject added
Change-Id: I6a766415a84b1e3c44ce65052ffa8ed4511d00dd
2021-09-13 15:07:16 +03:00
Michał Niewrzał
b7555980c4 satellite/metainfo: add zombiedeletion chore
This chore will be deleting zombie objects and it's segments.

Change-Id: Ia5d92dd6c3f52e178d4d43d9603732310761e027
2021-08-24 08:22:06 +00:00