Commit Graph

534 Commits

Author SHA1 Message Date
Cameron
4815cfc09b satellite/{console,satellitedb}: add PublicID to Project, db method GetByPublicID
github issue: https://github.com/storj/storj/issues/4861

Change-Id: Ia83635c0de751a77cd5a49d641da19ed76132c46
2022-07-21 19:49:26 +00:00
Lizzy Thomson
c237468ac9 satellite/console: allow user to update project when limits are above paid defaults
When a user's bandwidth/storage limits are manually set to exceed the
paid tier defaults, attempting to update their project via the satellite
UI (e.g. to change the name/description) would result in an error.
This change modifies the limit checks for updating a project to remove
this issue.

https://github.com/storj/storj/issues/4892

Change-Id: I48853a3289b0ac51587f268a18c1b25743123fcf
2022-07-21 08:40:18 -06:00
Ivan Fraixedes
7aaab3c4c4 satellite/console/consoleweb: Trace HTTP endpoints
Trace all the requests that the HTTP API endpoints receive.

We want to trace them with Monkit because we want to break them down by
request type and response code for seeing if they succeeded or failed.

Also log them with DEBUG level with the IP client.

Change-Id: Ia7b013351c788f131e775818f27091f3014ea861
2022-07-18 13:27:50 +00:00
Vitalii
69dc9a4731 satellite/console: added new email which is sent on unknown password reset
Added new email html template.
It is sent when user tries to reset password with unknown or unverified account.
Made a couple of minor config changes.

Issue: https://github.com/storj/storj/issues/4913

Change-Id: I730f48b3478e302d1e38e1f8a27c75f66a8ba6fd
2022-07-14 14:32:59 +00:00
Egon Elbre
4e31c96836 satellite/console: don't use global log
Change-Id: I2464f2aca3cdb97c19de29dac59499cc9d5ff2bb
2022-07-06 16:50:04 +00:00
Ivan Fraixedes
1fbc8f1f40 satellite/console/consoleweb: Delete old TODO comment
Delete an old TODO comment which doesn't apply anymore.

Closes https://github.com/storj/storj/issues/4958

Change-Id: Ie4be51afb6a39b3f5e5822ac04d1c8fd9d86ee3a
2022-07-01 11:31:19 +00:00
Egon Elbre
05e165283f storagenode/console/consoleapi: use fixed time.Now()
It seems the tests relied on time.Now(), which might cause some
discrepancies in calculations. Use a fixed time.Now() rather than
recalculating.

As a sidefix, remove "Test" prefix from t.Run. These are unnecessary.

Change-Id: I1de903fcf0fcf46fc8e3acf2463e17239b8e3cc6
2022-07-01 12:36:01 +03:00
littleskunk
cf750716ea
satellite/console: enable new access grant flow (#4934) 2022-06-30 00:06:39 +02:00
Ivan Fraixedes
0051298eec satellite/console: Classify errors activation tokens
Classify errors related to invalid tokens for activating user accounts
for returning 400 status code rather than 500 status code.

Don't log all the errors with "error" level, only the ones related to
internal server errors and the rest log them with "debug" level because
they pollute the production satellite errors with errors that are
misguiding.

Change-Id: Id2bd737edba8550ce08965b51b8bf2540bd13ca4
2022-06-28 11:07:57 +00:00
prerna-parashar
6a1d7c8747
satellite/analytics: Added tracks calls for product activity metrics (#4907)
satellite/analytics: Added tracks calls for product activity metrics
2022-06-17 12:57:10 -07:00
Jeremy Wharton
5ce7d980af private/apigen: Make API generation deterministic
This change fixes the issue where the API generator would produce
different Go code for the same API definition upon each invocation
due to the random nature of map iteration.

Change-Id: I6770a10faf06311c24f541611c25d0b2b0f8e521
2022-06-17 12:06:08 -05:00
Cameron
240b70b828 satellite/console: use new type UpdateUserRequest as arg to db users.Update
The users.Update method in the satellitedb package takes a console.User
as an argument. It reads some of the fields on this struct and assigns
the value to dbx.User_Update_Fields. However, you cannot optionally
update only some of the fields. They all will always be updated. This means
that if you only want to update FullName, you still need to read the
user info from the DB to avoid updating the rest of the fields to zero.
This is not good because concurrent updates can overwrite each other.

This change introduces a new struct type, UpdateUserRequest, which
contains pointers for all the fields that are updated by satellite db
users.Update. Now the update method will check if a field is nil before
assigning the value to be updated in the db, so you only need to set the
field you want updated. For nullable columns, the respective field is a
double pointer. This allows us to update a column to NULL if the outer
pointer is not nil, but the inner pointer is.

Change-Id: I27f842d283c2711e24d51dcab622e57eeb9157f1
2022-06-14 09:28:03 -04:00
Jeremy Wharton
58c5d44f44 satellite/console: integrate sessions into satellite UI
This change integrates the session management database functionality
with the web application. Claim-based authentication has been removed
in favor of session token-based authentication.

Change-Id: I62a4f5354a3ed8ca80272814aad2448f901eab1b
2022-06-13 08:02:02 +00:00
prerna-parashar
cc0518f473
satellite/analytics: Added segment.io page calls to track all the pages (#4880)
satellite/analytics: send analytics 'page visit' api requests when the user navigates around the UI
2022-06-09 11:54:23 -07:00
Cameron
5cfa7ca460 satellite/console: add cutoff to email reminders
There are multiple entries in the users table with the same email
address. This is because in the past users were able to register
multiple times if the email was not verified. This is no longer
the case. If a user tries to register with an unverified email
already in the DB, we send a verification email instead of
creating another entry. However, since these old entries in the
table with duplicate emails were never cleaned up, the email
reminder chore will send out email verification reminders to them.
A single person will get one separate email per entry in the DB
with their email and where status = 0.

Since the multiple entries with the same email problem was solved
a while ago, just add a constraint to GetUnverifiedNeedingReminder
to only select users created after a cutoff. Once the DB is
migrated to remove the duplicate emails, we can remove the cutoff.

github issue: https://github.com/storj/storj/issues/4853

Change-Id: I07d77d43109bcacc8909df61d4fb49165a99527c
2022-06-09 16:45:35 +00:00
Vitalii
f56504de2a apigen: project delete endpoint
Implemented project delete endpoint for REST API.
Added project usage status check service method to indicate if project can be deleted.
Updated project invoice status check method to indicate if project can be deleted.

Change-Id: I57dc96efb072517144252001ab5405446c9cdeb4
2022-06-07 12:23:24 +03:00
Vitalii
f0b28d6326 apigen: endpoint to get user
Implemented new GET user by request context endpoint.
Updated docs.

Change-Id: Iebb493e55f9456b89d7dbd234bb0b939b82b0ced
2022-06-06 16:31:19 +00:00
Vitalii
ba58530089 {satellite}/web,console,testsuite: remove old navigation structure
Removed old satellite UI navigation structure.
Removed old feature flag.

Change-Id: Ic998886cf2e30ebd44e67a20fc53888103fe4b8d
2022-06-06 16:43:05 +03:00
dlamarmorgan
270204f352 satellite/{payments/storjscan,satellitedb}: Add wallet implementation
Add storjscan wallets implementation to the satellite. The wallets interface allows you to add and claim new wallets as called by the API. The storjscan specific implementation of this interface uses a wallets DB to associate the user to a wallet address, as well as a storjscan client to request and associate new wallets to the satellite.

Change-Id: I54081edb5545d4e3ee07cf1cce3d3e87cc00c4a1
2022-06-03 11:45:47 +00:00
cl-mitch
cbaca8b17e
web/satellite added new billing screen feature flag (#4836)
New feature flag implemented to enable the new billing flow.

Co-authored-by: Maximillian von Briesen <mobyvb@gmail.com>
2022-06-02 09:30:27 -05:00
Egon Elbre
763c04770f satellite/console/consoleweb: fix CSP values
script-src-elem is preferred over script-src in certain scenarios.
If it's absent, then the browser always uses script-src. By adding
script-src-elem it ended up blocking google recaptcha.

Change-Id: I9cf96e71e69054c4a034ca189db84fbe8903a59b
2022-06-02 10:44:48 +03:00
Vitalii
07e65cd338 satellite/projectaccounting: sum up bucket usages for daily usage query
Fixed daily usage query returning single bucket usage.
We sum up bucket usages now.
Also fixed https://github.com/storj/storj/issues/4559.

Change-Id: I2eb6299f1ef500d68150879195011b6fbb5f37ed
2022-06-01 12:50:10 +00:00
littleskunk
6cdd250019
satellite/console: enable new object flow / onboarding (#4851) 2022-06-01 14:04:03 +02:00
littleskunk
3d1b469f35
satellite/console: disable account activation reminder email (#4844)
* satellite/console: disable account activation reminder email
2022-05-31 17:36:40 +02:00
Vitalii
69ad49f473 satellite/server: fixed CSP for hcaptcha
Fixed CSP errors for hcaptcha

Change-Id: Ie928e206c652c97d36bcbdaf8436ae4a33afed8d
2022-05-27 14:28:31 +00:00
Vitalii
d916b26e0e apigen: create api key endpoint
Implemented new service method for generating API keys.
Implemented new endpoint.
Improved multiple endpoint groups handling.

Change-Id: Iba26fbf9123707b5b4c2d5e8c5a35d507404f24a
2022-05-26 16:00:23 +00:00
Cameron
55821605e8 satellite/console: add monkit metrics around user registraion/login
github issue: https://github.com/storj/storj/issues/4807

Change-Id: Id56ec73ec91b07b639b8011f0f916b4adbb01be6
2022-05-26 10:44:47 -04:00
Márton Elek
c136796308 test: make http client in TestActivationRouting thread-safe
testplanet executes cockroach and postgress tests parallel, therefore using http.DefaultClient is safe only as long as we don't modify it.

TestActivationRouting modifies it (client.CheckRedirect=...), therefore it should use a local version instead of the default one.

Problem reported by a jenkins build:

```
==================
WARNING: DATA RACE
Write at 0x000003486af0 by goroutine 143:
  storj.io/storj/satellite/console/consoleweb_test.TestActivationRouting.func1()
      /home/jenkins/workspace/storj-testing-experiments/satellite/console/consoleweb/server_test.go:66 +0x378
  storj.io/storj/private/testplanet.Run.func1.1()
...

Previous read at 0x000003486af0 by goroutine 104:
  net/http.(*Client).checkRedirect()
      /usr/local/go/src/net/http/client.go:494 +0xd73
  net/http.(*Client).do()
      /usr/local/go/src/net/http/client.go:691 +0xd31
  net/http.(*Client).Do()
      /usr/local/go/src/net/http/client.go:593 +0x204
  storj.io/storj/satellite/console/consoleweb_test.TestActivationRouting.func1.1()
      /home/jenkins/workspace/storj-testing-experiments/satellite/console/consoleweb/server_test.go:48 +0x1e5
  storj.io/storj/satellite/console/consoleweb_test.TestActivationRouting.func1()
      /home/jenkins/workspace/storj-testing-experiments/satellite/console/consoleweb/server_test.go:74 +0x49d
  storj.io/storj/private/testplanet.Run.func1.1()
...

```

Change-Id: I73319a5a593e067b906ec1fda70a44ca1e5a49a2
2022-05-25 15:27:02 +00:00
Cameron
87f6a3dcda {web/satelliite, satellite/console}: logo redirects to homepage
logo redirects to homepage on login, signup, forgot password, reset
password, and activate account pages

Change-Id: I992aeae197004d620addd8d515cae1c1ca80a778
2022-05-23 16:26:19 -04:00
Márton Elek
55de50eea7 console: stub endpoints for native token payments
Change-Id: I3f99ab44332eb8c9e2a3834a932eee72f44bd490
2022-05-23 13:28:00 +00:00
NickolaiYurchenko
99237d5c78 web/satellite: new bucket creation flow
old bucket creation flow removed
new flow added
name and passphrase splitted into separate views
demo bucket will not be created automatically
bucket creation progress bar added

Change-Id: I2a1d7d77c3038caaafb3c06bdb0ac5dd1ad17599
2022-05-22 11:55:55 +00:00
Malcolm Bouzi
087e57d037 web/satellite: create chore that will resend verification emails to unverified users
We want to remind unverified users to verify their emails:
once after 24 hours has passed and again after 5 days has passed.

Add mailservice.Service to satellite core because it is needed by the
chore for sending emails. To add the mailservice.Service to the core,
we create a helper function in satellite/peer.go to avoid duplicating
the code in both api.go and core.go. In addition to the chore, this
change adds methods to users.DB to get unverified users in need of
reminder.

Change-Id: I4e515bdf43f922788b4f965b2efb34fa32288bd1
2022-05-18 08:08:33 +00:00
JT Olio
5fb9ee3cfa uplink, satellite: use bgp hostnames
Change-Id: I58f5011d3019f8267fa8cbd3096b2cfe42eb5f8b
2022-05-16 18:14:37 +00:00
Cameron
bfad64a6f0 satellite/satellitedb: add verification_reminders column to users table
When a new user registers, we send a verification request to their
email. Currently, if they do not verify their email, we take no further
action. We want to send these users reminders: one after about one day
and one after about 5 days. To do this we will use this new
verification_reminders column.

It will look something like this:

```
SELECT email FROM users
WHERE status = 0
   AND (
          (verification_reminders = 0 AND created_at < now() - 'INTERVAL 1d')
          OR (verification_reminders = 1 AND created_at < now() - 'INTERVAL 5d')
   )
```

Change-Id: If0620e08c97e9e337c9563481d665c5bd462693b
2022-05-16 16:37:16 +00:00
Cameron
772397172f satellite/console: implement WebappSessions db
Create WebappSessions interface in consoleauth package.
Interface implements the DB for webapp_sessions table.

https://github.com/storj/storj/blob/main/docs/blueprints/webapp-session-management.md

Change-Id: Ib56f238c20b58f2877046fc384232add253ee82b
2022-05-16 15:06:04 +00:00
Cameron
0633aca607 satellite/console: create new consoleauth service
We want to send email verification reminders to users from the satellite
core, but some of the functionality required to do so exists in the
satellite console service. We could simply import the console service
into the core to achieve this, but the service requires a lot of
dependencies that would go unused just to be able to send these emails.

Instead, we break out the needed functionality into a new service which
can be imported separately by the console service and the future email
chore.

The consoleauth service creates, signs, and checks the expiration of auth
tokens.

Change-Id: I2ad794b7fd256f8af24c1a8d73a203d508069078
2022-05-13 16:27:07 +00:00
Moby von Briesen
763bfc0913 satellite/console,web/satellite: Implement hCaptcha
Adds a new configuration for hcaptcha enabled, secretkey, and sitekey.
If both reCAPTCHA and hCaptcha are configured as "enabled", reCAPTCHA
will be used.

Change-Id: I73cc6e133d8da3555e0ed8b2b377cf9eb263e6dc
2022-05-13 14:57:45 +00:00
Vitalii
95ae739d9c satellite/projectaccounting: fetch bucket usage since bucket creation instead of project creation
Fix for this customer issue
https://github.com/storj/customer-issues/issues/34

By this change we fetch bucket usage since its creation instead of using project's createdAt timestamp.

Change-Id: Ic0ea5d169056a5bd64ed143d13954d794da6e1d2
2022-05-13 13:27:42 +00:00
Vitalii
dedccbd2e4 satellite/console, web/satellite: limit failed login attempts
Added account locking on 3 or more login attempts.
Includes both password and MFA failed attempts on login.
Unlock account on successful password reset.

Change-Id: If4899b40ab4a77d531c1f18bfe22cee2cffa72e0
2022-05-11 14:49:11 +00:00
Moby von Briesen
9fc6484600 private/apigen: Reorganize and rename files
Change-Id: I430d620bb0940f2186cb884b00cb5eb9729b9744
2022-05-09 15:30:54 +00:00
Ivan Fraixedes
05d9c7940d
satellite/.../consoleapi: Respond with 401 on unauth req (#4781)
Respond with the appropriate HTTP status code when a request to the
analytics trigger event handler receive an authorized request.

A part of fixing the response status code this will stop to log these
response with ERROR level in our satellite logs.

Example of error message found in our satellite logs:

  {
    "insertId": "0ljf1cfn4xroxfd6",
    "jsonPayload": {
      "N": "console:endpoint",
      "T": "2022-05-06T13:31:35.415Z",
      "errorVerbose": "unauthorized: http: named cookie not present\n\tstorj.io/storj/satellite/console.GetAuth:72\n\tstorj.io/storj/satellite/console/consoleweb/consoleapi.(*Analytics).EventTriggered:60\n\tnet/http.HandlerFunc.ServeHTTP:2047\n\tstorj.io/storj/satellite/console/consoleweb.(*Server).withAuth.func1:488\n\tnet/http.HandlerFunc.ServeHTTP:2047\n\tgithub.com/gorilla/mux.(*Router).ServeHTTP:210\n\tstorj.io/storj/satellite/console/consoleweb.(*Server).withRequest.func1:495\n\tnet/http.HandlerFunc.ServeHTTP:2047\n\tnet/http.serverHandler.ServeHTTP:2879\n\tnet/http.(*conn).serve:1930",
      "L": "ERROR",
      "error": "unauthorized: http: named cookie not present",
      "message": "unauthorized: http: named cookie not present",
      "code": 500,
      "S": "storj.io/storj/satellite/console/consoleweb/consoleapi.serveCustomJSONError\n\t/go/src/storj.io/storj/satellite/console/consoleweb/consoleapi/common.go:37\nstorj.io/storj/satellite/console/consoleweb/consoleapi.serveJSONError\n\t/go/src/storj.io/storj/satellite/console/consoleweb/consoleapi/common.go:23\nstorj.io/storj/satellite/console/consoleweb/consoleapi.(*Analytics).serveJSONError\n\t/go/src/storj.io/storj/satellite/console/consoleweb/consoleapi/analytics.go:75\nstorj.io/storj/satellite/console/consoleweb/consoleapi.(*Analytics).EventTriggered\n\t/go/src/storj.io/storj/satellite/console/consoleweb/consoleapi/analytics.go:62\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2047\nstorj.io/storj/satellite/console/consoleweb.(*Server).withAuth.func1\n\t/go/src/storj.io/storj/satellite/console/consoleweb/server.go:488\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2047\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/pkg/mod/github.com/gorilla/mux@v1.8.0/mux.go:210\nstorj.io/storj/satellite/console/consoleweb.(*Server).withRequest.func1\n\t/go/src/storj.io/storj/satellite/console/consoleweb/server.go:495\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2047\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2879\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:1930",
      "M": "returning error to client"
    },
    "resource": {
      "type": "k8s_container",
      "labels": {
        "location": "us-central1",
        "pod_name": "us-central1-satellite-api-77c47f5c5-dzrpj",
        "project_id": "storj-prod",
        "namespace_name": "satellite",
        "container_name": "satellite",
        "cluster_name": "us-central1-gke-manatee"
      }
    },
    "timestamp": "2022-05-06T13:31:35.416050390Z",
    "severity": "ERROR",
    "labels": {
      "k8s-pod/version": "v3",
      "k8s-pod/app": "us-central1-satellite-api",
      "compute.googleapis.com/resource_name": "gke-us-central1-gke--terraform-202110-97ff1891-t0fv",
      "k8s-pod/service": "api",
      "k8s-pod/pod-template-hash": "77c47f5c5"
    },
    "logName": "projects/storj-prod/logs/stderr",
    "receiveTimestamp": "2022-05-06T13:31:37.419991630Z"
  }

Change-Id: I7cfcfb500b7878c59b1d259683c92e8963e2dc3f

Co-authored-by: Stefan Benten <mail@stefan-benten.de>
2022-05-08 12:35:42 +02:00
hovex023
58f957bd1d
web/satellite: Add new access grants flow (#4765)
* Added new feature Flag for new Access Grant Flow. 

* Added 3 cards to access grant view for S3, CLI and Access grant to replace old header

* Added new formatting, text and Icon for Access Grant Delete Popup modal
2022-04-29 10:31:52 -05:00
Vitalii
96411ba56a rest-api: endpoint reworkings
Added documentation.
Replaced PUT request with POST request.
Added inline param support for PATCH request.
Replaced unix timestamps handling with RFC-3339 timestampts handling.
Added 'Bearer' method requirement for Authorization header.

Change-Id: I4faa3864051dd18826c2c583ada53666d4aaec44
2022-04-28 18:17:54 +00:00
prerna-parashar
894b7b1cf3
satellite/analytics: Track event "Credit Card Added" (#4749)
satellite/analytics: Track event "Credit Card Added"
2022-04-28 11:12:42 -07:00
Mya
5cebbdee03 web/satellite: add consent screen for oauth
When an application wants to interact with resources on behalf of
an end-user, it needs to be granted access. In OAuth, this is done
when a user submits the consent screen.

Change-Id: Id838772f76999f63f5c9dbdda0995697b41c123a
2022-04-27 14:33:07 +00:00
Moby von Briesen
ed5ebb2527 satellite: Rename "acct mgmt api" to "rest api"
"REST API" is a more accurate descriptor of the generated API in the
console package than "account management API". The generated API is very
flexible and will allow us to implement many more endpoints outside the
scope of "account management", and "account management" is not very well
defined to begin with.

Change-Id: Ie87faeaa3c743ef4371eaf0edd2826303d592da7
2022-04-25 18:51:46 +00:00
NickolaiYurchenko
c32ca6e67f apigen: endpint to update project
Implemented new endpoint for project update using apigen.
Implemented new service method compatible with new generated api.

Change-Id: Ic0a7e0bbf3ea942275bd927d6e30cfb7e721e9c1
2022-04-14 22:21:08 +00:00
Vitalii Shpital
b4cec4fd0b satellite/satellitedb: add columns to users table to control failed login attempts
Added failed_login_count and login_lockout_expiration columns to users table to control users failed login attempts.
We want to prevent brute forcing of user login so this is the first step.

Change-Id: I06b0b9f5415a1922e08cd9908893b2fd3c26bca0
2022-04-12 08:37:07 +00:00
Vitalii
3b39399905 apigen: endpoint to create new Project
Implemented new endpoint for project creation using apigen.
Implemented new service method compatible with new generated api.

Change-Id: I2bae22c8b046f21ec5bb6522f09b9c4e74bdba0c
2022-04-06 17:49:46 +00:00
Vitalii
9b695525c6 satellite/console: fixed small email validation issues
Moved invalid email testing to separate test.
Made all the emails used to have .test domain.
Added links to regex resources.

Change-Id: I26920ba7360064528256a6aeaea947bbe56ef618
2022-04-04 09:43:33 +00:00
Vitalii
67b5b07730 apigen: api key authentication implemented
Implemented account management api key authentication.
Extended IsAuthenticated service method to include both cookie and api key authorization.

Change-Id: I6f2d01fdc6115cb860f2e49c74980a39155afe7e
2022-04-01 15:17:38 +00:00
Moby von Briesen
0018d62837 satellite/analytics: Associate Hubspot token with new user
If a visitor has accepted cookies on www.storj.io, there might be a
"hubspotutk" cookie in their browser upon account creation. This allows
Hubspot to link website activity with a newly created user.

Change-Id: If06c67fb4d2e5dd3cf46c1fe80a0e9d7f25d6e58
2022-03-29 16:25:33 -04:00
Cameron
84b522bc06 satellite/console: create account management api keys service
We are in the process of creating an api to allow users to manage their
accounts programmatically. We would like to use api keys for
authorization. We were originally going to create an entirely new table
for these api keys, but seeing as we already have 2 other tables for
keys/tokens, api_keys and oauth_tokens, we thought it might be better to
use one of these. We're using oauth_tokens.

We create a new oidc.OAuthTokenKind for account management api keys:
KindAccountManagementTokenV0. We made the key versioned because we
likely want to improve the implementation in the future, but we want to
get something functional out the door ASAP because the account management
api feature is highly desired.

Add a new method to oidc.OAuthTokens interface for revoking v0 account
management api keys, RevokeAccountManagementTokenV0. Add update method
to dbx implementation to allow updating the expiration. We will revoke
these keys by setting the expiration to 0 so they are expired.

Change-Id: Ideb8ae04b23aa55d5825b064b5e43e32eadc1fba
2022-03-23 17:02:20 +00:00
Vitalii Shpital
2ccfd13d7f apigen: endpoint to get all buckets usage by project ID
Added new endpoint to get all bucket rollups by bucket ID.

Example of response:
vitalii:~/Documents$ ./testapi.sh
HTTP/1.1 200 OK
Content-Type: application/json
Date: Mon, 07 Mar 2022 11:18:55 GMT
Content-Length: 671

[{"projectID":"a9b2b1b6-714a-4c49-99f1-6a53d0852525","bucketName":"demo-bucket","totalStoredData":0.0026272243089674662,"totalSegments":0.05000107166666666,"objectCount":0.03333373083333333,"metadataSize":1.6750359008333334e-9,"repairEgress":0,"getEgress":0,"auditEgress":0,"since":"2022-03-01T11:00:00Z","before":"2022-03-07T11:17:07Z"},{"projectID":"a9b2b1b6-714a-4c49-99f1-6a53d0852525","bucketName":"qwe","totalStoredData":0.000018436725422435552,"totalSegments":0.016667081388888887,"objectCount":0.016667081388888887,"metadataSize":1.933381441111111e-9,"repairEgress":0,"getEgress":0,"auditEgress":0,"since":"2022-03-01T11:00:00Z","before":"2022-03-07T11:17:07Z"}]

Change-Id: I8b04b24dbc67b78be5c309ce542bf03d6f67e65d
2022-03-23 15:12:27 +00:00
Egon Elbre
0d2d59f884 all: fix linting issues
Change-Id: Idfc93948e59a181321d79b365e638d63e256a16f
2022-03-21 15:26:42 +00:00
Mya
98f4fae02c satellite/oidc: add integration test
This change adds an integration test that performs an OAuth
workflow and verifies the OIDC endpoints are functioning as
expected.

Change-Id: I18a8968b4f0385a1e4de6784dee68e1b51df86f7
2022-03-18 16:14:18 +00:00
Mya
4a110b266e satellite/console: added oidc endpoints
This change adds endpoints for supporting OpenID Connect (OIDC) and
OAuth requests. This allows application developers to easily
develop apps with Storj using common mechanisms for authentication
and authorization.

Change-Id: I2a76d48bd1241367aa2d1e3309f6f65d6d6ea4dc
2022-03-16 12:01:26 +00:00
Vitalii Shpital
60b209e47d web/satellite, satellite/console: reworked registration email validation
Reworked email validation for new users (for old users trying to login or reset password validation remains the same).
Regular expression was built according to RFC 5322 and then extended to include international characters.

Change-Id: Id0224fee21a1ec0f8a2dcca5b8431197dee6b9d3
2022-03-16 09:40:53 +00:00
Mya
75be1c0a28 satellite/console: support reading apikeys by name and project id
When performing re-authorizations for OAuth, we need to pull up an
APIKey using it's project id and name. This change also updates the
APIKeyInfo struct to return the head value associated with an API
key.

Change-Id: I4b40f7f13fb9b58a1927dd283b42a39015ea550e
2022-03-14 13:21:21 -05:00
dlamarmorgan
715077d04a satellite/{console,satellitedb}: increase project limit on paid tier upgrade
Update the user to the default paid tier project limit, which is currently 3 projects, when the user upgrades to a paid account.

Change-Id: I95b19d62cebc7d878b716355f2ebcaf0b51ca3f7
2022-03-14 16:10:23 +00:00
Vitalii Shpital
1245283637 apigen: new endpoint to get project's single bucket rollup
Added new endpoint to get project's single bucket usage rollup.
Extended generation code to handle service method args.

Change-Id: Ief768632a801c047c66e0617056fbd7b30427b33
2022-03-04 17:33:38 +00:00
Michał Niewrzał
39ac90835f satellite/satellitedb: change default segment limit to 100M for paying users
We decided that we want to have segment limit for paying users high
enough to not have to change it too often.

Fixes https://github.com/storj/storj/issues/4590

Change-Id: Ic1c38bf3e2fcc000548ff4c7e7004647b39fbecf
2022-03-04 13:18:14 +00:00
Jeremy Wharton
66e6a75e2a satellite/console,web/satellite: Add MFA to password reset
Users will be required to enter a MFA passcode or recovery code
upon attempting a password reset for an account with MFA enabled.

Change-Id: I08d07597035d5a25849dbc70f7fd686753530610
2022-03-03 17:27:04 +00:00
NickolaiYurchenko
64176aaca4 web/satellite: registration success redirect from config
This change allows us to send newly registered users to a configured URL
to help us track user conversions for marketing campaigns.
Brave conversions continue to be tracked using the /signup-success page
within the satellite app.

Change-Id: I9b451947ce0f39d3c99b233cb4b806d361151823
2022-03-03 10:13:21 -05:00
Vitalii Shpital
9b5904cd49 satellite/{projectaccounting, console}:query to get single bucket rollup
Added new projectaccounting query to get project's single bucket usage rollup.
Added new service method to call new query.
Added implementation for IsAuthenticated method which is used by new generated API.

Change-Id: I7cde5656d489953b6c7d109f236362eb465fa64a
2022-03-03 12:04:29 +00:00
Vitalii Shpital
ba6956db0f console/server, apigen: feature flag for new generated console api
Added a feture flag which will be used to indicate if new generated console api is used.
Fixed some comments from previous PR.

Change-Id: Ice31c998b0b347028a491c971a648fd1269bfd49
2022-02-28 23:00:12 +00:00
Ivan Fraixedes
8caa4c4557 satellite/console: Don't lose ErrValiation error class
There was a defined type (`validationErrors`) for gathering several
validation errors and classify them with the `ErrValdiation errs.Class`.

`errs.Combine` doesn't maintain the classes of the errors to combine,
for example

```
var myClass errs.Class = "My error class"

err1 := myClass.Wrap(erros.New("error 1"))
err2 := myClass.Wrap(erros.New("error 2"))
err3 := errors.New("error 3")

combinedErr := errs.Combine(err1, err2, err3)
myClass.Has(combinedErr) // It returns false

// Even only passing errors with a class and with the same one for all
// of them
combinedErr := errs.Combine(err1, err2)
myClass.Has(combinedErr) // It returns false
```

Hence `validationErrors` didn't return what we expected to return when
calling its `Combine` method.

This commit delete the type and it replaces by `errs.Group` when there
are more than one error, and wrapping the `errs.Group.Err` returned
error with `ErrValiation` error class.

The bug caused the HTTP API server to return a 500 status code as you
can seee in the following log message extracted from the satellite
production logs:

```
code: 500
error: "console service: validation: full name can not be empty; validation: Your password needs at least 6 characters long; validation: mail: no address"
errorVerbose: "console service: validation: full name can not be empty; validation: Your password needs at least 6 characters long; validation: mail: no address
        storj.io/storj/satellite/console.(*Service).CreateUser:593
        storj.io/storj/satellite/console/consoleweb/consoleapi.(*Auth).Register:250
        net/http.HandlerFunc.ServeHTTP:2047
        storj.io/storj/private/web.(*RateLimiter).Limit.func1:90
        net/http.HandlerFunc.ServeHTTP:2047
        github.com/gorilla/mux.(*Router).ServeHTTP:210
        storj.io/storj/satellite/console/consoleweb.(*Server).withRequest.func1:464
        net/http.HandlerFunc.ServeHTTP:2047
        net/http.serverHandler.ServeHTTP:2879
        net/http.(*conn).serve:1930"
message: "There was an error processing your request"
```

The issues was that not being classified with `ErrValidation` class it
was not picked by the correct switch branch of the
`consoleapi.Auth.getStatusCode` method which is in the call chain to
`consoleapi.Auth.Register` method when it calls
`console.Service.CreateUser` and returns an error.

These changes should return the appropriated HTTP status code (Bad
Request) when `console.Service.CreateUser` returns a validation error.

Returning the appropriated HTTP statsus code also makes not to show this
as an error in the server logs because the Bad Request sttatus code gets
logged with debug level.

Change-Id: I869ea85788992ae0865c373860fbf93a40d2d387
2022-02-28 11:00:56 +00:00
Moby von Briesen
d12bb19972 satellite/console/consoleweb: Update CSP directives
Update the Content Security Policy to whitelist `blob:` for the img-src
and media-src directives. This is necessary to prevent CSP errors in the
object browser while loading previews and object maps.

Change-Id: Ic32bf0954f300c77ec4f0fe11fae63f0c7b622da
2022-02-23 17:30:35 -05:00
NickolaiYurchenko
70c8ccb435 web/satellite: inactivity timer to dashboard added
added InactivityTimerEnabled flag to enable/disable feature
added InactivityTimerDelay to configure delay time in seconds
default timer set up to 10 minutes
reset dom events: keypress, mouseover, mousedown, touchmove

Change-Id: Idb66067c2902b2cdbe1a972225319c8abff97927
2022-02-16 07:16:00 +00:00
Cameron
eefef2eb17 satellite/console: pass entire auth.User into users.Update in service.UpdateAccount
The Update method of the usersDB takes a console.User as an argument.
To update the columns in the DB, we have to migrate the fields from the
console.Users struct into a special dbx struct. If one of these fields
is left empty, then the zero value of that field's type will be used to
update the respective column.

In most cases where the users.Update method is called, the entire
console.User is apparently retrieved first, fields are updated, then it
is passed to users.Update. This is not the case for
service.UpdateAccount. Because these fields are not populated in the
user struct in UpdateAccount before it is passed into users.Update,
their respective columns in the database are overwritten with zero:

ProjectLimit, ProjectStorageLimit, ProjectBandwidthLimit,
ProjectSegmentLimit, PaidTier, MfaEnabled, MfaRecoveryCodes, MfaSecretKey

Solution: Do what is done in other places which call users.Update. Take
the console.User from the auth context, update the relevant fields on
that, then pass that in.

Change-Id: I3cbd560e8ea5397e5c27711fb40bb3907d987028
2022-02-11 15:33:04 +00:00
Vitalii Shpital
07c71e34c2 satellite/console{gen}: GetUsersProjects endpoint
Initial implementation of auto-generated GetUsersProjects endpoint

Change-Id: If41bff2ea3ff9cfc87afeda9e5e5b3f586cbab33
2022-02-10 08:03:02 +00:00
Vitalii Shpital
150be885b7 satellitedb/projectaccounting, web/satellite: reworked bandwidth chart to show both allocated and settled bandwidth
Extended DB query.
Reworked charts UI functionality to show 2 lines if necessary.

Change-Id: I8ac4e4fa07676fc9fa7e9c078ecdeed62233b8e2
2022-02-08 22:07:59 +00:00
Mya
0164682c37 satellite/oidc: move oidc into common package
Change-Id: I77702e0e46f15a09fee315b9076638e1412836f7
2022-02-08 09:46:54 -06:00
Stefan Benten
06944f062d satellite/{admin,payments,satellitedb}: add checks for deletion of free tier accounts
This change adds some more checks to the deletion process for projects and
users, since we ran into a race condition during invoicing, where projects
have been deleted before the invoicing was finished, leading to missing
references.
This PR changes the logic to block user deletion if we are in exactly that period,
while also allowing the deletion of projects/users on free tier during the month.

Change-Id: Ic0735205e6633762fb7e3c2fa13e744cdfa5ec32
2022-02-08 10:11:31 +00:00
Vitalii Shpital
b3e1be37ff satellite/projectaccounting: query to get daily project usage by date range
Finished implementing queries for both bandwidth and storage using pgx.Batch.
Fixed CSP styling issue.

Change-Id: I5f9e10abe8096be3115b4e1f6ed3b13f1e7232df
2022-02-04 22:32:37 +00:00
Vitalii Shpital
3c8e41e665 web/satellite: get object map and preview by signed request.
There is a sev-2 issue to add more browser caching.
In this PR I made object map and object preview to be fetched by signed request with non-public credentials using AWS SignatureV4 package.

Change-Id: Ib5013fa6d6af3faa97eed5168c11a13f9629cd87
2022-02-03 11:31:09 +00:00
Mya
4d0dd737b0 satellite/admin: add endpoints for oauth clients
Change-Id: I26aa81266f494be8aab0b5523217bad9405037a0
2022-02-02 09:47:52 -06:00
Vitalii Shpital
b8c55fdd87 satellite/projectaccounting, satellite/console, web/satellite: implemented backend for bandwidth chart
Implemented endpoint and query to get bandwidth chart data for new project dashboard.
Connected backend with frontend.
Storage chart data is mocked right now.

Change-Id: Ib24d28614dc74bcc31b81ee3b8aa68b9898fa87b
2022-01-26 15:09:22 +00:00
Vitalii Shpital
5d6ee506b0 private/apigen: initial typings and methods
Initial typings and methods for the future API Management implementation.
Blueprint - https://review.dev.storj.io/c/storj/storj/+/6341

Change-Id: I9afe66a7610a3a80e764b317ddd211f125b1ea43
2022-01-26 14:39:22 +00:00
Malcolm Bouzi
3451162583 web/console: Save timestamp when creating users to mark their first email verification notification.
This value will be updated whenever we send a reminder for a user to verify their email.

Change-Id: I7f08b16eccc6633b8eb3662eb7efed1f1f780f44
2022-01-24 16:53:51 +00:00
Malcolm Bouzi
c30c7def44 satellite/{console,payments},web/satellite: save signup promo code to newly registered users
Users signing up through a url containing a promo code will have that code applied to their stripe account instead of the free tier coupon.

Change-Id: I071041b0934648ef3f5bdb05b6ec97c400f89ae4
2022-01-21 15:02:22 +00:00
Moby von Briesen
464b4b02fa satellite/console/consoleweb: Only replace some kinds of characters in name
The previous change (59648dc272) ends up removing a lot of characters
for valid non-English names. Instead, only replace URL characters such
as slashes, colons, and periods. Since someone may use these characters
to separate two parts of a name, e.g. Name1/Name2, replace these
characters with a hyphen.

Change-Id: I4cc3d1bdb05d525a83970cf1b42479414c9678e7
2022-01-20 11:13:26 -05:00
Moby von Briesen
59648dc272 satellite/console: Filter special characters out of name
When a user is created, but before verification or forgot password email
is sent, remove any special characters in the provided name. This
protects us against certain phishing attacks.

Change-Id: Ieddd3479da20eb80b9f1b56eb86c8f46bca2642c
2022-01-20 00:18:56 +00:00
Michał Niewrzał
95750b9d32 satellite/console: verify limits while test
Additional test case to be sure that update operation went well.

Change-Id: I7dbf1e3110f716712ba9747a1915b54112b89801
2022-01-19 16:39:36 +00:00
Michał Niewrzał
270c6e24f2 satellite/console: fix default limit for free-tier segment usage limit
Free-tier segment usage limit was defined as 150k, not 140k. This change
is correcting that.

https://github.com/storj/team-metainfo/issues/8

Change-Id: I71ec0961930b19fd09b2b996e01acd406a8dcf8f
2022-01-19 09:21:13 +00:00
Qweder93
8be655c487 satellite/metainfo: add segment limit validation, caching
We want to be able to limit the number of segments per project for users.
To limit this we need to check limit value associated with project
and value of used segments already in BeginMoveObject, BeginMoveSegment
and increment cache segments usage after each CommitSegment call.

Resolves https://github.com/storj/team-metainfo/issues/1

Change-Id: I6290e67c095a174b9d101c4521802d9bfe0453b8
2022-01-10 14:33:26 +02:00
Egon Elbre
04d04c0306 web/satellite: remove old object browser
Change-Id: Ief16a3887444df093c644831d86a9b3e865e5867
2022-01-05 16:29:19 +02:00
Malcolm Bouzi
6a059b7f9d web/satellite: remove all references to the old onboarding flow
Change-Id: I1f42fabdb407a7eb9c34f0341d332e66c3ce288d
2022-01-04 18:40:08 +00:00
Moby von Briesen
5351c43757 satellite/analytics: track origin header and referer on signup
This allows us to distinguish between accounts created from the signup
page vs. from www.storj.io.
Also set a field `account_created=true` when we send so
that we can see when existing leads have created an account.

Change-Id: Ibef34825a08b6c68b8f2869625e576bb837520e5
2022-01-03 17:47:48 +00:00
Cameron Ayer
2a4ff6f469 satellite/satellitedb: select user_agent in apikeys method GetPagedByProjectID
The user_agent column was missing from the SQL query in
apikeys.GetPagedByProjectID. Add it.

Change-Id: I2201b0ecd1767194a6d2339dba25a0b58e1ee30a
2022-01-03 11:58:26 -05:00
Moby von Briesen
1bd74658df satellite/console: Disable segment limit checking during project update
This is not supported on the client side yet.

Change-Id: Id2df75278efe4b3a48be21d0992ee835e804fc3a
2021-12-23 14:35:45 -05:00
Michał Niewrzał
3f0babffe8 satellite/{console,satellitedb}: add project segment limit to user
All limits we have for projects have also parent limits stored
with user data. New created project is first taking limits from
owner (user) limits.
This change is extending users table with project_segment_limit
column and adds functionality to get and set value for this
column.

Change-Id: Iff5e36c62b517652390b649fc05992475916ecff
2021-12-16 08:46:01 +00:00
Jeremy Wharton
9d13c649a2 satellite/{console,satellitedb}: Forbid creating users with used email
This change disallows creation of users possessing the same email.
If a user attempts to create an account with an email address
that's already used - whether it belongs to an active account or not -
he will be notified of unsuccessful account creation. If he attempts to
log in using an email address belonging to an inactive account,
he will be presented with a link allowing him to re-send the
verification email. Attempting to register with an email address
belonging to an existing account triggers a password reset email.

Change-Id: Iefd8c3bef00ecb1dd9e8504594607aa0dca7d82e
2021-12-15 17:24:35 +00:00
prerna-parashar
a41758bba5
Track user events through HubSpot events API (#4300)
Track user events through HubSpot events API
2021-12-09 12:52:51 -08:00
Michał Niewrzał
d94d8d1775 satellite/accounting: expose project segment limit
Exposes functionality to get and update project segment
limit. It will be used to limit number of segments per project
while uploading object.

Change-Id: I971d48eebb4e7db8b01535c3091829e73437f48d
2021-12-08 11:27:21 +00:00
Vitalii Shpital
b78f65e83b satellite/console, web/satellite: added object and segment count per project info to new project dashboard
Added new query to get project object and segment count.
Added appropriate object and segment count view for new project dashboard.

Change-Id: I69a2e55442f318c51dc365c0c578b964f2f06c7f
2021-12-01 17:46:07 +02:00
Jeremy Wharton
984792fd1e satellite/satellitedb: Add GetByEmailWithUnverified to users table
Allows us to handle duplicate emails better.

Change-Id: I266057900725e50d1c47977da307714fd32d9081
2021-11-29 20:35:47 +00:00
Igor
4daaa9503e
satellite/console: enable object flow by default (#4298) 2021-11-29 16:43:37 +02:00
Vitalii Shpital
3de7f8d5af satellite/console, web/satellite: feature flag for new project dashboard
Added a feature flag to switch between old and new project dashboard

Change-Id: Ibb1b04ccbddb43e3f41116e3513f917deba95699
2021-11-17 16:02:47 +02:00
Mya
814e3126fa satellite/buckets: add new buckets service
The main motivation is to wrap the bucket DB and metainfo DB, so we
could check if a bucket is empty before applying geofencing config.

Change-Id: I8bac21555e01d51a663fb557bc1acfc8106bc2e1
2021-11-16 12:36:17 +02:00