Commit Graph

268 Commits

Author SHA1 Message Date
littleskunk
2437d5b171
satellite/access-grants: default auth service url (#4002)
* satellite/access-grants: default auth service url
2020-12-17 23:38:16 +01:00
VitaliiShpital
f4bbd0f5df web/satellite: use brotli instead of gzip
WHAT:
we'll use brotli instead of gzip from now on

WHY:
better compression

Change-Id: Ibeadd6bfc783e9c15cf3f62f719af692071a7721
2020-12-17 19:23:44 +00:00
Jessica Grebenschikov
3cc98de3ee satellite/console/wasm: reduce size to <9MB
Make changes so that we only import the necessary files from the console package so that the generated wasm code is as small as possible.

This change gets the compiled wasm code down to 8.6MB uncompressed and 2MB when compressed with `gzip --best`.

https://review.dev.storj.io/c/storj/storj/+/3396

Change-Id: Ifdd4be285810757b46bbbe43327c0d0139e5f8f7
2020-12-14 16:41:39 +00:00
Stefan Benten
8fe829d5fd
build: add wasm bits to Dockerfile and bump to go v1.15.6 (#3992) 2020-12-11 02:23:39 +01:00
Stefan Benten
494bd5db81
all: golangci-lint v1.33.0 fixes (#3985) 2020-12-05 17:01:42 +01:00
Yingrong Zhao
d8ba7b3057 satellite/console: only allow project member to get all bucket names
Change-Id: I8ceb0b7eb19e221072b4ff3411a4ec1a7817d16f
2020-11-30 15:41:35 -05:00
VitaliiShpital
0771cdb0b1 web/satellite: create access grant: generate gateway credentials step
WHAT:
generate gateway credentials step for create access grant flow

WHY:
part of the flow

Change-Id: I6496712b43f78a818ba0582b586cfae3a44683e6
2020-11-30 10:36:29 +00:00
VitaliiShpital
bb7677a85f web/satellite: get gateway credentials request using url from config
WHAT:
POST request to get gateway credentials using access grant.
Put request url to config and use it for request.

WHY:
to show gateway credentials on UI

Change-Id: I15ef43ecdeed69b0961d5796aacb47f36d560b1b
2020-11-30 10:36:23 +00:00
VitaliiShpital
51a712f9e8 satellite/console: get all bucket names endpoint and service method
WHAT:
new endpoint for fetching all bucket names

WHY:
used by new access grant flow

Change-Id: I356a3381359665fd2726120139b34b1e611fe3c4
2020-11-16 17:51:40 +02:00
Yaroslav Vorobiov
1b4bfbb9d2 multinode/console: nodes addition and removal
Change-Id: I60c685953a8d0e24f78b1414c34a28d4b87863b0
2020-11-12 20:26:08 +02:00
Jessica Grebenschikov
226e13e616 satellite/cosole: add tests for wasm access code
Change-Id: I78f71b2f0bef03b6e87cd7d79ccaef5f45393b55
2020-11-12 08:03:36 -08:00
Jeff Wendling
31533ed1a1 satellite/console/wasm: remove storj.io/uplink deependency
Change-Id: Iee95389e4ba24618e31aff7be44d05377b2e2419
2020-11-11 16:51:14 +00:00
Yingrong Zhao
2ce3170bb4 satellite/console/wasm: expose method to add caveats in the browser
This PR does the following three things:
    1. Defines a high-level interface for this wasm package
        - All return value from this package will be wrapped with an
          result object that contains a value field and an error field
    2. Exposes two new functions to allow users to add permissions for a
       given API key
        - newPermission()
        - setAPIKeyPermission()
    3. Adds API documentation for the newly added API functions

Change-Id: Id995189702b369bba18fa344bef4ddfb0f3f1f44
2020-11-10 20:10:53 +00:00
VitaliiShpital
f8c3848c78 satellite/console: change user's email endpoint/feature
WHAT:
change user's email endpoint and appropriate service method was implemented

WHY:
make it possible to change user's email for temporary filezilla account

Change-Id: Ieea41bf49819a42b5f433e8dfaeec24c6d5ddc9f
2020-11-06 11:54:07 +00:00
jessicagreben
c4c29e370a wasm: add webassembly code for creating access grant in console web UI
Change-Id: I3c6d9afc660f3d959d6138db84341e9460b877a1
2020-11-04 12:08:30 -08:00
Egon Elbre
7183dca6cb all: fix defers in loop
defer should not be called in a loop.

Change-Id: Ifa5a25a56402814b974bcdfb0c2fce56df8e7e59
2020-11-02 15:06:38 +02:00
Egon Elbre
e0dca4042d all: add pprof labels for debugger
By using pprof.Labels debugger is able to show service/peer names in
goroutine names.

Change-Id: I5f55253470f7cc7e556f8e8b87f746394e41675f
2020-10-29 15:10:07 +00:00
Ivan Fraixedes
4b61ca638b
satellite/console/consoleweb/consoleapi: Fix & add test DeleteAccount
Fix the DeleteAccount handler to return 501 HTTP status code because
it's what corresponds for a "Not Implemented" status.

Add a black box test for the DeleteAccount to ensure that always return
an error response because, at this time, we don't allow to delete
accounts through the API.

This test was not added to the corresponding commit
https://review.dev.storj.io/c/storj/storj/+/2712 due to the rush to
fix it.

Change-Id: Ibcf09e2ec52f182a8a580d606c457328d94c8b60
2020-10-23 09:14:50 +02:00
Ivan Fraixedes
9abdcc05e5 satellite/console/consoleweb/consoleapi: report err to monkit
Report the "Not Implemented" error response returned by DeleteAccount
API handler to monkit.

Change-Id: I17e319639c458cbe803b65b5a34111b8f74daece
2020-10-22 17:07:13 +00:00
Ivan Fraixedes
46b12c96bd satellite/console/consoleweb/consoleql: Fix typo
Fix a typo in the GraphQL mutation testing function.

Change-Id: I1c474795bfbaa3151b04cb768dfc506e654557ab
2020-10-22 13:30:20 +00:00
Ivan Fraixedes
979ee762ba
satellite/console/consoleweb: Fix typo in method name
Fix a typo in the graphQL handler method name.

Change-Id: I038c7783073f7bed95353f56a8a24520c724a5b6
2020-10-21 11:58:37 +02:00
VitaliiShpital
59d85aab5b web/satellite: take project amount limit from db instead of config
WHAT:
Now project amount limit is taken from users db instead of config. But if db value is 0 then default config value will be used instead.

WHY:
this will allow us to change user's project limit by changing db value.

Change-Id: I9edcd0bf9eaae5fe40e90a44cac82d9ce8519274
2020-10-14 14:17:45 +00:00
Kaloyan Raev
830817ec0d cmd/storj-sim: run gateway without --access flag
This makes it possible to remove of this obsolete flag from the
multi-tenant gateway.

As a consequence, displaying the GATEWAY_0_ACCESS env var will always
require a running storj-sim. Until now, it was required only the first
time. Then the value was stored in the 'access' config. But this is now
not possible anymore.

The changes in StripeMock are required to fix failures in integration
tests. StripeMock is in-memory and its data does not survive restarts of
storj-sim. The second and following starts of storj-sim had invalid
state of StripeMock, which failed requests that were required to
populate the GATEWAY_0_ACCESS env var. The changes in StripeMock makes
it repopulate the Stripe customers from the database.

Change-Id: I981a208172b76577f12ecdaae485f5ae4ea269bc
2020-10-13 14:45:04 +00:00
Egon Elbre
2268cc1df3 all: fix linter complaints
Change-Id: Ia01404dbb6bdd19a146fa10ff7302e08f87a8c95
2020-10-13 15:59:01 +03:00
Stefan Benten
1d3b728766 satellite/{console/payments/satellitedb}: add validation for deletion of account and project
The same was that our Admin API handles project and account deletions currently, we would like
to have the same checks on the user-facing API. This PR adds the same checks to the console service.
General more applicable checks have been moved directly into the payments service.

In addition it adds the BucketsDB to the console DB, to have easier access and avoiding import cycles with
the metainfo package.

A small cleanup around our unnecessary monkit imports made it in as well.

Change-Id: I8769b01c2271c1687fbd2269a738a41764216e51
2020-10-13 07:55:26 +00:00
Stefan Benten
b3cf12f567 satellite/console: Add more validation for console requests
Adds membership checks for the following calls:
- GetProject

Add ownership checks for the following calls:
- DeleteProject

It also disables the API endpoint to delete a project.

Furthermore it adds tests for the console service.

Change-Id: I1ffc8dcb44746a74ad06a7dbd064a29c57c25272
2020-10-07 15:33:28 +00:00
Kaloyan Raev
4280142b24 satellite/console: remove unnecessary Error.Wrap
Change-Id: If851ccce7932cbf72c2fff3b51f4f9f2ea07c124
2020-10-07 09:22:41 +00:00
Stefan Benten
14a2050b8d pkg/auth: move package to consoleauth
To avoid further name collisions, the very broad named package gets moved into
the consoleauth package where its also mainly being used.

Change-Id: Ie563c9700adbf0553baca2b7b8ba4a1d9c29d144
2020-10-06 14:15:07 +02:00
Stefan Benten
44bd65795b satellite/console: ensure only project members can remove other project members
Change-Id: I815eb85f37631aaa65b5dc4cafa6851f241ca0f0
2020-10-06 11:03:12 +00:00
Stefan Benten
9deea2ffe2 satellite/console: disable account deletion via API
Change-Id: Ia8e43284c90fb2b833eb601e2c8f701cb5a4d9c0
2020-10-06 13:01:46 +02:00
Stefan Benten
0aaad88a44 satellite/{admin, console}: add test for projectLimit increase and update README
This change adds the capabilities to adjust the users project limit via the Admin API.
Adds a test for the new added function of the API and updates the existing tests.
It renames the json field on the user struct to be more consistent.

Change-Id: I9018acd80dae0af68d1d50526f20987132c654f3
2020-10-05 11:54:37 +00:00
Stefan Benten
79eb682f9c satellite/console: allow coupons to be a valid payment option
Currently a user is only able to create a project if either
a STORJ deposit or CC was added to his account. With this change, an existing
coupon is also valid to let the user proceed.

Change-Id: I7be8d2d9ec58a15c50755b3fe33af04d2fd64ea2
2020-09-28 21:24:04 +00:00
Jessica Grebenschikov
4a2c66fa06 satellite/accounting: add cache for getting project storage and bw limits
This PR adds the following items:
1) an in-memory read-only cache thats stores project limit info for projectIDs

This cache is stored in-memory since this is expected to be a small amount of data. In this implementation we are only storing in the cache projects that have been accessed. Currently for the largest Satellite (eu-west) there is about 4500 total projects. So storing the storage limit (int64) and the bandwidth limit (int64), this would end up being about 200kb (including the 32 byte project ID) if all 4500 projectIDs were in the cache. So this all fits in memory for the time being. At some point it may not as usage grows, but that seems years out.

The cache is a read only cache. When requests come in to upload/download a file, we will read from the cache what the current limits are for that project. If the cache does not contain the projectID, it will get the info from the database (satellitedb project table), then add it to the cache.

The only time the values in the cache are modified is when either a) the project ID is not in the cache, or b) the item in the cache has expired (default 10mins), then the data gets refreshed out of the database. This occurs by default every 10 mins. This means that if we update the usage limits in the database, that change might not show up in the cache for 10 mins which mean it will not be reflected to limit end users uploading/downloading files for that time period..

Change-Id: I3fd7056cf963676009834fcbcf9c4a0922ca4a8f
2020-09-25 16:28:49 +00:00
Stefan Benten
9d0d0ad728 satellite/console: enable multiple projects all users
Change-Id: I42cc9f48cac387e1a67d21c1dd394f28cc5ff399
2020-09-23 16:18:28 +00:00
Stefan Benten
2f648fd981 satellite: make limits be nullable
Currently we have no way to actually set one
of the following limits to 0 (meaning not usable):

- maxBuckets
- usageLimit
- bandwidthLimit

With having the field nullable,
NULL corresponds to the global default,
0 now actually 0 and
a set value determines a custom limit.

Change-Id: I92bb77529dcbd0881ae8368921be9d246eb0919e
2020-09-21 19:34:19 +00:00
JT Olio
f46161cf20 consoleweb: log index template failures
Change-Id: I286ded309fed6198f1c450a2a31df36b6a015551
2020-09-16 16:26:14 +00:00
VitaliiShpital
7d5e0259f6 satellite/projects: initial update project name functionality implemented
WHAT:
added functionality for user to update project name. Logic only, without actual GUI updates.

WHY:
better user experience

Change-Id: I1e38e33ba827b0bdf2c89e29de24e4e87edb474a
2020-09-15 12:21:56 +03:00
VitaliiShpital
c4d6f472fc web/satellite: notification bar for reaching projects count limit
WHAT:
notification bar added to project dashboard page. It is shown when projects count limit is reached.
Create project button is removed after creating last available project

WHY:
inform user that their projects count limit was reached

Change-Id: If0d67148003be40cc9eb4d8b25cc17f8204008d4
2020-09-08 15:48:27 +00:00
JT Olio
249f98f4c1 audit logging: log important user activity
Change-Id: Ic423dab4b6c513fa44a48cb6fe2903c70fe916fb
2020-09-07 15:26:34 +00:00
Egon Elbre
94a09ce20b all: add missing dots
Change-Id: I93b86c9fb3398c5d3c9121b8859dad1c615fa23a
2020-08-11 17:50:01 +03:00
Michal Niewrzal
4561d9bdb0 satellite/console/consoleweb: add support for partner parameter for
signup page

With this change partner id can be associated with user during creating
account by adding `?partner=partner_name` parameter to signup page url
e.g. https://tardigrade.io/signup?partner=mongodb

https://storjlabs.atlassian.net/browse/USR-999

Change-Id: I12a5ebec92a6f5135909447172ef24da57fb1c68
2020-07-30 11:29:31 +00:00
Bill Thorp
b265b7f555 satellite/console: make paywall optional
Add a config so that some percent of users require credit cards /
account balances
in order to create a project or have a promotional coupon applied

UI was updated to match needed paywall status

At this point we decided not to use a field to store if a user is in an
A/B
test, and instead just use math to see if they're in a test.  We decided
to use MD5 (because its in Postgres too) and User UUID for that math.

Change-Id: I0fcd80707dc29afc668632d078e1b5a7a24f3bb3
2020-07-28 10:57:49 +00:00
Kaloyan Raev
cb0caa2e25 satellites/payments: resolve data race in StripeMock
Change-Id: I0dbcf043e88edab12704742ad9653c62fd7b08f4
2020-07-28 09:53:50 +00:00
Kaloyan Raev
4bcf308a04 satellite/payments: fetch old deposit bonuses from Stripe metadata
Jira: https://storjlabs.atlassian.net/browse/USR-822

The balance history in Satellite GUI display the deposit bonuses as
separate rows. These bonuses used to be stored in the satellite DB. We
recently started depositing the bonus directly to the Stripe balance and
migrated old bonuses to Stripe metadata.

This change displays all billing history entirely from Stripe, so we can
remove the `credits` and `credits_spendings` DB tables in a next step.

Change-Id: I14c304c66ec47c6a51f5b8508f11470cf36c4e24
2020-07-23 12:11:17 +00:00
Kaloyan Raev
96c83eb0b8 satellite/payments: log returned error
This makes debugging of issues easier as the error in the web browser
does not show all the info.

Change-Id: Ia79902b7275103a78a21d43ba14243bc32bb8be5
2020-07-22 12:00:27 +00:00
Egon Elbre
ce7adc8d85 satellite/console: remove need for nolint
Change-Id: I1b7ab00720f630c4ad3f984adc5698b41368613b
2020-07-21 09:13:57 +00:00
Egon Elbre
d8dcae3075 all: fix error checking
Change-Id: Ia0da1bbd6ce695139922f94096c2419281905e32
2020-07-16 19:13:14 +03:00
Egon Elbre
e70da5cd4e all: fix comments
Change-Id: I2d2307e3fab87de47a72b3595d051e2c95ff4f8a
2020-07-16 19:13:14 +03:00
Egon Elbre
080ba47a06 all: fix dots
Change-Id: I6a419c62700c568254ff67ae5b73efed2fc98aa2
2020-07-16 14:58:28 +00:00
stefanbenten
9ace375ee0 satellite/{console,satellitedb}: change project limiting based on new users field
This change switches the backend logic to use the new DB column on the users table to restrict project creation.
Furthermore it back fills the existing limits from registration tokens to the new column to ensure no users are reset to the new default.

UI is updated to reflect ability to create several projects

Change-Id: Ie29157430ae6b065411ca4c4557c9f1be69cdc4f
2020-07-16 10:57:47 +00:00