When a new user registers, we send a verification request to their
email. Currently, if they do not verify their email, we take no further
action. We want to send these users reminders: one after about one day
and one after about 5 days. To do this we will use this new
verification_reminders column.
It will look something like this:
```
SELECT email FROM users
WHERE status = 0
AND (
(verification_reminders = 0 AND created_at < now() - 'INTERVAL 1d')
OR (verification_reminders = 1 AND created_at < now() - 'INTERVAL 5d')
)
```
Change-Id: If0620e08c97e9e337c9563481d665c5bd462693b
Initialy we wanted to put segment usage into cache values retrieved
directly from metabase but it cause performance issues. Now we
will be collecting segment usage from tally object loop and those
values will be put into a cache but becuse we cannot get those values
on demand we shouldn't expire cache value at all because objects
loop requires sustencial amount of time to be executed.
Part of solution for https://github.com/storj/storj/issues/4744
Change-Id: I3b37e23badeecebed0c95064156e85b38038bfe2
We want to send email verification reminders to users from the satellite
core, but some of the functionality required to do so exists in the
satellite console service. We could simply import the console service
into the core to achieve this, but the service requires a lot of
dependencies that would go unused just to be able to send these emails.
Instead, we break out the needed functionality into a new service which
can be imported separately by the console service and the future email
chore.
The consoleauth service creates, signs, and checks the expiration of auth
tokens.
Change-Id: I2ad794b7fd256f8af24c1a8d73a203d508069078
Adds a new configuration for hcaptcha enabled, secretkey, and sitekey.
If both reCAPTCHA and hCaptcha are configured as "enabled", reCAPTCHA
will be used.
Change-Id: I73cc6e133d8da3555e0ed8b2b377cf9eb263e6dc
Fix for this customer issue
https://github.com/storj/customer-issues/issues/34
By this change we fetch bucket usage since its creation instead of using project's createdAt timestamp.
Change-Id: Ic0ea5d169056a5bd64ed143d13954d794da6e1d2
Things that make debugging easier.
* Added logging to automatic link clicking to make it obvious, when it
fails.
* Added monitoring to oidc.
* Made dbx create calls noreturn for oauth_*
Change-Id: I37397b4e84ce5bfd82954aed9c38fdfd52595f24
This will apply an appropriate "subsystem" label to goroutines which are
part of the core, api, repairer, admin, or gc subsystems.
It will also label goroutines whose job it is to watch for slow shutdown
of lifecycle groups (there are a lot of these).
Finally, this will also label goroutines whose job it is to wait on the
toplevel errgroup of a subsystem.
Change-Id: I560b5fff4a0101300d6c9a67609c2d80d7424486
Added account locking on 3 or more login attempts.
Includes both password and MFA failed attempts on login.
Unlock account on successful password reset.
Change-Id: If4899b40ab4a77d531c1f18bfe22cee2cffa72e0
Implement a buffer for inserting repair items into the queue in a batch.
Part of https://github.com/storj/storj/issues/4727
Change-Id: I718472b2f2b1f4993c3d6f15c44923776407155a
TRUNCATE is faster than DELETE when deleting all rows.
As almost every metabase test case calls TestingDeleteAll, this change
should give some slight test speed-up.
Change-Id: Ib477962b6deb93edd60d6db2f1be6ede1b4b2381
Create an error class for the "pending object error" for distinguishing
it from other errors for allowing to return it as a "Not Found" DRPC
status code instead an "Internal" status code.
"Internal" errors are logged in the satellite error so this was
polluting the server logs aside of returning an inappropriate status
code.
Change-Id: I10a81adfc887c030c08a228158adc8815834b23c
Respond with the appropriate HTTP status code when a request to the
analytics trigger event handler receive an authorized request.
A part of fixing the response status code this will stop to log these
response with ERROR level in our satellite logs.
Example of error message found in our satellite logs:
{
"insertId": "0ljf1cfn4xroxfd6",
"jsonPayload": {
"N": "console:endpoint",
"T": "2022-05-06T13:31:35.415Z",
"errorVerbose": "unauthorized: http: named cookie not present\n\tstorj.io/storj/satellite/console.GetAuth:72\n\tstorj.io/storj/satellite/console/consoleweb/consoleapi.(*Analytics).EventTriggered:60\n\tnet/http.HandlerFunc.ServeHTTP:2047\n\tstorj.io/storj/satellite/console/consoleweb.(*Server).withAuth.func1:488\n\tnet/http.HandlerFunc.ServeHTTP:2047\n\tgithub.com/gorilla/mux.(*Router).ServeHTTP:210\n\tstorj.io/storj/satellite/console/consoleweb.(*Server).withRequest.func1:495\n\tnet/http.HandlerFunc.ServeHTTP:2047\n\tnet/http.serverHandler.ServeHTTP:2879\n\tnet/http.(*conn).serve:1930",
"L": "ERROR",
"error": "unauthorized: http: named cookie not present",
"message": "unauthorized: http: named cookie not present",
"code": 500,
"S": "storj.io/storj/satellite/console/consoleweb/consoleapi.serveCustomJSONError\n\t/go/src/storj.io/storj/satellite/console/consoleweb/consoleapi/common.go:37\nstorj.io/storj/satellite/console/consoleweb/consoleapi.serveJSONError\n\t/go/src/storj.io/storj/satellite/console/consoleweb/consoleapi/common.go:23\nstorj.io/storj/satellite/console/consoleweb/consoleapi.(*Analytics).serveJSONError\n\t/go/src/storj.io/storj/satellite/console/consoleweb/consoleapi/analytics.go:75\nstorj.io/storj/satellite/console/consoleweb/consoleapi.(*Analytics).EventTriggered\n\t/go/src/storj.io/storj/satellite/console/consoleweb/consoleapi/analytics.go:62\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2047\nstorj.io/storj/satellite/console/consoleweb.(*Server).withAuth.func1\n\t/go/src/storj.io/storj/satellite/console/consoleweb/server.go:488\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2047\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/go/pkg/mod/github.com/gorilla/mux@v1.8.0/mux.go:210\nstorj.io/storj/satellite/console/consoleweb.(*Server).withRequest.func1\n\t/go/src/storj.io/storj/satellite/console/consoleweb/server.go:495\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2047\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2879\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:1930",
"M": "returning error to client"
},
"resource": {
"type": "k8s_container",
"labels": {
"location": "us-central1",
"pod_name": "us-central1-satellite-api-77c47f5c5-dzrpj",
"project_id": "storj-prod",
"namespace_name": "satellite",
"container_name": "satellite",
"cluster_name": "us-central1-gke-manatee"
}
},
"timestamp": "2022-05-06T13:31:35.416050390Z",
"severity": "ERROR",
"labels": {
"k8s-pod/version": "v3",
"k8s-pod/app": "us-central1-satellite-api",
"compute.googleapis.com/resource_name": "gke-us-central1-gke--terraform-202110-97ff1891-t0fv",
"k8s-pod/service": "api",
"k8s-pod/pod-template-hash": "77c47f5c5"
},
"logName": "projects/storj-prod/logs/stderr",
"receiveTimestamp": "2022-05-06T13:31:37.419991630Z"
}
Change-Id: I7cfcfb500b7878c59b1d259683c92e8963e2dc3f
Co-authored-by: Stefan Benten <mail@stefan-benten.de>
return storage and segment totals as a single result, instead of returning only storage
and bandwidth and segment values are filtered out, https://github.com/storj/storj/issues/4744
Change-Id: I624d67ed5205ae21ecd5a2f39775f63ed042e629
TestSegmentInExcludedCountriesRepair and TestSegmentInExcludedCountriesRepairIrreparable are using 20 storage nodes.
This change make them use 7 by adjusting the test redundancy scheme.
Change-Id: I1a44aa8b997d6edcc9a3305fdd0dac57e4d525b5
* Added new feature Flag for new Access Grant Flow.
* Added 3 cards to access grant view for S3, CLI and Access grant to replace old header
* Added new formatting, text and Icon for Access Grant Delete Popup modal
Added documentation.
Replaced PUT request with POST request.
Added inline param support for PATCH request.
Replaced unix timestamps handling with RFC-3339 timestampts handling.
Added 'Bearer' method requirement for Authorization header.
Change-Id: I4faa3864051dd18826c2c583ada53666d4aaec44
When an application wants to interact with resources on behalf of
an end-user, it needs to be granted access. In OAuth, this is done
when a user submits the consent screen.
Change-Id: Id838772f76999f63f5c9dbdda0995697b41c123a
Version collector previously returned errors and logged them in the
calling code. It is cleaner to log inside version collector.
Change-Id: I52cb49a1ef53f3f1f51692ddb26ec095cfd0f100
We were already able to override (or not) metadata with this method
but to be explicit we are introducting new option to control storing
metadata with object. Separate option should be less error prone.
https://github.com/storj/team-metainfo/issues/105
Change-Id: I4c5bce953a633a0009b05c5ca84266ca6ceefc26
"REST API" is a more accurate descriptor of the generated API in the
console package than "account management API". The generated API is very
flexible and will allow us to implement many more endpoints outside the
scope of "account management", and "account management" is not very well
defined to begin with.
Change-Id: Ie87faeaa3c743ef4371eaf0edd2826303d592da7
Extended user update query so prod owner can change user's paid tier status, bandwidth, storage and segment limits.
Change-Id: I82768afd1e50f653a50f7020310ce1e91578d746
it's a bit weird that these code definitions are in storj/storj
instead of storj/crypthopper-go, but as it stands, we should make
sure this package knows about all the codes in use.
Change-Id: I8df4666a015098e2d2e536d2f6c8ca5317a4369c
We implemented server-side copy feature and we would like to
confirm that it is not affecting expired deletion service.
Resolves: https://github.com/storj/storj/issues/4698
Change-Id: Ia8ca27a7ab7764a48a0c85dc7be80a58bfc83729
Last part of backwards compatible db migration to remove "suspended" column.
Removes exeption which removes "suspended" column in tests from `migrate_test.go`.
Adds DB migration to remove "suspended" column from 'nodes' and 'reputations' tables.
Change-Id: I02051279f6f4181e966c567919af0e774583f165
Set disqualification reason when reputations stats are updated on DB.Update.
Added tests for DisqualifyNode and for disqualification cases which happens during Update.
Change-Id: I00130ab5d9722422805159ad2f183c205de60f7e
When an api server is processing a graceful exit (node is connected and
getting lists of pieces to transfer), and the api server is shut down,
it was incorrectly marking all pending graceful exits as complete. The
GE then either passed or failed depending on the ratio of successfully
transferred pieces to unsuccessful pieces. In at least one case, _no_
pieces were transferred at all before the GE was marked a success.
Change-Id: I62cfab54a2296572c2e654eb460b62f772b7a60b
Attribution is attached to bucket usage, but that's more granular than
necessary for the attribution report. This change iterates over the
bucket attributions, parses the user agent, converts the first entry
to lower case, and uses that as the key to a map which holds the
attribution totals for each unique user agent.
Change-Id: Ib2962ba0f57daa8a7298f11fcb1ac44a8bb97875
Implemented new endpoint for project update using apigen.
Implemented new service method compatible with new generated api.
Change-Id: Ic0a7e0bbf3ea942275bd927d6e30cfb7e721e9c1
Migrate free tier users to have default limits if their limits were set to 0.
They were affected by incorrect working of Update user query.
Change-Id: I4c49c8d99b12dba2b9b0ab61b2175085976dcc95
