To have the same UploadID returned from BeginUpload, ListObjects
and ListPendingObjectStreams we need to set StreamID.Versioned field
everywhere.
Change-Id: I1328d9c476767559b8feb7c5bcd5afb154f7cee3
Add idempotency key for every create/update invoice item request.
Key consists of public project ID, line item description and formatted invoicing period.
This should eliminate the possibility of making duplicate create/update actions.
Also, with this change, we mark a project record as consumed only after the appropriate invoice items are created or the record itself is skipped.
Issue:
https://github.com/storj/storj/issues/6501
Change-Id: I7506a8f3886d7f575bcc0facc3f107513352a312
This change fixes the behavior of legal freeze; it now does not allow
lists and deletes for users in that freeze. It also fixes an issue
where users who have been legal/violation frozen will have their limits
upgraded, partially exiting freeze status when they pay with STORJ
tokens.
Issue: storj/storj-private/#517
storj/storj-private/#515
Change-Id: I6fa2b6353159984883c60a2888617da1ee51ce0a
We couldn't delete API keys by name whose name contained slashes because
Gorilla Mux router interpreted the as path separator and didn't resolve
to the right endpoint.
To fix the issue the name is sent as a query parameter rather than as a
path parameter.
Change-Id: Ica67d6b9f047d7c33a5350457afc822cb8d4c7a1
This is the final change to completely drop the type
column from the nodes table.
All code references are removed and deployed, dbx changes
have been merged and deployed.
Updates https://github.com/storj/storj/issues/5426
Change-Id: I1b76fe92f25e350c1c44730c008f0e9107852509
Current value is too big and can cause memory issues. With 500 bloom
filters where single filter can go up to 2MB we can end up with 1GB zip
which is more problematic if we don't want to download it before
processing.
Change-Id: I4b983afbcac39c718afbbeb764a30d32c4e80c08
This change adds a new flag to toggle whether to use the v2 app signup
flow. If disabled, the v2 signup page will redirect to the current
app's signup page.
Issue: #6552
Change-Id: I0d48ca713dc77cb2879630613f92123baced3ca3
New instance based satellites are reporting durability alerts like `class=last_net, value=192.168.0.0`.
Bus factor will be reported as `class=bus_factor value=last_net`.
Change-Id: I0130866fa8e50b3d2876f580a78383bb1c609159
Display a banner on the dashboard of all projects for bot accounts, with a prompt to submit a support ticket.
Issue:
https://github.com/storj/storj-private/issues/503
Change-Id: Id5f2aae0394b4fdca968a4de93bda19d6b0eb37a
The project ID must never be sent out from the server because old
projects used that ID as salt for creating access grants.
We must always expose the public ID and the server must retrieve the ID
when the public ID is sent in to the server and needs it for performing
some DB queries.
Change-Id: I4a9d94049ef84f578b62907827d6c09dfd1db4db
Added new admin functionality to disable account bot restriction by activating it.
This must be used for accounts with status PendingBotVerification.
Note: PendingBotVerification status is automatically applied for accounts with high captcha score right after activation.
Issue:
https://github.com/storj/storj-private/issues/503
Change-Id: I4c9ee834075a7abaf221ac054a455a8d27debf40
Added new captcha score cutoff threshold config value (default is 0.8).
Added new user status PendingBotVerification which is applied right after account activation if signup captcha score is above threshold.
Restricted project creation/joining if user's status is PendingBotVerification.
Issue:
https://github.com/storj/storj-private/issues/503
Change-Id: I9fa9932ffad48ea4f5ce8235178bd4af45a1bc48
Create an API generator middleware for being able to hook the new
satellite admin authorization in the endpoints.
The commit fixes a bug found in the API generator that caused that
fields of types of the same package of the generated code where wrongly
added. Concretely:
- The package matching was missing in the function middlewareFields,
hence it was generating code that referenced types with the package
name.
- middlewareFields function was not adding the pointer symbol (*) when
the type was from the same package where the generated code is
written.
There is also an accidental enhancement in the API generator because I
thought that the bug commented above corresponded to it, rather than
removing it, I though that was worthwhile to keep it because it was
already implemented. This enhancement allows to use fields in the
middleware with packages whose last path part contains `-` or `.`, using
a package rename in the import statement.
Change-Id: Ie98b303226a8e8845e494f25054867f95a283aa0
Add a new method to the Authorizer to use it with the API midleware that
we are going to implement for injecting it into the handler generated by
the API generator.
This new method will reduce the lines of code to generate and avoiding
errors that are more difficult to test in generated code.
The commit deletes the Middleware method because we won't used due to
the API generator doesn't support "standard" middlewares and allows
their customization via code generated and injected inside the handler
base logic generated by the API generator.
Change-Id: Ie427eb2eea94797913e2c357cf097ecf1e2e63ef
We don't need to send all the information that the endpoint currently
provides - name and email is sufficient.
Change-Id: I088d61edfc022a4a40c96e1faebd48808af5b00f
This change modifies the register endpoint handler to not require name
for signups from the v2 app and adds a new endpoint for completing
account information (e.g. name). This is to support the new signup and
account setup flow of the v2 app.
Issue: #6470
Change-Id: I256e1c804fcdbc8ce05aa82d6bc4b0263f55daa5
This change implements account activation using OTP code. Based on
whether this activation method is activated, signup will generate a
6-digit code and send it via email. A new endpoint is added to validate
this code and activate the user's account and log them in.
Issue: #6428
Change-Id: Ia78bb123258021bce78ab9e98dce2c900328057a
Test was flaky because we asserted if we have 15 classes:
6 email (for each used (!!!) nodes)
6 last_net (for each used (!!!) nodes)
1 wallet
1 country ("HU")
1 empty value
But there was a very low chance to use only 5 nodes, out of the 6 (RS.Success=5, RS.Total=6).
In that specific case, we had only 12 classes, as we didn't see all the used emails, as we iterated over the used nodes only (and one node was not used).
https://github.com/storj/storj/issues/6549
Change-Id: I66882d5fa9b0d5f5b2397ea856494037972d4b81
This inserts additional information provided in error messages from
Hubspot into the Go error that is eventually logged out.
Before, we would see the a generic "sending event failed" log.
With this change, we will see more detailed information in the log, such
as a list of required fields that were not submitted.
Change-Id: I24da0646bca62f459377abe6281403020fb54c49
This change adds a search field to the new admin UI through which user
email addresses may be submitted. If the email belongs to a verified
user, the client will be redirected to the Account Details page which
is populated with the user's information.
Resolves#6469Resolves#6475
Change-Id: Icbf3cb3f8374f2764e73a523f111c5ecf3d06569
Inject the middleware code after the query and path parameters parsing
because in the case that a middleware needs to execute a more expensive
logic, it isn't executed if there are malformed or missing parameters.
Change-Id: I883ade8ee42d25accb153c78d9f583d883730d6a
Added new optional '--aggregate' flag for billing.generate-invoices and billing.prepare-invoice-records commands to toggle invoice items aggregation.
Added new explicit command billing.create-aggregated-project-invoice-items which should be used after preparing aggregated invoice records (in cases when invoice generation happens step-by-step).
Change-Id: I04fc0110be5263edb959306d5314a4a1a8eec3ba
This change adds an endpoint to the back office API that returns user
info based on email address.
References #6503
Change-Id: Ib48d30b0b6c6862887b3f8114f50538b3deca57b
Add a README document with some general API guidelines that we wrote in
a Slack canvas to discuss them.
Change-Id: Iec933edeb7622b78a98155512b25267d12879837
If the API description or version is empty, don't print out in the docs
a placeholder for them.
In commit bcde51138d a space between the
placeholder and description was removed because it was printing a
trailing space if the description was empty, but that introduced another
bug, which is not adding the space when the description is empty.
This commit fixes the bug, no printing the placeholder if the
description is empty, otherwise print it with the corresponding blank
space in between the placeholder and the description.
Change-Id: I19c92c3dd7dfb38b8b2a43821bafc726ffad59d7
Fix a case where it was possible to create a delete marker when only
pending object was present.
The solution is not pretty, but we have a TODO note to cleanup the
precommit code, so let's fix the bug first.
Change-Id: I0ab66d745443c9dccbf29ef32389dd912b2d9caf
The API generator doesn't have a way to customize each Go handler
endpoint unless that the Go generator is modified.
This commit adds a way to customize each endpoint injecting instances of
types that implement an interface (Middleware) that return the code to inject.
To show how it works, the commit get rid of the 2 fields that we used to
customize the authentication request with the logic that the
satellite/console/consoleweb/consoleapi needs and replace the hardcoded
customization using this new way to customize handlers.
This new way should allow to hook the satellite/admin/back-office
authorization into the handlers using a Middleware implementation.
Change-Id: I894aa0026b30fa2f4a5604a6c34c22e0ed582e2b
The code to get daily storage usage is a little confusing. There is a
portion of the query which looks like it is trying to sum the storage
for buckets by each day, but it doesn't work. There is also a section
which sums the buckets by reading the returned rows, but it assumes that
the rows are ordered by the date, which they are not. This results in
the potential for multiple entries for the same day being created: one
for each bucket. The UI doesn't know about this, so the entries for the
same day overwrite each other and result in the appearance of less
stored data on the graph.
To fix it, don't group by bucket and allow the query to sum the buckets
together.
Change-Id: Id3ec25238e6e6346b93b10f834bdc9c70023f1e1
This change adds a new column to the user table, which will hold a
random code for account activation. And another to hold the signup
request ID as another layer of verification.
Issue: #6428
Change-Id: Icd46cb5d8fc76102264d599aca27686cd8b2e84e
Currently we have large set of nodes that are already disqualified and
we are not sending bloom filters to them. The issue is that we are still
generating filters for them while garbage collection process. Even if
we have only segment with one piece which was stored on this node. This
consumes additional memory and processing powers.
This change is changing logic behind `AllPieceCounts` (renamed to
ActiveNodesPieceCounts) to return piece count for all nodes except disqualified one (even with piece count = 0). With this change we can
modify GC observer to skip nodes that where not returned by
ActiveNodesPieceCounts.
Change-Id: Ic75159135abe535084d8aeee560bb801a4a03e17
This change gives the proper type to TS class fields generated from
nillable Go struct fields. Previously, Go struct fields having a nil
representation ([]Type, *Type, etc.) were translated into TypeScript as
"Type | undefined". This isn't correct because these fields, when nil,
are given the value "null" when marshalled into JSON. This change fixes
this issue by giving these fields the type "Type | null".
Change-Id: I5a1a83eb3810a3cba10895bb2f0f75ca5fd7d1b5
the new bus_factor calculation doesn't make sense with different classes, as we have overlaps.
For example: it can detect a risk if we loose one country and one different subnet (with possible overlap).
It's better to calculate the stat and bus_factor per class (net, country, ...).
It also makes it easier to measure execution time per class.
Change-Id: I7d4d5f7cb811cd50c5831077b43e001908aab96b
For S3 compatibility we should not allow returning pure delete
marker object. If metabase returns delete marker metainfo will
return MethodNotAllowed rpc status.
https://github.com/storj/storj/issues/6522
Change-Id: I89804b2bd22da0e5beec8f106e74b74733e19a52
