This change fixes data races in tests for the account freeze chore
caused by the chore service being set while in use.
Change-Id: Ibcb94a947c405314deae0d77419e69b768feab40
Once this is merged and deployed, we can go ahead
and change the db schema.
Note that this will stop updating the `type` column in the nodes
table but it shouldn't be a problem because this column has a
default value set.
Updates https://github.com/storj/storj/issues/5426
Change-Id: I2470ebacbcb0e60cf894617eb69b593227357283
This change adds a new column to the account_freeze_events table,
days_till_escalation, which is the number of days till the freeze event
is escalated. E.g.: 30 days from billing warning event to billing
freeze event.
Issue: #6382
Change-Id: I15cfd6f1208e641d8c380bef61717031befdad73
To get better performance, we pre-load all nodealias/node information at the beginning of the segment loop.
It's possible that we receive a new node alias from the segment table what we are not fully aware of (yet).
The easiest solution is just ignoring. New risks/threats can be detected by a new execution cycle.
Change-Id: Ib54f7edc46eedbab6d13b4d651aaac1425994940
Added client side logic to disable billing features depending on config value.
Disabled billing endpoints if billing is disabled.
Issue:
https://github.com/storj/storj-private/issues/464
Change-Id: I6e70dc5e2372953b613ddab9f19cb94f008935ce
The overlay.NodeDossier struct only tracks information about a
storagenode, the field is deprecated and no longer needed.
This is a kademlia left-over.
Updates https://github.com/storj/storj/issues/5426
Change-Id: Ie278ffd88d1b9a9fde6c81eb5f0e287bab8c9ef0
`overlay.NodeDossier` contains information that satellite tracks
about a storagenode.
As part of the work to remove the type column from the nodes
table https://github.com/storj/storj/issues/5426, we would have
to remove the `Type` field in the `overlay.NodeDossier` since it
is only about the storagenode.
The `Local` method on the contact service is also removed because
it is unused. When needed we can create a seperate `SatelliteInfo`
struct which the `Local` method will return.
Change-Id: If0c1a25d9df397a9492bbf1d7f33ba5b6a918878
This change limits payment attempts to
1. Card updates when billing frozen/warned
2. Right before billing freezing a warned account.
Issue: https://github.com/storj/storj-private/issues/457
Change-Id: Ic6d5c649cdac38d5c3b7365e20a4ceb3b6199ee8
This change adds a flag to the satellite config indicating whether
unregistered email addresses should receive project invitation emails.
Change-Id: I0396f25574ddae3f9adaea32a6e7cd15b931bf12
This change prevents multiple project invitation records from being
created from a single API request.
Change-Id: I01268fcc0e2f7b5f24870b032cb53f03c7ad0800
While the index shouldn't be necessary as long as our implementation is
correct, it still provides some additional checks for mistakes in the
implementation.
Change-Id: I7ed71ac99a979e375d7f94c8898e6f83ac623cb6
By using a separate function for deleting the latest object and
fetching the latest version we can simplify some of the code.
However, there can be more performant approaches, such as using
ON CONFLICT for updating the existing object or using select and delete
in the same query in databases that support it.
Change-Id: I52bc3f9fa025f44d05ee010723ffb81f5bd2a2d7
Allow user to download detailed usage report from Billing -> Overview screen.
Report is a CSV file containing usage data for all the projects user owns.
Issue:
https://github.com/storj/storj/issues/6154
Change-Id: I3109002bf37b1313652a2be3447aaa7bc6204887
There are several different object types in a versioned table,
which will determine the exact behaviour.
The object type states are:
* Pending - the object is yet to be committed and is being uploaded.
* Committed - the object has been finished and can be read.
* DeleteMarker - indicates that the object should be treated as not
present when is at the top of the version stack.
There are also versioning states:
* Unversioned - only one unversioned object is allowed per object key.
* Versioned - multiple objects with the same key are allowed.
Change-Id: I65dfa781e8da253a4e5d572b799d53c351196eee
I believe that this change in semantics won't break anything, because
ListVerifySegments is only used by cmd/tools/segment-verify (which only
needs to operate on non-expired segments) and various tests, none of
which expect ListVerifySegments to include expired segments.
Change-Id: I037f43b16bc5750ed914bc32949418e001df1a8c
This change adds user ID rate limiting to the endpoint responsible for
sending project invitations.
Resolves storj-private#462
Change-Id: Icf0be7d7bb7f2765725ba3e152a2195bc02484e2
ClassifySegmentPieces uses custom set implementation instead map.
Side note, for custom set implementation I also checked int8 bit set but
it didn't give better performance so I used simpler implementation.
Benchmark results (compared against part 2 optimization change):
name old time/op new time/op delta
RemoteSegment/healthy_segment-8 21.7µs ± 8% 15.4µs ±16% -29.38% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
RemoteSegment/healthy_segment-8 7.41kB ± 0% 1.87kB ± 0% -74.83% (p=0.000 n=5+4)
name old allocs/op new allocs/op delta
RemoteSegment/healthy_segment-8 150 ± 0% 130 ± 0% -13.33% (p=0.008 n=5+5)
Change-Id: I21feca9ec6ac0a2558ac5ce8894451c54f69e52d
Given the placement.txt with the content:
```
9:exclude(placement(10))
10:country("DE")
```
Without patch:
```
placement-test --placement=/tmp/placement.txt countrycode=DE
Error: type mismatch: reflect: Call using zero Value argument
```
With the patch:
```
placement-test --placement=/tmp/placement.txt countrycode=DE
Error: Error in line 'exclude(placement(10))' when placement rule is parsed: Placement 10 is referenced before defined. Please define it first!
```
Change-Id: I9ad81016d4a57fdb32f3ff9031b5591f9a7cd2a6
It would appear that we have been making concurrent accesses to
statsCollector for a long, long time (we expect there to be multiple
calls to `Repair()` at the same time on the same instance of
`SegmentRepairer`, up to `config.MaxRepair`, and before this change
there was no sort of synchronization guarding accesses to the
`statsCollector.stats` map.
Refs: https://github.com/storj/storj/issues/6402
Change-Id: I5bcdd13c88913a8d66f6dd906c9037c588960cc9
This change incorporates the UI type (legacy or Vuetify) in the
properties attached to analytics events originating from the satellite
UI.
Resolves#6363
Change-Id: Ie3627bc24e4349407376e28460a5a830d211b47b
This change adds two new admin endpoints to freeze users for ToS
violation and to remove them from that state,
Issue: https://github.com/storj/storj-private/issues/386
Change-Id: I49c922377c9cdb315ce2777fcd35dcad432b0539
Optimizing collecting monkit metrics:
* initialize metrics once at the begining
* avoid using string in map for getting stats structs per redundancy
Benchmark results (compared against part 1 optimization change):
name old time/op new time/op delta
RemoteSegment/Cockroach/healthy_segment-8 31.4µs ± 6% 21.7µs ± 8% -30.73% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
RemoteSegment/healthy_segment-8 10.2kB ± 0% 7.4kB ± 0% -27.03% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
RemoteSegment/healthy_segment-8 250 ± 0% 150 ± 0% -40.00% (p=0.008 n=5+5)
Change-Id: Ie09476eb469a4d6c09e52550c8ba92b3b4b34271
This change extends the account freeze functionality account for
violation freezes as well.
Also, debug level logs in the freeze chore have been changed to info.
It adds an analytics event for when an invoice is found that belongs to
a user frozen for violation.
And finally adds whether a user is frozen for violation to the
/account/freezestatus response.
Issue: https://github.com/storj/storj-private/issues/386
Change-Id: Id8e40282dc8fd8f242da52791ab8ddbbef3da2bc
Account freeze excludes accounts who have invoices which are unpaid and
are above a configured value. This change updates that configured value
from $100 to $1000.
Change-Id: Iff381deeb73ef2d8fccaf7a1612a33e8aeb08698
Add billing commands to manually complete or fail storjscan invoice
payments that are stuck in a pending state.
Change-Id: Ia19f0a2597201d9d17aad0889eaedff095d706b9
The API generators rely on the Name and Prefix fields of the
EndpointGroup type to generate code.
Conventional naming code requires using upper or lower case for types,
functions, etc, however requiring the user to set this fields with the
correct casing seems cumbersome for them because they can be adjusted
depending where those values are used on the generated code.
This commit lifts the restriction for the user and adjust the casing of
them according to where they are used.
Change-Id: I700a879d13b4789b4d6ba0519b4d7508061eac73
the existing test was missing a case where deleting the geofencing
should set the placement back to default.
this also changes EveryCountry to DefaultPlacement, as the former is
deprecated.
Change-Id: I29f4f1c3ae1d05e8f5ecaa03a70ac4f17574a475
deleteGeofenceForProject wasn't able to work correctly, because
Console().Projects().Update() declines to update default_placement when
the input value is 0.
This introduces a Console().Projects().UpdateDefaultPlacement() method,
congruent to the method of the same name on Console().Users().
deleteGeofenceForProject now uses this new method, so that specifying a
new placement of 0 will work correctly.
Change-Id: I4589b36707f7e4f1cfdc66543520b0d4205c1a84
This column is no longer used, but it is referenced in dbx. This change
removes those references and adds an exception to the migration test to
disregard this column in schema comparison.
After this change is deployed, we will need to follow up with a
migration to remove the column in a later release, and remove the
exception in the migration test.
Related to https://github.com/storj/storj/issues/5432
Change-Id: I168fb57244b347901d1ed9c7813c1338554ee644
Rename the Endpoint fields MethodName and RequestName because they were
confusing for what they are used.
This commit also adds some validations for these fields values and other
validations for Endpoint and EndpointGroup to avoid generating invalid
code.
It also include some tests for these new validations.
Closes https://github.com/storj/storj/issues/6333
Change-Id: Iaabfc33935517889e3729c8b37be51a55eea366c
this fixes cases where it's possible to update a user and the
DefaultPlacement field gets overwritten to the zero value.
it also adds UpdateDefaultPlacement which can be used to set
DefaultPlacement directly. This is needed for the geofencing
endpoints in satellite admin to set the DefaultPlacement back
to zero to delete geofencing for a user.
Change-Id: If2c798dabfa6773ed6023fb8257bf00ec7bc2e68
NodeSelection struct is used to make decisions (and assertions) related to node selection.
Usually we don't use email and wallet for placement decision, as they are not reliable.
But there are cases, when we know that the email address is confirmed. Also, it can be used for upper-bound estimations (if same wallet is used for too many pieces in a segment, it's a sign of a risk, even if not all the risks can be detected with this approach, as one owner can use different wallets).
Long story short: let's put wallet and email to the SelectedNode.
Change-Id: I922185e3769d43eb7762b8d60d88ecd3d50991bb
These test cases are the parts of the testplan for the Graceful Exit
Revamp which are automateable but not yet automated.
I'm not entirely sure why we have to reject graceful exit from nodes
that are suspended, but implementing that was probably easier than
convincing everybody that it's not necessary.
Refs: https://github.com/storj/storj/issues/6369
Change-Id: I0261b37f7e010d72d84332cde5dd8689f7c41580
