`overlay.NodeDossier` contains information that satellite tracks
about a storagenode.
As part of the work to remove the type column from the nodes
table https://github.com/storj/storj/issues/5426, we would have
to remove the `Type` field in the `overlay.NodeDossier` since it
is only about the storagenode.
The `Local` method on the contact service is also removed because
it is unused. When needed we can create a seperate `SatelliteInfo`
struct which the `Local` method will return.
Change-Id: If0c1a25d9df397a9492bbf1d7f33ba5b6a918878
This change limits payment attempts to
1. Card updates when billing frozen/warned
2. Right before billing freezing a warned account.
Issue: https://github.com/storj/storj-private/issues/457
Change-Id: Ic6d5c649cdac38d5c3b7365e20a4ceb3b6199ee8
This change adds a flag to the satellite config indicating whether
unregistered email addresses should receive project invitation emails.
Change-Id: I0396f25574ddae3f9adaea32a6e7cd15b931bf12
This change prevents multiple project invitation records from being
created from a single API request.
Change-Id: I01268fcc0e2f7b5f24870b032cb53f03c7ad0800
While the index shouldn't be necessary as long as our implementation is
correct, it still provides some additional checks for mistakes in the
implementation.
Change-Id: I7ed71ac99a979e375d7f94c8898e6f83ac623cb6
By using a separate function for deleting the latest object and
fetching the latest version we can simplify some of the code.
However, there can be more performant approaches, such as using
ON CONFLICT for updating the existing object or using select and delete
in the same query in databases that support it.
Change-Id: I52bc3f9fa025f44d05ee010723ffb81f5bd2a2d7
Allow user to download detailed usage report from Billing -> Overview screen.
Report is a CSV file containing usage data for all the projects user owns.
Issue:
https://github.com/storj/storj/issues/6154
Change-Id: I3109002bf37b1313652a2be3447aaa7bc6204887
There are several different object types in a versioned table,
which will determine the exact behaviour.
The object type states are:
* Pending - the object is yet to be committed and is being uploaded.
* Committed - the object has been finished and can be read.
* DeleteMarker - indicates that the object should be treated as not
present when is at the top of the version stack.
There are also versioning states:
* Unversioned - only one unversioned object is allowed per object key.
* Versioned - multiple objects with the same key are allowed.
Change-Id: I65dfa781e8da253a4e5d572b799d53c351196eee
I believe that this change in semantics won't break anything, because
ListVerifySegments is only used by cmd/tools/segment-verify (which only
needs to operate on non-expired segments) and various tests, none of
which expect ListVerifySegments to include expired segments.
Change-Id: I037f43b16bc5750ed914bc32949418e001df1a8c
This change adds user ID rate limiting to the endpoint responsible for
sending project invitations.
Resolves storj-private#462
Change-Id: Icf0be7d7bb7f2765725ba3e152a2195bc02484e2
ClassifySegmentPieces uses custom set implementation instead map.
Side note, for custom set implementation I also checked int8 bit set but
it didn't give better performance so I used simpler implementation.
Benchmark results (compared against part 2 optimization change):
name old time/op new time/op delta
RemoteSegment/healthy_segment-8 21.7µs ± 8% 15.4µs ±16% -29.38% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
RemoteSegment/healthy_segment-8 7.41kB ± 0% 1.87kB ± 0% -74.83% (p=0.000 n=5+4)
name old allocs/op new allocs/op delta
RemoteSegment/healthy_segment-8 150 ± 0% 130 ± 0% -13.33% (p=0.008 n=5+5)
Change-Id: I21feca9ec6ac0a2558ac5ce8894451c54f69e52d
Given the placement.txt with the content:
```
9:exclude(placement(10))
10:country("DE")
```
Without patch:
```
placement-test --placement=/tmp/placement.txt countrycode=DE
Error: type mismatch: reflect: Call using zero Value argument
```
With the patch:
```
placement-test --placement=/tmp/placement.txt countrycode=DE
Error: Error in line 'exclude(placement(10))' when placement rule is parsed: Placement 10 is referenced before defined. Please define it first!
```
Change-Id: I9ad81016d4a57fdb32f3ff9031b5591f9a7cd2a6
It would appear that we have been making concurrent accesses to
statsCollector for a long, long time (we expect there to be multiple
calls to `Repair()` at the same time on the same instance of
`SegmentRepairer`, up to `config.MaxRepair`, and before this change
there was no sort of synchronization guarding accesses to the
`statsCollector.stats` map.
Refs: https://github.com/storj/storj/issues/6402
Change-Id: I5bcdd13c88913a8d66f6dd906c9037c588960cc9
This change incorporates the UI type (legacy or Vuetify) in the
properties attached to analytics events originating from the satellite
UI.
Resolves#6363
Change-Id: Ie3627bc24e4349407376e28460a5a830d211b47b
This change adds two new admin endpoints to freeze users for ToS
violation and to remove them from that state,
Issue: https://github.com/storj/storj-private/issues/386
Change-Id: I49c922377c9cdb315ce2777fcd35dcad432b0539
Optimizing collecting monkit metrics:
* initialize metrics once at the begining
* avoid using string in map for getting stats structs per redundancy
Benchmark results (compared against part 1 optimization change):
name old time/op new time/op delta
RemoteSegment/Cockroach/healthy_segment-8 31.4µs ± 6% 21.7µs ± 8% -30.73% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
RemoteSegment/healthy_segment-8 10.2kB ± 0% 7.4kB ± 0% -27.03% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
RemoteSegment/healthy_segment-8 250 ± 0% 150 ± 0% -40.00% (p=0.008 n=5+5)
Change-Id: Ie09476eb469a4d6c09e52550c8ba92b3b4b34271
This change extends the account freeze functionality account for
violation freezes as well.
Also, debug level logs in the freeze chore have been changed to info.
It adds an analytics event for when an invoice is found that belongs to
a user frozen for violation.
And finally adds whether a user is frozen for violation to the
/account/freezestatus response.
Issue: https://github.com/storj/storj-private/issues/386
Change-Id: Id8e40282dc8fd8f242da52791ab8ddbbef3da2bc
Account freeze excludes accounts who have invoices which are unpaid and
are above a configured value. This change updates that configured value
from $100 to $1000.
Change-Id: Iff381deeb73ef2d8fccaf7a1612a33e8aeb08698
Add billing commands to manually complete or fail storjscan invoice
payments that are stuck in a pending state.
Change-Id: Ia19f0a2597201d9d17aad0889eaedff095d706b9
The API generators rely on the Name and Prefix fields of the
EndpointGroup type to generate code.
Conventional naming code requires using upper or lower case for types,
functions, etc, however requiring the user to set this fields with the
correct casing seems cumbersome for them because they can be adjusted
depending where those values are used on the generated code.
This commit lifts the restriction for the user and adjust the casing of
them according to where they are used.
Change-Id: I700a879d13b4789b4d6ba0519b4d7508061eac73
the existing test was missing a case where deleting the geofencing
should set the placement back to default.
this also changes EveryCountry to DefaultPlacement, as the former is
deprecated.
Change-Id: I29f4f1c3ae1d05e8f5ecaa03a70ac4f17574a475
deleteGeofenceForProject wasn't able to work correctly, because
Console().Projects().Update() declines to update default_placement when
the input value is 0.
This introduces a Console().Projects().UpdateDefaultPlacement() method,
congruent to the method of the same name on Console().Users().
deleteGeofenceForProject now uses this new method, so that specifying a
new placement of 0 will work correctly.
Change-Id: I4589b36707f7e4f1cfdc66543520b0d4205c1a84
This column is no longer used, but it is referenced in dbx. This change
removes those references and adds an exception to the migration test to
disregard this column in schema comparison.
After this change is deployed, we will need to follow up with a
migration to remove the column in a later release, and remove the
exception in the migration test.
Related to https://github.com/storj/storj/issues/5432
Change-Id: I168fb57244b347901d1ed9c7813c1338554ee644
Rename the Endpoint fields MethodName and RequestName because they were
confusing for what they are used.
This commit also adds some validations for these fields values and other
validations for Endpoint and EndpointGroup to avoid generating invalid
code.
It also include some tests for these new validations.
Closes https://github.com/storj/storj/issues/6333
Change-Id: Iaabfc33935517889e3729c8b37be51a55eea366c
this fixes cases where it's possible to update a user and the
DefaultPlacement field gets overwritten to the zero value.
it also adds UpdateDefaultPlacement which can be used to set
DefaultPlacement directly. This is needed for the geofencing
endpoints in satellite admin to set the DefaultPlacement back
to zero to delete geofencing for a user.
Change-Id: If2c798dabfa6773ed6023fb8257bf00ec7bc2e68
NodeSelection struct is used to make decisions (and assertions) related to node selection.
Usually we don't use email and wallet for placement decision, as they are not reliable.
But there are cases, when we know that the email address is confirmed. Also, it can be used for upper-bound estimations (if same wallet is used for too many pieces in a segment, it's a sign of a risk, even if not all the risks can be detected with this approach, as one owner can use different wallets).
Long story short: let's put wallet and email to the SelectedNode.
Change-Id: I922185e3769d43eb7762b8d60d88ecd3d50991bb
These test cases are the parts of the testplan for the Graceful Exit
Revamp which are automateable but not yet automated.
I'm not entirely sure why we have to reject graceful exit from nodes
that are suspended, but implementing that was probably easier than
convincing everybody that it's not necessary.
Refs: https://github.com/storj/storj/issues/6369
Change-Id: I0261b37f7e010d72d84332cde5dd8689f7c41580
Currently, pending invoice payments that are made using a users token
balance can get stuck in a pending state if the invoice is not able
to be paid appropriately in stripe. This change addresses these stuck
token invoice payments by attempting to transition them to failed
if the invoice cannot be paid.
Change-Id: I2b70a11c97ae5c733d05c918a1082e85bb7f73f3
This change modifies wallet payments endpoints to return empty lists
instead of returning a 404 error if a wallet is not found for a user.
Change-Id: Ic765fecbc8183d14f179ce1d510ae512d8e0c4a9
Current bucket_metainfos table schema was far from optimal. All
operations are using project_id and name to find bucket but primary key
was on id field which is completely not used. Turns out that secondary
index was not as fast as it could be because each select was also joining
primary index which was slowing down whole query.
Primary key is changed to project_id/name. Other indexes are dropped
and id column is now nullable which gives us space for dropping it
completely from schema and code later.
Change-Id: I477056a4243d39e39489f1473de01ded1ed24bbb
This change updates the version of the stripe-go dependency from v73 to
v75 in order to improve performance (see stripe/stripe-go#1728).
Resolves#6287
Change-Id: I727ac08dcaa90d7138ba30d907711cc44daf0b7a
This change updates the version of the stripe-go dependency from v72 to
v73. This is part of a process to reach v75, which contains performance
improvements.
References #6287
Change-Id: I95f132378fe05e506f0388a0fe039cb1c2db58e2
Removed code which was responsible for tracking regular user activity in hubspot.
The only direct event we send to Hubspot is 'user created'.
Issue:
https://github.com/storj/storj-private/issues/441
Change-Id: Ia6da30374f91369372af8fe5b5990ec5672fc03b
Fix the API generator to generate valid TypeScript code when using
slices an arrays of any type (base types, struct types, anonymous struct
types, etc.).
Closes https://github.com/storj/storj/issues/6323
Change-Id: I580ae5305c58f65c2e4f4a35d14ca4ee509a9250
With the upcoming versioning changes `BeginObjectExactVersion` makes
only sense for testing. Currently this does not rename the options
struct or move it into `metabasetest`, because it would create a
significant amount of merge/rebase noise.
Change-Id: Iafa2f81a05ae66320bc6a839828217ec94c63e1f
To live count observer is emitting metric with number of segments
already processed. It's handy to see current progress.
Change-Id: Id6766b5ec9b05bce8f6376e9afcae91fcf7df6fd
This change introduces a new freeze event, ViolationFreeze, for ToS
violations and also prepends other events with Billing to clarify what
they signify.
Issue: https://github.com/storj/storj-private/issues/386
Change-Id: Ieef2abbbb03d06377dc7a73ba5ef0742ada75e8e
