This change addresses an issue where the /charges endpoint will take a
while to respond due to a project having a large number of buckets.
The queries responsible for this have been merged into a single query to
get all data needed at a go and potentially improve performance.
Benchmarks indicate that this new way is more performant than the
previous.
name old ms/op new ms/op
Postgres/sum_all_partner_usages 7.534 0.622
Postgres/individual_partner_usages 6.104 0.588
Cockroach/sum_all_partner_usages 14.813 3.057
Cockroach/individual_partner_usages 16.046 2.852
Issue: https://github.com/storj/storj-private/issues/277
Change-Id: Ibb7f867ab6610b3cb1ba203961f7d6aef6bfda4a
This change removes the obsolete project member paging code.
Previously, we implemented functionality for including project
invitations in pages of project members. However, the satellite
frontend still expected API responses to use the old paging style, so
the related code could not be removed right away. Now that the frontend
has been updated, this code is no longer necessary.
References #5855
Change-Id: I12fdaaeb869977c4d87a0d50b9a7b11c68552c82
Added new endpoint, service method and DB query to get all API key names by provided project ID.
Issue:
https://github.com/storj/storj/issues/5693
Change-Id: I62e4e8ae660bd81234b75aa159a472a5aa9d5a48
The console service method responsible for removing members from a
project has been extended to remove project member invitations as well.
This will allow invitations to be deleted through the satellite
frontend.
References #5855
Change-Id: I90ca042cc6fb9a75fcd9b391e317caabb1c828f2
A method has been implemented that allows for paged searching through
project members and project member invitations. In the future, this
will be used to display invitations in the Team page of the satellite
frontend.
References #5855
Change-Id: I0937c425f60f1318e55202bf30b44a33ff695414
An index has been added on the project_id column of the project_members
satellite database table so that we can retrieve members of a project
without performing a full table scan.
References #5855
Change-Id: I1cc30686f836c8fd1aa319247ce857a2392e7a52
There were some dbx-generated queries on the old satellite version that
still reference this column, so we need to add it back until the next
version.
Change-Id: I78b19336d9ca0384936d6cc11f5c50e579b4f2ab
This change includes STORJ bonuses to the list of transactions returned
by the /wallet/payments endpoint.
Issue: https://github.com/storj/storj/issues/5755
Change-Id: Icc95c2cb9dd9fc5ee7a373e68c1cf8a991e1aa58
Methods SelectAllStorageNodesUpload and SelectAllStorageNodesDownload
are not returning full info with overlay.SelectedNode because its
missing CountryCode.
Change-Id: Ie3cb396bf28d7ec4c6ab8927e5bb560236036aa6
Tallies are now created for buckets with no objects. Previously, the
bucket tally collector skipped empty buckets since it created tallies
only using information from the objects table. Methods that used
bucket tallies when calculating usage costs would return incorrect
results because of this.
Change-Id: I0b37fe7159a11cc02a51562000dad9258555d9f9
The last code referencing these columns was removed as of satellite
release v1.79, so it is safe to remove them now.
Resolves https://github.com/storj/storj/issues/5432
Change-Id: I2e9d641b2511a61e0b9482ef0f4955a73c290709
During billing, before invoice creation, check if users are part of a
package plan. If so, and if the package plan is expired, remove unused
credit from the user's balance. If the user has credit in addition to
the package credit, send an analytics event to notify someone to handle
the credit removal manually.
Change-Id: Iad71d791f67c9733f9d9e42f962c64b2780264cc
The project member invitations table has been modified to contain a
column for the ID of the user who sent the invitation. This ID is
required for us to return information about the inviter to the
satellite frontend.
References #5855
Change-Id: I928d987a8db2340f731ca65ce30173d4f90a9837
87d0789691 replaces offset usage with cursors.
But to continue the interation from a specific cursor, we need to iterate over ordered records.
(at least this is what I understood based on the failing tests)
87d0789691
Change-Id: Ic4da3a7c5f03386dd4c373c05102f05871900a3a
This change updates the replacer in satellite/satellitedb/dbx/gen/main.go
to work with an updated dbx.
Change-Id: I08e89d6d27e6f1d435416105fe5f622009add7ad
Stripe invoice project records while listing are causing full table scan
because of OFFSET caluse. This change is refactoring query to list using
cursor.
Change-Id: I6b73b9b2815173d7ef02cf615408778476eb3b7b
We have method which is getting projects owned by specific user but it's
causing full table scan because we don't have index on owner_id column.
Change-Id: Icb71c9ac5b73104a52241ed8ba126c995c10811f
The console DB cleanup chore has been extended to remove old project
member invitation records.
Resolves#5816
Change-Id: Id0a748e40f5acf03b9b903265c653b072846ba19
There is still a reference to partner_id in a query, which we cannot
move until dropping the "not null" constraint for it. This change adds
that migration.
Related to https://github.com/storj/storj/issues/5432
Change-Id: I98802a6e1bd59f3d9214de3db6688d9daf664a70
A chore responsible for purging data from the console DB has been
implemented. Currently, it removes old records for unverified user
accounts. We plan to extend this functionality to include expired
project member invitations in the future.
Resolves#5790
References #5816
Change-Id: I1f3ef62fc96c10a42a383804b3b1d2846d7813f7
This change adds the user's passphrase prompt setting to the
/account/settings endpoints.
Issue: https://github.com/storj/storj/issues/5616
Change-Id: I48d470d49e82096fd090b74da323b279e342546e
This change immplements methods for interacting with the project member
invitations table.
Resolves#5766
Change-Id: I0090c50f9fde5bcdae4ebdaa72cdcaa84d341b54
A table for storing pending project member invitations has been added.
This table is required to satisfy our new project invitation UX revamp.
References #5766
Change-Id: I6f948de66ed5b4dc81532564958ff7f48533cad2
This field is deprecated in favor of UserAgent; Removing these
references is the final step necessary before dropping the columns from
the database.
https: //github.com/storj/storj/issues/5432
Change-Id: I3a6619170dcf382f82dc8eddb73b6547eaf636f0
New SQL queries for GetProjectObjectsSegments turns out to introduce
full table scan. This is fix for this problem.
Change-Id: Ieac22aafeb780168523a97e27c9283c9ac6a24c8
Query to list (with pages) stripe customers were doing full table scan
because Offset clause was used. This refactoring changed listing to
use cursor instead Offset.
Change-Id: I14688e6c533bc932ba0d209a061562f080b4cf54
We avoid putting more than one piece of a segment on the same /24
network (or /64 for ipv6). However, it is possible for multiple pieces
of the same segment to move to the same network over time. Nodes can
change addresses, or segments could be uploaded with dev settings, etc.
We will call such pieces "clumped", as they are clumped into the same
net, and are much more likely to be lost or preserved together.
This change teaches the repair checker to recognize segments which have
clumped pieces, and put them in the repair queue. It also teaches the
repair worker to repair such segments (treating clumped pieces as
"retrievable but unhealthy"; i.e., they will be replaced on new nodes if
possible).
Refs: https://github.com/storj/storj/issues/5391
Change-Id: Iaa9e339fee8f80f4ad39895438e9f18606338908
add ability to sort api keys on the access management page by ascending
or descending name or creation date. Additionally, edit api keys query
when ordering by name to order by lower(name) so names starting with
capital letters are not treated differently from lower case when
ordering.
Change-Id: I81dbb87587a24fb7097313f76bad116b1f20d306
Add columns package_plan and purchased_package_at to stripe_customers
table and add methods to update and select these values from console
service and payments accounts.
Change-Id: I1e89909055cc3054bfb7baa33c9dca3dfdc7336e
Implementing https://github.com/storj/storj/issues/5702 means adding a
bonus billing transaction for each storjscan transaction being recorded.
To do this idempotently, we need to the ability to for both the
storjscan and bonus transaction to be committed together.
This change updates the billing database to allow multiple billing
transactions to be inserted under the same database transaction.
Change-Id: I941864f47fc64d65aab076eec2e96fd04fcc7aac
FileWalker implements methods to walk over pieces in
in a storage directory.
This is just a refactor to separate filewalker functions
from pieces.Store. This is needed to simplify the work
to create a separate filewalker subprocess and reduce the
number of config flags passed to the subprocess.
You might want to check https://review.dev.storj.io/c/storj/storj/+/9773
Change-Id: I4e9567024e54fc7c0bb21a7c27182ef745839fff
