This reverts commit 676178299f.
Reason for revert:
The new query used by this commit performs a full table scan.
It's been reverted pending a fix for that.
Change-Id: Idc53954459aa6f5a692056232b8674b11d1928ce
Some of tally queries are not passing bucket name as byte but as string.
If bucket contains some special characters encoding to bytea can fail.
This change makes sure all parts of tally passes bucket name correctly.
Change-Id: I7330d546b44d86a2e4614c814580e9e5262370ed
This change addresses an issue where the /charges endpoint will take a
while to respond due to a project having a large number of buckets.
The queries responsible for this have been merged into a single query to
get all data needed at a go and potentially improve performance.
Benchmarks indicate that this new way is more performant than the
previous.
name old ms/op new ms/op
Postgres/sum_all_partner_usages 7.534 0.622
Postgres/individual_partner_usages 6.104 0.588
Cockroach/sum_all_partner_usages 14.813 3.057
Cockroach/individual_partner_usages 16.046 2.852
Issue: https://github.com/storj/storj-private/issues/277
Change-Id: Ibb7f867ab6610b3cb1ba203961f7d6aef6bfda4a
Tallies are now created for buckets with no objects. Previously, the
bucket tally collector skipped empty buckets since it created tallies
only using information from the objects table. Methods that used
bucket tallies when calculating usage costs would return incorrect
results because of this.
Change-Id: I0b37fe7159a11cc02a51562000dad9258555d9f9
New SQL queries for GetProjectObjectsSegments turns out to introduce
full table scan. This is fix for this problem.
Change-Id: Ieac22aafeb780168523a97e27c9283c9ac6a24c8
Commit fb59974 disabled usage price overrides because of a failing
test. This change reenables it while resolving the issue that caused
the test to fail.
The previous version of the test passed Gerrit verification and was
merged, but it failed for the primary Jenkins pipeline after merge.
This is due to a difference in how the Jenkins build runs Cockroach
and Postgres for each pipeline.
This commit rewrites the test to be safe for concurrent execution by
ensuring any mutable variables are defined within each test so that
shared state across tests is reduced.
Change-Id: Ia4566c9cd2d698afdb2caa4b7e2808b17e18de4e
Project usage price overriding has been removed because it produces
incorrect results when tested. It should not be re-implemented until
the issues it causes are resolved.
Change-Id: Ic92eff374c9af4fea3bf32782a72303a7978b055
This change causes the bucket's partner info to be used rather than the
user's when calculating project usage prices. This ensures that users
who own differently-partnered buckets will be charged correctly for
usage based on the specific bucket they are utilizing.
according to the bucket's partner.
Related to storj/storj-private#90
Change-Id: Ieeedfcc5451e254216918dcc9f096758be6a8961
bucket_bandwidth_rollups table
We have performance problems with updating bucket_bandwidth_rollups. To
improve situation we can stop storing allocated bandwidth in this table.
This should reduce large number of updates which are comming from
metainfo endpoints, repair workers and audit.
Next step will be to drop `allocated` column completely from
bucket_bandwidth_rollups.
Allocated GET bandwidth is all we need and we are keeping it in
bucket_bandwidth_rollups table.
Change-Id: Ifdd26a89ba8262acbca6d794a6c02883ad0c0c9b
This change modifies the method responsible for returning project
usage summaries such that the end date of the given time period
is excluded to prevent overlap.
Change-Id: If06155efff5c6fce3865f5f6e4344873abe3e432
Fixed daily usage query returning single bucket usage.
We sum up bucket usages now.
Also fixed https://github.com/storj/storj/issues/4559.
Change-Id: I2eb6299f1ef500d68150879195011b6fbb5f37ed
We were already able to override (or not) metadata with this method
but to be explicit we are introducting new option to control storing
metadata with object. Separate option should be less error prone.
https://github.com/storj/team-metainfo/issues/105
Change-Id: I4c5bce953a633a0009b05c5ca84266ca6ceefc26
Added new projectaccounting query to get project's single bucket usage rollup.
Added new service method to call new query.
Added implementation for IsAuthenticated method which is used by new generated API.
Change-Id: I7cde5656d489953b6c7d109f236362eb465fa64a
Finished implementing queries for both bandwidth and storage using pgx.Batch.
Fixed CSP styling issue.
Change-Id: I5f9e10abe8096be3115b4e1f6ed3b13f1e7232df
