JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

satellite/orders: don't store non user bandwidth actions for bucket

Browse Source 
For bucket_bandwidth_rollups we are trying to insert lots of entries
with empty bucket name and project id. Those are inserts from orders
created by repair, audit and GE. High load on the same primary key
(the same range) is causing many retries and that's affect all inserts
as we are putting 1000 entries into DB a once.

This change solves this problem by not storing into
bucket_bandwidth_rollups other actions then GET and PUT. Those actions
are only important from bucket bandwidth usage perspective because
those are actions performed by users. Other actions (repair, audit or
GE) are also stored in storagenode_bandwdith_rollups so we will still
have access to them e.g. for statistic purposes.

https://github.com/storj/storj/issues/5332

Change-Id: Ibb5bf0a4c869b0439dc65da1c9342a38ca2890ba
This commit is contained in:
Michal Niewrzal 2023-02-01 11:03:39 +01:00 committed by Storj Robot
parent 7dfa379d77
commit 15508d270c
1 changed files with 10 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
satellite/orders/endpoint.go
View File

@ -303,6 +303,15 @@ func (endpoint *Endpoint) SettlementWithWindowFinal(stream pb.DRPCOrders_Settlem
storagenodeSettled[int32(orderLimit.Action)] += order.Amount
// user can do only two actions which are important for bucket bandwidth usage
userAction := orderLimit.Action == pb.PieceAction_PUT || orderLimit.Action == pb.PieceAction_GET
// don't store anything else than user actions in bucket_bandwidth_rollups table. amounts for other
// actions will be stored in storagenode_bandwidth_rollups.
if !userAction {
continue
}
metadata, err := endpoint.ordersService.DecryptOrderMetadata(ctx, orderLimit)
if err != nil {
log.Debug("decrypt order metadata err:", zap.Error(err))
@ -332,13 +341,9 @@ func (endpoint *Endpoint) SettlementWithWindowFinal(stream pb.DRPCOrders_Settlem
continue
}
satelliteAction := orderLimit.Action == pb.PieceAction_GET_AUDIT ||
orderLimit.Action == pb.PieceAction_GET_REPAIR ||
orderLimit.Action == pb.PieceAction_PUT_REPAIR
// log error only for orders created by users, for satellite actions order limits are created
// without bucket name and project ID because segments loop doesn't have access to it
if !satelliteAction && (bucketInfo.BucketName == "" || bucketInfo.ProjectID.IsZero()) {
if bucketInfo.BucketName == "" || bucketInfo.ProjectID.IsZero() {
log.Warn("decrypt order: bucketName or projectID not set",
zap.String("bucketName", bucketInfo.BucketName),
zap.String("projectID", bucketInfo.ProjectID.String()),