Commit Graph

10 Commits

Author SHA1 Message Date
Michal Niewrzal
8e030f4e30 satellite/accounting/tally: fix looping over all buckets
We have a bug where if number of buckets in the system will be
multiplication of batch size (2500) then loop that is going over
all buckets can run indefinitely.

Fixes https://github.com/storj/storj/issues/5374

Change-Id: Idd4d97c638db83e46528acb9abf223c98ad46223
2022-12-07 10:32:23 +00:00
Qweder93
5e5d6ecf6c satellite/{accounting, buckets}: added feature that allows to use custom SQL query instead of object loop for buckets accounting
since amount of objects is growing and looping through all of them
starts taking lot of time, we are switching for SQL query to do it
in chunks of tallies per bucket. 2nd part of issue fix.

Closes https://github.com/storj/team-metainfo/issues/125

Change-Id: Ia26bcac0a7e2c6503df9ebbf4817a636841d3284
2022-11-02 09:01:33 +00:00
Márton Elek
76c2228fbd satellite/metainfo: propagate geofencing between buckets and stream id
Github: https://github.com/storj/storj/issues/4245

Change-Id: I83d34367aab1f3c0d46a044f54980b2d50174b19
2021-11-24 08:05:05 +00:00
Mya
814e3126fa satellite/buckets: add new buckets service
The main motivation is to wrap the bucket DB and metainfo DB, so we
could check if a bucket is empty before applying geofencing config.

Change-Id: I8bac21555e01d51a663fb557bc1acfc8106bc2e1
2021-11-16 12:36:17 +02:00
Michał Niewrzał
4a146000cc satellite/metainfo: read from DB only needed columns fro bucket
Uplink needs only part of columns we are reading from DB.
To improve performance we should read only those that are
realy needed.

Change-Id: Ib39259318169c46afe5fa4c6ce2184da82e960c8
2021-10-19 11:53:42 +00:00
dlamarmorgan
4bbf667ad1 satellite/{satellitedb,attribution,console}: value attribution changes that add userAgent field to buckets table and all tables that have partner_id
Change-Id: I36a13bb651b86bfc14fe5a0a2258f719e6cd2b48
2021-10-18 13:56:19 +00:00
Egon Elbre
267506bb20 satellite/metabase: move package one level higher
metabase has become a central concept and it's more suitable for it to
be directly nested under satellite rather than being part of metainfo.

metainfo is going to be the "endpoint" logic for handling requests.

Change-Id: I53770d6761ac1e9a1283b5aa68f471b21e784198
2021-04-21 15:54:22 +03:00
Jeff Wendling
2462ac5917 metainfo+satellite/satellitedb: use HasBucket when possible
Change-Id: I8318f5581eef559f0dfbf20d07ea1fe36f81ce87
2021-04-02 13:20:36 -04:00
Michał Niewrzał
d995fb497f Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: I367da03351ab80f7343332420490dde9282aa47a
2021-02-23 12:31:31 +01:00
Egon Elbre
1137620baf satellite/satellitedb: move tests to their domains
Testing interfaces is slightly clearer when it's in the package needing
the database rather than each individual implementation.

Change-Id: I10334c214a205f7e510b939b4359a2214c4e060a
2021-02-19 17:29:15 +02:00