storj/satellite/gc
Michal Niewrzal 158eb2381e satellite/gc/bloomfilter: uploading bloom filters
We would like to have separate process/command to collect bloom
filters from source different than production DBs. Such process will
use segment loop to build bloom filters for all storage nodes and
will send it to Storj bucket.

This change add main logic to new service. After collecting all bloom
filters with segment loop and piece tracker all filters are marshaled
and packed into zip files. Each zip contains up to "ZipBatchSize" bloom
filters and it's uploaded to specified in configuration bucket.

All uploaded objects have specified expiration time to not delete them
manually.

Updates https://github.com/storj/team-metainfo/issues/120

Change-Id: I2b6bc02a7dd7c3a639e75810fd013ae4afdc80a2
2022-09-12 08:33:53 +00:00
..
bloomfilter satellite/gc/bloomfilter: uploading bloom filters 2022-09-12 08:33:53 +00:00
doc.go satellite/gc: move GC to segments loop 2021-06-01 20:36:02 +00:00
gc_test.go satellite/gc: improve test for copies 2022-04-11 11:06:01 +00:00
piecetracker.go satellite: fix segment loop observers metrics 2022-08-10 14:13:16 +00:00
service.go satellite/gc: remove skip first 2021-06-21 18:30:38 +00:00