JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
158eb2381e
storj/satellite
History
Michal Niewrzal 158eb2381e satellite/gc/bloomfilter: uploading bloom filters 
We would like to have separate process/command to collect bloom
filters from source different than production DBs. Such process will
use segment loop to build bloom filters for all storage nodes and
will send it to Storj bucket.

This change add main logic to new service. After collecting all bloom
filters with segment loop and piece tracker all filters are marshaled
and packed into zip files. Each zip contains up to "ZipBatchSize" bloom
filters and it's uploaded to specified in configuration bucket.

All uploaded objects have specified expiration time to not delete them
manually.

Updates https://github.com/storj/team-metainfo/issues/120

Change-Id: I2b6bc02a7dd7c3a639e75810fd013ae4afdc80a2
2022-09-12 08:33:53 +00:00
..
accounting satellite: fix segment loop observers metrics 2022-08-10 14:13:16 +00:00
admin satellite/admin: add Burst field to Update Project Limits params 2022-08-17 10:40:28 -04:00
analytics satellite/analytics: Re-added user click events in Access Management flow (#5151) 2022-09-06 09:55:33 -07:00
attribution {cmd/satellite/reports, satellite/attribution}: type and variable name adjustments 2022-04-26 20:12:38 +00:00
audit satellite/reputation: reconfigure lambda and alpha 2022-08-17 18:52:53 +00:00
buckets satellite/metainfo: propagate geofencing between buckets and stream id 2021-11-24 08:05:05 +00:00
compensation all: reformat comments as required by gofmt 1.19 2022-08-10 18:24:55 +00:00
console satellite/console,web/satellite: limit new password length to 128 2022-09-08 13:29:47 +00:00
contact satellite/contact: swap net.IP.IsPrivateIP with isPrivateIP 2022-06-13 01:01:44 +02:00
gc satellite/gc/bloomfilter: uploading bloom filters 2022-09-12 08:33:53 +00:00
geoip satellite/geoip: update node check-in to associate a country code 2021-11-10 16:44:41 +01:00
gracefulexit satellite: fix segment loop observers metrics 2022-08-10 14:13:16 +00:00
inspector {satellite/metabase, satellite/metainfo, satellite/inspector} : Use metabase.GetObjectLastCommitted instead metabase.GetObjectExactVersion 2022-09-08 07:27:22 +00:00
internalpb satellite/internalpb: new protobuf for retain filters 2022-08-30 12:37:40 +02:00
mailservice satellite/consoleweb: fix flaky TestAuth tests 2022-08-04 19:06:07 +00:00
metabase satellite/metabase: BeginObjectNextVersion returns object 2022-09-12 07:54:14 +00:00
metainfo satellite/metabase: BeginObjectNextVersion returns object 2022-09-12 07:54:14 +00:00
metrics satellite: fix segment loop observers metrics 2022-08-10 14:13:16 +00:00
nodeapiversion satellite/nodeapiversion: new table for tracking node api usage 2020-07-09 15:02:25 +00:00
nodeselection/uploadselection satellite/repairer: handle excluded countries 2022-03-14 10:59:36 -04:00
nodestats satellite: return interval_end_time in DailyStorageUsage endpoint 2022-07-27 18:24:27 +00:00
oidc satellite/console,web/satellite: invalidate sessions after inactivity 2022-08-23 15:51:05 +00:00
orders satellite/{orders,overlay}: use cache for downloads 2022-07-12 11:04:34 +00:00
overlay satellite/overlay: don't insert new nodes if contact check failed 2022-08-26 09:05:04 +00:00
payments satellite/payments/monetary: remove pkg and all its references 2022-09-07 12:58:00 +00:00
repair storagenode/satellite: support different piece hash algorithms 2022-08-23 18:15:06 +00:00
reputation satellite/reputation: reconfigure lambda and alpha 2022-08-17 18:52:53 +00:00
revocation satellite/satellitedb: move tests to their domains 2021-02-19 17:29:15 +02:00
rewards satellite/rewards: adding SeaweedFS to partners list (#4230) 2021-10-19 21:30:31 +02:00
satellitedb satellite/payments/monetary: remove pkg and all its references 2022-09-07 12:58:00 +00:00
snopayouts all: fix error naming 2021-04-29 15:38:21 +03:00
admin.go satellite/payments/stripecoinpayments: storjscan invoice generation 2022-08-03 13:24:26 -07:00
api.go satellitedb: add token balance to API endpoint 2022-08-31 22:55:16 +00:00
configlock_test.go all: fix linter complaints 2020-10-13 15:59:01 +03:00
core.go satellite/payments/storjscan: add billing update to chore 2022-08-16 11:18:15 -07:00
gc-bf.go satellite/gc/bloomfilter: add service to collect bloom filters 2022-09-02 08:10:46 +00:00
gc.go satellite: more detailed goroutine labels 2022-05-11 17:50:55 +00:00
peer.go satellite/gc/bloomfilter: add service to collect bloom filters 2022-09-02 08:10:46 +00:00
repairer.go satellite/reputation: offset write times by random, not by satelliteID 2022-08-03 21:14:06 +00:00