JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
601cadb92c
storj/scripts
History
Michal Niewrzal 158eb2381e satellite/gc/bloomfilter: uploading bloom filters 
We would like to have separate process/command to collect bloom
filters from source different than production DBs. Such process will
use segment loop to build bloom filters for all storage nodes and
will send it to Storj bucket.

This change add main logic to new service. After collecting all bloom
filters with segment loop and piece tracker all filters are marshaled
and packed into zip files. Each zip contains up to "ZipBatchSize" bloom
filters and it's uploaded to specified in configuration bucket.

All uploaded objects have specified expiration time to not delete them
manually.

Updates https://github.com/storj/team-metainfo/issues/120

Change-Id: I2b6bc02a7dd7c3a639e75810fd013ae4afdc80a2
2022-09-12 08:33:53 +00:00
..
testdata satellite/gc/bloomfilter: uploading bloom filters 2022-09-12 08:33:53 +00:00
tests test-sim-versions: write what bash step is executing 2022-06-10 19:33:03 +00:00
waitlist Waitlist script (#573) 2018-11-06 18:43:20 +01:00
build-wasm.sh satellite/admin/ui: avoid needing an additional copy 2022-03-25 11:42:22 +02:00
cert-gating.sh Delete Bootstrap and Kademlia (#2974) 2019-10-04 16:48:41 -04:00
check-package-lock.sh web/: add check for change to eslint import 2021-12-21 15:59:23 +00:00
deploy-storagenode.sh scripts: Change Storagenode arm32v6 to arm32v5 2022-06-21 11:54:38 +00:00
deploy-watchtower.sh Script for deploying the Docker manifest for watchtower (#3015) 2019-09-12 17:38:48 +03:00
draft-release.sh scripts/draft-release: add sleep to avoid race condition on uploading the binaries (#4647) 2022-03-21 15:24:20 +01:00
gerrit-cla.py gerrit: add cla 2019-06-19 18:28:53 +00:00
gerrit-status.sh build: premerge/verify Jenkinsfile (subsets of Jenkinsifle.public) 2022-06-23 09:15:43 +02:00
install-awscli.sh Travis with go1.11 and modules (#479) 2018-10-16 20:02:00 +03:00
lint.go scripts: Fix lint to show out on error 2022-05-27 09:32:48 +00:00
pg-to-crdb.go all: fix linting issues 2022-03-21 15:26:42 +00:00
postgres-dev.sh satellite/testing: Change testing to use PG 12.3 (#3913) 2020-06-25 20:17:39 +03:00
redis-server.sh scripts: Do not use Docker for test-sim Redis test 2021-03-22 08:43:53 +00:00
release.sh scripts/release.sh: making clear if builing a production or development version (#4304) 2021-12-09 12:38:23 +01:00
tag-release.sh private/version: minimal fix for tag-release.sh 2020-04-01 13:46:45 +00:00
test-backwards.sh cmd/uplinkng: become cmd/uplink 2022-02-09 17:02:21 +00:00
test-billing.sh tests: fix calculation of previous month for billing integration test 2021-03-29 10:38:59 +02:00
test-certificates.sh fix certificates auth export command (#3110) 2019-09-24 10:38:18 -06:00
test-network-stalls.go all: fix linting issues 2022-03-21 15:26:42 +00:00
test-sim-backwards-dev.sh metainfo-migration: basic pointerdb->metabase migrator 2021-01-12 12:59:53 +00:00
test-sim-backwards.sh ci: switch back to pulling gateway@main 2021-09-28 15:01:04 +00:00
test-sim-dev.sh scripts: use postgres script with all tests (#3404) 2019-10-31 07:03:54 -07:00
test-sim-redis-up-and-down-dev.sh scripts: Do not use Docker for test-sim Redis test 2021-03-22 08:43:53 +00:00
test-sim-redis-up-and-down.sh scripts: Do not use Docker for test-sim Redis test 2021-03-22 08:43:53 +00:00
test-sim.sh cmd/uplinkng: become cmd/uplink 2022-02-09 17:02:21 +00:00
test-uplink-redis-up-and-down.sh cmd/uplinkng: become cmd/uplink 2022-02-09 17:02:21 +00:00
test-uplink-rs-download.sh cmd/uplinkng: become cmd/uplink 2022-02-09 17:02:21 +00:00
test-uplink-rs-upload.sh cmd/uplinkng: become cmd/uplink 2022-02-09 17:02:21 +00:00
test-uplink-share.sh cmd/uplinkng: become cmd/uplink 2022-02-09 17:02:21 +00:00
test-uplink.sh scripts: add 'uplink cp' tests for stdin/stdout 2022-05-18 08:42:04 +00:00
test-wasm-size.sh satellite/console/wasm: add test to confirm wasm size isnt growing 2021-01-21 15:48:49 +00:00
test-wasm.sh build: fix conditional run of web builds 2022-08-11 14:34:44 +00:00
update-access.go all: fix linting issues 2022-03-21 15:26:42 +00:00
utils.sh cmd/uplink: add ranged download to uplink cli 2021-10-25 13:13:33 +00:00
validate-jenkinsfile.sh build: premerge/verify Jenkinsfile (subsets of Jenkinsifle.public) 2022-06-23 09:15:43 +02:00