0649d2b930
We migrated satelliteDB off of Postgres and over to CockroachDB (crdb), but there was way too high contention for the injuredsegments table so we had to rollback to Postgres for the repair queue. A couple things contributed to this problem: 1) crdb doesn't support `FOR UPDATE SKIP LOCKED` 2) the original crdb Select query was doing 2 full table scans and not using any indexes 3) the SLC Satellite (where we were doing the migration) was running 48 repair worker processes, each of which run up to 5 goroutines which all are trying to select out of the repair queue and this was causing a ton of contention. The changes in this PR should help to reduce that contention and improve performance on CRDB. The changes include: 1) Use an update/set query instead of select/update to capitalize on the new `UPDATE` implicit row locking ability in CRDB. - Details: As of CRDB v20.2.2, there is implicit row locking with update/set queries (contention reduction and performance gains are described in this blog post: https://www.cockroachlabs.com/blog/when-and-why-to-use-select-for-update-in-cockroachdb/). 2) Remove the `ORDER BY` clause since this was causing a full table scan and also prevented the use of the row locking capability. - While long term it is very important to `ORDER BY segment_health`, the change here is only suppose to be a temporary bandaid to get us migrated over to CRDB quickly. Since segment_health has been set to infinity for some time now (re: https://review.dev.storj.io/c/storj/storj/+/3224), it seems like it might be ok to continue not making use of this for the short term. However, long term this needs to be fixed with a redesign of the repair workers, possible in the trusted delegated repair design (https://review.dev.storj.io/c/storj/storj/+/2602) or something similar to what is recommended here on how to implement a queue on CRDB https://dev.to/ajwerner/quick-and-easy-exactly-once-distributed-work-queues-using-serializable-transactions-jdp, or migrate to rabbit MQ priority queue or something similar.. This PRs improved query uses the index to avoid full scans and also locks the row its going to update and CRDB retries for us if there are any lock errors. Change-Id: Id29faad2186627872fbeb0f31536c4f55f860f23 |
||
|---|---|---|
| .github | ||
| certificate | ||
| cmd | ||
| docs | ||
| installer/windows | ||
| multinode | ||
| pkg | ||
| private | ||
| resources | ||
| satellite | ||
| scripts | ||
| storage | ||
| storagenode | ||
| versioncontrol | ||
| web | ||
| .clabot | ||
| .dockerignore | ||
| .gitignore | ||
| CODE_OF_CONDUCT.md | ||
| docker-compose.yaml | ||
| go.mod | ||
| go.sum | ||
| Jenkinsfile | ||
| Jenkinsfile.public | ||
| LICENSE | ||
| Makefile | ||
| monkit.lock | ||
| package-lock.json | ||
| README.md | ||
Storj V3 Network
Storj is building a decentralized cloud storage network. Check out our white paper for more info!
Storj is an S3-compatible platform and suite of decentralized applications that allows you to store data in a secure and decentralized manner. Your files are encrypted, broken into little pieces and stored in a global decentralized network of computers. Luckily, we also support allowing you (and only you) to retrieve those files!
Table of Contents
Contributing to Storj
All of our code for Storj v3 is open source. Have a code change you think would make Storj better? Please send a pull request along! Make sure to sign our Contributor License Agreement (CLA) first. See our license section for more details.
Have comments or bug reports? Want to propose a PR before hand-crafting it? Jump on to our forum and join the Engineering Discussions to say hi to the developer community and to talk to the Storj core team.
Want to vote on or suggest new features? Post it on ideas.storj.io.
Issue tracking and roadmap
See the breakdown of what we're building by checking out the following resources:
Install required packages
To get started running Storj locally, download and install the latest release of Go (at least Go 1.13) at golang.org.
You will also need Git. (brew install git, apt-get install git, etc).
If you're building on Windows, you also need to install and have gcc setup correctly.
We support Linux, Mac, and Windows operating systems. Other operating systems supported by Go should also be able to run Storj.
Download and compile Storj
Aside about GOPATH: Go 1.11 supports a new feature called Go modules, and Storj has adopted Go module support. If you've used previous Go versions, Go modules no longer require a GOPATH environment variable. Go by default falls back to the old behavior if you check out code inside of the directory referenced by your GOPATH variable, so make sure to use another directory,
unset GOPATHentirely, or setGO111MODULE=onbefore continuing with these instructions.
First, fork our repo and clone your copy of our repository.
git clone git@github.com:<your-username>/storj storj
cd storj
Then, let's install Storj.
go install -v ./cmd/...
Make changes and test
Make the changes you want to see! Once you're done, you can run all of the unit tests:
go test -v ./...
You can also execute only a single test package if you like. For example:
go test ./pkg/identity. Add -v for more informations about the executed unit
tests.
Push up a pull request
Use Git to push your changes to your fork:
git commit -a -m 'my changes!'
git push origin master
Use Github to open a pull request!
A Note about Versioning
While we are practicing semantic versioning for our client libraries such as uplink, we are not practicing semantic versioning in this repo, as we do not intend for it to be used via Go modules. We may have backwards-incompatible changes between minor and patch releases in this repo.
Start using Storj
Our wiki has documentation and tutorials. Check out these three tutorials:
License
The network under construction (this repo) is currently licensed with the AGPLv3 license. Once the network reaches beta phase, we will be licensing all client-side code via the Apache v2 license.
For code released under the AGPLv3, we request that contributors sign our Contributor License Agreement (CLA) so that we can relicense the code under Apache v2, or other licenses in the future.
Support
If you have any questions or suggestions please reach out to us on our community forum or email us at support@tardigrade.io.