ed7c82439d
Calling stat() (really, lstat()) on every file during a directory walk is the step that takes up the most time. Furthermore, not all directory walk uses _need_ to have a stat done on every file. Therefore, in this commit we avoid doing the stat at the lowest level of walkNamespaceInPath. The stat will still be done when it is requested, with the Stat() method on the blobInfo object. The major upside of this is that we can avoid the stat call on most files during a Retain operation. This should speed up garbage collection considerably. The major downside is that walkNamespaceInPath will no longer automatically skip over directories that are named like blob files, or blob files which are deleted between readdir() and stat(). Callers to walkNamespaceInPath and its variants (WalkNamespace, WalkSatellitePieces, etc) are now expected to handle these cases individually. Thanks to forum member Toyoo for the insight that this would speed up garbage collection. Refs: https://github.com/storj/storj/issues/5454 Change-Id: I72930573d58928fa25057ed89cd4ec474b884199 |
||
|---|---|---|
| .github | ||
| certificate | ||
| cmd | ||
| crashcollect | ||
| docs | ||
| installer/windows | ||
| multinode | ||
| private | ||
| resources | ||
| satellite | ||
| scripts | ||
| storage | ||
| storagenode | ||
| testsuite | ||
| versioncontrol | ||
| web | ||
| .dockerignore | ||
| .earthlyignore | ||
| .gitattributes | ||
| .gitignore | ||
| .gitreview | ||
| CODE_OF_CONDUCT.md | ||
| CODEOWNERS | ||
| CONTRIBUTING.md | ||
| DEVELOPING.md | ||
| docker-compose.tests.yaml | ||
| Earthfile | ||
| go.mod | ||
| go.sum | ||
| Jenkinsfile | ||
| Jenkinsfile.premerge | ||
| Jenkinsfile.public | ||
| Jenkinsfile.verify | ||
| LICENSE | ||
| MAINTAINERS.md | ||
| Makefile | ||
| monkit.lock | ||
| proto.lock | ||
| README.md | ||
Storj V3 Network
Storj is building a decentralized cloud storage network. Check out our white paper for more info!
Storj is an S3-compatible platform and suite of decentralized applications that allows you to store data in a secure and decentralized manner. Your files are encrypted, broken into little pieces and stored in a global decentralized network of computers. Luckily, we also support allowing you (and only you) to retrieve those files!
Table of Contents
Contributing to Storj
All of our code for Storj v3 is open source. If anything feels off, or if you feel that some functionality is missing, please check out the contributing page. There you will find instructions for sharing your feedback, building the tool locally, and submitting pull requests to the project.
A Note about Versioning
While we are practicing semantic versioning for our client libraries such as uplink, we are not practicing semantic versioning in this repo, as we do not intend for it to be used via Go modules. We may have backwards-incompatible changes between minor and patch releases in this repo.
Start using Storj
Our wiki has documentation and tutorials. Check out these three tutorials:
License
This repository is currently licensed with the AGPLv3 license.
For code released under the AGPLv3, we request that contributors sign our Contributor License Agreement (CLA) so that we can relicense the code under Apache v2, or other licenses in the future.
Support
If you have any questions or suggestions please reach out to us on our community forum or file a ticket at https://support.storj.io/.