Go to file
Isaac Hess 7d1e28ea30 storagenode: Include trash space when calculating space used
This commit adds functionality to include the space used in the trash
directory when calculating available space on the node.

It also includes this trash value in the space used cache, with methods
to keep the cache up-to-date as files are trashed, restored, and
emptied.

As part of the commit, the RestoreTrash and EmptyTrash methods have
slightly changed signatures. RestoreTrash now also returns the keys that
were restored, while EmptyTrash also returns the total disk space
recovered. Each of these changes makes it possible to keep the cache
up-to-date and know how much space is being used/recovered.

Also changed is the signature of PieceStoreAccess.ContentSize method.
Previously this method returns only the content size of the blob,
removing the size of any header data. This method has been renamed
`Size` and returns both the full disk size and content size of the blob.
This allows us to only stat the file once, and in some instances (i.e.
cache) knowing the full file size is useful.

Note: This commit simply adds the trash size data to the piece size data
we were already collecting. The piece size data is not accurate for all
use-cases (e.g. because it does not contain piece header data); however,
this commit does not fix that problem. Now that the ContentSize (Size)
method returns the full size of the file, it should be easier to fix
this problem in a future commit.

Change-Id: I4a6cae09e262c8452a618116d1dc66b687f59f85
2019-12-23 19:07:03 -07:00
.github pr template: migrations run concurrently with api servers now 2019-10-31 09:27:46 -06:00
certificate pkg/rpc: replace methods with direct calls to pb 2019-12-22 17:12:43 +02:00
cmd pkg/rpc: replace methods with direct calls to pb 2019-12-22 17:12:43 +02:00
docs storagenode/trust: wire up list into pool 2019-12-13 20:32:50 +00:00
examples uplink/eestream: move Pad to encryption package to break dependency to eestream 2019-12-20 19:32:12 +00:00
installer/windows cmd/storagenode-updater, pkg/process: Fix logging timestamp 2019-12-11 10:05:48 +00:00
lib lib/uplinkc: add user agent 2019-12-20 13:11:25 +00:00
linksharing private: rename internal to private (#3573) 2019-11-14 21:46:15 +02:00
pkg pkr/rpc: remove RawConn 2019-12-22 18:01:01 +02:00
private storagenode: Include trash space when calculating space used 2019-12-23 19:07:03 -07:00
resources Add LE CA to satellite and storagenode images (#1688) 2019-04-08 13:06:01 -04:00
satellite pkg/rpc: replace methods with direct calls to pb 2019-12-22 17:12:43 +02:00
scripts all: remove code to default to grpc 2019-12-20 20:12:04 +02:00
storage storagenode: Include trash space when calculating space used 2019-12-23 19:07:03 -07:00
storagenode storagenode: Include trash space when calculating space used 2019-12-23 19:07:03 -07:00
uplink pkg/rpc: replace methods with direct calls to pb 2019-12-22 17:12:43 +02:00
versioncontrol private: rename internal to private (#3573) 2019-11-14 21:46:15 +02:00
web web/storagenode: notification markup 2019-12-19 17:38:26 +00:00
.clabot adding Igor Gaidaenko our new team member! (#3726) 2019-12-11 13:54:12 -05:00
.dockerignore Forward-port release-alpha8 build script issues (#1726) 2019-04-09 23:01:10 -06:00
.gitignore mobile: build libuplink aar (#3165) 2019-10-04 01:56:42 -07:00
.golangci.yml jenkins: update to golangci-lint 1.21.0 (#3289) 2019-10-17 10:42:22 +03:00
CODE_OF_CONDUCT.md Adding CODE_OF_CONDUCT to storj/storj repo (#779) 2018-12-07 15:10:02 -05:00
docker-compose.yaml Use Postgres 9.6 for Jenkins builds (#3729) 2019-12-13 10:03:06 -05:00
go.mod ci: use external repository 2019-12-19 12:05:49 +00:00
go.sum ci: use external repository 2019-12-19 12:05:49 +00:00
Jenkinsfile scripts: add test-versions stage to private Jenkins 2019-12-20 15:52:54 +00:00
Jenkinsfile.public satellite/satellitedb: improve Cockroach migrate test 2019-12-21 10:58:43 +00:00
LICENSE license code with agplv3 (#126) 2018-07-05 10:24:26 -04:00
Makefile all: remove code to default to grpc 2019-12-20 20:12:04 +02:00
monkit.lock satellite/metainfo, satellite/repair, uplink/eestream: add metric for download failed due to not enough pieces available (#3665) 2019-12-04 16:24:36 -05:00
package-lock.json Satellite api keys frontend (#1039) 2019-02-01 18:19:30 +02:00
proto.lock storagenode: Add new endpoint to receive satellite requests for… (#3590) 2019-11-26 18:47:19 +01:00
README.md add sourcerer hall-of-fame to README 2019-11-19 16:08:16 -07:00

Storj V3 Network

Go Report Card Go Doc Coverage Status Alpha

Storj is building a decentralized cloud storage network. Check out our white paper for more info!


Storj is an S3-compatible platform and suite of decentralized applications that allows you to store data in a secure and decentralized manner. Your files are encrypted, broken into little pieces and stored in a global decentralized network of computers. Luckily, we also support allowing you (and only you) to retrieve those files!

Table of Contents

Contributing to Storj

All of our code for Storj v3 is open source. Have a code change you think would make Storj better? Please send a pull request along! Make sure to sign our Contributor License Agreement (CLA) first. See our license section for more details.

Have comments or bug reports? Want to propose a PR before hand-crafting it? Jump on to our forum and join the Engineering Discussions to say hi to the developer community and to talk to the Storj core team.

Want to vote on or suggest new features? Post it on ideas.storj.io.

Issue tracking and roadmap

See the breakdown of what we're building by checking out the following resources:

Install required packages

To get started running Storj locally, download and install the latest release of Go (at least Go 1.13) at golang.org.

You will also need Git. (brew install git, apt-get install git, etc). If you're building on Windows, you also need to install and have gcc setup correctly.

We support Linux, Mac, and Windows operating systems. Other operating systems supported by Go should also be able to run Storj.

Download and compile Storj

Aside about GOPATH: Go 1.11 supports a new feature called Go modules, and Storj has adopted Go module support. If you've used previous Go versions, Go modules no longer require a GOPATH environment variable. Go by default falls back to the old behavior if you check out code inside of the directory referenced by your GOPATH variable, so make sure to use another directory, unset GOPATH entirely, or set GO111MODULE=on before continuing with these instructions.

First, fork our repo and clone your copy of our repository.

git clone git@github.com:<your-username>/storj storj
cd storj

Then, let's install Storj.

go install -v ./cmd/...

Make changes and test

Make the changes you want to see! Once you're done, you can run all of the unit tests:

go test -v ./...

You can also execute only a single test package if you like. For example: go test ./pkg/identity. Add -v for more informations about the executed unit tests.

Push up a pull request

Use Git to push your changes to your fork:

git commit -a -m 'my changes!'
git push origin master

Use Github to open a pull request!

Start using Storj

Our wiki has documentation and tutorials. Check out these three tutorials:

License

The network under construction (this repo) is currently licensed with the AGPLv3 license. Once the network reaches beta phase, we will be licensing all client-side code via the Apache v2 license.

For code released under the AGPLv3, we request that contributors sign our Contributor License Agreement (CLA) so that we can relicense the code under Apache v2, or other licenses in the future.

Support

If you have any questions or suggestions please reach out to us on our community forum or email us at support@tardigrade.io.