Go to file
Natalie Villasana ff65663867
adds Bucket Store (#213)
* adds foundation for bucketStore

* adds prefixedObjStore to buckets package, adjusts gateway-storj accordingly

* fixes multi value assignment problems in gateway-storj

* fixes more multi value assignment errors in gateway-storj

* starts changing miniogw tests to accommodate buckets

* creates bucket store mock

* wip - fixing test cases in object tests

* adds get, put, and list object tests, comments out two test cases

* adds happy scenario tests for bucket methods

* fixes bug in list, removes redundant parts from gateway tests

* fixes nit

* Clean up tests from #188

* Fix bug with timestamp conversion in segment store

* fixes segments.Meta test

* Fix regression in listing objects in a bucket

* adds check to see if bucket is empty before deleting

* updates DeleteBucket test to account for empty/full bucket

* adds TODOs for DeleteBucket and MakeBucket for some cases, adjusts tests, filters out minio errors in logging.go

* adds checks for if buckets already exist or not in DeleteBucket and MakeBucket functions; adjusts tests

* adds BucketNotFound error check in bucket store, removes todo

* adds make_bucket to Travis test, updates boltdb client constructor to always create a bucket (table)
2018-08-16 10:32:28 -04:00
cmd Fix the sha3 error with the build (#230) 2018-08-15 14:49:30 -04:00
docs Add error handling section 2018-04-11 07:43:23 -05:00
examples Consolidate command configuration and setup (#221) 2018-08-13 09:07:05 -06:00
internal Integrating on disk routing table with kademlia (#166) 2018-08-09 15:20:39 -04:00
logo wip on structure 2018-04-06 12:32:34 -04:00
pkg adds Bucket Store (#213) 2018-08-16 10:32:28 -04:00
pointerdb/auth Refactor List in PointerDB (#163) 2018-07-27 09:02:59 +03:00
protos adds stripe_size field to pointerdb proto file (#218) 2018-08-14 11:15:22 -04:00
scripts adds Bucket Store (#213) 2018-08-16 10:32:28 -04:00
static Cache (#67) 2018-06-05 17:06:37 -04:00
storage adds Bucket Store (#213) 2018-08-16 10:32:28 -04:00
test Add files for testing builds in docker (#161) 2018-07-25 17:47:02 -04:00
.clabot add phutchins to clabot (#220) 2018-08-10 18:11:44 -04:00
.gitignore Node Identity (#193) 2018-08-13 10:39:45 +02:00
.travis.yml Test captplanet with travis (#226) 2018-08-14 12:58:16 -04:00
CODE_OF_CONDUCT.md styling changes to the code of conduct (#234) 2018-08-16 10:07:30 -04:00
docker-compose.yaml Add files for testing builds in docker (#161) 2018-07-25 17:47:02 -04:00
go.mod pkg/kademlia tests and restructuring (#97) 2018-06-22 09:33:57 -04:00
Gopkg.lock Fix the sha3 error with the build (#230) 2018-08-15 14:49:30 -04:00
Gopkg.toml Fix the sha3 error with the build (#230) 2018-08-15 14:49:30 -04:00
index.html Cache (#67) 2018-06-05 17:06:37 -04:00
Jenkinsfile Adding auto deploy to staging in kube (#219) 2018-08-14 10:00:39 -05:00
LICENSE license code with agplv3 (#126) 2018-07-05 10:24:26 -04:00
Makefile Fix the sha3 error with the build (#230) 2018-08-15 14:49:30 -04:00
README.md spelling fix for read me (#217) 2018-08-10 09:21:43 -07:00

Storj

Go Report Card Go Doc Coverage Status

Storj is in the midst of a rearchitecture. Please stay tuned for our v3 whitepaper!


Storj is a platform, token, and suite of decentralized applications that allows you to store data in a secure and decentralized manner. Your files are encrypted, shredded into little pieces and stored in a global decentralized network of computers. Luckily, we also support allowing you (and only you) to recover them!

Start Using Storj

Download the latest release

Go here to download the latest build // TODO: add link when a build is released // TODO for how to run the release

Configure AWS CLI

In a new terminal session:

Download and install the AWS CLI: https://docs.aws.amazon.com/cli/latest/userguide/installing.html

Configure the AWS CLI:

$ aws configure
AWS Access Key ID [None]: insecure-dev-access-key
AWS Secret Access Key [None]: insecure-dev-secret-key
Default region name [None]: us-east-1
Default output format [None]: 
$ aws configure set default.s3.multipart_threshold 1TB  # until we support multipart

Test some commands:

Upload an Object

$ aws s3 --endpoint=http://localhost:7777/ cp ~/Desktop/your-large-file.mp4 s3://bucket/your-large-file.mp4

Download an Object

$ aws s3 --endpoint=http://localhost:7777/ cp s3://bucket/your-large-file.mp4 ~/Desktop/your-large-file.mp4

List Objects

aws s3 --endpoint=http://localhost:7777/ ls s3://bucket/ --recursive

Delete Objects in a Bucket

aws s3 --endpoint=http://localhost:7777/ rm --recursive  s3://bucket/

Generate a URL for an Object

aws s3 --endpoint=http://localhost:7777/ presign s3://bucket/your-large-file.mp4

For more information about the AWS s3 CLI visit: https://docs.aws.amazon.com/cli/latest/reference/s3/index.html

Start Contributing to Storj

Install required packages

First, install git and golang. We currently support Debian-based and Mac operating systems

Debian based (like Ubuntu)

Download and install the latest release of go https://golang.org/

apt-get install git golang
echo 'export STORJDEV="$HOME/storj"' >> $HOME/.bashrc
echo 'export GOPATH="$STORJDEV:$STORJDEV/vendor"' >> $HOME/.bashrc
echo 'export PATH="$PATH:$STORJDEV/bin"' >> $HOME/.bashrc
source $HOME/.bashrc

Mac OSX

brew install git go
if test -e $HOME/.bash_profile
then
	echo 'export STORJDEV="$HOME/storj"' >> $HOME/.bash_profile
	echo 'export GOPATH="$STORJDEV:$STORJDEV/vendor"' >> $HOME/.bash_profile
	echo 'export PATH="$PATH:$STORJDEV/bin"' >> $HOME/.bash_profile
	source $HOME/.bash_profile
else
	echo 'export STORJDEV="$HOME/storj"' >> $HOME/.profile
	echo 'export GOPATH="$STORJDEV:$STORJDEV/vendor"' >> $HOME/.profile
	echo 'export PATH="$PATH:$STORJDEV/bin"' >> $HOME/.profile
	source $HOME/.profile
fi

Install storj

Clone the storj repository. You may want to clone your own fork and branch.

mkdir -p $STORJDEV/src/storj.io
git clone https://github.com/storj/storj $STORJDEV/src/storj.io/storj

Install all dependencies

git clone --recursive https://github.com/storj/storj-vendor $STORJDEV/vendor
rm -rf $STORJDEV/vendor/src/github.com/minio/minio/vendor/github.com/minio/cli
rm -rf $STORJDEV/vendor/src/github.com/minio/minio/vendor/golang.org/x/net/trace

Start the network

$ go install -v storj.io/storj/cmd/captplanet
$ captplanet setup
$ captplanet run

Try out some commands via Storj CLI or AWS CLI

Run unit tests

go test storj.io/storj/...

You can execute only a single test package. For example: go test storj.io/storj/pkg/kademlia. Add -v for more informations about the executed unit tests.

Support

If you have any questions or suggestions please reach out to us on Rocketchat or Twitter.