* Updates to config handling
- Add functions to load in configs
- Standardize location and naming of config files
- Configuration over convention style of config file handling for each
service
* update config handling and correclty handle cli flags being set
* generate configs from default if no config is found
- renamed pointerdbDB to pointerdb for clarity in config file
- set sane default for pkg/overlay boltDB file
- set srvPort to default to 8082 to avoid port collision on default
setting
* linter updates
* move boltdb path vars into function
* update tests to handle config environment changes
* --fix exec test mocks
* update tests to use viper instead of flag library
* fix typo
* add redis-server to services in travis for tests
* update examples with new config env function signature
* fix tests
* WIP ObjectStore
* Remove methods for extended attributes
* List returns metadata too
* No real need to prepend "object" in path
* Serialize metadata
* List retuns []ListItem instead of []Path
* lays out SegmentStore functions to implement
* Merge branch 'master' into segment-store
* adds overlay calls to put
* allows SegmentStore Put to upload a file to ecclient, then save pointer to pointerdb
* Merge branch 'master' into segment-store
* removes new overlay client instance in Put
* fixes syntax
* fixes syntax again
* fixes imports
* fixes typo
* removes pointerdb client from segmentStore struct for now
* changes SegmentStore to segmentStore
* changing types in parameters to fit other function calls
* takes RedundancyStrategy out of Put params
* changes NewClient param back to take an interface (not pointer to interface)
* fixes types
* moves pointer into PutRequest in SegmentStore Put
* passes interfact, not pointer to interface to NewSegmentStore
* fixes some types
* Get returns an instance of Meta
* fixes PutRequest fields
* adds remotePieces slice to pointerdb PutRequest
* ecClient Put now takes *proto.Nodes instead of proto.Nodes
* fixes syntax
* changes ec client dial interface to use *proto.Node
* changes other instances of proto.Node to *proto.Node in ecclient pkg
* adds *proto.Node to Get and Delete functions in ecclient pkg
* changes proto.Node to pointer in ec client_test
* changes proto.Node to pointer in ec client_test
* adds ecclient and pointerdb client to the segmentstore constructor
* adds ecclient and pointerDBClient to segmentStore constructor
* port changes
* Task monitor and setup merge from the staging
* Restructure + additional interface
* Add NewOverlayClient
* integrated DHT client interface
* added test for interface
* PR comments addressed
* lint issue
* added generated protobuf
* adding new interface
* added the interface framework
* deleted file
* fixes compilation errors and integrates new dhtcclient interface
* merged netstat latest changes and dht new interface chagnes
* fixed the address's port
* adding comments
* PR comments addressed
* netclient interface dial method added
* rename and integrated transportclient with minio gateway
* rename and code clean up
* made changes based on the Dennis's changes on the kad-client
* Code review comment changes based on kaloyan review comments
* reverted the changes to be similar to master
* removed unused file
* renamed to transportclient
* added the review changes
* store the address of the client
* updates per the code review comments, changes-> added error retry connection attempt logic, added error conditions including nil parameters
* updated the test case to test the bad address passed condition
* updated the code per code review comments
* Bolt backed overlay cache (#94)
* wip
* add separate `Process` tests for bolt and redis-backed overlay
* more testing
* fix gitignore
* fix linter error
* goimports goimports GOIMPORTS GoImPortS!!!!
* fix port madness
* forgot to add
* add `mux` as handler and shorten context timeouts
* gofreakingimports
* fix comments
* refactor test & add logger/monkit registry
* debugging travis
* add comment
* Set redisAddress to empty string for bolt-test
* travis experiment
* refactoring tests
* Merge remote-tracking branch 'upstream/master' into bolt-backed-overlay-cache
* Automatically build, tag and push docker images on merge to master (#103)
* port changes
* build overlay on successful merge to master
* fixes to Makefile
* permissions
* dep ensure
* gopath
* let's try vgo
* remove dep
* maybe alpine is the issue
* tagging go version on build
* stupid vgo
* vgo
* adding tags to push
* quotes
* local linting fixes & stupid travis
* prepend storjlabs to docker tag (#108)
* port changes
* fixing tag name
* Use continue instead of return in table tests (#106)
I did a dumb mistake for some of the table tests, which made some of the
test cases not being executed.
* pkg/kademlia tests and restructuring (#97)
* port changes
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* files created
* Merge remote-tracking branch 'upstream/master' into coyle/kad-tests
* wip
* Merge remote-tracking branch 'upstream/master' into coyle/kad-tests
* wip
* remove bkad dependencie from tests
* wip
* wip
* wip
* wip
* wip
* updated coyle/kademlia
* wip
* cleanup
* ports
* overlay upgraded
* linter fixes
* piecestore kademlia newID
* add changes from kad demo
* PR comments addresses
* go func
* force travis build
* fixed merge conflicts
* fixed merge conflicts
* Merge branch 'coyle/kad-tests' of https://github.com/coyle/storj into coyle/kad-tests
* linter issues
* linting issues
* fixed merge conflicts
* linter is stupid
* Coyle/docker fix (#109)
* port changes
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of https://github.com/storj/storj
* fixing tag name
* no idea
* testing
* changes
* testing on travis
* testing
* changes to travis build
* new approach
* Merge branch 'master' into coyle/docker-fix
* hardcode version (#111)
* hardcode version
* adding coveralls / code coverage (#112)
* adding coveralls
* adding code coverage badge
* fixing badges
* verbose
* swap tests and coverage
* extra line
* maybe
* maybe
* moar
* gover maybe
* testing
* cleanup
* protos/netstate: remove stuff we're not using (#100)
* protos/netstate: remove stuff we're not using
* protos/netstate: add metadata field for segmentstore
* fix netstate client test
* pkg/process: start replacing pkg/process with cobra helpers (#98)
* Implement psclient interface (#107)
* Implement psclient interface
* Add string method to pieceID type
* try to fix linter errors
* Whoops missed an error
* More linter errors
* Typo
* Lol double typo
* Get everything working, begin adding tests for psclient rpc
* goimports
* Forgot to change the piecestore cli when changed the piecestore code
* Fix CLI
* remove ID length, added validator to pieceID
* Move grpc ranger to client
Change client PUT api to take a reader rather than return a writer
* GRPCRanger -> PieceRanger; Make PieceRanger a RangeCloser
* Forgot to remove offset
* Added message upon successful store
* Do that thing dennis and kaloyan wanted
* goimports
* Make closeConn a part of the interface for psclient
* Use interface
* Removed uneccessary new lines
* goimport
* Whoops
* Actually we don't want to use the interface in Piece Ranger
* Renamed piecestore in examples to piecestore-client; moved piecestore-cli to examples
* Make comments look nicer
* modified transport client based on the the design discussion
* modified transport client based on the the design discussion
* added the as discussed connection cache interface functionality
* added the as discussed connection cache interface functionality
* transport client changes
* transport client per code review changes
* per the code review comments
* transport client incorporates review comments
* fixes lint warnings
* lint warning fixes....client interface has to be Client
* initial draft of Objectstore
* transport client review changes
* client.go changes
* transport.go changes
* added test case
* added test cases
* streams iface
* comment fix
* object store changes
* comment fix
* initialized the objectstore in gw
* Added PutObject with test support for encryption file
* added object store test cases
* tested & integrated the objectstore with miniogw
* handled the ranger and paths
* indentation change
* Kalyon's code review comments resolution implemented after the 30min code review meeting
* Compilation error fix
* fixes the tavis build warnings
* corrects the ListObject return type to be slice of slice
* corrects the ListObject return type to be slice of slice
* added the serialization using protobuf
* added the unmarshalling of data in getobject()
* Jt's Review comments
* Kaloyan's review comments, moved the unmarshalling logic and other minor code indentation fixes
* more code reivew
* more code reivew
* Changes the expiration time to zeroTime and added error check in putObject function
* Changes the expiration time to zeroTime and added error check in putObject function
* minor warning fix- had to add a comment and fix the wording
* added a TODO comment per kaloyan
* code clean up removed unused variables
* WIP creating admin node service
- WIP changing the process pkg to accept multiple services
- WIP looping over services passed to process
- add netstate/service.go file and abstract it for service processing
* implement goroutine to launch each process
* goroutines working with multiple services
* code review fixes
* more code updates for review
* Add pkg lock and mod files back in
* code review updates
* update process.Main with better concurrent error handling
* Update error handling and pass ctx to StartService
* Update error handling with channel implementation
* Merge in upstream changes
- Simplify error handling channels
* updates
* Updates per reviewable
* fix test
* Setup test exec
* Scaffold test setup
* process main test working
* update admin process test
* Test multiple processes done
* Add error classes for testing, test main logger error
* Updates to tests
* Update how process.Main() handles configs
* Complete merge
* Update Gopkg and add Copyright
* Fix cyclical import issue
- Added .coverprofile to gitignore
- Update admin main.go function call
* remove unnecessary line
* Updates
* DRY up cmd/netstate package
* update service function calls
* updates
* Trying no-ops in examples
* rename netstate to pointerdb
* trying to fix merge
* dep ensure and run tests
* remove flag.Parse
* Update deps
* Skip offending test in pkg/process, to be fixed later
* adds netstate rpc server pagination, mocks pagination in test/util.go
* updates ns client example, combines ns client and server test to netstate_test, adds pagination to bolt client
* better organizes netstate test calls
* wip breaking netstate test into smaller tests
* wip modularizing netstate tests
* adds some test panics
* wip netstate test attempts
* testing bug in netstate TestDeleteAuth
* wip fixes global variable problem, still issues with list
* wip fixes get request params and args
* fixes bug in path when using MakePointers helper fn
* updates mockdb list func, adds test, changes Limit to int
* fixes merge conflicts
* fixes broken tests from merge
* remove unnecessary PointerEntry struct
* removes error when Get returns nil value from boltdb
* breaks boltdb client tests into smaller tests
* renames AssertNoErr test helper to HandleErr
* adds StartingKey and Limit parameters to redis list func, adds beginning of redis tests
* adds helper func for mockdb List function
* if no starting key provided for netstate List, the first value in storage will be used
* adds basic pagination for redis List function, adds tests
* adds list limit to call in overlay/server.go
* streamlines/fixes some nits from review
* removes use of obsolete EncryptedUnencryptedSize
* uses MockKeyValueStore instead of redis instance in redis client test
* changes test to expect nil returned for getting missing key
* remove error from `KeyValueStore#Get`
* fix bolt test
* Merge pull request #1 from bryanchriswhite/nat-pagination
remove error from `KeyValueStore#Get`
* adds Get returning error back to KeyValueStore interface and affected clients
* trying to appease travis: returns errors in Get calls in overlay/cache and cache_test
* handles redis get error when no key found
* port changes
* Task monitor and setup merge from the staging
* Restructure + additional interface
* Add NewOverlayClient
* integrated DHT client interface
* added test for interface
* PR comments addressed
* lint issue
* added generated protobuf
* adding new interface
* added the interface framework
* deleted file
* fixes compilation errors and integrates new dhtcclient interface
* merged netstat latest changes and dht new interface chagnes
* fixed the address's port
* adding comments
* PR comments addressed
* netclient interface dial method added
* rename and integrated transportclient with minio gateway
* rename and code clean up
* made changes based on the Dennis's changes on the kad-client
* Code review comment changes based on kaloyan review comments
* reverted the changes to be similar to master
* removed unused file
* renamed to transportclient
* added the review changes
* store the address of the client
* updates per the code review comments, changes-> added error retry connection attempt logic, added error conditions including nil parameters
* updated the test case to test the bad address passed condition
* updated the code per code review comments
* Bolt backed overlay cache (#94)
* wip
* add separate `Process` tests for bolt and redis-backed overlay
* more testing
* fix gitignore
* fix linter error
* goimports goimports GOIMPORTS GoImPortS!!!!
* fix port madness
* forgot to add
* add `mux` as handler and shorten context timeouts
* gofreakingimports
* fix comments
* refactor test & add logger/monkit registry
* debugging travis
* add comment
* Set redisAddress to empty string for bolt-test
* travis experiment
* refactoring tests
* Merge remote-tracking branch 'upstream/master' into bolt-backed-overlay-cache
* Automatically build, tag and push docker images on merge to master (#103)
* port changes
* build overlay on successful merge to master
* fixes to Makefile
* permissions
* dep ensure
* gopath
* let's try vgo
* remove dep
* maybe alpine is the issue
* tagging go version on build
* stupid vgo
* vgo
* adding tags to push
* quotes
* local linting fixes & stupid travis
* prepend storjlabs to docker tag (#108)
* port changes
* fixing tag name
* Use continue instead of return in table tests (#106)
I did a dumb mistake for some of the table tests, which made some of the
test cases not being executed.
* pkg/kademlia tests and restructuring (#97)
* port changes
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* files created
* Merge remote-tracking branch 'upstream/master' into coyle/kad-tests
* wip
* Merge remote-tracking branch 'upstream/master' into coyle/kad-tests
* wip
* remove bkad dependencie from tests
* wip
* wip
* wip
* wip
* wip
* updated coyle/kademlia
* wip
* cleanup
* ports
* overlay upgraded
* linter fixes
* piecestore kademlia newID
* add changes from kad demo
* PR comments addresses
* go func
* force travis build
* fixed merge conflicts
* fixed merge conflicts
* Merge branch 'coyle/kad-tests' of https://github.com/coyle/storj into coyle/kad-tests
* linter issues
* linting issues
* fixed merge conflicts
* linter is stupid
* Coyle/docker fix (#109)
* port changes
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of https://github.com/storj/storj
* fixing tag name
* no idea
* testing
* changes
* testing on travis
* testing
* changes to travis build
* new approach
* Merge branch 'master' into coyle/docker-fix
* hardcode version (#111)
* hardcode version
* adding coveralls / code coverage (#112)
* adding coveralls
* adding code coverage badge
* fixing badges
* verbose
* swap tests and coverage
* extra line
* maybe
* maybe
* moar
* gover maybe
* testing
* cleanup
* protos/netstate: remove stuff we're not using (#100)
* protos/netstate: remove stuff we're not using
* protos/netstate: add metadata field for segmentstore
* fix netstate client test
* pkg/process: start replacing pkg/process with cobra helpers (#98)
* Implement psclient interface (#107)
* Implement psclient interface
* Add string method to pieceID type
* try to fix linter errors
* Whoops missed an error
* More linter errors
* Typo
* Lol double typo
* Get everything working, begin adding tests for psclient rpc
* goimports
* Forgot to change the piecestore cli when changed the piecestore code
* Fix CLI
* remove ID length, added validator to pieceID
* Move grpc ranger to client
Change client PUT api to take a reader rather than return a writer
* GRPCRanger -> PieceRanger; Make PieceRanger a RangeCloser
* Forgot to remove offset
* Added message upon successful store
* Do that thing dennis and kaloyan wanted
* goimports
* Make closeConn a part of the interface for psclient
* Use interface
* Removed uneccessary new lines
* goimport
* Whoops
* Actually we don't want to use the interface in Piece Ranger
* Renamed piecestore in examples to piecestore-client; moved piecestore-cli to examples
* Make comments look nicer
* modified transport client based on the the design discussion
* modified transport client based on the the design discussion
* added the as discussed connection cache interface functionality
* added the as discussed connection cache interface functionality
* transport client changes
* transport client per code review changes
* per the code review comments
* transport client incorporates review comments
* fixes lint warnings
* lint warning fixes....client interface has to be Client
* client.go changes
* transport.go changes
* added test case
* added test cases
* comment fix
* comment fix
* Implement psclient interface
* Add string method to pieceID type
* try to fix linter errors
* Whoops missed an error
* More linter errors
* Typo
* Lol double typo
* Get everything working, begin adding tests for psclient rpc
* goimports
* Forgot to change the piecestore cli when changed the piecestore code
* Fix CLI
* remove ID length, added validator to pieceID
* Move grpc ranger to client
Change client PUT api to take a reader rather than return a writer
* GRPCRanger -> PieceRanger; Make PieceRanger a RangeCloser
* Forgot to remove offset
* Added message upon successful store
* Do that thing dennis and kaloyan wanted
* goimports
* Make closeConn a part of the interface for psclient
* Use interface
* Removed uneccessary new lines
* goimport
* Whoops
* Actually we don't want to use the interface in Piece Ranger
* Renamed piecestore in examples to piecestore-client; moved piecestore-cli to examples
* Make comments look nicer
* internal/test: switch errors to error classes
if you construct an error directly at package init time, you
won't get useful stack traces or anything. zeebo/errs expects that
you always construct an error (.New) when the error actually
happens. instead, you call Class at init time, then use (Class).Has
to test for error type membership.
* fix linter
* fix test
* piecestore: connect to kad
* piecestore: Linter errors
* piecestore: added pstore config command line utility
* piecestore: removed main.go, implement methods and structs
* piecestore: Import Cam's config code into piecestore-farmer
* piecestore: moving farmer to urfavecli
* piecestore: added create command
* piecestore: Removed old config, added server start code to cli
* piecestore: Get server code working
* piecestore: Changed default dir for storing piece store data; added ttl to config
* piecestore: Generate id; add bootstrap ip for kad
* piecestore: Separate kad port and server port
* piecestore: goimports
* piecestore: Removed print
* piecestore: use pkg/process
* piecestore: Better config file
* piecestore: base58 encode for id
* piecestore: base58 encode and clean up cli
* piecestore: Typo
* piecestore: removed unnecessary variable
* piecestore: Fixed more typos
* piecestore: Place data in a directory based on nodeid
* piecestore: base58 encode instead
* piecestore: Add dependency to go.mod
* piecestore: Fix typo in rpc server start; clear data on failed piece upload
* add reference to dht to overlay client struct
* wip
* wip
* Implement FindNode
* get nodes
* WIP
* Merge in Dennis kademlia code, get it working with our code
* ping and moar
* WIP trying to get cache working with kademlia
* WIP more wiring up
* WIP
* Update service cli commands
* WIP
* added GetNodes
* added nodes to Kbucket
* default transport changed to TCP
* GetBuckets interface changed
* filling in more routing
* timestamp methods
* removed store
* Added initial network overlay explorer page
* Updating and building with dockerfile
* Working on adding bootstrap node code
* WIP merging in dennis' code
* WIP
* connects cache to pkg/kademlia implementation
* WIP redis cache
* testing
* Add bootstrap network function for CLI usage
* cleanup
* call bootstrap on init network
* Add BootstrapNetwork function to interface
* Merge in dennis kad code
* WIP updates to redis/overlay client interface
* WIP trying to get the DHT connected to the cache
* go mod & test
* deps
* Bootstrap node now setting up correctly
- Need to pass it through CLI commands better
* WIP adding refresh and walk functions, added cli flags
- added cli flags for custom bootstrap port and ip
* PR comments addressed
* adding FindStorageNodes to overlay cache
* fix GetBucket
* using SplitHostPort
* Use JoinHostPort
* updates to findstoragenodes response and request
* WIP merge in progress, having issues with a panic
* wip
* adjustments
* update port for dht bootstrap test
* Docker
* wip
* dockerfile
* fixes
* makefile changes
* Update port in NewKademlia call
* Update local kademlia DHT config
* kubernetes yaml
* cleanup
* making tests pass
* k8s yaml
* lint issues
* Edit cli flags to allow for configurable bootstrap IP and Port args
* cleanup
* cache walking the network now
* Rough prototype of Walk function laid out
* Move walk function into bootstrap function
* Update dht.go
* changes to yaml
* goimports
* wip
* wip
* get nodes
* ping and moar
* added GetNodes
* added nodes to Kbucket
* default transport changed to TCP
* GetBuckets interface changed
* filling in more routing
* timestamp methods
* removed store
* testing
* cleanup
* go mod & test
* deps
* PR comments addressed
* fix GetBucket
* using SplitHostPort
* Use JoinHostPort
* added spawn scripts
* wip
* Make server not require length
* Added test for 0 length
* Added test for accepting length of 0
* linter errors
* Make store not take anything more than an id and maybe a ttl if you want
* initial commit for PUT request authorization
* inital auth for put request
* auth. request working for all req
* deleted library
* removed .db files
* work in progress for modifying test suite to accomodate credentials
* modified tests
* gofmt, fixed code based on suggestions; test passed
* gofmt again
* merged nat's update on pointers, passed tests, cleanup from git rebase
* fixed fmt
* fixed fmt on tests
* work in progress
* reduced code
* fixed naming conventions
* added line in code
* fixed server bug, merged new code to server, added env
* fixed linter; getting cright issues on piecestore
* added comments for what passes on the creds
* added spawn scripts
* Determine random id for storing
* Moved determine id to rpc example
* Added tests
* Better test
* goimports
* Updated tests
* Fix typos
* Added piecestore
* gofmt
* Added requested changes
* Added cli
* Removed ranger because I wanted something that can stand alone
* Add example of http server using piece store
* Changed piecestore code to make it more optial for error handelling
* Merged with piecestore
* Added missing package
* Forgot io import
* gofmt
* gofmt
* Forgot io
* Make path by hash exported
* updated to simplify again whoops
* Updated server to work real good
* Forgot ampersand
* Updated to match FilePiece
* Merged in cam's delete code
* Remove unused io
* Added RPC code
* Give the download request a reader
* Removed http server stuff; changed receive stream to say io.reader
* Added expiration date to shardInfo
* gRPC Ranger
* Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions
* Adapt to latest changes in piece store rpc api
* added ttl info request
* Initialize grpcRanger type with named fields
* Move scripts to http server pr; added close method for Retrieve api
* added rpc server tests for getting piece meta data and retrieval routes
* Adapt to PieceStreamReader now being a ReadCloser
* Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf
* Imported cams test
* Bump gometalinter deadline
* Adapt to package name changes
* Remove Garbage
* Adapt to latest changes in piece store rpc api
* NewCustomRoute constructor to allow mocking the gRPC client
* Name struct values in constructor.
* Added piecestore
* gofmt
* Added requested changes
* Added cli
* Removed ranger because I wanted something that can stand alone
* Add example of http server using piece store
* Changed piecestore code to make it more optial for error handelling
* Merged with piecestore
* Added missing package
* Forgot io import
* gofmt
* gofmt
* Forgot io
* Make path by hash exported
* updated to simplify again whoops
* Updated server to work real good
* Forgot ampersand
* Updated to match FilePiece
* Merged in cam's delete code
* Remove unused io
* Added RPC code
* Give the download request a reader
* Removed http server stuff; changed receive stream to say io.reader
* Added expiration date to shardInfo
* Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions
* added ttl info request
* Move scripts to http server pr; added close method for Retrieve api
* added rpc server tests for getting piece meta data and retrieval routes
* Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf
* Imported cams test
* Bump gometalinter deadline
* WIP adding tests
* added tests for store and delete routes
* Add changes as requested by Kaloyan, also cleaned up some code
* Get the code actually working whoops
* More cleanup
* Separating database calls from api.go
* need to rename expiration
* Added some changes requested by JT
* Fix read size
* Fixed total amount to read
* added tests
* Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages
* Moved rpc client and server to piece store
* Moved piecestore protobuf to the /protos folder
* Cleaned up messages
* Clean up linter errors
* Added missing sqlite import
* Add ability to do iterative reads and writes to pstore
* Incrementally read data
* Fix linter and import errors
* Solve linter Error
* Change return types
* begin test refactor
* refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go
* Cleaning up tests
* Added retrieve tests
* refactored delete tests
* Deleted old tests
* Updated cmd/piecestore to reflect changes to piecestore
* Refactored server tests and server/client store code
* gofmt
* WIP implementing TTL struct
* Read 4k at a time when Retrieving
* implemented ttl struct
* Accidentally removed fpiece dependency?
* Double resolve merge conflict
* Moved client to the examples since it is an example
* Change hash to id in protobuf. TODO: Change client and server to reflect these changes
* Update client and protobuf
* changed hash to id
* Handle eof properly in retrieve
* Id -> ID
* forgot to change import for client after moving
* Moved client and server main to examples
* Made changes requested by JT
* checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close()
* Print not fatal
* Handle overflow when reading from server
* added const IDLength
* removed types from comments
* Add reader/writer for download data from and uploading data to server
* goimports and comments
* fixed nits, casts, added OK const, DBCleanup now exits program on error
* Add stream reader and writer for server
* Fix errors
* i beforee except after c lol
* customizable data dir
* Forgot to rename variable
* customizable data dir
* Handle closing of server stream properly
* linter
* pass on inserting the same data twice
* linter
* linter
* Do the easy things JT asked for
* Handle overflow from reads properly; handle custom db path
* Handle overflow for server stream read; TODO Combine server and client stream reads
* Allow for TTL of 0 to stay forever
* Make Client cleaner for user
* integrated eestream's serve-pieces functionality
* added ref to http request function
* created dummy bucket list
* Initialized the buckets with files with hardcoded sample data
* supports upload Object(s)
* uploads to corresponding folders
* code cleanup for review
* updated based on code review comments
* updates based on missed code review comments
* updated with review comments
* implemented review comments
* merged latest and tested
* added filepath.Join()
* updates based on the comments
* fixes the eestreamer parameter due to merge
* Moved filepiece into storj
* Fix linter errors
* Seek comment for linter
* gofmt/golinter accidentally removed import
* Fix small typos
* Use the weird iota. P cool dude ✌️
* Do things the cool way
* Changes requested by kaloyan
* didn't need test main
* Path encryption library
* Use base64 instead of hex encoding
* Prepend version number to encrypted path segments
* Remove redundant var alias
* Simplified returns
* wrap errors
* Buffered eestream EncodeReader
* Extracted fillBuffer() helper function
* Slow channels will be closed if there are still at least k fast channels
* Doc comment for maxBufferMemory
* Use timer more efficiently
* Timer initialization should be inside the for-loop
* Parallel copy of encoded data to reader buffer channels
* Transfer input read errors to output encoded readers
* minimum and optimum thresholds
* Use time.AfterFunc
* Simplify error handling in constructor
* adds pointer to netstate proto file
* generated updated netstate proto
* changes boltdb netstate to save pointers as values
* updates netstate Put to save Pointers, updates client example to put a pointer, adds grpc status errors, updates tests, changes boltdb 'File' struct to 'PointerEntry'
* updates netstate client example and client test to save pointers, updates netstate List and Delete
* begins adding netstate-http tests
* removes netstate http service
* re-adds netstate auth
* updates boltdb netstate test
* changes encrypted_unencrypted_size from int64 to bytes in netstate proto
* updates READMEs
* Added piecestore
* gofmt
* Added requested changes
* Added cli
* Removed ranger because I wanted something that can stand alone
* Changed piecestore code to make it more optial for error handelling
* Added missing package
* Forgot io import
* gofmt
* Make path by hash exported
* updated to simplify again whoops
* Forgot ampersand
* Updated to match FilePiece
* Change store to use io.CopyN
* Clear golinter errors
* Updated go.mod
* Updated go.mod
* AES GCM implementation and unit test code
* modified and tested per the code review comments
* modified and tested per the code review comments
* updated to return err
* removed the debug printf commented code
* support of aes-gcm
* updated the renaming convention per GOLANG coding standards
* Initial Go & C biniding with libstorj
* Initial GO & C bindings with libstorj
* fixing the callback
* moved a millimeter :-) on c to go to gone....
* added error condition, per review comment
* removed the .idea directory and also movie.avi file
* deleting files not to be in this pull request
* Revert "deleting files not to be in this pull request"
This reverts commit 026b834fe00f6b20a7566e71973fe224c12f533f.
* deleting files not to be in this pull request
* resolving conflicts
* syncing the file to master
* Use aes gcm in eestream rs tests
* Use aes gcm in serve example
* Fixed comment
* adds proto files for netstate crud
* moves netstate grpc client lib into pkg/netstate where grpc netstate service is defined
* starts adding grpc client and server tests
* moves creation of grpc server into cmd/netstate/main.go, removes pkg/netstate/service.go, adds more client testing
* changed all 'Path' and 'Value' fields from strings to bytes, updated tests
* changes Get and Delete in proto file to receive 'requests' instead of 'file paths', adds tests for Get, List, and Delete
* changes netstate-routes to get 'fileValue' bytes not 'fileInfo'
* adds example rpc client in 'examples' and adds more specific debug logs
* adds readmes for netstate rpc services and updates netstate-routes
* WIP eestream avoid waiting for slow pieces
* Use non-blocking select to harvest all available inbufs
* Better way to check for elapsed time in tests
* Improve comment
* Determine EOF based on expected size
* Remove unused readerError type
* Configurable readers channel size
* Close properly decodedReader in tests
* Use context for properly closing the decodedReader
* Handle infectious errors using the new descendent classes
* Refactor decodedReader.Read() into helper functions
* Reenable TestRSErrors
* Test with Rangers
* Remove obsolete comment
* Decoder can tolerate readers with unexpected EOF
* Return EOF if required number of inbufs are at EOF
* Use existing randData() to generate random data for tests
* Test case for io.ErrUnexpectedEOF
* Add TransformReaderSize constructor
* Do not fail decoding on first read error
Try decoding with the rest successfully read inputs.
* some small code improvements
* Allocate map memory upfront
* HTTPRanger
* Migrate eestream library to use ReadCloser instead of Reader
* Use MultiReadCloser instead of MultiReader
* Adapt example cli cmds to latest serverbox changes
* Close LazyReadCloser only if only generated
* Close ReadClosers sequentially
* Close response body on unexpected status code
* Avoid double close of the pipe reader
* Better formatting for imports
* Exit instead of panicking
* Light client implementation with get-info and list-buckets commands
* Fix client package name
* Fix go.mod to work with vgo
* Use single `fmt.Printf` for `get-info` output
* Use unnamed import to client package
* Simplify usage of sha256 and sha512 sums
* Remove obsolete test code
* Use helper structs for unmarshalling bridge info
* Remove LGPL license files and adjust copyright headers
* Use github.com/zeebo/errs
* Use httptest for test http server
* Use viper for env var management
* Nested struct for swagger
* Add github.com/zeebo/errs to go.mod
* More bucket tests
* word wrap long line
* Use zeebo/errs for crypto errors
