Commit Graph

115 Commits

Author SHA1 Message Date
Kaloyan Raev
ad9b7e01a6
Path encryption library (#61)
* Path encryption library

* Use base64 instead of hex encoding

* Prepend version number to encrypted path segments

* Remove redundant var alias

* Simplified returns

* wrap errors
2018-05-30 18:33:27 +03:00
Kaloyan Raev
ed72a5e542
Optimize eestream upload for slow piece storage nodes (#48)
* Buffered eestream EncodeReader

* Extracted fillBuffer() helper function

* Slow channels will be closed if there are still at least k fast channels

* Doc comment for maxBufferMemory

* Use timer more efficiently

* Timer initialization should be inside the for-loop

* Parallel copy of encoded data to reader buffer channels

* Transfer input read errors to output encoded readers

* minimum and optimum thresholds

* Use time.AfterFunc

* Simplify error handling in constructor
2018-05-30 18:27:09 +03:00
JT Olio
22b1fe4e21 pkg/process: add pkg/telemetry plumbing (#47)
* pkg/process: add pkg/telemetry plumbing

* pkg/process: add debug endpoints

* fix linting
2018-05-30 10:03:44 -04:00
Natalie Villasana
6723064bfb
Enables netstate service to save pointers (#49)
* adds pointer to netstate proto file

* generated updated netstate proto

* changes boltdb netstate to save pointers as values

* updates netstate Put to save Pointers, updates client example to put a pointer, adds grpc status errors, updates tests, changes boltdb 'File' struct to 'PointerEntry'

* updates netstate client example and client test to save pointers, updates netstate List and Delete

* begins adding netstate-http tests

* removes netstate http service

* re-adds netstate auth

* updates boltdb netstate test

* changes encrypted_unencrypted_size from int64 to bytes in netstate proto

* updates READMEs
2018-05-29 22:47:40 -04:00
nfarah86
6b49aed88a Mutex/process api key (#29)
* initial commit

* created httpreq struct

* checked against env key

* wrapped up comparing xapikey to flag input

* added readme

* added example and updated code with viper

* added ConstantTimeCompare function

* fixed spacing

* updated with comments, and updated readme

* updated main.go with setting env variable

* fixed linter errors
2018-05-15 15:30:32 -04:00
aligeti
0376dc4bd2 AES GCM implementation and unit test code (#19)
* AES GCM implementation and unit test code

* modified and tested per the code review comments

* modified and tested per the code review comments

* updated to return err

* removed the debug printf commented code

* support of aes-gcm

* updated the renaming convention per GOLANG coding standards

* Initial Go & C biniding with libstorj

* Initial GO & C bindings with libstorj

* fixing the callback

* moved a millimeter :-) on c to go to gone....

*  added error condition, per review comment

* removed the .idea directory and also movie.avi file

* deleting files not to be in this pull request

* Revert "deleting files not to be in this pull request"

This reverts commit 026b834fe00f6b20a7566e71973fe224c12f533f.

* deleting files not to be in this pull request

* resolving conflicts

* syncing the file to master

* Use aes gcm in eestream rs tests

* Use aes gcm in serve example

* Fixed comment
2018-05-15 18:11:03 +03:00
Natalie Villasana
1a06c2b0c7 adding grpc interface to network state (#33)
* adds proto files for netstate crud

* moves netstate grpc client lib into pkg/netstate where grpc netstate service is defined

* starts adding grpc client and server tests

* moves creation of grpc server into cmd/netstate/main.go, removes pkg/netstate/service.go, adds more client testing

* changed all 'Path' and 'Value' fields from strings to bytes, updated tests

* changes Get and Delete in proto file to receive 'requests' instead of 'file paths', adds tests for Get, List, and Delete

* changes netstate-routes to get 'fileValue' bytes not 'fileInfo'

* adds example rpc client in 'examples' and adds more specific debug logs

* adds readmes for netstate rpc services and updates netstate-routes
2018-05-14 18:31:26 -06:00
Kaloyan Raev
90ca28f7bc eestream avoid waiting for slow pieces (#30)
* WIP eestream avoid waiting for slow pieces

* Use non-blocking select to harvest all available inbufs

* Better way to check for elapsed time in tests

* Improve comment

* Determine EOF based on expected size

* Remove unused readerError type

* Configurable readers channel size

* Close properly decodedReader in tests

* Use context for properly closing the decodedReader

* Handle infectious errors using the new descendent classes

* Refactor decodedReader.Read() into helper functions

* Reenable TestRSErrors

* Test with Rangers

* Remove obsolete comment
2018-05-14 14:30:57 -06:00
Dennis Coyle
a51087a343 merge conflicts 2018-05-02 15:10:40 -04:00
Kaloyan Raev
4635621a4d Move helper structs inside functions 2018-04-30 06:06:25 -06:00
Kaloyan Raev
acce2bfc08 Reduce network latency by initializing Rangers in parallel 2018-04-30 06:06:25 -06:00
Dennis Coyle
bb8d4f8d22 cleanup 2018-04-25 10:55:26 -04:00
JT Olds
00854b9736
pkg/ranger: FileRanger (#18) 2018-04-23 20:16:34 -06:00
Kaloyan Raev
f96cc9c955 HTTP Ranger (#11)
* HTTPRanger

* Migrate eestream library to use ReadCloser instead of Reader

* Use MultiReadCloser instead of MultiReader

* Adapt example cli cmds to latest serverbox changes

* Close LazyReadCloser only if only generated

* Close ReadClosers sequentially

* Close response body on unexpected status code

* Avoid double close of the pipe reader

* Better formatting for imports

* Exit instead of panicking
2018-04-17 07:39:14 -06:00
Dennis Coyle
b236d7dffb keep 2018-04-06 12:36:55 -04:00