Commit Graph

1029 Commits

Author SHA1 Message Date
Kaloyan Raev
90ca28f7bc eestream avoid waiting for slow pieces (#30)
* WIP eestream avoid waiting for slow pieces

* Use non-blocking select to harvest all available inbufs

* Better way to check for elapsed time in tests

* Improve comment

* Determine EOF based on expected size

* Remove unused readerError type

* Configurable readers channel size

* Close properly decodedReader in tests

* Use context for properly closing the decodedReader

* Handle infectious errors using the new descendent classes

* Refactor decodedReader.Read() into helper functions

* Reenable TestRSErrors

* Test with Rangers

* Remove obsolete comment
2018-05-14 14:30:57 -06:00
Dennis Coyle
4be0503f4a DHT interfaces (#28)
* first pass at interface for dht

* wip

* comments addressed from @jtolds

* adjust GetBuckets

* missed comment

* addressed PR comments from @jtolds
2018-05-09 08:04:51 -06:00
JT Olds
fc7cfde9f7 pkg/miniogw: a minio-based s3 gateway for storj. just stubs right now (#34) 2018-05-08 19:49:54 -04:00
JT Olio
78989a2d5a fix hanging travis test 2018-05-08 17:16:16 -06:00
JT Olds
2ef8e581b2 fix travis, and get a clean lint (#32)
* fix travis

* lint fixes

* travis: enable tests

* fix some tests, disable a broken eestream test for now
2018-05-08 18:02:01 -04:00
Natalie Villasana
c73a4f9d68
Merge pull request #20 from coyle/clean-lint
Clean lint!
2018-05-07 14:14:40 -04:00
Dennis Coyle
5e9b3f7d6a merge conflicts 2018-05-07 14:04:53 -04:00
JT Olds
5be1da982f pkg/telemetry: telemetry package using zeebo/admission (#26)
* pkg/telemetry: use zeebo/admission

* pkg/telemetry: updated to use latest zeebo/admission changes
2018-05-07 14:03:40 -04:00
Kaloyan Raev
1c56a4cf5c Decoder can tolerate readers with unexpected EOF and random data (#27)
* Decoder can tolerate readers with unexpected EOF

* Return EOF if required number of inbufs are at EOF

* Use existing randData() to generate random data for tests

* Test case for io.ErrUnexpectedEOF

* Add TransformReaderSize constructor
2018-05-04 14:52:49 -06:00
Dennis Coyle
a51087a343 merge conflicts 2018-05-02 15:10:40 -04:00
JT Olds
4f40280c71 some small import cleanups, go generate overlay protos (#25)
checking in go generated stuff is recommended by
https://blog.golang.org/generate
2018-05-01 13:24:32 -04:00
Dennis Coyle
5d8d0cdda0 renamed ranger.Closer -> ranger.RangeCloser 2018-04-30 10:49:04 -04:00
Kaloyan Raev
8c96f4e5a5 Avoid extra Range requests for content type detection 2018-04-30 06:08:41 -06:00
Kaloyan Raev
4635621a4d Move helper structs inside functions 2018-04-30 06:06:25 -06:00
Kaloyan Raev
acce2bfc08 Reduce network latency by initializing Rangers in parallel 2018-04-30 06:06:25 -06:00
Dennis Coyle
bb8d4f8d22 cleanup 2018-04-25 10:55:26 -04:00
Kaloyan Raev
69239e9e17 Do not fail decoding on first read error (#17)
* Do not fail decoding on first read error

Try decoding with the rest successfully read inputs.

* some small code improvements

* Allocate map memory upfront
2018-04-23 20:17:50 -06:00
JT Olds
00854b9736
pkg/ranger: FileRanger (#18) 2018-04-23 20:16:34 -06:00
Dennis Coyle
ecbd5f08c3 Adding gRPC server/clients for overlay (#14)
* overlay proto

* grpc server and client

* fix import to storj

* tests

* change imports

* imports cleanup/comments

* PR comments addressed from @jtolds
2018-04-23 09:54:22 -06:00
Kaloyan Raev
f96cc9c955 HTTP Ranger (#11)
* HTTPRanger

* Migrate eestream library to use ReadCloser instead of Reader

* Use MultiReadCloser instead of MultiReader

* Adapt example cli cmds to latest serverbox changes

* Close LazyReadCloser only if only generated

* Close ReadClosers sequentially

* Close response body on unexpected status code

* Avoid double close of the pipe reader

* Better formatting for imports

* Exit instead of panicking
2018-04-17 07:39:14 -06:00
Dennis Coyle
2c08e56b20 fix import to storj 2018-04-16 10:09:01 -06:00
Dennis Coyle
1975b98eba grpc server and client 2018-04-16 10:09:01 -06:00
Kaloyan Raev
7fde8b908a
Light client implementation with get-info and list-buckets commands (#2)
* Light client implementation with get-info and list-buckets commands

* Fix client package name

* Fix go.mod to work with vgo

* Use single `fmt.Printf` for `get-info` output

* Use unnamed import to client package

* Simplify usage of sha256 and sha512 sums

* Remove obsolete test code

* Use helper structs for unmarshalling bridge info

* Remove LGPL license files and adjust copyright headers

* Use github.com/zeebo/errs

* Use httptest for test http server

* Use viper for env var management

* Nested struct for swagger

* Add github.com/zeebo/errs to go.mod

* More bucket tests

* word wrap long line

* Use zeebo/errs for crypto errors
2018-04-16 16:42:06 +03:00
JT Olio
c90143a58d eestream: make secretbox example harder to fall victim to nonce reuse 2018-04-16 06:52:23 -06:00
Kaloyan Raev
6c2c4b92e4
Merge pull request #1 from jtolds/master
merge eestream
2018-04-11 22:32:06 +03:00
JT Olio
d84757d49a pkg/ranger: fix linting errors 2018-04-11 08:05:41 -06:00
JT Olio
80fa10a06e pkg/eestream: initial 2018-04-11 07:46:34 -06:00
JT Olio
21d8a20037 ranger package 2018-04-11 06:55:40 -06:00
Dennis Coyle
b236d7dffb keep 2018-04-06 12:36:55 -04:00