Alexander Leitner
8f07c4d746
Keep track of used space ( #338 )
...
* Keep track of space at all times
* Create index on expires
* Updated mock psclient
2018-09-11 08:40:45 -04:00
Jennifer Li Johnson
d0f87f0de1
Overlay Bulk Lookup ( #279 )
...
* implements bulk node lookup from overlay cache
2018-09-11 00:52:14 -04:00
Michal Niewrzal
dfc52e4394
[WIP] Piece Rangers should connect lazily to storage nodes (second approach) ( #321 )
...
* Piece Rangers should connect lazily to storage nodes
* code reorganization
* fixed unit tests
* remove if
2018-09-08 16:52:19 +03:00
JT Olio
a4d1070d68
storage/ec: fix some Get socket leaks ( #308 )
...
* storage/ec: fix some Get socket leaks
* fix linters
2018-09-05 10:08:39 -07:00
Alexander Leitner
899e1e68f1
Add functions for signing and verifying during bandwidth exchange ( #246 )
...
* Added initial functions for signing and verifying
* whoops
* Get client up to speed
* Added initial functions for signing and verifying
* whoops
* Get client up to speed
* wip
* wip
* actual signatures in tests
(cherry picked from commit 1464853b737f1d712d64fbf90147f535525c8fd9)
* bugfixing
* Generate private key in example
* Generate signatures for pieceranger tests
* Update examples to use TLS
* Use private key from identity inside of example
* Use crypto.PrivateKey interface
* Change err name in defers
* Pass tests
* Pass identity Key to PSClient
* Get tests passing on travis
* Resolve linter complaints
2018-08-27 14:35:27 -04:00
Kaloyan Raev
130acf57a5
Make PSClient a Closer, so it can be closed with utils.LogClose() ( #247 )
2018-08-20 18:11:54 +03:00
Alexander Leitner
b0db33f919
Bandwidth accounting ( #134 )
...
* captplanet standalone farmer setup
* Bandwidth Allocation
* utils.Close method changed to utils.LogClose
* Get build temporarily working
* Get/Put for PSClient should take payer bandwidth allocations rather than the NewPSClient function
* Update example client to reflect changes in client API
* Update ecclient to use latest PSClient, Make NewPSClient return error also
* Updated pieceranger tests to check for errors; sign method should take byte array
* Handle defers in store.go better
* Fix defer functions in psdb.go
* fun times
* Protobuf bandwidthallocation data is now a byte array
* Remove psservice package and merge it into pstore server
* Write wrapper for database calls
* Change all expiration names in protobuf to be more informative; add defer in retrieve; remove old comment
* Make PSDB tests implementation independent rather than method independent
* get rid of payer, renter in ecclient
* add context monitoring in store and retrieve
2018-08-17 13:40:15 -04:00
Bryan White
5d20cf8829
Node Identity ( #193 )
...
* peertls: don't log errors for double close
understood that this part of the code is undergoing heavy change
right now, but just want to make sure this fix gets incorporated
somewhere
* git cleanup: node-id stuff
* cleanup
* rename identity_util.go
* wip `CertificateAuthority` refactor
* refactoring
* gitignore update
* wip
* Merge remote-tracking branch 'storj/doubleclose' into node-id3
* storj/doubleclose:
peertls: don't log errors for double close
* add peertls tests & gomports
* wip:
+ refactor
+ style changes
+ cleanup
+ [wip] add version to CA and identity configs
+ [wip] heavy client setup
* refactor
* wip:
+ refactor
+ style changes
+ add `CAConfig.Load`
+ add `CAConfig.Save`
* wip:
+ add `LoadOrCreate` and `Create` to CA and Identity configs
+ add overwrite to CA and identity configs
+ heavy client setup
+ refactor
+ style changes
+ cleanup
* wip
* fixing things
* fixing things
* wip hc setup
* hc setup:
+ refactor
+ bugfixing
* improvements based on reveiw feedback
* goimports
* improvements:
+ responding to review feedback
+ refactor
* feedback-based improvements
* feedback-based improvements
* feedback-based improvements
* feedback-based improvements
* feedback-based improvements
* feedback-based improvements
* cleanup
* refactoring CA and Identity structs
* Merge branch 'master' into node-id3
* move version field to setup config structs for CA and identity
* fix typo
* responding to revieiw feedback
* responding to revieiw feedback
* responding to revieiw feedback
* responding to revieiw feedback
* responding to revieiw feedback
* responding to revieiw feedback
* Merge branch 'master' into node-id3
* fix gateway setup finally
* go imports
* fix `FullCertificateAuthority.GenerateIdentity`
* cleanup overlay tests
* bugfixing
* update ca/identity setup
* go imports
* fix peertls test copy/paste fail
* responding to review feedback
* setup tweaking
* update farmer setup
2018-08-13 10:39:45 +02:00
Kaloyan Raev
ffaebb57df
Improved error message in ECClient ( #214 )
...
* Improved error message in ECClient
* Fixed broken test
2018-08-10 12:14:23 +03:00
Natalie Villasana
9ad4949825
adds segment store tests ( #198 )
...
* starts adding segmentStore tests
* adds mocked interfaces for segmentStore tests
* adds tests for put, get, delete, and list
* regenerates pointerdb mock and updates calls to accommodate new changes
2018-08-07 11:52:11 -04:00
Kaloyan Raev
aac7c6fbd1
ECClient pads data to fit RS block size ( #199 )
2018-08-06 17:24:30 +03:00
Kaloyan Raev
85eee94814
Fixes V3-173 - ECClient check if given nodes are unique ( #186 )
2018-08-02 18:12:19 +03:00
Cameron
6463b87ebe
Errcheck ( #133 )
...
* add errcheck
* fixed linter errors
* fixes
* errcheck fixes in pkg/paths
* Fix errchecks in PieceID.Derive
* Fix ecclient tests
* Move closeConn a little bit above in the exectution flow
* fix new lint errors
* Fatalf -> Printf
* address eclipsed errors
* rename err to closeErr
* rename err to closeErr for differentiation
2018-07-16 15:22:34 -04:00
Natalie Villasana
d82486b85b
adds SegmentStore Put functionality ( #138 )
...
* lays out SegmentStore functions to implement
* Merge branch 'master' into segment-store
* adds overlay calls to put
* allows SegmentStore Put to upload a file to ecclient, then save pointer to pointerdb
* Merge branch 'master' into segment-store
* removes new overlay client instance in Put
* fixes syntax
* fixes syntax again
* fixes imports
* fixes typo
* removes pointerdb client from segmentStore struct for now
* changes SegmentStore to segmentStore
* changing types in parameters to fit other function calls
* takes RedundancyStrategy out of Put params
* changes NewClient param back to take an interface (not pointer to interface)
* fixes types
* moves pointer into PutRequest in SegmentStore Put
* passes interfact, not pointer to interface to NewSegmentStore
* fixes some types
* Get returns an instance of Meta
* fixes PutRequest fields
* adds remotePieces slice to pointerdb PutRequest
* ecClient Put now takes *proto.Nodes instead of proto.Nodes
* fixes syntax
* changes ec client dial interface to use *proto.Node
* changes other instances of proto.Node to *proto.Node in ecclient pkg
* adds *proto.Node to Get and Delete functions in ecclient pkg
* changes proto.Node to pointer in ec client_test
* changes proto.Node to pointer in ec client_test
* adds ecclient and pointerdb client to the segmentstore constructor
* adds ecclient and pointerDBClient to segmentStore constructor
2018-07-12 18:37:50 -04:00
Bryan White
5b913c45b9
Transport security ( #63 )
...
* wip initial transport security
* wip: transport security (add tests / refactor)
* wip tests
* refactoring - still wip
* refactor, improve tests
* wip tls testing
* fix typo
* wip testing
* wip testing
* wip
* tls_test passing
* code-style improvemente / refactor; service and tls tests passing!
* code-style auto-format
* add TestNewServer_LoadTLS
* refactor; test improvements
* refactor
* add client cert
* port changes
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* files created
* Merge remote-tracking branch 'upstream/master' into coyle/kad-tests
* wip
* add separate `Process` tests for bolt and redis-backed overlay
* more testing
* fix gitignore
* fix linter error
* goimports goimports GOIMPORTS GoImPortS!!!!
* wip
* fix port madness
* forgot to add
* add `mux` as handler and shorten context timeouts
* gofreakingimports
* fix comments
* refactor test & add logger/monkit registry
* debugging travis
* add comment
* Set redisAddress to empty string for bolt-test
* Merge remote-tracking branch 'upstream/master' into coyle/kad-tests
* Merge branch 'tls' into tls-upstream
* tls:
add client cert
refactor
refactor; test improvements
add TestNewServer_LoadTLS
code-style auto-format
code-style improvemente / refactor; service and tls tests passing!
tls_test passing
wip
wip testing
wip testing
fix typo
wip tls testing
refactor, improve tests
refactoring - still wip
wip tests
wip: transport security (add tests / refactor)
wip initial transport security
* fixing linter things
* wip
* remove bkad dependencie from tests
* wip
* wip
* wip
* wip
* wip
* updated coyle/kademlia
* wip
* cleanup
* ports
* overlay upgraded
* linter fixes
* piecestore kademlia newID
* Merge branch 'master' into tls-upstream
* master:
Add error to the return values of Ranger.Range method (#90 )
udp-forwarding: demo week work! (#84 )
* Merge branch 'kad-tests' into tls-upstream
* kad-tests:
piecestore kademlia newID
linter fixes
overlay upgraded
ports
cleanup
wip
updated coyle/kademlia
wip
wip
wip
wip
wip
remove bkad dependencie from tests
wip
wip
files created
port changes
* wip
* finish merging service tests
* add test for different client/server certs
* wip
* Merge branch 'master' into tls-upstream
* master:
Add context to Ranger.Range method (#99 )
Coyle/kad client (#91 )
* wip
* wip; refactoring/cleanup
* wip
* Merge branch 'master' into tls
* master:
Bolt backed overlay cache (#94 )
internal/test: switch errors to error classes (#96 )
* wip - test passing
* cleanup
* remove port.go
* cleanup
* Merge branch 'master' into tls
* master:
hardcode version (#111 )
Coyle/docker fix (#109 )
pkg/kademlia tests and restructuring (#97 )
Use continue instead of return in table tests (#106 )
prepend storjlabs to docker tag (#108 )
Automatically build, tag and push docker images on merge to master (#103 )
* more belated merging
* more belated merging
* more belated merging
* add copyrights
* cleanup
* goimports
* refactoring
* wip
* wip
* implement `TLSFileOptions#loadTLS`, refactoring:
`peertls.TestNewClient_LoadTLS` is the failing holdout; Still trying to figure out why I'm getting ECDSA verification is failing.
* not sure if actually working:
Tests are now passing (no more "ECDSA verification failed"); however,
`len(*tls.Certificates.Certificate) == 1` which I don't think should be
the case if the root and leaf are being created correctly.
* Experimenting/bugfixing?:
I think leaf certs should be properly signed by the parent now but not
entirely certain. It's also unclear to me why in
`VerifyPeerCertificate`, `len(rawCerts) == 1` when the certs should
contain both the root and leaf afaik.
* Properly write/read certificate chain (root/leaf):
I think I'm now properly reading and writing the root and leaf
certificate chain such that they're both being received by
`VerifyPeerCertificate`.
The next step is to parse the certificates with `x509.ParseCertificate`
(or similar) and verify that the public keys and signatures match.
* Add tls certificate chain signature veification (spike):
+ `VerifyPeerCertificate` verifies signatures of certificates using the
key of it's parent if there is one; otherwise, it verifies the
certificate is self-signed
+ TODO: refactor
+ TODO: test
* refactoring `VerifyPeerCertificate`
* cleanup
* refactor
* Merge branch 'master' into tls
* master:
Remove some structural folders we don't seem to be using. (#125 )
license code with agplv3 (#126 )
Update .clabot (#124 )
added team memebers (#123 )
clabot file added (#121 )
ECClient (#110 )
docker image issue fixed (#118 )
Piecestore Farmer CLI (#92 )
Define Path type (#101 )
adds netstate pagination (#95 )
Transport Client (#89 )
Implement psclient interface (#107 )
pkg/process: start replacing pkg/process with cobra helpers (#98 )
protos/netstate: remove stuff we're not using (#100 )
adding coveralls / code coverage (#112 )
* responding to review feedback / cleanup / add copywrite headers
* suggestions
* realitive
* Merge pull request #1 from coyle/coyle/tls
suggestions
* remove unnecessary `_`s
* Merge branch 'tls' of github.com:bryanchriswhite/storj into tls
* 'tls' of github.com:bryanchriswhite/storj:
realitive
suggestions
* Responding to review feedback:
+ refactor `VerifyPeerCertificate`
* remove tls expiration
* remove "hosts" and "clien option" from tls options
* goimports
* linter fixes
2018-07-09 19:43:13 +02:00
Kaloyan Raev
a6bafa993a
Obscure piece id across piece store nodes ( #120 )
...
* Obscure piece id across piece store nodes
* Add line separator in imports
* Avoid potential panic if hash < 32 bytes
2018-07-06 11:51:13 +03:00
Kaloyan Raev
d8f1ec1db6
ECClient ( #110 )
...
* WIP ECClient
* Get returns RangeCloser
* Introduce RedundancyStrategy
* Constructor takes max buffer memory
* Remove unnecessary NopCloser wrapper
* Added telemetry
* Tests
* Adapt to PSClient from master
* Decode should report error if empty rrs map is passed
* collectErrors helper
* Move to /pkg/storage
* Move to /pkg/storage/ec
* Rename ecclient.go to client.go
* Better logging
* Rename ec.ECClient to ec.Client
* Fix some test execution
* Adopt Transport Client from master
2018-07-03 11:35:01 +03:00
