Maximillian von Briesen
6c655d117b
Filter nodes by reputation and IP address (node selection) ( #720 )
...
* Pulls statdb stats into overlay cache whenever cache.Put() is called
* Updates overlay.FindStorageNodes()/overlayClient.Choose() to filter based on node stats
* Updates overlay.FindStorageNodes()/overlayClient.Choose() to exclude duplicate IP addresses
2018-12-04 15:18:26 -05:00
Bryan White
2a0c4e60d2
preparing for use of customtype gogo extension with NodeID type ( #693 )
...
* preparing for use of `customtype` gogo extension with `NodeID` type
* review changes
* preparing for use of `customtype` gogo extension with `NodeID` type
* review changes
* wip
* tests passing
* wip fixing tests
* more wip test fixing
* remove NodeIDList from proto files
* linter fixes
* linter fixes
* linter/review fixes
* more freaking linter fixes
* omg just kill me - linterrrrrrrr
* travis linter, i will muder you and your family in your sleep
* goimports everything - burn in hell travis
* goimports update
* go mod tidy
2018-11-29 19:39:27 +01:00
Egon Elbre
0ae05cf834
Ensure that we only choose storage nodes ( #732 )
2018-11-29 16:57:00 +02:00
Maximillian von Briesen
7cf16503b8
Add statdb to overlay cache (node selection) ( #711 )
...
* make overlay node rep consistent with statdb node rep
* add statdb to testplanet
* add statdb to overlay cache
* tests
2018-11-27 12:46:12 -05:00
Bryan White
dabd87cb36
swap golang/protobuf out for gogo/protobuf ( #692 )
...
* swap golang/protobuf out for gogo/protobuf
* revert go.mod and go.sum changes
* linter fixes
2018-11-20 19:29:07 +01:00
Egon Elbre
6fe16e48ba
Simplify and fix tests ( #660 )
2018-11-16 18:31:14 +02:00
Bryan White
ee62e2a9d8
Use transport client and cleanup all the clients ( #574 )
...
* wip
* linter fixes
* linter fixes
* test fixes
* linter fixes
* fix merge + restructure piecestore packages
* review feedback
* linter fixes
* linter fixes
* remove unnecessary aliases to piecestore
* more merge fixing
2018-11-06 18:49:17 +01:00
Bryan White
8b9711cb5e
better waitlist-gating ( #557 )
...
* better waitlist-gating
(cherry picked from commit 490fe02b7c3558da18678dfb651c92ec9c4a75b5)
* fix broken test
* linter fixes
* linter fixes
* make extension verification optional
* add certifcate gating script for captplanet
* fixing tests
* linter fixes
* linter fixes?
* moar linter fixes
* Revert "moar linter fixes"
This reverts commit 8139ccbd73cbbead987b7667567844f50f7df2c8.
* just kill me
* refactor
* refactor tests
* liniter...
* cleanup
2018-11-01 16:48:43 +01:00
Cameron
939515e6ee
Increase test coverage for overlay client Choose method ( #508 )
...
* add server option to NewMockServer
* add test to check nodes returned by Choose against list of excluded IDs
2018-10-19 10:05:31 -04:00
Cameron
f5c115596e
filter out excluded nodes when repairing ( #426 )
...
* add filter field into OverlayOptions message
* chooseFiltered method, add excluded parameter in populate method
* change excluded type to []dht.NodeID in ChooseFiltered, change comment
* change name filter to excluded_nodes in proto
* implement helper function contains
* delete ChooseFiltered and add its functionality into Choose method to keep original author's history, add excluded argument into Choose calls
* regenerate mock_client.go
* regenerate protobuf
* update test case to use new IDFromString function
* remove old kademlia test code
2018-10-15 13:42:36 -04:00
Dennis Coyle
dee2c137c8
Remove BKAD dependency from pkg/kademlia ( #294 )
...
* slowly but surely
* hardcode ID for tests so we can get predictable results
* skipping bad test
* removing tests of bkad
* wip
* new algorithm for worker
* clean up
* remove skipped test
* changes
* uncomment
* fixed conflicts
* maybe done ?
* cleanup
* boot bkad
* wip
* cleanup
* undo change
* fixes
* wip
* wip
* moving nodeID around
* wip
* wip
* fixes
* fixes after merge
* added TODO
* fixed tests post identity
* linter fixes
* wip
* PR review comments
* wip
* fixing tests
* fix tests
* force db directory
* bad test
* fixes race condition
* small cleanups
* adding db folder
* testing
* wip
* cleanup
* cleanup
* linters
* export Restrict
* add timeout
* testing
* linters
* forgot one
* moar fixes from master merge
* PR comments
* moar PR comments
* removed stun flag
* remove duplicate declaration
* remove old tests
* remove timeout
* fix tests
* missed one
* changed StringToID >> IDFromString
* PR comments
* stupid linter
* moevd overlay mock
* fixed merge conflicts
* fixes
* linter
2018-10-08 11:09:37 -04:00
Egon Elbre
b6b6111173
Flatten proto definitions into a single package ( #360 )
...
* protos: move streams to pb
* protos: move overlay to pb
* protos: move pointerdb to pb
* protos: move piecestore to pb
* fix statdb import naming
2018-09-18 07:39:06 +03:00
Egon Elbre
e7e2d4d7c9
use golangci-lint and add missing error checks ( #340 )
2018-09-11 16:13:25 +03:00
Egon Elbre
6ee6f0fdf5
Remove duplicated mock store implementations ( #337 )
2018-09-11 10:27:12 +03:00
Jennifer Li Johnson
d0f87f0de1
Overlay Bulk Lookup ( #279 )
...
* implements bulk node lookup from overlay cache
2018-09-11 00:52:14 -04:00
Egon Elbre
706a6ac26e
Use better test ports ( #311 )
...
Other interface or ports may be blocked or require permissions when running tests.
2018-09-07 12:42:04 +03:00
Egon Elbre
0f5a2f4ef5
Enable more linters ( #272 )
...
* enable more linters
* Run gofmt -s
* run goimports
* run unconvert
* fix naked return
* fix misspellings
* fix ineffectual assigments
* fix missing declaration
* don't use deprecated grpc.Errof
* check errors in tests
* run gofmt -w -r "assert.Nil(err) -> assert.NoError(err)"
* fix directory permissions
* don't use nil Context
* simplify boolean expressions
* use bytes.Equal instead of bytes.Compare
* merge variable declarations, remove redundant returns
* fix some golint errors
* run goimports
* handle more errors
* delete empty TestMain
* delete empty TestMain
* ignore examples for now
* fix lint errors
* remove unused values
* more fixes
* run gofmt -w -s .
* add more comments
* fix naming
* more lint fixes
* try switching travis to go1.11
* fix unnecessary conversions
* fix deprecated methods
* use go1.10 and disable gofmt/goimports for now
* switch to 1.10
* don't re-enable gofmt and goimports
* switch covermode to atomic because of -race
* gofmt
2018-08-27 11:28:16 -06:00
Maximillian von Briesen
553def620a
TLS Support ( #252 )
...
* begin adding tls
* remove incomplete line in gw/main.go
* identity fixes+:
+ fix `peertls.NewCert` public key issue
+ fix `peertls.verfiyChain` issue
+ fix identity dial option
+ rename `GenerateCA` to `NewCA` and `generateCAWorker` to `newCAWorker` for better consistency/convention
* use pdbclient instead of pointerdb in miniogw
* fix tests
* go fmt
* make review changes
* modify how context.Background() is used
* more context stuff
2018-08-23 22:01:03 -06:00
Kaloyan Raev
445044b856
Adjust Segment Store to the updated interface ( #160 )
...
* Adjust Segment Store to the updated interface
* Move /pkg/storage/segment to /pkg/storage/segments
* Fix overlay client tests
* Revert changes in NewOverlayClient return value
* Rename `rem` to `seg`
* Implement Meta()
2018-07-23 14:05:02 -06:00
Bryan White
5b913c45b9
Transport security ( #63 )
...
* wip initial transport security
* wip: transport security (add tests / refactor)
* wip tests
* refactoring - still wip
* refactor, improve tests
* wip tls testing
* fix typo
* wip testing
* wip testing
* wip
* tls_test passing
* code-style improvemente / refactor; service and tls tests passing!
* code-style auto-format
* add TestNewServer_LoadTLS
* refactor; test improvements
* refactor
* add client cert
* port changes
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* files created
* Merge remote-tracking branch 'upstream/master' into coyle/kad-tests
* wip
* add separate `Process` tests for bolt and redis-backed overlay
* more testing
* fix gitignore
* fix linter error
* goimports goimports GOIMPORTS GoImPortS!!!!
* wip
* fix port madness
* forgot to add
* add `mux` as handler and shorten context timeouts
* gofreakingimports
* fix comments
* refactor test & add logger/monkit registry
* debugging travis
* add comment
* Set redisAddress to empty string for bolt-test
* Merge remote-tracking branch 'upstream/master' into coyle/kad-tests
* Merge branch 'tls' into tls-upstream
* tls:
add client cert
refactor
refactor; test improvements
add TestNewServer_LoadTLS
code-style auto-format
code-style improvemente / refactor; service and tls tests passing!
tls_test passing
wip
wip testing
wip testing
fix typo
wip tls testing
refactor, improve tests
refactoring - still wip
wip tests
wip: transport security (add tests / refactor)
wip initial transport security
* fixing linter things
* wip
* remove bkad dependencie from tests
* wip
* wip
* wip
* wip
* wip
* updated coyle/kademlia
* wip
* cleanup
* ports
* overlay upgraded
* linter fixes
* piecestore kademlia newID
* Merge branch 'master' into tls-upstream
* master:
Add error to the return values of Ranger.Range method (#90 )
udp-forwarding: demo week work! (#84 )
* Merge branch 'kad-tests' into tls-upstream
* kad-tests:
piecestore kademlia newID
linter fixes
overlay upgraded
ports
cleanup
wip
updated coyle/kademlia
wip
wip
wip
wip
wip
remove bkad dependencie from tests
wip
wip
files created
port changes
* wip
* finish merging service tests
* add test for different client/server certs
* wip
* Merge branch 'master' into tls-upstream
* master:
Add context to Ranger.Range method (#99 )
Coyle/kad client (#91 )
* wip
* wip; refactoring/cleanup
* wip
* Merge branch 'master' into tls
* master:
Bolt backed overlay cache (#94 )
internal/test: switch errors to error classes (#96 )
* wip - test passing
* cleanup
* remove port.go
* cleanup
* Merge branch 'master' into tls
* master:
hardcode version (#111 )
Coyle/docker fix (#109 )
pkg/kademlia tests and restructuring (#97 )
Use continue instead of return in table tests (#106 )
prepend storjlabs to docker tag (#108 )
Automatically build, tag and push docker images on merge to master (#103 )
* more belated merging
* more belated merging
* more belated merging
* add copyrights
* cleanup
* goimports
* refactoring
* wip
* wip
* implement `TLSFileOptions#loadTLS`, refactoring:
`peertls.TestNewClient_LoadTLS` is the failing holdout; Still trying to figure out why I'm getting ECDSA verification is failing.
* not sure if actually working:
Tests are now passing (no more "ECDSA verification failed"); however,
`len(*tls.Certificates.Certificate) == 1` which I don't think should be
the case if the root and leaf are being created correctly.
* Experimenting/bugfixing?:
I think leaf certs should be properly signed by the parent now but not
entirely certain. It's also unclear to me why in
`VerifyPeerCertificate`, `len(rawCerts) == 1` when the certs should
contain both the root and leaf afaik.
* Properly write/read certificate chain (root/leaf):
I think I'm now properly reading and writing the root and leaf
certificate chain such that they're both being received by
`VerifyPeerCertificate`.
The next step is to parse the certificates with `x509.ParseCertificate`
(or similar) and verify that the public keys and signatures match.
* Add tls certificate chain signature veification (spike):
+ `VerifyPeerCertificate` verifies signatures of certificates using the
key of it's parent if there is one; otherwise, it verifies the
certificate is self-signed
+ TODO: refactor
+ TODO: test
* refactoring `VerifyPeerCertificate`
* cleanup
* refactor
* Merge branch 'master' into tls
* master:
Remove some structural folders we don't seem to be using. (#125 )
license code with agplv3 (#126 )
Update .clabot (#124 )
added team memebers (#123 )
clabot file added (#121 )
ECClient (#110 )
docker image issue fixed (#118 )
Piecestore Farmer CLI (#92 )
Define Path type (#101 )
adds netstate pagination (#95 )
Transport Client (#89 )
Implement psclient interface (#107 )
pkg/process: start replacing pkg/process with cobra helpers (#98 )
protos/netstate: remove stuff we're not using (#100 )
adding coveralls / code coverage (#112 )
* responding to review feedback / cleanup / add copywrite headers
* suggestions
* realitive
* Merge pull request #1 from coyle/coyle/tls
suggestions
* remove unnecessary `_`s
* Merge branch 'tls' of github.com:bryanchriswhite/storj into tls
* 'tls' of github.com:bryanchriswhite/storj:
realitive
suggestions
* Responding to review feedback:
+ refactor `VerifyPeerCertificate`
* remove tls expiration
* remove "hosts" and "clien option" from tls options
* goimports
* linter fixes
2018-07-09 19:43:13 +02:00
Dennis Coyle
9e40141cbc
Coyle/kad client ( #91 )
...
* port changes
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Restructure + additional interface
* Add NewOverlayClient
* added test for interface
* PR comments addressed
* lint issue
* added generated protobuf
* fixed merge conflicts
* adding comments
* PR comments addressed
2018-06-19 10:00:15 -04:00
