storj/pkg/transport/transport_test.go

255 lines
6.3 KiB
Go
Raw Normal View History

2019-01-24 20:15:10 +00:00
// Copyright (C) 2019 Storj Labs, Inc.
Transport Client (#89) * port changes * Task monitor and setup merge from the staging * Restructure + additional interface * Add NewOverlayClient * integrated DHT client interface * added test for interface * PR comments addressed * lint issue * added generated protobuf * adding new interface * added the interface framework * deleted file * fixes compilation errors and integrates new dhtcclient interface * merged netstat latest changes and dht new interface chagnes * fixed the address's port * adding comments * PR comments addressed * netclient interface dial method added * rename and integrated transportclient with minio gateway * rename and code clean up * made changes based on the Dennis's changes on the kad-client * Code review comment changes based on kaloyan review comments * reverted the changes to be similar to master * removed unused file * renamed to transportclient * added the review changes * store the address of the client * updates per the code review comments, changes-> added error retry connection attempt logic, added error conditions including nil parameters * updated the test case to test the bad address passed condition * updated the code per code review comments * Bolt backed overlay cache (#94) * wip * add separate `Process` tests for bolt and redis-backed overlay * more testing * fix gitignore * fix linter error * goimports goimports GOIMPORTS GoImPortS!!!! * fix port madness * forgot to add * add `mux` as handler and shorten context timeouts * gofreakingimports * fix comments * refactor test & add logger/monkit registry * debugging travis * add comment * Set redisAddress to empty string for bolt-test * travis experiment * refactoring tests * Merge remote-tracking branch 'upstream/master' into bolt-backed-overlay-cache * Automatically build, tag and push docker images on merge to master (#103) * port changes * build overlay on successful merge to master * fixes to Makefile * permissions * dep ensure * gopath * let's try vgo * remove dep * maybe alpine is the issue * tagging go version on build * stupid vgo * vgo * adding tags to push * quotes * local linting fixes & stupid travis * prepend storjlabs to docker tag (#108) * port changes * fixing tag name * Use continue instead of return in table tests (#106) I did a dumb mistake for some of the table tests, which made some of the test cases not being executed. * pkg/kademlia tests and restructuring (#97) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * files created * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * remove bkad dependencie from tests * wip * wip * wip * wip * wip * updated coyle/kademlia * wip * cleanup * ports * overlay upgraded * linter fixes * piecestore kademlia newID * add changes from kad demo * PR comments addresses * go func * force travis build * fixed merge conflicts * fixed merge conflicts * Merge branch 'coyle/kad-tests' of https://github.com/coyle/storj into coyle/kad-tests * linter issues * linting issues * fixed merge conflicts * linter is stupid * Coyle/docker fix (#109) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge branch 'master' of https://github.com/storj/storj * fixing tag name * no idea * testing * changes * testing on travis * testing * changes to travis build * new approach * Merge branch 'master' into coyle/docker-fix * hardcode version (#111) * hardcode version * adding coveralls / code coverage (#112) * adding coveralls * adding code coverage badge * fixing badges * verbose * swap tests and coverage * extra line * maybe * maybe * moar * gover maybe * testing * cleanup * protos/netstate: remove stuff we're not using (#100) * protos/netstate: remove stuff we're not using * protos/netstate: add metadata field for segmentstore * fix netstate client test * pkg/process: start replacing pkg/process with cobra helpers (#98) * Implement psclient interface (#107) * Implement psclient interface * Add string method to pieceID type * try to fix linter errors * Whoops missed an error * More linter errors * Typo * Lol double typo * Get everything working, begin adding tests for psclient rpc * goimports * Forgot to change the piecestore cli when changed the piecestore code * Fix CLI * remove ID length, added validator to pieceID * Move grpc ranger to client Change client PUT api to take a reader rather than return a writer * GRPCRanger -> PieceRanger; Make PieceRanger a RangeCloser * Forgot to remove offset * Added message upon successful store * Do that thing dennis and kaloyan wanted * goimports * Make closeConn a part of the interface for psclient * Use interface * Removed uneccessary new lines * goimport * Whoops * Actually we don't want to use the interface in Piece Ranger * Renamed piecestore in examples to piecestore-client; moved piecestore-cli to examples * Make comments look nicer * modified transport client based on the the design discussion * modified transport client based on the the design discussion * added the as discussed connection cache interface functionality * added the as discussed connection cache interface functionality * transport client changes * transport client per code review changes * per the code review comments * transport client incorporates review comments * fixes lint warnings * lint warning fixes....client interface has to be Client * client.go changes * transport.go changes * added test case * added test cases * comment fix * comment fix
2018-06-29 19:28:06 +01:00
// See LICENSE for copying information.
2018-12-14 10:44:48 +00:00
package transport_test
Transport Client (#89) * port changes * Task monitor and setup merge from the staging * Restructure + additional interface * Add NewOverlayClient * integrated DHT client interface * added test for interface * PR comments addressed * lint issue * added generated protobuf * adding new interface * added the interface framework * deleted file * fixes compilation errors and integrates new dhtcclient interface * merged netstat latest changes and dht new interface chagnes * fixed the address's port * adding comments * PR comments addressed * netclient interface dial method added * rename and integrated transportclient with minio gateway * rename and code clean up * made changes based on the Dennis's changes on the kad-client * Code review comment changes based on kaloyan review comments * reverted the changes to be similar to master * removed unused file * renamed to transportclient * added the review changes * store the address of the client * updates per the code review comments, changes-> added error retry connection attempt logic, added error conditions including nil parameters * updated the test case to test the bad address passed condition * updated the code per code review comments * Bolt backed overlay cache (#94) * wip * add separate `Process` tests for bolt and redis-backed overlay * more testing * fix gitignore * fix linter error * goimports goimports GOIMPORTS GoImPortS!!!! * fix port madness * forgot to add * add `mux` as handler and shorten context timeouts * gofreakingimports * fix comments * refactor test & add logger/monkit registry * debugging travis * add comment * Set redisAddress to empty string for bolt-test * travis experiment * refactoring tests * Merge remote-tracking branch 'upstream/master' into bolt-backed-overlay-cache * Automatically build, tag and push docker images on merge to master (#103) * port changes * build overlay on successful merge to master * fixes to Makefile * permissions * dep ensure * gopath * let's try vgo * remove dep * maybe alpine is the issue * tagging go version on build * stupid vgo * vgo * adding tags to push * quotes * local linting fixes & stupid travis * prepend storjlabs to docker tag (#108) * port changes * fixing tag name * Use continue instead of return in table tests (#106) I did a dumb mistake for some of the table tests, which made some of the test cases not being executed. * pkg/kademlia tests and restructuring (#97) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * files created * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * remove bkad dependencie from tests * wip * wip * wip * wip * wip * updated coyle/kademlia * wip * cleanup * ports * overlay upgraded * linter fixes * piecestore kademlia newID * add changes from kad demo * PR comments addresses * go func * force travis build * fixed merge conflicts * fixed merge conflicts * Merge branch 'coyle/kad-tests' of https://github.com/coyle/storj into coyle/kad-tests * linter issues * linting issues * fixed merge conflicts * linter is stupid * Coyle/docker fix (#109) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge branch 'master' of https://github.com/storj/storj * fixing tag name * no idea * testing * changes * testing on travis * testing * changes to travis build * new approach * Merge branch 'master' into coyle/docker-fix * hardcode version (#111) * hardcode version * adding coveralls / code coverage (#112) * adding coveralls * adding code coverage badge * fixing badges * verbose * swap tests and coverage * extra line * maybe * maybe * moar * gover maybe * testing * cleanup * protos/netstate: remove stuff we're not using (#100) * protos/netstate: remove stuff we're not using * protos/netstate: add metadata field for segmentstore * fix netstate client test * pkg/process: start replacing pkg/process with cobra helpers (#98) * Implement psclient interface (#107) * Implement psclient interface * Add string method to pieceID type * try to fix linter errors * Whoops missed an error * More linter errors * Typo * Lol double typo * Get everything working, begin adding tests for psclient rpc * goimports * Forgot to change the piecestore cli when changed the piecestore code * Fix CLI * remove ID length, added validator to pieceID * Move grpc ranger to client Change client PUT api to take a reader rather than return a writer * GRPCRanger -> PieceRanger; Make PieceRanger a RangeCloser * Forgot to remove offset * Added message upon successful store * Do that thing dennis and kaloyan wanted * goimports * Make closeConn a part of the interface for psclient * Use interface * Removed uneccessary new lines * goimport * Whoops * Actually we don't want to use the interface in Piece Ranger * Renamed piecestore in examples to piecestore-client; moved piecestore-cli to examples * Make comments look nicer * modified transport client based on the the design discussion * modified transport client based on the the design discussion * added the as discussed connection cache interface functionality * added the as discussed connection cache interface functionality * transport client changes * transport client per code review changes * per the code review comments * transport client incorporates review comments * fixes lint warnings * lint warning fixes....client interface has to be Client * client.go changes * transport.go changes * added test case * added test cases * comment fix * comment fix
2018-06-29 19:28:06 +01:00
import (
"context"
2018-12-14 10:44:48 +00:00
"fmt"
Transport Client (#89) * port changes * Task monitor and setup merge from the staging * Restructure + additional interface * Add NewOverlayClient * integrated DHT client interface * added test for interface * PR comments addressed * lint issue * added generated protobuf * adding new interface * added the interface framework * deleted file * fixes compilation errors and integrates new dhtcclient interface * merged netstat latest changes and dht new interface chagnes * fixed the address's port * adding comments * PR comments addressed * netclient interface dial method added * rename and integrated transportclient with minio gateway * rename and code clean up * made changes based on the Dennis's changes on the kad-client * Code review comment changes based on kaloyan review comments * reverted the changes to be similar to master * removed unused file * renamed to transportclient * added the review changes * store the address of the client * updates per the code review comments, changes-> added error retry connection attempt logic, added error conditions including nil parameters * updated the test case to test the bad address passed condition * updated the code per code review comments * Bolt backed overlay cache (#94) * wip * add separate `Process` tests for bolt and redis-backed overlay * more testing * fix gitignore * fix linter error * goimports goimports GOIMPORTS GoImPortS!!!! * fix port madness * forgot to add * add `mux` as handler and shorten context timeouts * gofreakingimports * fix comments * refactor test & add logger/monkit registry * debugging travis * add comment * Set redisAddress to empty string for bolt-test * travis experiment * refactoring tests * Merge remote-tracking branch 'upstream/master' into bolt-backed-overlay-cache * Automatically build, tag and push docker images on merge to master (#103) * port changes * build overlay on successful merge to master * fixes to Makefile * permissions * dep ensure * gopath * let's try vgo * remove dep * maybe alpine is the issue * tagging go version on build * stupid vgo * vgo * adding tags to push * quotes * local linting fixes & stupid travis * prepend storjlabs to docker tag (#108) * port changes * fixing tag name * Use continue instead of return in table tests (#106) I did a dumb mistake for some of the table tests, which made some of the test cases not being executed. * pkg/kademlia tests and restructuring (#97) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * files created * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * remove bkad dependencie from tests * wip * wip * wip * wip * wip * updated coyle/kademlia * wip * cleanup * ports * overlay upgraded * linter fixes * piecestore kademlia newID * add changes from kad demo * PR comments addresses * go func * force travis build * fixed merge conflicts * fixed merge conflicts * Merge branch 'coyle/kad-tests' of https://github.com/coyle/storj into coyle/kad-tests * linter issues * linting issues * fixed merge conflicts * linter is stupid * Coyle/docker fix (#109) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge branch 'master' of https://github.com/storj/storj * fixing tag name * no idea * testing * changes * testing on travis * testing * changes to travis build * new approach * Merge branch 'master' into coyle/docker-fix * hardcode version (#111) * hardcode version * adding coveralls / code coverage (#112) * adding coveralls * adding code coverage badge * fixing badges * verbose * swap tests and coverage * extra line * maybe * maybe * moar * gover maybe * testing * cleanup * protos/netstate: remove stuff we're not using (#100) * protos/netstate: remove stuff we're not using * protos/netstate: add metadata field for segmentstore * fix netstate client test * pkg/process: start replacing pkg/process with cobra helpers (#98) * Implement psclient interface (#107) * Implement psclient interface * Add string method to pieceID type * try to fix linter errors * Whoops missed an error * More linter errors * Typo * Lol double typo * Get everything working, begin adding tests for psclient rpc * goimports * Forgot to change the piecestore cli when changed the piecestore code * Fix CLI * remove ID length, added validator to pieceID * Move grpc ranger to client Change client PUT api to take a reader rather than return a writer * GRPCRanger -> PieceRanger; Make PieceRanger a RangeCloser * Forgot to remove offset * Added message upon successful store * Do that thing dennis and kaloyan wanted * goimports * Make closeConn a part of the interface for psclient * Use interface * Removed uneccessary new lines * goimport * Whoops * Actually we don't want to use the interface in Piece Ranger * Renamed piecestore in examples to piecestore-client; moved piecestore-cli to examples * Make comments look nicer * modified transport client based on the the design discussion * modified transport client based on the the design discussion * added the as discussed connection cache interface functionality * added the as discussed connection cache interface functionality * transport client changes * transport client per code review changes * per the code review comments * transport client incorporates review comments * fixes lint warnings * lint warning fixes....client interface has to be Client * client.go changes * transport.go changes * added test case * added test cases * comment fix * comment fix
2018-06-29 19:28:06 +01:00
"testing"
2018-12-14 10:44:48 +00:00
"time"
Transport Client (#89) * port changes * Task monitor and setup merge from the staging * Restructure + additional interface * Add NewOverlayClient * integrated DHT client interface * added test for interface * PR comments addressed * lint issue * added generated protobuf * adding new interface * added the interface framework * deleted file * fixes compilation errors and integrates new dhtcclient interface * merged netstat latest changes and dht new interface chagnes * fixed the address's port * adding comments * PR comments addressed * netclient interface dial method added * rename and integrated transportclient with minio gateway * rename and code clean up * made changes based on the Dennis's changes on the kad-client * Code review comment changes based on kaloyan review comments * reverted the changes to be similar to master * removed unused file * renamed to transportclient * added the review changes * store the address of the client * updates per the code review comments, changes-> added error retry connection attempt logic, added error conditions including nil parameters * updated the test case to test the bad address passed condition * updated the code per code review comments * Bolt backed overlay cache (#94) * wip * add separate `Process` tests for bolt and redis-backed overlay * more testing * fix gitignore * fix linter error * goimports goimports GOIMPORTS GoImPortS!!!! * fix port madness * forgot to add * add `mux` as handler and shorten context timeouts * gofreakingimports * fix comments * refactor test & add logger/monkit registry * debugging travis * add comment * Set redisAddress to empty string for bolt-test * travis experiment * refactoring tests * Merge remote-tracking branch 'upstream/master' into bolt-backed-overlay-cache * Automatically build, tag and push docker images on merge to master (#103) * port changes * build overlay on successful merge to master * fixes to Makefile * permissions * dep ensure * gopath * let's try vgo * remove dep * maybe alpine is the issue * tagging go version on build * stupid vgo * vgo * adding tags to push * quotes * local linting fixes & stupid travis * prepend storjlabs to docker tag (#108) * port changes * fixing tag name * Use continue instead of return in table tests (#106) I did a dumb mistake for some of the table tests, which made some of the test cases not being executed. * pkg/kademlia tests and restructuring (#97) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * files created * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * remove bkad dependencie from tests * wip * wip * wip * wip * wip * updated coyle/kademlia * wip * cleanup * ports * overlay upgraded * linter fixes * piecestore kademlia newID * add changes from kad demo * PR comments addresses * go func * force travis build * fixed merge conflicts * fixed merge conflicts * Merge branch 'coyle/kad-tests' of https://github.com/coyle/storj into coyle/kad-tests * linter issues * linting issues * fixed merge conflicts * linter is stupid * Coyle/docker fix (#109) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge branch 'master' of https://github.com/storj/storj * fixing tag name * no idea * testing * changes * testing on travis * testing * changes to travis build * new approach * Merge branch 'master' into coyle/docker-fix * hardcode version (#111) * hardcode version * adding coveralls / code coverage (#112) * adding coveralls * adding code coverage badge * fixing badges * verbose * swap tests and coverage * extra line * maybe * maybe * moar * gover maybe * testing * cleanup * protos/netstate: remove stuff we're not using (#100) * protos/netstate: remove stuff we're not using * protos/netstate: add metadata field for segmentstore * fix netstate client test * pkg/process: start replacing pkg/process with cobra helpers (#98) * Implement psclient interface (#107) * Implement psclient interface * Add string method to pieceID type * try to fix linter errors * Whoops missed an error * More linter errors * Typo * Lol double typo * Get everything working, begin adding tests for psclient rpc * goimports * Forgot to change the piecestore cli when changed the piecestore code * Fix CLI * remove ID length, added validator to pieceID * Move grpc ranger to client Change client PUT api to take a reader rather than return a writer * GRPCRanger -> PieceRanger; Make PieceRanger a RangeCloser * Forgot to remove offset * Added message upon successful store * Do that thing dennis and kaloyan wanted * goimports * Make closeConn a part of the interface for psclient * Use interface * Removed uneccessary new lines * goimport * Whoops * Actually we don't want to use the interface in Piece Ranger * Renamed piecestore in examples to piecestore-client; moved piecestore-cli to examples * Make comments look nicer * modified transport client based on the the design discussion * modified transport client based on the the design discussion * added the as discussed connection cache interface functionality * added the as discussed connection cache interface functionality * transport client changes * transport client per code review changes * per the code review comments * transport client incorporates review comments * fixes lint warnings * lint warning fixes....client interface has to be Client * client.go changes * transport.go changes * added test case * added test cases * comment fix * comment fix
2018-06-29 19:28:06 +01:00
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/zap"
2018-12-14 10:44:48 +00:00
"storj.io/storj/internal/testcontext"
2019-04-08 19:15:19 +01:00
"storj.io/storj/internal/testidentity"
2018-12-14 10:44:48 +00:00
"storj.io/storj/internal/testplanet"
"storj.io/storj/pkg/pb"
"storj.io/storj/pkg/peertls/tlsopts"
2018-12-14 10:44:48 +00:00
"storj.io/storj/pkg/storj"
Transport Client (#89) * port changes * Task monitor and setup merge from the staging * Restructure + additional interface * Add NewOverlayClient * integrated DHT client interface * added test for interface * PR comments addressed * lint issue * added generated protobuf * adding new interface * added the interface framework * deleted file * fixes compilation errors and integrates new dhtcclient interface * merged netstat latest changes and dht new interface chagnes * fixed the address's port * adding comments * PR comments addressed * netclient interface dial method added * rename and integrated transportclient with minio gateway * rename and code clean up * made changes based on the Dennis's changes on the kad-client * Code review comment changes based on kaloyan review comments * reverted the changes to be similar to master * removed unused file * renamed to transportclient * added the review changes * store the address of the client * updates per the code review comments, changes-> added error retry connection attempt logic, added error conditions including nil parameters * updated the test case to test the bad address passed condition * updated the code per code review comments * Bolt backed overlay cache (#94) * wip * add separate `Process` tests for bolt and redis-backed overlay * more testing * fix gitignore * fix linter error * goimports goimports GOIMPORTS GoImPortS!!!! * fix port madness * forgot to add * add `mux` as handler and shorten context timeouts * gofreakingimports * fix comments * refactor test & add logger/monkit registry * debugging travis * add comment * Set redisAddress to empty string for bolt-test * travis experiment * refactoring tests * Merge remote-tracking branch 'upstream/master' into bolt-backed-overlay-cache * Automatically build, tag and push docker images on merge to master (#103) * port changes * build overlay on successful merge to master * fixes to Makefile * permissions * dep ensure * gopath * let's try vgo * remove dep * maybe alpine is the issue * tagging go version on build * stupid vgo * vgo * adding tags to push * quotes * local linting fixes & stupid travis * prepend storjlabs to docker tag (#108) * port changes * fixing tag name * Use continue instead of return in table tests (#106) I did a dumb mistake for some of the table tests, which made some of the test cases not being executed. * pkg/kademlia tests and restructuring (#97) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * files created * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * remove bkad dependencie from tests * wip * wip * wip * wip * wip * updated coyle/kademlia * wip * cleanup * ports * overlay upgraded * linter fixes * piecestore kademlia newID * add changes from kad demo * PR comments addresses * go func * force travis build * fixed merge conflicts * fixed merge conflicts * Merge branch 'coyle/kad-tests' of https://github.com/coyle/storj into coyle/kad-tests * linter issues * linting issues * fixed merge conflicts * linter is stupid * Coyle/docker fix (#109) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge branch 'master' of https://github.com/storj/storj * fixing tag name * no idea * testing * changes * testing on travis * testing * changes to travis build * new approach * Merge branch 'master' into coyle/docker-fix * hardcode version (#111) * hardcode version * adding coveralls / code coverage (#112) * adding coveralls * adding code coverage badge * fixing badges * verbose * swap tests and coverage * extra line * maybe * maybe * moar * gover maybe * testing * cleanup * protos/netstate: remove stuff we're not using (#100) * protos/netstate: remove stuff we're not using * protos/netstate: add metadata field for segmentstore * fix netstate client test * pkg/process: start replacing pkg/process with cobra helpers (#98) * Implement psclient interface (#107) * Implement psclient interface * Add string method to pieceID type * try to fix linter errors * Whoops missed an error * More linter errors * Typo * Lol double typo * Get everything working, begin adding tests for psclient rpc * goimports * Forgot to change the piecestore cli when changed the piecestore code * Fix CLI * remove ID length, added validator to pieceID * Move grpc ranger to client Change client PUT api to take a reader rather than return a writer * GRPCRanger -> PieceRanger; Make PieceRanger a RangeCloser * Forgot to remove offset * Added message upon successful store * Do that thing dennis and kaloyan wanted * goimports * Make closeConn a part of the interface for psclient * Use interface * Removed uneccessary new lines * goimport * Whoops * Actually we don't want to use the interface in Piece Ranger * Renamed piecestore in examples to piecestore-client; moved piecestore-cli to examples * Make comments look nicer * modified transport client based on the the design discussion * modified transport client based on the the design discussion * added the as discussed connection cache interface functionality * added the as discussed connection cache interface functionality * transport client changes * transport client per code review changes * per the code review comments * transport client incorporates review comments * fixes lint warnings * lint warning fixes....client interface has to be Client * client.go changes * transport.go changes * added test case * added test cases * comment fix * comment fix
2018-06-29 19:28:06 +01:00
)
func TestDialNode(t *testing.T) {
2018-12-14 10:44:48 +00:00
ctx := testcontext.New(t)
defer ctx.Cleanup()
2018-12-14 10:44:48 +00:00
planet, err := testplanet.New(t, 0, 2, 0)
if err != nil {
t.Fatal(err)
}
2018-12-14 10:44:48 +00:00
defer ctx.Check(planet.Shutdown)
2019-04-08 19:15:19 +01:00
whitelistPath, err := planet.WriteWhitelist(storj.LatestIDVersion())
require.NoError(t, err)
2018-12-14 10:44:48 +00:00
planet.Start(ctx)
client := planet.StorageNodes[0].Transport
Transport Client (#89) * port changes * Task monitor and setup merge from the staging * Restructure + additional interface * Add NewOverlayClient * integrated DHT client interface * added test for interface * PR comments addressed * lint issue * added generated protobuf * adding new interface * added the interface framework * deleted file * fixes compilation errors and integrates new dhtcclient interface * merged netstat latest changes and dht new interface chagnes * fixed the address's port * adding comments * PR comments addressed * netclient interface dial method added * rename and integrated transportclient with minio gateway * rename and code clean up * made changes based on the Dennis's changes on the kad-client * Code review comment changes based on kaloyan review comments * reverted the changes to be similar to master * removed unused file * renamed to transportclient * added the review changes * store the address of the client * updates per the code review comments, changes-> added error retry connection attempt logic, added error conditions including nil parameters * updated the test case to test the bad address passed condition * updated the code per code review comments * Bolt backed overlay cache (#94) * wip * add separate `Process` tests for bolt and redis-backed overlay * more testing * fix gitignore * fix linter error * goimports goimports GOIMPORTS GoImPortS!!!! * fix port madness * forgot to add * add `mux` as handler and shorten context timeouts * gofreakingimports * fix comments * refactor test & add logger/monkit registry * debugging travis * add comment * Set redisAddress to empty string for bolt-test * travis experiment * refactoring tests * Merge remote-tracking branch 'upstream/master' into bolt-backed-overlay-cache * Automatically build, tag and push docker images on merge to master (#103) * port changes * build overlay on successful merge to master * fixes to Makefile * permissions * dep ensure * gopath * let's try vgo * remove dep * maybe alpine is the issue * tagging go version on build * stupid vgo * vgo * adding tags to push * quotes * local linting fixes & stupid travis * prepend storjlabs to docker tag (#108) * port changes * fixing tag name * Use continue instead of return in table tests (#106) I did a dumb mistake for some of the table tests, which made some of the test cases not being executed. * pkg/kademlia tests and restructuring (#97) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * files created * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * remove bkad dependencie from tests * wip * wip * wip * wip * wip * updated coyle/kademlia * wip * cleanup * ports * overlay upgraded * linter fixes * piecestore kademlia newID * add changes from kad demo * PR comments addresses * go func * force travis build * fixed merge conflicts * fixed merge conflicts * Merge branch 'coyle/kad-tests' of https://github.com/coyle/storj into coyle/kad-tests * linter issues * linting issues * fixed merge conflicts * linter is stupid * Coyle/docker fix (#109) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge branch 'master' of https://github.com/storj/storj * fixing tag name * no idea * testing * changes * testing on travis * testing * changes to travis build * new approach * Merge branch 'master' into coyle/docker-fix * hardcode version (#111) * hardcode version * adding coveralls / code coverage (#112) * adding coveralls * adding code coverage badge * fixing badges * verbose * swap tests and coverage * extra line * maybe * maybe * moar * gover maybe * testing * cleanup * protos/netstate: remove stuff we're not using (#100) * protos/netstate: remove stuff we're not using * protos/netstate: add metadata field for segmentstore * fix netstate client test * pkg/process: start replacing pkg/process with cobra helpers (#98) * Implement psclient interface (#107) * Implement psclient interface * Add string method to pieceID type * try to fix linter errors * Whoops missed an error * More linter errors * Typo * Lol double typo * Get everything working, begin adding tests for psclient rpc * goimports * Forgot to change the piecestore cli when changed the piecestore code * Fix CLI * remove ID length, added validator to pieceID * Move grpc ranger to client Change client PUT api to take a reader rather than return a writer * GRPCRanger -> PieceRanger; Make PieceRanger a RangeCloser * Forgot to remove offset * Added message upon successful store * Do that thing dennis and kaloyan wanted * goimports * Make closeConn a part of the interface for psclient * Use interface * Removed uneccessary new lines * goimport * Whoops * Actually we don't want to use the interface in Piece Ranger * Renamed piecestore in examples to piecestore-client; moved piecestore-cli to examples * Make comments look nicer * modified transport client based on the the design discussion * modified transport client based on the the design discussion * added the as discussed connection cache interface functionality * added the as discussed connection cache interface functionality * transport client changes * transport client per code review changes * per the code review comments * transport client incorporates review comments * fixes lint warnings * lint warning fixes....client interface has to be Client * client.go changes * transport.go changes * added test case * added test cases * comment fix * comment fix
2018-06-29 19:28:06 +01:00
2019-04-08 19:15:19 +01:00
unsignedIdent, err := testidentity.PregeneratedIdentity(0, storj.LatestIDVersion())
require.NoError(t, err)
2019-04-08 19:15:19 +01:00
signedIdent, err := testidentity.PregeneratedSignedIdentity(0, storj.LatestIDVersion())
require.NoError(t, err)
opts, err := tlsopts.NewOptions(signedIdent, tlsopts.Config{
UsePeerCAWhitelist: true,
PeerCAWhitelistPath: whitelistPath,
2019-04-09 18:01:45 +01:00
PeerIDVersions: "*",
})
require.NoError(t, err)
2019-04-08 19:15:19 +01:00
unsignedClientOpts, err := tlsopts.NewOptions(unsignedIdent, tlsopts.Config{
2019-04-09 18:01:45 +01:00
PeerIDVersions: "*",
2019-04-08 19:15:19 +01:00
})
require.NoError(t, err)
t.Run("DialNode with invalid targets", func(t *testing.T) {
2018-12-14 10:44:48 +00:00
targets := []*pb.Node{
{
Id: storj.NodeID{},
Address: nil,
},
{
Id: storj.NodeID{},
Address: &pb.NodeAddress{
Transport: pb.NodeTransport_TCP_TLS_GRPC,
},
},
{
Id: storj.NodeID{123},
Address: &pb.NodeAddress{
Transport: pb.NodeTransport_TCP_TLS_GRPC,
Address: "127.0.0.1:100",
},
},
{
Id: storj.NodeID{},
Address: &pb.NodeAddress{
Transport: pb.NodeTransport_TCP_TLS_GRPC,
Address: planet.StorageNodes[1].Addr(),
},
},
2018-12-14 10:44:48 +00:00
}
for _, target := range targets {
tag := fmt.Sprintf("%+v", target)
timedCtx, cancel := context.WithTimeout(ctx, time.Second)
conn, err := client.DialNode(timedCtx, target)
2018-12-14 10:44:48 +00:00
cancel()
assert.Error(t, err, tag)
assert.Nil(t, conn, tag)
}
})
2018-12-14 10:44:48 +00:00
t.Run("DialNode with valid target", func(t *testing.T) {
target := &pb.Node{
2018-12-14 10:44:48 +00:00
Id: planet.StorageNodes[1].ID(),
Address: &pb.NodeAddress{
Transport: pb.NodeTransport_TCP_TLS_GRPC,
Address: planet.StorageNodes[1].Addr(),
},
}
timedCtx, cancel := context.WithTimeout(ctx, time.Second)
conn, err := client.DialNode(timedCtx, target)
cancel()
assert.NoError(t, err)
require.NotNil(t, conn)
assert.NoError(t, conn.Close())
})
t.Run("DialNode with valid signed target", func(t *testing.T) {
target := &pb.Node{
Id: planet.StorageNodes[1].ID(),
Address: &pb.NodeAddress{
Transport: pb.NodeTransport_TCP_TLS_GRPC,
Address: planet.StorageNodes[1].Addr(),
},
}
dialOption, err := opts.DialOption(target.Id)
require.NoError(t, err)
timedCtx, cancel := context.WithTimeout(ctx, time.Second)
conn, err := client.DialNode(timedCtx, target, dialOption)
2018-12-14 10:44:48 +00:00
cancel()
assert.NoError(t, err)
require.NotNil(t, conn)
2018-12-14 10:44:48 +00:00
assert.NoError(t, conn.Close())
})
2018-12-14 10:44:48 +00:00
t.Run("DialNode with unsigned identity", func(t *testing.T) {
target := &pb.Node{
Id: planet.StorageNodes[1].ID(),
Address: &pb.NodeAddress{
Transport: pb.NodeTransport_TCP_TLS_GRPC,
Address: planet.StorageNodes[1].Addr(),
},
}
timedCtx, cancel := context.WithTimeout(ctx, time.Second)
dialOption, err := unsignedClientOpts.DialOption(target.Id)
require.NoError(t, err)
conn, err := client.DialNode(
timedCtx, target, dialOption,
)
cancel()
assert.NotNil(t, conn)
require.NoError(t, err)
assert.NoError(t, conn.Close())
})
t.Run("DialAddress with unsigned identity", func(t *testing.T) {
timedCtx, cancel := context.WithTimeout(ctx, time.Second)
dialOption := unsignedClientOpts.DialUnverifiedIDOption()
conn, err := client.DialAddress(
timedCtx, planet.StorageNodes[1].Addr(), dialOption,
)
cancel()
assert.NotNil(t, conn)
require.NoError(t, err)
assert.NoError(t, conn.Close())
})
t.Run("DialAddress with valid address", func(t *testing.T) {
2018-12-14 10:44:48 +00:00
timedCtx, cancel := context.WithTimeout(ctx, time.Second)
conn, err := client.DialAddress(timedCtx, planet.StorageNodes[1].Addr())
2018-12-14 10:44:48 +00:00
cancel()
assert.NoError(t, err)
require.NotNil(t, conn)
2018-12-14 10:44:48 +00:00
assert.NoError(t, conn.Close())
})
}
func TestDialNode_BadServerCertificate(t *testing.T) {
ctx := testcontext.New(t)
defer ctx.Cleanup()
planet, err := testplanet.NewCustom(
zap.L(),
testplanet.Config{
SatelliteCount: 0,
StorageNodeCount: 2,
UplinkCount: 0,
Reconfigure: testplanet.DisablePeerCAWhitelist,
2019-04-08 19:15:19 +01:00
Identities: testidentity.NewPregeneratedIdentities(storj.LatestIDVersion()),
},
)
if err != nil {
t.Fatal(err)
Transport Client (#89) * port changes * Task monitor and setup merge from the staging * Restructure + additional interface * Add NewOverlayClient * integrated DHT client interface * added test for interface * PR comments addressed * lint issue * added generated protobuf * adding new interface * added the interface framework * deleted file * fixes compilation errors and integrates new dhtcclient interface * merged netstat latest changes and dht new interface chagnes * fixed the address's port * adding comments * PR comments addressed * netclient interface dial method added * rename and integrated transportclient with minio gateway * rename and code clean up * made changes based on the Dennis's changes on the kad-client * Code review comment changes based on kaloyan review comments * reverted the changes to be similar to master * removed unused file * renamed to transportclient * added the review changes * store the address of the client * updates per the code review comments, changes-> added error retry connection attempt logic, added error conditions including nil parameters * updated the test case to test the bad address passed condition * updated the code per code review comments * Bolt backed overlay cache (#94) * wip * add separate `Process` tests for bolt and redis-backed overlay * more testing * fix gitignore * fix linter error * goimports goimports GOIMPORTS GoImPortS!!!! * fix port madness * forgot to add * add `mux` as handler and shorten context timeouts * gofreakingimports * fix comments * refactor test & add logger/monkit registry * debugging travis * add comment * Set redisAddress to empty string for bolt-test * travis experiment * refactoring tests * Merge remote-tracking branch 'upstream/master' into bolt-backed-overlay-cache * Automatically build, tag and push docker images on merge to master (#103) * port changes * build overlay on successful merge to master * fixes to Makefile * permissions * dep ensure * gopath * let's try vgo * remove dep * maybe alpine is the issue * tagging go version on build * stupid vgo * vgo * adding tags to push * quotes * local linting fixes & stupid travis * prepend storjlabs to docker tag (#108) * port changes * fixing tag name * Use continue instead of return in table tests (#106) I did a dumb mistake for some of the table tests, which made some of the test cases not being executed. * pkg/kademlia tests and restructuring (#97) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * files created * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * remove bkad dependencie from tests * wip * wip * wip * wip * wip * updated coyle/kademlia * wip * cleanup * ports * overlay upgraded * linter fixes * piecestore kademlia newID * add changes from kad demo * PR comments addresses * go func * force travis build * fixed merge conflicts * fixed merge conflicts * Merge branch 'coyle/kad-tests' of https://github.com/coyle/storj into coyle/kad-tests * linter issues * linting issues * fixed merge conflicts * linter is stupid * Coyle/docker fix (#109) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge branch 'master' of https://github.com/storj/storj * fixing tag name * no idea * testing * changes * testing on travis * testing * changes to travis build * new approach * Merge branch 'master' into coyle/docker-fix * hardcode version (#111) * hardcode version * adding coveralls / code coverage (#112) * adding coveralls * adding code coverage badge * fixing badges * verbose * swap tests and coverage * extra line * maybe * maybe * moar * gover maybe * testing * cleanup * protos/netstate: remove stuff we're not using (#100) * protos/netstate: remove stuff we're not using * protos/netstate: add metadata field for segmentstore * fix netstate client test * pkg/process: start replacing pkg/process with cobra helpers (#98) * Implement psclient interface (#107) * Implement psclient interface * Add string method to pieceID type * try to fix linter errors * Whoops missed an error * More linter errors * Typo * Lol double typo * Get everything working, begin adding tests for psclient rpc * goimports * Forgot to change the piecestore cli when changed the piecestore code * Fix CLI * remove ID length, added validator to pieceID * Move grpc ranger to client Change client PUT api to take a reader rather than return a writer * GRPCRanger -> PieceRanger; Make PieceRanger a RangeCloser * Forgot to remove offset * Added message upon successful store * Do that thing dennis and kaloyan wanted * goimports * Make closeConn a part of the interface for psclient * Use interface * Removed uneccessary new lines * goimport * Whoops * Actually we don't want to use the interface in Piece Ranger * Renamed piecestore in examples to piecestore-client; moved piecestore-cli to examples * Make comments look nicer * modified transport client based on the the design discussion * modified transport client based on the the design discussion * added the as discussed connection cache interface functionality * added the as discussed connection cache interface functionality * transport client changes * transport client per code review changes * per the code review comments * transport client incorporates review comments * fixes lint warnings * lint warning fixes....client interface has to be Client * client.go changes * transport.go changes * added test case * added test cases * comment fix * comment fix
2018-06-29 19:28:06 +01:00
}
defer ctx.Check(planet.Shutdown)
2019-04-08 19:15:19 +01:00
whitelistPath, err := planet.WriteWhitelist(storj.LatestIDVersion())
require.NoError(t, err)
planet.Start(ctx)
client := planet.StorageNodes[0].Transport
2019-04-08 19:15:19 +01:00
ident, err := testidentity.PregeneratedSignedIdentity(0, storj.LatestIDVersion())
require.NoError(t, err)
opts, err := tlsopts.NewOptions(ident, tlsopts.Config{
UsePeerCAWhitelist: true,
PeerCAWhitelistPath: whitelistPath,
})
require.NoError(t, err)
t.Run("DialNode with bad server certificate", func(t *testing.T) {
target := &pb.Node{
Id: planet.StorageNodes[1].ID(),
Address: &pb.NodeAddress{
Transport: pb.NodeTransport_TCP_TLS_GRPC,
Address: planet.StorageNodes[1].Addr(),
},
}
timedCtx, cancel := context.WithTimeout(ctx, time.Second)
dialOption, err := opts.DialOption(target.Id)
require.NoError(t, err)
conn, err := client.DialNode(timedCtx, target, dialOption)
cancel()
tag := fmt.Sprintf("%+v", target)
assert.Nil(t, conn, tag)
require.Error(t, err, tag)
assert.Contains(t, err.Error(), "not signed by any CA in the whitelist")
})
t.Run("DialAddress with bad server certificate", func(t *testing.T) {
timedCtx, cancel := context.WithTimeout(ctx, time.Second)
dialOption, err := opts.DialOption(planet.StorageNodes[1].ID())
require.NoError(t, err)
conn, err := client.DialAddress(timedCtx, planet.StorageNodes[1].Addr(), dialOption)
cancel()
assert.Nil(t, conn)
require.Error(t, err)
assert.Contains(t, err.Error(), "not signed by any CA in the whitelist")
})
Transport Client (#89) * port changes * Task monitor and setup merge from the staging * Restructure + additional interface * Add NewOverlayClient * integrated DHT client interface * added test for interface * PR comments addressed * lint issue * added generated protobuf * adding new interface * added the interface framework * deleted file * fixes compilation errors and integrates new dhtcclient interface * merged netstat latest changes and dht new interface chagnes * fixed the address's port * adding comments * PR comments addressed * netclient interface dial method added * rename and integrated transportclient with minio gateway * rename and code clean up * made changes based on the Dennis's changes on the kad-client * Code review comment changes based on kaloyan review comments * reverted the changes to be similar to master * removed unused file * renamed to transportclient * added the review changes * store the address of the client * updates per the code review comments, changes-> added error retry connection attempt logic, added error conditions including nil parameters * updated the test case to test the bad address passed condition * updated the code per code review comments * Bolt backed overlay cache (#94) * wip * add separate `Process` tests for bolt and redis-backed overlay * more testing * fix gitignore * fix linter error * goimports goimports GOIMPORTS GoImPortS!!!! * fix port madness * forgot to add * add `mux` as handler and shorten context timeouts * gofreakingimports * fix comments * refactor test & add logger/monkit registry * debugging travis * add comment * Set redisAddress to empty string for bolt-test * travis experiment * refactoring tests * Merge remote-tracking branch 'upstream/master' into bolt-backed-overlay-cache * Automatically build, tag and push docker images on merge to master (#103) * port changes * build overlay on successful merge to master * fixes to Makefile * permissions * dep ensure * gopath * let's try vgo * remove dep * maybe alpine is the issue * tagging go version on build * stupid vgo * vgo * adding tags to push * quotes * local linting fixes & stupid travis * prepend storjlabs to docker tag (#108) * port changes * fixing tag name * Use continue instead of return in table tests (#106) I did a dumb mistake for some of the table tests, which made some of the test cases not being executed. * pkg/kademlia tests and restructuring (#97) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * files created * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * Merge remote-tracking branch 'upstream/master' into coyle/kad-tests * wip * remove bkad dependencie from tests * wip * wip * wip * wip * wip * updated coyle/kademlia * wip * cleanup * ports * overlay upgraded * linter fixes * piecestore kademlia newID * add changes from kad demo * PR comments addresses * go func * force travis build * fixed merge conflicts * fixed merge conflicts * Merge branch 'coyle/kad-tests' of https://github.com/coyle/storj into coyle/kad-tests * linter issues * linting issues * fixed merge conflicts * linter is stupid * Coyle/docker fix (#109) * port changes * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge remote-tracking branch 'upstream/master' * Merge branch 'master' of https://github.com/storj/storj * fixing tag name * no idea * testing * changes * testing on travis * testing * changes to travis build * new approach * Merge branch 'master' into coyle/docker-fix * hardcode version (#111) * hardcode version * adding coveralls / code coverage (#112) * adding coveralls * adding code coverage badge * fixing badges * verbose * swap tests and coverage * extra line * maybe * maybe * moar * gover maybe * testing * cleanup * protos/netstate: remove stuff we're not using (#100) * protos/netstate: remove stuff we're not using * protos/netstate: add metadata field for segmentstore * fix netstate client test * pkg/process: start replacing pkg/process with cobra helpers (#98) * Implement psclient interface (#107) * Implement psclient interface * Add string method to pieceID type * try to fix linter errors * Whoops missed an error * More linter errors * Typo * Lol double typo * Get everything working, begin adding tests for psclient rpc * goimports * Forgot to change the piecestore cli when changed the piecestore code * Fix CLI * remove ID length, added validator to pieceID * Move grpc ranger to client Change client PUT api to take a reader rather than return a writer * GRPCRanger -> PieceRanger; Make PieceRanger a RangeCloser * Forgot to remove offset * Added message upon successful store * Do that thing dennis and kaloyan wanted * goimports * Make closeConn a part of the interface for psclient * Use interface * Removed uneccessary new lines * goimport * Whoops * Actually we don't want to use the interface in Piece Ranger * Renamed piecestore in examples to piecestore-client; moved piecestore-cli to examples * Make comments look nicer * modified transport client based on the the design discussion * modified transport client based on the the design discussion * added the as discussed connection cache interface functionality * added the as discussed connection cache interface functionality * transport client changes * transport client per code review changes * per the code review comments * transport client incorporates review comments * fixes lint warnings * lint warning fixes....client interface has to be Client * client.go changes * transport.go changes * added test case * added test cases * comment fix * comment fix
2018-06-29 19:28:06 +01:00
}