Bryan White
fde0020c68
dial node/address with bad server cert ( #1342 )
2019-02-26 19:35:16 +01:00
Bryan White
b652b6a522
use peer ca whitelist in testplanet ( #1337 )
2019-02-25 09:38:03 +02:00
Natalie Villasana
c3d3f41d30
removes some SignedMessage use ( #1258 )
...
Removes most instances of pb.SignedMessage (there's more to take out but they shouldn't hurt anyone as is).
There used to be places in psserver where a PieceID was hmac'd with the SatelliteID, which was gotten from a SignedMessage. This PR makes it so some functions access the SatelliteID from the Payer Bandwidth Allocation instead.
This requires passing a SatelliteID into psserver functions where they weren't before, so the following proto messages have been changed:
* PieceId - satellite_id field added
This is so the psserver.Piece function has access to the SatelliteID when it needs to get the namespaced pieceID.
This proto message should probably be renamed to PieceRequest, or a new PieceRequest message should be created so this isn't misnamed.
* PieceDelete - satellite_id field added
This is so the psserver.Delete function has access to the SatelliteID when receiving a request to Delete.
2019-02-19 23:36:08 -06:00
Michal Niewrzal
8d685217e4
Storagenode migrations ( #1299 )
...
* creates initial migration for psdb
* add test mechanism to validate migration to every version
* fix few small issues in versions.go and context.go
2019-02-19 10:39:04 +01:00
Egon Elbre
12261c0624
Satellite migrations ( #1301 )
2019-02-14 23:55:21 +02:00
Egon Elbre
497fb756fe
create dbutil package for migration testing ( #1305 )
2019-02-13 18:06:34 +02:00
JT Olio
2a59679766
pkg/transport: require tls configuration for dialing ( #1286 )
...
* separate TLS options from server options (because we need them for dialing too)
* stop creating transports in multiple places
* ensure that we actually check revocation, whitelists, certificate signing, etc, for all connections.
2019-02-11 13:17:32 +02:00
JT Olio
ea28a9a581
pkg/transport: fail on bad certificates ( #1285 )
2019-02-11 13:02:49 +02:00
Egon Elbre
17a7f9d002
Enable planet Merging test ( #1281 )
2019-02-08 22:35:59 +02:00
Egon Elbre
12bdc0ef21
Disable revocation table for testplanet. ( #1283 )
2019-02-08 21:27:31 +02:00
Egon Elbre
e37e0c1b5f
Fix server config usage ( #1282 )
2019-02-08 20:57:17 +02:00
Michal Niewrzal
c7ad427d95
Move metainfo config ( #1234 )
...
* Move metainfo config
* move config to uplink
* reorganize code
* use satellite apikey
* comment update
2019-02-08 13:57:35 +01:00
Egon Elbre
9c1e299f3c
Ensure everyone sees everyone else ( #1275 )
2019-02-08 11:25:13 +02:00
paul cannon
ef61c170b1
Consolidate key/cert/signature encoding and decoding ( #1243 )
2019-02-07 12:40:28 -06:00
Egon Elbre
bb11d83ed0
Proper planet shutdown ( #1249 )
2019-02-06 15:19:14 +02:00
Egon Elbre
fdbe2db273
Remove node package and simplify DHT interface ( #1233 )
2019-02-06 14:37:17 +02:00
Egon Elbre
ab83f0f077
Fix parallel execution sharing ( #1247 )
2019-02-06 11:16:05 +02:00
Egon Elbre
b5d86aa0b4
Make testplanet.Run and satellitedbtest.Run parallel by default ( #1231 )
2019-02-05 21:44:00 +02:00
Egon Elbre
07412698a9
Use lower bcrypt cost for console.Service when running tests and disable flaky tests ( #1236 )
2019-02-05 19:31:53 +02:00
Yaroslav Vorobiov
2ff0d9d435
Add bucket to project relationship on satellite ( #1143 )
...
* integrate console api keys with buckets in pointerdb
* fix test
* fix tests kvmetainfo
* linter fix
* disable account activation
* fix test
* review fixes
* fix comments
* little refactoring
* remove debug println
* fix typo
* disable activation in a propper way
* fix test
* fix imports
* fix uplink count in testplanet
* move key creation to planet.newUplink
2019-02-05 19:22:17 +02:00
Egon Elbre
6564357e13
Ensure that tally tests run against postgres ( #1227 )
2019-02-05 16:23:45 +02:00
Egon Elbre
87d6410b50
Revert "Remove node package and simplify DHT interface."
...
This reverts commit 03ec1ff92d
2019-02-05 10:38:48 +02:00
Egon Elbre
03ec1ff92d
Remove node package and simplify DHT interface.
2019-02-05 10:37:24 +02:00
Dylan Lott
21a58d0ece
node graveyard resurrection ( #1186 )
...
* deletes nodes from cache if unresponsive
* update error messages
* WIP restructuring discovery tickers
* passes tests
* Removes log messages
* Passes up error in graveyard ticker
2019-02-04 14:52:06 -07:00
Egon Elbre
c37e0c1b6d
testplanet.Run ( #1222 )
2019-02-04 22:37:46 +02:00
Michal Niewrzal
2f8b3da1af
Upload/download for testplanet Uplink ( #1183 )
...
* Upload/download for testplanet Uplink
* check error in tests
* cleanup
* refactor node -> uplink
* add missing test file
* rest of refactoring
* workaround to resolve cycles in tests
* rename method
* add missing comments
* review comments
* use KiB
2019-02-04 17:56:10 +01:00
Egon Elbre
d1cc0dd85b
testplanet: add capability to stop peers ( #1221 )
2019-02-04 17:40:37 +02:00
Dylan Lott
c77a647174
Updates transport dialopts to use WithBlock() ( #1201 )
...
* Updates transport opts to use WithBlock()
* Removes unnecessary withBlock's and adds WithBlock to DialAddress
2019-02-01 10:40:35 -07:00
Egon Elbre
b91d77436f
Test merging planets ( #1181 )
2019-02-01 15:32:28 +02:00
Egon Elbre
d5346982c2
Delete provider package ( #1177 )
2019-01-30 22:47:21 +02:00
Dylan Lott
5e1ab841cf
Cache refresher ( #1171 )
...
* got tests passed
* wire up paginate function for cache node retrieval
* Add tests for paginate but they're failing
* fix the test arguments
* Updates paginate function to return more variable
* Updates
* Some test and logic tweaks
* improves config handling in discovery
* adds refresh offset to discovery struct
2019-01-30 11:29:18 -05:00
Natalie Villasana
8d7944bcf8
new node selection in overlay ( #1136 )
2019-01-29 14:42:43 -05:00
Egon Elbre
1df81b1460
Separate garbage collect logic from psdb ( #1167 )
2019-01-29 17:41:01 +02:00
Egon Elbre
f7e18882c4
Kademlia closing race ( #1139 )
2019-01-26 00:33:20 +02:00
Jennifer Li Johnson
5397efe5c1
removes standalone payments service ( #1134 )
...
* remove standalone payments service - will integrate within satellite directly
* regenerate proto files
2019-01-25 11:23:25 -05:00
Egon Elbre
187e9b2138
Code consistency between peers ( #1126 )
2019-01-24 22:28:06 +02:00
Jennifer Li Johnson
856b98997c
updates copyright 2018 to 2019 ( #1133 )
2019-01-24 15:15:10 -05:00
Yaroslav Vorobiov
f437effaef
Add console to satellite peer ( #1120 )
2019-01-24 18:26:36 +02:00
Egon Elbre
5de7f8af7f
Satellite Peer ( #1119 )
2019-01-23 21:58:44 +02:00
Michal Niewrzal
3fdb47e31c
Try accessing bootstrap node for verification ( #1089 )
2019-01-23 17:48:46 +02:00
Egon Elbre
b6c61cdd55
Use storagenode.Peer for storagenode ( #1107 )
2019-01-23 12:39:03 +02:00
Egon Elbre
78dc02b758
Satellite Peer ( #1034 )
...
* add satellite peer
* Add overlay
* reorganize kademlia
* add RunRefresh
* add refresh to storagenode.Peer
* add discovery
* add agreements and metainfo
* rename
* add datarepair checker
* add repair
* add todo notes for audit
* add testing interface
* add into testplanet
* fixes
* fix compilation errors
* fix compilation errors
* make testplanet run
* remove audit refrences
* ensure that audit tests run
* dev
* checker tests compilable
* fix discovery
* fix compilation
* fix
* fix
* dev
* fix
* disable auth
* fixes
* revert go.mod/sum
* fix linter errors
* fix
* fix copyright
* Add address param for SN dashboard (#1076 )
* Rename storj-sdk to storj-sim (#1078 )
* Storagenode logs and config improvements (#1075 )
* Add more info to SN logs
* remove config-dir from user config
* add output where config was stored
* add message for successful connection
* fix linter
* remove storage.path from user config
* resolve config path
* move success message to info
* log improvements
* Remove captplanet (#1070 )
* pkg/server: include production cert (#1082 )
Change-Id: Ie8e6fe78550be83c3bd797db7a1e58d37c684792
* Generate Payments Report (#1079 )
* memory.Size: autoformat sizes based on value entropy (#1081 )
* Jj/bytes (#1085 )
* run tally and rollup
* sets dev default tally and rollup intervals
* nonessential storj-sim edits (#1086 )
* Closing context doesn't stop storage node (#1084 )
* Print when cancelled
* Close properly
* Don't log nil
* Don't print error when closing dashboard
* Fix panic in inspector if ping fails (#1088 )
* Consolidate identity management to identity cli commands (#1083 )
* Consolidate identity management:
Move identity cretaion/signing out of storagenode setup command.
* fixes
* linters
* Consolidate identity management:
Move identity cretaion/signing out of storagenode setup command.
* fixes
* sava backups before saving signed certs
* add "-prebuilt-test-cmds" test flag
* linters
* prepare cli tests for travis
* linter fixes
* more fixes
* linter gods
* sp/sdk/sim
* remove ca.difficulty
* remove unused difficulty
* return setup to its rightful place
* wip travis
* Revert "wip travis"
This reverts commit 56834849dcf066d3cc0a4f139033fc3f6d7188ca.
* typo in travis.yaml
* remove tests
* remove more
* make it only create one identity at a time for consistency
* add config-dir for consitency
* add identity creation to storj-sim
* add flags
* simplify
* fix nolint and compile
* prevent overwrite and pass difficulty, concurrency, and parent creds
* goimports
2019-01-18 08:54:08 -05:00
Egon Elbre
f8906ce000
Change overlay cache database interface ( #1047 )
2019-01-15 11:08:45 -05:00
Alexander Leitner
bfde515391
Clean up Storage node setup ( #1013 )
...
* Edit config on Setup
* Default to 1TiB storage space and 500GiB bandwidth
* Use human readable formats
* Use memory
* units of 1024 are measured with KiB/MiB etc
* pkg/cfgstruct: allow values to be configured with human readable sizes
Change-Id: Ic4e9ae461516d1d26fb81f6e44c5ac5cfccf777f
* Modify tests
* Removed comments
* More merge conflict stuff resolved
* Fix lint
* test fixin
Change-Id: I3a008206bf03a4446da19f642a2f9c1f9acaae36
* Remove commented code but secretly leave it in the histroy forever
* Move flag definition to struct
2019-01-14 16:19:15 -05:00
Egon Elbre
8893884044
convert piecestorage into a struct ( #1024 )
2019-01-11 13:26:39 +02:00
Egon Elbre
eb69ecadec
Storage Node Peer ( #1005 )
2019-01-10 15:13:27 +02:00
Jennifer Li Johnson
a2fa5c4c5a
Proper NodeType Handling ( #873 )
...
* adds enums to nodetype
* updating nodetype todos
* ran pb updates
* reorder nodetypes
* adding checks
* wip
* wip
* wip
* bug in test-captplanet
* wip
* add values to storagenode, satellite, captplanet binaries
* Cleanup
* more cleanup
* wip
* lint
* lint
* wip
* fixes bug
* regenerate protos
Change-Id: Id270212e8c7479e52641058042cf23b5317ab773
* limit node type changes to kademlia
Change-Id: I9c1a6cc4a79e05086627f0fdeb5028c62ce754f4
* dpanic
Change-Id: Id952a2ad13c807ebaea0ec0a875405e267d81c3e
* review comments
Change-Id: I7f9b77ef22779dd012fd490375b136014f51f834
2019-01-02 11:47:34 -07:00
Egon Elbre
89db0fe9f5
storj-sdk base code ( #799 )
2019-01-02 20:07:49 +02:00
Egon Elbre
4346cd060f
Implement mutex around satellitedb ( #932 )
2018-12-27 11:56:25 +02:00
Dylan Lott
7569b7d71b
Node lifecycle hooks ( #920 )
...
* wires up first draft of lifecycle methods
* creates interface on transport
* node lifecycle hooks works
* linter fixes
* adds error log at connection success
* chnages Observer interface to use context
* Makes Discovery take its own logger
* WIP
* linter fixes
* Test fixes
* adds in ConnFailure code for cache
2018-12-21 21:51:42 -07:00
Dylan Lott
2f6835a3be
Adds a logger to RoutingTable ( #922 )
...
* adds delete functions and tests for cache
* adds logger to routing table struct and passes it through to logger
* add loggers
2018-12-21 17:48:53 -07:00
Bill Thorp
3fa094f3da
Add the ability for Kademlia to do a refresh for each bucket ( #908 )
...
* added random id in range, added refresh
2018-12-20 16:45:06 -05:00
Egon Elbre
d9b9ae6ffa
Cleanup overlay methods and names. ( #914 )
2018-12-20 15:57:54 +02:00
Egon Elbre
d9a13667fc
Use fixed logger for Kademlia and make not being able to connect in discovery debug statement. ( #899 )
2018-12-18 17:13:32 +02:00
Natalie Villasana
17c60c1f06
moves node selection config setup from uplink to satellite ( #891 )
2018-12-17 16:05:05 -05:00
Cameron
0d17c21a1b
Add logger to piecestore server/node started log ( #856 )
...
* add logger to psserver struct
* node started log
* rename initialize to NewEndpoint
* return err from WriteBandwidthAllocToDB
2018-12-17 10:23:02 -05:00
Egon Elbre
d8df4b5f6b
Simplify provider tests ( #862 )
2018-12-17 17:09:52 +02:00
aligeti
5e1b02ca8b
Statdb master db v3 848 ( #830 )
...
* intial changes to migrate statdb to masterdb framework
* statdb refactor compiles
* added TestCreateDoesNotExist testcase
* Initial port of statdb to masterdb framework working
* refactored statdb proto def to pkg/statdb
* removed statdb/proto folder
* moved pb.Node to storj.NodeID
* CreateEntryIfNotExistsRequest moved pd.Node to storj.NodeID
* moved the fields from pb.Node to statdb.UpdateRequest
ported TestUpdateExists, TestUpdateUptimeExists, TestUpdateAuditSuccessExists TestUpdateBatchExists
2018-12-14 15:17:30 -05:00
Dylan Lott
9bdee7b106
Discovery service ( #805 )
...
* WIP possible discovery service impl
* Adds discovery service to CaptPlanet
* Updates the config and server for discovery service
* updates testplanet to use discovery package
* update satellite imports
* Removes unnecessary cache test
* linter fixes
* adds discovery startup to captplanet
* invoke refresh
* updates to discovery refresh cycle
* Make implementation more consistent with previous implementation
* add wait before trying to upload
* sleep a bit more
* remove kademlia bootstrap
* updates
* remove comments
2018-12-14 12:23:54 -07:00
Egon Elbre
969fcabf72
Remove utils.LogClose from identity saving ( #837 )
2018-12-13 15:50:05 +02:00
Egon Elbre
7a80e7bf2f
Use testplanet in node tests ( #841 )
2018-12-12 17:40:33 +02:00
Kaloyan Raev
c025c13706
Fix deadlock on testplanet shutdown ( #798 )
2018-12-07 20:31:59 +02:00
Bryan White
228aa34ff6
TLS extension processing ( #771 )
2018-12-07 14:44:25 +01:00
Maximillian von Briesen
6c655d117b
Filter nodes by reputation and IP address (node selection) ( #720 )
...
* Pulls statdb stats into overlay cache whenever cache.Put() is called
* Updates overlay.FindStorageNodes()/overlayClient.Choose() to filter based on node stats
* Updates overlay.FindStorageNodes()/overlayClient.Choose() to exclude duplicate IP addresses
2018-12-04 15:18:26 -05:00
Maximillian von Briesen
2ab15196d9
Get rid of statdb grpc stuff ( #761 )
...
* remove sdbclient
* do not register statdb grpc server/remove apikey from statdb
* start audit service after statdb in captplanet
2018-12-04 13:47:58 -05:00
Egon Elbre
1e4556f88a
Fix import groupings ( #739 )
2018-11-30 15:40:13 +02:00
Bryan White
2a0c4e60d2
preparing for use of customtype gogo extension with NodeID type ( #693 )
...
* preparing for use of `customtype` gogo extension with `NodeID` type
* review changes
* preparing for use of `customtype` gogo extension with `NodeID` type
* review changes
* wip
* tests passing
* wip fixing tests
* more wip test fixing
* remove NodeIDList from proto files
* linter fixes
* linter fixes
* linter/review fixes
* more freaking linter fixes
* omg just kill me - linterrrrrrrr
* travis linter, i will muder you and your family in your sleep
* goimports everything - burn in hell travis
* goimports update
* go mod tidy
2018-11-29 19:39:27 +01:00
Egon Elbre
0ae05cf834
Ensure that we only choose storage nodes ( #732 )
2018-11-29 16:57:00 +02:00
Maximillian von Briesen
7cf16503b8
Add statdb to overlay cache (node selection) ( #711 )
...
* make overlay node rep consistent with statdb node rep
* add statdb to testplanet
* add statdb to overlay cache
* tests
2018-11-27 12:46:12 -05:00
Dennis Coyle
e7e0d1daaa
Coyle/disable mock ( #627 )
...
* disabled mock overlay from captplanet
2018-11-20 11:54:52 -05:00
Egon Elbre
76af9f5171
Add overlay server to testplanet ( #677 )
2018-11-19 22:39:25 +02:00
Egon Elbre
d07433c150
Use testplanet to test Refresh ( #678 )
2018-11-19 16:40:01 +02:00
Egon Elbre
21a102a05f
testplanet: add logger ( #649 )
2018-11-15 10:57:47 +02:00
Bryan White
ee62e2a9d8
Use transport client and cleanup all the clients ( #574 )
...
* wip
* linter fixes
* linter fixes
* test fixes
* linter fixes
* fix merge + restructure piecestore packages
* review feedback
* linter fixes
* linter fixes
* remove unnecessary aliases to piecestore
* more merge fixing
2018-11-06 18:49:17 +01:00
Michal Niewrzal
1129ead6ef
Fix 'message not signed by the satellite' problem ( #560 )
...
* Fix 'message not signed by the satellite' problem
* fixed build
* fix errors
2018-11-05 16:12:19 +01:00
Egon Elbre
c765dd98de
internal/testplanet ( #528 )
2018-11-03 14:17:14 +02:00
