Commit Graph

79 Commits

Author SHA1 Message Date
Bill Thorp
e4b3dab446
Add config editing via the CLI for storage nodes (#919)
* added editor and updated storagenode cmd
* switched to custom yaml gen
2019-01-02 10:12:07 -05:00
Egon Elbre
4346cd060f
Implement mutex around satellitedb (#932) 2018-12-27 11:56:25 +02:00
Dylan Lott
7569b7d71b
Node lifecycle hooks (#920)
* wires up first draft of lifecycle methods

* creates interface on transport

* node lifecycle hooks works

* linter fixes

* adds error log at connection success

* chnages Observer interface to use context

* Makes Discovery take its own logger

* WIP

* linter fixes

* Test fixes

* adds in ConnFailure code for cache
2018-12-21 21:51:42 -07:00
Dylan Lott
2f6835a3be
Adds a logger to RoutingTable (#922)
* adds delete functions and tests for cache

* adds logger to routing table struct and passes it through to logger

* add loggers
2018-12-21 17:48:53 -07:00
Bill Thorp
3fa094f3da
Add the ability for Kademlia to do a refresh for each bucket (#908)
* added random id in range, added refresh
2018-12-20 16:45:06 -05:00
Egon Elbre
1feef7105b
Implement memory and file pipe (#894) 2018-12-20 16:51:39 +02:00
Egon Elbre
d9b9ae6ffa
Cleanup overlay methods and names. (#914) 2018-12-20 15:57:54 +02:00
Egon Elbre
d9a13667fc
Use fixed logger for Kademlia and make not being able to connect in discovery debug statement. (#899) 2018-12-18 17:13:32 +02:00
Natalie Villasana
17c60c1f06
moves node selection config setup from uplink to satellite (#891) 2018-12-17 16:05:05 -05:00
Cameron
0d17c21a1b
Add logger to piecestore server/node started log (#856)
* add logger to psserver struct

* node started log

* rename initialize to NewEndpoint

* return err from WriteBandwidthAllocToDB
2018-12-17 10:23:02 -05:00
Egon Elbre
d8df4b5f6b
Simplify provider tests (#862) 2018-12-17 17:09:52 +02:00
aligeti
5e1b02ca8b
Statdb master db v3 848 (#830)
* intial changes to migrate statdb to masterdb framework

* statdb refactor compiles

* added TestCreateDoesNotExist testcase

* Initial port of statdb to masterdb framework working

* refactored statdb proto def to pkg/statdb

* removed statdb/proto folder

* moved pb.Node to storj.NodeID

* CreateEntryIfNotExistsRequest moved pd.Node to storj.NodeID

* moved the fields from pb.Node to statdb.UpdateRequest

ported TestUpdateExists, TestUpdateUptimeExists, TestUpdateAuditSuccessExists TestUpdateBatchExists
2018-12-14 15:17:30 -05:00
Dylan Lott
9bdee7b106
Discovery service (#805)
* WIP possible discovery service impl

* Adds discovery service to CaptPlanet

* Updates the config and server for discovery service

* updates testplanet to use discovery package

* update satellite imports

* Removes unnecessary cache test

* linter fixes

* adds discovery startup to captplanet

* invoke refresh

* updates to discovery refresh cycle

* Make implementation more consistent with previous implementation

* add wait before trying to upload

* sleep a bit more

* remove kademlia bootstrap

* updates

* remove comments
2018-12-14 12:23:54 -07:00
Bryan White
2016ce9fd6
Certificate revocation (#836)
* wip certificate revocation

* refactor tests

* wip testing

* testing

* review fixes

* integration fix attempt #1

* review fixes

* integration fix attempt #2

* linter fixes

* add copywrite

* integration fix attemp #3

* more testing

* more tests

* go mod tidy

* review fixes

* linter fixes
2018-12-13 21:01:43 +01:00
Egon Elbre
969fcabf72
Remove utils.LogClose from identity saving (#837) 2018-12-13 15:50:05 +02:00
Egon Elbre
7a80e7bf2f
Use testplanet in node tests (#841) 2018-12-12 17:40:33 +02:00
Kaloyan Raev
c025c13706
Fix deadlock on testplanet shutdown (#798) 2018-12-07 20:31:59 +02:00
Bryan White
228aa34ff6
TLS extension processing (#771) 2018-12-07 14:44:25 +01:00
Maximillian von Briesen
6c655d117b
Filter nodes by reputation and IP address (node selection) (#720)
*   Pulls statdb stats into overlay cache whenever cache.Put() is called
* Updates overlay.FindStorageNodes()/overlayClient.Choose() to filter based on node stats
* Updates overlay.FindStorageNodes()/overlayClient.Choose() to exclude duplicate IP addresses
2018-12-04 15:18:26 -05:00
Maximillian von Briesen
2ab15196d9
Get rid of statdb grpc stuff (#761)
* remove sdbclient

* do not register statdb grpc server/remove apikey from statdb

* start audit service after statdb in captplanet
2018-12-04 13:47:58 -05:00
Michal Niewrzal
06c196d28c
Allow config generation into empty directory (#752)
* Allow config generation into empty directory

* unifed messages

* move main method to fpath

* fix linter problems
2018-12-04 12:42:16 +01:00
Cameron
226cf48c41
move clients to internal/s3api (#749)
* move clients to internal/s3client

* move Config and Client into common.go
2018-12-03 14:50:05 -05:00
Michal Niewrzal
1206ef9126
Use OS related default setup directory (#741)
* Use OS related default setup directory

* added missing comment

* adjust test-captplanet.sh

* ApplicationDir moved to fpath
2018-12-03 16:51:56 +01:00
Kaloyan Raev
66476da1ae
metainfo: basic upload (#726) 2018-11-30 15:50:52 +02:00
Egon Elbre
1e4556f88a
Fix import groupings (#739) 2018-11-30 15:40:13 +02:00
Bryan White
2a0c4e60d2
preparing for use of customtype gogo extension with NodeID type (#693)
* preparing for use of `customtype` gogo extension with `NodeID` type

* review changes

* preparing for use of `customtype` gogo extension with `NodeID` type

* review changes

* wip

* tests passing

* wip fixing tests

* more wip test fixing

* remove NodeIDList from proto files

* linter fixes

* linter fixes

* linter/review fixes

* more freaking linter fixes

* omg just kill me - linterrrrrrrr

* travis linter, i will muder you and your family in your sleep

* goimports everything - burn in hell travis

* goimports update

* go mod tidy
2018-11-29 19:39:27 +01:00
Egon Elbre
0ae05cf834
Ensure that we only choose storage nodes (#732) 2018-11-29 16:57:00 +02:00
Maximillian von Briesen
7cf16503b8
Add statdb to overlay cache (node selection) (#711)
* make overlay node rep consistent with statdb node rep

* add statdb to testplanet

* add statdb to overlay cache

* tests
2018-11-27 12:46:12 -05:00
Egon Elbre
ed86059829
Add stack-trace and remove testcase with testing (#716) 2018-11-27 17:50:40 +02:00
Dennis Coyle
e7e0d1daaa
Coyle/disable mock (#627)
* disabled mock overlay from captplanet
2018-11-20 11:54:52 -05:00
Egon Elbre
7d70842d53
Detect goroutines running for too long in testcontext (#685) 2018-11-20 11:32:18 +02:00
Egon Elbre
76af9f5171
Add overlay server to testplanet (#677) 2018-11-19 22:39:25 +02:00
Egon Elbre
d07433c150
Use testplanet to test Refresh (#678) 2018-11-19 16:40:01 +02:00
Egon Elbre
c4b90f84dd
Use better defaults and naming for postgres database (#659) 2018-11-15 20:36:57 +02:00
Egon Elbre
21a102a05f
testplanet: add logger (#649) 2018-11-15 10:57:47 +02:00
Egon Elbre
6066ae7567
Check schemas while creating tables (#609) 2018-11-10 00:15:35 +02:00
Bryan White
ee62e2a9d8
Use transport client and cleanup all the clients (#574)
* wip

* linter fixes

* linter fixes

* test fixes

* linter fixes

* fix merge + restructure piecestore packages

* review feedback

* linter fixes

* linter fixes

* remove unnecessary aliases to piecestore

* more merge fixing
2018-11-06 18:49:17 +01:00
Michal Niewrzal
1129ead6ef
Fix 'message not signed by the satellite' problem (#560)
* Fix 'message not signed by the satellite' problem

* fixed build

* fix errors
2018-11-05 16:12:19 +01:00
Egon Elbre
c765dd98de
internal/testplanet (#528) 2018-11-03 14:17:14 +02:00
Egon Elbre
fd99de5834
Don't try to remove empty directory (#553) 2018-10-31 20:55:14 +02:00
Egon Elbre
47d056d627
testcontext package (#545) 2018-10-29 16:16:36 +02:00
Egon Elbre
61632095e9
Flatten internal/pkg and remove unused internal/pkg/node (#526) 2018-10-25 11:24:39 +03:00
Egon Elbre
8efb4f0e89
Fix repairing run (#523)
* Fix repairing run
* Fix concurrency bugs
* Add sync2.Limiter concurrency primitive
2018-10-24 15:35:59 +03:00
Stefan Benten
1e569e3d4d
Add S3 Scheme as valid URL Scheme (#483)
* Add s3 Scheme as valid URL Scheme

* Merge from Master

* Remove Test Case and fix Error

* Add invalid Test Case

* add valid s3 Test Case for good measure

* Improve Speed and Resource Handling
2018-10-16 21:50:37 +02:00
Stefan Benten
586ed1b58d
Improve path handling + filepath handling framework (#423)
* Initial Layout

* Commit to test File Handling OS independed

* Hide struct properties to prevent manual interaction

* Fix Linting Errors

* 1st Working Windows Version

* Add missing Error Handling

* Fix Linting Errors

* Remove dependencies

* Further Improvements

* Remove commented code

* Improve comments and error messages

* No pointers to FPath

* Improve comment

* Do not filepath.ToSlash URL path

* Extract helper functions for parsing local path and Storj path

* Minor Improvements based on PR Comments

* Fix Linting Error and make Regex private

* Improve Layout

* Rework FPath and add tests

* Add more tests cases for windows

* Use for-loop instead of goto

* Use FPath in all uplink commands

* Add guard checks

* Add Test Cases and add comments
2018-10-15 16:10:20 +02:00
Egon Elbre
caecb2801d
Cleanups related to transport (#475)
* Remove DialUnauthenticated method
* Use more specific node transport enum
* Formatting
* Rename Provider.g -> Provider.grpc
* Fix naming in dialer
2018-10-15 15:04:21 +03:00
Dennis Coyle
dee2c137c8
Remove BKAD dependency from pkg/kademlia (#294)
* slowly but surely

* hardcode ID for tests so we can get predictable results

* skipping bad test

* removing tests of bkad

* wip

* new algorithm for worker

* clean up

* remove skipped test

* changes

* uncomment

* fixed conflicts

* maybe done ?

* cleanup

* boot bkad

* wip

* cleanup

* undo change

* fixes

* wip

* wip

* moving nodeID around

* wip

* wip

* fixes

* fixes after merge

* added TODO

* fixed tests post identity

* linter fixes

* wip

* PR review comments

* wip

* fixing tests

* fix tests

* force db directory

* bad test

* fixes race condition

* small cleanups

* adding db folder

* testing

* wip

* cleanup

* cleanup

* linters

* export Restrict

* add timeout

* testing

* linters

* forgot one

* moar fixes from master merge

* PR comments

* moar PR comments

* removed stun flag

* remove duplicate declaration

* remove old tests

* remove timeout

* fix tests

* missed one

* changed StringToID >> IDFromString

* PR comments

* stupid linter

* moevd overlay mock

* fixed merge conflicts

* fixes

* linter
2018-10-08 11:09:37 -04:00
Egon Elbre
ab64aa0b97
add missing copyright headers (#403) 2018-10-01 14:29:27 +03:00
Egon Elbre
e83502f33d
s3-benchmark: add aws-cli support (#386) 2018-09-28 15:40:08 +03:00
Egon Elbre
f6c48dcb60
Fix windows linter errors (#355) 2018-09-14 20:56:18 +03:00