Commit Graph

24 Commits

Author SHA1 Message Date
Michal Niewrzal
794254b0b2
mobile: build libuplink aar (#3165) 2019-10-04 01:56:42 -07:00
Kaloyan Raev
eb73a3d35f
Windows installer for storage nodes (#2921)
Co-authored-by: Michal Niewrzal <michal@storj.io>
2019-09-23 23:43:32 +03:00
Jess G
3e121b13e0
docs/design: satellite service separation (#2815)
* add md extension to dd process doc

* rm old format of dd process doc

* wip design doc

* add diagram, add implementation

* adjust format

* fix format

* add updates per CR and arch review meeting

* update diagram sizes

* make image smaller

* try using svg instead of png

* replace png with svg, mv to docs/images dir

* updates per CR comments

* more CR comment changes

* update sa design img

* grammar fixes

* a few more updates

* fixings nits

* fix spelling err

* fix spelling, change binary to process, rm process doc

* add changes to design imagees per CR comments
2019-08-28 09:28:53 -07:00
Egon Elbre
0c52f40d97
cmd/internal/assets: package for embedding resources (#2374) 2019-07-01 17:11:23 +03:00
Egon Elbre
0aae26d78b
gitignore: cleanup old things (#2197) 2019-06-14 12:42:37 +03:00
Jennifer Li Johnson
8d779d3d3e Disconnects nodeclient, routing table dbs when done with kademlia (#507)
* disconnect from nodeclient

* cleanup connections in tests

* kademlia disconnects from nodeclient

* updating disconnect method for mocks

* creates separate disconnect and removeAll methods for tests

* adds init to connection pool

* fix folder cleanup and disconnect

* creates and cleans up test db files and disconnects kad

* removes db/.keep

* includes disconnect within cleanup methods

* creates public init method on connection pool to handle mutex copy issues

* remove all after disconnect

* pair creation and destruction

* checks disconnect error

* remove ctx

* fixes mock kad
2018-10-26 10:07:02 -04:00
Matt Robinson
bcd18a44bc
give our appearence on windows some love (#429) 2018-10-16 14:48:17 -04:00
Dennis Coyle
dee2c137c8
Remove BKAD dependency from pkg/kademlia (#294)
* slowly but surely

* hardcode ID for tests so we can get predictable results

* skipping bad test

* removing tests of bkad

* wip

* new algorithm for worker

* clean up

* remove skipped test

* changes

* uncomment

* fixed conflicts

* maybe done ?

* cleanup

* boot bkad

* wip

* cleanup

* undo change

* fixes

* wip

* wip

* moving nodeID around

* wip

* wip

* fixes

* fixes after merge

* added TODO

* fixed tests post identity

* linter fixes

* wip

* PR review comments

* wip

* fixing tests

* fix tests

* force db directory

* bad test

* fixes race condition

* small cleanups

* adding db folder

* testing

* wip

* cleanup

* cleanup

* linters

* export Restrict

* add timeout

* testing

* linters

* forgot one

* moar fixes from master merge

* PR comments

* moar PR comments

* removed stun flag

* remove duplicate declaration

* remove old tests

* remove timeout

* fix tests

* missed one

* changed StringToID >> IDFromString

* PR comments

* stupid linter

* moevd overlay mock

* fixed merge conflicts

* fixes

* linter
2018-10-08 11:09:37 -04:00
Egon Elbre
3b98b93f05
captplanet: add ctrl+\ for dumping goroutines (#416) 2018-10-03 15:27:51 +03:00
Egon Elbre
10d9420f24
command-line benchmark and visualization (#342) 2018-09-11 18:35:46 +03:00
Egon Elbre
7ecca9988a
add reed-solomon encoding benchmark (#307) 2018-09-05 17:33:20 +03:00
Matt Robinson
0a78e741ec
Build and upload binaries (#296)
* First pass at building binaries

* Finished building images

* Reworked some bits to save binaries in a better place.

* First pass at uploading binaries

* Builds for freebsd fail currently

* Add ignore file for docker builds to not invalidate the cache as often

* Docker image for 1.11-alpine exists now

* Name windows binaries correctly

* The makefile is used in tests.

* Take a swing at updating the tests for 1.11

* Switch to something with glibc so the race detector works.

* Remove unused .PHONY targets
2018-08-31 11:21:44 -04:00
JT Olio
58c810519f
cross-compilation: fix darwin (#293)
* cross-compilation: fix darwin

* minio without fsnotify

* update Gopkg.lock

* make release target
2018-08-27 15:06:55 -06:00
Bryan White
5d20cf8829
Node Identity (#193)
* peertls: don't log errors for double close

understood that this part of the code is undergoing heavy change
right now, but just want to make sure this fix gets incorporated
somewhere

* git cleanup: node-id stuff

* cleanup

* rename identity_util.go

* wip `CertificateAuthority` refactor

* refactoring

* gitignore update

* wip

* Merge remote-tracking branch 'storj/doubleclose' into node-id3

* storj/doubleclose:
  peertls: don't log errors for double close

* add peertls tests & gomports

* wip:

+ refactor
+ style changes
+ cleanup
+ [wip] add version to CA and identity configs
+ [wip] heavy client setup

* refactor

* wip:

+ refactor
+ style changes
+ add `CAConfig.Load`
+ add `CAConfig.Save`

* wip:

+ add `LoadOrCreate` and `Create` to CA and Identity configs
+ add overwrite to CA and identity configs
+ heavy client setup
+ refactor
+ style changes
+ cleanup

* wip

* fixing things

* fixing things

* wip hc setup

* hc setup:

+ refactor
+ bugfixing

* improvements based on reveiw feedback

* goimports

* improvements:

+ responding to review feedback
+ refactor

* feedback-based improvements

* feedback-based improvements

* feedback-based improvements

* feedback-based improvements

* feedback-based improvements

* feedback-based improvements

* cleanup

* refactoring CA and Identity structs

* Merge branch 'master' into node-id3

* move version field to setup config structs for CA and identity

* fix typo

* responding to revieiw feedback

* responding to revieiw feedback

* responding to revieiw feedback

* responding to revieiw feedback

* responding to revieiw feedback

* responding to revieiw feedback

* Merge branch 'master' into node-id3

* fix gateway setup finally

* go imports

* fix `FullCertificateAuthority.GenerateIdentity`

* cleanup overlay tests

* bugfixing

* update ca/identity setup

* go imports

* fix peertls test copy/paste fail

* responding to review feedback

* setup tweaking

* update farmer setup
2018-08-13 10:39:45 +02:00
Dennis Coyle
5de5428d3a
First pass at node restrictions (#146)
* WIP First pass at node restrictions

* adjustments to storage clients

* fix cover

* undo previous import change

* adding copyright

* fix the tests

* check for keys

* moar fixes to tests

* linter

* change how we handle restrictions

* add generated pb.go file

* PR comments addressed

* MockKeyValueStore

* pr comments addressed

* missing )

* past my bedtime

* moar merge issues

* cleanup
2018-08-01 10:15:38 -04:00
Dylan Lott
15960d4269
Admin node (#131)
* WIP creating admin node service

- WIP changing the process pkg to accept multiple services
- WIP looping over services passed to process
- add netstate/service.go file and abstract it for service processing

* implement goroutine to launch each process

* goroutines working with multiple services

* code review fixes

* more code updates for review

* Add pkg lock and mod files back in

* code review updates

* update process.Main with better concurrent error handling

* Update error handling and pass ctx to StartService

* Update error handling with channel implementation

* Merge in upstream changes

- Simplify error handling channels

* updates

* Updates per reviewable

* fix test

* Setup test exec

* Scaffold test setup

* process main test working

* update admin process test

* Test multiple processes done

* Add error classes for testing, test main logger error

* Updates to tests

* Update how process.Main() handles configs

* Complete merge

* Update Gopkg and add Copyright

* Fix cyclical import issue

- Added .coverprofile to gitignore
- Update admin main.go function call

* remove unnecessary line

* Updates

* DRY up cmd/netstate package

* update service function calls

* updates

* Trying no-ops in examples

* rename netstate to pointerdb

* trying to fix merge

* dep ensure and run tests

* remove flag.Parse

* Update deps

* Skip offending test in pkg/process, to be fixed later
2018-07-09 16:43:32 -06:00
Bryan White
c6f8144221 Bolt backed overlay cache (#94)
* wip

* add separate `Process` tests for bolt and redis-backed overlay

* more testing

* fix gitignore

* fix linter error

* goimports goimports GOIMPORTS GoImPortS!!!!

* fix port madness

* forgot to add

* add `mux` as handler and shorten context timeouts

* gofreakingimports

* fix comments

* refactor test & add logger/monkit registry

* debugging travis

* add comment

* Set redisAddress to empty string for bolt-test

* travis experiment

* refactoring tests

* Merge remote-tracking branch 'upstream/master' into bolt-backed-overlay-cache
2018-06-20 10:28:46 -04:00
Dylan Lott
325a70d514 Cache (#67)
* add reference to dht to overlay client struct

* wip

* wip

* Implement FindNode

* get nodes

* WIP

* Merge in Dennis kademlia code, get it working with our code

* ping and moar

* WIP trying to get cache working with kademlia

* WIP more wiring up

* WIP

* Update service cli commands

* WIP

* added GetNodes

* added nodes to Kbucket

* default transport changed to TCP

* GetBuckets interface changed

* filling in more routing

* timestamp methods

* removed store

* Added initial network overlay explorer page

* Updating and building with dockerfile

* Working on adding bootstrap node code

* WIP merging in dennis' code

* WIP

* connects cache to pkg/kademlia implementation

* WIP redis cache

* testing

* Add bootstrap network function for CLI usage

* cleanup

* call bootstrap on init network

* Add BootstrapNetwork function to interface

* Merge in dennis kad code

* WIP updates to redis/overlay client interface

* WIP trying to get the DHT connected to the cache

* go mod & test

* deps

* Bootstrap node now setting up correctly

- Need to pass it through CLI commands better

* WIP adding refresh and walk functions, added cli flags

- added cli flags for custom bootstrap port and ip

* PR comments addressed

* adding FindStorageNodes to overlay cache

* fix GetBucket

* using SplitHostPort

* Use JoinHostPort

* updates to findstoragenodes response and request

* WIP merge in progress, having issues with a panic

* wip

* adjustments

* update port for dht bootstrap test

* Docker

* wip

* dockerfile

* fixes

* makefile changes

* Update port in NewKademlia call

* Update local kademlia DHT config

* kubernetes yaml

* cleanup

* making tests pass

* k8s yaml

* lint issues

* Edit cli flags to allow for configurable bootstrap IP and Port args

* cleanup

* cache walking the network now

* Rough prototype of Walk function laid out

* Move walk function into bootstrap function

* Update dht.go

* changes to yaml

* goimports
2018-06-05 17:06:37 -04:00
nfarah86
f9af3b6ee4
Mutex/auth grpc (#50)
* initial commit for PUT request authorization

* inital auth for put request

* auth. request working for all req

* deleted library

* removed .db files

* work in progress for modifying test suite to accomodate credentials

* modified tests

* gofmt, fixed code based on suggestions; test passed

* gofmt again

* merged nat's update on pointers, passed tests, cleanup from git rebase

* fixed fmt

* fixed fmt on tests

* work in progress

* reduced code

* fixed naming conventions

* added line in code

* fixed server bug, merged new code to server, added env

* fixed linter; getting cright issues on piecestore

* added comments for what passes on the creds
2018-06-04 09:45:07 -07:00
Bryan White
9aeff0ab57 dep init (#56)
ignoring jenkins
2018-05-29 20:28:46 -06:00
Kaloyan Raev
a10ffc2ae6 Ignore VSCode settings and debug files (#41) 2018-05-10 12:23:26 -04:00
JT Olds
4f40280c71 some small import cleanups, go generate overlay protos (#25)
checking in go generated stuff is recommended by
https://blog.golang.org/generate
2018-05-01 13:24:32 -04:00
Dennis Coyle
9e31cfc7a0 overlay proto 2018-04-16 10:09:01 -06:00
Dennis Coyle
250b36f27b wip on structure 2018-04-06 12:32:34 -04:00