JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,257 Commits 5 Branches 0 Tags 94 MiB
60035caade
Commit Graph

363 Commits

Author SHA1 Message Date
Egon Elbre
4117600f32
Cleanup repository a bit (#1203) 2019-02-01 20:21:00 +02:00
Bryan White
2a88642c55 yea, that's really all it takes to fix this (#1202) 2019-01-31 22:57:14 -05:00
Kaloyan Raev
5ae47c0de4
V3-1127 Properly initialize default config dir in storagenode binary (#1193) 2019-01-31 18:57:44 +02:00
Cameron
1403b15cc7
payments accesses database directly (#1135) 
* move payments command into satellite/main.go

* flag for db connection string in paymentsCmd

* refactor payments to satellite subcommand

* reports command, add payments arg descriptions

* report data prints to stdout unless --out is set

* fix small error in csv columns
 2019-01-30 16:44:50 -05:00
Egon Elbre
d5346982c2
Delete provider package (#1177) 2019-01-30 22:47:21 +02:00
Alexander Leitner
7c741c16d1
Readable sizes in dashboard (#1180) 
* Readable sizes

* Use tabwriter

* Added more addresses to dashboard stats

* Use kad to look up addresses

* Move dashboard command to dashboard.go
 2019-01-30 14:49:25 -05:00
Egon Elbre
471ed4ab2c
Remove empty file (#1175) 2019-01-30 12:40:53 +02:00
Jennifer Li Johnson
6355ae7a35
closes db in satellite, storage node and bootstrap cmdRun (#1159) 2019-01-30 00:22:58 -05:00
Egon Elbre
6132ce86b7
Remove utils.LogClose (#1169) 2019-01-29 22:42:27 +02:00
Egon Elbre
1df81b1460
Separate garbage collect logic from psdb (#1167) 2019-01-29 17:41:01 +02:00
Egon Elbre
f49436268a
storj-sim: read in config values for gateway (#1162) 2019-01-29 16:23:30 +02:00
Dylan Lott
bb2588e9ae
Refactors kademlia inspector to use FindNear (#1157) 
* WIP

* update identity paths to get dump-nodes working

* Remove commented out code

* remove comment

* updates from code reviews

* updates linter
 2019-01-28 15:53:37 -07:00
Bill Thorp
60946c2024
make bandwidth agreements sensible: without []byte's (#1152) 
removed []byte's from bandwidth agreement protocol buffers
 2019-01-28 14:45:25 -05:00
Egon Elbre
04d3699107
storj-sim configs (#1140) 2019-01-28 16:48:49 +02:00
Timothy Adams
2dbc06f7e1
fixing entrypoint script looking for invalid flag (#1147) 
* fixing entrypoint script looking for invalid flag

* fixed other entrypoints

* removed ca_difficulty entries from compose.yaml
 2019-01-28 07:53:16 -05:00
Bryan White
2b20acbec9
identity cleanup (#1145) 2019-01-26 15:59:53 +01:00
Dylan Lott
1385a2d6bd
Storagenode online confirmation ping (#1144) 
* initial commit of dashboard connection tools

* Currently working against default configs

* Update configs and remove hardcoded addresses

* Update config handling

* adds external address param for tunneling nodes

* remove unnecessary address check logic
 2019-01-25 10:46:09 -07:00
Bryan White
7bed8050aa
Improve identity cli ux: (#1142) 2019-01-25 17:55:45 +01:00
Jennifer Li Johnson
5397efe5c1
removes standalone payments service (#1134) 
* remove standalone payments service - will integrate within satellite directly

* regenerate proto files
 2019-01-25 11:23:25 -05:00
Egon Elbre
85b43926b4
Separate identity from server config (#1138) 2019-01-25 16:54:54 +02:00
Egon Elbre
a1027c7836
Make inmemory a separate command (#1127) 2019-01-24 23:04:29 +02:00
Egon Elbre
187e9b2138
Code consistency between peers (#1126) 2019-01-24 22:28:06 +02:00
Jennifer Li Johnson
856b98997c
updates copyright 2018 to 2019 (#1133) 2019-01-24 15:15:10 -05:00
Jennifer Li Johnson
70940b8710
Making sure payments works (#1122) 
* byte hours

* updates comment

* removes payment test data methods

* wip

* adds tally and rollup to peer

* remove sim test for payment (for now)

* lint error
 2019-01-24 13:44:14 -05:00
JT Olio
d87aa11fd0
Identity cleanups (#1129) 
* identity: fix unexpected control-c nil-dereference stacktrace

Change-Id: I3366c26908736f96b3c0828d0fd7b5586f3cc2c2

* identity: newlines

Change-Id: Ia31b1a0066b7f9d84043f3fc5669c6efc710b546

* identity: make default signer address

Change-Id: Ibb051397756cb76721b98851fcfa3d9e62b580db
 2019-01-24 10:23:45 -07:00
Yaroslav Vorobiov
f437effaef
Add console to satellite peer (#1120) 2019-01-24 18:26:36 +02:00
Egon Elbre
28427e7fad
Change identity command (#1128) 2019-01-24 17:41:16 +02:00
Egon Elbre
5de7f8af7f
Satellite Peer (#1119) 2019-01-23 21:58:44 +02:00
Michal Niewrzal
3fdb47e31c Try accessing bootstrap node for verification (#1089) 2019-01-23 17:48:46 +02:00
Egon Elbre
05b96f95a3
Change default difficulty to 30 (#1113) 2019-01-23 13:36:19 +02:00
Egon Elbre
b6c61cdd55
Use storagenode.Peer for storagenode (#1107) 2019-01-23 12:39:03 +02:00
Egon Elbre
99d3b7a3c8
Fix import grouping (#1111) 2019-01-22 17:48:23 +02:00
Bryan White
322a2813f7
oops (#1110) 2019-01-22 15:34:40 +01:00
Egon Elbre
6c48d69c38
Disable mounting for bsd-like systems (#1109) 
* Disable mounting for bsd-like
* Remove debug code
 2019-01-22 15:28:57 +02:00
Bryan White
8edfd43abb
certificates config and --certs-dir fixes (#1093) 2019-01-22 13:35:48 +01:00
Alexander Leitner
aabcceef52 Don't create identity when running dash (#1098) 
* Don't create identity when running dash

* only use configs for connecting with dash, delete unused client example

* use server address
 2019-01-18 18:08:45 +01:00
Dennis Coyle
369ca7d6fe
Coyle/cert inspector (#1094) 
* change default BootstrapAddr

* load cert files

* goimports

* linting gods appeased
 2019-01-18 10:02:35 -05:00
paul cannon
d6c2737521
fix some email list parsing in certificates tool (#1095) 
this avoids getting an "userID cannot be empty" error when using an
emails list that has a newline at the end.
 2019-01-18 10:01:55 -05:00
Dylan Lott
7bc1944a08 Dashboard updates (#1087) 2019-01-18 05:41:25 -05:00
Egon Elbre
bbf81f2479 Consolidate identity management to identity cli commands (#1083) 
* Consolidate identity management:

Move identity cretaion/signing out of storagenode setup command.

* fixes

* linters

* Consolidate identity management:

Move identity cretaion/signing out of storagenode setup command.

* fixes

* sava backups before saving signed certs

* add "-prebuilt-test-cmds" test flag

* linters

* prepare cli tests for travis

* linter fixes

* more fixes

* linter gods

* sp/sdk/sim

* remove ca.difficulty

* remove unused difficulty

* return setup to its rightful place

* wip travis

* Revert "wip travis"

This reverts commit 56834849dcf066d3cc0a4f139033fc3f6d7188ca.

* typo in travis.yaml

* remove tests

* remove more

* make it only create one identity at a time for consistency

* add config-dir for consitency

* add identity creation to storj-sim

* add flags

* simplify

* fix nolint and compile

* prevent overwrite and pass difficulty, concurrency, and parent creds

* goimports
 2019-01-18 11:36:58 +01:00
Kaloyan Raev
82f5bb6072 Fix panic in inspector if ping fails (#1088) 2019-01-17 16:14:34 -05:00
Natalie Villasana
3bf7c7d43b
nonessential storj-sim edits (#1086) 2019-01-17 15:59:26 -05:00
Jennifer Li Johnson
c4b876ddc5
Jj/bytes (#1085) 
* run tally and rollup

* sets dev default tally and rollup intervals
 2019-01-17 14:39:32 -05:00
Jennifer Li Johnson
e6fbf63620
Generate Payments Report (#1079) 2019-01-17 13:34:13 -05:00
Egon Elbre
c89609abbe
Remove captplanet (#1070) 2019-01-17 04:35:22 -05:00
Michal Niewrzal
f9abb4584c
Storagenode logs and config improvements (#1075) 
* Add more info to SN logs

* remove config-dir from user config

* add output where config was stored

* add message for successful connection

* fix linter

* remove storage.path from user config

* resolve config path

* move success  message to info

* log improvements
 2019-01-16 21:41:43 -05:00
Egon Elbre
9dbb6a101d
Rename storj-sdk to storj-sim (#1078) 2019-01-16 18:09:57 -05:00
Michal Niewrzal
677edccab7 Add address param for SN dashboard (#1076) 2019-01-16 16:52:17 -05:00
Dylan Lott
82b9b54695
storagenode dashboard is working and passing tests locally (#1072) 
* storagenode dashboard is working and passing tests locally

* linter fixes

* linter fixes

* moar linter fixes god

* review fixes and updates

* linter fixes
 2019-01-16 16:09:13 -05:00
paul cannon
681d77c41a
use _real_ cert-path to get node ID for metrics (#1058) 
apparently, the presence of an "Identity" attribute in a config struct
does not imply that anything will fill it in or use it.

i tested all of these executables manually and individually this time.
 2019-01-16 15:43:29 -05:00
Natalie Villasana
5aa2b02930
adds pretty printing for inspector kad dump-nodes (#1074) 2019-01-16 15:40:30 -05:00
Maximillian von Briesen
f97883f990
remove overlay count from inspector kad count command (#1073) 2019-01-16 14:44:26 -05:00
Michal Niewrzal
ab0c18de3a
Fix SN overrided params (#1049) 
* Fix SN overrided params

* remove one more override

* fix missing var

* overriding default for storage node

* restore removed code

* fix docker
 2019-01-15 10:43:10 -05:00
paul cannon
248ee6438f
use Node ID for metrics ID (#1052) 
This change removes automatic metrics reporting for everything going
through process.Exec(), and re-adds metrics reporting for those commands
which are expected to be long-lived. Other commands (which may have been
intermittently sending metrics before this, if they ran unusually long)
will no longer send any metrics.

For commands where it makes sense, a node ID is used as the metrics ID.
 2019-01-15 10:02:54 -05:00
Kaloyan Raev
a63abf8fab
Save only user-specific flags to storage node's config.yaml (#1051) 2019-01-15 15:55:33 +02:00
Yehor Butko
e38cf8f50d
Renaming and moving pkg/satellite to satellite/console (#1054) 
* [WIP] V3-853 Merge the satellite DB into the master database

* Removing consoleDB from satelliteDB

* Fixing tests for satellite/console

* fixing linter

* sorting imports in satellite/console

* fixing console config

* fixing linter
 2019-01-15 15:03:24 +02:00
Alexander Leitner
bfde515391
Clean up Storage node setup (#1013) 
* Edit config on Setup

* Default to 1TiB storage space and 500GiB bandwidth

* Use human readable formats

* Use memory

* units of 1024 are measured with KiB/MiB etc

* pkg/cfgstruct: allow values to be configured with human readable sizes

Change-Id: Ic4e9ae461516d1d26fb81f6e44c5ac5cfccf777f

* Modify tests

* Removed comments

* More merge conflict stuff resolved

* Fix lint

* test fixin

Change-Id: I3a008206bf03a4446da19f642a2f9c1f9acaae36

* Remove commented code but secretly leave it in the histroy forever

* Move flag definition to struct
 2019-01-14 16:19:15 -05:00
JT Olio
87925789de pkg/inspector: split functionality to be per-service (#1029) 
now kad inspector features exist on every server that has
kademlia running. likewise, overlay and statdb.

this means kad inspection features are now available on
storage nodes

Change-Id: I343c873552341de13302bfb7a5d79cccc84fc6b8
 2019-01-14 11:47:22 -07:00
Maximillian von Briesen
43ed9dea86
Remove overwrite flags (#1040) 
remove overwrite flags from uplink, satellite, storagenode, captplanet, bootstrap

https://storjlabs.atlassian.net/browse/V3-1013
 2019-01-14 10:57:58 -05:00
Michal Niewrzal
de5f592e90 Fix kademlia bootstrap address ignored (#1038) 2019-01-14 10:15:26 -05:00
Bryan White
b1b8f66462
Improvements to certificates commands: (#1012) 2019-01-14 15:28:52 +01:00
Philip Hutchins
b5f51014d3
Adding additional required values for satellite configuration (#1036) 
* Adding other top level vars to be parsed into config

* Adding additional vars for parsing

* Fixing shell script conditionals in remaining entrypoints

* Adding null default for remaining env variable if statements
 2019-01-11 22:47:18 -05:00
Egon Elbre
10005d869c
Fail integration tests on panic (#1032) 2019-01-11 18:18:16 +02:00
Bryan White
9e1ec633d8
Improve cert signing integration test+: (#997) 2019-01-11 15:59:35 +01:00
Michal Niewrzal
e13f7c7a7d
Fix unstarted pointerdb responsibility (#1033) 2019-01-11 13:45:11 +01:00
Egon Elbre
8893884044
convert piecestorage into a struct (#1024) 2019-01-11 13:26:39 +02:00
Egon Elbre
34125fe614 Reduce test CA difficulty (#1010) 
* reduce test CA difficulty

* reduce docker ca difficulty
 2019-01-10 15:23:33 +01:00
Bill Thorp
fb4a11ebb0
Differentiate GET and PUT operations in the tally service (#965) 
* draft of new bandwidth agreement types
* updated storagenode report for new types
* use correct pba types
 2019-01-10 06:41:57 -05:00
Egon Elbre
1d741f29a7
Fix captplanet race (#1021) 2019-01-10 13:26:34 +02:00
Philip Hutchins
c5549befd2 Reordering when we set the API_KEY env var in the docker entrypoint (#1017) 2019-01-10 12:35:06 +02:00
Egon Elbre
ff2e77ff7a
Fix captplanet race (#1014) 2019-01-09 23:18:26 +02:00
JT Olio
f8ee5f88e5 cmd/bootstrap: a kademlia-only server (#1001) 2019-01-09 17:59:51 +02:00
Bill Thorp
ea47d27c1b
Get satellites to send BW agreements to via KAD (#985) 
* moved from hardcoded overlay IP to kad
 2019-01-09 09:41:50 -05:00
Egon Elbre
2ee4eed465
Add missing satellite arguments (#998) 2019-01-09 01:41:01 +02:00
Egon Elbre
387fb14e23
storj-sdk (#969) 2019-01-08 17:24:15 +02:00
Egon Elbre
db5a990719
lint imports grouping (#993) 2019-01-08 16:05:14 +02:00
Michal Niewrzal
58b16c2ada
Fix for uplink setup command (#989) 2019-01-08 14:30:07 +01:00
JT Olio
1faeeb49d5 prepare key generation for launch (#979) 
* pkg/identity: use sha256 instead of sha3 for pow

Change-Id: I9b7a4f2c3e624a6e248a233e3653eaccaf23c6f3

* pkg/identity: restructure key generation a bit

Change-Id: I0061a5cc62f04b0c86ffbf046519d5c0a154e896

* cmd/identity: indefinite key generation command

you can start this command and leave it running and it will fill up your
hard drive with node certificate authority private keys ordered by
difficulty.

Change-Id: I61c7a3438b9ff6656e74b8d74fef61e557e4d95a

* pkg/storj: more node id difficulty testing

Change-Id: Ie56b1859aa14ec6ef5973caf42aacb4c494b87c7

* review comments

Change-Id: Iff019aa8121a7804f10c248bf2e578189e5b829d
 2019-01-07 13:02:22 -05:00
Michal Niewrzal
4e857ea133
Add setup tag for config structs (#968) 
* Fix ignored setup arguments

* fix linter errors

* small params cleanup

* fix integration tests

* cleanup in configs

* Add `setup` tag for config structs

* fix broken if statement

* cleanup captplanet config

* remove reduntant return

* add missing Signer config

* review comments

* local variable renamed

* remove unused var
 2019-01-07 12:06:10 +01:00
Michal Niewrzal
4be58e2a76
Fix ignored setup arguments (#944) 
* Fix ignored setup arguments

* fix linter errors

* small params cleanup

* fix integration tests

* cleanup in configs

* Restart Travis
 2019-01-07 10:48:16 +01:00
Kaloyan Raev
41fdc31ec0
Fix output message when overwriting storagenode configuration (#982) 2019-01-07 11:12:49 +02:00
Bryan White
b6611e2800
automate certificate signing in storage node setup (#954) 2019-01-04 18:23:23 +01:00
Matt Robinson
c0e6b62708
Test all-in-one (#900) 
* Add test for aio

* Don't trust the user to have images built for a version

* Make travis run the aio test

* Add missing values to docker-compose, sort some things, consider the gateway image

* today's changes

* config changed, again

* more fixes

* Expose satellite port on localhost:7778

* Add retries and a timeout around the big-testfile test in AIO

* Another config value changed

* Make this error message a little more useful

* Fix nil condition
 2019-01-03 14:54:27 -05:00
Jennifer Li Johnson
a2fa5c4c5a Proper NodeType Handling (#873) 
* adds enums to nodetype

* updating nodetype todos

* ran pb updates

* reorder nodetypes

* adding checks

* wip

* wip

* wip

* bug in test-captplanet

* wip

* add values to storagenode, satellite, captplanet binaries

* Cleanup

* more cleanup

* wip

* lint

* lint

* wip

* fixes bug

* regenerate protos

Change-Id: Id270212e8c7479e52641058042cf23b5317ab773

* limit node type changes to kademlia

Change-Id: I9c1a6cc4a79e05086627f0fdeb5028c62ce754f4

* dpanic

Change-Id: Id952a2ad13c807ebaea0ec0a875405e267d81c3e

* review comments

Change-Id: I7f9b77ef22779dd012fd490375b136014f51f834
 2019-01-02 11:47:34 -07:00
Egon Elbre
89db0fe9f5
storj-sdk base code (#799) 2019-01-02 20:07:49 +02:00
Bill Thorp
e4b3dab446
Add config editing via the CLI for storage nodes (#919) 
* added editor and updated storagenode cmd
* switched to custom yaml gen
 2019-01-02 10:12:07 -05:00
JT Olio
4a0749e44a Remove farmer references (#949) 2019-01-02 12:31:49 +02:00
JT Olio
2c916a04c3 pkg/provider: split into pkg/server, pkg/identity (#953) 2019-01-02 12:23:25 +02:00
Egon Elbre
cc8cce58dd
statreceiver nitpicking (#926) 2019-01-01 11:41:27 +02:00
Bryan White
4efb5c0a75
Authorization improvements (#925) 2018-12-31 10:45:43 -05:00
littleskunk
ea4c91b72d
repair storagenode diag command (#941) 
* repair storagenode diag command
 2018-12-28 11:01:02 +01:00
Egon Elbre
4346cd060f
Implement mutex around satellitedb (#932) 2018-12-27 11:56:25 +02:00
Matt Robinson
5224e92646
Fix detection of existing config in docker images (#929) 
* If the directory is somehow persistent, but the config file itself isn't created yet, create it

* Other improvements to thd storagenode Docker image

* Other improvements to thd satellite Docker image

* Other improvements to thd gateway Docker image

* Other improvements to thd uplink Docker image
 2018-12-21 11:56:57 -05:00
Cameron
f70b826fd4
repair queue masterDB support (#865) 
* add injuredsegment model to satellitedb.dbx

* add context to queue.RepairQueue interface

* use queue.RepairQueue interface, use masterdb
 2018-12-21 10:11:19 -05:00
Egon Elbre
ffbc9ea186
captplanet: add sleep after satellite startup (#927) 2018-12-21 15:32:50 +02:00
Bryan White
398379b149
CSR service (part 1): authorizations (#906) 2018-12-20 19:29:05 +01:00
JT Olio
5c222157da
Update README.md 2018-12-20 10:01:28 -07:00
Egon Elbre
d9b9ae6ffa
Cleanup overlay methods and names. (#914) 2018-12-20 15:57:54 +02:00
littleskunk
e1c3f11cfa
add overwrite flag (#913) 
* add storagenode setup --overwrite flag
* exit setup with error code
 2018-12-20 14:27:51 +01:00
Michal Niewrzal
bbe1dbb92c
Fix overlay commands after master db merge (#895) 
* Fix overlay commands after master db merge

* remove unused var

* remove unused comment
 2018-12-18 21:41:31 +01:00