Commit Graph

428 Commits

Author SHA1 Message Date
paul cannon
c35b93766d
Unite all cryptographic signing and verifying (#1244)
this change removes the cryptopasta dependency.

a couple possible sources of problem with this change:

 * the encoding used for ECDSA signatures on SignedMessage has changed.
   the encoding employed by cryptopasta was workable, but not the same
   as the encoding used for such signatures in the rest of the world
   (most particularly, on ECDSA signatures in X.509 certificates). I
   think we'll be best served by using one ECDSA signature encoding from
   here on, but if we need to use the old encoding for backwards
   compatibility with existing nodes, that can be arranged.

 * since there's already a breaking change in SignedMessage, I changed
   it to send and receive public keys in raw PKIX format, instead of
   PEM. PEM just adds unhelpful overhead for this case.
2019-02-07 14:39:20 -06:00
paul cannon
ef61c170b1
Consolidate key/cert/signature encoding and decoding (#1243) 2019-02-07 12:40:28 -06:00
Kaloyan Raev
0ffdaeebce
Disable colors in dashboard by default. --color flag to enable them (#1264) 2019-02-07 19:54:46 +02:00
paul cannon
32882daf38
SHA256Hash() and TBSBytes() needn't return error (#1242) 2019-02-07 11:08:52 -06:00
Stefan Benten
f5ab9e34e6 fix difficulty display (#1256) 2019-02-07 11:36:19 +02:00
paul cannon
0032147665 regroup things related to public-key cryptography (#1241) 2019-02-07 10:04:29 +01:00
Bryan White
3b54cf0e15
identity improvements: (#1215) 2019-02-06 17:40:55 +01:00
JT Olio
0c96353078
storagenode: fix port in entrypoint (#1252)
Change-Id: I07fa4cec447f2d6a3a3e308ad9b63bdc9dc1b161
2019-02-06 09:19:58 -07:00
Egon Elbre
60b0275a44
Enable ipv6 tests (#1204) 2019-02-06 14:47:00 +02:00
Bryan White
00554f0876
add certificates claims+ (#1216) 2019-02-06 12:00:26 +01:00
Bryan White
7b7e6c43f8
better batch-generation (#1219) 2019-02-06 09:04:12 +01:00
Yaroslav Vorobiov
2ff0d9d435
Add bucket to project relationship on satellite (#1143)
* integrate console api keys with buckets in pointerdb

* fix test

* fix tests kvmetainfo

* linter fix

* disable account activation

* fix test

* review fixes

* fix comments

* little refactoring

* remove debug println

* fix typo

* disable activation in a propper way

* fix test

* fix imports

* fix uplink count in testplanet

* move key creation to planet.newUplink
2019-02-05 19:22:17 +02:00
Michal Niewrzal
383cb05e8e
Divide uplink and gateway params set (#1026)
* Divide uplink and gateway params set

* attempt to fix docker

* attempt to fix all in one

* test

* more reorganization

* fix compilation error

* fix imports order

* fix dependency

* rename structs

* keep minio params for now

* review comments

* remove manual flag check
2019-02-04 16:50:06 +01:00
Bill Thorp
0f662b8e38 sql based tally (#1166)
* WIP

* wacky changes

* more

* it builds.... IT BUILDS!!!

* fixed SQL, broke out saving tallies for testing

* shorter lines

* fixed SQL, moved tally_test to testplanet

* lint

* WIP logic error preventing PUT and GETs to same serialnum

* fixed BWA test

* fixed temporary brain failure

* eliminated magic numbers

* cleaned up satellite uplink stats

* use errs.Combine instead

* thrashing

* fixed tally erroneous error msg

* fixed tally test

* lint

* SQL syntax attempt to fix

* spelling error

* made bwa db test resist old postgres data

* postgres pk error msg hunting

* postgres pk error msg hunting

* postgres

* err might be nil?

* fixed error logging bug

* hopefully solved postgres issue

* using rebind

* moved tests to _test package

* fixing test dirs

* finally made sense of Egons package name feedback

* UTC, array fixes
2019-02-01 13:50:12 -05:00
Egon Elbre
f7c692f844 Use base10 for all values (#1187) 2019-02-01 11:29:37 -07:00
Egon Elbre
4117600f32
Cleanup repository a bit (#1203) 2019-02-01 20:21:00 +02:00
Bryan White
2a88642c55 yea, that's really all it takes to fix this (#1202) 2019-01-31 22:57:14 -05:00
Kaloyan Raev
5ae47c0de4
V3-1127 Properly initialize default config dir in storagenode binary (#1193) 2019-01-31 18:57:44 +02:00
Cameron
1403b15cc7
payments accesses database directly (#1135)
* move payments command into satellite/main.go

* flag for db connection string in paymentsCmd

* refactor payments to satellite subcommand

* reports command, add payments arg descriptions

* report data prints to stdout unless --out is set

* fix small error in csv columns
2019-01-30 16:44:50 -05:00
Egon Elbre
d5346982c2
Delete provider package (#1177) 2019-01-30 22:47:21 +02:00
Alexander Leitner
7c741c16d1
Readable sizes in dashboard (#1180)
* Readable sizes

* Use tabwriter

* Added more addresses to dashboard stats

* Use kad to look up addresses

* Move dashboard command to dashboard.go
2019-01-30 14:49:25 -05:00
Egon Elbre
471ed4ab2c
Remove empty file (#1175) 2019-01-30 12:40:53 +02:00
Jennifer Li Johnson
6355ae7a35
closes db in satellite, storage node and bootstrap cmdRun (#1159) 2019-01-30 00:22:58 -05:00
Egon Elbre
6132ce86b7
Remove utils.LogClose (#1169) 2019-01-29 22:42:27 +02:00
Egon Elbre
1df81b1460
Separate garbage collect logic from psdb (#1167) 2019-01-29 17:41:01 +02:00
Egon Elbre
f49436268a
storj-sim: read in config values for gateway (#1162) 2019-01-29 16:23:30 +02:00
Dylan Lott
bb2588e9ae
Refactors kademlia inspector to use FindNear (#1157)
* WIP

* update identity paths to get dump-nodes working

* Remove commented out code

* remove comment

* updates from code reviews

* updates linter
2019-01-28 15:53:37 -07:00
Bill Thorp
60946c2024
make bandwidth agreements sensible: without []byte's (#1152)
removed []byte's from bandwidth agreement protocol buffers
2019-01-28 14:45:25 -05:00
Egon Elbre
04d3699107
storj-sim configs (#1140) 2019-01-28 16:48:49 +02:00
Timothy Adams
2dbc06f7e1
fixing entrypoint script looking for invalid flag (#1147)
* fixing entrypoint script looking for invalid flag

* fixed other entrypoints

* removed ca_difficulty entries from compose.yaml
2019-01-28 07:53:16 -05:00
Bryan White
2b20acbec9
identity cleanup (#1145) 2019-01-26 15:59:53 +01:00
Dylan Lott
1385a2d6bd
Storagenode online confirmation ping (#1144)
* initial commit of dashboard connection tools

* Currently working against default configs

* Update configs and remove hardcoded addresses

* Update config handling

* adds external address param for tunneling nodes

* remove unnecessary address check logic
2019-01-25 10:46:09 -07:00
Bryan White
7bed8050aa
Improve identity cli ux: (#1142) 2019-01-25 17:55:45 +01:00
Jennifer Li Johnson
5397efe5c1
removes standalone payments service (#1134)
* remove standalone payments service - will integrate within satellite directly

* regenerate proto files
2019-01-25 11:23:25 -05:00
Egon Elbre
85b43926b4
Separate identity from server config (#1138) 2019-01-25 16:54:54 +02:00
Egon Elbre
a1027c7836
Make inmemory a separate command (#1127) 2019-01-24 23:04:29 +02:00
Egon Elbre
187e9b2138
Code consistency between peers (#1126) 2019-01-24 22:28:06 +02:00
Jennifer Li Johnson
856b98997c
updates copyright 2018 to 2019 (#1133) 2019-01-24 15:15:10 -05:00
Jennifer Li Johnson
70940b8710
Making sure payments works (#1122)
* byte hours

* updates comment

* removes payment test data methods

* wip

* adds tally and rollup to peer

* remove sim test for payment (for now)

* lint error
2019-01-24 13:44:14 -05:00
JT Olio
d87aa11fd0
Identity cleanups (#1129)
* identity: fix unexpected control-c nil-dereference stacktrace

Change-Id: I3366c26908736f96b3c0828d0fd7b5586f3cc2c2

* identity: newlines

Change-Id: Ia31b1a0066b7f9d84043f3fc5669c6efc710b546

* identity: make default signer address

Change-Id: Ibb051397756cb76721b98851fcfa3d9e62b580db
2019-01-24 10:23:45 -07:00
Yaroslav Vorobiov
f437effaef
Add console to satellite peer (#1120) 2019-01-24 18:26:36 +02:00
Egon Elbre
28427e7fad
Change identity command (#1128) 2019-01-24 17:41:16 +02:00
Egon Elbre
5de7f8af7f
Satellite Peer (#1119) 2019-01-23 21:58:44 +02:00
Michal Niewrzal
3fdb47e31c Try accessing bootstrap node for verification (#1089) 2019-01-23 17:48:46 +02:00
Egon Elbre
05b96f95a3
Change default difficulty to 30 (#1113) 2019-01-23 13:36:19 +02:00
Egon Elbre
b6c61cdd55
Use storagenode.Peer for storagenode (#1107) 2019-01-23 12:39:03 +02:00
Egon Elbre
99d3b7a3c8
Fix import grouping (#1111) 2019-01-22 17:48:23 +02:00
Bryan White
322a2813f7
oops (#1110) 2019-01-22 15:34:40 +01:00
Egon Elbre
6c48d69c38
Disable mounting for bsd-like systems (#1109)
* Disable mounting for bsd-like
* Remove debug code
2019-01-22 15:28:57 +02:00
Bryan White
8edfd43abb
certificates config and --certs-dir fixes (#1093) 2019-01-22 13:35:48 +01:00
Alexander Leitner
aabcceef52 Don't create identity when running dash (#1098)
* Don't create identity when running dash

* only use configs for connecting with dash, delete unused client example

* use server address
2019-01-18 18:08:45 +01:00
Dennis Coyle
369ca7d6fe
Coyle/cert inspector (#1094)
* change default BootstrapAddr

* load cert files

* goimports

* linting gods appeased
2019-01-18 10:02:35 -05:00
paul cannon
d6c2737521
fix some email list parsing in certificates tool (#1095)
this avoids getting an "userID cannot be empty" error when using an
emails list that has a newline at the end.
2019-01-18 10:01:55 -05:00
Dylan Lott
7bc1944a08 Dashboard updates (#1087) 2019-01-18 05:41:25 -05:00
Egon Elbre
bbf81f2479 Consolidate identity management to identity cli commands (#1083)
* Consolidate identity management:

Move identity cretaion/signing out of storagenode setup command.

* fixes

* linters

* Consolidate identity management:

Move identity cretaion/signing out of storagenode setup command.

* fixes

* sava backups before saving signed certs

* add "-prebuilt-test-cmds" test flag

* linters

* prepare cli tests for travis

* linter fixes

* more fixes

* linter gods

* sp/sdk/sim

* remove ca.difficulty

* remove unused difficulty

* return setup to its rightful place

* wip travis

* Revert "wip travis"

This reverts commit 56834849dcf066d3cc0a4f139033fc3f6d7188ca.

* typo in travis.yaml

* remove tests

* remove more

* make it only create one identity at a time for consistency

* add config-dir for consitency

* add identity creation to storj-sim

* add flags

* simplify

* fix nolint and compile

* prevent overwrite and pass difficulty, concurrency, and parent creds

* goimports
2019-01-18 11:36:58 +01:00
Kaloyan Raev
82f5bb6072 Fix panic in inspector if ping fails (#1088) 2019-01-17 16:14:34 -05:00
Natalie Villasana
3bf7c7d43b
nonessential storj-sim edits (#1086) 2019-01-17 15:59:26 -05:00
Jennifer Li Johnson
c4b876ddc5
Jj/bytes (#1085)
* run tally and rollup

* sets dev default tally and rollup intervals
2019-01-17 14:39:32 -05:00
Jennifer Li Johnson
e6fbf63620
Generate Payments Report (#1079) 2019-01-17 13:34:13 -05:00
Egon Elbre
c89609abbe
Remove captplanet (#1070) 2019-01-17 04:35:22 -05:00
Michal Niewrzal
f9abb4584c
Storagenode logs and config improvements (#1075)
* Add more info to SN logs

* remove config-dir from user config

* add output where config was stored

* add message for successful connection

* fix linter

* remove storage.path from user config

* resolve config path

* move success  message to info

* log improvements
2019-01-16 21:41:43 -05:00
Egon Elbre
9dbb6a101d
Rename storj-sdk to storj-sim (#1078) 2019-01-16 18:09:57 -05:00
Michal Niewrzal
677edccab7 Add address param for SN dashboard (#1076) 2019-01-16 16:52:17 -05:00
Dylan Lott
82b9b54695
storagenode dashboard is working and passing tests locally (#1072)
* storagenode dashboard is working and passing tests locally

* linter fixes

* linter fixes

* moar linter fixes god

* review fixes and updates

* linter fixes
2019-01-16 16:09:13 -05:00
paul cannon
681d77c41a
use _real_ cert-path to get node ID for metrics (#1058)
apparently, the presence of an "Identity" attribute in a config struct
does not imply that anything will fill it in or use it.

i tested all of these executables manually and individually this time.
2019-01-16 15:43:29 -05:00
Natalie Villasana
5aa2b02930
adds pretty printing for inspector kad dump-nodes (#1074) 2019-01-16 15:40:30 -05:00
Maximillian von Briesen
f97883f990
remove overlay count from inspector kad count command (#1073) 2019-01-16 14:44:26 -05:00
Michal Niewrzal
ab0c18de3a
Fix SN overrided params (#1049)
* Fix SN overrided params

* remove one more override

* fix missing var

* overriding default for storage node

* restore removed code

* fix docker
2019-01-15 10:43:10 -05:00
paul cannon
248ee6438f
use Node ID for metrics ID (#1052)
This change removes automatic metrics reporting for everything going
through process.Exec(), and re-adds metrics reporting for those commands
which are expected to be long-lived. Other commands (which may have been
intermittently sending metrics before this, if they ran unusually long)
will no longer send any metrics.

For commands where it makes sense, a node ID is used as the metrics ID.
2019-01-15 10:02:54 -05:00
Kaloyan Raev
a63abf8fab
Save only user-specific flags to storage node's config.yaml (#1051) 2019-01-15 15:55:33 +02:00
Yehor Butko
e38cf8f50d
Renaming and moving pkg/satellite to satellite/console (#1054)
* [WIP] V3-853 Merge the satellite DB into the master database

* Removing consoleDB from satelliteDB

* Fixing tests for satellite/console

* fixing linter

* sorting imports in satellite/console

* fixing console config

* fixing linter
2019-01-15 15:03:24 +02:00
Alexander Leitner
bfde515391
Clean up Storage node setup (#1013)
* Edit config on Setup

* Default to 1TiB storage space and 500GiB bandwidth

* Use human readable formats

* Use memory

* units of 1024 are measured with KiB/MiB etc

* pkg/cfgstruct: allow values to be configured with human readable sizes

Change-Id: Ic4e9ae461516d1d26fb81f6e44c5ac5cfccf777f

* Modify tests

* Removed comments

* More merge conflict stuff resolved

* Fix lint

* test fixin

Change-Id: I3a008206bf03a4446da19f642a2f9c1f9acaae36

* Remove commented code but secretly leave it in the histroy forever

* Move flag definition to struct
2019-01-14 16:19:15 -05:00
JT Olio
87925789de pkg/inspector: split functionality to be per-service (#1029)
now kad inspector features exist on every server that has
kademlia running. likewise, overlay and statdb.

this means kad inspection features are now available on
storage nodes

Change-Id: I343c873552341de13302bfb7a5d79cccc84fc6b8
2019-01-14 11:47:22 -07:00
Maximillian von Briesen
43ed9dea86
Remove overwrite flags (#1040)
remove overwrite flags from uplink, satellite, storagenode, captplanet, bootstrap

https://storjlabs.atlassian.net/browse/V3-1013
2019-01-14 10:57:58 -05:00
Michal Niewrzal
de5f592e90 Fix kademlia bootstrap address ignored (#1038) 2019-01-14 10:15:26 -05:00
Bryan White
b1b8f66462
Improvements to certificates commands: (#1012) 2019-01-14 15:28:52 +01:00
Philip Hutchins
b5f51014d3
Adding additional required values for satellite configuration (#1036)
* Adding other top level vars to be parsed into config

* Adding additional vars for parsing

* Fixing shell script conditionals in remaining entrypoints

* Adding null default for remaining env variable if statements
2019-01-11 22:47:18 -05:00
Egon Elbre
10005d869c
Fail integration tests on panic (#1032) 2019-01-11 18:18:16 +02:00
Bryan White
9e1ec633d8
Improve cert signing integration test+: (#997) 2019-01-11 15:59:35 +01:00
Michal Niewrzal
e13f7c7a7d
Fix unstarted pointerdb responsibility (#1033) 2019-01-11 13:45:11 +01:00
Egon Elbre
8893884044
convert piecestorage into a struct (#1024) 2019-01-11 13:26:39 +02:00
Egon Elbre
34125fe614 Reduce test CA difficulty (#1010)
* reduce test CA difficulty

* reduce docker ca difficulty
2019-01-10 15:23:33 +01:00
Bill Thorp
fb4a11ebb0
Differentiate GET and PUT operations in the tally service (#965)
* draft of new bandwidth agreement types
* updated storagenode report for new types
* use correct pba types
2019-01-10 06:41:57 -05:00
Egon Elbre
1d741f29a7
Fix captplanet race (#1021) 2019-01-10 13:26:34 +02:00
Philip Hutchins
c5549befd2 Reordering when we set the API_KEY env var in the docker entrypoint (#1017) 2019-01-10 12:35:06 +02:00
Egon Elbre
ff2e77ff7a
Fix captplanet race (#1014) 2019-01-09 23:18:26 +02:00
JT Olio
f8ee5f88e5 cmd/bootstrap: a kademlia-only server (#1001) 2019-01-09 17:59:51 +02:00
Bill Thorp
ea47d27c1b
Get satellites to send BW agreements to via KAD (#985)
* moved from hardcoded overlay IP to kad
2019-01-09 09:41:50 -05:00
Egon Elbre
2ee4eed465
Add missing satellite arguments (#998) 2019-01-09 01:41:01 +02:00
Egon Elbre
387fb14e23
storj-sdk (#969) 2019-01-08 17:24:15 +02:00
Egon Elbre
db5a990719
lint imports grouping (#993) 2019-01-08 16:05:14 +02:00
Michal Niewrzal
58b16c2ada
Fix for uplink setup command (#989) 2019-01-08 14:30:07 +01:00
JT Olio
1faeeb49d5 prepare key generation for launch (#979)
* pkg/identity: use sha256 instead of sha3 for pow

Change-Id: I9b7a4f2c3e624a6e248a233e3653eaccaf23c6f3

* pkg/identity: restructure key generation a bit

Change-Id: I0061a5cc62f04b0c86ffbf046519d5c0a154e896

* cmd/identity: indefinite key generation command

you can start this command and leave it running and it will fill up your
hard drive with node certificate authority private keys ordered by
difficulty.

Change-Id: I61c7a3438b9ff6656e74b8d74fef61e557e4d95a

* pkg/storj: more node id difficulty testing

Change-Id: Ie56b1859aa14ec6ef5973caf42aacb4c494b87c7

* review comments

Change-Id: Iff019aa8121a7804f10c248bf2e578189e5b829d
2019-01-07 13:02:22 -05:00
Michal Niewrzal
4e857ea133
Add setup tag for config structs (#968)
* Fix ignored setup arguments

* fix linter errors

* small params cleanup

* fix integration tests

* cleanup in configs

* Add `setup` tag for config structs

* fix broken if statement

* cleanup captplanet config

* remove reduntant return

* add missing Signer config

* review comments

* local variable renamed

* remove unused var
2019-01-07 12:06:10 +01:00
Michal Niewrzal
4be58e2a76
Fix ignored setup arguments (#944)
* Fix ignored setup arguments

* fix linter errors

* small params cleanup

* fix integration tests

* cleanup in configs

* Restart Travis
2019-01-07 10:48:16 +01:00
Kaloyan Raev
41fdc31ec0
Fix output message when overwriting storagenode configuration (#982) 2019-01-07 11:12:49 +02:00
Bryan White
b6611e2800
automate certificate signing in storage node setup (#954) 2019-01-04 18:23:23 +01:00
Matt Robinson
c0e6b62708
Test all-in-one (#900)
* Add test for aio

* Don't trust the user to have images built for a version

* Make travis run the aio test

* Add missing values to docker-compose, sort some things, consider the gateway image

* today's changes

* config changed, again

* more fixes

* Expose satellite port on localhost:7778

* Add retries and a timeout around the big-testfile test in AIO

* Another config value changed

* Make this error message a little more useful

* Fix nil condition
2019-01-03 14:54:27 -05:00
Jennifer Li Johnson
a2fa5c4c5a Proper NodeType Handling (#873)
* adds enums to nodetype

* updating nodetype todos

* ran pb updates

* reorder nodetypes

* adding checks

* wip

* wip

* wip

* bug in test-captplanet

* wip

* add values to storagenode, satellite, captplanet binaries

* Cleanup

* more cleanup

* wip

* lint

* lint

* wip

* fixes bug

* regenerate protos

Change-Id: Id270212e8c7479e52641058042cf23b5317ab773

* limit node type changes to kademlia

Change-Id: I9c1a6cc4a79e05086627f0fdeb5028c62ce754f4

* dpanic

Change-Id: Id952a2ad13c807ebaea0ec0a875405e267d81c3e

* review comments

Change-Id: I7f9b77ef22779dd012fd490375b136014f51f834
2019-01-02 11:47:34 -07:00
Egon Elbre
89db0fe9f5
storj-sdk base code (#799) 2019-01-02 20:07:49 +02:00
Bill Thorp
e4b3dab446
Add config editing via the CLI for storage nodes (#919)
* added editor and updated storagenode cmd
* switched to custom yaml gen
2019-01-02 10:12:07 -05:00
JT Olio
4a0749e44a Remove farmer references (#949) 2019-01-02 12:31:49 +02:00
JT Olio
2c916a04c3 pkg/provider: split into pkg/server, pkg/identity (#953) 2019-01-02 12:23:25 +02:00
Egon Elbre
cc8cce58dd
statreceiver nitpicking (#926) 2019-01-01 11:41:27 +02:00
Bryan White
4efb5c0a75
Authorization improvements (#925) 2018-12-31 10:45:43 -05:00
littleskunk
ea4c91b72d
repair storagenode diag command (#941)
* repair storagenode diag command
2018-12-28 11:01:02 +01:00
Egon Elbre
4346cd060f
Implement mutex around satellitedb (#932) 2018-12-27 11:56:25 +02:00
Matt Robinson
5224e92646
Fix detection of existing config in docker images (#929)
* If the directory is somehow persistent, but the config file itself isn't created yet, create it

* Other improvements to thd storagenode Docker image

* Other improvements to thd satellite Docker image

* Other improvements to thd gateway Docker image

* Other improvements to thd uplink Docker image
2018-12-21 11:56:57 -05:00
Cameron
f70b826fd4
repair queue masterDB support (#865)
* add injuredsegment model to satellitedb.dbx

* add context to queue.RepairQueue interface

* use queue.RepairQueue interface, use masterdb
2018-12-21 10:11:19 -05:00
Egon Elbre
ffbc9ea186
captplanet: add sleep after satellite startup (#927) 2018-12-21 15:32:50 +02:00
Bryan White
398379b149
CSR service (part 1): authorizations (#906) 2018-12-20 19:29:05 +01:00
JT Olio
5c222157da
Update README.md 2018-12-20 10:01:28 -07:00
Egon Elbre
d9b9ae6ffa
Cleanup overlay methods and names. (#914) 2018-12-20 15:57:54 +02:00
littleskunk
e1c3f11cfa
add overwrite flag (#913)
* add storagenode setup --overwrite flag
* exit setup with error code
2018-12-20 14:27:51 +01:00
Michal Niewrzal
bbe1dbb92c
Fix overlay commands after master db merge (#895)
* Fix overlay commands after master db merge

* remove unused var

* remove unused comment
2018-12-18 21:41:31 +01:00
Michal Niewrzal
a7489c2760
Validate farmers wallet and email configuration (#897)
* Validate farmers wallet and email configuration

* remove usless part of regexp

* move to zap logger

* logging valid wallet and email
2018-12-18 18:37:56 +01:00
Matt Robinson
23fa3ac24a
Fix all-in-one, again (#896) 2018-12-18 09:27:46 -05:00
Bryan White
4eb55017c8
Cert revocation CLI (#848)
* wip

* allow identity and CA configs to save cert/key separately

* fixes

* linter and default path fixes

* review fixes

* fixes:

+ review fixes
+ bug fixes
+ add extensions command

* linter fixes

* fix ca revoke description

* review fixes
2018-12-18 12:55:55 +01:00
Matt Robinson
7c9ff4fc32
Fix segfault too (#883) 2018-12-17 14:27:19 -05:00
Matt Robinson
a5af25c9c2
Limit Jenkins to only one build at a time (#885)
* Fix docker images, again

* Limit jenkins to only one build of a branch at a time
2018-12-17 11:54:32 -05:00
littleskunk
4cd4ecb7c4
unify logging messages (#879) 2018-12-17 16:40:22 +01:00
Kaloyan Raev
c0e348d4fa
Default config for storagenode binary to join captplanet (#854) 2018-12-17 17:38:38 +02:00
Michal Niewrzal
f0662675bc
Add '--dir' param for all CLI parts (replace --base-path) (#833)
* Add '--dir' param for all CLI parts (replace --base-path)

* FindDirParam method moved

* fix compilation error

* make param global

* remove unused fields

* rename param

* remove config flag

* goimports
2018-12-14 22:14:59 +01:00
Bryan White
d8db7c3049
RevocationDB fixes (#866) 2018-12-14 21:45:53 +01:00
Bryan White
52629d3b85
fix defaultConfDir initialization (#870) 2018-12-14 21:45:15 +01:00
aligeti
5e1b02ca8b
Statdb master db v3 848 (#830)
* intial changes to migrate statdb to masterdb framework

* statdb refactor compiles

* added TestCreateDoesNotExist testcase

* Initial port of statdb to masterdb framework working

* refactored statdb proto def to pkg/statdb

* removed statdb/proto folder

* moved pb.Node to storj.NodeID

* CreateEntryIfNotExistsRequest moved pd.Node to storj.NodeID

* moved the fields from pb.Node to statdb.UpdateRequest

ported TestUpdateExists, TestUpdateUptimeExists, TestUpdateAuditSuccessExists TestUpdateBatchExists
2018-12-14 15:17:30 -05:00
Dylan Lott
9bdee7b106
Discovery service (#805)
* WIP possible discovery service impl

* Adds discovery service to CaptPlanet

* Updates the config and server for discovery service

* updates testplanet to use discovery package

* update satellite imports

* Removes unnecessary cache test

* linter fixes

* adds discovery startup to captplanet

* invoke refresh

* updates to discovery refresh cycle

* Make implementation more consistent with previous implementation

* add wait before trying to upload

* sleep a bit more

* remove kademlia bootstrap

* updates

* remove comments
2018-12-14 12:23:54 -07:00
Bill Thorp
f9845e7e92
changes to masterdb to support accountingDB (#846)
* added rollup to captplanet, moved accountingDB to masterdb
2018-12-14 09:27:21 -05:00
Egon Elbre
7363e6cfd8
Better transport tests (#849) 2018-12-14 12:44:48 +02:00
Bill Thorp
020b283b33
fixed revocation db setup in capt (#858) 2018-12-13 20:49:04 -05:00
Dylan Lott
f71a4557b3
change port inspector connects to (#857) 2018-12-13 16:31:52 -07:00
Bryan White
2016ce9fd6
Certificate revocation (#836)
* wip certificate revocation

* refactor tests

* wip testing

* testing

* review fixes

* integration fix attempt #1

* review fixes

* integration fix attempt #2

* linter fixes

* add copywrite

* integration fix attemp #3

* more testing

* more tests

* go mod tidy

* review fixes

* linter fixes
2018-12-13 21:01:43 +01:00
Natalie Villasana
6f634d9f02
pdbinspect time format fix (#855) 2018-12-13 12:02:59 -05:00
Egon Elbre
c56307f5b9
datarepair: remove unneeded debug message (#839)
* datarepair: remove unneeded debug message
* Make overlay-cache and routing table logs more specific
2018-12-12 17:39:16 +02:00
Bill Thorp
ba5f71810e
added tally to capt run.go (#751)
* added tally to capt run.go, ParseURL -> SplitDBURL
2018-12-12 08:15:34 -05:00
JT Olio
4a4f6ad53e
cfgstruct.Bind: support nested config paths (#785)
this will allow some config cleanups in a future pr

Change-Id: Ie873bcee567a72956d9337dfc13ab6ba46c9d1a0
2018-12-11 11:41:19 -07:00
JT Olio
ceb590fa67
capt: reduce nodes to 10 (#793)
Change-Id: Ief380fe29e3043657705cd7505c266fd774181a4
2018-12-11 11:40:54 -07:00
JT Olio
362f447d9f
cmd/statreceiver: lua-scriptable stat receiver (#636)
* cmd/statreceiver: lua-scriptable stat receiver

Change-Id: I3ce0fe3f1ef4b1f4f27eed90bac0e91cfecf22d7

* some updates

Change-Id: I7c3485adcda1278fce01ae077b4761b3ddb9fb7a

* more comments

Change-Id: I0bb22993cd934c3d40fc1da80d07e49e686b80dd

* linter fixes

Change-Id: Ied014304ecb9aadcf00a6b66ad28f856a428d150

* catch errors

Change-Id: I6e1920f1fd941e66199b30bc427285c19769fc70

* review feedback

Change-Id: I9d4051851eab18970c5f5ddcf4ff265508e541d3

* errorgroup improvements

Change-Id: I4699dda3022f0485fbb50c9dafe692d3921734ff

* too tricky

the previous thing was better for memory with lots of errors at a time
but https://play.golang.org/p/RweTMRjoSCt is too much of a foot gun

Change-Id: I23f0b3d77dd4288fcc20b3756a7110359576bf44
2018-12-11 11:24:31 -07:00
Matt Robinson
32fdce7735
Fixes for latest code changes (#817) 2018-12-11 08:05:33 -05:00
Michal Niewrzal
7e2f66ee24
Small fixes around master db (#822) 2018-12-11 10:30:09 +01:00
Kaloyan Raev
4cc49a8fa3
Better error handling of stream.Upload.Close (#811) 2018-12-10 16:35:22 +02:00
Kaloyan Raev
a4dec2da93
Fix "key not found" error on uploading files with uplink cli (#807) 2018-12-10 12:13:30 +02:00
Maximillian von Briesen
72a7726513
enable audit service on satellite (#803) 2018-12-07 13:52:58 -05:00
Kaloyan Raev
72a20af569
Migrate Minio Gateway to the new Metainfo API (#775) 2018-12-07 20:31:29 +02:00
Dylan Lott
147f86bba4
add dump-nodes command to inspector under kad sub command (#789) 2018-12-07 11:26:33 -07:00
JT Olio
9a06de0407
capt: remove heavy client naming (#790) 2018-12-07 08:36:09 -07:00
JT Olio
7d8c92473a
change default encryption key to be more obviously bad (#787) 2018-12-07 08:35:57 -07:00
JT Olio
06b23f7dae
capt: default to debug logs (#792)
Change-Id: Ib94b10b8480e63ca9a317b70dd9cd4ab84e52747
2018-12-07 07:48:04 -07:00
Bryan White
228aa34ff6
TLS extension processing (#771) 2018-12-07 14:44:25 +01:00
Michal Niewrzal
034a34069a
Merge bwagreement db into satellite master db (#770)
* Merge bwagreement db into satellite master db

* adjust to recent tally changes

* linter problems

* linter problems

* returning db structs in more optimal way

* added pointer for assignment

* error message changed

* better param message
2018-12-07 10:59:31 +01:00