Bill Thorp
40e7fae564
not overflow int32 ( #1594 )
2019-03-28 13:08:32 +02:00
Kaloyan Raev
d1639c4157
Merge statdb pkg into overlay pkg ( #1570 )
2019-03-25 18:25:09 -04:00
Bill Thorp
dbd97aaaf9
SNs should NOT be audited for expired segments ( #1559 )
...
delete expired pointers during audit
2019-03-23 06:52:51 -04:00
Egon Elbre
694b6dc1da
make tests run faster ( #1553 )
2019-03-22 15:14:17 +02:00
Natalie Villasana
e390605b81
rearranges code in verifier.go ( #1529 )
2019-03-20 11:54:37 +01:00
Natalie Villasana
a09f568fdb
audit service won't hang on bad storage node conns ( #1503 )
2019-03-19 13:37:26 -04:00
Egon Elbre
05d148aeb5
Storage node and upload/download protocol refactor ( #1422 )
...
refactor storage node server
refactor upload and download protocol
2019-03-18 12:55:06 +02:00
Natalie Villasana
036b2f3b49
allocates 1kb for audit bandwidth agreements instead of 256kb ( #1404 )
2019-03-05 18:58:25 -05:00
Bill Thorp
373b301736
BWA aliases ( #1333 )
...
aliased RBAs and PBAs
2019-02-22 16:17:35 -05:00
Natalie Villasana
c3d3f41d30
removes some SignedMessage use ( #1258 )
...
Removes most instances of pb.SignedMessage (there's more to take out but they shouldn't hurt anyone as is).
There used to be places in psserver where a PieceID was hmac'd with the SatelliteID, which was gotten from a SignedMessage. This PR makes it so some functions access the SatelliteID from the Payer Bandwidth Allocation instead.
This requires passing a SatelliteID into psserver functions where they weren't before, so the following proto messages have been changed:
* PieceId - satellite_id field added
This is so the psserver.Piece function has access to the SatelliteID when it needs to get the namespaced pieceID.
This proto message should probably be renamed to PieceRequest, or a new PieceRequest message should be created so this isn't misnamed.
* PieceDelete - satellite_id field added
This is so the psserver.Delete function has access to the SatelliteID when receiving a request to Delete.
2019-02-19 23:36:08 -06:00
Egon Elbre
9c1e299f3c
Ensure everyone sees everyone else ( #1275 )
2019-02-08 11:25:13 +02:00
Egon Elbre
94ee200e35
Ensure cursor tests run against postgres ( #1228 )
2019-02-05 17:45:09 +02:00
Egon Elbre
5a63c00442
Fix issues with blocking during startup ( #1212 )
2019-02-01 19:28:40 +02:00
Egon Elbre
7a85c5c1c6
Fix audit node ids ( #1207 )
2019-02-01 16:48:57 +02:00
Egon Elbre
d5346982c2
Delete provider package ( #1177 )
2019-01-30 22:47:21 +02:00
Egon Elbre
6132ce86b7
Remove utils.LogClose ( #1169 )
2019-01-29 22:42:27 +02:00
Bill Thorp
60946c2024
make bandwidth agreements sensible: without []byte's ( #1152 )
...
removed []byte's from bandwidth agreement protocol buffers
2019-01-28 14:45:25 -05:00
Jennifer Li Johnson
856b98997c
updates copyright 2018 to 2019 ( #1133 )
2019-01-24 15:15:10 -05:00
Egon Elbre
5de7f8af7f
Satellite Peer ( #1119 )
2019-01-23 21:58:44 +02:00
Egon Elbre
ad49f4b994
Enable linters ( #1105 )
2019-01-22 17:09:02 +02:00
Michal Niewrzal
eb2d858ffc
Use pointerdb.Service instead of pointerdb.Server ( #1099 )
2019-01-19 13:58:53 -05:00
Egon Elbre
78dc02b758
Satellite Peer ( #1034 )
...
* add satellite peer
* Add overlay
* reorganize kademlia
* add RunRefresh
* add refresh to storagenode.Peer
* add discovery
* add agreements and metainfo
* rename
* add datarepair checker
* add repair
* add todo notes for audit
* add testing interface
* add into testplanet
* fixes
* fix compilation errors
* fix compilation errors
* make testplanet run
* remove audit refrences
* ensure that audit tests run
* dev
* checker tests compilable
* fix discovery
* fix compilation
* fix
* fix
* dev
* fix
* disable auth
* fixes
* revert go.mod/sum
* fix linter errors
* fix
* fix copyright
* Add address param for SN dashboard (#1076 )
* Rename storj-sdk to storj-sim (#1078 )
* Storagenode logs and config improvements (#1075 )
* Add more info to SN logs
* remove config-dir from user config
* add output where config was stored
* add message for successful connection
* fix linter
* remove storage.path from user config
* resolve config path
* move success message to info
* log improvements
* Remove captplanet (#1070 )
* pkg/server: include production cert (#1082 )
Change-Id: Ie8e6fe78550be83c3bd797db7a1e58d37c684792
* Generate Payments Report (#1079 )
* memory.Size: autoformat sizes based on value entropy (#1081 )
* Jj/bytes (#1085 )
* run tally and rollup
* sets dev default tally and rollup intervals
* nonessential storj-sim edits (#1086 )
* Closing context doesn't stop storage node (#1084 )
* Print when cancelled
* Close properly
* Don't log nil
* Don't print error when closing dashboard
* Fix panic in inspector if ping fails (#1088 )
* Consolidate identity management to identity cli commands (#1083 )
* Consolidate identity management:
Move identity cretaion/signing out of storagenode setup command.
* fixes
* linters
* Consolidate identity management:
Move identity cretaion/signing out of storagenode setup command.
* fixes
* sava backups before saving signed certs
* add "-prebuilt-test-cmds" test flag
* linters
* prepare cli tests for travis
* linter fixes
* more fixes
* linter gods
* sp/sdk/sim
* remove ca.difficulty
* remove unused difficulty
* return setup to its rightful place
* wip travis
* Revert "wip travis"
This reverts commit 56834849dcf066d3cc0a4f139033fc3f6d7188ca.
* typo in travis.yaml
* remove tests
* remove more
* make it only create one identity at a time for consistency
* add config-dir for consitency
* add identity creation to storj-sim
* add flags
* simplify
* fix nolint and compile
* prevent overwrite and pass difficulty, concurrency, and parent creds
* goimports
2019-01-18 08:54:08 -05:00
Egon Elbre
f8906ce000
Change overlay cache database interface ( #1047 )
2019-01-15 11:08:45 -05:00
Michal Niewrzal
e13f7c7a7d
Fix unstarted pointerdb responsibility ( #1033 )
2019-01-11 13:45:11 +01:00
Maximillian von Briesen
058ef585aa
do not try to dial nil node from audit verifier ( #1027 )
...
* do not try to dial nil node from audit verifier
* add todo
2019-01-10 15:13:40 -05:00
Maximillian von Briesen
0642592420
Replace pdb client with server in audit service ( #1016 )
...
* add logger to audit service
* use pointerdb instead of pdbclient in audit
* linter fixes
2019-01-10 11:35:18 -05:00
Bill Thorp
fb4a11ebb0
Differentiate GET and PUT operations in the tally service ( #965 )
...
* draft of new bandwidth agreement types
* updated storagenode report for new types
* use correct pba types
2019-01-10 06:41:57 -05:00
Michal Niewrzal
bacc1b13b4
Verifier should use payer bandwidth alloc from satellite ( #960 )
...
* Verifier should use payer bandwidth alloc from satellite
* unit test added
* fix typo
* review comments applied
* fix renamed field
2019-01-06 19:51:01 +01:00
Jennifer Li Johnson
a2fa5c4c5a
Proper NodeType Handling ( #873 )
...
* adds enums to nodetype
* updating nodetype todos
* ran pb updates
* reorder nodetypes
* adding checks
* wip
* wip
* wip
* bug in test-captplanet
* wip
* add values to storagenode, satellite, captplanet binaries
* Cleanup
* more cleanup
* wip
* lint
* lint
* wip
* fixes bug
* regenerate protos
Change-Id: Id270212e8c7479e52641058042cf23b5317ab773
* limit node type changes to kademlia
Change-Id: I9c1a6cc4a79e05086627f0fdeb5028c62ce754f4
* dpanic
Change-Id: Id952a2ad13c807ebaea0ec0a875405e267d81c3e
* review comments
Change-Id: I7f9b77ef22779dd012fd490375b136014f51f834
2019-01-02 11:47:34 -07:00
Egon Elbre
89db0fe9f5
storj-sdk base code ( #799 )
2019-01-02 20:07:49 +02:00
Bryan White
249244536a
CSR Service (part 2): cert signing rpc ( #950 )
...
* CSR Service:
+ implement certificate sign rpc method
+ implement certificate signer client/server
+ refactor `AuthorizationDB#Create`
+ refactor `NewTestIdentity`
+ add `AuthorizationDB#Claim`
+ add `Token#Equal`
+ fix `Authorizations#Marshal` when marshaling identities and certificates
+ tweak `Authorization#String` format
+ cert debugging improvements (jsondiff)
+ receive context arg in `NewTestIdentity`
+ misc. fixes
2019-01-02 12:39:17 -05:00
Egon Elbre
d9b9ae6ffa
Cleanup overlay methods and names. ( #914 )
2018-12-20 15:57:54 +02:00
Maximillian von Briesen
f18c38628e
Refactor statdb ( #887 )
...
Update statdb args/return values to minimize structs
Simplify statdb.Update() to update all stats instead of an arbitrary subset determined by flags
Remove CreateIfNotExists logic from statdb.Update()
Simplify audit code structure
2018-12-19 13:44:03 -05:00
aligeti
5e1b02ca8b
Statdb master db v3 848 ( #830 )
...
* intial changes to migrate statdb to masterdb framework
* statdb refactor compiles
* added TestCreateDoesNotExist testcase
* Initial port of statdb to masterdb framework working
* refactored statdb proto def to pkg/statdb
* removed statdb/proto folder
* moved pb.Node to storj.NodeID
* CreateEntryIfNotExistsRequest moved pd.Node to storj.NodeID
* moved the fields from pb.Node to statdb.UpdateRequest
ported TestUpdateExists, TestUpdateUptimeExists, TestUpdateAuditSuccessExists TestUpdateBatchExists
2018-12-14 15:17:30 -05:00
Maximillian von Briesen
2ab15196d9
Get rid of statdb grpc stuff ( #761 )
...
* remove sdbclient
* do not register statdb grpc server/remove apikey from statdb
* start audit service after statdb in captplanet
2018-12-04 13:47:58 -05:00
Egon Elbre
1e4556f88a
Fix import groupings ( #739 )
2018-11-30 15:40:13 +02:00
Bryan White
2a0c4e60d2
preparing for use of customtype gogo extension with NodeID type ( #693 )
...
* preparing for use of `customtype` gogo extension with `NodeID` type
* review changes
* preparing for use of `customtype` gogo extension with `NodeID` type
* review changes
* wip
* tests passing
* wip fixing tests
* more wip test fixing
* remove NodeIDList from proto files
* linter fixes
* linter fixes
* linter/review fixes
* more freaking linter fixes
* omg just kill me - linterrrrrrrr
* travis linter, i will muder you and your family in your sleep
* goimports everything - burn in hell travis
* goimports update
* go mod tidy
2018-11-29 19:39:27 +01:00
Egon Elbre
0ae05cf834
Ensure that we only choose storage nodes ( #732 )
2018-11-29 16:57:00 +02:00
Kaloyan Raev
f46d43b0d0
Fix panic in Audit service ( #713 )
2018-11-28 09:33:17 +02:00
Maximillian von Briesen
7cf16503b8
Add statdb to overlay cache (node selection) ( #711 )
...
* make overlay node rep consistent with statdb node rep
* add statdb to testplanet
* add statdb to overlay cache
* tests
2018-11-27 12:46:12 -05:00
Michal Niewrzal
8dd669b37f
Bandwidth allocations need method designation (GET/PUT) ( #708 )
...
* Bandwidth allocations need method designation (GET/PUT)
* add method comment
* goimports
2018-11-26 19:21:44 +01:00
Maximillian von Briesen
5014a785a0
Node selection 0/4: Update statdb auth/functionality ( #698 )
...
* remove api key from statdb server reqs; add statdb UpdateUptime and UpdateAuditSuccess to server
* update api key authentication in statdb server
* add todos for future statdb updates
* add UpdateUptime and UpdateAuditSuccess to statdb server
* fix apikey stuff in config.go and statdb_test.go
* fix tests
* update sdbclient.NewClient call in audit package
* fix UpdateUptime and UpdateAuditSuccess in sdbclient
* set api key from statdb/config.go
* change package for statdb tests
* linter fixes
* remove todo comments
* fix sdbclient err checking
* move validate auth functionality to auth package
* update description for statdb api key
* remove import
2018-11-26 12:08:29 -05:00
Bryan White
54ccb460a9
rename size fields in protobuf messages for gogo compatibility ( #690 )
...
* rename `size` fields in protobuf messages for gogo compatibility
* linter fixes
2018-11-20 18:09:35 +01:00
Egon Elbre
6fe16e48ba
Simplify and fix tests ( #660 )
2018-11-16 18:31:14 +02:00
Natalie Villasana
f11f4653e4
removes audit erroring when no stripe is given ( #603 )
2018-11-08 22:36:44 +00:00
Alexander Leitner
320c93368a
Add creation time to Payer Bandwidth Allocation ( #581 )
...
* Add creation time to Payer Bandwidth Allocation
* goimports
2018-11-08 11:04:52 -05:00
Maximillian von Briesen
66ee332446
Fix audit stuff breaking test-captplanet ( #594 )
...
* fix audit stripe selector to work if last segment is smaller than stripe size
* fix audit bug related to indexing an incomplete list of nodes returned by overlay
2018-11-07 16:23:05 -05:00
Egon Elbre
d5b3d5f395
Don't panic in case of a 0-size pointer ( #590 )
2018-11-07 18:51:36 +02:00
Natalie Villasana
3b572264ca
adds audit job ( #521 )
2018-11-07 01:16:43 +00:00
Bryan White
ee62e2a9d8
Use transport client and cleanup all the clients ( #574 )
...
* wip
* linter fixes
* linter fixes
* test fixes
* linter fixes
* fix merge + restructure piecestore packages
* review feedback
* linter fixes
* linter fixes
* remove unnecessary aliases to piecestore
* more merge fixing
2018-11-06 18:49:17 +01:00
