Jennifer Li Johnson
a2fa5c4c5a
Proper NodeType Handling ( #873 )
...
* adds enums to nodetype
* updating nodetype todos
* ran pb updates
* reorder nodetypes
* adding checks
* wip
* wip
* wip
* bug in test-captplanet
* wip
* add values to storagenode, satellite, captplanet binaries
* Cleanup
* more cleanup
* wip
* lint
* lint
* wip
* fixes bug
* regenerate protos
Change-Id: Id270212e8c7479e52641058042cf23b5317ab773
* limit node type changes to kademlia
Change-Id: I9c1a6cc4a79e05086627f0fdeb5028c62ce754f4
* dpanic
Change-Id: Id952a2ad13c807ebaea0ec0a875405e267d81c3e
* review comments
Change-Id: I7f9b77ef22779dd012fd490375b136014f51f834
2019-01-02 11:47:34 -07:00
Egon Elbre
89db0fe9f5
storj-sdk base code ( #799 )
2019-01-02 20:07:49 +02:00
Egon Elbre
0ca03b41e2
Use generated locking database implementation ( #947 )
2019-01-02 19:53:27 +02:00
Bryan White
249244536a
CSR Service (part 2): cert signing rpc ( #950 )
...
* CSR Service:
+ implement certificate sign rpc method
+ implement certificate signer client/server
+ refactor `AuthorizationDB#Create`
+ refactor `NewTestIdentity`
+ add `AuthorizationDB#Claim`
+ add `Token#Equal`
+ fix `Authorizations#Marshal` when marshaling identities and certificates
+ tweak `Authorization#String` format
+ cert debugging improvements (jsondiff)
+ receive context arg in `NewTestIdentity`
+ misc. fixes
2019-01-02 12:39:17 -05:00
Michal Niewrzal
b712fbcbb0
Fix 'empty queue' error when satellite starts ( #939 )
2019-01-02 17:00:32 +01:00
Bill Thorp
e4b3dab446
Add config editing via the CLI for storage nodes ( #919 )
...
* added editor and updated storagenode cmd
* switched to custom yaml gen
2019-01-02 10:12:07 -05:00
Matt Robinson
cbc5fdf99a
Fix CA Timeout and show progress to the user ( #915 )
...
* Fix idle after CA timeout and show progress to the user so they don't think we've hung
* fix timeout again since it was lost in a rebase
2019-01-02 09:41:01 -05:00
JT Olio
3fbd8c7d15
kademlia: remove a redundant argument ( #951 )
2019-01-02 12:57:06 +02:00
JT Olio
d4d0b0d13b
inspector: use existing identity ( #952 )
2019-01-02 12:45:47 +02:00
JT Olio
4a0749e44a
Remove farmer references ( #949 )
2019-01-02 12:31:49 +02:00
JT Olio
2c916a04c3
pkg/provider: split into pkg/server, pkg/identity ( #953 )
2019-01-02 12:23:25 +02:00
Egon Elbre
cc8cce58dd
statreceiver nitpicking ( #926 )
2019-01-01 11:41:27 +02:00
Bryan White
4efb5c0a75
Authorization improvements ( #925 )
2018-12-31 10:45:43 -05:00
Yehor Butko
e82edc68c8
V3-976 Create db query for filtering team mebers ( #940 )
...
* V3-976 Create db query for filtering team mebers
* fixing linter
* fixing linter
* sql injection fixed
* getOrder renamed, tests added
2018-12-28 14:07:35 +02:00
Yaroslav Vorobiov
d65cefcac7
Satellite api keys api ( #936 )
2018-12-27 17:30:15 +02:00
Egon Elbre
4346cd060f
Implement mutex around satellitedb ( #932 )
2018-12-27 11:56:25 +02:00
Michal Niewrzal
a9572b7169
Fix ignored config logger settings ( #938 )
2018-12-27 10:38:11 +01:00
Nikolay Yurchenko
312fd5a5fd
[V3-963] Satellite team members add dialog ( #935 )
...
* Implemented team member add dialog without api.
* logic changed according to api
* added initial app reducer and team members deletion fixed
2018-12-26 17:05:33 +02:00
Yaroslav Vorobiov
3b9f5fa633
satellite api keys repo and service method added ( #931 )
2018-12-26 16:00:53 +02:00
Yehor Butko
5ef427265e
web/satellite notifications added, users and projects api updated ( #934 )
...
* web/satellite notifications added, users and projects api updated
* fix users api
2018-12-24 14:52:52 +02:00
Dylan Lott
7569b7d71b
Node lifecycle hooks ( #920 )
...
* wires up first draft of lifecycle methods
* creates interface on transport
* node lifecycle hooks works
* linter fixes
* adds error log at connection success
* chnages Observer interface to use context
* Makes Discovery take its own logger
* WIP
* linter fixes
* Test fixes
* adds in ConnFailure code for cache
2018-12-21 21:51:42 -07:00
Dylan Lott
2f6835a3be
Adds a logger to RoutingTable ( #922 )
...
* adds delete functions and tests for cache
* adds logger to routing table struct and passes it through to logger
* add loggers
2018-12-21 17:48:53 -07:00
Dylan Lott
05c1644fec
adds delete functions and tests for cache ( #921 )
2018-12-21 11:35:21 -07:00
Yaroslav Vorobiov
16db288c85
Satellite add/delete project members by email api ( #928 )
...
* Satellite add/delete project members by email api
2018-12-21 17:41:53 +02:00
Cameron
f70b826fd4
repair queue masterDB support ( #865 )
...
* add injuredsegment model to satellitedb.dbx
* add context to queue.RepairQueue interface
* use queue.RepairQueue interface, use masterdb
2018-12-21 10:11:19 -05:00
Bill Thorp
3fa094f3da
Add the ability for Kademlia to do a refresh for each bucket ( #908 )
...
* added random id in range, added refresh
2018-12-20 16:45:06 -05:00
Dennis Coyle
18c4972205
add monkit to Satellite DB Service ( #903 )
...
* add monkit to Satelite DB Service
* goimports
2018-12-20 15:10:27 -05:00
Bryan White
398379b149
CSR service (part 1): authorizations ( #906 )
2018-12-20 19:29:05 +01:00
Yehor Butko
ac860c1984
Removing company and project ownership ( #917 )
2018-12-20 18:18:08 +02:00
Yaroslav Vorobiov
1c119c6215
( #916 ) delete and add projectmember api now accepts list of users
...
* delete and add projectmember api now accepts list of users
* review changes
* simplification
* use errs.Combine
2018-12-20 17:36:32 +02:00
Egon Elbre
d9b9ae6ffa
Cleanup overlay methods and names. ( #914 )
2018-12-20 15:57:54 +02:00
Maximillian von Briesen
f18c38628e
Refactor statdb ( #887 )
...
Update statdb args/return values to minimize structs
Simplify statdb.Update() to update all stats instead of an arbitrary subset determined by flags
Remove CreateIfNotExists logic from statdb.Update()
Simplify audit code structure
2018-12-19 13:44:03 -05:00
Yaroslav Vorobiov
76018231c6
satellite project members pagination api ( #907 )
...
* satellite project members pagination api
* rename GetByProjectIDPaged to GetByProjectID
* add comment to max limit
2018-12-19 15:03:12 +02:00
Dennis Coyle
1b7ab9600d
Do not error if Discovery can not find Node ( #902 )
2018-12-18 15:01:15 -05:00
aligeti
3c7f62be73
KBucket clean up ( #898 )
2018-12-18 13:04:46 -05:00
Yaroslav Vorobiov
efdfff9704
updated project and user deletion ( #884 )
...
* update project and user deletion
* linter fixes
2018-12-18 19:43:02 +02:00
Jennifer Li Johnson
01df85704c
Updates accounting tables and allows tally to update at raw table for at rest data ( #874 )
2018-12-18 12:18:42 -05:00
Egon Elbre
d9a13667fc
Use fixed logger for Kademlia and make not being able to connect in discovery debug statement. ( #899 )
2018-12-18 17:13:32 +02:00
Bryan White
4eb55017c8
Cert revocation CLI ( #848 )
...
* wip
* allow identity and CA configs to save cert/key separately
* fixes
* linter and default path fixes
* review fixes
* fixes:
+ review fixes
+ bug fixes
+ add extensions command
* linter fixes
* fix ca revoke description
* review fixes
2018-12-18 12:55:55 +01:00
Natalie Villasana
17c60c1f06
moves node selection config setup from uplink to satellite ( #891 )
2018-12-17 16:05:05 -05:00
Michal Niewrzal
a819d819e3
Overlay Cache master db merge ( #877 )
...
* Overlay Cache master db merge
* test update
* review comments
* Fix transaction usage
* removed unused variable
* added better limit handling
* better error handling
2018-12-17 21:14:16 +01:00
Egon Elbre
6b5945139f
Overlay error handling ( #880 )
2018-12-17 20:47:26 +02:00
Egon Elbre
5c26aa3e4a
Use storj.NodeID directly in statdb tests ( #881 )
2018-12-17 20:47:04 +02:00
Natalie Villasana
d97e27ed80
removes a few todos ( #867 )
2018-12-17 13:11:29 -05:00
Michal Niewrzal
1d2523cf60
Payments API protobuf definition ( #876 )
2018-12-17 18:36:50 +01:00
Dylan Lott
706f20611d
adds discovery function to overlay for random lookups ( #860 )
...
* adds discovery function to overlay for random lookups
* uses different variable name
* linter fixes
2018-12-17 09:31:14 -07:00
Cameron
0d17c21a1b
Add logger to piecestore server/node started log ( #856 )
...
* add logger to psserver struct
* node started log
* rename initialize to NewEndpoint
* return err from WriteBandwidthAllocToDB
2018-12-17 10:23:02 -05:00
Egon Elbre
d8df4b5f6b
Simplify provider tests ( #862 )
2018-12-17 17:09:52 +02:00
Yehor Butko
90f79e585c
V3-975 Create db pagination query for team members ( #871 )
...
* V3-975 Create db pagination query for team members
* fix go mod tidy
* Methods renamed. Picked go.sum from storj/master.
* fixing linter
2018-12-17 16:28:58 +02:00
Michal Niewrzal
f0662675bc
Add '--dir' param for all CLI parts (replace --base-path) ( #833 )
...
* Add '--dir' param for all CLI parts (replace --base-path)
* FindDirParam method moved
* fix compilation error
* make param global
* remove unused fields
* rename param
* remove config flag
* goimports
2018-12-14 22:14:59 +01:00
Bryan White
d8db7c3049
RevocationDB fixes ( #866 )
2018-12-14 21:45:53 +01:00
aligeti
5e1b02ca8b
Statdb master db v3 848 ( #830 )
...
* intial changes to migrate statdb to masterdb framework
* statdb refactor compiles
* added TestCreateDoesNotExist testcase
* Initial port of statdb to masterdb framework working
* refactored statdb proto def to pkg/statdb
* removed statdb/proto folder
* moved pb.Node to storj.NodeID
* CreateEntryIfNotExistsRequest moved pd.Node to storj.NodeID
* moved the fields from pb.Node to statdb.UpdateRequest
ported TestUpdateExists, TestUpdateUptimeExists, TestUpdateAuditSuccessExists TestUpdateBatchExists
2018-12-14 15:17:30 -05:00
Dylan Lott
9bdee7b106
Discovery service ( #805 )
...
* WIP possible discovery service impl
* Adds discovery service to CaptPlanet
* Updates the config and server for discovery service
* updates testplanet to use discovery package
* update satellite imports
* Removes unnecessary cache test
* linter fixes
* adds discovery startup to captplanet
* invoke refresh
* updates to discovery refresh cycle
* Make implementation more consistent with previous implementation
* add wait before trying to upload
* sleep a bit more
* remove kademlia bootstrap
* updates
* remove comments
2018-12-14 12:23:54 -07:00
Yaroslav Vorobiov
60fb655db2
satellite user delete updated to require user password ( #852 )
2018-12-14 18:14:17 +02:00
Michal Niewrzal
725ed44ce0
Add gen.go for protobuf ( #863 )
2018-12-14 16:55:55 +01:00
Michal Niewrzal
db5a3e0d0a
Parameters cleanup ( #864 )
...
* Parameters cleanup
* goimports
2018-12-14 16:55:45 +01:00
Bill Thorp
f9845e7e92
changes to masterdb to support accountingDB ( #846 )
...
* added rollup to captplanet, moved accountingDB to masterdb
2018-12-14 09:27:21 -05:00
Egon Elbre
7363e6cfd8
Better transport tests ( #849 )
2018-12-14 12:44:48 +02:00
Bill Thorp
c17a6ba584
fixed temp dir issues w/ rev db ( #859 )
2018-12-13 21:24:35 -05:00
Bryan White
2016ce9fd6
Certificate revocation ( #836 )
...
* wip certificate revocation
* refactor tests
* wip testing
* testing
* review fixes
* integration fix attempt #1
* review fixes
* integration fix attempt #2
* linter fixes
* add copywrite
* integration fix attemp #3
* more testing
* more tests
* go mod tidy
* review fixes
* linter fixes
2018-12-13 21:01:43 +01:00
Egon Elbre
08394389d3
Fix ConfNested on Windows ( #851 )
2018-12-13 07:34:38 -07:00
Egon Elbre
969fcabf72
Remove utils.LogClose from identity saving ( #837 )
2018-12-13 15:50:05 +02:00
Kaloyan Raev
2a8fef8062
Data repairer should use the redundancy strategy from segment's pointer ( #838 )
2018-12-13 09:12:36 +02:00
Jennifer Li Johnson
6642f97142
Jj/accounting updates ( #820 )
...
* tallies up data stored on each node in pointerdb
* adds comments for data type enums
* changes Open to BeginTx because Go convention
* removes online status check from identify active nodes
* changes identifyactivenodes to calculatestaticdata
* updates accounting dbx names
2018-12-12 16:24:08 -05:00
Maximillian von Briesen
a6c7306350
Cut off piecestore Puts if they exceed alloced bandwidth/space ( #819 )
...
* add bandwidth/storage limits to StreamWriter
* add StreamWriter tests for bandwidth/storage limits
2018-12-12 14:14:51 -05:00
Egon Elbre
7a80e7bf2f
Use testplanet in node tests ( #841 )
2018-12-12 17:40:33 +02:00
Egon Elbre
c56307f5b9
datarepair: remove unneeded debug message ( #839 )
...
* datarepair: remove unneeded debug message
* Make overlay-cache and routing table logs more specific
2018-12-12 17:39:16 +02:00
Egon Elbre
38f72df8b5
Faster tests ( #834 )
2018-12-12 16:05:47 +02:00
Bill Thorp
ba5f71810e
added tally to capt run.go ( #751 )
...
* added tally to capt run.go, ParseURL -> SplitDBURL
2018-12-12 08:15:34 -05:00
JT Olio
4a4f6ad53e
cfgstruct.Bind: support nested config paths ( #785 )
...
this will allow some config cleanups in a future pr
Change-Id: Ie873bcee567a72956d9337dfc13ab6ba46c9d1a0
2018-12-11 11:41:19 -07:00
JT Olio
ceb590fa67
capt: reduce nodes to 10 ( #793 )
...
Change-Id: Ief380fe29e3043657705cd7505c266fd774181a4
2018-12-11 11:40:54 -07:00
JT Olio
362f447d9f
cmd/statreceiver: lua-scriptable stat receiver ( #636 )
...
* cmd/statreceiver: lua-scriptable stat receiver
Change-Id: I3ce0fe3f1ef4b1f4f27eed90bac0e91cfecf22d7
* some updates
Change-Id: I7c3485adcda1278fce01ae077b4761b3ddb9fb7a
* more comments
Change-Id: I0bb22993cd934c3d40fc1da80d07e49e686b80dd
* linter fixes
Change-Id: Ied014304ecb9aadcf00a6b66ad28f856a428d150
* catch errors
Change-Id: I6e1920f1fd941e66199b30bc427285c19769fc70
* review feedback
Change-Id: I9d4051851eab18970c5f5ddcf4ff265508e541d3
* errorgroup improvements
Change-Id: I4699dda3022f0485fbb50c9dafe692d3921734ff
* too tricky
the previous thing was better for memory with lots of errors at a time
but https://play.golang.org/p/RweTMRjoSCt is too much of a foot gun
Change-Id: I23f0b3d77dd4288fcc20b3756a7110359576bf44
2018-12-11 11:24:31 -07:00
Natalie Villasana
9e1ec97b31
adds node selection config ( #782 )
2018-12-11 12:30:14 -05:00
Kaloyan Raev
9449010c15
Enable overlay by default ( #835 )
2018-12-11 18:52:45 +02:00
Kaloyan Raev
252da15f0d
Randomize node selection during GETs ( #827 )
2018-12-11 18:05:14 +02:00
Yaroslav Vorobiov
175e25f93a
Satellite graphql: take authorized user id if no id was provided in iput ( #828 )
...
satellite graphql: take authorized user id if no id was provided by client
2018-12-11 17:54:45 +02:00
Egon Elbre
492639aef7
Skip TestUploadDownload ( #826 )
2018-12-11 16:52:03 +02:00
Egon Elbre
8e14d2cb94
Reduce ranger test output ( #825 )
2018-12-11 11:38:56 +02:00
Michal Niewrzal
7e2f66ee24
Small fixes around master db ( #822 )
2018-12-11 10:30:09 +01:00
Egon Elbre
26dbf788ad
Optimize compilation by removing logging tests ( #816 )
2018-12-11 10:46:52 +02:00
aligeti
d27bcddaab
Merge the irreparable DB into the master database ( #777 )
...
* initial refactoring to merge irreparabledb to master db
* refactor
* code review comment updates
2018-12-10 14:08:45 -05:00
Yehor Butko
a8a082f4c4
V3-808 Implement graphql queries for team members page ( #815 )
2018-12-10 19:32:15 +02:00
Yaroslav Vorobiov
63998b4a62
updated user update api, separated company and user creation ( #767 )
2018-12-10 17:57:06 +02:00
Kaloyan Raev
3175eb74a6
Improve piece retrieving log with offset and length ( #813 )
2018-12-10 17:22:49 +02:00
Michal Niewrzal
a15099d5f5
Simplify testing with all supported DBs ( #809 )
...
* Simplify testing with all supported DBs
* move method to separate package
* method renamed to Run
* fixed comment
2018-12-10 15:50:12 +01:00
Kaloyan Raev
3cd0b4adb2
Verify bucket exists before starting multipart upload ( #812 )
2018-12-10 16:41:43 +02:00
Kaloyan Raev
4cc49a8fa3
Better error handling of stream.Upload.Close ( #811 )
2018-12-10 16:35:22 +02:00
Yehor Butko
b1605a7331
Updating users repository ( #808 )
2018-12-10 15:47:48 +02:00
Yehor Butko
678eb81d54
V3-898 Create ability for querying db in transaction scope ( #781 )
...
* V3-898 Create ability for querying db in transaction scope
* handling errors
* fixing linter
* Transaction updated
* fixing empty lines linter
* tx signatures changed
* fixing comments
2018-12-10 14:29:01 +02:00
Yaroslav Vorobiov
8ff6cea5a9
[WIP] get project members api ( #801 )
2018-12-10 13:38:42 +02:00
Egon Elbre
c7cf97cc23
Fix sqlite registration ( #806 )
2018-12-10 10:32:49 +02:00
Cameron
f46fcd0303
upload/download tests with test-planet ( #718 )
...
* add upload/download test
* use minio gateway for upload download
* move test into miniogw package
* logging edits, remove logging_test.go
2018-12-07 14:26:39 -05:00
Kaloyan Raev
c025c13706
Fix deadlock on testplanet shutdown ( #798 )
2018-12-07 20:31:59 +02:00
Kaloyan Raev
72a20af569
Migrate Minio Gateway to the new Metainfo API ( #775 )
2018-12-07 20:31:29 +02:00
Egon Elbre
554daf17e6
Remove mock kademlia ( #802 )
2018-12-07 19:36:20 +02:00
JT Olio
890c87914f
pstore: fix blob permissions ( #788 )
...
Change-Id: I004747657221c6ac53894fc5c5917e3f92436990
2018-12-07 07:47:45 -07:00
JT Olio
1c96db01ba
better database error handling ( #784 )
...
* better database error handling
Change-Id: I28dbd69cf6c2fa268e02405521ff6e6c1a68a702
* missing comments added
* missing comment added
2018-12-07 07:46:42 -07:00
Brandon Iglesias
023eb95a9b
cleaning up todos in Kademlia pkg ( #763 )
...
* updating comments in KAD code
* deleting todo comment because they are not needed anymore
2018-12-07 08:57:42 -05:00
Bryan White
228aa34ff6
TLS extension processing ( #771 )
2018-12-07 14:44:25 +01:00
Michal Niewrzal
4608976526
Bwagreement tests cleanup ( #795 )
2018-12-07 13:11:35 +01:00
Michal Niewrzal
d62896bbaf
Use testcontext with tally tests ( #796 )
2018-12-07 12:55:25 +01:00
Michal Niewrzal
034a34069a
Merge bwagreement db into satellite master db ( #770 )
...
* Merge bwagreement db into satellite master db
* adjust to recent tally changes
* linter problems
* linter problems
* returning db structs in more optimal way
* added pointer for assignment
* error message changed
* better param message
2018-12-07 10:59:31 +01:00
Jennifer Li Johnson
dac79fd762
removes check for mock overlay in data repair checker config ( #794 )
2018-12-07 00:42:16 -05:00
JT Olio
b6c936340b
clean up uplink config section ( #786 )
2018-12-06 17:18:11 -07:00
Alexander Leitner
04ff6b8e11
Use audit results in checker ( #754 )
...
* Use statdb to get audit results
* Find invalid nodes
2018-12-06 13:51:23 -05:00
Kaloyan Raev
b043fd76dd
Fix race in connection pool ( #778 )
2018-12-06 18:05:57 +02:00
Yehor Butko
0c95fba4c8
Satellite Projects db entity and api updated ( #772 )
...
* Satellite Projects db entity and api updated
* get users projects query updated with dbx
2018-12-06 17:19:47 +02:00
Yaroslav Vorobiov
d6f6ab56a1
satellite projectmembers add and delete api ( #776 )
2018-12-06 16:40:32 +02:00
Egon Elbre
fc8e9c6572
Move irreparabledb under datarepair ( #764 )
2018-12-06 14:32:21 +02:00
Maximillian von Briesen
828a8b6907
Change statdb.FindValidNodes to FindInvalidNodes ( #774 )
...
* Remove statdb.FindValidNodes
* Add statdb.FindInvalidNodes for getting nodes that fall below a reputation threshold
2018-12-05 15:23:32 -05:00
Dennis Coyle
fffb093056
SelfAddr -> ExternalAddress ( #768 )
2018-12-05 11:36:03 -05:00
Egon Elbre
c4033b15af
Simplify peer discovery implementation ( #765 )
2018-12-05 16:32:37 +02:00
Bill Thorp
b2a8a10b16
redo tally query ( #742 )
...
* redo tally query
* broken, sharing w/ Jen
* tally tests added, failiing
* fix txn issue in tally
* fix lint
2018-12-05 09:03:23 -05:00
Dennis Coyle
e138056943
listen to configured address ( #755 )
2018-12-05 05:37:03 -05:00
Dylan Lott
69281552b3
Pretty prints node output from kad lookup tool ( #762 )
...
* Pretty prints node output from kad lookup tool
* Linter fix
2018-12-04 17:23:39 -07:00
Dylan Lott
c6d790d58e
Improvements to Kad Lookups ( #731 )
...
* adds channel for getting node out of lookup
* WIP adding the channels to lookups
* WIP adding channel to node lookups
* Wires up FindNodes method with channels
* WIP adds a test suite for lookup - tests are still failing
* WIP wires up use of testplanet for kademlia lookup tests
* WIP merging in node id changes
* Merges in pkg/storj node type changes
* Tests passing
* Lookup node working via Inspector now
* updates
* WIP working on getting tests passing
* WIP getting tests passing
* FindNode works
* Linter fix
* Adds copyrights to lookup_test
* removes a fmt.Printf I missed
* Removes commented out lines
2018-12-04 14:39:28 -07:00
Matt Robinson
4237afb1c1
Attempt at fixing the all-in-one environment ( #736 )
2018-12-04 16:10:23 -05:00
Maximillian von Briesen
6c655d117b
Filter nodes by reputation and IP address (node selection) ( #720 )
...
* Pulls statdb stats into overlay cache whenever cache.Put() is called
* Updates overlay.FindStorageNodes()/overlayClient.Choose() to filter based on node stats
* Updates overlay.FindStorageNodes()/overlayClient.Choose() to exclude duplicate IP addresses
2018-12-04 15:18:26 -05:00
Maximillian von Briesen
2ab15196d9
Get rid of statdb grpc stuff ( #761 )
...
* remove sdbclient
* do not register statdb grpc server/remove apikey from statdb
* start audit service after statdb in captplanet
2018-12-04 13:47:58 -05:00
Kaloyan Raev
b7c45c7c16
Migrate Uplink CLI to Metainfo API ( #745 )
2018-12-04 18:46:02 +02:00
aligeti
9f6e5d6731
Keep track of 'irreparable' segments in a database ( #686 )
...
* initial irreparable development
* added the Open, Insert, Update, Get, Delete, Close support
* added the Unit test cases
* code review changes
2018-12-04 11:26:30 -05:00
Egon Elbre
ae790dfd9f
Fix queue implementation ( #758 )
2018-12-04 17:46:53 +02:00
Egon Elbre
efe475b448
Ensure that restrictions are consistent. ( #759 )
2018-12-04 16:02:03 +02:00
Bryan White
24f6da2dd5
Kad inspector fixes ( #746 )
...
* pretty print kad inspector
* fix format string
* cleanup
* cleanup
* usage, args, print improvements
* kad inspector tweaks
* pretty print kad inspector
* cleanup
* cleanup
* kad inspector tweaks
* linter fixes
2018-12-03 21:01:54 +01:00
Maximillian von Briesen
c6dd70541e
move sdb apikey to config struct ( #753 )
2018-12-03 13:40:03 -05:00
Kaloyan Raev
0018ebf63e
Object.Bucket as storj.Bucket instead of string ( #747 )
2018-12-03 16:38:03 +02:00
Bryan White
120d875e06
consolidate NodeRep and NodeStats ( #744 )
...
* consolidate `NodeRep` and `NodeStats`
* regenerate
* review fixes
* go travis
2018-12-03 14:23:12 +01:00
Bryan White
fe8c1823b7
Cleanup debug code
2018-12-03 12:45:27 +02:00
Bryan White
71ef93feed
pretty print kad inspector ( #743 )
...
* pretty print kad inspector
* fix format string
* cleanup
* cleanup
* usage, args, print improvements
2018-11-30 18:36:05 +01:00
Egon Elbre
3e461df470
Linting protobuf files ( #676 )
2018-11-30 17:02:01 +02:00
Kaloyan Raev
66476da1ae
metainfo: basic upload ( #726 )
2018-11-30 15:50:52 +02:00
Egon Elbre
1e4556f88a
Fix import groupings ( #739 )
2018-11-30 15:40:13 +02:00
Bryan White
66b3a20595
cleanup and bugfix ( #737 )
2018-11-30 14:04:59 +01:00
Egon Elbre
9ac3517432
Reduce noise in logging ( #733 )
2018-11-29 22:59:26 +02:00
Bryan White
2a0c4e60d2
preparing for use of customtype gogo extension with NodeID type ( #693 )
...
* preparing for use of `customtype` gogo extension with `NodeID` type
* review changes
* preparing for use of `customtype` gogo extension with `NodeID` type
* review changes
* wip
* tests passing
* wip fixing tests
* more wip test fixing
* remove NodeIDList from proto files
* linter fixes
* linter fixes
* linter/review fixes
* more freaking linter fixes
* omg just kill me - linterrrrrrrr
* travis linter, i will muder you and your family in your sleep
* goimports everything - burn in hell travis
* goimports update
* go mod tidy
2018-11-29 19:39:27 +01:00
Yaroslav Vorobiov
a21855ac37
satellite user validation ( #729 )
2018-11-29 18:23:44 +02:00
Egon Elbre
0ae05cf834
Ensure that we only choose storage nodes ( #732 )
2018-11-29 16:57:00 +02:00
Yehor Butko
7b628e8af6
V3-763 Create logic for project details ( #728 )
...
* V3-763 Create logic for project details
* updating markup
* icon fixed
2018-11-28 18:20:23 +02:00
Yaroslav Vorobiov
c27431d64f
company update api ( #725 )
2018-11-28 14:30:38 +02:00
Yaroslav Vorobiov
24159216bc
update user api ( #715 )
2018-11-28 12:31:15 +02:00
Kaloyan Raev
f46d43b0d0
Fix panic in Audit service ( #713 )
2018-11-28 09:33:17 +02:00
Dylan Lott
a245aa7827
Inspector - Kad PING ( #722 )
...
* initial commit of inspector gadget wireup
* change name of comman dline tool, setup grpc server
* Get inspector cli working with grpc client
* Wired up CountNodes command
* WIP getting buckets response working
* Added GetBucket command
* WIP working on get buckets command
* WIP working on bucket list
* Still WIP
* WIP getting bucket counts to work
* Some clean up of unnecessary changes
* List Buckets and Get Bucket are working
* Removing logs, getting ready for review
* initial commit of inspector gadget wireup
* change name of comman dline tool, setup grpc server
* Get inspector cli working with grpc client
* Wired up CountNodes command
* WIP getting buckets response working
* Added GetBucket command
* WIP working on get buckets command
* WIP working on bucket list
* Still WIP
* WIP getting bucket counts to work
* Some clean up of unnecessary changes
* List Buckets and Get Bucket are working
* Removing logs, getting ready for review
* Fix error return
* Trying to get tests passing
* Adds method on dht mock for tests
* Add dbx files back
* Fix package import error in dbx file
* Adds copyrights to pass linter
* tidy go mod
* Updates from code review
* Updates inspector to take flag arguments for address
* Format list-buckets output more prettier
* Wiring up PING in kad inspector tools
2018-11-27 15:50:52 -07:00
Dylan Lott
fd42795456
updates dbx/gen.go to name generated package correctly ( #723 )
2018-11-27 13:52:19 -07:00
Maximillian von Briesen
7cf16503b8
Add statdb to overlay cache (node selection) ( #711 )
...
* make overlay node rep consistent with statdb node rep
* add statdb to testplanet
* add statdb to overlay cache
* tests
2018-11-27 12:46:12 -05:00
Michal Niewrzal
376bd74bed
Stops logging repairer errors for empty queue ( #717 )
2018-11-27 16:57:51 +01:00
Yaroslav Vorobiov
a0d84ccd67
Satellite user delete ( #707 )
...
* auth from context
* user delete api
2018-11-27 16:20:58 +02:00
Yehor Butko
784f3a887d
V3-760 Create logic for New project popup ( #714 )
...
* V3-760 Create logic for New project popup
* removing path to static files from gateway
* added missing copyrights
2018-11-27 13:14:10 +00:00
Kaloyan Raev
ea9bf38d85
eestream avoids waiting for pending readers if lots of errors ( #709 )
2018-11-27 10:17:57 +02:00
Bill Thorp
b8b0d75cc6
made accounting use migrate ( #688 )
...
* made accounting use migrate
2018-11-26 16:49:55 -05:00
Michal Niewrzal
8dd669b37f
Bandwidth allocations need method designation (GET/PUT) ( #708 )
...
* Bandwidth allocations need method designation (GET/PUT)
* add method comment
* goimports
2018-11-26 19:21:44 +01:00
Maximillian von Briesen
5014a785a0
Node selection 0/4: Update statdb auth/functionality ( #698 )
...
* remove api key from statdb server reqs; add statdb UpdateUptime and UpdateAuditSuccess to server
* update api key authentication in statdb server
* add todos for future statdb updates
* add UpdateUptime and UpdateAuditSuccess to statdb server
* fix apikey stuff in config.go and statdb_test.go
* fix tests
* update sdbclient.NewClient call in audit package
* fix UpdateUptime and UpdateAuditSuccess in sdbclient
* set api key from statdb/config.go
* change package for statdb tests
* linter fixes
* remove todo comments
* fix sdbclient err checking
* move validate auth functionality to auth package
* update description for statdb api key
* remove import
2018-11-26 12:08:29 -05:00
Yehor Butko
1f1f9f5875
Satellitedb company-user relation changed to 1-to-1 ( #706 )
...
* Satellitedb company-user relation changed to 1-to-1
* fixing comments
2018-11-26 14:57:55 +00:00
Yehor Butko
a998665491
Satellite projects api ( #703 )
...
* V3-645 Create Project API
* fixing linter
* update project refactored
2018-11-26 10:47:23 +00:00
Kaloyan Raev
67db26b36e
Metainfo stream download ( #697 )
2018-11-26 09:39:05 +02:00
Maximillian von Briesen
b884ee4626
Update protos to fit style guide (easy review) ( #702 )
...
* fix overlay.proto and piecestore.proto to meet style guide
* update code to be consistent with regenerated protos
2018-11-23 21:46:53 -05:00
Yaroslav Vorobiov
bdebe36ae0
updated login api ( #699 )
2018-11-22 12:38:58 +02:00
Maximillian von Briesen
06cf1c71ed
Statdb inspection tool ( #626 )
...
* begin adding stadb inspection tool
* add command for creating statdb nodes from csv
* add getCSVStats and create helper func for getting sdb client
* fix createcsvstats, add createstats
* fix linter errors
2018-11-21 17:01:09 -05:00
Dylan Lott
0d05cb26bf
Kademlia Inspector CLI ( #657 )
...
* initial commit of inspector gadget wireup
* change name of comman dline tool, setup grpc server
* Get inspector cli working with grpc client
* Wired up CountNodes command
* WIP getting buckets response working
* Added GetBucket command
* WIP working on get buckets command
* WIP working on bucket list
* Still WIP
* WIP getting bucket counts to work
* Some clean up of unnecessary changes
* List Buckets and Get Bucket are working
* Removing logs, getting ready for review
* initial commit of inspector gadget wireup
* change name of comman dline tool, setup grpc server
* Get inspector cli working with grpc client
* Wired up CountNodes command
* WIP getting buckets response working
* Added GetBucket command
* WIP working on get buckets command
* WIP working on bucket list
* Still WIP
* WIP getting bucket counts to work
* Some clean up of unnecessary changes
* List Buckets and Get Bucket are working
* Removing logs, getting ready for review
* Fix error return
* Trying to get tests passing
* Adds method on dht mock for tests
* Add dbx files back
* Fix package import error in dbx file
* Adds copyrights to pass linter
* tidy go mod
* Updates from code review
* Updates inspector to take flag arguments for address
* Format list-buckets output more prettier
2018-11-21 10:31:27 -07:00
Yaroslav Vorobiov
c829835dc1
updated createuser api ( #687 )
2018-11-21 17:51:43 +02:00
Michal Niewrzal
f72832ee69
Add metadata to pb.Node (email and wallet) ( #671 )
...
* Add metadata to pb.Node (email and wallet)
* goimports
* reorganize config
* fix typo
* remove 'foo' id
* add basic test
2018-11-21 16:07:18 +01:00
Kaloyan Raev
aec2c41911
Metainfo remote segment test ( #696 )
2018-11-21 16:35:53 +02:00
Kaloyan Raev
8c56198090
Use designated error classes for common bucket and object errors ( #691 )
2018-11-21 13:17:28 +02:00
Bryan White
dabd87cb36
swap golang/protobuf out for gogo/protobuf ( #692 )
...
* swap golang/protobuf out for gogo/protobuf
* revert go.mod and go.sum changes
* linter fixes
2018-11-20 19:29:07 +01:00
Bryan White
54ccb460a9
rename size fields in protobuf messages for gogo compatibility ( #690 )
...
* rename `size` fields in protobuf messages for gogo compatibility
* linter fixes
2018-11-20 18:09:35 +01:00
Dennis Coyle
e7e0d1daaa
Coyle/disable mock ( #627 )
...
* disabled mock overlay from captplanet
2018-11-20 11:54:52 -05:00
Jennifer Li Johnson
93c5f385a8
Enable checker in captplanet and staging ( #643 )
...
* enable checker
* add option to use mock overlay in checker
* adds logs to checker
* appease linter
2018-11-20 10:54:22 -05:00
Yehor Butko
4734a7447b
V3-805 Extend satellitedb with Project members entity ( #683 )
...
* V3-805 Extend satellitedb with Project members entity
* fixing linter
* small refactoring
* tests refactoring
2018-11-20 14:50:47 +00:00
Egon Elbre
76af9f5171
Add overlay server to testplanet ( #677 )
2018-11-19 22:39:25 +02:00
Egon Elbre
832317b0ee
Remove unused workers ( #640 )
2018-11-19 17:07:24 +02:00
Egon Elbre
d07433c150
Use testplanet to test Refresh ( #678 )
2018-11-19 16:40:01 +02:00
Kaloyan Raev
bb1cf151e3
Metainfo handle inline segments in ReadOnlyStream ( #674 )
2018-11-19 13:08:28 +02:00
Egon Elbre
6fe16e48ba
Simplify and fix tests ( #660 )
2018-11-16 18:31:14 +02:00
Yaroslav Vorobiov
3e99861e04
operationName and variables fields added to graphql json parsing ( #668 )
2018-11-16 16:34:35 +02:00
Kaloyan Raev
f730ce451a
Telemetry in metainfo implementation ( #670 )
2018-11-16 15:59:27 +02:00
aligeti
7958994ae2
Diagnostic tool to inspect repair queue ( #656 )
...
* initial repair queue diag tool development
* fixes linter warnings
* code review updates
2018-11-16 08:31:33 -05:00
Alexander Leitner
5e9377524f
Refactor basystem ( #641 )
...
* Signature verification
* Clean up agreement sender to have less errors
* overlay address in captnplanet
* Refactor bandwidth.proto to not use streams
* Make sure the send worked
* Handle connection to satellite
* Save renter public key inside of renter bandwidth allocations
* Default diag to sqlite. Make configurable
* Separate bw server and dbm; regenerate dbx files
* Make sure test uses protobufs
* Demonstrate creating bandwidth allocations
2018-11-15 14:06:09 -05:00
Egon Elbre
c4b90f84dd
Use better defaults and naming for postgres database ( #659 )
2018-11-15 20:36:57 +02:00
Kaloyan Raev
0357e61bbd
metainfo objects tests ( #662 )
2018-11-15 17:31:33 +02:00
Yaroslav Vorobiov
92b1bceca7
added copyright to files in satelliteauth pkg ( #663 )
2018-11-15 14:25:04 +02:00
Yaroslav Vorobiov
3f2ce11315
added copyright to every file in satellite pkg ( #661 )
2018-11-15 14:00:08 +02:00
Egon Elbre
21a102a05f
testplanet: add logger ( #649 )
2018-11-15 10:57:47 +02:00
Maximillian von Briesen
2c36dcc2c8
Update statdb ( #650 )
...
* add sdbclient.UpdateUptime; update args for sdbclient.CreateEntryIfNotExists
* add auditcount to node stats; restructure statdb.CreateEntryIfNotExists
* add noop mock sdbclient
* add the ability to create a node in statdb without "default" stats
* update statdb.CreateEntryIfNotExists
* take fewer args for sdbclient.CreateWithStats/FindValidNodes
* add sdbclient.UpdateAuditSuccess
* update sdbclient.Update so that all fields are updated when called (reduce args)
* update error checking in statdb.Create
2018-11-14 19:03:19 -05:00
Bill Thorp
d3c347a0ac
Verify overlay cache is accurately and fully tested ( #540 )
...
* added intro node id, moved port range to 1024, listen in go routines
2018-11-14 16:30:07 -05:00
Yaroslav Vorobiov
f28af4cdfb
added creation date to graphql user and removed redundant check from graphql api handler ( #654 )
2018-11-14 17:13:13 +02:00
Yehor Butko
8990fea63c
Satellitedb refactoring ( #647 )
2018-11-14 12:45:49 +00:00
Yaroslav Vorobiov
c442205b3a
account api: infrastructure, login, register, getUser ( #611 )
2018-11-14 12:50:15 +02:00
Egon Elbre
17519a7532
metainfo: Fix listing paths ( #645 )
2018-11-14 12:40:53 +02:00
Kaloyan Raev
5cdb03515f
Configure path encryption in metainfo API ( #644 )
2018-11-14 11:26:18 +02:00
Jennifer Li Johnson
377832c705
Create Tally and Rollup pkgs in accounting ( #642 )
...
* creates separate tally and rollup packages and writes skeleton for rollup
* TODO add rollupDB and rawDB to rollup struct
* TODO add rawDB to tally struct
2018-11-13 20:22:18 -05:00
Alexander Leitner
50d657af11
Cleanup agreement sender ( #630 )
...
* Signature verification
* code review comments & lint warnings
* Clean up agreement sender to have less errors
* overlay address in captnplanet
2018-11-13 10:37:49 -05:00
Kaloyan Raev
514ca50611
Configurable encryption type for paths ( #637 )
2018-11-13 14:21:52 +02:00
Yehor Butko
75213fa633
V3-663 Extend database with Project entity ( #622 )
...
* V3-663 Extend database with Project entity
* fixing linter
2018-11-13 10:27:42 +02:00
aligeti
6576d31b29
Signature verification ( #604 )
...
* Signature verification
* code review comments & lint warnings
2018-11-12 16:59:30 -05:00
Bill Thorp
ad39201ba9
Issues in overlay rebuild allowed to continue ( #625 )
...
* Issues in overlay rebuild allowed to continue
2018-11-12 14:53:20 -05:00
Cameron
ace82bc834
remove bad nodes from originalNodes list when repairing ( #597 )
...
* remove bad nodes from list
* create slice for healthy nodes
* add contains helper, clean up loop
2018-11-12 11:10:44 -05:00
Kaloyan Raev
9afa9b6b52
Metainfo buckets tests ( #607 )
2018-11-12 15:23:19 +02:00
Yehor Butko
5352778b3c
refactored byte to uuid converting ( #610 )
...
* refactored byte to uuid converting
* linter fixed
* error messages updated
2018-11-12 09:14:16 +00:00
Egon Elbre
6066ae7567
Check schemas while creating tables ( #609 )
2018-11-10 00:15:35 +02:00
Dennis Coyle
e4baec8fce
Coyle/tls verify ( #612 )
...
* add nodeID to dial opts
* fix tests to use real identity
2018-11-09 17:08:33 -05:00
Maximillian von Briesen
741a6bc8c3
update dbx generation script for statdb ( #602 )
2018-11-09 16:01:51 -05:00
