Dylan Lott
05c1644fec
adds delete functions and tests for cache ( #921 )
2018-12-21 11:35:21 -07:00
Yaroslav Vorobiov
16db288c85
Satellite add/delete project members by email api ( #928 )
...
* Satellite add/delete project members by email api
2018-12-21 17:41:53 +02:00
Cameron
f70b826fd4
repair queue masterDB support ( #865 )
...
* add injuredsegment model to satellitedb.dbx
* add context to queue.RepairQueue interface
* use queue.RepairQueue interface, use masterdb
2018-12-21 10:11:19 -05:00
Bill Thorp
3fa094f3da
Add the ability for Kademlia to do a refresh for each bucket ( #908 )
...
* added random id in range, added refresh
2018-12-20 16:45:06 -05:00
Dennis Coyle
18c4972205
add monkit to Satellite DB Service ( #903 )
...
* add monkit to Satelite DB Service
* goimports
2018-12-20 15:10:27 -05:00
Bryan White
398379b149
CSR service (part 1): authorizations ( #906 )
2018-12-20 19:29:05 +01:00
Yehor Butko
ac860c1984
Removing company and project ownership ( #917 )
2018-12-20 18:18:08 +02:00
Yaroslav Vorobiov
1c119c6215
( #916 ) delete and add projectmember api now accepts list of users
...
* delete and add projectmember api now accepts list of users
* review changes
* simplification
* use errs.Combine
2018-12-20 17:36:32 +02:00
Egon Elbre
d9b9ae6ffa
Cleanup overlay methods and names. ( #914 )
2018-12-20 15:57:54 +02:00
Maximillian von Briesen
f18c38628e
Refactor statdb ( #887 )
...
Update statdb args/return values to minimize structs
Simplify statdb.Update() to update all stats instead of an arbitrary subset determined by flags
Remove CreateIfNotExists logic from statdb.Update()
Simplify audit code structure
2018-12-19 13:44:03 -05:00
Yaroslav Vorobiov
76018231c6
satellite project members pagination api ( #907 )
...
* satellite project members pagination api
* rename GetByProjectIDPaged to GetByProjectID
* add comment to max limit
2018-12-19 15:03:12 +02:00
Dennis Coyle
1b7ab9600d
Do not error if Discovery can not find Node ( #902 )
2018-12-18 15:01:15 -05:00
aligeti
3c7f62be73
KBucket clean up ( #898 )
2018-12-18 13:04:46 -05:00
Yaroslav Vorobiov
efdfff9704
updated project and user deletion ( #884 )
...
* update project and user deletion
* linter fixes
2018-12-18 19:43:02 +02:00
Jennifer Li Johnson
01df85704c
Updates accounting tables and allows tally to update at raw table for at rest data ( #874 )
2018-12-18 12:18:42 -05:00
Egon Elbre
d9a13667fc
Use fixed logger for Kademlia and make not being able to connect in discovery debug statement. ( #899 )
2018-12-18 17:13:32 +02:00
Bryan White
4eb55017c8
Cert revocation CLI ( #848 )
...
* wip
* allow identity and CA configs to save cert/key separately
* fixes
* linter and default path fixes
* review fixes
* fixes:
+ review fixes
+ bug fixes
+ add extensions command
* linter fixes
* fix ca revoke description
* review fixes
2018-12-18 12:55:55 +01:00
Natalie Villasana
17c60c1f06
moves node selection config setup from uplink to satellite ( #891 )
2018-12-17 16:05:05 -05:00
Michal Niewrzal
a819d819e3
Overlay Cache master db merge ( #877 )
...
* Overlay Cache master db merge
* test update
* review comments
* Fix transaction usage
* removed unused variable
* added better limit handling
* better error handling
2018-12-17 21:14:16 +01:00
Egon Elbre
6b5945139f
Overlay error handling ( #880 )
2018-12-17 20:47:26 +02:00
Egon Elbre
5c26aa3e4a
Use storj.NodeID directly in statdb tests ( #881 )
2018-12-17 20:47:04 +02:00
Natalie Villasana
d97e27ed80
removes a few todos ( #867 )
2018-12-17 13:11:29 -05:00
Michal Niewrzal
1d2523cf60
Payments API protobuf definition ( #876 )
2018-12-17 18:36:50 +01:00
Dylan Lott
706f20611d
adds discovery function to overlay for random lookups ( #860 )
...
* adds discovery function to overlay for random lookups
* uses different variable name
* linter fixes
2018-12-17 09:31:14 -07:00
Cameron
0d17c21a1b
Add logger to piecestore server/node started log ( #856 )
...
* add logger to psserver struct
* node started log
* rename initialize to NewEndpoint
* return err from WriteBandwidthAllocToDB
2018-12-17 10:23:02 -05:00
Egon Elbre
d8df4b5f6b
Simplify provider tests ( #862 )
2018-12-17 17:09:52 +02:00
Yehor Butko
90f79e585c
V3-975 Create db pagination query for team members ( #871 )
...
* V3-975 Create db pagination query for team members
* fix go mod tidy
* Methods renamed. Picked go.sum from storj/master.
* fixing linter
2018-12-17 16:28:58 +02:00
Michal Niewrzal
f0662675bc
Add '--dir' param for all CLI parts (replace --base-path) ( #833 )
...
* Add '--dir' param for all CLI parts (replace --base-path)
* FindDirParam method moved
* fix compilation error
* make param global
* remove unused fields
* rename param
* remove config flag
* goimports
2018-12-14 22:14:59 +01:00
Bryan White
d8db7c3049
RevocationDB fixes ( #866 )
2018-12-14 21:45:53 +01:00
aligeti
5e1b02ca8b
Statdb master db v3 848 ( #830 )
...
* intial changes to migrate statdb to masterdb framework
* statdb refactor compiles
* added TestCreateDoesNotExist testcase
* Initial port of statdb to masterdb framework working
* refactored statdb proto def to pkg/statdb
* removed statdb/proto folder
* moved pb.Node to storj.NodeID
* CreateEntryIfNotExistsRequest moved pd.Node to storj.NodeID
* moved the fields from pb.Node to statdb.UpdateRequest
ported TestUpdateExists, TestUpdateUptimeExists, TestUpdateAuditSuccessExists TestUpdateBatchExists
2018-12-14 15:17:30 -05:00
Dylan Lott
9bdee7b106
Discovery service ( #805 )
...
* WIP possible discovery service impl
* Adds discovery service to CaptPlanet
* Updates the config and server for discovery service
* updates testplanet to use discovery package
* update satellite imports
* Removes unnecessary cache test
* linter fixes
* adds discovery startup to captplanet
* invoke refresh
* updates to discovery refresh cycle
* Make implementation more consistent with previous implementation
* add wait before trying to upload
* sleep a bit more
* remove kademlia bootstrap
* updates
* remove comments
2018-12-14 12:23:54 -07:00
Yaroslav Vorobiov
60fb655db2
satellite user delete updated to require user password ( #852 )
2018-12-14 18:14:17 +02:00
Michal Niewrzal
725ed44ce0
Add gen.go for protobuf ( #863 )
2018-12-14 16:55:55 +01:00
Michal Niewrzal
db5a3e0d0a
Parameters cleanup ( #864 )
...
* Parameters cleanup
* goimports
2018-12-14 16:55:45 +01:00
Bill Thorp
f9845e7e92
changes to masterdb to support accountingDB ( #846 )
...
* added rollup to captplanet, moved accountingDB to masterdb
2018-12-14 09:27:21 -05:00
Egon Elbre
7363e6cfd8
Better transport tests ( #849 )
2018-12-14 12:44:48 +02:00
Bill Thorp
c17a6ba584
fixed temp dir issues w/ rev db ( #859 )
2018-12-13 21:24:35 -05:00
Bryan White
2016ce9fd6
Certificate revocation ( #836 )
...
* wip certificate revocation
* refactor tests
* wip testing
* testing
* review fixes
* integration fix attempt #1
* review fixes
* integration fix attempt #2
* linter fixes
* add copywrite
* integration fix attemp #3
* more testing
* more tests
* go mod tidy
* review fixes
* linter fixes
2018-12-13 21:01:43 +01:00
Egon Elbre
08394389d3
Fix ConfNested on Windows ( #851 )
2018-12-13 07:34:38 -07:00
Egon Elbre
969fcabf72
Remove utils.LogClose from identity saving ( #837 )
2018-12-13 15:50:05 +02:00
Kaloyan Raev
2a8fef8062
Data repairer should use the redundancy strategy from segment's pointer ( #838 )
2018-12-13 09:12:36 +02:00
Jennifer Li Johnson
6642f97142
Jj/accounting updates ( #820 )
...
* tallies up data stored on each node in pointerdb
* adds comments for data type enums
* changes Open to BeginTx because Go convention
* removes online status check from identify active nodes
* changes identifyactivenodes to calculatestaticdata
* updates accounting dbx names
2018-12-12 16:24:08 -05:00
Maximillian von Briesen
a6c7306350
Cut off piecestore Puts if they exceed alloced bandwidth/space ( #819 )
...
* add bandwidth/storage limits to StreamWriter
* add StreamWriter tests for bandwidth/storage limits
2018-12-12 14:14:51 -05:00
Egon Elbre
7a80e7bf2f
Use testplanet in node tests ( #841 )
2018-12-12 17:40:33 +02:00
Egon Elbre
c56307f5b9
datarepair: remove unneeded debug message ( #839 )
...
* datarepair: remove unneeded debug message
* Make overlay-cache and routing table logs more specific
2018-12-12 17:39:16 +02:00
Egon Elbre
38f72df8b5
Faster tests ( #834 )
2018-12-12 16:05:47 +02:00
Bill Thorp
ba5f71810e
added tally to capt run.go ( #751 )
...
* added tally to capt run.go, ParseURL -> SplitDBURL
2018-12-12 08:15:34 -05:00
JT Olio
4a4f6ad53e
cfgstruct.Bind: support nested config paths ( #785 )
...
this will allow some config cleanups in a future pr
Change-Id: Ie873bcee567a72956d9337dfc13ab6ba46c9d1a0
2018-12-11 11:41:19 -07:00
JT Olio
ceb590fa67
capt: reduce nodes to 10 ( #793 )
...
Change-Id: Ief380fe29e3043657705cd7505c266fd774181a4
2018-12-11 11:40:54 -07:00
JT Olio
362f447d9f
cmd/statreceiver: lua-scriptable stat receiver ( #636 )
...
* cmd/statreceiver: lua-scriptable stat receiver
Change-Id: I3ce0fe3f1ef4b1f4f27eed90bac0e91cfecf22d7
* some updates
Change-Id: I7c3485adcda1278fce01ae077b4761b3ddb9fb7a
* more comments
Change-Id: I0bb22993cd934c3d40fc1da80d07e49e686b80dd
* linter fixes
Change-Id: Ied014304ecb9aadcf00a6b66ad28f856a428d150
* catch errors
Change-Id: I6e1920f1fd941e66199b30bc427285c19769fc70
* review feedback
Change-Id: I9d4051851eab18970c5f5ddcf4ff265508e541d3
* errorgroup improvements
Change-Id: I4699dda3022f0485fbb50c9dafe692d3921734ff
* too tricky
the previous thing was better for memory with lots of errors at a time
but https://play.golang.org/p/RweTMRjoSCt is too much of a foot gun
Change-Id: I23f0b3d77dd4288fcc20b3756a7110359576bf44
2018-12-11 11:24:31 -07:00
Natalie Villasana
9e1ec97b31
adds node selection config ( #782 )
2018-12-11 12:30:14 -05:00
Kaloyan Raev
9449010c15
Enable overlay by default ( #835 )
2018-12-11 18:52:45 +02:00
Kaloyan Raev
252da15f0d
Randomize node selection during GETs ( #827 )
2018-12-11 18:05:14 +02:00
Yaroslav Vorobiov
175e25f93a
Satellite graphql: take authorized user id if no id was provided in iput ( #828 )
...
satellite graphql: take authorized user id if no id was provided by client
2018-12-11 17:54:45 +02:00
Egon Elbre
492639aef7
Skip TestUploadDownload ( #826 )
2018-12-11 16:52:03 +02:00
Egon Elbre
8e14d2cb94
Reduce ranger test output ( #825 )
2018-12-11 11:38:56 +02:00
Michal Niewrzal
7e2f66ee24
Small fixes around master db ( #822 )
2018-12-11 10:30:09 +01:00
Egon Elbre
26dbf788ad
Optimize compilation by removing logging tests ( #816 )
2018-12-11 10:46:52 +02:00
aligeti
d27bcddaab
Merge the irreparable DB into the master database ( #777 )
...
* initial refactoring to merge irreparabledb to master db
* refactor
* code review comment updates
2018-12-10 14:08:45 -05:00
Yehor Butko
a8a082f4c4
V3-808 Implement graphql queries for team members page ( #815 )
2018-12-10 19:32:15 +02:00
Yaroslav Vorobiov
63998b4a62
updated user update api, separated company and user creation ( #767 )
2018-12-10 17:57:06 +02:00
Kaloyan Raev
3175eb74a6
Improve piece retrieving log with offset and length ( #813 )
2018-12-10 17:22:49 +02:00
Michal Niewrzal
a15099d5f5
Simplify testing with all supported DBs ( #809 )
...
* Simplify testing with all supported DBs
* move method to separate package
* method renamed to Run
* fixed comment
2018-12-10 15:50:12 +01:00
Kaloyan Raev
3cd0b4adb2
Verify bucket exists before starting multipart upload ( #812 )
2018-12-10 16:41:43 +02:00
Kaloyan Raev
4cc49a8fa3
Better error handling of stream.Upload.Close ( #811 )
2018-12-10 16:35:22 +02:00
Yehor Butko
b1605a7331
Updating users repository ( #808 )
2018-12-10 15:47:48 +02:00
Yehor Butko
678eb81d54
V3-898 Create ability for querying db in transaction scope ( #781 )
...
* V3-898 Create ability for querying db in transaction scope
* handling errors
* fixing linter
* Transaction updated
* fixing empty lines linter
* tx signatures changed
* fixing comments
2018-12-10 14:29:01 +02:00
Yaroslav Vorobiov
8ff6cea5a9
[WIP] get project members api ( #801 )
2018-12-10 13:38:42 +02:00
Egon Elbre
c7cf97cc23
Fix sqlite registration ( #806 )
2018-12-10 10:32:49 +02:00
Cameron
f46fcd0303
upload/download tests with test-planet ( #718 )
...
* add upload/download test
* use minio gateway for upload download
* move test into miniogw package
* logging edits, remove logging_test.go
2018-12-07 14:26:39 -05:00
Kaloyan Raev
c025c13706
Fix deadlock on testplanet shutdown ( #798 )
2018-12-07 20:31:59 +02:00
Kaloyan Raev
72a20af569
Migrate Minio Gateway to the new Metainfo API ( #775 )
2018-12-07 20:31:29 +02:00
Egon Elbre
554daf17e6
Remove mock kademlia ( #802 )
2018-12-07 19:36:20 +02:00
JT Olio
890c87914f
pstore: fix blob permissions ( #788 )
...
Change-Id: I004747657221c6ac53894fc5c5917e3f92436990
2018-12-07 07:47:45 -07:00
JT Olio
1c96db01ba
better database error handling ( #784 )
...
* better database error handling
Change-Id: I28dbd69cf6c2fa268e02405521ff6e6c1a68a702
* missing comments added
* missing comment added
2018-12-07 07:46:42 -07:00
Brandon Iglesias
023eb95a9b
cleaning up todos in Kademlia pkg ( #763 )
...
* updating comments in KAD code
* deleting todo comment because they are not needed anymore
2018-12-07 08:57:42 -05:00
Bryan White
228aa34ff6
TLS extension processing ( #771 )
2018-12-07 14:44:25 +01:00
Michal Niewrzal
4608976526
Bwagreement tests cleanup ( #795 )
2018-12-07 13:11:35 +01:00
Michal Niewrzal
d62896bbaf
Use testcontext with tally tests ( #796 )
2018-12-07 12:55:25 +01:00
Michal Niewrzal
034a34069a
Merge bwagreement db into satellite master db ( #770 )
...
* Merge bwagreement db into satellite master db
* adjust to recent tally changes
* linter problems
* linter problems
* returning db structs in more optimal way
* added pointer for assignment
* error message changed
* better param message
2018-12-07 10:59:31 +01:00
Jennifer Li Johnson
dac79fd762
removes check for mock overlay in data repair checker config ( #794 )
2018-12-07 00:42:16 -05:00
JT Olio
b6c936340b
clean up uplink config section ( #786 )
2018-12-06 17:18:11 -07:00
Alexander Leitner
04ff6b8e11
Use audit results in checker ( #754 )
...
* Use statdb to get audit results
* Find invalid nodes
2018-12-06 13:51:23 -05:00
Kaloyan Raev
b043fd76dd
Fix race in connection pool ( #778 )
2018-12-06 18:05:57 +02:00
Yehor Butko
0c95fba4c8
Satellite Projects db entity and api updated ( #772 )
...
* Satellite Projects db entity and api updated
* get users projects query updated with dbx
2018-12-06 17:19:47 +02:00
Yaroslav Vorobiov
d6f6ab56a1
satellite projectmembers add and delete api ( #776 )
2018-12-06 16:40:32 +02:00
Egon Elbre
fc8e9c6572
Move irreparabledb under datarepair ( #764 )
2018-12-06 14:32:21 +02:00
Maximillian von Briesen
828a8b6907
Change statdb.FindValidNodes to FindInvalidNodes ( #774 )
...
* Remove statdb.FindValidNodes
* Add statdb.FindInvalidNodes for getting nodes that fall below a reputation threshold
2018-12-05 15:23:32 -05:00
Dennis Coyle
fffb093056
SelfAddr -> ExternalAddress ( #768 )
2018-12-05 11:36:03 -05:00
Egon Elbre
c4033b15af
Simplify peer discovery implementation ( #765 )
2018-12-05 16:32:37 +02:00
Bill Thorp
b2a8a10b16
redo tally query ( #742 )
...
* redo tally query
* broken, sharing w/ Jen
* tally tests added, failiing
* fix txn issue in tally
* fix lint
2018-12-05 09:03:23 -05:00
Dennis Coyle
e138056943
listen to configured address ( #755 )
2018-12-05 05:37:03 -05:00
Dylan Lott
69281552b3
Pretty prints node output from kad lookup tool ( #762 )
...
* Pretty prints node output from kad lookup tool
* Linter fix
2018-12-04 17:23:39 -07:00
Dylan Lott
c6d790d58e
Improvements to Kad Lookups ( #731 )
...
* adds channel for getting node out of lookup
* WIP adding the channels to lookups
* WIP adding channel to node lookups
* Wires up FindNodes method with channels
* WIP adds a test suite for lookup - tests are still failing
* WIP wires up use of testplanet for kademlia lookup tests
* WIP merging in node id changes
* Merges in pkg/storj node type changes
* Tests passing
* Lookup node working via Inspector now
* updates
* WIP working on getting tests passing
* WIP getting tests passing
* FindNode works
* Linter fix
* Adds copyrights to lookup_test
* removes a fmt.Printf I missed
* Removes commented out lines
2018-12-04 14:39:28 -07:00
Matt Robinson
4237afb1c1
Attempt at fixing the all-in-one environment ( #736 )
2018-12-04 16:10:23 -05:00
Maximillian von Briesen
6c655d117b
Filter nodes by reputation and IP address (node selection) ( #720 )
...
* Pulls statdb stats into overlay cache whenever cache.Put() is called
* Updates overlay.FindStorageNodes()/overlayClient.Choose() to filter based on node stats
* Updates overlay.FindStorageNodes()/overlayClient.Choose() to exclude duplicate IP addresses
2018-12-04 15:18:26 -05:00
Maximillian von Briesen
2ab15196d9
Get rid of statdb grpc stuff ( #761 )
...
* remove sdbclient
* do not register statdb grpc server/remove apikey from statdb
* start audit service after statdb in captplanet
2018-12-04 13:47:58 -05:00
Kaloyan Raev
b7c45c7c16
Migrate Uplink CLI to Metainfo API ( #745 )
2018-12-04 18:46:02 +02:00
aligeti
9f6e5d6731
Keep track of 'irreparable' segments in a database ( #686 )
...
* initial irreparable development
* added the Open, Insert, Update, Get, Delete, Close support
* added the Unit test cases
* code review changes
2018-12-04 11:26:30 -05:00
Egon Elbre
ae790dfd9f
Fix queue implementation ( #758 )
2018-12-04 17:46:53 +02:00
