Commit Graph

1161 Commits

Author SHA1 Message Date
Jennifer Li Johnson
53d96be44a
Stylistic Go Cleanup (#2524) 2019-07-22 15:10:04 -04:00
Michal Niewrzal
6f2b85603d
Metainfo RPC segment methods (part 1) (#2567) 2019-07-22 16:45:18 +02:00
Egon Elbre
63c1a050fc
pkg/dht: remove (#2599) 2019-07-19 20:34:00 +03:00
Egon Elbre
5fe95fe8fa kademlia: restrict maximum find near query (#2598) 2019-07-19 18:46:08 +02:00
aligeti
29b576961f
value attribution merge fix and more test cases (#2588)
* value attribution merge fix and more test cases
2019-07-19 11:17:34 -04:00
Bill Thorp
a7cc940776
Nodes should not be able to fail the same audit multiple times (#2404)
update pointer on audit failure
2019-07-18 14:08:15 -04:00
Jess G
3af9250659
update irreparableDB.GetLimited query to use where instead of offset (#2585)
* update query to use where instead of offset, update tests

* update cmd/inspector irreparable

* add comment for offset
2019-07-18 09:21:21 -07:00
Simon Guindon
91f0adef10
Add the ability to set dial and request timeouts from the cmd/uplink CLI to libuplink. (#2439)
* Added the ability to pass timeout settings from cmd/uplink to libuplink.

* Removed commented out code.

* Updated 2min timeouts for the uplink CLI.

* Removed comment.

* Made transport defaultDialTimeout and defaultRequestTimeout public

* Added comments to describe where these defaults apply.

* Added a new defaults to libuplink and added tests.

* Added a new defaults to libuplink and added tests.
2019-07-18 11:13:59 -04:00
Andrew Harding
416fa80e85
Link Sharing Service (#2431)
Link sharing service. See `docs/design/link-sharing-service.md` for the design and `cmd/linksharing/README.md` for operational instructions.
2019-07-18 06:26:09 -06:00
Kaloyan Raev
0931f0e71b Log fatal errors with respective severity (#2581) 2019-07-17 14:39:10 -04:00
Egon Elbre
f6f65a80d7
storagenode/trust: implement fetching peer identity without kademlia and endpoint (#2584) 2019-07-17 21:14:44 +03:00
Michal Niewrzal
260d9c49a8
Metainfo RPC objects methods (#2534) 2019-07-16 12:39:23 +02:00
Cameron
8b2d46a974
add vouchers to QueryRequest (#2559) 2019-07-15 18:02:22 -04:00
Jeff Wendling
3a34a0df7b repair: fix data race in reliability cache (#2561) 2019-07-15 22:58:39 +03:00
Egon Elbre
002d9748ec
signing: ensure we don't break signatures (#2542) 2019-07-12 16:41:19 -04:00
aligeti
daa3b32ee2
Add Attribution Columns to appropriate tables for OSPP referral link (#2516)
* adds "partner_id" column to user, project, api_key & bucket_metainfo tables
2019-07-12 13:59:19 -04:00
Jess G
07b23b7cc3
update tally to rm bucket count (#2539)
* update tally to rm bucket count

* rm old comment
2019-07-12 12:19:45 -04:00
Alexander Leitner
64b2769de3
discovery: parallelize refresh (#2535)
* parallelize discovery refresh

* add paginateQualifiedtest, address pr comments

* Remove duplicate uptime update

* Lower concurrency in Testplanet for discovery
2019-07-12 10:35:48 -04:00
Jess G
f11bf46a11
Jg/1967 mv bucket metadata uplink (#2505)
* add bucketstore, add init uplink bucket

* update uplink to use bucket rpc

* fix tests

* wrap metainfo client errors

* add allowedBucket struct, fix tests

* update comment

* add paging

* updates per CR

* add test for pagination

* fix lint

* fix uplink test so its easier tyo understand

* fix gateway pagination bug

* changes per cr

* fix bug w/allowedBuckets, add test to catch
2019-07-12 08:57:02 -04:00
Ivan Fraixedes
f420b29d35
[V3-1927] Repairer uploads to max threshold instead of success… (#2423)
* pkg/datarepair: Add test to check num upload pieces
  Add a new test for ensuring the number of pieces that the repair process
  upload when a segment is injured.
* satellite/orders: Don't create "put order limits" over total
  Repair must not create "put order limits" more than the total count.
* pkg/datarepair: Update upload repair pieces test
  Update the test which checks the number of pieces which are uploaded
  during a repair for using the same excess over the success threshold
  value than the implementation.
* satellites/orders: Limit repair put order for not being total
  Limit the number of put orders to be used by repair for only uploading
  pieces to a % excess over the successful threshold.
* pkg/datarepair: Change DataRepair test to pass again
  Make some changes in the DataRepair test to make pass again after the
  repair upload repaired pieces only until a % excess over success
  threshold.
  Also update the steps description of the DataRepair test after it has been
  changed, to match on what's now, besides to leave it more generic for
  avoiding having to update it on minimal future refactorings.
* satellite: Make repair excess optimal threshold configurable
  Add a new configuration parameter to the satellite for being able to
  configure the percentage excess over the optimal threshold, used for
  determining how many pieces should be repaired/uploaded, rather than
  having the value hard coded.
* repairer: Add configurable param to segments/repairer
  Add a new parameters to the segment/repairer to calculate the maximum
  number of excess nodes, based on the optimal threshold, that repaired
  pieces can be uploaded.
  This new parameter has been added for not returning more nodes than the
  number of upload orders for data repair satellite service calculate for
  repairing pieces.
* pkg/storage/ec: Update log message in clien.Repair
* satellite: Update configuration lock file
2019-07-12 00:44:47 +02:00
Michal Niewrzal
8a9c63809b tests for PiecePublicKey/PiecePrivateKey (#2526) 2019-07-11 17:11:04 -04:00
Egon Elbre
d52f764e54
protocol: implement new piece signing and verification (#2525) 2019-07-11 16:51:40 -04:00
Maximillian von Briesen
8b507f3d73 Address concerns with storagenode Retain endpoint (#2527) 2019-07-11 16:04:21 -04:00
Michal Niewrzal
268c629ba8
Replace base64 encoding for path segments (#2345) 2019-07-11 13:26:07 -04:00
Kaloyan Raev
0f36b160fc Higher dial timeout for TestDownloadSharesDownloadTimeout (#2500) 2019-07-10 17:45:09 -04:00
Maximillian von Briesen
de85d17069
Add checker metrics (#2487)
checker_segment_total_count - Number of total segments in pointer during checker iteration
checker_segment_healthy_count - Number of healthy segments in pointer during checker iterationn
time_since_checker_queue - Seconds elapsed between checker queue and beginning repair
time_for_repair - Seconds elapsed between beginning repair and ending repair/dequeueing
2019-07-10 17:27:46 -04:00
Jeff Wendling
10547cc1ea
segments: send in the object path to the initial CreateSegment… (#2518)
otherwise, api key restictions will fail because we look like we're
asking to put to the bucket metadata path.
2019-07-10 11:33:55 -04:00
Cameron
c29f033e7d
Move kademlia dialer into separate package (#2466)
* move kademlia.Dialer into kademliaclient package
2019-07-10 10:36:37 -04:00
JT Olio
a79c7d77f3 overlay cache: slight modification of node-is-online rules (#2490) 2019-07-09 22:36:09 -04:00
Alexander Leitner
1c5db71faf
Change protobuf expirations to use time.Time (#2509)
* Change protobuf expirations to use time.Time instead of timestamp.Timestamp
2019-07-09 17:54:00 -04:00
Jeff Wendling
671648012a Update password based key derivation design doc (#2367) 2019-07-09 00:07:03 -04:00
Ivan Fraixedes
50d601ab07 pkg/identity: Use identity error class (#2488) 2019-07-08 22:52:52 -04:00
JT Olio
887b784f54 discovery: use fetch info to ping (#2491) 2019-07-08 22:10:17 -04:00
Jess G
f9696d6c5e
satellite/metainfo: add buckets RPC and database (#2460)
* add db interface and methods, add sa metainfo endpoints and svc

* add bucket metainfo svc funcs

* add sadb bucekts

* bucket list gets all buckets

* filter buckets list on macaroon restrictions

* update pb cipher suite to be enum

* add conversion funcs

* updates per comments

* bucket settings should say default

* add direction to list buckets, add tests

* fix test bucket names

* lint err

* only support forward direction

* add comments

* minor refactoring

* make sure list up to limit

* update test

* update protolock file

* fix lint

* change per PR
2019-07-08 15:32:18 -07:00
Alexander Leitner
3587e1a579 Change pointerdb pointer to use time.Time for Creation date (#2483) 2019-07-09 00:16:50 +02:00
Egon Elbre
674742d1a7
satellite/datarepair: use reliability cache (#1976) 2019-07-09 01:04:35 +03:00
Ivan Fraixedes
a786e4249c pkg: Align errs Class messages (#2485)
Align 2 errs Class messages with the rest of them.
2019-07-08 17:37:12 -04:00
Alexander Leitner
19ab9852f2
Update node.proto to use time.Time instead of timestamp (#2482) 2019-07-08 14:24:42 -04:00
Alexander Leitner
dcf8e2936b
Update vouchers to use time.Time instead of timestamp (#2478)
* Update vouchers to use time.Time instead of timestamp
2019-07-08 13:07:30 -04:00
Simon Guindon
8d15d774b6
Removing trace of encodedPiece.Read() because it's fast. (#2477)
* Removing trace of encodedPiece.Read() because it's fast.

* Added comment about why tracing was removed.
2019-07-08 11:16:13 -04:00
Stefan Benten
ccef5eee46
Add proper Version Handling to Identity, Gateway and Uplink Binary (#2471) 2019-07-08 10:45:20 -04:00
Yaroslav Vorobiov
524eb24c83 storagenode/nodestats: combine stats into single RPC call (#2455)
* change satellite nodestats endpoint
2019-07-08 17:33:43 +03:00
Alexander Leitner
88732188cb
Update inspector timestamp to time.time (#2464)
* Update inspector timestamp
2019-07-08 10:06:12 -04:00
Michal Niewrzal
094e1b8b90
Add 'Old' suffix to some metainfo methods/messages (#2462) 2019-07-08 15:33:15 +02:00
Yaroslav Vorobiov
7aca0eb284 storagenode/dashboard: show console address (#2456) 2019-07-06 15:40:58 +02:00
Ivan Fraixedes
ea9a8b37bc pkg/datarepair/checker: Fix typo in doc comment (#2461) 2019-07-05 22:20:58 +02:00
Kaloyan Raev
f9ed0dc1a8
Improve stability of TestDownloadSharesDownloadTimeout (#2210) 2019-07-05 19:04:15 +03:00
Ivan Fraixedes
e0a8937c6c
pkg/storage/ec: Make minor improvements in client.Repair (#2454)
* Fix some log message to actually report the number of pieces needed to
  repaired for reaching the successful/optimal threshold.
* Remove some unneeded `nil` check conditional.
2019-07-05 12:39:10 +02:00
Jeff Wendling
8b07df37f5 refactor some store encryption stuff (#2434) 2019-07-05 11:36:35 +03:00
Yaroslav Vorobiov
42c125d697
Satellite nodestats endpoint add audit check (#2426) 2019-07-04 13:41:40 +03:00