Commit Graph

2083 Commits

Author SHA1 Message Date
Kaloyan Raev
d453cd148e Consider encryption overhead when validating max segment size (#2569) 2019-07-16 15:30:10 +02:00
Michal Niewrzal
260d9c49a8
Metainfo RPC objects methods (#2534) 2019-07-16 12:39:23 +02:00
Jeff Wendling
b51d3a69da enable utccheck in all tests by default (#2565)
* enable utccheck in all tests by default
2019-07-16 09:42:19 +09:00
Yingrong Zhao
c12a4aed3b satellite/console: add redeem reward method (#2484)
* add RedeemRewards method

* remove redeem from reward.db

* add redeemable cap check in redeem

* rename offerCap to redeemableCap

* remove redeem test

* update error message

* fix build

* Trigger Jenkins

* use correct credit setting for redeem

* fix comment

* change create qury to get redeemable_cap from offers table

* change referredBy to a pointer in user credit struct
2019-07-15 19:30:00 -04:00
Cameron
8b2d46a974
add vouchers to QueryRequest (#2559) 2019-07-15 18:02:22 -04:00
Jeff Wendling
3a34a0df7b repair: fix data race in reliability cache (#2561) 2019-07-15 22:58:39 +03:00
Jeff Wendling
b9d8ddaad1
storagenode: remove datetime calls in favor of UTC (#2557)
* storagenode: remove datetime calls in favor of UTC

datetime only has second level granularity whereas string
comparisons don't. Since we're wiping everything anyway, it's
easier to just use UTC everywhere rather than migrate to
datetime calls.

* add utcdb to check that arguments are utc

* storagenodedb: add trivial tests to ensure calls work

This at least tests that all of the timestamps passed in are
in the UTC timezone.

* fix truncated comment and change migrations to be UTC
2019-07-15 13:38:08 -04:00
Jess G
aa99482fa7
Jg/add tests (#2547)
* add a test to confirm only objects in pointerDB

* fix lint

* fix link
2019-07-15 09:30:26 -07:00
paul cannon
0d1dce508e
ensure uplink is sending correct size with PieceHash (#2555)
If we verify that the size matches reality, we can then expect to use
the filesystem to store the piece size as used in the signed PieceHash
from the uplink. Otherwise, the uplink might send a garbage size value,
leaving the storagenode with no good way to verify the uplink signature
on the piece at a later date.

Also fix the code in uplink/piecestore/ so that it sends a valid size,
because it was being rude and sending 0.
2019-07-15 11:26:18 -04:00
Michal Niewrzal
5bec820145
Fix monkit leaking (#2553) 2019-07-13 11:04:54 -04:00
Egon Elbre
002d9748ec
signing: ensure we don't break signatures (#2542) 2019-07-12 16:41:19 -04:00
Jeff Wendling
a2418b22af storagenodedb: optimize index usage and queries (#2545)
- Drops some unused indexes
- Applies a computed index to timestamp columns
- Applies a partial index for expired pieces
- Uses BETWEEN to avoid some datetime calls
- Filters expired piece search by those that aren't NULL
2019-07-12 15:29:09 -04:00
aligeti
3a01106ab4
quick fix (#2548) 2019-07-12 14:12:02 -04:00
aligeti
daa3b32ee2
Add Attribution Columns to appropriate tables for OSPP referral link (#2516)
* adds "partner_id" column to user, project, api_key & bucket_metainfo tables
2019-07-12 13:59:19 -04:00
Maximillian von Briesen
b590e53d64
Order by attempted time in injured segments select (#2533) 2019-07-12 13:35:20 -04:00
Alexander Leitner
3cc45a02fb
Update vouchers.go (#2544) 2019-07-12 12:55:48 -04:00
Jess G
07b23b7cc3
update tally to rm bucket count (#2539)
* update tally to rm bucket count

* rm old comment
2019-07-12 12:19:45 -04:00
Alexander Leitner
64b2769de3
discovery: parallelize refresh (#2535)
* parallelize discovery refresh

* add paginateQualifiedtest, address pr comments

* Remove duplicate uptime update

* Lower concurrency in Testplanet for discovery
2019-07-12 10:35:48 -04:00
Yingrong Zhao
d887ffec62
satellite/satellitedb: add default offer for offers table (#2522)
* add default offer for offers table

* fix migration test

* Trigger Jenkins

* set the default value to be correct type

* skip soon will deleted test

* fix test data

* add orderby for ListAll

* change durations, redeemable cap to be a nullable field

* remove unecessary code
2019-07-12 10:19:38 -04:00
Faris Huskovic
003828c348
web/marketing/pages: partner UI table (#2523)
* add UI table for partner offers

* remove award credit from partner table, add referral link column
2019-07-12 09:52:00 -04:00
Yingrong Zhao
6e57b102c5
satellite/satellitedb: remove num_redeemed in offers table (#2510)
* remove num_redeemed

* remove the field frol Offer struct

* remove the field in the UI templates

* fix merge conflict

* fix migration

* fix merge conflict

* fix typo
2019-07-12 09:19:18 -04:00
Jess G
f11bf46a11
Jg/1967 mv bucket metadata uplink (#2505)
* add bucketstore, add init uplink bucket

* update uplink to use bucket rpc

* fix tests

* wrap metainfo client errors

* add allowedBucket struct, fix tests

* update comment

* add paging

* updates per CR

* add test for pagination

* fix lint

* fix uplink test so its easier tyo understand

* fix gateway pagination bug

* changes per cr

* fix bug w/allowedBuckets, add test to catch
2019-07-12 08:57:02 -04:00
Ivan Fraixedes
f420b29d35
[V3-1927] Repairer uploads to max threshold instead of success… (#2423)
* pkg/datarepair: Add test to check num upload pieces
  Add a new test for ensuring the number of pieces that the repair process
  upload when a segment is injured.
* satellite/orders: Don't create "put order limits" over total
  Repair must not create "put order limits" more than the total count.
* pkg/datarepair: Update upload repair pieces test
  Update the test which checks the number of pieces which are uploaded
  during a repair for using the same excess over the success threshold
  value than the implementation.
* satellites/orders: Limit repair put order for not being total
  Limit the number of put orders to be used by repair for only uploading
  pieces to a % excess over the successful threshold.
* pkg/datarepair: Change DataRepair test to pass again
  Make some changes in the DataRepair test to make pass again after the
  repair upload repaired pieces only until a % excess over success
  threshold.
  Also update the steps description of the DataRepair test after it has been
  changed, to match on what's now, besides to leave it more generic for
  avoiding having to update it on minimal future refactorings.
* satellite: Make repair excess optimal threshold configurable
  Add a new configuration parameter to the satellite for being able to
  configure the percentage excess over the optimal threshold, used for
  determining how many pieces should be repaired/uploaded, rather than
  having the value hard coded.
* repairer: Add configurable param to segments/repairer
  Add a new parameters to the segment/repairer to calculate the maximum
  number of excess nodes, based on the optimal threshold, that repaired
  pieces can be uploaded.
  This new parameter has been added for not returning more nodes than the
  number of upload orders for data repair satellite service calculate for
  repairing pieces.
* pkg/storage/ec: Update log message in clien.Repair
* satellite: Update configuration lock file
2019-07-12 00:44:47 +02:00
Michal Niewrzal
8a9c63809b tests for PiecePublicKey/PiecePrivateKey (#2526) 2019-07-11 17:11:04 -04:00
Egon Elbre
d52f764e54
protocol: implement new piece signing and verification (#2525) 2019-07-11 16:51:40 -04:00
Maximillian von Briesen
8b507f3d73 Address concerns with storagenode Retain endpoint (#2527) 2019-07-11 16:04:21 -04:00
Michal Niewrzal
268c629ba8
Replace base64 encoding for path segments (#2345) 2019-07-11 13:26:07 -04:00
Kaloyan Raev
0f36b160fc Higher dial timeout for TestDownloadSharesDownloadTimeout (#2500) 2019-07-10 17:45:09 -04:00
Maximillian von Briesen
de85d17069
Add checker metrics (#2487)
checker_segment_total_count - Number of total segments in pointer during checker iteration
checker_segment_healthy_count - Number of healthy segments in pointer during checker iterationn
time_since_checker_queue - Seconds elapsed between checker queue and beginning repair
time_for_repair - Seconds elapsed between beginning repair and ending repair/dequeueing
2019-07-10 17:27:46 -04:00
Bill Thorp
0e463dccfd
7 day validity window for order limits (#2520)
* 7 day limit
2019-07-10 17:17:00 -04:00
Bogdan Artemenko
32e0227c45
Project Payment methods (#2037) 2019-07-10 23:29:26 +03:00
Jeff Wendling
02565db73a storagenode: migration to drop unused index and used_serials data (#2508) 2019-07-10 15:16:13 -04:00
ethanadams
f06aec06fb
Move int64s to top of struct to resolve alignment issue on ARM (#2521)
* move int64s to top of struct to resolve alignment issue on ARM
2019-07-10 13:47:22 -04:00
Faris Huskovic
0d294103e9
satellite/rewards: nicer offers handling (#2390)
* organize offers

* revert changes to go.mod and go.sum

* change OfferStatus enums back to original

* revert modified auto-gen files

* don't render empty row if offers is empty

* change return val of ListAll to Offers

* fix build

* add method to check for empty offer when rendering template

* fix typo

* fix lint and typos

* lean out IsEmpty

* dont use named return vals

* better clarify offer statuses

* change back order of setting offer.Status

* lint

* satellite/marketingweb: allow disabling rewards (#2392)

* implement handler for stop offer endpoint

* use proper text and fix data-target for free-credit stop modal
2019-07-10 13:12:40 -04:00
Jeff Wendling
94eeb58b45 mobile: add a way to get an encryption access at some path root (#2519)
* mobile: add a way to get an encryption access at some path root

this exposes a way to have keys deeper than the bucket root

* APIKey + Caveat exported

* use safeError

* return nil
2019-07-10 12:01:56 -04:00
Jeff Wendling
10547cc1ea
segments: send in the object path to the initial CreateSegment… (#2518)
otherwise, api key restictions will fail because we look like we're
asking to put to the bucket metadata path.
2019-07-10 11:33:55 -04:00
Jeff Wendling
7886a4d7b9 storagenodedb: use datetime functions in sqlite queries (#2512)
This way comparison happens on the actual time rather than the
string representation of the time which may change depending on
the time zone.
2019-07-10 10:47:59 -04:00
Cameron
c29f033e7d
Move kademlia dialer into separate package (#2466)
* move kademlia.Dialer into kademliaclient package
2019-07-10 10:36:37 -04:00
Fadila
fa1f5c8d7f garbage collection endpoint on storage node (#2424) 2019-07-10 09:41:47 -04:00
Nikolay Yurchenko
8f024b3db8 scss beautification (#2499) 2019-07-10 12:55:40 +03:00
Michal Niewrzal
13e917f9ed
[Mobile] Add method for EncryptionAccessWithDefaultKey (#2479) 2019-07-10 04:45:48 +02:00
JT Olio
a79c7d77f3 overlay cache: slight modification of node-is-online rules (#2490) 2019-07-09 22:36:09 -04:00
Alexander Leitner
1c5db71faf
Change protobuf expirations to use time.Time (#2509)
* Change protobuf expirations to use time.Time instead of timestamp.Timestamp
2019-07-09 17:54:00 -04:00
Michal Niewrzal
bbc25a2bf7 Drop SN certifiates table from DB (#2498) 2019-07-09 17:33:45 -04:00
Jeff Wendling
d616be8ae0 storagenode: use minimum time in the order for expiration (#2504) 2019-07-09 17:16:30 -04:00
Egon Elbre
910e4744ad
satellite/metainfo: optimize pointerdb size (#2506) 2019-07-09 16:36:18 -04:00
ethanadams
4faed2098d
method name changer per PR 2469 (#2494) 2019-07-09 16:05:14 -04:00
Natalie Villasana
6aefa1ae80 adds metainfo rpc doc (#1957) 2019-07-09 21:44:01 +02:00
Egon Elbre
203f36a41f
rename order.go to order.pb.go (#2496) 2019-07-09 13:24:41 -04:00
Egon Elbre
c5de36ed1f
update to go 1.12.7 (#2493) 2019-07-09 11:56:08 -04:00