Commit Graph

2022 Commits

Author SHA1 Message Date
Jeff Wendling
a2418b22af storagenodedb: optimize index usage and queries (#2545)
- Drops some unused indexes
- Applies a computed index to timestamp columns
- Applies a partial index for expired pieces
- Uses BETWEEN to avoid some datetime calls
- Filters expired piece search by those that aren't NULL
2019-07-12 15:29:09 -04:00
aligeti
3a01106ab4
quick fix (#2548) 2019-07-12 14:12:02 -04:00
aligeti
daa3b32ee2
Add Attribution Columns to appropriate tables for OSPP referral link (#2516)
* adds "partner_id" column to user, project, api_key & bucket_metainfo tables
2019-07-12 13:59:19 -04:00
Maximillian von Briesen
b590e53d64
Order by attempted time in injured segments select (#2533) 2019-07-12 13:35:20 -04:00
Alexander Leitner
3cc45a02fb
Update vouchers.go (#2544) 2019-07-12 12:55:48 -04:00
Jess G
07b23b7cc3
update tally to rm bucket count (#2539)
* update tally to rm bucket count

* rm old comment
2019-07-12 12:19:45 -04:00
Alexander Leitner
64b2769de3
discovery: parallelize refresh (#2535)
* parallelize discovery refresh

* add paginateQualifiedtest, address pr comments

* Remove duplicate uptime update

* Lower concurrency in Testplanet for discovery
2019-07-12 10:35:48 -04:00
Yingrong Zhao
d887ffec62
satellite/satellitedb: add default offer for offers table (#2522)
* add default offer for offers table

* fix migration test

* Trigger Jenkins

* set the default value to be correct type

* skip soon will deleted test

* fix test data

* add orderby for ListAll

* change durations, redeemable cap to be a nullable field

* remove unecessary code
2019-07-12 10:19:38 -04:00
Faris Huskovic
003828c348
web/marketing/pages: partner UI table (#2523)
* add UI table for partner offers

* remove award credit from partner table, add referral link column
2019-07-12 09:52:00 -04:00
Yingrong Zhao
6e57b102c5
satellite/satellitedb: remove num_redeemed in offers table (#2510)
* remove num_redeemed

* remove the field frol Offer struct

* remove the field in the UI templates

* fix merge conflict

* fix migration

* fix merge conflict

* fix typo
2019-07-12 09:19:18 -04:00
Jess G
f11bf46a11
Jg/1967 mv bucket metadata uplink (#2505)
* add bucketstore, add init uplink bucket

* update uplink to use bucket rpc

* fix tests

* wrap metainfo client errors

* add allowedBucket struct, fix tests

* update comment

* add paging

* updates per CR

* add test for pagination

* fix lint

* fix uplink test so its easier tyo understand

* fix gateway pagination bug

* changes per cr

* fix bug w/allowedBuckets, add test to catch
2019-07-12 08:57:02 -04:00
Ivan Fraixedes
f420b29d35
[V3-1927] Repairer uploads to max threshold instead of success… (#2423)
* pkg/datarepair: Add test to check num upload pieces
  Add a new test for ensuring the number of pieces that the repair process
  upload when a segment is injured.
* satellite/orders: Don't create "put order limits" over total
  Repair must not create "put order limits" more than the total count.
* pkg/datarepair: Update upload repair pieces test
  Update the test which checks the number of pieces which are uploaded
  during a repair for using the same excess over the success threshold
  value than the implementation.
* satellites/orders: Limit repair put order for not being total
  Limit the number of put orders to be used by repair for only uploading
  pieces to a % excess over the successful threshold.
* pkg/datarepair: Change DataRepair test to pass again
  Make some changes in the DataRepair test to make pass again after the
  repair upload repaired pieces only until a % excess over success
  threshold.
  Also update the steps description of the DataRepair test after it has been
  changed, to match on what's now, besides to leave it more generic for
  avoiding having to update it on minimal future refactorings.
* satellite: Make repair excess optimal threshold configurable
  Add a new configuration parameter to the satellite for being able to
  configure the percentage excess over the optimal threshold, used for
  determining how many pieces should be repaired/uploaded, rather than
  having the value hard coded.
* repairer: Add configurable param to segments/repairer
  Add a new parameters to the segment/repairer to calculate the maximum
  number of excess nodes, based on the optimal threshold, that repaired
  pieces can be uploaded.
  This new parameter has been added for not returning more nodes than the
  number of upload orders for data repair satellite service calculate for
  repairing pieces.
* pkg/storage/ec: Update log message in clien.Repair
* satellite: Update configuration lock file
2019-07-12 00:44:47 +02:00
Michal Niewrzal
8a9c63809b tests for PiecePublicKey/PiecePrivateKey (#2526) 2019-07-11 17:11:04 -04:00
Egon Elbre
d52f764e54
protocol: implement new piece signing and verification (#2525) 2019-07-11 16:51:40 -04:00
Maximillian von Briesen
8b507f3d73 Address concerns with storagenode Retain endpoint (#2527) 2019-07-11 16:04:21 -04:00
Michal Niewrzal
268c629ba8
Replace base64 encoding for path segments (#2345) 2019-07-11 13:26:07 -04:00
Kaloyan Raev
0f36b160fc Higher dial timeout for TestDownloadSharesDownloadTimeout (#2500) 2019-07-10 17:45:09 -04:00
Maximillian von Briesen
de85d17069
Add checker metrics (#2487)
checker_segment_total_count - Number of total segments in pointer during checker iteration
checker_segment_healthy_count - Number of healthy segments in pointer during checker iterationn
time_since_checker_queue - Seconds elapsed between checker queue and beginning repair
time_for_repair - Seconds elapsed between beginning repair and ending repair/dequeueing
2019-07-10 17:27:46 -04:00
Bill Thorp
0e463dccfd
7 day validity window for order limits (#2520)
* 7 day limit
2019-07-10 17:17:00 -04:00
Bogdan Artemenko
32e0227c45
Project Payment methods (#2037) 2019-07-10 23:29:26 +03:00
Jeff Wendling
02565db73a storagenode: migration to drop unused index and used_serials data (#2508) 2019-07-10 15:16:13 -04:00
ethanadams
f06aec06fb
Move int64s to top of struct to resolve alignment issue on ARM (#2521)
* move int64s to top of struct to resolve alignment issue on ARM
2019-07-10 13:47:22 -04:00
Faris Huskovic
0d294103e9
satellite/rewards: nicer offers handling (#2390)
* organize offers

* revert changes to go.mod and go.sum

* change OfferStatus enums back to original

* revert modified auto-gen files

* don't render empty row if offers is empty

* change return val of ListAll to Offers

* fix build

* add method to check for empty offer when rendering template

* fix typo

* fix lint and typos

* lean out IsEmpty

* dont use named return vals

* better clarify offer statuses

* change back order of setting offer.Status

* lint

* satellite/marketingweb: allow disabling rewards (#2392)

* implement handler for stop offer endpoint

* use proper text and fix data-target for free-credit stop modal
2019-07-10 13:12:40 -04:00
Jeff Wendling
94eeb58b45 mobile: add a way to get an encryption access at some path root (#2519)
* mobile: add a way to get an encryption access at some path root

this exposes a way to have keys deeper than the bucket root

* APIKey + Caveat exported

* use safeError

* return nil
2019-07-10 12:01:56 -04:00
Jeff Wendling
10547cc1ea
segments: send in the object path to the initial CreateSegment… (#2518)
otherwise, api key restictions will fail because we look like we're
asking to put to the bucket metadata path.
2019-07-10 11:33:55 -04:00
Jeff Wendling
7886a4d7b9 storagenodedb: use datetime functions in sqlite queries (#2512)
This way comparison happens on the actual time rather than the
string representation of the time which may change depending on
the time zone.
2019-07-10 10:47:59 -04:00
Cameron
c29f033e7d
Move kademlia dialer into separate package (#2466)
* move kademlia.Dialer into kademliaclient package
2019-07-10 10:36:37 -04:00
Fadila
fa1f5c8d7f garbage collection endpoint on storage node (#2424) 2019-07-10 09:41:47 -04:00
Nikolay Yurchenko
8f024b3db8 scss beautification (#2499) 2019-07-10 12:55:40 +03:00
Michal Niewrzal
13e917f9ed
[Mobile] Add method for EncryptionAccessWithDefaultKey (#2479) 2019-07-10 04:45:48 +02:00
JT Olio
a79c7d77f3 overlay cache: slight modification of node-is-online rules (#2490) 2019-07-09 22:36:09 -04:00
Alexander Leitner
1c5db71faf
Change protobuf expirations to use time.Time (#2509)
* Change protobuf expirations to use time.Time instead of timestamp.Timestamp
2019-07-09 17:54:00 -04:00
Michal Niewrzal
bbc25a2bf7 Drop SN certifiates table from DB (#2498) 2019-07-09 17:33:45 -04:00
Jeff Wendling
d616be8ae0 storagenode: use minimum time in the order for expiration (#2504) 2019-07-09 17:16:30 -04:00
Egon Elbre
910e4744ad
satellite/metainfo: optimize pointerdb size (#2506) 2019-07-09 16:36:18 -04:00
ethanadams
4faed2098d
method name changer per PR 2469 (#2494) 2019-07-09 16:05:14 -04:00
Natalie Villasana
6aefa1ae80 adds metainfo rpc doc (#1957) 2019-07-09 21:44:01 +02:00
Egon Elbre
203f36a41f
rename order.go to order.pb.go (#2496) 2019-07-09 13:24:41 -04:00
Egon Elbre
c5de36ed1f
update to go 1.12.7 (#2493) 2019-07-09 11:56:08 -04:00
Andrew Harding
7212423c52
Link Sharing service design (#2360) 2019-07-09 09:36:08 -06:00
Vitalii Shpital
aefb775061
frontend template tabs fixed (#2497) 2019-07-09 18:04:51 +03:00
Bogdan Artemenko
e81e25c271
Web satellite master fix (#2495) 2019-07-09 16:51:33 +03:00
Jeff Wendling
671648012a Update password based key derivation design doc (#2367) 2019-07-09 00:07:03 -04:00
Ivan Fraixedes
50d601ab07 pkg/identity: Use identity error class (#2488) 2019-07-08 22:52:52 -04:00
Vitalii Shpital
93cd05d600 error icon changed (#2475) 2019-07-08 22:46:57 -04:00
JT Olio
887b784f54 discovery: use fetch info to ping (#2491) 2019-07-08 22:10:17 -04:00
ethanadams
ff6f1d1b32
storagenode: add in-memory tracking for bandwidth and disk usage (#2469)
* Add in-memory cache for bandwidth and space usage monitoring

* moved some structs around and added error handling for get piece size query

* added to existing bandwidth test.  fixed typo

* added test, updates from PR review, added monkit for new methods

* PR review updates. renamed space used methods

* changed bw cache so that only Add updates the cache and it only overwrites when the date moves forward

* moved bandwidth usage to bw and space usage to pieceinfodb

* fixed interface comment

* removed pointer from sync.Once
2019-07-08 20:33:50 -04:00
Jess G
f9696d6c5e
satellite/metainfo: add buckets RPC and database (#2460)
* add db interface and methods, add sa metainfo endpoints and svc

* add bucket metainfo svc funcs

* add sadb bucekts

* bucket list gets all buckets

* filter buckets list on macaroon restrictions

* update pb cipher suite to be enum

* add conversion funcs

* updates per comments

* bucket settings should say default

* add direction to list buckets, add tests

* fix test bucket names

* lint err

* only support forward direction

* add comments

* minor refactoring

* make sure list up to limit

* update test

* update protolock file

* fix lint

* change per PR
2019-07-08 15:32:18 -07:00
Stefan Benten
16156e3b3d
Ensure we force a segment size and account storage before committing them (#2473) 2019-07-08 18:24:38 -04:00
Alexander Leitner
3587e1a579 Change pointerdb pointer to use time.Time for Creation date (#2483) 2019-07-09 00:16:50 +02:00