Commit Graph

423 Commits

Author SHA1 Message Date
Isaac Hess
e34b2c553c
Reduce UpdateAddress calls with address cache (#2681) 2019-08-06 16:56:12 -06:00
Egon Elbre
c8edeb0257
satellite/overlay: rename overlay.Cache to overlay.Service (#2717) 2019-08-06 19:35:59 +03:00
Michal Niewrzal
de7dddbe59
metainfo: Batch request (#2694) 2019-08-06 16:56:23 +02:00
Yingrong Zhao
c32fc283c3
make display order consistent (#2705) 2019-08-06 10:41:25 -04:00
Bill Thorp
fcbc9d71da
satellite/repair: add shouldDelete (#2702)
* add shouldDelete to repair
2019-08-05 11:09:16 -04:00
Michal Niewrzal
688d932d93
Make one implementation for SetAttribution/SetBucketAttribution (#2683) 2019-08-05 09:07:40 +02:00
aligeti
65932ad692
Updating the certdb to support storage of multiple public keys for same node ID (#2692)
* updating the certdb to support storage of multiple public keys for same node ID
2019-08-02 13:47:35 -04:00
Maximillian von Briesen
4547084f26
Implement checker observer (#2620)
Integrate checker into metainfo loop
2019-08-01 14:44:32 -04:00
Yingrong Zhao
51833d0650
satellite/satellitedb: get active offer for partners (#2664)
* get partner offer

* fix lint

* fix deleting user account

* fix sqlite query

* add comments

* fix migration

* fix query_test

* add error logs

* add tests for user credits
2019-08-01 13:46:33 -04:00
Michal Niewrzal
287fdf9936
Integrate new Metainfo calls (server side) (#2682) 2019-08-01 11:04:31 +02:00
Natalie Villasana
238e264a8f
satellite confirms success threshold reached (#2657) 2019-07-31 14:28:43 -04:00
ethanadams
c9b46f2fe2
V3-1987: Optimize audits stats persistence (#2632)
* Added batch update stats for recordAuditSuccessStatus
* Added batch update stats to recordAuditFailStatus
* added configurable batch size
* build individual update/delete statements so the statements can be batched into 1 call to the DB
* notified #config-changes channel and ran make update-satellite-config-lock
* updated tests to use batch update stats
2019-07-31 13:21:06 -04:00
Yingrong Zhao
4adafd056c
satellite/rewards: add generate referral link logic (#2655)
* add GeneratePartnerLink function

* remove unnecessary code

* add copyright

* fix typo and url path creation logic

* re-organize partners map

* fix creating offer for partners
2019-07-31 12:01:18 -04:00
Egon Elbre
4f0d39cc64
don't use global loggers (#2675) 2019-07-31 17:38:44 +03:00
Egon Elbre
ec3d5c0bdd
don't use global loggers (#2671)
* pkg/server: don't use global logger
* satellite/overlay: use correct logger
* pkg/kademlia: use correct logger
* linksharing: use conventional way to pass in logger
* use zaptest in tests
2019-07-31 15:09:45 +03:00
aligeti
6dee5a3918
added more logging info (#2667) 2019-07-30 12:38:10 -04:00
aligeti
55e3a37d19
Adds more detailed error logging information V3-2239 (#2654)
* added more logging info
2019-07-30 12:03:25 -04:00
Alexander Leitner
4632ab0a67
Delete irreparable segments (#2642)
* Delete irreparable segments
2019-07-30 11:38:25 -04:00
Alexander Leitner
159ad439b1
Add count to repair queue (#2661)
* Add count to repair queue
2019-07-30 11:21:40 -04:00
Yingrong Zhao
efa2c776b7
satellite/satellitedb: add updateEarnedCredits method for user_credits table (#2609)
* parent 13dd501042
author Yingrong Zhao <yingrong.zhao@gmail.com> 1563560530 -0400
committer Yingrong Zhao <yingrong.zhao@gmail.com> 1563581673 -0400

parent 13dd501042
author Yingrong Zhao <yingrong.zhao@gmail.com> 1563560530 -0400
committer Yingrong Zhao <yingrong.zhao@gmail.com> 1563581428 -0400

satellite/console: add referral link logic (#2576)

* setup referral route

* referredBy

* add user id

* modify user query

* separate optional field from userInfo

* get current reward on init of satellite gui

* remove unsed code

* fix format

* only apply 0 credit on registration

* only pass required information for rewards

* fix time parsing

* fix test and linter

* rename method

* add todo

* remove user referral logic

* add null check and fix format

* get current offer

* remove partnerID on CreateUser struct

* fix storj-sim user creation

* only redeem credit when there's an offer

* fix default offer configuration

* fix migration

* Add helper function for get correct credit duration

* add comment

* only store userid into user_credit table

* add check for partner id to set correct offer type

* change free credit to use invitee credits

* remove unecessary code

* add credit update in activateAccount

* remove unused code

* fix format

* close reader and fix front-end build

* move create credit logic into CreateUser method

* when there's no offer set, user flow shouldn't be interrupted by referral program

* add appropriate error messages

* remove unused code

* add comment

* add error class for no current offer error

* add error class for credits update

* add comment for migration

* only log secret when it's in debug level

* fix typo

* add testdata
2019-07-30 09:21:00 -04:00
Yehor Butko
518a198221
CSP implementation for satellite console (#2644)
* CSP implementation for console
2019-07-30 13:13:24 +03:00
Marc Schubert
64199d37bd
Improve Node Selection Query (#2648)
* Update overlaycache.go

Removes one select statement and columns gets filtered in first query. 

Needs to be tested agains real database that this query is working and faster!

* Correct linting

reorder scans that this fit to new sql result order
2019-07-29 14:32:43 +02:00
Egon Elbre
e75813d094 satellite/repair: move segment repairer to satellite and simplify (#2651) 2019-07-29 13:24:56 +02:00
Egon Elbre
dd7c8610bb
satellite/repair: move test files (#2649) 2019-07-28 12:15:34 +03:00
Egon Elbre
5d0816430f
rename all the things (#2531)
* rename pkg/linksharing to linksharing
* rename pkg/httpserver to linksharing/httpserver
* rename pkg/eestream to uplink/eestream
* rename pkg/stream to uplink/stream
* rename pkg/metainfo/kvmetainfo to uplink/metainfo/kvmetainfo
* rename pkg/auth/signing to pkg/signing
* rename pkg/storage to uplink/storage
* rename pkg/accounting to satellite/accounting
* rename pkg/audit to satellite/audit
* rename pkg/certdb to satellite/certdb
* rename pkg/discovery to satellite/discovery
* rename pkg/overlay to satellite/overlay
* rename pkg/datarepair to satellite/repair
2019-07-28 08:55:36 +03:00
Yingrong Zhao
3b6c69c594
fix mutation (#2641) 2019-07-26 10:58:17 -07:00
Faris Huskovic
2144181c99
satellite/marketingweb: Add Partner Offer Type (#2540) 2019-07-25 18:06:23 -04:00
Stefan Benten
e260f9f1cc
Remove unnecessary UNION and Selects (#2624) 2019-07-25 15:17:12 -04:00
Kaloyan Raev
734c793deb
Add UpdatePieces method to Metainfo service (#2572) 2019-07-25 19:59:46 +03:00
Egon Elbre
0cdeae1922 add missing error handling (#2630) 2019-07-25 17:01:44 +02:00
Natalie Villasana
f11413bc8e Implement garbage collection on satellite (#2577)
* Added a gc package at satellite/gc, which contains the gc.Service, which runs garbage collection integrated with the metainfoloop, and the gc PieceTracker, which implements the metainfo loop Observer interface and stores all of the filters (about which pieces are good) for each node.
* Added a gc config located at satellite/gc/service.go (loop disabled by default in release)
* Creates bloom filters with pieces to be retained inside the metainfo loop
* Sends RetainRequests (or filters with good piece ids) to all storage nodes.
2019-07-24 13:26:43 -04:00
Michal Niewrzal
5710dc3a32
Metainfo RPC segment methods (part 2) (#2616) 2019-07-24 13:33:23 +02:00
Ivan Fraixedes
b24e60a33f
satellite/satellitedb: Use var block for single variable declar… (#2622)
* satellite/satellitedb: User var block for Error
  To follow with the code style of the majority of the sources of the
  current code base the Error variable should be in a block.
  Replacing a single var expression to a block one makes the godoc more
  consistent across the repository.

* satellite/satellitedb: Remove empty spaces end of line
2019-07-23 19:58:43 +02:00
Yingrong Zhao
aaf3283b4e
satellite/console: referral link for individual users (#2614)
* init

* change give credit to be invitee credits for free credits

* remove unused variables

* fix tests
2019-07-23 12:08:07 -04:00
Jess G
353b089927
update testplanet with libuplink (#2618)
* update testplanet uplink upload with libuplink

* add libuplink to testplanet download

* update createbucket and delete obj with libuplink

* update downloadStream, fix tests

* fix test

* updates for CR comments
2019-07-23 07:58:45 -07:00
Ivan Fraixedes
3c8f1370d2
[v3 2137] - Add more info to find out repair failures (#2623)
* pkg/datarepair/repairer: Track always time for repair
  Make a minor change in the worker function of the repairer, that when
  successful, always track the metric time for repair independently if the
  time since checker queue metric can be tracked.

* storage/postgreskv: Wrap error in Get func
  Wrap the returned error of the Get function as it is done when the
  query doesn't return any row.

* satellite/metainfo: Move debug msg to the right place
  NewStore function was writing a debug log message when the DB was
  connected, however it was always writing it out despite if an error
  happened when getting the connection.

* pkg/datarepair/repairer: Wrap error before logging it
  Wrap the error returned by process which is executed by the Run method
  of the repairer service to add context to the error log message.

* pkg/datarepair/repairer: Make errors more specific in worker
  Make the error messages of the "worker" method of the Service more
  specific and the logged message for such errors.

* pkg/storage/repair: Improve error reporting Repair
  In order of improving the error reporting by the
  pkg/storage/repair.Repair method, several errors of this method and
  functions/methods which this one relies one have been updated to be
  wrapper into their corresponding classes.

* pkg/storage/segments: Track path param of Repair method
  Track in monkit the path parameter passed to the Repair method.

* satellite/satellitedb: Wrap Error returned by Delete
  Wrap the error returned by repairQueue.Delete method to enhance the
  error with a class and stack and the
  pkg/storage/segments.Repairer.Repair method get a more contextualized
  error from it.
2019-07-23 16:28:06 +02:00
Michal Niewrzal
cba008d7df
Add GetObject method to Metainfo (#2611) 2019-07-23 13:09:12 +02:00
Jennifer Li Johnson
53d96be44a
Stylistic Go Cleanup (#2524) 2019-07-22 15:10:04 -04:00
Michal Niewrzal
6f2b85603d
Metainfo RPC segment methods (part 1) (#2567) 2019-07-22 16:45:18 +02:00
Maximillian von Briesen
537d1cf09f
Add test to ensure correct order of repair queue selection (#2551) 2019-07-22 10:10:42 -04:00
Maximillian von Briesen
6c1c3fb4a7
Add metainfo loop service (#2563)
Add a metainfo loop service on the satellite that can be subscribed to by various services that need to make use of metainfo information
2019-07-22 09:34:12 -04:00
Yingrong Zhao
6778caf846
satellite/console: add referral link logic (#2576)
* setup referral route

* referredBy

* add user id

* modify user query

* separate optional field from userInfo

* get current reward on init of satellite gui

* remove unsed code

* fix format

* only apply 0 credit on registration

* only pass required information for rewards

* fix time parsing

* fix test and linter

* rename method

* add todo

* remove user referral logic

* add null check and fix format

* get current offer

* remove partnerID on CreateUser struct

* fix storj-sim user creation

* only redeem credit when there's an offer

* fix default offer configuration

* fix migration

* Add helper function for get correct credit duration

* add comment

* only store userid into user_credit table

* add check for partner id to set correct offer type

* change free credit to use invitee credits

* remove unecessary code
2019-07-19 14:22:10 -04:00
aligeti
29b576961f
value attribution merge fix and more test cases (#2588)
* value attribution merge fix and more test cases
2019-07-19 11:17:34 -04:00
Bill Thorp
a7cc940776
Nodes should not be able to fail the same audit multiple times (#2404)
update pointer on audit failure
2019-07-18 14:08:15 -04:00
Yingrong Zhao
3e9f90b7ee
satellite/satellitedb: fix nil value from offers table (#2587)
* fix nil value

* Oops
2019-07-18 13:26:04 -04:00
Jess G
3af9250659
update irreparableDB.GetLimited query to use where instead of offset (#2585)
* update query to use where instead of offset, update tests

* update cmd/inspector irreparable

* add comment for offset
2019-07-18 09:21:21 -07:00
ethanadams
af7ffb8072 v3-2156: Add partner_id on user creation (#2571)
* Add partnerID on user creation

* added support for partner ID on create user in consoleql User

* add partner ID to api key if the user creating it has a partner ID associated with it

* updates for consoleal user and userinfo
2019-07-17 16:53:14 -04:00
Michal Niewrzal
0b827f1375
Speedup metainfo tests (#2556) 2019-07-16 21:16:41 +02:00
Kaloyan Raev
d453cd148e Consider encryption overhead when validating max segment size (#2569) 2019-07-16 15:30:10 +02:00
Michal Niewrzal
260d9c49a8
Metainfo RPC objects methods (#2534) 2019-07-16 12:39:23 +02:00