Commit Graph

66 Commits

Author SHA1 Message Date
Yaroslav Vorobiov
35edc2bcc3 satellite/payments: invoice creation (#3468) 2019-11-05 15:16:02 +02:00
Yehor Butko
0c2e498f09
satellite/satellitedb: console tables archview comments updated (#3465) 2019-11-04 16:37:39 +02:00
Yaroslav Vorobiov
30a3205745
satellite/payments: update account balance (#3379) 2019-10-29 18:04:34 +02:00
Ethan Adams
9905f2c61e add piece num to transfer queue PK (#3390) 2019-10-28 11:08:33 -04:00
Natalie Villasana
45c35d7c3f
satellite/satellitedb: add exit_status column to nodes table (#3301) 2019-10-17 11:01:39 -04:00
Yaroslav Vorobiov
24e72f35d3
satellite/payments: token deposit (#3283) 2019-10-17 17:04:50 +03:00
Yehor Butko
0cc23add5b
satellite/payments - payment account setup (#3187) 2019-10-10 20:12:23 +03:00
Ethan Adams
9edfb6efe0
satellite/satellitedb: Initial GE Satellite DB Implementation (#3049)
Initial GE Satellite DB impl
Add basic CRUD operations for graceful_exit_progress and graceful_exit_transfer_queue tables.
2019-09-25 11:12:44 -06:00
Jess G
93788e5218
remove kademlia: create upsert query to update uptime (#2999)
* create upsert query for check-in method

* add tests

* fix lint err

* add benchmark test for db query

* fix lint and tests

* add a unit test, fix lint

* add address to tests

* replace print w/ b.Fatal

* refactor query per CR comments

* fix disqualified, only set if null

* fix query

* add version to updatecheckin query

* fix version

* fix tests

* change version for tests

* add version to tests

* add IP, add transport, mv unit test

* use node.address as arg

* add last ip

* fix lint
2019-09-19 11:37:31 -07:00
Ethan Adams
886041e0ba
satellite/satellitedb: add new graceful exit tables and add graceful exit fields to nodes (#3033)
DB schema changes for satellite to support Graceful Exit
2019-09-13 12:57:32 -04:00
Bogdan Artemenko
2c7813d40d
satellite/console: Added email normalization to users table (#2586) 2019-09-10 17:00:33 +03:00
aligeti
33aff71959 satellitedb/overlay: add database for storing peer identities (#2764) 2019-08-26 19:49:42 +03:00
Bryan White
6400d63a6c
satellite/satellitedb: Add piece count column to nodes table (#2795) 2019-08-19 12:58:13 +02:00
aligeti
32f95a14fd
satellite/certdb: remove certdb that was used to store uplink certificates (#2760)
* satellitedb/certDB: refactors of the node certificate storage DB table

The existing implementation doesnt allow to store the complete certificate chain of uplinkIDs or storagenodeIDs, so the current table is dropped and new table will be added which addresses the storage and retrieval of certificates

pkg/identity: fixes spelling mistakes that I missed on PR#2754

Fixes V3-1992/V3-2388
2019-08-12 10:41:34 -04:00
Yehor Butko
b0e5964718
satellite/console: create owner_id column for project table (#2706)
* satellite/console: create owner_id column for project table
2019-08-07 15:28:13 +03:00
aligeti
65932ad692
Updating the certdb to support storage of multiple public keys for same node ID (#2692)
* updating the certdb to support storage of multiple public keys for same node ID
2019-08-02 13:47:35 -04:00
Yingrong Zhao
51833d0650
satellite/satellitedb: get active offer for partners (#2664)
* get partner offer

* fix lint

* fix deleting user account

* fix sqlite query

* add comments

* fix migration

* fix query_test

* add error logs

* add tests for user credits
2019-08-01 13:46:33 -04:00
Yingrong Zhao
efa2c776b7
satellite/satellitedb: add updateEarnedCredits method for user_credits table (#2609)
* parent 13dd501042
author Yingrong Zhao <yingrong.zhao@gmail.com> 1563560530 -0400
committer Yingrong Zhao <yingrong.zhao@gmail.com> 1563581673 -0400

parent 13dd501042
author Yingrong Zhao <yingrong.zhao@gmail.com> 1563560530 -0400
committer Yingrong Zhao <yingrong.zhao@gmail.com> 1563581428 -0400

satellite/console: add referral link logic (#2576)

* setup referral route

* referredBy

* add user id

* modify user query

* separate optional field from userInfo

* get current reward on init of satellite gui

* remove unsed code

* fix format

* only apply 0 credit on registration

* only pass required information for rewards

* fix time parsing

* fix test and linter

* rename method

* add todo

* remove user referral logic

* add null check and fix format

* get current offer

* remove partnerID on CreateUser struct

* fix storj-sim user creation

* only redeem credit when there's an offer

* fix default offer configuration

* fix migration

* Add helper function for get correct credit duration

* add comment

* only store userid into user_credit table

* add check for partner id to set correct offer type

* change free credit to use invitee credits

* remove unecessary code

* add credit update in activateAccount

* remove unused code

* fix format

* close reader and fix front-end build

* move create credit logic into CreateUser method

* when there's no offer set, user flow shouldn't be interrupted by referral program

* add appropriate error messages

* remove unused code

* add comment

* add error class for no current offer error

* add error class for credits update

* add comment for migration

* only log secret when it's in debug level

* fix typo

* add testdata
2019-07-30 09:21:00 -04:00
Yingrong Zhao
6778caf846
satellite/console: add referral link logic (#2576)
* setup referral route

* referredBy

* add user id

* modify user query

* separate optional field from userInfo

* get current reward on init of satellite gui

* remove unsed code

* fix format

* only apply 0 credit on registration

* only pass required information for rewards

* fix time parsing

* fix test and linter

* rename method

* add todo

* remove user referral logic

* add null check and fix format

* get current offer

* remove partnerID on CreateUser struct

* fix storj-sim user creation

* only redeem credit when there's an offer

* fix default offer configuration

* fix migration

* Add helper function for get correct credit duration

* add comment

* only store userid into user_credit table

* add check for partner id to set correct offer type

* change free credit to use invitee credits

* remove unecessary code
2019-07-19 14:22:10 -04:00
Bill Thorp
a7cc940776
Nodes should not be able to fail the same audit multiple times (#2404)
update pointer on audit failure
2019-07-18 14:08:15 -04:00
aligeti
daa3b32ee2
Add Attribution Columns to appropriate tables for OSPP referral link (#2516)
* adds "partner_id" column to user, project, api_key & bucket_metainfo tables
2019-07-12 13:59:19 -04:00
Maximillian von Briesen
b590e53d64
Order by attempted time in injured segments select (#2533) 2019-07-12 13:35:20 -04:00
Yingrong Zhao
d887ffec62
satellite/satellitedb: add default offer for offers table (#2522)
* add default offer for offers table

* fix migration test

* Trigger Jenkins

* set the default value to be correct type

* skip soon will deleted test

* fix test data

* add orderby for ListAll

* change durations, redeemable cap to be a nullable field

* remove unecessary code
2019-07-12 10:19:38 -04:00
Yingrong Zhao
6e57b102c5
satellite/satellitedb: remove num_redeemed in offers table (#2510)
* remove num_redeemed

* remove the field frol Offer struct

* remove the field in the UI templates

* fix merge conflict

* fix migration

* fix merge conflict

* fix typo
2019-07-12 09:19:18 -04:00
Michal Niewrzal
268c629ba8
Replace base64 encoding for path segments (#2345) 2019-07-11 13:26:07 -04:00
Bogdan Artemenko
32e0227c45
Project Payment methods (#2037) 2019-07-10 23:29:26 +03:00
Jess G
0587dd79d6 change bucket table to fix conflict with pointerDB (#2452)
* change bucket table to fix conflict with pointerDB

* update init query

* update migration for dbx psql

* fix migration testdata
2019-07-04 00:03:56 +02:00
littleskunk
a2362f92dc
Rollback uptime disqualification (#2417) 2019-07-02 10:39:36 +02:00
Jess G
0158b1eb5a
add bucket metadata table in SA masterDB (#2369)
* add bucket metadata table in SA masterDB

* fix indentation

* update db model per CR comments

* update testdata

* add missing field on sql testdata

* fix args to testdata

* unique bucket name

* fix fkey constraint for test

* fix one too many commas

*  update timestamp type

* Trigger Jenkins

* Trigger Jenkins yet again
2019-07-01 13:45:21 -07:00
ethanadams
605e3fb730
v3-2023: Add migration for project_id change in V3-2010 (#2332)
* v3-2023: add project_id migration for bucket_storage_tallies and bucket_bandwidth_rollups

* added test data for migration 37

* corrected data format

* test sql update

* migrate script updates

* adding previous data
2019-06-25 15:21:34 -04:00
Stefan Benten
01beaa289a
Mask IP Addresses to subnets (#2305) 2019-06-24 17:33:18 +02:00
Bill Thorp
d1e9829e84 preserve reputation migration (#2295)
Adds a migration step to pull in old reputation success / total counts into modern alpha / beta scores

If audit success count is less than 50, audit alpha will be set to 50
If uptime success count is less than 100, uptime alpha will be set to 100
This helps us deal with cases where nodes have not been audited or checked for uptime yet, in which case alpha/beta values of 0/0 would cause a node to be considered disqualified.

A node with audit alpha/beta of 50/0 will be disqualified on the 19th check
A node with uptime alpha/beta of 100/0 will be disqualified on the 44th check

This does not affect brand new nodes (nodes that were not in the database before this change). The alpha/beta values for those nodes will be set to 1/0 as before
2019-06-22 21:18:54 -04:00
Bill Thorp
8f47fca5d3
Remove audit / uptime ratio fields (#2247)
* removed ratios
2019-06-21 13:14:53 -04:00
Bill Thorp
81134e97cc
nodes db - adds alpha and beta reputation fields (#2215)
* node dbx and migrate adding alpha and beta reputation
2019-06-18 09:45:02 -04:00
Ivan Fraixedes
35c8648330
[v3-1914] Storage node disqualification: Change type from bool to timestamp (#2212)
* satellite/satellitedb: Alter nodes disqualification column
  Change the type of the 'disqualification' column of the nodes table from
  boolean to timestamp.
* overlay/cache: Change Disqualified field type
  Change the Disqualified field type the NodeDossier struct type from bool
  to time.Time to match with the disqualified type used by the DB layer.
* satellite/satellitedb: Update queries uses disqualified
  Update the queries which uses the disqualified column due to the column
  type has been changed from boolean to nullable timestamp.
* docs/design: Update disqualification due impl changes
  Update the disqualification design document to contain the architectural
  change required to be able to restore unfair disqualified nodes in case
  of an unexpected cause (bug, mistake, hard network disconnection, etc.).
2019-06-18 11:14:31 +02:00
Egon Elbre
f4f776d092 Use mail.test as domain in emails (#2224) 2019-06-18 02:28:40 +02:00
Faris Huskovic
c0fe061a67 add user credits table (#2085)
* add user credits table

* change primary key, change type for credit_type, and change relation kind of foreign keys from cascade to restrict

* modify table and query methods

* modify schema

* add dbx queries

* add migration file

* add orderby to read available credit entries
2019-06-13 09:52:33 -04:00
ethanadams
380684e766 Removing bucket_id and adding project_id and bucket_name as primary key (#2182)
* removing bucket_id and adding project_id and bucket_name as primary key

* cleanup
2019-06-13 08:28:55 -04:00
Yaroslav Vorobiov
51db703b2b
Console add payments to service (#2100) 2019-06-06 19:07:14 +03:00
Egon Elbre
28a1201590 pkg/bwagreements: remove service (#2138) 2019-06-06 15:57:58 +02:00
Dennis Coyle
b9d586901e
Value Attribution DBX Model (#2116)
* adds model to satellite dbx

* cleans up model spacing

* generated golang from dbx

* added migration steps

* Added testdata

* changed node_id -> bucket_id

* adds -- NEW DATA -- to testdata

* more testdata changes

* adds -- NEW DATA -- line

* dbx makes the table plural

* missed a singular value_attribution

* restart jenkins

* Update satellitedb.dbx

* adjust to PR comments

* autogenerated dbx models

* restart jenkins
2019-06-05 12:06:14 -04:00
Yingrong Zhao
09b0c2a630
create db implementation for offer table (#2031)
* init marketing service

Fix linting error

Create offerdb implementation

Create offers service

Add update method

Create offer table and migration

Fix linting error

fix conflicts

Insert new data

Change duration to have clear indication to be based on days

add error wrapper

Change from using uuid to int for id field

* Create Marketing service

* make error virable name more readable

* add condition in update service method to check offer status

* generate lock file

Change get to listAllOffers

* Add method for getting current offer

wip

* add check for expires_at in update method

* Fix conflicts

* add copyright header

* Fix linting error

* only allow update to active offers

* add isDefault argument to GetCurrent

* Update lock file

* add migration file

* finish migrate for adding credit_in_cents for both award and invitee

* save 100 years as expiration date for default offers

* create crud test for offers

* add GetCurrent test

* modify doc

* Fix GetCurrent to work with default offer

* fix linting issue

* add more tests and address feedbacks

* fix migration file

* add type column back to match with mockup design

* add type column back to match with mockup design

* move doc changes to new pr

* add comments

* change GetCurrent to GetCurrentByType

* fix typo
2019-06-04 15:17:01 -04:00
Natalie Villasana
6db9388082 add disqualified column to nodes table (#2086)
* add disqualified column to nodes table, update migrate script and testdata

* fix crazy formatting of postgres.v25.sql
2019-05-30 17:38:23 -04:00
Michal Niewrzal
f731267e8c Per-project usage limiting (#2036)
What: Changes to support custom usage limit for the project. With this implementation by default project usage limit is taken from configuration flag. If project DB field usage_limit will be set to value larger than 0 it will become custom usage limit and we will be used to verify is limit was exceeded.

Whats changed:

usage_limit (bigint) field added to projects table (with migration)
things related to project usage moved from metainfo endpoint to project usage type
accounting.ProjectAccounting extended with GetProjectUsageLimits() method
Why: We need to have different usage limits per project. https://storjlabs.atlassian.net/browse/V3-1814
2019-05-28 09:36:52 -06:00
Jeff Wendling
1bd52b9f90 server side macaroons (#1945)
What: Adds macaroon support to the server side

Why: So that api keys are now macaroons
2019-05-24 10:51:27 -06:00
Yingrong Zhao
f9045f8385 Create offer table in satellitedb for free credit program (#1975) 2019-05-22 23:41:55 +02:00
Cameron
4058c29ca4
filter duplicate node IPs (#1890)
* add last_ip field to dbx model node, generate dbx

* add last_ip to node proto, generate pb

* migrate

* resolve address in transport.DialNode, update lastIp in cache.UpdateAddress

* use net.SplitHostPort to isolate host address from port

* define DistinctIPs flag

* add test for GetIP

* select last_ip when querying for nodes

* if distinctIPs flag == true, query for nodes with distinct IPs

* some basic tests

* change last_ip to field 14 in proto

* remove comments

* check err

* change distinctIPs to distinctIP

* exclude IPs from newNodes in query for reputable nodes

* add index on last_ip

* only add to excludedIPs if flag is true

* test half new nodes returns distinct IPs

* fix alignment

* add test

* rework ip filter query, add retry logic, add switch for database driver

* add retry to SelectNewNodes

* change discovery intervals so IPs don't get overwritten

* remove TestGetIP

* edit updating node stats in test

* split exclude into nodeIDs and IPs

* separate non-distinct IP query into other function

* trigger checks

* remove else block
2019-05-22 16:06:27 -04:00
Maximillian von Briesen
cc020dfdea
Create containment mode database table and migrate scripts (#1970) 2019-05-16 10:11:15 -04:00
Bogdan Artemenko
f46487b015
ResetPassword Table and all CRUD methods. (#1916) 2019-05-13 18:53:52 +03:00
Jennifer Li Johnson
5395ff5fe6
Refactor accountingdb interface (#1897)
* splits accounting db into storagenodeaccounting and projectaccounting interfaces and renames methods to match
2019-05-10 15:05:42 -04:00