JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,866 Commits 5 Branches 0 Tags 94 MiB
6b23380d6b
Commit Graph

29 Commits

Author SHA1 Message Date
Jess G
e5c48fab74
fix ordersDB methods to take correct args (#2314) 
* fix orderdDB methods to take correct args

* update tally to save projectID in correct format

* update var names in splitBucket test

* changes per CR comments
 2019-06-25 08:58:42 -07:00
Michal Niewrzal
f731267e8c Per-project usage limiting (#2036) 
What: Changes to support custom usage limit for the project. With this implementation by default project usage limit is taken from configuration flag. If project DB field usage_limit will be set to value larger than 0 it will become custom usage limit and we will be used to verify is limit was exceeded.

Whats changed:

usage_limit (bigint) field added to projects table (with migration)
things related to project usage moved from metainfo endpoint to project usage type
accounting.ProjectAccounting extended with GetProjectUsageLimits() method
Why: We need to have different usage limits per project. https://storjlabs.atlassian.net/browse/V3-1814
 2019-05-28 09:36:52 -06:00
Jennifer Li Johnson
5395ff5fe6
Refactor accountingdb interface (#1897) 
* splits accounting db into storagenodeaccounting and projectaccounting interfaces and renames methods to match
 2019-05-10 15:05:42 -04:00
Jess G
55bfe507e5
add tests for tally and accounting db (#1675) 
* add tally calcAtRestData test

* add account db tests

* move tests from sadb to accounting pkg

* add func to calcEncSize

* remove unneeded pad changes

* update comment

* add changes to comments
 2019-04-09 09:48:35 -04:00
Jess G
5dfe28a8c3
use allocated instead of settled (#1700) 
* use allocated instead of settled

* add expansion factor

* changes per CR
 2019-04-08 14:35:54 -07:00
Jennifer Li Johnson
8549421385
Remove bw from tally service + query bandwidth in rollup service (#1618) 2019-04-04 11:20:59 -04:00
Jess G
d51bdf14df
project usage limiting (#1561) 
* reorg uplink cmd files for consistency

* init implementation of usage limiting

* Revert "reorg uplink cmd files for consistency"

This reverts commit 91ced7639bf36fc8af1db237b01e233ca92f1890.

* add changes per CR comments

* fix custom query to use rebind

* updates per convo about what to limit on

* changes per comments

* fix syntax and comments

* add integration test, add db methods for test

* update migration, add rebind to query

* update testdata for psql

* remove unneeded drop index statement

* fix migrations, fix calculate usage limit

* fix comment

* add audit test back

* change methods to use bucketName/projectID, fix tests

* add changes per CR comments

* add test for uplink upload and err ssg

* changes per CR comments

* check get/put limit separately
 2019-04-02 11:21:18 -07:00
Cameron
6d43832c4f
record bucket data into bucket_storage_tally table (#1595) 
* add MetadataSize to stats

* add logic for accumulating bucket stats in calculateAtRestData

* rename stats to BucketTally, move to accounting package

* define method on accountingDB for inserting bucketTallies

* insert bucketTallies into bucket_storage_tally table
 2019-04-01 09:42:17 -04:00
Kaloyan Raev
d1639c4157 Merge statdb pkg into overlay pkg (#1570) 2019-03-25 18:25:09 -04:00
Bill Thorp
665fd33e3c
Repair queue isolation level fix (#1466) 
Implemented custom SQLite and Postgres Repairqueue Dequeue handlers
 2019-03-14 17:12:47 -04:00
Bill Thorp
1f4e1a3ab0
framework to delete raw accounting tallies (#1439) 
framework to delete raw accounting tallies, renamed Query
 2019-03-12 09:08:23 -04:00
Cameron
bfe8060f4d
Rollup tests (#1224) 
* createdAt field no longer autoinsert

* rollup tests
 2019-02-06 23:16:24 -05:00
Bill Thorp
0f662b8e38 sql based tally (#1166) 
* WIP

* wacky changes

* more

* it builds.... IT BUILDS!!!

* fixed SQL, broke out saving tallies for testing

* shorter lines

* fixed SQL, moved tally_test to testplanet

* lint

* WIP logic error preventing PUT and GETs to same serialnum

* fixed BWA test

* fixed temporary brain failure

* eliminated magic numbers

* cleaned up satellite uplink stats

* use errs.Combine instead

* thrashing

* fixed tally erroneous error msg

* fixed tally test

* lint

* SQL syntax attempt to fix

* spelling error

* made bwa db test resist old postgres data

* postgres pk error msg hunting

* postgres pk error msg hunting

* postgres

* err might be nil?

* fixed error logging bug

* hopefully solved postgres issue

* using rebind

* moved tests to _test package

* fixing test dirs

* finally made sense of Egons package name feedback

* UTC, array fixes
 2019-02-01 13:50:12 -05:00
Bill Thorp
60946c2024
make bandwidth agreements sensible: without []byte's (#1152) 
removed []byte's from bandwidth agreement protocol buffers
 2019-01-28 14:45:25 -05:00
Cameron
f5b1e77cf2 rollup timestamp fix (#1132) 
* create timestamp if isNew

* pass isNil into SaveRollup
 2019-01-24 15:41:22 -05:00
Jennifer Li Johnson
856b98997c
updates copyright 2018 to 2019 (#1133) 2019-01-24 15:15:10 -05:00
Jennifer Li Johnson
70940b8710
Making sure payments works (#1122) 
* byte hours

* updates comment

* removes payment test data methods

* wip

* adds tally and rollup to peer

* remove sim test for payment (for now)

* lint error
 2019-01-24 13:44:14 -05:00
Cameron
ecd704692e
wip testing and timestamp fix (#1100) 
* wip testing and timestamp fix

* fixed import cycle
 2019-01-18 11:53:23 -05:00
Jennifer Li Johnson
e6fbf63620
Generate Payments Report (#1079) 2019-01-17 13:34:13 -05:00
Bill Thorp
342dc857f5 rollup query (#1056) 
* implemention notes

* more notes

* starting rollup query

* not working yet

* fixed build

* fixed cfg bug

* change context cancelled errs to debugs

* using byte hours for at rest tally

* revert changes to go.mod

* comment fixes

* prevent double recording tallies in rollup

* linting

* stop leaking dbx

* nodeid changes

* fix build
 2019-01-16 14:30:33 -05:00
Bill Thorp
fb4a11ebb0
Differentiate GET and PUT operations in the tally service (#965) 
* draft of new bandwidth agreement types
* updated storagenode report for new types
* use correct pba types
 2019-01-10 06:41:57 -05:00
Egon Elbre
0ca03b41e2
Use generated locking database implementation (#947) 2019-01-02 19:53:27 +02:00
Jennifer Li Johnson
01df85704c
Updates accounting tables and allows tally to update at raw table for at rest data (#874) 2018-12-18 12:18:42 -05:00
Bill Thorp
f9845e7e92
changes to masterdb to support accountingDB (#846) 
* added rollup to captplanet, moved accountingDB to masterdb
 2018-12-14 09:27:21 -05:00
Jennifer Li Johnson
6642f97142
Jj/accounting updates (#820) 
* tallies up data stored on each node in pointerdb

* adds comments for data type enums

* changes Open to BeginTx because Go convention

* removes online status check from identify active nodes

* changes identifyactivenodes to calculatestaticdata

* updates accounting dbx names
 2018-12-12 16:24:08 -05:00
Bill Thorp
ba5f71810e
added tally to capt run.go (#751) 
* added tally to capt run.go, ParseURL -> SplitDBURL
 2018-12-12 08:15:34 -05:00
JT Olio
1c96db01ba
better database error handling (#784) 
* better database error handling

Change-Id: I28dbd69cf6c2fa268e02405521ff6e6c1a68a702

* missing comments added

* missing comment added
 2018-12-07 07:46:42 -07:00
Bill Thorp
b2a8a10b16
redo tally query (#742) 
* redo tally query

* broken, sharing w/ Jen

* tally tests added, failiing

* fix txn issue in tally

* fix lint
 2018-12-05 09:03:23 -05:00
Bill Thorp
b8b0d75cc6
made accounting use migrate (#688) 
* made accounting use migrate
 2018-11-26 16:49:55 -05:00