Commit Graph

32 Commits

Author SHA1 Message Date
JT Olio
9c5708da32 pkg/*: add monkit task to missing places (#2109) 2019-06-04 13:36:27 +02:00
Jennifer Li Johnson
5395ff5fe6
Refactor accountingdb interface (#1897)
* splits accounting db into storagenodeaccounting and projectaccounting interfaces and renames methods to match
2019-05-10 15:05:42 -04:00
Bill Thorp
ea978dd674
hopefully sensible satellite defaults (#1888)
* hopefully sensible satellite defaults
2019-05-07 10:44:47 -04:00
Jennifer Li Johnson
db6a2a46a9
set delete-tallies default to false (#1844) 2019-04-25 15:30:37 -04:00
Cameron
9f6b010748 rollup interval fixes (#1739)
* set all intervals to UTC in rollupStats map, only delete latest day after both rollups

* clean up usage of interval, use intervalEndTime rather than createdAt

* change some variable names, add comments

* add flag for tally deletion

* adds deletetallies flag to testplanet

* space

* Removes println:

* adds test for deletes false
2019-04-23 15:21:30 -04:00
Jennifer Li Johnson
253cbe5fa0
updates rollup default interval (#1800) 2019-04-22 15:18:16 -04:00
JT Olio
2744a26b60
pkg/cfgstruct: tie defaults to releases (#1787)
* tie defaults to releases

this change makes it so that by default, the flag defaults are
chosen based on whether the build was built as a release build or
an ordinary build. release builds by default get release defaults,
whereas ordinary builds by default get dev defaults.

any binary can have its defaults changed by specifying

 --defaults=dev

or

 --defaults=release

Change-Id: I6d216aa345d211c69ad913159d492fac77b12c64

* make release defaults more clear

this change extends cfgstruct structs to support either
a 'default' tag, or a pair of 'devDefault' and 'releaseDefault'
tags, but not both, for added clarity

Change-Id: Ia098be1fa84b932fdfe90a4a4d027ffb95e249c6

* clarify cfgstruct.DefaultsFlag

Change-Id: I55f2ff9080ebbc0ce83abf956e085242a92f883e
2019-04-19 12:17:30 -06:00
Cameron
32192aca10
convert times to UTC before entering as utimestamp (#1708) 2019-04-09 15:12:58 -04:00
Jennifer Li Johnson
8549421385
Remove bw from tally service + query bandwidth in rollup service (#1618) 2019-04-04 11:20:59 -04:00
Cameron
34a439a99c
Delete accounting raws after rollup (#1646)
* enable deletion of raws

* assert deletion of raws during TestRollupRaws

* add some unit tests for DeleteRawBefore
2019-04-02 23:55:24 -04:00
Jess G
d51bdf14df
project usage limiting (#1561)
* reorg uplink cmd files for consistency

* init implementation of usage limiting

* Revert "reorg uplink cmd files for consistency"

This reverts commit 91ced7639bf36fc8af1db237b01e233ca92f1890.

* add changes per CR comments

* fix custom query to use rebind

* updates per convo about what to limit on

* changes per comments

* fix syntax and comments

* add integration test, add db methods for test

* update migration, add rebind to query

* update testdata for psql

* remove unneeded drop index statement

* fix migrations, fix calculate usage limit

* fix comment

* add audit test back

* change methods to use bucketName/projectID, fix tests

* add changes per CR comments

* add test for uplink upload and err ssg

* changes per CR comments

* check get/put limit separately
2019-04-02 11:21:18 -07:00
Bill Thorp
665fd33e3c
Repair queue isolation level fix (#1466)
Implemented custom SQLite and Postgres Repairqueue Dequeue handlers
2019-03-14 17:12:47 -04:00
Bill Thorp
1f4e1a3ab0
framework to delete raw accounting tallies (#1439)
framework to delete raw accounting tallies, renamed Query
2019-03-12 09:08:23 -04:00
Bill Thorp
66718cc5e6
Development defaults for configuration (#1430)
added --dev command line option, cfgstruct.DevFlag(), and cfgstruct.SetupFlag()
2019-03-12 08:51:06 -04:00
Egon Elbre
9c1e299f3c
Ensure everyone sees everyone else (#1275) 2019-02-08 11:25:13 +02:00
Bill Thorp
0b35762105
Convert Payments to use SQL, for SUM() and Wallet (#1266)
* payments query no longer DBX, using SQL

* sum in SQL

* removed old function

* fixed rollup test

* wrap errors

* removed DBX code
2019-02-07 15:26:55 -05:00
Cameron
bfe8060f4d
Rollup tests (#1224)
* createdAt field no longer autoinsert

* rollup tests
2019-02-06 23:16:24 -05:00
Bill Thorp
0f662b8e38 sql based tally (#1166)
* WIP

* wacky changes

* more

* it builds.... IT BUILDS!!!

* fixed SQL, broke out saving tallies for testing

* shorter lines

* fixed SQL, moved tally_test to testplanet

* lint

* WIP logic error preventing PUT and GETs to same serialnum

* fixed BWA test

* fixed temporary brain failure

* eliminated magic numbers

* cleaned up satellite uplink stats

* use errs.Combine instead

* thrashing

* fixed tally erroneous error msg

* fixed tally test

* lint

* SQL syntax attempt to fix

* spelling error

* made bwa db test resist old postgres data

* postgres pk error msg hunting

* postgres pk error msg hunting

* postgres

* err might be nil?

* fixed error logging bug

* hopefully solved postgres issue

* using rebind

* moved tests to _test package

* fixing test dirs

* finally made sense of Egons package name feedback

* UTC, array fixes
2019-02-01 13:50:12 -05:00
Cameron
f5b1e77cf2 rollup timestamp fix (#1132)
* create timestamp if isNew

* pass isNil into SaveRollup
2019-01-24 15:41:22 -05:00
Jennifer Li Johnson
856b98997c
updates copyright 2018 to 2019 (#1133) 2019-01-24 15:15:10 -05:00
Jennifer Li Johnson
030d294399
adds logs in rollup and tally run (#1131) 2019-01-24 15:05:53 -05:00
Egon Elbre
5de7f8af7f
Satellite Peer (#1119) 2019-01-23 21:58:44 +02:00
Cameron
ecd704692e
wip testing and timestamp fix (#1100)
* wip testing and timestamp fix

* fixed import cycle
2019-01-18 11:53:23 -05:00
Jennifer Li Johnson
c4b876ddc5
Jj/bytes (#1085)
* run tally and rollup

* sets dev default tally and rollup intervals
2019-01-17 14:39:32 -05:00
Bill Thorp
342dc857f5 rollup query (#1056)
* implemention notes

* more notes

* starting rollup query

* not working yet

* fixed build

* fixed cfg bug

* change context cancelled errs to debugs

* using byte hours for at rest tally

* revert changes to go.mod

* comment fixes

* prevent double recording tallies in rollup

* linting

* stop leaking dbx

* nodeid changes

* fix build
2019-01-16 14:30:33 -05:00
Bill Thorp
f9845e7e92
changes to masterdb to support accountingDB (#846)
* added rollup to captplanet, moved accountingDB to masterdb
2018-12-14 09:27:21 -05:00
Jennifer Li Johnson
6642f97142
Jj/accounting updates (#820)
* tallies up data stored on each node in pointerdb

* adds comments for data type enums

* changes Open to BeginTx because Go convention

* removes online status check from identify active nodes

* changes identifyactivenodes to calculatestaticdata

* updates accounting dbx names
2018-12-12 16:24:08 -05:00
Bill Thorp
ba5f71810e
added tally to capt run.go (#751)
* added tally to capt run.go, ParseURL -> SplitDBURL
2018-12-12 08:15:34 -05:00
Bryan White
2a0c4e60d2
preparing for use of customtype gogo extension with NodeID type (#693)
* preparing for use of `customtype` gogo extension with `NodeID` type

* review changes

* preparing for use of `customtype` gogo extension with `NodeID` type

* review changes

* wip

* tests passing

* wip fixing tests

* more wip test fixing

* remove NodeIDList from proto files

* linter fixes

* linter fixes

* linter/review fixes

* more freaking linter fixes

* omg just kill me - linterrrrrrrr

* travis linter, i will muder you and your family in your sleep

* goimports everything - burn in hell travis

* goimports update

* go mod tidy
2018-11-29 19:39:27 +01:00
Bill Thorp
b8b0d75cc6
made accounting use migrate (#688)
* made accounting use migrate
2018-11-26 16:49:55 -05:00
Jennifer Li Johnson
93c5f385a8
Enable checker in captplanet and staging (#643)
* enable checker

* add option to use mock overlay in checker

* adds logs to checker

* appease linter
2018-11-20 10:54:22 -05:00
Jennifer Li Johnson
377832c705
Create Tally and Rollup pkgs in accounting (#642)
* creates separate tally and rollup packages and writes skeleton for rollup

* TODO add rollupDB and rawDB to rollup struct

* TODO add rawDB to tally struct
2018-11-13 20:22:18 -05:00