Bill Thorp
0b35762105
Convert Payments to use SQL, for SUM() and Wallet ( #1266 )
...
* payments query no longer DBX, using SQL
* sum in SQL
* removed old function
* fixed rollup test
* wrap errors
* removed DBX code
2019-02-07 15:26:55 -05:00
aligeti
b736ae4823
Store the uplinks public key on the satellite so that it can verify bandwidth requests in the future ( #1042 )
...
* integrated with bwagreement & psserver
* integrated with pointerdb
* code review updates
* refactor after code review
* uplinkdb rename to certdb
* Code review changes
2019-02-07 14:22:49 -05:00
Cameron
bfe8060f4d
Rollup tests ( #1224 )
...
* createdAt field no longer autoinsert
* rollup tests
2019-02-06 23:16:24 -05:00
Bill Thorp
690e8b2061
storage node cert cache ( #1226 )
...
* draft
* still errors
* double close fix
* added tests
* weird, goimports must not be working
* renames
* missed one
* forgot to save:
2019-02-05 12:57:56 -05:00
Egon Elbre
6564357e13
Ensure that tally tests run against postgres ( #1227 )
2019-02-05 16:23:45 +02:00
paul cannon
0238a7785b
make a map object before inserting into it ( #1217 )
...
I think this is causing panics.
2019-02-03 18:42:20 -06:00
Bill Thorp
0f662b8e38
sql based tally ( #1166 )
...
* WIP
* wacky changes
* more
* it builds.... IT BUILDS!!!
* fixed SQL, broke out saving tallies for testing
* shorter lines
* fixed SQL, moved tally_test to testplanet
* lint
* WIP logic error preventing PUT and GETs to same serialnum
* fixed BWA test
* fixed temporary brain failure
* eliminated magic numbers
* cleaned up satellite uplink stats
* use errs.Combine instead
* thrashing
* fixed tally erroneous error msg
* fixed tally test
* lint
* SQL syntax attempt to fix
* spelling error
* made bwa db test resist old postgres data
* postgres pk error msg hunting
* postgres pk error msg hunting
* postgres
* err might be nil?
* fixed error logging bug
* hopefully solved postgres issue
* using rebind
* moved tests to _test package
* fixing test dirs
* finally made sense of Egons package name feedback
* UTC, array fixes
2019-02-01 13:50:12 -05:00
Bill Thorp
60946c2024
make bandwidth agreements sensible: without []byte's ( #1152 )
...
removed []byte's from bandwidth agreement protocol buffers
2019-01-28 14:45:25 -05:00
Egon Elbre
49dacb662c
Improve bwagreement tests ( #1149 )
2019-01-26 01:06:38 +02:00
Bill Thorp
8ba4b61e59
Make Bandwidth Agreements Secure / Trustable ( #1117 )
...
* Added cert chains and nodeid verification to bandwidth agreement
2019-01-25 13:05:21 -05:00
Cameron
f5b1e77cf2
rollup timestamp fix ( #1132 )
...
* create timestamp if isNew
* pass isNil into SaveRollup
2019-01-24 15:41:22 -05:00
Jennifer Li Johnson
856b98997c
updates copyright 2018 to 2019 ( #1133 )
2019-01-24 15:15:10 -05:00
Jennifer Li Johnson
030d294399
adds logs in rollup and tally run ( #1131 )
2019-01-24 15:05:53 -05:00
Jennifer Li Johnson
70940b8710
Making sure payments works ( #1122 )
...
* byte hours
* updates comment
* removes payment test data methods
* wip
* adds tally and rollup to peer
* remove sim test for payment (for now)
* lint error
2019-01-24 13:44:14 -05:00
Egon Elbre
5de7f8af7f
Satellite Peer ( #1119 )
2019-01-23 21:58:44 +02:00
Michal Niewrzal
eb2d858ffc
Use pointerdb.Service instead of pointerdb.Server ( #1099 )
2019-01-19 13:58:53 -05:00
Cameron
ecd704692e
wip testing and timestamp fix ( #1100 )
...
* wip testing and timestamp fix
* fixed import cycle
2019-01-18 11:53:23 -05:00
Michal Niewrzal
cc1bdef8b7
Move pointerdb logic to a service ( #1037 )
...
* test
* Move pointerdb logic to service
* tset
* reorder constructor params
* restore field
2019-01-18 10:10:21 -05:00
Jennifer Li Johnson
c4b876ddc5
Jj/bytes ( #1085 )
...
* run tally and rollup
* sets dev default tally and rollup intervals
2019-01-17 14:39:32 -05:00
Jennifer Li Johnson
e6fbf63620
Generate Payments Report ( #1079 )
2019-01-17 13:34:13 -05:00
Bill Thorp
342dc857f5
rollup query ( #1056 )
...
* implemention notes
* more notes
* starting rollup query
* not working yet
* fixed build
* fixed cfg bug
* change context cancelled errs to debugs
* using byte hours for at rest tally
* revert changes to go.mod
* comment fixes
* prevent double recording tallies in rollup
* linting
* stop leaking dbx
* nodeid changes
* fix build
2019-01-16 14:30:33 -05:00
littleskunk
2aebbeed34
add more bwagreement unit tests ( #1046 )
...
* add more bwagreement unit tests
2019-01-15 05:44:56 +01:00
Michal Niewrzal
e13f7c7a7d
Fix unstarted pointerdb responsibility ( #1033 )
2019-01-11 13:45:11 +01:00
Cameron
ef50bbf8b6
satellites reject expired BWAs ( #1015 )
...
* add 45 day expiration to PBAs
* add expiration field to relevant areas, DeleteExpired placeholder
* reject expired BWAs
* test for expired BWAs
* add BwExpiration config value
2019-01-10 13:30:55 -05:00
Bill Thorp
fb4a11ebb0
Differentiate GET and PUT operations in the tally service ( #965 )
...
* draft of new bandwidth agreement types
* updated storagenode report for new types
* use correct pba types
2019-01-10 06:41:57 -05:00
aligeti
457f909a74
Satellite accepts manipulated paychecks V3-1023 ( #972 )
...
* fix - Satellite crashing on receiving a manipulated bandwidthagreement
* provider.PeerIdentityFromContext called twice. Remove one
* add storage node ID to serial number
* remove serialNum query and transaction
* add uuid to GeneratePayerBandwidthAllocation for testing
* enable expected failure on duplicate serialnum cases
* Revert "enable expected failure on duplicate serialnum cases"
This reverts commit 5948f43ed1741c280f0bb34a86c1c490365417bc.
* enable expected failure on duplicate serialnum cases
2019-01-09 10:02:03 -05:00
littleskunk
9a536f2cc4
Test bwagreement serial number ( #978 )
...
* bwagreement serial number tests
2019-01-07 16:19:05 +01:00
Jennifer Li Johnson
4e3e1be1a0
returns nil if no nodeData present in CalculateAtRestRaw ( #976 )
2019-01-04 14:54:54 -05:00
aligeti
ce1db97ce6
fix manipulated paychecks V3-1023 ( #961 )
...
Satellite crashing on receiving a manipulated bandwidthagreement V3-1022
2019-01-04 11:26:26 -05:00
Jennifer Li Johnson
a2fa5c4c5a
Proper NodeType Handling ( #873 )
...
* adds enums to nodetype
* updating nodetype todos
* ran pb updates
* reorder nodetypes
* adding checks
* wip
* wip
* wip
* bug in test-captplanet
* wip
* add values to storagenode, satellite, captplanet binaries
* Cleanup
* more cleanup
* wip
* lint
* lint
* wip
* fixes bug
* regenerate protos
Change-Id: Id270212e8c7479e52641058042cf23b5317ab773
* limit node type changes to kademlia
Change-Id: I9c1a6cc4a79e05086627f0fdeb5028c62ce754f4
* dpanic
Change-Id: Id952a2ad13c807ebaea0ec0a875405e267d81c3e
* review comments
Change-Id: I7f9b77ef22779dd012fd490375b136014f51f834
2019-01-02 11:47:34 -07:00
Egon Elbre
0ca03b41e2
Use generated locking database implementation ( #947 )
2019-01-02 19:53:27 +02:00
Bryan White
249244536a
CSR Service (part 2): cert signing rpc ( #950 )
...
* CSR Service:
+ implement certificate sign rpc method
+ implement certificate signer client/server
+ refactor `AuthorizationDB#Create`
+ refactor `NewTestIdentity`
+ add `AuthorizationDB#Claim`
+ add `Token#Equal`
+ fix `Authorizations#Marshal` when marshaling identities and certificates
+ tweak `Authorization#String` format
+ cert debugging improvements (jsondiff)
+ receive context arg in `NewTestIdentity`
+ misc. fixes
2019-01-02 12:39:17 -05:00
Egon Elbre
4346cd060f
Implement mutex around satellitedb ( #932 )
2018-12-27 11:56:25 +02:00
Jennifer Li Johnson
01df85704c
Updates accounting tables and allows tally to update at raw table for at rest data ( #874 )
2018-12-18 12:18:42 -05:00
Natalie Villasana
d97e27ed80
removes a few todos ( #867 )
2018-12-17 13:11:29 -05:00
Bill Thorp
f9845e7e92
changes to masterdb to support accountingDB ( #846 )
...
* added rollup to captplanet, moved accountingDB to masterdb
2018-12-14 09:27:21 -05:00
Jennifer Li Johnson
6642f97142
Jj/accounting updates ( #820 )
...
* tallies up data stored on each node in pointerdb
* adds comments for data type enums
* changes Open to BeginTx because Go convention
* removes online status check from identify active nodes
* changes identifyactivenodes to calculatestaticdata
* updates accounting dbx names
2018-12-12 16:24:08 -05:00
Bill Thorp
ba5f71810e
added tally to capt run.go ( #751 )
...
* added tally to capt run.go, ParseURL -> SplitDBURL
2018-12-12 08:15:34 -05:00
Michal Niewrzal
7e2f66ee24
Small fixes around master db ( #822 )
2018-12-11 10:30:09 +01:00
JT Olio
1c96db01ba
better database error handling ( #784 )
...
* better database error handling
Change-Id: I28dbd69cf6c2fa268e02405521ff6e6c1a68a702
* missing comments added
* missing comment added
2018-12-07 07:46:42 -07:00
Michal Niewrzal
d62896bbaf
Use testcontext with tally tests ( #796 )
2018-12-07 12:55:25 +01:00
Michal Niewrzal
034a34069a
Merge bwagreement db into satellite master db ( #770 )
...
* Merge bwagreement db into satellite master db
* adjust to recent tally changes
* linter problems
* linter problems
* returning db structs in more optimal way
* added pointer for assignment
* error message changed
* better param message
2018-12-07 10:59:31 +01:00
Bill Thorp
b2a8a10b16
redo tally query ( #742 )
...
* redo tally query
* broken, sharing w/ Jen
* tally tests added, failiing
* fix txn issue in tally
* fix lint
2018-12-05 09:03:23 -05:00
Egon Elbre
9ac3517432
Reduce noise in logging ( #733 )
2018-11-29 22:59:26 +02:00
Bryan White
2a0c4e60d2
preparing for use of customtype gogo extension with NodeID type ( #693 )
...
* preparing for use of `customtype` gogo extension with `NodeID` type
* review changes
* preparing for use of `customtype` gogo extension with `NodeID` type
* review changes
* wip
* tests passing
* wip fixing tests
* more wip test fixing
* remove NodeIDList from proto files
* linter fixes
* linter fixes
* linter/review fixes
* more freaking linter fixes
* omg just kill me - linterrrrrrrr
* travis linter, i will muder you and your family in your sleep
* goimports everything - burn in hell travis
* goimports update
* go mod tidy
2018-11-29 19:39:27 +01:00
Egon Elbre
0ae05cf834
Ensure that we only choose storage nodes ( #732 )
2018-11-29 16:57:00 +02:00
Bill Thorp
b8b0d75cc6
made accounting use migrate ( #688 )
...
* made accounting use migrate
2018-11-26 16:49:55 -05:00
Bryan White
54ccb460a9
rename size fields in protobuf messages for gogo compatibility ( #690 )
...
* rename `size` fields in protobuf messages for gogo compatibility
* linter fixes
2018-11-20 18:09:35 +01:00
Jennifer Li Johnson
93c5f385a8
Enable checker in captplanet and staging ( #643 )
...
* enable checker
* add option to use mock overlay in checker
* adds logs to checker
* appease linter
2018-11-20 10:54:22 -05:00
Jennifer Li Johnson
377832c705
Create Tally and Rollup pkgs in accounting ( #642 )
...
* creates separate tally and rollup packages and writes skeleton for rollup
* TODO add rollupDB and rawDB to rollup struct
* TODO add rawDB to tally struct
2018-11-13 20:22:18 -05:00
Jennifer Li Johnson
ad4df839a8
Data at rest accounting database ( #601 )
...
* starts at rest storage db
* generates aggregate and granular models with dbx
* reverts gomod
* reverts gosum
2018-11-09 15:02:28 -05:00
Jennifer Li Johnson
e678e52229
Creates Accounting Pkg to tally at rest node storage ( #568 )
...
* creates accounting package with tally service
* adds cancel on context
* test online nodes
2018-11-08 11:18:28 -05:00
