Egon Elbre
bb11d83ed0
Proper planet shutdown ( #1249 )
2019-02-06 15:19:14 +02:00
Michal Niewrzal
aa3836c06a
Add checker irreparable test ( #1246 )
...
* Add checker irreparable test
* use RepairUnixSec
* wait for second check
* use storagenode.ID()
* Update comments
2019-02-06 14:03:38 +01:00
Egon Elbre
fdbe2db273
Remove node package and simplify DHT interface ( #1233 )
2019-02-06 14:37:17 +02:00
Egon Elbre
ab83f0f077
Fix parallel execution sharing ( #1247 )
2019-02-06 11:16:05 +02:00
Egon Elbre
b5d86aa0b4
Make testplanet.Run and satellitedbtest.Run parallel by default ( #1231 )
2019-02-05 21:44:00 +02:00
Egon Elbre
07412698a9
Use lower bcrypt cost for console.Service when running tests and disable flaky tests ( #1236 )
2019-02-05 19:31:53 +02:00
Yaroslav Vorobiov
2ff0d9d435
Add bucket to project relationship on satellite ( #1143 )
...
* integrate console api keys with buckets in pointerdb
* fix test
* fix tests kvmetainfo
* linter fix
* disable account activation
* fix test
* review fixes
* fix comments
* little refactoring
* remove debug println
* fix typo
* disable activation in a propper way
* fix test
* fix imports
* fix uplink count in testplanet
* move key creation to planet.newUplink
2019-02-05 19:22:17 +02:00
paul cannon
1573cbf4c4
pass UTC times to db so that sqlite3 understands ( #1237 )
...
* pass UTC times to db so that sqlite3 understands
this fixes the pkg/accounting/tally tests, at least for me.
(credit to Bill Thorp)
* also fix this one
2019-02-05 11:33:14 -05:00
Egon Elbre
87d6410b50
Revert "Remove node package and simplify DHT interface."
...
This reverts commit 03ec1ff92d
2019-02-05 10:38:48 +02:00
Egon Elbre
03ec1ff92d
Remove node package and simplify DHT interface.
2019-02-05 10:37:24 +02:00
Egon Elbre
c37e0c1b6d
testplanet.Run ( #1222 )
2019-02-04 22:37:46 +02:00
Michal Niewrzal
383cb05e8e
Divide uplink and gateway params set ( #1026 )
...
* Divide uplink and gateway params set
* attempt to fix docker
* attempt to fix all in one
* test
* more reorganization
* fix compilation error
* fix imports order
* fix dependency
* rename structs
* keep minio params for now
* review comments
* remove manual flag check
2019-02-04 16:50:06 +01:00
Bill Thorp
0f662b8e38
sql based tally ( #1166 )
...
* WIP
* wacky changes
* more
* it builds.... IT BUILDS!!!
* fixed SQL, broke out saving tallies for testing
* shorter lines
* fixed SQL, moved tally_test to testplanet
* lint
* WIP logic error preventing PUT and GETs to same serialnum
* fixed BWA test
* fixed temporary brain failure
* eliminated magic numbers
* cleaned up satellite uplink stats
* use errs.Combine instead
* thrashing
* fixed tally erroneous error msg
* fixed tally test
* lint
* SQL syntax attempt to fix
* spelling error
* made bwa db test resist old postgres data
* postgres pk error msg hunting
* postgres pk error msg hunting
* postgres
* err might be nil?
* fixed error logging bug
* hopefully solved postgres issue
* using rebind
* moved tests to _test package
* fixing test dirs
* finally made sense of Egons package name feedback
* UTC, array fixes
2019-02-01 13:50:12 -05:00
Egon Elbre
5a63c00442
Fix issues with blocking during startup ( #1212 )
2019-02-01 19:28:40 +02:00
Yaroslav Vorobiov
ba07f337c0
Satellite api keys frontend ( #1039 )
...
* api keys frontend
* fix linter
* fixes, copy to clipboard, disable account activation
* activation is removed in https://github.com/storj/storj/pull/1143
2019-02-01 18:19:30 +02:00
Egon Elbre
5a0306ce0c
Fix typo - second edition ( #1206 )
2019-02-01 16:09:34 +02:00
Egon Elbre
b91d77436f
Test merging planets ( #1181 )
2019-02-01 15:32:28 +02:00
Egon Elbre
73ccd8411a
Fix select nodes equation ( #1200 )
2019-01-31 22:45:37 +02:00
Maximillian von Briesen
c46864449d
Order nodes by random ( #1197 )
2019-01-31 14:33:07 -05:00
Egon Elbre
ba5097c13d
Postgres schemas for testing ( #1194 )
...
* add schema creation code
2019-01-31 14:17:12 -05:00
Egon Elbre
0c366c1ed2
Clearer code for node selection ( #1173 )
2019-01-31 13:49:00 -05:00
Bogdan Artemenko
cec2e2d0d5
[V3-1071] Enhanced project members search to full name. ( #1192 )
2019-01-31 16:11:53 +02:00
Egon Elbre
5e27d6ec59
Console postgres tests ( #1188 )
2019-01-31 15:01:13 +02:00
Dylan Lott
5e1ab841cf
Cache refresher ( #1171 )
...
* got tests passed
* wire up paginate function for cache node retrieval
* Add tests for paginate but they're failing
* fix the test arguments
* Updates paginate function to return more variable
* Updates
* Some test and logic tweaks
* improves config handling in discovery
* adds refresh offset to discovery struct
2019-01-30 11:29:18 -05:00
Yehor Butko
19bc01c19a
V3-1091 Extend Users table with IsActive functionality ( #1170 )
...
* V3-1091 Extend Users table with IsActive functionality
* fixed review comments
2019-01-30 17:04:40 +02:00
Natalie Villasana
8d7944bcf8
new node selection in overlay ( #1136 )
2019-01-29 14:42:43 -05:00
Egon Elbre
e1a8bbdcb6
Kademlia flags cleanup ( #1137 )
2019-01-29 08:51:07 +02:00
Bill Thorp
e94f6408ff
bw agreement db refactor ( #1158 )
2019-01-28 16:16:21 -05:00
Bill Thorp
60946c2024
make bandwidth agreements sensible: without []byte's ( #1152 )
...
removed []byte's from bandwidth agreement protocol buffers
2019-01-28 14:45:25 -05:00
Egon Elbre
d50c07e56c
Implement WorkGroup ( #1151 )
2019-01-28 21:04:42 +02:00
Bogdan Artemenko
b75f06e229
Removed TermsAccepted in project from satellite console and web. ( #1155 )
...
* Removed TermsAccepted from satellite console and web.
2019-01-28 20:20:33 +02:00
Egon Elbre
cecd4b0816
Remove server aliases ( #1154 )
2019-01-28 17:04:53 +02:00
Egon Elbre
f7e18882c4
Kademlia closing race ( #1139 )
2019-01-26 00:33:20 +02:00
Bill Thorp
8ba4b61e59
Make Bandwidth Agreements Secure / Trustable ( #1117 )
...
* Added cert chains and nodeid verification to bandwidth agreement
2019-01-25 13:05:21 -05:00
Jennifer Li Johnson
5397efe5c1
removes standalone payments service ( #1134 )
...
* remove standalone payments service - will integrate within satellite directly
* regenerate proto files
2019-01-25 11:23:25 -05:00
Cameron
f5b1e77cf2
rollup timestamp fix ( #1132 )
...
* create timestamp if isNew
* pass isNil into SaveRollup
2019-01-24 15:41:22 -05:00
Egon Elbre
187e9b2138
Code consistency between peers ( #1126 )
2019-01-24 22:28:06 +02:00
Jennifer Li Johnson
856b98997c
updates copyright 2018 to 2019 ( #1133 )
2019-01-24 15:15:10 -05:00
Jennifer Li Johnson
b424345edf
Jj/audit ( #1130 )
2019-01-24 14:52:16 -05:00
Jennifer Li Johnson
70940b8710
Making sure payments works ( #1122 )
...
* byte hours
* updates comment
* removes payment test data methods
* wip
* adds tally and rollup to peer
* remove sim test for payment (for now)
* lint error
2019-01-24 13:44:14 -05:00
Yaroslav Vorobiov
f437effaef
Add console to satellite peer ( #1120 )
2019-01-24 18:26:36 +02:00
Egon Elbre
5de7f8af7f
Satellite Peer ( #1119 )
2019-01-23 21:58:44 +02:00
Egon Elbre
99d3b7a3c8
Fix import grouping ( #1111 )
2019-01-22 17:48:23 +02:00
Michal Niewrzal
eb2d858ffc
Use pointerdb.Service instead of pointerdb.Server ( #1099 )
2019-01-19 13:58:53 -05:00
Cameron
ecd704692e
wip testing and timestamp fix ( #1100 )
...
* wip testing and timestamp fix
* fixed import cycle
2019-01-18 11:53:23 -05:00
Michal Niewrzal
cc1bdef8b7
Move pointerdb logic to a service ( #1037 )
...
* test
* Move pointerdb logic to service
* tset
* reorder constructor params
* restore field
2019-01-18 10:10:21 -05:00
Egon Elbre
78dc02b758
Satellite Peer ( #1034 )
...
* add satellite peer
* Add overlay
* reorganize kademlia
* add RunRefresh
* add refresh to storagenode.Peer
* add discovery
* add agreements and metainfo
* rename
* add datarepair checker
* add repair
* add todo notes for audit
* add testing interface
* add into testplanet
* fixes
* fix compilation errors
* fix compilation errors
* make testplanet run
* remove audit refrences
* ensure that audit tests run
* dev
* checker tests compilable
* fix discovery
* fix compilation
* fix
* fix
* dev
* fix
* disable auth
* fixes
* revert go.mod/sum
* fix linter errors
* fix
* fix copyright
* Add address param for SN dashboard (#1076 )
* Rename storj-sdk to storj-sim (#1078 )
* Storagenode logs and config improvements (#1075 )
* Add more info to SN logs
* remove config-dir from user config
* add output where config was stored
* add message for successful connection
* fix linter
* remove storage.path from user config
* resolve config path
* move success message to info
* log improvements
* Remove captplanet (#1070 )
* pkg/server: include production cert (#1082 )
Change-Id: Ie8e6fe78550be83c3bd797db7a1e58d37c684792
* Generate Payments Report (#1079 )
* memory.Size: autoformat sizes based on value entropy (#1081 )
* Jj/bytes (#1085 )
* run tally and rollup
* sets dev default tally and rollup intervals
* nonessential storj-sim edits (#1086 )
* Closing context doesn't stop storage node (#1084 )
* Print when cancelled
* Close properly
* Don't log nil
* Don't print error when closing dashboard
* Fix panic in inspector if ping fails (#1088 )
* Consolidate identity management to identity cli commands (#1083 )
* Consolidate identity management:
Move identity cretaion/signing out of storagenode setup command.
* fixes
* linters
* Consolidate identity management:
Move identity cretaion/signing out of storagenode setup command.
* fixes
* sava backups before saving signed certs
* add "-prebuilt-test-cmds" test flag
* linters
* prepare cli tests for travis
* linter fixes
* more fixes
* linter gods
* sp/sdk/sim
* remove ca.difficulty
* remove unused difficulty
* return setup to its rightful place
* wip travis
* Revert "wip travis"
This reverts commit 56834849dcf066d3cc0a4f139033fc3f6d7188ca.
* typo in travis.yaml
* remove tests
* remove more
* make it only create one identity at a time for consistency
* add config-dir for consitency
* add identity creation to storj-sim
* add flags
* simplify
* fix nolint and compile
* prevent overwrite and pass difficulty, concurrency, and parent creds
* goimports
2019-01-18 08:54:08 -05:00
Jennifer Li Johnson
e6fbf63620
Generate Payments Report ( #1079 )
2019-01-17 13:34:13 -05:00
Yaroslav Vorobiov
d832789481
ConsoleDB merge and buckets table creation ( #1066 )
...
* init
* db merge
* bucket table renamed to bucket info
* remove id and fix comments
* fix imports
2019-01-16 15:23:28 -05:00
Bill Thorp
342dc857f5
rollup query ( #1056 )
...
* implemention notes
* more notes
* starting rollup query
* not working yet
* fixed build
* fixed cfg bug
* change context cancelled errs to debugs
* using byte hours for at rest tally
* revert changes to go.mod
* comment fixes
* prevent double recording tallies in rollup
* linting
* stop leaking dbx
* nodeid changes
* fix build
2019-01-16 14:30:33 -05:00
Egon Elbre
f8906ce000
Change overlay cache database interface ( #1047 )
2019-01-15 11:08:45 -05:00
Yehor Butko
e38cf8f50d
Renaming and moving pkg/satellite to satellite/console ( #1054 )
...
* [WIP] V3-853 Merge the satellite DB into the master database
* Removing consoleDB from satelliteDB
* Fixing tests for satellite/console
* fixing linter
* sorting imports in satellite/console
* fixing console config
* fixing linter
2019-01-15 15:03:24 +02:00
Egon Elbre
12eec57abf
Add arguments to lockedgen for using it in other places ( #1030 )
2019-01-11 18:07:26 +02:00
Cameron
ef50bbf8b6
satellites reject expired BWAs ( #1015 )
...
* add 45 day expiration to PBAs
* add expiration field to relevant areas, DeleteExpired placeholder
* reject expired BWAs
* test for expired BWAs
* add BwExpiration config value
2019-01-10 13:30:55 -05:00
Bill Thorp
fb4a11ebb0
Differentiate GET and PUT operations in the tally service ( #965 )
...
* draft of new bandwidth agreement types
* updated storagenode report for new types
* use correct pba types
2019-01-10 06:41:57 -05:00
aligeti
457f909a74
Satellite accepts manipulated paychecks V3-1023 ( #972 )
...
* fix - Satellite crashing on receiving a manipulated bandwidthagreement
* provider.PeerIdentityFromContext called twice. Remove one
* add storage node ID to serial number
* remove serialNum query and transaction
* add uuid to GeneratePayerBandwidthAllocation for testing
* enable expected failure on duplicate serialnum cases
* Revert "enable expected failure on duplicate serialnum cases"
This reverts commit 5948f43ed1741c280f0bb34a86c1c490365417bc.
* enable expected failure on duplicate serialnum cases
2019-01-09 10:02:03 -05:00
Egon Elbre
0ca03b41e2
Use generated locking database implementation ( #947 )
2019-01-02 19:53:27 +02:00
Michal Niewrzal
b712fbcbb0
Fix 'empty queue' error when satellite starts ( #939 )
2019-01-02 17:00:32 +01:00
JT Olio
4a0749e44a
Remove farmer references ( #949 )
2019-01-02 12:31:49 +02:00
Michal Niewrzal
8c00b2109e
Format satellite.dbx ( #930 )
...
* Format satellite.dbx
* remove empty lines
2018-12-27 22:21:50 +01:00
Egon Elbre
4346cd060f
Implement mutex around satellitedb ( #932 )
2018-12-27 11:56:25 +02:00
Cameron
f70b826fd4
repair queue masterDB support ( #865 )
...
* add injuredsegment model to satellitedb.dbx
* add context to queue.RepairQueue interface
* use queue.RepairQueue interface, use masterdb
2018-12-21 10:11:19 -05:00
Maximillian von Briesen
1aa8bb03ef
Statdb transactions ( #911 )
...
add sql transactions to statdb
update statdb errors to use satellitedb error class
2018-12-19 17:34:20 -05:00
Maximillian von Briesen
f18c38628e
Refactor statdb ( #887 )
...
Update statdb args/return values to minimize structs
Simplify statdb.Update() to update all stats instead of an arbitrary subset determined by flags
Remove CreateIfNotExists logic from statdb.Update()
Simplify audit code structure
2018-12-19 13:44:03 -05:00
Michal Niewrzal
bbe1dbb92c
Fix overlay commands after master db merge ( #895 )
...
* Fix overlay commands after master db merge
* remove unused var
* remove unused comment
2018-12-18 21:41:31 +01:00
Jennifer Li Johnson
01df85704c
Updates accounting tables and allows tally to update at raw table for at rest data ( #874 )
2018-12-18 12:18:42 -05:00
Michal Niewrzal
a819d819e3
Overlay Cache master db merge ( #877 )
...
* Overlay Cache master db merge
* test update
* review comments
* Fix transaction usage
* removed unused variable
* added better limit handling
* better error handling
2018-12-17 21:14:16 +01:00
Michal Niewrzal
de4ab85ebe
Fix irreparable db transaction usage ( #888 )
2018-12-17 19:54:46 +01:00
aligeti
5e1b02ca8b
Statdb master db v3 848 ( #830 )
...
* intial changes to migrate statdb to masterdb framework
* statdb refactor compiles
* added TestCreateDoesNotExist testcase
* Initial port of statdb to masterdb framework working
* refactored statdb proto def to pkg/statdb
* removed statdb/proto folder
* moved pb.Node to storj.NodeID
* CreateEntryIfNotExistsRequest moved pd.Node to storj.NodeID
* moved the fields from pb.Node to statdb.UpdateRequest
ported TestUpdateExists, TestUpdateUptimeExists, TestUpdateAuditSuccessExists TestUpdateBatchExists
2018-12-14 15:17:30 -05:00
Bill Thorp
f9845e7e92
changes to masterdb to support accountingDB ( #846 )
...
* added rollup to captplanet, moved accountingDB to masterdb
2018-12-14 09:27:21 -05:00
Bill Thorp
ba5f71810e
added tally to capt run.go ( #751 )
...
* added tally to capt run.go, ParseURL -> SplitDBURL
2018-12-12 08:15:34 -05:00
Michal Niewrzal
7e2f66ee24
Small fixes around master db ( #822 )
2018-12-11 10:30:09 +01:00
aligeti
d27bcddaab
Merge the irreparable DB into the master database ( #777 )
...
* initial refactoring to merge irreparabledb to master db
* refactor
* code review comment updates
2018-12-10 14:08:45 -05:00
Kevin Leffew
4a128e65d9
Removed unnecessary comment ( #804 )
2018-12-10 13:54:52 -05:00
Michal Niewrzal
a15099d5f5
Simplify testing with all supported DBs ( #809 )
...
* Simplify testing with all supported DBs
* move method to separate package
* method renamed to Run
* fixed comment
2018-12-10 15:50:12 +01:00
Egon Elbre
c7cf97cc23
Fix sqlite registration ( #806 )
2018-12-10 10:32:49 +02:00
JT Olio
1c96db01ba
better database error handling ( #784 )
...
* better database error handling
Change-Id: I28dbd69cf6c2fa268e02405521ff6e6c1a68a702
* missing comments added
* missing comment added
2018-12-07 07:46:42 -07:00
Michal Niewrzal
034a34069a
Merge bwagreement db into satellite master db ( #770 )
...
* Merge bwagreement db into satellite master db
* adjust to recent tally changes
* linter problems
* linter problems
* returning db structs in more optimal way
* added pointer for assignment
* error message changed
* better param message
2018-12-07 10:59:31 +01:00
Michal Niewrzal
44a6cfa1c1
Initial satellite master database setup ( #735 )
...
* Initial satellite master database setup
* fixed unsed ctx and /nolint added for temp change
* tests for two db drivers + connection param renamed
2018-12-05 10:35:50 +01:00
