Commit Graph

13 Commits

Author SHA1 Message Date
ethanadams
8f8b13abb9
Re-enable SN bandwidth rollups. Fix SN bandwidth rollup unique constraint issue. Re-organize service code (#2617)
* re-organizing into bandwidth service. re-enable rollup loop
* Prevent uniqueness failure in bandwidth rollup
* Add test to make sure the rollup select date range works correctly
* add bandwidth config for rollup interval
2019-07-29 10:07:52 -04:00
Jeff Wendling
89afe3ee37
remove struct to ensure 64bit alignment for atomics (#2578)
Change-Id: Id2a4b740b2486a844673f69ce2e54c8c1e3187e2
2019-07-16 14:53:58 -06:00
ethanadams
d044613679
SN DB Optimization: Add rollups to bandwidth usage (#2541)
* V3-2119: Add storagenode bandwidth usage rollup
2019-07-16 10:58:58 -04:00
Jeff Wendling
b9d8ddaad1
storagenode: remove datetime calls in favor of UTC (#2557)
* storagenode: remove datetime calls in favor of UTC

datetime only has second level granularity whereas string
comparisons don't. Since we're wiping everything anyway, it's
easier to just use UTC everywhere rather than migrate to
datetime calls.

* add utcdb to check that arguments are utc

* storagenodedb: add trivial tests to ensure calls work

This at least tests that all of the timestamps passed in are
in the UTC timezone.

* fix truncated comment and change migrations to be UTC
2019-07-15 13:38:08 -04:00
Jeff Wendling
a2418b22af storagenodedb: optimize index usage and queries (#2545)
- Drops some unused indexes
- Applies a computed index to timestamp columns
- Applies a partial index for expired pieces
- Uses BETWEEN to avoid some datetime calls
- Filters expired piece search by those that aren't NULL
2019-07-12 15:29:09 -04:00
ethanadams
f06aec06fb
Move int64s to top of struct to resolve alignment issue on ARM (#2521)
* move int64s to top of struct to resolve alignment issue on ARM
2019-07-10 13:47:22 -04:00
Jeff Wendling
7886a4d7b9 storagenodedb: use datetime functions in sqlite queries (#2512)
This way comparison happens on the actual time rather than the
string representation of the time which may change depending on
the time zone.
2019-07-10 10:47:59 -04:00
ethanadams
4faed2098d
method name changer per PR 2469 (#2494) 2019-07-09 16:05:14 -04:00
ethanadams
ff6f1d1b32
storagenode: add in-memory tracking for bandwidth and disk usage (#2469)
* Add in-memory cache for bandwidth and space usage monitoring

* moved some structs around and added error handling for get piece size query

* added to existing bandwidth test.  fixed typo

* added test, updates from PR review, added monkit for new methods

* PR review updates. renamed space used methods

* changed bw cache so that only Add updates the cache and it only overwrites when the date moves forward

* moved bandwidth usage to bw and space usage to pieceinfodb

* fixed interface comment

* removed pointer from sync.Once
2019-07-08 20:33:50 -04:00
ethanadams
47e4584fbe
V3-1989: Storage node database is locked for several minutes while submiting orders (#2410)
* remove infodb locks and give a unique name for each in memory created.

* changed max idle and open to 1 for memory DBs.  fixes table locking errors

* fixed race condition

* added file based infodb test

* added busy timeout parameter to the file based infodb for testing

* fixed imports

* removed db.locked() after merge from master
2019-07-02 17:23:02 -04:00
JT Olio
c4bb84f209 storagenode: add monkit task to missing places (#2107) 2019-06-04 14:31:38 +02:00
Egon Elbre
fba9a5f945 migration tests for storagenodedb infodb (#1628) 2019-04-02 09:54:09 +02:00
Egon Elbre
05d148aeb5
Storage node and upload/download protocol refactor (#1422)
refactor storage node server
refactor upload and download protocol
2019-03-18 12:55:06 +02:00