Commit Graph

11 Commits

Author SHA1 Message Date
Egon Elbre
25b76fe63f storagenode/storagenodedb: use tagsql
Change-Id: Iba3b34a97b982deb4f72ce55517a294f249b6b55
2020-01-19 14:39:16 +02:00
Egon Elbre
81d53b8097 storagenode/storagenodedb: fixes to row handling
Change-Id: I3813310b48337428f13678a9fcba5c8a0e0b2b2a
2020-01-16 15:08:37 +00:00
Egon Elbre
64fb2d3d2f Revert "dbutil: statically require all databases accesses to use contexts"
This reverts commit 8e242cd012.

Revert because lib/pq has known issues with context cancellation.
These issues need to be resolved before these changes can be merged.

Change-Id: I160af51dbc2d67c5449aafa406a403e5367bb555
2020-01-15 07:28:00 +00:00
JT Olio
8e242cd012 dbutil: statically require all databases accesses to use contexts
this will allow for some nice runtime analysis down the road.
also, this allows for wrapping database handles in a way that
can interact with these contexts

requires https://review.dev.storj.io/c/storj/dbx/+/514

Change-Id: Ib087b7cd73296dd2c1e0331314da34d861f61d2b
2020-01-14 18:20:47 -05:00
Egon Elbre
6615ecc9b6 common: separate repository
Change-Id: Ibb89c42060450e3839481a7e495bbe3ad940610a
2019-12-27 14:11:15 +02:00
Jeff Wendling
fb8e78132d storagenodedb: reenable utccheck in tests
Change-Id: If7d64dd4ae58e4b656ff9122ae3195b2a5173cb3
2019-12-10 23:17:14 +00:00
Bill Thorp
89c59d06f9
storagenode/storagenodedb: add SQL receiver logic for graceful exit (#3067)
* added graceful exit db methods
2019-10-01 10:34:03 -04:00
Simon Guindon
a2b1e9fa95
storagenode/storagenodedb: refactor both data access objects and migrations to support multiple DB connections (#3057)
* Split the info.db database into multiple DBs using Backup API.

* Remove location. Prev refactor assumed we would need this but don't.

* Added VACUUM to reclaim space after splitting storage node databases.

* Added unique names to SQLite3 connection hooks to fix testplanet.

* Moving DB closing to the migration step.

* Removing the closing of the versions DB. It's already getting closed.

* Swapping the database connection references on reconnect.

* Moved sqlite closing logic away from the boltdb closing logic.

* Moved sqlite closing logic away from the boltdb closing logic.

* Remove certificate and vouchers from DB split migration.

* Removed vouchers and bumped up the migration version.

* Use same constructor in tests for storage node databases.

* Use same constructor in tests for storage node databases.

* Adding method to access underlining SQL database connections and cleanup

* Adding logging for migration diagnostics.

* Moved migration closing database logic to minimize disk usage.

* Cleaning up error handling.

* Fix missing copyright.

* Fix linting error.

* Add test for migration 21 (#3012)

* Refactoring migration code into a nicer to use object.

* Refactoring migration code into a nicer to use object.

* Fixing broken migration test.

* Removed unnecessary code that is no longer needed now that we close DBs.

* Removed unnecessary code that is no longer needed now that we close DBs.

* Fixed bug where an invalid database path was being opened.

* Fixed linting errors.

* Renamed VersionsDB to LegacyInfoDB and refactored DB lookup keys.

* Renamed VersionsDB to LegacyInfoDB and refactored DB lookup keys.

* Fix migration test. NOTE: This change does not address new tables satellites and satellite_exit_progress

* Removing v22 migration to move into it's own PR.

* Removing v22 migration to move into it's own PR.

* Refactored schema, rebind and configure functions to be re-useable.

* Renamed LegacyInfoDB to DeprecatedInfoDB.

* Cleaned up closeDatabase function.

* Renamed storageNodeSQLDB to migratableDB.

* Switched from using errs.Combine() to errs.Group in closeDatabases func.

* Removed constructors from storage node data access objects.

* Reformatted usage of const.

* Fixed broken test snapshots.

* Fixed linting error.
2019-09-18 12:17:28 -04:00
Yaroslav Vorobiov
f7403f97b0
storagenode/storageusage: add summary, rename timestamp to interval_start (#2911) 2019-09-04 17:13:43 +03:00
Simon Guindon
476fbf919a
storagenode/storagenodedb: refactor SQLite3 database connection initialization. (#2732)
* Rebasing changes against master.

* Added back withTx().

* Fix using new error type.

* Moving back database initialization back into the struct.

* Fix failing migration tests.

* Fix linting errors.

* Renamed database object names to be consistent.

* Fixing linting error in imports.

* Rebasing changes against master.

* Added back withTx().

* Fix using new error type.

* Moving back database initialization back into the struct.

* Fix failing migration tests.

* Fix linting errors.

* Renamed database object names to be consistent.

* Fixing linting error in imports.

* Adding missing change from merge.

* Fix error name.
2019-08-21 10:32:25 -04:00
Yaroslav Vorobiov
28a7778e9e
storagenode/nodestats: cache node stats (#2543) 2019-08-08 16:47:04 +03:00