storj/satellite/satellitedb
paul cannon 294d253923 satellite/payments: chore to migrate big.Float values out of db
All code on known satellites at this moment in time should know how to
populate and use the new numeric columns on the
stripecoinpayments_tx_conversion_rates and coinpayments_transactions
tables in the satellite db. However, there are still gob-encoded
big.Float values in the database from before these columns existed. To
get rid of those values, so that we can excise the gob-decoding code
from the relevant sections, however, we need something to read the gob
bytestrings and convert them to numeric values, a few at a time, until
they're all gone.

To accomplish that, this change adds two chores to be run in the
satellite core process- one for the coinpayments_transactions table, and
one for the stripecoinpayments_tx_conversion_rates table. They should
run relatively infrequently, so that we do not impose any undue load on
processing resources or the db.

Both of these chores work without using explicit sql transactions, but
should still be concurrent-safe, since they work by way of
compare-and-swap type operations.

If the satellite core process needs to be restarted, both of these
chores will start scanning for migrateable rows from the beginning of
the id space again. This is not ideal, but shouldn't be a problem (as
far as I can tell, there are only a few thousand rows at most in either
of these tables on any production satellite).

Change-Id: I733b7cd96760d506a1cf52735f598c6c3aa19735
2022-02-16 23:48:30 +00:00
..
dbx satellite/satellitedb: phase 2 getting rid of gob encodings in db 2022-02-09 12:25:46 +00:00
satellitedbtest satellite/metabase: limit maximum number of parts and size 2021-10-22 10:16:37 +00:00
testdata satellite/satellitedb: phase 2 getting rid of gob encodings in db 2022-02-09 12:25:46 +00:00
apikeys.go satellite/satellitedb: select user_agent in apikeys method GetPagedByProjectID 2022-01-03 11:58:26 -05:00
attribution.go satellite/{satellitedb,attribution,console}: value attribution changes that add userAgent field to buckets table and all tables that have partner_id 2021-10-18 13:56:19 +00:00
audithistory.go satellite/satellitedb: remove referrence to audit information in 2021-08-13 21:11:28 +00:00
bucketsdb.go satellite/metainfo: propagate geofencing between buckets and stream id 2021-11-24 08:05:05 +00:00
coinpaymentsmigration_test.go satellite/payments: chore to migrate big.Float values out of db 2022-02-16 23:48:30 +00:00
coinpaymentsmigration.go satellite/payments: chore to migrate big.Float values out of db 2022-02-16 23:48:30 +00:00
coinpaymentstxs.go satellite/satellitedb: phase 2 getting rid of gob encodings in db 2022-02-09 12:25:46 +00:00
compensation.go satellite/compensation: smaller txns for RecordPeriod 2021-03-02 20:14:25 +00:00
consoledb_test.go satellite/satellitedb/satellitedbtest: pass ctx as an argument 2020-01-20 16:35:42 +02:00
consoledb.go satellite/oidc: move oidc into common package 2022-02-08 09:46:54 -06:00
containment.go satellite/satellitedb: remove references to contained column in nodes table 2021-10-14 19:17:46 +00:00
customers.go satellite/billing: make stripe invoice generation work with multiregion satellites 2021-05-06 11:51:03 -06:00
database.go satellite/oidc: move oidc into common package 2022-02-08 09:46:54 -06:00
gracefulexit.go satellite/gracefulexit: drop unused column in graceful_exit_progress 2021-10-22 16:41:27 +00:00
invoiceprojectrecords.go satellite: use segment count for billing 2021-10-27 19:21:50 +00:00
migrate_gen.go private/dbutil: use dbutil and tagsql from storj.io/private 2021-04-23 14:36:52 +03:00
migrate_test.go satellite/satellitedb: drop contained from nodes and reputations 2022-01-20 19:46:48 +00:00
migrate.go satellite/satellitedb: phase 2 getting rid of gob encodings in db 2022-02-09 12:25:46 +00:00
migratez.go satellite/satellitedb: phase 2 getting rid of gob encodings in db 2022-02-09 12:25:46 +00:00
nodeapiversion.go ci: fix lint errors 2021-05-17 13:37:31 +00:00
nodeselection.go satellite/satellitedb: simplify select nodes query construction 2021-10-22 07:41:07 +00:00
orders.go satellite/{accounting,orders,satellitedb}: group bucket bandwidth rollups by time window 2022-01-05 20:24:48 +00:00
overlaycache.go satellite/{audit, reputation}: fix potential nodes reputation status 2022-01-06 21:05:59 +00:00
payouts.go satellite/compensation: smaller txns for RecordPeriod 2021-03-02 20:14:25 +00:00
peeridentities.go all: fix dots 2020-07-16 14:58:28 +00:00
projectaccounting_test.go satellitedb/projectaccounting, web/satellite: reworked bandwidth chart to show both allocated and settled bandwidth 2022-02-08 22:07:59 +00:00
projectaccounting.go satellitedb/projectaccounting, web/satellite: reworked bandwidth chart to show both allocated and settled bandwidth 2022-02-08 22:07:59 +00:00
projectmembers_test.go web/satellite: project members sorting fixed (#3231) 2019-10-15 15:24:53 +03:00
projectmembers.go all: fix linter complaints 2020-10-13 15:59:01 +03:00
projects_test.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
projects.go satellite/{console,satellitedb}: add project segment limit to user 2021-12-16 08:46:01 +00:00
regtokens.go all: fix dots 2020-07-16 14:58:28 +00:00
repairqueue.go satellite/repair: migrate to new repair_queue table 2021-06-30 17:12:24 +02:00
reputations.go satellite/{audit, reputation}: fix potential nodes reputation status 2022-01-06 21:05:59 +00:00
resetpasstokens.go all: fix dots 2020-07-16 14:58:28 +00:00
revocation.go private/lrucache: import from common 2021-08-16 10:04:32 +00:00
storagenodeaccounting.go satellite/{metabase,satellitedb}: deduplicate AS OF SYSTEM TIME code 2021-05-11 12:40:36 +03:00
stripecoinpaymentsdb.go satellite/{payments,satellitedb}: Remove custom coupon implementation 2021-10-11 19:47:00 +00:00
users_test.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
users.go satellite/{admin,payments,satellitedb}: add checks for deletion of free tier accounts 2022-02-08 10:11:31 +00:00