storj/satellite/satellitedb/coinpaymentstxs.go

142 lines
5.2 KiB
Go
Raw Normal View History

// Copyright (C) 2019 Storj Labs, Inc.
// See LICENSE for copying information.
package satellitedb
import (
"context"
"time"
"github.com/shopspring/decimal"
"github.com/zeebo/errs"
"storj.io/common/currency"
"storj.io/common/uuid"
"storj.io/storj/satellite/payments/coinpayments"
"storj.io/storj/satellite/payments/stripe"
"storj.io/storj/satellite/satellitedb/dbx"
)
// ensure that coinpaymentsTransactions implements stripecoinpayments.TransactionsDB.
var _ stripe.TransactionsDB = (*coinPaymentsTransactions)(nil)
// coinPaymentsTransactions is CoinPayments transactions DB.
//
// architecture: Database
type coinPaymentsTransactions struct {
satellite/satellitedb: unexport satellitedb.DB Backstory: I needed a better way to pass around information about the underlying driver and implementation to all the various db-using things in satellitedb (at least until some new "cockroach driver" support makes it to DBX). After hitting a few dead ends, I decided I wanted to have a type that could act like a *dbx.DB but which would also carry information about the implementation, etc. Then I could pass around that type to all the things in satellitedb that previously wanted *dbx.DB. But then I realized that *satellitedb.DB was, essentially, exactly that already. One thing that might have kept *satellitedb.DB from being directly usable was that embedding a *dbx.DB inside it would make a lot of dbx methods publicly available on a *satellitedb.DB instance that previously were nicely encapsulated and hidden. But after a quick look, I realized that _nothing_ outside of satellite/satellitedb even needs to use satellitedb.DB at all. It didn't even need to be exported, except for some trivially-replaceable code in migrate_postgres_test.go. And once I made it unexported, any concerns about exposing new methods on it were entirely moot. So I have here changed the exported *satellitedb.DB type into the unexported *satellitedb.satelliteDB type, and I have changed all the places here that wanted raw dbx.DB handles to use this new type instead. Now they can just take a gander at the implementation member on it and know all they need to know about the underlying database. This will make it possible for some other pending code here to differentiate between postgres and cockroach backends. Change-Id: I27af99f8ae23b50782333da5277b553b34634edc
2019-12-14 02:29:54 +00:00
db *satelliteDB
}
// GetLockedRate returns locked conversion rate for transaction or error if non exists.
func (db *coinPaymentsTransactions) GetLockedRate(ctx context.Context, id coinpayments.TransactionID) (rate decimal.Decimal, err error) {
defer mon.Task()(&ctx)(&err)
dbxRate, err := db.db.Get_StripecoinpaymentsTxConversionRate_By_TxId(ctx,
dbx.StripecoinpaymentsTxConversionRate_TxId(id.String()),
)
if err != nil {
return decimal.Decimal{}, err
}
rate = decimal.NewFromFloat(dbxRate.RateNumeric)
return rate, nil
satellite/satellitedb: prepare to remove big.Float from db Why: big.Float is not an ideal type for dealing with monetary amounts, because no matter how high the precision, some non-integer decimal values can not be represented exactly in base-2 floating point. Also, storing gob-encoded big.Float values in the database makes it very hard to use those values in meaningful queries, making it difficult to do any sort of analysis on billing. Now that we have amounts represented using monetary.Amount, we can simply store them in the database using integers (as given by the .BaseUnits() method on monetary.Amount). We should move toward storing the currency along with any monetary amount, wherever we are storing amounts, because satellites might want to deal with currencies other than STORJ and USD. Even better, it becomes much clearer what currency each monetary value is _supposed_ to be in (I had to dig through code to find that out for our current monetary columns). Deployment ---------- Getting rid of the big.Float columns will take multiple deployment steps. There does not seem to be any way to make the change in a way that lets existing queries continue to work on CockroachDB (it could be done with rules and triggers and a stored procedure that knows how to gob-decode big.Float objects, but CockroachDB doesn't have rules _or_ triggers _or_ stored procedures). Instead, in this first step, we make no changes to the database schema, but add code that knows how to deal with the planned changes to the schema when they are made in a future "step 2" deployment. All functions that deal with the coinbase_transactions table have been taught to recognize the "undefined column" error, and when it is seen, to call a separate "transition shim" function to accomplish the task. Once all the services are running this code, and the step 2 deployment makes breaking changes to the schema, any services that are still running and connected to the database will keep working correctly because of the fallback code included here. The step 2 deployment can be made without these transition shims included, because it will apply the database schema changes before any of its code runs. Step 1: No schema changes; just include code that recognizes the "undefined column" error when dealing with the coinbase_transactions or stripecoinpayments_tx_conversion_rates tables, and if found, assumes that the column changes from Step 2 have already been made. Step 2: In coinbase_transactions: * change the names of the 'amount' and 'received' columns to 'amount_gob' and 'received_gob' respectively * add new 'amount_numeric' and 'received_numeric' columns with INT8 type. In stripecoinpayments_tx_conversion_rates: * change the name of the 'rate' column to 'rate_gob' * add new 'rate_numeric' column with NUMERIC(8, 8) type Code reading from either of these tables must query both the X_gob and X_numeric columns. If X_numeric is not null, its value should be used; otherwise, the gob-encoded big.Float in X_gob should be used. A chore might be included in this step that transitions values from X_gob to X_numeric a few rows at a time. Step 3: Once all prod satellites have no values left in the _gob columns, we can drop those columns and add NOT NULL constraints to the _numeric columns. Change-Id: Id6db304b404e6fde44f5a8c23cdaeeaaa2324f20
2021-08-10 23:30:23 +01:00
}
// ListAccount returns all transaction for specific user.
func (db *coinPaymentsTransactions) ListAccount(ctx context.Context, userID uuid.UUID) (_ []stripe.Transaction, err error) {
defer mon.Task()(&ctx)(&err)
dbxTXs, err := db.db.All_CoinpaymentsTransaction_By_UserId_OrderBy_Desc_CreatedAt(ctx,
dbx.CoinpaymentsTransaction_UserId(userID[:]),
)
if err != nil {
return nil, err
}
txs, err := convertSlice(dbxTXs, fromDBXCoinpaymentsTransaction)
return txs, Error.Wrap(err)
satellite/satellitedb: prepare to remove big.Float from db Why: big.Float is not an ideal type for dealing with monetary amounts, because no matter how high the precision, some non-integer decimal values can not be represented exactly in base-2 floating point. Also, storing gob-encoded big.Float values in the database makes it very hard to use those values in meaningful queries, making it difficult to do any sort of analysis on billing. Now that we have amounts represented using monetary.Amount, we can simply store them in the database using integers (as given by the .BaseUnits() method on monetary.Amount). We should move toward storing the currency along with any monetary amount, wherever we are storing amounts, because satellites might want to deal with currencies other than STORJ and USD. Even better, it becomes much clearer what currency each monetary value is _supposed_ to be in (I had to dig through code to find that out for our current monetary columns). Deployment ---------- Getting rid of the big.Float columns will take multiple deployment steps. There does not seem to be any way to make the change in a way that lets existing queries continue to work on CockroachDB (it could be done with rules and triggers and a stored procedure that knows how to gob-decode big.Float objects, but CockroachDB doesn't have rules _or_ triggers _or_ stored procedures). Instead, in this first step, we make no changes to the database schema, but add code that knows how to deal with the planned changes to the schema when they are made in a future "step 2" deployment. All functions that deal with the coinbase_transactions table have been taught to recognize the "undefined column" error, and when it is seen, to call a separate "transition shim" function to accomplish the task. Once all the services are running this code, and the step 2 deployment makes breaking changes to the schema, any services that are still running and connected to the database will keep working correctly because of the fallback code included here. The step 2 deployment can be made without these transition shims included, because it will apply the database schema changes before any of its code runs. Step 1: No schema changes; just include code that recognizes the "undefined column" error when dealing with the coinbase_transactions or stripecoinpayments_tx_conversion_rates tables, and if found, assumes that the column changes from Step 2 have already been made. Step 2: In coinbase_transactions: * change the names of the 'amount' and 'received' columns to 'amount_gob' and 'received_gob' respectively * add new 'amount_numeric' and 'received_numeric' columns with INT8 type. In stripecoinpayments_tx_conversion_rates: * change the name of the 'rate' column to 'rate_gob' * add new 'rate_numeric' column with NUMERIC(8, 8) type Code reading from either of these tables must query both the X_gob and X_numeric columns. If X_numeric is not null, its value should be used; otherwise, the gob-encoded big.Float in X_gob should be used. A chore might be included in this step that transitions values from X_gob to X_numeric a few rows at a time. Step 3: Once all prod satellites have no values left in the _gob columns, we can drop those columns and add NOT NULL constraints to the _numeric columns. Change-Id: Id6db304b404e6fde44f5a8c23cdaeeaaa2324f20
2021-08-10 23:30:23 +01:00
}
// TestInsert inserts new coinpayments transaction into DB.
func (db *coinPaymentsTransactions) TestInsert(ctx context.Context, tx stripe.Transaction) (createTime time.Time, err error) {
defer mon.Task()(&ctx)(&err)
dbxCPTX, err := db.db.Create_CoinpaymentsTransaction(ctx,
dbx.CoinpaymentsTransaction_Id(tx.ID.String()),
dbx.CoinpaymentsTransaction_UserId(tx.AccountID[:]),
dbx.CoinpaymentsTransaction_Address(tx.Address),
dbx.CoinpaymentsTransaction_AmountNumeric(tx.Amount.BaseUnits()),
dbx.CoinpaymentsTransaction_ReceivedNumeric(tx.Received.BaseUnits()),
dbx.CoinpaymentsTransaction_Status(tx.Status.Int()),
dbx.CoinpaymentsTransaction_Key(tx.Key),
dbx.CoinpaymentsTransaction_Timeout(int(tx.Timeout.Seconds())),
)
if err != nil {
return time.Time{}, err
}
return dbxCPTX.CreatedAt, nil
}
// TestLockRate locks conversion rate for transaction.
func (db *coinPaymentsTransactions) TestLockRate(ctx context.Context, id coinpayments.TransactionID, rate decimal.Decimal) (err error) {
defer mon.Task()(&ctx)(&err)
rateFloat, exact := rate.Float64()
if !exact {
// It's not clear at the time of writing whether this
// inexactness will ever be something we need to worry about.
// According to the example in the API docs for
// coinpayments.net, exchange rates are given to 24 decimal
// places (!!), which is several digits more precision than we
// can represent exactly in IEEE754 double-precision floating
// point. However, that might not matter, since an exchange rate
// that is correct to ~15 decimal places multiplied by a precise
// monetary.Amount should give results that are correct to
// around 15 decimal places still. At current exchange rates,
// for example, a USD transaction would need to have a value of
// more than $1,000,000,000,000 USD before a calculation using
// this "inexact" rate would get the equivalent number of BTC
// wrong by a single satoshi (10^-8 BTC).
//
// We could avoid all of this by preserving the exact rates as
// given by our provider, but this would involve either (a)
// abuse of the SQL schema (e.g. storing rates as decimal values
// in VARCHAR), (b) storing rates in a way that is opaque to the
// db engine (e.g. gob-encoding, decimal coefficient with
// separate exponents), or (c) adding support for parameterized
// types like NUMERIC to dbx. None of those are very ideal
// either.
delta, _ := rate.Sub(decimal.NewFromFloat(rateFloat)).Float64()
mon.FloatVal("inexact-float64-exchange-rate-delta").Observe(delta)
}
_, err = db.db.Create_StripecoinpaymentsTxConversionRate(ctx,
dbx.StripecoinpaymentsTxConversionRate_TxId(id.String()),
dbx.StripecoinpaymentsTxConversionRate_RateNumeric(rateFloat),
)
satellite/satellitedb: prepare to remove big.Float from db Why: big.Float is not an ideal type for dealing with monetary amounts, because no matter how high the precision, some non-integer decimal values can not be represented exactly in base-2 floating point. Also, storing gob-encoded big.Float values in the database makes it very hard to use those values in meaningful queries, making it difficult to do any sort of analysis on billing. Now that we have amounts represented using monetary.Amount, we can simply store them in the database using integers (as given by the .BaseUnits() method on monetary.Amount). We should move toward storing the currency along with any monetary amount, wherever we are storing amounts, because satellites might want to deal with currencies other than STORJ and USD. Even better, it becomes much clearer what currency each monetary value is _supposed_ to be in (I had to dig through code to find that out for our current monetary columns). Deployment ---------- Getting rid of the big.Float columns will take multiple deployment steps. There does not seem to be any way to make the change in a way that lets existing queries continue to work on CockroachDB (it could be done with rules and triggers and a stored procedure that knows how to gob-decode big.Float objects, but CockroachDB doesn't have rules _or_ triggers _or_ stored procedures). Instead, in this first step, we make no changes to the database schema, but add code that knows how to deal with the planned changes to the schema when they are made in a future "step 2" deployment. All functions that deal with the coinbase_transactions table have been taught to recognize the "undefined column" error, and when it is seen, to call a separate "transition shim" function to accomplish the task. Once all the services are running this code, and the step 2 deployment makes breaking changes to the schema, any services that are still running and connected to the database will keep working correctly because of the fallback code included here. The step 2 deployment can be made without these transition shims included, because it will apply the database schema changes before any of its code runs. Step 1: No schema changes; just include code that recognizes the "undefined column" error when dealing with the coinbase_transactions or stripecoinpayments_tx_conversion_rates tables, and if found, assumes that the column changes from Step 2 have already been made. Step 2: In coinbase_transactions: * change the names of the 'amount' and 'received' columns to 'amount_gob' and 'received_gob' respectively * add new 'amount_numeric' and 'received_numeric' columns with INT8 type. In stripecoinpayments_tx_conversion_rates: * change the name of the 'rate' column to 'rate_gob' * add new 'rate_numeric' column with NUMERIC(8, 8) type Code reading from either of these tables must query both the X_gob and X_numeric columns. If X_numeric is not null, its value should be used; otherwise, the gob-encoded big.Float in X_gob should be used. A chore might be included in this step that transitions values from X_gob to X_numeric a few rows at a time. Step 3: Once all prod satellites have no values left in the _gob columns, we can drop those columns and add NOT NULL constraints to the _numeric columns. Change-Id: Id6db304b404e6fde44f5a8c23cdaeeaaa2324f20
2021-08-10 23:30:23 +01:00
return Error.Wrap(err)
}
// fromDBXCoinpaymentsTransaction converts *dbx.CoinpaymentsTransaction to stripecoinpayments.Transaction.
func fromDBXCoinpaymentsTransaction(dbxCPTX *dbx.CoinpaymentsTransaction) (stripe.Transaction, error) {
userID, err := uuid.FromBytes(dbxCPTX.UserId)
if err != nil {
return stripe.Transaction{}, errs.Wrap(err)
}
// TODO: the currency here should be passed in to this function or stored
// in the database.
return stripe.Transaction{
ID: coinpayments.TransactionID(dbxCPTX.Id),
AccountID: userID,
Address: dbxCPTX.Address,
Amount: currency.AmountFromBaseUnits(dbxCPTX.AmountNumeric, currency.StorjToken),
Received: currency.AmountFromBaseUnits(dbxCPTX.ReceivedNumeric, currency.StorjToken),
Status: coinpayments.Status(dbxCPTX.Status),
Key: dbxCPTX.Key,
Timeout: time.Second * time.Duration(dbxCPTX.Timeout),
CreatedAt: dbxCPTX.CreatedAt,
}, nil
}