2019-10-17 15:04:50 +01:00
|
|
|
// Copyright (C) 2019 Storj Labs, Inc.
|
|
|
|
// See LICENSE for copying information.
|
|
|
|
|
|
|
|
package satellitedb
|
|
|
|
|
|
|
|
import (
|
|
|
|
"context"
|
2019-10-23 13:04:54 +01:00
|
|
|
"time"
|
2019-10-17 15:04:50 +01:00
|
|
|
|
satellite/payments: specialized type for monetary amounts
Why: big.Float is not an ideal type for dealing with monetary amounts,
because no matter how high the precision, some non-integer decimal
values can not be represented exactly in base-2 floating point. Also,
storing gob-encoded big.Float values in the database makes it very hard
to use those values in meaningful queries, making it difficult to do
any sort of analysis on billing.
For better accuracy, then, we can just represent monetary values as
integers (in whatever base units are appropriate for the currency). For
example, STORJ tokens or Bitcoins can not be split into pieces smaller
than 10^-8, so we can store amounts of STORJ or BTC with precision
simply by moving the decimal point 8 digits to the right. For USD values
(assuming we don't want to deal with fractional cents), we can move the
decimal point 2 digits to the right.
To make it easier and less error-prone to deal with the math involved, I
introduce here a new type, monetary.Amount, instances of which have an
associated value _and_ a currency.
Change-Id: I03395d52f0e2473cf301361f6033722b54640265
2021-08-10 23:29:50 +01:00
|
|
|
"github.com/shopspring/decimal"
|
2019-10-17 15:04:50 +01:00
|
|
|
"github.com/zeebo/errs"
|
|
|
|
|
2022-09-06 13:43:09 +01:00
|
|
|
"storj.io/common/currency"
|
2020-03-30 10:08:50 +01:00
|
|
|
"storj.io/common/uuid"
|
2019-10-17 15:04:50 +01:00
|
|
|
"storj.io/storj/satellite/payments/coinpayments"
|
|
|
|
"storj.io/storj/satellite/payments/stripecoinpayments"
|
2020-01-15 02:29:51 +00:00
|
|
|
"storj.io/storj/satellite/satellitedb/dbx"
|
2019-10-17 15:04:50 +01:00
|
|
|
)
|
|
|
|
|
2019-11-05 13:16:02 +00:00
|
|
|
// ensure that coinpaymentsTransactions implements stripecoinpayments.TransactionsDB.
|
|
|
|
var _ stripecoinpayments.TransactionsDB = (*coinPaymentsTransactions)(nil)
|
2019-10-17 15:04:50 +01:00
|
|
|
|
2019-11-05 13:16:02 +00:00
|
|
|
// coinPaymentsTransactions is CoinPayments transactions DB.
|
2019-10-17 15:04:50 +01:00
|
|
|
//
|
|
|
|
// architecture: Database
|
2019-11-05 13:16:02 +00:00
|
|
|
type coinPaymentsTransactions struct {
|
2019-12-14 02:29:54 +00:00
|
|
|
db *satelliteDB
|
2019-10-17 15:04:50 +01:00
|
|
|
}
|
|
|
|
|
2022-11-29 12:36:41 +00:00
|
|
|
// GetLockedRate returns locked conversion rate for transaction or error if non exists.
|
|
|
|
func (db *coinPaymentsTransactions) GetLockedRate(ctx context.Context, id coinpayments.TransactionID) (rate decimal.Decimal, err error) {
|
2019-11-15 14:59:39 +00:00
|
|
|
defer mon.Task()(&ctx)(&err)
|
|
|
|
|
2022-11-29 12:36:41 +00:00
|
|
|
dbxRate, err := db.db.Get_StripecoinpaymentsTxConversionRate_By_TxId(ctx,
|
|
|
|
dbx.StripecoinpaymentsTxConversionRate_TxId(id.String()),
|
2019-10-17 15:04:50 +01:00
|
|
|
)
|
|
|
|
if err != nil {
|
2022-11-29 12:36:41 +00:00
|
|
|
return decimal.Decimal{}, err
|
2019-10-17 15:04:50 +01:00
|
|
|
}
|
2022-11-29 12:36:41 +00:00
|
|
|
|
|
|
|
rate = decimal.NewFromFloat(dbxRate.RateNumeric)
|
|
|
|
return rate, nil
|
satellite/satellitedb: prepare to remove big.Float from db
Why: big.Float is not an ideal type for dealing with monetary amounts,
because no matter how high the precision, some non-integer decimal
values can not be represented exactly in base-2 floating point. Also,
storing gob-encoded big.Float values in the database makes it very hard
to use those values in meaningful queries, making it difficult to do
any sort of analysis on billing.
Now that we have amounts represented using monetary.Amount, we can
simply store them in the database using integers (as given by the
.BaseUnits() method on monetary.Amount).
We should move toward storing the currency along with any monetary
amount, wherever we are storing amounts, because satellites might want
to deal with currencies other than STORJ and USD. Even better, it
becomes much clearer what currency each monetary value is _supposed_ to
be in (I had to dig through code to find that out for our current
monetary columns).
Deployment
----------
Getting rid of the big.Float columns will take multiple deployment
steps. There does not seem to be any way to make the change in a way
that lets existing queries continue to work on CockroachDB (it could be
done with rules and triggers and a stored procedure that knows how to
gob-decode big.Float objects, but CockroachDB doesn't have rules _or_
triggers _or_ stored procedures). Instead, in this first step, we make
no changes to the database schema, but add code that knows how to deal
with the planned changes to the schema when they are made in a future
"step 2" deployment. All functions that deal with the
coinbase_transactions table have been taught to recognize the "undefined
column" error, and when it is seen, to call a separate "transition shim"
function to accomplish the task. Once all the services are running this
code, and the step 2 deployment makes breaking changes to the schema,
any services that are still running and connected to the database will
keep working correctly because of the fallback code included here. The
step 2 deployment can be made without these transition shims included,
because it will apply the database schema changes before any of its code
runs.
Step 1:
No schema changes; just include code that recognizes the
"undefined column" error when dealing with the
coinbase_transactions or stripecoinpayments_tx_conversion_rates
tables, and if found, assumes that the column changes from Step
2 have already been made.
Step 2:
In coinbase_transactions:
* change the names of the 'amount' and 'received' columns to
'amount_gob' and 'received_gob' respectively
* add new 'amount_numeric' and 'received_numeric' columns with
INT8 type.
In stripecoinpayments_tx_conversion_rates:
* change the name of the 'rate' column to 'rate_gob'
* add new 'rate_numeric' column with NUMERIC(8, 8) type
Code reading from either of these tables must query both the X_gob
and X_numeric columns. If X_numeric is not null, its value should
be used; otherwise, the gob-encoded big.Float in X_gob should be
used. A chore might be included in this step that transitions values
from X_gob to X_numeric a few rows at a time.
Step 3:
Once all prod satellites have no values left in the _gob columns, we
can drop those columns and add NOT NULL constraints to the _numeric
columns.
Change-Id: Id6db304b404e6fde44f5a8c23cdaeeaaa2324f20
2021-08-10 23:30:23 +01:00
|
|
|
}
|
2019-10-17 15:04:50 +01:00
|
|
|
|
2022-11-29 12:36:41 +00:00
|
|
|
// ListAccount returns all transaction for specific user.
|
|
|
|
func (db *coinPaymentsTransactions) ListAccount(ctx context.Context, userID uuid.UUID) (_ []stripecoinpayments.Transaction, err error) {
|
2019-11-15 14:59:39 +00:00
|
|
|
defer mon.Task()(&ctx)(&err)
|
|
|
|
|
2022-11-29 12:36:41 +00:00
|
|
|
dbxTXs, err := db.db.All_CoinpaymentsTransaction_By_UserId_OrderBy_Desc_CreatedAt(ctx,
|
|
|
|
dbx.CoinpaymentsTransaction_UserId(userID[:]),
|
|
|
|
)
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
2019-10-23 13:04:54 +01:00
|
|
|
}
|
|
|
|
|
2022-11-29 12:36:41 +00:00
|
|
|
var txs []stripecoinpayments.Transaction
|
|
|
|
for _, dbxTX := range dbxTXs {
|
|
|
|
tx, err := fromDBXCoinpaymentsTransaction(dbxTX)
|
|
|
|
if err != nil {
|
|
|
|
return nil, errs.Wrap(err)
|
2019-10-29 16:04:34 +00:00
|
|
|
}
|
|
|
|
|
2022-11-29 12:36:41 +00:00
|
|
|
txs = append(txs, *tx)
|
|
|
|
}
|
satellite/satellitedb: prepare to remove big.Float from db
Why: big.Float is not an ideal type for dealing with monetary amounts,
because no matter how high the precision, some non-integer decimal
values can not be represented exactly in base-2 floating point. Also,
storing gob-encoded big.Float values in the database makes it very hard
to use those values in meaningful queries, making it difficult to do
any sort of analysis on billing.
Now that we have amounts represented using monetary.Amount, we can
simply store them in the database using integers (as given by the
.BaseUnits() method on monetary.Amount).
We should move toward storing the currency along with any monetary
amount, wherever we are storing amounts, because satellites might want
to deal with currencies other than STORJ and USD. Even better, it
becomes much clearer what currency each monetary value is _supposed_ to
be in (I had to dig through code to find that out for our current
monetary columns).
Deployment
----------
Getting rid of the big.Float columns will take multiple deployment
steps. There does not seem to be any way to make the change in a way
that lets existing queries continue to work on CockroachDB (it could be
done with rules and triggers and a stored procedure that knows how to
gob-decode big.Float objects, but CockroachDB doesn't have rules _or_
triggers _or_ stored procedures). Instead, in this first step, we make
no changes to the database schema, but add code that knows how to deal
with the planned changes to the schema when they are made in a future
"step 2" deployment. All functions that deal with the
coinbase_transactions table have been taught to recognize the "undefined
column" error, and when it is seen, to call a separate "transition shim"
function to accomplish the task. Once all the services are running this
code, and the step 2 deployment makes breaking changes to the schema,
any services that are still running and connected to the database will
keep working correctly because of the fallback code included here. The
step 2 deployment can be made without these transition shims included,
because it will apply the database schema changes before any of its code
runs.
Step 1:
No schema changes; just include code that recognizes the
"undefined column" error when dealing with the
coinbase_transactions or stripecoinpayments_tx_conversion_rates
tables, and if found, assumes that the column changes from Step
2 have already been made.
Step 2:
In coinbase_transactions:
* change the names of the 'amount' and 'received' columns to
'amount_gob' and 'received_gob' respectively
* add new 'amount_numeric' and 'received_numeric' columns with
INT8 type.
In stripecoinpayments_tx_conversion_rates:
* change the name of the 'rate' column to 'rate_gob'
* add new 'rate_numeric' column with NUMERIC(8, 8) type
Code reading from either of these tables must query both the X_gob
and X_numeric columns. If X_numeric is not null, its value should
be used; otherwise, the gob-encoded big.Float in X_gob should be
used. A chore might be included in this step that transitions values
from X_gob to X_numeric a few rows at a time.
Step 3:
Once all prod satellites have no values left in the _gob columns, we
can drop those columns and add NOT NULL constraints to the _numeric
columns.
Change-Id: Id6db304b404e6fde44f5a8c23cdaeeaaa2324f20
2021-08-10 23:30:23 +01:00
|
|
|
|
2022-11-29 12:36:41 +00:00
|
|
|
return txs, nil
|
satellite/satellitedb: prepare to remove big.Float from db
Why: big.Float is not an ideal type for dealing with monetary amounts,
because no matter how high the precision, some non-integer decimal
values can not be represented exactly in base-2 floating point. Also,
storing gob-encoded big.Float values in the database makes it very hard
to use those values in meaningful queries, making it difficult to do
any sort of analysis on billing.
Now that we have amounts represented using monetary.Amount, we can
simply store them in the database using integers (as given by the
.BaseUnits() method on monetary.Amount).
We should move toward storing the currency along with any monetary
amount, wherever we are storing amounts, because satellites might want
to deal with currencies other than STORJ and USD. Even better, it
becomes much clearer what currency each monetary value is _supposed_ to
be in (I had to dig through code to find that out for our current
monetary columns).
Deployment
----------
Getting rid of the big.Float columns will take multiple deployment
steps. There does not seem to be any way to make the change in a way
that lets existing queries continue to work on CockroachDB (it could be
done with rules and triggers and a stored procedure that knows how to
gob-decode big.Float objects, but CockroachDB doesn't have rules _or_
triggers _or_ stored procedures). Instead, in this first step, we make
no changes to the database schema, but add code that knows how to deal
with the planned changes to the schema when they are made in a future
"step 2" deployment. All functions that deal with the
coinbase_transactions table have been taught to recognize the "undefined
column" error, and when it is seen, to call a separate "transition shim"
function to accomplish the task. Once all the services are running this
code, and the step 2 deployment makes breaking changes to the schema,
any services that are still running and connected to the database will
keep working correctly because of the fallback code included here. The
step 2 deployment can be made without these transition shims included,
because it will apply the database schema changes before any of its code
runs.
Step 1:
No schema changes; just include code that recognizes the
"undefined column" error when dealing with the
coinbase_transactions or stripecoinpayments_tx_conversion_rates
tables, and if found, assumes that the column changes from Step
2 have already been made.
Step 2:
In coinbase_transactions:
* change the names of the 'amount' and 'received' columns to
'amount_gob' and 'received_gob' respectively
* add new 'amount_numeric' and 'received_numeric' columns with
INT8 type.
In stripecoinpayments_tx_conversion_rates:
* change the name of the 'rate' column to 'rate_gob'
* add new 'rate_numeric' column with NUMERIC(8, 8) type
Code reading from either of these tables must query both the X_gob
and X_numeric columns. If X_numeric is not null, its value should
be used; otherwise, the gob-encoded big.Float in X_gob should be
used. A chore might be included in this step that transitions values
from X_gob to X_numeric a few rows at a time.
Step 3:
Once all prod satellites have no values left in the _gob columns, we
can drop those columns and add NOT NULL constraints to the _numeric
columns.
Change-Id: Id6db304b404e6fde44f5a8c23cdaeeaaa2324f20
2021-08-10 23:30:23 +01:00
|
|
|
}
|
|
|
|
|
2022-11-29 12:36:41 +00:00
|
|
|
// TestInsert inserts new coinpayments transaction into DB.
|
|
|
|
func (db *coinPaymentsTransactions) TestInsert(ctx context.Context, tx stripecoinpayments.Transaction) (createTime time.Time, err error) {
|
2019-11-15 14:59:39 +00:00
|
|
|
defer mon.Task()(&ctx)(&err)
|
|
|
|
|
2022-11-29 12:36:41 +00:00
|
|
|
dbxCPTX, err := db.db.Create_CoinpaymentsTransaction(ctx,
|
|
|
|
dbx.CoinpaymentsTransaction_Id(tx.ID.String()),
|
|
|
|
dbx.CoinpaymentsTransaction_UserId(tx.AccountID[:]),
|
|
|
|
dbx.CoinpaymentsTransaction_Address(tx.Address),
|
|
|
|
dbx.CoinpaymentsTransaction_AmountNumeric(tx.Amount.BaseUnits()),
|
|
|
|
dbx.CoinpaymentsTransaction_ReceivedNumeric(tx.Received.BaseUnits()),
|
|
|
|
dbx.CoinpaymentsTransaction_Status(tx.Status.Int()),
|
|
|
|
dbx.CoinpaymentsTransaction_Key(tx.Key),
|
|
|
|
dbx.CoinpaymentsTransaction_Timeout(int(tx.Timeout.Seconds())),
|
|
|
|
)
|
|
|
|
if err != nil {
|
|
|
|
return time.Time{}, err
|
2020-02-13 18:08:45 +00:00
|
|
|
}
|
2022-11-29 12:36:41 +00:00
|
|
|
return dbxCPTX.CreatedAt, nil
|
2019-10-29 16:04:34 +00:00
|
|
|
}
|
|
|
|
|
2022-11-29 12:36:41 +00:00
|
|
|
// TestLockRate locks conversion rate for transaction.
|
|
|
|
func (db *coinPaymentsTransactions) TestLockRate(ctx context.Context, id coinpayments.TransactionID, rate decimal.Decimal) (err error) {
|
2019-11-15 14:59:39 +00:00
|
|
|
defer mon.Task()(&ctx)(&err)
|
|
|
|
|
2021-07-29 02:33:05 +01:00
|
|
|
rateFloat, exact := rate.Float64()
|
|
|
|
if !exact {
|
|
|
|
// It's not clear at the time of writing whether this
|
|
|
|
// inexactness will ever be something we need to worry about.
|
|
|
|
// According to the example in the API docs for
|
|
|
|
// coinpayments.net, exchange rates are given to 24 decimal
|
|
|
|
// places (!!), which is several digits more precision than we
|
|
|
|
// can represent exactly in IEEE754 double-precision floating
|
|
|
|
// point. However, that might not matter, since an exchange rate
|
|
|
|
// that is correct to ~15 decimal places multiplied by a precise
|
|
|
|
// monetary.Amount should give results that are correct to
|
|
|
|
// around 15 decimal places still. At current exchange rates,
|
|
|
|
// for example, a USD transaction would need to have a value of
|
|
|
|
// more than $1,000,000,000,000 USD before a calculation using
|
|
|
|
// this "inexact" rate would get the equivalent number of BTC
|
|
|
|
// wrong by a single satoshi (10^-8 BTC).
|
|
|
|
//
|
|
|
|
// We could avoid all of this by preserving the exact rates as
|
|
|
|
// given by our provider, but this would involve either (a)
|
|
|
|
// abuse of the SQL schema (e.g. storing rates as decimal values
|
|
|
|
// in VARCHAR), (b) storing rates in a way that is opaque to the
|
|
|
|
// db engine (e.g. gob-encoding, decimal coefficient with
|
|
|
|
// separate exponents), or (c) adding support for parameterized
|
|
|
|
// types like NUMERIC to dbx. None of those are very ideal
|
|
|
|
// either.
|
|
|
|
delta, _ := rate.Sub(decimal.NewFromFloat(rateFloat)).Float64()
|
|
|
|
mon.FloatVal("inexact-float64-exchange-rate-delta").Observe(delta)
|
2019-11-15 14:59:39 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
_, err = db.db.Create_StripecoinpaymentsTxConversionRate(ctx,
|
|
|
|
dbx.StripecoinpaymentsTxConversionRate_TxId(id.String()),
|
2022-03-24 16:19:47 +00:00
|
|
|
dbx.StripecoinpaymentsTxConversionRate_RateNumeric(rateFloat),
|
|
|
|
)
|
satellite/satellitedb: prepare to remove big.Float from db
Why: big.Float is not an ideal type for dealing with monetary amounts,
because no matter how high the precision, some non-integer decimal
values can not be represented exactly in base-2 floating point. Also,
storing gob-encoded big.Float values in the database makes it very hard
to use those values in meaningful queries, making it difficult to do
any sort of analysis on billing.
Now that we have amounts represented using monetary.Amount, we can
simply store them in the database using integers (as given by the
.BaseUnits() method on monetary.Amount).
We should move toward storing the currency along with any monetary
amount, wherever we are storing amounts, because satellites might want
to deal with currencies other than STORJ and USD. Even better, it
becomes much clearer what currency each monetary value is _supposed_ to
be in (I had to dig through code to find that out for our current
monetary columns).
Deployment
----------
Getting rid of the big.Float columns will take multiple deployment
steps. There does not seem to be any way to make the change in a way
that lets existing queries continue to work on CockroachDB (it could be
done with rules and triggers and a stored procedure that knows how to
gob-decode big.Float objects, but CockroachDB doesn't have rules _or_
triggers _or_ stored procedures). Instead, in this first step, we make
no changes to the database schema, but add code that knows how to deal
with the planned changes to the schema when they are made in a future
"step 2" deployment. All functions that deal with the
coinbase_transactions table have been taught to recognize the "undefined
column" error, and when it is seen, to call a separate "transition shim"
function to accomplish the task. Once all the services are running this
code, and the step 2 deployment makes breaking changes to the schema,
any services that are still running and connected to the database will
keep working correctly because of the fallback code included here. The
step 2 deployment can be made without these transition shims included,
because it will apply the database schema changes before any of its code
runs.
Step 1:
No schema changes; just include code that recognizes the
"undefined column" error when dealing with the
coinbase_transactions or stripecoinpayments_tx_conversion_rates
tables, and if found, assumes that the column changes from Step
2 have already been made.
Step 2:
In coinbase_transactions:
* change the names of the 'amount' and 'received' columns to
'amount_gob' and 'received_gob' respectively
* add new 'amount_numeric' and 'received_numeric' columns with
INT8 type.
In stripecoinpayments_tx_conversion_rates:
* change the name of the 'rate' column to 'rate_gob'
* add new 'rate_numeric' column with NUMERIC(8, 8) type
Code reading from either of these tables must query both the X_gob
and X_numeric columns. If X_numeric is not null, its value should
be used; otherwise, the gob-encoded big.Float in X_gob should be
used. A chore might be included in this step that transitions values
from X_gob to X_numeric a few rows at a time.
Step 3:
Once all prod satellites have no values left in the _gob columns, we
can drop those columns and add NOT NULL constraints to the _numeric
columns.
Change-Id: Id6db304b404e6fde44f5a8c23cdaeeaaa2324f20
2021-08-10 23:30:23 +01:00
|
|
|
return Error.Wrap(err)
|
2019-11-15 14:59:39 +00:00
|
|
|
}
|
|
|
|
|
2019-10-17 15:04:50 +01:00
|
|
|
// fromDBXCoinpaymentsTransaction converts *dbx.CoinpaymentsTransaction to *stripecoinpayments.Transaction.
|
|
|
|
func fromDBXCoinpaymentsTransaction(dbxCPTX *dbx.CoinpaymentsTransaction) (*stripecoinpayments.Transaction, error) {
|
2020-03-31 17:49:16 +01:00
|
|
|
userID, err := uuid.FromBytes(dbxCPTX.UserId)
|
2019-10-17 15:04:50 +01:00
|
|
|
if err != nil {
|
|
|
|
return nil, errs.Wrap(err)
|
|
|
|
}
|
|
|
|
|
satellite/payments: specialized type for monetary amounts
Why: big.Float is not an ideal type for dealing with monetary amounts,
because no matter how high the precision, some non-integer decimal
values can not be represented exactly in base-2 floating point. Also,
storing gob-encoded big.Float values in the database makes it very hard
to use those values in meaningful queries, making it difficult to do
any sort of analysis on billing.
For better accuracy, then, we can just represent monetary values as
integers (in whatever base units are appropriate for the currency). For
example, STORJ tokens or Bitcoins can not be split into pieces smaller
than 10^-8, so we can store amounts of STORJ or BTC with precision
simply by moving the decimal point 8 digits to the right. For USD values
(assuming we don't want to deal with fractional cents), we can move the
decimal point 2 digits to the right.
To make it easier and less error-prone to deal with the math involved, I
introduce here a new type, monetary.Amount, instances of which have an
associated value _and_ a currency.
Change-Id: I03395d52f0e2473cf301361f6033722b54640265
2021-08-10 23:29:50 +01:00
|
|
|
// TODO: the currency here should be passed in to this function or stored
|
|
|
|
// in the database.
|
|
|
|
|
2019-10-17 15:04:50 +01:00
|
|
|
return &stripecoinpayments.Transaction{
|
|
|
|
ID: coinpayments.TransactionID(dbxCPTX.Id),
|
|
|
|
AccountID: userID,
|
|
|
|
Address: dbxCPTX.Address,
|
2022-09-06 13:43:09 +01:00
|
|
|
Amount: currency.AmountFromBaseUnits(dbxCPTX.AmountNumeric, currency.StorjToken),
|
|
|
|
Received: currency.AmountFromBaseUnits(dbxCPTX.ReceivedNumeric, currency.StorjToken),
|
2019-10-17 15:04:50 +01:00
|
|
|
Status: coinpayments.Status(dbxCPTX.Status),
|
|
|
|
Key: dbxCPTX.Key,
|
2019-11-15 14:59:39 +00:00
|
|
|
Timeout: time.Second * time.Duration(dbxCPTX.Timeout),
|
2019-10-17 15:04:50 +01:00
|
|
|
CreatedAt: dbxCPTX.CreatedAt,
|
|
|
|
}, nil
|
|
|
|
}
|