2019-10-17 15:04:50 +01:00
|
|
|
// Copyright (C) 2019 Storj Labs, Inc.
|
|
|
|
// See LICENSE for copying information.
|
|
|
|
|
|
|
|
package stripecoinpayments
|
|
|
|
|
|
|
|
import (
|
|
|
|
"context"
|
|
|
|
"time"
|
|
|
|
|
satellite/payments: specialized type for monetary amounts
Why: big.Float is not an ideal type for dealing with monetary amounts,
because no matter how high the precision, some non-integer decimal
values can not be represented exactly in base-2 floating point. Also,
storing gob-encoded big.Float values in the database makes it very hard
to use those values in meaningful queries, making it difficult to do
any sort of analysis on billing.
For better accuracy, then, we can just represent monetary values as
integers (in whatever base units are appropriate for the currency). For
example, STORJ tokens or Bitcoins can not be split into pieces smaller
than 10^-8, so we can store amounts of STORJ or BTC with precision
simply by moving the decimal point 8 digits to the right. For USD values
(assuming we don't want to deal with fractional cents), we can move the
decimal point 2 digits to the right.
To make it easier and less error-prone to deal with the math involved, I
introduce here a new type, monetary.Amount, instances of which have an
associated value _and_ a currency.
Change-Id: I03395d52f0e2473cf301361f6033722b54640265
2021-08-10 23:29:50 +01:00
|
|
|
"github.com/shopspring/decimal"
|
2020-02-13 18:08:45 +00:00
|
|
|
"github.com/zeebo/errs"
|
2019-10-17 15:04:50 +01:00
|
|
|
|
2020-03-30 10:08:50 +01:00
|
|
|
"storj.io/common/uuid"
|
2019-10-17 15:04:50 +01:00
|
|
|
"storj.io/storj/satellite/payments/coinpayments"
|
satellite/payments: specialized type for monetary amounts
Why: big.Float is not an ideal type for dealing with monetary amounts,
because no matter how high the precision, some non-integer decimal
values can not be represented exactly in base-2 floating point. Also,
storing gob-encoded big.Float values in the database makes it very hard
to use those values in meaningful queries, making it difficult to do
any sort of analysis on billing.
For better accuracy, then, we can just represent monetary values as
integers (in whatever base units are appropriate for the currency). For
example, STORJ tokens or Bitcoins can not be split into pieces smaller
than 10^-8, so we can store amounts of STORJ or BTC with precision
simply by moving the decimal point 8 digits to the right. For USD values
(assuming we don't want to deal with fractional cents), we can move the
decimal point 2 digits to the right.
To make it easier and less error-prone to deal with the math involved, I
introduce here a new type, monetary.Amount, instances of which have an
associated value _and_ a currency.
Change-Id: I03395d52f0e2473cf301361f6033722b54640265
2021-08-10 23:29:50 +01:00
|
|
|
"storj.io/storj/satellite/payments/monetary"
|
2019-10-17 15:04:50 +01:00
|
|
|
)
|
|
|
|
|
2020-02-13 18:08:45 +00:00
|
|
|
// ErrTransactionConsumed is thrown when trying to consume already consumed transaction.
|
|
|
|
var ErrTransactionConsumed = errs.New("error transaction already consumed")
|
|
|
|
|
2019-10-17 15:04:50 +01:00
|
|
|
// TransactionsDB is an interface which defines functionality
|
|
|
|
// of DB which stores coinpayments transactions.
|
|
|
|
//
|
|
|
|
// architecture: Database
|
|
|
|
type TransactionsDB interface {
|
|
|
|
// Insert inserts new coinpayments transaction into DB.
|
satellite/satellitedb: prepare to remove big.Float from db
Why: big.Float is not an ideal type for dealing with monetary amounts,
because no matter how high the precision, some non-integer decimal
values can not be represented exactly in base-2 floating point. Also,
storing gob-encoded big.Float values in the database makes it very hard
to use those values in meaningful queries, making it difficult to do
any sort of analysis on billing.
Now that we have amounts represented using monetary.Amount, we can
simply store them in the database using integers (as given by the
.BaseUnits() method on monetary.Amount).
We should move toward storing the currency along with any monetary
amount, wherever we are storing amounts, because satellites might want
to deal with currencies other than STORJ and USD. Even better, it
becomes much clearer what currency each monetary value is _supposed_ to
be in (I had to dig through code to find that out for our current
monetary columns).
Deployment
----------
Getting rid of the big.Float columns will take multiple deployment
steps. There does not seem to be any way to make the change in a way
that lets existing queries continue to work on CockroachDB (it could be
done with rules and triggers and a stored procedure that knows how to
gob-decode big.Float objects, but CockroachDB doesn't have rules _or_
triggers _or_ stored procedures). Instead, in this first step, we make
no changes to the database schema, but add code that knows how to deal
with the planned changes to the schema when they are made in a future
"step 2" deployment. All functions that deal with the
coinbase_transactions table have been taught to recognize the "undefined
column" error, and when it is seen, to call a separate "transition shim"
function to accomplish the task. Once all the services are running this
code, and the step 2 deployment makes breaking changes to the schema,
any services that are still running and connected to the database will
keep working correctly because of the fallback code included here. The
step 2 deployment can be made without these transition shims included,
because it will apply the database schema changes before any of its code
runs.
Step 1:
No schema changes; just include code that recognizes the
"undefined column" error when dealing with the
coinbase_transactions or stripecoinpayments_tx_conversion_rates
tables, and if found, assumes that the column changes from Step
2 have already been made.
Step 2:
In coinbase_transactions:
* change the names of the 'amount' and 'received' columns to
'amount_gob' and 'received_gob' respectively
* add new 'amount_numeric' and 'received_numeric' columns with
INT8 type.
In stripecoinpayments_tx_conversion_rates:
* change the name of the 'rate' column to 'rate_gob'
* add new 'rate_numeric' column with NUMERIC(8, 8) type
Code reading from either of these tables must query both the X_gob
and X_numeric columns. If X_numeric is not null, its value should
be used; otherwise, the gob-encoded big.Float in X_gob should be
used. A chore might be included in this step that transitions values
from X_gob to X_numeric a few rows at a time.
Step 3:
Once all prod satellites have no values left in the _gob columns, we
can drop those columns and add NOT NULL constraints to the _numeric
columns.
Change-Id: Id6db304b404e6fde44f5a8c23cdaeeaaa2324f20
2021-08-10 23:30:23 +01:00
|
|
|
Insert(ctx context.Context, tx Transaction) (time.Time, error)
|
2019-10-23 13:04:54 +01:00
|
|
|
// Update updates status and received for set of transactions.
|
2019-10-29 16:04:34 +00:00
|
|
|
Update(ctx context.Context, updates []TransactionUpdate, applies coinpayments.TransactionIDList) error
|
|
|
|
// Consume marks transaction as consumed, so it won't participate in apply account balance loop.
|
|
|
|
Consume(ctx context.Context, id coinpayments.TransactionID) error
|
2019-11-15 14:59:39 +00:00
|
|
|
// LockRate locks conversion rate for transaction.
|
satellite/payments: specialized type for monetary amounts
Why: big.Float is not an ideal type for dealing with monetary amounts,
because no matter how high the precision, some non-integer decimal
values can not be represented exactly in base-2 floating point. Also,
storing gob-encoded big.Float values in the database makes it very hard
to use those values in meaningful queries, making it difficult to do
any sort of analysis on billing.
For better accuracy, then, we can just represent monetary values as
integers (in whatever base units are appropriate for the currency). For
example, STORJ tokens or Bitcoins can not be split into pieces smaller
than 10^-8, so we can store amounts of STORJ or BTC with precision
simply by moving the decimal point 8 digits to the right. For USD values
(assuming we don't want to deal with fractional cents), we can move the
decimal point 2 digits to the right.
To make it easier and less error-prone to deal with the math involved, I
introduce here a new type, monetary.Amount, instances of which have an
associated value _and_ a currency.
Change-Id: I03395d52f0e2473cf301361f6033722b54640265
2021-08-10 23:29:50 +01:00
|
|
|
LockRate(ctx context.Context, id coinpayments.TransactionID, rate decimal.Decimal) error
|
2019-11-15 14:59:39 +00:00
|
|
|
// GetLockedRate returns locked conversion rate for transaction or error if non exists.
|
satellite/payments: specialized type for monetary amounts
Why: big.Float is not an ideal type for dealing with monetary amounts,
because no matter how high the precision, some non-integer decimal
values can not be represented exactly in base-2 floating point. Also,
storing gob-encoded big.Float values in the database makes it very hard
to use those values in meaningful queries, making it difficult to do
any sort of analysis on billing.
For better accuracy, then, we can just represent monetary values as
integers (in whatever base units are appropriate for the currency). For
example, STORJ tokens or Bitcoins can not be split into pieces smaller
than 10^-8, so we can store amounts of STORJ or BTC with precision
simply by moving the decimal point 8 digits to the right. For USD values
(assuming we don't want to deal with fractional cents), we can move the
decimal point 2 digits to the right.
To make it easier and less error-prone to deal with the math involved, I
introduce here a new type, monetary.Amount, instances of which have an
associated value _and_ a currency.
Change-Id: I03395d52f0e2473cf301361f6033722b54640265
2021-08-10 23:29:50 +01:00
|
|
|
GetLockedRate(ctx context.Context, id coinpayments.TransactionID) (decimal.Decimal, error)
|
2019-11-12 11:14:34 +00:00
|
|
|
// ListAccount returns all transaction for specific user.
|
|
|
|
ListAccount(ctx context.Context, userID uuid.UUID) ([]Transaction, error)
|
2019-10-23 13:04:54 +01:00
|
|
|
// ListPending returns TransactionsPage with pending transactions.
|
|
|
|
ListPending(ctx context.Context, offset int64, limit int, before time.Time) (TransactionsPage, error)
|
satellite/payments: chore to migrate big.Float values out of db
All code on known satellites at this moment in time should know how to
populate and use the new numeric columns on the
stripecoinpayments_tx_conversion_rates and coinpayments_transactions
tables in the satellite db. However, there are still gob-encoded
big.Float values in the database from before these columns existed. To
get rid of those values, so that we can excise the gob-decoding code
from the relevant sections, however, we need something to read the gob
bytestrings and convert them to numeric values, a few at a time, until
they're all gone.
To accomplish that, this change adds two chores to be run in the
satellite core process- one for the coinpayments_transactions table, and
one for the stripecoinpayments_tx_conversion_rates table. They should
run relatively infrequently, so that we do not impose any undue load on
processing resources or the db.
Both of these chores work without using explicit sql transactions, but
should still be concurrent-safe, since they work by way of
compare-and-swap type operations.
If the satellite core process needs to be restarted, both of these
chores will start scanning for migrateable rows from the beginning of
the id space again. This is not ideal, but shouldn't be a problem (as
far as I can tell, there are only a few thousand rows at most in either
of these tables on any production satellite).
Change-Id: I733b7cd96760d506a1cf52735f598c6c3aa19735
2021-10-29 14:32:59 +01:00
|
|
|
// ListUnapplied returns TransactionsPage with completed transaction that should be applied to account balance.
|
2019-10-29 16:04:34 +00:00
|
|
|
ListUnapplied(ctx context.Context, offset int64, limit int, before time.Time) (TransactionsPage, error)
|
2019-10-17 15:04:50 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
// Transaction defines coinpayments transaction info that is stored in the DB.
|
|
|
|
type Transaction struct {
|
|
|
|
ID coinpayments.TransactionID
|
|
|
|
AccountID uuid.UUID
|
|
|
|
Address string
|
satellite/payments: specialized type for monetary amounts
Why: big.Float is not an ideal type for dealing with monetary amounts,
because no matter how high the precision, some non-integer decimal
values can not be represented exactly in base-2 floating point. Also,
storing gob-encoded big.Float values in the database makes it very hard
to use those values in meaningful queries, making it difficult to do
any sort of analysis on billing.
For better accuracy, then, we can just represent monetary values as
integers (in whatever base units are appropriate for the currency). For
example, STORJ tokens or Bitcoins can not be split into pieces smaller
than 10^-8, so we can store amounts of STORJ or BTC with precision
simply by moving the decimal point 8 digits to the right. For USD values
(assuming we don't want to deal with fractional cents), we can move the
decimal point 2 digits to the right.
To make it easier and less error-prone to deal with the math involved, I
introduce here a new type, monetary.Amount, instances of which have an
associated value _and_ a currency.
Change-Id: I03395d52f0e2473cf301361f6033722b54640265
2021-08-10 23:29:50 +01:00
|
|
|
Amount monetary.Amount
|
|
|
|
Received monetary.Amount
|
2019-10-17 15:04:50 +01:00
|
|
|
Status coinpayments.Status
|
|
|
|
Key string
|
2019-11-15 14:59:39 +00:00
|
|
|
Timeout time.Duration
|
2019-10-17 15:04:50 +01:00
|
|
|
CreatedAt time.Time
|
|
|
|
}
|
2019-10-23 13:04:54 +01:00
|
|
|
|
|
|
|
// TransactionUpdate holds transaction update info.
|
|
|
|
type TransactionUpdate struct {
|
|
|
|
TransactionID coinpayments.TransactionID
|
|
|
|
Status coinpayments.Status
|
satellite/payments: specialized type for monetary amounts
Why: big.Float is not an ideal type for dealing with monetary amounts,
because no matter how high the precision, some non-integer decimal
values can not be represented exactly in base-2 floating point. Also,
storing gob-encoded big.Float values in the database makes it very hard
to use those values in meaningful queries, making it difficult to do
any sort of analysis on billing.
For better accuracy, then, we can just represent monetary values as
integers (in whatever base units are appropriate for the currency). For
example, STORJ tokens or Bitcoins can not be split into pieces smaller
than 10^-8, so we can store amounts of STORJ or BTC with precision
simply by moving the decimal point 8 digits to the right. For USD values
(assuming we don't want to deal with fractional cents), we can move the
decimal point 2 digits to the right.
To make it easier and less error-prone to deal with the math involved, I
introduce here a new type, monetary.Amount, instances of which have an
associated value _and_ a currency.
Change-Id: I03395d52f0e2473cf301361f6033722b54640265
2021-08-10 23:29:50 +01:00
|
|
|
Received monetary.Amount
|
2019-10-23 13:04:54 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
// TransactionsPage holds set of transaction and indicates if
|
|
|
|
// there are more transactions to fetch.
|
|
|
|
type TransactionsPage struct {
|
|
|
|
Transactions []Transaction
|
|
|
|
Next bool
|
|
|
|
NextOffset int64
|
|
|
|
}
|
|
|
|
|
|
|
|
// IDList returns transaction id list of page's transactions.
|
2020-01-29 00:57:15 +00:00
|
|
|
func (page *TransactionsPage) IDList() TransactionAndUserList {
|
2022-08-01 13:41:12 +01:00
|
|
|
ids := make(TransactionAndUserList)
|
2019-10-23 13:04:54 +01:00
|
|
|
for _, tx := range page.Transactions {
|
2020-01-29 00:57:15 +00:00
|
|
|
ids[tx.ID] = tx.AccountID
|
|
|
|
}
|
|
|
|
return ids
|
|
|
|
}
|
|
|
|
|
2020-06-25 22:16:39 +01:00
|
|
|
// CreationTimes returns a map of creation times of page's transactions.
|
|
|
|
func (page *TransactionsPage) CreationTimes() map[coinpayments.TransactionID]time.Time {
|
2020-06-30 22:49:29 +01:00
|
|
|
creationTimes := make(map[coinpayments.TransactionID]time.Time)
|
2020-06-25 22:16:39 +01:00
|
|
|
for _, tx := range page.Transactions {
|
|
|
|
creationTimes[tx.ID] = tx.CreatedAt
|
|
|
|
}
|
|
|
|
return creationTimes
|
|
|
|
}
|
|
|
|
|
2020-07-16 15:18:02 +01:00
|
|
|
// TransactionAndUserList is a composite type for storing userID and txID.
|
2020-01-29 00:57:15 +00:00
|
|
|
type TransactionAndUserList map[coinpayments.TransactionID]uuid.UUID
|
|
|
|
|
|
|
|
// IDList returns transaction id list.
|
|
|
|
func (idMap TransactionAndUserList) IDList() coinpayments.TransactionIDList {
|
|
|
|
var list coinpayments.TransactionIDList
|
|
|
|
for transactionID := range idMap {
|
|
|
|
list = append(list, transactionID)
|
2019-10-23 13:04:54 +01:00
|
|
|
}
|
|
|
|
return list
|
|
|
|
}
|