storj/satellite/satellitedb
Jeff Wendling 78c6d5bb32 satellite/satellitedb: reported_serials table for processing orders
this commit introduces the reported_serials table. its purpose is
to allow for blind writes into it as nodes report in so that we have
minimal contention. in order to continue to accurately account for
used bandwidth, though, we cannot immediately add the settled amount.
if we did, we would have to give up on blind writes.

the table's primary key is structured precisely so that we can quickly
find expired orders and so that we maximally benefit from rocksdb
path prefix compression. we do this by rounding the expires at time
forward to the next day, effectively giving us storagenode petnames
for free. and since there's no secondary index or foreign key
constraints, this design should use significantly less space than
the current used_serials table while also reducing contention.

after inserting the orders into the table, we have a chore that
periodically consumes all of the expired orders in it and inserts
them into the existing rollups tables. this is as if we changed
the nodes to report as the order expired rather than as soon as
possible, so the belief in correctness of the refactor is higher.

since we are able to process large batches of orders (typically
a day's worth), we can use the code to maximally batch inserts into
the rollup tables to make inserts as friendly as possible to
cockroach.

Change-Id: I25d609ca2679b8331979184f16c6d46d4f74c1a6
2020-01-15 19:21:21 -07:00
..
dbx satellite/satellitedb: reported_serials table for processing orders 2020-01-15 19:21:21 -07:00
pbold common: separate repository 2019-12-27 14:11:15 +02:00
satellitedbtest satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
testdata satellite/satellitedb: reported_serials table for processing orders 2020-01-15 19:21:21 -07:00
apikeys.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
attribution_test.go common: separate repository 2019-12-27 14:11:15 +02:00
attribution.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
buckets.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
coinpaymentstxs.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
consoledb_test.go satellite/console: use transaction helpers in consoledb 2020-01-07 17:59:10 +00:00
consoledb.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
containment.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
coupons.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
customers.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
database.go satellite/satellitedb: reported_serials table for processing orders 2020-01-15 19:21:21 -07:00
downtime.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
gracefulexit.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
invoiceprojectrecords.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
irreparabledb.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
migrate_cockroach_test.go satellite/satellitedb: improve Cockroach migrate test 2019-12-21 10:58:43 +00:00
migrate_postgres_test.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
migrate.go satellite/satellitedb: reported_serials table for processing orders 2020-01-15 19:21:21 -07:00
offers.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
orders.go satellite/satellitedb: reported_serials table for processing orders 2020-01-15 19:21:21 -07:00
overlaycache.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
peeridentities.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
projectaccounting.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
projectmembers_test.go web/satellite: project members sorting fixed (#3231) 2019-10-15 15:24:53 +03:00
projectmembers.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
projects_test.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
projects.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
regtokens.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
repairqueue.go satellite/satellitedb: don't use crdb.ExecuteTx with postgres 2020-01-06 23:51:35 +00:00
resetpasstokens.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
storagenodeaccounting.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
stripecoinpaymentsdb.go satellite/satellitedb: unexport satellitedb.DB 2019-12-16 19:09:30 +00:00
usercredits.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
users_test.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
users.go satellite/satellitedb/dbx: name the package dbx 2020-01-15 15:16:39 -07:00
utils_test.go common: separate repository 2019-12-27 14:11:15 +02:00
utils.go common: separate repository 2019-12-27 14:11:15 +02:00