storj/satellite/accounting
Jeff Wendling 91698207cf storagenode: live tracking of order window usage
This change accomplishes multiple things:

1. Instead of having a max in flight time, which means
   we effectively have a minimum bandwidth for uploads
   and downloads, we keep track of what windows have
   active requests happening in them.

2. We don't double check when we save the order to see if it
   is too old: by then, it's too late. A malicious uplink
   could just submit orders outside of the grace window and
   receive all the data, but the node would just not commit
   it, so the uplink gets free traffic. Because the endpoints
   also check for the order being too old, this would be a
   very tight race that depends on knowledge of the node system
   clock, but best to not have the race exist. Instead, we piggy
   back off of the in flight tracking and do the check when
   we start to handle the order, and commit at the end.

3. Change the functions that send orders and list unsent
   orders to accept a time at which that operation is
   happening. This way, in tests, we can pretend we're
   listing or sending far into the future after the windows
   are available to send, rather than exposing test functions
   to modify internal state about the grace period to get
   the desired effect. This brings tests closer to actual
   usage in production.

4. Change the calculation for if an order is allowed to be
   enqueued due to the grace period to just look at the
   order creation time, rather than some computation involving
   the window it will be in. In this way, you can easily
   answer the question of "will this order be accepted?" by
   asking "is it older than X?" where X is the grace period.

5. Increases the frequency we check to send up orders to once
   every 5 minutes instead of once every hour because we already
   have hour-long buffering due to the windows. This decreases
   the maximum latency that an order will be reported back to
   the satellite by 55 minutes.

Change-Id: Ie08b90d139d45ee89b82347e191a2f8db1b88036
2020-08-19 19:42:33 +00:00
..
live all: add missing dots 2020-08-11 17:50:01 +03:00
projectbwcleanup satellite/accounting: Add chore to cleanup old project bandwidth rollups data 2020-07-22 18:56:49 +00:00
reportedrollup all: add missing dots 2020-08-11 17:50:01 +03:00
rollup satellite: fix scoping, formatting 2020-07-16 19:13:14 +03:00
tally all: fix dots 2020-07-16 14:58:28 +00:00
billing_test.go storagenode: live tracking of order window usage 2020-08-19 19:42:33 +00:00
bucketstats.go all: fix dots 2020-07-16 14:58:28 +00:00
bucketusage.go all: fix dots 2020-07-16 14:58:28 +00:00
common.go all: add missing dots 2020-08-11 17:50:01 +03:00
db_test.go satellite/storageusage: Group accounting rollups at_rest_total by day 2020-08-10 15:03:15 +00:00
db.go satellite/accounting: Add chore to cleanup old project bandwidth rollups data 2020-07-22 18:56:49 +00:00
projectusage_test.go satellite: prevents uplink from creating a bucket once it exceeds the max bucket allocation. 2020-07-15 17:27:05 +00:00
projectusage.go all: add missing dots 2020-08-11 17:50:01 +03:00