storj/storagenode
Jeff Wendling 91698207cf storagenode: live tracking of order window usage
This change accomplishes multiple things:

1. Instead of having a max in flight time, which means
   we effectively have a minimum bandwidth for uploads
   and downloads, we keep track of what windows have
   active requests happening in them.

2. We don't double check when we save the order to see if it
   is too old: by then, it's too late. A malicious uplink
   could just submit orders outside of the grace window and
   receive all the data, but the node would just not commit
   it, so the uplink gets free traffic. Because the endpoints
   also check for the order being too old, this would be a
   very tight race that depends on knowledge of the node system
   clock, but best to not have the race exist. Instead, we piggy
   back off of the in flight tracking and do the check when
   we start to handle the order, and commit at the end.

3. Change the functions that send orders and list unsent
   orders to accept a time at which that operation is
   happening. This way, in tests, we can pretend we're
   listing or sending far into the future after the windows
   are available to send, rather than exposing test functions
   to modify internal state about the grace period to get
   the desired effect. This brings tests closer to actual
   usage in production.

4. Change the calculation for if an order is allowed to be
   enqueued due to the grace period to just look at the
   order creation time, rather than some computation involving
   the window it will be in. In this way, you can easily
   answer the question of "will this order be accepted?" by
   asking "is it older than X?" where X is the grace period.

5. Increases the frequency we check to send up orders to once
   every 5 minutes instead of once every hour because we already
   have hour-long buffering due to the windows. This decreases
   the maximum latency that an order will be reported back to
   the satellite by 55 minutes.

Change-Id: Ie08b90d139d45ee89b82347e191a2f8db1b88036
2020-08-19 19:42:33 +00:00
..
bandwidth all: fix dots 2020-07-16 14:58:28 +00:00
collector error handling in heldamount cash and collector delete fixed 2020-07-22 12:26:13 +00:00
console storagenode/console: added list of all audits per satellite to sno dashboard/satellites 2020-08-10 12:55:07 +00:00
contact all: add missing dots 2020-08-11 17:50:01 +03:00
gracefulexit storagenode/gracefulexit: Wait for the worker delete and transfer goroutines to finish before completing the exit 2020-08-19 14:20:26 +00:00
heldamount web/storagenode: payout history table 2020-08-13 12:05:56 +00:00
inspector all: add missing dots 2020-08-11 17:50:01 +03:00
monitor private/testplanet, storagenode/{monitor,pieces}: write storage dir verification file on run and verify on loop 2020-08-19 19:12:21 +00:00
nodestats all: add missing dots 2020-08-11 17:50:01 +03:00
notifications all: add missing dots 2020-08-11 17:50:01 +03:00
orders storagenode: live tracking of order window usage 2020-08-19 19:42:33 +00:00
pieces private/testblobs, storage, storage/filestore: add storage dir verification to filestore 2020-08-19 17:18:14 +00:00
piecestore storagenode: live tracking of order window usage 2020-08-19 19:42:33 +00:00
preflight all: fix dots 2020-07-16 14:58:28 +00:00
pricing storagenode/nodestats: add pricing model, storagenode/cache: add paystub history storing 2020-04-14 19:04:00 +03:00
reputation all: fix dots 2020-07-16 14:58:28 +00:00
retain storagenode: fixed logging about piece being moved to trash, and added logging when piece was actually deleted 2020-08-02 20:00:05 +03:00
satellites all: add missing dots 2020-08-11 17:50:01 +03:00
storagenodedb storagenode: live tracking of order window usage 2020-08-19 19:42:33 +00:00
storageusage storagenode/storagenodedb/storagenodedbtest: pass ctx as an argument 2020-01-20 16:56:12 +02:00
trust all: add missing dots 2020-08-11 17:50:01 +03:00
version storagenode/version: version chore test fix 2020-07-24 20:17:35 +03:00
operator.go all: fix dots 2020-07-16 14:58:28 +00:00
peer.go storagenode: live tracking of order window usage 2020-08-19 19:42:33 +00:00