storj/satellite
Jeff Wendling 91698207cf storagenode: live tracking of order window usage
This change accomplishes multiple things:

1. Instead of having a max in flight time, which means
   we effectively have a minimum bandwidth for uploads
   and downloads, we keep track of what windows have
   active requests happening in them.

2. We don't double check when we save the order to see if it
   is too old: by then, it's too late. A malicious uplink
   could just submit orders outside of the grace window and
   receive all the data, but the node would just not commit
   it, so the uplink gets free traffic. Because the endpoints
   also check for the order being too old, this would be a
   very tight race that depends on knowledge of the node system
   clock, but best to not have the race exist. Instead, we piggy
   back off of the in flight tracking and do the check when
   we start to handle the order, and commit at the end.

3. Change the functions that send orders and list unsent
   orders to accept a time at which that operation is
   happening. This way, in tests, we can pretend we're
   listing or sending far into the future after the windows
   are available to send, rather than exposing test functions
   to modify internal state about the grace period to get
   the desired effect. This brings tests closer to actual
   usage in production.

4. Change the calculation for if an order is allowed to be
   enqueued due to the grace period to just look at the
   order creation time, rather than some computation involving
   the window it will be in. In this way, you can easily
   answer the question of "will this order be accepted?" by
   asking "is it older than X?" where X is the grace period.

5. Increases the frequency we check to send up orders to once
   every 5 minutes instead of once every hour because we already
   have hour-long buffering due to the windows. This decreases
   the maximum latency that an order will be reported back to
   the satellite by 55 minutes.

Change-Id: Ie08b90d139d45ee89b82347e191a2f8db1b88036
2020-08-19 19:42:33 +00:00
..
accounting storagenode: live tracking of order window usage 2020-08-19 19:42:33 +00:00
admin satellite/admin: add check project usage endpoint and fix some leftover http.Error handling 2020-08-13 13:48:34 +00:00
attribution all: fix dots 2020-07-16 14:58:28 +00:00
audit satellite/audit: verifier checks if node made sucess GE before auditing 2020-08-17 23:37:57 +03:00
compensation all: add missing dots 2020-08-11 17:50:01 +03:00
console all: add missing dots 2020-08-11 17:50:01 +03:00
contact all: fix dots 2020-07-16 14:58:28 +00:00
dbcleanup satellite/dbcleanup: run cleanup more frequently 2020-06-10 18:32:15 +00:00
downtime all: fix comments 2020-07-16 19:13:14 +03:00
gc all: add missing dots 2020-08-11 17:50:01 +03:00
gracefulexit all: add missing dots 2020-08-11 17:50:01 +03:00
heldamount error handling in heldamount cash and collector delete fixed 2020-07-22 12:26:13 +00:00
inspector all: add missing dots 2020-08-11 17:50:01 +03:00
mailservice all: fix dots 2020-07-16 14:58:28 +00:00
marketingweb all: fix dots 2020-07-16 14:58:28 +00:00
metainfo storagenode: live tracking of order window usage 2020-08-19 19:42:33 +00:00
metrics satellite: fix string(int) conversions 2020-06-12 06:41:34 +00:00
nodeapiversion satellite/nodeapiversion: new table for tracking node api usage 2020-07-09 15:02:25 +00:00
nodeselection all: add missing dots 2020-08-11 17:50:01 +03:00
nodestats all: fix dots 2020-07-16 14:58:28 +00:00
orders storagenode: live tracking of order window usage 2020-08-19 19:42:33 +00:00
overlay satellite/audit: verifier checks if node made sucess GE before auditing 2020-08-17 23:37:57 +03:00
payments satellite/metainfo: use user PartnerID for bucket attribution 2020-08-06 13:14:07 +00:00
referrals all: fix dots 2020-07-16 14:58:28 +00:00
repair satellite/{repair,overlay}: Use overlay NodeSelectionCache for repair uploads 2020-08-04 12:50:12 -04:00
revocation all: fix dots 2020-07-16 14:58:28 +00:00
rewards all: add missing dots 2020-08-11 17:50:01 +03:00
satellitedb all: add missing dots 2020-08-11 17:50:01 +03:00
admin.go satellite/console: make paywall optional 2020-07-28 10:57:49 +00:00
api.go satellite/piecedeletion: move node info retrieval into the service 2020-08-13 16:07:54 +00:00
config-lock_test.go satellite: add Go test version of satellite-config-lock tests 2020-05-13 08:14:24 +00:00
core.go satellite/console: make paywall optional 2020-07-28 10:57:49 +00:00
gc.go all: fix dots 2020-07-16 14:58:28 +00:00
peer.go satellite/accounting: Add chore to cleanup old project bandwidth rollups data 2020-07-22 18:56:49 +00:00
repairer.go satellite/orders: add buckets db to service 2020-07-24 17:36:49 +03:00