7b06575f6f
This combines the ListStreamPositions and GetSegmentByPosition calls with a ListSegments call that now knows how to return only the segments within a Range, just like ListStreamPositions. It would theoretically be possible to also include the GetObjectLastCommitted call by having it do one of three queries based on the incoming request range, but that would mean duplicating the data for the object in every single row that is returned for each segment in the range. One gross thing that ListSegments has to do now is update the first segment returned with the information from any ancestor segment because GetSegmentByPosition used to do that. It only updates the first segment so that it doesn't do O(N) database queries. It seems difficult to have it do a single query to update all of the segments at once. I'm not certain this change should be merged on this basis alone. This change has made me think a couple of things should happen: 1. Server side copy with ancestor segments strikes again making the code less clear and potentially more buggy or inefficient for a rare case (empirically <0.1%) 2. The download code requests individual segments from the satellite lazily as part of its download which requires the satellite telling it the locations of all of the segments which requires the satellite querying the locations of all of the segments. Instead the download RPC could return the orders for all of the segments for a range and the download code could issue N download calls rather than 1 download call and N get segment calls. I believe both sides of the code paths would be simpler and more efficient this way. 3. In looking at the timing information for downloads when testing this, we really need to focus on getting the auth key and bandwidth limit verification times down. Here's the timing I saw: - 42ms: validate auth - 52ms: bandwidth usage checking - 14ms: get object info - 26ms: get segment position info - 26ms: getting the first segment full info - 20ms: unaccounted for by spans - 6ms: creating the orders This change will remove 26ms, but there's a good 90ms in just validation. With improved semantics hitting the database only once and improved validation, a download rpc taking ~30ms seems doable compared to our current ~200ms. Change-Id: I4109dba082eaedb79e634c61dbf86efa93ab1222 |
||
---|---|---|
.. | ||
abtesting | ||
accounting | ||
admin | ||
analytics | ||
attribution | ||
audit | ||
buckets | ||
compensation | ||
console | ||
contact | ||
gc | ||
geoip | ||
gracefulexit | ||
inspector | ||
internalpb | ||
mailservice | ||
metabase | ||
metainfo | ||
metrics | ||
nodeapiversion | ||
nodeevents | ||
nodeselection/uploadselection | ||
nodestats | ||
oidc | ||
orders | ||
overlay | ||
payments | ||
repair | ||
reputation | ||
revocation | ||
satellitedb | ||
snopayouts | ||
admin.go | ||
api.go | ||
auditor.go | ||
configlock_test.go | ||
core.go | ||
gc-bf_test.go | ||
gc-bf.go | ||
gc.go | ||
peer.go | ||
rangedloop.go | ||
repairer.go |