storj/satellite
Michał Niewrzał 7390f389cd satellite/metabase: non recursive listing optimization
With pointerdb listing objects operation was optimized to skip
objects from prefixes for non recursive listing. This change it
adopting this optimiaztion from old code.

Main change is to drop current page results if we detect a prefix
that needs to be skipped and jump with next listing query after this
prefix by setting cursor to "some-prefix(byte('/')+1) which is
effectively "some-prefix0".

Benchmark:
name                                                 old time/op  new time/op  delta
NonRecursiveListing/Postgres/listing_no_prefix-8      960µs ±11%   257µs ±12%  -73.19%  (p=0.008 n=5+5)
NonRecursiveListing/Postgres/listing_with_prefix-8    945µs ±11%   671µs ±12%  -28.97%  (p=0.008 n=5+5)
NonRecursiveListing/Cockroach/listing_no_prefix-8    4.31ms ± 8%  1.19ms ± 7%  -72.44%  (p=0.008 n=5+5)
NonRecursiveListing/Cockroach/listing_with_prefix-8  4.97ms ± 8%  3.35ms ±15%  -32.67%  (p=0.008 n=5+5)

Fixes https://github.com/storj/team-metainfo/issues/115

Change-Id: Iafdf3600d058abbaf441f792d32a7fc17cc08696
2022-07-27 11:26:35 +00:00
..
accounting satellite/metainfo: usage limits for copy 2022-07-25 12:35:27 +02:00
admin satellite/{console,satellitedb}: add PublicID to Project, db method GetByPublicID 2022-07-21 19:49:26 +00:00
analytics satellite/analytics: Track have_sales_contact differently 2022-07-22 19:22:05 +00:00
attribution {cmd/satellite/reports, satellite/attribution}: type and variable name adjustments 2022-04-26 20:12:38 +00:00
audit satellite/reputation: add a reputation write cache 2022-07-14 21:40:16 +00:00
buckets satellite/metainfo: propagate geofencing between buckets and stream id 2021-11-24 08:05:05 +00:00
compensation satellite/compensation: add a code that crypthopper-go now uses 2022-04-25 10:46:51 +00:00
console satellite/{console,satellitedb}: add PublicID to Project, db method GetByPublicID 2022-07-21 19:49:26 +00:00
contact satellite/contact: swap net.IP.IsPrivateIP with isPrivateIP 2022-06-13 01:01:44 +02:00
gc satellite: use more optimal monkit call for loop observers methods 2022-05-20 11:03:41 +00:00
geoip satellite/geoip: update node check-in to associate a country code 2021-11-10 16:44:41 +01:00
gracefulexit satellite: use more optimal monkit call for loop observers methods 2022-05-20 11:03:41 +00:00
inspector satellite/metabase: drop GetObjectLatestVersion method 2022-02-02 09:40:53 +00:00
internalpb satellite/reputation: don't need 3 identical AuditHistory types 2022-05-24 05:48:46 +00:00
mailservice satellite/{mailservice,oidc}: improvements to debugging 2022-05-11 19:59:42 +00:00
metabase satellite/metabase: non recursive listing optimization 2022-07-27 11:26:35 +00:00
metainfo satellite/metainfo: usage limits for copy 2022-07-25 12:35:27 +02:00
metrics satellite/{audit,metrics}: optimize loop methods 2022-05-05 15:10:56 +00:00
nodeapiversion satellite/nodeapiversion: new table for tracking node api usage 2020-07-09 15:02:25 +00:00
nodeselection/uploadselection satellite/repairer: handle excluded countries 2022-03-14 10:59:36 -04:00
nodestats satellite/reputation: don't need 3 identical AuditHistory types 2022-05-24 05:48:46 +00:00
oidc satellite/console: integrate sessions into satellite UI 2022-06-13 08:02:02 +00:00
orders satellite/{orders,overlay}: use cache for downloads 2022-07-12 11:04:34 +00:00
overlay satellite/reputation: add a reputation write cache 2022-07-14 21:40:16 +00:00
payments satellite: wire storjscan chore to core process 2022-07-14 15:07:52 +00:00
repair satellite/repair: avoid retrying GET_REPAIR incorrectly 2022-07-18 05:11:56 +00:00
reputation satellite/reputation: add a reputation write cache 2022-07-14 21:40:16 +00:00
revocation satellite/satellitedb: move tests to their domains 2021-02-19 17:29:15 +02:00
rewards satellite/rewards: adding SeaweedFS to partners list (#4230) 2021-10-19 21:30:31 +02:00
satellitedb satellite/{console,satellitedb}: add PublicID to Project, db method GetByPublicID 2022-07-21 19:49:26 +00:00
snopayouts all: fix error naming 2021-04-29 15:38:21 +03:00
admin.go satellite/admin: fix console config handling 2022-05-27 22:26:06 +00:00
api.go satellite/metainfo/piecedeletion: use nodes cache 2022-07-21 00:43:24 +00:00
configlock_test.go all: fix linter complaints 2020-10-13 15:59:01 +03:00
core.go satellite/satellitedb: batch delete storage node tallies 2022-07-20 15:42:35 +03:00
gc.go satellite: more detailed goroutine labels 2022-05-11 17:50:55 +00:00
peer.go satellite/payments/storjscan: add payments DB 2022-06-10 13:44:27 +01:00
repairer.go satellite/reputation: add a reputation write cache 2022-07-14 21:40:16 +00:00