storj/satellite
Michal Niewrzal db1409eea6 satellite/metabase: use SUBSTRING with objects iterator
Until this change we were stripping prefix from object key on satellite side. Because of that we were transferring over network unnecessary data
from DB. This change adjusts iterator SQL queries to use SUBSTRING to
remove prefix on DB side and avoid sending it to satellite.

Benchmark against 'main':
unfortunately "time/op" is very unstable while doing local bench in this
case and sometimes  there is no difference in time and sometimes its up to 18%. I never saw results when old solution is faster then new one. Results for "alloc/op" and "allocs/op" are rather consistent.

name                                                 old time/op    new time/op    delta
NonRecursiveListing/Cockroach/listing_no_prefix-8      1.98ms ± 6%    2.05ms ±23%     ~     (p=1.000 n=9+10)
NonRecursiveListing/Cockroach/listing_with_prefix-8    3.97ms ± 8%    3.42ms ±20%  -13.86%  (p=0.005 n=10+10)
NonRecursiveListing/Cockroach/listing_only_prefix-8    8.42ms ±16%    7.58ms ± 5%   -9.91%  (p=0.002 n=10+10)

name                                                 old alloc/op   new alloc/op   delta
NonRecursiveListing/Cockroach/listing_no_prefix-8      16.7kB ± 0%    16.9kB ± 0%   +1.16%  (p=0.000 n=10+10)
NonRecursiveListing/Cockroach/listing_with_prefix-8    27.3kB ± 0%    28.2kB ± 0%   +3.31%  (p=0.000 n=10+10)
NonRecursiveListing/Cockroach/listing_only_prefix-8    60.0kB ± 0%    62.4kB ± 0%   +3.93%  (p=0.000 n=10+8)

name                                                 old allocs/op  new allocs/op  delta
NonRecursiveListing/Cockroach/listing_no_prefix-8         312 ± 0%       315 ± 0%   +0.96%  (p=0.000 n=10+10)
NonRecursiveListing/Cockroach/listing_with_prefix-8       526 ± 0%       541 ± 0%   +2.85%  (p=0.000 n=10+10)
NonRecursiveListing/Cockroach/listing_only_prefix-8     1.16k ± 0%     1.23k ± 0%   +5.24%  (p=0.000 n=10+10)

Change-Id: I23e501494ededafb2dd5ea903e8e4e313b42e956
2022-10-10 14:27:26 +00:00
..
accounting satellite/{accounting/tally,metabase}: remove monkit for fast methods 2022-10-05 08:38:48 +00:00
admin satellite/admin/ui: use a valid go.mod stub 2022-09-28 12:41:30 +00:00
analytics satellite/analytics: Added analytics for user behavior on Billing page (#5205) 2022-09-29 09:20:24 +03:00
attribution {cmd/satellite/reports, satellite/attribution}: type and variable name adjustments 2022-04-26 20:12:38 +00:00
audit satellite/audit: optimize loop observer 2022-10-02 22:24:37 +00:00
buckets satellite/metainfo: propagate geofencing between buckets and stream id 2021-11-24 08:05:05 +00:00
compensation all: reformat comments as required by gofmt 1.19 2022-08-10 18:24:55 +00:00
console web/satellite: increment login failed_login_count in sql 2022-10-07 16:46:29 +00:00
contact satellite/contact: swap net.IP.IsPrivateIP with isPrivateIP 2022-06-13 01:01:44 +02:00
gc satellite/gc/sender: concurrency 2022-09-27 08:10:10 +00:00
geoip satellite/geoip: update node check-in to associate a country code 2021-11-10 16:44:41 +01:00
gracefulexit satellite: fix segment loop observers metrics 2022-08-10 14:13:16 +00:00
inspector {satellite/metabase, satellite/metainfo, satellite/inspector} : Use metabase.GetObjectLastCommitted instead metabase.GetObjectExactVersion 2022-09-08 07:27:22 +00:00
internalpb satellite/internalpb: new protobuf for retain filters 2022-08-30 12:37:40 +02:00
mailservice satellite/consoleweb: fix flaky TestAuth tests 2022-08-04 19:06:07 +00:00
metabase satellite/metabase: use SUBSTRING with objects iterator 2022-10-10 14:27:26 +00:00
metainfo satellite/metainfo: use multiple object versions internally 2022-10-06 15:19:02 +00:00
metrics satellite: fix segment loop observers metrics 2022-08-10 14:13:16 +00:00
nodeapiversion satellite/nodeapiversion: new table for tracking node api usage 2020-07-09 15:02:25 +00:00
nodeselection/uploadselection satellite/repairer: handle excluded countries 2022-03-14 10:59:36 -04:00
nodestats satellite: return interval_end_time in DailyStorageUsage endpoint 2022-07-27 18:24:27 +00:00
oidc satellite/oidc: include nodeURL in oidc configuration 2022-09-13 13:46:30 +00:00
orders satellite/orders: remove unused service dependency 2022-09-28 08:56:36 +00:00
overlay satellite/gc/bloomfilter: use int64 to count pieces 2022-09-22 09:39:53 +00:00
payments satellite/payments/stripecoinpayments/service.go: add date to pay invoices command 2022-10-03 21:28:37 -07:00
repair satellite/repair/checker: remove monitoring from fast methods 2022-10-03 12:15:03 +00:00
reputation satellite/reputation: reconfigure lambda and alpha 2022-08-17 18:52:53 +00:00
revocation satellite/satellitedb: move tests to their domains 2021-02-19 17:29:15 +02:00
rewards satellite/rewards: adding SeaweedFS to partners list (#4230) 2021-10-19 21:30:31 +02:00
satellitedb web/satellite: increment login failed_login_count in sql 2022-10-07 16:46:29 +00:00
snopayouts all: fix error naming 2021-04-29 15:38:21 +03:00
admin.go satellite/payments/stripecoinpayments: storjscan invoice generation 2022-08-03 13:24:26 -07:00
api.go satellite/orders: remove unused service dependency 2022-09-28 08:56:36 +00:00
configlock_test.go all: fix linter complaints 2020-10-13 15:59:01 +03:00
core.go satellite/orders: remove unused service dependency 2022-09-28 08:56:36 +00:00
gc-bf.go satellite/gc/bloomfilter: add service to collect bloom filters 2022-09-02 08:10:46 +00:00
gc.go satellite/gc/sender: new service to send retain filters 2022-09-20 11:49:40 +00:00
peer.go satellite/gc/sender: new service to send retain filters 2022-09-20 11:49:40 +00:00
repairer.go satellite/orders: remove unused service dependency 2022-09-28 08:56:36 +00:00