JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7788170234
storj/satellite
History
Fadila Khadar 7788170234 satellite/metabase: ListObjects 
Current metainfo.ListObjects implementation is using metabase iterator to list objects.
In the non-recursive case, it used to retrieve all the corresponding rows and then discarded the entries that did not fit the listing request.
This can lead in some edge cases (each prefix contains more than batchsize objects/sub-prefixes) to make unecessary calls to the db.

This change defines the metabase.ListObjects and aims at retrieving only prefixes (but not objects under it) and objects by modifying the SQL query.

In this version, it is not optimized on the database side. Cockroach will still have to go through all rows under a prefix, so there is still room for improvement.

metainfo.ListObjects is not currently using this method as we would like to assess its performance on the QA satellite first.

Fixes https://github.com/storj/storj/issues/5088

Change-Id: Ied3a9210871871d9d4a3096888d3e40c2dceed61
2022-09-20 11:15:44 +00:00
..
accounting satellite/accounting: use single call to get project limits 2022-09-13 21:50:52 +00:00
admin satellite/admin: add Burst field to Update Project Limits params 2022-08-17 10:40:28 -04:00
analytics satellite/analytics: Re-added user click events in Access Management flow (#5151) 2022-09-06 09:55:33 -07:00
attribution {cmd/satellite/reports, satellite/attribution}: type and variable name adjustments 2022-04-26 20:12:38 +00:00
audit satellite/reputation: reconfigure lambda and alpha 2022-08-17 18:52:53 +00:00
buckets satellite/metainfo: propagate geofencing between buckets and stream id 2021-11-24 08:05:05 +00:00
compensation all: reformat comments as required by gofmt 1.19 2022-08-10 18:24:55 +00:00
console satellite/oidc: include nodeURL in oidc configuration 2022-09-13 13:46:30 +00:00
contact satellite/contact: swap net.IP.IsPrivateIP with isPrivateIP 2022-06-13 01:01:44 +02:00
gc satellite/gc/bloomfilter: take CreationDate from latest segment 2022-09-15 11:59:53 +00:00
geoip satellite/geoip: update node check-in to associate a country code 2021-11-10 16:44:41 +01:00
gracefulexit satellite: fix segment loop observers metrics 2022-08-10 14:13:16 +00:00
inspector {satellite/metabase, satellite/metainfo, satellite/inspector} : Use metabase.GetObjectLastCommitted instead metabase.GetObjectExactVersion 2022-09-08 07:27:22 +00:00
internalpb satellite/internalpb: new protobuf for retain filters 2022-08-30 12:37:40 +02:00
mailservice satellite/consoleweb: fix flaky TestAuth tests 2022-08-04 19:06:07 +00:00
metabase satellite/metabase: ListObjects 2022-09-20 11:15:44 +00:00
metainfo satellite/metainfo: fix deletion tests 2022-09-12 11:23:53 +00:00
metrics satellite: fix segment loop observers metrics 2022-08-10 14:13:16 +00:00
nodeapiversion satellite/nodeapiversion: new table for tracking node api usage 2020-07-09 15:02:25 +00:00
nodeselection/uploadselection satellite/repairer: handle excluded countries 2022-03-14 10:59:36 -04:00
nodestats satellite: return interval_end_time in DailyStorageUsage endpoint 2022-07-27 18:24:27 +00:00
oidc satellite/oidc: include nodeURL in oidc configuration 2022-09-13 13:46:30 +00:00
orders satellite/{orders,overlay}: use cache for downloads 2022-07-12 11:04:34 +00:00
overlay cmd/tools/segment-verify: wire up overlay logic 2022-09-19 11:32:18 +03:00
payments satellite/payments/stripecoinpayments: update invoicing to use credit note 2022-09-16 13:40:38 +00:00
repair storagenode/satellite: support different piece hash algorithms 2022-08-23 18:15:06 +00:00
reputation satellite/reputation: reconfigure lambda and alpha 2022-08-17 18:52:53 +00:00
revocation satellite/satellitedb: move tests to their domains 2021-02-19 17:29:15 +02:00
rewards satellite/rewards: adding SeaweedFS to partners list (#4230) 2021-10-19 21:30:31 +02:00
satellitedb satellite/payments/monetary: remove pkg and all its references 2022-09-07 12:58:00 +00:00
snopayouts all: fix error naming 2021-04-29 15:38:21 +03:00
admin.go satellite/payments/stripecoinpayments: storjscan invoice generation 2022-08-03 13:24:26 -07:00
api.go satellitedb: add token balance to API endpoint 2022-08-31 22:55:16 +00:00
configlock_test.go all: fix linter complaints 2020-10-13 15:59:01 +03:00
core.go satellite/payments/storjscan: add billing update to chore 2022-08-16 11:18:15 -07:00
gc-bf.go satellite/gc/bloomfilter: add service to collect bloom filters 2022-09-02 08:10:46 +00:00
gc.go satellite: more detailed goroutine labels 2022-05-11 17:50:55 +00:00
peer.go satellite/gc/bloomfilter: add service to collect bloom filters 2022-09-02 08:10:46 +00:00
repairer.go satellite/reputation: offset write times by random, not by satelliteID 2022-08-03 21:14:06 +00:00