Commit Graph

2 Commits

Author SHA1 Message Date
Michal Niewrzal
41bcc6bb62 satellite/metainfo: fix duplicates while listing committed objects
We have an issue where object can appear in two different listing pages.
It's because protobuf listing cursor doesn't have version included and
now we can have internally versions higher than 1. On satellite side
version 1 was always used as a default cursor version.

As a workaround for existing implementation of libuplink library we will
use always maximum version for listing cursor on satellite side.

Fixing protobuf and libuplink implementation will happen later.

https://github.com/storj/storj/issues/5570

Change-Id: Ibd27b174556c9d8b8bd60fab8cff7862fd11e994
2023-02-14 14:47:27 +01:00
Fadila Khadar
7788170234 satellite/metabase: ListObjects
Current metainfo.ListObjects implementation is using metabase iterator to list objects.
In the non-recursive case, it used to retrieve all the corresponding rows and then discarded the entries that did not fit the listing request.
This can lead in some edge cases (each prefix contains more than batchsize objects/sub-prefixes) to make unecessary calls to the db.

This change defines the metabase.ListObjects and aims at retrieving only prefixes (but not objects under it) and objects by modifying the SQL query.

In this version, it is not optimized on the database side. Cockroach will still have to go through all rows under a prefix, so there is still room for improvement.

metainfo.ListObjects is not currently using this method as we would like to assess its performance on the QA satellite first.

Fixes https://github.com/storj/storj/issues/5088

Change-Id: Ied3a9210871871d9d4a3096888d3e40c2dceed61
2022-09-20 11:15:44 +00:00