Commit Graph

16 Commits

Author SHA1 Message Date
Egon Elbre
7e72a231c2 satellite/metainfo/metabase: cast bucket_name to []byte
Currently our metabase assumption is that it may contain arbitrary
bucket names and endpoint applies the naming constraints as it sees fit.
However by passing bucket_name as TEXT pg and crdb automatically try to
convert it to []byte, which may or not may work as intended... or in
some cases not work at all.

Cast all bucket name arguments to []byte to make it work.

Change-Id: I44650f5c873010997398bb0163d7f56ff6d9b5cf
2021-03-02 13:22:49 +02:00
Egon Elbre
ba0197a9b7 satellite/metainfo/metabase: avoid full table scan
Change-Id: Id47ffb5e1287d9303ce6fb530e87dbdc23cf8307
2021-03-01 23:34:53 +02:00
Egon Elbre
994854dcda satellite/metainfo/metabase: use key with prefix as the next cursor
We shouldn't use a trimmed object key as the next cursor.

Change-Id: I8513f7d94ed366dff9e9eca732c0e1ded6f5cd71
2021-02-28 22:05:42 +02:00
Egon Elbre
261a4c1c09 satellite/metainfo/metabase: fix iterator boundaries
Currently the old encrypted keys may not match the path component
encoding. Change the iterator such that the prefixes handle arbitrary
byte sequences.

Change-Id: I0a50049f4ef9887e1c4df6f9692f967a054430eb
2021-02-28 21:19:51 +02:00
Fadila Khadar
f34519c969 metainfo/metabase: iterate through pending objects by key
Change-Id: I451453d52278a2ff15ade1a6fccede136ab314be
2021-02-03 11:44:05 +01:00
Michal Niewrzal
66d4d5eb48 satellite/metainfo/metabase: implement IterateObjectsAllVersions for
pending/committed objects

Change-Id: Ibf390821b6a23919078de4f18c2653e308320904
2020-12-22 10:27:09 +01:00
Michal Niewrzal
dad8360b39 satellite/metainfo/metabase: rename IterateObjectsAllVersions to
IterateObjectsAllVersionsWithStatus

We need different implementation for IterateObjectsAllVersions because
we want to iterate over all object without specifying object status.
Existing method will have new name but implementation details are not
changed.

Change-Id: I01b987996772fa7f8fd73da9910d52db2d1aa0d7
2020-12-21 16:47:32 +00:00
Michal Niewrzal
83e7cd2a40 satellite/metainfo/metabase: replace SQL substring with go code
We have some issues with SUBSTRING function on cockroachdb so for now we
are removing it from SQL query and replacing with go code.

Change-Id: I5be921211067d42e7d1a4997076bcfdbed9617a1
2020-12-02 10:47:18 +01:00
Michal Niewrzal
cfb45a785c satellite/metainfo/metabase: add TotalPlainSize to objects table
We need to know how large is object in general and we shouldn't query
segments for that.

Change-Id: Ie316e1ec38bfc39065e6046b79b55d58f3ce9d53
2020-11-25 14:35:28 +01:00
Egon Elbre
72fed3b3fb satellite/metainfo/metabase: add slow non-recursive listing
Change-Id: Ide60640400caea6474465af5224010fde60826c7
2020-11-17 16:47:29 +02:00
Egon Elbre
2e426ef0a8 satellite/metainfo/metabase: iterator compatibility
Old iterator returns object keys without prefixes, this helps to reduce
the bandwidth from the database. The endpoint also doesn't send the
prefixes.

Change-Id: I77d85dae671ee3a16abe75db14e19674e80abaf4
2020-11-17 15:50:39 +02:00
Fadila Khadar
c9bbd83f96 satellite/metainfo/metabase: align object status to those already in use
Change-Id: I33b2ce7282838e1172b7cb92d4e76169c96fddcb
2020-11-17 14:07:42 +01:00
Michal Niewrzal
b7df41a0a1 satellite/metainfo/metabase: add EncryptedMetainfoEncryptedKey parameter
to metabase

* EncryptedMetainfoEncryptedKey added to CommitSegment and
UpdateMetadata request
* EncryptedMetainfoEncryptedKey returned with GetObject response and all
delete responses
* EncryptedMetainfoEncryptedKey returned with object iterator results

Change-Id: I917541ab5f3e1863bc8f238d17a15fbf72a23025
2020-11-17 10:19:12 +00:00
Egon Elbre
402cfcb7c6 satellite/metainfo/metabase: add prefix to iterator
Change-Id: I2dd7b7917aed9def0be3f131ca60ab12f2077d83
2020-11-16 17:38:59 +02:00
Egon Elbre
7aba265db0 satellite/metainfo/metabase: add status option to iteration
Change-Id: Ie9d1a3704d8d08e3421ba2244a1f3be57d15ed8f
2020-11-16 15:41:20 +02:00
Fadila Khadar
a749ac9f47 satellite/metainfo/metabase: iterateObjectsAllVersions
Iterate through all objects in a bucket
recursive, no prefix support

Change-Id: Ieaad9fbd0b5c1593554db9d543f4ee04851bac28
2020-11-13 13:50:08 +01:00