Commit Graph

11 Commits

Author SHA1 Message Date
Egon Elbre
41e16bc398 satellite/metabase: versioning, add new statuses
There are several different object types in a versioned table,
which will determine the exact behaviour.

The object type states are:
* Pending   - the object is yet to be committed and is being uploaded.
* Committed - the object has been finished and can be read.
* DeleteMarker - indicates that the object should be treated as not
  present when is at the top of the version stack.

There are also versioning states:
* Unversioned - only one unversioned object is allowed per object key.
* Versioned - multiple objects with the same key are allowed.

Change-Id: I65dfa781e8da253a4e5d572b799d53c351196eee
2023-10-16 17:50:18 +00:00
Michal Niewrzal
ae2cba1d23 satellite/metabase: add IteratePendingObjectsByKeyNew method
New metabase method IteratePendingObjectsByKeyNew to iterate
over entries in pending_objects table with the same object key.
Implementation and tests are mostly copy of code for
IteratePendingObjectsByKey. Main difference is that pending_objects
table have StreamID column part of primary key instead Version.

Method will be used to support new table in
metainfo.ListPendingObjectStreams request.

After full transition to pending_objects table we should remove 'New'
suffix from methods names.

Part of https://github.com/storj/storj/issues/6047

Change-Id: Ifc1ecbc534f8510fbd70c4ec676cf2bf8abb94cb
2023-08-21 08:08:03 +00:00
Michal Niewrzal
929dc80091 satellite/metabase: add IteratePendingObjects method
New metabase method IteratePendingObjects to iterate over entries in
pending_objects table. Implementation and tests are mostly copy of
code that we are using to iterate over objects table. Main difference
is that pending_objects table have StreamID column part of primary
key instead Version. Also structure of pending object is smaller
than the one from object table but it's a detail.

Method will be used to support new table in metainfo.ListObjects
request.

Next step will be to port rest of iterator implementation to support
pending_objects table in metainfo.ListPendingObjectStreams.

Part of https://github.com/storj/storj/issues/6047

Change-Id: Ia578182f88840539f3668d4a242953e061eace02
2023-08-21 08:07:16 +00:00
Michał Niewrzał
c934f45bfc satellite/metabase: drop IterateObjectsAllVersions
We are not using this method and most probably we
won't need to list objects with all statuses at once.
Removing for now.

Change-Id: I7aa0468c5f635ee2fb1fe51db382595c6343dd9c
2022-05-26 14:12:20 +00:00
Erik van Velzen
91158fd3bf satellite: omit system metadata fields if not requested
To reduce database load, add option to omit as many object properties
as possible when listing objects.

Change-Id: I817633801b00629a4042d1d1bd2389ee581953de
2021-11-08 09:19:44 +01:00
Michał Niewrzał
1fdb0eaa5b Revert "satellite/metabase: use storj.Nonce instead []byte"
This change introduce problems with server side move so
let's revert it for now. Problem was found when latest
version of storj/storj was used in uplink tests.

This reverts commit 1ef06fae99.

Change-Id: I4d4fad5d1ea04ba15ff9d7bd765f7e078e9187c2
2021-10-12 15:39:54 +02:00
Michał Niewrzał
1ef06fae99 satellite/metabase: use storj.Nonce instead []byte
We were using mixed types for nonce fields. Protobuf
have storj.Nonce, metabase have []byte. This change
is a refactoring to have everywere its possible only
storj.Nonce.

Change-Id: Id54bd8481f30c721cdaf3df79206d25e7cfdab55
2021-10-11 16:13:34 +00:00
Clement Sam
6613133a1c satellite/metabase: return metadata from IterateObjectsAllVersionsWithStatus on demand
Change-Id: I6bb717cf941e2b5fd7c411bd1e5a77fad0835995
2021-08-06 16:20:39 +00:00
Michał Niewrzał
2e9d3a737c satellite/metabase: fix listing prefixes with cursor set
We were not skipping the initial prefix from the cursor.

Change-Id: I2bb472e960b92cae77fd1226de0b26fac79c429b
2021-07-07 14:30:46 +00:00
Michał Niewrzał
033006403f satellite/metainfo: fix setting object ZombieDeletionDeadline
So far we were setting ZombieDeletionDeadline alwasy as nil and because of that DB default was never set. This change adds separate query for inserting object if deadline is not set.

Change-Id: I3d6a16570e7c74b5304e13edad8c7adcd021340c
2021-05-11 07:26:42 +00:00
Egon Elbre
267506bb20 satellite/metabase: move package one level higher
metabase has become a central concept and it's more suitable for it to
be directly nested under satellite rather than being part of metainfo.

metainfo is going to be the "endpoint" logic for handling requests.

Change-Id: I53770d6761ac1e9a1283b5aa68f471b21e784198
2021-04-21 15:54:22 +03:00