There are several different object types in a versioned table,
which will determine the exact behaviour.
The object type states are:
* Pending - the object is yet to be committed and is being uploaded.
* Committed - the object has been finished and can be read.
* DeleteMarker - indicates that the object should be treated as not
present when is at the top of the version stack.
There are also versioning states:
* Unversioned - only one unversioned object is allowed per object key.
* Versioned - multiple objects with the same key are allowed.
Change-Id: I65dfa781e8da253a4e5d572b799d53c351196eee
With zombie deletion chore we are removing inactive pending objects from
objects table but new we need also to do this for pending_objects table.
https://github.com/storj/storj/issues/6050
Change-Id: Ia29116c103673a1d9e10c2f16654022572210a8a
By mistake AOST was added to query in deleteInactiveObjectsAndSegments
in DeleteZombieObjects. Delete statement is not supporting it.
Unfortunately unit tests didn't cover this case. This change removes
AOST from mentioned method and it adding AOST cases to unit tests.
Change-Id: Ib7f65134290df08c490c96b7e367d12f497a3373
* optimize SQL for zombie objects deletion query by reducing some direct
selects to segments table
* set AOST for expired/zombie object deletion (was 0 since now)
https://github.com/storj/storj/issues/5881
Change-Id: I50482151d056a86fe0e31678a463f413d410759d
We noticed that in the system we have undeleted very old pending
objects. General rule is to delete them after some inactivity. Turns
out that all those objects are objects migrated to metabase from
previous DB schema. During this migration we didn't set
zombie_deletion_deadline to any value.
This change takes into account pending objects with zombie deletion
deadline set to nil during zombie deletion process.
I also checked accross all production satellites and youngest pending
objects with nil zombie_deletion_deadline are from 2021 so it is safe
to delete them.
Change-Id: Ie2b6a4b4e203c1750cf8408ee281c0631b263082
Don't terminate the expired objects loop or the zombie objects loop when
there is a DB error when selecting the objects for deleting them because
it isn't critical and the loops will pick them up again in the next
iteration.
The exception is if the DB rows scan method returns an error because
that's a symptom of the passed arguments to the method don't match with
the columns order, number, or type of the query, or there is invalid
data in the DB.
Don't also terminate these loops if the there is a DB error when
deleting the objects because the loops will pick them up in the next
iteration.
Because we don't return those errors now for not terminating the loop,
we have to log them.
Change-Id: I86bcf83d619345255840ae8f3db61620f044d2af
We would like to verify if zombie object/segment works fine.
We need some metric for that. Figuring out number of deleted
objects is harder so let's for that later.
Change-Id: Ic99e2ce93256130b7c51f514824fddc009655075
Added options flag to define after which object won't be marked as inactive. All segments CreatedAt
time needs to be bellow this flag to treat object as inactive.
Change-Id: Ib5cffc776c6ee1b62b51eb8595438f968b42528c
Currently it's difficult to gather how many objects and segments are
being inserted. Adding separate monitoring counters make this easier.
Change-Id: I986cd82f03e99d2aa6fc76028255ee1090d1b294
Currently we were duplicating code for AS OF SYSTEM TIME in several
places. This replaces the code with using a method on
dbutil.Implementation.
As a consequence it's more useful to use a shorter name for
implementation - 'impl' should be sufficiently clear in the context.
Similarly, using AsOfSystemInterval and AsOfSystemTime to distinguish
between the two modes is useful and slightly shorter without causing
confusion.
Change-Id: Idefe55528efa758b6176591017b6572a8d443e3d
