Commit Graph

13 Commits

Author SHA1 Message Date
Michał Niewrzał
9d52451464 satellite/metainfo/expireddeletion: add config value to set list limit
We want to control how many expired objects we will query at once.

Change-Id: Iba7c79a6236c6b720b53bb8f078d55514690d8e5
2021-03-23 09:51:56 +00:00
Fadila Khadar
a8b5f8f990 satellite/metainfo/metabase: select then delete expired objects
We were deleting expired objects by directly executing a delete query.
With this change, we first select the objects to be deleted and then
delete them (as recommended by cockroachdb for deleting using a non indexed
column).

Change-Id: Ied150fbdc7031a343a74e0b9dab316598188ef66
2021-03-22 17:39:13 +01:00
Michał Niewrzał
65957c7525 satellite/metainfo/expireddeletion: temporarily log errors instead failing whole system
At the moment we are trying to optimize deletion queries but its hard to verify deletion performance. Until we are sure that the queries are good we will just log errors instead shutting down whole satellite core.

Change-Id: I5625251d4518c35f0d46d6bf37b2f3ea7950675e
2021-03-15 16:00:20 +01:00
Michal Niewrzal
dad8360b39 satellite/metainfo/metabase: rename IterateObjectsAllVersions to
IterateObjectsAllVersionsWithStatus

We need different implementation for IterateObjectsAllVersions because
we want to iterate over all object without specifying object status.
Existing method will have new name but implementation details are not
changed.

Change-Id: I01b987996772fa7f8fd73da9910d52db2d1aa0d7
2020-12-21 16:47:32 +00:00
Michal Niewrzal
218bbeaffa Merge 'master' branch
Change-Id: Ica5c25607a951076dd9f77e35e308062f71ce3f0
2020-12-07 15:05:52 +01:00
Stefan Benten
494bd5db81
all: golangci-lint v1.33.0 fixes (#3985) 2020-12-05 17:01:42 +01:00
Kaloyan Raev
46839b992a satellite/metainfo: expired deletion service to use Metabase
It also sets the Expires time correctly in the Metabase when uploading
objects.

Change-Id: Iec1b3ec8d11346a91dfc2ba23a1b08edec4a84d3
2020-12-01 09:12:14 +00:00
Kaloyan Raev
92a2be2abd satellite/metainfo: get away from using pb.Pointer in Metainfo Loop
As part of the Metainfo Refactoring, we need to make the Metainfo Loop
working with both the current PointerDB and the new Metabase. Thus, the
Metainfo Loop should pass to the Observer interface more specific Object
and Segment types instead of pb.Pointer.

After this change, there are still a couple of use cases that require
access to the pb.Pointer (hence we have it as a field in the
metainfo.Segment type):
1. Expired Deletion Service
2. Repair Service

It would require additional refactoring in these two services before we
are able to clean this.

Change-Id: Ib3eb6b7507ed89d5ba745ffbb6b37524ef10ed9f
2020-10-27 13:06:47 +00:00
Michal Niewrzal
9202295348 satellite/metainfo: replace ScopedPath with metabase.SegmentLocation
Change-Id: I7e89c9e8eaeae58be828a32ad47ed3028501f4c7
2020-09-04 10:06:52 +00:00
Michal Niewrzal
aa47e70f03 satellite/metainfo: use metabase.SegmentKey with metainfo.Service
Instead of using string or []byte we will be using dedicated type
SegmentKey.

Change-Id: I6ca8039f0741f6f9837c69a6d070228ed10f2220
2020-09-03 15:11:32 +00:00
Egon Elbre
94a09ce20b all: add missing dots
Change-Id: I93b86c9fb3398c5d3c9121b8859dad1c615fa23a
2020-08-11 17:50:01 +03:00
Egon Elbre
080ba47a06 all: fix dots
Change-Id: I6a419c62700c568254ff67ae5b73efed2fc98aa2
2020-07-16 14:58:28 +00:00
Moby von Briesen
178aa8b5e0 satellite/{metainfo,repair}: Delete expired segments from metainfo
* Delete expired segments in expired segments service using metainfo
loop
* Add test to verify expired segments service deletes expired segments
* Ignore expired segments in checker observer
* Modify checker tests to verify that expired segments are ignored
* Ignore expired segments in segment repairer and drop from repair queue
* Add repair test to verify that a segment that expires after being
added to the repair queue is ignored and dropped from the repair queue

Change-Id: Ib2b0934db525fef58325583d2a7ca859b88ea60d
2020-04-22 13:02:31 +00:00