Commit Graph

25 Commits

Author SHA1 Message Date
Michał Niewrzał
bc161794fc satellite/metabase: drop DeleteObjectLatestVersion method
This method was never used, except tests.

Change-Id: Idc1e69b2e2971995b5c4e6cf78a2b5fc69f39ad2
2022-02-02 14:33:48 +00:00
Michał Niewrzał
5041ee0abc satellite/metabase: drop GetObjectLatestVersion method
We don't have different version of object than 1 so at the moment
this method is not needed. Also using GetObjectExactVersion
should be slightly more performant.

Change-Id: I78235d8ae22594cc1d6345dabcc915f41cd7797b
2022-02-02 09:40:53 +00:00
Michał Niewrzał
60d03b9e42 satellite/metainfo: drop GetSegmentByLocation
Method is not used anymore. We need it during transition
from metainfo loop into segment loop.

Change-Id: Id32590ff9b02e06025efce1c32869e2cf9cd0bc5
2021-12-16 16:32:45 +01:00
Michał Niewrzał
de8464e848 satellite/metabase: add GetProjectSegmentCount method
This new method will be used with mechanism to limit number
of segments per project, similar to limiting buckets or bandwidth.
This is only one of multiple changes we will do to implement this
limitation.

Change-Id: Ia516c2a006ad1d7b4431d780679be9d809848f4b
2021-11-24 13:20:02 +00:00
Qweder93
50baefa10e satellite/metabase: limit maximum number of parts and size
Multipart upload limits added. Last part has no size limit.
Max number of parts: 10000, min part size: 5 MiB

Change-Id: Ic2262ce25f989b34d92f662bde720d4c4d0dc93d
2021-10-22 10:16:37 +00:00
Michał Niewrzał
f5f2398431 satellite/metabase: remove code related to part deletion
It is not used currently.

Change-Id: I32e9fa357814a3159ee8659a607cd6986f361fbf
2021-10-19 10:11:49 +00:00
Michał Niewrzał
1fdb0eaa5b Revert "satellite/metabase: use storj.Nonce instead []byte"
This change introduce problems with server side move so
let's revert it for now. Problem was found when latest
version of storj/storj was used in uplink tests.

This reverts commit 1ef06fae99.

Change-Id: I4d4fad5d1ea04ba15ff9d7bd765f7e078e9187c2
2021-10-12 15:39:54 +02:00
Michał Niewrzał
1ef06fae99 satellite/metabase: use storj.Nonce instead []byte
We were using mixed types for nonce fields. Protobuf
have storj.Nonce, metabase have []byte. This change
is a refactoring to have everywere its possible only
storj.Nonce.

Change-Id: Id54bd8481f30c721cdaf3df79206d25e7cfdab55
2021-10-11 16:13:34 +00:00
crawter
5e4b196b11 satellite/metainfo: finish move object
Second method needed to perform server-side move. It updates
metadata key and nonce and all segments key and nonces.

Change-Id: Ia43b26622a13048269f0ae9e1524b345db112adb
2021-09-20 16:21:01 +03:00
crawter
07311cdd6f satellite/metainfo: begin move object added
First from two methods needed to perform server-side move. It gets
metadata key and nonce and all segments key and nonces and returns
all of that to uplink.

Change-Id: Ied2c79559e77d3f63091c4d61948f2d6a2147d67
2021-09-14 13:18:21 +00:00
Michał Niewrzał
67a33e4c06 satellite/metabase: drop unused IterateLoopStreams method
We were using this method for metainfo loop but now
segment loop is not using it so we can drop it.

Change-Id: I60c5b4f86a619259906d8c2ba76e665b8715be75
2021-09-09 14:12:44 +00:00
Clement Sam
1f353f3231 segment/{metabase,repair}: change segment created_at column to not accept nulls
This change adds a NOT NULL constraint to the created_at column in the segment table.
All occurrences of CreatedAt as a pointer are changed to non pointer version (metabase, segment loop, etc)

Change-Id: I3efd476ebd1edd3327b69c9223d9edc800e1cc52
2021-08-06 08:16:28 +00:00
Kaloyan Raev
a767aed591
satellite/{metabase,metainfo}: require StreamID for UpdateObjectMetadata
This avoids corrupting objects if reuploads and metadata updates happen
concurrently.
2021-07-08 17:50:37 +03:00
Kaloyan Raev
a6086d421f satellite/metabase: implement SetObjectMetadataLatestVersion
It replaces UpdateMetadataObject that hasn't been used anywhere yet.

Change-Id: I1b2131acd75924c487a1c3ca099674251c5c9b56
2021-07-06 01:06:16 +00:00
Egon Elbre
0df4a27bf7 satellite/metabase: add method to get table statistics
In loop we need to start verifying it's correctness. This allows to
gather these stats.

Change-Id: I146fb50e2b3658b6f3c2682cdc1983e6abd73c29
2021-06-16 18:34:55 +03:00
JT Olio
6949dc0bac satellite/metaloop: missing monitoring on observers
Change-Id: I630fbb0448c8d08b426486b3e49abfbca03332a6
2021-06-15 13:39:13 +00:00
Michał Niewrzał
9a113da361 satellite/metabase: expose expires_at with loops
Loops are using custom structs to provide
segments while iterating/looping so we need
to expose new field there too.

Change-Id: I12c8f4a01afeac171bf638d278253999fa90a8cb
2021-06-11 16:51:37 +00:00
Michał Niewrzał
bc79f01aaa satellite/metabase: set expires_at while committing segment
We added expires_at column to segments table and now
we need to populate this column while committing segment.
We still need to migrate existing segments with
separate tool.

Change-Id: Ibac8c63d97201dd98cc2cb9db385f4cb73bc3f7e
2021-06-10 16:32:28 +00:00
Michał Niewrzał
3af91e7a90 satellite/metabase: add iteration over segments
Method IterateLoopSegments can be used to iterate over all segments in metabase without touching objects.

Change-Id: I3cc0e783884b603b47ef3f8233e357aa8a391250
2021-05-26 09:59:33 +00:00
Michał Niewrzał
547a6e9930 satellite/metabase: add DeletePart method
Method will delete all segments for given part.

Change-Id: I11bdfdf30176749aee03cea0d1d2fa467f7cebea
2021-05-11 22:45:43 +02:00
Egon Elbre
2ae80690cb satellite/metabase/metabasetest: remove STORJ_TEST_DATABASES
Initially metabase was developed separately and it was useful to have a
separate environment flag for tests, however, it's more convenient to
use the same as rest of the testsuite.

Change-Id: Ia4d79be27ce5911cbae68d57cdf0b30f63459444
2021-05-11 13:31:01 +00:00
Michał Niewrzał
033006403f satellite/metainfo: fix setting object ZombieDeletionDeadline
So far we were setting ZombieDeletionDeadline alwasy as nil and because of that DB default was never set. This change adds separate query for inserting object if deadline is not set.

Change-Id: I3d6a16570e7c74b5304e13edad8c7adcd021340c
2021-05-11 07:26:42 +00:00
Egon Elbre
6161436d8b satellite/metabase/metabasetest: sort results
Deletion results are not guaranteed to be sorted, hence we need to sort
them before comparing.

Change-Id: I6b571dd812d4256327c6c64fff74c780c62b2948
2021-05-07 12:36:02 +03:00
Michał Niewrzał
e8ef689929 satellite/metabase: add DeleteZombieObjects method
We will use this method to delete objects where zombie_deletion_deadline passed.

Change-Id: Id24322e40dad5b8bceb0f84080922c746c0b802d
2021-05-04 07:08:16 +00:00
Egon Elbre
948529a600 satellite/metabase/metabasetest: move test utils to a separate package
Change-Id: Iaa607996478497d5c97c6a6973d099b315b3abf0
2021-04-26 13:12:58 +03:00