Commit Graph

17 Commits

Author SHA1 Message Date
Erik van Velzen
87e1528104 satellite/metabase: error if source object changed
An object copy/move is done by 2 DRPC calls. It's possible a new object was uploaded ore moved to the source location between these calls. For copy, in that case the segments end up with the wrong keys. This change adds an explicit check for that by comparing the streamId supplied by the user with the streamId in the database.

Fixes https://github.com/storj/storj/issues/4930

Change-Id: Id600456ce78fb4069b93644828a0b3eb85e23e16
2022-07-19 13:27:55 +02:00
Erik van Velzen
74f4f6e765 satellite/metabase: fix copy to ancestor location
Previously copying an object to it's ancestor location (copy of copy)
broke the object and all copies.

This fixes this by calling the existing delete method rather than a
custom one when there is an existing object at the copy destination.

The check for existing object at destination has been moved to an
earlier point in FinishCopy.

metabase.DeleteObject exposes a transaction parameter so that it can be
reused within metabase.

Closes https://github.com/storj/storj/issues/4707

Uplink test at https://review.dev.storj.io/c/storj/uplink/+/7557

Change-Id: I418fc3337fa9f30146ccc1db456af168ae41c326
2022-06-28 03:53:01 +02:00
Qweder93
04b22f24eb satellite/metabase: copy expires_at to segments while FinishCopyObject
Resolves: https://github.com/storj/storj/issues/4726

Change-Id: I0bb222b7d5d3c66fe96c78a8aa1ea8f3b0a33f2b
2022-04-27 18:39:54 +00:00
Michał Niewrzał
6e5a94698e satellite/metabase: add option to override metadata with CommitObject
We were already able to override (or not) metadata with this method
but to be explicit we are introducting new option to control storing
metadata with object. Separate option should be less error prone.

https://github.com/storj/team-metainfo/issues/105

Change-Id: I4c5bce953a633a0009b05c5ca84266ca6ceefc26
2022-04-26 08:03:52 +00:00
Fadila Khadar
3d9329a5b1 satellite/metabase: overwrite existing destination on copy
s3 allows for overwriting an object when using server-side copy.
This change makes overwriting the destination part of the atomic server-side copy operation so that
if copy fails, the old object is still available.

All the segments of the existing destination are deleted. If this destination object is an ancestor of another object, a new ancestor is promoted.

Fixes https://github.com/storj/storj/issues/4607

Change-Id: I85d1250850bb71867586ac230c8275a0cc1b63c3
2022-04-07 06:28:51 +00:00
Michał Niewrzał
2014e4655a satellite/metabase: bring back copy/move validation for key and nonce
Uplink is fixed and now we should always get both key and nonce
or both empty.

Fixes https://github.com/storj/storj/issues/4646

Change-Id: I65dca2d4d5a10787c2fecad39e301121f1ae242a
2022-04-01 07:23:22 +00:00
Michał Niewrzał
4da3dc8170 satellite/metabase: improve processing of key/nonce while finish copy/move
So far we assumes that metadata key/nonce cannot be empty at all
but at some point we adjusted code to accept empty metadata/key/nonce
to save DB space.

This change is adjusting how we are processing nonce while
FinishMoveObject/FinishCopyObject. We can use storj.Nonce directly
which makes code cleaner. It's also fixing issue in FinishMoveObject
where we didn't convert nonce correctly to []byte.

Part of change is disabling validation for key and nonce until
uplink will be adjusted. We need change uplink to send always
both key and nonce or non of them. Validation will be restored
as soon as change for uplink will be merged.

https://github.com/storj/storj/issues/4646

Change-Id: Ia1772bc430ae591f54c6a9ae0308a4968aa30bed
2022-03-28 14:34:21 +00:00
Fadila Khadar
cb37971ce4 satellite/metabase: copy of inline segments only object not in segment_copies
Fixes  https://github.com/storj/storj/issues/4561

Change-Id: I8db0f984cc94ac9b95c1d746310a3b1d49728c7a
2022-03-23 22:41:06 +01:00
Fadila Khadar
f6b4d522be satellite/metabase: flatten copy references when copying
If B is a copy of A, and C is a copy of B, then in the segment_copies table, it should appear that C is a copy of A.

Fixes https://github.com/storj/storj/issues/4538

Change-Id: I7e6b03f7584597cf616cd1e0cd0156386771d207
2022-03-20 10:29:37 +00:00
Fadila Khadar
93f8c6a159 satellite/metabase: require non-zero newStreamID for FinishCopyObject
We were not checking if we were provided an empty StreamID.
Furthermore, this changes returns the object copy with the correct createdAt field.

Change-Id: Iefc563c34ae9d8c1e233895155c1718bf905df91
2022-03-16 15:47:52 +00:00
Michał Niewrzał
ccd16bbef6 satellite/metabase: handle NewEncryptedMetadata while coping object
Copy object functionality should support setting new metadata for
copy. This change is adjusting FinishCopyObject method to set new
metadata when OverrideMetadata field is set to true.

Fixes https://github.com/storj/storj/issues/4483

Change-Id: Ica37cb57e8edae301cdc483fbda4f3ddba5d2702
2022-03-09 12:31:33 +01:00
Erik van Velzen
56d1b8bbca satellite/metabasetest: improve test copy creation
Return the newly created segments separately so we can make better
assertion to test deletion.

Change-Id: I33414df4455f87db62d486957425d4984b66c0b2
2022-03-03 13:36:52 +01:00
Erik van Velzen
7a01a2a134 satellite/metabase/metabasetest/create: return segments
Return segments when creating a test object so that it can be checked
that the correct segments are remaining after a delete action.

Change-Id: Ifc245948935ba278806e887672c03abc5f2c2654
2022-02-28 13:40:46 +00:00
Michał Niewrzał
c0297bae78 satellite/metabase: return object metadata with FinishCopyObject
Updates metadata and metainfo to return object metadata with
FinishCopyObject request.

https://github.com/storj/storj/issues/4474

Change-Id: I32cba5c20a943272e9b5964df1b3d6463ad212dc
2022-02-25 12:46:30 +00:00
Fadila Khadar
332e6afed8 satellite/metabasetest: order RawSegments by streamID and position
Change-Id: I669c0c2361c2c7989c8e946eb1fbf2f52074ce0e
2022-02-25 11:19:11 +00:00
Fadila Khadar
5c7c4fedde satellite/metabasetest: include copies in raw state.
It can be useful to compare object copies created during unit tests.
They appear in the segment_copies table as couple (stream_id, ancestor_stream_id).

Change-Id: Id335c3ff7084fe30346456d27e670aff329154ea
2022-02-18 12:48:42 +00:00
Fadila Khadar
0358a08d6b satellite/metabase: BeginCopyObject and FinishCopyObject
Part of server-side copy implementation.

Creates the methods BeginCopyObject and FinishCopyObject in satellite/metabase/copy_object.go.

This commit makes it possible to copy objects and their segments in metabase.

https://github.com/storj/team-metainfo/issues/79

Change-Id: Icc64afce16e5e0e15b83c43cff36797bb9bd39bc
2022-02-17 13:37:39 +01:00