method similar to metabase.DeleteObjectExactVersion which will delete last committed object
Closes https://github.com/storj/storj/issues/4872
Change-Id: Ia9f8c227dc59575bf8ed297886b35536097028b4
We are preparing to use object versions internally and to do
that we need to prepare different parts of the system to handle
object versions different than '1'. This change adjust code
responsible for server-side move and copy.
What was done:
* begin methods for move and copy are now using GetObjectLastCommitted
to find object
* results from begin move and copy operation contains now version to
be able to map object correctly with finish operation
* begin methods are putting version into satellite stream id and
finish methods are using this version as parameter instead hardcoded
value
Fixes https://github.com/storj/storj/issues/4867
Change-Id: I1380911279c21e10a3fff0342793efd2e73eafad
Currently we are tracking loop time with RunOnce method
metrics but it may give inaccurate values if loop if waiting
for observer for a ver long time. We had such case with GC
which is only loop observer in GC peer and its joning loop
every 5 days.
Change-Id: I08546c912e00c3641488de6a5d75948fe75c8e99
Updated metabase.UpdateObjectMetadata method to update set metdata always for last committed object
Closes https://github.com/storj/storj/issues/4870
Change-Id: I060683e31efcaf3e2531fea143cf0567e5ff5f73
This change reverts satellite/metabase/iterator.go of 7390f389c to the
previous version (without optimization) but leaves added benchmarks in
place.
We noticed that this optimization doesn't work and actually elevates
listing times for most buckets, hence the revert until we come up with a
better idea.
Benchmarks:
name old time/op new time/op delta
NonRecursiveListing/Postgres/listing_no_prefix-8 1.30ms ± 4% 4.52ms ± 4% +246.92% (p=0.008 n=5+5)
NonRecursiveListing/Postgres/listing_with_prefix-8 3.26ms ± 3% 4.44ms ± 2% +36.19% (p=0.008 n=5+5)
NonRecursiveListing/Cockroach/listing_no_prefix-8 618µs ± 3% 2225µs ± 2% +259.94% (p=0.008 n=5+5)
NonRecursiveListing/Cockroach/listing_with_prefix-8 1.81ms ± 5% 2.60ms ± 5% +43.96% (p=0.008 n=5+5)
Updates storj/team-metainfo#115
Change-Id: I96e4e7a563b188df478f8489027dc0042469b839
This is longest metabase at the moment and would nice to speed
up is a bit to improve overal tests execution time.
Change-Id: I86da8e0e593d20024b3ec778cbeab34a4613151f
Restored GetObjectLatestVersion and renamed it to GetObjectLastCommitted
Add test cases to cover server-side copy
Closes https://github.com/storj/storj/issues/4866
Change-Id: I343b339a60152b8fb92fda97baf80bd8fe60d631
I don't know why the go people thought this was a good idea, because
this automatic reformatting is bound to do the wrong thing sometimes,
which is very annoying. But I don't see a way to turn it off, so best to
get this change out of the way.
Change-Id: Ib5dbbca6a6f6fc944d76c9b511b8c904f796e4f3
We noticed that in the system we have undeleted very old pending
objects. General rule is to delete them after some inactivity. Turns
out that all those objects are objects migrated to metabase from
previous DB schema. During this migration we didn't set
zombie_deletion_deadline to any value.
This change takes into account pending objects with zombie deletion
deadline set to nil during zombie deletion process.
I also checked accross all production satellites and youngest pending
objects with nil zombie_deletion_deadline are from 2021 so it is safe
to delete them.
Change-Id: Ie2b6a4b4e203c1750cf8408ee281c0631b263082
Split out the function to delete a batch of objects from a bucket, so
that we get metrics which give a rough indication how long this operation
takes.
Part of https://github.com/storj/storj/issues/4957
Change-Id: I20a4ed5894217f4cd0b2f25aee297f0ecda57ab5
Don't terminate the expired objects loop or the zombie objects loop when
there is a DB error when selecting the objects for deleting them because
it isn't critical and the loops will pick them up again in the next
iteration.
The exception is if the DB rows scan method returns an error because
that's a symptom of the passed arguments to the method don't match with
the columns order, number, or type of the query, or there is invalid
data in the DB.
Don't also terminate these loops if the there is a DB error when
deleting the objects because the loops will pick them up in the next
iteration.
Because we don't return those errors now for not terminating the loop,
we have to log them.
Change-Id: I86bcf83d619345255840ae8f3db61620f044d2af
metabasetest package utils can be used by both tests and benchmarks
if we will use interface TestingT from require package. This change
adjusts metabasettest.CreateObject method
Change-Id: I3c138e2ef9873b804ab5b3402804efa409397a9f
With pointerdb listing objects operation was optimized to skip
objects from prefixes for non recursive listing. This change it
adopting this optimiaztion from old code.
Main change is to drop current page results if we detect a prefix
that needs to be skipped and jump with next listing query after this
prefix by setting cursor to "some-prefix(byte('/')+1) which is
effectively "some-prefix0".
Benchmark:
name old time/op new time/op delta
NonRecursiveListing/Postgres/listing_no_prefix-8 960µs ±11% 257µs ±12% -73.19% (p=0.008 n=5+5)
NonRecursiveListing/Postgres/listing_with_prefix-8 945µs ±11% 671µs ±12% -28.97% (p=0.008 n=5+5)
NonRecursiveListing/Cockroach/listing_no_prefix-8 4.31ms ± 8% 1.19ms ± 7% -72.44% (p=0.008 n=5+5)
NonRecursiveListing/Cockroach/listing_with_prefix-8 4.97ms ± 8% 3.35ms ±15% -32.67% (p=0.008 n=5+5)
Fixes https://github.com/storj/team-metainfo/issues/115
Change-Id: Iafdf3600d058abbaf441f792d32a7fc17cc08696
Previously there was no realtime administration of the storage usage
during copies. Now there is.
Closes https://github.com/storj/storj/issues/4719
Change-Id: I0d536bf551d16208116c3aceac89ed590ec473bf
An object copy/move is done by 2 DRPC calls. It's possible a new object was uploaded ore moved to the source location between these calls. For copy, in that case the segments end up with the wrong keys. This change adds an explicit check for that by comparing the streamId supplied by the user with the streamId in the database.
Fixes https://github.com/storj/storj/issues/4930
Change-Id: Id600456ce78fb4069b93644828a0b3eb85e23e16
Previously copying an object to it's ancestor location (copy of copy)
broke the object and all copies.
This fixes this by calling the existing delete method rather than a
custom one when there is an existing object at the copy destination.
The check for existing object at destination has been moved to an
earlier point in FinishCopy.
metabase.DeleteObject exposes a transaction parameter so that it can be
reused within metabase.
Closes https://github.com/storj/storj/issues/4707
Uplink test at https://review.dev.storj.io/c/storj/uplink/+/7557
Change-Id: I418fc3337fa9f30146ccc1db456af168ae41c326
- instead of closing over the outer err variable, potentially
overwriting some errors or something, declare local variables.
- double check that we got the number of rows we expected to get
and error otherwise. this prevents a possible source of inserting
bogus rows into the database.
Change-Id: I30662be2727afe0a90e4215a182fedc2648d1169
Part of the delete query cause a full table scan of segment_copies. This
slowed down the system. This change should have the same semantics but
improved performance.
Part of https://github.com/storj/storj/issues/4898
Change-Id: I4afe23df05467eafc9c91591f47a7251a0f3dd31
Read the source object and write the destination object in the same
transaction, to prevent breaking the object because it was deleted
simultaneously.
This is probably the root cause of the metainfo loop halting from
2022-06-21 onwards, where 2 objects lost their root_piece_id during
copying.
Part of https://github.com/storj/storj/issues/4930
Change-Id: I9c45d56a7bfb48ecd5f4906ee1cca42922901e90
To avoid regression with old versions of uplink objects move we need to
remove FinishMoveObject check for key and nonce, in SQL
in FinishMoveObject do check if metadata is nil then
don't set key and nonce. The same UPDATE clause should return metadata and
if metadata != nil we should do the same validation for key and nonce
to avoid putting broken key and nonce while doing move
Resolves: https://github.com/storj/team-metainfo/issues/108
Change-Id: If723dfad899e9235f53559b71ee1c7fe49deb8b8
TRUNCATE requires table recreation which involves 'online schema change' with crdb.
(with psql it might be fater than DROP, that was the motivation of the original change)
`online schema change` is an async operation with crdb and it's eventually very slow therefore we try to avoid it.
This·reverts·commit·15bed0ed0e81d54fe4ffac9928bdf648f5e06ec6.¬
Change-Id: I93e1ab470962be77e3458d74c8787442c9d7bee0
when deleting an object that has been copied multiple times, we look for an ancestor_stream_id by taking the min of all copies stream_id.
This change simplifies this process by picking any stream_id as a new ancestor by using 'distinct on'.
Fixes https://github.com/storj/storj/issues/4745
Change-Id: Iffb519b82d2ae2ed73af48fa0e86f87384e0158f
We are not using this method and most probably we
won't need to list objects with all statuses at once.
Removing for now.
Change-Id: I7aa0468c5f635ee2fb1fe51db382595c6343dd9c
Tests are intermittently fail with similar error:
```
--- FAIL: TestDeletePendingObject/Cockroach (15.85s)
test.go:380:
Error Trace: test.go:380
delete_test.go:221
Error: Should be zero, but was metabase.DeleteObjectResult{
Objects: []metabase.Object{
{
ObjectStream: {ProjectID: {0x0f, 0x40, 0x70, 0x41, ...}, BucketName: "txxywyg4", ObjectKey: "\xbb+$\x17\x80\xc6\xcaC\xa3\xdb\xc3z*\xa8\xbe\xaf", Version: 1, ...},
- CreatedAt: s"2022-05-20 14:40:15.995376773 +0200 CEST",
+ CreatedAt: s"2022-05-20 14:40:21.04949 +0200 CEST",
ExpiresAt: nil,
Status: 1,
... // 9 identical fields
},
},
Segments: {{RootPieceID: {0x01, 0x00, 0x00, 0x00, ...}, Pieces: {{...}}}, {RootPieceID: {0x01, 0x00, 0x00, 0x00, ...}, Pieces: {{...}}}},
}
Test: TestDeletePendingObject/Cockroach/with_segments
--- FAIL: TestDeletePendingObject/Cockroach/with_segments (0.68s)
```
Looks like we shouldn't have an assumption that all tests can be finished in 5 seconds, especially not in highly parallel environment.
These tests use `time.Now` at the beginning and compare the time saved in the database (usually filled by the database).
The difference shouldn't be higher than 20 seconds (before this commit: 5 seconds) which assumes that the records are saved in this timeframe...
Change-Id: Ia6f52897d13f88c6857c05d728bf8e72ab863c9b
If TestCommitInlineSegment tests are taking longer time
then zombieDeadline created at the beginning of test
can be too far in the past. Creating zombieDeadline for each case
should avoid flakines.
Change-Id: Ieb011e8e470f6f1c32cf9365c8ae819317de6738
TRUNCATE is faster than DELETE when deleting all rows.
As almost every metabase test case calls TestingDeleteAll, this change
should give some slight test speed-up.
Change-Id: Ib477962b6deb93edd60d6db2f1be6ede1b4b2381
Create an error class for the "pending object error" for distinguishing
it from other errors for allowing to return it as a "Not Found" DRPC
status code instead an "Internal" status code.
"Internal" errors are logged in the satellite error so this was
polluting the server logs aside of returning an inappropriate status
code.
Change-Id: I10a81adfc887c030c08a228158adc8815834b23c
We were already able to override (or not) metadata with this method
but to be explicit we are introducting new option to control storing
metadata with object. Separate option should be less error prone.
https://github.com/storj/team-metainfo/issues/105
Change-Id: I4c5bce953a633a0009b05c5ca84266ca6ceefc26
Use the same query when deleting a single object or multiple.
I have chosen not to deduplicate the row "scan" logic because
it is less complicated code and this change would expand to other
parts of the codebase.
Part of https://github.com/storj/storj/issues/4700
Change-Id: I7a958c78c903b2bddd72ca217971f7e8e02a0d0c
s3 allows for overwriting an object when using server-side copy.
This change makes overwriting the destination part of the atomic server-side copy operation so that
if copy fails, the old object is still available.
All the segments of the existing destination are deleted. If this destination object is an ancestor of another object, a new ancestor is promoted.
Fixes https://github.com/storj/storj/issues/4607
Change-Id: I85d1250850bb71867586ac230c8275a0cc1b63c3
When deleting a bucket, make sure that object copies in other buckets are
promoted to new ancestor and left in a working state.
Closes https://github.com/storj/storj/issues/4591
Change-Id: I019d916cd6de5ed51dd0dd25f47c35d0ec666af6
Uplink is fixed and now we should always get both key and nonce
or both empty.
Fixes https://github.com/storj/storj/issues/4646
Change-Id: I65dca2d4d5a10787c2fecad39e301121f1ae242a
Latest CRDB version did't work with our server-side copy deletion
query and we had to rewrite it.
Fixes https://github.com/storj/storj/issues/4655
Change-Id: I66d5c4abfc3c3f53dea8bc01ae11cd2b020e4289
Methods was never used in production and it's not sure that
it will be used at all. Let's drop it and restore if will be needed.
Fixes https://github.com/storj/storj/issues/4480
Change-Id: Ifd780d0096b67be7e72dff84bdcf1d957e0b48b5
So far we assumes that metadata key/nonce cannot be empty at all
but at some point we adjusted code to accept empty metadata/key/nonce
to save DB space.
This change is adjusting how we are processing nonce while
FinishMoveObject/FinishCopyObject. We can use storj.Nonce directly
which makes code cleaner. It's also fixing issue in FinishMoveObject
where we didn't convert nonce correctly to []byte.
Part of change is disabling validation for key and nonce until
uplink will be adjusted. We need change uplink to send always
both key and nonce or non of them. Validation will be restored
as soon as change for uplink will be merged.
https://github.com/storj/storj/issues/4646
Change-Id: Ia1772bc430ae591f54c6a9ae0308a4968aa30bed
We have an issue with latest CRDB. Single query cannot modify
the same table multiple times. Now build is blocked.
This change is unblocking build by:
* adjusting query for inserting into repair queue
* temporary removing code for deletion for server-side copy
* temporary disable backward compatibility tests for CRDB
Change-Id: Idd9744ebd228e5dc05bdaf65cfc8f779472a975d
