JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
704cad8da4
storj/satellite/metabase
History
Michał Niewrzał 260a673327 satellite/metabase: don't delete pieces until query results are received 
To optimize memory consumption we where consuming
segment data during processing results from delete
query. Turns out that there is a chance that query will be
rolled-back if something will go wrong while reading
results. In such case its possible to delete pices but
object/segment will be still in DB.
This change removed piece deletion from problematic
place. Pieces are still deleted in batches but are not
limited at the moment. To avoid memory issues object
deletion batch was decreased.

Change-Id: Icb3667220f9c25f64b73cf71d0cf3fdc7e5107c5
2021-08-08 03:43:17 +00:00
..
metabasetest segment/{metabase,repair}: change segment created_at column to not accept nulls 2021-08-06 08:16:28 +00:00
segmentloop satellite/metabase/segmentloop: don't shutdown satellite on loop error 2021-07-30 06:49:10 +00:00
alias_test.go ci: fix lint errors 2021-05-17 13:37:31 +00:00
alias.go satellite/metabase: drop alias migration code 2021-05-07 12:13:28 +03:00
aliascache_test.go satellite/metabase/metabasetest: move test utils to a separate package 2021-04-26 13:12:58 +03:00
aliascache.go satellite/metabase: move package one level higher 2021-04-21 15:54:22 +03:00
aliaspiece_test.go satellite/metabase: move package one level higher 2021-04-21 15:54:22 +03:00
aliaspiece.go satellite/metabase: move package one level higher 2021-04-21 15:54:22 +03:00
bench_expired_test.go satellite/metabase/metabasetest: move test utils to a separate package 2021-04-26 13:12:58 +03:00
bench_test.go satellite/metabase: return metadata from IterateObjectsAllVersionsWithStatus on demand 2021-08-06 16:20:39 +00:00
commit_object_test.go segment/{metabase,repair}: change segment created_at column to not accept nulls 2021-08-06 08:16:28 +00:00
commit_object.go satellite/metabase: use db methods that pass context correctly 2021-07-29 09:32:31 +00:00
commit_test.go segment/{metabase,repair}: change segment created_at column to not accept nulls 2021-08-06 08:16:28 +00:00
commit.go satellite/metabase: use db methods that pass context correctly 2021-07-29 09:32:31 +00:00
common_test.go segment/{metabase,repair}: add dedicated methods on metabase.Pieces 2021-08-03 15:12:03 +00:00
common.go segment/{metabase,repair}: add dedicated methods on metabase.Pieces 2021-08-03 15:12:03 +00:00
consts_test.go satellite/metabase: move package one level higher 2021-04-21 15:54:22 +03:00
db_internal_test.go satellite/metabase/segmentloop: limit max interval 2021-06-10 17:06:33 +00:00
db_test.go satellite/metabase: drop alias migration code 2021-05-07 12:13:28 +03:00
db.go segment/{metabase,repair}: change segment created_at column to not accept nulls 2021-08-06 08:16:28 +00:00
delete_bucket_test.go satellite/metabase: don't delete pieces until query results are received 2021-08-08 03:43:17 +00:00
delete_bucket.go satellite/metabase: don't delete pieces until query results are received 2021-08-08 03:43:17 +00:00
delete_objects_test.go segment/{metabase,repair}: change segment created_at column to not accept nulls 2021-08-06 08:16:28 +00:00
delete_objects.go satellite/metabase: add intLimitRange 2021-07-19 07:51:16 +00:00
delete_part_test.go satellite/metabase: add DeletePart method 2021-05-11 22:45:43 +02:00
delete_part.go satellite/metabase: add monitoring for objects and segments 2021-06-04 20:01:10 +03:00
delete_test.go segment/{metabase,repair}: change segment created_at column to not accept nulls 2021-08-06 08:16:28 +00:00
delete.go satellite/metabase: use db methods that pass context correctly 2021-07-29 09:32:31 +00:00
encoding.go satellite/metabase: add intLimitRange 2021-07-19 07:51:16 +00:00
get_test.go segment/{metabase,repair}: change segment created_at column to not accept nulls 2021-08-06 08:16:28 +00:00
get.go satellite/metabase: return metadata from IterateObjectsAllVersionsWithStatus on demand 2021-08-06 16:20:39 +00:00
iterator_test.go satellite/metabase: return metadata from IterateObjectsAllVersionsWithStatus on demand 2021-08-06 16:20:39 +00:00
iterator_util_test.go satellite/metabase: fix listing prefixes with cursor set 2021-07-07 14:30:46 +00:00
iterator.go satellite/metabase: return metadata from IterateObjectsAllVersionsWithStatus on demand 2021-08-06 16:20:39 +00:00
list_segments_test.go segment/{metabase,repair}: change segment created_at column to not accept nulls 2021-08-06 08:16:28 +00:00
list_segments.go satellite/metabase: use db methods that pass context correctly 2021-07-29 09:32:31 +00:00
list.go satellite/metabase: return metadata from IterateObjectsAllVersionsWithStatus on demand 2021-08-06 16:20:39 +00:00
loop_test.go segment/{metabase,repair}: change segment created_at column to not accept nulls 2021-08-06 08:16:28 +00:00
loop.go segment/{metabase,repair}: change segment created_at column to not accept nulls 2021-08-06 08:16:28 +00:00
metadata_test.go satellite/{metabase,metainfo}: require StreamID for UpdateObjectMetadata 2021-07-08 17:50:37 +03:00
metadata.go satellite/{metabase,metainfo}: require StreamID for UpdateObjectMetadata 2021-07-08 17:50:37 +03:00
raw.go segment/{metabase,repair}: change segment created_at column to not accept nulls 2021-08-06 08:16:28 +00:00
stats_test.go satellite/metabase: add method to get table statistics 2021-06-16 18:34:55 +03:00
stats.go satellite/metabase: use db methods that pass context correctly 2021-07-29 09:32:31 +00:00
streamstat_test.go satellite/metabase/metabasetest: move test utils to a separate package 2021-04-26 13:12:58 +03:00
streamstat.go satellite/metabase: use db methods that pass context correctly 2021-07-29 09:32:31 +00:00
update_test.go segment/{metabase,repair}: change segment created_at column to not accept nulls 2021-08-06 08:16:28 +00:00
update.go satellite/metabase: use db methods that pass context correctly 2021-07-29 09:32:31 +00:00
util.go satellite/metabase: add intLimitRange 2021-07-19 07:51:16 +00:00