JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,141 Commits 5 Branches 0 Tags 94 MiB
646cf229a2
Commit Graph

26 Commits

Author SHA1 Message Date
Michał Niewrzał
646cf229a2 satellite/metainfo/metabase: rename FullIterateObjects method to IterateLoopObjects 
We want to have custom loop iterator to avoid reading all object fields to reduce memory consumpion. This is first step to just rename existing iterator to IterateLoopObjects.

Change-Id: I8878ff21a49ba224db2d497cc8f9076e75c7609e
 2021-03-01 17:19:05 +01:00
Michał Niewrzał
95b78e8011 satellite/metainfo: select segments in batches for metainfo loop 
Segments are not read in batches. For each batch of objects
we are reading all segments for those objects.

Change-Id: Idaf19bbe4d4b095065d59399dd326e22c57499a6
 2021-02-23 11:38:44 +00:00
Egon Elbre
adf687aebb satellite/metainfo/metabase: add full iterator 
Add FullIterateObjects that iterates over all objects in the metabase.

Change-Id: I4b8dbda32dfce2d7729e9574af79e4f20faed98a
 2021-02-18 16:51:35 +02:00
Egon Elbre
322c3a167b satellite/metainfo/metabase: add node alias table 
This is first step in supporting node aliases. It adds a table
that automatically assigns aliases to nodes inserted into the table.

Change-Id: Ibdf40097c3c1e5b371500203f8db203505a48adc
 2021-02-05 14:34:15 +02:00
Fadila Khadar
f34519c969 metainfo/metabase: iterate through pending objects by key 
Change-Id: I451453d52278a2ff15ade1a6fccede136ab314be
 2021-02-03 11:44:05 +01:00
Kaloyan Raev
b519bb377d satellite/metainfo/metabase: GetSegmentByLocation 
We need this method to fix repairing pending objects. In another PR, it
will replace the GetObjectLatestVersion + GetSegmentByPosition calls
that are currently executed.

Change-Id: I4c5c2ab604edf898452b6fd21b86d4d3f970ce79
 2021-02-01 21:52:12 +02:00
Egon Elbre
a700a1bdab satellite/metainfo/metabase: add benchmark 
Change-Id: I4289aac252f9c951548444aa5a82e007e9f47806
 2021-01-25 14:59:24 +00:00
Kaloyan Raev
ea48322dd3 satellite/metainfo: use deterministic signing for satStreamID 
So we can have stable UploadID for multipart uploads.

Change-Id: Iac6780394c8cc0f96c0b9c4b850b92ed3627a9b0
 2021-01-19 15:56:29 +00:00
Michał Niewrzał
38beecc7ad satellite/metainfo: Override pending object on upload 
On upload we need to override pending and committed object. This change is adjusting DeleteObjectAllVersions to delete both.

Change-Id: Ib66c2af207c618119f7bf0de7fa9d3e5145d8641
 2021-01-14 15:16:02 +00:00
Michal Niewrzal
66d4d5eb48 satellite/metainfo/metabase: implement IterateObjectsAllVersions for 
pending/committed objects

Change-Id: Ibf390821b6a23919078de4f18c2653e308320904
 2020-12-22 10:27:09 +01:00
Michal Niewrzal
dad8360b39 satellite/metainfo/metabase: rename IterateObjectsAllVersions to 
IterateObjectsAllVersionsWithStatus

We need different implementation for IterateObjectsAllVersions because
we want to iterate over all object without specifying object status.
Existing method will have new name but implementation details are not
changed.

Change-Id: I01b987996772fa7f8fd73da9910d52db2d1aa0d7
 2020-12-21 16:47:32 +00:00
Egon Elbre
1728e45e55 satellite/metainfo/metabase: optimize DeleteBucket 
Change-Id: If34cdeae0f688cb96717905fc8287c66ea3034be
 2020-12-16 14:39:33 +00:00
Michal Niewrzal
e7e6985ae9 satellite/metainfo/metabase: add UpdateSegmentPieces method 
We need to be able to update just remote_pieces column in DB. This is
needed at least for repair process.

Change-Id: I20dcc9b06babfefbbf102f32b1d14946379f26c2
 2020-12-14 10:25:09 +00:00
Egon Elbre
365410d10b satellite/metainfo/metabase: add commit with specifying a list of segments 
Change-Id: Ibb9999545691d150f36e20ce70ac00a3802ad7b2
 2020-12-09 15:16:38 +02:00
Fadila Khadar
fd7cc20a7e satellite/metainfo/metabase: add DeletePendingObject 
Change-Id: I7ba7d3d7f179712940e91b05501310279bbb97a4
 2020-12-03 10:21:04 +00:00
Michal Niewrzal
983b1737c2 satellite/metainfo: implement IsBucketEmpty with metabase 
Use objects iterator to check if bucket is empty.

Change-Id: I590bb7258a47dd816f079039a1151df809dd2637
 2020-11-25 12:20:30 +00:00
Kaloyan Raev
ff32268310 satellite/metainfo/metabase: DeleteExpiredObjects 
Change-Id: I16293ce1a33f714aa005022daf3d40670cc66b04
 2020-11-13 15:31:31 +02:00
Fadila Khadar
a749ac9f47 satellite/metainfo/metabase: iterateObjectsAllVersions 
Iterate through all objects in a bucket
recursive, no prefix support

Change-Id: Ieaad9fbd0b5c1593554db9d543f4ee04851bac28
 2020-11-13 13:50:08 +01:00
Michal Niewrzal
9d25b3a7d3 satellite/metainfo/metabase: add ListSegments request 
Change-Id: Ie777a0f791685beab3973e76b10a9ce76398e7d7
 2020-11-12 11:46:17 +01:00
Michal Niewrzal
92f9251074 satellite/metainfo/metabase: add GetSegmentByOffset request 
Change-Id: Iea2ce439ec1f7285e447d590c1297294204edc2e
 2020-11-10 11:33:34 +00:00
Kaloyan Raev
56e6bc884c metabase: add DeleteObjectsAllVersions method 
Change-Id: I3252dd0aac595b233f4861c3c1375622326d955d
 2020-11-05 12:17:54 +02:00
Michal Niewrzal
2398afe985 satellite/metainfo/metabase: add GetLatestObjectLastSegment request 
Change-Id: I50c452e379f78864b38921f31e53a0554642ab02
 2020-11-05 08:20:34 +00:00
Michal Niewrzal
4134100d80 satellite/metainfo/metabase: add UpdateObjectMetadata request 
Change-Id: Iec73000e58f373294f37ef3e4470412d1c5f1340
 2020-11-03 11:08:41 +00:00
Kaloyan Raev
f7aa5b4e39 metabase: CommitObject return metabase.Object 
An immediate purpose is to be used in test cases for deleting objects

Change-Id: I481d3be5800e55da231ecb628ef7eb3ea011d196
 2020-11-03 09:57:44 +00:00
Michal Niewrzal
ae17ae73d7 satellite/metainfo/metabase: add metadata and nonce to CommitObject 
request

Change-Id: I584775800963fc2167c749f3da20475036fa5325
 2020-11-02 16:49:18 +01:00
Michal Niewrzal
809eb14ac2 satellite/metainfo/metabase: move metainfo PoC into storj repo 
Change-Id: I39356d8bc7305b4a8ea0c1fb5603010ad72a68b9
 2020-10-29 09:44:55 +01:00