JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,621 Commits 5 Branches 0 Tags 94 MiB
5be4d413b0
Commit Graph

4621 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Michal Niewrzal
5be4d413b0 satellite/metainfo: set PlainSize for inline segment 
Plain size is required by metabase implementation.

Change-Id: I6739744bff4ce485e0066f2a49acdf38560d31fb
 2020-11-20 13:54:14 +01:00
Kaloyan Raev
05fe497a6f satellite/metainfo: add stream id when listing pending objects 
Change-Id: Idcdffd70b82c0fe4f79635fc193604b66101ef76
 2020-11-19 14:22:09 +02:00
Michal Niewrzal
1465539220 satellite/metainfo: ignore empty inline segments for backward 
compatibility

Change-Id: I2567d77e172391945dee24cea8e110fa1963d663
 2020-11-18 14:12:03 +00:00
Michal Niewrzal
70b1c7aa17 satellite/metainfo: check if EncryptedMetadataNonce was set 
It possible that old uplinks won't set it.

Change-Id: Ifd3c4d4a45c5636cfeb70b0a37bcf1305a097727
 2020-11-18 14:04:31 +01:00
Kaloyan Raev
a30f5d7ec0 satellite/metainfo: use Object.Status from ObjectListRequest 
Change-Id: Ie2e48d2eedf69bd0f04b7c9d6e7e217fec6b0267
 2020-11-18 13:31:52 +02:00
Michal Niewrzal
9bdc8ebf75 satellite/metainfo: adjust GetObject to handle 
EncryptedMetadataEncryptedKey

Change-Id: If159fdcc9f37e9449a7d8459a277366d6d6b5802
 2020-11-17 16:38:10 +00:00
Egon Elbre
72fed3b3fb satellite/metainfo/metabase: add slow non-recursive listing 
Change-Id: Ide60640400caea6474465af5224010fde60826c7
 2020-11-17 16:47:29 +02:00
Egon Elbre
5e7d47b9cd satellite/metainfo: code cleaning 
Rename iterator tests to iterator_test.go.
Add Delimiter to metabase package.

Change-Id: Icea6542b54176d716daba4afad901713f1dcac90
 2020-11-17 15:50:40 +02:00
Egon Elbre
2e426ef0a8 satellite/metainfo/metabase: iterator compatibility 
Old iterator returns object keys without prefixes, this helps to reduce
the bandwidth from the database. The endpoint also doesn't send the
prefixes.

Change-Id: I77d85dae671ee3a16abe75db14e19674e80abaf4
 2020-11-17 15:50:39 +02:00
Kaloyan Raev
402a581b86 ci: fail build if metabase tests fail 
Change-Id: Ic263ad2dcf777d1fb7c03bbddc9edd09a2165c83
 2020-11-17 13:30:15 +00:00
Fadila Khadar
c9bbd83f96 satellite/metainfo/metabase: align object status to those already in use 
Change-Id: I33b2ce7282838e1172b7cb92d4e76169c96fddcb
 2020-11-17 14:07:42 +01:00
Michal Niewrzal
b7df41a0a1 satellite/metainfo/metabase: add EncryptedMetainfoEncryptedKey parameter 
to metabase

* EncryptedMetainfoEncryptedKey added to CommitSegment and
UpdateMetadata request
* EncryptedMetainfoEncryptedKey returned with GetObject response and all
delete responses
* EncryptedMetainfoEncryptedKey returned with object iterator results

Change-Id: I917541ab5f3e1863bc8f238d17a15fbf72a23025
 2020-11-17 10:19:12 +00:00
Egon Elbre
402cfcb7c6 satellite/metainfo/metabase: add prefix to iterator 
Change-Id: I2dd7b7917aed9def0be3f131ca60ab12f2077d83
 2020-11-16 17:38:59 +02:00
Egon Elbre
7aba265db0 satellite/metainfo/metabase: add status option to iteration 
Change-Id: Ie9d1a3704d8d08e3421ba2244a1f3be57d15ed8f
 2020-11-16 15:41:20 +02:00
Kaloyan Raev
a015f41927 satellite/metainfo: ListObjects to use Metabase API 
Change-Id: If75d04b9a44f08515be717a85f397b66f8a5c56f
 2020-11-16 13:27:17 +02:00
Michal Niewrzal
7c384c8293 Merge 'master' branch 
Change-Id: I1eefd5a56449e577820977d61fa4a22bdd4fc230
 2020-11-16 10:02:54 +01:00
Michal Niewrzal
cd7d911b26 satellite/metainfo: implement ListSegments with metabase 
Change-Id: I78fb2400fed0bb01c5e05bd5bbcddef5e4a3bdf0
 2020-11-16 08:33:35 +00:00
Kaloyan Raev
ff32268310 satellite/metainfo/metabase: DeleteExpiredObjects 
Change-Id: I16293ce1a33f714aa005022daf3d40670cc66b04
 2020-11-13 15:31:31 +02:00
Fadila Khadar
a749ac9f47 satellite/metainfo/metabase: iterateObjectsAllVersions 
Iterate through all objects in a bucket
recursive, no prefix support

Change-Id: Ieaad9fbd0b5c1593554db9d543f4ee04851bac28
 2020-11-13 13:50:08 +01:00
Egon Elbre
8182d8a726 satellite/metainfo/metabase: use txutil.WithTx in delete 
Change-Id: I75965c4dcf57479ace2367f5d3069d785628e86a
 2020-11-12 14:38:36 +02:00
Egon Elbre
bc460cd62d satellite/metainfo/metabase: use txutil.WithTx in commit 
Change-Id: I84347291179c7042e22bda703a1c9bd80a1513a0
 2020-11-12 14:38:33 +02:00
Michal Niewrzal
9d25b3a7d3 satellite/metainfo/metabase: add ListSegments request 
Change-Id: Ie777a0f791685beab3973e76b10a9ce76398e7d7
 2020-11-12 11:46:17 +01:00
Michal Niewrzal
fd707392e7 satellite/metainfo: set RedundancyScheme for begin segment 
response

Multipart upload needs it for uploading parts.

Change-Id: I34886a9dabd17ecb7d55d4c6e19914a1c821f638
 2020-11-11 17:25:52 +00:00
Kaloyan Raev
9dcd0936d7 satellite/metainfo: MakeInlineSegment to use metabase 
Change-Id: Ib32c1d9e91e3f13b4e44abefb2b5d0352b64bd79
 2020-11-11 15:24:21 +02:00
Michal Niewrzal
3fe16f4003 satellite/metainfo: upload/download with metabase 
This change is adjusting metainfo endpoint to use metabase for uploading
and downloading remote objects. Inline segments will be added later.

Change-Id: I109d45bf644cd48096c47361043ebd8dfeaea0f3
 2020-11-11 12:13:52 +00:00
Fadila Khadar
2bd239bb75 satellite/metainfo/metabase: tests commit inline segment 
Change-Id: Ifad461041c7b9d8c78fc9f998a4e708853087b02
 2020-11-10 15:00:35 +00:00
Michal Niewrzal
48f1f7132e satellite/metainfo: set RS defaults 
While resolving conflicts with `master` I missed this change which is
needed e.g. to run storj-sim.

Change-Id: I56a548ed92b978510526c26c81af03051acfde2f
 2020-11-10 13:57:27 +01:00
Michal Niewrzal
7dde184cb5 Merge 'master' branch 
Change-Id: I6070089128a150a4dd501bbc62a1f8b394aa643e
 2020-11-10 11:58:59 +00:00
Michal Niewrzal
92f9251074 satellite/metainfo/metabase: add GetSegmentByOffset request 
Change-Id: Iea2ce439ec1f7285e447d590c1297294204edc2e
 2020-11-10 11:33:34 +00:00
Kaloyan Raev
3ed4183e52 satellite/metainfo: delete object to use metabase 
Change-Id: I2ab63a719fdbc1f8a7fbb4ad73d51a2d2dcfadc6
 2020-11-10 09:55:23 +00:00
Michal Niewrzal
dbf9f121ed ci: mark failures as unstable 
Change-Id: I3cfa07a35cb789f409d95d4778c07ca3355c0f3c
 2020-11-09 16:48:38 +01:00
Egon Elbre
184e1ffa9d satellite/metainfo/metabase: commit object without proofs 
Change-Id: I788111b9e5bdc961be9e731f8203e52ca9da9dca
 2020-11-09 15:18:07 +00:00
Egon Elbre
754838f5d0 ci: convert test failures to unstable 
Change-Id: Ib0252c666c086379bd6e2a3a47011479fe3988ed
 2020-11-09 11:18:12 +00:00
Egon Elbre
0a77deec81 satellite/metainfo/metabase: basic migrate 
Change-Id: Ia34f38e763af9ba88f75038b8ecfc9eb6550eae0
 2020-11-05 16:31:02 +02:00
Kaloyan Raev
56e6bc884c metabase: add DeleteObjectsAllVersions method 
Change-Id: I3252dd0aac595b233f4861c3c1375622326d955d
 2020-11-05 12:17:54 +02:00
Michal Niewrzal
2398afe985 satellite/metainfo/metabase: add GetLatestObjectLastSegment request 
Change-Id: I50c452e379f78864b38921f31e53a0554642ab02
 2020-11-05 08:20:34 +00:00
Kaloyan Raev
f986fdfe72 metabase: optimize delete segments 
Segments are deleted in a single DB query instead of a separate one for
each segment.

Change-Id: Icb9ae92b4b8405030c9d9ef57ba30bee6a35515c
 2020-11-04 11:59:56 +02:00
Michal Niewrzal
4134100d80 satellite/metainfo/metabase: add UpdateObjectMetadata request 
Change-Id: Iec73000e58f373294f37ef3e4470412d1c5f1340
 2020-11-03 11:08:41 +00:00
Kaloyan Raev
89cefa9045 metabase: add Objects info to DeleteObjectResult 
Change-Id: If7ed2ed0bbba08ad971c64852d51ff1a066c4f1e
 2020-11-03 12:45:49 +02:00
Kaloyan Raev
f7aa5b4e39 metabase: CommitObject return metabase.Object 
An immediate purpose is to be used in test cases for deleting objects

Change-Id: I481d3be5800e55da231ecb628ef7eb3ea011d196
 2020-11-03 09:57:44 +00:00
Kaloyan Raev
7e9ac2ec0a metabase: use storj.ErrObjectNotFound instead of custom error 
Change-Id: Ie90e438f7d7b58e6ea986359849dd8319433ae9f
 2020-11-03 09:35:22 +00:00
Michal Niewrzal
ae17ae73d7 satellite/metainfo/metabase: add metadata and nonce to CommitObject 
request

Change-Id: I584775800963fc2167c749f3da20475036fa5325
 2020-11-02 16:49:18 +01:00
Egon Elbre
716068a1e0 Merge branch 'master'. 
Change-Id: Ic14325edc291573582dce0cea3e04991a820b48b
 2020-11-02 13:02:01 +02:00
Egon Elbre
fd8e697ab2 {satellite,storagenode}/internalpb: use specific package name 
Ensure we don't register types with the same name into protobuf.

Change-Id: I53d025863fff8c91a067ca5819befa87eb5e35bb
 2020-10-30 17:31:08 +02:00
Egon Elbre
9674ce40e6 mod: bump dependencies 
Change-Id: I61beaf9793609f7614478cfa604b2721d9010b52
 2020-10-30 16:17:55 +02:00
Michal Niewrzal
0205f0d807 satellite/metainfo: fix usage of types from internalpb 
After moving SatStreamID and SatSegmentID from common I missed changing
some methods in metainfo endpoint. This change is a fix for that.

Change-Id: I34e121fce47371ee4cfd92cce03809520b68859f
 2020-10-30 16:03:45 +02:00
Egon Elbre
77c4f99fa0 satellite/internalpb: move delegated_repair.proto 
Change-Id: If4f37c52b151e09cf35d2145b463ef1e9ab529ae
 2020-10-30 15:31:32 +02:00
Egon Elbre
11338e9beb satellite/internalpb: move audithistory.pb 
Change-Id: I8eee84d49ed90459168ddaf04ae57f790c2a22c4
 2020-10-30 15:30:11 +02:00
Egon Elbre
1903b15474 storagenode/internalpb: move gracefulexit.proto 
Change-Id: Ia3614846ed49a39c8f39331516d16d45a695240b
 2020-10-30 15:24:56 +02:00
Michal Niewrzal
517212bfa8 satellite/metainfo: fix usage of types from internalpb 
After moving SatStreamID and SatSegmentID from common I missed changing
some methods in metainfo endpoint. This change is a fix for that.

Change-Id: I3344623dc7acfa73db6c20cd3212301e74335857
 2020-10-30 13:20:58 +00:00