JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,860 Commits 5 Branches 0 Tags 94 MiB
c4578eb3ec
Commit Graph

286 Commits

Author SHA1 Message Date
Michal Niewrzal
57f374af24 Merge 'master' branch 
Change-Id: Idf6b10ea7ca94e4d232e6a3b6a38ef2e646ba197
 2020-12-15 08:26:53 +01:00
Qweder93
c630037f34 multinodepb: diskspace, reputation and status added 
Change-Id: I470fa8b59ce7f00f2fbedbd0c0878fb5fff0590c
 2020-12-14 17:01:16 +00:00
Michal Niewrzal
b3acc1101a Merge 'master' branch 
Change-Id: Iee99400c7095770e61cde94b3b2c8eb0ddec463d
 2020-12-10 15:42:52 +01:00
Egon Elbre
588dff6d78 Upgrade to uplink v1.4.2 
Change-Id: Ib1f5e70c76710d9c49a39347e4ed95bd4671efab
 2020-12-10 15:47:11 +02:00
Kaloyan Raev
3269401515 deps: update to latest uplink from multipart-upload branch 
This resolves an issue in Uplink CLI with listing size 0 for files
uploaded with multipart upload.

Change-Id: I80e0b11a96f87ed6a87eb5301034c08dbc09e8aa
 2020-12-07 14:54:51 +00:00
Michal Niewrzal
218bbeaffa Merge 'master' branch 
Change-Id: Ica5c25607a951076dd9f77e35e308062f71ce3f0
 2020-12-07 15:05:52 +01:00
Caleb Case
c1bdb88d3c
Upgrade to uplink v1.4.1 
Change-Id: Ie823f9b72d41f3da8e929d96a870fffc2e7ec12b
 2020-12-04 11:38:19 -05:00
Caleb Case
6ba8dd58e6 Update to uplink v1.4.0 
Change-Id: Ia01fec012031ceb2867b80a37e6a6c2a7d179cca
 2020-12-02 13:44:32 +00:00
Michal Niewrzal
5a7bc9657d Merge 'master' branch 
Change-Id: If583132a821274dc4b78cf5f72b853ba8460c619
 2020-11-30 12:57:22 +01:00
Michal Niewrzal
21602e0494 satellite/metainfo: enable commented test 
Test was commented to make uplink refactoring possible. Now we can bring
back this test.

Change-Id: I0511b76073efaafed8aac97f8e845dcec93dd059
 2020-11-30 10:49:23 +01:00
Michal Niewrzal
cfb45a785c satellite/metainfo/metabase: add TotalPlainSize to objects table 
We need to know how large is object in general and we shouldn't query
segments for that.

Change-Id: Ie316e1ec38bfc39065e6046b79b55d58f3ce9d53
 2020-11-25 14:35:28 +01:00
Michal Niewrzal
46102c1942 satellite/metainfo: change ListSegments required permission to 
Read/Download

We should treat listing segments as a part of download process, not
listing.

Change-Id: Iae72c97c77976c427f668a96486af613d58da3b9
 2020-11-24 17:50:33 +01:00
Michal Niewrzal
5be4d413b0 satellite/metainfo: set PlainSize for inline segment 
Plain size is required by metabase implementation.

Change-Id: I6739744bff4ce485e0066f2a49acdf38560d31fb
 2020-11-20 13:54:14 +01:00
Kaloyan Raev
05fe497a6f satellite/metainfo: add stream id when listing pending objects 
Change-Id: Idcdffd70b82c0fe4f79635fc193604b66101ef76
 2020-11-19 14:22:09 +02:00
Michal Niewrzal
9bdc8ebf75 satellite/metainfo: adjust GetObject to handle 
EncryptedMetadataEncryptedKey

Change-Id: If159fdcc9f37e9449a7d8459a277366d6d6b5802
 2020-11-17 16:38:10 +00:00
Michal Niewrzal
cd7d911b26 satellite/metainfo: implement ListSegments with metabase 
Change-Id: I78fb2400fed0bb01c5e05bd5bbcddef5e4a3bdf0
 2020-11-16 08:33:35 +00:00
Michal Niewrzal
3fe16f4003 satellite/metainfo: upload/download with metabase 
This change is adjusting metainfo endpoint to use metabase for uploading
and downloading remote objects. Inline segments will be added later.

Change-Id: I109d45bf644cd48096c47361043ebd8dfeaea0f3
 2020-11-11 12:13:52 +00:00
Michal Niewrzal
7dde184cb5 Merge 'master' branch 
Change-Id: I6070089128a150a4dd501bbc62a1f8b394aa643e
 2020-11-10 11:58:59 +00:00
Egon Elbre
b892a00143 mod: bump dependencies and reenable test 
We shouldn't have any EOF issues with recent drpc fix, let's reenable
and see whether it's still flaky.

Change-Id: I0de312bcb087c7f70ec9d3281d73d86f971845d5
 2020-11-10 10:32:21 +00:00
jessicagreben
c4c29e370a wasm: add webassembly code for creating access grant in console web UI 
Change-Id: I3c6d9afc660f3d959d6138db84341e9460b877a1
 2020-11-04 12:08:30 -08:00
Egon Elbre
716068a1e0 Merge branch 'master'. 
Change-Id: Ic14325edc291573582dce0cea3e04991a820b48b
 2020-11-02 13:02:01 +02:00
Egon Elbre
9674ce40e6 mod: bump dependencies 
Change-Id: I61beaf9793609f7614478cfa604b2721d9010b52
 2020-10-30 16:17:55 +02:00
Egon Elbre
995900e02f Merge branch 'master' into multipart-upload 
Change-Id: I8717f9f7cc54f3934402a98f21129e16841b25bd
 2020-10-30 11:48:05 +02:00
Michal Niewrzal
8f26f66da0 internalpb: move satellite specific protobuf types storj/storj 
We have some types that are only valid for satellite usage. Such types
are SatStreamID and SatSegmentID. This change moves those types to
storj/storj and adds basic infrastructure for generating code.

Change-Id: I1e643844f947ce06b13e51ff16b7e671267cea64
 2020-10-30 08:49:16 +00:00
Qweder93
624255e8ba storagenode/secret: db tests added, small renaming fixes added 
Change-Id: I7eae1a9a64c20a39c97e81fa741cfc9b9e1e615a
 2020-10-29 14:23:04 +02:00
Michal Niewrzal
809eb14ac2 satellite/metainfo/metabase: move metainfo PoC into storj repo 
Change-Id: I39356d8bc7305b4a8ea0c1fb5603010ad72a68b9
 2020-10-29 09:44:55 +01:00
Caleb Case
be84616e69
Update to uplink v1.3.1 
Change-Id: Iae1e6a63c00d07fee9047638ddc9a414780ce0e9
 2020-10-14 09:39:29 -04:00
Egon Elbre
2f4bb114d4 go.mod: bump common to remove sha256-simd 
Change-Id: I77aeb3b4bea87e8d5bb83b05e86b61a80a695f2a
 2020-10-14 12:47:37 +03:00
Caleb Case
cf1748158a
Bump Dependencies 
Change-Id: I4c8a4438e74379a490a19f1f88ea9dac7715dbbd
 2020-10-09 09:33:49 -04:00
Yingrong Zhao
f13084983d bump common to latest 
Change-Id: I55ca40a7b2c58d70225da07b7a62d5ad60451881
 2020-10-01 15:39:09 -04:00
Yingrong Zhao
c085a17a52 bump common and uplink to latest 
Change-Id: I717f0214dd9973acd51b7732c5d64587f610c805
 2020-10-01 15:38:58 +00:00
Egon Elbre
139ff3427b go.mod: bump dependencies 
Change-Id: I2b1bf014a60ef6ddc4b1b546f58393c34ffe22fc
 2020-09-30 11:28:37 +03:00
Egon Elbre
c23a8e3b81 go.mod: update pgx to v4.9.0 
Fix query to use TextArray instead of VarcharArray.
Fix queries to use the correct type.

Change-Id: Ibb7e55adba277d05778118d81ca697470e72c374
 2020-09-29 19:03:08 +00:00
Kaloyan Raev
b409b53f7f cmd/satellite: command for verifying piece hashes 
Jira: https://storjlabs.atlassian.net/browse/PG-69

There are a number of segments with piece_hashes_verified = false in
their metadata) on US-Central-1, Europe-West-1, and Asia-East-1
satellites. Most probably, this happened due to a bug we had in the
past. We want to verify them before executing the main migration to
metabase. This would simplify the main migration to metabase with one
less issue to think about.

Change-Id: I8831af1a254c560d45bb87d7104e49abd8242236
 2020-09-29 10:58:24 +00:00
Egon Elbre
888bfaae4b cmd/satellite: only add google profiler to satellite 
Previously uplink, storagenode etc. included google cloud profiler,
however they don't need it.

Change-Id: Ibc95cb03d667a3844672eecd49fa455a6acc3866
 2020-09-25 18:56:59 +03:00
JT Olio
1a094fd6da storj: bump storj.io/private 
Change-Id: Ia2927010332bb2e1d140603299341a8549093233
 2020-09-10 16:21:12 -06:00
Egon Elbre
b45cad5eed bump common to latest 
This makes tests pass on Go 1.15.

Change-Id: I8407f16dcfc0b306aa685da30dcfedea6d3b515d
 2020-09-05 10:51:58 +03:00
Yingrong Zhao
cccd62fff9 bump uplink version to v1.3.0 
Change-Id: I9e7833713ca00f19bfb662eeaf61ef57bdc3bbbd
 2020-09-04 09:57:35 -04:00
Yingrong Zhao
af773ec8a6 cmd/uplink: use DeleteBucketWithObjects for bucket force deletion 
This PR updates `uplink rb --force` command to use the new libuplink API
`DeleteBucketWithObjects`.
It also updates `DeleteBucket` endpoint to return a specific error
message when a given bucket has concurrent writes while being deleted.

Change-Id: Ic9593d55b0c27b26cd8966dd1bc8cd1e02a6666e
 2020-09-02 16:39:20 +00:00
Qweder93
36d752e92d storagenode/reputation: offline_under_review_at added 
Change-Id: Ia7ec79b2d6f20fe29de0c36223f9485380d2845c
 2020-09-02 18:48:28 +03:00
Qweder93
7d9897b7af storagenode/nodestats: online_score added 
Change-Id: I84b50a6cace306e5f10d53a2073fe8810d4d2960
 2020-09-02 17:45:01 +03:00
Isaac Hess
ba1a113e2e satellite/metainfo: Add test for GetObjectIPs 
This PR adds the integration test for GetObjectIPs, testing all repos.

Change-Id: Id4d4f19c427a2e320b2a82efa68150fa7f5f86fb
 2020-08-31 10:27:56 -06:00
Cameron Ayer
3e343b683b cmd/segment-reaper: add metrics for zombie segments count 
Change-Id: I106c6795946283165ba3de8465e5898346da1a3f
 2020-08-26 18:42:59 +00:00
Yingrong Zhao
bd5213f68b satellite/metainfo: implement batch delete for DeleteBucket 
This PR changes DeleteBucket to be able to delete all objects within a
bucket if `DeleteAll` is set in `BucketDeleteRequest`.
It also changes `DeleteBucket` API to treat `ErrBucketNotFound` as a
successful delete operation instead of returning an error back to the
client.

Change-Id: I3a22c16224c7894f2d0c2a40ba1ae8717fa1005f
 2020-08-24 13:28:09 +00:00
Isaac Hess
a14887e20b satellite/metainfo: Add GetObjectIPs method 
This adds the unimplemented GetObjectIPs method to metainfo endpoint so
we can import new common protobuf definitions.

Change-Id: I154f26baccb6bb3c66de3eb25611930545c9754b
 2020-08-11 11:39:15 -06:00
Caleb Case
00f9882ad5 bump to uplink v1.2.0 
Also fixup config and related tests for storj.io/common
bee93690ab909580993f402b594ad5a14760c39f

Change-Id: I6897d925cd28b139c1fc2b43e63b08c0cc3fb688
 2020-08-10 17:11:09 +00:00
Qweder93
4ee1b2d45a storagenode/console: added list of all audits per satellite to sno dashboard/satellites 
Change-Id: I52e58748d6467f372d9a308347fc77e400d137e2
 2020-08-10 12:55:07 +00:00
Caleb Case
7735b797c0
Bump dependencies for various fixes. 
Change-Id: I79e4a985aef38f48ef49683eb19ce9dc0b3a2f85
 2020-07-29 11:27:35 -04:00
Jeff Wendling
bdb3a10d39 mod: bump drpc 
Change-Id: I584a2f55a391ce26e9fb1675e9f8a678febe649f
 2020-07-29 14:24:07 +00:00
JT Olio
c89746a383 storagenode: use hardcoded metric application name 
Change-Id: If11f226d03bc4bf70e2465569f8d578750718ae7
 2020-07-28 16:24:15 +00:00