JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,593 Commits 5 Branches 0 Tags 94 MiB
92f9251074
Commit Graph

4593 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Michal Niewrzal
92f9251074 satellite/metainfo/metabase: add GetSegmentByOffset request 
Change-Id: Iea2ce439ec1f7285e447d590c1297294204edc2e
 2020-11-10 11:33:34 +00:00
Kaloyan Raev
3ed4183e52 satellite/metainfo: delete object to use metabase 
Change-Id: I2ab63a719fdbc1f8a7fbb4ad73d51a2d2dcfadc6
 2020-11-10 09:55:23 +00:00
Michal Niewrzal
dbf9f121ed ci: mark failures as unstable 
Change-Id: I3cfa07a35cb789f409d95d4778c07ca3355c0f3c
 2020-11-09 16:48:38 +01:00
Egon Elbre
184e1ffa9d satellite/metainfo/metabase: commit object without proofs 
Change-Id: I788111b9e5bdc961be9e731f8203e52ca9da9dca
 2020-11-09 15:18:07 +00:00
Egon Elbre
754838f5d0 ci: convert test failures to unstable 
Change-Id: Ib0252c666c086379bd6e2a3a47011479fe3988ed
 2020-11-09 11:18:12 +00:00
Egon Elbre
0a77deec81 satellite/metainfo/metabase: basic migrate 
Change-Id: Ia34f38e763af9ba88f75038b8ecfc9eb6550eae0
 2020-11-05 16:31:02 +02:00
Kaloyan Raev
56e6bc884c metabase: add DeleteObjectsAllVersions method 
Change-Id: I3252dd0aac595b233f4861c3c1375622326d955d
 2020-11-05 12:17:54 +02:00
Michal Niewrzal
2398afe985 satellite/metainfo/metabase: add GetLatestObjectLastSegment request 
Change-Id: I50c452e379f78864b38921f31e53a0554642ab02
 2020-11-05 08:20:34 +00:00
Kaloyan Raev
f986fdfe72 metabase: optimize delete segments 
Segments are deleted in a single DB query instead of a separate one for
each segment.

Change-Id: Icb9ae92b4b8405030c9d9ef57ba30bee6a35515c
 2020-11-04 11:59:56 +02:00
Michal Niewrzal
4134100d80 satellite/metainfo/metabase: add UpdateObjectMetadata request 
Change-Id: Iec73000e58f373294f37ef3e4470412d1c5f1340
 2020-11-03 11:08:41 +00:00
Kaloyan Raev
89cefa9045 metabase: add Objects info to DeleteObjectResult 
Change-Id: If7ed2ed0bbba08ad971c64852d51ff1a066c4f1e
 2020-11-03 12:45:49 +02:00
Kaloyan Raev
f7aa5b4e39 metabase: CommitObject return metabase.Object 
An immediate purpose is to be used in test cases for deleting objects

Change-Id: I481d3be5800e55da231ecb628ef7eb3ea011d196
 2020-11-03 09:57:44 +00:00
Kaloyan Raev
7e9ac2ec0a metabase: use storj.ErrObjectNotFound instead of custom error 
Change-Id: Ie90e438f7d7b58e6ea986359849dd8319433ae9f
 2020-11-03 09:35:22 +00:00
Michal Niewrzal
ae17ae73d7 satellite/metainfo/metabase: add metadata and nonce to CommitObject 
request

Change-Id: I584775800963fc2167c749f3da20475036fa5325
 2020-11-02 16:49:18 +01:00
Egon Elbre
716068a1e0 Merge branch 'master'. 
Change-Id: Ic14325edc291573582dce0cea3e04991a820b48b
 2020-11-02 13:02:01 +02:00
Egon Elbre
fd8e697ab2 {satellite,storagenode}/internalpb: use specific package name 
Ensure we don't register types with the same name into protobuf.

Change-Id: I53d025863fff8c91a067ca5819befa87eb5e35bb
 2020-10-30 17:31:08 +02:00
Egon Elbre
9674ce40e6 mod: bump dependencies 
Change-Id: I61beaf9793609f7614478cfa604b2721d9010b52
 2020-10-30 16:17:55 +02:00
Michal Niewrzal
0205f0d807 satellite/metainfo: fix usage of types from internalpb 
After moving SatStreamID and SatSegmentID from common I missed changing
some methods in metainfo endpoint. This change is a fix for that.

Change-Id: I34e121fce47371ee4cfd92cce03809520b68859f
 2020-10-30 16:03:45 +02:00
Egon Elbre
77c4f99fa0 satellite/internalpb: move delegated_repair.proto 
Change-Id: If4f37c52b151e09cf35d2145b463ef1e9ab529ae
 2020-10-30 15:31:32 +02:00
Egon Elbre
11338e9beb satellite/internalpb: move audithistory.pb 
Change-Id: I8eee84d49ed90459168ddaf04ae57f790c2a22c4
 2020-10-30 15:30:11 +02:00
Egon Elbre
1903b15474 storagenode/internalpb: move gracefulexit.proto 
Change-Id: Ia3614846ed49a39c8f39331516d16d45a695240b
 2020-10-30 15:24:56 +02:00
Michal Niewrzal
517212bfa8 satellite/metainfo: fix usage of types from internalpb 
After moving SatStreamID and SatSegmentID from common I missed changing
some methods in metainfo endpoint. This change is a fix for that.

Change-Id: I3344623dc7acfa73db6c20cd3212301e74335857
 2020-10-30 13:20:58 +00:00
Egon Elbre
cda67a659a storagenode/internalpb: move inspector.proto 
Change-Id: I951379c3b2ff00d1bc09d6a49c026a7e723432d6
 2020-10-30 14:51:26 +02:00
Egon Elbre
7ce372c686 satellite/internalpb: add inspectors 
Change-Id: Ib688e43d05135c0c31ae95df533f1e4535ea396a
 2020-10-30 13:28:17 +02:00
Egon Elbre
004e610d0f satellite/internalpb: move datarepair.pb to internal 
Change-Id: If901d9ff4e5ee6715b963eeeb46513a602a44b3d
 2020-10-30 13:28:14 +02:00
Kaloyan Raev
b8c6fb764c satellite/metainfo: add metabase to metainfo service 
Change-Id: Ie3ff238b138d8a57d99e32b13f7a71aa624d53e3
 2020-10-30 12:49:47 +02:00
Egon Elbre
995900e02f Merge branch 'master' into multipart-upload 
Change-Id: I8717f9f7cc54f3934402a98f21129e16841b25bd
 2020-10-30 11:48:05 +02:00
Michal Niewrzal
8f26f66da0 internalpb: move satellite specific protobuf types storj/storj 
We have some types that are only valid for satellite usage. Such types
are SatStreamID and SatSegmentID. This change moves those types to
storj/storj and adds basic infrastructure for generating code.

Change-Id: I1e643844f947ce06b13e51ff16b7e671267cea64
 2020-10-30 08:49:16 +00:00
Qweder93
f5ba8b8009 storagenode/suspensions: added offline-suspension notificatio chore + tests 
Change-Id: I2521cd2e7d08a1dd379e717a554a026c7508c18f
 2020-10-29 19:44:22 +02:00
Egon Elbre
e0dca4042d all: add pprof labels for debugger 
By using pprof.Labels debugger is able to show service/peer names in
goroutine names.

Change-Id: I5f55253470f7cc7e556f8e8b87f746394e41675f
 2020-10-29 15:10:07 +00:00
Egon Elbre
53b396b901 satellite/metainfo/metabase: move Encryption to Begin 
Change-Id: Ic1fb935ce7668e901f6cbe14a67c071a1f16a9ad
 2020-10-29 16:04:21 +02:00
Qweder93
624255e8ba storagenode/secret: db tests added, small renaming fixes added 
Change-Id: I7eae1a9a64c20a39c97e81fa741cfc9b9e1e615a
 2020-10-29 14:23:04 +02:00
Egon Elbre
caefde6b32 private/{dbutil,tagsql}: pass ctx to database opening 
Database opening usually dial and hence we should pass ctx to them.

Change-Id: Iaa2875981570d83e65be3710f841cf30349f807b
 2020-10-29 10:51:29 +00:00
Egon Elbre
e3985799a1 storage/{cockroachkv,postgreskv}: add ctx to opening 
Database opening usually dial and hence we should pass ctx to them.

Change-Id: Iecf41241aaa94d54506cbc80b0e53449848d8819
 2020-10-29 10:49:08 +00:00
Michal Niewrzal
809eb14ac2 satellite/metainfo/metabase: move metainfo PoC into storj repo 
Change-Id: I39356d8bc7305b4a8ea0c1fb5603010ad72a68b9
 2020-10-29 09:44:55 +01:00
Egon Elbre
89ce1fe626 storagenode/storagenodedb: add ctx to OpenNew and OpenExisting 
Database opening usually dial and hence we should pass ctx to them.

Change-Id: I9160ae95829f22f347bd525904898a47279a7427
 2020-10-29 09:52:37 +02:00
Egon Elbre
096445bc1c certificate/authorization: add ctx to OpenDB 
Database opening usually dial and hence we should pass ctx to them.

Change-Id: I1362783568f66383c46f07be7549327bb1aaa39e
 2020-10-29 09:46:23 +02:00
Egon Elbre
d0beaa4a87 pkg/revocation: pass ctx into opening the database 
Opening a databases requires ctx, this is first step to passing ctx
to the appropriate level.

Change-Id: I12700f39a320206d8a2a4e054452319f8585b44b
 2020-10-29 07:15:36 +00:00
Egon Elbre
9b2e00a38b satellite: pass ctx into satellitedb.Open 
Opening a database requires ctx, this is first step to passing ctx
to the appropriate level.

Change-Id: Ic303e69f868ef3449ae36377937a29670cf635e2
 2020-10-29 06:38:37 +00:00
littleskunk
ed1f6d7973
satellite/config: move repair override from config to default (#3958) 
Co-authored-by: Igor <38665104+ihaid@users.noreply.github.com>
 2020-10-28 17:24:39 +02:00
Michal Niewrzal
cb1fea87f8 satellite/metainfo: mark unused methods as 'not implemented' 
Some of metainfo endpoint methods are not used but we still have
implementation there. This change removes unused code and returns
unimplemented error for those methods.

Change-Id: I74e75e0caff76a4f5d119ee989b687b4e9d6e6f9
 2020-10-28 12:42:47 +00:00
Michal Niewrzal
1adb497a71 satellite/metainfo: remove unused code 
This change removed unused 'createRequests' struct. As far I remember it
was used to help validating old metainfo beginObject/commitObject flow.

Change-Id: I0f139b9934196d73f26eafa347ba5605722f3a55
 2020-10-28 12:40:14 +01:00
Egon Elbre
76f4619a9c {satellite,storagenode}/gracefulexit: ensure client is closed 
Change-Id: I576a955a5578caf7fcbee832beca28cef2b0c83e
 2020-10-27 23:27:07 +02:00
Jessica Grebenschikov
99c88efbbf scripts/tests: fix gateway tests 
Change-Id: I9a23ef08794043ad615066ae5929df9ff3a02d69
 2020-10-27 08:21:28 -07:00
Kaloyan Raev
92a2be2abd satellite/metainfo: get away from using pb.Pointer in Metainfo Loop 
As part of the Metainfo Refactoring, we need to make the Metainfo Loop
working with both the current PointerDB and the new Metabase. Thus, the
Metainfo Loop should pass to the Observer interface more specific Object
and Segment types instead of pb.Pointer.

After this change, there are still a couple of use cases that require
access to the pb.Pointer (hence we have it as a field in the
metainfo.Segment type):
1. Expired Deletion Service
2. Repair Service

It would require additional refactoring in these two services before we
are able to clean this.

Change-Id: Ib3eb6b7507ed89d5ba745ffbb6b37524ef10ed9f
 2020-10-27 13:06:47 +00:00
Cameron Ayer
bb7be23115 satellite/{audit,overlay,satellitedb}: enable reporting offline audits 
- Remove flag for switching off offline audit reporting.
- Change the overlay method used from UpdateUptime to BatchUpdateStats, as this
is where the new online scoring is done.
- Add a new overlay.AuditOutcome type: AuditOffline. Since we now use the same
method to record offline audits as success, failure, and unknown, we need to
distinguish offline audits from the rest.

Change-Id: Iadcfe10cf13466fa1a1c2dc542db8994a6423355
 2020-10-27 10:44:46 +00:00
Moby von Briesen
2fbb4095b2 storagenode/orders/ordersfile: Handle remaining pb.Unmarshal errors 
Missed one case of Unmarshal in the previous commit for V0 files (0f4e4969b7)
In V1, unmarshalling was being attempted before the checksum was
verified, so this commit moves those calls to the end of the V1 ReadOne
function.

Change-Id: Ic0b49f0bbc91fb61fb28af6003060994d0af22ed
 2020-10-26 20:27:05 +00:00
Egon Elbre
9adde49e1a satellite/gracefulexit: ensure test doesn't timeout on failure 
Change-Id: Id004f8a075592ffc19b12a9d666058b60cb7724d
 2020-10-26 21:16:48 +02:00
Egon Elbre
22ec940f7e storage/filestore: defer closing 
Change-Id: Iccd6d1c64c1b7a6eecaa4c675bb7b554b381d0f5
 2020-10-26 21:09:58 +02:00
VitaliiShpital
2f4e383997 web/satellite: project edit page: back button click area decreased 
WHAT:
decreased back button's clickable area to avoid missclicking

WHY:
bug fixing

Change-Id: Ia38ff076b4204b00822cf97c7cca52dbef38baf5
 2020-10-26 18:02:27 +00:00