JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,414 Commits 5 Branches 0 Tags 94 MiB
c4d6f472fc
Commit Graph

27 Commits

Author SHA1 Message Date
Michal Niewrzal
9202295348 satellite/metainfo: replace ScopedPath with metabase.SegmentLocation 
Change-Id: I7e89c9e8eaeae58be828a32ad47ed3028501f4c7
 2020-09-04 10:06:52 +00:00
Natalie Villasana
95ff29cce1 satellite/metainfo: reduce lookupLimit default to 2500 
Change-Id: I6569c6d1f145b127a9e8e1a65e4344dd62c989bb
 2020-09-01 12:04:48 -04:00
Michal Niewrzal
0604a672c1 satellite/metainfo: use metabase in loop 
Change-Id: I1bb0c6fe0a762895fde950690b06f7dd9d77e178
 2020-09-01 10:06:16 +00:00
Egon Elbre
080ba47a06 all: fix dots 
Change-Id: I6a419c62700c568254ff67ae5b73efed2fc98aa2
 2020-07-16 14:58:28 +00:00
Egon Elbre
d98b8f6e23 satellite/metainfo,storage: use different limit for metainfo loop 
Change-Id: I5ef7233930679b977b33f7b3e1dda45c907dcfad
 2020-05-05 10:37:20 +00:00
Egon Elbre
93e39a6b33 satellite/metainfo: support co-joining the loop 
This adds support for observers to join the loop together. This allows
to ensure that when multiple observers join, they will be part of the
same loop iteration.

Change-Id: Ie887d4cedfb074b65c782690a2c09c1704f56dfe
 2020-04-16 13:31:17 +03:00
Egon Elbre
11a44cdd88 all: don't depend on gogo/proto directly 
Change-Id: I8822dea0d1b7b99e0b828e0373a0308a42dde2be
 2020-04-08 17:32:15 +00:00
Egon Elbre
1024bf9ce1 all: simplify uuid usage 
Instead of uuid.Parse, use uuid.FromString.
This removes a bunch of pointer management logic.

Change-Id: Id25bd174eb43c71d00b450158a198abafd8958f2
 2020-04-02 13:45:19 +00:00
Egon Elbre
0a69da4ff1 all: switch to storj.io/common/uuid 
Change-Id: I178a0a8dac691e57bce317b91411292fb3c40c9f
 2020-03-31 19:16:41 +03:00
Egon Elbre
89e5c77d83 satellite/metainfo: track observer timing 
Measure total time spent in each observer and distribution of handling
pointers by pointer type.

Change-Id: I2d125dfce8dbbb17225029fa35557bc106491151
 2020-02-26 17:42:56 +00:00
JT Olio
2ae9978304 satellite/gc: skip first gc run 
rationale: if GC kills the satellite, it would be nice to make
it through a repair checker sweep first

Change-Id: Id56171dc8e13940cfb6481e36a910bad077a01ed
 2020-02-13 13:41:15 +02:00
Egon Elbre
c1c878efcf all: fix import groupings 
check-imports was broken and didn't complain about things.

Change-Id: I38adafd16b4aba86f0eb4f53427b4393f9a6c710
 2020-01-20 17:47:44 +00:00
Simon Guindon
80b41af8f1 satellite/metainfo: Fixed bug that discarded context cancellation errors 
When the context was being cancelled the error was being discarded within the rate limiting error handling which caused tests to fail.
Change-Id: I5c6458c16da09a11531233ea0ee80d914969cb3f
 2020-01-03 22:48:12 -05:00
Simon Guindon
e1e7cebe49 satellite/metainfo: added rate limiting support to the metainfo loop. 
As per discussed we decided to rate limit how fast we iterate through
the metainfo database in the metainfo loop. This puts in place a
mechanism for rate limiting and burst limiting if need be in the future.

The default for this rate limiting is still no limits so it stays the
same as our previous functionality.

Change-Id: I950f7192962b0e49f082d2c4284e2d52b0a925c7
 2020-01-03 15:00:29 -05:00
Egon Elbre
6615ecc9b6 common: separate repository 
Change-Id: Ibb89c42060450e3839481a7e495bbe3ad940610a
 2019-12-27 14:11:15 +02:00
Ivan Fraixedes
48a557eb2b satellite/metainfo: Fix misspelling in comment (#3636) 2019-11-24 13:04:25 -08:00
Michal Niewrzal
d96df2691a satellite/metainfo: improve Loop comments (#3595) 2019-11-21 15:24:17 -05:00
JT Olio
40012e5790 satellite/metainfo: continue instead of return (fixing my bad advice) 
Change-Id: I57ad3de8e3f705429bad98ce976879c4d5e905c9
 2019-11-20 09:37:48 -07:00
Ivan Fraixedes
c2e605e81e satellite/metainfo: Don't return error in loop when path has less than 4 parts (#3616) 
* satellite/metainfo: Rollback path parts check in loop

We have to rollback the changes applied in checking the rawPath parts
from 4 to 3 because the production prointerDB is still storing buckets.

* satellite/metainfo: Don't return path parts less 4

Don't return an error in the metainfo loop iterator when a path doesn't
have 4 parts because it belongs to bucket metadata, not an actual
object.
 2019-11-20 09:29:52 -07:00
Michal Niewrzal
ec41a51bbb
metainfo/loop: refactor iterator to be reusable (#3578) 2019-11-18 07:26:48 -08:00
Egon Elbre
a744fdfef0 satellite/metainfo: remove Iterate from service (#3196) 
* satellite/metainfo: remove Iterate from service

* fix test
 2019-10-08 07:39:23 -07:00
Egon Elbre
4bd1ce868f
satellite/metainfo: close loop separately to avoid logical races (#3100) 2019-09-23 22:14:39 +03:00
Michal Niewrzal
1c72e80e40 uplink/satellite: fix for case when inline segment is last one (#3062) 
* uplink/satellite: fix when inline seg is last one

* review comments
 2019-09-19 01:18:14 +02:00
Egon Elbre
7240e6cbb2
satellite: remove remote/inline file from BucketTally (#3041) 2019-09-13 16:51:41 +03:00
Egon Elbre
8b668ab1f8
satellite/metainfo.Loop: use a parsed path for observers (#3003) 2019-09-12 13:38:49 +03:00
Egon Elbre
a801fab66a
all: add archview annotations (#2964) 2019-09-10 16:24:16 +03:00
Maximillian von Briesen
6c1c3fb4a7
Add metainfo loop service (#2563) 
Add a metainfo loop service on the satellite that can be subscribed to by various services that need to make use of metainfo information
 2019-07-22 09:34:12 -04:00