Commit Graph

20 Commits

Author SHA1 Message Date
Erik van Velzen
b574ee5e6d satellite/metabase/rangedloop: service skeleton
Create skeleton for multi-threaded segment loop, peer, cmd command for rangedloop.

Change-Id: I52c78a313f15070d43207c52ea94e53169821654
2022-11-22 15:21:41 +02:00
Michal Niewrzal
11d1e623b5 satellite/metabase/segmentloop: don't do rate limiting if disabled
We have a code to  limit segments loop in case it will hit DB to hard
but so far we didn't use this loop feature  in production. This is a
simple change to avoid logic responsible for rate limiting and its
monitoring if limiting is disabled (RateLimit = 0)

Change-Id: I43e07b407c6e65cf252303159d052eef250d1bea
2022-10-11 10:55:30 +00:00
Michal Niewrzal
6fcc5c5cb8 satellite/metabase/segmentloop: remove unused monkit calls
Monkit calls for fast methods which are executed very frequently can
slowdown whole process. This change removes monkit calls which are not
used.

See https://review.dev.storj.io/c/storj/storj/+/8498 as an example of
speed improvement after removing monkit calls.

Change-Id: If6567d80e05b748e6393b58a5142e43013107c61
2022-10-03 14:07:18 +00:00
Michal Niewrzal
2163d60037 satellite/metabase/segmentloop: add missing monitoring
Currently we are tracking loop time with RunOnce method
metrics but it may give inaccurate values if loop if waiting
for observer for a ver long time. We had such case with GC
which is only loop observer in GC peer and its joning loop
every 5 days.

Change-Id: I08546c912e00c3641488de6a5d75948fe75c8e99
2022-08-26 14:01:22 +02:00
Egon Elbre
6d3fd33ca3 satellite/metabase/segmentloop: start immediately on manual trigger
Currently loops wait for the coalesce duration for TriggerWait.
Let's skip the coalesce when we trigger it manually.

Change-Id: If5bacd4e263d233f1f3ea41b989922d2ed5a48d4
2021-09-28 18:32:03 +00:00
Egon Elbre
32cee1e572 satellite/metabase/segmentloop: ensure we shutdown on ctx err
Not all errors from RunOnce can be retried. The context can be cancelled
with several different errors, e.g. timeout. Ensure we stop the loop
when context has errored, because none of the queries will succeed
when it has failed.

Change-Id: If3ff11f11a6f43c0d67633be1cfaf23e3e9e55f3
2021-09-21 10:02:00 +00:00
Michał Niewrzał
84a383638b satellite/metabase: increase maximum batch size for loop
Current default is equal to hardcoded maximum of batch size
for loop listing. With this change we will bump maximum
batch size but current default won't change. We will be able to
increase it later without need for source code change.

Change-Id: I2744a87be28af4157f58ede73455682f61733bc1
2021-09-10 09:33:55 +00:00
Michał Niewrzał
c258f4bbac private/testplanet: move Metabase outside Metainfo for satellite
At some point we moved metabase package outside Metainfo
but we didn't do that for satellite structure. This change
refactors only tests.
When uplink will be adjusted we can remove old entries in
Metainfo struct.

Change-Id: I2b66ed29f539b0ec0f490cad42c72840e0351bcb
2021-09-09 07:15:51 +00:00
Artur M. Wolff
a5371353bf satellite/metabase/segmentloop: fix rate.NewLimiter call
This change fixes the call to rate.NewLimiter to account for the
"infinite" case.

Change-Id: Ib3f914ca33ad8b981157fd224f077e6ad8d8c644
2021-08-19 17:23:13 +00:00
Michał Niewrzał
55d7bcc59b satellite/metabase/segmentloop: don't shutdown satellite on loop error
We made decision to avoid satellite shutdown when segment loop
will return error. Loop still can reeturn error but it will be logged
and we will make monitoring/alert around that error.

Change-Id: I6aa8e284406edf644a09d6b1fe00c3155c5430c9
2021-07-30 06:49:10 +00:00
Michał Niewrzał
b12d29935a satellite/metabase: remove metaloop package
We moved everything to segment loop so we can now
remove metaloop from code.

Change-Id: I9bd8d2349e5638d7cdad50f2f313f9bd89a8165c
2021-07-22 13:00:45 +00:00
Egon Elbre
c248651f3f satellite/metabase/{meta,segment}loop: fix test flakiness
When the delta is very small from the bounds then the ratio calculation
doesn't work that well. Let's allow 100 from the bounds, since that
would be expected in any case.

We won't add a configuration for it, since it's not that useful.

Change-Id: I049066a42470b825f430b7f32ebe92d544c6cc8b
2021-07-02 19:16:11 +03:00
Egon Elbre
5f901c8066 satellite/metabase/metaloop: add as of system interval to stats
Table stats query should use AS OF SYSTEM INTERVAL in the query to optimize
the behavior.

Change-Id: Ie14f5441062d3d264bb3c626081e77d6edc7b724
2021-06-17 23:43:41 +03:00
Egon Elbre
9640cc2c06 satellite/metabase/segmentloop: verify processed count
This adds verification for the processed count and before and after
segment/objects table counts.

This adds new flag:

  metainfo.segment-loop.suspicious-processed-ratio: 0.03

This defaults to 3%, which at 100M segments is 3M segments.

Change-Id: I5ee03e913ddc4e67e94010ced126a2a9ea51f41b
2021-06-17 19:46:55 +03:00
Michał Niewrzał
9a113da361 satellite/metabase: expose expires_at with loops
Loops are using custom structs to provide
segments while iterating/looping so we need
to expose new field there too.

Change-Id: I12c8f4a01afeac171bf638d278253999fa90a8cb
2021-06-11 16:51:37 +00:00
Egon Elbre
f3a52d1da5 satellite/metabase/segmentloop: limit max interval
Ensure that we don't query too far in the history, which slows things
down.

Change-Id: Ia77aa522f7f4c5d43629d51bb9a51a49fab6fa14
2021-06-10 17:06:33 +00:00
Egon Elbre
4469d229f8 satellite/metabase/{meta,segment}loop: avoid passing config
Currently the iterate is being called in only one location so there's no
benefit in passing them as arguments over using the receiver.

Change-Id: I433a5d8b795b1bcc1f1e9320d87b10820cf537f1
2021-06-02 15:55:59 +00:00
Egon Elbre
2cf10a7bf4 satellite/metabase/{segment,meta}loop: avoid no observers error
In a rare case it's possible to start the loop iteration without
observers. The most likely case is that the observer is cancelled and
the coalesce timer trigger asynchronously, although being stopped.

Nevertheless, all the observers may also exit during the iteration, in
either case it should not result in an error.

If there's a probem with the observers, then they can report their own
error as they see fit.

Change-Id: Ie423fec41e6295be05536a4b7b0b6623ffebf2fb
2021-06-02 11:57:59 +00:00
JT Olio
da9ca0c650 testplanet/satellite: reduce the number of places default values need to be configured
Satellites set their configuration values to default values using
cfgstruct, however, it turns out our tests don't test these values
at all! Instead, they have a completely separate definition system
that is easy to forget about.

As is to be expected, these values have drifted, and it appears
in a few cases test planet is testing unreasonable values that we
won't see in production, or perhaps worse, features enabled in
production were missed and weren't enabled in testplanet.

This change makes it so all values are configured the same,
systematic way, so it's easy to see when test values are different
than dev values or release values, and it's less hard to forget
to enable features in testplanet.

In terms of reviewing, this change should be actually fairly
easy to review, considering private/testplanet/satellite.go keeps
the current config system and the new one and confirms that they
result in identical configurations, so you can be certain that
nothing was missed and the config is all correct.
You can also check the config lock to see what actual config
values changed.

Change-Id: I6715d0794887f577e21742afcf56fd2b9d12170e
2021-06-01 22:14:17 +00:00
Michał Niewrzał
053e58b683 satellite/metabase: add segmentloop service
We want to move some of current metainfo loop observers to
segment loop. This change adds new service, similar to metainfo
loop but which is iterating only over segments.

Change-Id: I67f7f461781723a4476e2b83377f31736d7c4870
2021-06-01 11:15:07 +00:00