802ff18bd8
We have an alert on `not_enough_shares_for_audit` which fires too frequently. Every time so far, it has been because of a network blip of some nature on the satellite side. Satellite operators are expected to have other means in place for alerting on network problems and fixing them, so it's not necessary for the audit framework to act in that way. Instead, in this change, we add three new metrics, `audit_not_enough_nodes_online`, `audit_not_enough_shares_acquired`, and `audit_suspected_network_problem`. When an audit fails, and emits `not_enough_shares_for_audit`, we will now determine whether it looks like we are having network problems (most errors are connection failures, possibly also some successful connections which subsequently time out) or whether something else has happened. After this is deployed, we can remove the alert on `not_enough_shares_for_audit` and add new alerts on `audit_not_enough_nodes_online` and `audit_not_enough_shares_acquired`. `audit_suspected_network_problem` does not need an alert. Refs: https://github.com/storj/storj/issues/4669 Change-Id: Ibb256bc19d2578904f71f5229111ac98e5212fcb |
||
---|---|---|
.. | ||
audit_test.go | ||
chore.go | ||
collector_test.go | ||
collector.go | ||
containment_test.go | ||
containment.go | ||
cryptosource.go | ||
disqualification_test.go | ||
getshare_test.go | ||
integration_test.go | ||
pieces_test.go | ||
pieces.go | ||
queue_test.go | ||
queue.go | ||
reporter_test.go | ||
reporter.go | ||
reservoir_test.go | ||
reservoir.go | ||
reverify_test.go | ||
verifier_test.go | ||
verifier_unit_test.go | ||
verifier.go | ||
worker.go |