915f3952af
We avoid putting more than one piece of a segment on the same /24 network (or /64 for ipv6). However, it is possible for multiple pieces of the same segment to move to the same network over time. Nodes can change addresses, or segments could be uploaded with dev settings, etc. We will call such pieces "clumped", as they are clumped into the same net, and are much more likely to be lost or preserved together. This change teaches the repair checker to recognize segments which have clumped pieces, and put them in the repair queue. It also teaches the repair worker to repair such segments (treating clumped pieces as "retrievable but unhealthy"; i.e., they will be replaced on new nodes if possible). Refs: https://github.com/storj/storj/issues/5391 Change-Id: Iaa9e339fee8f80f4ad39895438e9f18606338908
28 lines
922 B
Go
28 lines
922 B
Go
// Copyright (C) 2023 Storj Labs, Inc.
|
|
// See LICENSE for copying information.
|
|
|
|
package repair
|
|
|
|
import "storj.io/storj/satellite/metabase"
|
|
|
|
// FindClumpedPieces finds pieces that are stored in the same last_net (i.e., the same /24 network
|
|
// in the IPv4 case). The first piece for a given last_net is fine, but any subsequent pieces in
|
|
// the same last_net will be returned as part of the 'clumped' list.
|
|
//
|
|
// lastNets must be a slice of the same length as pieces; lastNets[i] corresponds to pieces[i].
|
|
func FindClumpedPieces(pieces metabase.Pieces, lastNets []string) (clumped metabase.Pieces) {
|
|
lastNetSet := make(map[string]struct{})
|
|
for i, p := range pieces {
|
|
lastNet := lastNets[i]
|
|
_, ok := lastNetSet[lastNet]
|
|
if ok {
|
|
// this last_net was already seen
|
|
clumped = append(clumped, p)
|
|
} else {
|
|
// add this last_net to the set of seen nets
|
|
lastNetSet[lastNet] = struct{}{}
|
|
}
|
|
}
|
|
return clumped
|
|
}
|