storj/satellite/satellitedb
paul cannon 2522ff09b6 satellite/overlay: configurable meaning of last_net
Up to now, we have been implementing the DistinctIP preference with code
in two places:

 1. On check-in, the last_net is determined by taking the /24 or /64
    (in ResolveIPAndNetwork()) and we store it with the node record.
 2. On node selection, a preference parameter defines whether to return
    results that are distinct on last_net.

It can be observed that we have never yet had the need to switch from
DistinctIP to !DistinctIP, or from !DistinctIP to DistinctIP, on the
same satellite, and we will probably never need to do so in an automated
way. It can also be observed that this arrangement makes tests more
complicated, because we often have to arrange for test nodes to have IP
addresses in different /24 networks (a particular pain on macOS).

Those two considerations, plus some pending work on the repair framework
that will make repair take last_net into consideration, motivate this
change.

With this change, in the #2 place, we will _always_ return results that
are distinct on last_net. We implement the DistinctIP preference, then,
by making the #1 place (ResolveIPAndNetwork()) more flexible. When
DistinctIP is enabled, last_net will be calculated as it was before. But
when DistinctIP is _off_, last_net can be the same as address (IP and
port). That will effectively implement !DistinctIP because every
record will have a distinct last_net already.

As a side effect, this flexibility will allow us to change the rules
about last_net construction arbitrarily. We can do tests where last_net
is set to the source IP, or to a /30 prefix, or a /16 prefix, etc., and
be able to exercise the production logic without requiring a virtual
network bridge.

This change should be safe to make without any migration code, because
all known production satellite deployments use DistinctIP, and the
associated last_net values will not change for them. They will only
change for satellites with !DistinctIP, which are mostly test
deployments that can be recreated trivially. For those satellites which
are both permanent and !DistinctIP, node selection will suddenly start
acting as though DistinctIP is enabled, until the operator runs a single
SQL update "UPDATE nodes SET last_net = last_ip_port". That can be done
either before or after deploying software with this change.

I also assert that this will not hurt performance for production
deployments. It's true that adding the distinct requirement to node
selection makes things a little slower, but the distinct requirement is
already present for all production deployments, and they will see no
change.

Refs: https://github.com/storj/storj/issues/5391
Change-Id: I0e7e92498c3da768df5b4d5fb213dcd2d4862924
2023-03-09 02:20:12 +00:00
..
dbx satellite/satellitedb: change bucket_bandwidth_rollups primary key 2023-03-03 16:25:35 +00:00
satellitedbtest satellite/satellitedb: cleanup testing access 2023-02-06 14:36:11 +02:00
testdata satellite/satellitedb: change bucket_bandwidth_rollups primary key 2023-03-03 16:25:35 +00:00
accountfreezeevents_test.go satellite/{console,satellitedb}: add account freeze service 2022-12-16 22:24:58 +00:00
accountfreezeevents.go satellite/{db,analytics,payments}: add chore for auto account freeze 2023-03-01 09:31:27 +00:00
apikeys.go satellite: remove rewards package 2023-01-31 11:46:50 +00:00
attribution.go satellite: remove rewards package 2023-01-31 11:46:50 +00:00
audithistory.go satellite/reputation: new ApplyUpdates() method 2022-06-07 15:22:25 +00:00
billingdb.go satellite/payments/monetary: remove pkg and all its references 2022-09-07 12:58:00 +00:00
bucketsdb.go satellite/accounting/tally: fix looping over all buckets 2022-12-07 10:32:23 +00:00
coinpaymentstxs.go satellite: remove unused coinpayments code and chores 2022-11-30 16:24:48 +00:00
compensation.go satellite/compensation: smaller txns for RecordPeriod 2021-03-02 20:14:25 +00:00
consoledb_test.go satellite/satellitedb/satellitedbtest: pass ctx as an argument 2020-01-20 16:35:42 +02:00
consoledb.go satellite/{console,satellitedb}: implement account freeze event DB 2022-12-15 14:16:24 +00:00
containment.go satellite/audit: add GetAllContainedNodes method to ReverifyQueue 2023-02-02 00:39:29 +00:00
customers.go satellite/{db,analytics,payments}: add chore for auto account freeze 2023-03-01 09:31:27 +00:00
database.go satellite/satellitedb: cleanup testing access 2023-02-06 14:36:11 +02:00
gracefulexit.go sql: capitalize keywords 2022-03-01 15:19:38 +00:00
invoiceprojectrecords.go satellite/satellitedb: use tx instead of db.db in transactions 2022-06-14 17:35:44 +00:00
migrate_gen.go all: fix deprecated ioutil commands 2022-10-11 15:27:29 +00:00
migrate_test.go satellite/satellitedb: remove gob migration exceptions 2023-02-10 15:05:18 +02:00
migrate.go satellite/satellitedb: change bucket_bandwidth_rollups primary key 2023-03-03 16:25:35 +00:00
migratez.go satellite/satellitedb: change bucket_bandwidth_rollups primary key 2023-03-03 16:25:35 +00:00
nodeapiversion.go ci: fix lint errors 2021-05-17 13:37:31 +00:00
nodeevents_test.go satellite/nodeevents: take last_attempted into account when selecting 2022-12-01 20:08:38 +00:00
nodeevents.go satellite/nodeevents: take last_attempted into account when selecting 2022-12-01 20:08:38 +00:00
nodeselection.go satellite/overlay: configurable meaning of last_net 2023-03-09 02:20:12 +00:00
orders_test.go satellite/orders: cleanup after altering primary key 2023-03-06 16:03:11 +00:00
orders.go satellite/orders: cleanup after altering primary key 2023-03-06 16:03:11 +00:00
overlaycache_test.go satellite/overlay: add SetAllContainedNodes method to overlay.DB 2023-02-06 10:18:54 +00:00
overlaycache.go satellite/overlay: add SetAllContainedNodes method to overlay.DB 2023-02-06 10:18:54 +00:00
payouts.go satellite/compensation: smaller txns for RecordPeriod 2021-03-02 20:14:25 +00:00
peeridentities.go all: fix dots 2020-07-16 14:58:28 +00:00
projectaccounting_test.go satellite/payments/stripecoinpayments: undo price override removal 2023-02-24 16:54:51 +00:00
projectaccounting.go satellite/orders: cleanup after altering primary key 2023-03-06 16:03:11 +00:00
projectfromdbx_test.go satellite/{console,satellitedb}: add PublicID to Project, db method GetByPublicID 2022-07-21 19:49:26 +00:00
projectmembers_test.go web/satellite: project members sorting fixed (#3231) 2019-10-15 15:24:53 +03:00
projectmembers.go all: fix linter complaints 2020-10-13 15:59:01 +03:00
projects_test.go satellite/{console,satellitedb}: add account freeze service 2022-12-16 22:24:58 +00:00
projects.go satellite: remove rewards package 2023-01-31 11:46:50 +00:00
regtokens.go all: fix dots 2020-07-16 14:58:28 +00:00
repairqueue.go satellite/repair: implementation of batch insert 2022-05-09 12:41:22 +00:00
reputations.go satellite/reputation: reconfigure lambda and alpha 2022-08-17 18:52:53 +00:00
resetpasstokens.go all: fix dots 2020-07-16 14:58:28 +00:00
reverifyqueue_test.go satellite/audit: test delay before Reverify 2023-02-15 23:16:23 +00:00
reverifyqueue.go satellite/audit: test delay before Reverify 2023-02-15 23:16:23 +00:00
revocation.go sql: capitalize keywords 2022-03-01 15:19:38 +00:00
storagenodeaccounting.go satellite/accounting/nodetally: add ranged loop 2023-01-17 13:50:18 +01:00
storjscanpayments.go satellite/payments/monetary: remove pkg and all its references 2022-09-07 12:58:00 +00:00
storjscanwallets.go satellitedb: add token balance to API endpoint 2022-08-31 22:55:16 +00:00
stripecoinpaymentsdb.go satellite/{payments,satellitedb}: Remove custom coupon implementation 2021-10-11 19:47:00 +00:00
userfromdbx_test.go satellite/console: add cutoff to email reminders 2022-06-09 16:45:35 +00:00
users_test.go satellite/{console,satellitedb}: migration, methods for user settings 2023-02-07 21:48:29 +00:00
users.go satellite/{console,satellitedb}: migration, methods for user settings 2023-02-07 21:48:29 +00:00
verifyqueue_test.go satellite/audit: help performance of pushing to audit queue 2022-11-29 15:37:49 +00:00
verifyqueue.go satellite/audit: help performance of pushing to audit queue 2022-11-29 15:37:49 +00:00
webappsessions_test.go satellite/console: implement WebappSessions db 2022-05-16 15:06:04 +00:00
webappsessions.go satellite/console,web/satellite: invalidate sessions after inactivity 2022-08-23 15:51:05 +00:00