A Multi-Path Bidirectional Layer 3 Proxy

Setup Notes

Linux

Policy Based Routing

ip route flush 10
ip route add table 10 to 1.1.1.0/24 dev eth1
ip rule add from 1.1.1.4 table 10 priority 10

ip route flush 11
ip route add table 11 to 1.1.1.0/24 dev eth2
ip rule add from 1.1.1.5 table 11 priority 11

ARP Flux

sysctl -w net.ipv4.conf.all.arp_announce=1
sysctl -w net.ipv4.conf.all.arp_ignore=1

See http://kb.linuxvirtualserver.org/wiki/Using_arp_announce/arp_ignore_to_disable_ARP

Setup Scripts

These are functional setup scripts that make the application run as intended on Linux. They should later be split into component parts, or incorporated into the main application.

Remote Portal

#!/bin/bash
set -e

# IPv4 Forwarding
sysctl -w net.ipv4.ip_forward=1
sysctl -w net.ipv4.conf.eth0.proxy_arp=1

# Tunnel addr/up
ip addr add 172.19.152.2/31 dev nc0
ip link set up nc0

# Deliberately break local routing
ip rule add from all table local priority 20
ip rule del 0 || true

# Route packets to the interface but for nc to this host
ip rule add to 1.1.1.3 dport 1234 table local priority 9

# Route packets to the interface but not for nc via the tunnel
ip route flush 10
ip route add table 10 to 1.1.1.3 via 172.19.152.3 dev nc0
ip rule add to 1.1.1.3 table 10 priority 10

Local Portal

#!/bin/bash
set -e

# Fix ARP
sysctl -w net.ipv4.conf.all.arp_announce=1
sysctl -w net.ipv4.conf.all.arp_ignore=1

# IPv4 Forwarding
sysctl -w net.ipv4.ip_forward=1

# Tunnel addr/up
ip addr add 172.19.152.3/31 dev nc0
ip link set up nc0

# Fix routing out of the correct interfaces
ip route flush 10
ip route add table 10 to 1.1.1.0/24 dev eth1
ip rule add from 1.1.1.4 table 10 priority 10

ip route flush 11
ip route add table 11 to 1.1.1.0/24 dev eth2
ip rule add from 1.1.1.5 table 11 priority 11

# Route packets from the remote portal's address on the client interface via the tunnel
ip route flush 12
ip route add table 12 to 1.1.1.0/24 via 172.19.152.2 dev nc0
ip rule add from 1.1.1.3 iif eth3 table 12 priority 12

# Route packets to the remote portal's address out of the client interface
ip route flush 13
ip route add table 13 to 1.1.1.3 dev eth3
ip rule add to 1.1.1.3 table 13 priority 13

Client

No configuration needed. Simply set the IP to that of the remote server/32 with a gateway of 192.168.1.1.