A Multi-Path Bidirectional Layer 3 Proxy

Setup Notes

Linux

Policy Based Routing

ip route flush 10
ip route add table 10 to 1.1.1.0/24 dev eth1
ip rule add from 1.1.1.4 table 10 priority 10

ip route flush 11
ip route add table 11 to 1.1.1.0/24 dev eth2
ip rule add from 1.1.1.5 table 11 priority 11

ARP Flux

sysctl -w net.ipv4.conf.all.arp_announce=1
sysctl -w net.ipv4.conf.all.arp_ignore=1

See http://kb.linuxvirtualserver.org/wiki/Using_arp_announce/arp_ignore_to_disable_ARP

Setup Scripts

These are functional setup scripts that make the application run as intended on Linux.

Remote Portal

Pre-Start

#!/bin/bash
set -e

## Set up variables
REMOTE_PORTAL_ADDRESS=A.B.C.D

## IPv4 Forwarding
sysctl -w net.ipv4.ip_forward=1
sysctl -w net.ipv4.conf.eth0.proxy_arp=1

## Transfer the local routing table to a much lower priority
(ip rule show | grep '20:') > /dev/null || ip rule add from all table local priority 20
ip rule del 0 2> /dev/null || true

## Ports to route locally

### MPBL3P
ip rule del 1 2> /dev/null || true
ip rule add to "$REMOTE_PORTAL_ADDRESS" dport 1234 table local priority 1

### SSH
ip rule del 2 2> /dev/null || true
ip rule add to "$REMOTE_PORTAL_ADDRESS" dport 22   table local priority 2

Post-Start

#!/bin/bash
set -e

## Set up variables
REMOTE_PORTAL_ADDRESS=A.B.C.D

## Tunnel addr/up
ip addr add 172.19.152.2/31 dev nc0
ip link set up nc0

# Route packets to the interface but not for nc via the tunnel
ip route flush 19
ip route add table 19 to "$REMOTE_PORTAL_ADDRESS" via 172.19.152.3 dev nc0
ip rule del 19 2> /dev/null || true
ip rule add to "$REMOTE_PORTAL_ADDRESS" table 19 priority 19

Local Portal

Pre-Start

#!/bin/bash
set -e

## Set up variables
GATEWAY_INTERFACE=eth0
GATEWAY_ADDRESS=10.36.12.1

## Fix ARP
sysctl -w net.ipv4.conf.all.arp_announce=1
sysctl -w net.ipv4.conf.all.arp_ignore=1

## IPv4 Forwarding
sysctl -w net.ipv4.ip_forward=1

## Gateway Interface Setup
ip addr add "$GATEWAY_ADDRESS"/32 dev "$GATEWAY_INTERFACE"
ip link set up "$GATEWAY_INTERFACE"

## Per-Interface Routing Tables

### 10.10.0.0/24
ip route flush 10
ip route add table 10 default via 10.10.0.1
ip rule del 10 2> /dev/null || true
ip rule add from 10.10.0.0/24 table 10 priority 10

### 192.168.0.0/24
ip route flush 11
ip route add table 11 default via 192.168.0.1
ip rule del 11 2> /dev/null || true
ip rule add from 192.168.0.0/24 table 11 priority 11

Post-Start

#!/bin/bash
set -e

## Set up variables
REMOTE_PORTAL_ADDRESS=A.B.C.D
GATEWAY_INTERFACE=eth0

## Tunnel Address and Enable
ip addr add 172.19.152.3/31 dev nc0
ip link set up nc0

## Route Outbound Packets Correctly
ip route flush 20
ip route add table 20 default via 172.19.152.2 dev nc0
ip rule del 20 2> /dev/null || true
ip rule add from "$REMOTE_PORTAL_ADDRESS" iif "$GATEWAY_INTERFACE" table 20 priority 20

## Route Inbound Packets Correctly
ip route flush 21
ip route add table 21 to "$REMOTE_PORTAL_ADDRESS" dev "$GATEWAY_INTERFACE"
ip rule del 21 2> /dev/null || true
ip rule add to "$REMOTE_PORTAL_ADDRESS" table 21 priority 21

Client

Connect to GATEWAY_INTERFACE and set the IP to REMOTE_PORTAL_ADDRESS/32 with a gateway of GATEWAY_ADDRESS.