325 lines
25 KiB
TeX
325 lines
25 KiB
TeX
%!TEX root = ../thesis.tex
|
|
%*******************************************************************************
|
|
%****************************** Fourth Chapter *********************************
|
|
%*******************************************************************************
|
|
\chapter{Evaluation}
|
|
|
|
% **************************** Define Graphics Path **************************
|
|
\ifpdf
|
|
\graphicspath{{4_Evaluation/Figs/Raster/}{4_Evaluation/Figs/PDF/}{4_Evaluation/Figs/}}
|
|
\else
|
|
\graphicspath{{4_Evaluation/Figs/Vector/}{4_Evaluation/Figs/}}
|
|
\fi
|
|
|
|
This chapter will discuss the methods used to evaluate my project and the results gained. The results will be discussed in the context of the success criteria laid out in the Project Proposal.
|
|
|
|
This evaluation shows that a network using my method of combining Internet connections can see vastly superior network performance to one without. It will show the benefits to throughput, availability, and adaptability.
|
|
|
|
\section{Evaluation Methodology}
|
|
|
|
I performed my experiments on a local Proxmox\footnote{\url{https://proxmox.com}} server. To encourage frequent and thorough testing, a harness was built in Python, allowing tests to be added easily and repeated with any code changes.
|
|
|
|
Proxmox was chosen due to its RESTful API, for integration with Python. It provides the required tools to limit connection speeds and disable connections. The server that ran these tests holds only a single other virtual machine which handles routing. This limits the effect of external factors on the tests.
|
|
|
|
The tests are performed on a Dell R710 Server with the following specifications:
|
|
|
|
\vspace{5mm}
|
|
\begin{tabular}{ll}
|
|
\textbf{CPU(s)} & 16 x Intel(R) Xeon(R) CPU X5667 @ 3.07GHz (2 Sockets) \\
|
|
\textbf{Memory} & 6 x 2GB DDR3 ECC RDIMMS \\
|
|
\textbf{Kernel} & Linux 5.4 LTS
|
|
\end{tabular}
|
|
|
|
\subsection{Data Gathering}
|
|
|
|
To generate these results, a fresh set of VMs (Virtual Machines) are created and the software installed on them. Once this is complete, each test begins, and is repeated a fixed number of times. When visualising the data produced, unless otherwise specified, the error bars will represent the inter-quartile range of the data, and the plotted point the median.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{tikzpicture}[
|
|
squarednode/.style={rectangle, draw=black!60, fill=red!5, very thick, minimum size=5mm},
|
|
]
|
|
|
|
% Nodes
|
|
\node[squarednode] at (0,0) (speedtest) {Speed Test Server};
|
|
\node[squarednode] at (4,0) (remoteportal) {Remote Portal};
|
|
\node[squarednode] at (8,0) (localportal) {Local Portal};
|
|
\node[squarednode] at (11,0) (client) {Client};
|
|
|
|
% Edges
|
|
\draw[->] ([yshift=6mm]speedtest.north) -- (speedtest.north);
|
|
\draw[->] ([yshift=6mm]remoteportal.north) -- (remoteportal.north);
|
|
\draw[->] ([xshift=-7mm,yshift=6mm]localportal.north) -- ([xshift=-7mm]localportal.north);
|
|
\draw[->] ([yshift=6mm]localportal.north) -- (localportal.north);
|
|
\draw[->] ([xshift=7mm,yshift=6mm]localportal.north) -- ([xshift=7mm]localportal.north);
|
|
\draw[->] ([yshift=6mm]client.north) -- (client.north);
|
|
|
|
\draw[-] ([yshift=6mm]speedtest.north) -- ([yshift=6mm]localportal.north);
|
|
\draw[-] ([xshift=7mm,yshift=6mm]localportal.north) -- ([yshift=6mm]client.north);
|
|
|
|
% Edge Label
|
|
\node at ([xshift=-3.5mm,yshift=9mm]localportal.north) {0 .. N};
|
|
\end{tikzpicture}
|
|
|
|
\caption{The network structure of standard tests}
|
|
\label{fig:standard-network-structure}
|
|
\end{figure}
|
|
|
|
The network structure of all standard tests is shown in figure \ref{fig:standard-network-structure}. Any deviations from this structure will be mentioned. The Local Portal has as many interfaces as referenced in any test, plus one to connect to the client. All Virtual Machines also have an additional interface for management, but this has no effect on the tests.
|
|
|
|
\section{Success Criteria}
|
|
|
|
\subsection{Flow Maintained}
|
|
|
|
Demonstrating that a flow is maintained under connection loss has two cases: TCP flows and UDP flows. For TCP flows, the success criteria will be met if a TCP flow can continue to provide reliable transmission under a connection loss. UDP flows are less standardised, so will be split into two categories: an artificial iperf3 test, and a less artificial SIP phone call. The iperf3 test can be used to study the packet behaviour at the time of connection loss, while the SIP phone call provides a representative example of a UDP flow that would be dropped under normal conditions, but is not dropped under this proxy.
|
|
|
|
\subsubsection{TCP}
|
|
|
|
To test whether a TCP flow is maintained, a pair of small Python scripts were written. The first creates a TCP server and listens on a port. When it receives a message, it reads the first 12 bytes as a UTF-8 string, replaces the second character with an `o', and sends back the string. The client sends messages of the form \verb'ping%s', where \verb'%s' is an 8-byte nonce. It therefore receives back messages of the form \verb'pong%s', where \verb'%s' is the nonce that it just sent. This allows the client to check that the connection to the server is actively responding.
|
|
|
|
\mynote{Fill in missing graph.}
|
|
|
|
\subsubsection{UDP}
|
|
|
|
Firstly, the effect of connection loss on a UDP flow will be judged by the packet loss statistics of an iperf3 test. The bandwidth of the test will be kept sufficiently low that packet loss would not be expected, given that loss is only introduced within the test network when bandwidth exceeds the limit. This bandwidth is chosen as 128KBps, which is sufficiently low as to not hit the bandwidth limits, but far higher than a UDP flow such as a SIP call.
|
|
|
|
If the proxy can maintain a flow under a connection loss, the expected result is a peak in packet loss of no more than 50\%, which rapidly returns to 0\%. This represents the small portion of packets needing to be sent to note that the internal flow is offline, and thus stop sending packets to it to be lost. This loss should not exceed 50\%, as the connections are of equal bandwidth and thus should receive half of the packets each.
|
|
|
|
\mynote{Fill in missing graph.}
|
|
|
|
The second test is qualitative, involving making a call and checking for disconnection. Firstly, the call is made and both connections for the local proxy disconnected. This represents a failure scenario, as no connections continue existing. It should be confirmed that the call is dropped. Secondly, a call is made from which only one connection is disconnected. To pass this success criteria, the call should continue, though may experience some minor disruption as the connection is disconnected.
|
|
|
|
\mynote{Fill in Wireshark traces and analysis.}
|
|
|
|
\subsection{Bidirectional Performance Gains}
|
|
|
|
To demonstrate that all performance gains are bidirectional, I will provide graphs both inbound and outbound to the client for each performance test executed in this evaluation. This will sufficiently show the performance gains in each case. Inbound tests occur with the test server running on the proxy client and the test client running outside, while outbound tests occur with the test server running outside of the proxy and reaching in.
|
|
|
|
To demonstrate this somewhat succinctly, a pair of graphs for the same test in a common case will be shown. To demonstrate that this requirement is satisfied for all cases, for each graph of results presented in this evaluation, the graph for the alternative direction will be provided in appendix \ref{appendix:outbound-graphs}.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/bidirectional-inbound}
|
|
\caption{Throughput of proxied connections inbound to the client.}
|
|
\label{fig:bidirectional-inbound}
|
|
\end{subfigure}
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/bidirectional-outbound}
|
|
\caption{Throughput of proxied connections outbound from the client.}
|
|
\label{fig:bidirectional-outbound}
|
|
\end{subfigure}
|
|
\caption{Throughput results both inbound to the client and outbound from the client.}
|
|
\label{fig:bidirectional-gains}
|
|
\end{figure}
|
|
|
|
Figure \ref{fig:bidirectional-gains} shows two graphs of the same set of tests - one for the inbound performance and one for the outbound. It can be seen that both graphs show the same shape, satisfying that the performance gains of this proxy apply in both directions.
|
|
|
|
\subsection{IP Spoofing}
|
|
\label{section:ip-spoofing-evaluation}
|
|
|
|
To demonstrate that the IP of the client can be set to the IP of the remote portal, the network structure shown in figure \ref{fig:standard-network-structure}, used for most of these tests, can be examined further. This will demonstrate that it is possible to set the IP as such, as all of the tests in this section did so.
|
|
|
|
In the given network structure, the speed test server, remote portal and local portal are each connected to one virtual switch, which acts as a mock Internet. There is then a separate virtual switch, which connects an additional interface of the local portal to the client. The IP addresses of the interfaces shown in figure \ref{fig:standard-network-structure} are listed in figure \ref{fig:standard-network-structure-ips}. The IP addresses of the public interfaces are represented by letters, as they use arbitrary public IP addresses to ensure no local network firewall rules impact the configuration.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{tabular}{c|c|c}
|
|
Machine & Interface & IP Address \\
|
|
\hline
|
|
Speed Test Server & eth0 & \emph{A} \\
|
|
\hline
|
|
Remote Portal & eth0 & \emph{B} \\
|
|
\hline
|
|
\multirow{5}{*}{Local Portal} & eth0 & \emph{C0} \\
|
|
& eth1 & \emph{C1} \\
|
|
& \vdots & \vdots \\
|
|
& ethN & \emph{CN} \\
|
|
& eth\{N+1\} & 192.168.1.1 \\
|
|
\hline
|
|
Client & eth0 & \emph{B}
|
|
\end{tabular}
|
|
\caption{The IP layout of the standard test network structure.}
|
|
\label{fig:standard-network-structure-ips}
|
|
\end{figure}
|
|
|
|
It is shown that the client in this testing setup shares an IP address with the remote portal. To achieve this, the client configuration is particularly simple. A static route is added for 192.168.1.1 from the eth0 interface, and this then set as the default gateway. The IP address is set as the IP address of the remote portal. The details of this configuration are provided in figure \ref{section:implementation-system-configuration}.
|
|
|
|
Given that the client shares the IP address of the remote portal in these cases, it is demonstrated that this success criteria is met. Sharing the IP of the remote portal allows most routers to be configured behind the local portal as a client, allowing it to act as a standard Internet connection. An alternative approach, where the local portal acts as a router, is detailed in section \ref{section:real-world-testing}.
|
|
|
|
\subsection{Security}
|
|
|
|
Success in terms of security involves providing security no worse than a standard connection. To demonstrate that this is satisfied, I refer back to section \ref{section:layered-security}, in which I describe the ability for this proxy to be layered with other security software. Specifically, the ability to run this proxy behind the VPN solution Wireguard. By setting up a Wireguard tunnel for each connection and using a separate IP range in each, configuring the proxy to run behind Wireguard is no more complicated than the IP routing necessary. Section \label{section:layered-security} provides an argument that the proxying of packets is made secure by operating in a secure overlay network.
|
|
|
|
Therefore, to provide security no worse than a standard connection, it is sufficient to show that the security provided is better than a standard connection. If Wireguard provides security better than a standard connection, then it is possible for this proxy to be configured such that it provides security no worse than a standard connection. Further, if any solution which this can be configured behind, such as IPsec Authentication Headers, provides the correct security guarantees, then the security is no worse.
|
|
|
|
I also presented additional security mechanisms that do not rely on other software. However, given the difficulty of proving the comparative security, I will be relying on the ability to improve security with layering to satisfy this success criteria.
|
|
|
|
\subsection{More Bandwidth over Two Equal Connections}
|
|
|
|
To demonstrate that more bandwidth is available over two equal connections through this proxy than one without, I will compare the iperf3 throughput between the two cases. Further, I will provide a comparison point against a single connection of the higher bandwidth, as this is the maximum theoretical performance of combining the two lower bandwidth connections.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-equal-a-inbound}
|
|
\caption{Throughput of proxied connections inbound to the client.}
|
|
\label{fig:more-bandwidth-equal-lesser}
|
|
\end{subfigure}
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-equal-b-inbound}
|
|
\caption{Throughput of proxied connections outbound from the client.}
|
|
\label{fig:more-bandwidth-equal-greater}
|
|
\end{subfigure}
|
|
\caption{Graphs demonstrating that the throughput of two connections proxied lie between one connection of the same speed and one connection of double the speed}
|
|
\label{fig:more-bandwidth-equal}
|
|
\end{figure}
|
|
|
|
The results of these tests are given in figure \ref{fig:more-bandwidth-equal}, for both a pair of 1MBps connections and a pair of 2MBps connections. To satisfy this success criteria, the proxied bar on each graph should exceed the throughput of the direct bar of equal bandwidth. It can be seen in both cases that this occurs, and thus the success criteria is met. The throughput far exceeds the single direct connection, and is closer to the single double bandwidth connection than the single equal bandwidth connection, demonstrating a good portion of the maximum performance is achieved.
|
|
|
|
\section{Extended Goals}
|
|
|
|
\subsection{More Bandwidth over Unequal Connections}
|
|
|
|
For showing improved throughput over connections which are not equal, three results will be compared. Connections of speed $x+x$, speeds $x+y$, and speeds $y+y$ will be shown, where $x < y$. To show that unequal connections exceed the performance of a pair of slower connections, the results for speeds $x+y$ should lie between $x+x$ and $y+y$. Further, to show that percentage throughput is invariant to the balance of connection throughput, the unequal connections should lie halfway between the two equal connection results.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-unequal-a-inbound}
|
|
\caption{Throughput of proxied connections inbound to the client.}
|
|
\label{fig:more-bandwidth-unequal-lesser}
|
|
\end{subfigure}
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-unequal-b-inbound}
|
|
\caption{Throughput of proxied connections outbound from the client.}
|
|
\label{fig:more-bandwidth-unequal-greater}
|
|
\end{subfigure}
|
|
\caption{Graphs demonstrating that the throughput of two two connections proxied lie between one connection of the same speed and one connection of double the speed}
|
|
\label{fig:more-bandwidth-unequal}
|
|
\end{figure}
|
|
|
|
Two sets of results are provided - one for 1MBps and 2MBps connections, and another for 2MBps and 4MBps connections. In both cases, it can be seen that the proxy with unequal connections lies between the equal connection proxies. Further, it can be seen that both unequal proxied connections lie approximately halfway between the equal pairs. This suggests that the proxy design is successful in being invariant to the static balance of connection throughput.
|
|
|
|
\subsection{More Bandwidth over Four Equal Connections}
|
|
|
|
This criteria expands on the scalability in terms of number of connections of the proxy. Specifically, comparing the performance of three connections against four. To fulfil this, the results for each of two, three and four connections are included on each graph. This allows the trend of performance with an increasing number of connections to begin being visualised, which is expanded upon further in section \ref{section:number-of-connections-scaling}.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-four-a-inbound}
|
|
\caption{1MB connections}
|
|
\label{fig:more-bandwidth-four-a}
|
|
\end{subfigure}
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-four-b-inbound}
|
|
\caption{2MB connections}
|
|
\label{fig:more-bandwidth-four-b}
|
|
\end{subfigure}
|
|
\caption{Scaling of equal connections}
|
|
\label{fig:more-bandwidth-four}
|
|
\end{figure}
|
|
|
|
Provided in figure \ref{fig:more-bandwidth-four} are results for both 1MBps and 2MBps connections. Firstly, it is clear that the proxy consisting of 4 connections exceeds the throughput of the proxy consisting of 3 connections in both cases. Secondly, it appears that a linear trend is forming. This trends will be further evaluated in section \ref{section:number-of-connections-scaling}, but suggests that the structure of the proxy suffers little loss in efficiency from adding further connections.
|
|
|
|
\subsection{Bandwidth Variation}
|
|
|
|
This criteria judges the adaptability of the congestion control system in changing network conditions. To test this, the bandwidth of one of the local portal's connections is varied during an iperf3 throughput test. Thus far, bar graphs have been sufficient to show the results of each test. In this case, as the performance should now be time sensitive, I will be presenting a line graph. The error bars on the x-axis represent the range of continuous time results included in each discrete plotted point, while the y-axis error bars again represent the inter-quartile range of the gathered data. The target rates will be plotted as a fixed line for each of the speeds, as opposed to time-series. The error bar for these series will be omitted, as they occlude much of the graph, and are visible in figure (ref needed).
|
|
|
|
\mynote{Reference the graph which shows the errors.}
|
|
|
|
The criteria will be met if the following are true: the throughput begins at the rate of a time constant connection; the throughput stabilises at the altered rate after alteration; the throughput returns to the original rate after the rate is reset.
|
|
|
|
\mynote{Re-gather data and include graph.}
|
|
|
|
Two graphs are presented here. Figure \ref{fig:bandwidth-variation-down} presents a situation where the speed of a connection decreases, before returning to its original rate. This test begins with two 2MBps connections, changing to 1MBps + 2MBps at $t=10$, and returning to two 2MBps connections at $t=20$. Figure \ref{fig:bandwidth-variation-up} presents a situation where the speed of a connection increases, before returning to its original rate. This test begins with two 2MBps connections, changing to 3MBps + 2MBps at $t=10$, and returning to two 2MBps connections at $t=20$.
|
|
|
|
\subsection{Connection Loss}
|
|
|
|
This criteria judges the ability of the proxy as a whole to handle a complete connection loss while maintaining proportional throughput. As the proxy has redundant connections, it is feasible for this to cause a minimal loss of service. Unfortunately, losing a connection causes significant instability with the proxy, so this extended goal has not been met.
|
|
|
|
\mynote{Re-evaluate with UDP.}
|
|
|
|
\subsection{Single Interface Remote Portal}
|
|
|
|
Similarly to section \ref{section:ip-spoofing-evaluation}, a remote portal with a single interface is employed within the standard testing structure for this section, using techniques detailed in section \ref{section:implementation-system-configuration}. By altering the routing tables such that all local traffic for the remote portal is sent to the local portal via the proxy, excluding the traffic for the proxy itself, the packets can be further forwarded from the local portal to the client which holds that IP address. As the standard testing structure employs a remote portal with a single interface, it is shown in each test result that this is a supported configuration, and thus this success criteria is met.
|
|
|
|
\subsection{Connection Metric Values}
|
|
|
|
The extended goal of connection metric values has not been implemented. Instead, peers which only transfer data in one direction were implemented, which covers some of the use cases for metric values. Though metric values for connections would have been useful in some cases, they do not represent the standard usage of the software, and the added complexity of managing live peers was deemed unnecessary for the core software. Instead, I would consider providing a better interface to control the software externally, which would allow a separate piece of software to manage live peers. This has not been completed at this time.
|
|
|
|
\section{Stretch Goals}
|
|
|
|
\subsection{IPv4/IPv6 Support}
|
|
|
|
The project is only tested with IPv4.
|
|
|
|
\subsection{UDP Proxy Flows}
|
|
|
|
Although UDP proxy flows are implemented, they are unable to provide improved performance over a TCP connection.
|
|
|
|
\subsection{IP Proxy Packets}
|
|
|
|
The proxy only supports TCP and UDP flows for carrying the proxied data.
|
|
|
|
\section{Performance Evaluation}
|
|
\label{section:performance-evaluation}
|
|
|
|
The discussion of success criteria above used relatively slow network connections to test scaling in certain situations. This section will focus on testing how the solution scales, in terms of faster individual connections, and with many more connections. Further, all of the above tests were automated and carried out entirely on virtual hardware. This section will show some `real-world' data, using a Raspberry Pi 4B and real Internet connections.
|
|
|
|
\subsection{Faster Connections Scaling}
|
|
|
|
\mynote{Once automated testing is up, grab the data for this and plot a couple of graphs.}
|
|
|
|
\subsection{Number of Connections Scaling}
|
|
\label{section:number-of-connections-scaling}
|
|
|
|
\mynote{Needs more data.}
|
|
|
|
\subsection{Real World Testing}
|
|
\label{section:real-world-testing}
|
|
|
|
Although the success criteria of this project revolve around virtual hardware, it extends into the real world. This section will describe the application of this proxy to my network, including the considerations made in network design.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{tikzpicture}[
|
|
rednode/.style={rectangle, draw=black!60, fill=red!5, very thick, minimum size=5mm},
|
|
bluenode/.style={rectangle, draw=black!60, fill=blue!5, very thick, minimum size=5mm},
|
|
]
|
|
|
|
% Nodes
|
|
\node[rednode] at (0,2) (modema) {Modem A};
|
|
\node[rednode] at (0,0) (modemb) {Modem B};
|
|
|
|
\node[bluenode] at (4,1) (router) {Multi-WAN Router};
|
|
|
|
\node[rednode] at (8,2.5) (laptop) {Laptop};
|
|
\node[bluenode] at (8,1) (portal) {Local Portal};
|
|
\node[rednode] at (8,-0.5) (tablet) {Tablet};
|
|
|
|
\node[rednode] at (12,2) (server) {Server};
|
|
\node[rednode] at (12,0) (phone) {Desk Phone};
|
|
|
|
% Edges
|
|
\draw[->] (modema.east) -- (router.west);
|
|
\draw[->] (modemb.east) -- (router.west);
|
|
|
|
\draw[->] (router.east) -- (laptop.west);
|
|
\draw[->] (router.east) -- (portal.west);
|
|
\draw[->] (router.east) -- (tablet.west);
|
|
|
|
\draw[->] (portal.east) -- (server.west);
|
|
\draw[->] (portal.east) -- (phone.west);
|
|
\end{tikzpicture}
|
|
\caption{Real world proxy implementation.}
|
|
\label{fig:real-world-network-diagram}
|
|
\end{figure}
|
|
|
|
Figure \ref{fig:real-world-network-diagram} presents the real world deployment as setup in my network. It begins with two modems providing two Internet connections to the multi-WAN router. The multi-WAN router uses session based load balancing to achieve usage of both connections simultaneously, across multiple devices. For some devices, such as laptops and tablets, this is sufficient. To avoid the extra cost of proxying traffics, such devices access the Internet directly via the multi-WAN router. Thus far, the configuration is standard for a multi-ISP setup.
|
|
|
|
Behind the multi-WAN router lies the local portal. Using a multi-WAN router here simplifies the configuration of the local portal. By forwarding a port from both WANs on the router to the local portal's internal host, it can receive connections via both WANs with a single IP address. The remote portal is then configured to connect via both IP addresses. The flexibility of the system allows either portal to listen for or initiate connections, which simplifies setup in this use case.
|
|
|
|
The local portal in this configuration runs FreeBSD\footnote{\url{https://www.freebsd.org/}} 13 on a Raspberry Pi 4. To make the most use of the proxied IP address, the local portal is configured as a NAT router. Source NAT is configured to translate the source address of outgoing packets to the address of the remote portal, allowing them to be proxied. Destination NAT is used to forward ports from the remote portal's IP address to the devices on the internal network.
|