233 lines
19 KiB
TeX
233 lines
19 KiB
TeX
%!TEX root = ../thesis.tex
|
|
%*******************************************************************************
|
|
%****************************** Fourth Chapter *********************************
|
|
%*******************************************************************************
|
|
\chapter{Evaluation}
|
|
|
|
% **************************** Define Graphics Path **************************
|
|
\ifpdf
|
|
\graphicspath{{4_Evaluation/Figs/Raster/}{4_Evaluation/Figs/PDF/}{4_Evaluation/Figs/}}
|
|
\else
|
|
\graphicspath{{4_Evaluation/Figs/Vector/}{4_Evaluation/Figs/}}
|
|
\fi
|
|
|
|
This chapter will discuss the methods used to evaluate my project and the results gained. The results will be discussed in the context of the success criteria laid out in the Project Proposal (Appendix \ref{appendix:project-proposal}). This evaluation shows that a network using my method of combining Internet connections can see vastly superior network performance to one without. It will show the benefits to throughput, availability, and adaptability.
|
|
|
|
The tests are performed on a Dell R710 Server with the following specifications:
|
|
|
|
\vspace{5mm}
|
|
\begin{tabular}{ll}
|
|
\textbf{CPU(s)} & 16 x Intel(R) Xeon(R) CPU X5667 @ 3.07GHz (2 Sockets) \\
|
|
\textbf{Memory} & 6 x 2GB DDR3 ECC RDIMMS \\
|
|
\textbf{Kernel} & Linux 5.4 LTS
|
|
\end{tabular}
|
|
|
|
When presenting data, error bars are given of the Inter-Quartile Range (IQR) of the data, with the plotted point being the median.
|
|
|
|
\section{Success Criteria}
|
|
|
|
\subsection{Flow Maintained}
|
|
|
|
Demonstrating that a flow is maintained under connection loss has two cases: TCP flows and UDP flows. For TCP flows, the success criteria will be met if a TCP flow can continue to provide reliable transmission under a connection loss. UDP flows are less standardised, so will be split into two categories: an artificial iperf3 test, and a less artificial SIP phone call. The iperf3 test can be used to study the packet behaviour at the time of connection loss, while the SIP phone call provides a representative example of a UDP flow that would be dropped under normal conditions, but is not dropped under this proxy.
|
|
|
|
\subsubsection{TCP}
|
|
|
|
To test whether a TCP flow is maintained, a pair of small Python scripts were written. The first creates a TCP server and listens on a port. When it receives a message, it reads the first 12 bytes as a UTF-8 string, replaces the second character with an `o', and sends back the string. The client sends messages of the form \verb'ping%s', where \verb'%s' is an 8-byte nonce. It therefore receives back messages of the form \verb'pong%s', where \verb'%s' is the nonce that it just sent. This allows the client to check that the connection to the server is actively responding.
|
|
|
|
\mynote{Fill in missing graph.}
|
|
|
|
\subsubsection{UDP}
|
|
|
|
Firstly, the effect of connection loss on a UDP flow will be judged by the packet loss statistics of an iperf3 test. The bandwidth of the test will be kept sufficiently low that packet loss would not be expected, given that loss is only introduced within the test network when bandwidth exceeds the limit. This bandwidth is chosen as 128KBps, which is sufficiently low as to not hit the bandwidth limits, but far higher than a UDP flow such as a SIP call.
|
|
|
|
If the proxy can maintain a flow under a connection loss, the expected result is a peak in packet loss of no more than 50\%, which rapidly returns to 0\%. This represents the small portion of packets needing to be sent to note that the internal flow is offline, and thus stop sending packets to it to be lost. This loss should not exceed 50\%, as the connections are of equal bandwidth and thus should receive half of the packets each.
|
|
|
|
\mynote{Fill in missing graph.}
|
|
|
|
The second test is qualitative, involving making a call and checking for disconnection. Firstly, the call is made and both connections for the local proxy disconnected. This represents a failure scenario, as no connections continue existing. It should be confirmed that the call is dropped. Secondly, a call is made from which only one connection is disconnected. To pass this success criteria, the call should continue, though may experience some minor disruption as the connection is disconnected.
|
|
|
|
\mynote{Fill in Wireshark traces and analysis.}
|
|
|
|
\subsection{Bidirectional Performance Gains}
|
|
|
|
To demonstrate that all performance gains are bidirectional, I will provide graphs both inbound and outbound to the client for each performance test executed in this evaluation. This will sufficiently show the performance gains in each case. Inbound tests occur with the test server running on the proxy client and the test client running outside, while outbound tests occur with the test server running outside of the proxy and reaching in.
|
|
|
|
To demonstrate this somewhat succinctly, a pair of graphs for the same test in a common case will be shown. To demonstrate that this requirement is satisfied for all cases, for each graph of results presented for the basic success criteria, the graph for the alternative direction will be provided in appendix \ref{appendix:outbound-graphs}.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/bidirectional-inbound}
|
|
\caption{Throughput of proxied connections inbound to the client.}
|
|
\label{fig:bidirectional-inbound}
|
|
\end{subfigure}
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/bidirectional-outbound}
|
|
\caption{Throughput of proxied connections outbound from the client.}
|
|
\label{fig:bidirectional-outbound}
|
|
\end{subfigure}
|
|
\caption{Throughput results both inbound to the client and outbound from the client.}
|
|
\label{fig:bidirectional-gains}
|
|
\end{figure}
|
|
|
|
Figure \ref{fig:bidirectional-gains} has two series for the same set of tests - one for the inbound (reaching in to the client, download) performance and one for the outbound (the client reaching out, upload). It can be seen that both graphs show the same shape, satisfying that the performance gains of this proxy apply in both directions.
|
|
|
|
\subsection{IP Spoofing}
|
|
\label{section:ip-spoofing-evaluation}
|
|
|
|
To demonstrate that the IP of the client can be set to the IP of the remote proxy, the network structure shown in Figure \ref{fig:standard-network-structure}, used for most of these tests, can be examined further. This will demonstrate that it is possible to set the IP as such, as all of the tests in this section did so.
|
|
|
|
In the given network structure, the speed test server, remote proxy and local proxy are each connected to one virtual switch, which acts as a mock Internet. There is then a separate virtual switch, which connects an additional interface of the local proxy to the client. The IP addresses of the interfaces shown in Figure \ref{fig:standard-network-structure} are listed in Figure \ref{fig:standard-network-structure-ips}. The IP addresses of the public interfaces are represented by letters, as they use arbitrary public IP addresses to ensure no local network firewall rules impact the configuration.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{tabular}{c|c|c}
|
|
Machine & Interface & IP Address \\
|
|
\hline
|
|
Speed Test Server & eth0 & \emph{A} \\
|
|
\hline
|
|
Remote Proxy & eth0 & \emph{B} \\
|
|
\hline
|
|
\multirow{5}{*}{Local Proxy} & eth0 & \emph{C0} \\
|
|
& eth1 & \emph{C1} \\
|
|
& \vdots & \vdots \\
|
|
& ethN & \emph{CN} \\
|
|
& eth\{N+1\} & 192.168.1.1 \\
|
|
\hline
|
|
Client & eth0 & \emph{B}
|
|
\end{tabular}
|
|
\caption{The IP layout of the test network structure.}
|
|
\label{fig:standard-network-structure-ips}
|
|
\end{figure}
|
|
|
|
It is shown that the client in this testing setup shares an IP address with the remote proxy. To achieve this, the client configuration is particularly simple. A static route is added for 192.168.1.1 from the eth0 interface, and this then set as the default gateway. The IP address is set as the IP address of the remote proxy. The details of this configuration are provided in Figure \ref{section:implementation-system-configuration}.
|
|
|
|
Given that the client shares the IP address of the remote proxy in these cases, it is demonstrated that this success criteria is met. Sharing the IP of the remote proxy allows most routers to be configured behind the local proxy as a client, allowing it to act as a standard Internet connection. An alternative approach, where the local proxy acts as a router, is detailed in section \ref{section:real-world-testing}.
|
|
|
|
\subsection{Security}
|
|
|
|
Success for security involves providing security no worse than a standard connection.
|
|
|
|
Success in terms of security involves providing security no worse than a standard connection. To demonstrate that this is satisfied, I refer back to Section \ref{section:layered-security}, in which I describe the ability for this proxy to be layered with other security software. Specifically, the ability to run this proxy behind the VPN solution Wireguard. By setting up a Wireguard tunnel for each connection and using a separate IP range in each, configuring the proxy to run behind Wireguard is no more complicated than the IP routing necessary. Section \ref{section:layered-security} provides an argument that the proxying of packets is made secure by operating in a secure overlay network.
|
|
|
|
Therefore, to provide security no worse than a standard connection, it is sufficient to show that the security provided is better than a standard connection. If Wireguard provides security better than a standard connection, then it is possible for this proxy to be configured such that it provides security no worse than a standard connection. Further, if any solution which this can be configured behind, such as IPsec Authentication Headers, provides the correct security guarantees, then the security is no worse.
|
|
|
|
I also presented additional security mechanisms that do not rely on other software. However, given the difficulty of proving the comparative security, I will be relying on the ability to improve security with layering to satisfy this success criteria.
|
|
|
|
\subsection{More Bandwidth over Two Equal Connections}
|
|
|
|
To demonstrate that more bandwidth is available over two equal connections through this proxy than one without, I will compare the iperf3 throughput between the two cases. Further, I will provide a comparison point against a single connection of the higher bandwidth, as this is the maximum theoretical performance of combining the two lower bandwidth connections.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-equal-a-inbound}
|
|
\caption{Throughput of proxied connections inbound to the client.}
|
|
\label{fig:more-bandwidth-equal-lesser}
|
|
\end{subfigure}
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-equal-b-inbound}
|
|
\caption{Throughput of proxied connections outbound from the client.}
|
|
\label{fig:more-bandwidth-equal-greater}
|
|
\end{subfigure}
|
|
\caption{Graphs demonstrating that the throughput of two connections proxied lie between one connection of the same speed and one connection of double the speed}
|
|
\label{fig:more-bandwidth-equal}
|
|
\end{figure}
|
|
|
|
The results of these tests are given in Figure \ref{fig:more-bandwidth-equal}, for both a pair of 1MBps connections and a pair of 2MBps connections. To satisfy this success criteria, the proxied bar on each graph should exceed the throughput of the direct bar of equal bandwidth. It can be seen in both cases that this occurs, and thus the success criteria is met. The throughput far exceeds the single direct connection, and is closer to the single double bandwidth connection than the single equal bandwidth connection, demonstrating a good portion of the maximum performance is achieved.
|
|
|
|
\section{Extended Goals}
|
|
|
|
\subsection{More Bandwidth over Unequal Connections}
|
|
|
|
For showing improved throughput over connections which are not equal, three results will be compared. Connections of speed $x+x$, speeds $x+y$, and speeds $y+y$ will be shown, where $x < y$. To show that unequal connections exceed the performance of a pair of slower connections, the results for speeds $x+y$ should lie between $x+x$ and $y+y$. Further, to show that percentage throughput is invariant to the balance of connection throughput, the unequal connections should lie halfway between the two equal connection results.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-unequal-a-inbound}
|
|
\caption{Throughput of proxied connections inbound to the client.}
|
|
\label{fig:more-bandwidth-unequal-lesser}
|
|
\end{subfigure}
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-unequal-b-inbound}
|
|
\caption{Throughput of proxied connections outbound from the client.}
|
|
\label{fig:more-bandwidth-unequal-greater}
|
|
\end{subfigure}
|
|
\caption{Graphs demonstrating that the throughput of two two connections proxied lie between one connection of the same speed and one connection of double the speed}
|
|
\label{fig:more-bandwidth-unequal}
|
|
\end{figure}
|
|
|
|
Two sets of results are provided - one for 1MBps and 2MBps connections, and another for 2MBps and 4MBps connections. In both cases, it can be seen that the proxy with unequal connections lies between the equal connection proxies. Further, it can be seen that both unequal proxied connections lie approximately halfway between the equal pairs. This suggests that the proxy design is successful in being invariant to the static balance of connection throughput.
|
|
|
|
\subsection{More Bandwidth over Four Equal Connections}
|
|
|
|
This criteria expands on the scalability in terms of number of connections of the proxy. Specifically, comparing the performance of three connections against four. To fulfil this, the results for each of two, three and four connections are included on each graph. This allows the trend of performance with an increasing number of connections to begin being visualised, which is expanded upon further in section \ref{section:number-of-connections-scaling}.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-four-a-inbound}
|
|
\caption{1MB connections}
|
|
\label{fig:more-bandwidth-four-a}
|
|
\end{subfigure}
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-four-b-inbound}
|
|
\caption{2MB connections}
|
|
\label{fig:more-bandwidth-four-b}
|
|
\end{subfigure}
|
|
\caption{Scaling of equal connections}
|
|
\label{fig:more-bandwidth-four}
|
|
\end{figure}
|
|
|
|
Provided in Figure \ref{fig:more-bandwidth-four} are results for both 1MBps and 2MBps connections. Firstly, it is clear that the proxy consisting of 4 connections exceeds the throughput of the proxy consisting of 3 connections in both cases. Secondly, it appears that a linear trend is forming. This trends will be further evaluated in section \ref{section:number-of-connections-scaling}, but suggests that the structure of the proxy suffers little loss in efficiency from adding further connections.
|
|
|
|
\subsection{Bandwidth Variation}
|
|
|
|
This criteria judges the adaptability of the congestion control system in changing network conditions. To test this, the bandwidth of one of the local portal's connections is varied during an iperf3 throughput test. Thus far, bar graphs have been sufficient to show the results of each test. In this case, as the performance should now be time sensitive, I will be presenting a line graph. The error bars on the x-axis represent the range of continuous time results included in each discrete plotted point, while the y-axis error bars again represent the inter-quartile range of the gathered data. The target rates will be plotted as a fixed line for each of the speeds, as opposed to time-series. The error bar for these series will be omitted, as they occlude much of the graph, and are visible in Figure (ref needed).
|
|
|
|
\mynote{Reference the graph which shows the errors.}
|
|
|
|
The criteria will be met if the following are true: the throughput begins at the rate of a time constant connection; the throughput stabilises at the altered rate after alteration; the throughput returns to the original rate after the rate is reset.
|
|
|
|
\mynote{Re-gather data and include graph.}
|
|
|
|
Two graphs are presented here. Figure \ref{fig:bandwidth-variation-down} presents a situation where the speed of a connection decreases, before returning to its original rate. This test begins with two 2MBps connections, changing to 1MBps + 2MBps at $t=10$, and returning to two 2MBps connections at $t=20$. Figure \ref{fig:bandwidth-variation-up} presents a situation where the speed of a connection increases, before returning to its original rate. This test begins with two 2MBps connections, changing to 3MBps + 2MBps at $t=10$, and returning to two 2MBps connections at $t=20$.
|
|
|
|
\subsection{Connection Loss}
|
|
|
|
This criteria judges the ability of the proxy as a whole to handle a complete connection loss while maintaining proportional throughput. As the proxy has redundant connections, it is feasible for this to cause a minimal loss of service. Unfortunately, losing a connection causes significant instability with the proxy, so this extended goal has not been met.
|
|
|
|
\mynote{Re-evaluate with UDP.}
|
|
|
|
\subsection{Single Interface Remote Portal}
|
|
|
|
Similarly to section \ref{section:ip-spoofing-evaluation}, a remote portal with a single interface is employed within the standard testing structure for this section, using techniques detailed in section \ref{section:implementation-system-configuration}. By altering the routing tables such that all local traffic for the remote portal is sent to the local portal via the proxy, excluding the traffic for the proxy itself, the packets can be further forwarded from the local portal to the client which holds that IP address. As the standard testing structure employs a remote portal with a single interface, it is shown in each test result that this is a supported configuration, and thus this success criteria is met.
|
|
|
|
\subsection{Connection Metric Values}
|
|
|
|
The extended goal of connection metric values has not been implemented. Instead, peers which only transfer data in one direction were implemented, which covers some of the use cases for metric values. Though metric values for connections would have been useful in some cases, they do not represent the standard usage of the software, and the added complexity of managing live peers was deemed unnecessary for the core software. Instead, I would consider providing a better interface to control the software externally, which would allow a separate piece of software to manage live peers. This has not been completed at this time.
|
|
|
|
\section{Stretch Goals}
|
|
|
|
\subsection{IPv4/IPv6 Support}
|
|
|
|
The project is only tested with IPv4.
|
|
|
|
\subsection{UDP Proxy Flows}
|
|
|
|
Although UDP proxy flows are implemented, they are unable to provide improved performance over a TCP connection.
|
|
|
|
\subsection{IP Proxy Packets}
|
|
|
|
The proxy only supports TCP and UDP flows for carrying the proxied data.
|
|
|
|
\section{Performance Evaluation}
|
|
\label{section:performance-evaluation}
|
|
|
|
The discussion of success criteria above used relatively slow network connections to test scaling in certain situations. This section provides a brief analysis of how this solution would scale to providing a higher bandwidth connection, specifically by adding network connections.
|
|
|
|
|
|
|
|
\mynote{Needs more data.}
|