266 lines
17 KiB
TeX
266 lines
17 KiB
TeX
%!TEX root = ../thesis.tex
|
|
%*******************************************************************************
|
|
%****************************** Fourth Chapter *********************************
|
|
%*******************************************************************************
|
|
\chapter{Evaluation}
|
|
|
|
% **************************** Define Graphics Path **************************
|
|
\ifpdf
|
|
\graphicspath{{Evaluation/Figs/Raster/}{Evaluation/Figs/PDF/}{Evaluation/Figs/}}
|
|
\else
|
|
\graphicspath{{Evaluation/Figs/Vector/}{Evaluation/Figs/}}
|
|
\fi
|
|
|
|
This chapter will discuss the methods used to evaluate my project and the results gained. The results will be discussed in the context of the success criteria laid out in the Project Proposal.
|
|
|
|
This evaluation shows that a network using my method of combining Internet connections can see vastly superior network performance to one without. It will show the benefits to throughput, availability, and adaptability.
|
|
|
|
\section{Evaluation Methodology}
|
|
|
|
I performed my experiments on a local Proxmox\footnote{\url{https://proxmox.com}} server. To encourage frequent and thorough testing, a harness was built in Python, allowing tests to be added easily and repeated with any code changes.
|
|
|
|
Proxmox was chosen due to its RESTful API, for integration with Python. It provides the required tools to limit connection speeds and disable connections. The server that ran these tests holds only a single other virtual machine which handles routing. This limits the effect of external factors on the tests.
|
|
|
|
The tests are performed on a Dell R710 Server with the following specifications:
|
|
|
|
\vspace{5mm} \noindent \textbf{CPU(s)} 16 x Intel(R) Xeon(R) CPU X5667 @ 3.07GHz (2 Sockets)
|
|
\\ \noindent \textbf{Memory} 6 x 2GB DDR3 ECC RDIMMS
|
|
\\ \noindent \textbf{Kernel} Linux 5.4 LTS
|
|
|
|
\section{Graphs}
|
|
|
|
\subsection{Bar Graphs}
|
|
|
|
The majority of data presented in this chapter will be in the form of bar graphs.
|
|
|
|
\subsection{Data Gathering}
|
|
|
|
To generate these results, a fresh set of VMs (Virtual Machines) are created and the software installed on them. Once this is complete, each test is repeated until the coefficient of variance ($\sigma / \mu$, where $\mu$ is the arithmetic mean and $\sigma$ the standard deviation) is below a desired level, or too many attempts have been completed. The number of attempts taken for each series will be shown in the legend of each graph.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{tikzpicture}[
|
|
squarednode/.style={rectangle, draw=black!60, fill=red!5, very thick, minimum size=5mm},
|
|
]
|
|
|
|
% Nodes
|
|
\node[squarednode] at (0,0) (speedtest) {Speed Test Server};
|
|
\node[squarednode] at (4,0) (remoteportal) {Remote Portal};
|
|
\node[squarednode] at (8,0) (localportal) {Local Portal};
|
|
\node[squarednode] at (11,0) (client) {Client};
|
|
|
|
% Edges
|
|
\draw[->] ([yshift=6mm]speedtest.north) -- (speedtest.north);
|
|
\draw[->] ([yshift=6mm]remoteportal.north) -- (remoteportal.north);
|
|
\draw[->] ([xshift=-7mm,yshift=6mm]localportal.north) -- ([xshift=-7mm]localportal.north);
|
|
\draw[->] ([yshift=6mm]localportal.north) -- (localportal.north);
|
|
\draw[->] ([xshift=7mm,yshift=6mm]localportal.north) -- ([xshift=7mm]localportal.north);
|
|
\draw[->] ([yshift=6mm]client.north) -- (client.north);
|
|
|
|
\draw[-] ([yshift=6mm]speedtest.north) -- ([yshift=6mm]localportal.north);
|
|
\draw[-] ([xshift=7mm,yshift=6mm]localportal.north) -- ([yshift=6mm]client.north);
|
|
|
|
% Edge Label
|
|
\node at ([xshift=-3.5mm,yshift=9mm]localportal.north) {0 .. N};
|
|
\end{tikzpicture}
|
|
|
|
\caption{The network structure of standard tests}
|
|
\label{fig:standard-network-structure}
|
|
\end{figure}
|
|
|
|
The network structure of all standard tests is shown in figure \ref{fig:standard-network-structure}. Any deviations from this structure will be mentioned. The Local Portal has as many interfaces as referenced in any test, plus one to connect to the client. All Virtual Machines also have an additional interface for management, but this has no effect on the tests.
|
|
|
|
\section{Success Criteria}
|
|
|
|
\subsection{Flow Maintained}
|
|
|
|
\mynote{Write and gather results.}
|
|
|
|
\subsection{Bidirectional Performance Gains}
|
|
|
|
To demonstrate that all performance gains are bidirectional, I will provide graphs both inbound and outbound to the client for each performance test executed in this evaluation. This will sufficiently show the performance gains in each case. Inbound tests occur with the test server running on the proxy client and the test client running outside, while outbound tests occur with the test server running outside of the proxy and reaching in.
|
|
|
|
To demonstrate this somewhat succinctly, a pair of graphs for the same test in a common case will be shown. To demonstrate that this requirement is satisfied for all cases, for each graph of results presented in this evaluation, the graph for the alternative direction will be provided in appendix \ref{appendix:outbound-graphs}.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/bidirectional-inbound}
|
|
\caption{Throughput of proxied connections inbound to the client.}
|
|
\label{fig:bidirectional-inbound}
|
|
\end{subfigure}
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/bidirectional-outbound}
|
|
\caption{Throughput of proxied connections outbound from the client.}
|
|
\label{fig:bidirectional-outbound}
|
|
\end{subfigure}
|
|
\caption{Throughput results both inbound to the client and outbound from the client.}
|
|
\label{fig:bidirectional-gains}
|
|
\end{figure}
|
|
|
|
Figure \ref{fig:bidirectional-gains} shows two graphs of the same set of tests - one for the inbound performance and one for the outbound. It can be seen that both graphs show the same shape, satisfying that the performance gains of this proxy apply in both directions.
|
|
|
|
\subsection{IP Spoofing}
|
|
\label{section:ip-spoofing-evaluation}
|
|
|
|
To demonstrate that the IP of the client can be set to the IP of the remote portal, the network structure shown in figure \ref{fig:standard-network-structure}, used for most of these tests, can be examined further. This will demonstrate that it is possible to set the IP as such, as all of the tests in this section did so.
|
|
|
|
In the given network structure, the speed test server, remote portal and local portal are each connected to one virtual switch, which acts as a mock Internet. There is then a separate virtual switch, which connects an additional interface of the local portal to the client. The IP addresses of the interfaces shown in figure \ref{fig:standard-network-structure} are listed in \ref{fig:standard-network-structure-ips}. The IP addresses of the public interfaces are represented by letters, as they use arbitrary public IP addresses to ensure no local network firewall rules impact the configuration.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{tabular}{c|c|c}
|
|
Machine & Interface & IP Address \\
|
|
\hline
|
|
Speed Test Server & eth0 & \emph{A} \\
|
|
\hline
|
|
Remote Portal & eth0 & \emph{B} \\
|
|
\hline
|
|
\multirow{5}{*}{Local Portal} & eth0 & \emph{C0} \\
|
|
& eth1 & \emph{C1} \\
|
|
& \vdots & \vdots \\
|
|
& ethN & \emph{CN} \\
|
|
& eth\{N+1\} & 192.168.1.1 \\
|
|
\hline
|
|
Client & eth0 & \emph{B}
|
|
\end{tabular}
|
|
\caption{The IP layout of the standard test network structure.}
|
|
\label{fig:standard-network-structure-ips}
|
|
\end{figure}
|
|
|
|
It is shown that the client in this testing setup shares an IP address with the remote portal. To achieve this, the client configuration is particularly simple. A static route is added for 192.168.1.1 from the eth0 interface, and this then set as the default gateway. The IP address is set as the IP address of the remote portal. The details of this configuration are provided in \ref{section:implementation-systems-routing}.
|
|
|
|
Given that the client shares the IP address of the remote portal in these cases, it is demonstrated that this success criteria is met. Sharing the IP of the remote portal allows most routers to be configured behind the local portal as a client, allowing it to act as a standard Internet connection. An alternative approach, where the local portal acts as a router, is detailed in section \ref{section:real-world-testing}.
|
|
|
|
\subsection{Security}
|
|
|
|
\mynote{Write this.}
|
|
|
|
\subsection{More Bandwidth over Two Equal Connections}
|
|
|
|
To demonstrate that more bandwidth is available over two equal connections through this proxy than one without, I will compare the performance between the two cases. Further, I will provide a comparison point against a single connection of the higher bandwidth, as this is the maximum theoretical performance of combining the two lower bandwidth connections.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-equal-a-inbound}
|
|
\caption{Throughput of proxied connections inbound to the client.}
|
|
\label{fig:more-bandwidth-equal-lesser}
|
|
\end{subfigure}
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-equal-b-inbound}
|
|
\caption{Throughput of proxied connections outbound from the client.}
|
|
\label{fig:more-bandwidth-equal-greater}
|
|
\end{subfigure}
|
|
\caption{Graphs demonstrating that the throughput of two connections proxied lie between one connection of the same speed and one connection of double the speed}
|
|
\label{fig:more-bandwidth-equal}
|
|
\end{figure}
|
|
|
|
The results of these tests are given in figure \ref{fig:more-bandwidth-equal}, for both a pair of 1MBps connections and a pair of 2MBps connections. To satisfy this success criteria, the proxied bar on each graph should exceed the throughput of the direct bar of equal bandwidth. It can be seen in both cases that this occurs, and thus the success criteria is met. The throughput far exceeds the single direct connection, and is closer to the single double bandwidth connection than the single equal bandwidth connection, demonstrating a good portion of the maximum performance is achieved.
|
|
|
|
\section{Extended Goals}
|
|
|
|
\subsection{More Bandwidth over Unequal Connections}
|
|
|
|
For showing improved throughput over connections which are not equal, three results will be compared. That of two pairs of equal connections, one lower and one higher, and one of unequal connections, composed of the one of each of the pairs. To show that unequal connections exceed the performance of a pair of lower connections, the results for the unequal proxied connections should lie between the other two. Further, to show that percentage throughput is invariant to the balance of connection throughput, the unequal connections should lie halfway between the two equal connection results.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-unequal-a-inbound}
|
|
\caption{Throughput of proxied connections inbound to the client.}
|
|
\label{fig:more-bandwidth-unequal-lesser}
|
|
\end{subfigure}
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-unequal-b-inbound}
|
|
\caption{Throughput of proxied connections outbound from the client.}
|
|
\label{fig:more-bandwidth-unequal-greater}
|
|
\end{subfigure}
|
|
\caption{Graphs demonstrating that the throughput of two two connections proxied lie between one connection of the same speed and one connection of double the speed}
|
|
\label{fig:more-bandwidth-unequal}
|
|
\end{figure}
|
|
|
|
Two sets of results are provided - one for 1MBps and 2MBps connections, and another for 2MBps and 4MBps connections. In both cases, it can be seen that the proxy with unequal connections lies between the equal connection proxies. Further, it can be seen that both unequal proxied connections lie approximately halfway between the equal pairs. This suggests that the proxy design is successful in being invariant to the static balance of connection throughput.
|
|
|
|
\subsection{More Bandwidth over Four Equal Connections}
|
|
|
|
This criteria expands on the scalability in terms of number of connections of the proxy. Specifically, comparing the performance of three connections against four. To fulfil this, the results for each of two, three and four connections are included on each graph. This allows the trend of performance with an increasing number of connections to begin being visualised, which is expanded upon further in section \ref{section:number-of-connections-scaling}.
|
|
|
|
\begin{figure}
|
|
\centering
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-four-a-inbound}
|
|
\caption{1MB connections}
|
|
\label{fig:more-bandwidth-four-a}
|
|
\end{subfigure}
|
|
\begin{subfigure}{.49\textwidth}
|
|
\includegraphics[width=0.9\linewidth]{graphs/more-bandwidth-four-b-inbound}
|
|
\caption{2MB connections}
|
|
\label{fig:more-bandwidth-four-b}
|
|
\end{subfigure}
|
|
\caption{Scaling of equal connections}
|
|
\label{fig:more-bandwidth-four}
|
|
\end{figure}
|
|
|
|
Provided in figure \ref{fig:more-bandwidth-four} are results for both 1MBps and 2MBps connections. Firstly, it is clear that the proxy consisting of 4 connections exceeds the throughput of the proxy consisting of 3 connections in both cases. Secondly, it appears that a linear trend is forming. This trends will be further evaluated in section \ref{section:number-of-connections-scaling}, but suggests that the structure of the proxy suffers little loss in performance from adding further connections.
|
|
|
|
\subsection{Bandwidth Variation}
|
|
|
|
This criteria judges the adaptability of the congestion control system in changing network conditions. To test this, the bandwidth of one of the local portal's connections is varied during an iperf3 bandwidth test. Thus far, bar graphs have been sufficient to show the results of each test. In this case, as the performance should now be time sensitive, I will be presenting a line graph. The error bars on the x-axis represent the range of continuous time results included in each discrete plotted point, while the y-axis error bars again represent the inter-quartile range of the gathered data. The target rates will be plotted as a fixed line for each of the speeds, as opposed to time-series. The error bar for these series will be omitted, as they occlude much of the graph, and are visible in figure (ref needed).
|
|
|
|
The criteria will be met if the following are true: the throughput begins at the rate of a time constant connection; the throughput stabilises at the altered rate after alteration; the throughput returns to the original rate after the rate is reset.
|
|
|
|
\mynote{Re-gather data and include graph.}
|
|
|
|
Two graphs are presented here. Figure \ref{fig:bandwidth-variation-down} presents a situation where the speed of a connection decreases, before returning to its original rate. This test begins with two 2MBps connections, changing to 1MBps + 2MBps at $t=10$, and returning to two 2MBps connections at $t=20$. Figure \ref{fig:bandwidth-variation-up} presents a situation where the speed of a connection increases, before returning to its original rate. This test begins with two 2MBps connections, changing to 3MBps + 2MBps at $t=10$, and returning to two 2MBps connections at $t=20$.
|
|
|
|
\subsection{Connection Loss}
|
|
|
|
This criteria judges the ability of the proxy as a whole to handle a complete connection loss. As the proxy has redundant connections, it is feasible for this to cause a minimal loss of service. Unfortunately, losing a connection causes significant instability with the proxy, so this extended goal has not been met.
|
|
|
|
\mynote{Re-evaluate with UDP.}
|
|
|
|
\subsection{Single Interface Remote Portal}
|
|
|
|
Similarly to section \ref{section:ip-spoofing-evaluation}, a remote portal with a single interface is employed within the standard testing structure for this section, using techniques detailed in section \ref{section:implementation-systems-routing}. By altering the routing tables such that all local traffic for the remote portal is sent to the local portal via the proxy, excluding the traffic for the proxy itself, the packets can be further forwarded from the local portal to the client which holds that IP address.
|
|
|
|
As the standard testing structure employs a remote portal with a single interface, it is shown in each test result that this is a supported configuration, and thus this success criteria is met.
|
|
|
|
\subsection{Connection Metric Values}
|
|
|
|
The extended goal of connection metric values has not been implemented at this point.
|
|
|
|
\section{Stretch Goals}
|
|
|
|
\subsection{IPv4/IPv6 Support}
|
|
|
|
The project is only tested with IPv4.
|
|
|
|
\subsection{UDP Proxy Flows}
|
|
|
|
Although UDP proxy flows are implemented, they are unable to provide improved performance over a TCP connection.
|
|
|
|
\subsection{IP Proxy Packets}
|
|
|
|
The project only supports TCP and UDP flows for carrying the proxied data.
|
|
|
|
\section{Performance Evaluation}
|
|
\label{section:performance-evaluation}
|
|
|
|
The discussion of success criteria above used relatively slow network connections to test scaling in certain situations. This section will focus on testing how the solution scales, in terms of faster individual connections, and with many more connections. Further, all of the above tests were automated and carried out entirely on virtual hardware. This section will show some 'real-world' data, using a Raspberry Pi 4B and real Internet connections.
|
|
|
|
\subsection{Faster Connections Scaling}
|
|
|
|
\mynote{Write this.}
|
|
|
|
\subsection{Number of Connections Scaling}
|
|
\label{section:number-of-connections-scaling}
|
|
|
|
\mynote{Write this.}
|
|
|
|
\subsection{Real World Testing}
|
|
\label{section:real-world-testing}
|
|
|
|
\mynote{Write this.}
|
|
|