%!TEX root = ../thesis.tex
%*******************************************************************************
%****************************** Fourth Chapter *********************************
%*******************************************************************************
\chapter{Evaluation}

% **************************** Define Graphics Path **************************
\ifpdf
    \graphicspath{{Evaluation/Figs/Raster/}{Evaluation/Figs/PDF/}{Evaluation/Figs/}}
\else
    \graphicspath{{Evaluation/Figs/Vector/}{Evaluation/Figs/}}
\fi

This chapter will discuss the methods used to evaluate my project and the results gained. The results will be discussed in the context of the success criteria laid out in the Project Proposal.

This evaluation shows that a network using my method of combining Internet connections can see vastly superior network performance to one without. It will show the benefits to throughput, availability, and adaptability.

\section{Evaluation Methodology}

I performed my experiments on a local Proxmox\footnote{\url{https://proxmox.com}} server. To encourage frequent and thorough testing, a harness was built in Python, allowing tests to be added easily and repeated with any code changes.

Proxmox was chosen due to its RESTful API, for integration with Python. It provides the required tools to limit connection speeds and disable connections. The server that ran these tests holds only a single other virtual machine which handles routing. This limits the effect of external factors on the tests.

The tests are performed on a Dell R710 Server with the following specifications:

\vspace{5mm} \noindent \textbf{CPU(s)} 16 x Intel(R) Xeon(R) CPU X5667 @ 3.07GHz (2 Sockets)
\\ \noindent \textbf{Memory} 6 x 2GB DDR3 ECC RDIMMS
\\ \noindent \textbf{Kernel} Linux 5.4 LTS

\section{Graphs}

\subsection{Bar Graphs}

The majority of data presented in this chapter will be in the form of bar graphs.

\subsection{Line Graphs}

Time sensitive data will be presented in the form of line graphs. These are generated in a consistent format, using a script found in appendix (Possibly Appendix).

\begin{figure}
  \centering
  \begin{subfigure}{.3\textwidth}
    \includegraphics[width=0.9\linewidth]{graphs/TIS0-R0-1R1-1R2-1R3-1T10S1-R0-1R1-1R2-1T10S2-R0-1R1-1T10}
    \caption{No error bars}
    \label{fig:errorbars-none}
  \end{subfigure}
  \begin{subfigure}{.3\textwidth}
    \includegraphics[width=0.9\linewidth]{graphs/TIExS0-R0-1R1-1R2-1R3-1T10S1-R0-1R1-1R2-1T10S2-R0-1R1-1T10}
    \caption{X error bars}
    \label{fig:errorbars-x}
  \end{subfigure}
  \begin{subfigure}{.3\textwidth}
    \includegraphics[width=0.9\linewidth]{graphs/TIEyS0-R0-1R1-1R2-1R3-1T10S1-R0-1R1-1R2-1T10S2-R0-1R1-1T10}
    \caption{Y error bars}
    \label{fig:errorbars-y}
  \end{subfigure}
  \caption{Line graphs produced with varying error bars enabled}
  \label{fig:errorbars}
\end{figure}

In figure \ref{fig:errorbars}, examples are shown of the same graph without any error bars, with error bars on the X axis, and with error bars on the Y axis. Error bars for the X axis are plotted as the range of all of the results, while error bars on the Y axis are plotted as $1.5*\sigma$, where $\sigma$ represents the standard deviation of the results.

In figure \ref{fig:errorbars-x}, it is shown that the range of the timestamps provided is incredibly tight. For this reason, I will not be including error bars in the X axis on the graphs shown from this point onwards.

In figure \ref{fig:errorbars-y}, it can be seen that the error bars on the Y axis are far more significant. Thus, error bars will continue to be included in the Y axis.

To generate these results, a fresh set of VMs (Virtual Machines) are created and the software installed on them. Once this is complete, each test is repeated until the coefficient of variance ($\sigma / \mu$, where $\mu$ is the arithmetic mean and $\sigma$ the standard deviation) is below a desired level, or too many attempts have been completed. The number of attempts taken for each series will be shown in the legend of each graph.

\begin{figure}
  \centering
  \begin{tikzpicture}[
      squarednode/.style={rectangle, draw=black!60, fill=red!5, very thick, minimum size=5mm},
    ]

    % Nodes
    \node[squarednode] at (0,0) (speedtest)      {Speed Test Server};
    \node[squarednode] at (4,0) (remoteportal)   {Remote Portal};
    \node[squarednode] at (8,0) (localportal)    {Local Portal};
    \node[squarednode] at (11,0) (client)         {Client};

    % Edges
    \draw[->] ([yshift=6mm]speedtest.north) -- (speedtest.north);
    \draw[->] ([yshift=6mm]remoteportal.north) -- (remoteportal.north);
    \draw[->] ([xshift=-7mm,yshift=6mm]localportal.north) -- ([xshift=-7mm]localportal.north);
    \draw[->] ([yshift=6mm]localportal.north) -- (localportal.north);
    \draw[->] ([xshift=7mm,yshift=6mm]localportal.north) -- ([xshift=7mm]localportal.north);
    \draw[->] ([yshift=6mm]client.north) -- (client.north);

    \draw[-] ([yshift=6mm]speedtest.north) -- ([yshift=6mm]localportal.north);
    \draw[-] ([xshift=7mm,yshift=6mm]localportal.north) -- ([yshift=6mm]client.north);

    % Edge Label
    \node at ([xshift=-3.5mm,yshift=9mm]localportal.north) {0 .. N};
  \end{tikzpicture}

  \caption{The network structure of standard tests}
  \label{fig:standard-network-structure}
\end{figure}

The network structure of all standard tests is shown in figure \ref{fig:standard-network-structure}. Any deviations from this structure will be mentioned. The Local Portal has as many interfaces as referenced in any test, plus one to connect to the client. All Virtual Machines also have an additional interface for management, but this has no effect on the tests.

\section{Success Criteria}

\subsection{Flow Maintained}

TODO.

\subsection{Bidirectional Performance Gains}

The performance gains measured are visible in both directions (inbound and outbound to the client). The graphs shown in this evaluation section are inbound unless stated otherwise, with the outbound graphs being available in Appendix \ref{appendix:outbound-graphs}.

\begin{figure}
  \centering
  \begin{subfigure}{.49\textwidth}
    \includegraphics[width=0.9\linewidth]{graphs/IES0-R0-1R1-1T10S1-R0-1R1-2T10S2-R0-2R1-2T10}
    \caption{The inbound graph}
    \label{fig:example-inbound}
  \end{subfigure}
  \begin{subfigure}{.49\textwidth}
    \includegraphics[width=0.9\linewidth]{graphs/OES0-R0-1R1-1T10S1-R0-1R1-2T10S2-R0-2R1-2T10}
    \caption{The outbound graph}
    \label{fig:example-outbound}
  \end{subfigure}
  \caption{The same test performed both inbound and outbound}
  \label{fig:example-inbound-outbound}
\end{figure}

Figure \ref{fig:example-inbound-outbound} shows two graphs of the same test - one for the inbound performance and one for the outbound. It can be seen that both graphs show the same shape.

\subsection{IP Spoofing}

\begin{figure}
  \begin{minted}{shell-session}
#IPv4 Forwarding
sysctl -w net.ipv4.ip_forward=1

# Route packets from the remote portal address on the client interface via the tunnel
ip route flush 12
ip route add table 12 to 1.1.1.0/24 via 172.19.152.2 dev nc0
ip rule add from 1.1.1.3 iif eth3 table 12 priority 12

# Route packets to the remote portal address out of the client interface
ip route flush 13
ip route add table 13 to 1.1.1.3 dev eth3
ip rule add to 1.1.1.3 table 13 priority 13
  \end{minted}
  \caption{The script necessary for the Local Portal to accept packets from a client with the spoofed IP.}
  \label{fig:policy-based-routing-script-local-portal}
\end{figure}

This goal was to ensure the Client could use its network interface as if it really had that IP. This is achieved through Policy Based Routing. Example scripts are shown in figure \ref{fig:policy-based-routing-script-local-portal}. Linux also requires the kernel parameter \mintinline{shell-session}{net.ipv4.ip_forward} to be set to 1.

\subsection{Security}

TODO.

\subsection{More Bandwidth over Two Equal Connections}

This success criteria is satisfied, as seen in figure \ref{fig:two-equal-connections}. This graph compares the combined performance of two equal connections to one connection of the same speed and one connection of double the speed. It can be seen that the connection comes very close to the double speed connection, showing that this is an effective method of combining, in terms of raw TCP throughput. These tests are conducted using the TCP method of congestion control.

\begin{figure}
    \centering
    \includegraphics[width=10cm]{graphs/IES0-DR0-1T10S1-R0-1R1-1T10S2-DR0-2T10}
    \caption{A graph demonstrating that two connections proxied lie between one connection of the same speed and one connection of double the speed in terms of throughput}
    \label{fig:two-equal-connections}
\end{figure}

\section{Extended Goals}

\subsection{More Bandwidth over Unequal Connections}

\begin{figure}
  \centering
  \includegraphics[width=10cm]{graphs/IES0-R0-2R1-2T10S1-R0-1R1-2T10S2-R0-1R1-1T10}
  \caption{A graph demonstrating that two unequal connections proxied lie between two connections of the lower bandwidth and two connections of the higher bandwidth}
  \label{fig:unequal-connections}
\end{figure}

This is demonstrated by showing that $1x1MB + 1x2MB$ connections can exceed the performance of $2x1MB$ connections. The results for this can be seen in figure \ref{fig:unequal-connections}, compared against $2x2MB$ and $1x2MB$. It can be seen that the uneven connections fall between the two, which is as expected.

\subsection{More Bandwidth over Four Equal Connections}

\begin{figure}
  \centering
  \begin{subfigure}{.49\textwidth}
    \includegraphics[width=0.9\linewidth]{graphs/IES0-R0-1R1-1R2-1R3-1T10S1-R0-1R1-1R2-1T10S2-R0-1R1-1T10}
    \caption{1MB connections}
    \label{fig:four-equal-connections-1MB}
  \end{subfigure}
  \begin{subfigure}{.49\textwidth}
    \includegraphics[width=0.9\linewidth]{graphs/IES0-R0-2R1-2R2-2R3-2T10S1-R0-2R1-2R2-2T10S2-R0-2R1-2T10}
    \caption{2MB connections}
    \label{fig:four-equal-connections-2MB}
  \end{subfigure}
  \caption{Scaling of equal connections}
  \label{fig:four-equal-connections}
\end{figure}

This criteria is about throughput increasing with the number of equal connections added. It is demonstrated by comparing the throughput of $2x1MB$, $3x1MB$ and $4x1MB$ connections. This can be seen in figure \ref{fig:four-equal-connections-1MB}. A further example is provided of $2x2MB$, $3x2MB$ and $4x2MB$ in figure \ref{fig:four-equal-connections-2MB}.

\subsection{Bandwidth Variation}

This criteria is about the adaptability of the congestion control system. This can be seen in figure \ref{fig:bandwidth-variation}, a graph in which the capacity of one of the two links is decreased. For $0 < t \leq 10$, both links are $2MB$. For $10 < t \leq 20$, one link is decreased to $1MB$ and the other remains at $2MB$. For $20 < t \leq 30$, both links are returned to $2MB$.

\begin{figure}
    \centering
    \includegraphics[width=10cm]{graphs/TIEyS0-R0-2R1-2E10R0-1E20R0-2T30}
    \caption{A graph demonstrating the effect of decreasing the bandwidth of one of the two connections}
    \label{fig:bandwidth-variation}
\end{figure}

It can be seen in the graph that the bandwidth initially stabilises at approximately $30Mbps$, before decreasing at just after $t = 10$. The connection then quickly recovers at $t=20$, to the initial rate.

\subsection{Connection Loss}

TODO.

\subsection{Single Interface Remote Portal}

\begin{figure}
  \begin{minted}{shell-session}
# IPv4 Forwarding
sysctl -w net.ipv4.ip_forward=1
sysctl -w net.ipv4.conf.eth0.proxy_arp=1

# Deliberately break local routing
ip rule add from all table local priority 20
ip rule del 0 || true

# Route packets to the interface but for nc to this host
ip rule add to 1.1.1.3 dport 1234 table local priority 9

# Route packets to the interface but not for nc via the tunnel
ip route flush 10
ip route add table 10 to 1.1.1.3 via 172.19.152.3 dev nc0
ip rule add to 1.1.1.3 table 10 priority 10
  \end{minted}
  \caption{The scripts necessary to allow the Remote Portal to only use a single interface.}
  \label{fig:policy-based-routing-script-remote-portal}
\end{figure}

The single interface Remote Portal is achieved using a similar set of commands to IP Spoofing. The majority of the work is again done by policy based routing, with some kernel parameters needing to be set too. A sample script is shown in figure \ref{fig:policy-based-routing-script-remote-portal}.

\subsection{Connection Metric Values}

Not implemented yet.

\section{Stretch Goals}

\subsection{IPv4/IPv6 Support}

The project is only tested with IPv4.

\subsection{UDP Proxy Flows}

TODO

\subsection{IP Proxy Packets}

The project only supports TCP and UDP flows for carrying the proxied data.

\section{Performance Evaluation}
\label{section:performance-evaluation}

The discussion of success criteria above used slow network connections to test scaling in certain situations. This section will focus on testing how the solution scales, in terms of faster individual connections, and with many more connections. Further, all of the above tests were automated and carried out entirely on virtual hardware. This section will show some 'real-world' data, using a Raspberry Pi 4B and real Internet connections.

\subsection{Faster Connections Scaling}

TODO

\subsection{Number of Connections Scaling}

TODO

\subsection{Real World Testing}

TODO

