Began evaluation

Evaluated many of the success criteria that already have the required
supporting evidence available.

.gitignore

@@ -1,3 +1,4 @@
+_minted-thesis/
 *.tdo*
 .DS_Store*
 *.aux*

Appendix1/appendix1.tex

@@ -1,132 +0,0 @@
-%!TEX root = ../thesis.tex
-% ******************************* Thesis Appendix A ****************************
-\chapter{How to install \LaTeX} 
-
-\section*{Windows OS}
-
-\subsection*{TeXLive package - full version}
-\begin{enumerate}
-\item	Download the TeXLive ISO (2.2GB) from\\
-\href{https://www.tug.org/texlive/}{https://www.tug.org/texlive/}
-\item	Download WinCDEmu (if you don't have a virtual drive) from \\
-\href{http://wincdemu.sysprogs.org/download/}
-{http://wincdemu.sysprogs.org/download/}
-\item	To install Windows CD Emulator follow the instructions at\\
-\href{http://wincdemu.sysprogs.org/tutorials/install/}
-{http://wincdemu.sysprogs.org/tutorials/install/}
-\item	Right click the iso and mount it using the WinCDEmu as shown in \\
-\href{http://wincdemu.sysprogs.org/tutorials/mount/}{
-http://wincdemu.sysprogs.org/tutorials/mount/}
-\item	Open your virtual drive and run setup.pl
-\end{enumerate}
-
-or
-
-\subsection*{Basic MikTeX - \TeX~ distribution}
-\begin{enumerate}
-\item	Download Basic-MiK\TeX (32bit or 64bit) from\\
-\href{http://miktex.org/download}{http://miktex.org/download}
-\item	Run the installer 
-\item	To add a new package go to Start >> All Programs >> MikTex >> Maintenance (Admin) and choose Package Manager
-\item	Select or search for packages to install
-\end{enumerate}
-
-\subsection*{TexStudio - \TeX~ editor}
-\begin{enumerate}
-\item	Download TexStudio from\\
-\href{http://texstudio.sourceforge.net/\#downloads}
-{http://texstudio.sourceforge.net/\#downloads} 
-\item	Run the installer
-\end{enumerate}
-
-\section*{Mac OS X}
-\subsection*{MacTeX - \TeX~ distribution}
-\begin{enumerate}
-\item	Download the file from\\
-\href{https://www.tug.org/mactex/}{https://www.tug.org/mactex/}
-\item	Extract and double click to run the installer. It does the entire configuration, sit back and relax.
-\end{enumerate}
-
-\subsection*{TexStudio - \TeX~ editor}
-\begin{enumerate}
-\item	Download TexStudio from\\
-\href{http://texstudio.sourceforge.net/\#downloads}
-{http://texstudio.sourceforge.net/\#downloads} 
-\item	Extract and Start
-\end{enumerate}
-
-
-\section*{Unix/Linux}
-\subsection*{TeXLive - \TeX~ distribution}
-\subsubsection*{Getting the distribution:}
-\begin{enumerate}
-\item	TexLive can be downloaded from\\
-\href{http://www.tug.org/texlive/acquire-netinstall.html}
-{http://www.tug.org/texlive/acquire-netinstall.html}.
-\item	TexLive is provided by most operating system you can use (rpm,apt-get or yum) to get TexLive distributions
-\end{enumerate}
-
-\subsubsection*{Installation}
-\begin{enumerate}
-\item	Mount the ISO file in the mnt directory
-\begin{verbatim}
-mount -t iso9660 -o ro,loop,noauto /your/texlive####.iso /mnt
-\end{verbatim}
-
-\item	Install wget on your OS (use rpm, apt-get or yum install)
-\item	Run the installer script install-tl.
-\begin{verbatim}
-	cd /your/download/directory
-	./install-tl
-\end{verbatim}
-\item	Enter command `i' for installation
-
-\item	Post-Installation configuration:\\
-\href{http://www.tug.org/texlive/doc/texlive-en/texlive-en.html\#x1-320003.4.1}
-{http://www.tug.org/texlive/doc/texlive-en/texlive-en.html\#x1-320003.4.1} 
-\item	Set the path for the directory of TexLive binaries in your .bashrc file
-\end{enumerate}
-
-\subsubsection*{For 32bit OS}
-For Bourne-compatible shells such as bash, and using Intel x86 GNU/Linux and a default directory setup as an example, the file to edit might be \begin{verbatim}
-edit $~/.bashrc file and add following lines
-PATH=/usr/local/texlive/2011/bin/i386-linux:$PATH; 
-export PATH 
-MANPATH=/usr/local/texlive/2011/texmf/doc/man:$MANPATH;
-export MANPATH 
-INFOPATH=/usr/local/texlive/2011/texmf/doc/info:$INFOPATH;
-export INFOPATH
-\end{verbatim}
-\subsubsection*{For 64bit OS}
-\begin{verbatim}
-edit $~/.bashrc file and add following lines
-PATH=/usr/local/texlive/2011/bin/x86_64-linux:$PATH;
-export PATH 
-MANPATH=/usr/local/texlive/2011/texmf/doc/man:$MANPATH;
-export MANPATH 
-INFOPATH=/usr/local/texlive/2011/texmf/doc/info:$INFOPATH;
-export INFOPATH
-
-\end{verbatim}
-
-
-
-%\subsection{Installing directly using Linux packages} 
-\subsubsection*{Fedora/RedHat/CentOS:}
-\begin{verbatim} 
-sudo yum install texlive 
-sudo yum install psutils 
-\end{verbatim}
-
-
-\subsubsection*{SUSE:}
-\begin{verbatim}
-sudo zypper install texlive
-\end{verbatim}
-
-
-\subsubsection*{Debian/Ubuntu:}
-\begin{verbatim} 
-sudo apt-get install texlive texlive-latex-extra 
-sudo apt-get install psutils
-\end{verbatim}

Appendix2/appendix2.tex

@@ -1,13 +0,0 @@
-%!TEX root = ../thesis.tex
-% ******************************* Thesis Appendix B ********************************
-
-\chapter{Installing the CUED class file}
-
-\LaTeX.cls files can be accessed system-wide when they are placed in the
-<texmf>/tex/latex directory, where <texmf> is the root directory of the user’s \TeX installation. On systems that have a local texmf tree (<texmflocal>), which
-may be named ``texmf-local'' or ``localtexmf'', it may be advisable to install packages in <texmflocal>, rather than <texmf> as the contents of the former, unlike that of the latter, are preserved after the \LaTeX system is reinstalled and/or upgraded.
-
-It is recommended that the user create a subdirectory <texmf>/tex/latex/CUED for all CUED related \LaTeX class and package files. On some \LaTeX systems, the directory look-up tables will need to be refreshed after making additions or deletions to the system files. For \TeX Live systems this is accomplished via executing ``texhash'' as root. MIK\TeX users can run ``initexmf -u'' to accomplish the same thing.
-
-Users not willing or able to install the files system-wide can install them in their personal directories, but will then have to provide the path (full or relative) in addition to the filename when referring to them in \LaTeX.
-

Evaluation/evaluation.tex

@@ -11,3 +11,220 @@
     \graphicspath{{Evaluation/Figs/Vector/}{Evaluation/Figs/}}
 \fi
 
+This chapter will discuss the methods used to evaluate my project and the results gained. The results will be discussed in the context of the success criteria, laid out in the Project Proposal.
+
+This evaluation shows that a network using my method of combining Internet connections can see vastly superior network performance to one without. It will show the benefits to throughput, redundancy, and adaptability.
+
+\section{Evaluation Methodology}
+
+I performed my experiments on a local Proxmox\footnote{\url{https://proxmox.com}} server. To encourage frequent and thorough testing, a harness was built in Python, allowing tests to be added with very small difficulty and repeated with any code changes.
+
+Proxmox was chosen due to its RESTful API, for integration with Python. It provides the required tools to limit connection speeds and disable connections. The server that ran these tests holds only a single other virtual machine which handles routing. This limits the effects of external factors on the tests.
+
+The tests are performed on a Dell R710 Server with the following specifications:
+
+\vspace{5mm} \noindent \textbf{CPU(s)} 16 x Intel(R) Xeon(R) CPU X5667 @ 3.07GHz (2 Sockets)
+\\ \noindent \textbf{Memory} 6 x 2GB DDR3 ECC RDIMMS
+\\ \noindent \textbf{Kernel} Linux 5.4 LTS
+
+
+
+\section{Line Graphs}
+
+The majority of data presented in this section will be in the form of line graphs. These are generated in a consistent format, using a script found in appendix \ref{appendix:graph_generation}.
+
+\begin{figure}
+  \begin{subfigure}{.3\textwidth}
+    \includegraphics[width=0.9\linewidth]{graphs/IS0-R0-1R1-1R2-1R3-1T10S1-R0-1R1-1R2-1T10S2-R0-1R1-1T10}
+    \caption{No error bars}
+    \label{fig:errorbars-none}
+  \end{subfigure}
+  \begin{subfigure}{.3\textwidth}
+    \includegraphics[width=0.9\linewidth]{graphs/IExS0-R0-1R1-1R2-1R3-1T10S1-R0-1R1-1R2-1T10S2-R0-1R1-1T10}
+    \caption{X error bars}
+    \label{fig:errorbars-x}
+  \end{subfigure}
+  \begin{subfigure}{.3\textwidth}
+    \includegraphics[width=0.9\linewidth]{graphs/IEyS0-R0-1R1-1R2-1R3-1T10S1-R0-1R1-1R2-1T10S2-R0-1R1-1T10}
+    \caption{Y error bars}
+    \label{fig:errorbars-y}
+  \end{subfigure}
+  \caption{The structure of graphs throughout this section}
+  \label{fig:errorbars}
+\end{figure}
+
+In figure \ref{fig:errorbars}, examples are shown of the same graph without any error bars, with error bars on the X axis, and with error bars on the Y axis. Error bars for the X axis are plotted as the range of all of the results, while error bars on the Y axis are plotted as $1.5*\sigma$, where $\sigma$ represents the standard deviation of the results.
+
+In figure \ref{fig:errorbars-x}, it is shown that the range of the timestamps provided is incredibly tight. For this reason, I will not be including error bars in the X axis on the graphs shown from this point onwards.
+
+In figure \ref{fig:errorbars-y}, it can be seen that the error bars on the Y axis are far more significant. Thus, error bars will continue to be included in the Y axis.
+
+To generate these results, a fresh set of VMs (Virtual Machines) are created and the software installed on them. Once this is complete, each test is repeated until the coefficient of variance ($\sigma / \mu$, where $\mu$ is the arithmetic mean and $\sigma$ the standard deviation) is below a desired level, or too many attempts have been completed. The number of attempts taken for each series will be shown in the legend of each graph.
+
+\begin{figure}
+  \centering
+  \begin{tikzpicture}[
+      squarednode/.style={rectangle, draw=black!60, fill=red!5, very thick, minimum size=5mm},
+    ]
+
+    % Nodes
+    \node[squarednode] at (0,0) (speedtest)      {Speed Test Server};
+    \node[squarednode] at (4,0) (remoteportal)   {Remote Portal};
+    \node[squarednode] at (8,0) (localportal)    {Local Portal};
+    \node[squarednode] at (12,0) (client)         {Client};
+
+    % Edges
+    \draw[->] ([yshift=6mm]speedtest.north) -- (speedtest.north);
+    \draw[->] ([yshift=6mm]remoteportal.north) -- (remoteportal.north);
+    \draw[->] ([xshift=-7mm,yshift=6mm]localportal.north) -- ([xshift=-7mm]localportal.north);
+    \draw[->] ([yshift=6mm]localportal.north) -- (localportal.north);
+    \draw[->] ([xshift=7mm,yshift=6mm]localportal.north) -- ([xshift=7mm]localportal.north);
+    \draw[->] ([yshift=6mm]client.north) -- (client.north);
+
+    \draw[-] ([yshift=6mm]speedtest.north) -- ([yshift=6mm]localportal.north);
+    \draw[-] ([xshift=7mm,yshift=6mm]localportal.north) -- ([yshift=6mm]client.north);
+
+    % Edge Label
+    \node at ([xshift=-3.5mm,yshift=9mm]localportal.north) {0 .. N};
+  \end{tikzpicture}
+
+  \caption{The network structure of standard tests}
+  \label{fig:standard-network-structure}
+\end{figure}
+
+The network structure of all standard tests is shown in figure \ref{fig:standard-network-structure}. Any deviations from this structure will be mentioned. The Local Portal has as many interfaces as referenced in any test, plus one to connect to the client. All Virtual Machines also have an additional interface for management, but it is not relevant to the tests.
+
+
+
+\section{Success Criteria}
+
+\subsection{Flow Maintained}
+
+TODO.
+
+\subsection{Bidirectional Performance Gains}
+
+The performance gains measured are visible in both directions (inbound and outbound to the client). The graphs shown in this evaluation section are inbound unless stated otherwise, with the outbound graphs being available in Appendix \ref{appendix:outbound-graphs}.
+
+\begin{figure}
+  \begin{subfigure}{.49\textwidth}
+    \includegraphics[width=0.9\linewidth]{graphs/IEyS0-R0-2R1-2T10S1-R0-1R1-2T10S2-R0-1R1-1T10}
+    \caption{The inbound graph}
+    \label{fig:example-inbound}
+  \end{subfigure}
+  \begin{subfigure}{.49\textwidth}
+    \includegraphics[width=0.9\linewidth]{graphs/OEyS0-R0-2R1-2T10S1-R0-1R1-2T10S2-R0-1R1-1T10}
+    \caption{The outbound graph}
+    \label{fig:example-outbound}
+  \end{subfigure}
+  \caption{The same test performed both inbound and outbound}
+  \label{fig:example-inbound-outbound}
+\end{figure}
+
+Figure \ref{fig:example-inbound-outbound} shows two graphs of the same test - one for the inbound performance and one for the outbound. It can be seen that both graphs show the same shape.
+
+\subsection{IP Spoofing}
+
+\begin{figure}
+  \begin{minted}[
+      fontsize=\footnotesize,
+      linenos,
+      breaklines,
+    ]{shell-session}
+#IPv4 Forwarding
+sysctl -w net.ipv4.ip_forward=1
+
+# Route packets from the remote portal's address on the client interface via the tunnel
+ip route flush 12
+ip route add table 12 to 1.1.1.0/24 via 172.19.152.2 dev nc0
+ip rule add from 1.1.1.3 iif eth3 table 12 priority 12
+
+# Route packets to the remote portal's address out of the client interface
+ip route flush 13
+ip route add table 13 to 1.1.1.3 dev eth3
+ip rule add to 1.1.1.3 table 13 priority 13
+  \end{minted}
+  \caption{The script necessary for the Local Portal to accept packets from a client with the spoofed IP.}
+  \label{fig:policy-based-routing-scripts}
+\end{figure}
+
+This goal was to ensure the Client could use its network interface as if it really had that IP. This is achieved through Policy Based Routing. Example scripts are shown in figure \ref{fig:policy-based-routing-scripts}. Linux also requires the kernel parameter \lstinline{net.ipv4.ip_forward} to be set to 1.
+
+\subsection{Security}
+
+TODO.
+
+\subsection{More Bandwidth over Two Equal Connections}
+
+TODO.
+
+
+
+\section{Extended Goals}
+
+\subsection{More Bandwidth over Unequal Connections}
+
+\begin{figure}
+  \centering
+  \includegraphics[height=7cm]{graphs/IEyS0-R0-2R1-2T10S1-R0-1R1-2T10S2-R0-1R1-1T10}
+  \caption{Unequal connections compared against equal of both sides}
+  \label{fig:unequal-connections}
+\end{figure}
+
+This is demonstrated by showing that $1x1MB + 1x2MB$ connections can exceed the performance of $2x1MB$ connections. The results for this can be seen in figure \ref{fig:unequal-connections}, compared against $2x2MB$ and $1x2MB$. It can be seen that the uneven connections fall between the two, which is as expected.
+
+\subsection{More Bandwidth over Four Equal Connections}
+
+\begin{figure}
+  \begin{subfigure}{.49\textwidth}
+    \includegraphics[width=0.9\linewidth]{graphs/IEyS0-R0-1R1-1R2-1R3-1T10S1-R0-1R1-1R2-1T10S2-R0-1R1-1T10}
+    \caption{1MB connections}
+    \label{fig:equal-connections-scaling-1MB}
+  \end{subfigure}
+  \begin{subfigure}{.49\textwidth}
+    \includegraphics[width=0.9\linewidth]{graphs/IEyS0-R0-2R1-2R2-2R3-2T10S1-R0-2R1-2R2-2T10S2-R0-2R1-2T10}
+    \caption{2MB connections}
+    \label{fig:equal-connections-scaling-2MB}
+  \end{subfigure}
+  \caption{Scaling of equal connections}
+  \label{fig:equal-connections-scaling}
+\end{figure}
+
+This criteria is about throughput increasing with the number of equal connections added. It is demonstrated by comparing the throughput of $2x1MB$, $3x1MB$ and $4x1MB$ connections. This can be seen in figure \ref{fig:equal-connections-scaling-1MB}. A further example is provided of $2x2MB$, $3x2MB$ and $4x2MB$ in figure \ref{fig:equal-connections-scaling-2MB}.
+
+\subsection{Bandwidth Variation}
+
+TODO.
+
+\subsection{Connection Loss}
+
+TODO.
+
+\subsection{Single Interface Remote Portal}
+
+TODO.
+
+\subsection{Connection Metric Values}
+
+TODO.
+
+
+
+\section{Stretch Goals}
+
+\subsection{IPv4/IPv6 Support}
+
+The project only supports IPv4.
+
+\subsection{UDP Proxy Datagrams}
+
+The project only supports TCP flows for carrying the proxied data.
+
+\subsection{IP Proxy Packets}
+
+The project only supports TCP flows for carrying the proxied data.
+
+\section{Performance Evaluation}
+
+TODO.
+

GraphGeneration/graphgeneration.tex

@@ -0,0 +1,74 @@
+%!TEX root = ../thesis.tex
+% ********************* Thesis Appendix A - Graph Generation **********************
+\chapter{Graph Generation Script}
+\label{appendix:graph_generation}
+
+\begin{minted}[
+    fontsize=\footnotesize,
+    linenos,
+    breaklines,
+  ]{python}
+from itertools import cycle
+import matplotlib.pyplot as plt
+
+def plot_iperf_results(
+        series: Dict[str, StandardTest],
+        title: str = None,
+        direction = 'inbound',
+        error_bars_x=False,
+        error_bars_y=False,
+        filename=None,
+        start_at_zero=True,
+):
+    if filename in ['png', 'eps']:
+        filename = 'graphs/{}{}{}{}.{}'.format(
+            'I' if direction == 'inbound' else 'O',
+            'Ex' if error_bars_x else '',
+            'Ey' if error_bars_y else '',
+            ''.join(['S{}-{}'.format(i,x.name()) for (i, x) in enumerate(series.values())]),
+            filename,
+        )
+
+    series = {
+        k: (directionInbound if direction == 'inbound' else directionOutbound)[v.name()] for (k, v) in series.items()
+    }
+
+    cycol = cycle('brgy')
+
+    fig = plt.figure()
+    axes = fig.add_axes([0,0,1,1])
+
+    if title is not None:
+        axes.set_title(title, pad=20.0 if True in [len(x.test.events) > 0 for x in series.values()] else None)
+
+    axes.set_xlabel('Time (s)')
+    axes.set_ylabel('Throughput (Mbps)')
+
+    for k, v in series.items():
+        data = v.summarise()
+
+        axes.errorbar(
+            data.keys(),
+            [x/1e6 for x in data.values()],
+            xerr=([x[0] for x in v.time_range().values()], [x[1] for x in v.time_range().values()]) if error_bars_x else None,
+            yerr=[x*1.5/1e6 for x in v.standard_deviation().values()] if error_bars_y else None,
+            capsize=3,
+            ecolor='grey',
+            color=next(cycol),
+            label=k,
+        )
+
+    legend = axes.legend()
+
+    if start_at_zero:
+        axes.set_ylim(bottom=0)
+        axes.set_xlim(left=0)
+
+    if False:
+        for k, v in events.items():
+            axes.axvline(k, linestyle='--', color='grey')
+            axes.annotate(v, (k, 1.02), xycoords=axes.get_xaxis_transform(), ha='center')
+
+    if filename is not None:
+        fig.savefig(filename, bbox_extra_artists=(legend,), bbox_inches='tight', pad_inches=0.3)
+\end{minted}

OutboundGraphs/outboundgraphs.tex

@@ -0,0 +1,8 @@
+%!TEX root = ../thesis.tex
+% ********************** Thesis Appendix B - Outbound Graphs *************************
+
+\chapter{Outbound Graphs}
+\label{appendix:outbound-graphs}
+
+The graphs shown in the evaluation section are Inbound to the Client (unless otherwise specified).
+This appendix contains the same tests but Outbound from the client.

Preamble/preamble.tex

@@ -47,7 +47,8 @@
 % ************************* Algorithms and Pseudocode **************************
 
 %\usepackage{algpseudocode}
-
+\usepackage{minted}
+\usepackage{listings}
 
 % ********************Captions and Hyperreferencing / URL **********************
 
@@ -76,6 +77,9 @@
 \usepackage{pdfpages}
 \usepackage{graphicx}
 
+\usepackage{tikz}
+\usetikzlibrary{positioning}
+
 % ********************************** Tables ************************************
 \usepackage{booktabs} % For professional looking tables
 \usepackage{multirow}

import-graphs.sh

@@ -0,0 +1,14 @@
+#!/bin/bash
+set -e
+
+# Clean up previous graphs
+rm /home/jake/repos/dissertation/4-dissertation/Evaluation/Figs/graphs/* || true
+rm /home/jake/repos/dissertation/4-dissertation/OutboundGraphs/Figs/graphs/* || true
+
+# Copy graphs for evaluation
+cp /home/jake/repos/dissertation/3-evaluation/graphs/I* /home/jake/repos/dissertation/4-dissertation/Evaluation/Figs/graphs/
+cp /home/jake/repos/dissertation/3-evaluation/graphs/OEyS0-R0-2R1-2T10S1-R0-1R1-2T10S2-R0-1R1-1T10.png /home/jake/repos/dissertation/4-dissertation/Evaluation/Figs/graphs/
+
+# Copy graphs for outbound graphs appendix
+cp /home/jake/repos/dissertation/3-evaluation/graphs/O* /home/jake/repos/dissertation/4-dissertation/OutboundGraphs/Figs/graphs/
+

thesis.tex

@@ -174,8 +174,8 @@
 
 \begin{appendices} % Using appendices environment for more functunality
 
-\include{Appendix1/appendix1}
+\include{GraphGeneration/graphgeneration}
-\include{Appendix2/appendix2}
+\include{OutboundGraphs/outboundgraphs}
 
 \end{appendices}