Update on Overleaf.
This commit is contained in:
overleaf
parent
5e5035fc68
commit
e2b8e1f8aa
@ -14,17 +14,17 @@
|
||||
|
||||
In this chapter...
|
||||
|
||||
TODO
|
||||
\mynote{Write this.}
|
||||
|
||||
% ------------------------- Project Structure ------------------------------ %
|
||||
\section{Structure}
|
||||
|
||||
TODO: How the project is structured overall, including the builder class, and a UML diagram.
|
||||
\mynote{How the project is structured overall, including the builder class, and a UML diagram.}
|
||||
|
||||
% ------------------------------- Proxy ------------------------------------ %
|
||||
\section{Proxy}
|
||||
|
||||
The central structure for the operation of the software is \verb'proxy/Proxy struct'. The proxy is defined by its source and sink, and provides methods for \verb'AddConsumer' and \verb'AddProducer'. The proxy coordinates the dispatching of sourced packets to consumers, and the delivery of produced packets to the sink. This follows the packet data path shown in figure \ref{fig:proxy-start-data-flow}.
|
||||
The central structure for the operation of the software is the \verb'proxy/Proxy' struct. The proxy is defined by its source and sink, and provides methods for \verb'AddConsumer' and \verb'AddProducer'. The proxy coordinates the dispatching of sourced packets to consumers, and the delivery of produced packets to the sink. This follows the packet data path shown in figure \ref{fig:proxy-start-data-flow}.
|
||||
|
||||
\begin{figure}
|
||||
\centering
|
||||
@ -62,9 +62,9 @@ The central structure for the operation of the software is \verb'proxy/Proxy str
|
||||
\label{fig:proxy-start-data-flow}
|
||||
\end{figure}
|
||||
|
||||
The proxy is implemented to take a consistent sink and source and accept consumers and producers that vary over the lifetime. This is due to the nature of producers and consumers, as a consumer may be either ephemeral or persistent, depending on the configuration. As the configuration is deliberately intended to be flexible, both of these can exist within the same proxy. An example is a device that accepts TCP connections and has outbound UDP connections. In such a case, the TCP producers and consumers would be ephemeral, existing only until they are closed by the far side. The UDP producers and consumers are persistent, as control of reconnection is handled by this proxy.
|
||||
The proxy is implemented to take a consistent sink and source and accept consumers and producers that vary over the lifetime. This is due to the nature of producers and consumers, as each may be either ephemeral or persistent, depending on the configuration. An example is a device that accepts TCP connections and has outbound UDP connections. In such a case, the TCP producers and consumers would be ephemeral, existing only until they are closed by the far side. The UDP producers and consumers are persistent, as control of reconnection is handled by this proxy. As the configuration is deliberately intended to be flexible, both of these can exist within the same proxy instance.
|
||||
|
||||
The structure of the proxy is built around the flow graph in figure \ref{fig:proxy-start-data-flow}. The data flow demonstrates the four transfers of data that occur: packet source to source queue, source queue to consumer, producer to sink queue, and sink queue to packet sink. For the former and latter, these exist once for an instance of the proxy. The others run once for each consumer or producer. As was mentioned previously, producers and consumers are considered ephemeral, and thus may come and go throughout the lifetime of the proxy. Basic examples of the logic applied for each flow are given in figure \ref{fig:proxy-loops}.
|
||||
The structure of the proxy is built around the flow graph in figure \ref{fig:proxy-start-data-flow}. The data flow demonstrates the four transfers of data that occur: packet source to source queue, source queue to consumer, producer to sink queue, and sink queue to packet sink. For the former and latter, these exist once for an instance of the proxy. The others run once for each consumer or producer. Basic examples of the logic applied for each flow are given in figure \ref{fig:proxy-loops}.
|
||||
|
||||
\begin{figure}
|
||||
\centering
|
||||
@ -104,7 +104,7 @@ while is_alive(producer):
|
||||
\label{fig:proxy-loops}
|
||||
\end{figure}
|
||||
|
||||
Although the pseudocode given in figure \ref{fig:proxy-loops} is incredibly simple, aside from error handling, this is as implemented in the Go code. Go's cooperative scheduler and lightweight Goroutines make this an efficient implementation. However, given the expected quantities of simultaneously connected consumers and producers is low, heavier OS threads would also be effective here. The queues are further trivial to implement in Go, as channels provide all of the necessary functionality, but can also be implemented in other languages.
|
||||
Although the pseudocode given in figure \ref{fig:proxy-loops} is incredibly simple, aside from error handling, this is as implemented in the Go code. Go's cooperative scheduler and lightweight Goroutines make this an efficient implementation. However, given the expected quantities of simultaneously connected consumers and producers is low, heavier OS threads would also be effective here. The queues are further trivial to implement in Go, as channels provide all of the necessary functionality, but can also be implemented in other languages. The lifetime of producers and consumers are controlled by the lifetime of the aforementioned loops, and are only referenced within them, such that the garbage collector can collect any producers and consumers for which the loops have exited.
|
||||
|
||||
Finally is the aforementioned ability for the central proxy to restart consumers or producers that support it (thus far, those initiated by the proxy in question). This causes the wrapping of the loops shown in figure \ref{fig:proxy-loops} in an additional layer. Pseudocode for the expansion to the consumer is shown in figure \ref{fig:proxy-loops-restart}, with the providers being expanded similarly.
|
||||
|
||||
@ -125,25 +125,25 @@ while is_reconnectable(consumer)
|
||||
% ------------------------- Builder / Config ------------------------------- %
|
||||
\section{Configuration}
|
||||
|
||||
The configuration format chosen was INI, extended with multiple duplicate names. Included is a single Host section, followed by multiple Peer sections specific to a method of communicating with the other side. Processing the configuration file is split into three parts: loading the configuration file into a Go struct, validating the configuration file, and building a proxy from the loaded configuration.
|
||||
The configuration format chosen was INI, extended with duplicate names. Included is a single Host section, followed by multiple Peer sections specific to a method of communicating with the other side. Processing the configuration file is split into three parts: loading the configuration file into a Go struct, validating the configuration file, and building a proxy from the loaded configuration.
|
||||
|
||||
Validation of the configuration file is included to discover configuration errors prior to building an invalid proxy. Firstly, this ensures that all parts of the program built from the configuration are given values which are invalid in context and easily verifiable, such as a TCP port of above 65,535. Secondly, and perhaps more importantly, catching errors in configuration before attempting to build the proxy constrains the errors of an invalid configuration to a single location. For a user, this might mean that an error such as \verb'Peer[1].LocalPort invalid: max 65535; given 74523' is shown, as opposed to \verb'tcp: invalid address', which shows the user's error as opposed to a bug in the code.
|
||||
Validation of the configuration file is included to discover configuration errors prior to building an invalid proxy. Firstly, this ensures that all parts of the program built from the configuration are given values which are invalid in context and easily verifiable, such as a TCP port of above 65,535. Secondly, catching errors in configuration before attempting to build the proxy constrains the errors of an invalid configuration to a single location. For a user, this might mean that an error such as \verb'Peer[1].LocalPort invalid: max 65535; given 74523' is shown, as opposed to \verb'tcp: invalid address', which shows the user's error as opposed to a bug in the code.
|
||||
|
||||
Once a configuration is validated, the proxy is built. This is a simple case of creating the proxy from the given data and adding the producers and consumers for its successful running.
|
||||
|
||||
TODO: This is shit.
|
||||
This builder structure is also useful for a Go project, as it helps avoid circular imports, which are banned in Go. An example is a TCP flow implementing \verb'proxy.Consumer'. The proxy package cannot import TCP to create flows, so this must be delegated to another package. A builder package can bridge the gap, while maintaining a close link between the configuration and a single place where it is built.
|
||||
|
||||
% ------------------------- Sources and Sinks ------------------------------ %
|
||||
\section{Sourcing and Sinking Packets}
|
||||
|
||||
A source and sink of packets are the names given to the method of entry for packets to tunnel and the method of exit for packets received. In the current implementation, both of these are achieved with a TUN adapter. A TUN adapter is a virtual network interface provided by the kernels of multiple unices, allowing userspace code to interact with Layer 3 networking.
|
||||
|
||||
TODO
|
||||
\mynote{Write this.}
|
||||
|
||||
% ---------------------- Providers and Consumers --------------------------- %
|
||||
\section{Providers and Consumers}
|
||||
\section{Producers and Consumers}
|
||||
|
||||
Providers and consumers were designed with flexibility in mind. It has already been mentioned that they can be either ephemeral or persistent, and this is achieved by implementation of an additional interface, \verb'Reconnectable'. The three relevant interfaces for producers and consumers are given in \ref{fig:producer-consumer-interfaces}.
|
||||
Producers and consumers were designed with flexibility in mind. It has already been mentioned that they can be either ephemeral or persistent, and this is achieved by implementation of an additional interface, \verb'Reconnectable'. The three relevant interfaces for producers and consumers are given in \ref{fig:producer-consumer-interfaces}.
|
||||
|
||||
\begin{figure}
|
||||
\begin{minted}{go}
|
||||
@ -165,60 +165,61 @@ type Reconnectable interface {
|
||||
\label{fig:producer-consumer-interfaces}
|
||||
\end{figure}
|
||||
|
||||
TODO
|
||||
\mynote{Write this.}
|
||||
|
||||
% --------------------------------- TCP ------------------------------------ %
|
||||
\section{TCP}
|
||||
|
||||
The base implementation for a producer and consumer takes advantage of TCP. The requirements for the load balancing given above to function are simple: flow control and congestion control. TCP provides both of these, so was an obvious base solution. However, TCP also provides unnecessary overhead, which will go on to be discussed further.
|
||||
The base implementation for producers and consumers takes advantage of TCP. The requirements for the load balancing given above to function are simple: flow control and congestion control. TCP provides both of these, so was an obvious base solution. However, TCP also provides unnecessary overhead, which will go on to be discussed further.
|
||||
|
||||
TCP is a stream oriented connection, while the packets to be sent are discrete units. That is, a TCP flow cannot be connected directly to a TUN adapter, as the TUN adapter expects discrete and formatted IP packets while the TCP connection sends a stream of bytes. To resolve this, each packet sent across a TCP flow is prefixed with the length of the packet. On the sending side, this involves writing the 32-bit length of the packet, followed by the packet itself. For the receiver, first four bytes are read to recover the length of the next packet, and then that many packets can be read. This successfully punctuates the stream oriented connection into a packet based connection.
|
||||
TCP is a stream oriented connection, while the packets to be sent are discrete units. That is, a TCP flow cannot be connected directly to a TUN adapter, as the TUN adapter expects discrete and formatted IP packets while the TCP connection sends a stream of bytes. To resolve this, each packet sent across a TCP flow is prefixed with the length of the packet. On the sending side, this involves writing the 32-bit length of the packet, followed by the packet itself. For the receiver, first four bytes are read to recover the length of the next packet, after which that many bytes are read. This successfully punctuates the stream oriented connection into a packet based connection.
|
||||
|
||||
However, using TCP to tunnel TCP packets (known as TCP-over-TCP) can cause a degradation in performance in non-ideal circumstances \citep{honda_understanding_2005}. Further, using TCP to tunnel IP packets provides a superset of the required guarantees, in that reliable delivery and ordering are guaranteed. Reliable delivery can cause a decrease in performance for tunnelled flows which do not require reliable delivery, such as a live video stream - a live stream does not wish to wait for a packet to be redelivered from a portion that is already played, and thus will spend longer buffering than if it received the up to date packets instead. Ordering can limit performance when tunnelling multiple streams, as a packet for a phone call could already be received, but instead has to wait in a buffer for a packet for a download to arrive, increasing latency unnecessarily.
|
||||
|
||||
Although the TCP implementation provides an excellent proof of concept and basic implementation, work therefore moved on to a second UDP implementation. However, the TCP implementation is functionally correct, so is left as an option, furthering the idea of flexibility maintained throughout this project.
|
||||
Although the TCP implementation provides an excellent proof of concept and basic implementation, work moved to a second UDP implementation, aiming to solve some of these problems. However, the TCP implementation is functionally correct, so is left as an option, furthering the idea of flexibility maintained throughout this project. In cases where a connection that suffers particularly high packet loss is combined with one which is more stable, TCP could be employed on the high loss connection to limit overall packet loss. The effectiveness of such a solution would be implementation specific, so is left for the architect to decide.
|
||||
|
||||
% --------------------------------- UDP ------------------------------------ %
|
||||
\section{UDP}
|
||||
|
||||
To resolve the issues seen with TCP, an implementation using UDP was built as an alternative. UDP differs from TCP in that it provides almost no guarantees, and is based on sending discrete datagrams as opposed to a stream of bytes. However, UDP datagrams don't provide the congestion control or flow control required, so this must be built on top of the protocol. As the flow itself can be managed in userspace, as opposed to the TCP flow which is managed in kernel space, more flexbility is available in implementation. This allows correct packets to be immediately dispatched, with little regard for ordering.
|
||||
To resolve the issues seen with TCP, an implementation using UDP was built as an alternative. UDP differs from TCP in that it provides almost no guarantees, and is based on sending discrete datagrams as opposed to a stream of bytes. However, UDP datagrams don't provide the congestion control or flow control required, so this must be built on top of the protocol. As the flow itself can be managed in userspace, opposed to the TCP flow which is managed in kernel space, more flexibility is available in implementation. This allows received packets to be immediately dispatched, with little regard for ordering.
|
||||
|
||||
\subsection{Congestion Control}
|
||||
|
||||
TODO
|
||||
\mynote{Write this.}
|
||||
|
||||
\subsubsection{New Reno}
|
||||
|
||||
TODO
|
||||
\mynote{Write this.}
|
||||
|
||||
% ------------------------------- Routing ---------------------------------- %
|
||||
\section{System Configuration}
|
||||
|
||||
TODO
|
||||
\mynote{Write this.}
|
||||
|
||||
\subsection{Routing}
|
||||
|
||||
TODO
|
||||
\mynote{Write this.}
|
||||
|
||||
\subsection{Buffers}
|
||||
|
||||
TODO
|
||||
\mynote{Write this.}
|
||||
|
||||
% ----------------------------- Security ----------------------------------- %
|
||||
\section{Security}
|
||||
|
||||
TODO
|
||||
\mynote{Write this.}
|
||||
|
||||
\subsection{Exchanges}
|
||||
|
||||
Cryptographic exchanges are represented by the interface given in figure \ref{fig:crypto-exchanges-interface}. The interface was chosen to be absolutely flexible, while maintaining simplicity. The first exchange implemented is the \verb'None' exchange, which should implement no exchange at all.
|
||||
Cryptographic exchanges are represented by the interface given in figure \ref{fig:crypto-exchanges-interface}. The interface was chosen to be absolutely flexible, while maintaining simplicity. The \verb'Next' method returns two byte arrays. The first contains the message that should be sent back to the far side, if there is one. The second contains data taken from the packet, if it contains any. This allows for exchanges such as the TCP three way handshake, in which the final message contains data as well as setup. The first exchange implemented is the \verb'None' exchange, which should implement no exchange at all, and is shown in figure \ref{fig:crypto-exchanges-none}.
|
||||
|
||||
\begin{figure}
|
||||
\centering
|
||||
\begin{minted}{go}
|
||||
type Exchange interface {
|
||||
First() []byte
|
||||
Next([]byte) ([]byte, []byte, error)
|
||||
IsDone() bool
|
||||
First() (out []byte, err error)
|
||||
Next(in []byte) (out []byte, data []byte, err error)
|
||||
}
|
||||
\end{minted}
|
||||
\caption{Cryptographic exchanges interface.}
|
||||
@ -230,18 +231,35 @@ type Exchange interface {
|
||||
\begin{minted}{go}
|
||||
type None struct{}
|
||||
|
||||
func (None) First() []byte {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (None) Next(b []byte) ([]byte, []byte, error) {
|
||||
return nil, b, nil
|
||||
}
|
||||
func (None) IsDone() bool { return true }
|
||||
func (None) First() ([]byte, error) { return nil, nil }
|
||||
func (None) Next(b []byte) ([]byte, []byte, error) { return nil, nil, nil }
|
||||
\end{minted}
|
||||
\caption{Cryptographic exchanges interface.}
|
||||
\caption{Empty cryptographic exchange implementation.}
|
||||
\label{fig:crypto-exchanges-none}
|
||||
\end{figure}
|
||||
|
||||
The exchanges are designed to be flexible for any packet transport method. The flow of using an exchange is shown in pseudocode in figure \ref{fig:crypto-exchanges-pseudocode}. The flow manages all packets exchanged by the connection, until the exchange confirms that it is done, or fails. The exchange can also produce data during the exchange, which is handed off to the normal packet handling mechanism, represented in the pseudocode by the \verb'yield' keyword.
|
||||
|
||||
\begin{figure}
|
||||
\centering
|
||||
\begin{minted}{python}
|
||||
if !exchange.IsDone():
|
||||
if this_side_goes_first:
|
||||
send(try(exchange.First()))
|
||||
|
||||
while !exchange.IsDone():
|
||||
packet = receive()
|
||||
next, out = try(exchange.Next(packet))
|
||||
if out != nil:
|
||||
yield out
|
||||
\end{minted}
|
||||
\caption{Pseudocode for implementing the cryptographic exchange interface for a transport.}
|
||||
\label{fig:crypto-exchanges-pseudocode}
|
||||
\end{figure}
|
||||
|
||||
\mynote{Reference implemented exchange from preparation once finalised.}
|
||||
|
||||
\subsection{Repeated Packets}
|
||||
\label{section:implementation-repeated-packets}
|
||||
|
||||
@ -287,12 +305,12 @@ The security features presented in this section form a hierarchy of attachment t
|
||||
% ----------------------------- Testing ------------------------------------ %
|
||||
\section{Testing and Evaluation}
|
||||
|
||||
TODO
|
||||
\mynote{Write this.}
|
||||
|
||||
% ------------------------ Repository Overview ----------------------------- %
|
||||
\section{Repository Overview}
|
||||
|
||||
TODO
|
||||
\mynote{Write this.}
|
||||
|
||||
\begin{figure}
|
||||
\dirtree{%
|
||||
@ -318,11 +336,15 @@ TODO
|
||||
\label{fig:repository-structure}
|
||||
\end{figure}
|
||||
|
||||
% ------------------------------ Summary ----------------------------------- %
|
||||
\section{Summary}
|
||||
|
||||
\mynote{Write this.}
|
||||
|
||||
% GARBAGE
|
||||
% DO NOT CONTINUE
|
||||
% OLD STUFF
|
||||
TODO: Delete from here to bottom.
|
||||
\mynote{Delete from here to bottom.}
|
||||
% ----------------------- Greedy Load Balancing ---------------------------- %
|
||||
\section{Greedy Load Balancing}
|
||||
|
||||
|
||||
@ -201,12 +201,15 @@
|
||||
|
||||
%\ifsetDraft
|
||||
% \usepackage[colorinlistoftodos]{todonotes}
|
||||
% \newcommand{\mynote}[1]{\todo[author=kks32,size=\small,inline,color=green!40]{#1}}
|
||||
% \newcommand{\mynote}[1]{\todo[author=jsh77,size=\small,inline,color=green!40]{#1}}
|
||||
%\else
|
||||
% \newcommand{\mynote}[1]{}
|
||||
% \newcommand{\listoftodos}{}
|
||||
%\fi
|
||||
|
||||
\usepackage[colorinlistoftodos]{todonotes}
|
||||
\newcommand{\mynote}[1]{\todo[author=jsh77,size=\small,inline,color=orange!40]{#1}}
|
||||
|
||||
% Example todo: \mynote{Hey! I have a note}
|
||||
|
||||
% ******************************** Highlighting Changes **********************************
|
||||
|
||||
Loading…
Reference in New Issue
Block a user