Haizea

While Haizea is running, it collects data that can be analysed offline (accepted/rejected leases, waiting times, etc.). This data is saved to disk when Haizea stops running so, for now, this information is (in practice) only useful for simulation experiments. In the future, Haizea will save data periodically to disk so it can also be analysed online.

The information that is collected can be specified through a series of \emph{probes}. For example, there is a \texttt{best-effort} probe that collects information relevant to best-effort leases, such as the time the lease had to wait in the queue until it was allocated resources. Haizea includes several probes (see Appendix~\ref{app:probes}) and also allows you to write your own probes (see Chapter~\ref{chap:accounting})

The file where the collected data will be saved and the probes to use are specified in the \texttt{[accounting]} section of the configuration file:

\begin{wideshellverbatim}

[accounting]

datafile: /var/tmp/haizea.dat

probes: ar best-effort immediate utilization

\end{wideshellverbatim}

This file is not human-readable, and there are two ways of accessing its contents: using the \texttt{haizea-convert-data} command or programmatically through Python. Both are described in this chapter.

\section{Type of data collected}

Accounting probes can collect three types of data:

\begin{description}

\item[Per-lease data]: Data attributable to individual leases or derived from how each lease was scheduled. For example, as mentioned earlier, the \texttt{best-effort} probe collects the waiting time of each best-effort lease.

\item[Per-run data]: Data from a single run of Haizea. For example, the \texttt{ar} probe collects the total number of AR leases that were accepted and rejected during the entire run.

\item[Counters]: A counter is a time-ordered list showing how some metric varied throughout a single run of Haizea. For example, the \texttt{best-effort} probe keeps track of the length of the queue. 

\end{description}

See Appendix~\ref{app:probes}) for a list of probes included with Haizea, and a description of the specific data they collect.

\section{The \texttt{haizea-convert-data} command}

\label{sec:haizea-convert-data}

The \texttt{haizea-convert-data} command will convert the contents of the data file into a CSV file. 

To print out all the per-lease data, simply run the following: 

\begin{shellverbatim}

haizea-convert-data -t per-lease /var/tmp/haizea.dat

\end{shellverbatim}

This will print out one line per lease, showing its lease ID and all data collected for that lease. Take into account that some fields will be empty, as a probe might collect data just for one specific type of lease (e.g., AR leases will have empty values for the `Waiting time' information collected by the \texttt{best-effort}).

To print out all the per-run data, run the following:

\begin{shellverbatim}

haizea-convert-data -t per-run /var/tmp/haizea.dat

\end{shellverbatim}

To print out a counter, run the following:

\begin{shellverbatim}

haizea-convert-data -t counter -c countername /var/tmp/haizea.dat

\end{shellverbatim}

Where \texttt{countername} should be substituted for the counter you want to access. If you do not know what counters are included in the file, the following will print out a list of counters:

\begin{shellverbatim}

haizea-convert-data -l /var/tmp/haizea.dat

\end{shellverbatim}

When running multiple simulations (as described in \ref{sec:multiplesim}), Haizea will generate one data file for each simulation profile, which are all stored in the same directory. \texttt{haizea-convert-data} can also be used to produce aggregate statistics from all these data files. For example:

\begin{shellverbatim}

haizea-convert-data -t per-run /var/tmp/results/*.dat

\end{shellverbatim}

This will print out one line per simulation run, each with the per-run data for the run along with the simulation profile, tracefile, and injection file used in that run. Similarly, you can run \texttt{haizea-convert-data} with \texttt{-t per-lease} or \texttt{-t counter} to print the per-lease data or a counter from multiple simulation runs, using the simulation profile, tracefile, and injection file columns to disambiguate the run the data originated from.

\section{Analysing data programmatically} 

The data file generated by Haizea is a Python-pickled \texttt{AccountingData} object. This object contains all the per-lease and per-run data, along with all the counters. You can analyse the data programmatically by unpickling the file from your own Python code and accessing the data contained in the \texttt{AccountingData} object (see the generated pydoc documentation linked from the Haizea website for details on the object's attributes). An example of how this file is unpickled, and some of its information accessed, can be found in function \texttt{haizea\_convert\_data} in module \texttt{haizea.cli.commands}.