annotate docs/ProjectSpecificationDocument.tex @ 1543:45363ee3632d

SMC paper: added boxplot graph, input XML, decreased itemize spacing (also changed index.html page title)

\documentclass{article}

\usepackage[margin=2cm]{geometry}
\usepackage{listings}

\begin{document}

\large APE Browser Tool - Project Specification Document

\section{Document}

An XML file containing all project information to load and execute the project on the client. Certain interfaces are optional, however others are mandatory. This guide should reflect the changes in the XML project and keep track of the versions. Hopwfully this can remain simple!

\section{Root}

The XML root must be \texttt{<BrowserEvalProjectDocument>}. This should be sufficiently identifiable in both itself and in the JavaScript decoding as it will create an object called the root name.

There must also be a \texttt{<version>} tag which has the attribute \texttt{id} containing a numerical representation of the version. Currently everything in this document can be assumed to be version 1. If future updates or corrections are made post delivery this should give the flexibility to ensure past projects still work.

The root will also contain the following tags: setup and tracks.

\section{Setup tag}

The setup tag specifies certain global test settings including: the interface type to use, the project return location and any other setup instructions.
Any general pre/post test questions must be specified in the relevant children tag. Any enabled metrics must also be specified in the metric child node.

\subsection{Attributes}
\begin{itemize}
\item \texttt{interface} - Mandatory, String. Defaults to APE, otherwise use to load any of the available interfaces. Currently only valid string is APE.
\item \texttt{projectReturn} - Mandatory, String. Specify the URL to return the test results. If null client will generate XML locally and prompt user to return the file.
\item \texttt{randomiseOrder} - Optional, default to false. Specify if the order of the tests can be randomised.
\item \texttt{collectMetrics} - Optional, Boolean. Default to false. Determine if the test metrics should be collected. These include how long each test session took etc. The full metrics list can be modified in the 'metrics' tag.
\end{itemize}

\subsection{Elements}
None

\section{AudioHolder tag}

There should be one audioHolder tag per test session, inside which each audioElement is specified as children. The audioHolder tag can help to generalise certain objects. Each audioHolder instance specifies a separate listening test to be paged, each with their own specific requirements.

\subsection{Attributes}
\begin{itemize}
\item \texttt{id} - Mandatory, String. Give an ID string or number to identify the test in the result.
\item \texttt{hostURL} - Optional, String. If all tracks are hosted from the same folder on a server, you can put in the lead here. For instance, if loading http://test.com/tracks/track1.wav and http://test.com/tracks/track2.wav, this could equal http://test.com/tracks/ and the url attribute in the track tag can be track1.wav or track2.wav. Equally http://test.com/ and then using tracks/track1.wav and tracks/track2.wav is valid.
\item \texttt{sampleRate} - Optional, Number. If your test requires a specific sample rate, this should be set to the desired sample rate in Hertz. This does not set the browser to the correct sample rate, but forces the browser to check the sample rate matches. If this is undefined, no sample rate matching will occur.
\item \texttt{randomiseOrder} - Optional, Boolean String. Defaults to false. Determine if the track order should be randomised. Must be true or false.
\item \texttt{repeatCount} - Optional, Number. Defaults to 0 (ie: no repeats). The number of times a test should be repeated.
\item \texttt{loop} - Optional, Boolean String. Defaults to false. Enable if audioElements should loop their playback or not.
\end{itemize}

\subsection{Elements}
Contain the audioElements tags and the interfaceSetup tag.

\section{audioElements tag}

This must reside as children in the audioHolder tag. There must be one audioElement tag per sound sample to load into the test.

\subsection{Attributes}
\begin{itemize}
\item \texttt{id} - Mandatory, String. Must give a string or number to identify each audio element. This id is used in the output to identify each track once randomised.
\item \texttt{url} - Mandatory, String. Contain the full URL to the track. If the Tracks tag hostURL is set, concatenate this tag with the hostURL attribute to obtain the full URL.
\end{itemize}

\section{interface tag}

This is contained within the audioHolder tag and outlines test instance specific requirements. These include the following children tags:
\begin{itemize}
\item 'title' - Contains the test title to be shown at the top of the page. Can only be one title node per interface.
\item 'scale' - Takes the attribute position to be a value between 0 and 100 indicating where on the scale to place the text contained inside. Can be multiple scale tags per interface.
\end{itemize}

\section {CommentQuestion tag}

This is a 1st level tag (same level as AudioHolder and setup). This allows another question and comment box to be presented on the page. The results of these are passed back in the results XML with both the comment and the question. The id attribute is set to keep track at the results XML.

\section {PreTest tag and PostTest tag}

These are 1st level tags. The PreTest tag allows for the specifying of pre test instructions and questions. These appear as a pop-up style window with next buttons and other automatic GUI. The postTest tag allows for specifying post test instructions, questions and resources. These appear as a pop-up style window after the submit button is pressed.

\subsection{Attributes}
None.

\subsection{Elements}
Takes the \texttt{statement} and \texttt{question} tags. The order these are presented in the XML define the order they appear on the screen.

\subsubsection{Statement}

The statement tag simply prints the included string verbatim on a 'pop-up' window with a next button.

\subsubsection{Question}

This allows for a question to be asked pre/post the test. This is added to the response XML in the same location as the other common/global questions. The response includes both the question asked and the response. This takes two attributes, id and mandatory. ID is a mandatory field. The same ID will be used in the results so it is important it is properly entered. Mandatory is optional. True means the field must be entered before continuing.

\subsubsection{Resource}

The resource tag is only available in the postTest tag. This allows for the linking to some external resource via the href attribute.

\section{Metric tag}
A 1st level tag, metrics must be declared in the setup tag. This takes a set of children 'metricEnable' to define which metrics to collect and present.

\subsection{metricEnable tag}
This takes a single attribute to determine which metric to enable for collection. Some of these are a global, per track or per test instance.
\begin{itemize}
\item testTimer - Return the global test timer and test instance timers. Measures the time between the first start and final submit.
\item elementTimer - Return the total time each audioElement in each test was listened too. Measures time between successive clicks on the track changer
\item elementTracker - Return the initial position of each track
\item elementTrackerFull - Return an enumerated pair of time and position. Track the entire movement of each element position. NOTE: Will override the elementTracker option above and throw an error into the browser console.
\item elementFlagListenedTo - Return a boolean per elementck to see if the element was listened to
\item elementFlagMoved - Return a boolean per element to see if the element slider was moved.
\item elementFlagComments - Return a boolean per element to see if the element has comments.
\end{itemize}

\section{Feature List}
\begin{itemize}
\item Paging listening tests - eg. Ask multiple questions in each experiment
\item Labels on X axis - scale
\item Input questions/comment at top to guide towards the question being asked.
\item Randomise track numbers -(inc. comment boxes and relate back to correct reference track)
\item Randomise order of individual tests
\item Save output XML file to remote server
\item Tests Metrics
\begin{itemize}
\item Duration of listening to each track
\item Time spent on each individual test
\item Start and end position of every track
\item Flags on each track, to ensure each track (but may not restrict users from submitting)
\begin{itemize}
\item Has been listened to 
\item Has been moved
\item Has comments about it
\end{itemize}
\end{itemize}
\end{itemize}

\subsection{Advanced feature list}
\begin{itemize}
\item Repeat each tests number of times (2 or 3?) to remove learning / experience bias and ensure that the order is consistent
\item Perform Loudness equalisation on all tracks
\item Selection of test type
\item Pre-test of some basic hearing test
\begin{itemize}
\item MUSHRA (with vertical slider per track)
\item APE (Single horizontal slider)
\item AB Test
\end{itemize}
\end{itemize}



\section{Example}

Here is an example XML structure

\begin{lstlisting}
<?xml version="1.0" encoding="utf-8"?>
<BrowserEvalProjectDocument>
	<setup interface="APE" projectReturn="null" randomiseOrder='true' collectMetrics='true'>
		<PreTest>
			<statement>Please listen to all mixes</statement>
			<question id="location" mandatory="true">Please enter your listening location</question>
		</PreTest>
		<PostTest>
			<statement>Thank you for taking this listening test.</statement>
			<question id="SessionID">Please enter your name.</question>
		</PostTest>
		<Metric>
			<metricEnable>testTimer</metricEnable>
			<metricEnable>elementTimer</metricEnable>
			<metricEnable>elementTracker</metricEnable>
			<metricEnable>elementFlagListenedTo</metricEnable>
			<metricEnable>elementFlagMoved</metricEnable>
		</Metric>
	</setup>
	<audioHolder id='0' hostURL="example_eval/" sampleRate="44100" randomiseOrder='true' repeatCount='1'>
		<interface>
			<title>Example Test Question</title>
			<scale position="0">Min</scale>
			<scale position="100">Max</scale>
			<scale position="50">Middle</scale>
			<scale position="20">20</scale>
		</interface>
		<audioElements url="0.wav" id="0"/>
		<audioElements url="1.wav" id="1"/>
		<audioElements url="2.wav" id="2"/>
		<audioElements url="3.wav" id="3"/>
		<audioElements url="4.wav" id="4"/>
		<audioElements url="5.wav" id="5"/>
		<audioElements url="6.wav" id="6"/>
		<audioElements url="7.wav" id="7"/>
		<audioElements url="8.wav" id="8"/>
		<audioElements url="9.wav" id="9"/>
		<audioElements url="10.wav" id="10"/>
		<CommentQuestion id='mixingExperiance'>What is your mixing experiance</CommentQuestion>
		<PreTest>
			<statement>Start the Test 3</statement>
		</PreTest>
		<PostTest>
			<statement>Please take a break before the next test</statement>
			<question id="testComment">How did you find the test</question>
		</PostTest>
	</audioHolder>
</BrowserEvalProjectDocument>
\end{lstlisting}



\end{document}