# HG changeset patch # User Dave Moffat # Date 1429814174 -3600 # Node ID 684cbfb0e696e9726d87ded089904ca646429fec # Parent 7586646a6a0572cebb652efe77e102f6a4ce7766# Parent 94eacf4b03a960ea4d2a706da55722f0a0aa1f06 Commit Merge diff -r 7586646a6a05 -r 684cbfb0e696 docs/SMC15/smc2015template.tex --- a/docs/SMC15/smc2015template.tex Thu Apr 23 19:34:01 2015 +0100 +++ b/docs/SMC15/smc2015template.tex Thu Apr 23 19:36:14 2015 +0100 @@ -197,16 +197,28 @@ \section{Architecture}\label{sec:architecture} % or implementation? -The tool runs entirely inside the browser through the new HTML5 Web Audio API. The API is supported by most major web browsers (with the exception of Internet Explorer) and allows for constructing a chain of audio processing elements to produce a high quality, real time signal process to manipulate audio streams. The API supports multi-channel processing and has an accurate playback timer for precise scheduled playback control. The Web Audio API is controlled through the browser JavaScript and is therefore highly configurable. The Web Audio API processing is all controlled in a separate thread to the main JavaScript thread, meaning there is no blocking due to real time processing. +The tool runs entirely inside the browser through the new HTML5 Web Audio API and is supported by most major web browsers (with the exception of Internet Explorer). The API allows for constructing audio processing elements and connecting them together to produce a high quality, real time signal process to manipulate audio streams. The API supports multi-channel processing and has an accurate playback timer for precise scheduled playback control. The API is controlled through the browser JavaScript engine and is therefore highly configurable. Processing is all performed in a separate, low latency thread to the main JavaScript thread, meaning there is no blocking due to real time processing. The web tool itself is split into several files to operate: \begin{itemize} -\item \texttt{index.html}: The main index file to load the scripts, this is the file the browser must request to load. %This should be renamed index.html, but will wait until the file is renamed in the repo. -\item \texttt{core.js}: Contains functions and objects to manage the audio control, audio objects for testing and loading of files. +\item \texttt{index.html}: The main index file to load the scripts, this is the file the browser must request to load. +\item \texttt{core.js}: Contains functions and objects to define the audio objects, audio playback engine and loading media files \item \texttt{ape.js}: Parses setup files to create the interface as instructed, following the same style chain as the MATLAB APE Tool \cite{deman2014b}. \end{itemize} The HTML file loads the \texttt{core.js} file along with a few other ancillary files (such as the jQuery JavaScript extensions)% should we cite jQuery.... https://jquery.com/ -, at which point the browser JavaScript begins to execute the on-page instructions, which gives the URL of the test setup XML document (outlined in the next section). \texttt{core.js} parses this document and executes the function in \texttt{ape.js} to build the web page with the given audio files. The reason for separating these two files is to allow for further interface designs (such as MUSHRA \cite{mushra} or A-B tests \cite{bech}) to be used, which would still require the same underlying core functions outlined in \texttt{core.js}, see also Section \ref{sec:interface}. +, at which point the browser JavaScript begins to execute the on-page instructions, which gives the URL of the test setup XML document (outlined in Section \ref{sec:setupresultsformats}). \texttt{core.js} parses this document and executes the function in \texttt{ape.js} to build the web page with the given audio files. The reason for separating these two files is to allow for further interface designs (such as MUSHRA \cite{mushra} or A-B tests \cite{bech}) to be used, which would still require the same underlying core functions outlined in \texttt{core.js}, see also Section \ref{sec:interface}. -The \texttt{ape.js} file has several main functions but the most important are \textit{loadInterface(xmlDoc)}, \textit{loadTest(id)}, \textit{pageXMLSave(testId)} and \textit{interfaceXMLSave()}. \textit{loadInterface(xmlDoc)} is called to decode the supplied project document in respect for the interface specified and define any global structures (such as the slider interface). It also identifies the number of pages in the test and randomises the order, if specified to do so. This is the only madatory function in any of the interface JavaScript files as this is called by \texttt{core.js} when the document is ready. The design style is such that \texttt{core.js} cannot 'see' any interface specific functions and therefore cannot assume any are available. Therefore the \textit{loadInterface(xmlDoc)} is very important to setup the entire test environment. It can therefore be assumed that the interface files can `s \ No newline at end of file +The \texttt{ape.js} file has several main functions but the most important are \textit{loadInterface(xmlDoc)}, \textit{loadTest(id)}, \textit{pageXMLSave(testId)} and \textit{interfaceXMLSave()}. \textit{loadInterface(xmlDoc)} is called to decode the supplied project document in respect for the interface specified and define any global structures (such as the slider interface). It also identifies the number of pages in the test and randomises the order, if specified to do so. This is the only madatory function in any of the interface files as this is called by \texttt{core.js} when the document is ready. The design style is such that \texttt{core.js} cannot 'see' any interface specific functions and therefore cannot assume any are available. Therefore the \textit{loadInterface(xmlDoc)} is very important to setup the entire test environment. It can be assumed that the interface files can `see' the \texttt{core.js} file and can therefore not only interact with it, but also modify it. +Each test page is loaded using \textit{loadTest(id)} which performs two major tasks: to populate the interface with the slider elements and comment boxes; and secondly to instruct the \textit{audioEngine} to load the audio fragments and construct the backend audio graph. The markers on the slider at the top of the page are positioned randomly, to minimise the bias that may be introduced when the initial positions are near the beginning, end or middle of the slider. While another approach is to place the markers outside of the slider bar at first and have the subject drag them in, the authors believe this doesn't encourage careful consideration and comparison of the different fragments as the implicit goal of the test becomes to audition and drag each fragment in just once, rather than to compare all fragments rigorously. +They consist of a \textit{bufferSourceNode} (a node which holds a buffer of audio samples for playback) and a \textit{gainNode}. There are various functions applied depending on metric collection which record the interaction with the audio element. These nodes are then connected to the \textit{audioEngine} (itself a custom web audio node) containing a \textit{gainNode} (where the various \textit{audioObject} connect to) for summation before passing the output to the \textit{destinationNode}, a permanent fixed node of the Web Audio API created as the master output where the browser then passes the audio information to the system sound device. % Does this now make sense? +% audio object/audioObject/Audio Object: -- should always be audioObject if talking about the JavaScript object, otherwise should say audio element or audio fragment. +When an \textit{audioObject} is created, it is given the URL of the audio sample to load. This is downloaded into the browser asynchronously using the \textit{XMLHttpRequest} object. This downloads any file into the JavaScript environment for further processing which is particularly useful for the Web Audio API because it supports downloading of files in their binary form for decoding. Once downloaded the file is decoded using the Web Audio API offline decoder. This uses the browser available decoding schemes to decode the audio files into raw float32 arrays, which are in turn passed to the relevant \textit{audioObject} for playback. + +Once each page of the test is completed, identified by pressing the Submit button, the \textit{pageXMLSave(testId)} is called to store all of the collected data until all pages of the test are completed. After the final test and any post-test questions are completed, the \textit{interfaceXMLSave()} function is called. This function generates the final XML file for submission as outlined in Section \ref{sec:setupresultsformats}. + +Browsers support various audio file formats and are not consistent in any format. Currently the Web Audio API is best supported in Chrome, Firefox, Opera and Safari. All of these support the use of the uncompressed WAV format. Although not a compact, web friendly format, most transport systems are of a high enough bandwidth this should not be a problem. Ogg Vorbis is another well supported format across the 4 supported major desktop browsers, as well as MP3 (although Firefox may not support all MP3 types) \cite{mozdevSupportedMedia}. %https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_media_formats +One issue of the Web Audio API is that the sample rate is assigned by the system sound device, rather than requested and does not have the ability to request a different one. % Does this make sense? The problem is across all audio files. + Therefore, the default operation when an audio file is loaded with a different sample rate to that of the system is to convert the sample rate. To provide a check for this, the desired sample rate can be supplied with the setup XML and checked against. If the sample rates do not match, a browser alert window is shown asking for the sample rate to be correctly adjusted. +\item \textbf{Repeat test}: Number of times test should be repeated (none by default), to allow familiarisation with the content and experiment, and to investigate consistency of user and variability due to familiarity. In the setup, each 'page' can be given a repeat count. These are all gathered before shuffling the order so repeated tests are not back-to-back if possible. +% explanation on how this is implemented? -- Is this better?, Nick \ No newline at end of file