Mercurial > hg > webaudioevaluationtool
changeset 985:d610dec1aa2e
Completed Abstract. Some minor corrections. Reshuffled order of intro paragraphs for clarity.
author | Nicholas Jillings <n.g.r.jillings@se14.qmul.ac.uk> |
---|---|
date | Mon, 27 Apr 2015 15:31:08 +0100 |
parents | 9f51f5c015a3 |
children | 04e80d491e88 |
files | docs/SMC15/smc2015template.bbl docs/SMC15/smc2015template.tex |
diffstat | 2 files changed, 134 insertions(+), 134 deletions(-) [+] |
line wrap: on
line diff
--- a/docs/SMC15/smc2015template.bbl Fri Apr 24 18:53:24 2015 +0100 +++ b/docs/SMC15/smc2015template.bbl Mon Apr 27 15:31:08 2015 +0100 @@ -1,111 +1,111 @@ -% Generated by IEEEtran.bst, version: 1.12 (2007/01/11) -\begin{thebibliography}{10} -\providecommand{\url}[1]{#1} -\csname url@samestyle\endcsname -\providecommand{\newblock}{\relax} -\providecommand{\bibinfo}[2]{#2} -\providecommand{\BIBentrySTDinterwordspacing}{\spaceskip=0pt\relax} -\providecommand{\BIBentryALTinterwordstretchfactor}{4} -\providecommand{\BIBentryALTinterwordspacing}{\spaceskip=\fontdimen2\font plus -\BIBentryALTinterwordstretchfactor\fontdimen3\font minus - \fontdimen4\font\relax} -\providecommand{\BIBforeignlanguage}[2]{{% -\expandafter\ifx\csname l@#1\endcsname\relax -\typeout{** WARNING: IEEEtran.bst: No hyphenation pattern has been}% -\typeout{** loaded for the language `#1'. Using the pattern for}% -\typeout{** the default language instead.}% -\else -\language=\csname l@#1\endcsname -\fi -#2}} -\providecommand{\BIBdecl}{\relax} -\BIBdecl - -\bibitem{schoeffler2013impact} -M.~Schoeffler and J.~Herre, ``About the impact of audio quality on overall - listening experience,'' in \emph{Proceedings of Sound and Music Computing - Conference}, 2013, pp. 48--53. - -\bibitem{repp} -R.~Repp, ``Recording quality ratings by music professionals,'' in - \emph{Proceedings of the 2006 International Computer Music Conference}, 2006, - pp. 468--474. - -\bibitem{de2013real} -A.~de~G{\"o}tzen, E.~Sikstr{\"o}m, F.~Grani, and S.~Serafin, ``Real, foley or - synthetic? {A}n evaluation of everyday walking sounds,'' in \emph{Proceedings - of SMC 2013 : 10th Sound and Music Computing Conference}, 2013. - -\bibitem{durr2015implementation} -G.~Durr, L.~Peixoto, M.~Souza, R.~Tanoue, and J.~D. Reiss, ``Implementation and - evaluation of dynamic level of audio detail,'' in \emph{Audio Engineering - Society Conference: 56th International Conference: Audio for Games}, February - 2015. - -\bibitem{deman2014a} -B.~De~Man and J.~D. Reiss, ``Adaptive control of amplitude distortion - effects,'' in \emph{53rd Conference of the Audio Engineering Society}, - January 2014. - -\bibitem{uhlereiss} -J.~D. Reiss and C.~Uhle, ``Determined source separation for microphone - recordings using {IIR} filters,'' in \emph{129th Convention of the Audio - Engineering Society}.\hskip 1em plus 0.5em minus 0.4em\relax Audio - Engineering Society, November 2010. - -\bibitem{song2013b} -Y.~Song, S.~Dixon, and A.~R. Halpern, ``Do online social tags predict perceived - or induced emotional responses to music?'' in \emph{Proceedings of the 14th - International Society for Music Information Retrieval (ISMIR2013)}, 2013. - -\bibitem{song2013a} -Y.~Song, S.~Dixon, M.~T. Pearce, and G.~Fazekas, ``Using tags to select stimuli - in the study of music and emotion,'' \emph{Proceedings of the 3rd - International Conference on Music \& Emotion (ICME)}, 2013. - -\bibitem{friberg2011comparison} -A.~Friberg and A.~Hedblad, ``A comparison of perceptual ratings and computed - audio features,'' in \emph{Proceedings of the SMC2011, 8th Sound and Music - Computing Conference}, 2011, pp. 122--127. - -\bibitem{mason2015compression} -A.~Mason, N.~Jillings, Z.~Ma, J.~D. Reiss, and F.~Melchior, ``Adaptive audio - reproduction using personalized compression,'' in \emph{Audio Engineering - Society Conference: 57th International Conference: The Future of Audio - Entertainment Technology -- Cinema, Television and the Internet}, March 2015. - -\bibitem{deman2014b} -B.~De~Man and J.~D. Reiss, ``{APE}: {A}udio {P}erceptual {E}valuation toolbox - for {MATLAB},'' in \emph{136th Convention of the Audio Engineering Society}, - April 2014. - -\bibitem{mushram} -E.~Vincent, M.~G. Jafari, and M.~D. Plumbley, ``Preliminary guidelines for - subjective evalutation of audio source separation algorithms,'' in \emph{UK - ICA Research Network Workshop}, 2006. - -\bibitem{whisper} -S.~Ciba, A.~Wlodarski, and H.-J. Maempel, ``Whis{PER} -- a new tool for - performing listening tests,'' in \emph{126th Convention of the Audio - Engineering Society}, May 7-10 2009. - -\bibitem{scale} -A.~V. Giner, ``Scale - a software tool for listening experiments,'' in - \emph{AIA/DAGA Conference on Acoustics, Merano (Italy)}, 2013. - -\bibitem{beaqlejs} -S.~Kraft and U.~Z{\"o}lzer, ``{BeaqleJS}: {HTML5} and {JavaScript} based - framework for the subjective evaluation of audio quality,'' in \emph{Linux - Audio Conference, Karlsruhe, DE}, 2014. - -\bibitem{mushra} -\emph{Method for the subjective assessment of intermediate quality level of - coding systems}.\hskip 1em plus 0.5em minus 0.4em\relax Recommendation {ITU-R - BS.1534-1}, 2003. - -\bibitem{bech} -S.~Bech and N.~Zacharov, \emph{Perceptual Audio Evaluation - Theory, Method and - Application}.\hskip 1em plus 0.5em minus 0.4em\relax John Wiley \& Sons, - 2007. - -\end{thebibliography} +% Generated by IEEEtran.bst, version: 1.12 (2007/01/11) +\begin{thebibliography}{10} +\providecommand{\url}[1]{#1} +\csname url@samestyle\endcsname +\providecommand{\newblock}{\relax} +\providecommand{\bibinfo}[2]{#2} +\providecommand{\BIBentrySTDinterwordspacing}{\spaceskip=0pt\relax} +\providecommand{\BIBentryALTinterwordstretchfactor}{4} +\providecommand{\BIBentryALTinterwordspacing}{\spaceskip=\fontdimen2\font plus +\BIBentryALTinterwordstretchfactor\fontdimen3\font minus + \fontdimen4\font\relax} +\providecommand{\BIBforeignlanguage}[2]{{% +\expandafter\ifx\csname l@#1\endcsname\relax +\typeout{** WARNING: IEEEtran.bst: No hyphenation pattern has been}% +\typeout{** loaded for the language `#1'. Using the pattern for}% +\typeout{** the default language instead.}% +\else +\language=\csname l@#1\endcsname +\fi +#2}} +\providecommand{\BIBdecl}{\relax} +\BIBdecl + +\bibitem{schoeffler2013impact} +M.~Schoeffler and J.~Herre, ``About the impact of audio quality on overall + listening experience,'' in \emph{Proceedings of Sound and Music Computing + Conference}, 2013, pp. 48--53. + +\bibitem{repp} +R.~Repp, ``Recording quality ratings by music professionals,'' in + \emph{Proceedings of the 2006 International Computer Music Conference}, 2006, + pp. 468--474. + +\bibitem{de2013real} +A.~de~G{\"o}tzen, E.~Sikstr{\"o}m, F.~Grani, and S.~Serafin, ``Real, foley or + synthetic? {A}n evaluation of everyday walking sounds,'' in \emph{Proceedings + of SMC 2013 : 10th Sound and Music Computing Conference}, 2013. + +\bibitem{durr2015implementation} +G.~Durr, L.~Peixoto, M.~Souza, R.~Tanoue, and J.~D. Reiss, ``Implementation and + evaluation of dynamic level of audio detail,'' in \emph{Audio Engineering + Society Conference: 56th International Conference: Audio for Games}, February + 2015. + +\bibitem{deman2014a} +B.~De~Man and J.~D. Reiss, ``Adaptive control of amplitude distortion + effects,'' in \emph{53rd Conference of the Audio Engineering Society}, + January 2014. + +\bibitem{uhlereiss} +J.~D. Reiss and C.~Uhle, ``Determined source separation for microphone + recordings using {IIR} filters,'' in \emph{129th Convention of the Audio + Engineering Society}.\hskip 1em plus 0.5em minus 0.4em\relax Audio + Engineering Society, November 2010. + +\bibitem{song2013b} +Y.~Song, S.~Dixon, and A.~R. Halpern, ``Do online social tags predict perceived + or induced emotional responses to music?'' in \emph{Proceedings of the 14th + International Society for Music Information Retrieval (ISMIR2013)}, 2013. + +\bibitem{song2013a} +Y.~Song, S.~Dixon, M.~T. Pearce, and G.~Fazekas, ``Using tags to select stimuli + in the study of music and emotion,'' \emph{Proceedings of the 3rd + International Conference on Music \& Emotion (ICME)}, 2013. + +\bibitem{friberg2011comparison} +A.~Friberg and A.~Hedblad, ``A comparison of perceptual ratings and computed + audio features,'' in \emph{Proceedings of the SMC2011, 8th Sound and Music + Computing Conference}, 2011, pp. 122--127. + +\bibitem{mason2015compression} +A.~Mason, N.~Jillings, Z.~Ma, J.~D. Reiss, and F.~Melchior, ``Adaptive audio + reproduction using personalized compression,'' in \emph{Audio Engineering + Society Conference: 57th International Conference: The Future of Audio + Entertainment Technology -- Cinema, Television and the Internet}, March 2015. + +\bibitem{deman2014b} +B.~De~Man and J.~D. Reiss, ``{APE}: {A}udio {P}erceptual {E}valuation toolbox + for {MATLAB},'' in \emph{136th Convention of the Audio Engineering Society}, + April 2014. + +\bibitem{mushram} +E.~Vincent, M.~G. Jafari, and M.~D. Plumbley, ``Preliminary guidelines for + subjective evalutation of audio source separation algorithms,'' in \emph{UK + ICA Research Network Workshop}, 2006. + +\bibitem{whisper} +S.~Ciba, A.~Wlodarski, and H.-J. Maempel, ``Whis{PER} -- a new tool for + performing listening tests,'' in \emph{126th Convention of the Audio + Engineering Society}, May 7-10 2009. + +\bibitem{scale} +A.~V. Giner, ``Scale - a software tool for listening experiments,'' in + \emph{AIA/DAGA Conference on Acoustics, Merano (Italy)}, 2013. + +\bibitem{beaqlejs} +S.~Kraft and U.~Z{\"o}lzer, ``{BeaqleJS}: {HTML5} and {JavaScript} based + framework for the subjective evaluation of audio quality,'' in \emph{Linux + Audio Conference, Karlsruhe, DE}, 2014. + +\bibitem{mushra} +\emph{Method for the subjective assessment of intermediate quality level of + coding systems}.\hskip 1em plus 0.5em minus 0.4em\relax Recommendation {ITU-R + BS.1534-1}, 2003. + +\bibitem{bech} +S.~Bech and N.~Zacharov, \emph{Perceptual Audio Evaluation - Theory, Method and + Application}.\hskip 1em plus 0.5em minus 0.4em\relax John Wiley \& Sons, + 2007. + +\end{thebibliography}
--- a/docs/SMC15/smc2015template.tex Fri Apr 24 18:53:24 2015 +0100 +++ b/docs/SMC15/smc2015template.tex Mon Apr 27 15:31:08 2015 +0100 @@ -134,10 +134,10 @@ \capstarttrue % \begin{abstract} -New functionality in HTML5, notably its Web Audio API, allows for increasingly powerful applications in the browser. % is this true? -Perceptual evaluation tests for audio, where the subject assesses certain qualities of different audio fragments through a graphical user interface and/or text boxes, require playback of audio and rapid switching between different files. % what else? -The advantage of a web application is easy deployment on any platform, without requiring any other application or library, easy storing of results on a server. -[...] % Need to expand here! +Perceptual evaluation tests for audio, where the subject assesses certain qualities of different audio fragments are an integral part of research evaluation. These evaluation tests require playback of audio and rapid switching between different files. Participants give vast amount of data from comments and ranking through a graphical user interface and/or text boxes. +New functionality in HTML5 with the Web Audio API, allows for increasingly powerful media applications in a browser independent environment. In this paper we propose a tool which enables conducting perceptual audio evaluation tests over the web with the ability for multiple test interfaces, custom configuration, metric and interface data collection and other features. +The advantage of a web application is easy deployment on any platform, without requiring any other application enabling multiple tests to be easily conducted across locations. Results are gathered automatically using web friendly formats for easy storing of results on a server. +% Currently at 150, don't think anything more needs to be done here?? %Place your abstract at the top left column on the first page. %Please write about 150-200 words that specifically highlight the purpose of your work, %its context, and provide a brief synopsis of your results. @@ -153,11 +153,7 @@ Perceptual evaluation of audio plays an important role in a wide range of research including audio quality \cite{schoeffler2013impact,repp}, sound synthesis \cite{de2013real,durr2015implementation}, audio effect design \cite{deman2014a}, source separation \cite{uhlereiss}, music and emotion analysis \cite{song2013b,song2013a}, and many others \cite{friberg2011comparison}. % codec design? -The Web Audio API is a high-level JavaScript Application Programming Interface (API) designed for real-time processing of audio inside the browser through various processing nodes\footnote{http://webaudio.github.io/web-audio-api/}. Various web sites have used the Web Audio API for either creative purposes, such as drum machines and score creation tools\footnote{http://webaudio.github.io/demo-list/}, -others from the list show real-time captured audio processing such as room reverberation tools and a phase vocoder from the system microphone. The BBC Radiophonic Workshop shows effects used on famous TV shows such as Doctor Who, being simulated inside the browser\footnote{http://webaudio.prototyping.bbc.co.uk/}. -Another example is the BBC R\&D personalised compressor which applies a dynamic range compressor on a radio station that dynamically adjusts the compressor settings to match the listener's environment \cite{mason2015compression}. - -This work is based in part on the APE audio perceptual evaluation interface for MATLAB \cite{deman2014b}. An important drawback of this toolbox is the need to have MATLAB to create a test and even to run (barring the use of an executable generated by MATLAB), and limited compatibility with both earlier and newer versions of MATLAB, which makes it hard to maintain. On the other hand, a web application generally has the advantage of running in most browsers on most applications, and we present a tool that requires no specialised software or even programming knowledge to set up. +This work is based in part on the APE audio perceptual evaluation interface for MATLAB \cite{deman2014b}. An important drawback of this toolbox is the need to have MATLAB to create a test and even to run (barring the use of an executable generated by MATLAB), and limited compatibility with both earlier and newer versions of MATLAB, which makes it hard to maintain. On the other hand, a web application generally has the advantage of running in most browsers on most applications. % IMPORTANT %[TO ADD: other interfaces for perceptual evaluation of audio, browser-based or not!] \\ @@ -169,19 +165,23 @@ In this paper, we provide a listening test back end that allows for easy set up of a wide variety of listening tests, highly flexible yet very simple and not requiring any programming skills. % [How is this one different from all these?] improve +The Web Audio API is a high-level JavaScript Application Programming Interface (API) designed for real-time processing of audio inside the browser through various processing nodes\footnote{http://webaudio.github.io/web-audio-api/}. Various web sites have used the Web Audio API for either creative purposes, such as drum machines and score creation tools\footnote{http://webaudio.github.io/demo-list/}, +others from the list show real-time captured audio processing such as room reverberation tools and a phase vocoder from the system microphone. The BBC Radiophonic Workshop shows effects used on famous TV shows such as Doctor Who, being simulated inside the browser\footnote{http://webaudio.prototyping.bbc.co.uk/}. +Another example is the BBC R\&D personalised compressor which applies a dynamic range compressor on a radio station that dynamically adjusts the compressor settings to match the listener's environment \cite{mason2015compression}. + We present a browser-based perceptual evaluation tool from which any kind of multiple stimulus audio evaluation tool where subjects need to rank, rate, select, or comment on different audio samples can be built. We also include an example of the multiple stimulus user interface included with the APE tool \cite{deman2014b}, which presents the subject with a number of axes on which a number of markers, corresponding to audio samples, can be moved to reflect any subjective quality, as well as corresponding comment boxes. However, other graphical user interfaces can be put on top of the engine that we provide with minimal or no modifications. Examples of this are the MUSHRA test \cite{mushra}, single or multiple stimulus evaluation with a two-dimensional interface (such as valence and arousal dimensions), or simple annotation (using free-form text, check boxes, radio buttons or drop-down menus) of one or more audio samples at a time. In some cases, such as method of adjustment, where the audio is processed by the user \cite{bech}, or AB test \cite{bech}, where the interface does not show all audio samples to be evaluated at once, the back end of the tool needs to be modified as well. -There are a number of advantages to building a web audio based listening test environment. The ability to easily deploy a flexible and scalable testing environment that requires no proprietary software to run makes the web audio evaluation tool a very flexible testing tool. The ability to host a single test server and create multiple clients not only allows multiple participants to be involved in a trial simultaneously, but also permits participants to be located anywhere in the world. There are also less user experience issues, since all users should have some experience with using existing web technologies. +There are a number of advantages to building a web audio based listening test environment. The ability to easily deploy a flexible and scalable testing environment that requires no proprietary software to run makes the web audio evaluation tool a very flexible testing tool. The ability to host a single test server not only allows multiple participants to be involved in a trial simultaneously, but also permits participants to be located anywhere in the world. There are also less user experience issues, since all users should have some experience with using existing web technologies. In the following sections, we describe the included interface in more detail, discuss the implementation, and cover considerations that were made in the design process of this tool. \section{Interface}\label{sec:interface} At this point, we have implemented the interface of the MATLAB-based APE (Audio Perceptual Evaluation) toolbox \cite{deman2014b}. This shows one marker for each simultaneously evaluated audio fragment on one or more horizontal axes, that can be moved to rate or rank the respective fragments in terms of any subjective quality, as well as a comment box for every marker, and any extra text boxes for extra comments. -The reason for such an interface, where all stimuli are presented on a single rating axis (or multiple axes if multiple subjective qualities need to be evaluated), is that it urges the subject to consider the rating and/or ranking of the stimuli relative to one another, as opposed to comparing each individual stimulus to a given reference, as is the case with e.g. a MUSHRA test \cite{mushra}. See \ref{fig:interface} for an example of the interface, with eleven fragments and one axis. %? change if a new interface is shown +The reason for such an interface, where all stimuli are presented on a single rating axis (or multiple axes if multiple subjective qualities need to be evaluated), is that it urges the subject to consider the rating and/or ranking of the stimuli relative to one another, as opposed to comparing each individual stimulus to a given reference, as is the case with e.g. a MUSHRA test \cite{mushra}. See Figure \ref{fig:interface} for an example of the interface, with eleven fragments and one axis. %? change if a new interface is shown However, the back end of this test environment allows for many more established and novel interfaces for listening tests, particularly ones where the subject only assesses audio without manipulating it (i.e. method of adjustment), which would require additional features to be implemented. @@ -196,23 +196,23 @@ \section{Architecture}\label{sec:architecture} % or implementation? -The tool runs entirely inside the browser through the new HTML5 Web Audio API and is supported by most major web browsers. The API allows for constructing audio processing elements and connecting them together to produce a high quality, real time signal process to manipulate audio streams. The API supports multichannel processing and has an accurate playback timer for precise scheduled playback control. The API is controlled through the browser JavaScript engine and is therefore highly configurable. Processing is all performed in a separate, low latency thread to the main JavaScript thread, so there is no blocking due to real time processing. +The tool uses entirely client side processing utilizing the new HTML5 Web Audio API, supported by most major web browsers. The API allows for constructing audio processing elements and connecting them together to produce a high quality, real time signal process to manipulate audio streams. The API supports multichannel processing and has an accurate playback timer for precise scheduled playback control. The API is controlled through the browser JavaScript engine and is therefore highly configurable. Processing is all performed in a separate, low latency thread to the main JavaScript thread, so there is no blocking due to real time processing. The web tool itself is split into several files to operate: \begin{itemize} \item \texttt{index.html}: The main index file to load the scripts, this is the file the browser must request to load. -\item \texttt{core.js}: Contains functions and objects to define the audio objects, audio playback engine and loading media files +\item \texttt{core.js}: Contains global functions and object prototypes to define the audio playback engine, audio objects and loading media files \item \texttt{ape.js}: Parses set up files to create the interface as instructed, following the same style chain as the MATLAB APE Tool \cite{deman2014b}. \end{itemize} -The HTML file loads the \texttt{core.js} file along with a few other ancillary files (such as the jQuery JavaScript extensions\footnote{http://jquery.com/}), at which point the browser JavaScript begins to execute the on-page instructions, which gives the URL of the test set up XML document (outlined in Section \ref{sec:setupresultsformats}). \texttt{core.js} parses this document and executes the function in \texttt{ape.js} to build the web page with the given audio files. The reason for separating these two files is to allow for further interface designs (such as MUSHRA \cite{mushra} or AB tests \cite{bech}) to be used, which would still require the same underlying core functions outlined in \texttt{core.js}, see also Section \ref{sec:interface}. +The HTML file loads the \texttt{core.js} file along with a few other ancillary files (such as the jQuery JavaScript extensions\footnote{http://jquery.com/}), at which point the browser JavaScript begins to execute the on-page instructions, which gives the URL of the test set up XML document (outlined in Section \ref{sec:setupresultsformats}). \texttt{core.js} parses this document and executes the functions in \texttt{ape.js} to build the web page. The reason for separating these two files is to allow for further interface designs (such as MUSHRA \cite{mushra} or AB tests \cite{bech}) to be used, which would still require the same underlying core functions outlined in \texttt{core.js}. -The \texttt{ape.js} file has several main functions but the most important are \textit{loadInterface(xmlDoc)}, \textit{loadTest(id)}, \textit{pageXMLSave(testId)} and \textit{interfaceXMLSave()}. \textit{loadInterface(xmlDoc)} is called to decode the supplied project document in respect for the interface specified and define any global structures (such as the slider interface). It also identifies the number of pages in the test and randomises the order, if specified to do so. This is the only madatory function in any of the interface files as this is called by \texttt{core.js} when the document is ready. The design style is such that \texttt{core.js} cannot 'see' any interface specific functions and therefore cannot assume any are available. Therefore the \textit{loadInterface(xmlDoc)} is very important to set up the entire test environment. It can be assumed that the interface files can `see' the \texttt{core.js} file and can therefore not only interact with it, but also modify it. +The \texttt{ape.js} file has several main functions but the most important are \textit{loadInterface(xmlDoc)}, \textit{loadTest(id)}, \textit{pageXMLSave(testId)} and \textit{interfaceXMLSave()}. \textit{loadInterface(xmlDoc)} is called to decode the supplied project document in respect for the interface specified and define any global structures (such as the slider interface). It also identifies the number of pages in the test and randomises the order, if specified to do so. This is the only madatory function in any of the interface files as this is called by \texttt{core.js} when the document is ready. \texttt{core.js} cannot 'see' any interface specific functions and therefore cannot assume any are available. Therefore the \textit{loadInterface(xmlDoc)} is very important to set up the entire test environment. Because the interface files are loaded by \texttt{core.js} and because the functions in \texttt{core.js} are global, the interface files can `see' the \texttt{core.js} file and can therefore not only interact with it, but also modify it. Each test page is loaded using \textit{loadTest(id)} which performs two major tasks: to populate the interface with the slider elements and comment boxes; and secondly to instruct the \textit{audioEngine} to load the audio fragments and construct the backend audio graph. The markers on the slider at the top of the page are positioned randomly, to minimise the bias that may be introduced when the initial positions are near the beginning, end or middle of the slider. Another approach is to place the markers outside of the slider bar at first and have the subject drag them in, but the authors believe this doesn't encourage careful consideration and comparison of the different fragments as the implicit goal of the test becomes to audition and drag each fragment in just once, rather than to compare all fragments rigorously. \textit{loadTest(id)} in \texttt{ape.js} also instructs the audio engine in \texttt{core.js} to create the \textit{audioObject} These are custom audio nodes, one representing each audio element specified in each page. -They consist of a \textit{bufferSourceNode} (a node which holds a buffer of audio samples for playback) and a \textit{gainNode}. There are various functions applied depending on metric collection which record the interaction with the audio element. These nodes are then connected to the \textit{audioEngine} (itself a custom web audio node) containing a \textit{gainNode} (where the various \textit{audioObject} connect to) for summation before passing the output to the \textit{destinationNode}, a permanent fixed node of the Web Audio API created as the master output where the browser then passes the audio information to the system sound device. % Does this now make sense? +They consist of a \textit{bufferSourceNode} (a node which holds a buffer of audio samples for playback) and a \textit{gainNode}, both of which are Web Audio API Nodes. There are various functions applied, depending on which metrics are enabled, to record the interaction with the audio element. These nodes are then connected to the \textit{audioEngine} (itself a custom web audio node) containing a \textit{gainNode} (where the various \textit{audioObject} connect to) for summation before passing the output to the \textit{destinationNode}, a permanent node of the Web Audio API created as the master output where the browser then passes the audio information to the system sound device. % Does this now make sense? % audio object/audioObject/Audio Object: -- should always be audioObject if talking about the JavaScript object, otherwise should say audio element or audio fragment. When an \textit{audioObject} is created, it is given the URL of the audio sample to load. This is downloaded into the browser asynchronously using the \textit{XMLHttpRequest} object. This downloads any file into the JavaScript environment for further processing which is particularly useful for the Web Audio API because it supports downloading of files in their binary form for decoding. Once downloaded the file is decoded using the Web Audio API offline decoder. This uses the browser available decoding schemes to decode the audio files into raw float32 arrays, which are in turn passed to the relevant \textit{audioObject} for playback. @@ -222,14 +222,14 @@ Browsers support various audio file formats and are not consistent in any format. Currently the Web Audio API is best supported in Chrome, Firefox, Opera and Safari. All of these support the use of the uncompressed WAV format. Although not a compact, web friendly format, most transport systems are of a high enough bandwidth this should not be a problem. Ogg Vorbis is another well supported format across the 4 supported major desktop browsers, as well as MP3 (although Firefox may not support all MP3 types) \footnote{https://developer.mozilla.org/en-US/docs/Web/HTML/\\Supported\_media\_formats}. %https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_media_formats One issue of the Web Audio API is that the sample rate is assigned by the system sound device, rather than requested and does not have the ability to request a different one. % Does this make sense? The problem is across all audio files. Therefore, the default operation when an audio file is loaded with a different sample rate to that of the system is to convert the sample rate. To provide a check for this, the desired sample rate can be supplied with the set up XML and checked against. If the sample rates do not match, a browser alert window is shown asking for the sample rate to be correctly adjusted. -As this happens before any loading or decoding of audio files, the system will only fetch files as soon as the system's sample rate meets any requirements, avoiding requests for large files until they are actually needed. +This happens before any loading or decoding of audio files so the browser will only be instructed to fetch files if the system sample rate meets the requirements, avoiding multiple requests for large files until they are actually needed. %During playback, the playback nodes loop indefinitely until playback is stopped. The gain nodes in the \textit{audioObject}s enable dynamic muting of nodes. When a bar in the sliding ranking is clicked, the audio engine mutes all \textit{audioObject}s and un-mutes the clicked one. Therefore, if the audio samples are perfectly aligned up and of the same sample length, they will remain perfectly aligned with each other. % Don't think this is relevant anymore \section{Input and result files}\label{sec:setupresultsformats} -The set up and result files both use the common XML document format to outline the various parameters. The set up file determines which interface to use, the location of audio files, how many pages and other general set up rules to define the testing environment. Having one document to modify allows for quick manipulation in a `human readable' form to create new tests, or adjust current ones, without needing to edit multiple web files. An example of this XML document is presented in Figure~\ref{fig:xmlIn}% I mean the .js and .html files, though not sure if any better. +The set up and result files both use the common XML document format to outline the various parameters. The set up file determines which interface to use, the location of audio files, how many pages and other parameters to define the testing environment. Having one document to modify allows for quick manipulation in a `human readable' form to create new tests, or adjust current ones, without needing to edit multiple web files. An example of this XML document is presented in Figure~\ref{fig:xmlIn}% I mean the .js and .html files, though not sure if any better. \subsection{Set up and configurability} @@ -241,7 +241,7 @@ \end{center} \end{figure} -The set up document has several defined nodes and structure which are documented with the source code. For example there is a section for general set up options where the pre-test and post-test questions and statements are set up and from the example in Figure~\ref{fig:xmlIn}, it can be seen that a question box should be generated, with the id 'location' and it is mandatory to answer. The question is in the PreTest node meaning it will appear before any testing will begin. When the result for the entire test is shown, then this will appear in the PreTest node of the response with the id 'location' allowing it to be found easily. This outlines the importance of having clear and meaningful ID values. Pre- and post-test dialog boxes allow for comments or questions to be presented before or after the test, to convey listening test instructions, gather information about the subject, listening environment, and overall experience of the test. +The set up document has several defined nodes and structure which are documented with the source code. For example there is a section for general set up options where any pre-test and post-test questions and statements can be defined. Pre- and post-test dialogue boxes allow for comments or questions to be presented before or after the test, to convey listening test instructions, gather information about the subject, listening environment, and overall experience of the test. From the example in Figure~\ref{fig:xmlIn}, it can be seen that a question box should be generated, with the id 'location' and it is mandatory to answer. The question is in the PreTest node meaning it will appear before any testing will begin. When the result for the entire test is shown, the response will appear in the PreTest node with the id 'location' allowing it to be found easily. This outlines the importance of having clear and meaningful ID values. We try to cater to a diverse audience with this toolbox, while ensuring the toolbox is simple, elegant and straightforward. To that end, we include the following options that can be easily switched on and off, by setting the value in the input XML file. @@ -283,9 +283,9 @@ \end{center} \end{figure} -The results also contain information collected by any defined pre/post questions. These are referenced against the set up XML by using the same ID so readable responses can be obtained. Taking from the earlier example of setting up a pre-test question, an example response can be seen in Figure \label{fig:xmlOut}. +The results also contain information collected by any defined pre/post questions. These are referenced against the set up XML by using the same ID so readable responses can be obtained. Taking from the earlier example of setting up a pre-test question, an example response can be seen in Figure \ref{fig:xmlOut}. -Each page of testing is returned with the results of the entire page included in the structure. One `audioElement' node is created per audio fragment per page, along with its ID. This includes several child nodes including the value holding the rating between 0 and 1, the comment, and any other collected metrics. The latter include how long the element was listened for, the initial position, boolean flags if the element was listened to, if the element was moved and if the element comment box had any comment. Furthermore, each user action (manipulation of any interface element, such as playback or moving a marker) is logged along with a the corresponding time code and stored or sent along with the results. +Each page of testing is returned with the results of the entire page included in the structure. One `audioElement' node is created per audio fragment per page, along with its ID. This includes several child nodes including the rating between 0 and 1, the comment, and any other collected metrics including how long the element was listened for, the initial position, boolean flags if the element was listened to, if the element was moved and if the element comment box had any comment. Furthermore, each user action (manipulation of any interface element, such as playback or moving a marker) can be logged along with a the corresponding time code. Furthermore, we also store session data such as the browser the tool was used in. We provide the option to store the results locally, and/or to have them sent to a server. @@ -318,8 +318,8 @@ %</metric> \\ %</audioelement>} -As can be seen from Figure~\ref{fig:xmlOut}, the parent tag \texttt{audioelement} holds the id of the element passed in from the set up document. The first child element is \texttt{comment} and holds both the question shown and the response from the comment box inside. -The child element \texttt{value} holds the normalised ranking value. Next comes the metric node structure, there is one \texttt{metricresult} node per metric event collected. The id of the node identifies the type of data it contains. For example, the first holds the id \textit{elementTimer} and the data contained represents how long, in seconds, the audio element was listened to. The next holds the id \textit{elementTrackerFull} and contains a pair of elements per entry. This represents the entire movement of the elements' slider giving the time the event took place in seconds from when the current test page started, and the new position. In our example there are three \texttt{timepos} children with their id representing their order. There is one of these \texttt{audioelement} tags per audio element outlined on each test page. +The parent tag \texttt{audioelement} holds the ID of the element passed in from the set up document. The first child element is \texttt{comment} and holds both the question shown and the response from the comment box inside. +The child element \texttt{value} holds the normalised ranking value. Next comes the metric node structure, there is one \texttt{metricresult} node per metric event collected. The id of the node identifies the type of data it contains. For example, the first holds the id \textit{elementTimer} and the data contained represents how long, in seconds, the audio element was listened to. There is one \texttt{audioelement} tag per audio element on each test page. \section{Conclusions and future work}\label{sec:conclusions}