Mercurial > hg > webaudioevaluationtool
comparison docs/Instructions/Instructions.tex @ 1447:cd05eb7921a6
Analysis page clean up. Instructions minor update. Minor fixes.
author | Brecht De Man <b.deman@qmul.ac.uk> |
---|---|
date | Mon, 21 Dec 2015 12:10:17 +0100 |
parents | 7831cfdc4cca |
children | 56ee920d5d85 |
comparison
equal
deleted
inserted
replaced
1446:7831cfdc4cca | 1447:cd05eb7921a6 |
---|---|
238 | 238 |
239 \clearpage | 239 \clearpage |
240 | 240 |
241 \section{Features} | 241 \section{Features} |
242 | 242 |
243 This section goes over the different features implemented in the Web Audio Evaluation Tool, how to use them, and what to know about them. | 243 This section covers the different features implemented in the Web Audio Evaluation Tool, how to use them, and what to know about them. |
244 | 244 |
245 Unless otherwise specified, \emph{each} feature described here is optional, i.e. it can be enabled or disabled and adjusted to some extent. | 245 Unless otherwise specified, \emph{each} feature described here is optional, i.e. it can be enabled or disabled and adjusted to some extent. |
246 | 246 |
247 As the example project showcases (nearly) all of these features, please refer to its configuration XML document for a demonstration of how to enable and adjust them. | 247 As the example project showcases (nearly) all of these features, please refer to its configuration XML document for a demonstration of how to enable and adjust them. |
248 | |
249 \subsection{Interface layout} | |
250 The \texttt{interface} node (child of \texttt{audioholder}) contains | |
251 | |
252 Example: | |
253 | |
254 \begin{lstlisting} | |
255 <interface name="quality"> | |
256 <title>Audio Quality</title> | |
257 <scale position="10">Poor</scale> | |
258 <scale position="90">Excellent</scale> | |
259 <commentBoxPrefix>Comment on fragment</commentBoxPrefix> | |
260 </interface> | |
261 \end{lstlisting} | |
262 | |
263 \subsubsection{Title} | |
264 Specifies the axis title as displayed on the interface. | |
265 | |
266 If this tag is absent, the title will default to `Axis \emph{[number]}'. Therefore, if no title is desired, just add the title tag (\texttt{<title/>}) without text. | |
267 | |
268 \subsubsection{Annotation} | |
269 Words or numbers can be placed on specific positions of the scale with the \texttt{scale} tag. The \texttt{position} attribute is a value from 0 to 100, corresponding to the percentage of the width/height of the scale where you want the string to be placed. | |
270 | |
271 \subsubsection{Comment box prefix} | |
272 If comment boxes corresponding with the fragments are enabled, this sets the comment box string after which the fragment number is appended. | |
273 | |
274 The default value is ``Comment on fragment''. So in this case, each comment box would have a header ``Comment on fragment \emph[number]''. | |
275 | |
276 \subsubsection{Multiple scales} | |
277 In the case of multiple rating scales, e.g. when the stimuli are to be rated in terms of attributes `timbre' and `spatial impression', multiple interface nodes will have to be added, each specifying the title and annotations. | |
278 | |
279 This is where the \texttt{interface}'s \texttt{name} attribute is particularly important: use this to retrieve the rating values, comments and metrics associated with the specified interface. | |
280 If none is given, you can still use the automatically given \texttt{interface-id}, which is the interface number starting with 0 and corresponding to the order in which the rating scales appear. | |
281 | |
248 | 282 |
249 \subsection{Surveys} | 283 \subsection{Surveys} |
250 Surveys are conducted through an in-page popup window which can collect data using various HTML functions, see Survey elements below for a list. Survey questions are placed into the \texttt{<pretest>} or \texttt{<posttest>} nodes. Appending these nodes to the \texttt{<setup>} node will have the survey options appear before any test pages (if in the \texttt{<pretest>} node) or after all test pages. Placing the survey options in the \texttt{<audioholder>} node will have them appear before or after the test page they are a child of. | 284 Surveys are conducted through an in-page popup window which can collect data using various HTML functions, see Survey elements below for a list. Survey questions are placed into the \texttt{<pretest>} or \texttt{<posttest>} nodes. Appending these nodes to the \texttt{<setup>} node will have the survey options appear before any test pages (if in the \texttt{<pretest>} node) or after all test pages. Placing the survey options in the \texttt{<audioholder>} node will have them appear before or after the test page they are a child of. |
251 \subsubsection{Survey elements} | 285 \subsubsection{Survey elements} |
252 All survey elements (which `pop up' in the centre of the browser) have an \texttt{id} attribute, for retrieval of the responses in post-processing of the results, and a \texttt{mandatory} attribute, which if set to ``true'' requires the subjects to respond before they can continue. | 286 All survey elements (which `pop up' in the centre of the browser) have an \texttt{id} attribute, for retrieval of the responses in post-processing of the results, and a \texttt{mandatory} attribute, which if set to ``true'' requires the subjects to respond before they can continue. |
291 <option name="bad">Bad</option> | 325 <option name="bad">Bad</option> |
292 <option name="poor">Poor</option> | 326 <option name="poor">Poor</option> |
293 <option name="good">Good</option> | 327 <option name="good">Good</option> |
294 <option name="great">Great</option> | 328 <option name="great">Great</option> |
295 </radio> | 329 </radio> |
296 <checkbox id="checkboxtest" mandatory="true"> | 330 <checkbox id="background" mandatory="true"> |
297 <statement>Please select with which activities you have any experience (example checkbox question)</statement> | 331 <statement>Please select with which activities you have any experience (example checkbox question)</statement> |
298 <option name="musician">Playing a musical instrument</option> | 332 <option name="musician">Playing a musical instrument</option> |
299 <option name="soundengineer">Recording or mixing audio</option> | 333 <option name="soundengineer">Recording or mixing audio</option> |
300 </checkbox> | 334 </checkbox> |
301 <statement>Thank you for taking this listening test. Please click 'Submit' and your results will appear in the 'saves/' folder.</statement> | 335 <statement>Thank you for taking this listening test. Please click 'Submit' and your results will appear in the 'saves/' folder.</statement> |
302 </PostTest> | 336 </PostTest> |
303 \end{lstlisting} | 337 \end{lstlisting} |
304 | 338 |
305 | 339 |
340 | |
306 \subsection{Randomisation} | 341 \subsection{Randomisation} |
342 [WORK IN PROGRESS] | |
307 | 343 |
308 \subsubsection{Randomisation of configuration XML files} | 344 \subsubsection{Randomisation of configuration XML files} |
309 The python server has a special function to automatically cycle through a list of test pages. Instead of directly requesting an XML, simply setting the url item in the browser URL to \texttt{pseudo.xml} will cycle through a list of XMLs. These XMLs must be in the local directory called \texttt{pseudo}. | 345 The python server has a special function to automatically cycle through a list of test pages. Instead of directly requesting an XML, simply setting the url item in the browser URL to \texttt{pseudo.xml} will cycle through a list of XMLs. These XMLs must be in the local directory called \texttt{pseudo}. |
310 % how to | 346 % how to |
311 % explain how this is implemented in the pythonServer | 347 % explain how this is implemented in the pythonServer |
435 Add \texttt{<check name="scalerange" min="25" max="75"/>} to the \texttt{interface} node. | 471 Add \texttt{<check name="scalerange" min="25" max="75"/>} to the \texttt{interface} node. |
436 | 472 |
437 \subsubsection{Note on the use of multiple rating axes} | 473 \subsubsection{Note on the use of multiple rating axes} |
438 I.e. what if more than one axis? How to specify which axis the checks relate to? %Nick? to add? | 474 I.e. what if more than one axis? How to specify which axis the checks relate to? %Nick? to add? |
439 | 475 |
440 \subsection{Layout options} | |
441 \texttt{title}, \texttt{scale}, \texttt{position}, \texttt{commentBoxPrefix} | |
442 | |
443 \subsection{Multiple sliders} | |
444 (APE example) | |
445 | |
446 \begin{lstlisting} | |
447 <interface name="preference"> | |
448 <title>Preference</title> | |
449 <scale position="0">Min</scale> | |
450 <scale position="100">Max</scale> | |
451 <scale position="50">Middle</scale> | |
452 <commentBoxPrefix>Comment on fragment</commentBoxPrefix> | |
453 </interface> | |
454 <interface name="depth"> | |
455 <title>Depth</title> | |
456 <scale position="0">Low</scale> | |
457 <scale position="100">High</scale> | |
458 <scale position="50">Middle</scale> | |
459 <commentBoxPrefix>Comment on fragment</commentBoxPrefix> | |
460 </interface> | |
461 \end{lstlisting} | |
462 where the \texttt{interface} nodes are children of the \texttt{audioholder} node. | |
463 | |
464 \subsection{Platform information} | 476 \subsection{Platform information} |
465 % what does it do, what does it look like | 477 % what does it do, what does it look like |
466 % limitations? | 478 % limitations? |
479 For troubleshooting and usage statistics purposes, information about the browser and the operating system is logged in the results XML file. This is especially useful in the case of remote tests, when it is not certain which operating system, browser and/or browser were used. Note that this information is not always available and/or accurate, e.g. when the subject has taken steps to be more anonymous, so it should be treated as a guide only. | |
480 | |
481 Example: | |
482 \begin{lstlisting} | |
483 <navigator> | |
484 <platform>MacIntel</platform> | |
485 <vendor>Google Inc.</vendor> | |
486 <uagent>Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.134 Safari/537.36</uagent> | |
487 </navigator> | |
488 \end{lstlisting} | |
467 | 489 |
468 \subsection{Show progress} | 490 \subsection{Show progress} |
469 Add \texttt{<option name="page-count"/>} to the \texttt{interface} node (see Section \ref{sec:checks}: Checks) to add the current page number and the total number of pages to the interface. | 491 Add \texttt{<option name="page-count"/>} to the \texttt{interface} node (see Section \ref{sec:checks}: Checks) to add the current page number and the total number of pages to the interface. |
470 | 492 |
471 \subsection{Gain} | 493 \subsection{Gain} |
660 As a guide to better understand the Instructions, and to expand them later, here is a list of terms that may be unclear or ambiguous unless properly defined. | 682 As a guide to better understand the Instructions, and to expand them later, here is a list of terms that may be unclear or ambiguous unless properly defined. |
661 \begin{description} | 683 \begin{description} |
662 \item[Subject] The word we use for a participant, user, ... of the test, i.e. not the experimenter who designs the test but the person who evaluates the audio under test as part of an experiment (or the preparation of one). | 684 \item[Subject] The word we use for a participant, user, ... of the test, i.e. not the experimenter who designs the test but the person who evaluates the audio under test as part of an experiment (or the preparation of one). |
663 \item[User] The person who uses the tool to configure, run and analyse the test - i.e. the experimenter, most likely a researcher - or at least | 685 \item[User] The person who uses the tool to configure, run and analyse the test - i.e. the experimenter, most likely a researcher - or at least |
664 \item[Page] A screen in a test; corresponds with an \texttt{audioholder} | 686 \item[Page] A screen in a test; corresponds with an \texttt{audioholder} |
665 \item[Fragment] An element or sample in a test; corresponds with an \texttt{audioelement} | 687 \item[Fragment] An element, stimulus or sample in a test; corresponds with an \texttt{audioelement} |
666 \item[Test] A complete test which can consist of several pages; corresponds with an entire configuration XML file | 688 \item[Test] A complete test which can consist of several pages; corresponds with an entire configuration XML file |
667 \item[Configuration XML file] The XML file containing the necessary information on interface, samples, survey questions, configurations, ... which the JavaScript modules read to produce the desired test. | 689 \item[Configuration XML file] The XML file containing the necessary information on interface, samples, survey questions, configurations, ... which the JavaScript modules read to produce the desired test. |
668 \item[Results XML file] The output of a successful test, including ratings, comments, survey responses, timing information, and the complete configuration XML file with which the test was generated in the first place. | 690 \item[Results XML file] The output of a successful test, including ratings, comments, survey responses, timing information, and the complete configuration XML file with which the test was generated in the first place. |
669 \end{description} | 691 \end{description} |
670 | 692 |