comparison docs/Instructions/Instructions.tex @ 1447:cd05eb7921a6

Analysis page clean up. Instructions minor update. Minor fixes.
author Brecht De Man <b.deman@qmul.ac.uk>
date Mon, 21 Dec 2015 12:10:17 +0100
parents 7831cfdc4cca
children 56ee920d5d85
comparison
equal deleted inserted replaced
1446:7831cfdc4cca 1447:cd05eb7921a6
238 238
239 \clearpage 239 \clearpage
240 240
241 \section{Features} 241 \section{Features}
242 242
243 This section goes over the different features implemented in the Web Audio Evaluation Tool, how to use them, and what to know about them. 243 This section covers the different features implemented in the Web Audio Evaluation Tool, how to use them, and what to know about them.
244 244
245 Unless otherwise specified, \emph{each} feature described here is optional, i.e. it can be enabled or disabled and adjusted to some extent. 245 Unless otherwise specified, \emph{each} feature described here is optional, i.e. it can be enabled or disabled and adjusted to some extent.
246 246
247 As the example project showcases (nearly) all of these features, please refer to its configuration XML document for a demonstration of how to enable and adjust them. 247 As the example project showcases (nearly) all of these features, please refer to its configuration XML document for a demonstration of how to enable and adjust them.
248
249 \subsection{Interface layout}
250 The \texttt{interface} node (child of \texttt{audioholder}) contains
251
252 Example:
253
254 \begin{lstlisting}
255 <interface name="quality">
256 <title>Audio Quality</title>
257 <scale position="10">Poor</scale>
258 <scale position="90">Excellent</scale>
259 <commentBoxPrefix>Comment on fragment</commentBoxPrefix>
260 </interface>
261 \end{lstlisting}
262
263 \subsubsection{Title}
264 Specifies the axis title as displayed on the interface.
265
266 If this tag is absent, the title will default to `Axis \emph{[number]}'. Therefore, if no title is desired, just add the title tag (\texttt{<title/>}) without text.
267
268 \subsubsection{Annotation}
269 Words or numbers can be placed on specific positions of the scale with the \texttt{scale} tag. The \texttt{position} attribute is a value from 0 to 100, corresponding to the percentage of the width/height of the scale where you want the string to be placed.
270
271 \subsubsection{Comment box prefix}
272 If comment boxes corresponding with the fragments are enabled, this sets the comment box string after which the fragment number is appended.
273
274 The default value is ``Comment on fragment''. So in this case, each comment box would have a header ``Comment on fragment \emph[number]''.
275
276 \subsubsection{Multiple scales}
277 In the case of multiple rating scales, e.g. when the stimuli are to be rated in terms of attributes `timbre' and `spatial impression', multiple interface nodes will have to be added, each specifying the title and annotations.
278
279 This is where the \texttt{interface}'s \texttt{name} attribute is particularly important: use this to retrieve the rating values, comments and metrics associated with the specified interface.
280 If none is given, you can still use the automatically given \texttt{interface-id}, which is the interface number starting with 0 and corresponding to the order in which the rating scales appear.
281
248 282
249 \subsection{Surveys} 283 \subsection{Surveys}
250 Surveys are conducted through an in-page popup window which can collect data using various HTML functions, see Survey elements below for a list. Survey questions are placed into the \texttt{<pretest>} or \texttt{<posttest>} nodes. Appending these nodes to the \texttt{<setup>} node will have the survey options appear before any test pages (if in the \texttt{<pretest>} node) or after all test pages. Placing the survey options in the \texttt{<audioholder>} node will have them appear before or after the test page they are a child of. 284 Surveys are conducted through an in-page popup window which can collect data using various HTML functions, see Survey elements below for a list. Survey questions are placed into the \texttt{<pretest>} or \texttt{<posttest>} nodes. Appending these nodes to the \texttt{<setup>} node will have the survey options appear before any test pages (if in the \texttt{<pretest>} node) or after all test pages. Placing the survey options in the \texttt{<audioholder>} node will have them appear before or after the test page they are a child of.
251 \subsubsection{Survey elements} 285 \subsubsection{Survey elements}
252 All survey elements (which `pop up' in the centre of the browser) have an \texttt{id} attribute, for retrieval of the responses in post-processing of the results, and a \texttt{mandatory} attribute, which if set to ``true'' requires the subjects to respond before they can continue. 286 All survey elements (which `pop up' in the centre of the browser) have an \texttt{id} attribute, for retrieval of the responses in post-processing of the results, and a \texttt{mandatory} attribute, which if set to ``true'' requires the subjects to respond before they can continue.
291 <option name="bad">Bad</option> 325 <option name="bad">Bad</option>
292 <option name="poor">Poor</option> 326 <option name="poor">Poor</option>
293 <option name="good">Good</option> 327 <option name="good">Good</option>
294 <option name="great">Great</option> 328 <option name="great">Great</option>
295 </radio> 329 </radio>
296 <checkbox id="checkboxtest" mandatory="true"> 330 <checkbox id="background" mandatory="true">
297 <statement>Please select with which activities you have any experience (example checkbox question)</statement> 331 <statement>Please select with which activities you have any experience (example checkbox question)</statement>
298 <option name="musician">Playing a musical instrument</option> 332 <option name="musician">Playing a musical instrument</option>
299 <option name="soundengineer">Recording or mixing audio</option> 333 <option name="soundengineer">Recording or mixing audio</option>
300 </checkbox> 334 </checkbox>
301 <statement>Thank you for taking this listening test. Please click 'Submit' and your results will appear in the 'saves/' folder.</statement> 335 <statement>Thank you for taking this listening test. Please click 'Submit' and your results will appear in the 'saves/' folder.</statement>
302 </PostTest> 336 </PostTest>
303 \end{lstlisting} 337 \end{lstlisting}
304 338
305 339
340
306 \subsection{Randomisation} 341 \subsection{Randomisation}
342 [WORK IN PROGRESS]
307 343
308 \subsubsection{Randomisation of configuration XML files} 344 \subsubsection{Randomisation of configuration XML files}
309 The python server has a special function to automatically cycle through a list of test pages. Instead of directly requesting an XML, simply setting the url item in the browser URL to \texttt{pseudo.xml} will cycle through a list of XMLs. These XMLs must be in the local directory called \texttt{pseudo}. 345 The python server has a special function to automatically cycle through a list of test pages. Instead of directly requesting an XML, simply setting the url item in the browser URL to \texttt{pseudo.xml} will cycle through a list of XMLs. These XMLs must be in the local directory called \texttt{pseudo}.
310 % how to 346 % how to
311 % explain how this is implemented in the pythonServer 347 % explain how this is implemented in the pythonServer
435 Add \texttt{<check name="scalerange" min="25" max="75"/>} to the \texttt{interface} node. 471 Add \texttt{<check name="scalerange" min="25" max="75"/>} to the \texttt{interface} node.
436 472
437 \subsubsection{Note on the use of multiple rating axes} 473 \subsubsection{Note on the use of multiple rating axes}
438 I.e. what if more than one axis? How to specify which axis the checks relate to? %Nick? to add? 474 I.e. what if more than one axis? How to specify which axis the checks relate to? %Nick? to add?
439 475
440 \subsection{Layout options}
441 \texttt{title}, \texttt{scale}, \texttt{position}, \texttt{commentBoxPrefix}
442
443 \subsection{Multiple sliders}
444 (APE example)
445
446 \begin{lstlisting}
447 <interface name="preference">
448 <title>Preference</title>
449 <scale position="0">Min</scale>
450 <scale position="100">Max</scale>
451 <scale position="50">Middle</scale>
452 <commentBoxPrefix>Comment on fragment</commentBoxPrefix>
453 </interface>
454 <interface name="depth">
455 <title>Depth</title>
456 <scale position="0">Low</scale>
457 <scale position="100">High</scale>
458 <scale position="50">Middle</scale>
459 <commentBoxPrefix>Comment on fragment</commentBoxPrefix>
460 </interface>
461 \end{lstlisting}
462 where the \texttt{interface} nodes are children of the \texttt{audioholder} node.
463
464 \subsection{Platform information} 476 \subsection{Platform information}
465 % what does it do, what does it look like 477 % what does it do, what does it look like
466 % limitations? 478 % limitations?
479 For troubleshooting and usage statistics purposes, information about the browser and the operating system is logged in the results XML file. This is especially useful in the case of remote tests, when it is not certain which operating system, browser and/or browser were used. Note that this information is not always available and/or accurate, e.g. when the subject has taken steps to be more anonymous, so it should be treated as a guide only.
480
481 Example:
482 \begin{lstlisting}
483 <navigator>
484 <platform>MacIntel</platform>
485 <vendor>Google Inc.</vendor>
486 <uagent>Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.134 Safari/537.36</uagent>
487 </navigator>
488 \end{lstlisting}
467 489
468 \subsection{Show progress} 490 \subsection{Show progress}
469 Add \texttt{<option name="page-count"/>} to the \texttt{interface} node (see Section \ref{sec:checks}: Checks) to add the current page number and the total number of pages to the interface. 491 Add \texttt{<option name="page-count"/>} to the \texttt{interface} node (see Section \ref{sec:checks}: Checks) to add the current page number and the total number of pages to the interface.
470 492
471 \subsection{Gain} 493 \subsection{Gain}
660 As a guide to better understand the Instructions, and to expand them later, here is a list of terms that may be unclear or ambiguous unless properly defined. 682 As a guide to better understand the Instructions, and to expand them later, here is a list of terms that may be unclear or ambiguous unless properly defined.
661 \begin{description} 683 \begin{description}
662 \item[Subject] The word we use for a participant, user, ... of the test, i.e. not the experimenter who designs the test but the person who evaluates the audio under test as part of an experiment (or the preparation of one). 684 \item[Subject] The word we use for a participant, user, ... of the test, i.e. not the experimenter who designs the test but the person who evaluates the audio under test as part of an experiment (or the preparation of one).
663 \item[User] The person who uses the tool to configure, run and analyse the test - i.e. the experimenter, most likely a researcher - or at least 685 \item[User] The person who uses the tool to configure, run and analyse the test - i.e. the experimenter, most likely a researcher - or at least
664 \item[Page] A screen in a test; corresponds with an \texttt{audioholder} 686 \item[Page] A screen in a test; corresponds with an \texttt{audioholder}
665 \item[Fragment] An element or sample in a test; corresponds with an \texttt{audioelement} 687 \item[Fragment] An element, stimulus or sample in a test; corresponds with an \texttt{audioelement}
666 \item[Test] A complete test which can consist of several pages; corresponds with an entire configuration XML file 688 \item[Test] A complete test which can consist of several pages; corresponds with an entire configuration XML file
667 \item[Configuration XML file] The XML file containing the necessary information on interface, samples, survey questions, configurations, ... which the JavaScript modules read to produce the desired test. 689 \item[Configuration XML file] The XML file containing the necessary information on interface, samples, survey questions, configurations, ... which the JavaScript modules read to produce the desired test.
668 \item[Results XML file] The output of a successful test, including ratings, comments, survey responses, timing information, and the complete configuration XML file with which the test was generated in the first place. 690 \item[Results XML file] The output of a successful test, including ratings, comments, survey responses, timing information, and the complete configuration XML file with which the test was generated in the first place.
669 \end{description} 691 \end{description}
670 692