annotate latex/scripts/paper_results.txt @ 4:3e666d0329b5 tip

updated code

____________________________

____ PRE-EVALUATION     ____

____________________________


compare different annotators


Significance of differences of pitch estimates obtained from different 
note segmentation annotators (t test p<0.01).

* Mean difference between sd, kf in semitones: -0.002
  Is significant: FALSE
* Mean difference between sd, mm in semitones: -0.017
  Is significant: FALSE
* Mean difference between mm, kf in semitones: 0.014
  Is significant: FALSE

____________________________

____ MAIN PAPER RESULTS ____

____________________________

This script assumes that the script data_import.R has been run.
This script generates statistics and figures used in the JASA submission.


____ METHOD SECTION ____

- data for self-assessment table:
      age               musical.background   sing.ability     sing.experience
 Min.   :13.00   none            : 1       poor    : 1    notanswered : 1    
 1st Qu.:25.75   amateur         :14       low     : 3    none        : 3    
 Median :32.50   semiprofessional: 7       medium  :14    some        : 6    
 Mean   :34.46   professional    : 2       high    : 4    alot        :13    
 3rd Qu.:40.75                             veryhigh: 2    professional: 1    
 Max.   :62.00                                                               
              choir.experience
 none                 : 5     
 childneverafter      : 3     
 sangbutnolongeractive: 5     
 active               :11     
                              
                              

____ ACCURACY SECTION ____

- Interval error significance tests:

  - Upward minor sixth is flat.
    Is the difference significant at significance level 0.0001? TRUE (p = 5.58618e-11, df=186, t = -6.96201)

  - Upward octave is flat.
    Is the difference significant at significance level 0.0001? TRUE (p = 1.65043e-16, df=183, t = -9.08791)

  - Prime (0 interval) is sharp by 0.29.
    Is the difference significant at significance level 0.0001? TRUE (p = 2.83605e-60, df=753, t = 17.9552)

- explaining pitch error by following interval or beginning of phrase

  - p: 3.04e-42, 2.56e-68
  - F = 254.937
  - DF: 4667
  - Variance explained: 9.8%
  - Sharpening in cents per note in interval: 1.3

  - Variance explained only by intercept and following interval: 6.2%
  - p: 1.15e-68
  - F = 316.791
  - DF: 4773

  - Variance explained only by intercept and beginning of phrase: 3.8%
  - p: 8.91e-41
  - F = 182.276
  - DF: 4668

- Error difference between ET and JI error:
  Is it significant? Paired t-test says:TRUE (p = 8.20175e-45, df: 4774, t = -14.1927)

 
ET (MAPE: 0.190394) fits better than JI (MAPE: 0.203976), by -0.0135824


____ RESULTS SECTION ____

Basic metrics statistics.
  - MAPE. Mean: 0.189;  median: 0.187;  sd: 0.051
  - MAIE. Mean: 0.263;  median: 0.267;  sd: 0.069
  - D. Mean: 0.074;  median: 0.069;  sd: 0.169
  - |D|. Mean: 0.138;  median: 0.111;  sd: 0.122

  - lin D. in cents: Mean: 0.097;  median: 0.096;  sd: 0.371
- Significance of drift:  - significant drifts: 25 of 72 (35%)
  - number of upward drifts: 50, of which 20 significantly so.
  - number of downward drifts: 22, of which 5 significantly so.

- Significance of linear drift  - significant drifts: 16 of 72 (22%) at conf. level 0.01, 27 of 72 (38%) at conf. level 0.05 
  - number of upward drifts: 40, of which 13 significantly so at 0.01.
  - number of downward drifts: 32, of which 3 significantly so at 0.01.

  - number of upward drifts: 40, of which 20 significantly so at 0.05.
  - number of downward drifts: 32, of which 7 significantly so at 0.05.

- Correlation table (LaTeX syntax)
  sg.abl& \textbf{ 0.40}& \textbf{ 0.31}& \textbf{ 0.54}& \textbf{-0.45}& \textbf{-0.46}&  0.11& -0.02&  0.06\\
  &sg.exp& -0.07& \textbf{ 0.42}& -0.16& -0.27&  0.20&  0.05&  0.11\\
  & &mus.bg& \textbf{ 0.34}& -0.16& -0.24&  0.10& -0.02&  0.05\\
  & & &ch.exp& \textbf{-0.37}& \textbf{-0.40}&  0.22&  0.01&  0.07\\
  & & & &MAIE& \textbf{ 0.93}& -0.19& -0.01& -0.06\\
  & & & & &MAPE& -0.19& -0.01& -0.04\\
  & & & & & &lin.D& \textbf{ 0.52}& \textbf{ 0.94}\\
  & & & & & & &abs.D& \textbf{ 0.54}\\
  & & & & & & & &D\\


- Variance explained by singer for different measures

  - MAPE: 85.38%, p-value: 4.4044e-13, df = 23, F = 12.1856
  - MAIE: 80.98%, p-value: 1.4267e-10, df = 23, F = 8.88731
  - lin.D: 56.87%, p-value: 0.0015462, df = 23, F = 2.75204
  - D: 47.53%, p-value: 0.031599, df = 23, F = 1.89015
  - |D|: 22.28%, p-value: 0.90865, df = 23, F = 0.598159

- Testing influence of singing condition (Normal/Masked/Imagined) via Kruskal-Wallis tests (sign. level 0.01):

  - MAPE significant difference due to condition? FALSE (p-value: 0.64045, df = 2, chisquared = 0.891172)
  - MAIE significant difference due to condition? FALSE (p-value: 0.29725, df = 2, chisquared = 2.42637)
  - lin. D significant difference due to condition? FALSE (p-value: 0.36367, df = 2, chisquared = 2.02302)
  - D significant difference due to condition? FALSE (p-value: 0.28387, df = 2, chisquared = 2.51846)
  - |D| significant difference due to condition? FALSE (p-value: 0.80985, df = 2, chisquared = 0.421804)

____ MODEL SECTION ____

- existence of 'spread'
  - notenumber: 31.29%, p-value: 5.8408e-07, df = 30.5138, F = 67

- inadequacy of mu = 0 model:

  - interval (error) variance: 0.147
  - expected variance of D (interval variance summed over 50 notes): 7.358
  - ... and the corresponding standard deviation: 2.713

  - In comparison the standard deviation of pitch erros is: 0.280

- Better parameter for mu, firstly via 'model MAPE':

  - the smallest error of 0.22 is achieved with model 'err0850'
  - error at mu = 0: 0.27
  - error at mu = 1: 0.29

- Estimate according to drift variance formula:

  - overall pitch error variance: 0.079
  - observed variance over 50-note gap: 0.029
  - estimated mu based on these: 0.915

- Stats for mu by singer:
  - range: 0.617 to 0.983
  - mean: 0.832, median: 0.850, sd: 0.105