|
emmanouil@7
|
1 Metrics Files for D-CASE Challenge
|
|
emmanouil@7
|
2 http://c4dm.eecs.qmul.ac.uk/sceneseventschallenge/
|
|
emmanouil@7
|
3
|
|
emmanouil@7
|
4 Copyright (c) 2013 Dimitrios Giannoulis, Emmanouil Benetos, Dan Stowell, Mark Plumbley and Queen Mary University of London
|
|
emmanouil@7
|
5
|
|
emmanouil@7
|
6 - for licence information see the file named COPYING.
|
|
emmanouil@7
|
7
|
|
emmanouil@7
|
8 For details on metric definitions, please see the following Technical Report: http://c4dm.eecs.qmul.ac.uk/sceneseventschallenge/EECSRR-13-01.pdf
|
|
emmanouil@7
|
9
|
|
emmanouil@7
|
10 ====
|
|
emmanouil@7
|
11
|
|
emmanouil@7
|
12 1. Scene Classification Metrics
|
|
emmanouil@7
|
13
|
|
emmanouil@7
|
14 [confusionMat, AccFolds, Acc, Std] = sceneClassificationMetrics_eval(numfolds, foldsGTlist, foldstestlist)
|
|
emmanouil@7
|
15
|
|
emmanouil@7
|
16 The function sceneClassificationMetrics_eval takes as input a number of folds (numfolds, e.g. 5), a file containing a list of ground-truth files which correspond to each fold (foldsGTlist - an example is given in foldGTlist.txt), and a file containing the list of folds in the scene classification system (foldstestlist). An example on the format of the foldstestlist file is given in foldtestlist.txt. Likewise, the internal structure of a fold file is givein in fold1_test_ubhjwk.txt.
|
|
emmanouil@7
|
17
|
|
emmanouil@7
|
18 For instructions on formatting the output files for the scene classification system, please see: http://c4dm.eecs.qmul.ac.uk/sceneseventschallenge/aasp_scenerec_spec.pdf.
|
|
emmanouil@7
|
19
|
|
emmanouil@7
|
20 The output of the metrics function contains the confusion matrix (confusionMat), the accuracy per fold (AccFolds), the overall accuracy (Acc), and the standard deviation (Std).
|
|
emmanouil@7
|
21
|
|
emmanouil@7
|
22 ===
|
|
emmanouil@7
|
23
|
|
emmanouil@7
|
24 2. Event Detection Metrics
|
|
emmanouil@7
|
25
|
|
emmanouil@7
|
26 In each case, 'outputFile' is a text file containing a list of detected events from the event detection system (see http://c4dm.eecs.qmul.ac.uk/sceneseventschallenge/aasp_eventdet_OL_spec.pdf for formatting guidelines). 'GTFile' is the corresponding list of ground-truth events, formatted in the same way.
|
|
emmanouil@7
|
27
|
|
emmanouil@7
|
28 a. Frame-based metrics: [results] = eventDetectionMetrics_frameBased(outputFile,GTFile)
|
|
emmanouil@7
|
29
|
|
emmanouil@7
|
30 b. Event-based metrics: [results] = eventDetectionMetrics_eventBased(outputFile,GTFile)
|
|
emmanouil@7
|
31
|
|
emmanouil@7
|
32 c. Class-wise event-based metrics: [results] = eventDetectionMetrics_classWiseEventBased(outputFile,GTFile)
|
|
emmanouil@7
|
33
|
|
emmanouil@7
|
34 In all cases, 'results' is a structure containing the corresponding metrics. For the frame-wise case, results contain Precision (Pre), Recall (Rec), F-Measure (F), and the AEER metric (AEER).
|
|
emmanouil@7
|
35
|
|
emmanouil@7
|
36
|
