Mercurial > hg > chime-home-dataset-annotation-and-baseline-evaluation-code
view README @ 5:b523456082ca tip
Update path to dataset and reflect modified chunk naming convention.
author | peterf |
---|---|
date | Mon, 01 Feb 2016 21:35:27 +0000 |
parents | cb535b80218a |
children |
line wrap: on
line source
Summary ------------------- This software relates to results reported in Foster et al. (2015). Scripts are included for analysing annotator agreement and for obtaining baseline GMM results, using the CHiME-Home dataset. Finally, the scripts are provided for obtaining the annotations. NB: The CHiME-Home dataset is not included. The dataset (recordings & annotations) is available at TODO. The scripts were developed and tested on Linux environments, using Python 2.7.6. Portability to other environments is not expected. Overview of directory structure and scripts ------------------- annotation_scripts/ Contains scripts used to obtain annotations annotationkit_create_annotation_protocol.py Prepares annotation protocol CSV from list of audio files annotationkit_create_annotation_protocol_wrapper.sh Wrapper shell script for annotationkit_create_annotation_protocol.py annotationkit_play_chunks.py Play back list of audio chunks, based on specified annotation protocol CSV and read annotation strings from terminal warmup_phase/ Contains preliminary scripts used to create protocol files for annotator warm-up phases (cf. Foster et al., 2015) analysis_of_annotations/evaluate_annotations.py Code relevant to analysis of CHiME-Home annotations (not audio) gmm_baseline_experiments/ Contains scripts for baseline GMM evaluation. run_experiments.py Main script for baseline GMM evaluation How-to ------------------- Reproduce analysis of annotations reported in Foster et al. (2015) :- 1) Modify evaluate_annotations.py, so that variable DatasetPath points to directory containing CHiME-Home dataset 2) Execute evaluate_annotations.py The script writes textual analysis data to standard output; a figure pdf is written to analysis_of_annotations/figures subdirectory. Reproduce baseline results reported in Foster et al. (2015) :- 1) Modify run_experiments.py, so that Settings['paths']['chime_home']['basepath'] points to Directory containing CHiME-Home dataset; Settings['paths']['resultsdir'] and Settings['paths']['featuresdir'] should respectively point to desired directories for storing pickle files of data structures containing results and features 2) Execute run_experiments.py The script writes a figure pdf to gmm_baseline_experiments/figures subdirectory. Adapt annotation scripts for alternative purposes :- 1) Modify annotationkit_create_annotation_protocol.py. The purpose of this script is to read a list of audio files from standard input and write CSV to standard output; as presently implemented, each input file is a audio segment from which a set of 4-second chunks are sampled (as defined in script header, and sampling without replacement). The list of chunks is written to an output CSV, and defines a `protocol' file for use with the annotation script annotationkit_play_chunks.py. Variables chunkDuration and sampleRate should match desired chunk duration and given audio sample rate. 2) Modify annotationkit_play_chunks_wrapper.sh so that parameters chunkDuration, sampleRate are consistent with annotationkit_create_annotation_protocol.py. Further, modify parameters PermittedAnnotationCharacters, LoneAnnotationCharacters, LabelMap to reflect chosen label alphabet. NB: As implemented, using the protocol CSV, annotationkit_play_chunks.py seeks within each audio segment to locate each audio chunk, thus no explicit segmentation of input audio is required. How to cite ------------------- P. Foster, S. Sigtia, S. Krstulovic, J. Barker, M. D. Plumbley. "CHiME-Home: A Dataset for Sound Source Recognition in a Domestic Environment," in Proceedings of the 11th Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2015.