Mercurial > hg > soundsoftware-icassp-2012

--- a/cannam.tex	Thu Sep 22 14:35:17 2011 +0100
+++ b/cannam.tex	Thu Sep 22 14:49:09 2011 +0100
@@ -53,9 +53,9 @@

 This presents two areas of difficulty.  First, researchers in the
 audio and music research community --- including within the group
-represented by the present authors, the Centre for Digital Music at
-Queen Mary University of London --- come from a wide range of
-backgrounds including signal processing, electronics, computer
+represented by the present authors, the Centre for Digital Music
+(C4DM) at Queen Mary University of London --- come from a wide range
+of backgrounds including signal processing, electronics, computer
 science, music, information sciences, dance and performance, and data
 sonification.  In many of these fields, researchers do not have the
 skills or desire to become involved in traditional software
@@ -79,8 +79,10 @@

 Some researchers have come to realize that traditional methods of
 disseminating research outputs are no longer sufficient for
-computational science research.  Dohono and colleagues at Stanford
-have, since the mid 1990’s, aimed to carry out ``Reproducible
+computational science research, because the algorithms and parameters
+involved are complex enough that the description in the paper is no
+longer sufficient to reproduce the results.  Dohono and colleagues at
+Stanford have, since the mid 1990’s, aimed to carry out ``Reproducible
 Research'' by providing the paper, source code, and data, sufficient
 for other researchers to reproduce the same results
 \cite{buckheit1995}.  Recent years have seen some moves to promote
@@ -95,11 +97,11 @@
 as these promote the idea that research results in signal processing
 should be presented not simply as a printed paper, but as a compendium
 including the paper, research data, and code.  Vandewalle et al have
-also created a Reproducible Research Repository (http://rr.epfl.ch/),
+also created a Reproducible Research Repository\footnote{\tt http://rr.epfl.ch/},
 designed to promote reproducible research by requiring the authors of
 a paper to upload the code and data used in the experiments. Readers
-can also give comments about a publication and evaluate the
-reproducibility of the work.
+can also comment on a publication and evaluate the reproducibility of
+the work.

 \subsection{Real-world limitations on software practice}
 \label{subsec:researchsoft}
@@ -163,16 +165,25 @@
 Our approach is to facilitate incremental improvements to the way
 software is managed during research, by identifying practical barriers
 to software reuse and providing means to reduce or eliminate them.
+
 While we support the goal of reproducible research and aim to
 encourage open publication of code and data linked with paper
 publications, we believe that this goal is more easily approached
-indirectly because researchers can appreciate improvements to software
-development practice regardless of their beliefs or intentions with
-regard to reproducible research.  By helping researchers to feel
-comfortable with managing provenance and versioning for software, with
-collaborative development of code, and with the perception of code as
-something that may readily be reused, we aim to prepare ground in
-which open and reproducible publication can naturally grow.
+incrementally because researchers will appreciate improvements to
+software development practice regardless of their beliefs or
+intentions with regard to reproducible research.  By helping
+researchers to feel comfortable with managing provenance and
+versioning for software, with collaborative development of code, and
+with the perception of code as something that may readily be reused,
+we aim to prepare ground in which open and reproducible publication
+can naturally grow.
+
+Further, the software lifecycle does not end with publication.
+Software that is to be used needs maintenance, and any proposal to
+help researchers reuse software more easily needs to address the
+problem that such software is not always in a reusable state.  The
+ultimate concern of our present work, therefore, is sustainability and
+reusability rather than reproducibility.

 \subsection{Education and Confidence With Code}

@@ -187,8 +198,8 @@
 materials \cite{softwarecarpentry}.  This week-long residential course
 for 20 audio and music researchers from groups around the UK taught
 fundamentals of software development and good practice including
-version control, unit testing, Python syntax and structure, and
-managing small result databases with sqlite.
+version control, unit testing and test-driven development, Python
+syntax and structure, and managing small result databases with sqlite.

 \subsubsection{Videos and Tutorials}

@@ -215,41 +226,54 @@
 \subsubsection{SoundSoftware Code Site}
 \label{sec:codesite}

-We provide the SoundSoftware code site
-(http://code.soundsoftware.ac.uk/) as a facility which audio
-and music researchers in the UK may use for collaborative development
-and as a version control and code hosting facility, if their
-institution is unable to help them or if they have a need to work with
-researchers at other institutions who would not be permitted access to
-their institution's facilities.  Of course the existence of this site
-also addresses the shortcomings in our own group's former ad-hoc use
-of version control.  The site is implemented using a custom version of
-the Redmine project management application, with Mercurial version
-control.  Any UK researcher in the field is permitted to register and
-to start their own collaborative projects using the version control,
-wiki, issue tracking, and other services provided.
+We provide the SoundSoftware code site\footnote{\tt http://code.soundsoftware.ac.uk/} as a facility which audio and music
+researchers in the UK may use for collaborative development and as a
+version control and code hosting facility, if their institution is
+unable to help them or if they have a need to work with researchers at
+other institutions who would not be permitted access to their
+institution's facilities.  Of course the existence of this site also
+addresses the shortcomings in our own group's former ad-hoc use of
+version control.  The site is implemented using a custom version of
+the Redmine [citation needed] project management application, with
+Mercurial version control.  Any UK researcher in the field is
+permitted to register and to start their own collaborative projects
+using the version control, wiki, issue tracking, and other services
+provided.

-Four aspects of our code site make it potentially more useful to audio
-and music researchers than a generic site such as SourceForge or
-Github:
+Four aspects of our code site contribute to sustainability and utility
+for researchers:

 \begin{enumerate}
-\item {\em Focus} --- We believe that the focus of our site on audio and music research increases the likelihood of software relevant to researchers being found and reused;
-\item {\em Public and private projects} --- Researchers can use this site for code management even for projects they do not intend to publish, or can start a project privately and open it later;
-\item {\em Linking publications with code} --- Users can associate publication records with code projects to make it easy for future researchers to find code when searching for a publication;
-\item {\em Tracking external projects} --- Researchers who use code hosting or project management facilities elsewhere can also make use of our site as a nexus for relevant projects, as the site does not require that code is hosted with it and can also track external repositories.
+\item {\em Focus} --- The focus of the site on audio and music
+  research makes it easier to locate and obtain code for reuse;
+\item {\em Linking publications with code} --- Users can associate
+  publication records with their projects so that readers can
+  immediately see what publications are relevant;
+\item {\em Public and private projects} --- Researchers can use the
+  site for code management even for projects they do not intend to
+  publish, or can start a project privately and open it later;
+\item {\em Tracking external projects} --- Researchers who use code
+  hosting or project management facilities elsewhere can also make use
+  of our site as a nexus for relevant projects, as the site does not
+  require that code is hosted with it and can also track external
+  repositories.
 \end{enumerate}

-Although we believe these features make the site useful, our goal is
-to encourage researchers to make best use of whatever facilities are
-available to them.  We offer it simply as one alternative.
+We hope these features together will encourage researchers to employ
+collaborative development early in their work, and to place themselves
+in a situation in which the outcomes of their work can be used in a
+sustainable way with relatively little extra effort.  That said, our
+primary goal is to encourage researchers to make best use of whatever
+facilities are available to them; this site is only one offering.

 \subsubsection{EasyMercurial}
 \label{sec:easyhg}

 Our attempt to address the difficulties faced in learning version
 control user interfaces is EasyMercurial, an application we developed
-based on existing work [citation needed].
+based on existing work [citation needed] in order to provide a user
+interface that we could teach easily to researchers across multiple
+operating system platforms.

 \subsection{Platforms and Reuse}

@@ -279,6 +303,21 @@

 \subsubsection{Beat Trackers}

+In section \ref{subsec:researchsoft} we gave example of beat tracking
+methods whose implementations had effectively been lost.  However, we
+have made some successful attempts at sustainability for other
+methods. An early work from C4DM in audio onset detection
+\cite{bello2003} was used in later work on beat tracking
+\cite{davies2005}. This algorithm, originally in MATLAB, was ported to
+SuperCollider by Collins at Cambridge and in our group into a
+cross-platform C++ Vamp plugin. It also inspired a real time Max/MSP
+beat tracking system \cite{robertson2007} and was used in
+beat-synchronous audio effects, developed in Matlab and ported to
+real-time VST plugins \cite{stark2008}.  Nearly all of these
+implementations are now available, linked to their references, through
+the SoundSoftware code repository described in section \ref{sec:codesite}.
+
+
 \subsubsection{Chordino and NNLS Chroma}
 \label{subsubsec:chordino}