annotate survey_appendix.tex @ 11:84ad0fe93204

survey results (work in progress)

In October 2010 we opened an online survey, advertised to a
number of senior researchers in other groups around the UK. This
survey asked for detailed information about the software usage and authorship practices of
researchers, with the aim of obtaining a number of individual case
points for further examination as well as some broad numerical
results. The survey, started in November 2010, closed in April 2011,
with 54 complete and 23 partially complete responses. There were
responses from at least 16 different institutions.


\subsection{Software Development}

Our survey shows that 66,6\% of researchers use more than one OS. Linux isn’t
used as a single operating system by anybody, which seems to indicate
that most Linux users use it for multi-platform developing or for
specific software needs. SuperCollider, Android SDK/NDK, NET, PRAAT (Speech Researcher), CUDA-C
(GPU Programming), Clojure, Presentation, R.

Most researchers (56\%) use version control. This kind of system is
more widely used by PHD students and Postdocs/Research
Assistants. When asked for what kind of technologies were used, SVN
(14) and GIT (7) were the most popular systems. CVS (5) and Mercurial
(4) were the other available options. Many users use more than one of
these systems simultaneously.

When asked for the usage of code hosting services, 52\% of the researchers
said their code stayed in their computers. The most used third-party
source code hosting services is SourceForge (6 users). 10 users are
using university source code version control tools. \textit{remove
  numbers, only leave percentages}

27 users do not produce or maintain software. 16 do, while 11 did not
answer. \textit{percentages\ldots}

Most users (57\%) do not plan to make any software available.

When asked “Do you develop any software that you do not intend to
publish?”, 52\% of the users answered no. Possible commercial use is
the main justification given for not publishing the software at this
point.

\subsection{Reproducible Research}

Most researchers (56\%) acknowledge they don't take the necessary
steps to ensure sustainable and reproducible research. Many do not
understand the concept of reproducible research. By analyzing this
accordingly to the current position, we can see that PhD students are
the ones that are less aware of the importance of reproducible
research (even the ones that are almost finishing their PhD).

Many of the researchers that ensure they do the steps necessary to
reproducibility say they only give the code and/or data to interested
researchers. Some researchers also say that they publish their code in
their own pages. At the same time, there are indications that this
procedure can lead to unsustainability itself. Many researchers
complain about the amount of time and/or complexity of making research
reproducible. Also many of them make only parts of their work
available. Some researchers also complain about copyright issues in
releasing data.

Many researchers do not understand the full concept of
reproducibility. Some assume that explaining the algorithm and the
tools used is enough for other researchers to be able to reproduce
their results. Finally, some typical (but not widely admitted) answers
justify the decision not to embrace reproducibility due to messy code
or code/data protection:


%%% Local Variables: 
%%% mode: latex
%%% TeX-master: "cannam"
%%% End: