Software Carpentry at QM

h1. Software Carpentry workshops for MAT/C4DM in QMUL

h2. Dates/Where

*  6th-7th February 2013 -- MAT lab

h2. MAT lab computer setup

Computer lacks:

* Login for Luis (and Steve?)

* hg and EasyMercurial

* Matplotlib

* iPython

* Nose

* pip

* scikits.audiolab

* Emacs with a window (as opposed to in terminal) - Aquamacs?

Computer does have:

* A text editor (TextWrangler)

* Python 2.7 system install

* NumPy

h2. Short schedule

*Day 1 - 9.30am to 3pm*

* Day 1 - Morning

** Intro and get settled down - 20 mins - with coffee?

** Shell - *Steve* - material from Software Tools For Engineers (ELE595) + Steve's head - 30 min to 10.30

** Python intro - *Luis* - some potential material "here":https://bitbucket.org/softwaresaved/boot-camp-oxford-2012/src/09d7142640725f78620511ec83ce756b491fdcdb/PythonIntroScript.docx from SSI - 10.30 to 12

* Day 1 - Lunch 12 - 12.30

* Day 1 - Afternoon

** Version control - *Chris* - material "here":/projects/easyhg/wiki/SC2012BootcampPlan using Bitbucket - 12.30 to 2pm

** Data management points - conversation points in the area outside the MAT lab - 2pm to 3pm

*Day 2 - 10am to 4pm*

* Day 2 - Morning

** NumPy - *Luis* - material from Luis, in preparation - takes 1 hour

** Audio file opening etc - *Luis?* - material "here":/projects/audio-bootcamp-planning from Codasign

* Day 2 - Lunch - 12.30 to 1.15

* Day 2 - Afternoon

** Testing, test-driven development - *Chris?* - based on "ISMIR":/projects/soundsoftware-ismir-2012 tutorial - up to 2 hours inc coffee at end

** Philosophical stuff - Feedback cycles - *Chris?* - mixture of "What We Know":https://bitbucket.org/softwaresaved/boot-camp-oxford-2012/src/09d7142640725f78620511ec83ce756b491fdcdb/WhatWeKnow.ppt?at=default and "DAFx tutorial stuff":https://code.soundsoftware.ac.uk/attachments/561/SoundSoftware-DAFx-Software.pdf with some of "Why Crunch Mode Doesn't Work":http://www.igda.org/why-crunch-modes-doesnt-work-six-lessons - takes 1 hour at most

* Post-workshop assessment exercise

** Register for soundsoftware site

** Take the exercise from the last bit of the afternoon (based on ISMIR, but not quite the same) and complete some aspect of it.

** Do this within version control on soundsoftware - add one of us as project member

h2. Day 1

9am (?) to 2pm.

Day 1 is rather short because of the MAT lab being booked by someone else from 2pm.

h3. Morning 

* Intro to Shell (SJW)

** Directories: pwd, cd, mkdir, ls

** Files: cp, mv, rm, cat, editors

** scripts #!/bin/bash

** access control: chmod

** Finding stuff: grep, find, locate, which

*** Regular expressions

*** Shell wildcards

** Background tasks: &, fg, bg

** Disk usage: du, df

** Tying things together > < |

** commands history, tab autocompletion

** variables

** doing stuff somewhere else: ssh

** doing serious stuff: sudo

* More possibilities:

** dot

** wget

** rsync

** mounting network (home) directories

** working on octave

* Version Control (CC) (exercise over lunch?)

** Creating repositories on SSW

h3. Afternoon

* Data Management (SJW) - Should content be integrated into discussion of shell / be before VC ?

** Rationale

** File formats - open vs. proprietary, lossy vs. lossless

*** Open formats should remain accessible in the future

*** The actual output from a lossy data format may vary depending upon how it's decoded

*** Converting files - ImageMagick convert, mplayer (to wav) + lame/flac/ogg, ffmpeg

** Backing up

** Archiving

*** "standard" compression tools good on text-like files especially if areas of content are similar

*** special compression needed for complex binary data which varies a lot e.g. FLAC

** Publishing

** Documenting

** Available facilities in C4DM (relevance for MAT ?)

h2. Day 2

10am to 4pm.

h3. Morning 

* Python (LF)

** Variables

*** strings, integers, floats, lists, sets and dictionaries

*** By reference vs. by value

** Control-of-flow

** Libraries and modules - pip, easy_install

** OOP, objects and classes ?

* Python for research

** NumPy

*** Matplotlib

*** SciPy

** Python for Audio/Media: audioop, ossaudiodev, wave, aifc, sunau, PyMedia, pygame (sndarray, mixer, mixer.music), libpd (?), Python Audio Tools, scikits.audiolab, Medussa

h3. Afternoon

* Software engineering

** feedback cycles

** Unit Tests and TDD

*** nose

*Note*: We need to have some sort of evaluation so that we can give attendees a mark (some exercise to be submitted via version control a week later?)

h2. Outcomes

* Students to have their own version control repository on soundsoftware.ac.uk and to know how to use it (command-line and easyHg)

* Removal of fear of the command-line

* Knowledge of basic UNIX shell commands

* Knowledge of basic Python

* Knowledge of (some) Python libraries for media research

* Understanding of the need for data management and how to manage data

* Understand facilities available at C4DM

* Know where to go for more info

** Software Carpentry - software-carpentry.org

** Version control / project management - soundsoftware.ac.uk

** Python - python.org

** Data management - SoDaMaT

* Improved practices for software engineering, souce code management, data management