Software Carpentry at QM

h1. Software Carpentry workshop for MAT at QMUL, Feb 2013

h2. Post-workshop links

* [[MATFeb2013AssessedExercise|Assessed exercise]] - [[MATFeb2013AssessmentEvaluation|How we marked the exercise]]

* [[MATFeb2013Material|Download links for workshop material]]

h2. Dates/Where

*  6th-7th February 2013 -- MAT lab

h2. Planned schedule

*Day 1 - 9.30am to 3pm*

* Day 1 - Morning

** *Introduction* - "material":/attachments/650/intro.pdf - 5 mins - *Chris*

** *Shell* - "material":/projects/jisc-training/repository/show/presentations/MAT%20SWC - 45 mins - *Steve*

** Short coffee break

** *Python* - "material":/documents/71 - 90 mins - *Luis*

* Day 1 - Lunch 12 - 12.30

* Day 1 - Afternoon

** *Version control* - "material":/projects/easyhg/wiki/SC2012BootcampPlan - 90 mins - *Chris*

** *Data management* - "material":/projects/jisc-training/repository/show/presentations/MAT%20SWC - conversation points in area outside lab - 60 mins - *Steve*

*Day 2 - 10am to 4pm*

* Day 2 - Morning

** *Python modules and NumPy* - material in preparation - 60 mins - *Luis*

** Coffee

** *Audio files and more NumPy* - material in preparation based on "this":/projects/audio-bootcamp-planning from Codasign - 80 mins - *Luis*

* Day 2 - Lunch - 12.30 to 1.15

* Day 2 - Afternoon

** *Testing* - [[SWC2013TDD|material]] - 90 mins - *Chris*

** Set the assessment exercise; coffee

** *What We Know* - "material":/documents/70 - 50 mins - *Chris*

* Post-workshop assessment exercise

** -Register for soundsoftware site- No, the exercise will use BitBucket as well (though we tell them about the SoundSoftware site)

** Take the exercise from the last bit of the afternoon (based on ISMIR, but not quite the same) and complete some aspect of it.

** Do this within version control on soundsoftware - add one of us as project member

h2. Actual schedule

The timings we actually ended up having.

*Day 1 - 9.30am to 3pm*

* Wait for people to arrive - 15 mins

* Short intro - Chris - 5 mins

* Shell - Steve - 60 mins

* Coffee - 15 mins

* Intro to Python - Luis - 50 mins

* Lunch - 40 mins (12.00 - 12.40)

* Version control - Chris - 80 mins

* Data - Steve - 60 mins (inc. words from Pat at the start)

*Day 2 - 10am to 4pm*

* Wait for people to arrive - 15 mins

* Python functions - Luis - 95 mins

* Coffee - 10 mins

* Audio in Python - Luis - 45 mins

* Lunch - 60 mins (12.45 - 13.45) (inc. words from Pat at the end)

* Testing - Chris - 60 mins

* Coffee - 20 mins

* Preview assessed exercise - 5 mins

* What We Know - Chris - 50 mins

h2. MAT lab computer setup

Computer lacks:

* Login for Luis (and Steve?) *fixed*

* hg and EasyMercurial *fixed*

* Matplotlib *fixed*

* iPython *fixed*

* Nose *fixed*

* pip _unsure_

* scikits.audiolab _unsure_

* Emacs with a window (as opposed to in terminal) - Aquamacs? *use textwrangler*

Computer does have:

* A text editor (TextWrangler)

* Python 2.7 system install

* NumPy

h1. Detailed plan for schedule

h2. Day 1

9am (?) to 2pm.

Day 1 is rather short because of the MAT lab being booked by someone else from 2pm.

h3. Morning 

* Intro to Shell (SJW)

** Directories: pwd, cd, mkdir, ls

** Files: cp, mv, rm, cat, editors

** scripts #!/bin/bash

** access control: chmod

** Finding stuff: grep, find, locate, which

*** Regular expressions

*** Shell wildcards

** Background tasks: &, fg, bg

** Disk usage: du, df

** Tying things together > < |

** commands history, tab autocompletion

** variables

** doing stuff somewhere else: ssh

** doing serious stuff: sudo

* More possibilities:

** dot

** wget

** rsync

** mounting network (home) directories

** working on octave

* Version Control (CC) (exercise over lunch?)

** Creating repositories on SSW

h3. Afternoon

* Data Management (SJW) - Should content be integrated into discussion of shell / be before VC ?

** Rationale

** File formats - open vs. proprietary, lossy vs. lossless

*** Open formats should remain accessible in the future

*** The actual output from a lossy data format may vary depending upon how it's decoded

*** Converting files - ImageMagick convert, mplayer (to wav) + lame/flac/ogg, ffmpeg

** Backing up

** Archiving

*** "standard" compression tools good on text-like files especially if areas of content are similar

*** special compression needed for complex binary data which varies a lot e.g. FLAC

** Publishing

** Documenting

** Available facilities in C4DM (relevance for MAT ?)

h2. Day 2

10am to 4pm.

h3. Morning 

* Python (LF)

** Variables

*** strings, integers, floats, lists, sets and dictionaries

*** By reference vs. by value

** Control-of-flow

** Libraries and modules - pip, easy_install

** OOP, objects and classes ?

* Python for research

** NumPy

*** Matplotlib

*** SciPy

** Python for Audio/Media: audioop, ossaudiodev, wave, aifc, sunau, PyMedia, pygame (sndarray, mixer, mixer.music), libpd (?), Python Audio Tools, scikits.audiolab, Medussa

h3. Afternoon

* Software engineering

** feedback cycles

** Unit Tests and TDD

*** nose

*Note*: We need to have some sort of evaluation so that we can give attendees a mark (some exercise to be submitted via version control a week later?)

h2. Outcomes

*Note* some of these are C4DM-specific for the C4DM version of the workshop

* Students to have their own version control repository on soundsoftware.ac.uk and to know how to use it (command-line and easyHg)

* Removal of fear of the command-line

* Knowledge of basic UNIX shell commands

* Knowledge of basic Python

* Knowledge of (some) Python libraries for media research

* Understanding of the need for data management and how to manage data

* Understand facilities available at C4DM

* Know where to go for more info

** Software Carpentry - software-carpentry.org

** Version control / project management - soundsoftware.ac.uk

** Python - python.org

** Data management - SoDaMaT

* Improved practices for software engineering, souce code management, data management