Tony: a tool for melody transcription

h1. Wiki

h2. Specification

The Tony tool will be a very simple user interface for the exact annotation of notes, note pitches and performance of human singing and other monophonic instruments.

h3. Users

We expect users to be unfamiliar with programming, but to be familiar with music and research in general. In order to illustrate this, these are some possible users:

# Friedrich (56) is a musicologist at a university in northern Germany. Sample publication: "Zur Ästhetik der Stimme bei Wagner - Eine vergleichende Analyse historischer und moderner Aufnahmen". He uses a Windows Vista laptop with, mainly for Email correspondence and keeping lists of audio recordings in an Excel spreadsheet. He listens to Music from CDs, noting down singing characteristics on a paper notepad. He plays the cello and organises outings to the Berlin philharmonics with his Seminar students.

# Sam (27) is an ethnomusicologist studying the music of Brazilian immigrants in Portugal. She records the immigrants' music on location in Lisbon and compares the rhythmic and 

# Peter (23) is part of a 

h3. Components

   * automatic pitch and note transcription methods

   * GUI for correction, and additional note-based or phrase-based annotations

   * export to RDF and graphics

GUI

The most important aspect of the GUI is that it should be simple because the users are unlikely to be familiar with complicated programs.

   * User A: German musicologist

      * is used to Windows 95 but recently acquired a laptop with Windows 7 installed

      * uses Word to write papers and Outlook for email, but no other computer programs

   * User B:

1. Signal processing: mainly pitch extraction via Yin, but including more outputs than the original Yin (in particular: frequency estimate, even if Yin thinks there's no pitch, and the underlying salience function).

1. Note alignment: align notes given a pitch file

1. Note detection: this may be implemented as a variant of the HMM I made, with three states per output.

1. Interactive GUI with these key features:

     * scrolling/zooming

          * scroll through piece horizontally (in time) and vertically (in pitch)

          * vertical zoom

     * Note editing

          * edit note start and end times,

          * edit continuous note pitch,

          * choose note pitch estimate (mean, median, manual)

          * turn note to rest (unpitched)

          * lock notes (i.e. prevent from re-estimation)

     * Audio Playback

          * playback original audio

          * sonify notes

          * loop around current note

     * Undo note and pitch track changes

     * Export notes and pitches

1. Further possible GUI features

     * automatic re-estimation after edits (kind of Maximum A Posteriori or something)

     * export as MIDI

1. Explicitly unwanted features

     * delete note (notes can only be made into rests -- we assume a contiguous, monophonic melody)

     * horizontal zoom

     * polyphonic processing