MATCH

Field Detail

• ADVANCE_BOTH

  protected static final int ADVANCE_BOTH

  See Also:

  Constant Field Values

• ADVANCE_OTHER

  protected static final int ADVANCE_OTHER

  See Also:

  Constant Field Values

• ADVANCE_THIS

  protected static final int ADVANCE_THIS

  Encoding of minimum-cost steps performed in DTW algorithm.

  See Also:

  Constant Field Values

• audioFile

  protected AudioFile audioFile

  The AudioFile object from the AligningAudioPlayer, or null if the command line version is being used.

• audioFileName

  protected java.lang.String audioFileName

  Source of input data. Later to be extended to include live input from the sound card.

• audioFormat

  protected javax.sound.sampled.AudioFormat audioFormat

  Format of the audio data in pcmInputStream

• audioOut

  protected javax.sound.sampled.SourceDataLine audioOut

  Line for audio output (only one PerformanceMatcher should set this!)

• audioOutputRequested

  protected boolean audioOutputRequested

  Flag to select audio output from this PerformanceMatcher

• batchMode

  public static boolean batchMode

• bestPathCost

  protected int[][] bestPathCost

  The best path cost matrix.

• blockSize

  protected int blockSize

  Width of the search band in FFT frames (see blockTime)

• blockTime

  protected double blockTime

  The width of the search band (error margin) around the current match position, measured in seconds. Strictly speaking the width is measured backwards from the current point, since the algorithm has to work causally.

• cbIndex

  protected int cbIndex

  The index of the next position to write in the circular buffer.

• channels

  protected int channels

  Number of channels of audio in audioFormat

• circBuffer

  protected double[] circBuffer

  Audio data is scaled to the range [0,1] and averaged to one channel and stored in a circular buffer for reuse (if hopTime < fftTime).

• decay

  protected static final double decay

  See Also:

  Constant Field Values

• distance

  protected byte[][] distance

  The distance matrix.

• events

  protected EventList events

• fftSize

  protected int fftSize

  The size of an FFT frame in samples (see fftTime)

• fftTime

  protected double fftTime

  The size of an FFT frame in seconds, as set by the command line option -f FFTTime. (Default = 0.04644s). Note that the value is not taken to be precise; it is adjusted so that fftSize is always a power of 2.

• fileLength

  protected long fileLength

  Total number of audio frames, or -1 for live or compressed input.

• first

  protected int[] first

  The bounds of each row of data in the distance and path cost matrices.

• firstPM

  protected boolean firstPM

  Indicates which performance is considered primary (the score). This is the performance shown on the vertical axis, and referred to as "this" in the codes for the direction of DTW steps.

• frameCount

  protected int frameCount

  The number of frames of audio data which have been read.

• frameRMS

  protected double frameRMS

  RMS amplitude of the current frame.

• frames

  protected double[][] frames

  A block of previously seen frames are stored in this structure for calculation of the distance matrix as the new frames are read in. One can think of the structure of the array as a circular buffer of vectors. The last element of each vector stores the total energy.

• freqMap

  protected int[] freqMap

  A mapping function for mapping FFT bins to final frequency bins. The mapping is linear (1-1) until the resolution reaches 2 points per semitone, then logarithmic with a semitone resolution. e.g. for 44.1kHz sampling rate and fftSize of 2048 (46ms), bin spacing is 21.5Hz, which is mapped linearly for bins 0-34 (0 to 732Hz), and logarithmically for the remaining bins (midi notes 79 to 127, bins 35 to 83), where all energy above note 127 is mapped into the final bin.

• freqMapSize

  protected int freqMapSize

  The number of entries in freqMap. Note that the length of the array is greater, because its size is not known at creation time.

• guiVisible

  public static boolean guiVisible

• hopSize

  protected int hopSize

  Spacing of audio frames in samples (see hopTime)

• hopTime

  protected double hopTime

  Spacing of audio frames (determines the amount of overlap or skip between frames). This value is expressed in seconds and can be set by the command line option -h hopTime. (Default = 0.020s)

• ignore

  protected boolean[] ignore

  The frames to ignore because they are not annotated.

• imBuffer

  protected double[] imBuffer

  The imaginary part of the data for the in-place FFT computation. Since input data is real, this initially contains zeros.

• inputBuffer

  protected byte[] inputBuffer

  Audio data is initially read in PCM format into this buffer.

• last

  protected int[] last

  The bounds of each row of data in the distance and path cost matrices.

• liveInputBufferSize

  public static final int liveInputBufferSize

  See Also:

  Constant Field Values

• liveWorm

  protected boolean liveWorm

• ltAverage

  protected double ltAverage

  Long term average frame energy (in frequency domain representation).

• MASK

  protected static final int MASK

  See Also:

  Constant Field Values

• matchFileName

  protected java.lang.String matchFileName

  For assessing the matching algorithm, match files are used, which give the times and velocities of all notes (as recorded by the Boesendorfer SE290).

• matchFileOffset

  protected double matchFileOffset

  Onset time of the first note in the audio file, in order to establish synchronisation between the match file and the audio data.

• matrixVisible

  public static boolean matrixVisible

• MAX_LENGTH

  protected static final int MAX_LENGTH

  See Also:

  Constant Field Values

• MAX_RUN_COUNT

  protected static final int MAX_RUN_COUNT

  See Also:

  Constant Field Values

• maxFrames

  protected int maxFrames

  The total number of frames of audio data to be read.

• metadata

  protected PerformanceMatcher.MetaType metadata

• newFrame

  protected double[] newFrame

• normalise1

  protected boolean normalise1

  Flag (command line options -n1 and -N1) indicating whether or not each frame of audio should be normalised to have a sum of 1. (Default = true).

• normalise2

  protected boolean normalise2

  Flag (command line options -n2 and -N2) indicating whether or not the distance metric for pairs of audio frames should be normalised by the sum of the two frames. (Default = false).

• normalise3

  protected boolean normalise3

  Flag (command line options -n3 and -N3) indicating whether or not each frame of audio should be normalised by the long term average of the summed energy. (Default = false; assumes normalise1 == false).

• normalise4

  protected boolean normalise4

  Flag (command line options -n4 and -N4) indicating whether or not the distance metric for pairs of audio frames should be normalised by the log of the sum of the frames. (Default = true; assumes normalise2 == false).

• normalise5

  protected boolean normalise5

  Flag (command line options -n5 and -N5) indicating whether or not the distance metric for pairs of audio frames should be set to zero between annotated positions. (Default = false).

• otherMatcher

  protected PerformanceMatcher otherMatcher

  Points to the other performance with which this one is being compared. The data for the distance metric and the dynamic time warping is shared between the two matchers. In the original version, only one of the two performance matchers contained the distance metric. (See firstPM)

• outputBufferSize

  public static final int outputBufferSize

  See Also:

  Constant Field Values

• outputFileName

  protected java.lang.String outputFileName

• outputType

  protected PerformanceMatcher.PathType outputType

• paused

  protected boolean paused

  Interactive control of the matching process allows pausing computation of the cost matrices in one direction.

• pcmInputStream

  protected javax.sound.sampled.AudioInputStream pcmInputStream

  Uncompressed version of rawInputStream. In the (normal) case where the input is already PCM data, rawInputStream == pcmInputStream

• prevFrame

  protected double[] prevFrame

  The most recent frame; used for calculating the frame to frame spectral difference.

• progressCallback

  protected at.ofai.music.match.GUI.FileNameSelection progressCallback

  GUI component which shows progress of alignment.

• rawInputStream

  protected javax.sound.sampled.AudioInputStream rawInputStream

  Input data for this performance (possibly in compressed format)

• reBuffer

  protected double[] reBuffer

  The real part of the data for the in-place FFT computation. Since input data is real, this initially contains the input data.

• referenceFrequency

  protected double referenceFrequency

• runCount

  protected int runCount

  The number of frames sequentially processed by this matcher, without a frame of the other matcher being processed.

• sampleRate

  protected float sampleRate

  Sample rate of audio in audioFormat

• scale

  protected double scale

  Scaling factor for distance metric; must guarantee that the final value fits in the data type used, that is, (unsigned) byte. (Default = 16).

• silenceThreshold

  protected static final double silenceThreshold

  See Also:

  Constant Field Values

• silent

  protected static boolean silent

  Disable or enable debugging output

• stop

  public static boolean stop

• useChromaFrequencyMap

  protected boolean useChromaFrequencyMap

• useSpectralDifference

  protected boolean useSpectralDifference

  Flag (command line options -d and -D) indicating whether or not the half-wave rectified spectral difference should be used in calculating the distance metric for pairs of audio frames, instead of the straight spectrum values. (Default = true).

• window

  protected double[] window

  The window function for the STFT, currently a Hamming window.

• wormHandler

  protected at.ofai.music.match.WormHandler wormHandler

Constructor Detail

• PerformanceMatcher

  public PerformanceMatcher(PerformanceMatcher p)

  Constructor for PerformanceMatcher.

  Parameters:

  p - The PerformanceMatcher representing the performance with which this one is going to be matched. Some information is shared between the two matchers (currently one possesses the distance matrix and optimal path matrix).

Method Detail

• print

  public void print()

  For debugging, outputs information about the PerformanceMatcher to standard error.

• toString

  public java.lang.String toString()

  Gives some basic `header' information about the PerformanceMatcher.

  Overrides:

  toString in class java.lang.Object

• setOtherMatcher

  public void setOtherMatcher(PerformanceMatcher p)

  Adds a link to the PerformanceMatcher object representing the performance which is going to be matched to this one.

  Parameters:

  p - the PerformanceMatcher representing the other performance

• setProgressCallback

  public void setProgressCallback(at.ofai.music.match.GUI.FileNameSelection c)

  Adds a link to the GUI component which shows the progress of matching.

  Parameters:

  c - the PerformanceMatcher representing the other performance

• setMatchFile

  public void setMatchFile(java.lang.String fileName,
                           double tStart,
                           boolean isWorm)

  Sets the match file for automatic evaluation of the PerformanceMatcher.

  Parameters:

  fileName - The path name of the match file

  tStart - The offset of the audio recording, that is, the time of the first note onset relative to the beginning of the audio file. This is required for precise synchronisation of the audio and match files.

  isWorm - Indicates whether the match file is in Worm file format.

• setMatchFile

  public void setMatchFile(java.lang.String fileName,
                           double tStart)

• setLabelFile

  public void setLabelFile(java.lang.String fileName)

• writeLabelFile

  public void writeLabelFile(at.ofai.music.match.ScrollingMatrix sm)

• writeMidiFile

  public void writeMidiFile(at.ofai.music.match.ScrollingMatrix sm)

• setLiveInput

  public void setLiveInput()

  Sets up the streams and buffers for live audio input (CD quality). If any Exception is thrown within this method, it is caught, and any opened streams are closed, and pcmInputStream is set to null, indicating that the method did not complete successfully.

• setInputFile

  public void setInputFile(AudioFile f)

• setInputFile

  public void setInputFile(java.lang.String fileName)

  Sets up the streams and buffers for audio file input. If any Exception is thrown within this method, it is caught, and any opened streams are closed, and pcmInputStream is set to null, indicating that the method did not complete successfully.

  Parameters:

  fileName - The path name of the input audio file.

• init

  protected void init()

• closeStreams

  public void closeStreams()

  Closes the input stream(s) associated with this object.

• makeFreqMap

  protected void makeFreqMap(int fftSize,
                             float sampleRate,
                             double refFreq)

• makeStandardFrequencyMap

  protected void makeStandardFrequencyMap(int fftSize,
                                          float sampleRate,
                                          double refFreq)

  Creates a map of FFT frequency bins to comparison bins. Where the spacing of FFT bins is less than 0.5 semitones, the mapping is one to one. Where the spacing is greater than 0.5 semitones, the FFT energy is mapped into semitone-wide bins. No scaling is performed; that is the energy is summed into the comparison bins. See also processFrame()

• makeChromaFrequencyMap

  protected void makeChromaFrequencyMap(int fftSize,
                                        float sR,
                                        double refFreq)

• getFrame

  public boolean getFrame()

  Reads a frame of input data, averages the channels to mono, scales to a maximum possible absolute value of 1, and stores the audio data in a circular input buffer. Assumes 16 bit PCM, any number of channels.

  Returns:

  true if a frame (or part of a frame, if it is the final frame) is read. If a complete frame cannot be read, the InputStream is set to null.

• processFrame

  protected void processFrame()

  Processes a frame of audio data by first computing the STFT with a Hamming window, then scaling the frequency axis with an arbitrary mapping, then (optionally) computing the half-wave rectified spectral difference from the previous frame, then (optionally) normalising to a sum of 1, then calculating the distance to all frames stored in the otherMatcher and storing them in the distance matrix, and finally updating the optimal path matrix using the dynamic time warping algorithm.

• calcDistance

  protected int calcDistance(double[] f1,
                             double[] f2)

  Calculates the Manhattan distance between two vectors, with an optional normalisation by the combined values in the vectors. Since the vectors contain energy, this could be considered as a squared Euclidean distance metric. Note that normalisation assumes the values are all non-negative.

  Parameters:

  f1 - one of the vectors involved in the distance calculation

  f2 - one of the vectors involved in the distance calculation

  Returns:

  the distance, scaled and truncated to an integer

• getValue

  protected int getValue(int i,
                         int j,
                         boolean firstAttempt)

  Retrieves values from the minimum cost matrix.

  Parameters:

  i - the frame number of this PerformanceMatcher

  j - the frame number of the other PerformanceMatcher

  Returns:

  the cost of the minimum cost path to this location

• setValue

  protected void setValue(int i,
                          int j,
                          int dir,
                          int value,
                          int dMN)

  Stores entries in the distance matrix and the optimal path matrix.

  Parameters:

  i - the frame number of this PerformanceMatcher

  j - the frame number of the other PerformanceMatcher

  dir - the direction from which this position is reached with minimum cost

  value - the cost of the minimum path except the current step

  dMN - the distance cost between the two frames

• evaluateMatch

  public java.util.LinkedList<at.ofai.music.match.PerformanceMatcher.Onset> evaluateMatch(PerformanceMatcher pm)

  Matches two match files, creating a composite list of corresponding onset times, consisting of the mean onset times of the notes in each notated score position. This is the basis for evaluating the alignment of the two performances. The asynchrony of notationally simultaneous notes limits the accuracy achievable by the PerformanceMatcher.

• doMatch

  public static void doMatch(PerformanceMatcher pm1,
                             PerformanceMatcher pm2,
                             at.ofai.music.match.ScrollingMatrix s)

  Tracks a performance by choosing a likely band for the optimal path.

• processArgs

  public static int processArgs(PerformanceMatcher pm1,
                                PerformanceMatcher pm2,
                                java.lang.String[] args)

  Processes command line arguments.

  Parameters:

  pm1 - The first PerformanceMatcher

  pm2 - The second PerformanceMatcher

  args - Command line arguments
  Usage: java PerformanceMatcher [optional-args] inputFile1 inputFile2
  Optional args are:

  • -h hopTime spacing of audio frames (in seconds, default 0.01)
  • -f frameTime size of FFT (in seconds, default 0.01161)
  • -x maxFrames stop after maxFrames frames have been processed
  • -m1 matchFile1 offset1 matchFile + start time for inputFile1
  • -m2 matchFile2 offset2 matchFile + start time for inputFile2
  • -w1 wormFile1 wormFile for inputFile1
  • -w2 wormFile2 wormFile for inputFile2
  • -[nN]k Various options for normalisation, where n switches on, N switches off each option. k can have the following values:
    • 1: normalise each FFT frame (sum of energy = 1) before comparison (default=true)
    • 2: normalise distance metric by sum of energies of both frames (default=false)
    • 3: normalise each FFT frame by the medium-term average energy (default=false)
    • 4: normalise distance metric by thresholded log of sum of frames (default=true)
    • 5: set distance to zero for non-annotated positions in either file (default=false)
  • -d use half-wave rectified spectral difference (default)
  • -D do not use half-wave rectified spectral difference
  • -s scale set scaling factor for distance metric
  • -b set batch mode
  • -B unset batch mode (default)
  • -smooth length set smoothing window size to length
  • -a audio out (file 1)
  • -A audio out (file 2)
  • -l live input (file 1)
  • -L live input (file 2)
  • -w live worm output (file 1)
  • -W live worm output (file 2)
  • -z fileName worm output (map file 1 to 2)
  • -Z fileName worm output (map file 2 to 1)
  • -ob fileName output backward match path
  • -of fileName output forward match path
  • -os fileName output smoothed match path
  • -rf1 value set A4 reference frequency (default 440) for pm1
  • -rf2 value set A4 reference frequency for pm2

• main

  public static void main(java.lang.String[] args)

  Entry point for command line version of performance matcher.

  See Also:

  processArgs(PerformanceMatcher,PerformanceMatcher,String[]) for documentation of command-line arguments.