# HG changeset patch # User Henrik Ekeus # Date 1328539242 0 # Node ID e0cbf34f2487a266e7b981a2506f3baeebaba059 # Parent 60edf73674571a5e8dc24c920c1958f22c1404c2 Completed information dynamics introduction section diff -r 60edf7367457 -r e0cbf34f2487 nime2012/mtriange.pdf Binary file nime2012/mtriange.pdf has changed diff -r 60edf7367457 -r e0cbf34f2487 nime2012/mtriange.tex --- a/nime2012/mtriange.tex Mon Feb 06 14:09:39 2012 +0000 +++ b/nime2012/mtriange.tex Mon Feb 06 14:40:42 2012 +0000 @@ -16,7 +16,6 @@ %TODO % -%more background on expectation in music. Include refs. %more background on IDyOM. Include refs %formal descriptions of redundancy, entropy rate, predictive information rate %discussion on its use as a composition assistant.. @@ -70,14 +69,13 @@ Music involves patterns in time. When listening to music we continually build and re-evaluate expectations of what is to come next. Composers commonly, consciously or not, play with this expectation by setting up expectations which may, or may not be fulfilled. This manipulation of expectation and surprise in the listener has been articulated by music theorist Meyer\cite{Meyer:1967} and Narmour\cite{Narmour:1977}. Core to this is the idea that music is not a static object presented as a whole, as a Lerdahl and Jackendo analysis\cite{Lerdahl:1983} would imply, but as a phenomenon that `unfolds' and is experienced \emph{in time}. -The research into Information Dynamics explores several different kinds of predictability in musical patterns, how human listeners might perceive these, and how they shape or affect the listening experience. [more on IDyOM project] - +The information dynamic approach\cite{Abdallah:2009p4089} considers several different kinds of predictability in musical patterns, how human listeners might perceive these, and how they shape or affect the listening experience. Central to this is the idea that listeners maintain a dynamically evolving statistical model that enables them to make predictions about how a piece of music will continue. They do this using both the immediate context of the piece, and on previous musical experience, and as the music unfolds they continually revise their model. \section{The Melody Triangle} %%%How we created the transition matrixes and created the triangle. The Melody Triangle is based on first order Markov chains, represented as transition matrixes, that generate streams of symbols. By mapping the symbols to individual notes, melodies are generated. Further by layering these streams of notes can result in intricate musical textures. The choice of notes or scale is not a part of the Melody Triangle's core functionality, in fact the symbols could be mapped to anything, even non sonic outputs. -Any sequence of symbols can be analysed and information theoretic measures taken from it. The novelty of the Melody Triangle lies in that we go 'backwards' - given desired values for these measures, as determined from the user interface, we return a stream of symbols that match those measures. The information measures used are redundancy, entropy rate and predictive information rate. +Any sequence of symbols can be analysed and information theoretic measures taken from it. The novelty of the Melody Triangle lies in that we go `backwards' - given desired values for these measures, as determined from the user interface, we return a stream of symbols that match those measures. The information measures used are redundancy, entropy rate and predictive information rate. \subsection{Information measures} \subsubsection{Redundancy} @@ -85,7 +83,7 @@ Redundancy tells us the difference in uncertainty before we look at the context (the fixed point distribution) and the uncertainty after we look at context. For instance a transition matrix with high redundancy, such as one that represents a long periodic sequence, would have high uncertainty before we look at the context but as soon as we look at the previous symbol, the uncertainty drops to zero because we now know what is coming next. \subsubsection{Entropy rate} [todo - a more formal description] -Entropy rate is the average uncertainty for the next symbol as we go through the sequence. A looping sequence has 0 entropy, a sequence that is difficult to predict has high entropy rate. Entropy rate is an average of 'surprisingness' over time. +Entropy rate is the average uncertainty for the next symbol as we go through the sequence. A looping sequence has 0 entropy, a sequence that is difficult to predict has high entropy rate. Entropy rate is an average of `surprisingness' over time. \subsubsection{Predictive Information Rate} [todo - a more formal description] @@ -107,7 +105,7 @@ \begin{figure} \centering \includegraphics[width=0.5\textwidth]{MatrixDistribution.png} -\caption{The population of transition matrixes distributed along three axes of redundancy, entropy rate and predictive -information rate. Note how the distribution makes a curved triangle-like plane floating in 3d space. \label{InfoDynEngine}} +\caption{The population of transition matrixes distributed along three axes of redundancy, entropy rate and predictive information rate. Note how the distribution makes a curved triangle-like plane floating in 3d space. \label{InfoDynEngine}} \end{figure} \begin{figure}[h] \centering @@ -120,14 +118,14 @@ -When we look at the distribution of randomly generated transition matrixes and plotted in this space, we see that it forms an arch shape that is fairly thin. It thus becomes a reasonable approximation to pretend that it is just a sheet in two dimensions; and so we stretch out this curved arc into a flat triangle. It is this triangular sheet that is our 'Melody Triangle' and forms the interface by which the system is controlled. +When we look at the distribution of randomly generated transition matrixes and plotted in this space, we see that it forms an arch shape that is fairly thin. It thus becomes a reasonable approximation to pretend that it is just a sheet in two dimensions; and so we stretch out this curved arc into a flat triangle. It is this triangular sheet that is our `Melody Triangle' and forms the interface by which the system is controlled. When the Melody Triangle is used, regardless of whether it is as a screen based system, or as an interactive installation, it involves a mapping to this statistical space. Then a transition matrix corresponding to this position in statistical space is returned. As can be seen in figure \ref{TheTriangle}, a position within the triangle maps to different measures of redundancy, entropy rate and predictive information rate. %%%paragraph explaining what the different parts of the triangle are like. - [todo improve, include example melodies?] Each corner corresponds to three different extremes of predictability and unpredictability, which could be loosely characterised as periodicity, noise and repetition. Melodies from the 'noise' corner have no discernible pattern; they have high entropy rate, low predictive information rate and low redundancy. These melodies are essentially totally random. A melody along the 'periodicity' to 'repetition' edge are all deterministic loops that get shorter as we approach the 'repetition' corner, until it becomes just one repeating note. It is the areas in between that provide the more interesting melodies, those that have some level of unpredictability, but are not completely random and conversely that are predictable, but not entirely so. This triangular space allows for an intuitive exploration of expectation and surprise in temporal sequences based on a simple model of how one might guess the next event given the previous one. + [todo improve, include example melodies?] Each corner corresponds to three different extremes of predictability and unpredictability, which could be loosely characterised as periodicity, noise and repetition. Melodies from the `noise' corner have no discernible pattern; they have high entropy rate, low predictive information rate and low redundancy. These melodies are essentially totally random. A melody along the `periodicity' to `repetition' edge are all deterministic loops that get shorter as we approach the `repetition' corner, until it becomes just one repeating note. It is the areas in between that provide the more interesting melodies, those that have some level of unpredictability, but are not completely random and conversely that are predictable, but not entirely so. This triangular space allows for an intuitive exploration of expectation and surprise in temporal sequences based on a simple model of how one might guess the next event given the previous one. @@ -148,7 +146,7 @@ \subsubsection{Tracking and Control} -Tracking and control was done using the OpenNI libraries' API and high level middle-ware for tracking with Kinect. This provided reliable blob tracking of humanoid forms in 2d space. By triangulating this to the Kinect's depth map it became possible to get reliable coordinate of visitors positions in the space. +Tracking and control was done using the OpenNI libraries' API\footnote{http://OpenNi.org/} and high level middle-ware for tracking with Kinect. This provided reliable blob tracking of humanoid forms in 2d space. By triangulating this to the Kinect's depth map it became possible to get reliable coordinate of visitors positions in the space. This system was extended to detect gestures. By detecting the bounding box of the 2d blobs of individuals in the space, and then normalising these based on the distance of the depth map it became possible to work out if an individual had an arm stretched out or if they were crouching. @@ -178,7 +176,7 @@ The collaborative nature of this installation is one area that merits attention. By not having one user be able to control the whole narrative, the participants would communicate verbally and direct each other in the goals of learning to use the system, and eventually towards finding interesting musical textures. The collaborative nature added an element of playfulness and enjoyment that was obviously apparent. -As an artefact this installation is an exploratory prototype, and occupies an ambiguous role in terms of purpose; it is in a nebulous middle ground between instrument and art installation[, and could also form a framework for a kind of dance performance]. One thing is clear is that as a vehicle for communicating ideas related to the expectation, pattern and predictability in music it is very effective. +As an artefact this installation is an exploratory prototype, and occupies an ambiguous role in terms of purpose; it is in a nebulous middle ground between instrument and art installation. One thing is clear is that as a vehicle for communicating ideas related to the expectation, pattern and predictability in music it is very effective. \subsection{The Screen Based Interface}