# HG changeset patch # User samer # Date 1328673244 0 # Node ID ea6a73f648c24247f1689b05319cf5f733a6a8be # Parent 01639248a31027b7af06792078648d72e4aa80ce Edits up to section 2.1. diff -r 01639248a310 -r ea6a73f648c2 nime2012/mtriange.pdf Binary file nime2012/mtriange.pdf has changed diff -r 01639248a310 -r ea6a73f648c2 nime2012/mtriange.tex --- a/nime2012/mtriange.tex Wed Feb 08 01:57:06 2012 +0000 +++ b/nime2012/mtriange.tex Wed Feb 08 03:54:04 2012 +0000 @@ -58,42 +58,94 @@ \begin{abstract} %The Melody Triangle is a Markov-chain based melody generator where the input - positions within a triangle - directly map to information theoretic measures of its output. -The Melody Triangle is an exploratory interface for the discovery of melodic content, where the input - positions within a triangle - directly map to information theoretic measures of the output. The measures are the entropy rate, redundancy and \emph{predictive information rate}\cite{Abdallah:2009p4089} of the melody. Predictive information rate is an information measure developed as part of the Information Dynamics of Music project\footnote{(IDyOM) http://www.idyom.org/}. It characterises temporal structure and is a way of modelling expectation and surprise in the perception of music. +The Melody Triangle is an exploratory interface for the discovery of melodic content, where the input---positions within a triangle---directly map to information theoretic measures associated with the output. The measures are the entropy rate, redundancy and \emph{predictive information rate}\cite{Abdallah:2009p4089} of the random process used to generate the sequnce of notes. These are all related to the \emph{predictability} of the the sequence and as such address the notions of expectation and surprise in +the perception of music. +%Predictive information rate is an information measure developed as part of the Information Dynamics of Music project\footnote{(IDyOM) http://www.idyom.org/}. It characterises a certain kind of temporal complexity in sequential random processes. -We describe the information dynamics approach, how it forms the basis of the Melody Triangle, and outline two of its incarnations. The first is a multi-user installation where collaboration in a performative setting provides a playful yet informative way to explore expectation and surprise in music. The second is a screen based interface where the Melody Triangle becomes a compositional tool for the generation of intricate musical textures using an abstract, high-level description of predictability. Finally we outline a pilot study where the screen-based interface was used under experimental conditions to determine how the three measures of predictive information rate, entropy and redundancy might relate to musical preference. +We describe some of the relevant ideas from information dynamics, how the Melody Triangle is defined in terms of these, and describe two physical incarnations of the Melody Triangle. The first is a multi-user installation where collaboration in a performative setting provides a playful yet informative way to explore expectation and surprise in music. The second is a screen based interface where the Melody Triangle becomes a compositional tool for the generation of musical textures where the user's control is at the abstract level of randomness and predictability. Finally we outline a pilot study where the screen-based interface was used under experimental conditions to determine how the three measures of predictive information rate, entropy and redundancy might relate to musical preference. \end{abstract} \keywords{Information dynamics, Markov chains, Collaborative performance, Aleatoric composition, Information theory} \section{Information dynamics} - Music involves patterns in time. When listening to music we continually build and re-evaluate expectations of what is to come next. Composers commonly, consciously or not, play with this expectation by setting up expectations which may, or may not be fulfilled. This manipulation of expectation and surprise in the listener has been articulated by music theorist Meyer\cite{Meyer:1967} and Narmour\cite{Narmour:1977}. Core to this is the idea that music is not a static object presented as a whole, as a Lerdahl and Jackendo analysis\cite{Lerdahl:1983} would imply, but as a phenomenon that `unfolds' and is experienced \emph{in time}. +Music involves patterns in time. When listening to music we continually build and re-evaluate expectations of what is to come next. Composers commonly, consciously or not, play with this process by setting up expectations which may, or may not be fulfilled, manipulating the expectations of the listener and inducing surprise or not as the music progresses +%and surprise in the listener has been articulated by music theorist Meyer +\cite{Meyer:1967,Narmour:1977}. Central to this is the idea that music is not a static object, presented as a whole, +%as the grammatical analysis of Lerdahl and Jackendoff \cite{Lerdahl:1983} might imply, +but as a phenomenon that `unfolds' and is experienced \emph{in time}. -The information dynamic approach\cite{Abdallah:2009p4089} considers several different kinds of predictability in musical patterns, how human listeners might perceive these, and how they shape or affect the listening experience. Central to this is the idea that listeners maintain a dynamically evolving statistical model that enables them to make predictions about how a piece of music will continue. They do this using both the immediate context of the piece as well as using previous musical experience. As the music unfolds listeners continually revise their model; in other words, they revise their own, subjective probabilistic belief state. +Information dynamics \cite{Abdallah:2009p4089} considers several different kinds of predictability in musical patterns, how these might be quantified using the tools of information theory, +%human listeners might perceive these, +and how they shape or affect the listening experience. Central to this is the idea that listeners maintain a dynamically evolving statistical model that enables them to make predictions about how a piece of music will continue. They do this using both the immediate context of the piece as well as using previous musical experience. As the music unfolds, listeners continually revise their model; in other words, they revise their own, subjective probabilistic belief state. These changes in probabilistic beliefs can be associated with +quantities of information; these are the focus of information dynamics. \section{The Melody Triangle} %%%How we created the transition matrixes and created the triangle. -The Melody Triangle enables the discovery of melodic content matching a set of information theoretic criteria. This criteria is the user input and maps to positions within a triangle. How exactly the triangle is formed relative to the information theoretic measures is outlined in section \ref{makingthetriangle}. The interface to the triangle may come in different forms; so far it has been realised as an interactive installation and as a traditional screen based interface. +The Melody Triangle enables the discovery of melodic content matching a set of information theoretic criteria. Positions within the triangle correspond with pairs of values of entropy rate and redundancy. The relationship with the predictive information rate is not explicitly controlled as this would require a three-dimensional interface, but an implicit relationship emerges, which is described in section \ref{makingthetriangle}. The physical interface to the Triangle has so far been realised in two forms: as an interactive installation and as a screen based interface. -The Melody Triangle does not generate the melodic content itself, but rather selects appropriate parameters for another system to generate the content. The implementations discussed in this paper use first order Markov chains as the content generator. However any generative system can be used, so long as it possible to define a listener model to calculate the appropriate information measures. +Given the information coordinates corresponding to a point in the triangle, we select from a pre-built +library of random processes, choosing one whose entropy rate and redundancy match the desired +values. The implementations discussed in this paper use first order Markov chains as the content generator, +since it is easy to compute the theoretically exact values of entropy rate, redundancy and predictive +information rate given the transition matrix of the Markov chain. However, in principle, any generative system could be used to create the library of sequences, given an appropriate probabilistic listener model supporting +the estimation of entropy rate and redundancy. -The Triangle operates on streams of symbols, and it is by mapping the symbols to individual notes that melodies are generated. Further by layering these streams intricate musical textures can be created. The choice of notes or scale is not a part of the Melody Triangle's core functionality, in fact the symbols could be mapped to anything, even non sonic outputs. +The Markov chain based implementation generates streams of symbols in the abstract; the alphabet of symbols is then mapped to a set of distinct sounds, such as pitched notes in a scale or a set of percussive +sounds. Further by layering these streams intricate musical textures can be created. The selection of +notes or sounds is arbitrary, as long as they are all distinguishable. +%)le is not a part of the Melody Triangle's core functionality, i +Indeed, the symbols could be mapped to even non sonic outputs such as visible shapes, colours, or movements. -Any sequence of symbols can be analysed and information theoretic measures taken from it. The novelty of the Melody Triangle lies in that we go `backwards' - given desired values for these measures, as determined from the user interface, we return a stream of symbols that match those measures. The information measures used are redundancy, entropy rate and predictive information rate. +Any sequence of symbols can be analysed and information theoretic measures estimated from it. +The novelty of the Melody Triangle lies in that we reverse this mapping: given desired values for these measures, as determined from the user interface, we return a stream of symbols with the desired properties. +In the next section we describe the three information theoretic measures that we use. -\subsection{Information measures} -\subsubsection{Redundancy} -[todo (Samer) - a more formal description] -Redundancy tells us the difference in uncertainty before we look at the context (the fixed point distribution) and the uncertainty after we look at context. For instance a transition matrix with high redundancy, such as one that represents a long periodic sequence, would have high uncertainty before we look at the context but as soon as we look at the previous symbol, the uncertainty drops to zero because we now know what is coming next. -\subsubsection{Entropy rate} -[todo (Samer) - a more formal description] -Entropy rate is the average uncertainty for the next symbol as we go through the sequence. A looping sequence has 0 entropy, a sequence that is difficult to predict has high entropy rate. Entropy rate is an average of `surprisingness' over time. +\subsection{Sequential information measures} -\subsubsection{Predictive Information Rate} -[todo (Samer) - a more formal description] -Predictive information rate tell us the average reduction in uncertainty upon perceiving a symbol; a system with high predictive information rate means that each symbol tells you more about the next one. +The \emph{entropy rate} of a random process is a basic measure of its randomness or +unpredictablity. Consider the viewpoint of an observer at a certain time, and split the +sequence into an infinite \emph{past}, as single symbol in the \emph{present}, and the +infinite \emph{future}. The entropy rate is a conditional entropy; informally: +\begin{equation} + \mathrm{EntropyRate} = H( \mathrm{Present} | \mathrm{Past}), +\end{equation} +that is, it represents our average uncertainty about the present symbol \emph{given} +that we have observed everything before it. Processes with zero entropy rate can +be predicted perfectly given enough of the preceeding context. -If we imagine a purely periodic sequence, each symbol tells you nothing about the next one that we didn't already know as we already know how the pattern is going. Similarly with a seemingly uncorrelated sequence, seeing the next symbol does not tell us anymore because they are completely independent anyway; there is no pattern. There is a subset of transition matrixes that have high predictive information rate, and it is neither the periodic ones, nor the completely un-corellated ones. Rather they tend to yield output that have certain characteristic patterns, however a listener can't necessarily know when they occur. However a certain sequence of symbols might tell us about which one of the characteristics patterns will show up next. Each symbols tell a us little bit about the future but nothing about the infinite future, we only learn about that as time goes on; there is continual building of prediction. +The \emph{redundancy} of the a process, in the sense we are using the term here, is +a measure of how much the predictability of the process depends on knowing the +preceeding context. It is the difference between the entropy of a single element of the +sequence in isolation (imagine chosing a note from a musical score at random with your +eyes closed and then trying to guess the note) and its entropy after taking into account +the preceeding context: +\begin{equation} + \mathrm{Redundancy} = H( \mathrm{Present} ) - H(\mathrm{Present} | \mathrm{Past}). +\end{equation} +If the previous symbols reduce our uncertainty about present symbol a great deal, then +the redundancy is high. For example, if we know that a sequence consists of a repeating +cycle such as $ \ldots b, c, d, a, b, c, d, a \ldots$, but we don't know which was the first +symbol, then the redundancy is high, as $H(\mathrm{Present})$ is high (because we +have no idea about the present symbol in isolation, but $H(\mathrm{Present}|\mathrm{Past})$ +is zero, because knowing the previous symbol immediately tells us what the present symbol is. + +The \emph{predictive information rate} (PIR) brings in our uncertainty about the future. It is a +measure of how much each symbol reduces our uncertainty about the future as it is +observed, \emph{given} that we have observed the past: +\begin{equation} + \mathrm{PIR} = H(\mathrm{Future} | \mathrm{Past}) - H(\mathrm{Future} | \mathrm{Present}, \mathrm{Past}). +\end{equation} +It is a measure of the \emph{new} information in each symbol +Notice that if the past completely determines both the present and the future (as in the cyclic +patter above) the PIR is zero, since the present symbol brings no new information. However, +if the symbols in a sequence are generated completely independently, e.g. by rolling a die for each +one, then again, the present symbol provides no information about the future and the PIR +is zero. However, there do exist processes that have high predictive information rates as compared +with their entropy rates: within the class of Markov chains, these are neither the periodic nor the sequentially uncorrellated ones. Rather they tend to yield sequences that have certain recognisable patterns or motifs, +but which occur at irregular times. A certain symbol might tell us about which one of the characteristic patterns will appear next. Each symbol tell a us little bit about the future; in order to make good predictions, +the listener must continually pay attention, building up expectations on the basis of each new observation. +% but only a limited amount about the infinite future, we only learn about that as time goes on; there is continual building of prediction.