view musicweb.tex @ 56:483df26a74d8 tip

minor corrections and changes to rebuttal

\documentclass{llncs}
\usepackage{graphicx}

\usepackage{amsmath,cite}
\usepackage{pifont}
\usepackage{listings}
\usepackage{courier}
\usepackage{adjustbox}
\usepackage{url}
\usepackage{parskip}
\usepackage[ngerman,english]{babel}
\usepackage[utf8]{inputenc}
\newcommand{\cmark}{\fontsize{14}{14}\textbullet\selectfont}
\usepackage[usenames,dvipsnames]{color}

\lstdefinelanguage{sparql} {
	morekeywords={SELECT, WHERE, a, COUNT, GROUP, ORDER, BY, HAVING, as, DISTINCT, OPTIONAL},
	sensitive=true,
	morecomment=[l][\color{ForestGreen}\bfseries\selectfont]{\#\ },
	moredelim=[s][\color{ForestGreen}\bfseries\selectfont]{?}{\ }, %variable
	moredelim=[s][\color{NavyBlue}\bfseries\selectfont]{@}{\ }, %prefix
	moredelim=[s][\color{OrangeRed}\bfseries\selectfont]{<}{>}, %red URIs
	morestring=[b][\color{ForestGreen}]\", %green strings
	morestring=[b]",
}

\lstdefinestyle{sparql}{
	language=sparql,
	basicstyle=\footnotesize\ttfamily\bfseries,
	% basicstyle=\ttfamily,
	keywordstyle=\color{blue}\bfseries,
	identifierstyle=,
	commentstyle=\color{ForestGreen}\bfseries,
	stringstyle=\color{BlueGreen}\ttfamily,
	showstringspaces=false,
	emph={context},emphstyle=\color{BrickRed}\bfseries,
	frame = lines,
	%aboveskip = 15pt,
	%framexbottommargin = 5pt,
	%framextopmargin = 5pt,
	aboveskip = 10pt,
	framexbottommargin = 2pt,
	framextopmargin = 2pt,
	captionpos=b,
	escapechar=~,
	breaklines=true,
	breakatwhitespace=true,
	breakindent=20pt,
	xleftmargin=5pt,
	xrightmargin=5pt,
	basewidth={0.55em,0.45em},
	%basewidth={0.50em,0.6em},
	fontadjust=true,
	% lineskip={-1.5pt},
	% columns=[c]spaceflexible, %make the long artist URI fit
	%float = h!,
}

\renewcommand{\lstlistingname}{{\bf Listing}}


\lstdefinelanguage{ttl} {
	morekeywords={a},
	sensitive=true,
	% morecomment=[l][\color{ForestGreen}\bfseries\fontsize{10}{2}\selectfont]{\#\ },
	% moredelim=[s][\color{ForestGreen}\bfseries\fontsize{10}{2}\selectfont]{\#\ }{\ },
	% moredelim=[s][\color{NavyBlue}\bfseries\fontsize{10}{2}\selectfont]{@}{\ }, %prefix
	% moredelim=[s][\color{OrangeRed}\bfseries\fontsize{10}{2}\selectfont]{<}{>}, %red URIs
	morecomment=[l][\color{ForestGreen}\ttfamily\bfseries\selectfont]{\#\ },
	moredelim=[s][\color{NavyBlue}\ttfamily\bfseries\selectfont]{@}{\ }, %prefix
	moredelim=[s][\color{OrangeRed}\ttfamily\bfseries\selectfont]{<}{>}, %red URIs
	morestring=[b][\color{ForestGreen}]\", %green strings
	morestring=[b]",
}

\lstdefinestyle{ttl}{
	language=ttl,
	basicstyle=\footnotesize\ttfamily\bfseries,
	keywordstyle=\color{blue}\bfseries,
	identifierstyle=,
	commentstyle=\color{ForestGreen}\bfseries,
	stringstyle=\color{BlueGreen}\bfseries,
	showstringspaces=false,
	emph={context},emphstyle=\color{BrickRed}\bfseries,
	frame = lines,
	framexbottommargin = 5pt,
	framextopmargin = 5pt,
	captionpos=b,
	escapechar=~,
	breaklines=true,
	breakatwhitespace=true,
	breakindent=20pt,
	aboveskip = 15pt,
	xleftmargin=5pt,
	xrightmargin=5pt,
	basewidth={0.55em,0.45em},
	% basewidth={0.50em,0.6em},
	fontadjust=true,
	lineskip={-1.5pt},
	% columns=[c]spaceflexible, %make the long artist URI fit
	% float = h!,
}

\lstdefinelanguage{json} {
	sensitive=true,
	%morecomment=[l]{@},
	moredelim=[s][\color{ForestGreen}]{?}{\ }, %green variables
	moredelim=[s][\color{OrangeRed}\fontsize{7}{2}\selectfont]{<}{>}, %red URIs
	morestring=[b][\color{Gray}]\",
	morestring=[b]",
}

\lstdefinestyle{json}{
	language=json,
	keywords={type,context},
	basicstyle=\fontsize{7}{8}\bfseries\ttfamily,
	keywordstyle=\color{blue}\bfseries,
	identifierstyle=,
	commentstyle=\color{Gray}\bfseries,
	stringstyle=\color{OliveGreen}\ttfamily,
	showstringspaces=false,
	emph={context},emphstyle=\color{BrickRed}\bfseries,
	frame = lines,
	framexbottommargin = 5pt,
	framextopmargin = 5pt,
	tabsize=4,
	captionpos=b,
	escapechar=~,
	breaklines=false,
	xleftmargin=5pt,
	xrightmargin=5pt,
	basewidth={0.50em,0.45em},
	%basewidth={0.50em,0.6em},
	fontadjust=true,
	columns=[c]spaceflexible, %make the long artist URI fit
	float = ht,
}

\renewcommand{\labelitemi}{$\bullet$}

\begin{document}

\mainmatter

\title{MusicWeb: an open linked semantic platform for music metadata}

\author{Mariano Mora-Mcginity \and Alo Allik \and Gy\"orgy Fazekas \and Mark Sandler }
%

\institute{Queen Mary University of London, \\
\email{\{m.mora-mcginity, a.allik, g.fazekas, mark.sandler\}@qmul.ac.uk}}

\maketitle

\begin{abstract}

% MusicWeb is a web site that provides users a browsing, searching and linking platform of music artist and group information by integrating open linked semantic metadata from various Semantic Web, music recommendation and social media data sources, including DBpedia.org, sameas.org, MusicBrainz, the Music Ontology, Last.FM, Youtube, and Echonest. The front portal includes suggested links to selected artists and a search functionality from where users can navigate to individual artists pages. Each artist page contains a biography, links to online audio and a video player with a side menu displaying a selection of Youtube videos. Further it provides lists of YAGO categories linking each artist to other artists by various commonalities such as style, geographical location, instrumentation, record label as well as more obscure categories, for example, artists who have received the same award, have shared the same fate, or belonged to the same organisation or religion. The artist connections are further enhanced by thematic analysis of journal articles and blog posts as well as content-based music information retrieval similarity measures.

This paper presents MusicWeb, a novel platform for linking music artists within a web-based application for discovering connections between them. MusicWeb provides a browsing experience using connections that are either extra-musical or tangential to music, such as the artists' political affiliation or social influence, or intra-musical, such as the artists' main instrument or most favoured musical key. The platform integrates open linked semantic metadata from various Semantic Web, music recommendation and social media data sources including DBpedia.org, sameas.org, MusicBrainz, the Music Ontology, Last.FM and Youtube as well as content-derived information. The front portal includes suggested links to selected artists and a search functionality from where users can navigate to individual artists pages. Each artist page contains a biography and links to online audio and a video resources. Connections are made using YAGO categories linking artist by various commonalities such as style, geographical location, instrumentation, record label as well as more obscure categories, for instance, artists who have received the same award, have shared the same fate, or belonged to the same organisation or religion. These connections are further enhanced by thematic analysis of journal articles and blog posts as well as content-based similarity measures focussing on high level musical categories.

\keywords{Semantic Web, Linked Open Data, music metadata, semantic audio analysis, music information retrieval }
\end{abstract}


\section{Introduction}\label{sec:introduction}
In recent years we have witnessed an explosion of information, a consequence of millions of users producing and consuming web resources. Researchers and industry have recognised the potential of this data, and have endeavoured to develop methods to handle such a vast amount of information: to understand and manage it, to transform into knowledge. Multimedia content providers have devoted a lot of energy to analysing consumer preference, in an effort to offer customised user experiences. Music stream services, for instance, carry out extensive analysis trying to identify patterns in user's listening habits, and researchers are striving to refine multimedia recommendation algorithms. There are two main approaches to music recommendation\cite{Song2012}: the first is known as \emph{collaborative filtering}\cite{Su2009}, which recommends music items based on the choices of similar users.  The second model is based on audio content analysis, or \emph{music information retrieval}. The task here is to extract low to high-level audio features such as tempo, key, metric structure, melodic and harmonic sequences, instrument recognition and song segmentation, which are then used to measure music similarity\cite{Aucoutourier2002}, to carry out genre classification or to identify the mood of the song\cite{Kim2010}. Music discovery websites such as Last.fm\footnote{http://www.last.fm}, Allmusic\footnote{http://www.allmusic.com} or Pandora\footnote{http://www.pandora.com} have successfully developed hybrid systems which combine both approaches.\\
There are, however, limitations in both approaches to music recommendation. Most users participating in (or whose data is used to analyse) collaborative filtering listen to a very small percentage of the music available, the so called ``short-tail'', whereas the much larger ``long-tail'' remains mainly unknown\cite{Celma2010}. These systems will show a bias towards music that is already consumed by many listeners. Suggesting already popular music will increase the likelihood of it being recommended to new users, thus creating a \emph{rich-club phenomenon}\cite{Zhou2004} or what is known as \emph{cumulative advantage}. Also, content analysis of audio features is mainly applied to songs: systems can recommend similar tracks, but generally know nothing about similar artists. Many music listeners follow artists because of their style and would be interested in music from similar artists. It is very hard to pinpoint what exactly makes two artists ``similar'': very often notions of similarity are based on social and cultural issues, rather than a precise definition of style.\\
 To many music lovers discovering new music, or music they weren't aware of, is an integral part of enjoying a musical experience, and appreciate expanding their musical taste. Lee and Price\cite{Lee2015} identify seven different \emph{personas} which tipify music service consumption. Two such personas, for instance, the ``active curator'' and the ``music epicurean'' characteristically spend a long time hunting for new music, whereas the ``wanderer'' enjoys the discovery process itself, trying out new things with an open mind.  Automatic musical discovery is a very challenging problem\cite{Jennings2007}. There are many different ways in which people are attracted to new artists: word of mouth, their network of friends, music magazines or blogs, songs heard in a movie or a T.V. commercial, they might be interested in a musician who has played with another artist or been mentioned as an influence,  etc. The route from listening to one artist and discovering a new one would sometimes seem very disconcerting were it to be drawn on paper. A listener is not so much following a map as exploring new territory, with many possible forks and shortcuts. Music discovery systems generally disregard this kind of information, often because it is very nuanced and difficult to parse and interpret.\\
MusicWeb is an application which offers the user the possibility of exploring editorial, cultural and musical links between artists. It gathers, extracts and manages musical metadata from many different sources and connects them in informative ways. This paper deals with the different ways in which musicweb collects these resources and shapes them into high-level information. We will first review various knowledege-based web resources available to musicweb. We will then introduce the application itself and detail the architecture to analyse and extract data. Before the final conclusions and discussion of future work we will analyse the experience of interfacing with the application and how users can explore and discover new musical paths.



\section{Background}\label{sec:background}
Researchers have realised the usefulness of musical metadata and have for some time tried to collect and exploit music metadata for knowledge representation. Several ontologies have been and continue to be developed to link music metadata, such as the music ontology\cite{DBLP:conf/ismir/RaimondASG07}, which defines all objects in the process of creation, interpretation and distribution of music; the similarity ontology\cite{jacobson2011}, which allows for associations based on similarity of all musical elements contained in the music ontology; the studio ontology, which can be used to describe all elements in music studio environments\cite{fazekas2011studio}; or the audio effects ontology\cite{wilmering2013}, permitting the description of audio effects employed in music production processes. Linked music metadata is full of promise. However, most attempts to make use of linked metadata to guide music discovery have stressed some aspects of metadata while ignoring others. Pachet\cite{Pachet2005} identifies three types of musical metadata:
\begin{enumerate}
  \item Editorial metadata: information that is provided manually by authoritative experts. There is a wide range of potential producers of this kind of data, from record labels to collaborative schemes, as well as different kinds of data, from which musician played in which song to tour info, to artists' biography.
  \item Cultural metadata: information which is produced by the environment or culture. This is data that is not explicitly entered into some information system, but rather is contained, and must be extracted from, other information sources, such as user trends, google searches, articles and magazines, word associations in blogs, etc.
  \item Acoustic metadata: data extracted from audio files using music information retrieval methods.
\end{enumerate}
Of these, only the first has been exploited to a significant degree. Web resources for music discovery which employ linked data such as musicbrainz or lastfm rely mostly on editorial metadata. Commercial recommendation systems make use of limited cultural metadata, mainly through collaborative filtering.
To our knowledge the first recommedation system based on linked data was proposed in \cite{celma2008foafing}, which used web crawling to gather data which could then be offered to the user. Recommendation was based on profiling the user's listening habits and \emph{foaf} connections. A further step was taken in \cite{heitmann2010}, in which the author addresses common problems in recommender system such the new item problem or the new user problem. \emph{dbrec}, a recommender system introduced in \cite{passant2010dbrec}, suggested music obtained from dbpedia by computing a measure of semantic distance as the number of indirect and distinct links between resources in a graph. The system offered the user an explanation for each recommendation, listing the resources shared by the artists recommended.
\cite{nguyen2015} explores the effectiveness of recommendation systems based on knowledge encyclopedias such as dbpedia and freenet. The authors compute several different similarity measures of linked data extracted from both datasets, which they then feed to a recommender system.\\
There are several web resources offering services similar to MusicWeb. One of them is musikipedia\footnote{http://musikipedia.org/}. The user can visit a page for an artist and listen to music or watch videos. The user can also link to other artists that are connected to the current one, and an explanation of the connection is offered. Links are extracted from dbpedia and offer all common categories between artists.\\
Dbpedia and freebase are two of the most common sources of linked data available. There are several other sources of music metadata. AcousticBrainz\footnote{https://acousticbrainz.org/} is an crowd source information resource which contains low and high level music metadata, including audio and editorial features. Acousticbrainz is participated by musicbrainz, which is also a major container of linked editorial metadata.



\section{MusicWeb architecture}

MusicWeb provides a browsing experience using connections that are either extra-musical or tangential to music, such as the artists' political affiliation or social influence, or intra-musical, such as the artists' main instrument or most favoured musical keys. It does this by pulling data from several different web knowledge content resources and presenting them for the user to navigate in a faceted manner\cite{Marchionini2006}. The listener can begin his journey by choosing or searching an artist. The application offers youtube videos, audio streams, photographs and album covers, as well as the artist's biography. The page also includes many box widgets with links to artists who are related to the current artist in different, and sometimes unexpected ways (figure \ref{fig:ella_page}). The user can then click on any of these artists and the search commences again, exploring a web of artists further and further.


\begin{figure}[!ht]
      \centering
	\includegraphics[width=\textwidth, height=9cm]{graphics/ella_page.png}
	\caption{Ella}
	\label{fig:ella_page}
\end{figure}


MusicWeb was originally conceived as a platform for collating metadata about music artists using already available online linked data resources. The core functionality of the platform relies on available SPARQL endpoints sa well as various commercial and community-run APIs. More recently, novel services complement the platform to provide alternative ways to forge connections using natural language processing and machine learning methods. %The front portal includes suggested links to selected artists and a search functionality from where users can navigate to individual artists pages. Each artist page contains a biography, a playlist of online audio and a selection of Youtube videos. Further it provides lists of categories linking each artist to other similar artists by various commonalities such as style, geographical location, instrumentation, record label as well as more obscure categories, for example, artists who have received the same award, have shared the same fate, or belonged to the same organisation or religion. The artist connections are further enhanced by thematic analysis of journal articles and blog posts, content-based music information retrieval similarity metrics and proximity measures in a 2-dimensional mood space.

The MusicWeb API uses a number of LOD resources and Semantic Web ontologies to process and aggregate information about artists:

\begin{itemize}

	\item[] \textbf{Musicbrainz}\footnote{\url{http://musicbrainz.org}} is an online, open, crowd-sourced music encyclopedia, that provides reliable and unambiguous identifiers for entities in music publishing metadata, including artists, releases, recordings, performances, etc.

	\item[] \textbf{DBPedia}\footnote{\url{http://dbpedia.org}} is a crowd-sourced community effort to extract structured information from Wikipedia and make it available on the Web.

	\item[] \textbf{Sameas.org}\footnote{\url{http://sameas.org}} manages URI co-references on Web of Data.

	\item[] \textbf{Youtube} API is used to query associated video content for the artist panel.

	\item[] \textbf{Echonest}\footnote{\url{http://the.echonest.com}} was a music metadata and information retrieval platform for developers and media companies, which has since been integrated into Spotify.

	\item[] \textbf{Last.fm}\footnote{\url{http://last.fm}} is an online music social network and recommender system that collects information about users listening habits and makes available crowd-sourced tagging data through an API.

	\item[] \textbf{YAGO}\cite{Suchanek:WWW:2007} is a semantic knowledge base that collates information and structure from Wikipedia, WordNet and GeoNames with high accuracy. The ontology makes use of the categories defined in Wikipedia as a principle for semantic linking of entities, while exploiting the clean taxonomy of concepts from WordNet.

	\item[] \textbf{the Music Ontology}\cite{DBLP:conf/ismir/RaimondASG07} provides main concepts and properties for describing musical entities, including artists, albums, tracks, performances, compositions, etc., on the Semantic Web

\end{itemize}

\begin{figure}[!ht]
	\centering
	\includegraphics[scale=0.5]{graphics/architecture.pdf}%\vspace{-5pt}
	\caption{MusicWeb architecture}\vspace{-10pt}
	\label{fig:layers}
\end{figure}

\begin{figure}[!ht]
	\centering
	\includegraphics[scale=0.5]{graphics/mw_flow.pdf}%\vspace{-5pt}
	\caption{MusicWeb architecture}\vspace{-10pt}
	\label{fig:layers}
\end{figure}

The global MusicBrainz identifiers enable convenient and concise means to disambiguate between potential duplicates or irregularities in metadata across resources, a problem which is all too common in systems relying on named entities. Besides identifiers, the MusicBrainz infrastructure is also used for the search functionality of MusicWeb. However, in order to query any information in DBpedia, the MusicBrainz identifiers need to be associated with a DBpedia resource, which is a different kind of identifier. This mapping is achieved by querying the Sameas.org co-reference service to retrieve the corresponding DBpedia URIs. The caveat in this process is that Sameas does not actually keep track of MusicBrainz artist URIs, however, by substituting the domain for the same artist's URI in the BBC domain\footnote{\url{http://www.bbc.co.uk/music/artists/}}, the service can get around this obstacle. Once the DBpedia artist identity is determined, the service proceeds to construct the majority of the profile, including the biography and most of the linking categories to other artists. The standard categories available include associated artists and artists from the same hometown, while music group membership and artist collaboration links are queried from MusicBrainz. The core of the Semantic Web linking functionality is provided by categories from YAGO. The Spotify\footnote{historically the Echonest} and Last.fm APIs are used for recommendations that are based on different similarity calculations, thus providing recommendations that do not necessarily overlap.

\section{Artist similarity}
Music does not lend itself easily to categorisation. There are many ways in which artist can be, and in fact are, considered to be related. Similarity may refer to whether artists' songs sound similar, or are perceived to be in the same style or genre. But it may also mean that they are followed by people from similar social backgrounds or political inclinations, or similar ages; or perhaps they are similar because they have played together, or participated in the same event, or their songs touch on similar themes. Linked data facilitates faceted searching and displaying of information\cite{Oren2006}: an artist may be similar to many other artists in one of the ways just mentioned, and to a completely different plethora of artists in other senses, all of which might contribute to music discovery. Semantic web technologies can help us gather different facets of data and shape them into representations of knowledge. MusicWeb does this by searching similarities in three different domains: socio-cultural, research and journalistic literature and content-based linkage.
\subsection{Socio-cultural linkage}

Socio-cultural connections between artists in MusicWeb are primarily derived from YAGO categories that are incorporated into entities in DBpedia. Many categories, in particular those that can be considered extra-musical or tangential to music, that emerge as users browse MusicWeb, stem from the particular methodology used to derive YAGO information from Wikipedia. While DBpedia extracts knowledge from the same source, YAGO leverages Wikipedia category pages to link entities without adapting the Wikipedia taxonomy of these categories. The hierarchy is created by adapting the Wikipedia categories to the WordNet concept structure. This enables linking each artist to other similar artists by various commonalities such as style, geographical location, instrumentation, record label as well as more obscure categories, for example, artists who have received the same award, have shared the same fate, or belonged to the same organisation or religion. YAGO categories can reveal connections between artists that traditional isolated music datasets would not be able to establish. For example, MusicWeb links Alice Coltrane to John McLaughlin as both artists are converts to Hinduism, or Ella Fitzgerald to Jerry Garcia because both belong to the category of American amputees, or further still, the guitarist of heavy metal band Pantera to South African singer and civil rights activist Miriam Makeba as both died on stage.


\subsection{Literature-based linking}
Often artists share a connection through literature topics. For example: a psychologist interested in self-image during adolescence might want to research the impact of artists like Miley Cyrus or Rihanna on young teenagers\cite{Lamb2013}. Or a historian researching class politics in the UK might write about The Sex Pistols and John Lennon\cite{Moliterno2012}. In order to extract these relations one must mine the data from texts using natural language processing. Our starting point is a large database of 100,000 artists. MusicWeb searches several sources and collects texts that mention each artist. It then carries out semantic analysis to identify connections between artists and higher-level topics. There are two main sources of texts:
\begin{enumerate}
\item Research articles. There are various web resources that allow querying their research literature databases. MusicWeb uses Mendeley\footnote{\url{http://dev.mendeley.com/}} and Elsevier\footnote{\url{http://dev.elsevier.com/}}. Both resources offer managed and largely curated data and search possibilities include keywords, authors and disciplines. Data comprehension varies, but most often it features an array of keywords, an abstract, readership categorised according to discipline and sometimes the article itself.
  \item Online publications, such as newspapers, music magazines and blogs focused on music. This is non-managed, non-curated data, it must be extracted from the body of the text. The data is accessed after having crawled websites searching for keywords or tags in the title, and then scraped. External links contained in the page are also followed.
\end{enumerate}
Many texts contain references to an artist name without actually being relevant to MusicWeb. A search for Madonna, for example, can yield many results from the fields of sculpture, art history or religion studies. The first step is to model the relevance of the text, and discard texts which are of no interest to music discovery. This is done through a two stage process:
\begin{enumerate}
\item Managed articles contain information about the number of readers per discipline. This data is analysed and texts are discarded if the readers belong mainly to disciplines not related to humanities.
\item The text is projected onto a \emph{tfd/idf} vector space model\cite{Manning1999} constructed from words appearing in a relatively small collection of already accepted articles. Cosine similarity between this corpus and the text of every potential is computed, and texts which exceed a threshold are rejected.
\end{enumerate}
All items that pass these tests are stored as potential articles in the shape of the graph depicted in figure \ref{fig:article_graph}. Potential articles can always be reviewed and discarded as the corpus of articles grows and the similarity is recomputed.

\begin{figure}[!ht]
  \centering
  \includegraphics[scale=0.4]{graphics/article_graph.pdf}
  \caption{Article graph}
  \label{fig:article_graph}

\end{figure}
Texts (or abstracts, in the case of research publications where the body is not available) are subjected to semantic analysis. The text as a bag of words is used to query the alchemy\footnote{AlchemyAPI is used under license from IBM Watson.} language analysis service for:
\begin{itemize}
\item Named entity recognition. The entity recogniser provides a list of names that appear mentioned in the text together with a measure of relevance. They can include toponyms, institutions, publications and persons. MusicWeb is interested in identifying artists, so every person mentioned is checked against the database. If the person is not included in MusicWeb's database then three resources are checked: dbpedia, musicbrainz and freebase. All three resources identify musicians using YAGO. It is important to align the artist properly, since the modeling process is largely unsupervised, and wrong identifications can skew the model. Musicians identified in texts are stored and linked to the article as \textbf{props:in\_article}.
\item Keyword extraction. Non-managed texts and research papers that don't include tags or keywords. Keywords are checked against wordnet for hypernyms and stored. Artists that share keywords or hypernyms are considered to be relevant to the same topic in the literature.
\end{itemize}
MusicWeb also offers links between artists who appear in different articles by the same author, as well as in the same journal.



\subsection{Content-based linking}\label{sec:mir}

Content-based Music Information Retrieval (MIR) \cite{casey08} facilitates applications that rely on perceptual, statistical, semantic or musical features derived from audio using digital signal processing and machine learning methods. These features may include statistical aggregates computed from time-frequency representations extracted over short time windows. For instance, spectral centroid is said to correlate with the perceived brightness of a sound \cite{Schubert:06}, therefore it may be used in the characterisation in timbral similarity between music pieces. More complex representations include features that are extracted using a perceptually motivated algorithm. Mel-Frequency Cepstral Coefficients (MFCCs) for instance are often used in speech recognition as well as in estimating music similarity \cite{logan2000mel}. Higher-level musical features include keys, chords, tempo, rhythm, as well as semantic features like genre or mood, with specific algorithms to extract this information from audio.
%
Content-based features are increasingly used in music recommendation systems to overcome issues such as infrequent access of lesser known pieces in large music catalogues (the ``long tail'' problem) or the difficulty of recommending new pieces without user ratings in systems that employ collaborative filtering (``cold start'' problem) \cite{Celma2010}.

In this work, we are interested in supporting music discovery by facilitating a user to engage in interesting journeys through the ``space of music artists''. Although similar to recommendation, this is in contrast with most recommender systems which operate on the level of individual music items. We aim at creating links between artists based on stylistic elements of their music derived from a collection of recordings and complement the social and cultural links discussed in the previous sections.

High-level stylistic descriptors are not easily estimated from audio but they can correlate with lower level features such as the average tempo of a track, the frequency of note onsets, the most commonly occurring keys or chords or the overall spectral envelope that characterises dominant voices or instrumentation. To exploit different types of similarity, we model each artist using three main categories of audio descriptors: rhythmic, harmonic and timbral. We compute the joint distribution of several low-level features in each category over a large collection of tracks from each artist. We then link artists exhibiting similar distributions of these features.

% for XXXX artists with a mean track count of YYY
We obtain audio features form the AcousticBrainz\footnote{https://acousticbrainz.org/} Web service which provides descriptors in each category of interest. Tracks are indexed by MusicBrainz identifiers enabling unambiguous linking to artists and other relevant metadata. For each artist in our database, we retrieve features for a large collection of their tracks in the above categories, including beats-per-minute and onset rate (rhythmic), chord histograms (harmonic) and MFCC (timbral) features.

For each artist, we fit a Gaussian Mixture Model (GMM) with full covariances on each set of aggregated features in each category across several tracks and compute pair-wise distances $D_{cat}$ for the selected category using Eq.\ref{eq:dist}
%
\begin{equation}\label{eq:dist}
D_{cat} = d_{skl}(artist\_model_{cat}(i), artist\_model_{cat}(j)),
\end{equation}
%
where $d_{skl}$ is the symmetrised Kullback-Leibler divergence obtained using a Variational Bayes approximation for mixture models\cite{hershey:07} and $artist\_model_{cat}(i)$ are the model parameters (GMM weighs, means and covariance matrices) for artist $i$ in the selected category. The divergences for each artist are then ranked and identifiers of the top $N$ closest artists are stored.


\subsection{Similarity by mood}

While automatic feature extraction has significantly enhanced organisation and categorisation of large music collections, it is still rather challenging to derive high level semantic information relating to mood or genre. Complementing signal processing and machine learning methods with crowd-sourced social tagging data from platforms like Last.fm can enrich and inform understanding of general listening habits and connections between artists.
Mood-based similarity is another experimental enhancement to MusicWeb. This method involves using the Semantic Web version of ILM10K music mood dataset that consists of over 4000 unique artists. The dataset is based on crowd-sourced mood tag statistics from Last.fm users, which have been transformed to numerical coordinates in a Cartesian space. Every track in the collection is associated to 2-dimensional coordinates reflecting energy and pleasantness respectively. The similarity between artists in this case is measured by first calculating the location of the target artist in the mood space by averaging the coordinates of all the associated tracks. The same procedure is repeated for all other artists which then enables computing Manhattan distances between the target from the rest and using the ranking as similarity metric. This process is illustrated by the example SPARQL query in Listing \ref{lst:sparql1}.


\section{Discussion}\label{sec:discussion}
Interacting with MusicWeb can be a surprising experience. Often, the artists visited are similar enough. It is not unexpected that Rihanna and Chris Brown are linked because they are both mentioned in the same news item. Or, for instance, Schumman, Von Weber and Berlioz are all identified in the same musicology paper. It often happens, however, that the user begins by searching an artist and, following some of the links offered, ends up listening to a completely different style of music. One such journey, for example, started with Madonna. By following links only suggested by the fact that both artists appear in the same text the user is directed to Theodor W. Adorno, and then to Gustav Mahler, and finally to Pierre Boulez. Taking two steps back, from Adorno the user can choose to visit Niezstche's page, both being German composers who are also philosophers, or to Albert Einstein's, since both are Jewish people who migrated to the U.S. fleeing Nazi Germany. One final example of following semantic links: a user who searches for Bach can then go to Mozart's page. From there he can proceed to Glenn Gould's, on to Thelonius Monk and finally John Coltrane.
This kind of discovery path would be very unlikely in any recommender system based on user profile data.

\vspace{-10pt}
\noindent\begin{minipage}[!ht]{\textwidth}
	\begin{lstlisting}[ style = sparql, label=lst:sparql1, tabsize=4,  caption={example SPARQL query to retrieve similar artists from the ILM10K mood dataset by Manhattan distance of valence-arousal coordinates.} ]
SELECT ?artist ?mbid
WHERE
{
	SELECT ?artist ?mbid (AVG(?valence) as ?avg_valence) (AVG(?arousal) as ?avg_arousal) ((ABS(AVG(?target_valence)-AVG(?valence)) + ABS(AVG(?target_arousal)-AVG(?arousal))) as ?diff)
	WHERE {
		{
			SELECT ?target_valence ?target_arousal
			WHERE {
				?target_coords mood:valence ?target_valence ;
					mood:arousal ?target_arousal ;
					mood:configuration mood:actfold4 .
				?target_lfmid mood:coordinates ?target_coords ;
					mood:artist_name ?target_artist .
				FILTER(?target_artist = "Roots Manuva")
			}
		}
		?coords mood:valence ?valence ;
			mood:arousal ?arousal ;
			mood:configuration mood:actfold4 .
		?lfmid mood:coordinates ?coords ;
			foaf:maker ?maker ;
			mood:artist_name ?artist .
		?maker mo:musicbrainz_guid ?mbid .
		FILTER(?artist != "Roots Manuva")
	} GROUP BY ?artist ?mbid ORDER BY ?diff
} LIMIT 20
	\end{lstlisting}
\end{minipage}


% \section{Conclusions}\label{sec:conclusions}

\section{Conclusions and Future work}\label{sec:conclusions}

% \subsection{Future work}
MusicWeb is an emerging application being developed at a research centre to explore the possibilities of linked data-based music discovery and tested with a small user base of student and staff. % As such, its user base is at the moment limited to member of the research group (roughly 30 people, students and staff). Hence there are no major technical infrastructure requirements which cannot be supported by the university. This does not mean that we do not intend to utilise very large datasets. It was conceived and it is being developed mainly as a research tool.
% actually C4DM is over 70 people!!
Our aim is to gather in one application various different approaches to music discovery and how they can benefit from linked music metadata. Our next steps are directed toward evaluating its potential acceptance by end users. It would be of great value to us to find out which linking methods listeners find most appealing or interesting, and which they would use more often.
As to the different methods of linking music metadata, our next steps will be:
\begin{itemize}
\item In literature-based linking:
\begin{enumerate}
\item To reinforce the model to filter texts. We want to explore different possibilities to make the selection of texts robuster and more reliable.
\item To investigate methods for reliable abstract concept identification. The use of valuable metadata such as discipline, journal or keywords offers the possibility of clustering topics under a hierarchy of abstract concepts.
\item To research the application of graph distance in artist similarity.
\end{enumerate}

\item In content-based linking:
\begin{enumerate}
\item Include more feature types
\item investigate correlation between linking categories (e.g. would content based similarity be correlated with cultural similarity)?
\item  model albums separately (e.g. many artists cross genres over a long career, does this have an effect influencing content based linking?)
\end{enumerate}
\end{itemize}

    \bibliographystyle{plain}
    \bibliography{musicweb}

\end{document}