Daniel@0: function demo4segmentation Daniel@0: % To get familiar with some approaches of segmentation of audio files Daniel@0: % using MIRtoolbox. Daniel@0: Daniel@0: % 1. Load an audio file (for instance, guitar.wav). Daniel@0: a = miraudio('guitar'); Daniel@0: Daniel@0: % 2. We will perform the segmentation strategy as proposed in (Foote & Daniel@0: % Cooper, 2003). First, decompose the file into successive frames of 50 ms Daniel@0: % without overlap. Daniel@0: help mirframe Daniel@0: fr = mirframe(a,0.05,1) Daniel@0: Daniel@0: % 3. Compute the spectrum representation (FFT) of the frames. Daniel@0: sp = mirspectrum(fr) Daniel@0: clear fr Daniel@0: % (Remove from the memory any data that will not be used any more.) Daniel@0: Daniel@0: % 4. Compute the similarity matrix that shows the similarity between the Daniel@0: % spectrum of different frames. Daniel@0: help mirsimatrix Daniel@0: sm = mirsimatrix(sp) Daniel@0: clear sp Daniel@0: % Look at the structures shown in the matrix and find the relation with the Daniel@0: % structure heard when listening to the extract. Daniel@0: Daniel@0: % 5. Estimate the novelty score related to the similarity matrix. It Daniel@0: % consists in a convolution of the diagonal of the matrix with a Daniel@0: % checker-board Gaussian kernel. Use the novelty function for that purpose. Daniel@0: help mirnovelty Daniel@0: nv = mirnovelty(sm) Daniel@0: Daniel@0: % 6. Detect the peaks in the novelty score. Daniel@0: help mirpeaks Daniel@0: p1 = mirpeaks(nv) Daniel@0: Daniel@0: % You can change the threshold value of the peak picker function in order to Daniel@0: % get better results. Daniel@0: p2 = mirpeaks(nv,'Contrast',0.01) Daniel@0: Daniel@0: clear nv Daniel@0: Daniel@0: % 7. Segment the original audio file using the peaks as position for Daniel@0: % segmentation. Daniel@0: help mirsegment Daniel@0: s1 = mirsegment(a,p1) Daniel@0: clear p1 Daniel@0: Daniel@0: % 8. Listen to the results. Daniel@0: mirplay(s1) Daniel@0: Daniel@0: %s2 = mirsegment(a,p2) Daniel@0: %clear p2 Daniel@0: %mirplay(s2) Daniel@0: Daniel@0: % 9. Compute the similarity matrix of this obtained segmentation, in order Daniel@0: % to view the relationships between the different segments and their Daniel@0: % possible clustering into higher-level groups. Daniel@0: mirsimatrix(s1,'Similarity') Daniel@0: clear s1 Daniel@0: %mirsimatrix(s2) Daniel@0: %clear s2 Daniel@0: Daniel@0: display('Strike any key to continue...'); Daniel@0: pause Daniel@0: close all Daniel@0: Daniel@0: % 10. Change the size of the kernel used in the novelty function, in order Daniel@0: % to obtain segmentations of different levels of detail, from detailed Daniel@0: % analysis of the local texture, to very simple segmentation of the whole Daniel@0: % piece. Daniel@0: n100 = mirnovelty(sm,'KernelSize',100) Daniel@0: n50 = mirnovelty(sm,'KernelSize',50) Daniel@0: n10 = mirnovelty(sm,'KernelSize',10) Daniel@0: clear sm Daniel@0: % As you can see, the smaller the gaussian kernel is, the more peaks can be Daniel@0: % found in the novelty score. Indeed, if the kernel is small, the cumulative Daniel@0: % multiplication of its elements with the superposed elements in the Daniel@0: % similarity matrix may vary more easily, throughout the progressive Daniel@0: % sliding of the kernel along the diagonal of the similarity matrix, and Daniel@0: % local change of texture may be more easily detected. On the contrary, Daniel@0: % when the kernel is large, only large-scale change of texture are Daniel@0: % detected. Daniel@0: Daniel@0: display('Strike any key to continue...'); Daniel@0: pause Daniel@0: close all Daniel@0: Daniel@0: p100 = mirpeaks(n100,'NoBegin','NoEnd') Daniel@0: clear n100 Daniel@0: p50 = mirpeaks(n50,'NoBegin','NoEnd') Daniel@0: clear n50 Daniel@0: p10 = mirpeaks(n10,'NoBegin','NoEnd') Daniel@0: clear n10 Daniel@0: s100 = mirsegment(a,p100) Daniel@0: clear p100 Daniel@0: mirplay(s100) Daniel@0: clear s100 Daniel@0: s50 = mirsegment(a,p50) Daniel@0: clear p50 Daniel@0: mirplay(s50) Daniel@0: clear s50 Daniel@0: s10 = mirsegment(a,p10) Daniel@0: clear p10 Daniel@0: mirplay(s10) Daniel@0: clear s10 Daniel@0: Daniel@0: display('Strike any key to continue...'); Daniel@0: pause Daniel@0: close all Daniel@0: Daniel@0: % One more compact way of writing these commands is as follows: Daniel@0: mirsegment(a,'Novelty') Daniel@0: mirsegment(a,'Novelty','Contrast',0.01) Daniel@0: mirsegment(a,'Novelty','KernelSize',100) Daniel@0: Daniel@0: display('Strike any key to continue...'); Daniel@0: pause Daniel@0: close all Daniel@0: Daniel@0: % Besides, if you want to see the novelty curve with the peaks, just add a Daniel@0: % second output: Daniel@0: [s50 p50] = mirsegment(a,'Novelty','KernelSize',50) Daniel@0: clear s50 p50 Daniel@0: [s10 p10] = mirsegment(a,'Novelty','KernelSize',10) Daniel@0: clear a s10 p10 Daniel@0: Daniel@0: display('Strike any key to continue...'); Daniel@0: pause Daniel@0: close all Daniel@0: Daniel@0: Daniel@0: % 11. Try the whole process with MFCC instead of spectrum analysis. Take the Daniel@0: % first ten MFCC for instance. Daniel@0: help mirsegment Daniel@0: % The segment function can simply be called as follows: Daniel@0: sc = mirsegment('czardas','Novelty','MFCC','Rank',1:10) Daniel@0: clear sc Daniel@0: Daniel@0: % Here are some other examples of use: Daniel@0: [ssp p m b] = mirsegment('valse_triste_happy','Spectrum',... Daniel@0: 'KernelSize',150,'Contrast',.1) Daniel@0: clear p m b Daniel@0: mirplay(ssp) Daniel@0: clear ssp Daniel@0: Daniel@0: display('Strike any key to continue...'); Daniel@0: pause Daniel@0: close all Daniel@0: Daniel@0: [smfcc2 p m a] = mirsegment('valse_triste_happy','MFCC',2:10,... Daniel@0: 'KernelSize',150,'Contrast',.1) Daniel@0: clear p m a Daniel@0: mirplay(smfcc2)