Structured sparsity for automatic music transcription

Transcription

1 Structured sparsity for automatic music transcription O'Hanon, K; Nagano, H; Pumbey, MD; Internationa Conference on Acoustics, Speech and Signa Processing (ICASSP 2012) For additiona information about this pubication cick this ink. Information about this research object was correct at the time of downoad; we occasionay make corrections to records, pease therefore check the pubished record when citing. For more information contact

2 STRUCTURED SPARSITY FOR AUTOMATIC MUSIC TRANSCRIPTION Ken O Hanon Hidehisa Nagano Mark D. Pumbey Queen Mary University of London Centre for Digita Music NTT Communication Science Laboratories NTT Corporation ABSTRACT Sparse representations have previousy been appied to the automatic music transcription (AMT) probem. Structured sparsity, such as group and moecuar sparsity aows the introduction of prior knowedge to sparse representations. Moecuar sparsity has previousy been proposed for AMT, however the use of greedy group sparsity has not previousy been proposed for this probem. We propose a greedy sparse pursuit based on nearest subspace cassification for groups with coherent bocks, based in a non-negative framework, and appy this to AMT. Further to this, we propose an enhanced moecuar variant of this group sparse agorithm and demonstrate the effectiveness of this approach. Index Terms Transcription, non-negative, structured sparsity 1. INTRODUCTION Sparse coding is the attempt to derive a representation vector, t, predominated by zero coefficients, or sparse, of a signa s from a given dictionary D, where s Dt. In this work, we consider sparse approximation in a non-negative framework. Formay stated, for the noisy case, the non-negative sparse representation probem seeks the minimisation: 1 min t 2 s Dt λ t 0 s.t. t 0. (1) Many agorithms have been proposed for performing sparse approximation, the most popuar famiies of which are greedy methods, which typicay add the atom most correated with the residua signa at each iteration, and goba optimisation methods, which reax the 0 norm sparse penaty for a 1 norm sparse penaty, aowing the probem to be soved with convex optimisation methods. Structured sparsity aows the introduction of prior knowedge to sparse approximation agorithms. This knowedge can be genera as in bock, or group sparsity [1][2], where the assumption is made that groups of atoms tend to be supported coincidentay, or appication specific, such as moecuar pursuits [3] which have been appied to audio signas seecting time-reated atoms at each iteration. Musica signas can be seen as inherenty sparse in that ony a sma subset of notes are seen to be active at a given This research is supported by ESPRC Leadership Feowship EP/G007144/1 and EU FET-Open Project FP7-ICT SMALL time. Automatic music transcription is the attempt to produce a mid-eve pitch-time representation, often caed a piano ro, which reates the note events in a musica piece. Resarch in AMT is often deineated into methods which use either offine training or onine training. For instance, a state-of-the art onine method using Bayesian NMF with time-smoothness and harmonic constraints is proposed in [4]. There exists severa works in the iterature which aim to use sparse representation methods with dictionaries earnt offine to tacke the AMT probem. For exampe, Abdaah and Pumbey [5] used a non-negative sparse coding agorithm based on a Bayesian formuation of sparsity to produce a piano ro using a dictionary of fu-spectrum basis atoms. Leveau et a. [6] proposed using Matching Pursuit (MP) with instrument and note specific dictionaries of harmonic atoms for instrument recognition, and aso used a moecuar representation for poyphonic transcription. We are currenty deveoping a non-negative sparse dictionary earning based AMT system, and the work presented here considers the sparse approximation step of the proposed system. It was observed in [5] that transcription is improved using severa atoms per note, better capturing the spectra dynamics of a note. To this end we propose using group sparsity, with groups of pitch-reated atoms. A variant of Non- Negative Basis Pursuit (NN-BP) which outputs group coefficients is considered. We propose a non-negative group greedy pursuit taiored for the coherent groups we use, which we term Non-negative Nearest Subspace OMP (NN-NS-OMP). We aso propose a moecuar version of NN-NS-OMP, promoting time persistence in the sparse representation. We demonstrate improved transcription performance using these structured approaches on a dataset of poyphonic piano pieces. 2. BACKGROUND 2.1. Non-negative sparse representations In [7], Hoyer proposed a non-negative sparse coding (NNSC) agorithm, based on a mutipicative update, [D T S] k,n t k,n t k,n [D T DT] k,n + λ which seeks to minimise the cost function min T 1 2 S DT 2 F + λ T 1,1 s.t. T 0 (2) which is seen to be simiar to the 1 reaxation of the sparse representation probem with an added non-negativity con-

3 straint. In [8], NN-BP was proposed, using severa iterations of NNSC, foowed by a hard threshoding. A non-negative variant of OMP (NN-OMP), outined in Agorithm 1 is proposed in [9] which differs from OMP by constraining the atom seection to positive coefficients, and by using non-negative east squares(nnls) to cacuate the coefficients of the supported atoms at each iteration. Agorithm 1 NN-OMP [9] + ; s R M + Initiaise i = 0; r 0 = S; t 0 = 0; Γ 0 = {}; ˆk = arg max k d k, r i 1 Γ i = Γ i 1 ˆk t Γ i = min z D Γ i z s 2 2; z 0 r i = s Dt i 2.2. Structured Sparsity Group sparsity assumes that groups of atoms tend to be active at the same time. Usuay it is assumed that the groups are of the same size and adjacent in a given dictionary, D R M K with K = L P where L is the number of groups and P is the amount of atoms per group, aowing us to define the set of group indices: G = {G G = {P ( 1) + 1,..., P } {1,..., L}}. Severa variants of OMP, such as Bock-OMP (B-OMP) [1] and Subspace Matching Pursuit (SMP) [2] have been proposed which incorporate the group assumption. These agorithms differ from OMP by using group seection criteria and by adding a members of a eected group to the sparse support at each iteration. In B-OMP the group seection criterion is given by ˆ = arg max D T G r i 1 2 (3) whie the SMP seection criterion is given by ˆ = arg min r i 1 π (r i 1 ) 2 (4) where π (y) is the projection operator of the signa y onto the subspace D G. Daudet [3] proposed Moecuar Matching Pursuit (MMP) for separation of transient, tona and noise eements of an audio signa. A modified matching pursuit seects transient moecues consisting of waveet trees, or tona moecues consisting of time-persisting fourier atoms. To isoate spurious noise in the Fourier domain, tona atoms were seected from a time-smoothed spectrogram with coefficents based on the vaues of severa time reated atoms. When a tona atom is seected, a moecue is grown by tracking in both time directions unti the coefficient vaues in the origina spectrogram disappeared beow a threshod. Subsequenty a atoms in the moecue are added to the sparse support Dictionary Training 3. METHOD We use a simpe dictionary training method, earning subdictionaries each of size P atoms using NMF[10] on STFTs of isoated notes from the MAPS [11] database. We earn a subdictionary for each of L = 88 notes corresponding to MIDI notes 21 to 108, composing our bock-based dictionaries from the note-specific subdictionaries. An aternative viewpoint is to regard each subdictionary as a subspace and the dictionary as a union of subspaces Non-negative structured sparse approximation We modify sighty the NN-BP agorithm, imposing group structure post-hoc by deriving a group coefficient matrix, GC R M L by summing the group coefficients. A hard threshod is then determined by a parameter δ and the argest entry in GC, to which it is then appied eementwise. Agorithm 2 NN-BP(GC) D R M K +, S R M N +, δ, T 0 = D T S [D T S] k,n t k,n t k,n [D T DT] k,n + λ GC,n = T G,n (, n) GC,n = 0 {, n } s.t. GC,n < δ GC, Our subdictionaries tend to dispay high inner-group coherence, or sub-coherence [1] due to both non-negativity and harmonic reationship. For this reason, the B-OMP [1] seection criteria (3) is unsuitabe for this probem. Due to the ack of a expicit projection operator, such as π(y) in (4), in the non-negative framework used in these experiments the SMP [2] seection criteria (4) is aso unsuitabe. We propose a simiar greedy group sparse agorithm, NN-NS-OMP, outined in Agorithm 3 which seeks the group containing the nearest subspace with non-negative coefficients at each iteration. Agorithm 3 NN-NS-OMP +, s R M + Initiaise Γ = {} r 0 = s i = 0 ˆ = arg min r i D G Θ 2 2 s.t. Θ 0 = {1,..., L} Γ i = Γ i 1 D Gˆ t Γi = arg min z s D Γi z 2 2 r i = s Dt i We aso propose a moecuar version of this agorithm, M-NN-NS-OMP, outined in Agorithm 4, which accepts an input binary atom support, Γ derived from GC output from NN-BP(GC). The reasons for inputting Γ are twofod; affording cheaper computation, as we need ony cacuate NNLS for atoms supported in Γ at each iteration, instead of a atoms

4 as in NN-NS-OMP. Aso we no onger rey on a threshod, as in MMP, to stop growing the moecue, instead using discontinuities in Γ to deineate the endpoints of a moecue. This agorithm perfoms sparse approximation on the matrix as a whoe, in which it differs from NN-NS-OMP which operates on vectors. In this case we do not seect a nearest subspace per se, but derive a group coefficient Θ from the sum of the NNLS coefficient vector x for the group at each time bin supported in Γ. Simiar to tona eements in MMP[3], a smoothed coeffcient matrix Θ is derived with a rectanguar window, of size α. We seect the argest atom coefficient from Θ, from which we grow a moecue using Γ to define the endpoints, as mentioned above. Agorithm 4 M-NN-NS-OMP +, S R M N +, Γ {0, 1} L N, G, α Initiaise i = 0; Φ = 0 L N ; B = {β n β n = {} n {1,.., N}} Get group coefs Θ and smoothed coefs Θ x G,n = arg min x r i n D G x 2 2 s.t. x 0 Γ n n+α 1 Θ,n = x G,n ; Θ,n = Θ,n /α n =n Seect initia atom and grow moecue {ˆ, ˆn} = arg max,n Θ,n n min = min n s.t. Γˆ,Ξ = 1, Ξ = { n,..., ˆn} n max = max n s.t. Γˆ,Ξ = 1, Ξ = {ˆn,..., n} β n = β n ˆ n Ξ = {n min,..., n max} Cacuate current coefficients and residua t Gβn,n = min t s n D Gβn t 2 2 n Ξ r i+1 n = s n D Gβn t Gβn n Ξ 4. EXPERIMENTS We perform transcription on a subset of MAPS [11], a database of midi-aigned piano pieces, aso used in [4]. The database contains many sets of piano pieces, most recorded using high quaity sampes, and a pieces come with a standardised ground truth. The subset we used was recorded ive on a Diskavier, and, simiar to [4] we use the first thirty seconds of each piece. We downsamped each piece in the subset to 22.05kHz, and used the STFT with a window size of 90ms with a 50% overap to produce a magnitude spectrogram. MAPS aso contains sampes of isoated chords and notes, from which we trained our subdictionaries. We trained dictionaries for various sizes of P = 1 to 5 in order to investigate the possibe effect on performance of group size. We note that when P = 1, the agorithms revert to non-group sparse impementations. We performed sparse approximation on the spectrograms using NN-BP(GC), NN-NS-OMP and M-NN-NS-OMP. We set NN-BP(GC) to run for 100 iterations. For NN-NS-OMP, we set three stopping conditions; at each time bin we aowed no more than 7 atoms to be seected; once the residua error norm dropped beow 5% of the origina signa norm; we aso Onset based Frame based Acc Rec F Acc Rec F NN-BP NN-BP (GC) NN-OMP NN-NS-OMP M-OMP M-NN-NS-OMP Marot[13] B-NMF[4] Tabe 1. Transcription resuts comparing group sparse methods (P = 3) with non-group sparse methods. Resuts for Marot method and B-NMF from [4] shown for comparision stopped when adding a new atom woud reduce the residua norm by ess than 2% of the origina signa norm. For the NN-BP and NN-NS-OMP, we perform some post-processing on GC. We assume a minimum note ength of 3 time bins and threshod out atoms which are continuousy supported in the time domain for a esser duration. We stopped the M-NN-NS-OMP, when Θ, dropped beow a threshod of 2, and a persistence factor α = 5 was used. We used both onset-based and a frame based measures to compare the performance of the agorithms. We detect onsets using a simpe threshod-based method simiar to [4]. An onset is detected when a threshod is exceeded and subsequenty sustained for a minimum duration of three time bins. We set the threshod to an ampitude of 5.5, noting that better resuts can be obtained with other threshods with respect to P. We register a true positive, tp when we find an onset for a note within 1 time bin of the ground truth, as ascertained from the aigned onset fies suppied in MAPS. A fase positive, fp is registered when the system detects an onset which is not present in the ground truth. Fase negatives, fn are recorded for each note in the ground truth which is not detected by the system. We aso record the frame-based performance, in which we compare the midi ground truth with the output from the sparse agorithms at each frame. Here we register a tp when a point in the time-frequency domain is supported by the ground truth and the transcription resuts. Simiary, points supported ony in the ground truth, and transcription resuts register tn and fn respectivey. The foowing metrics are then used to measure performance; accuracy Acc = tp 100 tp+fp the F -measure F = 2 (Acc Rec) Acc+Rec. 5. RESULTS %; reca Rec = tp 100 tp+fn % and In Tabe 1, we compare resuts for the group methods against their corresponding non-group methods. We note that the moecuar methods show the best resuts for F -vaue for both types of transcription, and that M-NN-NS-OMP improves upon the M-NN-OMP. NN-BP(GC) performs worse with group coefficients for a measures except reca, with very poor accuracy for frame based metrics. This can be expained by the fact that the agorithm does not expoit the group structure in the mutipicative update, but uses a post hoc summation of group

5 P BP(GC) NN-NS-OMP M-NN-NS-OMP 2 5 Acc Rec F Acc Rec F Tabe 2. Onset based metrics for group methods (abbreviated names) reative to group size P. P BP(GC) NN-NS-OMP M-NN-NS-OMP 2 5 Acc Rec F Acc Rec F Tabe 3. Frame Onset based metrics for group methods (abbreviated names) reative to group size P. coefficients. However the reca is high, which prompts good resuts from the moecuar approaches which input Γ from NN-BP(GC). In this way the M-NN-NS-OMP can be seen as a denoising step on the NN-BP(GC), capturing the saient parts of the signa. The NN-NS-OMP shows sighty poorer performance in the onset detection task, but an improvement in the frame based detection. Informa experiments suggest that resuts are superior to other group versions of OMP we have tried, but that is not described here. Finay a comparision is made with the resuts described for onset detection in [4]. Our resuts compare favouraby with the neura network-based method proposed by Marot in [13]. The resuts for the Bayesian NMF method are state-ofthe-art for a signa decomposition based method, and direct comparision with offine-earning based methods is unfair, except to show the reative difficuty of the probems. In Tabes 2 & 3, we show resuts for different vaues of P, which indicate the effects of group size on performance. In particuar we note the increase in amost a metrics for the moecuar approach with group size. 6. CONCLUSIONS AND FURTHER WORK We have a presented a structured non-negative sparse music transcription system, with promising resuts, demonstrating that group and moecuar sparsity may enhance transcription performance. We intend to test the proposed method with datasets and metrics used by other researchers in AMT to aow a fu comparision to be made. 7. REFERENCES [1] Y.C. Edar et a., Bock-sparse signas: Uncertainty reations and efficient recovery, IEEE Trans. SP, vo. 58, pp , [2] A. Ganesh et a., Separation of a subspace=sparse signa: Agorithms and conditions, in Proc. ICASSP Fig. 1. Transcriptions resuts for with denoted agorithms and ground truth from 5 second excerpt, with P = 3. Detected onsets cooured in back (extended forward in time). [3] L. Daudet, Sparse and structured decompositions of signas with the moecuar matching pursuit, IEEE Trans. ASLP, pp , [4] N. Bertin et a., Enforcing Harmonicity and Smoothness in Bayesian Non-negative Matrix Factorization Appied to Poyphonic Music Transcription, IEEE Trans. ASLP, vo. 18, pp , [5] S.A. Abdaah and M.D. Pumbey, Poyphonic transcription by non-negative sparse coding of power spectra, in Proc. ISMIR 2004, 2004, pp [6] P. Leveau et a., Instrument-specific harmonic atoms for mid-eve music representation, IEEE Trans. ASLP, pp , [7] P.O. Hoyer, Non-negative sparse coding, Proc. NNSP 2002, pp , [8] M. Aharon et a., K-svd and its non-negative variant for dictionary design, in Proc. of the SPIE conference waveets, 2005, pp [9] A.M. Bruckstein et a., On the uniqueness of nonnegative sparse soutions to underdetermined systems of equations, IEEE Trans. IT, vo. 54, pp , [10] D.D. Lee and H.S. Seung, Agorithms for non-negative matrix factorization, Proc. NIPS [11] V. Emiya et a., Mutipitch estimation of piano sounds using a new probabiistic spectra smoothness principe, IEEE Trans. ASLP, vo. 18, pp , Aug [12] E. Oja, Subspace Methods of Pattern Recognition, Res. Studies Press, [13] M. Marot, A connectionist approach to automatic transcription of poyphonic piano music, IEEE Trans on Mutimedia, pp , 2004.