Grading learning for blind source separation

Transcription

1 Vol. 46 No. 1 SCIENCE IN CHINA (Seres F) February 2003 Gradg learg for bld source separato ZHANG Xada ( ) 1,2, ZHU Xaolog (r ) 1 & BAO Zheg ( ) 1 1. Key Laboratory for Radar Sgal Processg, Xda Uversty, X a , Cha; 2. Departmet of Automato, State Key Laboratory of Itellget echology ad Systems, sghua Uversty, Bejg , Cha; Correspodece should be addressed to Zhag Xada (emal: zxd-dau@mal.tsghua.edu.c) Receved May 31, 2002 Abstract By geeralzg the learg rate parameter to a learg rate matrx, ths paper proposes a gradg learg algorthm for bld source separato. he whole learg process s dvded to three stages: tal stage, capturg stage ad tracg stage. I dfferet stages, dfferet learg rates are used for each output compoet, whch s determed by ts depedecy o other output compoets. It s show that the gradg learg algorthm s equvarat ad ca eep the separatg matrx from becomg sgular. Smulatos show that the proposed algorthm ca acheve faster covergece, better steady-state performace ad hgher umercal robustess, as compared wth the exstg algorthms usg fxed, tme-descedg ad adaptve learg rates. Keywords: bld source separato, depedet compoet aalyss, eural computato, adaptve learg. Bld source separato (BSS) cossts of recoverg mutually statstcally depedet but otherwse uobserved source sgals from ther lear mxtures wthout owg the mxg coeffcets. BSS has draw a lot of atteto sgal processg ad commucatos sce t s a fudametal problem ecoutered may practcal applcatos. he BSS algorthms may be off-le or o-le. Fxed-pot algorthm of Hyvare [1] s off-le, ad wors most satsfactorly, but t s ot avalable for real-tme applcatos. Moreover, ts performace deterorates serously as the umber of sources creases [2]. he exstg o-le BSS methods ca be dvded to three major categores: 1) etropy maxmzato (EM) [3] ;2) olear prcpal compoet aalyss (PCA) [4,5] ; ad 3) depedet compoet aalyss (ICA) [6 15]. It s show (see, e.g. ref. [9]) that the ICA ad the EM are equvalet. I ths paper, we pay our atteto to the ICA for BSS. As a mastream techque for BSS, the ICA has may effcet algorthms, such as the atural gradet algorthms of Amar et al. [9,14], the EASI algorthm of Cardoso ad Laheld [8],the exteded ICA algorthm [11], the flexble ICA algorthm [12], the teratve verso ICA algorthm [13] ad so o. Although the adaptve ICA algorthms have may dfferet forms, most of whch belog to the LMS-type algorthms, thus oe requres a careful choce of the learg rate to obta better performace. Whe the learg rate or step taes a costat, there s a coflct betwee covergece speed

2 32 SCIENCE IN CHINA (Seres F) Vol. 46 ad steady-state performace ay LMS-type algorthm. A small learg rate s useful for steady-state performace, but t leads to slow covergece. Cotrarly, the LMS algorthm may become ustable f the learg rate s too large. o overcome ths problem, a smple way s to decrease the learg rate [10,14,15] wth tme, but t would cause the ew problem: f the learg rate becomes very small whe some source sgal s ot yet separated wth others, the t may eed a very log tme for ths sgal to be separated, ad eve the sgal ca ever be separated. Aother alteratve s to use the adaptve learg rates, such as auxlary varable based adaptve step learg algorthms [16 19], ad gradet adaptve step sze algorthm [20]. o the best of our owledge, however, oe s sgal-adaptve amog the exstg learg rate rules. I ths paper, we propose a ovel gradg learg algorthm for BSS, whch the learg rates are sgal-adaptve ad determed automatcally by the separateess of outputs of the separato system. Usg the gradg learg, the covergece ad tracg of the BSS algorthm ca be cotrolled automatcally. 1 Adaptve o-le algorthms for BSS I eural etwors, sgal processg ad statstcs, oe cosders the stadard lear data model x = As, = 1,2, (1) Fg. 1. Bld source separato va a lear eural etwor. where x deotes the avalable sesor vector at tme, ads s the source vector whose compoets, called the source sgals, are uow, mutually statstcally depedet, ad at most oe s allowed to be Gaussa. he md mxg matrx A s a uow costat matrx wth full ra. As show fg. 1, we use a lear eural etwor for BSS. he BSS problem s to separate the orgal sources from the mxtures. For ths purpose, oe usually adjusts the Dm separatg matrx W such that y = Wx (2) has depedet compoets, or the mutual formato amog y s mmzed. he mutual formato ca be measured by the Kullbac-Lebler dvergece betwee the jot probablty desty fucto p( y, W) ad ts factorzed verso N p ( yw, ) = p( y, W), amely p( yw, ) I( W) = D[ p( y, W) p ( y, W)] = p( y, W)log d y. (3) p ( yw, ) he mutual formato s oegatve,.e. I(W)ƒ0. By Como [6], I(W) s a cotrast fucto for the ICA, meag = 1

3 No. 1 GRADING LEARNING FOR BLIND SOURCE SEPARAION 33 I ( W ) = 0 ff WA = P, (4) where s a osgular dagoal matrx, ad P s a D permutato matrx. hs mples that the mutual formato s equal to zero f ad oly f y = Wx = Ps s a scaled ad/or permuted verso of the source sgals. he followg are several typcal ICA algorthms for BSS. he atural gradet algorthms [9 12,14,15,21] : + = + η φ W 1 W [ I ( y ) y ] W. (5) ItsshowbyAmar [15] that the atural gradet s a exteso of the geeral gradet the Rema space, whle the geeral gradet s a specal example of the atural gradet the uform Eucldea space. he EASI algorthm [8] : + = + η φ + φ W 1 W [ I ( y ) y y ( y ) y y ] W. (6) he teratve verso algorthm [13] : + = + η φ ψ W 1 W [ I ( y ) ( y )] W. (7) I Algorthms (5) (7), η s the learg rate, φ( y ) = [ φ ( y ( )),, φ ( y ( ))] 1 1 ψ( y) = [ ψ1( y1( )),, ψ( y( ))], where the elemets of colum vectors φ( y ) ad ψ ( y ) 2 have several choces, such as φ ( y ( )) = y ( ) C y ( ), ψ ( y ( )) = sg(re( y ( ))) + j sg (Im( y ( ))). It s easly see that the above three algorthms ca be wrtte as the followg uform form: W+ 1 = W + ηg( y) W. (8) I the BSS, ay effcet algorthm should have two ey propertes: the equvarat property ad the property of eepg the separatg matrx from becomg sgular. he equvarat property guaratees that the BSS algorthm ca offer uform performace, meag ts behavors are depedet of the mxg process, whle the osgular property s the premse that the BSS algorthm wors steadly. A BSS algorthm s equvarat f the combed mxg-separatg matrx C = WA satsfes [8] C+ 1 = C +ηf( Cs) C, (9) where FCs ( ) s a matrx fucto of Cs such that the mxg matrx A does ot eter to the evoluto of C except as a tal codto. It s show [8 10,15] that the EASI algorthm ad the atural gradet oes are equvarat ad have the osgular property of W. As a comparso, the BSS algorthms based o geeral gradet do ot have these two ey performaces. ad

4 34 SCIENCE IN CHINA (Seres F) Vol A gradg learg algorthm for BSS I ths secto, we develop a ovel gradg learg rule for adaptvely selectg the learg step η the BSS algorthm (8). 2.1 Measure of sgal depedece As dscussed the prevous secto, the goal of the BSS s to determe W such that the outputs y of the separato system are mutually depedet. It s well ow that f y s depedet of y j, the for ay measurable fuctos g 1 ad g 2, oe has [22] Eg { ( y) g( y)} Eg { ( y)} Eg { ( y)} = 0, j (10) 1 2 j 1 2 j whch mples g1( y ) ad g2 ( yj ) are ucorrelated wth each other. herefore, to uderstad whether y ad y j are mutually depedet or ot, we should justfy the valdess of eq. (10) for all measurable fuctos g 1 ad g 2, whch s ot oly mpossble but also uecessary practcal applcatos. As a matter of fact, t s suffcet to cosder the secod- ad hgher-order correlatos betwee the output compoets y ad y j.ithspaper,wedefe sc = cov[ y ( ), y ( )] cov[ y ( )]cov[ y ( )], j j j (11) hc = cov[ φ ( y ( )), y ( )] cov[ φ ( y ( ))]cov[ y ( )], j j j (12) where cov[ x ( ), y ( )] = E{[ x ( ) x][ y ( ) y]}, cov[ x ( )] = E{[ x ( ) x] } ad x = Ex {()}. Clearly, sc j represets the secod-order correlato coeffcet betwee y ad y j.sce φ (C) s olear, we refer to hc j as the ormalzed hgher-order correlato coeffcet betwee y ad y j. Note that hcj hc j sce hc j s the cross-correlato betwee φ( y( )) ad y j, whle hc j s that betwee φ ( y ( )) ad y. j j I the BSS, the separateess of a par of source sgals ca be measured by ther depedece. For ay two sgals, we have the followg basc observatos: Sgals y ad y j are of strog depedece f ay of the secod-order correlato coeffcet sc j, the hgher-order oes hc j ad hc j s large ; y ad y j are of wea depedece f all of the sc j, hc j ad hc j are smaller ; y ad y j are almost depedet f sc j, hc j ad hc j are all small eough. Defto 1. he maxmum absolute value of sc j, hc j ad hc j s defed as the measure of sgal depedece betwee y ad y j,.e. D ( ) = max{ sc ( ), hc ( ), hc ( ) }. (13) j j j j Clearly, ths defto cocdes wth the above three observatos. hat s to say, the sgals y ad y j may be sad to be strogly depedet f D j () s large (e.g D j () 1), wealy depedet f t s smaller (e.g D j () 0.25 ), ad almost depedet f suffcetly small 2

5 No. 1 GRADING LEARNING FOR BLIND SOURCE SEPARAION 35 (e.g. 0 D j () 0.05 ). Partcularly, D j () = 0 for two depedet sgals y ad y j,add j () =1 for two coheret sgals. Note that Defto 1 assures D j () =D j (), whch s a expected characterstc sce the depedece of y o y j should be detcal to that of y j o y. Moreover, Defto 1 s avalable as well for two pre-whteed sgals, such a case, the secod-order coeffcet sc j s equal to zero, resultg D ( ) = max{ hc ( ), hc ( ) }. j j j I the BSS, order to measure the depedece of a sgal o the others ad the separateess of all the source sgals, we geeralze Defto 1 as follows. Defto 2. Defe D () as the depedece measure of sgal y () o all the other sgals, amely, D ( ) = max{ D ( )} = max{ sc ( ), hc ( ), hc ( ) }. (14) j j j j j, j j, j Defto 3. Let D() represet the depedece amog the outputs of the separato system, the t descrbes the separateess of all the source sgals, ad ca be defed as D ( ) = max{ D( )} = max { sc( ), hc( ), hc ( ) }. (15) j j j, j, j Sce the BSS s desred to be mplemeted o-le practcal applcatos, we eed to derve the adaptve updates of sc j () adhc j (). o ths ed, we assume that the output y s wde-sese statoary ad ergodc,.e. ts statstcal average ca be substtuted by ts tme average. It s easy to verfy that for ay statoary dscrete sgal x(), the sample mea at tme ca be wrtte as x ( ) = λ x ( ) = λ x ( 1) + x ( ), = 1 where 0 λ 1 s a forgettg factor. Lettg R = cov[ y ( ), y ( )], P = cov[ φ ( y ( )), y ( )], ad Q = cov[ φ ( y ( ))], j j j j we have the followg recursve formulae: 1 1 y( ) = λ y( 1) + y( ), (16) ( ) = y ( ) y ( 1), (17) 1 1 Rj ( ) = λ [ Rj ( 1) + ( ) j ( )] + [ y ( ) y ( )][ yj ( ) yj ( )], (18) φ 1 1 ( ) = ( 1) ( y( )), + (19) ( ) = φ ( ) φ ( 1), (20) 1 1 ( ) Pj = λ [ Pj ( 1) + ( ) j ( )] + [ φ ( y ( )) φ ( )][ yj ( ) yj ( )], (21)

6 36 SCIENCE IN CHINA (Seres F) Vol Q ( ) = λ [ Q ( 1) + ( )] + [ φ ( y ( )) φ ( )] (22) for, j = 1,,. Oce R j, P j ad Q are recursvely estmated, we ca use (11) ad (12) to compute drectly sc j ()adhc j (), ad thus obta D j (), D ()add(). 2.2 A gradg learg algorthm for BSS It s well ow that for the LMS-type algorthms, a fxed learg rate or step sze wll evtably lead to the cotradcto betwee the covergece speed ad the steady-state performace. o solve ths problem, oe may decrease the learg rate as tme [10,14,15] or use the adaptve learg rate [16 18], but oe s related drectly to the depedece or separateess of the source sgals, so the performace mprovemet s qute lmted. Furthermore, a scalar learg rate mples that all the source sgals use the same learg rate, whch s clearly ot optmal. hs s because a sgal already separated from the others should use a small learg rate to trac, whle a sgal ot separated from the others should use a large oe to speed up ts separato. Although a small learg rate s approprate for the separated sgals, t may be a uder-learg rate for other source sgals wth strog depedece o each other, whch slows dow ther separato. Cotrarly, a large learg rate sutable for some source sgals wth strog depedece may be a over-learg rate for the other separated sources, whch may shatter them. Based o these cosderatos, we propose a gradg learg algorthm for BSS whose cetral dea s to use a learg rate matrx Λ = { η ( )} stead of the orgal scalar η, j ad thus the algorthm (8) becomes W+ 1 = W + [ Λ G( y)] W, (23) where represets the Hadamard product of two matrx, amely, A B = { ab}, ad the elemets of Λ are determed by the values D j (), D ()add(). Resortg to D(), we dvde the whole separato process to the followg three stages: Ital stage: 0.25 D() 1;.apture stage: 0.05 D() 0.25; tracg stage: 0 D() We may cosder the tal stage as the pre-separato of the source sgals, the secod stage as coarse separato to capture each source sgal, ad the last stage as the fe separato to mprove the recovery qualty of the source sgals. For the ew BSS algorthm (23), we have the followg result. heorem 1. he gradg learg algorthm (23) s equvarat ad has the property of eepg the separatg matrx from becomg sgular. Proof. Let C = WA be the global mxg-separatg system, ad post-multply both sdes of (23) by A, we mmedately obta C + 1 = C + [ Λ G( y )] C whch s depedet of j j

7 No. 1 GRADING LEARNING FOR BLIND SOURCE SEPARAION 37 W ad A, so the gradg learg algorthm (23) s equvarat. O the other had, let XY, = trace( X Y), W = det( W). Mmcg ref. [9], we have d W W dw =, = WW,[ Λ G( y)] W dt W dt = W trace[ Λ G( y)] = W ηg. t herefore, we get Wt = W0 exp η ( ) ( )d, 0 τ G τ τ from whch t follows that f W0 0, = 1 the W 0, eepg W from becomg sgular. hs completes the proof of heorem 1. t 3 Comparsos wth the exstg algorthms I ths secto, we cosder the desg of Λ, ad compare the gradg learg algorthm (23) wth several typcal BSS algorthms usg fxed or tme-varyg learg rates. 3.1 Ital stage Sce the outputs of the separato system are of strog depedece the tal stage, we should use a hgh learg rate to speed up the separato. Clearly, t s approprate to assg the same learg rate to all the source sgals ths stage, thus we tae the learg rate matrx Λ as a costat matrx,.e. ηj ( ) = η for ay ad j, ad the gradg learg algorthm (23) s the smplfed to (8). he scalar η has may possble choces. A smplest choce s to use a costat learg rate η = η. A alteratve s to use a tme-descedg fucto as the chagg learg rate, such as the coolg scheme [15] 1,whchη =, 2 startg from a adequate tal value. Recetly, Yag [10] suggested usg the expoetal atteuato fucto η0, K0, η = Kd ( K0 ) η, K 0e 0, (24) where η 0, K 0 ad K d are costats. he tme-descedg learg rate has a drawbac: f there are some source sgals that have ot bee separated from the others, ad η has become very small, the these sources are much dffcult to be extracted. o overcome ths lmtato, the thrd choce s the adaptve learg rate. Oe example s the so-called gradet step sze [18 20] that s calculated as J η+ 1 = η + ρc W ) η (25) whch J(C) s some cost or cotrast fucto, ad ρ s a step sze of the step sze. As poted = 1

8 38 SCIENCE IN CHINA (Seres F) Vol. 46 out by Douglas et al. [18], the shortcomgs of ths method are ts poor robustess, ad the fact that the proper selecto of ρ s dffcult. I fact, t s stll a log way to go for the gradet step sze scheme to be used practce. Explotg the formato o how far the estmated separatg matrx s from ts optmum, Murata et al. [17] suggested the followg adaptve learg rate algorthm: Γ+ 1 = (1 δ) Γ δg( y) W, (26) η+ 1 = η + αη[ β Γ η], (27) where α, β ad 0 δ 1 are parameters to be determed, ad deotes the Frobeus orm of Γ. I practce, to assure the stablty of the BSS algorthm, the learg rate should be cofed to a fte rage,.e. η m η η max. o the best of our owledge, however, amog adaptve learg rate algorthms publshed prevously, oe s depedet drectly o the separato state of source sgals. Sce the tas of ths stage s to pre-separate the sources, we tae the scalar η as a lear fucto of D(): η0, K0, η = (28) a + ( η0 ad ) ( ), K0 ad D ( ) ƒ0.25, where a, η 0 ad K 0 are costats. hat η = η0 s tae for K 0 s based o the cosderato that the recursve estmates of sc j () adhc j () have larger bas for a shorter set of samples. O the other had, the source sgals are strogly correlated wth each other at the tal stage of the BSS, whch demostrates that a larger learg rate should be adopted. 3.2 Capture stage he ma tas of BSS ths stage s to capture all the source sgals. Due to the fact that oe or more sources may be completely or partly separated from the others, t s o loger optmal ths stage to assg the same learg rate to all the source sgals. o speed up the separato of the o-separated sources from the others, ad trac the separated sources at the same tme, dfferet source sgals should use dfferet learg rates. o ths ed, we tae the th row of Λ as the same value η ( ), = 1, f,. Based o ths specal structure, the gradg learg algorthm (23) becomes W + 1 = W + D G ( y ) W, (29) where D =dag[ η1 ( ),, η( )] s a dagoal matrx, whose elemets are determed by η ( ) = β( D ( )), = 1,, (30) whch β (C) s a olear fucto, see secto 4 for ts choce. Recetly, from the vewpot of the Fsher formato matrx, Yag [10] proposed a seral updatg rule for adaptve BSS, gve by 1 η µ Φ W + 1 = W + D [ I ( y ) y ] W, (31) Γ

9 No. 1 GRADING LEARNING FOR BLIND SOURCE SEPARAION 39 where Dµ = dag[ µ 1,, µ 2]. hs rule s based o the followg three costrats [10] : C1) All the dstrbutos of the source sgals are symmetrc. C2) he trplets ( µ, λ, v ) of Q = E{ vec( I Φ( y) y ) vec ( I Φ( y) y )} satsfy v = µ λ, where vec(b) deotes the colum vector obtaed by cascadg the colums of B from the left to the rght. C3) he mxg matrx A must be a square matrx. It should be stressed that the learg algorthm (31) has a form smlar to (29), but they have the followg substatal dffereces: (1) D 1 µ (31) s oly a dagoal matrx form, ts elemets have o relatoshp wth the sgal depedece, whle the dagoal elemets of D (29) are determed by the sgal depedece va D (). (2) Algorthm (29) has ot the costrats C1 C racg stage Oce all the source sgals are successfully captured, the remader s to trac them ad mprove the recovery qualty as much as possble. From the vewpot of the covergece of the eural etwor for BSS, the fal objectve of ths stage s to mae the separatg matrx coverge to the optmum oe. Usg the relatoshp betwee the equlbrum pots ad D j (), we tae β ( D ( )), = j, ηj ( ) = (32) β ( Dj ( )), j, where β (C) s the same olear fucto as that (30). We ote that Cchoc et al. [16] proposed a learg algorthm for BSS, whch each syaptc weght has ts ow (local) learg rate, gve by where d w ( ) j t = ηj () t wj () t φ ( y ()) t wpj () t yp () t, dt (33) p= 1 j j φ pj p p= 1 g () t = w () t ( y ()) t w () t y (), t (34) d vj ( t) τ 1 = vj () t + gj (), t dt (35) d ηj ( t) τ2 = ηj () t + γ vj (), t dt (36) whch τ 1, τ 2 ad γ are parameters to be carefully chose. Deote Λ = { η ( )} ad = φ G( y ) I ( y ) y, respectvely, the (33) ca be wrtte a matrx form as j

10 40 SCIENCE IN CHINA (Seres F) Vol. 46 W+ 1 = W + Λ [ G( y) W ] (37) whch form s much smlar to the gradg learg algorthm (23), but they are dfferet at least two aspects. Oe s the determato of the learg rate matrx Λ, the other s that algorthm (37) does ot satsfy the equvarat property. Post-multplyg both sdes of (37) by A yelds C+ 1 = C + { Λ [ G( y) W]} A. Clearly, the global system C depeds o A uless Λ s a costat matrx or all the rows of Λ have the same values. I geeral, algorthm (37) s ot equvarat, whle the equvarat property s the most essetal property to the BSS algorthm. 4 Smulatos ad dscussos I order to verfy the effectveess of the gradg learg algorthm (23) for adaptve BSS, we use t to separate several source sgals from ther mxtures. he uow source sgals are as follows: S1) Sg fucto sgal: sg(cos(2π155 t)); S2) hgh-frequecy susod sgal: s(2π800 t); S3) low-frequecy susod sgal: s(2π90 t); S4) phase-modulated sgal: s(2π300t 6cos(2π60 t)); S5) ampltude-modulated sgal: s(2π9 t)s(2π300 t); S6) radom sgal: uformly dstrbuted ose [ 1, 1]; S7) radom sgal: the square root of the Raylegh dstrbuted sgal wth parameter σ =1. I smulatos, the mxg matrx A s fxed each ru, but the elemets of whch are radomly geerated subject to the uform dstrbuto [ 1, 1] or the ormal dstrbuto wth zero mea ad varace 0.5. For smplcty, we deote t as A~U( 1, 1) ad A~N(0, 0.5), respectvely. he mxed source sgals are sampled at 10 Hz. As the performace dex of the BSS algorthm, we use the cross-talg error [9] E ct cl cl = 1 1 = 1 l= 1 max c + p l= 1 = 1max c pl p p, (38) where C = { c } = W A s the combed mxg-separatg matrx. l I smulatos, we use the olear fucto φ( y)=[ φ ( y ),, φ ( y )] suggested by Yag ad Amar [4],.e. the gradg rule (23), we tae 1 1 N = φ 2 3 ( y) ( 3, 4) y ( 3, 4) y G( y ) I ( y ) y, ad φ = α κ κ + β κ κ (39) N whch 3 3 E y ( ) κ = { } ad 4 4 E y ( ) κ = { } 3 are called the sewess ad the urtoss of y,re- spectvely; ad the fuctos α ( κ3, κ4) ad β ( κ3, κ4) are gve by

11 No. 1 GRADING LEARNING FOR BLIND SOURCE SEPARAION a ( κ3, κ4) = κ3 + κ3κ4, 2 4 (40) β ( κ3, κ4) = κ4 + ( κ3) + ( κ4) (41) he sewess ad the urtoss are updated usg the followg adaptve formulae: 3 3, 1 3, u 3, y, κ + = κ ( κ ), (42) 4 4, 1 4, u 4, y, κ + = κ ( κ + 3), (43) where =1.0D10 4 s s the samplg perod, ad µ s a costat, tae as 60D. For comparso, we execute the atural gradet algorthm wth several dfferet learg rates whose correspodg early optmal parameters for fast covergece ad good steady-state performace are as follows: (1) he fxed learg rate, called the Yag-Amar (Y-A) algorthm [9] : η =65D. (2) he tme-descedg learg rate (24), called the Yag algorthm [10] : η 0 =140D, K 0 = 500 ad K d =15D. (3) he adaptve learg rate (26), (27), called the MMZA algorthm [17] : η max = 140D, η m =5D, δ =5D, α =0.02ad β = 150 Γ / max Γ p. (4) he sgal-adaptve learg rate ths paper, called the gradg learg algorthm: η 0 = 140D, K 0 =500ada =40D are tae (28), ad the forgettg factor λ = (16) (22). he olear fucto β (C) (30) ad (32) must be a mootoc odecreasg ad bouded oegatve fucto. It may tae dfferet forms, such as the expoetal fucto, the sgmod fucto, the pecewse lear fucto ad so o. A better choce of β (C) should provde good capture ad tracg capabltes, ad thus we tae t as a covex fucto the capture stage ad a cocave fucto the tracg stage, resultg a sgmod fucto: x, 0 x 0.05, 0.7 β ( x) = ( x 0.05), 0.05 x 0.2, (44) 0.016, 0.2 x. o chec ad compare the performace of the above four algorthms uder dfferet codtos, we smulated the followg four cases: he source sgals cosst of the fve determstc sgals S1 S5 ad a radom ose S6, ad the mxg matrx A~U( 1, 1). he sx source sgals S1 S6, but A~N(0, 0.5). he sx source sgals S1 S6 plus aother radom ose S7, ad A~U( 1, 1). he seve source sgals S1 S7, but A~N(0, 0.5). p

12 42 SCIENCE IN CHINA (Seres F) Vol. 46 he separatg matrx taes W 0 =0.5I as the tal value. Fgs. 2 ad 3 show respectvely the meas ad the stadard devatos (SDs) of the performace dex E ct of successve 400 depedet rus (where 200 rus for A~U( 1, 1) ad aother 200 rus for A~N(0, 0.5)) for the source sgals S1 S6, whle fgs. 4 ad 5 show those for the source sgals S1 S7. Fg. 2. Average performace dces, = 6. 1, they-aalgo- rthm; 2, the Yag algorthm; 3, the MMZA algorthm; 4, ths paper s algorthm. Fg. 3. SDs of performace dex, = 6. 1, the Y-A algorthm; 2, the Yag algorthm; 3, the MMZA algorthm; 4, ths paper s algorthm. Fg. 4. Average performace dces, = 7. 1, they-aalgo- rthm; 2, the Yag algorthm; 3, the MMZA algorthm; 4, ths paper s algorthm. Fg. 5. SDs of performace dex, = 7. 1, the Y-A algorthm; 2, the Yag algorthm; 3, the MMZA algorthm; 4, ths paper s algorthm. able 1 lsts the falg separato umbers ad the correspodg percetages of the four algorthms the above four cases. By falg separato, we mea that at least oe source sgal could ot be separated from the others utl the 4000th sample. From table 1 ad fgs. 2 5, we ca see that the gradg learg algorthm (23) has the fastest covergece speed (the fastest descedg of E ct ), the best steady-state performace (the least average performace dex E ct after covergece) ad the best umercal robustess (the least SD of E ct ), as compared wth the other three algorthms.

13 No. 1 GRADING LEARNING FOR BLIND SOURCE SEPARAION 43 able 1 Falg separato umber (200 depedet rus/each case) Source ad chael Learg algorthm of the learg rate η Source umber Chael Yag-Amar Yag MMZA Gradg learg =6 A~U( 1, 1) 3(1.5%) 7(3.5%) 5(2.5%) 1(0.5%) =6 A~N(0, 0.5) 5(2.5%) 9(4.5%) 7(3.5%) 1(0.5%) =7 A~U( 1, 1) 15(7.5%) 40(20%) 20(10%) 3(1.5%) =7 A~N(0, 0.5) 15(7.5%) 38(19%) 22(11%) 3(1.5%) 5 Coclusos hs paper defed the measure of sgal depedece ad the separateess of the source sgals for the BSS system, ad proposed a gradg learg algorthm for adaptve BSS, whch the whole learg process s dvded to three stages: tal stage, capturg stage ad tracg stage. he orgal scalar learg rate s geeralzed to a learg rate matrx whose elemets are automatcally determed by the measure of sgal depedece, yeldg three sgal-adaptve learg rates. We compared the three sgal-adaptve learg rate selectos the gradg learg algorthm wth the prevously publshed fxed ad tme-varyg learg rate algorthms. It was show that the gradg learg algorthm has both the equvarace ad the property of eepg the separatg matrx from becomg sgular. Smulato results showed that the ew algorthm performs better tha the other exstg algorthms. Acowledgemets ). Refereces hs wor was supported by the Natoal Natural Scece Foudato of Cha (Grat No. 1. Hyvare, A., Fast ad robust fxed-pot algorthms for depedet compoet aalyss, IEEE ras. Neural Networs, 1999, 10(3): Gaaopoulos, V., Comparso of adaptve depedet compoet aalyss algorthms, Avalable at f/~xgaa/. 3. Bell, A. J., Sejows,. J., A formato-maxmzato approach to bld separato ad bld decovoluto, Neural Computatos, 1995, 7: Karhue, J., Joutsesalo, J., Represetato ad separato of sgals usg olear PCA type learg, Neural Networs, 1994, 7: Karhue, J., Pajue, P., Oja, E., he olear PCA crtero bld source separato: Relatos wth other approaches, Neural Computg, 1998, 22: Como, P., Idepedet compoet aalyss, a ew cocept? Sgal Processg, 1994, 36: Pham, D.., Bld separato of stataeous mxtures of sources va a depedet compoet aalyss, IEEE ras. Sgal Processg, 1996, 44: Cardoso, J. F., Laheld, B., Equvarat adaptve source separato, IEEE ras. Sgal Processg, 1996, 44: Yag, H. H., Amar, S., Adaptve o-le learg algorthms for bld separato maxmum etropy ad mmum mutual formato, Neural Computatos, 1997, 9: Yag, H. H., Seral updatg rule for bld separato derved from the method of scorg, IEEE ras. Sgal Processg, 1999, 47: Lee,. W., Grolam, M., Sejows,. J., Idepedet compoet aalyss usg a exteded fromax algorthm for mxed sub-gaussa ad super-gaussa sources, Neural Computatos, 1999, 11:

14 44 SCIENCE IN CHINA (Seres F) Vol Cho, S., Cchoc, A., Amar, S., Flexble depedet compoet aalyss, Joural of VLSI Sgal Processg, 2000, 26: Cruces, S., Cchoc, A., Castedo, L., A teratve verso approach to bld source separato, IEEE ras. Neural Networs, 2000, 11: Amar, S., Cchoc, A., Yag, H. H., A ew learg algorthm for bld sgal separato, Advaces NIPS, 1996, 8: Amar, S., Natural gradet wors effcetly learg, Neural Computatos, 1998, 10: Cchoc, A., Amar, S., Adach, M. et al., Self-adaptve eural etwors for bld separato of sources, Proc Iteratoal Symp. o Crcuts ad Systems, vol.2, New Yor: IEEE Press, May 1996, Murata, N., Muller, K. R., Zehe, A. et al., Adaptve o-le learg chagg evromets. Advaces NIPS 9, Cambrdge, MA: MI Press, 1997, Douglas, S. C., Cchoc, A., Adaptve step sze techques for decorrelato ad bld source separato, Proc. 32d Aslomar Cof. o Sgals, Systems ad Computers, Pacfc Grove, CA, vol.2, New Yor: IEEE Press, Nov. 1998, Jacobs, R. A., Icreased rates of covergece through learg rate adaptato, Neural Networs, 1988, 1: Mathews, V. J., Xe, Z., A stochastc gradet adpatve flter wth gradet adaptve step sze, IEEE ras. Sgal Processg, 1993, 41: Zhag, X. D., Bao, Z., Commucato Sgal Processg ( Chese), Bejg: Defese Idustry Press, 2000, Papouls, A., Probablty, Radom Varables, ad Stochastc Process, New Yor: McGraw-Hll, 3rd edto, 1991,