EIGENVALUE DISTRIBUTION OF THE CORRELATION MATRIX IN L`-FILTERS. Constantine Kotropoulos and Ioannis Pitas

Transcription

1 EIGEVALUE DISTRIBUTIO OF THE CORRELATIO MATRI I L`-FILTERS Constantne Kotropoulos and Ioanns Ptas Dept of Informatcs, Arstotle Unversty of Thessalonk Box, Thessalonk 0 0, GREECE fcostas, ptas g@zeuscsdauthgr ABSTRACT L`-flters are a fundamental flter class wthn the famly of order statstc flters In ths paper, startng from frst prncples, we derve the cumulatve densty functon and the probablty densty functon of tme and ranked ordered samples for ndependent dentcally/non-dentcally dstrbuted nput random varables The raw second moments and the product moments of the tme and ranked ordered samples are then computed for ndependent dentcally dstrbuted nput samples Based on the aforementoned moments the correlaton matrx of the tme and ranked ordered samples s derved and ts egenvalue dstrbuton s determned We present relatonshps between the egenvalues of the correlaton matrx of tme and ranked ordered samples and those of the correlaton matrx of the ordered samples ITRODUCTIO onlnear flters have become a very mportant tool n sgnal processng, and especally n mage analyss and computer vson For a revew of the nonlnear flter classes the reader may consult [] One of the best known nonlnear famles s based on the order statstcs It uses the concept of data orderng One of the major classes of order statstc flters s the L`-flter [, ] It has the form of a lnear combnaton of the observatons and t explots the combned rank and locaton nformaton nherent n the observatons Powerful extensons of the L`-flters have been proposed n the lterature, such as the order statstc flter banks [], the permutaton flter lattces [] Closely related estmators to the L`-estmator have been proposed for the estmaton of the mean usng order statstcs n [] In practce, adaptve desgns based on the Least Mean Squares algorthm (LMS of L`-flters and ther extensons preval [] In ths paper, startng from frst prncples, we derve the cumulatve densty functon and the probablty densty functon of tme and ranked ordered samples for ndependent dentcally/nondentcally dstrbuted nput random varables The raw second moments and the product moments of the tme and ranked ordered samples are then computed for ndependent dentcally dstrbuted (d nput samples Based on the aforementoned moments the correlaton matrx of the tme and ranked ordered samples s derved and ts egenvalue dstrbuton s determned Although L`flters loose some of ther advantage n an d envronment, lke permutaton flters [8], because all tme-rank orderngs are equally lkely, the propertes derved under ths assumpton gve a valuable nsght nto the operaton of adaptve LMS L`-flters n respect of ther convergence n the mean and n the mean-square Such an d envronment s the case of a constant sgnal corrupted by addtve zero-mean whte nose For an observaton vector of length, we prove that out of the egenvalues of the tme and ranked ordered samples are gven by the egenvalues of the correlaton matrx of the ranked ordered samples dvded by We also derve lower and upper bounds for the mnmal and maxmal egenvalue of the correlaton matrx of the tme and ranked ordered samples These bounds depend on the extreme egenvalues of the correlaton matrx of the ranked ordered samples The latter egenvalues are related through nequaltes wth the egenvalues of the ranked ordered nose samples The theoretcal results have been verfed by numercal computatons The outlne of the paper s as follows Secton brefly descrbes L`-estmators Secton deals wth the probablty densty functon of one tme and ranked ordered sample and the jont probablty densty functon of two tme and ranked ordered samples The egenvalues of the correlaton matrx of ndependent dentcally dstrbuted tme and ranked ordered samples are studed n Secton Upper and lower bounds on the extreme egenvalues are derved n ths secton as well Smlar bounds on the extreme egenvalues of the correlaton matrx of ranked ordered samples are presented n Secton L`-ESTIMATORS Let us consder that the observed sgnal fx(ng can be expressed as a sum of an arbtrary nose-free sgnal fs(ng plus zero-mean addtve whte nose fv(ng, n denotes dscrete-tme Let x`(n be the vector of nput observatons x`(n =(x (n;x (n;:::;x (n T ( and x L(n be the ranked ordered nput vector at n gven by x L(n = x( (n;x ( (n;:::;x ( (n T x ( (n» x ( (n» :::» x ( (n The vector x L(n s commonly referred as the vector of the order statstcs of x`(n Let us defne the temporal locaton vector [] ( ο ( (n =(ο (n;ο (n;:::;ο (n T ( ρ f x ο ( (n $ x j(n j(n = ( 0 otherwse In ( x ( (n $ x j(n denotes that the th order statstc occupes the jth temporal sample By usng the temporal locaton vector we create the followng vector x L`(n = x( (nο T ( (n j x ((nο T ( (n j jx ((n

2 ο T ( (n T = x( (n;:::;x ( (n j ::: j x ( (n;:::;x ( (n T : ( An estmate bs(n of the orgnal (nose-free sgnal can be obtaned by bs(n =c T x L`(n ( that defnes the so-called -L`-estmator [] Henceforth we call ths estmator L`-estmator for brevty Let us assume that x L`(n defnedn(ands(n are jontly statonary stochastc sgnals Based on the statonarty assumpton we are nterested n the dervaton of the -L`-flter that mnmzes the mean squared error " = Ef(s(n bs(n g = c T R L` c c T p L` +Efs (ng ( R L` =Efx L`(nx T L`(ng s the correlaton matrx of the tme and ranked ordered samples, x L`(n, andp L` = Efs(n x L`(ng s the cross-correlaton vector between the vector x L`(n and the desred response s(n Clearly, provded that R L` s not sngular, the optmal -L`-flter coeffcent vector c o s gven by c o = R L` p L`: (8 Approaches to determnng the L`-flter coeffcents c usng the LMS algorthm have been proposed n [, 9] The convergence n the mean and n the mean square of the LMS-based desgn approaches depends strongly on the egenvalue dstrbuton of R L` [0] The objectve of ths paper s to study the egenvalue dstrbuton of R L` n an d envronment PROBABILIT DESIT FUCTIO OF THE CORRELATIO MATRI OF TIME AD RAKED ORDERED SAMPLES Let g (t and G (t denote, respectvely, the probablty densty functon (pdf and the cumulatve densty (cdf functon the -th nput observaton The pdf of the random varable x (m for ndependent non-dentcally dstrbuted observatons s gven by the followng expresson: f (m (t = g (t (n ;n ;:::;n m n ;n ;:::;n m S n <n <:::<n m n l n l S ;n ;:::;n m G n (tg n (t :::G nm (t [ G nl (t] m =; ;:::; and =; ;:::; (9 S = f; ;:::;g, S = Sfg, ands ;n ;n ;:::;n m = S fn ;n ;:::;n mg In (9 the summaton extends over all permutatons (n ;n ;:::;n m of ; ;:::; ;+ ;:::; whch are m n total Eq (9 can be proved startng from frst prncples for =, and subsequently applyng mathematcal nducton Alternatvely, t can be obtaned as a specal case of the analyss n [8, ] For =and m = =(9 yelds f ( (t =g (t[ G (t] [ G (t] : (0 The cdf of the random varable x (m can be obtaned by Z t F (m (t = f (m (ψdψ: ( For d nput observatons, we have f (m (t = g(tg m (t[ G m (t] m = f (m(t ( f (m (t s the pdf of the m-th order statstc x (m [] For» k<l», ; j =; ;:::;, = j and t <t the jont pdf of the random varables x (k and x (lj s gven by f (k (lj (t ;t =g (t g j(t (n ;n ;:::;n lk n ;n ;:::;n lk S ;j;q ;:::;q k n <n <:::<n lk G n m(t ] k ρ= (q ;q ;:::;q k q ;q ;:::;q k S j q <q <:::<q k G q ρ (t f S ;j;q ;:::;q k ;n ;:::;n lk lk m= [G n m (t [ G f (t ] : ( For d nput observatons ( yelds ρ f (k (lj (t ;t ( f (k(l(t ;t f t <t = f (l(t f t t ( f (k(l (t ;t s the jont pdf of the order statstcs x (k and x (l EIGEVALUES OF THE CORRELATIO MATRI OF TIME AD RAKED ORDERED SAMPLES In the subsequent analyss we assume that the nput observatons are d random varables, e, the nose-free sgnal s a constant s Under ths assumpton, usng ( and (, we obtan the followng expressons for the second-order moment of the random varable x (m and the product moments of the random varables x (k and x (lj : Efx (kg = Efx (kg ( Efx (k x (lj g = ( Efx (kx (l g ( k < l, = j and k; l = ; ;:::; Let us denote by Q = R L = Efx Lx T Lg the correlaton matrx of the ranked ordered samples Smlarly let R L` =Efx L`x T L`g defne the correlaton matrx of tme and ranked ordered samples By defnton both R L` and R L are postve sem-defnte Followng smlar arguments to [, pp 90-9], we argue that the aforementoned matrces are postve defnte f the random varables of concern are lnearly ndependent It s trval to show that the sum of the egenvalues s the same n both R L` and Q,thats = (R L` = = (Q =(ff + s (

3 (Q s the -th egenvalue of Q and ff s the nose varance Let 0 denote an matrx of zeroes By employng elementary smlarty transformatons t can be shown that R L` s smlar to a matrx havng the followng structure: G = G 0 ::: 0 0 A ::: A 0 A ::: A Q ( Q ( Q ::: Q ::: (8 ( Q ( Q ( Q ( Q ::: Q (9 and A j, ; j =;:::; are approprate matrces The egenvalues of matrx G can be obtaned analytcally, e (G = (Q: (0 Accordngly, we have found that under the d assumpton out of the egenvalues of R L` can be obtaned from the egenvalues of R L dvded by The egenvalue dstrbuton of the correlaton matrx of ranked ordered samples R L and the tme and ranked ordered samples R L` s plotted n Fgure for unform, Gaussan and Laplacan nose dstrbuton havng zero mean and unt varance, when s =and = In the followng subsecton, we demonstrate that t s also possble to derve upper and lower bounds for the smallest and the largest egenvalue of R L` Upper and lower bounds on the extreme egenvalues of the correlaton matrx of tme and ranked ordered samples The correlaton matrx of the tme and ranked ordered samples can be decomposed as R L` = dag (Q;Q;:::;Q Ω I + B ( dag( denotes the dagonal matrx whose dagonal elements are those nsde parentheses, I s the dentty matrx, Ω denotes the Kronecker product and B = Ω W The matrces and W are gven by = W = 0 Q ::: Q Q 0 ::: Q Q Q ::: 0 ( T I ( ( s the vector of ones B has egenvalues whose sum equals zero The sum of the egenvalues of both and W equals zero as well Therefore, the smallest egenvalue of matrx s negatve whle ts largest egenvalue s postve It can be shown that W has two dstnct egenvalues, e, (W = wth multplcty and (W = ( wth multplcty ( Accordngly, the egenvalues of B are as follows: ( ( ; wth multplcty, =; ;:::; ( ; =; ;:::; ( ( are the egenvalues of matrx The prevous dscusson yelds the followng expressons: mn( max(b = max ( ; max( mn(b = mn max( ( ; mn( : ( Accordngly, we need to relate the egenvalues of matrces B and to the egenvalues of the correlaton matrx of ranked ordered samples R L = Q The latter matrx can be decomposed as follows: Q = + dag (Q ;Q ;:::;Q : ( Let us defne by max and mn the maxmum and mnmum dagonal element of the correlaton matrx of the ranked ordered samples, respectvely, e: max = max (Q ;Q ;:::;Q ( mn = mn (Q ;Q ;:::;Q : (8 By applyng Theorem 8 [, p 9] we obtan ff = max(q max ff» max(» f fl» mn(» ff (9 f = max(q mn fl = mn(q max ff = mn(q mn: (0 The applcaton of the same theorem yelds also max + mn(b» max(r L`» max + max(b mn + mn(b» mn(r L`» mn + max(b ( By combnng ( and the nequaltes (9, (0 and ( we get mn(r L max(rl + max mn» max(r L`» mn(r L ( max mn» mn(r L`» max(rl : ( The numercal computatons summarzed n Table ndcate that the extreme egenvalues of R L` are equal to the extreme egenvalues of R L dvded by, a result that s wthn the ntervals predcted by the analyss descrbed above As a consequence the egenvalue spread of the correlaton matrx of tme and ranked ordered samples s the same wth that of the correlaton matrx of ranked ordered samples

4 Correlaton matrx (=,s= 0 Correlaton matrx (=,s= 0 Correlaton matrx (=,s= Egenvalues 0 0 Egenvalues 0 0 Egenvalues Tme and Tme and Tme and (a (b (c Fg Egenvalue dstrbuton of ranked ordered samples R L and the tme and ranked ordered samples R L` when s =and =for (a unform, (b Gaussan, and (c Laplacan nose dstrbuton havng zero mean and unt varance Table Smallest and largest egenvalues of R L and R L` for unform, Gaussan and Laplacan parent dstrbuton havng mean s =:0 and unt varance parent dstrbuton mn(r L mn(r L` max(r L max(r L` unform Gaussan Laplacan EIGEVALUES OF THE CORRELATIO MATRI OF RAKED ORDERED SAMPLES In the precedng analyss we have employed the correlaton matrx of the ranked ordered (nosy observatons Under the assumpton of a constant sgnal corrupted by addtve whte nose, the latter correlaton matrx can be expressed n terms of the correlaton matrx of ranked ordered nose samples, Φ =Efv Lv T Lg, as follows: R L = Φ + s μ T + μ T + s T ( μ =Efv Lg s the vector of the expected values of the order statstcs of nose samples Subsequently, we analyze the behavor of the egenvalues of R L For symmetrc nose dstrbutons T about zero, t can be shown that the matrx H = μ T + μ s smlar to []» H 0 = : ( For odd, the matrces and n ( are gven, respectvely, by: = ~μ~ T + ~ ~μ T ( = ~ ~μ T ~μ~ T ( ~μ s the vector of the expected values of v (, v (, :::, v (, ~ s the ( -dmensonal vector of ones and s the matrx that has ones along the secondary dagonal and zeros else It can be further proved that the matrx defned n ( has ( zero egenvalues and the remanng two non-zero ones are gven by ( ;(H = ;(H 0 =± p ~μ T ~μ: ( Wthout any loss of generalty, f s>0, we obtan p ;(s H =± = ±s ~μ T ~μ: (8 By applyng Theorem 8 [, pp 9] t can be shown that mn(φ + s H» mn(φ + (9 max(φ + s H» max(φ + : (0 Theorem 88 [, pp 9] asserts that there exst nonnegatve coeffcents k, k,,k such that ( (R L= ( (Φ + s H +k s ( the egenvalues are arranged n ascendng order of magntude

5 If ( (R L= (+ (Φ + s H = ( (Φ; =; ;:::; ( then t can be shown that max(r L» s + ( (Φ + max(φ ( The assumpton ( has been verfed n numercal computatons, as can be seen n Table In the computatons we have used the tables from [] The valdty of the upper bound n ( s demon- Table Egenvalues of matrces Φ, Φ+sH,andR L for Gaussan nose dstrbuton havng zero mean and unt varance, when s = :0 ( (Φ ( (Φ + sh ( (R L strated n Table for unform, Gaussan and Laplacan parent dstrbutons havng mean s =:0 and unt varance Accordngly, the smallest egenvalue of R L and consequently R L` s controlled exclusvely by the nose statstcs, that s, the smallest egenvalue of the correlaton matrx of the ordered nose samples On the contrary, the largest egenvalue of both R L and R L` s nfluenced by the dmensonalty of the observaton wndow, the true constant sgnal value, and the nose varance REFERECES [] I Ptas and A Venetsanopoulos, onlnear Dgtal Flters: Prncples and Applcatons, Kluwer Academc Publ, orwell, MA, 990 [] F Palmer and CG Boncelet, Jr, L`-fltersa new class or order statstc flters, IEEE Transactons on Acoustcs, Speech, and Sgnal Processng, vol, no, pp 90, May 989 [] PP Gandh and SA Kassam, Desgn and performance of combnaton flters for sgnal restoraton, IEEE Transactons on Sgnal Processng, vol 9, no, pp 0, July 99 Table Valdty of the upper bound n ( for unform, Gaussan and Laplacan parent dstrbutons havng when s = :0 and unt varance parent dstrbuton max(r L Upper bound unform Gaussan Laplacan [] G Arce and M Tan, Order statstc flter banks, IEEE Transactons on Image Processng, vol, no, pp 8 8, June 99 [] -T Km and GR Arce, Permutaton flter lattces: A general order statstc flterng framework, IEEE Trans on Sgnal Processng, vol, no 9, pp, September 99 [] S Zozor, E Mosan, and P Amblard, Revstng the estmaton of the mean usng order statstcs, Sgnal Processng, vol 8, pp, 998 [] C Kotropoulos and I Ptas, Adaptve order statstc flterng of stll mages and vdeo sequences, n onlnear Model- Based Image/Vdeo Processng and Analyss, C Kotropoulos and I Ptas, Eds J Wley, ew ork, 00 [8] KE Barner and GR Arce, Permutaton flters: A class of nonlnear flters based on set permutatons, IEEE Transactons on Sgnal Processng, vol, no, pp 898, Aprl 99 [9] I Ptas and S Vougoukas, LMS order statstc flter adaptaton by backpropagaton, Sgnal Processng, vol, pp 9, 99 [0] S Haykn, Adaptve Flter Theory, Prentce Hall, Englewood Clffs, J, 98 [] M Prasad and H Lee, Stack flters and selecton probabltes, IEEE Transactons on Sgnal Processng, vol, no 0, pp 8, October 99 [] HA Davd, Order Statstcs, J Wley, ew ork, 980 [] A Papouls, Probabltes, Random Varables and Stochastc Processes, McGraw-Hll, ew ork, rd edton, 99 [] G Golub and CF van Loan, Matrx Computatons, The Johns Hopkns Unversty Press, Baltmore, rd edton, 99 [] A Canton and P Butler, Egenvalues and egenvectors of symmetrc centrosymmetrc matrces, Lnear Algebra and ts Applcatons, vol, pp 8, 9 [] D Techroew, Tables of expected values of order statstcs and products of order statstcs for samples of sze twenty and less from the normal dstrbuton, Ann Math Statst, vol, pp 0, 9