Independent component analysis for nonminimum phase systems using H filters

Transcription

1 Independen componen analysis for nonminimum phase sysems using H filers Shuichi Fukunaga, Kenji Fujimoo Deparmen of Mechanical Science and Engineering, Graduae Shool of Engineering, Nagoya Universiy, Furo-cho, Chikusa-ku, Nagoya 6-863, Japan Deparmen of Monodukuri Engineering, Tokyo Meropolian College of Indusrial Technology, 1-1- Higashioi, Shinagawa-ku, Tokyo, 1-11, Japan Absrac: This paper proposes an independen componen analysis (ICA) mehod using H filers for nonminimum phase sysems. In he basic ICA approach, he inpu signal is recovered by esimaing he parameer of he inverse of he mixing sysem. If he sysem is nonminimum phase, he esimaed parameer diverges due o he insabiliy of he inverse. For his problem, a inverse filer is consruced based on an H filer in order o esimae he sae of he given plan. The learning algorihm o esimae he parameer of he sysem is derived by minimizing he Kullback-Leibler divergence. Furhermore, a numerical simulaion demonsraes he effeciveness of he proposed mehod. 1. INTRODUCTION Independen componen analysis (ICA) [Hyvärinen e al., 21] is a ool for saisical daa analysis and signal processing ha is able o recover independen inpu signals from mixed signals where he mixing process is unknown. This mehod has many poenial applicaions in various fields such as speech signal processing, image processing biomedical signal processing and elecommunicaions. In he field of sysem conrol engineering, i has been applied o sysem idenificaion [Sugimoo and Nia, 25], disurbance suppression [Sugimoo e al., 25], faul deecion [Sugimoo e al., 25] and process monioring [Kano e al., 1997]. Typical problems reaed in ICA are called Blind Source Separaion (BSS). While a sandard BSS problem adops a saic mapping as he mixing process model, a blind deconvoluion problem employs a dynamical equaion insead. In sysem conrol engineering, we have o deal wih dynamical sysems, and hence have o solve he blind deconvoluion problem. To solve his problem, an FIR filer approach [Thi and Juen, 1995], [Gorokhov and Loubaon, 1999] and sae-space approaches [Waheed and Salem, 23], [Zhang and Cichocki, 2], [Fukunaga and Fujimoo, 26] were proposed. In hese mehods, he inpu signal is recovered by esimaing he parameer of he inverse of he mixing sysem. However, if he sysem is nonminimum phase, he esimaed parameer diverges due o he insabiliy of he inverse sysem. For his problem, Zhang e al. proposed an ICA mehod for nonminimum phase sysems using FIR filers [Zhang e al., 2]. This mehod approximae he inverse sysem using FIR filers. On he oher hand, several researchers proposed sable inversion echniques for nonminimum phase sysems [Devasia e al., 1996], [Hun e al., 1996]. George e al. proposed a sable dynamic model inversion echnique [Devasia e al., 1996], [Hun e al., 1996]. This mehod consruc an inverse filer using Kalman filers. We developed an ICA mehod for nonminimum phase sysems using Kalman filers. However, since a blind deconvoluion problem reas non-gaussian signals, we proposed an ICA mehod using H filers [Takaba and Kaayama, 199] which is no assumed o rea gaussian signals. This paper is organized as follows. In Secion 2, a design of H filers is presened. In Secion 3, we propose an ICA mehod for nonminimum phase sysems. Firs, an approximae inverse filer is consruced based on an H filer. Second, he parameers of he inverse filer is esimaed by a seepes descen mehod. In Secion, a numerical example is given. Finally, Secion 5 concludes he paper. 2. DESIGN OF H FILTERS Here, we refer o [Takaba and Kaayama, 199] for a design of H filers. Consider he following linear ime-varying sysem x 1 F x G w y H x v where x R n, y R p, w R r and v R p are he sae vecor, he measuremen, he process disurbance and he measuremen noise, respecively. Noe ha w R r and v R p are unknown and arbirary L 2 [, N] signals. We assume ha an esimae of he iniial sae x is given by ˆx. We also define z L x where z R q, L R q n.

2 The finie-horizon H filering problem is o find he esimaes ˆx and ẑ based on he measuremen signal y so ha he following inequaliy hold. z ẑ 2 sup x,v,w x ˆx 2 Σ v 2 R w 2 Q < γ 2 (1) where γ is a given consan and Σ, Q and R are posiive definie weighing marices. We define he following cos funcion J (ẑ ; x, v, w ) : z ẑ 2 γ 2 ( x ˆx 2 Σ v 2 R w 2 Q The inequaliy condiion (1) is equivalen o having J (ẑ ; x, v, w ) <. The problem is o esimae ẑ saisfying he following inequaliy max J (ẑ ; x, v, w ) < x,v,w ) (2) Thus, we consider he following minimax problem beween ẑ and x, v, w. min ẑ max J (ẑ ; x, v, w ) < x,v,w The opimal minimizer of he minimax problem is given by where ẑ L ˆx ˆx 1 F ˆx ˆx F ˆx K (y H ˆx ) K : P H T (H P H T R ) P 1 F P Ψ F T G Q G T (3) Ψ I (H T R H γ 2 L T L )P () If γ is sufficienly large, he second erm of equaion (2) becomes dominan. Thus, as γ ends o infiniy, he minimax problem reduces o he problem of minimizing J (x, v, w ) : x ˆx 2 Σ v 2 R w 2 Q wih respec o w and x. I is well-known ha his minimizaion problem is equivalen o he leas mean square (LMS) esimaion problem in he case where x is generaed by he gaussian disribuion N (ˆx, Σ ) and where w and v are he zero mean gaussian whie noises wih uni covariances. The opimal soluion of he LMS esimaion problem is given by he Kalman filer. 3. INDEPENDENT COMPONENT ANALYSIS FOR NONMINIMUM PHASE SYSTEMS 3.1 Problem seing Consider he following linear ime-invarian sysem x 1 Ax Bu (5) y Cx Du where x R n, u R m and y R m are he sae vecor of he sysem, he inpu signal and he measured signal, respecively. Here we make he following assumpion for he inpu. Assumpion 1. The elemens of he unknown inpu signal u are saionary zero-mean i.i.d. process and muually saisically independen. Here, y is known and A, B, C, D, x and u are known. The objecive is o esimae he inpu u only from he oupu y. This can be achieved by finding he esimaes Â, ˆB, Ĉ, ˆD and ˆx of A, B, C, D and x. 3.2 Design of an approximae inverse filer In he basic ICA approach, he inpu signal is recovered by esimaing he parameer of he inverse of he mixing sysem. However, if he sysem is nonminimum phase, he esimaed parameer diverges due o he insabiliy of he inverse. This subsecion consrucs an approximae inverse filer using H filers In he sable dynamic inversion echnique, he inverse filer is consruced based on he Kalman filer. We design he inverse filer using he H filer. Since A, B, C and D are unknown, we consider he following ime-varying sysems x 1 Âx ˆB u y Ĉx ˆD u (6) For his sysem, he H filer is consruced by where ˆx 1 ˆx K : P ( ˆx ˆx K ( ˆD Ĉ ) T ( P 1 ÂP Ψ Ψ I (( ˆD Ĉ ˆx ˆD Ĉ P ( ˆD y ) (7) ˆD Ĉ ) T R ) (8)  T ˆB Q ˆBT (9) ˆD Ĉ ) T R ˆD Ĉ γ 2 L T L )P (1) The inpu signal can be esimaed by using he inverse of he oupu equaion (6). Therefore, he inverse filer is given by where ˆx 1 Âs, ˆx ÂK ˆ D y û ˆ Cˆx ˆ D y (11)  s,  ÂK ˆ C ˆ C ˆD Ĉ ˆ D ˆD

3 If he marix D is singular, ˆD Ĉ and ˆD canno be calculae. We esimae ˆ C and ˆ D in order o avoid a singular poin. 3.3 Esimaion of he parameer of he inverse filer In his subsecion, we derive he algorihm for esimaing he parameers Â, ˆB, ˆ C, ˆ D of he inverse filer. A densed noaion of he oupus Y and ha of he esimaes of he inpus Û are defined by Y : (y T, y1 T,..., yn) T T Û : (û T, û T 1,..., û T N) T If he esimaes of he inpus Û are spaially muually independen and emporarily i.i.d. signals, hen he probabiliy densiy funcion p(û) of Û is facorized as follows: p(û) m i1 N p(û i, ) Saisical independence can be measured using Kullback- Leibler divergence beween p(û) and m i1 N p(û i, ) I(Û) : p(û) log p(û) m N i1 p(û dû (12) i, ) The objecive is o minimize I(Û) wih respec o he parameers Â, ˆB, ˆ C and ˆ D by he seepes descen mehod. We reformulae he cos funcion in order o implemen he on-line learning for he parameers of he inverse filer. According o he propery of he probabiliy densiy funcion, we derive he following relaion beween p(û) and p(y): p(y) p(û) (13) J where J is he deerminan of he Jacobian marix which is calculaed as follows ˆ D... ˆ D1... J de ˆ DN where denoes a nonzero value. Since he above Jacobian marix is lower block riangular, p(û) is described by p(û) p(y) N de ˆ D Subsiuing equaion (1) for equaion (12) gives N I(Û) E[log de ˆ D ] E[log p(y)] m i1 E[log p(û i, )] (1) Here E[log p(y)] is a consan. Therefore, he cos funcion can be simplified as Ĩ(û ) : log de ˆ D m log p(û i, ) (15) i1 The minimizaion of Ĩ(û ) requires he compuaion of is gradien wih respec o he parameers Â, ˆB, ˆ C and ˆ D. The gradiens are compued separaely in he parameers of he oupu equaion and hose of he dynamics. We can calculae he derivaive of Ĩ(û ) wih respec o he marices ˆ C and ˆ D as Ĩ(û ) ˆ C ϕ(û )ˆx T (16) Ĩ(û ) ˆ D T ˆ D ϕ(û )y T (17) where ϕ(û ) is defined by ( d log p(û1, ) ϕ(û ) :,..., d log p(û ) T m, ) (18) dû 1, dû m, The nonlinear funcion ϕ(û) depends on he probabiliy densiy funcion of he inpu signal, which is unknown in a blind deconvoluion seing. I is no necessary o esimae he probabiliy densiy funcion precisely. One imporan issue in deermining he nonlinear funcion is ha he sabiliy condiions of he learning algorihm mus be saisfied [Amari e al., 1997]. The gradien of he cos funcion in (17) requires calculaing he inverse of he marix ˆ D. Forunaely, a modificaion of sandard gradienbased procedures has been developed ha overcomes his difficuly in he BSS ask. This modificaion, ermed he naural gradien by [Amari e al., 1996], modifies he sandard gradien updae by a linear ransformaion whose elemens are deermined by he Riemannian meric ensor for he assumed parameer space. Then he modified search direcion is given by Ĩ(û ) ˆ D ˆ DT ˆ D (I ϕ(û )y T ˆ DT ) ˆ D (19) The gradiens of Ĩ(û ) wih respec o he marices  and ˆB can be calculaed as Ĩ(û )  Ĩ(û ) ˆB 2 Ĩ(û ) ˆx p, ˆx p,  (2) Ĩ(û ) ˆx p, ˆx p, ˆB 2 (21) (See he Appendix for he deails of his calculaion.) 3. Algorihm Summarizing he resuls of he previous secion, he block diagram of he proposed mehod is shown in Fig 1 and he proposed algorihm consiss of he following seps: Algorihm 1.

4 inpu oupu esimaed sae esimaed inpu u Sysem Eq. (5) y filer Eq. (7) x -1 Oupu inversion Eq. (11) u Parameer esimaor Eq. (22) - (25) Fig. 1. Block diagram of he proposed mehod sep1 Se he iniial value of he parameers Â, ˆB, ˆ C, ˆ D and P sep2 Compue he gain (8) sep3 Updae he filer equaion (7) sep Updae he marix (9) sep5 Updae he parameers Â, ˆB, ˆ C, ˆ D using he gradiens are described by equaions (2), (21), (16) and (19) Â 1 Â µ 1 ˆB 1 ˆB µ 2 n n Ĩ(û ) ˆx p, ˆx p, Â (22) Ĩ(û ) ˆx p, ˆx p, ˆB 2 (23) ˆ C 1 ˆ C µ 3 ϕ(û )ˆx T (2) ˆ D 1 ˆ D µ (I ϕ(û )y T ˆ DT ) ˆ D (25) where µ 1, µ 2, µ 3, µ > are he sep size parameers. sep6 Compue he esimae of he inpu (11). NUMERICAL EXAMPLE The parameers of he sysem (5) are A B C D The zeros are a.8651, ,.858,.83 ± i,.256 ±.789i,.2957,.858,.6776,.759±.9828i,.79, ,.53, and hence he sysem is nonminimum phase. The inpu signal u is chosen o be i.i.d signal uniformly disribued in he range (, 1). The nonlinear funcion is chosen as ϕ(û ) û 3. The iniial parameers are deermined as follows: Â ˆB ˆ C ˆ D Fig. 2 shows he source signals and he recovered signals. In he figure, he solid line denoe he recovered signal and he dashed line denoe he inpu signal. Fig. 3 shows he cos funcion Ĩ(û ). The compuaion of he cos funcion needs o use he probabiliy densiy funcion of û which is unknown. We solve equaion (18) wih nonlinear funcion ϕ(û ) û 3. The soluion of equaion (18) is given by ( 2 p(û ) Γ û ) 1 exp where Γ( ) is he gamma funcion. We observed ha he value of he cos funcion decreases and converges. This resul demonsraes he effeciveness of he proposed mehod. To illusrae he influence of γ, we used γ 1.15 and Fig. shows he cos funcion averaged over 2 seps. In he figure, he solid line denoes γ 1.15 and he dashed line denoes γ. The error bar denoes he sandard derivaion from he average of 1 rials. The value of he cos funcion a γ 1.15 is smaller han γ. We compare he performance of he proposed mehod and he Zhang s mehod [Zhang e al., 2]. The filer lengh of he Zhang s mehod is 1. Fig. 5 shows he cos funcion averaged over 2 seps. In he figure, he solid line denoes he proposed mehod and he dashed-doed line denoes he Zhang s mehod. The error bar denoes he sandard derivaion from he average of 1 rials. The proposed mehod converges faser han he Zhang s mehod. Furhermore i is seen ha he proposed mehod can obain a smaller cos funcion han he Zhang s mehod.

5 ime [s] ime [s] Fig. 2. Source signals and recovered signals ime [s] Fig. 3. Cos funcion Ĩ(û ) ime [s] Fig.. Cos funcion in case of γ 1.15 and 5. CONCLUSION In his paper, an independen componen analysis mehod for nonminimum phase sysems using H filers is proposed. An inverse filer is consruced based on an H filer in order o esimae he sae of he given plan Fig. 5. Cos funcion of he proposed mehod and he Zhang s mehod The learning algorihm o esimae he parameer of he sysem is derived by minimizing he Kullback-Leibler divergence. A numerical simulaion shows he effeciveness of he proposed mehod. REFERENCES S. Amari, T.P. Chen, and A. Cichocki. Sabiliy analysis of learning algorihms for blind source separaion. Neural Neworks, 1(8): , S. Amari, A. Cichocki, and H. H. Yang. A new learning algorihm for blind signal separaion. In Advances in Neural Informaion Processing Sysems 8, pages , S. Devasia, D. Chen, and B. Paden. Nonlinear inversionbased oupu racking. IEEE Transacions on Auomaic Conrol, 1(7):93 92, S. Fukunaga and K. Fujimoo. Nonlinear blind deconvoluion based on a sae-space model. In Proceedings of he 5h IEEE Conference on Decision and Conrol, pages , 26. A. Gorokhov and P. Loubaon. Blind idenificaion of MIMO-FIR sysems: a generalized linear predicion approach. Signal Processing, 73(1-2):15 12, ISSN L.R. Hun, G. Meyer, and R. Su. Noncausal inverses for linear sysems. IEEE Transacions on Auomaic Conrol, 1():68 611, A. Hyvärinen, J. Karhunen, and E. Oja. Independen Componen Analysis. John Wiley & Sons, 21. M. Kano, S. Tanaka, H. Marua, S. Hasebe, I. Hashimoo, and H. Ohno. Saisical process monioring wih exernal analysis and independen componen analysis. Transacions of he Sociey of Insrumen and Conrol Engineers, 38(11), (in Japanese). K. Sugimoo and M. Nia. Polynomial marix approach o independen componen analysis: (par i) basics. In IFAC World Congress, 25. K. Sugimoo, A. Suzuki, M. Nia, and N. Adachi. Polynomial marix approach o independen componen analysis: (par ii) applicaion. In IFAC World Congress, 25. K. Takaba and T. Kaayama. Finie-horizon minimax esimaion for linear discree-ime sysems. Transacions of Insiue of Sysems, Conrol and Informaion Engineers, 7(8): , 199. (in Japanese).

6 H.L. Nguyen Thi and C. Juen. Blind source separaion for convoluive mixures. Signal processing, 5:29 229, K. Waheed and F.M. Salem. Blind source recovery: A framework in he sae space. Journal of Machine Learning Research, :111 16, 23. L. Zhang and A. Cichocki. Blind deconvoluion of dynamical sysems: A sae space approach. Journal of Signal Processing, (2):111 13, 2. L. Zhang, A. Cichocki, and S. Amari. Mulichannel blind deconvoluion of nonminimum-phase sysems using filer decomposiion. IEEE Transacions on Signal Processing, 52(5):13 12, 2. Appendix A. CALCULATION OF THE GRADIENTS WITH RESPECT TO THE PARAMETERS  AND ˆB The gradiens wih respec o he parameers  and ˆB are calculaed by Ĩ(û ) Âij, ϕ(û ) T Ĩ(û ) ˆB ik, 2 ϕ(û ) T ˆ C p, ˆx p, Âij, ˆ C p, ˆx p, ˆB ik, 2 where ˆ Cp is he l-h column vecor of marix ˆ C. Here, ˆx p, / Âij, and ˆx p, / ˆB ik, 2 are described by ˆx 1 Âij, Âs, ˆx J ij ˆx Âij, ( ) J ij K ˆ C ˆx ˆ D y ˆx 1 ˆB ik, Âs, ˆx r, ˆB ik, K  ˆB ˆ C ˆx ˆ D y ik, where J ij is he single-enry marix wih 1 a (i, j) and zero elsewhere. K / ˆB ik, is given by K ˆB P ik, ˆB ˆ CT ( ˆ C P ˆ CT ik, P ˆ CT ( ˆ C P ˆ CT P 1 ˆB ik, J ik Q ˆBT R ) ˆ C P ˆB ik, ˆ CT ( ˆ C P ˆ CT ˆB Q J ikt R ) R )