Západočeská Univerzita v Plzni, Czech Republic and Groupe ESIEE Paris, France

Transcription

1 ADAPTIVE SIGNAL PROCESSING USING MAXIMUM ENTROPY ON THE MEAN METHOD AND MONTE CARLO ANALYSIS Pavla Holejšovsá, Ing. *), Z. Peroua, Ing. **), J.-F. Bercher, Prof. Assis. ***) Západočesá Univerzia v Plzni, Czech Republic and Groupe ESIEE Paris, France Absrac: Designed signal and image processing mehod enables o avoid impac of measuremen sysem on experimenal evidence, miigaes noises and increases precision of obained daa. The simulaions in Malab were used o verify he proposed algorihm consised of wo mehods. One of hem is he Maximum Enropy on he Mean Mehod (MEMM) ha deals wih he solving of he linear and noisy inverse problem of he form y=ax+b. The second one is he Expecaion-Minimizaion Algorihm (EM algorihm) consising of wo seps. The expecaion sep (E sep) compues esimaion xˆ and i is followed by he maximizaion sep (M sep) ha provides new esimaion. Boh seps have o be ieraed unil convergence. In he exponenial family, he E sep gives similar resuls as he MEMM ha is why we combine boh mehods in order o explore he advanages of boh. The measured daa (y) processed by proposed algorihm allows o eliminae he noise b and he influence of he measuremen sysem properies (presened by degradaion marix A). Thus, his algorihm enables o reconsruc real daa x ha objec produces. Imporance of his mehod in he signal and image processing and diagnosics is obvious. 1. MAXIMUM ENTROPY ON THE MEAN METHOD (MEMM) The Maximum Enropy on he Mean Mehod (MEMM) solve he linear and noisy inverse problem of he form of y=ax+b, where y are he observaions, A is he supposed degradaion marix, vecor x is he measured objec (ypically a signal or image) and b is he noise which has o be esimaed. The measured daa processed by his algorihm allows o eliminae he noise b and he influence of he measuremen sysem properies (presened by degradaion marix A) o he measured daa y and so o obain real daa ha he objec produces. Given reference measure μ defined on he objec x and noise b, he MEMM consiss of selecing he disribuion pˆ which is he closes o μ according o he Kullbac disance and which saisfy a given consrain, in his case he observaion equaion. The MEMM esimaion xˆ is he mean of he seleced disribuion pˆ. So, i is he minimizer of a convex cos funcion defined on x and b. Le have a linear inverse problem y=ax+b. The observaion marix A is supposed o be nown and some saisical characerisics of he noise b oo. When he observaion marix A is no regular or ill-condiioned, he problem is ill-posed. I means he convex consrain x C, (1) where C is a convex se, is necessary. In some specific problems where he lower and upper bound (a,b ) are nown, he convex se can be defined as N C = x R / x a, b, = 1.. N. (2) { } For he esimaion xˆ we selec he disribuion pˆ which is he closes o reference measure μ according o he Kullbac disance. The Kullbac disance D(p μ) is defined for a reference measure μ and probabiliy measure P by dp D(p µ ) = log dp. (3) dµ Thus, he MMEM mehod begins by he specificaion of he convex se C and he reference measure dμ(x) over i. The acual observaions y are considered as he mean E p (X) of he probabiliy disribuion P defined on C.

2 The disribuion P is seleced as he minimizer of he μ-enropy submied on he consrains of he mean AE p (x) = y in he noiseless case. I means he P is he neares disribuion respec o he Kullbac disance D(p μ) o he reference measure μ saisfying equaion AE p (x) = y. MEMM problem in he noiseless case is given by: dp p ˆ = arg min log ( x) dp( x), P dµ such ha y = A xdp( x) (4) and x = E p ( x) The soluion exiss if i is belongs he exponenial family. For each x C, he AE p (x) = y. We define he funcion F(x) as he opimum value of he Kullbac disance. F x = Inf D(p µ ), where P = P : E x = x. (5) { } ( ) ( ) P PX X Than he problem can be posed as: ( ) xˆ = arg min F x, x (6) such ha y = Ax. This mehod amouns o minimizing he convex crierion and so admis he dual formulaion. The dual funcion is defined as D λ = λ y F λ A, (7) ( ) ( ) and allows o calculae numerically he expecaion E p (x). The funcion F ( λ A) is he convex conjugae of he funcion F(x). I means ha dual formulaion of he problem ha have o be solved is o find esimaion λˆ by maximizing he dual funcion D(λ): ( λ y F ( λ A) ) ˆ λ = sup *, (8) λ The problem number one of he MEMM is o define he funcion F ( λ A) ha presens he opimum value of he dual funcion and o maximize i. The problem is ha in he noisy F λ A has o be separaed o wo funcions: case he funcion ( ) F ( λ A) F ( λ A) + F ( λ ) x b p =. (9) The firs one is he signal funcion Fx ( λ A) funcion F ( λ ) and he second one is he noise b. As you see he problem sar o be more complicaed because boh pars have o be defined and solved separaely. The soluion of his problem is really complicaed. In he exponenial family, he MEMM gives similar resuls as he E sep of he EM algorihm. Tha is why we ried o use he EM algorihm o eliminae he need of he funcion F ( λ A) separaion in he noisy case. 2. E-M ALGORITHM WITH MAXIMUM ENTROPY ON THE MEAN METHOD Expecaion-Minimizaion Algorihm (EM algorihm) consiss of wo seps: he expecaion sep followed by he maximizaion sep. The E sep compues an esimaion xˆ saisfying he condiions upon he observaions. Le have x he daa, he y observaions, f(x θ) probabiliy densiy funcion and θ se of parameers of he densiy, hen for E sep we compue: [ ] Q θ θ = E log f x θ y, θ. (10) ( ) [ ( ) ]

3 Then he M sep provides new esimaion: Q( λ, λ, y) = λ AE[ x y, θ ] (11) These wo seps mus be ieraed unil convergence occurs - i may be deermined as: [ ] [ 1] θ θ < ε. (12) In he exponenial family, he MEMM gives similar resuls as he E sep of he EM algorihm. Tha is why he new mehod of he ieraive algorihm was used o avoid he difficulies wih he funcion F ( λ A) separaion in he noisy case of MEMM and he boh mehods were combined. We compue he esimaion of he x by applying he Maximum Enropy on he Mean Mehod (MEMM) ino he EM algorihm. The firs sep is compues he esimaionλˆ by maximizing he dual funcion D(λ) using he MEMM and hen he EM algorihm sars by implemenaion of he esimaionλˆ - i means we compue: Q λ, λ, y = λ AE x y, λ. (13) λ ( ) [ ] + 1 = arg max λ AE[ x y, λ ] F ( λ A) λ. (14) And hese wo seps are ieraed unil he convergence condiion occurs. [ ] [ 1] λ λ < ε. (15) 3. RESULTS The simulaions were made o reconsruc real signal x (x rue ) from he measured observaions y (y measured ). Resuls of he daa reconsrucion are well seen in he simulaions below. Here you can see he resuls of he simulaions of he signal x ha is given by: x=abs((.9*cos(2*pi*[.5:1:n-.5]/n)+.05))'. (16) The noise is supposed o be Gaussian. Various values of he σ 2 have been considered. In his paper, σ 2 =0.01, σ 2 =0.2 a σ 2 =0.5 are presened in figures 1, 2 and 3. On he lef side, here is he comparison beween he real signal x (normal) and signal x ME (bold) compued by he MEMM wih EM algorihm. Experimenal daa y are presened by poins. On he righ side of each figure you can see he ieraion process of esimaion xˆ. The real signal x is presened by doed line, las ieraion of he compued signal x ME by bold. Fig. 1 Simulaion of he MEMM wih EM algorihm (Parameers: σ 2 =0.01, 100 ier. cycles)

4 Fig. 2 Simulaion of he MEMM wih EM algorihm (Parameers: σ 2 =0.2, 1000 ier. cycles) Fig. 3 Simulaion of he MEMM wih EM algorihm (Parameers: σ 2 =0.5, 10 ier. cycles) 4. MONTE CARLO ANALYSIS Various simulaions in Malab were made o reconsruc real signal x from he measured observaions y compued by he MEMM wih EM algorihm. If he signal is oo noisy, he resuls will be less saisfacory as you can see in he figure 4a),b). Fig. 4 Simulaions of he MEMM wih EM algorihm a) Parameers: σ 2 =0.3, 16 ier. cycles b) Parameers: σ 2 =0.4, 13 ier. cycles

5 The noise and is disribuion can lead o he reconsrucion fauls ha can cause he esimaed signal disorion. In his case, he Mone Carlo analysis is applied o avoid he mehod misaes and compuing fauls. I allows o reconsruc he esimaion xˆ of he real signal x from measured, very noisy observaions y wih high precision. The Mone Carlo analysis was applied o 30 simulaions wih he same parameers and he obained resuls are shown in he figures 5 and 6. On he lef side you can see all he simulaions and he resul of he Mone Carlo analysis (blac line) wih is faul range (blac doed line). You can see ha even if he simulaions can provide quie grea reconsrucion fauls, he resul of he Mone Carlo analysis corresponds well wih he real daa x (gray bold line). On he righ side, he comparison of he real signal x (gray bold line) and he mean of he compued esimaion xˆ of all he simulaions (blac hin line) wih is faul range (gray hin doed line) is presened. From he figures is eviden ha qui grea esimaion error is very saisfacorily eliminaed by he Mone Carlo analysis (blac line). Fig. 5 Mone Carlo Analysis of he MEMM wih EM algorihm (Parameers: σ 2 =0.3, 30 simulaions) Fig. 6 Mone Carlo Analysis of he MEMM wih EM algorihm (Parameers: σ 2 =0.4, 30 simulaions)

6 5. CONCLUSIONS The Maximum Enropy on he Mean Mehod (MEMM) is used o solve he linear and noisy inverse problem. Because of difficulies in he compuaion process in he noisy case, he MEMM is used as he firs sep of he Expecaion Maximizaion algorihm ha allows o converge o he real signal x by he ieraion process. The problem is o find he bes esimaion xˆ of he real daa and saisfy he given consrains. We combine wo mehods in order o gain he advanages of boh. The proposed algorihm reconsrucs he real daa x from measured, noisy observaions y. This mehod enables o miigae he noise b and he measuremen sysem impac. The simulaions in Malab were made o verify his new reconsrucion mehod based on he Maximum Enropy on he Mean Mehod combined wih EM algorihm. The noise is supposed o be Gaussian. Various values of he σ 2 have been considered. If he signal is oo noisy, he resuls will be less saisfacory. The noise and is disribuion can lead o he reconsrucion fauls ha can cause he esimaed signal disorion. In his case, he Mone Carlo analysis is applied o avoid he mehod misaes and compuing fauls. I allows o reconsruc he esimaion xˆ of he real signal x from measured, very noisy observaions y wih favorable precision. Imporance of his mehod in he signal and image processing and diagnosics is obvious. The mehod is sill in progress. ACKNOWLEDGEMENT The research was performed a Groupe ESIEE Paris, France and is parly suppored by projec FRVŠ 2328/2003/G1. REFERENCES [1] J.-F.Bercher, G.Le Besnerais, G.Demomen: The Maximum Enropy on he Mean Mehod, Noise and Sensiiviy. Maximum Enropy and Bayesian Mehods, Kluwer Academic Publishers, Cambridge,U.K.,1995. [2] C.Heinrich, J.-F.Bercher, G.Demomen: The Maximum Enropy on he Mean Mehod, Correlaions and Implemenaion Issues. In: IEEE Transacions on Informaion Theory,1995. [3] Todd K. Moon: The Expecaion-Maximizaion Algorihme.. In: IEEE Signal Processing Magazine. November 1996, IEEE1996. p /96. *) **) ***) Universiy of Wes Bohemia in Pilsen, Faculy of Elecrical Engineering Deparmen of Applied Elecronics Univerziní 8, Plzeň, Czech Republic holejsov@ae.zcu.cz Universiy of Wes Bohemia in Pilsen, Faculy of Elecrical Engineering Deparmen of Elecromechanics and Power Elecronics Univerziní 8, Plzeň, Czech Republic peroua@ieee.org École Supérieure d'ingénieurs en Élecroechnique e Élecronique (Groupe ESIEE), Déparemen Traiemen du signal e Télécommunicaions e Laboraoire Parole Signal Image (LPSI) Cié Descares, BP 99, Noisy le Grand Cedex