Stochastic Signals and Systems

Transcription

1 Sochasic Signals and Sysems Conens 1. Probabiliy Theory. Sochasic Processes 3. Parameer Esimaion 4. Signal Deecion 5. Specrum Analysis 6. Opimal Filering Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 1 6 Opimal Filering Inroducion 3 6. Mached Filering Mached Filering for Whie Noise Mached Filering as Correlaion Processing Wiener Filering Wiener-Hopf Equaion 6.3. Finie Wiener Filering Noncausal Wiener Filering Causal Wiener Filering Kalman Filering Sae Space Model Sae Esimaion Kalman Filer Approach 56 Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus

2 6 Opimal Filering 6.1 Inroducion A basic problem in he applicaion of sochasic processes is he esimaion of a signal in he presence of addiive noise. The signal may be random or deerminisic, and he noise may be colored or whie. The problem consiss of esablishing he presence of he signal or of esimaing is form. The soluion of his problem depends on he sae of prior knowledge concerning he signal and he noise, e.g. we may be able o specify signal and noise covariance funcions, power specra or probabiliy densiies. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 3 Furhermore, sysem consrains define he form of he soluion. For example we migh allow he sysem o be nonlinear/linear, ime-varian/invarian, realizable, ec. In he following, we shall be exclusively concerned wih linear ime-varian/invarian sysems bu will no necessarily require ha hey be realizable. 6. Mached Filering In chaper we considered sochasic processes and described he impac of linear sysems on hese processes. Now, we develop echniques for designing linear filers o minimize he effec of noise. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 4

3 X s U U h H() X s U U The signal X could be eiher a signal in noise or noise only. The signal s is assumed o be deerminisic. Addiionally, we suppose ha E(U ) = 0 and he specrum C UU () of he inpu noise U is known. Now, we wish o deermine he filer characerisics such ha he insananeous raio of he oupu signal power o he oupu noise power is maximized a sampling ime 0, i.e. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 5 S S max max s E h h 0 N N ou,max This problem ypically arises in sonar and radar applicaions, where we wish o esablish he presence and locaion of a signal s reurning from a disan arge. Remark: The mached filer does no preserve he waveform of he inpu signal. The objecive is o disor he waveform and filer he noise such ha a he sampling ime 0 he oupu signal level will be as large as possible compared o he oupu noise level. ou Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 6 U

4 Theorem: The mached filer ha maximizes S s E 0 N ou has a ransfer funcion given by * S( ) j 0 Hop( ) k e, CUU( ) where S( ) F s and C ( ) F c ( ) are he Fourier ransform of s and he specrum of U, respecively, k is an arbirary real consan and 0 is he sampling ime when (S/N) is evaluaed. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 7 U UU UU Exercise 6.-1: (Proof of he Theorem) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 8

5 Remarks: k is an arbirary consan since he signal and he noise a he inpu are boh muliplied by k. Thus k cancels in he relaion for (S/N) ou. The filer found may or may no be causal. If i is no causal, i has o be approximaed by a causal filer. The ransfer funcion of he opimum filer is proporional o he complex conjugae of he specrum of he inpu signal. Hence, we migh say ha he linear sysem is mached o he specified signal. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus Mached Filering for Whie Noise Theorem: Suppose he inpu noise is whie. Then he impulse response of he mached filer is given by hop, cs 0, where c is an arbirary real consan, 0 is he ime of he peak signal oupu, and s is he known inpu signal waveform. Consequenly, he impulse response of he mached filer is simply a ime reversed, complex conjugaed and by 0 ranslaed version of he known signal waveform. Therefore, he filer is said o be "mached o he signal". Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 10

6 Exercise 6.-: (Proof of he Theorem) X s Z Z h H() X s Z Z Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 11 The signal-o-noise raio a he oupu is given by S N ou where Es finie lengh T. 1 1 E S( ) d s, T 1 s 0 T 1 s Z Z 0 Z is he energy of he inpu signal of Remarks: The signal-o-noise raio a he oupu of he filer depends on he signal energy and power level of he noise and no on he paricular signal waveform used. To improve he signal-o-noise raio, we can increase he signal ampliude or he signal lengh. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 1

7 Example: We wan o find he mached filer for he known signal 1 if 1 s 0 elsewhere of finie exen T 1 + 1, as visualized below, 1 s 1 ha is imbedded in addiive whie noise. Hence, he impulse response of he mached filer is given by Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 13 h s s, op ( ) 0 0 In order o obain a causal mached filer, we have o require 0. Choosing 0, he impulse response of he mached filer is shown below. 1 s, The signal componen of he mached filer oupu, depiced in he following figure. s, is Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 14

8 s, T 0 T1 T The peak oupu level occurs a 0. The inpu signal waveform has been disored by he filer in order o peak up he oupu signal a 0. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 15 Exercise 6.-3: (Mached filer design for exponenially decaying signals) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 16

9 6.. Mached Filering as Correlaion Processing Consider a known signal waveform s of finie suppor { 1,, } embedded in whie noise Z, i.e. he signal exen is T The oupu of he mached filer a 0 is X 0 X h 0, op. Using he mached filer for whie noise wih 0, i.e. s if 0 0 T h, op 0 elsewhere he summaion is non-zero for 0 T T Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 17 and he convoluion can be reformulaed o , 0( 0 ) X Xh Xs Xs. op T1 T1 T Hence, mached filering can be inerpreed as a correlaion operaion which is illusraed in he following figure. X s Z 0 0 T1 X s Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 18

10 6.3 Wiener Filering The mached filer considered in he previous secion is an opimal filer in he sense ha i provides he highes SNR a he oupu for deecing he presence of a known signal. The Wiener filer considered now aims o provide an opimal esimaion of he realizaion of one sochasic process from observaions of anoher sochasic process. More specifically, we consider a sysem configuraion as shown in he figure below, where X, Y and denoe he sochasic process o be esimaed, he observed sochasic process and he error process, respecively. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 19 X no observable Y h H() observable The goal is o design a linear ime-invarian filer wih impulse response h such ha he expeced value of he squared-error process, i.e. he MSE, is minimized. The filer which minimizes he MSE is known as Wiener filer. For he following consideraions we suppose ha X and Y are real valued, zero-mean and joinly wide-sense saionary (WSS) sochasic processes. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 0

11 Since he processes X and Y are joinly WSS and he filer wih impulse response h is assumed o be sable, he error process is also WSS. Hence, he MSE, which is he second-order momen of, does no depend on he index. The MSE can be expressed in erms of he filer response h by qh ( ) E ( h) EX hy E( X ) h h E( ) E( ) 1 Y Y 1 h XY 1 c (0) h h c ( ) h c ( ). XX 1 1 XY 1 Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 1 The impulse response of he opimal (Wiener) filer h,op is defined by h, op argmin q( h ), h H where H denoes he se of all absoluely summable impulse responses Wiener-Hopf Equaion Now, we would like o solve he MSE problem The soluion of his problem is provided by exploiing he orhogonaliy principle saed in he following heorem. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus min qh ( ) mine X hy. hh hh

12 Theorem: (Orhogonaliy Principle) Suppose X and Y are joinly WSS. The impulse response h,op H minimizes he MSE if and only if For finding he soluion of he minimizaion problem a more convenien form of he orhogonaliy condiion is given by he following resul. Corollary: h,op H minimizes he MSE if and only if H E X h, opy hy 0 h. E X h, opy Y u 0 u. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 3 Exercise 6.3-1: (Proof of he Theorem) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 4

13 Using he corollary, we obain he equaion ha specifies he impulse response h,op of he opimum esimaor. Reformulaion of he orhogonaliy condiion E and finally, op u E X h Y Y 0 uu provides u, op u E XY h Y Y uu XY, op which is known as Wiener-Hopf equaion. c ( v) h c ( v ) v uv Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus Finie Wiener Filering We consider now he problem X no observable Y observable h H() where he sochasic process Y is observed only over a finie discree-ime inerval U { T 1,, T } wih T 1 T. I is desired o obain an esimae X for X by applying a linear filer wih impulse response h, i.e. T 1 X h Y hy T T Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 6 T 1.

14 The filer h is herefore of finie lengh as shown below. Y T 1 Y Y T h T1 h 0 h T X Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 7 As discussed in he previous secion, we wish o deermine he opimum soluion h,op, which minimizes he MSE of he esimae X. The relaion beween he discree-ime inerval U and he ime a which X should be esimaed gives rise o he following hree ypes of esimaion problems. Filering Suppose ha Y has been observed over he discreeime inerval U { T 1,, } wih T 1 > 0. Then X has o be esimaed from he mos recen observaions. The soluion o his problem provides a causal filer which can be implemened in real-ime. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 8

15 Smoohing If he observaions are aken over he discree-ime inerval U { T 1,, T } wih T 1,T > 0 hen X can be esimaed from pas and fuure observaions. This is applicable in pos-processing siuaions, when a realizaion of Y has been recorded and can be played back. Predicion Le Y be given over he discree-ime inerval U { T 1,, k } wih T 1 > k > 0. Then X has o be prediced from pas observaions. Since he filering procedure is defined by a linear operaion X represens a k-sep linear predicor for X. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 9 Now, he opimum soluion h,op has o saisfy he Wiener- Hopf equaion which can be expressed in marix form (also known as Yule-Walker equaion) as where and T 1 c v h c v vv { T,, T } XY, op 1 T h op Ch, op cxy ht, op cxy ( T ), cxy h T1, op cxy( T1) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 30

16 C c (0) c (1) c ( T T ) c c c T T c ( T1T) c ( T1T 1) c (0) 1 (1) (0) ( 1 1). Assuming ha C is posiive definie (covariance marices are a leas nonnegaive definie) he opimum impulse response is given by 1 h C c op XY. The opimum soluion h op can be efficienly compued by he Levinson-Durbin algorihm which explois he Toepliz srucure and he symmery of C. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus Noncausal Wiener Filering The following Wiener Filering approach is ermed noncausal because one wans o esimae X based on observaions Y for all U. Thus X h Y hy where he filering operaion is no necessarily causal, i.e. he impulse response may no saisfy h 0 for <0. Hence, he Wiener-Hopf equaion can be wrien as XY, op c ( v) h c ( v ) vv, where he righ side of he equaion represen a discree- Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 3,

17 convoluion of h,op and c (v) wih v...1,0,1,... Assuming ha he following Fourier ransforms exis j op( ), op, XY ( ) XY ( ) jv and v ( ) ( ) jv wih v H h e C c v e C c v e he Wiener-Hopf equaion becomes C ( ) H ( ) C ( ),, XY op where H op (), C XY () and C () denoe he ransfer funcion of he opimal filer, he power specral densiy of Y and he cross power specral densiy of X and Y. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 33 Hence, he Wiener-Hopf equaion can be easily solved for he ransfer funcion of he opimum filer, i.e. CXY ( ) Hop( ),, C ( ) from which he impulse response is obained by 1 CXY ( ) j h, op e d,. C ( ) The minimum mean square error (MMSE) can now be expressed by qh ( ) min qh ( ) minex hy, op hh hh Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 34

18 qh ( ) EX h Y EX h Y X Since cxy ( ) cyx ( ) and CYX ( ) CXY ( ) he infinie sum in he equaion above can be wrien as, op, op, op E( X ) h, ope( XY ) cxx (0) h, opcxy ( ). 1 h c ( ) h c ( ) H ( ) C ( ) d, op XY, op YX op YX 1 CXY ( ) 1 CXY ( ) CYX ( ) d d. C ( ) C ( ) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 35 Moreover, exploiing 1 cxx (0) CXX ( ) d he MMSE can be rewrien as 1 CXY ( ) qh (, op ) CXX ( ) d C ( ) where 1 XY ( ) XX ( ), 1 R C d CXY ( ) RXY ( ) C ( ) C ( ) denoes he so-called magniude squared coherence. XX Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 36

19 Applying he resuls saed in chaper.6.3, R XY () can be expressed by Cov dzx( ), dzy( ) RXY ( ) Var dz ( ) Var dz ( ) so ha R XY () can be inerpreed as he correlaion coefficien beween he random componens of X and Y a frequency. Thus, R XY ( ) 1,. Moreover, he equaliy sign holds for all [, ] if and only if X and Y are relaed by a linear ransformaion X hy X Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 37. Y Exercise 6.3-: (Signal esimaion in addiive noise, noncausal filering) X Desired Signal Y Observed Signal h H() X U Noise Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 38

20 6.3.4 Causal Wiener Filering Noncausal Wiener filering is improper for applicaions in which real-ime esimaion is required. Therefore, i is of pracical relevance o consider he causal Wiener filering problem in which we wish o esimae X based on observaions Y u for all u U { : }. Hence, wih he causal filer approach X h Y hy 0 he Wiener-Hopf equaion becomes Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 39 XY, op V 0 0 c ( v) h c ( v ) v. Since he Wiener-Hopf equaion is only defined for v 0 i can no be solved by Fourier ransform. Before we can derive he soluion we firs have o discuss he so-called specral facorizaion. Specral Facorizaion and Linear Represenaion Suppose Y has an absoluely coninuous and inegrable specral densiy C (). Then Y can be represened as noncausal filered whie noise wih, ( ) 0 and zz( ) Y g Z g E Z c and C () can be wrien as j C ( ) G( ) G( ) wih G( ) g e. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 40

21 Moreover, if C () saisfies he Paley-Wiener condiion an unique causal impulse response g wih g 0 > 0 and g exiss, such ha 0 j Gz ( ) gz wih Ge ( ) G( ) 0 log ( ) C has no zeros ouside he uni circle (minimum phase filer) and provides a facorizaion for C () in he form Furhermore, g implies ha no poles of 0 Gz ( ) are lying ouside he uni circle. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 41 C G G G G e ( ) ( ) ( ) j ( ) ( ). The Paley-Wiener condiion is a fairly weak condiion ha may hold in any siuaion of pracical ineres. However, if we impose he addiional consrains g and G( z) g z 0 z an explici expression can be obain for Gz ( ). The consrains imply ha all zeros and poles of Gz ( ) are lying wihin he uni circle. Thus, Gz ( ) can be inerpreed as he z-transform of a linear, causal, sable, minimum phase and inverible sable filer. Now, he specral densiy of X given by Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 4 C ( ) G( ) G( e ) j

22 can be analyically coninued 1 ( ) ( ) ( ) ( ) C z GzGz c z in an annulus ha conains z 1. Since C z C z 1 ( ) ( ) we can conclude ha if z are zeros and poles 0, i and z, j of C ( respecively hen z0, i, z0, i, z0, i and z, j, z, j, z z ), j are also zeros and poles of C (. z ) Hence, we obain he so-called canonical facorizaion C ( z) C ( z) C ( z), where ( ) C z and C ( z ) conain all zeros and poles of Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 43 C ( ha are lying wihin or ouside he uni circle respecively. Since he zeros and poles of C ( are rela- z ) z ) ed o he zeros and poles of C ( ) by mirroring a he z 1 uni circle we can wrie C ( z) C ( z ). Soluion of he Wiener-Hopf Equaion Due o he preceding assumpions and explanaions we are now able o solve he Wiener-Hopf equaion. For his purpose we define he sequence which has obviously o saisfy q v 0 for all v 0. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 44 v XY, op 0 q c ( v) h c ( v )

23 Afer applying he wo-sided z-transform we obain Qz ( ) C ( z) H ( zc ) ( z) XY op C z H C z C z XY ( ) op ( ) ( ), where he convoluion heorem for z-transforms and he canonical facorizaion has been exploied. Since q v is anicausal is z-transform Qz ( ) does no conain a consan componen and can only possess poles ouside he uni circle. Dividing he former equaion by C ( z ) we can wrie Qz ( ) CXY ( z) Hop( z) C ( z). C ( z) C ( z) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 45 Due o he aforemenioned properies of Qz ( ) and since C ( has only zeros ouside he uni circle we can infer z ) ha Qz ( ) C does no conain a consan componen ( z) and possesses only poles ouside he uni circle. Moreover, as he poles of H are lying wihin op( z) C ( z) he uni circle is inverse z-transform represens a causal sequence. Hence, afer defining he operaion Fz ( ) fz fz 0 we can sae ha Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 46

24 Q( z) C ( z) 0 and Hop( z) C ( z) Hop( z) C ( z). Finally, he laer provides ogeher wih Q( z) C ( z) CXY ( z) C ( z) Hop( z) C ( z) he desired soluion for he Wiener filer in he z-domain 1 CXY ( z) Hop ( z) C ( z) C ( z) which in he frequency domain can be expressed by j j 1 CXY ( e ) 1 CXY ( ) Hop( e ) Hop( ). j j C ( e ) C ( e ) C ( ) C ( ) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 47 Exercise 6.3-3: (Soluion of he Wiener-Hopf equaion for whie noise and is applicaion afer prewhiening) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 48

25 Exercise 6.3-4: (Signal esimaion in addiive whie noise, causal filering) X Desired Signal Y Observed Signal h H() X Z Whie Noise Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus Kalman Filering The Wiener approach considered in he previous secion solved he MMSE problem for filering, predicion and smoohing of scalar wide sense saionary processes, where he derivaion of he opimum filer could be primarily considered as an frequency domain approach. The Kalman approach addresses he filering, predicion and smoohing problem of no necessarily saionary and vecor valued processes. I provides soluions in he ime domain by virue of formulaing he problem in he sae space. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 50

26 6.4.1 Sae Space Model Discree-ime dynamic sysems can be represened by a sae equaion X f 1 ( X, U ), 0,1,, and a measuremen equaion Y h 1 ( X, V ), 0,1,,, where (f ) 0 and (h ) 0 are sequences of funcions, (X ) 0 is a sequence in p describing he saes of ineres, (U ) 0 is a sequence in q acing on (X ) 1 and where (Y ) 0 and (V ) 0 are sequences in r represening he measuremens and he measuremen noise, respecively. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 51 If he discree-ime sysem is linear he sae and measuremen equaions can be expressed by X 1 FX GU, 0,1,,, and Y HX V, 0,1,,, where F, G and H are pp, pq and rp marices, respecively, for each. Furhermore, if he sysem is ime-invarian he marices F, G and H become consan coefficien marices which are accordingly denoed by F, G and H. Hence, he sae and measuremen equaions simplify o X 1 FX GU and Y HX V, 0,1,,. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 5

27 Exercise 6.4-1: (Sae space represenaion of he one-dimensional moion of a paricle) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus Sae Esimaion Now, we suppose ha he sequence Y 0,...,Y has been observed and ha he sae X should be esimaed. This esimaion problem is known as a) filering problem if, b) smoohing problem if >, c) predicion problem if <. To esimae he sae X by means of realizaions of he measuremen sequence Y 0,...,Y in he minimum mean square error sense we have o find an esimaing funcion X ˆ ( y,, 0 y ) ha minimizes R ( ˆ ) E ˆ ( MSE X X X Y0,, Y ). Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 54

28 We know from chaper ha he opimum esimaing funcion is given by he condiional mean, i.e. X ˆ ( y,, y ) E X Y y,, Y y However, we are usually ineresed in generaing esimaes in real ime as increases. Since he daa increase linear wih, an efficien calculaion of he condiional mean will be impossible unless suiable resricions on he sysem model srucure are imposed, e.g. resricion o discree-ime linear sysems wih independen inpu and measuremen noise sequences of independen zero-mean Gaussian random vecors. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus Kalman Filer Approach Wihin he assumpions menioned above a compuaional efficien and simulaneous soluion of he filering and one sep predicion problem can be saed. Theorem: For he linear, finie-dimensional, discree-ime sysem X 1 FX GU and Y HX V, 0,1,,, where (U ) and (V ) are independen sequences of independen zero-mean Gaussian vecors which are independen of he Gaussian iniial condiion X 0 wih T T E( UU ) Σ ( ), E( VV ) Σ ( ), E( X ) μ, Cov( X ) Σ, UU VV Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 56

29 he esimaors X E( X Y,, Y) and X E( X Y,, Y) can be recursively deermined by he following equaions. X X 1 K Y HX 1 0,1,,, and X 1 FX 0,1,,, wih iniializaion X μ and Kalman gain marix where T 1 T K Σ 1 H H Σ 1 H Σ VV (), Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 57 is he covariance of he predicion error which can be compued joinly wih he filering error covariance by he recursion and Σ Cov( X Y,, Y ) Σ Σ 1 K H Σ, 1 0,1,,, T T Σ 1 F Σ F G Σ () G, 0,1,,, UU wih he iniializaion Σ Σ. E( X X )( X X ) Y,, Y T Σ Cov( X Y,, Y) E ( X X )( X X ) Y,, Y T Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 58

30 Exercise 6.4-: (Proof of he Theorem) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 59 The recursions consis of he following wo basic seps. Measuremen updae which updaes he sae esimae of X by incorporaing he new measuremen Y, filer error covariance marix. Time updae which provides he X X K Y HX and Σ Σ K H Σ one-sep predicion of he sae esimae, predicion error covariance marix X FX and Σ F Σ F G Σ ( ) G T T 1 1 UU Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 60

31 The measuremen updae equaion X X K Y HX 1 1 can be viewed as a combinaion of he prediced sae vecor and a correcion erm. Since Y E( Y Y,, Y ) H E( X Y,, Y ) E( V Y,, Y ) H X he difference in he correcion erm can be inerpreed as an error signal I Y Y Y HX which is known as innovaion. The erm innovaion comes from he fac ha I is he par of Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus , 1 1 Y Y I 1 ha canno be prediced and herefore conains he new informaion ha is gained by he curren observaion. Remarks: The innovaion sequence I is a sequence of independen no idenically disribued zero-mean Gaussian random vecors. Y consiss of a par, Y 1, compleely dependen and a par, I, compleely independen of he pas. Thus I provides a se of independen observaions ha forms suiably scaled he oupu of a prewhiening operaion. Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 6

32 Exercise 6.4-3: (Proof of he remarks) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 63 Kalman Filering Algorihm Iniialize he sae predicion X μ Iniialize he predicion error covariance marix Σ Σ Calculae he Innovaion I Y HX 1 Measuremen updae (Correcion) Calculae he Kalman gain marix T T () K Σ H H Σ H Σ VV Updae he sae vecor X X K I 1 Updae he error covariance marix Σ Σ K H Σ 1 1 Time updae (Predicion) Projecion of sae vecor ahead Projecion of error covariance marix ahead X FX 1 T T Σ 1 F Σ F G ΣUU() G Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 64

33 Exercise 6.4-4: (Scalar Kalman filer) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 65 Exercise 6.4-5: (Track-While-Scan Radar wih independen acceleraions from scan o scan) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 66

34 Exercise 6.4-6: (Track-While-Scan Radar wih dependen acceleraions from scan o scan) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 67 Exercise 6.4-7: (Sae space represenaion of AR(p)-, MA(q)-, and ARMA(p,q)-Processes) Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 68

35 References o Chaper 6 [1] B.D.O. Anderson and J.B. Moore, Opimal Filering, Dover, 005 [] A. Gelb, Applied Opimal Esimaion, MIT Press, 1974 [3] T. Kailah, A.H. Sayed and B. Hassibi, Linear Esimaion, Prenice Hall, 000 [4] S. Kay, Fundamenals of Saisical Signal Processing, Vol. 1: Esimaion Theory, Prenice Hall, 1993 [5] L. Ljung, Sysem Idenificaion: Theory for he User, Prenice Hall, 1998 [6] M.B. Priesley, Specral Analysis and Time Series, Academic Press, 1984 [7] H.V. Poor, An Inroducion o Signal Deecion and Esimaion, Springer, 1994 Chaper 6 / Sochasic Signals and Sysems / Prof. Dr.-Ing. Dieer Kraus 69