Modeling, Estimation and Optimal Filtering in Signal Processing

Transcription

1

2 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng

3

4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Mohmed jm

5 Fst ublshed n Fnce n 6 b emes Scence/Lvose enttled Modélston, estmton et fltge otml en ttement du sgnl Fst ublshed n Get Btn nd the Unted Sttes n 8 b ISE Ltd nd John Wle & Sons, Inc. At fom n f delng fo the uoses of esech o vte stud, o ctcsm o evew, s emtted unde the Coght, Desgns nd Ptents Act 988, ths ublcton m onl be eoduced, stoed o tnsmtted, n n fom o b n mens, wth the o emsson n wtng of the ublshes, o n the cse of eoghc eoducton n ccodnce wth the tems nd lcenses ssued b the CLA. Enques concenng eoducton outsde these tems should be sent to the ublshes t the undementoned ddess: ISE Ltd John Wle & Sons, Inc. 6 Ftzo Sque Rve Steet London W 5DX oboen, J 73 UK USA ISE Ltd, 8 LAVOISIER, 6 he ghts of Mohmed jm to be dentfed s the utho of ths wo hve been sseted b hm n ccodnce wth the Coght, Desgns nd Ptents Act 988. Lb of Congess Ctlogng-n-Publcton Dt jm, Mohmed. [Modélston, estmton et fltge otml en ttement du sgnl. Englsh] Modelng, Estmton nd Otml Flteng n Sgnl Pocessng / Mohmed jm.. cm. Includes bbloghcl efeences nd nde. ISB: Electc fltes, Dgtl.. Sgnl ocessng--dgtl technques. I. tle. K787.F '--dc Btsh Lb Ctlogung-n-Publcton Dt A CIP ecod fo ths boo s vlble fom the Btsh Lb ISB: Pnted nd bound n Get Btn b Anton Rowe Ltd, Chenhm, Wltshe.

6 ble of Contents Pefce... Chte. Pmetc Models..... Intoducton..... Dscete lne models he movng vege MA model he utoegessve AR model Obsevtons on stblt, sttont nd nvetblt AR model cse ARMA model cse he AR model o the ARMA model? Snusodl models he elevnce of the snusodl model Snusodl models Stte sce eesenttons Defntons Stte sce eesenttons bsed on dffeentl equton eesentton Resoluton of the stte equtons Stte equtons fo dscete-tme sstem Some oetes of sstems descbed n the stte sce Intoducton Obsevblt Contollblt Plult of the stte sce eesentton of the sstem Cse : stte sce eesentton of AR ocesses Cse : stte sce eesentton of MA ocesses Cse 3: stte sce eesentton of ARMA ocesses... 36

7 v Modelng, Estmton nd Otml Flteng n Sgnl Pocessng.6.9. Cse 4: stte sce eesentton of nos ocess An AR ocess dstubed b whte nose AR ocess dstubed b coloed nose tself modeled b nothe AR ocess AR ocess dstubed b coloed nose tself modeled b MA ocess Concluson Refeences Chte. Lest Sques Estmton of Pmetes of Lne Models Intoducton Lest sques estmton of AR metes Detemnton o estmton of metes? Recusve estmton of metes Imlementton of the lest sques lgothm he lest sques method wth weghtng fcto A ecusve weghted lest sques estmto Obsevtons on some vnts of the lest sques method he utocoelton method Levnson s lgothm he Dubn-Levnson lgothm Lttce fltes he covnce method Relton between the covnce method nd the lest sques method Effect of whte ddtve nose on the estmton of AR metes A method fo llevtng the bs on the estmton of the AR metes Genelzed lest sques method he etended lest sques method Selectng the ode of the models Refeences...3 Chte 3. Mtched nd Wene Fltes Intoducton Mtched flte Intoducton Mtched flte fo the cse of whte nose Mtched flte fo the cse of coloed nose Fomulton of oblem Phscll unelzble mtched flte...5

8 ble of Contents v A mtched flte soluton usng whtenng technques he Wene flte Intoducton Fomulton of oblem he Wene-of equton Eo clculton n contnuous hscll non-elzble Wene flte Phscll elzble contnuous Wene flte. Rtonl sect cse Dscete-tme Wene flte Fnte mulse esonse FIR Wene flte Infnte mulse esonse IIR Wene flte Alcton of non-cusl dscete Wene flte to seech enhncement Modfed flte eesson Eementl esults Enhncement usng combnton of AR model nd non-cusl Wene flte Refeences...46 Chte 4. Adtve Flteng Intoducton Recusve lest sques lgothm Ect RLS method Fogettng fcto RLS method he lest men sques lgothm Vnts of the LMS lgothm omlzed lest men sques LMS Affne ojecton lgothm APA Summ of the oetes of the dffeent dtve fltes Alcton: nose cncellton Refeences...8 Chte 5. Klmn Flteng Intoducton Devton of the Klmn flte Sttement of oblem Pogton ste: eltonsh between ˆ / nd ˆ / ; ecuence eltonsh between the eo covnce mtces P + / nd P / Udte ste: eltonsh between ˆ / nd ˆ / ; ecusve eltonsh between P / nd P /...89

9 v Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Eesson of the Klmn flte gn Imlementton of the flte he noton of nnovton Devton of the Klmn flte fo coelted ocesses Reltonsh between the Klmn flte nd the lest sques method wth fogettng fcto Alcton of the Klmn flte to mete estmton Estmton of the metes of n AR model Alcton to seech nlss onlne estmton Model lnezton: lnezed Klmn flte he etended Klmn flte EKF Alctons of the EKF Pmete estmton of nos seech sgnl Alcton to tcng fomnt tjectoes of seech sgnls Concluson Refeences... Chte 6. Alcton of the Klmn Flte to Sgnl Enhncement Intoducton Enhncement of seech sgnl dstubed b whte nose Stte sce eesentton of the nos seech sgnl Seech enhncement ocedue Stte of the t dedcted to the sngle-chnnel enhncement methods usng Klmn flteng Altentve methods bsed on ojecton between subsces Intoducton Pelmn obsevtons Relton between subsce-bsed dentfcton methods nd the Klmn lgothm Sgnl edcton usng the otml Klmn flte Klmn flteng nd/o smoothng combned wth subsce dentfcton methods Smulton esults Innovton-bsed oches Intoducton Klmn-flte bsed enhncement wthout dect estmton of vnces Q nd R Klmn-flte bsed enhncement usng subotml gn Altentve och to Klmn-flte bsed enhncement, usng the estmton of vnces Q nd R...44

10 ble of Contents 6.3. Klmn flte-bsed enhncement of sgnl dstubed b coloed nose Concluson Refeences...5 Chte 7. Estmton usng the Instumentl Vble echnque Intoducton Intoducton to the nstumentl vble technque Pncle Revew of estng nstumentl vble methods fo the estmton of AR metes Klmn flteng nd the nstumentl vble method Sgnl estmton usng nos obsevtons Estmton of AR metes usng the flteed sgnl Estmton of the vnces of the dvng ocess nd the obsevton nose Concludng obsevtons Cse stud Pelmn obsevtons Comtve stud. Cse : whte ddtve nose Concluson Refeences...8 Chte 8. Estmton: n Altentve to Klmn Flteng? Intoducton Intoducton to estmton Defnton of the nom flteng Rcct equton-bsed ecusve soluton of flteng Revew of the use of flteng n sgnl ocessng Estmton of AR metes usng flteng flteng fo the estmton of AR metes Dul estmton of the AR ocess nd ts metes Relevnce of flteng to seech enhncement Concluson Refeences...3 Chte 9. Intoducton to Ptcle Flteng Monte Clo methods Sequentl motnce smlng flte...3

11 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng 9.3. Revew of estng tcle flteng technques Refeences Aend A. Khunen Loeve nsfom Aend B. Subsce Decomoston fo Sectl Anlss...34 Aend C. Subsce Decomoston Aled to Seech Enhncement Aend D. Fom AR Pmetes to Lne Sectum P Aend E. Influence of n Addtve Whte ose on the Estmton of AR Pmetes Aend F. he Schu-Cohn Algothm Aend G. he Gdent Method Aend. An Altentve W of Undestndng Klmn Flteng Aend I. Clculton of the Klmn Gn usng the Meh Aoch..373 Aend J. Clculton of the Klmn Gn the Cew nd Belnge Method Aend K. he Unscented Klmn Flte UKF Inde...39

12 Pefce hee s no ol od to scence, nd onl those who do not ded the ftgung clmb of ts stee ths hve chnce of gnng ts lumnous summts. Kl M he coe of ths boo s ten fom sees of lectues I gve n Shngh n 983 nd 985. On ths occson, the Chnese Assocton of Scence nd echnolog CAS gve to me cllghc veson of hlosohe Kl M s fmous quotton. I hve lws found the bove quotton ve etnent nd often used t to e u m students n tmes of dscougement. I would le to thn Pofesso Yong Xng Lu, Pesdent of the Chnese Scences Acdem, who encouged me to use the quotton t the begnnng of ths boo.

13 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng he efomnces of comutes n genel, nd sgnl ocessos n tcul, hve been constntl evolvng towds smlle nd fste vesons wth hghe nd hghe stoge cbltes. hs moved efomnce llows tod s engnee to mlement lgothms tht e eve moe comle. hough ths boo, we wsh to gve the ede the sgnfcnt esults htheto obtned n the feld of metc sgnl modelng, nd to esent some new oches. o the best of ou nowledge, these esults e dsesed though vous tetboos, nd thee s no sngle comendum goung them ll togethe. Moeove, lge t of these esults e onl esented n jounl tcles often nccessble to the vege student. hs boo ttemts to fll ths g. We wll mostl consde sgnl estmton/dentfcton, tdtonll goued n the domn of contol engneeng. od, howeve, ths mete estmton/dentfcton deseves sttue of ts own n vew of ts ecent mtut. One emle whee the deseved motnce ws gven to t s the t-nnul confeences ognzed b the Intentonl Fedeton of Automtc Contol IFAC nd clled the Smosum on Identfcton nd Sstem Pmete Estmton. A ecent tend dung these confeences hs been the ncesng motnce gven to chllenges lng n the feld of sgnl ocessng. Ove the st 5 es o so, dentfcton n sgnl ocessng hs undegone fevent ctvt, even enssnce. One of the fundmentl dffeences between contol engneeng nd sgnl/mge ocessng comes fom the ntue of the nut sgnl. In the fome, the nut s nown whees n the ltte, t s lws unnown. Sevel oblems secfc to sgnl ocessng hve the oots n ths dffeence. Fo emle, the d develoment of dgtl communctons hs led to enewed nteest n the dentfcton of Sngle Inut Multle Outut SIMO nd Multle Inut Multle Outut MIMO sstems. Addtonll, equlzton nd blnd-deconvoluton ssues e lso ncesngl motnt. hs se n nteest s best ttested b the numbe nd qult of elted tcles n the IEEE nsctons on Sgnl Pocessng nd the ICASSP confeences. hs boo s ognzed nto 9 chtes. Chte stts wth bef ntoducton nd evew of the bsc theo of dscete lne models, notbl the AR nd ARMA models. We then nlze the shotcomngs of these models nd esent n ltentve comosed of snusodl models nd ARCOS models to chcteze eodc sgnls. he tem comle hee ncludes both the comutton cost nd the bstcton level nvolved n the ssocted mthemtcl oches.

14 Pefce Once we hve chosen the model nd ts ode, the estmton of ts metes hs to be ddessed. In Chte, we esent the lest sques method nd ts vnts n sgnl ocessng, nmel the utocoelton method nd the covnce method fo the AR model. Fo non-ecusve cse, we then defne the oml o Yule-Wle equtons. heefte, we te u the ecusve foms of the lest sques lgothm nd consde the lowe-comlet lgothms such s the Levnson nd Dubn-Levnson methods. hs ltte toc lso seves s the fmewo n whch we ntoduce the eflecton coeffcents nd the lttce lgothms. oweve, s we wll note n the ote lce, the lest sques method gves bsed estmtons fo coelted mesuements. o get unbsed estmtes, we ntoduce the genelzed lest sques method nd the etended lest sques method. Fnll, we nlze the effect of n ddtve whte mesuement nose on the lest sques estmton of AR metes. We lso esent evew of estng methods used to comenste fo the nfluence of the mesuement nose. In Chtes 3 to 5, we te u mete estmton usng otml fltes nd dtve fltes such s the LMS, RLS nd APA. Fo the dscete-tme cse, we esent the elton lnng the Wene flte to the lest sques method. o ut R. E. Klmn s contbuton nto esectve, we fst esent. Wene s ognl devton [] fo contnuous sgnls, ledng to the Wene of ntegl equton. he non-ecusve ntue of the Wene flte led Klmn to oose n ltentve ecusve soluton, the Klmn flte []. hs ltentve soluton conssted of the tnsfomton of the ntegl equton nto dffeentl stochstc equton, fo whch he then found ecusve soluton. In Chte 5, we deve the Klmn flte usng n lgebc och. Even though ths lgebc och m lc cetn elegnce, t s bsed on fundmentl notons of lne lgeb. hs esentton cn fom the sttng ont fo the nteested ede, ledng hm to the nnovton-bsed esentton of the Klmn flte esented b Klth et l. n the boo Lne Estmton [3]. We then esent the Etended Klmn Flte EKF fo nonlne cses. hs etended flte s useful when we hve to c out the jont estmton of the desed sgnl nd the coesondng model metes ssocted wth t. he EKF s, howeve, not the onl ossble soluton fo nonlne estmton cses. he uose of the followng chtes s to esent othe solutons, tetng cse often seen n sgnl ocessng: when the covnce mtces of the dvng ocess Q nd the nose R e not nown o. hus, n Chte 6, we estte the clssc methods oosed b Cew-Belnge nd R. K. Meh n the domn of contol n the el 97s. he use of sub-sce oches fo dentfcton fees us fom the constnt of hvng nown covnce mtces of Q nd R; ths llows us to see the sgnl enhncement

15 v Modelng, Estmton nd Otml Flteng n Sgnl Pocessng oblem s elzton ssue. We then nlze the elevnce of these oches to enhnce sgnl. As the test sgnl, we choose the seech sgnl becuse t combnes fetues such s qus-eodct n the cse of vowels nd ndomness fo consonnts. In ddton, oches such s Lne Pedctve Codng LPC, ntll deved fo seech sgnls, found wdesed use s genec technque n mn othe es. hs s lso the cse fo the wvelets n the e of sesmc sgnls. Chte 7 concens mete estmton technques usng nstumentl vbles. he e n ltentve to the lest sques methods nd ovde unbsed estmtons. Instumentl vbles technques eque the fomulton of n ntemedte mt whch s constucted, fo emle, usng the sstem nut n the cse of contol. oweve, nfomton on the nut s not vlble n seech ocessng, nd we thus oose n ltentve och bsed on two ntectve Klmn fltes. Moeove, to use the otml Klmn flte, we hve to el on numbe of ssumtons whch cnnot lws be esected n el cses. he flte nown s the flte mes t ossble to el these ssumtons. Moe secfcll, ths concens the ntue of the ndom ocesses nd the necesst of nowng the vnce mtces o. hus, Chte 8 s dedcted to ths flte. We fst ecll the wo done nd esults obtned so f, s concens sgnfcnt lctons n sgnl ocessng, s well s some ecent solutons. We then come the nd -bsed oches n the feld of sgnl enhncement. hs comson wll moeove justf ou decson to gde n ths boo the LMS lgothm n the ctego of otml fltes. Fo ths justfcton, we use the esults obtned t the Stnfod school, wheen t ws shown tht ths flte s -otml. o futhe ese the sttstcl ssumtons, we esent tcle flteng s n ltentve to Klmn flteng n Chte 9. he wo esented n ths boo s the esult of clsses gven n sevel unvestes nd the wo ced out b ou esech gou. I hve been fotunte to she m wshes wth m PhD students nd fellow ofessos. I would le to menton the followng eole fo the contbutons n wtng ths boo: Ec Gvel, who hs been nvolved n ths dventue snce ts outset; wthout hs contnuous vlblt, I would not hve been ble to comlete ths boo; Mcel Gbe, esentl t the Ecole de echnologe Suéeue de Montél n Cnd; nd Dvd Lbe to whom I m ndebted fo the mtel fom hs PhD dssetton, whch he ovded to wte Chtes 8 nd 9. I would lso le to etend m gttude to the followng eole fo the constuctve ctcsm nd

16 Pefce v suggestons dung the wtng of ths boo: Aude Gemus, Pee Blou, Al Zohlgd, col Chstov fom the Unvest of Llle nd Ezo odn fom the Unvest of Bologne. hs boo s the fst n new sees beng lunched b ISE, unde m decton, concenng sgnl ocessng. he fst t of ths boo s sutble s tetboo fo students n the fst e of Mstes ogms. he othe chtes e the esult of ecent wo efomed on the dffeent sects of model mete estmton n elstc scenos. Even though ths second t s the coe of ths boo, t s ccessble to wde edesh, ssumng tht the fundmentl theo s nown. Mohmed AJIM []. Wene, Etolton, Inteolton nd Smoothng t Stton me Sees, Wle nd Sons, ew Yo, 949. [] R. E. Klmn, A new och to lne flteng nd edcton oblems, ASME, Sees D, Jounl of Bss Eng., vol. 8, ges 35-45, Mch 96. [3]. Klth, A. Sed nd B. ssb, Lne Estmton, Pentce ll,.

17 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Mohmed jm Coght 8, ISE Ltd. Chte Pmetc Models.. Intoducton A sgnl coesonds to hscl quntt tht ves wth tme, sce, etc. A wde nge of metes e conveted to electcl sgnls. In n ndustl ocess, fo emle, sensos llow the tnslton of metes such s temetue, essue, lqud nd gs flow tes, etc., nto electcl sgnls. he vton of the suoundng essue due to eson seng though mcohone s tnslted nto n electcl sgnl. Gound essue vtons cn sometmes esult fom contolled events, such s n the cse of tfcl sesmolog when t s used fo ol eloton. oweve, vtons n gound essue could lso esult fom uncontolled events such s ethques. In such cse, sesmoghs ovde electcl sgnls to chcteze the henomenon. Sgnls, the closest omtons of hscl mgntudes, cn be detemnstc o ndom ocesses. If the sgnls e detemnstc, the cn ethe be eodc o non-eodc, o combntons of eodc nd ndom comonents. he sectl content cn be studed usng tnsfomtons such s the Foue tnsfom. Such eesenttons e sd to be non-metc, whethe the e n the tme o fequenc domn. he mjo dvntge comes fom the fct tht the e esl elotble. oweve, the develoment of dgtl ocessos the sed on the one hnd nd, on the othe, the nfluence of dentfcton methods develoed fo the nlss of sstems leds electoncs engnees to use economcl eesenttons of

18 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng sgnls,.e., eesenttons whch use onl fnte numbe of metes, such s the ARMA utoegessve movng vege, AR utoegessve nd MA movng vege models. he AR model s hghl oul. In seech ocessng, sevel codng schemes bsed on the Code-Ected Lne Pedcton CELP, [5] hve been used. hese el on the th o 6 th ode AR model of the seech sgnl Fgues. nd.. Fo seech enhncement, sevel oches hve been develoed. hese wll be dscussed n Chte 6. In seech ecognton, some of the sgnl s chctestcs e etcted usng metc oches. In moble communctons, the Rlegh fdng chnnel hs U-shed sectum Fgue.3. Even though t s bndlmted n fequenc, t s sometmes omted usng st o nd ode utoegessve model, mnl becuse of the smlct of ths model. Bddou et l. hve, moeove, nlzed the elevnce of ve hgh-ode utoegessve models fo chnnel smultons [6] [7]. AR-bsed metc oches lso fnd wdesed use n sectl nlss. Fo emle, whle delng wth bomedcl sgnls to nlze cdoesto sstem consstng of fluctutons n the het bet, esto movements nd blood essue, n AR-bsed sectl nlss of the electocdogm ECG llows the detecton of chnges n the fequenc-domn oetes of the ECG [4]. hus, the oeton of the cdoesto sstem nd ts vtons ove tme cn be studed. Smll, fetl bethng movements cn be studed usng the sgnls ecoded b sees of electodes lced n the mothe s womb []. Fnll, the chctezton of the fequenc esonse of electoencehlogm EEG sgnls usng n AR model could be foeseen fo the detecton of dseses [3]. Fo emle, ths och s used to detemne the bn es esonsble fo eletc sezues [35].

19 Pmetc Models mltude tme mltude feq Fgue.. Plot of n unvoced seech sgnl nd ts owe sectl denst PSD

20 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Fgue.. Plot of voced seech sgnl nd ts PSD

21 Pmetc Models 5 owe sectl denst of Rlegh chnnel nomlzed fequenc Fgue.3. Powe sectl denst of Rlegh fdng chnnel nomlzed mmum Dole fequenc equl to.5 oweve, the ARMA models e onl well-dted fo sgnls wth domnnt ndom comonents. A new model, whch lso contns snusodl comonents, hs to be oosed to ccommodte the fequenc-domn sngultes of sgnls wth stongl eodc segments. In ths chte, we wll fst consde the chctezton of the MA, AR nd ARMA models. heefte, we wll ntoduce snusodl models. Fnll, we wll befl evew the stte sce eesentton of sstems. We wll lso evew the oetes of dscete lne sstems, such s obsevblt nd contollblt. hs eesentton wll be used n Chtes 5 to 8... Dscete lne models he use of models such s ARMA cn be tced bc to the begnnng of the th centu. he ARMA model ws ntoduced b Yule n the 9s fo the stud of tme sees [49]. It s bsed ound the centl de tht tme sees contnng lge mounts of tetve o coelted nfomton cn be obtned b lne-flteng sees of ndeendent ndom vbles. When these vbles e ssumed to be zeo-men Gussn ndom vbles wth constnt vnces, s s mostl the cse, we se of whte Gussn nose. he theoetcl fmewo of these models cn be ttbuted to the mthemtcn Wold t the end of the 93s. Wold s decomoston theoem sttes

22 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng tht n egul nd stton ocess cn be eessed s the sum of two othogonl ocesses, one detemnstc nd the othe ndom [47]. Sgnls cn lso be modeled usng ltentve oches, ncludng those usng eonentl o othe tes of functons [] [34]. Let us consde n nlog sgnl t eesented b + smles coesondng to tme nstnts, s,, s whee s s the smlng eod. Suosng tht ths sgnl s geneted b usng whte ocess, denoted u nd chctezed b q+ smles: u u u q A dscete lne model of the sgnl cn be defned s lne combnton between the smles n n,... nd u n n,..., q, whch cn be eessed s follows: b u b u [.] q q hs mes u the ARMA model, whch s sd to be of the ode, q, whee,..., nd b,..., q e clled the tnsvesl metes. Equton [.] s ttctve becuse, nsted of svng n nfnte numbe of smles necess fo eesentng the sgnl, t onl uses fnte numbe of chctestc metes nd llows fo the econsttuton of the sgnl usng ths fnte numbe. he condtons fo ths econsttuton wll be dscussed lte. In the est of ths chte, we wll use the conventon wheeb =. Equton [.] s thus chnged to: A ocess s sd to be utoegessve movng vege wth eogenous nut ARMAX f t s defned s follows: b u b u b u q c v c v c v whee the dvng ocess u s zeo-men whte, nd v s n eogenous nut led to the sstem. hs model s often used n felds such s contol engneeng nd econometcs. q

23 Pmetc Models 7 q u b [.] he bltel z-tnsfom, Yz, of the sequence s defned b: z z Y [.3] whee z s comle vble ncluded n the convegence-domn of the sees. otng tht the z-tnsfom of l esects: z Y z z l l fo ll ntege vlues of l, equton [.] of the ARMA models, usng the z-tnsfomton of ts two consttuent elements, becomes: z U z b z U b z Y z z Y q q he tnsfe functon of the ARMA model s thus gven s follows: z z A z B z z z b z b b z U z Y q q. [.4] z s nown s the tnsfe functon. Fgue.4. nsfe functon of the ARMA model he ARMA model cn thus be undestood s flte wth tnsfe functon z. hs flte s fed wth n nut u whose z-tnsfom s denoted Uz, nd t delves n outut sgnl whose z-tnsfom s denoted Yz. he olnomls Az nd Bz e chctezed b the locton of the zeos n the z-lne. he zeos of Bz e the sme s the zeos of z whle the zeos of Az e the oles of z Fgue.5.

24 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng mgn t. -. zeos oles el t owe sectl denst of the ARMA ocess nomlzed fequenc Fgue.5. Emle of n ARMA ocess: the locton of the zeos nd oles, nd the owe sectl denst of the coesondng ocess At ths stge, the followng questons se: wht s the mnmum numbe of metes,..., nd b,..., q needed to stsfctol eesent the sgnl? f we consde dffeent elztons of the sgnl, do metes,..., nd b,..., q emn the sme? s t ossble to obtn model n whch some of the metes,..., b,..., q e zeo? how cn the ode, q of the model be detemned? o

25 Pmetc Models 9 o bette undestnd the nfluence of these metes, we wll fst te u two secfc cses of the ARMA model: the models nown s MA nd AR.... he movng vege MA model Fo the MA model, we wll suose tht ll the metes,..., e zeo, ecet =. Model [.] s thus nmed the movng vege model, nd s eessed sml s follows: b u b u bqu [.5] q Snce Az =, ths model s chctezed b the locton of ts zeos n the z-lne, gvng se to the nme of the ll-zeo model Fgue.6. q z b b z b q z [.6] Fgue.6. nsfe functon of the MA model he MA model cn lso be nteeted s the outut of Fnte Imulse Resonse FIR flte ected b n nut sgnl u. z s the tnsfe functon of ths flte. Fgues.7 nd.8 show the she of the owe sectl denst of MA ocess chctezed b the two followng comle conjugte zeos wthn the unt ccle n the z-lne: nd: z R e j R e j f s f z R e f [.7] j R e j f s In the followng, we wll ttenton to the evoluton of the she sectum gven b MA model s ts zeos move close to the unt ccle n the z-lne.

26 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Gven the oetes of u, the bove equton entls stud of the sque of f the modulus of z wth z e j. Fst, we set the nomlzed ngul f s fequenc to / 3 nd v the mgntude of the zeos fom.9 nd.99 n stes of.75. he modfcton n the locton of the zeos n the z-lne cn thus be obseved, s cn the sque of z mgn t ncesng zeo modulus el t tnsfe functon modulus n db ncesng zeo modulus nomlzed fequenc f Fgue.7. Reesentton of z wth z e j, fo zeos of z wth the sme f s gument nd dffeent mgntudes. MA ocess

27 Pmetc Models If the modulus of the zeos s et constnt t.95 nd f ves fom / 6 to 5 / 6 n stes of / 6, we cn obseve how the mnm of z chnge Fgue f Fgue.8. Reesentton of z wth z e j, fo zeos of z wth the sme f s mgntude nd dffeent guments. MA ocess... he utoegessve AR model When the tnsvesl metes b,..., q e ll zeo, ecet b =, the model s clled the utoegessve model nd s eessed sml s follows: u [.8] In equton [.4], the olnoml Bz educes to constnt vlue of Bz =, nd z contns onl oles, gvng se to the nme ll-ole model. he tnsfe functon z cn be wtten s follows, to hghlght the oles,..., z : z [.9] z

28 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng he AR model cn be thought of n the followng ws: the tem n equton [.8] cn be seen s edcton of bsed on the lst vlues of the ocess, nd u s n eo tem n such edcton; equton [.8] cn tself be seen s dffeence equton nd flteng of sgnl u b n Infnte Imulse Resonse IIR flte. Fgue.9. nsfe functon of the AR model he locton of the oles n the z-lne comletel defnes the flte nd, consequentl, the model ssocted wth ths flte. If we suose tht the dvng ocess u nd the sgnl e el, the oles,..., e ethe el o comle conjugtes snce ths s the onl cse n whch the coeffcents,..., of equton [.8] e el. Let us then consde second-ode AR model n whch u s el zeo-men Gussn whte ocess wth vnce of. If the tnsfe functon llows fo two comle conjugte oles n the z-lne, we cn wte: nd: R e j R e j f s j R e j f s f f [.] * R e he Foue tnsfom Yf of wll then be equl to the oduct of Uf nd f z t z e j, whee Uf s the Foue tnsfom of u,.e.: f s

29 Pmetc Models 3 s f s f j f f j f U f U f f Y e e * [.] Equvlentl: s f s f f j R f f f j R f U f Y e e [.] he owe sectl denst of deends on the vlues of R nd the nomlzed ngul fequenc s f f, nd could show moe o less sh esonnce. hs owe sectl denst wll hve the sme fom s the sque of the mgntude of the coesondng z wth f s f j z e. o llustte ths ont, let us fst f the nomlzed ngul fequenc to 3 /, nd let us v R fom to.99 n stes of.75. he evoluton of the locton of the dffeent oles n the z-lne, nd of z n such cse s shown n Fgue.. We notce tht two esonnces e, t nomlzed fequences of 6 / /. hese esonnces shen s R oches the vlue.

30 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng mgn t ncesng ole modulus el t 4 tnsfe functon modulus n db ncesng ole modulus nomlzed fequenc f Fgue.. Reesentton of z wth z e j, fo oles of z wth the sme f s gument nd dffeent mgntudes. AR ocess If R s mntned constnt t.95 nd f ves fom / 6 to 5 / 6 n stes of / 6, we cn see tht the esonnces e successvel t nomlzed fequences whch e multles of / 6 Fgue..

31 Pmetc Models mgn t chngng the of the comle conjugte oles el t 3 tnsfe functon modulus n db chngng the of the comle conjugte oles nomlzed fequenc f Fgue.. Reesentton of z wth z e j, fo oles of z wth the sme f s mgntude nd dffeent guments. AR ocess Moeove, the vnce of the dvng whte Gussn ocess u hs n effect on the owe sectl denst of the AR ocess : t undegoes n uwd shft s the vnce of u goes fom to Fgue..

32 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Fgue.. Effect of the vnce of the dvng ocess on the PSD of the AR ocess. Reesentton of the men sectl vlue nd the PSD.3. Obsevtons on stblt, sttont nd nvetblt Stblt nd cuslt consdetons e mosed on sstems due to ctcl constnts. A sstem s sd to be stble n the bounded-nut bounded-outut BIBO sense f thee s bounded outut fo eve bounded nut. A lne nvnt sstem s thus sd to be stble f nd onl f ts mulse esonse h esects the followng condton: h [.3] he tnsfe functon s the z tnsfom of ths mulse esonse; nd thus, fo eve z n the egon of convegence: z h z h z [.4]

33 Pmetc Models 7 oweve, on the unt ccle n the z-lne: thus gvng se to: h z h [.5] z f ze j f s [.6] Sevel stblt cte hve been oosed to stud nd chcteze stblt, such s the cte of Routh, Ju, Schu-Cohn, etc. Moeove, sstem s sd to be cusl f ts esonse neve ecedes ts nut. hs coesonds to the hlosohcl de of cuslt: cuse ecedes consequence. Fo cusl sstem, the necess nd suffcent condton fo stblt s tht ll the oles of the tnsfe functon be nsde the unt ccle wthn the z-lne..3.. AR model cse he smlest cse s tht of el fst-ode AR model defned s follows: u [.7] whee nut u s zeo-men el whte sequence wth vnce u. he utocoelton functons of nd u e nmed, esectvel, E n n nd uu Eu n u n whee E. denotes the mthemtcl eectton. uu u It s cle tht. Moeove, t s lso nown tht utocoelton s n even functon becuse ocess s el. Sttng wth the cse whee >, smmet llows us to deduce the cse whee <. Consdeng equton [.7], we obtn, on the one hnd: E n n E n un n un u [.8]

34 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng wth: u f. [.9] On the othe hnd: E n n E n un n [.] Gven eltons [.9] nd [.], we cn show b ecuence tht fo ll > : u [.] heefoe, fo ll vlues of u [.] Let us eess s functon of uu. o do so, we tnsfom the AR model nto MA model b eessng the smles of sgnl of equton [.7] n ecusve mnne. hus: nd: u u 3 u u u Fnll: u u u u 3 u 4 [.3] Usng equton [.3] bove: uu u [.4] 4

35 Pmetc Models 9 nd combnng equtons [.4] nd [.9], we cn deduce tht: [.5] hs mles tht <. hs s the cteon tht ssues the stblt of the model. Usng the z tnsfom of the model n equton [.7], we obtn: Y z U z z [.6] hs leds to the sme concluson,.e., <. All the oles should thus be nsde the unt ccle n the z-lne..3.. ARMA model cse As befoe, the ARMA model cn be elced b n AR model b vefng the eqult elton: B z. [.7] A z A' z he nvese of the model follows: A z A' z s obtned b olnoml dvson s B z A' z z z he followng obsevtons cn be mde: stble ARMA model does not necessl ml n nvetble model. Stblt n ths cse sml mens tht the zeos of Az e nsde the unt ccle n the z-lne. Invesblt eques the fulflment of nothe condton: the zeos of B z should lso be nsde the unt ccle; the wde-sense sttont of ndom ocess sgnfes tht the men vlue of the ocess s constnt ove tme. It lso mens tht the utocoelton functon deends onl on the eltve tme shft between the smles of the sgnl, nd not on the ognl ont fom whch ths shft s consdeed.

36 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng.4. he AR model o the ARMA model? Even though most n the sgnl ocessng communt efe to del wth the AR model, t s not es to ovde defnte nswe to ths queston. hs s tl due to the dffcult n estmtng the b,..., q metes n the cse of the ARMA models. o detemne these metes, the nut u would lso hve to be detemned. oweve, u s not necessl es to ccess o nlze n the cse of sgnl ocessng. In the cse of dentfcton n contol, the nut s ssumed to be o nown. he efomnces of the two models hve lso been comed n el-wold stutons nd lctons. Fo emle, fo EEG sgnls, Bohln dew the followng conclusons fo the ARMA model [] []: odes, q e not es to detemne; the eos entled n oundng off e hghe n the ARMA model; nd thee s el s of obtnng unstble models. he esults fo sectl nlss e, howeve, comble to the AR model. he nsuffcenc of the AR model hs been evoed b sevel uthos, notbl fo the cse of seech nlss. hese uthos hve oosed modfed ARMA models nd elted technques fo the estmton of the metes [36]. oweve, the hgh comlet of these modfed models does not lws do justce to the eected sgnl qult, both fo tnsmsson nd snthess. In the cse of seech snthess, the most notble se n the qult undoubtedl comes fom the mult-ulse ectton [5]. he lmttons of the Lne Pedcton Codng LPC technque fo seech snthess se fom the LPC model beng smle descton of the electcl sgnl nd not of the vocl tct. he model s,..., metes hve no smle o dect elton to the hscl metes of the vocl tct. A lttle futhe on, we wll consde modelng b fomnts.e., the esonnces of the vocl tct nd the tcng usng nonlne dentfcton technques. hs n no w detcts fom the dvntges of the LPC technque, whch hs been successfull led n mn domns, notbl n the snthess of sevel lnguges [38]. It ws mentoned bove tht the bsc tfll of the AR model esults fom the dffcult encounteed n modelng stong eodctes such s voced sounds n seech, eods of slee n EEGs, d sgnls, etc. In these cses, good esults hve been obtned usng models whch contn eodc tems. One such clss of models s the snusodl models, whch wll now be esented.

37 Pmetc Models.5. Snusodl models.5.. he elevnce of the snusodl model Usng the lne coustc theo of seech oducton develoed n the 96s s sttng ont, we cn choose n utoegessve modelng of th -ode fo the seech sgnl: u [.8] hs souce flte model coesonds to whte nut souce u ocessed b flte wth n mulse esonse h nd the followng tnsfe functon: z h z [.9] z In ctcl cses, s dscussed befoe, the vlue les between nd 6 fo the modelng of the fst fomnts of the seech sgnl. he noton of shot tem edcto s elevnt: cn be eessed s functon of lne edcton bsed on the lst smles of the seech sgnl. Unde these condtons, howeve, the sectum of the voced sounds such s vowels esents hmonc chcte whch s dffcult to quntf. One ossble soluton would be to ncese the AR model ode, but ths s offset becuse the m hee s to comess the seech. In ths cse, we nlze the seech sgnl fme b fme. hese fmes do not lst longe thn bout 3 ms, sgnfng ound 5 smles fo sgnl smled t 8 z. o mtgte ths oblem, one soluton could be n och wheen the dvng ocess s Dc comb wth eod = /f. Whle ocessng seech, f coesonds to the fundmentl fequenc whch cn v n the nge 8 z to 6 z. he nut nd the flteng contbute sgnfcntl to the sectum of the sgnl unde stud s: h u * [.3]

38 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng We obtn: f f Uf Y [.3] B tng n U f f, the desed eodct s ntoduced. n Fgue.3. he souce flte model: utoegessve och As the Dc comb s dffcult to mlement, we cn suose tht the nut stsfes the followng eltonsh: u u q v [.3] whee tends towds, v s whte Gussn sequence nd q s such tht q s. hs elton cn be thought of s long tem edcto snce q >>. he bove consdetons gve se to Fgue.4. hs modelng scheme s used fo seech codng n moble hones, wth the code-ected lne edcton codes CELP codes [5]. In the feld of seech enhncement, Z. Goh et l. hve lso ut fowd the de of usng eesentton wheen the nut s djusted ccodng to the fme beng nlzed [] []. An ltentve och conssts of the dect use of snusodl models. Ove the st es o so, the hve been used fo nlss, snthess, low te codng nd hgh qult osodc tnsfomtons of seech sgnls. he lso fnd lctons n the ocessng of muscl nd d sgnls s well s n communctons fo the modulton of the dt to be tnsmtted.

39 Pmetc Models 3 Fgue.4. he souce flte model: utoegessve och We now detl some estng snusodl models. Moe secfcll, we wll dscuss the cse whee the sgnl s eesented ethe s sum of comle eonentls o s sum of snusods..5.. Snusodl models he smlest eesentton of eodc sgnl conssts of sgnl whch s sum of L comle eonentls: L f e j [.33] f s hs We cn lso consde moe genel model wth dmng fctos. leds to sum of L dmed comle eonentls: L f e e j [.34] f s f s he use of ths model fo sectl nlss foms the coe of the Pon method [46].

40 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Fo ndom sgnl wth mltude tems hvng the fom: j e, the model of equton [.33] cn be ugmented b n ddtonl zeo-men tem b whch s genell whte. hs leds to modelng wth two comonents n ts sectum: one dscete nd the othe contnuous. L f e j b [.35] fs he model esented n equton [.35] s the coe of hgh esoluton sectum nlss technques such s Pseno s monc Decomoston, MUSIC Multle Sgnl Clssfcton nd ESPRI Estmton of Sgnl Pmetes v Rottonl Invnce echnques. hese technques e bsed on the nlss of the utocoelton mtces of the obseved dt, nmel, the seton of the domnnt egenvlues whch chcteze the sgnl subsce fom the smllest ones whch coesond to the vnce of the nose b. he sgnl s fequenc ofle s found b mng use of the othogonlt between the two subsces. oweve, t s not lws stghtfowd to fnd the domnnt egenvlues, esecll when the sgnl hs ch sectl content [44]. Fo futhe detls, see Aendces A nd B. he model n equton [.35] s lso used to eteve seech sgnls fom nos obsevtons [6] [7] [8] [9] [4] [5] [6] [48]. Aend C esents moe detls on ths toc. Snce t would be beond the m nd scoe of ths chte to estmte the metes of snusodl models, we efe the ede to Chstensen nd L s contbutons fo futhe detls [4] [3]. In the el 98s, McAul nd Qute ntoduced low-te sgnl codng bsed on n nlss-snthess scheme [33]. In t, the seech sgnl s modeled s sum of L snusodl comonents: L L f cos cos [.36] fs

41 ,, Pmetc Models 5 ee, nd f denote, esectvel, the nstntneous mltude, the nstntneous hse, the ognl hse nd the fequenc of the th comonent of the model. hs equton foms the sttng ont of sevel othe oches to modelng sgnls wth stong eodctes [] [37]. he model n [.36] cn be enched b ddng nose b to the obsevton: L L cos b f f cos b s [.37] Wthn the domn of nlss/snthess of muscl sgnls, ths model s nown s the sgnl + nose model [4]. It hs lso been used b Stlnou [4] [43] fo the nlss, snthess nd enhncement of seech sgnls. Amltude s unnown nd hence cn be modeled b n utoegessve ocess of ode :, l u [.38] l l whee u s whte zeo-men Gussn dvng ocess wth vnce nvese flte ssocted wth ech fequenc bnd s defned s: z l z l, z l, e j l, l l u. he [.39] whee f l,. f s When the numbe of fequenc bnds equls, ths model s nown s utoegessve mltude-modulted cosnusod ARCOS, nd s used n d lctons [9].

42 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng he sectum hs she sml to tht fo n ARMA model of ode,. he utocoelton functons R nd R coesondng, esectvel, to sgnl nd mltude, stsf the followng elton: f [.4] R cos f s R Fgue.5. Plot of n ARCOS ocess, nd ts coesondng sectum n db

43 Pmetc Models 7 ng the Foue tnsfom of equton [.4] gves: S [.4] 4 f S f f S f f oweve, consdeng equton [.38], the PSD cn be eessed usng nd the coeffcents elted to the nvese edcton coeffcents l, l,..., flte: S f * z f z z j / e f f z e s j f s [.4] Consequentl, we obtn the owe sectl denst S f of the sgnl s: f S f f B [.43] A Afte the esentton of the dffeent models, we wll ntoduce the stte sce eesentton whch wll be used lte n Chtes 5 to Stte sce eesenttons.6.. Defntons In tdtonl ccut nd sstem nlss, dffeentl equtons e used to eesent the sstem. hs te of eesentton dectl tnsltes the behvo of the sstem unde stud nd cn lso be conveted nto eesentton usng tnsfe functon, wthout necessl efect equvlence between the tme nd fequenc domn desctons. A stton sstem cn be descbed usng convoluton equton, whch s efectl equvlent to eesentton usng tnsfe functon becuse the tnsfe functon s the Llce tnsfom of the convoluton equton. he smlct of these eesenttons hs mde them hghl oul. evetheless, the e ll-dted to tnsto o non-stton cses. Moeove, eesentng sstem n stte sce llows us to beneft fom the mn esults on lne lgeb cuentl vlble, nd to esl dentf the slent oetes.

44 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Fstl, let us defne t s column vecto of dmenson contnng the comonents t,..., : t t. t [.44] t s clled the stte vecto, whch s the smllest of the comonents t,, t tht llows us to detemne the sstem t t > t fo n ntl vlue t nd nown nut ut. Othewse eessed, the stte of sstem s collecton of suffcent nfomton whch llows us to detemne the evoluton of the sstem f the nuts e nown. A sstem s sd to be descbed usng stte sce f t s govened b the followng dffeentl equton: wth: t A t t B t u t [.45] d t t [.46] dt he stte vecto t s not dectl ccessble. Genell, onl mesuements e vlble. he stsf: t t t. [.47] Fo the modelng to be elstc, t s necess to dd nose tem to equton [.47]..6.. Stte sce eesenttons bsed on dffeentl equton eesentton Let us consde lne sstem govened b the followng dffeentl equton: whee: t t t bu t [.48] d t t [.49] dt

45 Pmetc Models 9 Equtons [.48]-[.49] llow us to choose sevel ossble stte sce eesenttons. Fo emle, let us constuct stte vecto t n whch the fst comonent s t nd the - th comonent s the - th devtve of t: hus: t t t t t [.5] t t t t t t nd t u t We cn estte [.49] s follows: u [.5] t A t t B t u t, [.45] b defnng A nd B s follows: A nd B b [.5] Consdeng [.47], the obsevton mt ssumes the followng fom: [.53] Stte sce eesenttons hve been the subject of much esech effot. he followng e consdeed clssc efeences n the feld [] [3] [8] [3] [8] [3] [39] [4] [5]. ote: sstem s sd to be stton f mtces A, B nd e tme-nvnt Resoluton of the stte equtons Fo the cse of stton sstem, equtons [.45] nd [.47] cn be efomulted s follows:

46 3 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng t A t Bu t [.45] t t [.47] t stsfes the followng homogenous dffeentl equton: t A t [.54] he soluton of equton [.54] s thus gven b: A t t t e whee: t he genel soluton fo sstem wth nut ut, t t > t, s gven b: A t t ea t t ea t Bu d t e [.55] he tem t t t t. t t e A s clled the tnston mt nd s lso denoted.6.4. Stte equtons fo dscete-tme sstem Sttng fom the followng equtons fo contnuous sstems: t A t t B t u t [.45] t t t [.47] Fo dscete-tme sstems, we cn: ethe fomll wte the stte equtons s follows: G u [.56] v [.57] o dscetze the contnuous equtons.

47 Pmetc Models 3 o do the ltte, the followng omted elton s used to clculte the devtve of the stte vecto: s s s s, [.58] whee s s the smlng eod. It cn lso be smlfed s follows: [.59] s Equton [.45] s chnged to: A Bu s [.6] Rengng the tems of the bove equton, we obtn the followng fom: I A B u [.6] whee I s the dentt mt. s Comng equton [.6] wth equton [.56] gves: s I A nd G B [.6] s s hs esult s not uneected: t ws shown n secton.6.3 tht the tnston mt too the followng fom: t t e A t [.63] t Substtutng t = + s nd t = s, ths s chnged to: e A, [.64] s s oweve, f we consde the enson of the eonentl functon: As e A s I As [.65]

48 3 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng nd omte ths sees to be lmted to the st -ode tem, we obtn equton [.6] Some oetes of sstems descbed n the stte sce Intoducton Vous studes hve been conducted fo the chctezton of sstems eesented n the stte sce. wo of these e the notons of obsevblt nd contollblt. Obsevblt s motnt fo the modelng of sgnls nd contollblt chctezes sstems we wsh to contol Obsevblt Obsevblt stnds fo the ossblt of fndng out the stte of sstem b studng ts oututs. It cn be eessed n numbe of ws. he smlest defnton s: sstem s sd to be obsevble f mesuement of ts outut llows the detemnton of ts ntl stte. In the dscussons tht follow, we wll estblsh the condtons of obsevblt of such sstems. ng nose-fee obsevton, fo the dscete stton cse: [.66] he vton of the stte vecto s gven b: [.67] Fo ll,, cn be eessed s functon of the tnston mt, of the obsevton mt nd of the ntl stte of the stte vecto. In mtcl fom, the bove equton s denoted s follows: [.68]

49 Pmetc Models 33 Y [.69] wth the obsevblt mt gven b: [.7] In ode to detemne the ntl stte fom the obsevton vecto Y, the obsevblt mt should be non-sngul. heefoe, the condton of obsevblt cn be ut foth: fo sstem to be obsevble, the obsevblt mt must be of full n. hs noton, fst oosed b Klmn, s closel lned to the nveson nd deconvoluton of sstems [9] Contollblt he contollblt of sstems denotes the ossblt tht b mosng tcul nut, the sstem cn be dven fom n ntl stte to desed fnl stte wthn fnte duton of tme. Such oet s useful n contol engneeng, fo contol ocedues, gudng cft o bllstc devces, etc. Contollblt of sstems mght e to be of lmted nteest n sgnl nd mge ocessng, felds whose mjo ms e modelng nd dentfcton, nd whee the nut s lmost neve ccessble. evetheless, we wll descbe the condtons, n tems of nd G, n whch dscete-model sstem, govened b equtons [.45] nd [.47], must stsf the followng condton: the contollblt mt G G G should hve n Plult of the stte sce eesentton of the sstem Let us consde the followng equtons: G u [.56] v [.57]

50 34 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng A sstem cn hve n nfnte numbe of eesenttons, nd we cn coss ove fom one to nothe usng non-sngul tnsfomton. hs mles bsc chnge n the stte sce: v Gu [.7] Ecet fo the non-sngul tnsfomton, the tlet [, G, ] s sml to:, G G, [.7] A numbe of eesenttons, ll chctezed b the tlet, G nd, e thus ossble. he mjo eesenttons mong these e the followng: Jodn s eesentton; contollble eesentton; obsevble eesentton. Dffeent foms cn lso be doted to hghlght dffeent hscl metes of the ocess beng studed o to dentf metes whch l secl ole n the tcul lcton [3]. All these foms e of nteest n the feld of contol engneeng Cse : stte sce eesentton of AR ocesses Let us stt wth n AR model of ode : u [.73] We cn defne stte vecto b conctentng the lst vlues of the ocess, these vlues beng denoted b the stte vbles. = - + [.74]

51 Pmetc Models 35 he stte vecto cn be udted to: Gu [.75] whee tnston mt hs the followng fom: [.76] nd nut weght vecto G s defned s: G. [.77].6.7. Cse : stte sce eesentton of MA ocesses Let us stt wth th -ode MA model: u b u b [.78] o smlf the stte sce eesentton of ths sstem, we cn ntoduce the followng ocesses: j u b j j j,...,,. [.79] We see tht on the one hnd: u b u b l u b l u b l j l j [.8]

52 36 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng nd, on the othe hnd, fo ll,..., j, we hve: u b u b l u b u b u b u b j j j j l l j l j j j j j j [.8] he stte vecto fo the MA ocess cn be wtten s follows: [.8] Udtng ths stte vecto gves: u b b b [.83] nd: u b [.84].6.8. Cse 3: stte sce eesentton of ARMA ocesses Let us consde n ARMA model of ode, q nd stt wth the smlfcton q : u b u b [.85] o eesent ths model n the stte sce, we wll te u n obsevble fom ote to the estmton of the model s metes.

53 Pmetc Models 37 Followng elton [.], we cn wte b ntoducng ddtonl oututs denoted,,,. hese oututs e defned s follows: u b [.86] whee the fst ddtonl outut esects the followng condton: u b u b u b [.87] Consdeng equton [.86] t the nstnt -, equton [.87] s modfed to: u b b u b u b [.88] Smll, we cn defne the second ddtonl outut s follows: 3 u b u b u b [.89] Consdeng equton [.86] t the nstnt -, equton [.87] s modfed to: 3 3 u b b u b u b [.9] nd so on, to the th nstnt: 3 u b b [.9] All the othe ddtonl oututs cn be deduced n sml mnne: u b b u b [.9] hus, t nstnt : u b b [.93] Wtng the bove equtons n mtcl fom, oututs e to be the elements of the followng stte vecto t nstnt :

54 38 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng, hese elements e eessed s functon of the nut u. Fnll, we cn wte ll the bove equtons n the fom of stte equtons: u b b b b b b b b [.94] Equton [.86] becomes: u b u b [.95] We note tht equtons [.83] nd [.84] e the sme s [.94] nd [.95] wth,..., =. he stte equtons cn lso be consdeed s eesentton of dgtl flte wth nut u nd outut [7] [45]. hs s clled the obsevble fom of the equtons, wth the obsevblt mt beng non-sngul fo. he eesentton of the ARMA model [.] s thus defned b: Bu J [.96] u b [.97] wth J,, b b B

55 Pmetc Models 39 nd:. [.98] Mt J hs the followng oetes:. [.99] J. J [.].6.9. Cse 4: stte sce eesentton of nos ocess Fom Chte 5 onwds, we wll te u the modelng of sgnl dstubed b ddtve nose, s well s the eesentton of the coesondng sstem n the stte sce. We wll lso mlement technques fo the estmton of the stte vecto usng vlble nos obsevtons. o set the stge fo ths, we wll esent some ddtonl cses An AR ocess dstubed b whte nose Let us fst consde th -ode AR ocess, denoted s, whch s dstubed b n ddtve zeo-men whte nose b wth vnce R. he obsevton s defned s follows: s b [.] he stte vecto s then defned s follows: = s - + s. [.] he udtng equton of the stte vecto s the sme s n [.75], whle the obsevton equton s: b [.3]

56 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng whee: G [.4] AR ocess dstubed b coloed nose tself modeled b nothe AR ocess We sometmes hve to ocess sgnls dstubed b coloed noses. Fst, let us model ths ddtve coloed nose s q th -ode AR ocess. b q j c b j w j [.5] ee, the sequence w s whte, zeo-men nd Gussn, wth vnce of W. It s ndeendent of the ocess u whch genetes the sgnl s. he stte vecto s then comosed of the lst vlues of the sgnl nd the q lst vlues of the ddtve nose: whee: nd: whee: = s v [.6] s = s - + s [.7] b = b - q + b. [.8] he vlble obsevton s thus: s b [.9] he use of such stte sce eesentton wll be llustted n Chte 6, fo the enhncement of sgnl dstubed b coloed nose.

57 Pmetc Models 4 q [.] he equton udtng the stte vecto cn be eessed s follows: G. [.] whee the tnston mt s ttoned s follows: v s [.] wth: s [.3] nd: c c c q q v. [.4] Mt G s defned b: G [.5] he nose vecto s constucted s follows: w u = [.6]

58 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng A sstem, defned b sgnl contmnted wth n AR coloed nose, s sd to be eesented n the stte sce b efect mesuement eesentton. hs nme ses fom the fct tht the nose does not e elctl n elton [.9]. In ths cse, the obsevton s lne combnton of stte vbles. hee e no othe tems n the obsevton equton of the stte sce eesentton AR ocess dstubed b coloed nose tself modeled b MA ocess 3 Let us suose tht the utoegessve sgnl s s dstubed b nose b modeled b q th -ode MA ocess s follows: q w c b [.7] whee q c...,, e the MA metes nd w s zeo-men whte Gussn nose wth vnce w. As n secton.6.7, we ntoduce the followng tems: q j w c j q j j,...,, [.8] Usng equtons [.79], [.] nd [.7], we cn wte:. w c s w c w c s w c s q q [.9] Let us defne the ocess v = c w s zeo-men whte Gussn nose wth vnce w v c R. We cn ewte equton [.9] s follows: v s [.] We cn defne the stte vecto s follows: 3 he use of such stte sce eesentton wll be llustted n Chtes 6 nd 7, fo the enhncement of sgnl dstubed b coloed nose.

59 Pmetc Models 43 q s s, [.] he stte sce eesentton coesondng to the sstem n equtons [.79], [.] nd [.7] cn be wtten s: v Gu [.] Fnll, the mtces coesondng to ths stte sce eesentton of equton [.] e defned s:,, q O q O [.3] n q O O G,, [.4] q [.5] wth n c q c. Indeendentl of the vlues of the ostve nteges m nd n, m m s ow vecto wth m- zeos nd Om,n s zeo mt of dmensons n m. Moeove, the dvng ocess vecto u s defned b: w u u [.6]

60 44 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng u hs s zeo-men Gussn vecto wth covnce mt Q. w It should be noted tht ths stte eesentton s no longe efect mesuement eesentton..7. Concluson hs chte hs med t esentng two mjo fmles of models. One of these comses the AR, MA nd ARMA models nd the othe fml s comosed of snusodl models nd the vous foms whch encomss wde nge of sgnls. hese models e used n vet of lctons, fom bomedcl to seech ocessng nd moble communctons. In the net chte, we wll te u the ssue of estmtng the AR metes usng lest sques methods..8. Refeences [] D. Al nd I. Gohbeg Edtos, he Stte Sce Method Genelztons nd Alctons, Lne oetos nd lne sstems, Oeto heo Advnces nd Alctons, vol. 6, Bhüse Velg,. [] M.. Ansoun, J.. Ds, G. J. Bette nd K. Bodd, Autoegessve Sectl Estmton of Fetl Bethng Movement, IEEE ns. on Bomedcl Engneeng, vol. 36, no., , ovembe 989. [3] M. Ao, Stte Sce Modelng of me Sees, Snge-Velg, 987. [4] M. Anold, W.. R. Mltne,. Wtte, R. Bue nd C. Bun, Adtve AR Modellng of onstton me Sees b Mens of Klmn Flteng, IEEE ns. on Bomedcl Engneeng, vol. 45, no. 5, , M 998. [5] B. Atl nd J. Remde, A ew Model fo LPC Ectton fo Poducng tul Soundng Seech t Low Bt Rte, IEEE-ICASSP 8, Ps, Fnce, , 3-5 M 98. [6] K. E. Bddou nd. C. Beuleu, Autoegessve Models fo Fdng Chnnel Smulton, IEEE-GLOBECOM,. 87-9, ov.. [7] K. E. Bddou nd. C. Beuleu, Autoegessve Modelng fo Fdng Chnnel Smulton, IEEE ns. on Weless Commun., vol. 4, no.. 4, , Jul 5. [8] A.V. Blshn, Elements of Stte Sce heo of Sstems, Otmzton Softwe, Inc. Publctons Dvson, ew Yo, 983.

61 Pmetc Models 45 [9] O. Besson nd P. Stoc, Snusodl Sgnls wth Rndom Amltude: Lest Sques Estmtos nd the Sttstcl Anlss, IEEE ns. on Sgnl Pocessng, vol. 43, no., ovembe 995. []. Bohln, Anlss of Stton EEG Sgnls b the Mmum Lelhood nd Genelzed Lest Sques Methods, ehncl e P 8. Sstems Develoment Dvson IBM odc Lboto Sweden. []. Bohln, Comson of wo Methods of Modelng Stton EEG Sgnls IBM J. Res. Dev., vol. 7,. 94, 973. [] J. A. Cdzow nd h.. wng, Sgnl Reesentton: An Effcent Pocedue, IEEE ns. on Acoustcs Seech nd Sgnl Pocessng, vol. ASSP-5, no. 6, , 977. [3] C.. Chen, Intoducton to Lne Sstem heo, olt Rneht Wnston, ew Yo, 97. [4] M. G. Chstensen nd S.. Jensen, On Pecetul Dstoton Mnmzton nd onlne Lest-Sques Fequenc Estmton, IEEE ns. on Audo, Seech nd Lnguge Pocessng, vol. 4, no., Jnu 6. [5] Codng of seech t 8bs usng conjugte-stuctue lgebc code-ected lne edcton; Codge de l Pole à 8 bt/s Pédcton Lnée vec Ectton Séquences Codées à Stuctue Algébque Conjuguée, IU- Recommendton G.79, 996. [6] M. Dendnos, S. Bmds nd G. Cnns, Seech Enhncement fom ose: Regenetve fom ose, Seech Communctons, vol., no., , Febu 99. [7] S. Doclo nd M. Moonen, SVD-Bsed Otml Flteng Wth Alctons to ose Reducton n Seech Sgnls, IEEE-WASPAA 99, ew Yo, USA, Octobe 999. [8] S. Doclo nd M. Moonen, GSVD-Bsed Otml Flteng fo Sngle nd Multmcohone Seech Enhncement, IEEE ns. on Sgnl Pocessng, vol. 5, no. 9,. 3-44, Setembe. [9] Y. Ehm nd. L. Vn ees, A Sgnl Subsce Aoch fo Seech Enhncement, IEEE ns. on Seech Audo Pocessng, vol. 3, no. 4,. 5-66, Jul 995. [] E. B. Geoge nd M. J.. Smth, Seech Anlss/Snthess nd Modfcton Usng n Anlss-b-Snthess/Ovel-Add Snusodl Model, IEEE ns. on Seech nd Audo Pocessng, vol 5, no. 5, , Setembe 997. [] Z. Goh, K. C. n nd B.. G. n, Seech Enhncement Bsed On Voced- Unvoced Seech Model, IEEE-ICASSP 98, Settle, Wshngton, USA, vol.,. 4-44, -5 M 998. [] Z. Goh, K.-C. n nd B.. G. n, Klmn-Flteng Seech Enhncement Method Bsed on Voced-Unvoced Seech Model, IEEE ns. on Seech nd Audo Pocessng, vol. 7, no. 5,. 5-54, Setembe 999.

62 46 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng [3] I. Güle, M. K. Km, M. An nd A. Aln, AR Sectl Anlss of EEG Sgnl b usng Mmum Lelhood Estmton, Comutes n Bolog nd Medcne, no. 3, ,. [4] J. Jensen, R. endcs, R. eusdens nd S. Jensen, Smoothed Subsce Bsed ose Suesson wth Alcton to Seech Enhncement, Eus-EUSIPCO 5, Antl, ue, 4-8 Setembe 5. [5] S.. Jensen, P. C. nsen, S. D. nsen nd J. Soensen, A Sgnl Subsce Aoch fo ose Reducton of Seech Sgnls, Eus-EUSIPCO 94, Ednbugh, Scotlnd, , 3-6 Setembe 994. [6] S.. Jensen, P. C. nsen, S. D. nsen nd J. Soensen, Reducton of Bod Bnd ose n Seech b uncted QSVD, IEEE ns. on Seech nd Audo Pocessng, vol. 3, no. 6, , 995. [7] M. J. Jong, Methods of Dscete Sgnl nd Sstems Anlss, McGw-ll, 98. [8]. Klth, Lne Sstems, Pentce ll, Englewood Clffs, 98. [9] R. E. Klmn, On the Genel heo of Contol Sstems, st IFAC Congess, Moscow, USSR 96, Automton nd Remote Contol Buttewoths nd Co, London, , 96. [3] J. B. Lews, Lne Dnmc Sstems, Mt Publshes Inc., Chmgn, Illnos, 977. [3]. L, P. Stoc nd J. L, Comuttonll Effcent Pmete Estmton fo monc Snusodl Sgnls, Sgnl Pocessng, vol. 8, ,. [3] P. S. Mbec, Stochstc Models, Estmton nd Contol, vol., Acdemc Pess, ew Yo, 979. [33] R. J. McAul,. F. Qute, Seech Anlss/Snthess Bsed on Snusodl Reesentton, IEEE ns. on Acoust., Seech, Sgnl Pocessng, vol. 34, no. 4, August 986. [34] R.. McDonough nd W.. uggns, Best Lest Sques Reesentton of Sgnls b Eonentls, IEEE ns. on Automtc Contol, vol. AC3, no. 4,. 48-4, 97. [35] F. Mwech, A. Gl, S. Uchd,. A,., M. shd,. Meh, K. Kw, S. Sung nd. Shmzu, Imulse Resonse Functon Bsed on Multvte AR Model Cn Dffeentte Focl emshee n emol Lobe Eles, Eles Resech, vol. 6, no. -3, , Setembe-Octobe 4. [36]. Mow nd. Fujs, Sstem Identfcton of the Seech Poduced Pocess Bsed on the Stte Sce Reesentton, IEEE ns. on Acoust., Seech, Sgnl Pocessng, vol. ASSP-3,. 5-6, 984. [37] D. O Bn nd A. I. C. Monghn, Conctentve Snthess Bsed on monc Model, IEEE ns. on Seech nd Audo Pocessng, vol. 9, no.,. -, Jnu. [38] A. Rjoun, M. jm nd A. Moud, Snthess of Abc Seech b Lne Pedctve Codng, Wosho on Sgnl Pocessng nd ts Alctons, 9 Setembe- Octobe, Poto, 98.

63 Pmetc Models 47 [39] R. J. Schwtz nd B. Fenlnd, Lne Sstems, McGw-ll, ew Yo, 965. [4] X. Se nd J. Smth III, Sectl Modelng Snthess: A Sound Sstem Bsed on Detemnstc Plus Stochstc Decomoston, Comute Musc Jounl, vol. 4, no. 4, 99. [4]. Södestöm, Dscete-me Stochstc Sstems, Estmton nd Contol, nd edton, Snge,. [4] Y. Stlnou, On the Imlementton of the monc Plus ose Model fo Conctentve Seech Snthess, IEEE-ICASSP, Istnbul, 5-9 June. [43] Y. Stlnou, Alng the monc Plus ose Model n Conctentve Seech Snthess, IEEE ns. on Seech nd Audo Pocessng, vol. 9, no., Jnu. [44] C. W. heen, Dscete ndom sgnls nd sttstcl sgnl ocessng, Pentce ll 99. [45] S. A. ette, Intoducton to Dscete tme Sgnl Pocessng, John Wle, 976. [46] S. Vn uffel, Enhnced Resoluton Bsed on Mnmum Vnce Estmton nd Eonentl Dt Modelng, Sgnl Pocessng, vol. 33, no. 3, , Setembe 993. [47]. Wold, A Stud n the Anlss of Stton me Sees, Almqust nd Wcsells, Usl, Sweden 938. [48]. Y nd P. C. Lozou, A Genelzed Subsce Aoch fo Enhncng Seech Couted b Coloed ose, IEEE ns. on Seech nd Audo Pocessng, vol., no. 4, Jul 3. [49] G. U. Yule, On Methods of Investgtng Peodctes n Dstubed Sees wth Secl Refeence to Wölfe s Sunsot umbes, Phl. ns. Ro. Soc. London, vol. A6, , 97. [5] L. Zdeh nd C. Desoe, Lne Sstems: A Stte Sce Aoch, McGw-ll, 963.

64

65 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Mohmed jm Coght 8, ISE Ltd. Chte Lest Sques Estmton of Pmetes of Lne Models.. Intoducton he uose of ths chte s to esent dffeent methods used to obtn lest sques estmtes of the metes of lne models. o llustte these oches, we wll focus ou ttenton on the utoegessve model metes ntoduced n Chte. Fst, we consde the smle cse whee the obsevtons e not dstubed b mesuement nose. We esent non-ecusve technques, when vlble smles e ocessed s blocs of dt. hs leds to the Yule-Wle equtons. hese equtons cn be solved ecusvel usng the Dubn-Levnson lgothm []. heefte, we te u the ecusve lest sques RLS lgothm, nd successvel tet cses whee the utoegessve ocess s stton o non-stton. he second t of ths chte dels wth cses whee the obsevtons e etubed b n ddtve whte nose. ee, we wll fst nlze the effect of the mesuement nose on the estmton of the AR metes, nd then esent nonecusve nd ecusve methods whch gve se to unbsed estmtons of the AR metes... Lest sques estmton of AR metes he lest sques method s the sttng ont of vous methods fo the dentfcton nd estmton of metes. It ws ntoduced b Guss n 89, but

66 5 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng s sometmes ttbuted to Legende, who woed towds edctng the movements of lnets usng mesuements ten fom telescoe [6]. Let us stt wth ocess defned s sum of the evous mesuements, weghted b the metes,..., : [.] o smlf the nlss, we defne the two followng column vectos: [.] nd [.3] Equton [.] cn thus be wtten s follows: [.4] oweve, the mete vecto neve stsfes the ecse edcton of ll the vlues of the ocess usng the lst mesuements. heefoe, the ocess deends on edcton eo denoted u: ˆ u u u he gol s then to detemne the edcton coeffcents,..., mesuements of the ocess. [.5] fom the... Detemnton o estmton of metes? he wod detemnton s obbl note hee becuse t leds us to eoneousl beleve tht we cn clculte the model metes wthout n eo. As we wll see below, the clculton contns cetn eo. hus, we nomll se of the estmton of metes.

67 Lest Sques Estmton of Pmetes of Lne Models 5 We wll fomlze ths oblem of estmton usng equton [.5]. Intuton would led us to beleve tht f the numbe of obsevtons s ncesed, estmtng the coeffcents...,, educes to the esoluton of sstem of lne equtons. hus, we cn conduct sees of mesuements o obsevtons, wth >, nd wte: u u u [.6] B eessng the bove equtons n mt fom, we obtn: u u [.7] Let us substtute the followng smlfed nottons: Y [.8] u u U [.9] nd: [.] whee Y nd U e column vectos of sze nd s mt wth dmensons. Equton [.7] s thus educed to: U Y [.]

68 5 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng If U, the detemnton of the tems,..., the mt s nvetble: Y s tvl ovded tht [.] When, the sstem of equton [.] cn be ewtten s follows: U Y U [.3] he bove equton shows tht U dectl nfluences the detemnton of the metes. he m s to estmte the vecto of the metes, nd not to detemne t. Let obsevtons, nd let ~ ˆ be the column vecto of the metes estmted usng ~ be the coesondng estmton eo vecto. ˆ [.4] B denotng ~ the eo nvolved when edctng, the followng column eo vecto s ntoduced: ~ Y Y Yˆ Y ~ ˆ [.5] ~ he lest sques method conssts of estmtng the metes whch mnmze the otmzton cteon, J, whch s the sum of the sques of the element ~ : eos,.e.,..., J ˆ ~ [.6]

69 Lest Sques Estmton of Pmetes of Lne Models 53 Agn, consdeng equton [.5], the bove cteon cn be eessed usng the column eo vecto: J Y ˆ Y ˆ Y ~ ˆ Y Y ~ Y ˆ Y ˆ ˆ [.7] Snce J ˆ s functon of ˆ, t deends on the vbles,...,. he vecto ˆ should stsf the two followng condtons to gve mnmum n cteon [.7]: the fst of these condtons concens the gdent of J, denoted J, whch s defned s the column vecto of the tl devtves of J wth esect to the. hs gdent should stsf: AR metes,..., J ˆ [.8] whee denotes the zeo vecto. heefoe: J ˆ j j,..., [.9] the second condton concens the essn mt of J ˆ, whch s denoted J, hs dmensons, nd s comosed of the second tl devtves.e.: J of J ˆ. hs essn mt should be ostve defnte, j V J V ndeendentl of the vlue of the non-zeo vecto V. [.] Fo sngle-vble functon, condtons [.9] nd [.] led to the sech fo n etemum nd then locl mnmum.

70 54 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng We cn esl show tht fo n column vecto P : n n n P ˆ,..., [.] Consequentl: P P P P ˆ ˆ ˆ [.] Substtutng Y P n equton [.], we obtn: ˆ Y Y [.3] Smll, fo n smmetc mt R wth elements j, nd whch esects j j, we obtn: n n n n n m mn m R ˆ ˆ,..., [.4] Consequentl: R R ˆ ˆ ˆ [.5] ng R, we hve: ˆ ˆ ˆ [.6] hen, tng nto ccount eltons [.7], [.3] nd [.6], equton [.9] becomes: Wth beng the ow nd j the column.

71 Lest Sques Estmton of Pmetes of Lne Models 55 ˆ ˆ ˆ ˆ ˆ Y Y Y Y J [.7] hus: ˆ Y. [.8] If the mt s non-sngul, ths leds us to the followng eesson fo the lest sques estmton of : ˆ Y [.9] Let us consde cses whee the estmton of s bsed,.e. whee the followng neqult holds: E ˆ [.3] wth. E beng the mthemtcl eectton. Fo ths, we must ecll the eesson of the dffeence between the mete vecto nd ts estmton ˆ usng elton [.]. hus: ˆ U U Y [.3] Shftng to the left hnd sde of the equton, we obtn: ˆ U. [.3] Clcultng the mthemtcl eectton on both sdes of equton [.3] gves: ˆ U E E [.33]

72 56 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng When U nd e ndeendent of ech othe, ths cuent equton s modfed to: ˆ E EU E [.34] hus, f U E ˆ. E, the estmto s unbsed, becuse evetheless, the lest sques estmton of s bsed n two cses: f U nd e coelted sequences; f the men vlue of U s non-zeo. We lso hve to vef tht condton [.] s lso stsfed. J ˆ clculted s follows: cn be J ˆ ˆ ˆ Y Y Y ˆ ˆ Y ˆ ˆ ˆ [.35] Consdeng equton [.], we chec tht s ndeed ostve defnte mt of dmensons. We hve thus f esented fst och whch llows us to eess the mete vecto usng equton [.9]. hs och nvolves the mt nd eques the nveson of the mt. It s n nonecusve och. In the secton below, we wll develo ecusve och to estmte.... Recusve estmton of metes he m of ths secton s to ut foth ecusve ocedue to estmte the metes. Sttng fom ˆ, the estmto of bsed on mesuements, we now eess ˆ whch uses + mesuements.

73 Lest Sques Estmton of Pmetes of Lne Models 57 We now tht: U Y [.] In ddton, + th obsevton of the ocess, +, s vlble ccodng to equton [.5]: u. [.36] Fo the est of ths devton, we wll suose tht u s zeo-men whte sequence wth vnce u. We cn ntoduce new etended vecto Y consstng of + mesuements, nd defned b mens of the equtons [.] nd [.36]: U u Y Y [.37] nd: U Y [.38] B lng [.9], the estmton ˆ of the vecto of the AR metes usng + obsevtons { -+,, + } cn be eessed s follows: ˆ Y [.39] hs cn be ewtten n tems of nd + : ˆ Y [.4]

74 58 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Endng the mt multlcton on the ght-hnd sde of the bove equton, we get the followng fom: ˆ Y [.4] o comlete the devton, we use the nveson lemm fo mt. Let A be mt whch cn be decomosed s follows: CD B A [.4] A, the nvese mt of A, s gven b: B D C B D I C B B A [.43] Consequentl, f we choose: B nd D C lng the nveson lemm to equton [.4] gves: ˆ Y [.44] ng nto ccount equton [.3] fo ˆ, ths leds to the stghtfowd clculton: ˆ ˆ ˆ [.45]

75 Lest Sques Estmton of Pmetes of Lne Models 59 hs equton cn be condensed usng weghtng fcto to ccount fo the chnge bought bout b +. hs fcto s clled the gn, nd s denoted b K. ˆ ˆ K [.46] Equton [.46] shows tht to detemne ˆ, we onl eque ˆ nd the new mesuement +. he estmton of the metes s udted b the weghted dffeence between the effectve mesuement + nd the edctble mesuement,.e.: ˆ ˆ [.47] It s notewoth tht n the bsence of nose, ˆ would eesent the best edcton of the mesuement. In equton [.45], the denomnto tem coesondng to the gn, nmel, s scl quntt. he oeton of mt nveson s elced b the smle oeton of dvson. ˆ..3. Imlementton of the lest sques lgothm In ode to mlement the lest sques lgothm nd to fcltte comsons wth othe lgothms, esecll the Klmn flte esented n Chte 5, we wll dot the followng notton: P [.48] he gn K s thus gven b: P K P [.49] In ode to detemne ecusve equton fo the clculton of P, let us stt wth the defnton: P [.5]

76 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng On beng u nto the tems nd, we obtn: P Equvlentl: P P Alng the mt nveson lemm [.43] nd tng: A P, B P, C nd D, we cn deve the followng eltonsh between P nd P : I K P P [.5] he new lgothm cn thus be mlemented usng the followng set of thee equtons: P K P [.49] ˆ ˆ K [.46] I K P P [.5] he detemnton of P mes t ossble to clculte the gn K, whch n tun llows the estmton of ˆ t the + th nstnt. Fgue. shows eesenttve cse: the ecusve estmton of the metes of nd -ode AR model. In ddton, u s el whte zeo-men Gussn nose wth unt vnce. he metes of the model e =. nd =.7. hs fgue esents the esults veged ove elztons of the ocess. ˆ

77 Lest Sques Estmton of Pmetes of Lne Models 6.4 Pmetes Pmetes o m b e d 'té to n umbe of tetons Fgue.. An emle of ecusve estmton of the metes of nd ode AR ocess wth =. nd = he lest sques method wth weghtng fcto Let us ecll the eo-mnmzton cteon [.6]: J ˆ ~ [.6] J ˆ s the sum of element eos ll hvng the sme unt weght. o ncese the emhss on the fst o the lst mesuements, we cn dot cteon whch weghts ech element eo ~ b fcto, such tht: J ˆ ~ ~ [.5] he bove elton cn lso be wtten wth mtcl fom b ntoducng the dgonl weghtng mt W s follows: whee: J ˆ ~ ~ Y W Y [.53] W. [.54]

78 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Let us choose whee s scl entt nd whee the cuent tem s obtned usng geometc ogesson. In tht cse, tng fvos the ltest mesuements wth esect to the elest ones. hus, behves s fogettng fcto. he estmto cn be found b solvng the followng equton: ˆ J, o j J j,..., [.9] Snce: Y W Y J ˆ ˆ ˆ [.55] we obtn: W Y W W Y Y W Y J ˆ ˆ ˆ ˆ ˆ [.56] ng equtons [.9], [.] nd [.5] nto ccount, we chec tht the estmto stsfes the followng elton: ˆ W Y W Y W [.57] Rengng the tems of the bove equton, we obtn: ˆ Y W W [.58] B choosng mt W to be equl to R uu, the nvese of the utocoelton mt of the sequence u, we obtn the so-clled best lne lest sques estmto. Fo consstenc n the nmng conventon, we secf tht the mt W = R uu o, sml ut, W = R.

79 Lest Sques Estmton of Pmetes of Lne Models A ecusve weghted lest sques estmto Fo the weghted lest sques cse, the non-ecusve estmto s gven b: ˆ Y R R [.59] If we ecll equton [.48]: R P [.6] the ecusve estmto wll be defned b the followng set of thee equtons: ˆ ˆ ˆ P K I P K P P K [.6]..6. Obsevtons on some vnts of the lest sques method he lest sques method s wdel used to obtn model metes of vous tes of sgnls. In the feld of seech ocessng, t hs been dted unde vous foms: covnce methods, utocoelton, tl coelton co, etc. [3]. he nlss below, fst undeten b Atl nd nue [4], ms t fndng the coeffcents...,, tht mnmze the sum of the sques of the edcton eos. j j j e J [.6]

80 64 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Mnmzng cteon J wth esect to the coeffcents j j,..., gves: J j j j,..., hus: j j j,..., [.63] nd: Defnng the utocoelton functon of the sgnl b:, j j [.64], j j, [.65] we obtn: j,, j j,, [.66] Vng j fom to, equton [.66] cn be eessed n mt fom s follows:,,,,,, [.67] nd condensed to: R R [.68]

81 Lest Sques Estmton of Pmetes of Lne Models 65 wth: [.69] R denotes the utocoelton mt of ocess : R,,,, [.7] nd R s the utocoelton column vecto of ocess : R,, [.7] Consdeng the vlues of the edcton metes, nd ncootng equtons [.6] nd [.63], the mnmum qudtc eo E cn be eessed b: E [.7] In the followng secton, we wll descbe the bounds of summton mosed on cteon J. hs descton leds us to two clsscl methods, nmel the utocoelton method nd the covnce method he utocoelton method he ntoducton of the utocoelton method s genell ttbuted to Mel nd G []. hs method conssts of tng the summton lmts n equtons [.63]-[.7] to be nd. Equtons [.64] nd [.7] e thus modfed to:, j j [.73] E,, [.74]

82 66 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Equton [.73] defnes the utocoelton functon of the el sgnl. Moeove, f we ssume tht the sgnl s stton, the elements of the mt R nd those of the vecto R e such tht: j j, [.75] he utocoelton functon lso vefes the oet of smmet: j j [.76] Rewtng the mt of equton [.67] gves: R [.77] hs smmetc mt hs ecul stuctue, clled the oeltz stuctue, n whch ll dgonl elements e equl. hs oet cn be used to develo sevel lgothms fo the esoluton of equton [.68]. When lng ths method to the estmton of the metes of the sgnl model, we lws consde fnte numbe of smles of the sgnl. hus: fo nd hs chnges equton [.64] to: j j j j, [.78] If the men of the dvng ocess u s ssumed to be zeo, the edcton eo s s follows: E u [.79]

83 Lest Sques Estmton of Pmetes of Lne Models 67 whee u denotes the vnce of the whte sequence u elted to the th -ode AR ocess. ng nto ccount equtons [.67], [.79] nd [.77], we obtn the followng mtcl elton: u [.8] Fo th -ode AR model, the esoluton of sstem of + lne equtons, clled the noml o Yule-Wle equtons, wll ovde the estmton of the metes. he wod noml ses fom the equton we evousl estblshed j,..., : J j j j e whch mles tht -j nd e e othogonl. he esoluton of noml equtons s the subject of vst mount of ublshed wo. An ehustve evew of ths ltetue would be dffcult so we cte the followng notble contbutons: Robnson [8], Klth [6], Mof [4], Cnns [5] nd Leou-Gueguen [8]. he fst lgothm deved fo ecusve soluton on the ode of these equtons s clled the Levnson lgothm [9]. It ws subsequentl moved b Dubn [] Levnson s lgothm We estblshed n secton..6. tht the estmton of the metes of n AR model usng the lest sques cteon leds to the esoluton of the oml o Yule-Wle equtons. hs esoluton s bsed on the nveson of the R mt. he oeltz stuctue of ths mt llows us to obtn smlfed ocedue. he tdtonl esoluton of the sstem n [.68],.e., whee 3 mt s consdeed, entls comuttonl cost of the ode. Levnson s

84 68 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng lgothm, whch mes t ossble to educe ths comlet to, s bsed on ecusve och on the model ode. Sttng fom the soluton fo n ode, the soluton fo ode + cn be deduced. hs gves se to secl metes, clled the eflecton coeffcents, whch cn be nteeted n mn dffeent ws. hee ests n ltentve to Levnson s lgothm [5]. At the begnnng of the th centu, Schu ntoduced enttes clled the tl coeltons useful fo the sech of olnoml oots. B the end of the 94s, Levnson soe of set of eflecton coeffcents fo ecusve solutons to sstems of lne equtons. od, these tl coeltons nd eflecton coeffcents e wdel used n sgnl ocessng, n lctons such s nlss nd modelng of sgnls, snthess nd seech codng, to hgh esoluton sectl nlss. Let us deve Levnson s lgothm n two stes: fst of ll, we ecll the eessons fo the lne edcto nd fo the AR model; thus, we cn concentte on some esults concenng the edcton eo. Equton [.8] cn be wtten s below: u [.8] ee,,..., coesonds to the AR metes of th -ode utoegessve model. he th mete n the cse of ode s dffeent fom the th mete n the cse of ode +. hus, we wll denote them nd esectvel. We wll ncese the dmenson of the utocoelton mt b ddng one ow nd one column, chngng equton [.8] s follows: In Aend E, we esent the Schu-Cohn lgothm s well s the equvlence tht cn be estblshed between the Schu coeffcents nd Levnson s eflecton coeffcents.

85 Lest Sques Estmton of Pmetes of Lne Models 69 u [.8] nd, ncludng the notton ntoduced n equton [.77]: u R [.83] Fo equton [.83] to be comtble wth equton [.8], the tem must stsf the followng condton: [.84] Snce the oeltz mt R hs secl smmet, equton [.83] cn be ewtten s follows: u [.85] Multlng equton [.85] b whch wll be defned lte nd ddng the esult to elton [.83], we obtn:

86 7 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng u u [.86] B choosng such tht: u [.87] equton [.86] s modfed to: u [.88] Rewtng the Yule-Wle elton [.8] fo ode +, we obtn: u [.89]

87 Lest Sques Estmton of Pmetes of Lne Models 7 hus, comng equton [.88] wth [.89] leds to the followng ecusve eltons: [.9] o: [.9] nd:. [.9] Moeove, we cn deduce the elton between the vnces of eos when delng wth model odes nd +: u u [.93] Equton [.87] becomes: u u [.94] s clled the eflecton coeffcent. Snce the tems u nd u e necessl ostve, equton [.94] dcttes tht. As oosed to the AR models tnsvesl metes...,,, the eflecton coeffcents hve hscl menng. Fo emle, fo seech sgnl, the coesond to the eflecton coeffcents of the ogton wves long the vocl tct. hese eflectons e esult of the chnges n the mednce due to the tnstons n the vocl tct. In sesmolog, when modelng the Eth s stt, these coeffcents cn be undestood s the eflecton of coustc wves dung chnge n the ntue of the dffeent subtenen les.

88 7 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng he Dubn-Levnson lgothm Le the Levnson lgothm, the Dubn-Levnson lgothm s ecusve lgothm on the ode of n AR ocess. Recllng the Yule-Wle equtons fo th -ode AR ocess: R R [.68] he bove elton cn be eessed usng the vecto # whch stoes the AR metes n evese ode: R # R # [.95] Rewtng equton [.68] fo + th -ode the AR ocess elds: R R [.96] If we tton the R mt b the utocoelton mt R fo ocess, nd f we wte R s functon of the column utocoelton vecto R of ocess, we obtn:

89 Lest Sques Estmton of Pmetes of Lne Models 73 # # R R R R [.97] Equtng the dffeent blocs of sstem [.97]: # # R R R R [.98] Multlng the fst element of sstem [.98] b R, the fst coeffcents of the + th -ode AR ocess cn be wtten s: # R R R R [.99] Gven equtons [.68] nd [.95], the bove equton cn be smlfed s follows: # [.] Intoducng equton [.] nto the second t of sstem [.98], we obtn: # # R [.]

90 74 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng o equvlentl: nd: # # # R R Usng the followng two equltes: # # R R [.] [.3] # # R R, [.4] we obtn the followng eesson fo : # R [.5] R Obsevng equtons [.8] nd [.5], we notce tht the denomnto of the ltte coesonds to the vnce of the dvng ocess n the cse of th -ode egesson model. We now hve two ecusve eltons,.e., equtons [.] nd [.5], to efom the dentfcton. We cn futhe smlf ths lgothm thns to ecuence on the vnce of the dvng ocess: u u u R R R [.6] Substtutng the vous tems of equtons [.79], [.9], [.] nd [.5], we obtn modfed fom of equton [.6]:

91 Lest Sques Estmton of Pmetes of Lne Models 75 # # R R R u u u u u u [.7] he bove equton s the sme s elton [.94] obtned fo Levnson s lgothm. hs new lgothm s ntlzed b: [.8] he Levnson nd Dubn-Levnson lgothms ntoduce the eflecton coeffcents. We wll see n the followng secton tht the defne stuctue clled lttce fltes Lttce fltes Let us ecll tht equton [.5], gven below, esents the AR ocess s lne fowd edcton of, bsed on ts lst vlues wth n eo tem, u f : ˆ,,,, u u u f f f f [.5] Let us consde bcwd lne edcton of. hs s edcton bsed on futue smles,, : ˆ, #,,, u u u b b b b [.9]

92 76 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng he tem ˆ, f of equton [.5] s eessed s: ˆ, f [.] If we nset equton [.] djusted to ode - nd the eflecton coeffcent of equton [.9] nd [.9], we obtn: ˆ ˆ ˆ,, # #, b f f [.] hs leds to the fst ecuence between the eo tems, u f,, u f nd, u b : ˆ,,,, u u u b f f f [.] Smll, we cn detemne the second ecuence b wtng ˆ, b s: b #, ˆ [.3] Substtutng equton [.], we obtn:, # # ˆ b [.4] B djustng equton [.4] t nstnt nd usng equtons [.5] nd [.9], we obtn:

93 Lest Sques Estmton of Pmetes of Lne Models 77 ˆ ˆ ˆ,, #, f b b [.5] he second ecuence between eo tems, u f,, u b nd, u b s thus:,,, u u u f b b [.6] Fnll, the ecusve equtons [.] nd [.6] cn be eesented s lttce stuctue see Fgue Fgue.. he bsc cell of lttce stuctue he covnce method Develoed b Atl nd nue [4], the covnce method fnds n equvlent n the contet of dentfcton of sstems, nmel the Meh covnce lgothm []. hs llel wll be eloted n Chte 6 fo the develoment of new oches fo sgnl enhncement. Let us consde the cse whee the summton vlues on cteon [.6] le wthn the ntevl. Equton [.64] s modfed to:, j j [.7]

94 78 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng nd we stll hve:, j j, [.8] oweve, the condton of eqult of, j nd j, s no longe esected. We hve:, j j j l j j j l l l j j, j j [.9] hs eltonsh sgnfes tht the dgonl tems e no longe equl. hus, R s stll smmetc mt, but s no longe oeltz Relton between the covnce method nd the lest sques method Whle comng the two methods, we wll see tht nste of the dffeent nottons, the oches e equvlent. Recll tht the obsevton o mesuement s wtten s follows: u, [.5] wth: nd: [.]. [.3] If we hve vlble mesuements,,...,, we obtn: Y U [.]

95 Lest Sques Estmton of Pmetes of Lne Models 79 wth: U u u [.] In equton [.], we deved n eesson fo the lest sques estmton. he estmton fo nose-fee cse s gven b: thus: Y ˆ [.] ˆ Y [.3] he cteon to be mnmzed, denoted J, s gven b: J Y Y. [.4] Relcng Y b Y nd b n equton [.4], we get the covnce method []: nd: [.5] Y [.6]

96 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng We end u wth: J [.7] Substtutng Y fo Y nd fo n equton [.3], nd usng equton [.7], we obtn the followng eqult:,, ˆ,,,, [.8] o: ˆ, R R [.9] ng nto ccount the two nottons, we thus demonstte the equvlence between the covnce method nd the lest sques method. ow, to come both methods, we wll use the followng nottons: Y [.3]

97 Lest Sques Estmton of Pmetes of Lne Models 8 [.3] If we lce the bove nottons n equton [.7] fo the estmto of the lest sques method, wth = fo < nd > we obtn: J [.3] Smll, elcng Y wth Y nd wth n equton [.3], o b usng equtons [.75] nd [.78], t follows tht:

98 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng ˆ [.33] o: R, ˆ R [.34] Effect of whte ddtve nose on the estmton of AR metes In mn cses, the sgnl, modeled b th ode AR ocess, s dstubed b n ddtve mesuement nose b. he eesson fo the obsevtons o mesuements s thus modfed to see Fgue.3: z b hs s the cse, fo emle, n the feld of seech enhncement [3]. In ths secton, we wll nlze the nfluence of mesuement nose b, ssumed to be whte, Gussn, wth zeo-men nd vnce b. hs ddtve nose ovdes bsed estmton of the AR metes. In the net secton,..6.8, we wll esent dffeent oches to llevtng ths oblem,.e. to obtn unbsed estmtes. Fgue.3. Reesentton of n AR ocess dstubed b n ddtve nose o nlze the effect of b on the estmton of the AR metes, K hs oosed to me the comson between the sectl fltnesses, nd z, of the ocess nd z esectvel [7].

99 Lest Sques Estmton of Pmetes of Lne Models 83 Recll tht fo n gven ocess, ths ndcto of sectl fltness s defned s follows: e / lns f df / P [.35] / S df R / f wth S nd R beng, esectvel, the sectl denst nd the utocoelton functon of. hs ndcto hs the followng oetes: ; f nd onl f f S s constnt; f nd onl f the ofle of f S shows e. he fltness ndctos z nd stsf the followng neqult eltonsh 3 : z [.36] hs neqult mes t ossble to show tht n ddtve whte nose tends to fltten the sectl denst of the consdeed ocess. he lest sques estmton of the AR metes s bsed f t s bsed dectl on obsevtons dstubed b whte ddtve nose. he coesondng oles of the nos AR ocess tend to be locted close to the cente of the unt ccle n the z-lne. hs s shown n Fgue.4. 3 K s demonstton of ths neqult eltonsh wll be consdeed n Aend E.

100 84 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Fgue.4. Influence of n ddtve nose on the estmton of the metes of nd ode AR ocess. he SR ves fom 3 to 5dB n stes of 3dB denote the oles

101 Lest Sques Estmton of Pmetes of Lne Models A method fo llevtng the bs on the estmton of the AR metes o educe the bs on the AR mete estmton, one soluton s to vod the use of zz,.e. the utocoelton functon of the nos AR ocess fo lg equl to ; we get modfed o ovedetemned Yule-Wle MYW equtons 4 [6], [], []: zz zz zz zz zz zz zz zz zz zz zz zz [.37] As n ltentve, we cn use the ended nd so-clled ovedetemned Yule- Wle equtons, wth q>: zz zz zz q zz zz zz q zz zz zz hs new och entel foegoes the use of sstem of q equtons wth < q unnowns. zz zz zz [.38] q zz, nd s nsted bsed on Yet nothe ltentve och conssts of the ntoducton of nosecomensted Yule-Wle equtons whch, howeve, eque n estmton of the ddtve nose s vnce, lbeled [7]: b ˆ b zz zz zz zz zz zz zz zz zz zz ˆ b I [.39] zz zz q 4 It should be noted tht the MYW equtons e used n the v functon of Mtlb s Identfcton toolbo. In Chte 7, we wll elbote uon Fedlnde s nteetton of the MYW equtons s n nstumentl vble technque [].

102 86 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng zz zz [.4] zz wth =,, q zz Gven equton [.39] the AR metes cn be estmted s follows: ˆ R I R [.4] LSC zz b zz he equton bove s nonlne due to the esence of the genell-unnown b. It cn be solved usng the followng two technques: the fst conssts of the tetve nd ltentve estmton of metes nd vnce b. Emles of ths technque e found n the och oosed b Zheng et l. [3] nd sn et l. [4]; the second technque conssts of consdeng equton [.4] s genelzed egenvlue decomoston ssue [8]. Let us loo t these two lgothms n moe detl. he estmton of o b usng equton [.4] nd condtonl to b o s lne ts. Zheng et l. [3] theefoe esent n tetve och whch ltentvel estmtes nd b. Fo ths uose, the ntoduce n etended vecto of the AR metes: e. [.4] he coesondng lest sques estmton bs of e stsfes: e E ˆ LS e b e e R zz [.43] whee e R zz denotes the ++ utocoelton mt of the obsevtons. he lest sques estmton of the metes s thus coected s follows: e LS b e e R e ˆ [.44] zz

103 Lest Sques Estmton of Pmetes of Lne Models 87 Multlng both sdes of equton [.44] b gves: e LS e e e e e R R b zz b zz. [.45] Fom equton [.45], t follows tht the vnce follows: b cn be eessed s b e e R zz e ˆ LS [.46] Zheng et l. use equltes [.44] nd [.46] n n tetve mnne, to estmte the AR metes nd the vnce of the ddtve nose. hs s llustted n Fgue.5 below. he tetve ocess stos when the estmtons t tetons nd, denoted, esectvel, ˆ nd ˆ, e close to one nothe nd stsf: ˆ ˆ ˆ [.47] whee. stnds fo the stndd L nd nges fom -6 to -3 deendng on the desed ecson. It should be noted tht fste vesons of ths lgothm e esented n [9] [3]. Fgue.5. Bloc-level schemtc of the Zheng method [3]

104 88 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng ssn et l. esent nothe method, ntll develoed fo the nlss of vectol AR ocesses [4]. We wll use ths method fo scl AR ocess. In [4], the uthos deve second eltonsh between nd b, usng the outut t of the nvese flte Az utocoelton functon of tt tt z Et t b whose nut s z see Fgue.6. he t thus stsfes the followng equton: [.48] Fgue.6. Bloc-level scheme used n sn s method [4] o estmte nd b, the uthos use n tetve method whch oetes s follows: ˆ ; ˆ s fst detemned b flteng tt the nonlne equtons [.4]-[.48] of the unnown z wth the nvese flte, defned b the ewton-rhson ocedue, nd the vlue of vnce ˆ the coected estmton b e esolved usng s udted; ˆ of the AR metes s deduced fom [.4]. Dvl et l. see the soluton of the nose-comensted Yule-Wle equtons [.4] s genelzed egenvlue decomoston ssue [8]. In fct, the uthos stt wth n ovedetemned sstem b ddng q equtons to the sstem n [.4], ledng to: R zz LSC b B, [.49] LSC b

105 Lest Sques Estmton of Pmetes of Lne Models 89 wth: R zz zz zz zz zz zz q q zz zz zz zz zz q zz zz zz zz zz zz zz, [.5] zz q zz nd: B [.5] LSC [ LSC ] [.5] Checng equton [.49], the vnce egenvlue of the mtces b cn be seen s genelzed R zz nd B. he egen-subsce coesondng to the vecto of the AR metes s found b solvng the followng equvlent qudtc equton: wth A A b A b LSC [.53] zz, A R zz B B Rzz A R zz R. nd A B B he egenvlues of equton [.53] e thus comle conjugtes. In noseless envonment, the egenvlue s theoetcll zeo becuse the vnce of the ddtve nose s zeo. Fo nos cse, the nose vnce s dded to ll the egenvlues of R zz nd the soluton of the equton coesonds to the onl el vlue of b. o solve equton [.53], we need to solve the followng genelzed egenvlue decomoston oblem:

106 9 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Pv b Qv [.54] whee A LSC A A v, P nd Q b LSC I. I In ctce, howeve, the estmton of the utocoelton functon of contns comutton eos. Consequentl, the egenvlue we obtn s not lws el. evetheless, s ts mgn comonent s much smlle thn the mgn comonents of the othe egenvlues of R zz, Dvl et l. choose the subsce ssocted wth the egenvlue of the smllest module. Aoches othe thn those bsed on nose-comensted Yule-Wle equtons hve been oosed, the most notble beng tht of Deche et l. [9]. In ths och, the AR metes e estmted usng n tetve eecttonmmzton EM te lgothm. Fnll, sn et l. model the utocoelton functon of nos obsevtons usng sum of eonentll-decesng snusodl comonents EDS [5]. he EDS metes e then used to estmte the AR metes. Recentl, we hve oosed new w to estmte the AR metes, the vnces of the ddtve nose nd the dvng ocess b usng the eos-nvbles och. hs method ws deved n the feld of seech enhncement 5 nd Rlegh fdng chnnel estmton 6. In the followng, we wll esent comtve stud of these methods, bsed on obsevtons, usng two dffeent tests. est : the sntheszed AR ocess s chctezed b the followng s oles n the z-lne:.7e j..8e j.4.85e j.7., nd hs ocess s then dstubed b zeo-men whte Gussn nose such tht the sgnl-to-nose to s db. Fve dffeent methods e closel emned: the Yule-Wle equtons, the modfed Yule-Wle equtons, nd the thee bscoecton lgothms oosed b Zheng [3], Dvl [8] nd sn [4]. As 5 Seech Enhncement Combnng Otml Smoothng nd Eos-In-Vbles Identfcton of os AR Pocesses W. Bobllet, R. Dves, E. Gvel, R. Gudoz, M. jm nd U Soven, IEEE ns. on Sgnl Pocessng, Dec. 7, vol. 55, n., Eos-In-Vbles Bsed Aoch fo the Identfcton of AR me-vng Fdng Chnnels, A. Jmoos, E. Gvel, W. Bobllet nd R. Gudoz, IEEE Sgnl Pocessng Lettes, ov. 7, vol.4, no.,

107 Lest Sques Estmton of Pmetes of Lne Models 9 Fgue.7 shows, when the numbe of vlble smles s hgh, sevel thousnd fo nstnce, ll fve methods gve ccute estmtons of the AR metes. est : to test the bove lgothms n moe elstc condtons, we stud the behvo when the numbe of obsevtons s lmted. Let us te 3 smles of n AR ocess chctezed b the followng s oles:,.98 e j. 3,4.97 e, j.3 nd 5,6.8 e j.84. As n est, the sgnl s dstubed b whte, zeo-men Gussn nose such tht the SR s db. As Fgue.8 shows, Zhng nd Dvl s methods gve AR metes whch cn ende the sstem unstble, snce the coesondng oles e outsde the unt ccle n the z-lne. Eected oles Obtned oles Obtned sectum Eected sectum b c

108 9 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng d Fgue.7. est : hgh numbe of obsevtons. Locton of the oles n the z-lne nd the ssocted AR sect fo dffeent off-lne nd ecusve oches: Yule-Wle, MYW b, Zheng c, Dvl d, sn e e b

109 Lest Sques Estmton of Pmetes of Lne Models 93 c d Fgue.8. est : lmted numbe of obsevtons. Locton of the oles n the z-lne nd the ssocted AR sect fo dffeent off-lne nd ecusve oches. oseless Levnson Yule-Wle, MYW b, Zheng c, Dvl d, sn e e..7. Genelzed lest sques method As we sw n secton.., f the nose s coelted to the obsevton, the estmton of the metes s bsed. Even f the nose s n uncoelted whte nose, the estmton s genell bsed becuse the ARMA stuctue modfes ths mesuement nose to coelted sequence [7]. ng u the bsc noseless ARMA model descbed b the ecusve equton, whee, fo the se of smlct, we consde odes nd q to be equl: b u [.55]

110 94 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Fst, let us te the cse, shown n Fgue.9, n whch the obsevtons z e dstubed b n ddtve nose b : z b [.56] Fgue.9. Outut-efeed nose As n secton..6.7, b elcng b z b n equton [.55], we obtn: z b z b bu [.57] Equvlentl: z z b u [.58] whee s coelted ocess such tht: b wth [.59] Let us consde second cse, shown n Fgue., n whch n ddtve whte nose w cts on the nut u. Fgue.. Inut-efeed nose

111 Lest Sques Estmton of Pmetes of Lne Models 95 Addng ths nose to the ocess u of equton [.55], we obtn: b u w [.6] Equvlentl: b u [.6] whee s coelted nose such tht: b w [.6] hus, esectve of the cse, ddng nose to the nut o to the outut of the sstem coesonds to n ARMA model dstubed b coelted nose. In the est of ths nlss, we wll consde the cse of coloed dstubnce : bu [.63] he z tnsfom of equton [.63] gves: A z Y z B z U z z [.64] whee Y z, U z nd z e, esectvel, the z tnsfoms of, u nd. In ddton, A z nd B z e defned s follows: A z z [.65] B z b z [.66]

112 96 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Let us ssume tht the esdue cn be modeled b n AR ocess: c e [.67] whee e s whte nose whose z tnsfom E z s such tht: z E z C z c z [.68] ng equton [.68] nto ccount, we cn ewte equton [.64] s: A z Cz Y z B z Cz U z E z [.69] Equvlentl: C z Y z B z C z U z E A z z [.7] o: A z Y z B z U z E z [.7] hese equtons e equvlent to the followng temol eltons: bu e [.7] c [.7b] u c u [.7c] he nut u o outut of ths model cn be seen s the sgnl u o flteed b flte wth tnsfe functon C z.

113 Lest Sques Estmton of Pmetes of Lne Models 97 Let us loo t the estmton of the coeffcents c of equton [.67]. o do so, we cn l the stndd lest sques method. If we use the mesuements ced out fom to, we obtn the followng mtcl eltonsh: e e c c Wtng:, nd:, e We cn estmte the coeffcents c,, s follows:,,, c c e e e [.73] oweve, the tems, e unnown nd hve to be estmted. o do ths, we use the ocedue ntoduced n [7]. o smlf the equtons, we wll dot the followng notton heefte: Y,

114 98 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng u u, Y u u e e E nd: b b Equtons [.63] nd [.7] cn be wtten esectvel s follows: nd e If we hndle mesuements, we cn wte, n mtcl fom: Y [.74] nd:,, E Y [.75] fst, we estmte vecto ˆ usng ˆ nd ˆ u fo ˆ ˆ ˆ ˆ ˆ,,,, Y

115 Lest Sques Estmton of Pmetes of Lne Models 99 net, we estmte the esdul vecto ˆ usng equton [.74]: ˆ Y ˆ [.76] then, we cn constuct C ˆ z sttng fom ˆ s follows: cˆ ˆ ˆ ˆ ˆ [.73] cˆ we net flte nd u so s to obtn ˆ nd uˆ fo fnll, ˆ nd uˆ llow us to detemne the new estmton ˆ,, whch s n tun used to detemne the new eesson fo ˆ nd then we conduct the sme oeton gn sttng wth the fst ste...8. he etended lest sques method Let thee be lne sstem wth n ddtve nose ctng uon ts outut o efeed to ts nut. We sw tht the stuctue of the ARMA model genell contns coelted nose. hus: bu [.77] wth: [.78] ce wth c Cng out the z tnsfom on equton [.77] gves: z B z U z C z E A z Y z [.79] o stt wth, let us suose tht e s nown. We cn wte the bove equton s:

116 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng whee: e [.8] u u e e nd b b c c. If e s n fct nown, we cn use the ecusve lest sques method. If, on the othe hnd, e s n unnown, we cn oceed s follows: whee: we fst clculte the esdue: eˆ ˆ ˆ [.8] ˆ u u eˆ eˆ we then use the lest sques estmto: K P ˆ ˆ P ˆ [.8] ˆ ˆ K ˆ e [.83] P I K ˆ P [.84] hs lgothm goes b sevel nmes: the Pnus lgothm, the etendedmt method, the omte mmum lelhood AML method. It s used n wde nge of lctons..3. Selectng the ode of the models he toc of the detemnton of the ode of the models s dffcult one, wthout n stghtfowd soluton. We wll thus se t hee, nd llustte t wth some emles.

117 Lest Sques Estmton of Pmetes of Lne Models Whch ode s suffcent to effectvel undestnd the behvo of sgnl? hs queston s n mbguous one nd deends on the defnton of suffcent. As t ws ghtl stted b J. Mhoul n [], the model tself s moved s ts ode nceses. evetheless, whee do we sto wth ths ncese n the ode? If we nomlze the coeffcents, j of the covnce mt b dvdng them b, we obtn the nomlzed eo: E V [.85] Accodng to Ulch et l., the ode should be smlle thn /, whee s the numbe of vlble smles [7]. If the ode s too hgh, we c out n djustment wheeb the metes tend towds f [3]. he estent cte fo the ode estmton use the edcton eo. he dsdvntge stems fom the fct tht ths eo dmnshes s the ode nceses. oweve, beond cetn ont, ths dmnuton tes off. Ae develoed the followng two cte: the Fnl Pedcton Eo FPE nd the Ae Infomton Cteon AIC, n [] nd [] esectvel. he e defned s: FPE E [.86] AIC log E [.87] whee s the ode whch educes the cte to mnmum, s the numbe of smles, nd E s the sum of ll the edcton eos. he ognl eesson tht Ae ovded ws: AIC logm.lelhood [.88] Fo Gussn ocesses, ths eesson s educed to the evous one. Snce the numbe of vlble smles s lmted b the wdth of the nlss wndow, coecton fcto hs to be dded. hs coecton, n the cse of mmng

118 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng wndow, conssts of tng n effectve numbe of smles sgnls, the ode s genell oveestmted [3]. e. 4. Fo nos Fgue. shows n emle n whch the the ode of n AR model s estmted. he nose u s zeo-men whte Gussn nose wth vnce of one. he model metes e. nd. 7. FPE AIC Ode Ode Ode Ode Fgue.. Emle: estmtng the ode of the model It s genell cceted tht fo seech sgnls, the ode of the model s between nd 6. hs llows the modelng of the fst fomnts of the seech sgnl, the esonnces of the vocl tct. It ws seen n Chte tht the -th ode AR model llows us to ccount fo u to esonnces n the sectum n the fequenc nge f s / f s /. Even f ths eesentton llows fo good omton of the sectum s enveloe, hmonc descton of voced sgnl such s n // o n // eques hghe odes. In elt, f the see s humn beng nd f the fequenc of hs tch s f = z, the lowest ossble ode s f s / f. If fs 8 Kz, ths ode wll theoetcll be t lest 8. oweve, the length of the nlss fmes s equed to be between nd 4 ms fo the hothess of the qus-sttont of the sgnl to hold tue. hs lmts the numbe of vlble smles fo the estmton of the AR metes 8 to 56 fo smlng fequenc of fs 8 Kz. A tde-off thus hs to be mde between the numbe of metes bsed on the numbe of vlble smles nd the fttng of the model.

119 Lest Sques Estmton of Pmetes of Lne Models 3.4. Refeences []. Ae, Fttng Autoegessve Models fo Pedcton, Ann. Inst. Stts. Mtch., vol. no., , 969. []. Ae, A ew Loo t the Sttstcl Model Identfcton, IEEE ns. on Automtc Contol, vol. no. AC-9, no. 6, , 974. [3] K. J. Åstöm nd P. Ehoff, Sstem Identfcton: Suve Automtc, vol. 7, Issue,. 3-6, Mch 97. [4] B. Atl nd A. nue, Seech Anlss nd Snthess b Lne Pedcton of the Seech Wve, J. Acoust. Socet of Amec, JASA, vol. no. 5, , 97. [5] G. Cnns,. Kloutsds nd D. Mnols, Fst Recusve Algothms fo Clss of Lne Equtons, IEEE ns. on Acoustcs, Seech, nd Sgnl Pocessng, vol. ASSP-3, no., Al 98,. 7-39, 98. [6] Y.. Chn nd R. Lngfod, Sectl Estmton v the gh-ode Yule-Wle Equtons, IEEE ns. on Acoustcs, Seech, nd Sgnl Pocessng, vol. ASSP-3, , 98. [7] D. W. Cle, Genelzed Lest Sques Estmton of the Pmetes of Dnmc Model, Poceedngs of the IFAC, Smosum on Identfcton, Pgue, 967. Cted n M. jm, Modélston et Identfcton en tement du Sgnl, Edton Msson, 988. [8] C. E. Dvl, A Subsce Aoch to Estmton of Autoegessve Pmetes fom os Mesuements IEEE ns. on Sgnl Pocessng, vol. 46, no., , Febu 998. [9] M. Deche, AR Pmete Estmton Fom os Dt Usng the EM Algothm, IEEE-ICASSP 94, Adelde, Austl, vol. 4,. 69-7, 9- Al 994. [] J. Dubn, he Fttng of me Sees Models, Rev. Inst. de Stts., vol. no. 8, no. 3, , 96. [] B. Fedlnde, Instumentl Vble Methods fo ARMA Sectl Estmton, IEEE ns. on Acoustcs, Seech nd Sgnl Pocessng, vol. 3, no., , Al 983. [] D. Gngs, Estmton of the Autoegessve Pmete fom Obsevtons of ose- Couted Autoegessve me Sees, IEEE-ICASSP 8, Ps, 3-5, M 98. [3] D. Gue, Identfcton nd Adtve Flteng, Robet E. Kege Publshng Comn Mlb, Flod, 984. [4] K. sn, J. ossn nd que A., Pmete Estmton of Multchnnel Autoegessve Pocesses n ose, Sgnl Pocessng, vol. 83, no. 3,. 63-6, Jnu 3. [5] K. sn nd S. A. Ftth, Identfcton of os AR Sstems Usng Dmed Snusodl Model of Autocoelton Functon, IEEE Sgnl Pocessng Lettes, vol., no. 6,. 57-6, June 3.

120 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng [6]. Klth, A Vew of hee Decdes of Lne Flteng heo, IEEE ns. on Infomton heo, vol. I-9, , 973. [7] S. M. K, ose Comenston fo Autoegessve Sectl Estmtes, IEEE ns. on Acoustcs, Seech nd Sgnl Pocessng, vol. 8, no. 3,. 9-33, June 98. [8] J. Leou nd C. Gueguen, A Fed Pont Comutton of the Ptl Coelton Coeffcents, IEEE ns. on Acoustcs, Seech, nd Sgnl Pocessng, vol. ASSP-5, , 977. [9]. Levnson, he Wene RMS Eo Flte Desgn nd Pedcton, ublshed s n end to the wo. Wene: Etolton Inteolton nd Smoothng of Stton me Sees, John Wle, ew Yo, 949. [] J. Mhoul, Lne Pedcton: utol Revew, Poc. IEEE, vol. no. 63, no. 4, , A [] J. D. Mel nd A.. G, On Autocoelton Equton Aled to Seech Anlss, IEEE ns. on Audo nd Electocoustcs, vol. AU-, , 973. [] R. K. Meh, On Lne Identfcton of Lne Dnmc Sstems Wth Alctons to Klmn Flteng, IEEE ns. on Automtc Contol, vol. AC-6, no.,. -, 97. [3] J. L. Mels nd J. D. omc, Lne Pedcton Codng wth Addtve ose fo Alctons to Seech Dgtlston, 4 th Alleton Confeence on Ccuts nd Sstems, USA, Setembe 976. [4] M. Mof, Fst Algothms fo Multvble Sstems, thess, Stnfod Unvest, Stnfod, 974. Cted n M. jm, Modélston et Identfcton en tement du Sgnl, Edton Msson, 988. [5] I. Schu, Ube Potenzehem, de Innen des Enhetsess beschänt snd, J. fü Rene und Angew. Mth, 47,. 5-3, 97. [6]. W. Soensen, Lest Sques Estmton: Fom Guss to Klmn, IEEE Sectum, , Jul 97. [7]. J. Ulch nd R.. Clton, me Sees Modelng nd Mmum Ento, Phs. Eth, Plnet Int., vol. no.,. 88, 976. [8] R. A. Wggns nd E. A. Robnson, Recusve Soluton of the Multchnnel Flteng Poblems, J. Geohs. Res., , 965. [9] W. X. Zheng, Unbsed Identfcton of Autoegessve Sgnls Obseved n Coloed ose, IEEE-ICASSP 98, Settle, USA, vol. 4, , -5 M 998. [3] W. X. Zheng, Autoegessve Pmete Estmton fom os Dt, IEEE ns. on Ccuts nd Sstems II: Anlog nd Dgtl Sgnl Pocessng, vol. 47, no.,. 7-75, Jnu. [3] W. X. Zheng, Fst Identfcton of Autoegessve Sgnls fom os Obsevtons, IEEE ns. on Ccuts nd Sstems II: Eess Befs, vol. 5, no., , Jnu 5.

121 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Mohmed jm Coght 8, ISE Ltd. Chte 3 Mtched nd Wene Fltes 3.. Intoducton Ou mn uose n ths chte s to detemne the otml flte needed to etct the sgnl fom the nose. hs otml flte cn be defned s follows: t s mthemtcl descton of the sgnl ocessng oetons tht hve to be conducted on the nos sgnl. hs descton should esect the cte of otmlt tht wll be descbed n ths chte. As elude, the followng onts should be noted: the nuts of these fltes e ethe ndom sgnls, o combntons of ndom nd detemnstc sgnls; we wll onl cove stton lne sstems n ths chte. When the fnl m s to obtn hscl mlementton, we wll lso consde the elzblt ssues of the flte. We wll consde two mn tes of fltes: the mtched flte nd the Wene flte. hese two clsses e, esectvel, the soluton to the followng cses: detectng the desed sgnl, whose she s led nown, when t s dstubed b whte o coloed nose; etctng the sgnl when both the sgnl nd the nose e ndom ocesses. When desgnng these fltes, the utocoelton functons nd mtces e ssumed to be nown.

122 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng he chte s ognzed s follows. In the fst secton, we wll loo t mtched fltes nd tet the two successve cses whee the sgnl s dstubed b whte ddtve nose nd b coloed one. In the second secton, we ecll the tdtonl esentton of the Wene flte, fst fo contnuous-tme nd then fo dscete-tme sgnls. Fo ths ltte te of sgnl, we wll etensvel quote fom the wo ced out b obet Wene whch, though fst esented n clssfed eot n 94-4, ws onl mde ublc n 949. Wene ws mong the fst to vew sgnls nd noses s elztons of ndom ocesses. 3.. Mtched flte 3... Intoducton Let t be the obsevton, whch coesonds to the sum of el sgnl s t whose she s nown o nd stton ndom nose b t. hs obsevton s ssed though flte whose mulse esonse s denoted ht. Snce the flte s lne, we hve, t nstnt t t see Fgue 3.: whee: h * t h * s t h b t s t b t [3.] * t h s t s ht s * d [3.] t h b t b ht d b * [3.3] Fgue 3.. A flte wth n mulse esonse ht Ou uose s thus to dentf the flte whch best detects the esence of s. hs flte s the best n the sense tht t should ovde the mmum sgnl-to-nose t

123 Mtched nd Wene Fltes 7 to R, defned s the to of the nstntneous owe of the sgnl s t to tht of the nose b t tme t t [7] []: t s t R t [3.4] E b t ng equtons [3.] nd [3.3] nto ccount, the bove elton s chnged to: R t whee b Eb b b nose t s h t ht ht d bb d d [3.5] s the utocoelton functon of the ddtve b. Equton [3.5] cn onl be esl solved f ths utocoelton functon s nown. Fo ths eson, let us fst loo t the cse whee b t s whte nose Mtched flte fo the cse of whte nose If nose b t s zeo-men, whte nd stton, wth utocoelton functon bb b, the denomnto of equton [3.5] s modfed to: b b h b b ht ht ht ht t d d d d d [3.6] hus, equton [3.5] becomes: s h b h t t d R t [3.7] d We loo t ths oblem of otmzng the SR t the flte outut n the fequenc domn, whee t s es to solve.

124 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng, If we ntoduce B,,, s the Foue tnsfoms of h t, s t nd b t esectvel, the Foue tnsfoms F of equtons [3.] nd [3.3] e: nd: S S S nd f S Fst s t e jtdt F bb bb b [3.8] [3.9] Substtutng [3.8] n equton [3.4] modfes the ltte to: s t R t E b t F S bb F bb t t S t t [3.] Alng the Wene-Khntchne theoem to nose b t, we obtn: R t F F S t t S b b t [3.] Followng equton [3.9], the SR cn be eessed s: R t S ejt b d d [3.] In ode to mmze the numeto of neqult whch, fo n two functons R t, we wll use the Schwz F nd G, cn be defned s:

125 Mtched nd Wene Fltes 9 F G d F d G d [3.3] hs eqult onl holds tue f F nd G stsf the followng condton: F G el whee G* s the comle conjugte of G.,. [3.4] If we use neqult [3.3] n the numeto of equton [3.] b tng G S e jt, we obtn: F nd hus, R S jt d d S d e [3.5] R t s mmzed s follows: t b S d [3.6] t hs mmum of R s ndeendent of the mulse esonse h t, nd deends onl on the sgnl eneg nd the vnce R s b. heefoe, mmzed when eqult [3.4] s esected. he esultng otml flte hs the followng tnsfe functon: t ot S j e t [3.7] whee S denotes the comle conjugte of * S. B tng the nvese Foue tnsfom of equton [3.7] bove, the mulse esonse h ot t of the otml flte cn be eessed s follows:

126 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng h ot t ejt S * S ot d e j t t d * ej t t d s t t * [3.8] If s t s el: h ot t s t t [3.9] s t. hus, the mulse esonse of the otml flte deends onl on the sgnl heefoe ths flte s mtched to the wvefom of the sgnl, ledng to the nme mtched flte. Moeove, the fcto denotes onl the gn of the flte nd hs the sme effect on both the sgnl nd the nose. When =, f s t s nown functon, h ot t s lso nown. o ensue tht the flte h ot t cn be mlemented n ctce,.e., to ensue tht t vefes t fo t, we cn choose: h ot h ot t s t t fo t fo t [3.] Equton [3.] mens tht t nstnt t, sgnl flte. heefoe: s t h u s t udu h u s t u tu ot t h ot t s d Insetng eesson [3.] fo the mulse esonse ot s t hs ssed though the du h ot t : s t h us t udu s t us t u tu t ot s t t s d du [3.]

127 hs equton cn be elted to the utocoelton functon of Mtched nd Wene Fltes s t gven below: ss t lm s s t d [3.] o hghlght the dffcult n choosng t, let us consde smle emle: sgnl defned s the eonentl functon of tme constnt such tht: s t e t / fo t fo t [3.3] the mulse esonse of the mtched flte s then defned s follows: fo t t st t fo t h ot t e t t/ fo t t [3.4] fo t fo t = s t 3.5 t = 3.5 h ot mltude mltude t = t = t t Fgue 3.. Reesenttons of sgnl nd the mulse esonse of the mtched flte hus, the smlle the vlue of t wth esect to the totl duton of the sgnl, the less ect the omton of the flte mulse esonse wll be. o obtn the best flte,.e., the flte wth the hghest SR, t hs to be ve hgh dell nfnte. Unfotuntel, ths s not elstc fo el-tme lctons, whee t s eltvel smll. hus, the el flte wll be subotml.

128 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Mtched flte fo the cse of coloed nose Fomulton of oblem In ths secton, we wll tet el sgnl dstubed b coloed nose wth utocoelton functon. o stt, let us consde equton [3.5]: b b R t s E b t t s h ht ht b b t d d d [3.5] A mmzton of R t coesonds to mnmzton of the nose owe E b t fo gven vlue of s. hs cn be seen s oblem of t otmzton unde constnts whch entls the mnmzton of the followng quntt: t s t Q E b [3.5] whee s clled the Lgnge multle. Q cn lso be eessed usng h t [7]: Q b b s h ht ht t d d d [3.6] hs sech fo h ot t chctezng the otml flte cn be conducted usng tdtonl vtonl clculton technques. Fst, h t n equton [3.6] s elced b: t h t h t h ot [3.7] Mmzng R t s lso equvlent to mnmzng E b t fo gven vlue of s t. Such n och would lso led to otmzton unde constnt, wth the Lgnge multle.

129 whee s el vble nd h t s n bt ncement. Mtched nd Wene Fltes 3 hs leds to n eesson of Q s functon of, whch wll be denoted Substtutng h t of equton [3.7] n the eesson fo Q, we obtn: Q. Q b b h t h t h t h t h t h t s d ot ot ot dd [3.8] hus, Q Q s second-degee olnoml of such tht: Q A B. o mnmze Q, the followng condton, whch guntees the esence of n etemum, must be esected: Q [3.9] whee: Q s b b b b h t h t h t h t b b ot ot h t h t h t d dd dd dd [3.3] If we te nto ccount equton [3.9] nd the even oet of the,, we see tht: utocoelton functon,.e., bb bb

130 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Q bb s h t h t h t d ot dd [3.3] Equvlentl: Q h t h t d s ot bb d [3.3] Relton [3.3] must be stsfed esectve of the vlue of the ncement h. heefoe, we hve: h ot t d s bb fo t when consdeng elzble flte. he fcto chnges onl the gn of the flte, nd thus cts equll on both the sgnl nd the nose. If we te =, h ot t s full nown, bng constnt fcto. he genelzed equton fo mtched flte, n the cse of coloed nose s thus: hot t d bb s. [3.33] he bove equton belongs to the Fedholm equton fml. he soluton of ths equton ovdes the eesson of the otmum flte. Q o me sue tht s suffcent condton, besdes beng necess s shown bove, the sgn of the second devtve hs to be vefed: Q h t h t dd bb

131 Mtched nd Wene Fltes 5 Snce b b s ostve b defnton, Q wll be ostve o zeo Phscll unelzble mtched flte If thee s no need fo elzble flte, eesson [3.33] fo the mtched flte becomes: h ot t d s bb fo [3.34] hs convoluton equton s smlfed when teted n the fequenc domn. Alng the Foue tnsfomton to both sdes of equton [3.34] gves: s h ot t d e j bb e j d d [3.35] If S b b denotes the PSD, the Foue tnsfom of the nose utocoelton functon t, the bove equton s modfed to: b b h t e j S d S ot bb [3.36] Substtutng t, equton [3.36] becomes [7]: jt S h ej d S e bb ot [3.37] Equvlentl: jt S S e bb ot [3.38] Rengng the tems of ths cuent equton, we obtn: ot S e jt [3.39] S bb

132 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng hs mght consttute n omted soluton when usng ecodng dt,.e. whee the st nd the futue of the sgnl wth esect to n ont n tme e nown, nd the ntegl fom to s ght A mtched flte soluton usng whtenng technques In ctce, the eondent nd delcte chllenge s the soluton of equton [3.33], whee the mn dffcult ses fom the ue lmt of the ntegton beng t nd not +. We wll esent the whtenng ocess often used to solve the Fedholm equtons. hs technque ls n motnt t n the es of detecton nd estmton nd n theoetcl hscs [7]. We wll fst ntoduce the sectl fctozton nd then te u the esoluton of equton [3.36] tself. Fst nd foemost, let us suose tht S s the outut of flte wth tnsfe functon G ected b zeo-men whte nose wth vnce equl to. hs cse s llustted n Fgue 3.3. We thus hve: S G. [3.4] D.S.P. D.S.P. S ðw Fgue 3.3. Whte nose flteng Fo lne, nvnt flte wth loclzed constnts, G cn be eessed s: G b b n n n j j j j [3.4] * hus, the PSD t the flte outut, G G G wtten s: n D S, cn be S [3.4]

133 Snce the ocess s el, ts utocoelton functon beng even, t follows tht Mtched nd Wene Fltes 7 s lso el. Fom S s el. Consequentl, n nd D e two constnt-coeffcent olnomls wth degees n nd esectvel. It cn esl be shown tht S tes the followng fom [7]: S S S whee S S. he oles nd zeos of ght hlf nd those of S e locted n the S n the left hlf of the comle s-lne. It follows tht the flte, whose tnsfe functon s, s hscll elzble. S Let us consde the whtenng ocess, whch conssts of flteng ndom coloed ocess to obtn whte nose. Fgue 3.4. he whtenng ocess hus: Seeng Fgue 3.4, S * S S ; equvlentl: [3.43] Snce S S, the bove equton s stsfed f S * S S.

134 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng hus, the hscll elzble whtenng flte hs the followng tnsfe functon: S [3.44] o solve equton [3.33] fo genel cse,.e. when the sgnl s obseved n the esence of stton coloed nose, the followng soluton esents tself ntutvel: fst whten the coloed nose, nd then tet t s the detecton of sgnl dstubed b whte nose. We cn show tht the otml soluton s n fct the followng: use the fst flte, wth fequenc esonse, to whten the nose, followed b the second flte, wth fequenc esonse X w, mtched to sgnl dstubed b whte nose. A dgm of ths lgothm s dected n Fgue 3.5. t s b S S b b t S b b whtenng t s ' b' S' t S b ' b' Mtched flte fo whte nose t s b S S b b t Fgue 3.5. Bloc-level schemtc; coloed nose cse Accodng to equton [3.44], we hve: S S S ' [3.45] S b b Let us s tht the flte s mtched to sgnl * S' jt s' t, defned n equton [3.7]: e [3.46] he tnsfe functon of the mtched flte, usng equton [3.45], s modfed to: A new och bsed on nne/oute fctozton hs ecentl been oosed nd deved fo Rlegh fdng chnnel smulto nd tetue chctezton.

135 Mtched nd Wene Fltes 9 S e jt S bb * [3.47] hus, we obtn n elct eesson of the soluton whch leds to the tnsfe functon of the otml hscll elzble flte [4] [7]: ot S b b S S b b e jt t d e jt dt It cn lso be shown tht ths eesson s soluton to the followng equton: t h ot t d s bb [3.48] In ths fst t, we hve ntoduced mtched flteng when the sgnl s dstubed b whte nd coloed ddtve nose. hs flte s used n the feld of d ocessng nd telecommunctons fo nstnce. In the followng secton, we wll loo t Wene flteng when delng wth contnuous-tme nd dscete-tme sgnls he Wene flte Intoducton he Wene flte, descbed b the Wene-of equton, hs led n motnt ole n the ntoducton of the noton of otml fltes [9]. he motvton behnd descbng the Wene flte n ths chte s twofold: ths flte s non-ecusve soluton to ovde soluton to otml flteng, nd hs seved to develo the ecusve och of the Klmn flte. Even though the Klmn flte, esented n Chte 5, s bsed on n lgebc och, t s nstuctve to stte tht the hstocl och ntoduced b R. E. Klmn leds to efomulton of the otml flteng ssue nd to fomultng t s ecusve soluton of dffeentl stochstc equton, n ste of solvng Fedholm equton te. Moeove, n the cse of dscete-tme sgnls, we cn consde the Wene flte s metc modelng och bsed on the lest sques method. hs flte lso benefts fom close coesondence wth the stochstc gdent lgothm clled the lest men sques LMS, esented n Chte 4.

136 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Fomulton of oblem t s t b t b t ndom nd el s t ndom nd el ht sˆ t s t b t Fgue 3.6. Lne tme-nvnt flteng he sgnl t s the sum of the desed sgnl s t nd n ddtve nose b t. Both the sgnl nd the nose e ndom stton ocesses. We hve to fnd the lne flte, wth mulse esonse ht, whch gves the best estmton ˆ t of t. Othewse stted, we wnt: s s s ˆ t s t b t h t d [3.49] to be s close to s t s ossble. he eo s defned s the dffeence between the estmted sgnl desed sgnl s t : s t ˆ nd the t s t s t [3.5] e ˆ he Wene flte, whose mulse esonse s denoted b uon mnmzng the men sque eo MSE: whee e t E s t s ˆ t h ot t, s obtned J E [3.5] E. s the mthemtcl eectton. Fo egodc sgnls, equton [3.5] s modfed s follows: J lm e t dt [3.5]

137 Mtched nd Wene Fltes he Wene-of equton Cteon [3.5] s defned fom the sgnl s t whch s, howeve, not vlble. As we wll soon see, s t does not dectl ntevene n the devton of the flte, onl though ts utocoelton functon. We wll lso see tht o nowledge of ths functon s necess fo obtnng the Wene flte. Combnng equtons [3.49] nd [3.5], cteon J s eessed s follows: J E e t E s t h t d [3.53] Mng use of the lnet of the mthemtcl eectton, we obtn: J E s E t E s t h t d h h t t dd [3.54] Equvlentl: J E s t h Es t t h h E t t d dd [3.55] Let us ntoduce the followng thee functons: the utocoelton functon of sgnl s t, the utocoelton functon of obsevton t, nd the cosscoelton functon between s nd t. hese e denoted esectvel s: nd Es t s t s s Et t Es t t s t.,

138 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng ng nto ccount the smmet of the utocoelton functon 3,.e., the eesson fo cteon J s modfed to: J E e t h d h h dd ss s [3.56] o obtn the mnmum of ths cteon, we wll use ocedue sml to the one used fo the mtched flte n secton 3..3: t h t h t h ot [3.57] E e t becomes functon of, denoted b J. Fo mnmum, t s necess tht: J to hve J. [3.58] whee: Plcng eesson [3.57] of h t n equton [3.56], we obtn: J h h ss h h h h ot s ot d We cn eess J s second-degee olnoml of : J J C D d d ot [3.59] [3.6] h h D dd [3.6] 3 Let Eb t b t, Es t b t nd Es t t b b t s t b t s b s Snce, we hve s s bb sb b s. If s t nd b t e both stton nd uncoelted, s modfed to: becuse ss bb sb bs.

139 Mtched nd Wene Fltes 3 nd: C s h h d h h h s ot ot d d d d [3.6] he bove elton should be esected esectve of the vlue of the ncement h. hus, t follows tht: h d h * ot s ot [3.63] hs ntegl, Fedholm equton of the fst nd, s clled the Wene-of equton. o detemne the soluton h ot t of ths ntegl, we need to now the cosscoelton s nd the utocoelton. We wll consde the sgnl nd the nose to be uncoelted. We notce tht the sgnl nd the nose do not dectl ntevene n equton [3.63]. Equton [3.58] esents the necess condton fo mnmzng J hve: J. As we D, [3.64] we cn esl chec tht the suffcent condton s fullflled to ensue tht mnmzed. J s So f, we hve not dscussed whethe the flte cn be mlemented. Fo eltme ocessng, whee the flte hs to be elzble, the lowe lmt of the ntegl equton [3.63] hs to be chnged fom to. hs does not smlf the esoluton of the Wene-of equton, becuse we eclude the use of the Foue tnsfom. If the oblem of hscl elzton of the flte s not consdeed, we cn te the Foue tnsfom of equton [3.63]. he Wene-of equton chnges to:

140 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng S S [3.65] s ot Rengng the tems of the bove equton, we get the followng modfed fom: ot S s S S s S s s b [3.66] S If the sgnl s uncoelted to the nose, tht s f S equton chnges to: ot sb, the bove [3.67] S s s S s s S S nd b b whee s s S b b denote, esectvel, the owe sectl denstes of the sgnl s t nd nose b t Eo clculton n contnuous hscll non-elzble Wene flte ng eesson [3.56] fo the eo, nd wtng t fo h t h t, we hve: J mn E e s s t h s d h h d d ot ot ot ot [3.56] Usng the oet of evenness of the utocoelton functon, nd usng the Wene-of equton [3.63], we cn ntoduce the quntt s n the double ntegl of equton [3.56]: J mn ss h ot h ot h ot s d d d

141 Mtched nd Wene Fltes 5 Smlfng the bove equton, we obtn: d h J s ot s s mn Relcng ss nd s b the nvese Foue tnsfoms: d e S j s s [3.68] d S s s s s [3.69] we obtn: d S S d S d S d d e h S d S d d e S h d S J ot s s s ot s s s j ot s s s j s ot s s mn [3.7] he bove elton s smlfed f we use eesson [3.66] of ot : d S S S S d S S S S S d S S S S J s s s s s s s s s s s * * * * * mn [3.7]

142 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng If s t nd t b e not coelted, we get S S S [3.7] s s b b S S. [3.73] s s s heefoe, equton [3.7] s modfed s follows: S s s Sb b J mn d [3.74] S S s s * b b he bove equton s useful n elnng the esults obtned fo the Wene flte. If the PSDs of the sgnls s t nd b t e oveled, the oduct S s s S bb s zeo nd leds to zeo estmton eo. he Wene flte s then no longe equed to sete the sgnl fom the nose. A smle low-ss o hgh-ss flte wll elmnte the nose. hs smle cse s llustted n Fgue 3.7. In ctce, we notce n ovel of the sgnl nd nose sect. See Fgue 3.7b. hus, the Wene flte s the onl soluton towds obtnng sgnl * estmton. he oduct S s s S bb n the fequenc bnd of nteest s no longe zeo, nd, consequentl, the estmton eo s lso lws non-zeo. S Fgue 3.7. he owe sectl denstes of the sgnl nd the nose

143 Mtched nd Wene Fltes Phscll elzble contnuous Wene flte. Rtonl sect cse Let us te the Wene-of equton obtned n secton 3.3.3: h d ot s [3.63] Fo the flte to be hscll elzble, t s necess tht: hot s d [3.75] Let us ssume the obsevton to be whte nd to hve zeo men nd vnce,.e., nd. he bove equton becomes: S hot d s. [3.76] hs leds to the eesson of the otml Wene flte s functon of : s s t t hot t [3.77] t he bove hothess of whte nose s not elstc one. he ndom, stton sgnls hve PSD whch cn be omted b n even, tonl, nonnegtve functon. We thus hve to loo fo secfc soluton of the Wene-of equton fo the cse whee: S [3.78] D We cn use the foementoned esults obtned n secton on the whtenng flte h t whch tnsfoms sgnl t nto t. Sgnl t s whte nose wth S. he PSD of s then gven b: t

144 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng S S [3.79] nd, snce s whte nose wth PSD =, we obtn: t [3.8] S hs sgnl s then flteed b hscll elzble otml flte whose t mulse esonse s gven b: h s fo t t t fo t [3.8] We decomose the flte, whose mulse esonse s h, nto two cscded fltes. he fst, whose fequenc esonse s desgnted mes t ossble t to comenste fo the whtenng. he second s defned b fequenc esonse. he m behnd ths decomoston s to vod the use of the whtenng b flte. hus, s the nvese of. Flte s the otml flte: ot b b [3.8]

145 Mtched nd Wene Fltes 9 Fgue 3.8. Ovell schemtc of the ocess he PSD of t cn be omted to be tonl, even, non-negtve functon. heefoe: S whee z nd n j z j z [3.83] D d * j j e, esectvel, the th zeo nd the th ole of S. We choose the zeos z to le n the left hlf of the s-lne nd hlf the zeos on the mgn s, to fom olnoml. Smll, we cn fom the olnoml D. Flte cn thus be eessed s: *

146 3 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng D [3.84] S he eesson fo chnges to: [3.85] D o clculte the mulse esonse h of the flte, we fst need to clculte the coss-coelton functon Es t t hus: t s t h t d h whee: u t u du [3.86] h u s u du h s s Alng the Foue tnsfom to ths oduct, we obtn: S [3.87] D S s S s [3.88] S s whee olnoml s comsed of the zeos of S lng n the ght hlf of the comle s-lne nd hlf the zeo s lced on the mgn s. D s constucted n the sme w, usng the oles. Snce sgnl s whte nose wth PSD =, the otml flte n the tme domn s: t h s fo t t t fo t [3.89] hs s cusl flte. he coss-coelton functon t s obtned uon tng the cusl t of the nvese Foue tnsfom of owe nte-sectl denst S. s s

147 Mtched nd Wene Fltes 3 Moeove, decomosng s S gves: s s s S S S [3.9] whee the functons S s nd S s e detemned b combnng the tems of the Luent sees of s S nto two gous: S s coesonds to the oles of the left hlf-lne; nd S s to the oles of the ght hlf-lne. he nvese Foue tnsfom of S s leds to the cusl comonent t s whle the nvese Foue tnsfom of S s leds to the nt-cusl t t s. t s thus stsfes the followng condton: othewse t d e S t t j s s [3.9] Usng [3.89], we get t t h s whch, when tnslted nto the fequenc domn, gves: s s S D S [3.9] he otml flte thus esects the followng condton: s b ot S D D [3.93]

148 3 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Equvlentl: s ot S S S [3.94] If the sgnl t s nd the nose t b e not coelted: s s s S S nd: b b s s S S S. ng the bove two equtons nto ccount, equton [3.94] becomes [7]: b b s s s s b b s s ot S S S S S [3.95] So f, we hve ld emhss on the tonl sect, whch cn be used to omte the non-tonl sect. he one necess nd suffcent condton fo fctoblt s fo the ntegl d S log to be convegent. Let us detemne the otml flte nowng tht S b b.e., when b s zeo-men whte nose, nd j s s s s S. Uon tng, we hve: b b s s S S j j j j S S S

149 Mtched nd Wene Fltes 33 nd: nd: S hus: S s S S s s j j s S s S Fnll: ot j j j j j vng esented the Wene flte fo contnuous tme sgnls, we wll loo t the flte fo cses whee the sgnls e dscete-tme Dscete-tme Wene flte In ths secton, we wll descbe dgtl flte, wth mulse esonse h, whch fltes sgnl to oduce n outut d ˆ whch s closest to the desed outut d. d h d ˆ e Fgue 3.9. Dscete-tme Wene flte

150 34 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Fnte mulse esonse FIR Wene flte Let us te dgtl flte wth mulse esonse metes h,..., conctented n column vecto s follows: h h [3.96] Let us lso te smles of the nut sgnl, denoted, stoed n vecto s follows: X,, X [3.97] he outut sgnl d ˆ wll thus hve the followng eesson: d ˆ X X h. We cn defne the eo sgnl e between the desed esonse nd the flte outut: e d dˆ d X d X [3.98] he men sque eo MSE, denoted J, s defned s follows: e E d X J E [3.99] Endng the bove defnton nd usng the lnet of the mthemtcl eectton, we obtn: J E E d Ed X E X X d R d R whee R d Ed X desed esonse d nd the nut sgnl, nd R E X X [3.] s the coss-coelton vecto between the s the

151 Mtched nd Wene Fltes 35 utocoelton mt of the nut sgnl. It should be noted tht J s defnte qudtc fom of. We cn obtn the mnmum vlue of J b clcultng: J ot Equvlentl: J h J h ot ledng to: ot R R d o: ot R R [3.] d whee ot s the otml Wene soluton. hs bove equton, whch detemnes the mnmum MSE flte coeffcents, s the Wene-of o noml equton. It s the dscete-tme equvlent to the Wene-of equton fo contnuous-tme cses. he estmton of the sgnl utocoelton mt R nd the coss-coelton vecto R eques lge numbe of elztons, whch s not elstc n ctce. d ot We cn clculte the mnmum eo b elcng flte b ts equvlent eesson gven n equton [3.] nd nsetng t nto equton [3.]:

152 36 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng J mn E E E E d R R d R d R R d d R d R R d R d R d R d R R d ot d R d R R d R R d [3.] he eo, gven b equton [3.], cn be eessed n the followng fom [8]: ot ot R J J [3.3] mn Indeeed, let us end equton [3.3]: J ot ot J mn R R [3.4] ot ot R R All the tems of equton [3.4] e scl qunttes nd thus equl to the tnsoses. hus, we notce tht the lst two tems e equl to one nothe. Relcng J mn b ts eesson, equton [3.4] s modfed to: ot ot ot ot Ed R R R R J Smll, elcng J E E d ot b ts eesson [3.] gves: d R d R R d R R R d d R R d If we ntoduce the tem V flte follows: nd the otml Wene flte ot R d, whch s the dffeence between the ot, we cn ewte equton [3.3] s J J V R V [3.5] mn R d

153 Mtched nd Wene Fltes 37 R s ostve b defnton, snce t s n utocoelton mt. hus, V R V fo ll V. B consequence, J s ostve quntt. Consdeng equton [3.3]: ot ot R J J [3.3] mn J deends on the utocoelton mt R, secfcll on ts egenvlues nd the coesondng egenvectos,...,,..., We ecll tht:. R, nd the egenvlue decomoston of the utocoelton mt PDP R R gves: whee D s the dgonl mt of the egenvlues nd P s the mt constucted fom the othogonl egenvectos. We cn lso choose nomlzed egenvectos; ths mes P n othonoml mt. hs lst eesson s nown s the noml fom of R. Let us consde equton [3.98]: e d X Multlng both sdes b X nd then clcultng the mthemtcl eectton, we obtn: hus: X e X d X X E e X R d R [3.6]

154 38 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng We notce tht f the flte tes ts otml vlue,.e., f e X ot, then E. hs mens tht the eo s othogonl to the sgnl vecto X. Fo ths eson, the sstem of equtons R R d s clled the noml equtons Infnte mulse esonse IIR Wene flte In ths secton, ou uose s to fnd dgtl flte wth nfnte mulse esonse h ot. Gven the flte nut, the flte outut d ˆ stsfes: ot d ˆ h [3.7] Let us eess the men sque eo J, b comng the outut d ˆ of the dtve flte wth the desed outut d : dd J E e E d dˆ E d h Intoducng the tems d ot E, E E d, the bove equton s wtten s: ot j d [3.8] d nd J h h h j j [3.9] dd ot o mnmze J, we obtn: d ot hot j j j h j* j j j [3.] d ot d hs s the dscete Wene-of equton fo non-cusl fltes.

155 Mtched nd Wene Fltes 39 If we l the dscete Foue tnsfom to both sdes of equton [3.], the tnsfe functon of the otml flte becomes: S d ot [3.] S d j j whee S d e nd S e e, esectvel, the nte-sectl denst nd the owe sectl denst of the nut flte sgnl. Bsed on the bove theoetcl nlss, let us eecse ths otml flteng n the contet of seech enhncement n the followng secton Alcton of non-cusl dscete Wene flte to seech enhncement Modfed flte eesson Let thee be nos sgnl o messge defned b: s b [3.] whee s s the seech sgnl nd b zeo-men ddtve whte nose. hen, let us flte the nos sgnl usng non-cusl Wene flte to enhnce the sgnl s. hs flte s gven b equton [3.]: S s ot [3.3] S We suose tht the seech sgnl nd the nose e not coelted: S S S nd S S [3.4] ss bb he otml Wene flte s thus defned s follows: ot s S ss [3.5] S S ss bb o enhnce the seech sgnl, we fst use nnng wndow on the nos sgnl, gvng se to the wndowed sgnl. he lengths of the nlss fmes w ss

156 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng ves between nd 3 ms. he dffeent fmes m be totll ndeendent, o oveled. An ovel n the nge of 5% to 7% s genell doted. In Chte 6, we wll ovde moe detls on ths mtte. In ode to obtn the shot tem estmton Sˆ of the ognl sectum S w, we use the Wene flte on the nos wndowed sgnl w, denoted X w n the fequenc domn: Sˆ X [3.6] w ot w Fnll, usng n nvese Foue tnsfom llows us to estmte the wndowed sgnl. he econsttuton of the enhnced sgnl s done b lng the ddtonovel method. he Wene flte s zeo-hse flte. We thus omte the hse of the estmted sgnl to be the sme s the hse of the nos sgnl, becuse the humn e s nsenstve to the hse vtons of sgnl. o clculte the otml flte n equton [3.5], we need to estmte the PSD of the seech sgnl s well s the PSD of the nose. Snce the seech sgnl s nonstton, we cn consde S w, whee S w s the shot-tem Foue tnsfom of sgnl s w, nsted of the sgnl s PSD S. he nose PSD Sbb s omted b vegng vege s denoted nose b w. w ss B w ove sevel slent fmes. hs B w, whee B w s the shot-tem Foue tnsfom of ng nto ccount the bove consdetons, the eesson fo the Wene flte s modfed to: ot S w [3.7] S B w w Combnng equtons [3.6] nd [3.7] nd elcng the sgnl s PSD b ts estmted vlue, we obtn the followng eesson:

157 Mtched nd Wene Fltes 4 ˆ ˆ ˆ w w w w w X B S S S [3.8] he fst soluton of ths equton s the zeo soluton, whle the second s gven b: / ˆ w w w B X S [3.9] hs eesson s sml to tht obtned uon the technque clled sectl subtcton [] [3]. o llevte the muscl nose 4, nose-suesson fcto cn be consdeed such tht: ˆ ˆ ˆ / w w w w w X B S S S [3.] A vnt of the Wene flte, clled the modfed Wene flte, hs been develoed n []. he dvntge confeed b ths non-tetve och s tht we cn educe the ddtve nose wthout ntoducng n dstoton to the seech sgnl. In tht cse, the sgnl s sectl denst s estmted s follows: w b b ss X E E E S E E E S [3.] whee E nd b E e, esectvel, the eneges of the nos sgnl nd of the nose. Equtons [3.] nd [3.] fo = gve the followng eesson fo the Wene flte: 4 A muscl nose s n esdul nose whose tonl comonents e ndoml dsesed n tme nd fequenc.

158 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng X w ot [3.] E X w Bw E Eb Fo n gven vlue of, we cn wo n n tetve mnne [6]. At n / teton, we estmte the otml flte usng the estmton Sˆ w, : Sˆ w,, [3.3] Sˆ w, E Bw / We cn then deduce the estmted Sˆ w, s follows: Sˆ,, Y [3.4] w w Fgue 3. shows how to enhnce the seech. w Shot tem Foue nsfom X w S ˆ Attenuton t ech fequenc w Invese shot tem Foue nsfom sˆ w Cncellton ule Estmton dung slent ntevls B w Fgue 3.. Schemtc of seech enhncement

159 Mtched nd Wene Fltes Eementl esults Let us consde the emle of seech enhncement nsde hgh-end vehcle tvelng t 3 m/h 5 on the hghw. Fgue 3. shows the PSD of the nose ecoded n the vehcle. Fgues 3. nd 3. show, esectvel, ecodngs of the nos sgnl, of the Wene-flteed sgnl nd of the ognl sgnl. Amltude Fequenc Smles 5 6 Amltude b Fgue 3.. ose sectum nd tme-domn eesentton of the nos sgnl b 6 6 Amltude Amltude Smle Smle b Fgue 3.. Enhnced sgnl nd ognl seech sgnl b Enhncement usng combnton of AR model nd non-cusl Wene flte Let the sgnl be eesented b th -ode utoegessve ocess: s s u [3.5] 5 hese sgnls hve been ndl ovded b Mt Comn.

160 44 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng whee s s the seech sgnl,,..., e the edcton coeffcents, nd u s the whte Gussn zeo-men nut wth vnce u. Moeove, the ddtve nose b s whte Gussn zeo-men nose wth vnce b. Let the sgnl column vecto, the nos obsevton vecto nd the AR mete vecto be defned esectvel s follows: s [ s s ] [3.6] [ ] [3.7] [ ] [3.8] he method we oose conssts of gettng the mmum osteo estmton of the model metes. he estmton of the coeffcent vecto should thus mmze the osteo obblt: ˆ gm s [3.9] Lm ntoduces n tetve method whch leds to subotml soluton [6]. At ech teton, two stes e followed: the coeffcents of the AR model e fst estmted usng the technque of mmum osteo: ˆ g m ˆ ˆ n s s n-,, u, n [3.3] s ˆn- s the estmton of the seech vecto t teton n- nd ˆ u, n s the estmton of the vnce of ocess u dung teton n-. In ctcl cses, the coelton method s used to estmte the coeffcents. o udte the vnce of sgnl u, we use the Psevl theoem s follows:

161 Mtched nd Wene Fltes 45 ˆ ˆ u, n d ˆ b j, ne [3.3] ˆ b whee s the estmton of the ddtve nose s vnce, clculted o udted dung the slent ntevls; n the second ste, we estmte the sgnl vecto of the seech: gm s ˆ,, ˆ sˆ n n u,n s hs cn be ced out usng non-cusl Wene flte: b Sˆ ss,, [3.3] Sˆ, ˆ ss whee the estmton of the shot tem PSD of the seech, t teton n, s eessed s follows: Sˆ ss ˆ u, n ˆ, [3.33] j, ne he schemtc of ths method s gven n Fgue 3.3. b s s ˆ Wene flteng Estmton of coeffcents Fgue 3.3. A non-cusl Wene flte usng n utoegessve sgnl model

162 46 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Even though ths och does not dd n muscl nose, t esents two mjo dsdvntges: the numbe of tetons ves ccodng to the ntue of the fme beng teted voced 6, unvoced 7, slence, etc; dstoton of the seech n the enhnced sgnl cn mnfest tself f the numbe of tetons s ncesed beond cetn lmt. o llevte the bove oblems, the followng constnts e mosed on the coeffcents obtned fte ech teton, so s to obtn stble model: the oles must be close to the unt ccle nd to ech othe n the z-lne; thee should be no but chnge n the loctons of the oles fom one seech fme to nothe, nd fom one teton to nothe. he ltte condton s dffcult to stsf n ctce, becuse t eques the emnent clculton of oles t ech ont n tme. o educe the clculton costs, nsen nd Clements oose the use of lne sectum s [5]; see Aend D fo moe detls. Usng the oetes of the olnomls Pz nd Qz, nsen nd Clements oose two tes of constnts: ntelot nd ntlot. he ntelot constnts consst of smoothng the oston coeffcents usng tngul wndow. he wdth of the wndow s chosen deendng on the te of seech fme voced, unvoced, slence, etc. he oston coeffcents e closel elted to the ostons of the shes. Fo the fst coeffcent, the smllest wdth of the wndow s chosen so s not to dstub the ecetul chctestcs of the seech. As f s ntlot constnts e concened, the m be mosed on the oston coeffcents o on the utocoelton functon Refeences [] L. Asln, A. McCee, V. Vswnthn, ew Methods fo Adtve ose Suesson, IEEE-ICASSP 95, Detot, Mchgn, USA, vol.,. 8-85, 8- M 995. [] M. Beout, R. Schwtz, J. Mhoul, Enhncement of Seech Couted b Acoustc ose, IEEE-ICASSP 79, Wshngton DC, USA,. 8-, -4 Al 979. [3] S. F. Boll, Suesson of Acoustc ose n Seech Usng Sectl Substcton, IEEE ns. on Acoustcs, Seech nd Sgnl Pocessng, vol. no. 7,. 3-, Al A sound s clled voced f ts oducton s ccomned b vbton of the vocl chods. It ossesses qus-eodc chcte. Emles of such sounds e vowels. 7 A sound s clled unvoced f no vbton of the vocl chods ccomnes the oducton of the sounds. Emles e the sounds //, /t/ o //.

163 Mtched nd Wene Fltes 47 [4]. L. Vn ees, Detecton, Estmton nd Modulton heo, Pt I, John Wle & Sons, Inc., 968. [5] J.. L. nsen, M. A. Clements, Itetve Seech Enhncement wth Sectl Constnts, IEEE-ICASSP 87, Dlls, es, vol.,. 89-9, 6-9 Al 987. [6] J. S. Lm, A. V. Oenhem, Enhncement nd Bndwdth Comesson of os Seech, Poceedngs of the IEEE, vol. 67, no., , Decembe 979. [7] J. B. homs, An Intoducton to Sttstcl Sgnl Pocessng, John Wle & Sons, Inc., 969. [8] B. Wdow, S. D. Stens, Adtve Sgnl Pocessng, Englewood Clfs, Pentce ll, 985. [9]. Wene, Etolton, Inteolton nd Smoothng t Stton me Sees, Wle & Sons, ew Yo, 949. [] L. A. Zdeh, J. R. Rgzzn, Otmum Fltes fo the Detecton of the Sgnls n ose, Poc. IRE, vol. 4,. 3, 95.

164

165 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Mohmed jm Coght 8, ISE Ltd. Chte 4 Adtve Flteng 4.. Intoducton he clssfcton of dtve lgothms cn follow vous ules. onetheless, ll ecusve oches cn be wtten unde the followng genelzed fom: K F, [4.] whee ll the metes combned n vecto e udted usng the functon F.. hs functon s secfc fo ech tcul lgothm nd genell deends on stte vecto. he mete K s weghtng coeffcent. Its eesson deends on the tcul lgothm tht s studed. In ddton, K m be used to esect tcul otmzton cteon, to ensue the convegence of the lgothm, etc. hs chte s dedcted to ecusve lgothms whch eque no o nfomton. hese lgothms e vestle: the djust themselves ccodng to the sttstcl nlss ced out on the obseved sgnls. Fo ou uoses, n dtve flte wll be defned s dgtl flte whose coeffcents e udted ove tme ccodng to the ote cte. As shown n Fgue 4., X s the vecto whch conctentes the lst vlues, u to the nstnt, of the nut sgnl; denotes the flte outut nd d the desed

166 5 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng esonse. We hve to fnd the otml flte tht mes t ossble to mnmze the eo e, wth esect to ote cte, between the flte outut nd the desed esonse [3] [6] [3]. d X e Fgue 4.. Adtve flte he lest-sques cteon s the most often used fo ths flte becuse t leds to smle esults. he fltes cn ethe be fnte mulse esonse FIR fltes o nfnte mulse esonse IIR fltes. As fo the stuctue, t s ethe tnsvesl o lttce. In wht follows, we wll close ttenton to th -ode dtve FIR fltes. he ot m s to ogessvel educe the dffeence between the otml flte nd the dtve flte b successve tetons, sttng fom bt ntl vlues fo the dtve flte coeffcents. wo mn oches cn be consdeed fo solvng the lest sques queston: the fst of these s bsed on the ecusve lest sques RLS lgothm. hs lgothm s lso clled the ect lest sques lgothm ; the second och s bsed on the stochstc gdent method nd s genell nown n sgnl nd mge ocessng s the lest men sques LMS lgothm. We wll emhsze ths second och nd esent the followng two vnts of the LMS: - the nomlzed lest men sques LMS, - the ffne ojecton lgothm APA. We wll lso come the LMS to the ect lest sques lgothm. hs elboton s sml to tht of the ecusve estmton of the AR metes esented n Chte. o me use of the homogenous nottons, we wll befl te u the RLS estmto hee. Moeove, ths wll llow dect comson wth the efomnce of the second och esented bove.

167 Adtve Flteng 5 he stochstc gdent fml of lgothms s ve oul thns to the smlct of ts mlementton. hs omton s f too oblemtc to be mthemtcll legtmte. It hs been nlzed fom dffeent obblstc vewonts, ngng fom the mtngle theo to the theo of lge devtons the obblstc nlss of e events, b eltng t to the stochstc omton theo [] [] [4] o to the odn dffeentl equton ODE nlss [4] [5]. Moe ecentl, eseches t Stnfod hve shown tht LMS s otml n the sense of the nom []. We wll notce s we go long tht the dffeent lgothms e dstngushed b the: ecson; numecl comlet; convegence seed whle tetng the vton of the dffeence between the otml flte nd the dtve flte constucted fom successve tetons. Moeove, the behvo deends on the te of nut nd on the esence o bsence of ddtve nose. he use of these lgothms wll be llustted though the estmton of the AR metes nd though seech enhncement. 4.. Recusve lest sques lgothm We wll fst esent the RLS fml of lgothms. Snce 95, vst mount of wo hs been ced out on ths toc. It s genell cceted tht Plcett [9] fst ntoduced them. Snce the 97s, some fste vesons of ths lgothm hve been develoed Ect RLS method houghout ths secton, we wll use the followng notton: h h [4.] X [4.3]

168 5 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng In Fgue 4., we see tht the flte outut sgnl / t tme nstnt stsfes: / hl l l [4.4] X X he lest sques cteon J conssts of mnmzng the sum of the sques of the element eos between the desed esonse nd the flte outut,.e.: d / J [4.5] Combnng equtons [4.4] nd [4.5], we obtn: J d X X X d [4.6] Usng the sme och we evousl descbed n Chte, we cn detemne the flte J to be mnmzed: whch llows fstl, J, the gdent of J tl devtves of should be zeo: J h j, defned s the column vecto of the h j J wth esect to the flte coeffcents j,..., j,..., [4.7] moeove, the essn mt J comosed of the second tl J of J, must be ostve defnte. devtves h h j,

169 Adtve Flteng 53 whee: nd: ng nto ccount equtons [4.6] nd [4.7], we obtn the followng eqult : R R [4.8] R R d d X X [4.9] X d [4.] If R s nvetble, we cn eess s follows: R R [4.] d We must fnd ecusve estmton ocedue fo the flte. Moe secfcll, we must deve, whch consdes + mesuements, fom the mulse esonse of the flte bsed on mesuements. hus, we hve: whee: R R d [4.] R R X X X X [4.3] In ths cse, the essn mt of s theefoe ostve defnte. J coesonds to the utocoelton mt of, nd

170 54 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng nd: R d R d X d X d [4.4] ng equtons [4.3] nd [4.4] nto ccount, equton [4.] becomes: R X X [4.5] R X d d At ths stge, let us ntoduce the nveson lemm of mt. Consde mt A gven b: A B CD [4.6] whose nvese s gven b: A I D B C D B B B C [4.7] If we substtute: B R nd D X C lng the nveson lemm to the mt equton [4.5] gves: R X X of R X X R X R R X d d R X [4.8] A stghtfowd develoment of the clculton gves:

171 Adtve Flteng 55 X R X R d X X [4.9] he bove equton cn lso be wtten n comct fom b ntoducng weghtng fcto to ccount fo the udte bought bout b d. hs weghtng fcto s clled the gn nd denoted K : K d X [4.] Accodng to the bove equton, the estmton of the metes s udted usng onl the cuent mesuement d nd the gn K. hs flte, howeve, s not s effcent n non-stton envonment. o move ts tcng cbltes fo non-stton sgnls, two vnts of ths lgothm hve been oosed: the fogettng fcto RLS method nd the sldng wndow method. We wll now esent the fst of these two methods Fogettng fcto RLS method In equton [4.6] defnng the eo-mnmzton cteon J, we cn see tht ll the bsc eos hve the sme weght. We cn ttbute hghe weght to the lst mesuements b weghtng ech bsc eo e / b fcto such tht : J d / [4.] he mulse esonse vecto of the flte s gven b settng: J [4.]

172 56 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng hus, the flte tself s eessed b the followng elton: X d K [4.3] whee: X R X X R K [4.4] nd: R X K I R [4.5] Equton [4.3] coesonds to equton [.6] fo the lest-sques ecusve estmton of the AR metes. he coeffcents of the dtve flte thus coesond to the AR metes to be estmted he lest men sques lgothm Let us consde cteon J, whch ms t mnmzng the men sque eo between the desed esonse d nd the outut of the dtve flte : X X E d X E d X d E J EQM [4.6] whee. E s the mthemtcl eectton.

173 Adtve Flteng 57 We cn defne the otml flte b oceedng n mnne sml to those ot doted n Chtes nd 3. he otml flte should esect two condtons: tht the gdent of J, J, defned s the column vecto of the tl h j, s zeo,.e. devtves of J wth esect to the flte coeffcents j,..., J h j j,..., o J [4.7] tht the essn mt of J, comosed of the second tl devtves J h h j of J, s ostve defnte. Usng the two evous condtons, we obtn the followng elton fo the otml flte: ot E R X X EX d R d [4.8] whee R stnds fo the utocoelton mt of the nut sgnl nd R d s the coss-coelton vecto between the flte nut nd the desed esonse. Equton [4.8] cn be elted to the Yule-Wle equtons obtned n Chte, s well s to the Wene flte equton n Chte 3. he dtve lest men sques lgothm ws ntoduced b Wdow nd off n 959, nd t llows us to ecusvel obtn subotml flte b usng the gdent-te otmzton method. he steeest descent method mes t ossble to loo fo the mnmum n the men sque eo hesce, b gong n the ooste decton to tht of the gdent fo moe detls, the ede s efeed to Aend G. he flte coeffcent vecto s subsequentl udted s follows: J [4.9]

174 58 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng As we wll see below, the ste sze djusts the convegence of the lgothm. he gdent of the men sque eo s gven b the followng eesson: J E E X d EX X X d X [4.3] B combnng the two bove equtons, nd obsevng tht: e t follows tht: d X [4.3] EX e [4.3] oweve, ths udtng of the flte s coeffcents, whch coesonds to detemnstc gdent lgothm, hs lmted otentl becuse t s dffcult to evlute the quntt E X e. Fo ths eson, ths quntt s elced b ts nstntneous vlue. he dtve flte s coeffcents cn be udted s follows: X e [4.33] At fst sght, ths ocedue s debtble. oweve, snce the lgothm tself s ecusve, t clcultes the temol vegng b successve tetons. he convegence of the LMS lgothm hs been the focus of sevel esech effots, best suveed nd esumed n [6]. Fom equtons [4.3] nd [4.33], we notce tht the LMS lgothm benefts fom low clculton cost n tems of the numbe of ddton nd multlcton oetons. hs comlet s dectl ootonl to the flte ode. hus, fo flte ode of, t needs + multlctons nd ddtons t ech teton. As oosed to the detemnstc gdent, the LMS does not llow us to ttn the mnmum of the EQM, becuse of fluctutons nduced b the estmted gdent. hese fluctutons hve zeo men nd bounded vnce whch s ootonl to the ste sze [3]. In tems of the dnmc nge of the otml flte s dffeence, the LMS lgothm s unble to etn the vlue of the otml flte on emnent bss [7].

175 Adtve Flteng 59 Indeed, tng u equton [4.33] fo the flte nd elcng the eo b ts eesson [4.3], we obtn: X d X [4.34] Beng scl quntt, n n X n X n. heefoe, the bove equton s modfed to: d X X X [4.35] B ntoducng the otml flte ot, whch s the soluton fo the Wene- of equton, nd settng: ot [4.36] equton [4.35] gves se to: ot ot d X X X [4.37] Equvlentl: ot ot ot X d X X d X X X I d X X X I, [4.38] wth: X X I, [4.39] Moeove, ot m X d b coesonds to the model nose. If the lgothm ttns the Wene soluton t teton,.e. ot, the dffeence between the otml flte nd the dtve flte t the net teton s not zeo becuse:

176 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng X b [4.4] m hs dffeence between the dtve flte nd the Wene flte s llustted n Fgue 4.: b m ot + + d + e X - Fgue 4.. Bloc-level descton of the LMS dtve flteng Fom the bove stud of the dffeence between the LMS nd the otml fltes, we cn deduce the elton tht should stsf fo the lgothm to convege. B vegng the tems on both sdes of equton [4.35], we obtn: E ot ot I R E f nd onl f: nd: E [4.4] X X E X X If we defne D nd P s follows: D dg whee,..., E e the egenvlues of R. P e the unt-nom egenvectos, whee,...,

177 Adtve Flteng 6 equton [4.4] s modfed s follows: E ot P I D P E ot [4.4] Multlng both sdes b P P, we obtn: ot E P P I D P E I D P E ot ot [4.43] Substtutng U P E ot leds to: U I D U [4.44] he th comonent of vecto U, denoted, should stsf the elton: u u [, ] [4.45] he bove s scl fst-ode dffeence equton. B successve substtutons, we cn eess u sttng fom u s follows: u u [, ] [4.46] he demonstton of the men convegence of the LMS lgothm towds the ot otml soluton s thus educed to demonstton tht the eo u conveges towds zeo [, ]. he egenvlues,..., u of R, whch e el nd ostve becuse R s el, smmetc ostve defnte mt, should stsf the followng convegence condton: [, ] [4.47] Rengng the tems, we obtn: [, ] [4.48]

178 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng hus, the necess nd suffcent condton fo the convegence of the LMS lgothm mes use of the lgest egenvlue of the nut sgnl utocoelton mt R : [4.49] m We cn defne condton fo the choce of the ste sze whch would be esest to hndle. Let us fst ecll the tce of the nut sgnl utocoelton mt: m tce [4.5] R Gven [4.5], moe estctve defnton of equton [4.49] cn be wtten s follows: tce R m [4.5] Let us ntoduce j s tme-constnt tem defned s the tme needed so tht U j eu. Fo ech u nd ts ssocted egenvlue, we obtn: hus: u j e u j u [, ] [4.5] f log [4.53] j, log nd j. heefoe: j [4.54]

179 Adtve Flteng 63 he tme constnt j thus deends on the sed of the egenvlues, nd wll be dven b the smllest egenvlue of the sgnl utocoelton mt,.e. j. mn If sgnl s whte zeo-men sequence wth vnce condton s modfed to:. Indeed, we hve: tce R R E., the bove Ech egenvlue s ssocted wth mode of convegence clled the egenmode. It should be noted tht the egenmodes ssocted wth the smll egenvlues of R convege moe slowl thn those ssocted wth the lge egenvlues. he convegence seed, ootonl to the ste sze, s nvesel ootonl to the sed of the egenvlues nd s ndeendent of the ntl condtons. Consequentl, the seed of convegence s govened b the smllest egenvlue of R. In ddton, the convegence s not unfom. hs s mjo dwbc of the LMS lgothm. he choce of ste sze s the bss fo defnng two fmles of the LMS lgothms. Indeed, cn hve constnt vlue o cn be vble. Alcton emle: let us estmte the AR metes of second-ode AR ocess defned b nd, ssumng tht, smles of the 4 elzton of the sgnl e vlble.

180 64 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Result veged ove elztons Result fo one elzton umbe of tetons Fgue 4.3. LMS-bsed mete estmton fo second-ode AR model = -

181 Adtve Flteng umbe of tetons Fgue 4.4. LMS-bsed mete estmton of second-ode AR model = umbe of tetons Fgue 4.5. LMS-bsed mete estmton of second-ode AR model = Othe ntl condtons sml to the cse esented bove

182 66 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Fgues 4.3 nd 4.4 ovde n emle of the estmton of the metes of second-ode AR ocess fo two dffeent vlues of the ste sze. We note tht s nceses, so does the convegence seed. oweve, ths comes t the cost of hghe estmton vnce. hus, tde-off hs to be mde between the convegence seed nd the ecson of the estmton. Moeove, s cn be obseved n Fgue 4.5, the ntl condtons do not l sgnfcnt ole n the ecusve estmton of the AR metes. ote: LMS-bsed estmton of nos sgnl s AR metes If the AR ocess s dstubed b n ddtve zeo-men whte nose wth vnce b, the lest-sques estmton of the AR metes usng nos obsevtons z becomes bsed see Chte. o solve ths bs oblem, modfed vesons of the LMS flte hve been develoed ove the st few es. he wll be ntoduced n the followng nd e esectvel nmed: the -LMS lgothm; the -LMS lgothm; the -LMS lgothm. echle et l. hve oosed the -LMS lgothm []. hs s ecusve esoluton of the nose-comensted Yule-Wle equtons usng gdent-te lgothm. he -LMS lgothm s bsed on the udtng of the AR metes wth gdent-te equton [6]. hs equton s tself bsed on elmn estmton of the ocess usng the lst estmted AR metes. B mens of sttstcl stud, the uthos demonstte tht the vnce of the metes obtned usng the -LMS flte s lowe thn fo those obtned usng the -LMS lgothm. evetheless, to mlement the -LMS nd -LMS lgothms, the vnce of the ddtve nose should be nown befoehnd. o llevte ths, Zhng et l. [7] oose the jont estmton of the AR metes nd vnce n, usng the - LMS lgothm. hs leds to unbsed AR metes. evetheless, ll the bove ecusve oches eque lge numbe of smles,.e., n the ode of sevel thousnd, to comenste fo the effects of the nose. he seed of the convegence of the LMS lgothm deceses dl n cse of coelted nuts. One ltentve s to use lest-sques lgothms. Othe solutons e bsed on tnsfomtons n the fequenc domn [5] [7]. hs ltte clss of solutons bngs two movements: n ncese n the convegence seed nd

183 Adtve Flteng 67 educton n comlet. In the sme contet, we cn lso menton the use of othe tnsfoms such s the tle tnsfom, o the wvelet-bsed decomoston [4] [5]. When the nose hens to be ulse nose, ts nfluence cn be educed b usng combnton of dtve fltes nd nonlne fltes such s ode fltes. hs ovdes obust estmton of the nos sgnl metes besdes elmntng the ulse nose [] []. LMS hs vleged sttus n the fml of ecusve lgothms. It s the smlest n tems of mlementton nd does not eque o nfomton. Usull, t s not consdeed s t of the otml lgothm fml, but Klth et l. demonstted n 996 tht the LMS lgothm s otml s egds the nom [] [] Vnts of the LMS lgothm As we mentoned ele, the mn dvntge of the LMS s ts low clculton comlet. oweve, the tdtonl LMS suffes fom the followng oblems: the convegence seed deends on the ste sze. he smlle the, the slowe the convegence wll be. Moeove, the vnce of the esdul eo s lso ootonl to. A tde-off thus hs to be found whle choosng the ste sze: convegence seed vesus ecson. Moeove, ths eques o nfomton bout the nut sgnl utocoelton mt; even though the convegence of the LMS s ndeendent of the ntl condtons, t s not unfom nd deends on the egenvlue sed. hs elns n t the slow seed of the LMS to convege. o educe the effects of these two shotcomngs, sevel moved vesons of the LMS lgothm hve been ntoduced. he nomlzed lest men sques ws esented ndeendentl b guno nd od on the one hnd, nd b Albet nd Gdne n 969 on the othe. It ws onl n 98 tht the nme LMS ws oosed b Btmed nd Andeson []. he modfed lest men sques MLMS lgothm s nothe ecusve method fo the estmton of the nut sgnl owe. Fnll, we wll menton the ffne ojecton lgothm omlzed lest men sques LMS In 984, Goodwn nd Sn [9] vewed the LMS lgothm s the esoluton of constned otmzton oblem. Stted s foml defnton: gven n

184 68 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng obsevton vecto X nd the desed esonse d, the m s to detemne the coeffcent vecto of the dtve flte t teton, whle mnmzng the Euclden nom of the dffeence between nd, nd whle esectng the followng condton: d X. We should thus mnmze: j j j h h [4.55] wth the followng constnt: d j h d X j j [4.56] In ode to solve ths constned otmzton oblem, we use the Lgnge multle method. hs method conssts of ntoducng n unnown scl lned to the constnt. hs scl quntt s clled the Lgnge multle 3. We then consde cteon coesondng to lne combnton of cteon [4.55] nd constnt [4.56]. hus, the quntt tht we see to mnmze s: d j h h h J j j j j j [4.57] he flte h h,, s obtned b solvng the equton: h h h J, [4.58] 3 We hve led used constned otmzton och n Chte 3 bove, whle seeng the eesson of the Wene flte.

185 Adtve Flteng 69 o: h h, [4.59] Multlng both sdes of the bove equton b,, ddng the ntemedte esults, nd usng equton [4.57], we obtn: e X X d X h d [4.6] If we substtute the vlue of n equton [4.59], we obtn the followng ecusve equton: e X X [4.6] o, b ntoducng the scl quntt, we obtn: e X X [4.6] Fnll, b ntoducng nomlzed eo e usng n Euclden nom, we get the followng fom fo the lgothm: X d e [4.3] e X X [4.63] X [4.64]

186 7 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng oweve, when X s smll, stblt ssues hve to be consdeed due to the lmted ecson of the dgtl clcultons. o llevte ths, we ntoduce egulzton scl. s thus udted s follows: X e X [4.65] he bove equton justfes the tem nomlzed LMS. he oduct of the eo e nd the nut sgnl smle vecto X s nomlzed wth esect to the Euclden nom of vecto X. We cn stud the dnmc nge of the dffeence between X nd ot of the otml flte nd detemne the convegence condton of the LMS lgothm. Usng the sme ocedue s followed n equtons [4.36]-[4.5] bove, we obtn: I X X R X X b Moeove, mosng U P E U m ot leds to: X X D U I [4.66] whee D s the dgonl egenvlue mt of R nd P s the ssocted egenvlue mt. We cn show tht the lgothm conveges f nd onl f:, tce R Intoducng the lgest egenvlue estctve condton s gven b: m of the coelton mt R [4.67], moe m tce R [4.68]

187 Adtve Flteng 7 In ctce, nothe condton s used whch, though much moe estctve, eques less o nowledge on the sgnl: [4.69] Gven equtons [4.3], [4.64] nd [4.66], the LMS lgothm s slghtl moe comle thn the smle LMS. Indeed, t eques 3 ddtons, 3+ multlctons nd one dvson. hs clculton cost cn be educed f we vod the clculton of the nut vecto Euclden nom t ech teton. he bove-mentoned smlfcton s the bss of the modfed lest men sques lgothm: whee: X [4.7] X X e X [4.7] he MLMS lgothm s bsed on ecusve method of estmtng the nut sgnl s owe. In ctce, howeve, we vod stong the ddtonl vlue of the nut sgnl equed to goousl clculte, n ecusve mnne, the nut sgnl owe. Insted, we ntoduce fcto, lng between nd. he coeffcents cn be udted s follows: [4.7] [4.73] X e Fo emle, we cn choose. he lgothm eques onl +3 multlctons, + ddtons nd dvson.

188 7 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Affne ojecton lgothm APA Let us consde the LMS equton: d X X X X X X X I [4.74] he column vecto n X gves se to ojecton mt on the vecto sce. hs mt s gven b: X X X X. We cn defne the ojecton of n on ths vecto sce s follows: n n X n X n X n X. In the LMS lgothm, the udte of the flte s coeffcents cn be undestood s one-dmensonl ffne ojecton. he APA s genelzton of the LMS. It conssts of consdeng L obseves [8]. o elbote uon ths nlog, let us suose tht the esonse s no longe scl quntt, but vecto d L stong L consecutve desed esonses: L d d d L, X L s no longe vecto of consecutve smles of the nut sgnl, but mt wth ows nd L columns, defned s follows: 4 3 3, L L L L X L [4.75]

189 Adtve Flteng 73 As wth the LMS, the APA lgothm cn be descbed b the thee followng stes: fst, we clculte the eo vecto; then we deduce the nomlzed eo vecto; fnll, ths nomlzed vecto s used to udte the flte s coeffcents:, X d e L L L [4.76],, e I X X L L L L L [4.77] n X L L, [4.78] ee L s no longe scl, but vecto wth dmensons L. We cn obseve the dnmc nge of the dffeence between nd the otml flte. Le the LMS, the APA lgothm cnnot loc the otml flte s vlues. hus,,,,,, b I X X X m L L L L [4.79] wth:,,,,, X I X X X I L L L L [4.8] nd: ot [4.36] Even though ths lgothm coesonds to the otml Wene flte t teton, the Wene nd the dtve fltes e not the sme t the net teton. he APA does not loc tself but osclltes ound men vlue wth cetn vnce. As we noted ele, the stud of the convegence of dtve lgothms cn be bsed on the stud of the dnmc nge of the dffeence between the otml flte nd the dtve flte.

190 74 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Let L,..., nd,..., L denote, esectvel, the egenvlues of X, L X, L nd the ssocted egenvectos. Let us loo t the sngul vlue decomoston of the dt mt X X, L, L : U V [4.8] Mtces U nd V hve dmensons nd L esectvel. Addtonll, s dgonl mt comsed of -L zeos nd L sngul non-zeo vlues nged n decesng ode. Let Q be the mt eessed s follows:, L, L, L, L Q X [ X X I] X [4.8] Usng the egenvlue decomoston of X, L X, L nd the sngul vlue decomoston of X, we obtn:, L Q U L L U [4.83] mn,..., L If s consdeed to be neglgble comed to the smllest egenvlue, L, L of X X, t follows tht:

191 Adtve Flteng 75 U U Q [4.84] Consequentl, f we te the L= obsevton vectos nto ccount, we obtn: I IU U X I X X X L L L L ] [,,,,. [4.85] Imosng: ot U U, we obtn:,,,, b I X X X U U Q I U m L L L L [4.86] Moeove, when s neglgble comed to the smllest of the egenvlues of,, X X L L, L= nd the model s nose vecto, b m L s zeo, we hve: U U [4.87] Stll tng the model nose to be zeo, we cn lso loo t the convegence condtons tht the dtve ste sze should esect. Usng equtons [4.83] nd [4.86], we obtn:

192 76 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng U U U U [4.88] Usng n och nlogous to tht used fo LMS nd LMS devton, we cn show tht the convegence s ssued f nd onl f:, [4.89], L, L Agn, tng to be neglgble comed to the smllest egenvlue of X X, we e led to the followng condton:. [4.9] A fst veson of ths lgothm hs been develoed b G [8]. It must lso be noted tht unde cetn condtons, the APA flte cn behve le the RLS flte s f s convegence s concened. Fgue 4.6 dects the estmton of the coeffcents of the FIR flte h usng the APA lgothm. he flte nut s seech sgnl whle the desed sgnl d coesonds to the sum of the flte outut nd zeo-men, whte Gussn nose b : d h b. he sgnl-to-nose to s ssumed to be equl to 3dB. Moeove, = 56. Ou m hee s to estmte the coeffcents of the mulse esonse h usng the smles of the sgnls nd d. Moeove, we wll lso evlute the convegence seed s well s the ecson of the dtve flte beng used. o do ths, we wll loo t the dffeence between the coeffcents of the otml flte

193 Adtve Flteng 77 nd the dtve flte, fo gven numbe of tetons. he cteon s defned s follows: ot ˆ ot J log [4.9] ot ot whee s the column vecto whch combnes the coeffcents of the mulse esonse h. he smultons e efomed n the followng condtons: =.5, = 6 nd L =,, o 5 obseves Men Eeu métque metc moenne eo db L = L = L = L = Échntllon Smle uméo de l'téton umbe of tetons mulse esonse of the flte b J ccodng to L Fgue 4.6. Alcton: estmton of FIR flte metes usng the APA lgothm

194 78 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng 4.5. Summ of the oetes of the dffeent dtve fltes LMS MLMS APA RLS obseve: LMS Sevel obsevtons Slow convegence Senstvt to nut sgnl Slowe thn LMS Insenstve to nut sgnl sttstcs Slowe thn LMS Insenstve to nut sgnl Fste wth ncesng numbe of obseves Quc convegence ble 4.. Summ of the convegence of the lgothms: LMS, LMS, MLMS, APA nd RLS lgothms LMS MLMS APA obseve: LMS Sevel obseves tce R ble 4.. Summ of the convegence of the dffeent lgothms, ccodng to ste sze: LMS, LMS, MLMS, APA nd RLS lgothms Fgue 4.7 esents the convegence seed nd the ecson of the dffeent dtve fltes, s functon of the numbe of tetons. We select fo the LMS lgothm,. 995 fo the RLS lgothm nd fo the APA lgothm. We consde L =, 5 o 5 obseves.

195 Adtve Flteng 79 Men metc eo db Eeu métque moenne db APA, L = APA, L = 5 APA, L = 5 LMS RLS uméo de l'téton umbe of tetons Fgue 4.7. Emle: estmton of FIR flte metes usng vous lgothms 4.6. Alcton: nose cncellton Let thee be nos sgnl defned s follows: d s b [4.9] whee s s the seech sgnl nd b s the ddtve nose. d=s+b e h Fgue 4.8. Bloc-level eesentton of nose-cncellton sstem

196 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng he dtve flte n ths cse ovdes n estmton of the nose whch gves n estmton of the ddtve nose b. he flte coeffcents e udted usng the followng eo: e d s b Fstl, we wll suose tht s s not coelted to ethe b o. he smultons e ced out unde the followng condtons: =, 8. he globl sgnl-to-nose to s 7.3 db fo the nos sgnl d nd 7.8 db fo the enhnced sgnl e. Fgue 4.9. he nos seech sgnl nd ts sectogm

197 Adtve Flteng 8 Fgue 4.. he enhnced seech sgnl nd ts sectogm Fgue 4.. he noseless seech sgnl nd ts sectogm In ths chte we hve esented nd llustted the LMS, LMS, APA nd RLS. In the followng chte, we wll ntoduce the Klmn flte, whch wll be subsequentl llustted n Chte 6.

198 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng 4.7. Refeences [] A. Benvenste, M. Métve nd P. Pouet, Adtve Algothms nd Stochstc Aomtons, Snge Velg, 99, nslton of: Algothmes dttfs et omtons stochstque, Msson, Ps 987. [] R. R. Btmed nd B. D. O. Andeson, Pefomnce of Adtve Estmton n Deendnt Rndom Envonment, IEEE ns. on Automtc Contol, vol. AC 5, [3]. J. Beshd nd P. L. Fentuch, Anlss of the Fequenc Domn Adtve Flte, Poc. IEEE, vol. 67, , Decembe 979. [4] P. K. Bondodh, Alcton of Runnng tle nsfom n Adtve Flteng, Poc. IEEE, vol. 76, , Octobe 988. [5] M. Dentno, J. McCool nd B. Wdow, Adtve Flteng n the Fequenc Domn, Poc. IEEE, vol. 66, , Decembe 978. [6] E. R. Fe, Fst Imlementton of LMS Adtve Fltes, IEEE ns. on Acoutcs, Seech nd Sgnl Pocessng, vol. ASSP-8, , Al 98. [7] S. Flon nd. J. Beshd, A Weghted omlzed Fequenc Domn LMS Adtve Algothm, IEEE ns. on Acoustcs, Seech nd Sgnl Pocessng, vol. ASSP-36,. -7, Jul 988. [8] S. L. G nd S. vth, he Fst Affne Pojecton Algothm, IEEE-ICASSP 95, Detot, Mchgn, USA, , 9- M 995. [9] G. C. Goodwn nd K. S. Sn, Adtve Flteng, Pedcton, nd Contol, Pentce ll, 984. [] B. ssb nd. Klth, Med Lest-Men-Sques/ -Otml Adtve Flteng, Aslom Confeence on Sgnls, Sstems nd Comutes, vol., , August 996. [] B. ssb, A. Sed nd. Klth, Otmlt of the LMS Algothm, IEEE ns. on Sgnl Pocessng, vol. 44, no.,. 67-8, Febu 996. []. I. weel nd P. M. Clson, A Clss of Ode Sttstc LMS Algothms, IEEE ns. on Sgnl Pocessng, vol. SP-4, no.,. 44-5, Jnu 99. [3] S. n, Adtve Flte heo, Pentce ll, 996. [4] L. Ljung, On Postve Rel nsfe Functons nd the Convegence of Some Recusons, IEEE ns. on Automtc Contol, vol AC-, no. 4, , August 977. [5] L. Ljung, Anlss of Recusve Stochstc Algothms, IEEE ns. on Automtc Contol, vol. AC-, no. 4, , August 977. [6] O. Mcch, Adtve Pocessng: he Lest Men Sque Aoch wth Alctons n nsmsson, Wle, 995.

199 Adtve Flteng 83 [7] P. Mnsou nd A.. G J, Unconstned Fequenc Domn Adtve Flte, IEEE ns. on Acous. Seech nd Sgnl Pocessng, vol. ASSP 3, , Octobe 98. [8] K. Oze nd. Umed, An Adttve Flteng Algothm Usng n Othogonl Pojecton to n Affne Subsce nd ts Poetes, Electoncs nd Communcton n Jn, vol. 67 A, no.5, 984. [9] R. L. Plcett, Some heoems n Lest Sques, Bomet, 37, , 95. []. Robbns nd S. Mono, A Stochstc Aomton Method, Ann. Mth. Stt., vol.,. 4-47, 95. [] R. Settne, M. jm nd D. Ottvn, Ode Sttstc Fst Klmn Flte, IEEE- ISCAS 996, Chcgo, USA, [] J. R. echle, nsent nd Convegent Behvo of the Adtve Lne Enhnce, IEEE ns. on Acoustcs, Seech nd Sgnl Pocessng, vol. 7, no.,. 53-6, Febu 979. [3] J. R. echle, C. Rchd Johnson J nd M. G. Lmoe, heo nd Desgn of Adtve Fltes, Wle Intescences, 987. [4] Y. Z. sn, Foundtons of the heo of Lenng Sstems, Acdemc Pess, 973. [5].. Wong nd C. P. Kwong, Adtve Flteng Usng tle nsfom nd Ovel Sve Method, IEEE ns. on Sgnl Pocessng, vol. 39, no.7,. 78-7, Jul 99. [6] W-R. Wu nd P-C Chen, Adtve AR Modelng n Whte Gussn ose, IEEE ns. on Sgnl Pocessng, vol. 45, no.5,. 84-9, M 997. [7] Y. Zhng, C. Wen, Y. nd C. Soh, Unbsed LMS Flteng n the Pesence of Whte Mesuement ose wth Unnown Powe, IEEE ns. on Ccuts nd Sstems-II: Anlog nd Dgtl Sgnl Pocessng, vol. 47, no. 9, , Setembe.

200

201 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Mohmed jm Coght 8, ISE Ltd. Chte 5 Klmn Flteng 5.. Intoducton In the evous chtes, we esented dffeent technques fo the estmton of the metes of lne models. We noted the non-ecusve chcte of the Wene flte whch ws led to dentfcton. hs chte wll esent the Klmn flte, ecusve ltentve to the Wene flte. It ws fst ntoduced n the 96s [] [3]. Klmn tnsfomed the ntegl equton of the contnuous-tme Wene flte to dffeentl equtons [7]. Othe oches wee lso used to obtn the Klmn flte: Sge nd Mstes technque bsed on the lest sques method [3]; Athns nd se s technque bsed on the Pontgun mmum ncle [3]. We hoe tht ths chte wll seve s ede-fendl ntoducton to the Klmn flte. We esent ths flte usng n lgebc och whch, though t m not be the most elegnt, hs the dvntge of not equng n o nowledge. hs lgebc descton cn lso be the sttng ont fo moe foml desctons, such s tht esented b. Klth, A. Sed nd B. ssb n []. In ths descton, the Klmn flte s esented wth secl emhss on ts utlzton n the estmton of model metes. We lso esent the so-clled etended Klmn flte fo nonlne estmton.

202 86 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng 5.. Devton of the Klmn flte 5... Sttement of oblem Let thee be sstem whch s eesented n the stte sce domn s follows:, G u [5.] v [5.] he dvng ocess u nd the mesuement nose e both ssumed to be zeo-men, so tht: u E [5.3] v E [5.4] Moeove, u nd v e ndeendent of ech othe, whte nd hve covnce mt Q nd vnce R esectvel: E u v l Eu Ev l,l [5.5] E u u Q [5.6] v v R E [5.7] In ddton, both ocesses stsf the followng condtons: E u [5.8] v E [5.9] Snce u s zeo-men, the covnce mt of u s the sme s ts utocoelton mt.

203 Klmn Flteng 87 Ou ts hee s to estmte the stte vecto tng nto ccount the nfomton vlble t tme n, whch cn be befoe, fte o t the nstnt. In so dong, we wll consde thee dffeent cses: n : we hve to estmte the stte vecto b tng nto ccount ll the obsevtons vlble t tme ; n ths cse, flteng s ced out; n : we onl consde t of the totl vlble mesuements, nd c out n nteolton o smoothng; n : we hve to edct the stte vecto; ths coesonds to edcton o n etolton. Iesectve of whch of the bove cses s beng teted, ˆ / n denotes the estmton of the stte t nstnt, consdeng the nfomton vlble u to nstnt n. Ou m s to obtn ecusve estmton of the stte vecto,.e., gven new mesuement t nstnt, to ovde new estmton of the stte vecto usng ts evous estmton t nstnt. hs s commonl nown s the one-ste edcto. We wll loo closel t the ogton nd udte stes whch coesond, esectvel, to the eltonsh between ˆ / nd ˆ / nd between ˆ / nd ˆ /. If we consde the estmton eos to ogte though the eo covnce mt, the deducton of the Klmn flte equtons s getl smlfed Pogton ste: eltonsh between ˆ / nd ˆ /; ecuence eltonsh between the eo covnce mtces P / nd P / ng nto consdeton the lnet of equton [5.], the stte vecto estmton ˆ / s lso chctezed b the tnston mt,. Moe fomll, the estmton ˆ / cn be defned s follows: ˆ / E,, [5.] Fom equton [5.], t follows tht:

204 88 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng ˆ /, E G E,, u,,. [5.] As: u,, E, [5.] we obtn: ˆ /, ˆ /. [5.3] Let us ntoduce ~ / s the eo n estmtng the stte vecto t nstnt : ~ / ˆ /. [5.4] he osteo -eo coelton mt t s defned b: P / E E ˆ / ˆ / ~ ~ [5.5] Usng stte equtons [5.] nd [5.] nd the lne elton [5.3] between ˆ / nd ˆ /, we cn eess the o-eo coelton mt. Indeed, b subtctng equton [5.3] fom equton [5.], we get: ˆ / G u ˆ /, [5.6] Let P / be the o -eo coelton mt defned s follows: ˆ / ˆ / P / E [5.7] nd usng equton [5.6], t follows tht: he tems osteo nd o wll be justfed lttle futhe.

205 Klmn Flteng 89 P /, G E Equvlentl: E u u G ˆ / ˆ /, [5.8] P /, P /, G Q G [5.9] We hve thus estblshed ecusve elton between the o-eo coelton mt nd the osteo-eo coelton mt Udte ste: eltonsh between ˆ / nd ˆ / eltonsh between P / nd P / ; ecusve Let us consde the ecusve estmton of the stte vecto. eefte, we wll dot the followng lne fom of the stte vecto estmton: ˆ / ˆ / K ˆ / [5.] hs fom s nsed b the lne ecusve lgothm deved nltcll n the secton on the ecusve lest sques estmton n Chte. he osteo estmton ˆ / of the stte vecto coesonds to the edcton ˆ / udted b coectve tem,.e. the weghted dffeence between the ctul mesuement nd ts edcton. hs och s sml to the one we doted when devng the lest sques method n Chte. Fgue 5.. Klmn flte K s nown s the gn of the flte, o the Klmn gn. We would eect K to be ndeendent of the mesuement, to ensue lnet of the lgothm.

206 9 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng ng u the eesson fo the estmton eo / ~ nd elcng the estmted vlue / ˆ b ts eesson [5.], we obtn: / ˆ / ˆ / ˆ / ˆ / ˆ / ˆ / ~ v K K I v K K [5.] We cn then deduce the eesson fo the coelton mt of the osteo eo t tme, nmel / P, s functon of / P : v K K I v K K I E E P / ~ / ~ / ~ / ~ / Equvlentl: / / K R K K I P K I P [5.] We hve thus estblshed second ecusve eltonsh between the osteo eo coelton mt nd the o eo coelton mt Eesson of the Klmn flte gn o ensue tht the lgothm s otml, the gn K should be chosen so s to mnmze the men sque eo on the estmton of the stte vecto. he cteon J whch needs to be mnmzed cn be defned s follows: / tce / ~ / ~ tce / ~ / ~ E P J. [5.3] Consdeng equtons [5.3] nd [5.5], the Klmn flte s mnmumvnce te of flte. he otml gn stsfes the followng condton: K J [5.4]

207 Klmn Flteng 9 In the bove equton, we clculte the devtve of scl quntt J wth esect to vecto K. ng equton [5.] nto ccount, we obtn: / tce K K R K K I P K I [5.5] hs mounts to: / / R K P K P [5.6] hus: / / R P P K [5.7] Gven equton [5.] bove, the eltonsh between / P nd / P cn be ewtten s follows: / / / / / K R K K P K K P P K P P [5.8] Rengng the elements of ths equton gves: / / / / / K R P K K P P K P P [5.9] Usng the eesson fo the Klmn flte gn, we cn smlf the bove equton to: / / / / / K P K P P K P P [5.3]

208 9 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng nd consequentl: / / P K I P [5.3] Fo contnuous-tme stte sce eesentton, equton [5.8] hs the fom of the Rctt dffeentl equton. B nlog, ths equton s clled the dscete Rctt equton. B lng the mt nveson lemm to equton [5.3], we obtn: / / / K K P P P [5.3] Relcng the Klmn gn n the bove equton b ts eesson [5.7] gves: / / / / / / / R P P R P P P P P [5.33] Some of the elements of equton [5.33] cn be ewtten s follows, subject to the condton tht R s non-zeo: / / / / / R P R P R P P R P [5.34] Equton [5.33] s modfed to: / / R P P [5.35] he tems / / P P nd R R e equl to dentt mt I. Insetng these vlues n the gn equton, we obtn:

209 Klmn Flteng 93 K P / P / P / R R P / R P / P / P / R P / R [5.36] P In the bove equton, elcng / b ts eesson [5.35] gves: K P / R P P / R / R [5.37] hus: K P / R. [5.38] Consdeng the equton: ˆ / ˆ / K ˆ / [5.] he sgnfcnce of ech tem n equton [5.] cn be hghlghted b the followng qulttve esonng. Gven eesson [5.7] of the Klmn gn, we see tht: fo constnt R : f P / s smll, the gn wll lso be smll. he confdence ttbuted to the estmton obtned fom the model s ncesed. If, howeve, P / s hgh, sgnfng low degee of confdence n the stte vecto estmton, the gn wll be hgh. he contbuton fom the weghted coecton to the gn wll be hghe; fo constnt P / : f R s smll, the mesuements wll be slghtl nos. hese mesuements wll be weghted hghe becuse of the gn vlue. If, on the othe hnd, R s hgh, the gn wll be smlle. he motnce of the second tem n equton [5.] wll be smlle. J. S. Demet, n [4], hs shown tht the Klmn flte gn mnmzes not onl the tce of the eo covnce mt, but lso n lne combnton of the dgonl elements of ths mt. We cn conclude fom ths esult tht f the stte vecto contns hscl qunttes such s seed, oston, etc., we need not eoccu ouselves wth the bsence of hscl sgnfcnce of the sum of eos ssocted wth the qunttes.

210 94 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Imlementton of the flte o me use of the set of ecusve equtons whch chcteze the Klmn flte, we should choose the ntl condtons of the stte vecto s estmton, nmel ˆ /, s well s the covnce mt of the ssocted eo, denoted P /. If we hve no o nfomton whtsoeve on the stte, we dot the followng ntl stte fo the stte vecto: ˆ / E. [5.39] If E s not nown, we te t to be equl to zeo. In ddton: ˆ / ˆ / P / P E [5.4] Alng equtons [5.3], [5.38] nd [5.7] to P /, P / nd gn K, we obtn: P /, P / K P / G Q G, P / R [5.4] [5.7] I K P / P / [5.38] We cn evlute these thee qunttes ndeendentl of the mesuements, even befoe these mesuements e teted b the flte. o detemne the thee qunttes, we eque the o nowledge of P /. Fo, we clculte P /, K, ˆ / nd ˆ / s follows: P / whee P/, P /, G Q G K whee K P/ P/ R ˆ / whee ˆ /, ˆ / ˆ / whee ˆ / ˆ/ K ˆ/

211 Klmn Flteng 95 Fo, we oceed n the sme mnne doted to udte the stte vecto s o nd osteo estmtons, nd so on. he choce of the ntl vlues s delcte ocess becuse t needs to ensue the d convegence of the lgothm. Fo n vlue of ˆ /, even n bt one n the eteme cse, the lgothm ocessng the obseved mesuements ces out the necess coectons. o me u fo the lc of nfomton on P /, we dot P / I, whee I s the dentt mt nd n bt scl. Fgue 5.. A flowcht of the Klmn flte he flowcht n Fgue 5. dects how the Klmn flte wos. Let us te u the equton eltng to P / to P / : I K P / P / [5.38] We sw ele tht the Klmn flte s bsed on the ntoducton of gn tem whch mnmzes the cteon J defned s follows: J tce P /. [5.3] Combnng equtons [5.38] nd [5.3] gves: / tce P / tce K P / tce P [5.4]

212 96 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng oweve, mt K P / s ethe ostve defnte o ostve semdefnte. Consequentl, t follows tht ll the egenvlues e ethe ostve o zeo. hs leds us to the followng neqult: tce / tce P / P [5.43] hus, the estmton eo deceses s the lgothm oceeds he noton of nnovton Let us gn consde the equton fo the ecusve estmto: ˆ / ˆ / K ˆ / [5.44] If ths edcton s efect one, n the bsence of mesuement nose, the coecton s nduced b the nnovton e,.e.: e ˆ / [5.45] In moe goous och, when the edcton s efect nd the flte s consdeed to be otml, the nnovton s whte ocess. We cn lso ove tht f the flte s otml, the nnovton e s whte sequence wth zeo men. e no longe contns n nfomton tht m be coelted wth the obsevton, whch could move the udte of the stte vecto []. We cn vef the degee of otmlt nd the flte efomnce b testng the whteness of the nnovton. o do ths, Meh hs oosed the followng sttstcl test:.95ˆ ee ˆ ee j fo j [5.46] ee, ˆ ee j s the estmton of the nnovton s utocoelton functon, nd s the numbe of smles vlble. Altentvel, we could lso use the test oosed b Stoc [5]: j ˆ ee j+.65 j ˆ ee / [5.47] hs test s moe elble thn tht n [5.46].

213 Klmn Flteng 97 Model equton, G u Obsevton equton v A o nfomton u E nd E v E u v l, l E u u l Q l Ev v l R l E u E v Flte equtons ˆ /, ˆ / ˆ / ˆ / K ˆ / Gn eessons K P / P / R K P / R A osteo covnce mt P / I K P / A o covnce mt Intl condtons ˆ / E P /, P / G Q G ˆ/ ˆ/ P / P E ble 5.. Klmn flte equtons,

214 98 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng If the dvng ocess s not Gussn, the efomnce of the Klmn flte s degded nd thee m be el s of dvegence. A moe obust veson hs been oosed fo ths cse b s nd Kutz [6] Devton of the Klmn flte fo coelted ocesses So f, we hve consdeed the dvng ocess nd the nose to be uncoelted. In ths secton, we wll see how the sstem eesentton s modfed fo the cse of coelted ocesses. Fst of ll, let us ntoduce the followng nottons:, G u [5.48] v [5.49] We sw ele tht ocesses u nd v stsf: u l Q l E u [5.5] v l R l E v. [5.5] In some cses, u nd v m be coelted. We wll consde these two ocesses to be themselves geneted b whte sequences nd s follows: u, u G [5.5] 3 v, v G [5.53] nd hve zeo-mens nd the followng covnce mtces: 3 l Q l E [5.54]

215 Klmn Flteng 99 l Q33 l E [5.55] 3 3 Moeove, we suose tht these two nose vectos stsf: l Q3 l E [5.56] 3 Sttng fom these ssumtons, let us loo t the etended vecto whch combnes, u nd v s follows : u [5.57] v hs combnton llows us to eesent the dnmc nges of, u nd v. Moeove, we cn lso clel note tht the dvng ocesses of u nd v, whch e nd 3 esectvel, dectl ntevene n the eesson of vecto. ng nto ccount model [5.48] nd eessons [5.5] nd [5.53], we cn wte the model equton s follows:, G, G 3 G3 3, [5.58] o contct ths eesson, we cn ntoduce vecto w: w 3 [5.59]

216 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng whch s nown s the etended nut: l Q Q Q Q l w w E. [5.6] he qunttes,, Q nd G e defned s follows:,,,, G [5.6] Q Q Q Q Q [5.6] nd: 3 G G G [5.63] whee the eesent zeo mtces of ote dmensons. he ended stte-sstem equton thus becomes:, w G [5.64] nd the mesuement equton cn be wtten s: [5.65] We see tht n the eesson of mesuement [5.65], thee s no elct tem fo the ddtve nose. evetheless, we now tht the Klmn gn s gven b: / R P K [5.38] hs gn eesson s no longe vld becuse now R =. We thus hve to stt ove gn wth the clculton [4].

217 Klmn Flteng As we mentoned n Chte, ths eesentton of the sstem n the stte sce s clled the efect mesuement eesentton, o noseless mesuement. hee e sevel shotcomngs of ths eesentton. he mn dsdvntge s tht due to efect mesuement, the equtons fo the Klmn flte e those esented n ble 5. wth R =. he stte estmton, n ths cse, cn gve se to some numecl dffcultes f we cn no longe guntee tht mt P / s nvetble fo ll vlues of. In such stuton, thee e two solutons to ensue tht the Klmn flte functons oel: the fst of these s n d hoc soluton consstng of ddng coecton tem n the P / mt so tht t s stll nvetble; the second soluton ms t educng the ode of the stte model. o do so, soluton bsed on chnge of the bss vectos of the stte sce hs been oosed. It leds to stte vecto wth obsevton s one of ts elements. Fo futhe detls, the ede s efeed to [7] Reltonsh between the Klmn flte nd the lest sques method wth fogettng fcto We sw ele tht the o eo covnce mt hs the followng fom: P /, P / G Q G, [5.4] o enble dect comson between the Klmn flte nd the lest sques method wth fogettng fcto, we hve to use the sme model n the two cses. hs wll be cheved f the mtces, nd G e defned s follows:, I [5.66] G I [5.67] hs chnges equton [5.4] to: P / P / Q [5.68]

218 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng We lso sw tht: I K P / P / [5.3] If we elce the tem P / b ts eesson [5.68], we obtn: I K P / Q P / [5.69] We estblshed ele fo the lest sques method tht: P P I K [5.7] If we mose the followng eqult, the two lgothms e dentcl: P P Q [5.7] Rengng the tems of the bove equton gves: Q P [5.7] Afte the descton of the Klmn flte nd ts elton to the lest sques method, we oceed to use t fo the estmton of metes Alcton of the Klmn flte to mete estmton Estmton of the metes of n AR model Let us loo t the followng AR ocess: u [5.73] ee, s the sgnl t nstnt nd u s zeo-men whte Gussn nose wth unt vnce.

219 Klmn Flteng 3 We dot stte sce eesentton hghlghtng the metes whch need to be dentfed,.e., the edcton coeffcents,...,. We then choose the stte vecto such tht the comonents,..., coesond to the edcton coeffcents: [5.74] Moeove, we note tht: [5.75] he obsevton vecto s then constucted b stong the followng vlues of the obsevton: [5.76] Snce the model s ssumed to be stton, metes,..., cn be consdeed constnt. he stte sce model cn thus be defned b the followng two equtons: [5.77] u [5.78] Wth esect to the stte sce eesentton gven n equtons [5.] nd [5.], the bove equtons led to the followng choce fo the dvng ocess covnce mt nd the tnston mt: Q nd, I. [5.79] Snce the vnce of the ddtve nose s equl to, the ecusve lest sques lgothm nd the Klmn flte e both eessed usng the sme equtons. Let us then estmte the AR metes of second-ode AR ocess wth metes defned s follows:

220 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng nd [5.8] ecusve estmton of the AR metes mltude smles Fgue 5.3. Estmton of AR metes he dvntge of the Klmn flte ove the RLS ses fom ts blt to tc the metes. Fo ths uose, let us genete second-ode AR ocess whch wll subsequentl be subjected to n but chnge, so tht the AR metes ssume the followng vlues: nd fo smles to 7 [5.8] 4.35 nd.565 fo smles 7 to,4 [5.8].99 nd.98 fo smles,4 to, [5.83]

221 Klmn Flteng 5 he esectve oles ssocted wth these metes e the followng: 3 3, e j,, e j nd,.99 e j 3. In ths non-stton cse,. he equton to udte the stte thus tes one of the followng two foms: n the fst fom, the stte vecto s udted s follows: whee w w [5.84] w s zeo-men whte Gussn nose. Its vnce s chosen b the use. If t s chosen too smll, the tcng of the vton of the metes s not ssued; f the chosen vlue s too lge, the estmtons wll hve lge vnce; n the second fom, the nge of the vton s nown nd vefes the followng:, w [5.85] Unfotuntel, mt, s el nown. he detemnton of both the mt nd of stte vecto s now nonlne estmton ssue.

222 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng smles tcng the AR metes, Q= tcng the AR metes, Q= tcng the AR metes, Q=. Fgue 5.4. cng the AR metes

223 Klmn Flteng 7 If we subject the flte to stong vton of the metes, we see tht the estmton s not followed u []. On the othe hnd, f we nject the nose w, the tnston s closel tced even fo low vlues of vnce. hs movement cn be elned b the fct tht n the fst cse Q, the gn tends towds zeo fte sevel tetons. he flte consequentl loses ts dtblt Alcton to seech nlss he Klmn flte ws fst led to seech sgnls b Mtsu et l. [6]. At bout the sme tme, Gueguen nd Cnns lso too u the modelng of seech sgnls usng the Klmn flte [9]. Gbson nd Mels esented comtve stud of vous ecusve lgothms used fo estmton [5] [6]. Mc nd Jn ntoduced modfed veson of the Klmn flte fo bette tcng of metes [5]. Fnll, mn esech effots hve been ced out n the feld of communctons, nd esecll n equlzes fo dgtl tnsmssons. In ths contet, we cn cte two gound-beng effots, those of Godd [7] nd Mow [9]. We wll esent some esults fo the lcton of the Klmn flte to the nlss of seech sgnls []. Let ths seech sgnl be modeled usng n AR model: u [5.73] whee denotes the th smle of the seech. o mlement the flte, we must set some metes. We dot the followng nlss condtons nd ntl condtons: ode of the model: ; R fo voced sounds nd R fo unvoced sounds [5] [6]; P / I. he nlss s ced out ove tme equl to the eod of the tch. hs tch cn be estmted usng the method esented n [6]. We cn lso use the oches ntoduced b Gffn nd used n the IMBE code [8]. Fnll, we consde the followng vtons of the eo cte, fst used b Gueguen nd Cnns [9]. o do ths, we defne the edcton eo s follows: e ˆ [5.86]

224 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng nd the vton nom of the mete vecto s follows: ˆ ˆ ˆ ˆ [5.87] he metes estmted b the Klmn flte e sml to those obtned b the covnce nd utocoelton methods esented n Chte. Fgue 5.5. Seech nlss wthout tch detecton; the sgnl beng nlzed, the vton n the nom of the mete vecto, nd the ssocted edcton eo 5.4. onlne estmton In ctce, we often encounte sgnls modeled usng nonlne mthemtcl equtons. Fo emle, ths holds tue fo hse- o fequenc-modulton. In ll the bove sectons, we consdeed the models to be lne, both n the model eesentton nd n the stte sce eesentton. In wht follows, we wll consde nonlne ocesses n the stte sce, nd we wll develo nonlne flte clled the etended Klmn flte EKF. Fo ths, we dot the followng eesentton fo contnuous-tme sstems:

225 Klmn Flteng 9 t, t G t u t f t [5.88] whee f t, t denotes nonlne functon of the stte nd the model metes. he obsevton vecto m lso be nonlne: t, t v t h t [5.89] We ssume ut nd vt to be ndom whte ocesses wth the followng coeltons: E u t u t Q t, [5.9] E v t v t R t, [5.9] he new soluton conssts of the lnezton of equtons [5.88] nd [5.89] ound efeence ont o set of efeence onts, clled the efeence tjecto. hs nme comes fom the fst lctons of ths soluton, whch wee n the e of eosce Model lnezton: lnezed Klmn flte We wll lneze the model ound efeence tjecto ssumed to be nown. hs tjecto, denoted t, s the soluton to the followng homogenous equton [8]: t f t, t [5.9] We ssocte ths tjecto wth nomnl mesuement: t h t, t [5.93] We wll estblsh dstubed equton whch defnes the dffeence between the effectve stte nd the stte gven b the efeence tjecto. o stt, f we subtct [5.9] fom [5.88], we obtn: t, t f t, t G t u t t f t [5.94]

226 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng It would be beond the scoe of ths boo to descbe the ntegton of ths nonlne stochstc dffeentl equton. oweve, moe elstc ltentve, moe consstent wth ou m, would be to c out lo-sees develoment ound the efeence tjecto t :,,, t t t t f t t t t f t t f t t [5.95] hs develoment s ced out suosng tht the dffeence t t emns smll. In ddton, t contns ll the hghe-ode tems of the lo enson. Lmtng ouselves to the fst-ode omton, equton [5.94] cn be ewtten s:, t u t G t t t t t f t t t t [5.96] Intoducng the fcto defned s follows: t t t, [5.97] equton [5.96], whch estmtes ths fcto, s modfed to:, t u t G t t t t f t t t [5.98] o smlf the nlss, we wll denote:,, t t t t t f t t F. [5.99] Combnng the two equtons bove, we obtn:, t u t G t t t F t. [5.] If we develo the tem t t h, of the obsevton equton usng the lmted lo sees enson, ound the efeence tjecto nd u to the fst ode, we obtn:

227 Klmn Flteng h t, t h t, t Denotng: h t, t t t t t t t t. [5.] h t, t t, t, [5.] t the obsevton equton [5.89] s chnged to: t, t t v t t [5.3] he lnezton of model equtons ound the efeence tjecto leds to the followng: t, t t G t u t F t [5.4] t, t t v t t [5.5] he dscete-tme model ssocted wth the contnuous-tme model s fomll defned s follows: ;, G u [5.6] ; v [5.7] We wll te u the clculton of the tnston mt lte. Snce ths model s lne, we cn l the flte eessons we hve evousl deved fo the stndd Klmn flte nd thus estmte the new stte vecto. In ths cse, the bove equtons hve the followng fom: ;, ˆ / ˆ / [5.8] ; ; ˆ / ˆ / K ˆ / [5.9]

228 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng / ˆ / ˆ [5.] he gn of the flte s eessed s follows: ; / ; ; / ; R P P K [5.] he covnce mtces of the o nd the osteo eos e defned esectvel s:, ; /, ; / G Q G P P [5.] ; ; ; ; / ; ; / K R K K I P K I P [5.3] he etended Klmn flte EKF We cn now eect bette estmton b tng the lst estmton s the efeence tjecto of the stte, nd lnezng the model ound ths estmton. hs gves se to the etended Klmn flte EKF. Let: / ˆ [5.4] Equton [5.] cn be ewtten s: / ˆ / ˆ / ˆ [5.5] Gven the followng: / ˆ, [5.6] the flte equtons become:

229 Klmn Flteng 3 ˆ / ˆ / K ˆ / ; ˆ / [5.7] P / ˆ / ;, P / G Q G ˆ / ;, [5.8] K P / ˆ / ; ˆ / ; P / ˆ / ; R [5.9] P / I K ˆ / ; P / I K ˆ / ; K R K [5.] We cn dw the followng summ obsevtons: the etended Klmn flte s nonlne; the Klmn gn cnnot be detemned o n the cse of nonlne models; the metes of the flte e chnged to ndom functons whch deend on the estmton. he EKF tends to dvege f the tnston fom the old to the new estmton s outsde the lmts of the lne zone. In ths contet, we cn cte the wo ced out b Ljung on the convegence of the EKF [4]. he estmtons obtned usng ths flte e often bsed, nd new vnt of the EKF whch vods ths bs hs been ntoduced n [8]. A subotml technque whch foegoes the use of the nonlne flte hs been descbed b elson nd Ste []: the uthos decomose the estmton usng two sete nd dstnct estmtos Alctons of the EKF he ctul, nd otentl, lctons of the EKF cove the dstnct domns of estmton: the jont estmton of the stte nd the metes; the estmton of nos sgnls;

230 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng the estmton of the tnston mtces n the cse of non stton sgnls Pmete estmton of nos seech sgnl We mentoned bove tht the mete estmton of nos sgnls s bsed. Fo seech sgnls, ths s cse whee we hve sgnl s bued wthn suoundng nose. We thus hve jont estmton ts: on the one hnd, the estmton of the seech sgnl; on the othe hnd, the estmton of the metes []. Let us consde the modelng of seech sgnl usng n AR ocess: u s s s [5.] Let thee be vecto whch combnes the lst vlues of sgnl s: s s s [5.] he dnmc of the stte s then dven b the followng elton: u s s s s [5.3] nd the mesued sgnl whch mes u the obsevton s gven b: v s [5.4] whee v s the mesuement nose. In mt fom, the model cn be wtten s follows:, Gu s F s [5.5] v s G [5.6]

231 Klmn Flteng 5 whee tnston mt F +, s s follows: F, [5.7] nd: whee: G [5.8] he udtes n the edcton coeffcents e modeled b: [5.9] [5.3] nd s zeo-men whte nose. he ovell model whch descbes ou ts s thus: s F, s Gu [5.3] G s v he twn m s thus to estmte the AR metes on the one hnd nd the stte s on the othe. Fo such uose, we cn constuct the etended stte vecto s: s [5.3]

232 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng hs gves se to the followng model:, ; [5.33] whee: u [5.34] ng nto ccount the tcul fom of the model equtons, we cn wte:,, ; s F I, ; s I I s I whee I s the unt mt of ode., ; s thus eessed s functon of the stte vecto:, ; B A [5.35] wth: I I A [5.36]

233 Klmn Flteng 7 [5.37] B I. [5.38] he obsevton equton, whch lns the mesuement to the stte, s: v [5.39] he element B n the tnston mt, whch s tself qudtc n, elctl shows the nonlnet of ou ts. hus:, ˆ / ;, Equvlentl: ˆ / A B A B B ˆ / ˆ / / ;, A ˆ / B B ˆ. [5.4] / ; ˆ. [5.4] hus, b lng the esults obtned n the evous secton, the stte vecto cn be estmted usng the followng equtons: ˆ / Aˆ / ˆ / Bˆ / [5.4]

234 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng P / ˆ / ;, P / Q ˆ / ;, [5.43] K P / P / R ˆ / ˆ / K ˆ / [5.44] [5.45] I K P / I K K R K P / [5.46] whee: l Q l E [5.47] nd E v v l R l [5.48] We cn estmte the metes of the seech sgnl model usng the etended Klmn flte. he metes thus detemned fo the nos sgnl e ve close to those obtned b the tdtonl method fo noseless sgnl. hs hghlghts two sects: flteng of nos sgnl, nd modelng usng the etended Klmn flte Alcton to tcng fomnt tjectoes of seech sgnls Once we now the fundmentl fequenc nd the metes of the LPC model fo the seech sgnl, we cn snthesze the seech fo both lmted nd unlmted vocbules. oweve, n ths snthess, we use metes whch do not hve n ovet hscl nteetton. he onl qunttes whch hve hscl sgnfcnce e the fomnts, whch e the esonnce fequences of the vocl tct, nd the bndwdths whch e ssocted wth these fomnts. oweve, the detemnton of the fomnts s tself bsed on the LPC modelng. Seech sntheszes bsed on fomnt snthess gve hgh lstenng qult, quntfed b the ntellgblt tes.

235 Klmn Flteng 9 It s useful to menton nothe lcton of the EKF s concens fomnt tjectoes: the tcng of the nonlne metes of the model. hs lcton s ttbutble to Rgoll []. Let us te u the model of the seech sgnl usng n AR ocess: u [5.73] he tnsfe functon ssocted wth ths model s: z, z z [5.49] he sech fo fomnts f nd bndwdths b s ced out b loong fo the oles of z, usng second-ode esontos: whee: z, [5.5] m c z d z b cos f c e [5.5] s s nd: s d e b [5.5] Unfotuntel, the elton between the LPC model s metes nd the esontos metes f, b cn be eessed s follows: f,, f, b,, b u g m m [5.53] whee g s nonlne functon of f, b, whch e consdeed s the metes of the followng stte vecto:, b [5.54] f,, f m, b, m

236 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng At fom showng the lcton of the EKF to the chctezton of seech sgnls, ths emle hghlghts the sech fo the oles of olnoml. whee: Moeove, usng equton [5.73], we cn detl functon g: v [5.55] [5.56] [5.57] v u. [5.58] he sstem model thus becomes: f w [5.59] g v [5.6] Fo moe n-deth elnton, the ede s efeed to [] Concluson he stted m of ths chte ws to esent the smle nd etended vesons of the Klmn flte on the one hnd, nd to llustte the mlementton usng clssc lctons such s mete estmton. In the followng chte, we wll te u the use of Klmn flteng n sgnl enhncement.

237 Klmn Flteng 5.6. Refeences [] D. Aboutjdne,. Lhd Ghzl nd M. jm, Seech Anlss Usng Klmn Flteng, Poto Wosho on Sgnl Pocessng nd ts Alctons, Poto, Jul 98. [] D. Aboutjdne nd M. jm, Adtve Flte Stuctues fo Deconvoluton of Sesmc Sgnls, IEEE ns. on Geosc. nd Remote Sensng, vol.-ge-3, no.,. 7-73, 985. [3] M. Athns nd E. se, A Dect Devton of the Otml Lne Flte Usng the Mmum Pncle, IEEE ns. on Automtc Contol, vol.-ac-, , 967. [4] S. Demet, A ote on the tue of Otmlt n the Dscete Klmn Flte, IEEE ns. on Automtc Contol, vol.-ac-5, , 97. [5] J. D. Gbson, J. L. Mels nd S.K. Jones, Dgtl Seech Anlss Usng Sequentl Estmton, IEEE ns. on Acoustcs Seech nd Sgnl Pocessng, vol.-assp-3, no. 4,. 36, August 975. [6] J. D. Gbson nd J. L. Mels, Unfed Develoment of Algothms Used fo Lne Pedctve Codng of Seech Sgnls, Com. Elec. Eng., vol. 3,. 75-9, 976. [7] D. Godd, Chnnel Equlzton Usng Klmn Flte fo Fst nsmsson, IBM J. Res. Dev., , 974. [8] D. W. Gffn nd J. S. Lm, Multbnd Ectton Vocode, IEEE ns. on Acoustcs, Seech nd Sgnl Pocessng, vol. no. 36, no. 8,. 3-35, August 988. [9] C. Gueguen nd G. Cnns, Anlse de l ole fltge otml de Klmn, Automtsmes, vol. 8, no. 3,. 99-5, Mch 973. []. Klth, An Innovton Aoch to Lest-Sques Estmton. Pt I: Lne Flteng n Addtve Whte ose, IEEE ns. on Automtc Contol, vol.-ac-3, , 968. []. Klth, A. Sed nd B. ssb, Lne Estmton, Pentce ll,. [] R. E. Klmn nd R. S. Buc, ew Results n Lne Flteng nd Pedcton heo, ns. ASME, Sees D Jounl of Bss Eng., vol. 38,. 95-, 96. [3] R. E. Klmn, A ew Aoch to Lne Flteng nd Pedcton Poblems, ASME, Sees D, Jounl of Bss Eng., vol. 8, , 96. [4] L. Ljung, Asmtotc Behvou of the Etended Klmn Flte s Pmete Estmto fo Lne Sstems, IEEE ns. on Automtc Contol, vol-ac-4, Jnu 979. [5] G. A. Mc nd V. K. Jn, Seech Pmete Estmton b me-weghted-eo Klmn Flteng, IEEE ns. on Acoustcs Seech nd Sgnl Pocessng, vol.-assp- 3, no. 5,. 3-33, 983. [6] E. Mtsu,. jm,. Suzu nd M. Omu, An Adtve Method fo Seech Anlss Bsed on the Klmn Flteng heo, Electon, Jn UDC534,. -9, 97.

238 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng [7] P. S. Mbec, Stochstc Models, Estmton nd Contol, vol. I, Acdemc Pess, ew Yo, 979. [8] P. S. Mbec, Stochstc Models, Estmton nd Contol, vol. II, Acdemc Pess, ew Yo, 98. [9]. Mow, Quntzton nd ts Evluton of nsmsson Chctestcs of Fdng Chnnel n the Adtve Recevng Sstem bsed Klmn Flte, Electoncs nd Com. n Jn, vol-67b, no. 3,. 8-36, 984. [] L. Mels nd J.D. omc, Lne Pedctve Codng wth Addtve ose fo Alcton to Seech Dgtlston, 4 th Alleton Confeence on Ccut nd Sstems, Setembe 976, USA. [] L. W. elson nd. Ste, he Smultneous On-Lne Estmton of Pmetes nd Stte n Lne Sstems, IEEE ns. on Automtc Contol, , 976. [] G. Rgoll, A ew Algothm fo Estmton of Fomnt jectoes Dectl fom the Seech Sgnl Bsed on n Etended Klmn Flte, IEEE-ICASSP 86, oo, Jn, 7- Al 986. [3] A. P. Sge nd G. W. Mstes, Lest Sques Cuve Fttng nd Dscete Otmum Flteng, IEEE ns. on Educton, vol-e-, no.,. 9-36, 967. [4] A. P. Sge nd J. L. Mels, Estmton heo wth Alctons to Communctons nd Contol, McGw-ll, 97. [5] P. Stoc, A est fo Whteness, IEEE ns. on Automtc Contol. vol. AC-, , Decembe 977. [6] C. s nd L. Kutz, An Adtve Robustzng Aoch to Klmn Flteng, Automtc, vol. 9, no. 3, , 983. [7]. Wene, he Etolton, Inteolton nd Smoothng of Stton me Sees wth Engneeng Alctons, John Wle, ew Yo, 949. [8]. Yoshmu, K. Konsh nd. Soed, A Modfed Etended Klmn Flte fo Lne Dscete me Sstems wth Unnown Pmetes, Automtc, vol. 7, no. 4, , 98.

239 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Mohmed jm Coght 8, ISE Ltd. Chte 6 Alcton of the Klmn Flte to Sgnl Enhncement 6.. Intoducton he Klmn flte, ntoduced n the evous chte, s etensvel used n sgnl nlss, fo vet of lctons such s bomedcl, nvgton, gudnce, econometcs, etc. It hs been the toc of lge mount of esech. he ede s efeed to [] [5] [] [5] nd [3]. hs lst s b no mens n ehustve one. hs chte wll mnl be concened wth the use of Klmn flteng n the followng cse: gven sgnl dstubed b n ddtve nose, how cn we enhnce the sgnl when onl sngle sequence of the nos sgnl s vlble, nd when thee s no o nfomton ethe on the sgnl o on the nose? hs fomulton les, fo emle, to the cse of the sngle-chnnel enhncement of seech sgnl dstubed b n ddtve nose. We wll stt out wth the cse of sgnl dstubed b whte nose. Assumng tht the seech sgnl cn be modeled b th -ode utoegessve ocess, we wll loo t the stte sce eesentton of the sstem. heefte, we wll esent the Klmn flte whose conventonl fom eques o nowledge of the stte sce s dnmc metes nd of the vnces of both the dvng ocess nd the ddtve nose. We wll then evew the sngle-chnnel enhncement methods bsed on the Klmn flte. hen, we wll oose sevel ltentve oches whch foego the vnces of the dvng ocess nd the mesuement nose, tdtonll denoted Q nd R esectvel. Fo emle, we wll esent method whch s bsed

240 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng on Meh s wo ntll develoed n the feld of dentfcton [6] [7]. We wll then te dvntge of the och used b Cew nd Belnge [4] whch ws ele esented s n ltentve to Meh s och n the e of contol. et, we wll consde the enhncement s stochstc elzton ssue n the domn of dentfcton. Fo ths ltte cse, these new oches e bsed on the subsce methods of dentfcton, lso ognll oosed n contol [35] [36]. o conclude the chte, we wll tet the cse of coloed nose nd summl menton the wo ced out fo sgnls dstubed b mulse nose, such s [33]. 6.. Enhncement of seech sgnl dstubed b whte nose 6... Stte sce eesentton of the nos seech sgnl Let us ssume tht the seech sgnl s cn be modeled b th -ode AR ocess: s s u [6.] whee u s the dvng ocess. Moeove, the obsevton s combnton of seech comonent nd n ddtve zeo-men whte nose comonent wth vnce R such tht: s b [6.] As we sw n Chte 5, the Klmn flte llows ecusve ocedue fo the stte estmton. As ou objectve hee s to estmte the seech sgnl, we cn constuct the stte vecto b combnng the q lst vlues of the seech sgnl s. As stted n secton.6.9, these vlues e nown s the stte vbles. = s - q + s [6.3] heefoe, the stte sce eesentton of the sstem n equtons [6.] nd [6.] s defned s [3] [3]: Gu [6.4] b [6.5]

241 Alcton of the Klmn Flte to Sgnl Enhncement 5 ee, s the qq tnston mt hvng the followng fom: q [6.6] G nd e the nut nd obsevton vectos esectvel, nd e defned s follows: q G [6.7] As elned n Chte, ths s the cnoncl, contollble stte sce eesentton Seech enhncement ocedue he dffeent oches to sgnl enhncement usng the Klmn flte genell oete n the followng stes. Fst, the seech sgnl s wndowed.

242 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Fgue 6.. Fme-b-fme ocessng of seech sgnl Fo the se of smlct, ll the fmes hve the sme length, nomnll 3 ms to fulfll the hothess of qus-sttont of the sgnl. In ctce, we cn te 5% ovel nd nnng o mmng wndow defned b: w cos wth =,, -. [6.8] whee =.5 fo nnng wndow nd =.54 fo mmng wndow. Moeove, denotes the numbe of smles e fme. o estmte the sgnl usng the Klmn flte fo ech fme, we fst need to now the coesondng vnces Q of the dvng ocess nd R of the mesuement nose, s well s the dnmc stte sce mt tlet, G,. hus, the enhnced sgnl cn be defned s follows: sˆ ˆ / [6.9] whee ˆ / s the osteo edcton of the stte vecto bsed on the obsevtons {,,..., }. Snce we hve doted the contollble stte sce eesentton, onl the edcton coeffcents,..., need to be estmted. As the ocess s ssumed to be qusstton, these coeffcents e constnt ove the nlss fme.

243 Alcton of the Klmn Flte to Sgnl Enhncement 7 We cn now eteve the seech sgnl b lng Klmn lgothm flteng o smoothng on ech fme. Fnll, we econstuct the comlete enhnced sgnl usng n ddton-ovel method [3]. hs s llustted n Fgue 6.. he use of ths method s justfed b the followng esons. If we chose to foego the ovel, the fnl econsttuton of the sgnl would be bsed on the dvson of ech enhnced fme b the wndow. oweve, ths would ncese the estmton uncetnt t the begnnng nd the end of ech fme, whee the wndow vlue s close to zeo. A 5% ovel s dvntgeous becuse t gves two estmtons fo ech seech smle, whch cn subsequentl be combned. Let s ˆ be the th smle of the th lot, nd s ˆ the th smle of the + th lot. Both smles coesond to n s weghted b the wndow w s follows: estmton of the th smle w sˆ s [6.] w s s ˆ. [6.] Consdeng equtons [6.] nd [6.], we cn esl deve s s follows: sˆ sˆ s [6.] w w If d s the numbe of ecove smles between two djonng fmes, we cn esl eess s functon of : d [6.3] Moeove, f we te nto ccount the 5% ovel, gvng d=/ f s even, nd the wndow eesson ntoduced n [6.8], we cn lso smlf equton [6.]. hus, we hve: w w cos cos cos cos [6.4]

244 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Equton [6.] s modfed to the followng: s sˆ sˆ [6.5] he mjo dffeence between the vous estmton oches les n the estmton of the nose vnce nd the edcton coeffcents,..., of the seech sgnl Stte of the t dedcted to the sngle-chnnel enhncement methods usng Klmn flteng In ths secton, we wll evew estng seech enhncement methods tht we hve come coss so f, notbl n [3] [9] [] [] [6] [7] [] [3] [4] nd [4]. In the contet of sgnl enhncement, one of the oneeng oches oosed ws ntoduced b Plwl nd Bsu [3]. In t, the sgnl nd nose sequences e both ssumed to be nown, ndeendentl of one nothe. he model s stte sce metes e then estmted usng the seech sgnl, befoe the sgnl s contmnted wth the nose. As fo the vnce of the ddtve nose, t s estmted dectl fom the nose sequence. he uthos then use deled veson of the Klmn flte to estmte the seech sgnl. hs ocedue s dected n Fgue 6.. It should be noted tht ths och s lmted to theoetcl stud nd cnnot be mlemented fo el cses. Lte, the method oosed b Gbson et l. ovded sub-otml soluton whch s smlfed veson of the eectton-mmzton EM lgothm [3]. Moe secfcll, the edcton coeffcents,..., of the seech sgnl nd the vnce Q of the dvng ocess e estmted fom the nos seech sgnl. he vnce of the ddtve nose s estmted, o udted, dung the eods of slence. hs eques the use of vocl ctvt detecton VAD. o move the enhncement, the ognl nos seech sgnl s flteed two o thee tmes. At ech teton, the metes of the seech e estmted usng the enhnced sgnl. hs s ghcll shown n Fgue 6.3. oweve, n the esectve wos, Plwl [3] nd Gbson et l. [3] do not dentf the method used fo estmtng the sgnl metes. he ltte mlctl quote the ublcton of Fedlndle [7], nd stte tht the esoluton of modfed Yule-Wle equtons could be osectve soluton, even though the esults

245 Alcton of the Klmn Flte to Sgnl Enhncement 9 obtned could be unstsfcto, esecll fo wdebnd nos sgnls, when onl lmted numbe of smles e vlble. Fgue 6.. he och oosed b Plwl nd Bsu Fgue 6.3. he och oosed b Gbson et l.

246 3 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng In the och used b Oenhem et l., the stte vecto hs sze q=+ nd contns the + lst smles of the sgnl [9]. s s s - = [6.6] he AR metes e estmted fom the esoluton of Yule-Wle equtons usng the estmted vlues of the sgnl metes: ˆ Q R ss [6.7] whee ˆ R ss s n estmton of the sgnl ++ utocoelton mt. Indeed, ssumng tht the sgnl s egodc, we cn omte the sgnl utocoelton mt usng the followng mt estmton: s s ss R ˆ [6.8] ee, s the fogettng fcto whch mes t ossble to tc the nonstton fetue of the sgnl. / + / ˆ / ˆ = P s s s s [6.9] Moeove, the estmton of the vnce of the ddtve nose s eessed s: ˆ R [6.] whee s fogettng fcto. he element s ecusvel clculted s follows: Fo futhe detls, the ede s efeed to Chte, secton..6.

247 Alcton of the Klmn Flte to Sgnl Enhncement 3 ˆ s / + s [6.] hs method cn be used to enhnce seech ecoded n nos envonment s well s fo the ctve cncellton of nose [4]. he estmton of the sgnl AR metes fom nos obsevtons s one of the mn dffcultes n the mlementton of Klmn-flte bsed enhncement methods. Gbe ooses usng n EM-te lgothm such s the one oosed b Deche n [6] to estmte the edcton coeffcents,...,. hs lgothm tetvel mmzes the log-smlt lelhood of the metes to be estmted, ssumng tht the sgnl nd the dvng ocess e both Gussn. Recentl, Gnnot et l. hve ten u ths mmum lelhood mete estmton oblem []. he oose the use of Klmn smoothng n the EM lgothm to obtn the estmtons of the stte ˆ / nd the covnce mt P /. o do ths, the fst l Klmn flteng, nd then use the followng bcwd-ecusve lgothm [8]: ˆ / ˆ / A ˆ / -ˆ / [6.] whee: P / - P / A P / P / A [6.3] - A P / P / [6.4] ˆ / nd P / e nvolved n the mmzton ste whose uose s to estmte the edcton coeffcents,..., nd the vnces of the dvng ocess nd the mesuement nose. he equtons e sml to the Yule-Wle equtons. Goh et l. hve oosed Klmn flteng method n whch the ectton ves ccodng to the ntue of the segment beng nlzed [4] [5]. he ocedue n ths soluton s the followng. If the fme s unvoced, the stte sce eesentton of the sstem s sml to tht descbed so f. If, on the othe hnd, he ede s efeed to Chte, secton..6.7, whch coves the nfluence of n ddtve whte nose on the estmton of AR metes.

248 3 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng the fme s voced, ectton u s ssumed to be eodc, wth eod omtel equl to the eod of the tch. he dvng ocess cn be modeled s follows: u, u d [6.5] whee d s stton zeo-men whte nose wth covnce Q. In ddton, s the nomlzed nstntneous eod of the tch whch needs to be estmted, nd, s n ndcto of the nstntneous eodct. he bove equton cn ltentvel be wtten n the followng fom: m u, lu l d [6.6] l ee, m s constnt whose vlue s equl to the mmum of the nomlzed nstntneous eod of the tch. Fo emle, fo sgnl smled t 8 z, m cn be equl to 6 nd:, l, l. he stte sce dnmc of the sstem leds to n etended stte vecto defned s follows: wth: nd: u = s [6.7] u = m u - + u [6.8] s = s - + s [6.9] hs och s dted to the te of fme beng nlzed. oweve, t suffes fom two dsdvntges. Fstl, n mbgut n tems of the te of modelng s ntoduced when the lots e med. Secondl, n es soluton does not lws est fo the tch etcton nd the voced/unvoced decson n nos envonment. hese two concens e not ddessed n the wo esented n Goh et l. [4].

249 Alcton of the Klmn Flte to Sgnl Enhncement Altentve methods bsed on ojecton between subsces Intoducton he Klmn flteng-bsed oches tht we emned n the bove secton e centeed ound cnoncl stte sce eesentton. he mjo dffcultes n the ctcl mlementton le n the estmton of the sgnl s AR metes. Usng the subsce-bsed technques fo the stte sce dentfcton of the sstem, we hve oosed new enhncement och whch foegoes the elct modelng of the sgnl nd the esultng estmton of the model metes 3. Subsce methods fo dentfcton wee fst ntoduced b Vn Oveschee et l. nd ognll used n the feld of contol [35] [36]. he lso beneft fom close eltonsh wth the Klmn flte. he mjo dvntge s tht the stte sce mtces needed to l Klmn fltes e dectl obtned fom the nos obsevtons. Moeove, s oosed to the EM-te oches, no teton of the ocess s equed to move the estmton of the seech sgnl. Unle the oches esented n the ecedng secton, ll the methods tht we wll esent hee e bsed on n estmton of the set,, Q, R, n stte sce bse see Chte, secton ssocted wth the followng one: G u [6.3] b [6.3] whee s the tnston mt, s the obsevton vecto, Q s the coelton mt of G u nd R s the vnce of b Pelmn obsevtons Sttng wth mete, whch hs to be gete thn the sze of the stte vecto, we fst defne the sstem s etended obsevton mt, :, [6.3] 3 hs de s the esult of dscussons wth M. M. Vehegen dung the Mthemtcs fo Sstems Wosho, 5-3 Jul 997, Snt Emlon, Fnce. We dul cnowledge hm fo ths.

250 34 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng We lso conctente the nos obsevtons n j nel mt s follows: Y/ j 3 j j [6.33] Fnll, we ntoduce the noton of othogonl ojecton of one subsce on nothe: gven the two mtces L nd M, whch hve dmensons L j nd M j esectvel, we cn defne the ow subsce L / M s the ojecton of the ow subsce of L on the ow subsce of M. LM E MM M L / M E [6.34] j j he mthemtcl eectton wll hencefoth be omted b tme vege: E j. lm.. j j In ctce, snce onl fnte numbe of obsevtons e vlble, we wll elce E j. b the oeto. nd te j, n equtons [6.33] nd j [6.39]. s the numbe of smles of the nlzed fme Relton between subsce-bsed dentfcton methods nd the Klmn lgothm he vous vesons of subsce dentfcton methods [35] [36] [38] [39] hve the oots n the oneeng wos of Ae [] nd Kung [], whch del wth the elzton ssue. Gven the obsevtons, the stochstc elzton oblem conssts of detemnng the dnmc metes nd the nose vnces, n ths cse the set,, Q, R, fom the coelton functon of the obsevtons. Kung [] develos n lgothm tht ovdes the stte sce model fom nel mt contnng Mov metes, whch howeve e the dffcult to clculte n ctce. hs nel mt s then eessed s the oduct of the contollblt nd obsevblt mtces of the sstem.

251 Alcton of the Klmn Flte to Sgnl Enhncement 35 o vod constuctng the obsevton covnce mt, the subsce methods use othogonl ojectons between some ow subsces of the nel dt mt. hese ojectons, Y / / Y / nd Y / / Y/, cn be eessed ccodng to the etended obsevton mt nd the two sequences X, nd X :, Y / / Y/, X, [6.35] Y Y X [6.36] / / /,, Vn Oveschee et l. hve ut fowd n nteetton of these two sequences b estblshng ln wth the Klmn lgothms [35] [36]. he uthos hve shown tht X, nd X, cn be consdeed the oututs of bn of Klmn fltes t the th nd + th tetons. o do ths, the uthos ntoduce the followng qunttes: nd:, X, X, E [6.37], X, X, E [6.38] whee X, cn be eessed n the followng fom: X, ˆ ˆ /,... j / j... j [6.39] whee ˆ /,..., s the estmton of the stte vecto. It tes nto ccount the evous obsevtons,..., n the stte sce bse ssocted wth the set,, Q, R. hus, the subsce-ojecton dentfcton methods e bsed on edcton of the stte b the otml Klmn flte, gven the lst obsevtons [35] [36]. he uthos of [8] nd [9] lso oose tht these oches be efomulted to sut seech enhncement.

252 36 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Sgnl edcton usng the otml Klmn flte he th smle of the fme cn be estmted thns to the estmton ˆ of the stte vecto n the stte sce bse ssocted to,, Q, R. In fct, we hve: ˆ s [6.4] Equton [6.35] cn lso be wtten n the followng fom: X,, Y/ / Y / [6.4] ee,, denotes the seudo-nvese of mt,. We cn thus obtn the edcton of the stte usng the otml Klmn flte bsed on the lst obsevtons,.... oweve,, nd obsevton vecto e not nown nd thus hve to be estmted. Accodng to [6.35], we cn estmte, b usng the sngul vlue decomoston Y / / Y /, snce Y / / Y / s the oduct of the etended obsevblt mt nd the stte sequence X,, ccodng to equton [6.35]: nd: Y / / Y / UV [6.4], U. [6.43] Sttng fom equton [6.3], we cn thus etct, whch s the fst ow of the, mt. hs ocedue vods the dect estmton of the vnce of the dvng ocess, the nose vnce nd the tnston mt. Deste ths dvntge, ths dect efomulton s dffcult n tems of mlementton becuse the choce of mete s delcte one. o move the stte estmton nd consequentl the efomnce of the enhncement och, we cn foesee edcton of the th smle bsed on the fst obsevtons o smles of the fme.

253 Alcton of the Klmn Flte to Sgnl Enhncement Klmn flteng nd/o smoothng combned wth subsce dentfcton methods In ths secton, we oose the use of the subsce methods to obtn the set,, Q, R. hen, we wll use Klmn flte to estmte the seech sgnl. As we sw ele, we cn etct fom the etended obsevblt mt,. he stuctue of ths mt lso llows us to obtn the tnston mt s follows:,,,, [6.44] hus:,, [6.45] whee, s the seudo-nvese of the mt,. Moeove, equton [6.36] s equvlent to:,, Y / / Y/ X [6.46] We thus obtn the Q R, fom the sequences X, nd X, s follows [35]: X, X, Y / [6.47]

254 38 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng he covnce Q R, s deduced fom the esdue [6.48]. Q [6.48] j R he dffeent lgothms mlemented n [8] nd [9] eque n RQ fctozton nd sngul vlue decomoston. he comuttonl comlet of these lgothms s eltvel hgh. Eessons of the stte sequences Y / / Y/, X, Aomton of, Y / / Y/, X, X,, Y / / Y/ X,, Y/ / Y/, seudo-nvese of,, L X, X, Y / Etcton of dnmc metes Q nd R Etcton of nose vnces Fgue 6.4. Subssce methods fo dentfcton Smulton esults In ths secton, we wll c out the enhncement of the seech sgnl Le tbunl v bentôt ende son jugement, smled t 8 z. hs sgnl s dstubed b whte Gussn nose wth sgnl-to-nose to of 5 db.

255 Alcton of the Klmn Flte to Sgnl Enhncement 39 Fgue 6.5. me-domn eesentton of the nos sgnl nd the enhnced sgnl Fequenc z Fequenc z Ognl sgnl Fequenc z me s os sgnl Fequenc z me s Flteed sgnl me s Smoothed sgnl me s Fgue 6.6. Emle of sgnl enhncement; nut SR: 5 db

256 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng We see n the bove fgues tht esdul nose emns n the enhnced sgnl. Klmn smoothng cn ecbl educe ths nose [8] Innovton-bsed oches Intoducton In ths secton, we wll esent oches whch eque o nowledge of nethe the covnces of mesuement nose nd dvng ocess no of the tnston mt. We wll concentte on mete estmton n sstem dentfcton nd beneft fom the dvntges of esults obtned b Meh [6] [7], nd Cew nd Belnge [4]. he fst of the new oches s nsed b the wo tht Meh hs efomed n the feld of dentfcton [6] [7]. Meh ntll develoed method tht ms t obtnng unbsed nd consstent estmtes of the covnces of the dvng ocess nd the ddtve nose. In ddton, he showed tht the otml sted-stte Klmn gn could lws be estmted. Subsequentl, Gbe et l. [8] hve oosed efomulton of hs och n the feld of seech sgnls: the Klmn gn s clculted n n tetve w, s long s the nnovton sequence s not whte. hs ocedue eques nethe the covnce of the dvng ocess no the vnce of the ddtve nose nd uses n estmton of the utocoelton functon of the nnovton. oweve, the tnston mt nd the obsevton vecto e stll unnown nd thus must be estmted. When choosng the contollble cnoncl stte sce eesentton of the sstem, the Modfed Yule Wle equtons cn be solved to estmte the edcton coffcents,.... oweve, ths technque m led to unstsfcto esults fo wdebnd nos fmes. Fo ths eson subsce dentfcton methods cn be used to estmte the dnmc metes. A second och s bsed on the wo ced out b Cew nd Belnge [4]. In ths wo, the m s to c out n tetve ocedue fo the estmton of the otml Klmn gn fom subotml gn. Lstl, ths method ooses the estmton of the vnces of the ectton nd mesuement noses fte flteng the sgnl wth Klmn flte. Once the two vnces hve been estmted, the seech sgnl s tself estmted usng stndd Klmn flte.

257 Alcton of the Klmn Flte to Sgnl Enhncement Klmn-flte bsed enhncement wthout dect estmton of vnces Q nd R Let us te u the stte sce eesentton ssocted wth the AR model of equtons [6.4] nd [6.5]. he stte vecto hs sze, whch s equl to the AR ocess ode. he otml Klmn gn cn be clculted tetvel, wthout dectl usng the vnces of the dvng ocess nd the mesuement nose. In fct, the soluton conssts of clcultng the Klmn gn fom the nnovton s utocoelton functon ee j. he flteng stge s globl n the sense tht the gn sts constnt n n nlss fme. Fnll, b vefng tht the nnovton sequence s whte, we cn chec whethe the otml soluton hs been ttned [6] [7]. he mn stes n ths och consst of: wndowng the sgnl nd estmtng the coesondng tnston mt nd obsevton vecto; clcultng the utocoelton functon of the nnovton sequence; develong n tetve ocedue to obtn the otml Klmn gn. o do so, we fst flte the sgnl wth n ntl gn K nd then clculte the nnovton s follows: e ˆ / [6.49] he stte vecto s udted s follows: ˆ / ˆ / K e [6.5] nd the enhnced sgnl s deduced s gven: sˆ ˆ / [6.5] he nnovton sequence thus obtned s used to move the estmton of the Klmn gn. hs gn s eessed, t the j+ th teton, s follows: I - K j K j K j - I - K j j ˆ ee j ˆ ee / ˆ j ˆ ee j ee [6.5]

258 4 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng whee ˆee denotes the utocoelton functon of the nnovton. It s clculted wth the Klmn gn K j. he w the Klmn gn s udted s elned n Aend I o detled n [6] [7]. j It should be noted tht the bove clculton s eteted s long s the nnovton s not whte sequence. o vef whethe the sequence s ndeed whte, the followng sttstcl test oosed b Stoc 4 s used: j ee j ee,95 fo [6.53] he enhnced sgnl s then econstucted usng ddton-ovel. os seech sgnl nd enhnced seech sgnl Fgue 6.7. Emle of sgnl enhncement Klmn-flte bsed enhncement usng subotml gn Bsed on edcton of the stte vecto usng subotml gn, we oose n ltentve to the method oosed b Meh to clculte the otml Klmn gn [4]. As n the subsecton bove, the flteng s globl,.e., the gn s constnt ove the ente nlss wndow. hs new och s dvded nto fou successve stes: wndowng the sgnl nd estmtng the, ; 4 P. Stoc, A est fo Whteness, IEEE ns. on Automtc Contol, vol. AC-, , 977.

259 Alcton of the Klmn Flte to Sgnl Enhncement 43 clcultng the Klmn gn usng the Cew-Belnge och; flteng the obsevtons wth gn Kˆ j to egn the sgnl; econsttutng the ovell sgnl usng ddton-ovel methods. he Cew-Belnge och conssts of the tetve esoluton of set of thee equtons lnng the otml cse to the subotml cse [4]. Moe secfcll, these equtons detemne the ect eltonsh between the utocoelton functon of the nnovton sequence n the otml cse to the sme functon n the subotml cse. * Let / be the o estmton of when the gn K s subotml, nd ˆ / the o estmton of when gn Kˆ s otml. In the followng nlses, mt P * / s defned s follows: ˆ / / ˆ / / P / E [6.54] Moeove, f we ssume the flte conveges,.e., f: lm P * / P * [6.55] then the otml gn cn be obtned tetvel. At the j th teton, desgnted b the subsct j heefte, the clculton of the otml gn s descbed s follows: j ee * ee ˆ = - P * j [6.56] Kˆ j K * - I - K * * j P * I - K * - I - K / ˆ j ee * ee * ee / ˆ * ee j ee [6.57] P ˆ ˆ [6.58] * * * * j * * j+ I K Pj I K ˆ ee K j K K j K

260 44 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng whee the utocoelton functon of the nnovton, denoted ee, s estmted fom the smles e * wth, s follows: - * * * ee l e e l [6.59] l he ede s efeed to Aend J fo moe detls on the devton of equtons [6.56], [6.57] nd [6.58] Altentve och to Klmn-flte bsed enhncement, usng the estmton of vnces Q nd R hs och llows the nose vnces to be estmted. It s bsed on the oetes of the nnovton sequence. We wll fst te u the estmton of the nose vnce, followed b the vnce of the dvng ocess. Assumng the nos sgnl to hve been flteed wth subotml ntl gn K, we hve: P K ee ee I - K ee [6.6] - I - K ee * l oweve, ccodng to secton 5..6: ˆ / ~ / b e. he vnce R of the ddtve nose b s thus elted to the utocoelton functon of the estmton eo b the followng equton: R= ee -P [6.6] Usng the estmted vlue of tnston mt, equtons [6.6] nd [6.6] e esectvel modfed to:

261 Alcton of the Klmn Flte to Sgnl Enhncement 45 P K ˆ ee ˆ ˆ ee ˆ I - K ˆ ˆ ee - ˆ I - K ˆ ˆ ee [6.6] nd: ˆ ˆ P [6.63] R - ee he estmton of the vnce of the dvng ocess s no longe necess. We cn tetvel clculte the utocoelton mt of the edcton eo s follows: P / -= I-K - P / - I-K - + K - K - R GG Q [6.64] Whle lm P / - P nd lm K - K e stsfed, equton [6.64] s chnged to [6]: - K PI - K + KK R GG Q P = I [6.65] Endng ths fom of equton [6.6], we obtn: P = P KP P K P GG Q KP P K KP K +KRK RK K P Usng equton [6.6] of the nnovton s utocoelton functon: GG Q [6.66] P = P P If we note: GG Q KP GG Q K ee P K K KP P K K ee K [6.67] ee [ K K KP P K ] [6.68]

262 46 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng equton [6.65] s chnged to: P= P + GG Q [6.69] Relcng P of the bove equton b ts eesson nd tetng the ocedue, we obtn [6]: j j j- j- GG Q P = P + j [6.7] = = Multlng both sdes b nd b obtn: - j, nd gven tht Q s scl, we P - j = P + j- j- - j GG Q = j = - j [6.7] hus, we cn obtn the vnce Q of the dvng nose u b tng nto ccount the smmet of mt P nd usng eltonsh [6.7] between G nd : - j P P = Q [6.7] j- = j j- - j - j We wll now show tht the denomnto of the bove equton s zeo fo ll - j j, nd tht s zeo fo j : fst, gven the defnton of the obsevton vecto n the contet of seech -j enhncement, the oduct coesonds to the coeffcents, of j the nvese of mt ; moeove, gven the ve ecul stuctue of the tnston mt :

263 Alcton of the Klmn Flte to Sgnl Enhncement 47 [6.6] If we defne mt B s the oduct A, we see tht ts - fst ows e equl to the lst - ows of mt A. In the secl cse whee A =, t cn be esl shown tht - th ow of mt coesonds to the - th ow of, the -3 th ow 3 of nd the -j th j ow of wth j. Consequentl, the mno, of j j - mt wth j, element of. s necessl zeo, s s the Equton [6.7] cn thus be used when j s lowe thn. If j s chosen to be gete thn, the clculton cost goes u. Usng the estmted vlues of nd P, equton [6.7] s chnged to: j- ˆ - j - ˆ j ˆ ˆ ˆ j P P ˆ = Q [6.73] j- ˆ ˆ - j = In ths secton, we hve esented vous seech enhncement methods to be used when the ddtve nose s whte. In the net secton, we te u the cse of coloed nose Klmn flte-bsed enhncement of sgnl dstubed b coloed nose In ths secton, we wll te u elstc cse,.e. Klmn-bsed enhncement when the seech sgnl s dstubed b coloed nose. o ccomlsh ths, we stt b modelng both the seech sgnl nd the nose b b utoegessve ocesses of odes nd q esectvel. We wll esent some Klmn flte-bsed methods tht del wth ths ssue. he stte sce eesentton s tht descbed n Chte, secton.6.9. hs eesentton, howeve, leds to efect-mesuement stte sce eesentton, s mentoned n Chte 5, secton 5..7.

264 48 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng We oceed n the followng thee stes to enhnce seech sgnl dstubed b coloed nose: the nos seech sgnl s fst wndowed n fmes lstng omtel 3 ms ech. A vocl ctvt detecto s mlemented to detect whethe the wndowed sgnl coesonds to slent fme o not. hese eods of slence llow the dvng ocess vnce s well s the nose s dnmc metes to be estmted; the tnston mt s s then estmted fom the nos sgnl fmes; the seech sgnl s ecoveed usng Klmn flteng. he enhnced sgnl s econstucted usng the ddton-ovel method. he method esented n the cse of whte nose n Fgue 6.3 ws etended b Gbson fo coloed nose [3]. he nose metes e estmted dung the eods of slence. he sgnl metes e obtned tetvel. he method ognll develoed b Oenhem et l. [9] hs been genelzed b Vebout [37]. In tht cse, the vecto s comosed of the + lst smles of the sgnl nd the q lst smles of the ddtve nose. Lstl, Gnnot et l. hve lso oosed n EM-te soluton fo coloed nose []. he estmton of the tnston mt b nd s cn be done b usng the Yule-Wle equtons. Indeed, f we suose tht set of smles of the obseved sgnl s vlble nd tht ths set s ssocted wth zone of slence, we cn wte: q b c b j w fo m m [6.74] j whee m s the nde of the fst smle of the bloc. j ng the bove equton nto ccount, we cn wte the utocoelton of obsevton s: q j c j W j [6.75] As ths utocoelton s n even functon, the Yule-Wle equtons cn be eessed s follows:

265 Alcton of the Klmn Flte to Sgnl Enhncement 49 q q q W q c. [6.76] cq hus, ssumng tht the sgnl obseved n ths fme s egodc, we cn estmte j fo j q s follows: M + m- ˆ j l l j. [6.77] M lm j Equton [6.76] cn be ewtten s follows: ˆ ˆ ˆ q ˆ ˆ ˆ q ˆ q Wˆ ˆ q cˆ ˆ cˆ q [6.78] B solvng ths sstem of equtons usng the Levnson lgothm, we cn estmte the tnston mt ˆ v : ˆ v cˆ q cˆ c q- ˆ [6.79] he estmton of mt s s slghtl moe comlcted thn ˆ v becuse ths mt hs to be estmted fom nos seech sgnl smles. Snce sgnl s nd nose b e uncoelted, we eess the utocoelton of the obsevtons fom the utocoelton functons of the seech sgnl nd the nose; thus, the Yule- Wle equtons fo sgnl s me t ossble to estmte s. Let thee be bloc of smles of : s b wth m M M m [6.8]

266 5 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng he utocoelton of s defned b: j = E{ - j} = E{ s s - j}+ E{ b b - j} = ss j bb j [6.8] Consequentl, the utocoelton functons of the sgnl, the nose nd the nos obsevtons stsf the followng condton: ss j j j. [6.8] bb he tem bb j cn be estmted usng the vlues of ove the eod of slence m M m nd b ssumng the sgnl to be egodc: M + m- ˆbb j l l j [6.83] M lm j ng equtons [6.8] nd [6.83] nto ccount, we cn wte tht: ˆ ss m+ M + - j l l j - ˆ bb j [6.84] lmm j whee the fcto s chosen such tht the sgnl s utocoelton mt sts ostve defnte. At tht stge, the sgnl s AR metes e estmted b solvng the Yule- Wle equtons, usng the bove equton fo ˆ j. hus, we hve: ss ˆ ss ˆ ss ˆ ss ˆ ss ˆ ˆ ss ss ˆ ˆ ss Q ˆ ˆ ss ˆ ˆ ss [6.85] nd:

267 Alcton of the Klmn Flte to Sgnl Enhncement 5 ˆ s ˆ ˆ - ˆ [6.86] In the chtes tht follow, we wll te u othe cses of enhncement of sgnls dstubed b coloed nose, notbl modeled usng MA ocesses Concluson In ths chte, we hve descbed sevel methods whose m s to enhnce sgnls dstubed b n ddtve nose usng Klmn fltes. We hve hghlghted the ctcl dffcultes n the mlementton of these oches. hese dffcultes e lned to the estmton of the model metes,.e. the AR metes, the nose vnce, the dvng ocess vnce, etc. In the chtes tht follow, we wll esent ltentve oches whch m t llevtng the oblems mentoned bove. Moe secfcll, we wll cove two technques, nmel the nstumentl vbles nd flteng Refeences []. Ae, Movn Reesentton of Stochstc Pocesses b Cnoncl Vbles, SIAM Jounl of Contol, vol. 3,. 6-73, 975. [] J. Benest, S. Mno nd J. Chen, Seech Enhncement, Snge, 5. [3]. C, E. Gvel nd M. jm, A Dul Klmn Flte-Bsed Smoothe fo Seech Enhncement, IEEE-ICASSP 3, ong Kong, 6- Al 3. [4] B. Cew, nd P. R. Belnge, Identfcton of Otmum Flte Sted-Stte Gn fo Sstems wth Unnown ose Covnces, IEEE ns. on Automtc Contol. vol. AC-8, , Decembe 973. [5] C. K. Chu nd G. Chen, Klmn Flteng, wth Rel me Alctons, Snge Sees n Infomton Scences, Snge Velg, 99. [6] M. Deche, AR Pmete Estmton fom os Dt Usng the EM Algothm, IEEE-ICASSP 94, Adelde, Austl, vol. IV, , 9- Al 994. [7] B. Fedlndle nd B. Pot, he Modfed Yule Wle Method of ARMA Sectl Estmton, IEEE ns. on Aeosce nd Electonc Sstems, vol. AES-, no.,. 58-7, Mch 984. [8] M. Gbe, E. Gvel nd M. jm, A Sngle Mcohone Klmn Flte-Bsed ose Cncele, IEEE Sgnl Pocessng Lettes, , Mch 999.

268 5 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng [9] M. Gbe, Adtve Klmn Flteng-Bsed Seech Enhncement Algothm, Cndn Confeence on Electcl nd Comute Engneeng,3-6 M, vol.,. 5-56,. [] M. Gbe, Seech Sgnl Recove n Coloed ose Usng n Adtve Klmn Flteng, IEEE-CCECE. Cndn Confeence on Electcl nd Comute Engneeng, vol., , -5 M. [] M. Gbe, Robust Adtve Klmn Flteng-Bsed Seech Enhncement Algothm, IEEE-ICASSP 4, Montel, Cnd, vol I,. 3-34, 7- M 4. [] S. Gnnot, D. Buchten nd E. Wensten, Itetve nd Sequentl Klmn Flte-Bsed Seech Enhncement Algothms, IEEE ns. on Seech nd Audo Pocessng, , Jul 998. [3] J. D. Gbson, B. Koo nd S. D. G, Flteng of Coloed ose fo Seech Enhncement nd Codng, IEEE ns. on Sgnl Pocessng, vol. 39, no. 8, , August 99. [4] Z. Goh, K.-C. n nd B.. G. n, Seech Enhncement Bsed on Voced- Unvoced Seech Model, IEEE-ICASSP 98, Settle, Wshngton, USA, vol. no.,. 4-44, -5 M 998. [5] Z. Goh, K.-C. n nd B.. G. n, Klmn-Flteng Seech Enhncement Method Bsed on Voced-Unvoced Seech Model, IEEE ns. on Seech nd Audo Pocessng, Volume 7, no. 5,. 5-54, Setembe 999. [6] V. Gnchov, J. Smuelsson nd B. Klejn, Imoved Klmn flteng fo Seech Enhncement, IEEE-ICASSP 5, Phldelh, PA, USA, 8-3 Mch 5. [7] V. Gnchov, J. Smuelsson nd B. Klejn, On Cusl Algothms fo Seech Enhncement, IEEE nsctons on Seech nd Audo Pocessng, vol. 4, no. 3, , M 6. [8] E. Gvel, M. Gbe nd M. jm, Subsce Stte Sce Model Identfcton fo Seech Enhncement, IEEE-ICASSP 99, Phoen, Azon, USA, vol. no., , Mch 999. [9] E. Gvel, M. Gbe nd M. jm, Seech Enhncement s Relzton Issue, Sgnl Pocessng, vol. 8, no.,, [] S. n, Adtve Flte heo, Pentce ll nfomton nd sstem scences sees, homs Klth, sees edto. 996 [] S. K. Kung, A ew Low-Ode Aomton Algothm V Sngul Vlue Decomoston, th Aslom Confeence on Ccuts, Sstems nd Comutes, , 978. []. M, M. Bouchd nd R. A. Goubn, Pecetul Klmn Flteng fo Seech Enhncement n Coloed ose, IEEE-ICASSP 4, Montel, Cnd, vol. I, vol., M 4.

269 Alcton of the Klmn Flte to Sgnl Enhncement 53 [3]. M, M. Bouchd nd R. A. Goubn, A Pecetul Klmn Flteng-Bsed Aoch fo Seech Enhncement, 7th Intentonl Smosum on Sgnl Pocessng nd ts lctons, vol., , -4 Jul 3. [4]. M, M. Bouchd nd R. A. Goubn, Fequenc nd me Domn Audto Msng heshold Constned Klmn Flte fo Seech Enhncement, 7th Intentonl Confeence on Sgnl Pocessng ICSP 4. 4, vol.3, , 3 August-4 Setembe 4. [5] P. S. Mbec, Stochstc Models, Estmton, nd Contol, Volume, Acdemc Pess, Olndo 979. [6] R. K. Meh, On the Identfcton of Vnces nd Adtve Klmn Flteng, IEEE ns. on Automtc Contol. vol. AC-5, o., , Al 97. [7] R. K. Meh, On-Lne Identfcton of Lne Dnmc Sstems wth Alctons to Klmn Flteng, IEEE ns. on Automtc Contol. vol. AC-6, no.,. -, Febu 97. [8] J. M. Mendel, Lessons n Estmton heo fo Sgnl Pocessng, Communctons nd Contol, Pentce ll, 995. [9] A. V. Oenhem, E. Wensten, K.C. Zng, M. Fede, nd D. Guge, Sngle-Senso Actve ose Cncellton, IEEE ns. on Seech nd Audo Pocessng, vol., no., Al 994. [3] K. K. Plwl, nd A. Bsu, A Seech Enhncement Method Bsed on Klmn Flteng, IEEE-ICASSP 87, Dlls, USA,. 77-8, 987, Al 987. [3] L. R. Rbne nd R. W. Schfe, Dgtl Pocessng of Seech Sgnl, Englewood Clffs, Pentce ll, 978. [3] Secl Issue on Alctons of Klmn Flteng, IEEE ns. on Automtc Contol, vol. AC-8, no. 3, 983. [33] R. Settne, M. jm nd D. Ottvn, Ode Sttstc Fst Klmn Flte, IEEE-ISCAS 996, Chcgo, USA, [34] P. Stoc, A est fo Whteness, IEEE ns. on Automtc Contol, vol. AC-, , Decembe 977. [35] P. Vn Oveschee nd B. de Moo, Subsce Algothms fo the Stochstc Identfcton Poblem, Automtc, vol. 9, no. 3, , 993. [36] P. Vn Oveschee nd B. de Moo, 4SID: Susbsce Algothm fo the Identfcton of Combned Detemnstc nd Stochstc Sstems, Automtc, vol. 3, no., , 994. [37] S. M. Vebout, Sgnl Enhncement fo Automtc Recognton of os Seech, RLE echncl Reot, no. 584, MI, M 994. [38] M. Vehegen nd P. Dewlde, Subsce Model Identfcton, Pt I: he Outut-Eo Stte Sce Model Clss of Algothms, Int. Jounl. Contol, vol. no. 56, no. 5,. 87-, 99.

270 54 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng [39] M. Vehegen, Identfcton of the Detemnstc Pt of MIMO Stte Sce Models gven n Innovtons Fom fom Inut-Outut Dt, Automtc, vol. 3, no.,. 6-74, 994. [4] C.. You, S.. Koh nd S. Rhdj, Klmn Flteng Seech Enhncement Incootng Msng Poetes fo Moble Communcton n C Envonment, IEEE ICME 4. 4, 7-3 June 4, vol., , 4. [4] K. C. Zng, Otml Feedbc Contol Fomulton of the Actve ose Cncellton Poblem: Pontwse nd Dstbuted, RLE echncl Reot no. 583, MI, M 994.

271 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Mohmed jm Coght 8, ISE Ltd. Chte 7 Estmton usng the Instumentl Vble echnque 7.. Intoducton In ths chte, we esent the nstumentl vble IV technque, n ltentve to the genelzed lest sques estmton methods. he IV technque ovdes consstent estmtons fom nos obsevtons. It s bsed on the use of the sstem s eogenous vbles,.e. the nstumentl vbles. Instumentl vble technques hve been used mnl n contol engneeng, but hstocll the wee deved el n the 94s n econometcs [9]. he nstumentl vbles e obtned b ocessng the nut sequence, whch s ssumed to be nown. Fo ths uose, we cn consde fnte mulse esonse FIR flte whose coeffcents cn themselves be udted usng Klmn flte. Such n och, descbed usng bloc dgm n Fgue 7., ws oosed b Young [9] fo dentfcton n contol engneeng. It should be noted tht the efomnce of the nstumentl estmto deends on the choce of these nstumentl vbles.

272 56 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng ose b Known nut u Sstem to be dentfed Obsevton FIR flte FIR flte metes Instumentl Vbles Klmn flte IV Estmto Sstem metes Fgue 7.. Bloc-level descton of the dentfcton method oosed b Young [9] n the feld of contol engneeng oweve, n the feld of sgnl ocessng, we usull do not hve o nowledge of the nut sequence nd the onl dt vlble e the nos obsevtons. heefoe, the bove och hs to be modfed. hus, the method oosed b Fedlnde, Stoc nd Sodestöm [] cn be consdeed. he suggest obtnng the nstumentl vbles b e-flteng the nos obsevtons, fo nstnce b cng out Klmn flteng see Fgue 7.. In ths chte, we wll fst esent evew of the IV technques cuentl used fo AR mete estmton. heefte, we wll oose new och combnng the IV technques nd the Klmn flteng. Moe secfcll, the Klmn flte ovdes the nstumentl vbles,.e. flteed veson of the nos obsevtons, whch e then used fo the estmton of AR metes. oweve, n ode to use Klmn flte, the AR metes must be nown befoehnd. hs thus leds to nonlne estmton ssue,.e. the estmton of both the sgnl nd ts AR metes fom nos obsevtons. o vod usng the etended Klmn flte, two ntectve Klmn fltes cn be consdeed [5]. he fst of these fltes uses the lst vlble estmton of the AR metes nd gves the flteed veson of the sgnl, whees the second flte mes t ossble to udte the AR metes b usng the lst vlble veson of the flteed sgnl.

273 Estmton usng the Instumentl Vble echnque 57 Fgue 7.. Bloc-level descton of the oosed estmton method, n the feld of sgnl ocessng In the lst t of ths chte, we stud the elevnce of ths och when the ddtve mesuement nose s whte, b cng out comtve stud between the method develoed n [5] nd the estng methods [5] [7] [3] [5] [7] [3]. 7.. Intoducton to the nstumentl vble technque 7... Pncle Let us consde th -ode AR ocess defned n Chtes nd s follows: u [7.] whee u s zeo-men whte Gussn ocess. In the followng, the AR mete vecto wll be denoted. Let us ssume tht onl smles of the followng nos obsevton e vlble: z b [7.] whee b s zeo-men whte ddtve Gussn nose.

274 58 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng he nos obsevton z esects the followng egesson elton, s seen n secton..6.7 of Chte : z z [7.3] whee u b b. he bove equton cn be wtten n mt fom s follows: Z Z Z. [7.4] whee Z z. When the obsevtons e dstubed b n ddtve nose, the stndd lest sques oches led to bsed estmtons of the model metes. Mn studes hve been ced out to countect the effects of the bs. Insted of usng bs comenston estmton technques 3, nstumentl vble methods me t ossble to ween ths bs. hs te of method hs been hstocll desgned s n ltentve to the tdtonl LS technque to obtn consstent estmtes of the metes fom nos obsevtons [6]. he consst of usng the nstumentl vbles whch e eogenous to the sstem nd e smtotcll uncoelted to the mesuement nose. hese vbles e usull stoed n mt, clled the nstumentl mt nd denoted M. A sees of estmtos ostve vlues of, we hve: ˆ hs lmt n obblt, ˆ P lm, f fo ll el lm Pob m ˆ [7.5] Fo moe detls, the ede s efeed to secton..6.7 nd Aend E. hese sectons descbe the bs ntoduced b n ddtve nose to the mete estmton. 3 Fo moe detls, the ede s efeed to sectons..6.8 nd 4.3 whee vous bs comenston technques e esented.

275 Estmton usng the Instumentl Vble echnque 59 When lm P ˆ, the estmto s sd to be consstent. Fo the estmton oblem defned n equtons [7.]-[7.], the q mt M s sd to be nstumentl f t stsfes the followng condtons: Condton : M s smtotcll uncoelted to,.e., P lm M, fo ll vlues of ; Condton : M s nvetble nd ts nvese hs lmt n obblt. Pe-multlng both sdes of equton [7.4] b M, we obtn the followng eesson: M M Z M M he IV estmto [7.6] ˆ IV of metes s thus defned s follows: M M Z ˆ IV [7.7] he eo ssocted wth ths estmton s: M M ˆ IV [7.8] We show tht ˆ conveges n obblt towds. he estmto s IV theefoe consstent. It should be noted tht f we te M, estmto [7.7] coesonds to the lest sques estmto. he IV method cn thus be consdeed s genelzton of the lest sques method. oweve, Condton bove s no longe esected. A ceful choce of mt M s the e fcto n the mlementton of n IV method []. In the net secton, we wll evew vous ws tht hve been oosed to defne M.

276 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng 7... Revew of estng nstumentl vble methods fo the estmton of AR metes In 967, Wong nd Pol ublshed the fst tcle on the use of IV methods to chcteze lne sstems [6]. od, these methods e used n mn es, notbl n bomedcl lctons [3], ntenn flteng to estmte the decton of vl DOA [8] [], dtve estmton of the AR metes [], seech enhncement [4], estmton of hdden metes of Mov models [3], etc. In [], Stoc et l. nlze how the IV technques me t ossble to move the estmton of the snusodl model s metes. he ln between the IV technques nd the modfed Yule-Wle equtons 4 MYW hs been estblshed b Fedlnde [9]. In fct, f the M mt s chosen ccodng to the followng defnton: M Z Z Z [7.9] the estmto of equton [7.7] cn be vewed s the utocoelton functon of obsevton z when the numbe of smles s nfnte,.e. ˆ zz VI R R [7.] zz Equvlentl: R zz zz zz zz zz zz zz zz zz zz nd: R zz zz zz Equton [7.] coesonds to the modfed Yule-Wle equtons esented n Chte. he e used fo nstnce n the functon v n the Sstem 4 Chte, secton..6.8.

277 Estmton usng the Instumentl Vble echnque 6 Identfcton Mtlb toolbo fo the stndd mlementton of the IV estmtes. he efomnce of ths estmto led to n ARMA ocess s detled n []. In ddton to the bloc-bsed oches, sevel ecusve IV methods hve led been oosed [], notbl the ecusve method bsed on the MYW equtons [9]. Fedlnde et l. hve moeove suggested obtnng the nstumentl vbles b usng e-flteed veson of the nos obsevtons []. o do ths, Klmn flte cn be used. oweve, s ths flte eques o nowledge of the AR metes, ths leds to the so clled dul estmton oblem [8]. When the stte vecto n the stte sce eesentton of the sstem s defned s the conctenton of smles of the sgnl nd the AR metes, n etended Klmn flte EKF s needed to estmte the stte vecto. A nonlne soluton would consst of usng n EKF, but the convegence oetes of the EKF e not gunteed due to the omtons ntoduced b the lnezton of the oblem. In the fmewo of contol, elson et l. [7] hve used two successve Klmn fltes. he fst ms t estmtng the metes nd, once the convegence hs been eched, the sgnl s eteved b devng second nfnte-hozon Klmn flte bsed on the nnovton model nd the sted-stte Klmn gn. In eteme cses, when the sgnl-to-nose to s smll, the EKF m even dvege. o vod ths oblem, Chu et l. hve ut fowd the use of n EKF whose nomnl tjecto s clculted usng n ul lne Klmn flte [4]. In the net secton, we esent dul och, usng two mutull ntectve Klmn fltes [5] Klmn flteng nd the nstumentl vble method he och oosed hee s llustted n Fgue 7.3. It conssts of usng two mutull ntectve Klmn fltes [5]. Ech tme new obsevton s vlble, the sgnl s estmted usng the ltest estmted vlue of the metes, nd convesel the metes e estmted usng the ltest osteo sgnl estmte. In othe wods, fo ech tme ste, one Klmn flte ovdes the nstumentl vbles,.e. the sgnl estmtes, whle the second Klmn flte mes t ossble to ecusvel solve equton [7.7]. Both Klmn fltes e ll the moe mutull ntectve s the vnce of the nnovton of the fst flteng s used to dve the gn of the second flteng. It should be noted tht ths method etends the so-clled MISP Mutull Intectve Stte/Pmete estmton lgothm, ntll develoed b odn et l. [4] fo AutoRegessve Movng Avege exogeneous ARMAX model dentfcton, nd moe ecentl nvestgted b Mntovn et l. n [6].

278 6 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng z z Klmn flte : estmton of the stte vecto ˆ / Klmn flte : estmton of the stte vecto ˆ / Klmn flte : estmton of the AR metes ˆ / Klmn flte : estmton of the AR metes ˆ / Fgue 7.3. Pncle of the dul ntectve Klmn flte Sgnl estmton usng nos obsevtons Ou uose s to estmte the sgnl whch s modeled b th ode AR ocess. We stt wth the sstem s stte sce eesentton gven b equtons [7.]-[7.], wth the followng stte vecto: [7.] hs stte vecto stsfes the followng eltons:, Gu z b [7.] whee u s the dvng ocess, consdeed to be zeo-men whte Gussn ocess, wth constnt vnce Q u. Let us fst ssume tht the estmton of the AR metes s vlble. We cn then defne the tnston mt,, the nut vecto G nd the obsevton vecto s follows:

279 Estmton usng the Instumentl Vble echnque 63 ˆ ˆ, [7.3] nd: G [7.4] he fst Klmn flte of Fgue 7.3 ovdes the estmton of the stte vecto t nstnt, gven l obsevtons. hs estmton s denoted ˆ / l. In the followng, we denote P / l s the covnce mt of the eo ssocted wth ths estmton. he equtons fo udtng the flte e then gven b the followng eessons: ˆ / / ˆ / [7.5] P / / P / / GQG [7.5b] e ˆ / [7.5c] K P / P / R [7.5d] ˆ / ˆ / K e [7.5e] P / I K P / [7.5f] whee K s the Klmn flte gn nd e s the nnovton ocess. When the flte eches otmlt, the nnovton e becomes ndom zeomen whte ocess whose vnce C s eessed s follows []:

280 64 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng C P / R [7.6] Fnll, the estmton ˆ / s obtned s follows: ˆ / ˆ / [7.7] o estmte the AR metes, we oose usng second Klmn flte Estmton of AR metes usng the flteed sgnl When the AR ocess s consdeed to be stton, ts AR metes e constnt ove tme nd stsf:. [7.8] o estmte these metes usng the enhnced obsevtons, we cn eess the obsevton s estmton ˆ / s functon of s follows: ˆ /, ˆ / K e ˆ / K e e. [7.9] whee ˆ / s the obsevton vecto. he bove equton [7.9] s the e ste of ths dul-flte method. It onts out how the nstumentl vbles e used to estmte the AR metes. When the fst Klmn flte s otml, the ocess e K e s zeo-men ndom whte ocess uncoelted to. hus, equton [7.9] gves consstent estmton of the AR metes. Moeove, s we obseve n equton [7.9], the vnce R of ocess e s defned s follows: K C K. [7.] R Equtons [7.8]-[7.9] thus consttute stte sce eesentton fo the Klmn flte-bsed estmton of AR metes s gven below:

281 Estmton usng the Instumentl Vble echnque 65 ˆ / e. [7.] If ˆ / l s the estmton of the AR metes t nstnt, tng nto ccount l obsevtons, nd f P / l s the ssocted covnce mt, the equtons fo udtng the second Klmn flte e gven s follows: ˆ / ˆ / [7.] P / P / [7.b] K P / P / R [7.c] ˆ / ˆ / K e [7.d] I K P / P / [7.e] he estmton ˆ / s then fed nto the fst Klmn flte whch n tun estmtes the AR ocess t tme +. oweve, ths method s onl lcble f the vnces of the two ocesses u nd b e nown. he followng secton esents method fo the estmton of these vnces Estmton of the vnces of the dvng ocess nd the obsevton nose o estmte the vnce of the dvng ocess, we use the Rcct equton fom equtons [7.5b] nd [7.5f] bove,.e.: P /, P /, Q K P / [7.3]

282 66 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng As P / s smmetc mt nd gven equtons [7.5d] nd [7.6], P nd C e elted s follows: / C K [7.4] P / Combnng the bove two equtons, the covnce mt Q of the dvng ocess s eessed s follows: P P Q D /, /, K C K D [7.5] wth D G G G beng the seudo-nvese mt of G. Snce C s the vnce of the nnovton e Q s follows:, we cn ecusvel estmte Q ˆ Qˆ DL D [7.6] whee L P /, P /, K e K. he vnce of the ddtve nose b, denoted R, s deved fom equton [7.6] s follows: ˆ R Rˆ [7.7] wth e P / Concludng obsevtons In ddton to the nstumentl vble method tht we hve detled bove, sevel othe methods use the dul estmton of ocess nd ts metes. Fo emle, Oenhem et l. oosed the modelng of undesed nose usng n AR ocess [8] fo nose cncellton. he use Klmn flte to estmte the nose, whees LMS flte mes t ossble to udte the AR metes. In [7], Klmn smoothng s used to enhnce n AR ocess dstubed b nose, whch

283 Estmton usng the Instumentl Vble echnque 67 cn be ddtve, AR o mulse. he mete estmton ste s bsed on the Levnson lgothm o the ecusve lest sque lttce RLSL lgothm. oweve, the uthos do not eln how the enhnced sgnl s used to estmte the AR metes. All these technques e nheentl IV, but the method we esented n the evous secton dffes fom them n one motnt sect: the vnce of the nnovton s used to defne the stte sce eesentton of the AR metes Cse stud In ths secton, we oose to llustte the efomnces of the method esented n secton 7.3 fo the estmton of AR metes when the nose s whte, Gussn nd ddtve. Fo ths uose, we wll come t wth the bloc-bsed methods, the ecusve bs-comenston methods Pelmn obsevtons o set the stge fo the comson between the new method of secton 7.3 bove nd the estng methods, some elmn tests e conducted. he efomnce of the lgothm bsed on the mutull ntectve Klmn fltes s ecble when the SR s gete thn db. Moeove, the estmton of the obsevton nose s vnce R s e fcto hee becuse ths estmton detemnes the mount of nose to be cncelled. Fnll, s s the cse fo most ecusve methods, the tmedeendent behvo of the estmton deends on the dvng ocess nd the ntl condtons. We used equtons [7.6] nd [7.7] to estmte, esectvel, the mtces Q nd R. Fgues 7.4 nd 7.5 show the smulton esults obtned fo snthetc AR ocess wth smles. he oles of ths ocess e:.9e j.3., hs ocess s dstubed b zeo-men whte Gussn nose, wth SR of db. he dffeent methods e comed usng: the men estmton of the AR metes fo elztons of the ddtve nose; the locton of the estmted oles n the z-lne; the stud of the coesondng AR sectum.

284 68 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Fo elztons of the ddtve nose, the men estmted oles e: ˆ.97.7 e j.93.4., We note n Fgue 7.5 tht the new dul Klmn flte-bsed method cnnot dscmnte between the vnces of the dvng ocess nd the ddtve nose. hus, n the followng, we wll ssume tht R s estmted n the segments whch do not contn the sgnl. Imz Desed oles nd sect Rez Imz omlzed fequenc Levnson s lgothm Rez Imz omlzed fequenc Poosed method Rez omlzed fequenc Fgue 7.4. Locton of the oles n the z-lne, nd coesondng sect fo Levnson s lgothm nd the oosed method bsed on equtons [7.6] nd [7.7]

285 Estmton usng the Instumentl Vble echnque 69 Fgue 7.5. Convegence of the AR mete estmton nd vnces Q nd R, usng the new lgothm, fo one elzton of the sgnl hs eement ws conducted gn, tng nto ccount the vnce R whch s now nown o. We see fom Fgues 7.6 nd 7.7 tht the AR metes nd the vnce Q convege towds the desed vlues. oweve, the ocessng seed s low fo cetn combntons of metes. hs occus, fo emle, when the sgnl s sectum contns sh esonnces o neghbong esonnces,.e. when the dstnce between the two esonnces s lowe thn.. A lge numbe of smles s thus equed to ensue the convegence of the lgothm. Bsed lest sque estmtons of nd Q cn be used s the ntl vlues of the lgothm to seed u the convegence of the estmton.

286 7 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Imz Desed oles nd sect Rez Imz omlzed fequenc Poosed method Rez omlzed fequenc Fgue 7.6. Locton of the oles n the z-lne nd the coesondng AR sect; fo the oosed method usng equton [7.6], vnce R beng nown o Fgue 7.7. Convegence of the AR metes estmton nd the vnce Q usng Klmn flteng, fo one elzton, wth vnce R beng nown o

287 Estmton usng the Instumentl Vble echnque Comtve stud. Cse : whte ddtve nose In ths secton, we evew the dffeent estng methods fo the estmton of AR metes. Eght methods ncludng fou offlne methods wll be consdeed: Dvl method [5] offlne method, Zheng method [3] offlne method, sn method [3] offlne method, modfed Yule-Wle equtons [9] offlne method, -LMS lgothm [5] onlne method, -LMS lgothm [7] onlne method, Doblnge och [7] onlne method, the nstumentl vble och usng dul Klmn flteng. When the numbe of smles s eltvel hgh,.e. gete thn,, ll the bove methods ovde unbsed nd/o consstent estmtons of the AR metes. hus, to dstngush the efomnces, we wll consde the bodelne cses nd esent the esults fo the followng two tests. he fst conssts of nlzng the efomnce of the lgothms gven onl lmted numbe of smles,.e. sevel hunded, whch coesonds to the el cses of codng o seech enhncement. In the second test, we consde AR ocesses wth two sh es n the sectum. est no. : lmted numbe of smles of the nos obsevtons. Fo ths test, we genete 3 smles of n AR ocess whch s chctezed b the followng oles:..98 e j., e j.3, e j.7. hs ocess s then dstubed b n ddtve zeo-men whte Gussn nose such tht the SR s db. est no. : AR ocess wth two closel-sced sh es. ee, we genete 5 smles of n AR ocess chctezed b the followng oles:..98 e j., e j.3. hs ocess s then dstubed b n ddtve zeo-men whte Gussn nose such tht the SR s 5 db.

288 7 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Method Q Levnson Dvl Zheng sn MYW -LMS -LMS Doblnge Poosed Method Rejected cses Eected Vlue / ble 7.. est, men vlues of the estmted AR metes. he lst column esents the numbe of elztons unccounted fo n the vegng 5 he lst column gves the numbe of elztons unccounted fo whle clcultng the men. hese e ejected becuse the led to n unstble sstem.

289 Estmton usng the Instumentl Vble echnque 73 Eected oles 5 Imz 3 Rez Desed oles nd sect Eected sectum 6 Imz 4 omlzed fequenc Dvl method [5] Rez Imz omlzed fequenc Zheng method [3] Rez Imz omlzed fequenc sn method [3] Rez Imz omlzed fequenc MYW equtons [9] Rez omlzed fequenc Fgue 7.8. est, oles nd sect estmted b dffeent bloc-bsed methods

290 74 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Eected oles 5 Imz 3 Rez Desed oles nd sect Eected sectum 6 Imz 4 omlzed fequenc -LMS lgothm [5] Rez Imz omlzed fequenc -LMS lgothm [7] Rez Imz omlzed fequenc Doblnge method [7] Rez Imz omlzed fequenc Poosed method [5] Rez omlzed fequenc Fgue 7.9. est, oles nd sect estmted b dffeent ecusve methods

291 Estmton usng the Instumentl Vble echnque 75 Consdeng Fgues 7.8 nd 7.9 nd ble 7., we see tht when the numbe of smles s lmted, the bloc-bsed methods e one to dvegence. It should be noted tht Dves et l. [6] confm the dvegence nd nstblt of the tetve Zheng method. he sn method gves se to bsed estmtons of the AR metes. Method Q 3 4 Levnson Dvl Zheng sn MYW -LMS -LMS Doblnge IV Method unde stud Rejected cses Eected vlue / ble 7.. est, men estmted vlues of the AR metes

292 76 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng he ole estmtons obtned fom the ecusve methods le n the unt ccle n the z-lne. oweve, the methods bsed on the LMS lgothm emn n the tnsto zone. he Doblnge method nd the new dul Klmn flte-bsed method both llow the detecton of the fequences of the stongest es n the ocess s sectum. In the second eement, the model s ode s nown befoehnd to be 4. Fom Fgues 7. nd 7. nd ble 7., we see tht the Dvl method s hghl unstble: 6% of the elztons e not elgble fo consdeton. We note moeove tht not ll the methods convege. In ths eement, t s dffcult to choose the model s ode ccodng to the tdtonl cte 6. We thus stted wth ode =6. he stud conssts of comng the locton of the oles nd the shes of the AR sect. As Fgues 7. nd 7.3 show, the bloc-bsed methods e stll unstble. he Dolbnge method nd the new nstumentl vble method both llow the detemnton of the lowfequenc es n the sect. oweve, the she of the sectum s modfed n the hgh-fequenc egon. he clculton cost of the dul Klmn flte s of the ode of 3 clcultons e teton. hs vlue s comble to the comuttonl cost of the Doblnge method, but t emns hghe thn the LMS lgothm-bsed methods. Fo n AR ocess dstubed b whte nose, the dul Klmn flte-bsed method cheves the best tde-off between stblt nd estmton ccuc. hs comes t the cost of eltvel hgh clculton cost. 6 Fo moe detls on these cte, the ede s efeed to the lst secton of Chte.

293 Estmton usng the Instumentl Vble echnque 77 Eected oles Imz 3 Rez Desed oles nd sect Eected sectum Imz 4 omlzed fequenc Dvl method [5] Rez Imz omlzed fequenc Zheng method [3] Rez Imz omlzed fequenc sn method [3] Rez Imz omlzed fequenc MYW equtons [9] Rez omlzed fequenc Fgue 7.. est, =4, oles nd sect estmted b dffeent bloc methods

294 78 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Eected oles Imz 3 Rez Desed oles nd sect Eected sectum 4 Imz omlzed fequenc -LMS lgothm [5] Rez Imz omlzed fequenc -LMS lgothm [7] Rez Imz omlzed fequenc Doblnge method [7] Rez Imz omlzed fequenc Poosed method [5] Rez omlzed fequenc Fgue 7.. est, =4, oles nd sect estmted b ecusve methods

295 Estmton usng the Instumentl Vble echnque 79 Eected oles Imz 3 Rez Desed oles nd sect Eected sectum Imz 4 omlzed fequenc Dvl method [5] Rez Imz omlzed fequenc Zheng method [3] Rez Imz omlzed fequenc sn method [3] Rez Imz omlzed fequenc MYW equtons [9] Rez omlzed fequenc Fgue 7.. est, =6, oles nd sect estmted b bloc methods

296 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Eected oles Imz 3 Rez Desed oles nd sect Eected sectum Imz 4 omlzed fequenc -LMS lgothm [5] Rez Imz omlzed fequenc -LMS lgothm [7] Rez Imz omlzed fequenc Doblnge method [7] Rez Imz omlzed fequenc Poosed method [5] Rez omlzed fequenc Fgue 7.3. est, =6, oles nd sect estmted b ecusve methods

297 Estmton usng the Instumentl Vble echnque Concluson In ths chte, we hve esented the nstumentl vble technques. hese technques esent n ltentve to the tdtonl lest sques estmton methods. he e ste n the success of these methods s the choce of the nstumentl vbles. We hve lso oosed, fo the estmton of AR metes, new och n whch the nstumentl vbles e defned usng flteed veson of the AR ocess s nos obsevtons. hs flteed veson s obtned b Klmn flteng. he Klmn flte s otml onl f two ve stong ssumtons on dvng ocess nd obsevton nose e esected n the stte sce of the sstem. hese two ssumtons e the whteness nd Gussn ntue of the ocesses. hese hotheses e ve lmtng fo el-lfe stutons. In the net chte, we wll te u estmton technques whch ese these lmttons on the ocesses Refeences [] B. D. O. Andeson nd J. B. Mooe, Otml Flteng, Ed.. Klth, Pentce ll. Chte, 979. [] V. Buzenc-Settne nd M. jm, OLRIV: A ew Fst Algothm fo Rectngul- Bloc oeltz Sstems, IEEE ns. on Sgnl Pocessng, vol. 48, no. 9, , Setembe. [3] M. Chn, J. Agul-Mtn, P. Celcs nd J. P. Mc Vegnes, Instumentl Vble echnques n Ceebl Blood Flow Estmton Usng Ve Few Smles, IFAC Sm. on Identfcton nd Sstem Pmete Estmton, Jul 985. [4] C. K. Chu, G. Chen nd. C. Chu, Modfed Etended Klmn Flteng nd Rel- me Pllel Algothm fo Sstem Pmete Identfcton, IEEE ns. on Automtc Contol, vol. 35, no.,. -4, Jnu 99. [5] C. E. Dvl, A Subsce Aoch to Estmton of Autoegessve Pmetes fom os Mesuements, IEEE ns. on Sgnl Pocessng, vol. 46, no., , Febu 998. [6] R. Dves, U. Soven nd R. Gudoz, A ew Estmton Aoch fo AR Models n Pesence of ose, XVI th IFAC Wold Congess, Pgue, 5, 3-8 Jul 5. [7] G. Doblnge, Smoothng of os AR Sgnls Usng n Adtve Klmn Flte, EURASIP-EUSIPCO, vol., , Setembe 998. [8] K. Dognç, Bs Comenston fo the Bengs-Onl Pseudolne get c Estmto, IEEE ns. on Sgnl Pocessng, vol. 54, no., , Jnu 6. [9] B. Fedlnde, Instumentl Vble Methods fo ARMA Sectl Estmton, IEEE ns. on Acoustcs, Seech nd Sgnl Pocessng, vol. 3, no., , Al 983.

298 8 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng [] B. Fedlnde, he Ovedetemned Recusve Instumentl Vble Method, IEEE ns. on Automtc Contol, vol. AC-9, no. 4, , Al 984. [] B. Fedlnde nd K. C. Shmn, Pefomnce Evluton of the Modfed Yule- Wle Estmto, IEEE ns. on Acoustcs, Seech nd Sgnl Pocessng, vol. ASSP- 33, no. 3, , June 985. [] B. Fedlnde, P. Stoc nd. Södetöm, Instumentl Vble Methods fo ARMA Pmete Estmton, 7 th IFAC/IFORS Smosum on Identfcton nd Sstem Pmete Estmton, Yo, Englnd,. 9-36, Jul 985. [3] M. K. sn, J. ossn nd A. que, Pmete Estmton of Multchnnel Autoegessve Pocesses n ose, Sgnl Pocessng, vol. 83, no. 3,. 63-6, Jnu 3. [4] D. Lbe, E. Gvel, M. jm nd E. odn, wo-klmn Fltes Bsed Instumentl Vble echnques fo Seech Enhncement, IEEE-MMSP, Senn, Itl, 9 Setembe Octobe 4. [5] D. Lbe, E. Gvel, M. jm nd E. odn, Consstent Estmton of Autoegessve Pmetes fom os Obsevtons bsed on wo Intectng Klmn Fltes, Sgnl Pocessng, vol. 86, no., , Octobe 6. [6] P. Mntovn, A. Pstoe nd S. onellto, Recusve Estmton of Sstem Pmete n Envonmentl me Sees, M. Vch nd O. Otz Eds, Clssfcton nd Dt Anlss, Snge,. 3-38, 999. [7] L. W. elson nd E. Ste, he Smultneous On-Lne Estmton of Pmetes nd Sttes n Lne Sstems, IEEE ns. on Automtc Contol, vol., no., , Febu, 976. [8] A. V. Oenhem, E. Wensten, K. C. Zng, M. Fede nd D. Guge, Sngle-Senso Actve ose Cncellton, IEEE ns. on Acoustcs, Seech nd Sgnl Pocessng, vol., no.,. 85-9, Al 994. [9] O. Resol, Confluence Anlss b Mens of Lg Moments nd Othe Methods of Confluence Anlss, Econometc, vol. 9, no.,. -3, Jnu 94. []. Södetöm nd P. Stoc, Comson of some Instumentl Vble Methods Consstenc nd Accuc Asects, Automtc, vol. 7,. -5, Jnu 98. [] P. Stoc, M. Cedevll nd. Södetöm, Adtve Instumentl Vble Method fo Robust Decton-of-Avl Estmton, IEE Rd Son nd vgton, vol. 4, no., , Al 995. [] P. Stoc, B. Fedlnde nd. Södestöm, On Instumentl Vble Estmton of Snusod Fequences nd the Psmon Pncle, IEEE ns. on Automtc Contol, vol. AC-3, no. 8, , August 986. [3] J. S. hone nd J. B. Mooe, An Instumentl Vble Aoch fo Identfcton of dden Mov Models, 5 th Intentonl Smosum on Sgnl Pocessng nd ts Alctons ISSPA 99, Bsbne, Austl, vol.,. 3-6, -5 August 999.

299 Estmton usng the Instumentl Vble echnque 83 [4] E. odn, Mutull Intectve Stte/Pmete Estmton MISP Alcton of Klmn Flte to dolog, dulcs nd Wte Resouces, AGU Chmn Conf., Unvest of Pttsbug, M 978. [5] J. R. echle, nsent nd Convegent Behvo of the Adtve Lne Enhnce, IEEE ns. on Acoustcs, Seech nd Sgnl Pocessng, vol. 7, no.,. 53-6, Febu 979. [6] K. Y. Wong nd E. Pol, Identfcton of Lne Dscete me Sstems Usng the Instumentl Vble Method, IEEE ns. on Automtc Contol, vol., no. 6, , Decembe 967. [7] W.-R. Wu nd P.-C. Chen, Adtve AR Modelng n Whte Gussn ose, IEEE ns. on Sgnl Pocessng, vol. 45, no. 5,. 84-9, M 997. [8]. Yoshmu, K. Konsh nd. Soed, An Etended Klmn Flte fo Lne Dscete me Sstems wth Unnown Pmetes, Automtc, vol. 7, no. 4, , Jul 98. [9] P.-C. Young, An Instumentl Vble Method fo Rel-me Indentfcton of os Pocess, Automtc, vol. 6, no.,. 7-87, Mch 97. [3] W. X. Zheng, Autoegessve Pmete Estmton fom os Dt, IEEE ns. on Ccuts nd Sstems II: Anlog nd Dgtl Sgnl Pocessng, vol. 47, no.,. 7-75, Jnu.

300

301 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Mohmed jm Coght 8, ISE Ltd. Chte 8 Estmton: n Altentve to Klmn Flteng? 8.. Intoducton In the evous chte, metc oches hve oved to be oweful tools fo the esoluton of mn oblems n sgnl ocessng. evetheless, we must gud gnst ove-estmtng the usefulness nd te oe ccount of the lmttons. An gven model s t best n omton of the el wold nd modelng uncetntes lws est. he chllenge n ths omton s twofold: choosng the most ote eesentton of the sgnl, nd tng nto ccount the oetes of the nose whch often dstubs the obsevtons. It should be noted tht ths nose s tself modeled, ledng to ddtonl model uncetntes. Futhe eos e ntoduced dung the estmton of the model metes. hs estmton hevl deends on stong sttstcl ssumtons. In Klmn flteng, fo emle, the mmum lelhood estmton of the stte vecto s obtned f nd onl f the dvng ocess nd the obsevton nose e both whte, Gussn nd uncoelted. Moeove, the clsscl lgothms gve bsed o non-consstent estmtons when the obsevtons e dstubed b n ddtve mesuement nose. Fo futhe detls on ths mtte; see secton In ths chte, we nlze the elevnce of the -bsed oches n sgnl ocessng. he mjo dvntge of these oches s tht the ssumtons equed fo the mlementton e less estctve thn those needed fo the Klmn flte.

302 86 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng In the fst secton of ths chte, we wll ntoduce the ede to the subject of estmton bsed on the mnmzton of the nom. Fo moe detls on ths toc, see B. ssb, A. Sed nd. Klth [5]. he second t of the chte s dedcted to the estmton of AR metes usng flte to eteve sgnl fom nos obsevtons. Moe tcull, we oose method bsed on the combnton of two lgothms. hs method llows fo the consstent estmton of AR metes. Fnll, we stud the use of estmton technques n seech enhncement. 8.. Intoducton to estmton he nom ws ntoduced b Zmes [39] n 98 n the contet of contol engneeng. hs nom ovdes n ote fmewo fo the otml mngement of contol ms, whch e often contdcto to one nothe. he flteng cn be thought of s secfc cse whee the onl constnts e those lced on the efomnce. All estmton technques m t educng the estmton eo b ccountng fo the noses nd the modelng uncetntes. o do ths, these technques consde the wost-cse sceno. As stted b ssb et l. n [5], estmton s moe obust to uncetntes n the sstem eesentton thn Klmn flteng-bsed estmton. o o ssumton s equed hee, ethe fo the dvng ocess o fo the obsevton nose. hus, flteng s sml to the bounded eo estmton technques [7]. We wll see n the followng sectons tht the flteng theo cn be deved usng oeto theo, sstem theo nd gme theo [5]. hs mult-onged och to undestndng the theo s best descbed n the boo ublshed, n 999, b ssb, Sed nd Klth [6]. In ths wo, the uthos use the followng quotton fom Kmu [7]: It s emble tht contol llows such multtude of oches. It loos entel dffeent fom dffeent vewonts. hs fct cetnl mles tht contol s qute ch n logcl stuctue nd s vestle s n engneeng tool. oweve, the ognl queston of wht s the theoetcl coe of contol emns unnsweed. Indeed eve fundmentl noton mentoned hs method fo solvng the contol oblem ssocted wth t. Unfotuntel, howeve, length chns of esonng nd hghl techncl mnultons e the common chctestc fetues. I.e., the cse of the wost ossble dstubnces.

303 Estmton 87 We wll fst defne the nom nd then consde -bsed flteng. Secl ttenton wll be d to the ecusve flteng bsed on the Rcct equton Defnton of the nom Inut sequence u Outut sequence Fgue 8.. nsfe oeto In Fgue 8., the tnsfe oeto ms the nut sequence u nto the outut sequence. Its nom s defned s follows: su [8.] u u whee u, the l -nom of the cusl sequence u, cn be eessed s follows: u u. [8.] Bsed on the bove defnton, the nom cn be vewed s the mmum eneg gn fom the nut u to the outut flteng Let us consde sgnl s, whch s estmted usng obsevtons dstubed b n ddtve mesuement nose v. o estmte s, we ssume tht t s the outut of sstem ected wth the dvng ocess u. hs sceno he l nom of vecto combnng successve smles of the nut sequence u s defned b u / u. Moeove, u m u.

304 88 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng s dected n Fgue 8. nd cn be defned b the followng stte sce eesentton:, Gu v s L [8.3] ee, L s vecto whch connects the stte vecto to the hscl quntt tht needs to be estmted. In ou cse, ths quntt s the sgnl 3 s. If sˆ s n estmton of s, the estmton eo s defned s follows: e sˆ s [8.4] - Fgue 8.. Bsc schemtc eesentton fo estmton he technques thus m t mnmzng the nom of the sstem shown n Fgue 8.3 nd defned s follows: J su u, b, e Q e u R v ˆ ˆ [8.5] he oeton su. denotes the ue lmt nd s the numbe of vlble smles. Q nd R e two ostve weghtng scls to be tuned, n ode to djust the flte behvo. hese weghtng fctos esectvel l the sme ole s the 3 Genell, ths quntt s vecto. he mt L then lns the stte vecto to the vecto we wsh to estmte.

305 Estmton 89 vnces of the dvng ocess nd the obsevton nose when usng Klmn flteng [3]. Moeove, the tem ˆ ˆ mes t ossble to ccount fo the uncetnt n the ntl vlue of the estmton eo of the stte vecto. Fgue 8.3. nsfe oeto between the dstubnces nd the estmton eo As stted b Shed [7] nd ssb [5], t s often mossble to dectl mnmze cteon [8.5]. When t s ossble, closed-fom soluton of the bove flteng ssue does not lws est. hus, subotml soluton 5 s often sought. hs soluton conssts of mng cteon J smlle thn the followng ue lmt: J [8.6] Fcto of the bove lmt s clled the dstubnce ttenuton level [] [8] [8] [4] o estmton level [7]. he choce of wll be futhe dscussed n secton In the contet of ths chte, we focus ou ttenton on condton [8.6] whch gves the soluton of Rcct-te qudtc equton [5] [] [3]. he stuctue of the flte thus obtned s sml to the Klmn flte, theeb esng the comsons between the Klmn nd fltes. Consdeng equton of the stte sce eesentton [8.3], we note tht the mlementton of n -te och s bsed on the ssumton tht the mesuement nose, the dvng ocess nd the stte vecto e ll ndom. he vnces e denoted Q nd R esectvel. If these ocesses e ndom, the lne estmto of s usng the nos obsevtons,..., s bsed on the 5 Condton [8.6] s moe oblem of fesbllt thn one of otmzton, s stted b Vlo et l. [34] [35]. hee e sevel solutons. Addtonl cn lso be mosed, ledng to hbd / oches.

306 9 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng mnmzton of the men of the eo eneg e leds to the Klmn flte esented n Chte 5.. he esoluton of ths oblem In the och, the ddtve nose, the dvng ocess nd the ntl stte vecto need not be consdeed ndom. o sttstcl ssumtons hve thus to be mde on the dvng ocess nd the obsevton nose. he e just ssumed to hve fnte eneg. Fom equton [8.5] nd the eesentton of the nom, the uose of these methods s to mnmze the wost ossble effects of the dstubnces on the estmton eo. As mentoned b Gmble et l. n [4], the otml estmton oblem conssts of desgnng n estmto tht mnmzes the e eo owe n the fequenc domn whees Klmn flteng ms t mnmzng the vege eo owe. he bove obsevtons justf the nme dstubnce ttenuton level gven to fcto. he estmton eo s eneg must theefoe be smlle thn n ue lmt whch deends on tem wth the followng stuctue: Q u R v ˆ ˆ he estmton oblem cn be fomulted fom gme theo vewont. In fct, gven equton [8.6], let the sgnl ocessng engnee be the fst le, whose ts s to mnmze the eo on the sgnl estmton. s vl, the dstubnces, m t ncesng the estmton eo. he sech fo the estmto s thus equvlent to the esoluton of the followng mnm oblem: mn m sˆ, u, v ˆ ˆ s sˆ Q u R v [8.7] whee s scl quntt whch defnes the nfluence of dstubnces u, v nd ˆ ˆ. he soluton of the bove equton s sml to tht obtned b solvng Rcct equton. It should be noted tht the wos esented b Yesh n [38], heodo n [3], C n [], Zhung n [4], etc. ll concen ths nteetton. Accodng to L et l. [], the flteng cn be vewed fom dffeent ngles. We wll not te u fequenc-domn nlss, such s the hstocl oches of Gmble et l., [] [3], the nteolton methods [8] o the J-sectl fctozton [3] hee becuse the le outsde the scoe of ths boo.

307 Estmton 9 hee e othe oches fo the esoluton of equton [8.6]. hese e genell bsed on the esoluton of constned otmzton oblem. hus, n contol engneeng, cteon [8.6] s efomulted to enble esoluton usng lne mt neqult LMI technques. he LMI, sometmes thought of s stblt condton, cn be solved usng n otmzton ocedue. hs ocedue s vlble to the engnee n the LMI nd otmzton toolboes of Mtlb. he bove methods eque n-deth nowledge beond the scoe of ths boo, nd wll thus not be develoed futhe. he nteested ede s efeed to [] [] nd [4] fo futhe detls Rcct equton-bsed ecusve soluton of flteng Fo gven dstubnce ttenuton level, the estmton s ˆ whch esects condton [8.6] cn be obtned ovded tht: P / L L [8.8] whee the mt P / stsfes Rcct ecuson, whch cn be slt nto the two followng eltons: P /, P /, GQG [8.9] P / P / P / L M P / L [8.] R wth M P / L L ee, s oosed to the Klmn flte cse, mt P / does not denote the covnce of the estmton eo. Accodng to Yesh et l. [38], mt P / coesonds to n ue bound of the Klmn flte eo covnce mt: E ˆ / ˆ / P /. [8.]

308 9 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Gven the smltes between the Klmn nd the fltes, ths mt P / coesonds to the covnce mt of the eo n the wost-cse sceno. o clculton P / K ccountng fo the mesuement ˆ / =+ osteo clculton P / Fgue 8.4. Stuctue of the flte If condton [8.8] holds, the osteo estmtons of the stte vecto nd of e udted s follows: whee: ˆ / Lˆ / [8.] ˆ / ˆ / K e [8.3] ˆ /, ˆ / [8.4] K P / s the flte gn nd e ˆ / s the nnovton. R P /

309 Estmton 93 Model equton, G u Obsevton equton v Lne combnton of Pmetes to be djusted Equton fo udtng the estmted stte vecto s L Choce of weghtng scls Q nd R Detemnton of the dstubnce ttenuton level ˆ /, ˆ / ˆ / ˆ / K ˆ / Gn K P / P / R Rcct ecuson P /, P /, GQG P / P / P / L M P L R M P / L L wth Intl condtons ˆ / nd P / ble 8.. Equtons of the flte he followng obsevtons cn be mde on the equtons of the flte n ble 8.: the hs stuctue sml to the Klmn flte. Moe secfcll, both fltes e defned b Rcct ecusve equtons. Fo the flte, equton [8.] contns the ddtonl tem fo the dstubnce ttenuton level. Moeove, when tends towds nfnt, the eessons of the flte coesond to the Klmn flte [5]; the Rcct equton [8.9-8.] does not lws hve ostve defnte soluton, consdeng the ve defnton of mt M nd fcto contned n ths

310 94 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng mt. Unle the Klmn flte, the estence of the flte s subject to condton [8.8]; the choce of L s cucl hee becuse of the ole t ls n the Rcct equton. As we sw n the chte on Klmn flteng, the estmton of lne combnton of the stte vecto elements s lne combnton of the estmtons of the stte vbles. Moeove, the otml Klmn gn mnmzes the lne combntons of the dgonl elements of the eo s covnce mt. Fo flteng, we l most motnce on the edefned lne combnton L of the stte vbles; the vlue of s of me motnce becuse t defnes the nom of the sstem beng studed. In ths cse, ths s the sme s gunteeng the followng condton: P / L L hus, whle lng the flte to the estmton of utoegessve metes, we need to estmte the vecto of the AR metes. L thus coesonds to dentt mt I. At tme nstnce, the condton fo the estence of the flte s [8.8]: P / I [8.5] Accodng to the bove equton, the egenvlues of the I mt,.e. should be smlle thn the egenvlues of the mt P /,.e., should be gete thn the egenvlues of / P followng neqult: P /,, ledng to the m eg [8.6] whee egm m s the lgest egenvlue of the M mt. Sttng fom ths, the uthos of [3] oose ecusve contol of mete usng the followng udtng equton: P / m eg [8.7] whee s gete thn.

311 Estmton Revew of the use of flteng n sgnl ocessng Whle the theo hs been wdel led n the feld of contol [36] [37], ts use n sgnl ocessng s subject to moe nd moe ttenton. Ove the st few es, the sgnl ocessng communt hs shown n ncesng nteest. o the best of ou nowledge, nom-bsed technques hve so f been used n es such s dtve nose cncellton [5] [6], flte bn desgn [33], equlzton [6] [6] [4], llel smbol-chnnel estmton [] [8], nd mlementton of mult-use detectos fo communcton sstems wth nte-use ntefeences [34]. he uthos usull justf the choce of the technques b the fct tht the e well-dted to stutons whee the sttstcl chctestcs of the dstubnces e ethe unnown o dffcult to model nd nlze. he uthos lso hghlght the fct tht the technques gve esults whch e qute close to the esults obtned usng technques 6. Sevel uthos, notbl Shen et l. [8] [9] [3] nd Shmzu et l. [3], oose the use of flteng fo cses whee the sgnl s modeled b n AR ocess dstubed b n ddtve nose. he oches ten b these uthos dffe fom one nothe n the w the AR metes e estmted nd n the w the weght s contolled. hus, Shmzu et l. [3] te the tetve och nd oose the use of totl lest sques method bsed on the Go lgothm [9] fo the devton of the AR metes fom the nos obsevtons. On the othe hnd, Shen et l. [] [3] oose dul flteng usng two fltes connected n sees. he fst of these two fltes gves the estmton of the AR metes vecto ˆ, whle the second estmted the AR ocess usng the vlble nos obsevtons.... ˆ flte flte s ˆ... Fgue 8.5. he Shen et l. method [] [3]. s the numbe of vlble smles he Shen och, shown n Fgue 8.5, hs been led n es such s sgnl enhncement [3] nd weless communctons fo the smultneous estmton of the tnsmsson chnnel nd the tnsmtted dt []. 6 In the sttstcl sense of the tem.

312 96 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng In the followng secton, we wll consde sevel cses whch wll show tht ths cscde of two fltes s not necessl the most elevnt soluton, nd tht n ltentve och conssts of the use of llel combnton of two fltes Estmton of AR metes usng flteng In ths secton, we fst test the behvo of flteng led to nos obsevtons fo the estmton of utoegessve metes. hs test wll lso hghlght the lmttons of ths te of flteng. heefte, we oose n ltentve method, consstng of the combnton of two mutull ntectve estmtos flteng fo the estmton of AR metes Among the esech effots towds the use of flteng fo mete estmton, the wo of Gmble et l., descbed n [4], dels wth the dentfcton of noseless ARMAX 8 ocess usng flteng. he esults obtned b the uthos e qute close to those obtned usng the Klmn flte. Let s be th -ode AR ocess defned s follows: s s u [8.8] whee s the vecto of the AR metes, nd u the dvng ocess. Onl the followng obsevton of the nos ocess s vlble: s b. [8.9] he stte sce eesentton of the sstem of equtons [8.8]-[8.9] s thus defned s follows: 7 Dul Algothms fo Sgnl Pocessng Alcton to Seech Enhncement, D. Lbe, E. Gvel, M. jm nd. Chstov, IEEE ns. on Sgnl Pocessng, vol. 55, no., Fo defnton of the ARMAX model, see Chte.

313 Estmton 97 Y L [8.] whee L coesonds to the dentt mt. hs bove eesentton s sometmes sd to be degeneted becuse the dvng ocess hs zeo vlue. Elements Y nd e esectvel defned s follows: nd: Y [8.] u b b [8.] An -bsed flteng technque bsed on eesentton [8.] llows us to obtn ecusve estmton of the stte vecto. Pocess s coloed nd no o nfomton s vlble on t. Due to ths lc of nfomton, the use of the flte seems moe ote, comed to Klmn flteng, fo the estmton of the AR metes usng nos obsevtons. o nlze the behvo of the flte, we conduct the followng two tests on snthetc AR sgnls [9]. Eement : lge numbe of smles e vlble We genete, smles of n AR ocess chctezed b the followng oles n the z-lne:.75e j.,.8e j.4 nd, 3,4 5,6.85e j.7. he dvng ocess u s zeo-men, whte nd Gussn, wth vnce u. he AR ocess s then dstubed b zeo-men whte Gussn nose sequence. he esultng sgnl-to-nose to s equl to db.

314 98 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Eement : onl smll numbe of smles e vlble. Fo ths eement, we genete 5 smles of n AR ocess chctezed b the followng oles:.98e j.,.97e j.3 nd, 3,4 5,6.8e j.7. he dvng ocess u s gn zeo-men, whte nd Gussn, wth vnce u. As fo Eement, the AR ocess s then dstubed b zeomen, whte Gussn nose sequence. he esultng sgnl-to-nose to s equl to db. Method Klmn flte flte u Eected vlue ble 8.. Eement, men vlues nd vtons of the estmted AR metes bsed on elztons of the ddtve obsevton nose bles 8. nd 8.3 show tht the estmtons of the AR metes e bsed n both cses, nd the esults obtned e omtel the sme fo Klmn nd fltes. hs smlt cn be ttbuted to the fct tht the obsevton nose n the stte sce eesentton deends on the metes tht we see to estmte. Method Klmn flte u flte -..6 Eected vlue ble 8.3. Eement, men vlues nd vtons of the estmted AR metes bsed on elztons of the ddtve obsevton nose

315 Estmton 99 he followng soluton, oosed b Lbe et l., nd bsed on two wong n llel, llevtes the bs oblem: the fst flte enbles the enhncement of the nos obsevtons b usng the lst vlble estmton of the AR metes; the second flte llows the udte of the estmton, b usng the lst vlble estmton of the AR ocess nd the nnovton ocess, both ten fom the fst estmto Dul estmton of the AR ocess nd ts metes Let us consde the bsc bloc-level schemtc gven n Fgue 8.6. o estmte the AR ocess, we fst constuct the stte vecto s follows: s s [8.3] he stte sce eesentton of the sstem of equtons [8.8]-[8.9] s thus defned b the followng set of equtons:, Gu b s L [8.4] whee:, nd: L G

316 3 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Gven the ttenuton level, flte llows us to obtn flteed estmton of the sgnl: sˆ / Lˆ / ˆ / [8.5] he AR metes, howeve, e not nown nd hve to be estmted. Let us ssume them to be constnt ove n nlss fme,.e.: [8.6] he flteed veson of the sgnl cn be eessed n tems of the AR metes s follows: sˆ / ˆ / e K e [8.7] whee ˆ / nd e K e. ng equtons [8.6] nd [8.7] nto ccount, the stte sce eesentton of the AR metes s wtten s follows: sˆ / e [8.8] L whee L s the dentt mt. If we ssume the ttenuton level of the second flte to be nown, ths second flteng oeton mes t ossble to udte the estmton of the AR metes. A new weght, R, s now ntoduced.

317 Estmton 3 flte ˆ / flte ˆ / Q R Q R flte ˆ / flte ˆ / R R Sstem stte t tme - Sstem stte t tme - Sstem stte t tme tme Sstem stte t tme + Fgue 8.6. Bloc-level descton of the dul -bsed estmton unng the weghts R, Q nd R s delcte ocess. In [3], Shen et l. use n EM och, but do not detl how t s mlemented. hus, we oose n ltentve to ths och. We sw n secton 8.. bove tht these weghtng tems e equvlent to the vnces of the dvng ocess nd obsevton nose vnces n the Klmn flte. Moeove, ecllng the smlt between the stuctues of the Klmn nd fltes nd the nteetton of the P / mt, the weghtng mtces cn be djusted usng method nlogous to the dul och esented n secton 7.3. Fst, let us ssume tht the chctestcs of the ddtve obsevton nose e lmost tme-nvnt. he R mt cn thus be detemned o udted n the fmes whee thee s no sgnl.

318 3 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng o udte the weght Q, we oose the followng method: Q Q DM D [8.9] whee: M P /, P /, K e K nd: D In ddton, R s defned s: R K C K [8.3] wth C P / R. In the est of ths secton, we test the new och b lng t to nos snthetc AR dt. We then come the new method to the dul Klmn flteng esented n secton 7.3. he tests conducted e the sme s the two descbed n secton 8.3. bove. Method Dul Klmn flteng Dul flteng u Eected vlue ble 8.4. Eement, men vlues nd vtons of the estmted AR metes bsed on elztons of the ddtve obsevton nose

319 Estmton 33 Gven the esults shown n bles 8.4 nd 8.5, we cn dw the followng conclusons: comed to the dect lcton of the flte on the nos dt, the lgothm bsed on the dul stuctue llows the educton of the eos n the estmton of the AR metes; sml efomnces e obtned fo the dul methods bsed ethe on Klmn flteng o the flte. hs confms the sttements of [6] [6] [33]. Method Dul Klmn flteng Dul flteng Eected vlues ble 8.5. Eement, men vlues nd vtons of the estmted AR metes bsed on elztons of the ddtve obsevton nose o futhe move the estmton of the sgnl s, we cn elce the flteng oeton b smoothng. o mlement ths smoothng ocedue, we must fnd n estmton ˆ / M of the stte vecto fo ll vlues of M, tng,..., M obsevtons nto ccount, whee M s fed ntege. It hs evousl been shown tht the Klmn nd smoothng oetons e dentcl [5]. In [] [], the uthos concentte on fed-del smoothng. Fo ths te of smoothng, we must fnd n estmton ˆ m / of the stte vecto ˆ fo ll vlues of, tng,..., nto ccount, wth m beng fed ntege. o obtn ths estmton, we defne the stte vecto s follows: wth s s m [8.3] m. he AR model hs n ode. o deduce the smoothed estmton of the sgnl sˆ m / fom the estmton ˆ /, we defne the vecto L s follows: L. [8.3] m

320 34 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng In the followng secton, we stud the lcblt of the flteng n el cses, b tng the emle of seech sgnl enhncement Relevnce of flteng to seech enhncement Seech sgnls ehbt vous sectl chctestcs. In fct, thee cn be nosele sounds nown s unvoced sounds such s the consonnts //, /m/, etc., seudoeodc sounds,.e. voced sounds such s the vowels //, /e/, etc. nd med sounds, clled fctve sounds, whch e combntons of voced nd unvoced sounds, such s the consonnts /z/, /v/, etc. Moeove, the ddtve obsevton nose cn be coloed o whte, stton o non-stton. heefoe, seech enhncement mes t ossble to cove wde nge of sgnl ocessng cses. We consde the followng two otocols [9]: Potocol : we wll fst consde the smle tetboo cse whee both the noseless seech sgnl s nd the obsevton nose b e ssumed to be vlble. Once the AR metes e detemned fom s, we mlement n flte to enhnce the seech sgnl fom the nos obsevtons s b. We wll lso nlze the elevnce of the fed-ntevl smoothng oeton. Potocol : n ode to evlute the flte wth esect to the eos n the estmton of the AR metes, we fst estmte the AR metes usng the nos obsevtons. Comed to Potocol, ths estmton ntoduces ddtonl eos. We then mlement the tetve och descbed n Fgue 8.7. he flteng oeton cn be mlemented usng ethe Klmn flte o n flte. he ntl condtons fo the eements e s follows. he sgnl /WAZIWAZA/ 9, smled t 6 z, s dstubed b n ddtve nose. he esultng SR s equl to 5,, o 5 db. We stud thee dffeent tes of ddtve nose: ose : zeo-men whte Gussn nose elztons. ose : coloed movng vege MA nose geneted fom the followng.8e j..8e j.9 elztons. zeos:, ose 3: nose ecoded n c gong t m/h elzton. 9 he uthos would le to thn the Sgnl Pocessng Detment of ES Ps fo ovdng us wth the seech sgnl.

321 Estmton 35 Seech enhncement ocedue s bsed on fme-b-fme nlss, wth fme ovel of 5%. In ddton, mmng wndow s consdeed. Fo moe nfomton, see Chte 6. he wdth of ech fme s fed = 5 smles. he ode of the AR ocess used to model the seech sgnl s set to. he qult of the enhnced sgnl s mesued b the followng thee cte: nfoml subjectve tests IS, SR movement nd sectl nlss. Estmton of AR metes Estmton of AR metes ˆ ˆ Flteng s ˆ : Flteng s ˆ : : Fgue 8.7. Bloc-level esentton of the seech enhncement method used n Potocol When the ddtve obsevton nose s coloed, t cn be modeled b MA ocess. In ou smulton tests, the model ode s ssgned to 4 fo the cse nose nd to 6 fo nose 3. he stte sce eesentton equed to mlement the Klmn flte s descbed n secton.6.9. of Chte. Fo whte Gussn noses, the Klmn flte ovdes mmum lelhood otml estmton of the stte vecto. he ttenuton level s ve hgh. As the esults show n the tbles below, the ddton of -bsed flteng o smoothng does not ecbl move the efomnce.

322 36 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng Potocol Inut SR db 5 5 Klmn flte flte Klmn smoothng smoothng Potocol Inut SR db 5 5 Klmn flte flte ble 8.6. Men SR movement, nose Potocol Inut SR db 5 5 Klmn flte flte Klmn smoothng smoothng Potocol Inut SR db 5 5 Klmn flte flte ble 8.7. Men SR movement, nose

323 Estmton 37 me s b me s me s c d me s Fgue 8.8. Enhncement of voced seech segment vowel /A/. Potocol nd nose, ognl sgnl, b nos sgnl db, c sgnl obtned fte flteng, d sgnl obtned fte smoothng Potocol Inut SR db 5 5 Klmn flte.6..5 flte Klmn smoothng smoothng Potocol Inut SR db 5 5 Klmn flte flte ble 8.8. Men SR movement, nose 3

324 38 Modelng, Estmton nd Otml Flteng n Sgnl Pocessng As eoted n bles 8.7 nd 8.8, nd -bsed methods ovde sml esults when the ddtve obsevton nose s coloed. Accodng to IS cte, the sgnl qult s lmost the sme wth both methods. he use of flteng o smoothng s dvntgeous becuse t does not eque the metc modelng of the ddtve nose nd thus hs lowe clculton cost. me s me-domn eesentton of the nos sgnl me s me-domn eesentton of the sgnl enhnced b Klmn flte me s me-domn eesentton of the sgnl enhnced b flte

325 Estmton 39 Fequenc Kz Fequenc Kz Ognl sgnl Fequenc Kz me s Fequenc Kz os sgnl me s me s Klmn flteed sgnl me s flteed sgnl Fgue 8.9. Emle of seech enhncement. Potocol nd nose he mutull ntectve flte-bsed och esented n the secton bove cn be tested fo the sme lcton nd s comed to: the Shen och, descbed n [3], bsed on two sees-connected fltes; the mutull ntectve Klmn flte-bsed method, oosed b Lbe et l. n []. We notce fom ble 8.9 tht the methods bsed on llel-connected fltes gve hghe SR gns thn the och oosed b Shen et l. [3]. Fo the cse of whte ddtve nose, sml esults e obtned when usng mutull ntectve Klmn nd fltes. Fo coloed nose, the stuctue bsed on the two fltes gves gns slghtl lowe thn those obtned b the mutull ntectve Klmn fltes. oweve, the clculton comlet s lowe becuse o modelng fo the mesuement nose s not needed.