Robust Adaptive Volterra Filter under Maximum Correntropy Criteria in Impulsive Environments

Size: px
Start display at page:

Download "Robust Adaptive Volterra Filter under Maximum Correntropy Criteria in Impulsive Environments"

Transcription

1 Robust Adatve Volterra Flter uder Maxmum Corretroy Crtera Imulsve Evromets Weyua Wag Haqua Zhao Badog Che Abstract: As a robust adatato crtero the maxmum corretroy crtero (MCC) has gaed creased atteto due to ts successful alcato the adatato esecally olear ad o-gaussa stuatos. I ths aer the secod-order Volterra (SOV) flter based o MCC s derved whch s called MCC-SOV. It combes the advatages of the SOV flter ad MCC. However smlar to the covetoal adatve algorthm the roosed MCC- SOV flter has a tradeoff betwee covergece rate ad steady-state error. I order to solve ths roblem we reset a combato of the MCC-SOV flter by combg Volterra kerels whch s called CK-MCC-SOV. I addto a weght trasfer method s aled to mrove the erformace of the roosed flter terms of the covergece rate ad the steady-state error. Fally smulatos are carred out to demostrate the advatages of the roosed flters. Keywords: Adatve flterg Covex combato Maxmum corretroy crtera Secod-order Volterra flter Imulsve ose Itroducto As we all kow the cocet of lear system detfcato lays a mortat role statstcal sgal rocessg [8]. However sce may real alcatos requre comlex olear models modelg by meas of the lear flter may result oor erformace. I such case the olear system detfcato techques have become a hot toc over the last decades. To accurately model ad characterze the olear system lots of olear aroaches have bee roosed such as Haqua Zhao () hqzhao@home.swjtu.edu.c Weyua Wag weyuawag@my.swjtu.edu.c Badog Che chebd@mal.xjtu.edu.c School of Electrcal Egeerg Southwest Jaotog Uversty Chegdu Cha. School of Electroc ad Iformato Egeerg X a Jaotog Uversty X a Cha

2 olyomal exaso [34] eural etworks [4] Kalma flter [ 30] ad so o. Sce eural etworks have outstadg erformace the aroxmato of olear fucto they have bee wdely used olear system alcatos. Furthermore dfferet tyes of olear flters based o eural etworks have bee roosed ad aled to lots of aers. To obta better erformace for dscrete-tme Markova jum fuzzy eural etworks wth tme delays authors reseted the mxed H-Ifty ad Passve Flterg [4]. Besdes to address the sem-markova jum system authors reseted a ew method of sldg mode cotrol [5]. Comared wth methods of eural etworks ad Kalma flter olyomal exaso method has lower comutatoal comlexty. As oe of the most famous methods of olyomal exaso algorthm the Volterra flter ca coe wth a geeral class of olear systems [34 35] because most olear systems ca be aroxmately modeled by olyomals. Moreover sce the arameters of Volterra system are learly assocated wth the outut detfcato of ths model s a lear estmato roblem. Because the Volterra flter eeds a large umber of coeffcets for accurately modelg olear systems oly the secod-order Volterra (SOV) ad thrd-order Volterra (TOV) flters ca be used for the mlemetato. Smlar to the covetoal adatve flterg algorthms the adatve SOV flter usually uses the mmum mea square error (MMSE) crtero as the adatato crtero [4 7]. The MMSE crtero has a otmal soluto uder Gaussa ose evromet. However whe the outut sgals are corruted by the mulsve ose ths crtero would ot be robust. To address ths ssue the maxmum corretroy crtero (MCC) has bee roosed [5 6]. Corretroy whch has already bee used may areas such as adatve flterg classfcato ad robust regresso s a ew kd of measure to estmate the smlarty betwee two radom varables [ ]. Comared wth MMSE crtero the MCC ca rovde the strog atjammg caablty to mulsve terfereces though t may suffer from erformace degradg uder the Gaussa ose evromet. I [9] authors used corretroy as a cost fucto lear adatve flters. I [36] authors aled the MCC to kerel adatve flter whch was robust agast the o-gaussa measuremet ose. Nevertheless the aforemetoed algorthms have the tradeoff betwee covergece rate ad steady-state error o the choce of the fxed ste sze. To overcome the roblem above the covex combato of adatve flters has bee roosed [ ]. Due to combg the good erformace of dfferet adatve flters the covex combato method ca offer comlemetary caabltes. Sce ths method has such vrtue t has bee aled several felds of sgal rocessg cludg bld equalzato sgal characterzato ad so o [ ]. Referece [5] troduced a robust adatve algorthm by covexly combg two MCC-based adatve algorthms wth dfferet ste szes whch outerforms each of the dvdual flters. I [6] authors roosed a ovel affe rojecto sg algorthm (APSA) by adatvely combg two APSA flters to rovde good trackg erformace o-gaussa ose evromet. I ths aer a ew SOV flter based o MCC whch s called MCC-SOV flter s reseted. It s robust agast the mulsve ose. We vestgate the erformace of roosed MCC-SOV flter comarso wth the covetoal SOV flter uder the

3 mulsve ose crcumstace. Comared wth the algorthms [5] ad [4] whch are also robustess agast the o-gaussa ose ad ca oly obta good erformace secfc systems the roosed MCC-SOV flter ca be aled geeral olear system. Ufortuately there s a heret tradeoff betwee covergece rate ad steady-state error MCC-SOV flter. To overcome ths drawback a ovel algorthm derved by covexly combg two MCC-SOV flters s roosed where the scheme of the combato s obtaed by combg kerels (CK). It s called CK-MCC-SOV flter. The roosed algorthm rovdes sgfcatly comutatoal savgs comared wth the combato of Volterra flters (CVF) [3]. To rovde good robustess agast the o-gaussa mulsve terferece the mxg factor s obtaed by maxmzg the corretroy. Addtoally order to further mrove the erformace of combatos of kerel we med a weght trasfer method roosed [3] ad aly t to reseted algorthm. The rest of the aer s orgazed as follows. The bref revew of corretroy s gve Secto. I Secto 3 we reset MCC-SOV flter. The covex combato of the kerel of the SOV flter s rovded Secto 4. The smulato results are gve Secto 5. Fally the cocluso s draw Secto 6. Bref Revew of Corretroy The cocet of corretroy was reseted the lear adatve flter to deal wth o-gaussa ose esecally mulsve ose. The corretroy s a method to estmate the smlarty betwee two arbtrary radom varables X [ x... x ] T N ad Y y y N [... ] T. The corretroy s usually defed by [9] where E[ ] V ( X Y) E ( X Y) ( x y) df ( x y) () deotes the exectato oerator XY s a symmetrc ostve defte kerel FXY ( x y ) deotes the jot dstrbuto fucto ad s the kerel wdth. I ths aer we use a ormalzed Gaussa kerel to exress corretroy [6] e ( xy ) ex () where e x y ad reresets the kerel sze of corretroy. I ractce stuatos oly a fte umber of samles ( x y ) of the varables X ad Y are avalable. Therefore t s more commo to use the samle estmator for the exectato oerator N V( X Y) ( x y). (3) N Substtutg () to () ad alyg Taylor seres exaso yeld

4 ( X Y) V ( X Y) E ex ( X Y) E 0!. (4) Obvously corretroy ca be see as a measure of quatfyg how dfferet radom varable X from radom varable Y robablty cotrolled by the kerel wdth. Also corretroy s a geeralzed correlato fucto for two radom rocesses whch cludes more tha two order momets of the radom varable. Utlzg the corretroy as a cost fucto s effectve uder mulsve ose evromet. I the secfc case of adatve flterg alcato the weght udate equato ca be obtaed by maxmzg the corretroy betwee the outut of the flter ad observed sgal. We call ths crtero the maxmum corretroy crtero. Sce corretroy s sestve to outlers the MCC based adatve flterg algorthm ca offer a mactful mechasm to relef the ll effects of the large outlers resultg from mulsve ose. E X Y 3 Proosed MCC-SOV flter 3. Volterra flter algorthm v ( ) z ( ) x ( ) d ( ) Nolear System (Plat) e ( ) Model (SOV flter) y ( ) Adatve algorthm Fg. Nolear system detfcato model based o olyomal flter Sce the Volterra seres model has may advatages t s the most dffusely used model for olear systems. I artcular ths model s wdely used the olear adatve flter because the covetoal coceto or crtera of lear adatve flters ca be easly exteded to sut ths model. The structure of olear system detfcato based o adatve olyomal flter s show Fg where x() z() e() d() ad v() rereset the ut sgal lat outut error sgal desred sgal ad the mulsve ose resectvely. I ths aer we maly study the trucated SOV flter whose outut ca be rereseted as

5 N y h ( m ) x( m ) m 0 w N N h ( m m ) x( m ) x( m ). (5) m 0 m m T u where h( m ) ad h ( m m ) are frst ad secod-order kerels resectvely. The ta weght vector The ut vector w u s gve by The ta weght vector w [ h (0) h ()... h ( N ) h (00) s exressed as... h ( N N )] T u [ x... x( N ) x... x ( N )] T. (7) w (6) of SOV flter ca be udated by the least mea squares (LMS) algorthm whch the MMSE crtero s used as the cost fucto T where e d w u. J E e (8) MMSE Usg the method of stochastc gradet descet yelds [8] e w( ) w w. (9) Arragg (9) leads to the LMS-SOV flter as follows w( ) w e u (0) where s the ste sze. It s brght to have dfferet covergece factors for the frst- ad secod-order kerels of the Volterra flter []. I ths case the weght udate equato of the LMS- SOV flter are gve by w( ) w e ul () w( ) w e uq where w [ h (0) h ()... h ( N )] s the lear ta weght vector w [ h (00)... h ( N N )] T s the quadratc ta weght vector u l [ x... x( N )] s the ut vector for frst-order kerel the ut u ( ) [ ( )... ( )] T s the ut vector for secod-order kerel ad x x N q e d w ul w u q. ad lear ad quadratc kerels resectvely. () are the ste szes for

6 3. Proosed MCC-SOV flter However o-gaussa ose esecally mulsve ose the error sgal varace may be fte. Therefore the LMS algorthm s mroer. From [9] we ca get that the MCC s more robust agast mulsve ose tha the MMSE crtera. Thus we modfy the cost fucto as follows e ( ) J MCC d( ) y( ) ex. N N N N () Rearragg () yelds d( ) w ( ) ul w( ) uq( ) J MCC ex. N N (3) Smlar to MMSE crtero usg a teratve gradet ascet aroach to search the otmal soluto we have w( ) w w J MCC (4) w( ) w w J MCC (5) where ad are ostve costat. Substtutg (3) to (4) ad (5) yelds w( ) w N N e () ex (6) w e () ex w ( ) w. (7) N N w The gradets (6) ad (7) ca be derved as follows e () ex e () e () ex w w e () e ( ) ex e ( ) w e () d( ) w( ) ul( ) w( ) uq( ) ex e ( ) w e () ex e ( ) ul ( ) (8)

7 e () ex w e () ex w e () e ( ) ex e ( ) w e () d( ) w( ) ul( ) w( ) uq( ) e () ex e ( ) w e () ex e ( ) uq ( ). (9) Table. Summary of the roosed MCC-SOV flter Parameters settg: Italzato: w (0) w (0) For =.. do y w u w u ) l ) e d y e ( ) 3) w( ) w eex u ( ) l q e ( ) 4) w( ) w eex u ( ) q Ed for Cosequetly we ca get the MCC-SOV flter e () w( ) w ex e( ) ( ) 3 u l (0) N N e () w( ) w ex e( ) ( ) 3 u q. () N N Usg the curret value (.e. N =) to aroxmate the sum term (0) ad () yelds e ( ) w( ) w eex u ( ) l ()

8 e ( ) w( ) w eex u ( ) q (3) where ad 3 are the ste szes for the frst- ad 3 secod-order kerels resectvely. The summary of the roosed MCC-SOV flter s show Table. 4 Proosed CK-MCC-SOV flter 4. Proosed CK-MCC-SOV flter Accordg to [ 8 33] there s a heret tradeoff betwee covergece rate ad steady-state errors all adatve flters. I other words the large ste sze would lead to large steady-state error ad fast covergece rate. Ad vce versa. Thereby the roosed MCC-SOV flter has the drawback of coflctg requremet betwee fast covergece rate ad small steady-state error though t s more robust agast mulsve ose tha the covetoal algorthms. I order to overcome ths roblem we focus o the covex combato of two MCC-SOV flters whch the combato of two kerels wth dfferet ste szes s used to relace the corresodg kerel. The block dagram of the roosed combato of the SOV flters s llustrated Fg.. v ( ) z ( ) x ( ) d ( ) Nolear System + + e ( ) e () Lear kerel y w e Lear kerel w( ) e y + e ( ) y + y ( ) Quadratc kerel w Quadratc kerel w e y y + e ( ) y Fg.. Block dagram of the roosed combato of MCC Volterra flters scheme

9 For the sake of keeg the heret roertes of all comoet flters dvdual kerels should be deedetly adated accordg to ther ow crtera. I the sequel two lear kerels of the roosed scheme are adated through the followg recurso e w( ) w e ex u ( ) l e w' ( ) w' e ex u ( ) l (4) where the ad are the ste szes of lear kerel. The recurso of the quadratc kerels Fg. are gve by e w ( ) w e ex ( ) uq e w' ( ) w' e ex ( ) uq (5) where are the ste szes of the quadratc kerel. (4) ad (5) ad e deotes the ovel quadratc error sgals where = ad =. I order to make the dvdual kerels be adated deedetly the quadratc error sgals are defed as. e ( ) ad e ( ) are the outut error sgals of the comoet flters e d y y ' (6) ' where y deotes the combato of two costtuet kerels ad y s the outut of kerel. The relatosh betwee y ( ) ad y ( ) s gve by where ( ) y y y (7) y y y s the mxg arameter the rage of 0 to. Ths arameter cotrols the combato of the two flters at each terato ad comes from a sgmodal actvato fucto gve by sgm a (8) a e where a ( ) s the mxg arameter. I order to make the roosed CK-MCC-SOV flter robust agast mulsve ose a ew stochastc ostve gradet method based o the corretroy maxmzato crtera s utlzed to derve the udate equato for a as follows

10 a ( ) a a e ex a e a a ex e a e e a ex e ( ) ( ) a a e a a e ( ) ( ) ( ) ( ) ex ( e e ) where / s a ste-sze arameter. a a Although the udate equato (9) s robust agast the mulsve ose t s dffcult to choose the arorate ste sze [4]. To overcome ths dffculty a ormalzed rule s aled to adat the mxg arameter whch ca make selecto of ste sze smle ad lead to the more stable erformace of the flter. The the mxg arameter udate equato (9) ca be modfed as follows e ex a a( ) a r a (30) where r s gve by a e a e e ex e ( ) ( ) r s the estmate of where s the forgettg factor. e e a. A recursve way for comutg r (9) r r ( ) ( ) e e (3) The erformace of the roosed scheme by usg the way of the trasfer of the coeffcet [3] we adot ad modfy ths method as follows. ad are defed as wdow legth. The f mod N0 0 ad N 0 N 0 a ( ) a we use the followg equato to calculate w ' ( ) w' ( ) w ( ). (3)

11 Ste Italzato Table. Summary of the roosed CK-MCC-SOV flter Algorthm w (0) 0 m N 0 Calculate m y w' (0) 0 a (0) N 0 ad e w (0) 0 a 0 w' (0) 0 usg (6) ad (7) Udate fast flter accordg to (4) ad (5). Let. m m Calculate a m m ad ( ) usg (30) ad (8). If a ( ) a let a ( ) a ( ) 0. Else f a ( ) let a ( ) ( ).If also a m 0 let a w' ( ) w ( ) ad m N0 If the codtos ste 4 are ot met udate usg (4). w ' ( ) 6 If a ( ) a let a ( ) a ( ) 0. Else f a ( ) a let a ( ) a ( ). 7 If also m 0 let w' ( ) w( ) ad m N. 8 0 If the codtos ste 7 are ot met udate w ' ( ) usg (5). 9 Let ad retur to ste. Smlar to the lear kerel the quadratc kerel ca be udated as follows If mod N0 0 ad a ( ) a the followg equato s aled to comute w ' ( ) w' ( ) w ( ). (33) Comared wth the stadard combato method the addtoal oeratos of ths method are the mod N0 ad mod oeratos. However whe equatos (3) ad (33) are used to comute w ' ( ) ad w ' ( ) resectvely the comlexty of ths method s actually lower tha that of the orgal combato method. The summary of the roosed CK-MCC-SOV flter s exhbted Table. N 0 4. Covexly costraed mxtures of the roosed algorthm Accordg to the method [9 0-] the Taylor seres exaso of the error sgal e s exressed as e e ( ) e a a o a ( ) ( ) e a a (34)

12 where o a ( ( )) deotes the hgher order terms of the Taylor seres exaso. Equato (30) mles that e y y ( ) a ad a e ( ) a ex ( ) e y y ( ) r (36) Substtutg (35) ad (36) to (34) ad omttg the hgher order momet yeld a e ( ) e ( ) e ex ( ) y y (37) r I order to make the error sgal close to zero whe teds to fty the error sgal e satsfes e a e ( ) e ( ) e y y ex (38) r The we obta a e ( ) y y ex ( ) r (39) Accordgly the rages of the mxg arameters roosed algorthm are exressed as r 0 a ( ) e y y ex a (35) (40) 5 Smulatos To evaluate the effectveess of two roosed algorthms smulato exermets are carred out olear system detfcato uder mulsve ose evromet. The smulato results show Fgs. 3-7 are obtaed by averagg over 00 deedet trals. To assess the estmato erformace we use the followg ormalzed kerel error (NKE) [9] w Ψ NKE Ψ (4) w Ψ F NKE Ψ F where Ψ s the lear kerel of system Ψ s the quadratc kerel of system s the Eucldea -orm ad F s the Frobeus orm. The mea square devato

13 (MSD) s also used as a measure of the erformace whch s defed as MSD 0 log w w. s the otmal weght vector. 0 o w o 5. Exermet I ths exermet the erformace of the MCC-SOV flter s comared wth that of LMS-SOV [7] ad LMP-SOV [3] flters. Ths exermet s o the olear system detfcato where the olear system has the followg form [3] y x 0.8 x( ).9 x( 3) 0.95 x (4). x x( ) 0.63 x( ) x( 3) The ut sgals are obtaed by a zero-mea whte Gaussa rocess wth ut varace. The outut sgal-to-ose rato (SNR) s set to 30 db. The mulsve terferece s geerated as c A [7] where s a Beroull c ( ) rocess wth the robablty desty fucto descrbed by c c P (wth ( ) 0 r terferece) ad A ( ) T o P r ( ) Pr ad beg the robablty of the occurrece of the mulsve s a zero mea whte Gaussa ose wth varace A 00 E ( ) u w. I ths exermet Pr s set to 0.0. The ste szes of the MCC-SOV ad LMP-SOV flters are chose such that the tal covergece rates of the MCC-SOV ad LMP-SOV flters are aroxmately same. The ste sze of the LMP-SOV flter s same as that of MCC-SOV flter. The learg curves of ormalzed lear kerel error ad ormalzed quadratc kerel error are show Fg. 3. As we ca see both MCC-SOV ad LMP-SOV flters are robust agast the mulsve ose whle LMS-SOV flter has large fluctuatos. Sce the egevalue sread of the correlato matrx for quadratc terms s larger tha that of lear terms the estmato error for the lear kerel s less tha that of the quadratc kerel both methods whch ca be see Fg. 3. It s also foud that the roosed MCC- SOV flter has the less estmato error tha LMP-SOV flter.

14 Fg.3. Learg curve of MCC-SOV LMS-SOV ad LMP-SOV flters 5. Exermet I ths subsecto the erformace of the roosed CK-MCC-SOV flter s comared wth that of MCC-SOV flter. The ukow system ad the mulsve terferece are the same as those the frst exermet. The ut sgal s a zero-mea uform ose wth ut varat. The ste szes of slow flter s lear kerel ad quadratc kerel are set to 0.0 ad 0.0 resectvely. The ste szes of fast flter s lear kerel ad quadratc kerel are set to 0.5. ad Pr are set to 4 ad 0.0 resectvely. Besdes the ste sze of the mxture arameter s set to 0.. The arameter s set to 0.9. As show Fg. 4 the CK-MCC-SOV ad MCC-SOV flters are robust agast mulsve terferece. It s also see that the CK-MCC-SOV flter outerforms the MCC-SOV flter terms of the covergece rate ad the steady-state error. a

15 Fg.4. Learg curve of MCC-SOV ad CK-MCC-SOV flter Table 3. Covergece rates of MCC-SOV ad CK-MCC-SOV flter 34 Algorthm Covergece Rate/Iteratos (5.5dB for Lear kerel ad 0 db for Quadratc kerel) MCC (small ste sze) Lear Quadratc kerel kerel MCC (large ste sze) Lear Quadratc kerel kerel CK-MCC (o trasfer) Lear Quadratc kerel kerel Lear kerel CK-MCC Quadratc kerel Covergece Rate/Iteratos (38dB for Lear kerel ad 30 db for Quadratc kerel) The covergece rates of the MCC-SOV ad CK-MCC-SOV flters are also lsted Table 3 where covergece rates are deoted by the terato umber frstly 3 The MCC-SOV ad CK-MCC-SOV flters are abbrevated as MCC ad CK-MCC resectvely. 4 The covergece rates are measured by the terato umber where the algorthms frstly reach the corresodg MSD

16 Algorthm reachg the corresodg MSD. To comare the steady-state MSD we calculate the steady-state MSD by averagg over more tha 500 stataeous MSD values the steady state for each algorthm. The results of steady-state MSD are gve Table 4. It s see from Table 3 ad Table 4 that CK-MCC-SOV flter wth trasfer has fastest covergece rate ad lowest MSD. I the other word the roosed CK-MCC-SOV flter does mrove the drawback of MCC-SOV flter. Table 4. Steady-state MSD of MCC-SOV ad CK-MCC-SOV flter MCC(small ste sze) Lear Quadratc kerel kerel MCC(large ste sze) Lear Quadratc kerel kerel CK-MCC (o trasfer) Lear kerel Quadratc kerel Lear kerel CK-MCC Quadratc kerel Steady-state MSD (db) The comutatoal tme of the MCC-SOV ad CK-MCC-SOV flters s show Table 5. It s see that the comutato tme s ot related to the value of ste sze. I addto the CK-MCC-SOV flter wth trasfer has lower comutato tme tha the CK-MCC-SOV flter wthout trasfer. Although CK-MCC-SOV flter has loger comutato tme tha the MCC-SOV flter t has the great better erformace tha MCC-SOV flter. Observg Table 5 aga we ca also get that the large ste sze would result large steady-state MSD ad fast covergece rate. O the cotrary the large ste sze would lead to small steady-state MSD ad slow covergece rate whch s corresodg to the cocluso at the begg of Secto 4. I a word there s a tradeoff betwee the comutato tme the covergece rate steady-state MSD ad the ste sze. Table 5. Comutatoal tme of the MCC-SOV ad CK-MCC-SOV flters for obtag Fg. 4 by usg MATLAB R04b o Itel(R) Core(TM)5 CPU 4460 at 3.0 GHz rocessor wth 8.00 GB RAM Algorthm MCC(small ste sze) MCC(large ste sze) CK-MCC (o trasfer) CK-MCC Comutato tme (s) Fg. 5 evaluates the erformace of the CK-MCC-SOV flter wth dfferet levels of mulsveess. Pr s set to 0.0ad 0.. It s foud that the CK-MCC-SOV flter algorthm s robust agast mulsveess varous levels of mulsveess. We also exame the erformace of the CK-MCC-SOV flter wth dfferet levels of SNR Fg.6. The results dcate that CK-MCC-SOV flter outerforms the MCC- SOV flter terms of the covergece rate ad the steady-state error varous levels

17 of SNR. I ths exermet we also comare the erformace of the CK-MCC-SOV flter wth that of CK-SOV flter [3]. The results are show Fg.7. As oe ca see the MCC-SOV flter s robust agast mulsve terferece whle the CK algorthm s dverget. Fg.5. Learg curve of MCC-SOV ad CK-MCC-SOV flters dfferet levels of mulsveess

18 Fg.6. Learg curve of MCC-SOV ad CK-MCC-SOV flters dfferet levels of SNR Fg.7. Learg curve of CK-LMS-SOV ad CK-MCC-SOV flters 6 Cocluso The maxmum corretroy crtero (MCC) s robust agast the mulsve ose. Motvated by ths dea ths aer resets the MCC-SOV ad CK-MCC-SOV flters

19 for olear system detfcato. Addtoally a weght trasfer aroach s aled to mrove the erformace of the roosed algorthm. Dfferet from the methods of flter desg reseted [3 5 4] the roosed algorthms ths aer are adatve flter algorthm where arameters of flters are tme-varat. I addto the roosed algorthms ca be aled a geeral olear system whle algorthms [5 4] oly address the sem-markova jum system or Markova jum system. Fally smulato exermets are carred out to demostrate the advatages of the roosed algorthms. Exermets results show that the MCC-SOV flter could obta the sueror erformace over the covetoal LMS-SOV ad LMP-SOV flters whe the measuremet ose s mulsve. It s also foud that the CK-MCC-SOV flter ca acheve hgh covergece rate wthout sacrfcg steady-state MSE erformace comarso wth the MCC-SOV flter. Ackowledgmets Ths work was artally suorted by Natoal Scece Foudato of the P.R. Cha (Grat: ad 64330). Referece. J. Areas-García A. R. Fgueras-Vdal Adatve combato of ormalsed flters for robust system detfcato. Electro. Lett. 4(5) (005). J. Areas-García A. R. Fgueras-Vdal A. H. Sayed Mea-square erformace of a covex combato of two adatve flters. IEEE Tras. Sgal Process. 54(3) (006) 3. L. A. Azcueta-Ruz M. Zeller A. R. Fgueras-Vdal J. Areas-García W. Kellerma Adatve combato of Volterra kerels ad ts alcato to olear acoustc echo cacellato. IEEE Tras. Audo Seech Lag. Process. 9() 97-0 (0) 4. L. A. Azcueta-Ruz A. R. Fgueras-Vdal J. Areas-García A ormalzed adatato scheme for the covex combato of two adatve flters Proc. IEEE It. Cof. Acoustcs Seech Sgal Process. (008) R. J. Bessa V. Mrada J. Gama Etroy ad corretroy agast mmum square error offle ad ole three-day ahead wd ower forecastg. IEEE Tras Power Syst. 4(4) (009) 6. B. Che X. Lu H. Zhao J. C. Prce Maxmum Corretroy Kalma Flter. Automatca (07) 7. B. Che L. Xg H. Zhao N. Zheg J. C. Prce Geeralzed corretroy for robust adatve flterg. IEEE Tras. o Sgal Processg 64(3) (06) 8. B. Che J. Wag H. Zhao N. Zheg J. C. Prce Covergece of a Fxed-Pot Algorthm uder Maxmum Corretroy Crtero. IEEE Sgal Processg Letters (0) (05) 9. B. Che L. Xg J. Lag N. Zheg J. C. Prce Steady-state Mea-square Error Aalyss for Adatve Flterg uder the Maxmum Corretroy Crtero. IEEE Sgal Processg Letters (7) (04) 0. B. Che J. C. Prce Maxmum corretroy estmato s a smoothed MAP estmato IEEE Sgal Processg Letters 9(8) (0)

20 . P. S. R. Dz Adatve flterg algorthms ad ractcal mlemetato 4th ed. (Srger New York 03). G. A. Ecke L. B. Whte Robust exteded Kalma flterg IEEE tas. Sgal rocess. 47 (9) (999) 3. M. M. Foud M. G.Mostafa A.S.Mashat T. F. Gharb WSAWF: A weghted sldg wdow flterg algorthm for frequet weghted termets mg. It. J. Iov. Comut. If. Cotrol ( 4) (05) 4. T. Koh E. J. Powers Secod-order Volterra flterg ad ts alcato to olear system detfcato. IEEE Tras. Audo Seech Lag. Process. 33(6) (985) 5. F. L L. Wu P. Sh C. Lm State estmato ad sldg mode cotrol for sem- Markova jum systems wth msmatched ucertates. Automatcs (05) 6. W. Lu P. P. Pokharel J. C. Príce Corretroy: roertes ad alcatos o- Gaussa sgal rocessg. IEEE Tras. Sgal Process. 55() (007) 7. L. Lu H. Zhao K. L ad B. Che A ovel ormalzed sg algorthm for system detfcato uder mulsve ose terferece Crcuts Systems ad Sgal Processg 35(9) (06) 8. L. Lu H. Zhao Actve mulsve ose cotrol usg maxmum corretroy wth adatve kerel sze Mechacal Systems ad Sgal Processg (07) 9. L. Lu H. Zhao B. Che Collaboratve adatve Volterra flters for olear system detfcato α-stable ose evromets. Joural of the Frakl Isttute (06) 0. D. P. Madc NNGD algorthm for eural adatve flters Electro. Lett. 39(6) (000). D. P. Madc J. A. Chambers Toward the otmal learg rate for backroagato Neural Process. Lett. () (000). D. P. Madc A. I. Haa M. Razaz A ormalzed gradet descet algorthm for olear adatve flters usg a gradet adatve ste sze IEEE Sgal Process. Lett. 8() (00) 3. V. H. Nascmeto R. C. De. Lamare A low-comlexty strategy for seedg u the covergece of covex combatos of adatve flters I Proc. IEEE It. Cof. Acoustcs Seech Sgal Process. (0) P. Sh Y. Zhag M. Chadl R. K. Agarwal Mxed H-Ifty ad Passve Flterg for Dscrete Fuzzy Neural Networks Wth Stochastc Jums ad Tme Delays. IEEE Tras. Neural Netw. Lear. Syst. 7(4) (06) 5. L. Sh Y. L Covex Combato of Adatve Flters uder the Maxmum Corretroy Crtero Imulsve Iterferece. IEEE Sgal Process. Lett. () (04) 6. L. Sh Y. L X. Xe Combato of affe rojecto sg algorthms for robust adatve flterg o-gaussa mulsve terferece. Electro. Lett. 50(6) (04) 7. M. Sayad F. Faech M. Najm A LMS adatve secod-order Volterra flter wth a zeroth-order term: steady-state erformace aalyss a tme-varyg evromet. IEEE Tras. Sgal Process. 47(3) (999) 8. A. H. Sayed Fudametal of Adatve Flterg (Wley New Jersey 003) 9. A. Sgh J. C. Prce Usg corretroy as a cost fucto lear adatve flters IEEE Iteratoal Coferece o Acoustcs Seech ad Sgal Processg (009) S. Wag G. Q L. Wag Hgh degree cubature H-fty flter for a class of olear dscrete-tme systems. It. J. Iov. Comut. If. Cotrol ( ) (05)

21 3. B. Weg K. E. Barer Nolear system detfcato mulsve evromets. IEEE Tras. Sgal Process. 53(7) (005) 3. Y. Yu H. Zhao Adatve Combato of Proortoate NSAF wth the Ta-Weghts Feedback for Acoustc Echo Cacellato Wreless Pers. Commu (07) 33. Y. Yu H. Zhao B. Che A ew ormalzed subbad adatve flter algorthm wth dvdual varable ste szes Crcuts Systems ad Sgal Processg 35 (4) H. Zhao J. Zhag A ovel adatve olear flter-based eled feedforward secodorder Volterra archtecture IEEE Tras. Sgal Process.57() (009) 35. H. Zhao X. Zeg Z. He T. L ad W. J Adatve exteded eled secod-order Volterra flter for olear actve ose cotroller IEEE Tras. Audo. Seech. Lag. Process. 0(4) (0) 36. S. Zhao B. Che J. C. Prce Kerel adatve flterg wth maxmum corretroy crtero Proceedgs of Iteratoal Jot Coferece o Neural Networks (0). 0 07

A Robust Total Least Mean Square Algorithm For Nonlinear Adaptive Filter

A Robust Total Least Mean Square Algorithm For Nonlinear Adaptive Filter A Robust otal east Mea Square Algorthm For Nolear Adaptve Flter Ruxua We School of Electroc ad Iformato Egeerg X'a Jaotog Uversty X'a 70049, P.R. Cha rxwe@chare.com Chogzhao Ha, azhe u School of Electroc

More information

STRONG CONSISTENCY FOR SIMPLE LINEAR EV MODEL WITH v/ -MIXING

STRONG CONSISTENCY FOR SIMPLE LINEAR EV MODEL WITH v/ -MIXING Joural of tatstcs: Advaces Theory ad Alcatos Volume 5, Number, 6, Pages 3- Avalable at htt://scetfcadvaces.co. DOI: htt://d.do.org/.864/jsata_7678 TRONG CONITENCY FOR IMPLE LINEAR EV MODEL WITH v/ -MIXING

More information

Recursive linear estimation for discrete time systems in the presence of different multiplicative observation noises

Recursive linear estimation for discrete time systems in the presence of different multiplicative observation noises Recursve lear estmato for dscrete tme systems the resece of dfferet multlcatve observato oses C. Sáchez Gozález,*,.M. García Muñoz Deartameto de Métodos Cuattatvos ara la Ecoomía y la Emresa, Facultad

More information

Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur

Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur Aalyss of Varace ad Desg of Exermets-I MODULE II LECTURE - GENERAL LINEAR HYPOTHESIS AND ANALYSIS OF VARIANCE Dr Shalabh Deartmet of Mathematcs ad Statstcs Ida Isttute of Techology Kaur Tukey s rocedure

More information

Functions of Random Variables

Functions of Random Variables Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,

More information

2. Independence and Bernoulli Trials

2. Independence and Bernoulli Trials . Ideedece ad Beroull Trals Ideedece: Evets ad B are deedet f B B. - It s easy to show that, B deedet mles, B;, B are all deedet ars. For examle, ad so that B or B B B B B φ,.e., ad B are deedet evets.,

More information

D KL (P Q) := p i ln p i q i

D KL (P Q) := p i ln p i q i Cheroff-Bouds 1 The Geeral Boud Let P 1,, m ) ad Q q 1,, q m ) be two dstrbutos o m elemets, e,, q 0, for 1,, m, ad m 1 m 1 q 1 The Kullback-Lebler dvergece or relatve etroy of P ad Q s defed as m D KL

More information

Introduction to local (nonparametric) density estimation. methods

Introduction to local (nonparametric) density estimation. methods Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest

More information

best estimate (mean) for X uncertainty or error in the measurement (systematic, random or statistical) best

best estimate (mean) for X uncertainty or error in the measurement (systematic, random or statistical) best Error Aalyss Preamble Wheever a measuremet s made, the result followg from that measuremet s always subject to ucertaty The ucertaty ca be reduced by makg several measuremets of the same quatty or by mprovg

More information

Research on Efficient Turbo Frequency Domain Equalization in STBC-MIMO System

Research on Efficient Turbo Frequency Domain Equalization in STBC-MIMO System Research o Effcet urbo Freuecy Doma Eualzato SBC-MIMO System Wau uag Bejg echology ad Busess Uversty Bejg 00048.R. Cha Abstract. A effcet urbo Freuecy Doma Eualzato FDE based o symbol-wse mmum mea-suare

More information

Training Sample Model: Given n observations, [[( Yi, x i the sample model can be expressed as (1) where, zero and variance σ

Training Sample Model: Given n observations, [[( Yi, x i the sample model can be expressed as (1) where, zero and variance σ Stat 74 Estmato for Geeral Lear Model Prof. Goel Broad Outle Geeral Lear Model (GLM): Trag Samle Model: Gve observatos, [[( Y, x ), x = ( x,, xr )], =,,, the samle model ca be exressed as Y = µ ( x, x,,

More information

An Introduction to. Support Vector Machine

An Introduction to. Support Vector Machine A Itroducto to Support Vector Mache Support Vector Mache (SVM) A classfer derved from statstcal learg theory by Vapk, et al. 99 SVM became famous whe, usg mages as put, t gave accuracy comparable to eural-etwork

More information

Quantum Plain and Carry Look-Ahead Adders

Quantum Plain and Carry Look-Ahead Adders Quatum Pla ad Carry Look-Ahead Adders Ka-We Cheg u8984@cc.kfust.edu.tw Che-Cheg Tseg tcc@ccms.kfust.edu.tw Deartmet of Comuter ad Commucato Egeerg, Natoal Kaohsug Frst Uversty of Scece ad Techology, Yechao,

More information

Estimation of Stress- Strength Reliability model using finite mixture of exponential distributions

Estimation of Stress- Strength Reliability model using finite mixture of exponential distributions Iteratoal Joural of Computatoal Egeerg Research Vol, 0 Issue, Estmato of Stress- Stregth Relablty model usg fte mxture of expoetal dstrbutos K.Sadhya, T.S.Umamaheswar Departmet of Mathematcs, Lal Bhadur

More information

Channel Models with Memory. Channel Models with Memory. Channel Models with Memory. Channel Models with Memory

Channel Models with Memory. Channel Models with Memory. Channel Models with Memory. Channel Models with Memory Chael Models wth Memory Chael Models wth Memory Hayder radha Electrcal ad Comuter Egeerg Mchga State Uversty I may ractcal etworkg scearos (cludg the Iteret ad wreless etworks), the uderlyg chaels are

More information

Two Fuzzy Probability Measures

Two Fuzzy Probability Measures Two Fuzzy robablty Measures Zdeěk Karíšek Isttute of Mathematcs Faculty of Mechacal Egeerg Bro Uversty of Techology Techcká 2 66 69 Bro Czech Reublc e-mal: karsek@umfmevutbrcz Karel Slavíček System dmstrato

More information

2006 Jamie Trahan, Autar Kaw, Kevin Martin University of South Florida United States of America

2006 Jamie Trahan, Autar Kaw, Kevin Martin University of South Florida United States of America SOLUTION OF SYSTEMS OF SIMULTANEOUS LINEAR EQUATIONS Gauss-Sedel Method 006 Jame Traha, Autar Kaw, Kev Mart Uversty of South Florda Uted States of Amerca kaw@eg.usf.edu Itroducto Ths worksheet demostrates

More information

å 1 13 Practice Final Examination Solutions - = CS109 Dec 5, 2018

å 1 13 Practice Final Examination Solutions - = CS109 Dec 5, 2018 Chrs Pech Fal Practce CS09 Dec 5, 08 Practce Fal Examato Solutos. Aswer: 4/5 8/7. There are multle ways to obta ths aswer; here are two: The frst commo method s to sum over all ossbltes for the rak of

More information

Lecture 9. Some Useful Discrete Distributions. Some Useful Discrete Distributions. The observations generated by different experiments have

Lecture 9. Some Useful Discrete Distributions. Some Useful Discrete Distributions. The observations generated by different experiments have NM 7 Lecture 9 Some Useful Dscrete Dstrbutos Some Useful Dscrete Dstrbutos The observatos geerated by dfferet eermets have the same geeral tye of behavor. Cosequetly, radom varables assocated wth these

More information

IS 709/809: Computational Methods in IS Research. Simple Markovian Queueing Model

IS 709/809: Computational Methods in IS Research. Simple Markovian Queueing Model IS 79/89: Comutatoal Methods IS Research Smle Marova Queueg Model Nrmalya Roy Deartmet of Iformato Systems Uversty of Marylad Baltmore Couty www.umbc.edu Queueg Theory Software QtsPlus software The software

More information

Lecture Notes Types of economic variables

Lecture Notes Types of economic variables Lecture Notes 3 1. Types of ecoomc varables () Cotuous varable takes o a cotuum the sample space, such as all pots o a le or all real umbers Example: GDP, Polluto cocetrato, etc. () Dscrete varables fte

More information

A Method for Damping Estimation Based On Least Square Fit

A Method for Damping Estimation Based On Least Square Fit Amerca Joural of Egeerg Research (AJER) 5 Amerca Joural of Egeerg Research (AJER) e-issn: 3-847 p-issn : 3-936 Volume-4, Issue-7, pp-5-9 www.ajer.org Research Paper Ope Access A Method for Dampg Estmato

More information

KLT Tracker. Alignment. 1. Detect Harris corners in the first frame. 2. For each Harris corner compute motion between consecutive frames

KLT Tracker. Alignment. 1. Detect Harris corners in the first frame. 2. For each Harris corner compute motion between consecutive frames KLT Tracker Tracker. Detect Harrs corers the frst frame 2. For each Harrs corer compute moto betwee cosecutve frames (Algmet). 3. Lk moto vectors successve frames to get a track 4. Itroduce ew Harrs pots

More information

Chapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements

Chapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements Aoucemets No-Parametrc Desty Estmato Techques HW assged Most of ths lecture was o the blacboard. These sldes cover the same materal as preseted DHS Bometrcs CSE 90-a Lecture 7 CSE90a Fall 06 CSE90a Fall

More information

Nonparametric Density Estimation Intro

Nonparametric Density Estimation Intro Noarametrc Desty Estmato Itro Parze Wdows No-Parametrc Methods Nether robablty dstrbuto or dscrmat fucto s kow Haes qute ofte All we have s labeled data a lot s kow easer salmo bass salmo salmo Estmate

More information

Comparison of Dual to Ratio-Cum-Product Estimators of Population Mean

Comparison of Dual to Ratio-Cum-Product Estimators of Population Mean Research Joural of Mathematcal ad Statstcal Sceces ISS 30 6047 Vol. 1(), 5-1, ovember (013) Res. J. Mathematcal ad Statstcal Sc. Comparso of Dual to Rato-Cum-Product Estmators of Populato Mea Abstract

More information

Random Variables. ECE 313 Probability with Engineering Applications Lecture 8 Professor Ravi K. Iyer University of Illinois

Random Variables. ECE 313 Probability with Engineering Applications Lecture 8 Professor Ravi K. Iyer University of Illinois Radom Varables ECE 313 Probablty wth Egeerg Alcatos Lecture 8 Professor Rav K. Iyer Uversty of Illos Iyer - Lecture 8 ECE 313 Fall 013 Today s Tocs Revew o Radom Varables Cumulatve Dstrbuto Fucto (CDF

More information

Application of Generating Functions to the Theory of Success Runs

Application of Generating Functions to the Theory of Success Runs Aled Mathematcal Sceces, Vol. 10, 2016, o. 50, 2491-2495 HIKARI Ltd, www.m-hkar.com htt://dx.do.org/10.12988/ams.2016.66197 Alcato of Geeratg Fuctos to the Theory of Success Rus B.M. Bekker, O.A. Ivaov

More information

13. Parametric and Non-Parametric Uncertainties, Radial Basis Functions and Neural Network Approximations

13. Parametric and Non-Parametric Uncertainties, Radial Basis Functions and Neural Network Approximations Lecture 7 3. Parametrc ad No-Parametrc Ucertates, Radal Bass Fuctos ad Neural Network Approxmatos he parameter estmato algorthms descrbed prevous sectos were based o the assumpto that the system ucertates

More information

Analysis of Lagrange Interpolation Formula

Analysis of Lagrange Interpolation Formula P IJISET - Iteratoal Joural of Iovatve Scece, Egeerg & Techology, Vol. Issue, December 4. www.jset.com ISS 348 7968 Aalyss of Lagrage Iterpolato Formula Vjay Dahya PDepartmet of MathematcsMaharaja Surajmal

More information

Econometric Methods. Review of Estimation

Econometric Methods. Review of Estimation Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators

More information

2SLS Estimates ECON In this case, begin with the assumption that E[ i

2SLS Estimates ECON In this case, begin with the assumption that E[ i SLS Estmates ECON 3033 Bll Evas Fall 05 Two-Stage Least Squares (SLS Cosder a stadard lear bvarate regresso model y 0 x. I ths case, beg wth the assumto that E[ x] 0 whch meas that OLS estmates of wll

More information

Analysis of Variance with Weibull Data

Analysis of Variance with Weibull Data Aalyss of Varace wth Webull Data Lahaa Watthaacheewaul Abstract I statstcal data aalyss by aalyss of varace, the usual basc assumptos are that the model s addtve ad the errors are radomly, depedetly, ad

More information

CHAPTER 6. d. With success = observation greater than 10, x = # of successes = 4, and

CHAPTER 6. d. With success = observation greater than 10, x = # of successes = 4, and CHAPTR 6 Secto 6.. a. We use the samle mea, to estmate the oulato mea µ. Σ 9.80 µ 8.407 7 ~ 7. b. We use the samle meda, 7 (the mddle observato whe arraged ascedg order. c. We use the samle stadard devato,

More information

Parameter Estimation

Parameter Estimation arameter Estmato robabltes Notatoal Coveto Mass dscrete fucto: catal letters Desty cotuous fucto: small letters Vector vs. scalar Scalar: la Vector: bold D: small Hgher dmeso: catal Notes a cotuous state

More information

Comparing Different Estimators of three Parameters for Transmuted Weibull Distribution

Comparing Different Estimators of three Parameters for Transmuted Weibull Distribution Global Joural of Pure ad Appled Mathematcs. ISSN 0973-768 Volume 3, Number 9 (207), pp. 55-528 Research Ida Publcatos http://www.rpublcato.com Comparg Dfferet Estmators of three Parameters for Trasmuted

More information

Nonlinear Blind Source Separation Using Hybrid Neural Networks*

Nonlinear Blind Source Separation Using Hybrid Neural Networks* Nolear Bld Source Separato Usg Hybrd Neural Networks* Chu-Hou Zheg,2, Zh-Ka Huag,2, chael R. Lyu 3, ad Tat-g Lok 4 Itellget Computg Lab, Isttute of Itellget aches, Chese Academy of Sceces, P.O.Box 3, Hefe,

More information

Ordinary Least Squares Regression. Simple Regression. Algebra and Assumptions.

Ordinary Least Squares Regression. Simple Regression. Algebra and Assumptions. Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos

More information

Simulation Output Analysis

Simulation Output Analysis Smulato Output Aalyss Summary Examples Parameter Estmato Sample Mea ad Varace Pot ad Iterval Estmato ermatg ad o-ermatg Smulato Mea Square Errors Example: Sgle Server Queueg System x(t) S 4 S 4 S 3 S 5

More information

BASIC PRINCIPLES OF STATISTICS

BASIC PRINCIPLES OF STATISTICS BASIC PRINCIPLES OF STATISTICS PROBABILITY DENSITY DISTRIBUTIONS DISCRETE VARIABLES BINOMIAL DISTRIBUTION ~ B 0 0 umber of successes trals Pr E [ ] Var[ ] ; BINOMIAL DISTRIBUTION B7 0. B30 0.3 B50 0.5

More information

CS 2750 Machine Learning Lecture 5. Density estimation. Density estimation

CS 2750 Machine Learning Lecture 5. Density estimation. Density estimation CS 750 Mache Learg Lecture 5 esty estmato Mlos Hausrecht mlos@tt.edu 539 Seott Square esty estmato esty estmato: s a usuervsed learg roblem Goal: Lear a model that rereset the relatos amog attrbutes the

More information

Bayes Estimator for Exponential Distribution with Extension of Jeffery Prior Information

Bayes Estimator for Exponential Distribution with Extension of Jeffery Prior Information Malaysa Joural of Mathematcal Sceces (): 97- (9) Bayes Estmator for Expoetal Dstrbuto wth Exteso of Jeffery Pror Iformato Hadeel Salm Al-Kutub ad Noor Akma Ibrahm Isttute for Mathematcal Research, Uverst

More information

Solving Constrained Flow-Shop Scheduling. Problems with Three Machines

Solving Constrained Flow-Shop Scheduling. Problems with Three Machines It J Cotemp Math Sceces, Vol 5, 2010, o 19, 921-929 Solvg Costraed Flow-Shop Schedulg Problems wth Three Maches P Pada ad P Rajedra Departmet of Mathematcs, School of Advaced Sceces, VIT Uversty, Vellore-632

More information

Lecture 3 Probability review (cont d)

Lecture 3 Probability review (cont d) STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto

More information

Beam Warming Second-Order Upwind Method

Beam Warming Second-Order Upwind Method Beam Warmg Secod-Order Upwd Method Petr Valeta Jauary 6, 015 Ths documet s a part of the assessmet work for the subject 1DRP Dfferetal Equatos o Computer lectured o FNSPE CTU Prague. Abstract Ths documet

More information

Multiple Choice Test. Chapter Adequacy of Models for Regression

Multiple Choice Test. Chapter Adequacy of Models for Regression Multple Choce Test Chapter 06.0 Adequac of Models for Regresso. For a lear regresso model to be cosdered adequate, the percetage of scaled resduals that eed to be the rage [-,] s greater tha or equal to

More information

Modified Cosine Similarity Measure between Intuitionistic Fuzzy Sets

Modified Cosine Similarity Measure between Intuitionistic Fuzzy Sets Modfed ose mlarty Measure betwee Itutostc Fuzzy ets hao-mg wag ad M-he Yag,* Deartmet of led Mathematcs, hese ulture Uversty, Tae, Tawa Deartmet of led Mathematcs, hug Yua hrsta Uversty, hug-l, Tawa msyag@math.cycu.edu.tw

More information

Computations with large numbers

Computations with large numbers Comutatos wth large umbers Wehu Hog, Det. of Math, Clayto State Uversty, 2 Clayto State lvd, Morrow, G 326, Mgshe Wu, Det. of Mathematcs, Statstcs, ad Comuter Scece, Uversty of Wscos-Stout, Meomoe, WI

More information

ESS Line Fitting

ESS Line Fitting ESS 5 014 17. Le Fttg A very commo problem data aalyss s lookg for relatoshpetwee dfferet parameters ad fttg les or surfaces to data. The smplest example s fttg a straght le ad we wll dscuss that here

More information

{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:

{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution: Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed

More information

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted

More information

Bayesian Classification. CS690L Data Mining: Classification(2) Bayesian Theorem: Basics. Bayesian Theorem. Training dataset. Naïve Bayes Classifier

Bayesian Classification. CS690L Data Mining: Classification(2) Bayesian Theorem: Basics. Bayesian Theorem. Training dataset. Naïve Bayes Classifier Baa Classfcato CS6L Data Mg: Classfcato() Referece: J. Ha ad M. Kamber, Data Mg: Cocepts ad Techques robablstc learg: Calculate explct probabltes for hypothess, amog the most practcal approaches to certa

More information

arxiv: v1 [cs.lg] 22 Feb 2015

arxiv: v1 [cs.lg] 22 Feb 2015 SDCA wthout Dualty Sha Shalev-Shwartz arxv:50.0677v cs.lg Feb 05 Abstract Stochastc Dual Coordate Ascet s a popular method for solvg regularzed loss mmzato for the case of covex losses. I ths paper we

More information

Chapter 5 Properties of a Random Sample

Chapter 5 Properties of a Random Sample Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample

More information

Bayes (Naïve or not) Classifiers: Generative Approach

Bayes (Naïve or not) Classifiers: Generative Approach Logstc regresso Bayes (Naïve or ot) Classfers: Geeratve Approach What do we mea by Geeratve approach: Lear p(y), p(x y) ad the apply bayes rule to compute p(y x) for makg predctos Ths s essetally makg

More information

Bayesian Inferences for Two Parameter Weibull Distribution Kipkoech W. Cheruiyot 1, Abel Ouko 2, Emily Kirimi 3

Bayesian Inferences for Two Parameter Weibull Distribution Kipkoech W. Cheruiyot 1, Abel Ouko 2, Emily Kirimi 3 IOSR Joural of Mathematcs IOSR-JM e-issn: 78-578, p-issn: 9-765X. Volume, Issue Ver. II Ja - Feb. 05, PP 4- www.osrjourals.org Bayesa Ifereces for Two Parameter Webull Dstrbuto Kpkoech W. Cheruyot, Abel

More information

A Combination of Adaptive and Line Intercept Sampling Applicable in Agricultural and Environmental Studies

A Combination of Adaptive and Line Intercept Sampling Applicable in Agricultural and Environmental Studies ISSN 1684-8403 Joural of Statstcs Volume 15, 008, pp. 44-53 Abstract A Combato of Adaptve ad Le Itercept Samplg Applcable Agrcultural ad Evrometal Studes Azmer Kha 1 A adaptve procedure s descrbed for

More information

Summary of the lecture in Biostatistics

Summary of the lecture in Biostatistics Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the

More information

A New Family of Transformations for Lifetime Data

A New Family of Transformations for Lifetime Data Proceedgs of the World Cogress o Egeerg 4 Vol I, WCE 4, July - 4, 4, Lodo, U.K. A New Famly of Trasformatos for Lfetme Data Lakhaa Watthaacheewakul Abstract A famly of trasformatos s the oe of several

More information

ENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections

ENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections ENGI 441 Jot Probablty Dstrbutos Page 7-01 Jot Probablty Dstrbutos [Navd sectos.5 ad.6; Devore sectos 5.1-5.] The jot probablty mass fucto of two dscrete radom quattes, s, P ad p x y x y The margal probablty

More information

Point Estimation: definition of estimators

Point Estimation: definition of estimators Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.

More information

Module 7. Lecture 7: Statistical parameter estimation

Module 7. Lecture 7: Statistical parameter estimation Lecture 7: Statstcal parameter estmato Parameter Estmato Methods of Parameter Estmato 1) Method of Matchg Pots ) Method of Momets 3) Mamum Lkelhood method Populato Parameter Sample Parameter Ubased estmato

More information

C-1: Aerodynamics of Airfoils 1 C-2: Aerodynamics of Airfoils 2 C-3: Panel Methods C-4: Thin Airfoil Theory

C-1: Aerodynamics of Airfoils 1 C-2: Aerodynamics of Airfoils 2 C-3: Panel Methods C-4: Thin Airfoil Theory ROAD MAP... AE301 Aerodyamcs I UNIT C: 2-D Arfols C-1: Aerodyamcs of Arfols 1 C-2: Aerodyamcs of Arfols 2 C-3: Pael Methods C-4: Th Arfol Theory AE301 Aerodyamcs I Ut C-3: Lst of Subects Problem Solutos?

More information

Maximum Likelihood Estimation

Maximum Likelihood Estimation Marquette Uverst Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 08 b Marquette Uverst Maxmum Lkelhood Estmato We have bee sag that ~

More information

Some queue models with different service rates. Július REBO, Žilinská univerzita, DP Prievidza

Some queue models with different service rates. Július REBO, Žilinská univerzita, DP Prievidza Some queue models wth dfferet servce rates Júlus REBO, Žlsá uverzta, DP Prevdza Itroducto: They are well ow models the queue theory Kedall s classfcato deoted as M/M//N wth equal rates of servce each servce

More information

Analyzing Fuzzy System Reliability Using Vague Set Theory

Analyzing Fuzzy System Reliability Using Vague Set Theory Iteratoal Joural of Appled Scece ad Egeerg 2003., : 82-88 Aalyzg Fuzzy System Relablty sg Vague Set Theory Shy-Mg Che Departmet of Computer Scece ad Iformato Egeerg, Natoal Tawa versty of Scece ad Techology,

More information

Periodic Table of Elements. EE105 - Spring 2007 Microelectronic Devices and Circuits. The Diamond Structure. Electronic Properties of Silicon

Periodic Table of Elements. EE105 - Spring 2007 Microelectronic Devices and Circuits. The Diamond Structure. Electronic Properties of Silicon EE105 - Srg 007 Mcroelectroc Devces ad Crcuts Perodc Table of Elemets Lecture Semcoductor Bascs Electroc Proertes of Slco Slco s Grou IV (atomc umber 14) Atom electroc structure: 1s s 6 3s 3 Crystal electroc

More information

Principal Components. Analysis. Basic Intuition. A Method of Self Organized Learning

Principal Components. Analysis. Basic Intuition. A Method of Self Organized Learning Prcpal Compoets Aalss A Method of Self Orgazed Learg Prcpal Compoets Aalss Stadard techque for data reducto statstcal patter matchg ad sgal processg Usupervsed learg: lear from examples wthout a teacher

More information

STK4011 and STK9011 Autumn 2016

STK4011 and STK9011 Autumn 2016 STK4 ad STK9 Autum 6 Pot estmato Covers (most of the followg materal from chapter 7: Secto 7.: pages 3-3 Secto 7..: pages 3-33 Secto 7..: pages 35-3 Secto 7..3: pages 34-35 Secto 7.3.: pages 33-33 Secto

More information

Mean is only appropriate for interval or ratio scales, not ordinal or nominal.

Mean is only appropriate for interval or ratio scales, not ordinal or nominal. Mea Same as ordary average Sum all the data values ad dvde by the sample sze. x = ( x + x +... + x Usg summato otato, we wrte ths as x = x = x = = ) x Mea s oly approprate for terval or rato scales, ot

More information

BERNSTEIN COLLOCATION METHOD FOR SOLVING NONLINEAR DIFFERENTIAL EQUATIONS. Aysegul Akyuz Dascioglu and Nese Isler

BERNSTEIN COLLOCATION METHOD FOR SOLVING NONLINEAR DIFFERENTIAL EQUATIONS. Aysegul Akyuz Dascioglu and Nese Isler Mathematcal ad Computatoal Applcatos, Vol. 8, No. 3, pp. 293-300, 203 BERNSTEIN COLLOCATION METHOD FOR SOLVING NONLINEAR DIFFERENTIAL EQUATIONS Aysegul Ayuz Dascoglu ad Nese Isler Departmet of Mathematcs,

More information

Lecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model

Lecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The

More information

MEASURES OF DISPERSION

MEASURES OF DISPERSION MEASURES OF DISPERSION Measure of Cetral Tedecy: Measures of Cetral Tedecy ad Dsperso ) Mathematcal Average: a) Arthmetc mea (A.M.) b) Geometrc mea (G.M.) c) Harmoc mea (H.M.) ) Averages of Posto: a) Meda

More information

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Exam: ECON430 Statstcs Date of exam: Frday, December 8, 07 Grades are gve: Jauary 4, 08 Tme for exam: 0900 am 00 oo The problem set covers 5 pages Resources allowed:

More information

Kernel-based Methods and Support Vector Machines

Kernel-based Methods and Support Vector Machines Kerel-based Methods ad Support Vector Maches Larr Holder CptS 570 Mache Learg School of Electrcal Egeerg ad Computer Scece Washgto State Uverst Refereces Muller et al. A Itroducto to Kerel-Based Learg

More information

Soft Computing Similarity measures between interval neutrosophic sets and their multicriteria decisionmaking

Soft Computing Similarity measures between interval neutrosophic sets and their multicriteria decisionmaking Soft omutg Smlarty measures betwee terval eutrosohc sets ad ther multcrtera decsomakg method --Mauscrt Draft-- Mauscrt Number: ull tle: rtcle ye: Keywords: bstract: SOO-D--00309 Smlarty measures betwee

More information

ECE 595, Section 10 Numerical Simulations Lecture 19: FEM for Electronic Transport. Prof. Peter Bermel February 22, 2013

ECE 595, Section 10 Numerical Simulations Lecture 19: FEM for Electronic Transport. Prof. Peter Bermel February 22, 2013 ECE 595, Secto 0 Numercal Smulatos Lecture 9: FEM for Electroc Trasport Prof. Peter Bermel February, 03 Outle Recap from Wedesday Physcs-based devce modelg Electroc trasport theory FEM electroc trasport

More information

ε. Therefore, the estimate

ε. Therefore, the estimate Suggested Aswers, Problem Set 3 ECON 333 Da Hugerma. Ths s ot a very good dea. We kow from the secod FOC problem b) that ( ) SSE / = y x x = ( ) Whch ca be reduced to read y x x = ε x = ( ) The OLS model

More information

Unsupervised Learning and Other Neural Networks

Unsupervised Learning and Other Neural Networks CSE 53 Soft Computg NOT PART OF THE FINAL Usupervsed Learg ad Other Neural Networs Itroducto Mture Destes ad Idetfablty ML Estmates Applcato to Normal Mtures Other Neural Networs Itroducto Prevously, all

More information

BAYESIAN INFERENCES FOR TWO PARAMETER WEIBULL DISTRIBUTION

BAYESIAN INFERENCES FOR TWO PARAMETER WEIBULL DISTRIBUTION Iteratoal Joural of Mathematcs ad Statstcs Studes Vol.4, No.3, pp.5-39, Jue 06 Publshed by Europea Cetre for Research Trag ad Developmet UK (www.eajourals.org BAYESIAN INFERENCES FOR TWO PARAMETER WEIBULL

More information

( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model

( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch

More information

Tokyo Institute of Technology Tokyo Institute of Technology

Tokyo Institute of Technology Tokyo Institute of Technology Outle ult-aget Search usg oroo Partto ad oroo D eermet Revew Itroducto Decreasg desty fucto Stablty Cocluso Fujta Lab, Det. of Cotrol ad System Egeerg, FL07--: July 09,007 Davd Ask ork rogress:. Smulato

More information

Diagnosing Problems of Distribution-Free Multivariate Control Chart

Diagnosing Problems of Distribution-Free Multivariate Control Chart Advaced Materals Research Ole: 4-6-5 ISSN: 66-8985, Vols. 97-973, 6-66 do:.48/www.scetfc.et/amr.97-973.6 4 ras ech Publcatos, Swtzerlad Dagosg Problems of Dstrbuto-Free Multvarate Cotrol Chart Wel Sh,

More information

Generative classification models

Generative classification models CS 75 Mache Learg Lecture Geeratve classfcato models Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Data: D { d, d,.., d} d, Classfcato represets a dscrete class value Goal: lear f : X Y Bar classfcato

More information

Unimodality Tests for Global Optimization of Single Variable Functions Using Statistical Methods

Unimodality Tests for Global Optimization of Single Variable Functions Using Statistical Methods Malaysa Umodalty Joural Tests of Mathematcal for Global Optmzato Sceces (): of 05 Sgle - 5 Varable (007) Fuctos Usg Statstcal Methods Umodalty Tests for Global Optmzato of Sgle Varable Fuctos Usg Statstcal

More information

Smoothing of Ultrasound Images with the p-lag FIR Structures

Smoothing of Ultrasound Images with the p-lag FIR Structures Recet Researches Telecommucatos, Iformatcs, Electrocs ad Sgal Processg Smoothg of Ultrasoud Images wth the -lag FIR Structures L. J. Morales-Medoza, Y. S. Shmaly, R. F. Vázquez-Bautsta ad O. G. Ibarra-Mazao

More information

Median as a Weighted Arithmetic Mean of All Sample Observations

Median as a Weighted Arithmetic Mean of All Sample Observations Meda as a Weghted Arthmetc Mea of All Sample Observatos SK Mshra Dept. of Ecoomcs NEHU, Shllog (Ida). Itroducto: Iumerably may textbooks Statstcs explctly meto that oe of the weakesses (or propertes) of

More information

X X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then

X X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers

More information

Research Article A New Derivation and Recursive Algorithm Based on Wronskian Matrix for Vandermonde Inverse Matrix

Research Article A New Derivation and Recursive Algorithm Based on Wronskian Matrix for Vandermonde Inverse Matrix Mathematcal Problems Egeerg Volume 05 Artcle ID 94757 7 pages http://ddoorg/055/05/94757 Research Artcle A New Dervato ad Recursve Algorthm Based o Wroska Matr for Vadermode Iverse Matr Qu Zhou Xja Zhag

More information

A tighter lower bound on the circuit size of the hardest Boolean functions

A tighter lower bound on the circuit size of the hardest Boolean functions Electroc Colloquum o Computatoal Complexty, Report No. 86 2011) A tghter lower boud o the crcut sze of the hardest Boolea fuctos Masak Yamamoto Abstract I [IPL2005], Fradse ad Mlterse mproved bouds o the

More information

Bounds on the expected entropy and KL-divergence of sampled multinomial distributions. Brandon C. Roy

Bounds on the expected entropy and KL-divergence of sampled multinomial distributions. Brandon C. Roy Bouds o the expected etropy ad KL-dvergece of sampled multomal dstrbutos Brado C. Roy bcroy@meda.mt.edu Orgal: May 18, 2011 Revsed: Jue 6, 2011 Abstract Iformato theoretc quattes calculated from a sampled

More information

Part 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971))

Part 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971)) art 4b Asymptotc Results for MRR usg RESS Recall that the RESS statstc s a specal type of cross valdato procedure (see Alle (97)) partcular to the regresso problem ad volves fdg Y $,, the estmate at the

More information

7.0 Equality Contraints: Lagrange Multipliers

7.0 Equality Contraints: Lagrange Multipliers Systes Optzato 7.0 Equalty Cotrats: Lagrage Multplers Cosder the zato of a o-lear fucto subject to equalty costrats: g f() R ( ) 0 ( ) (7.) where the g ( ) are possbly also olear fuctos, ad < otherwse

More information

13. Artificial Neural Networks for Function Approximation

13. Artificial Neural Networks for Function Approximation Lecture 7 3. Artfcal eural etworks for Fucto Approxmato Motvato. A typcal cotrol desg process starts wth modelg, whch s bascally the process of costructg a mathematcal descrpto (such as a set of ODE-s)

More information

Application of Calibration Approach for Regression Coefficient Estimation under Two-stage Sampling Design

Application of Calibration Approach for Regression Coefficient Estimation under Two-stage Sampling Design Authors: Pradp Basak, Kaustav Adtya, Hukum Chadra ad U.C. Sud Applcato of Calbrato Approach for Regresso Coeffcet Estmato uder Two-stage Samplg Desg Pradp Basak, Kaustav Adtya, Hukum Chadra ad U.C. Sud

More information

Variable Step LMS Algorithm Using the Accumulated Instantaneous Error Concept

Variable Step LMS Algorithm Using the Accumulated Instantaneous Error Concept Proceedgs of the World Cogress o Egeerg 2008 Vol I WCE 2008, July 2-4, 2008, Lodo, U.K. Varable Step LMS Algorthm Usg the Accumulated Istataeous Error Cocept Khaled F. Abusalem, Studet Member, IEEE. Yu

More information

Multivariate Transformation of Variables and Maximum Likelihood Estimation

Multivariate Transformation of Variables and Maximum Likelihood Estimation Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty

More information

ON BIVARIATE GEOMETRIC DISTRIBUTION. K. Jayakumar, D.A. Mundassery 1. INTRODUCTION

ON BIVARIATE GEOMETRIC DISTRIBUTION. K. Jayakumar, D.A. Mundassery 1. INTRODUCTION STATISTICA, ao LXVII, 4, 007 O BIVARIATE GEOMETRIC DISTRIBUTIO ITRODUCTIO Probablty dstrbutos of radom sums of deedetly ad detcally dstrbuted radom varables are maly aled modelg ractcal roblems that deal

More information

Multiple Regression. More than 2 variables! Grade on Final. Multiple Regression 11/21/2012. Exam 2 Grades. Exam 2 Re-grades

Multiple Regression. More than 2 variables! Grade on Final. Multiple Regression 11/21/2012. Exam 2 Grades. Exam 2 Re-grades STAT 101 Dr. Kar Lock Morga 11/20/12 Exam 2 Grades Multple Regresso SECTIONS 9.2, 10.1, 10.2 Multple explaatory varables (10.1) Parttog varablty R 2, ANOVA (9.2) Codtos resdual plot (10.2) Trasformatos

More information

Rademacher Complexity. Examples

Rademacher Complexity. Examples Algorthmc Foudatos of Learg Lecture 3 Rademacher Complexty. Examples Lecturer: Patrck Rebesch Verso: October 16th 018 3.1 Itroducto I the last lecture we troduced the oto of Rademacher complexty ad showed

More information