(Al. Jerozolimskie 202, Warszawa, Poland,

Transcription

1 th IMEKO TC Wokhop o Techical Diagotic Advaced meauemet tool i techical diagotic fo ytem' eliability ad afety Jue 6-7, 4, Waaw, Polad Semi-paametic etimatio of the chage-poit of mea value of o-gauia adom equece by polyomial maximizatio method Sehii W. Zabolotii, Zygmut Lech Waza Chekay State Uiveity of Techology, Ukaie (zaboloti@uk.et) Reeach Ititute of Automatio ad Meauemet PIAP (Al. Jeozolimkie, -486 Wazawa, Polad, zlw@op.pl) Abtact- A applicatio of the maximizatio techique i the ythei of polyomial adaptive algoithm fo a poteio (etopective) etimatio of the chage-poit of the mea value of adom equece i peeted. Statitical imulatio how a igificat iceae i the accuacy of polyomial etimate, which i achieved by takig ito accout the o-gauia chaacte of tatitical data. I. Itoductio Oe of the impotat tak of the diagoi of adom pocee i the meauemet of the poit at which the popetie of the obeved poce ae ubject to a chage (diode). Thi poit i called "the chage-poit". Statitical method of detectig the chage-poit ca be ued i eal time o a poteioi. The latte oe ae alo called the etopective method. A poteio tatitical etimatio i baed o the aalyi of a fixed volume ample of all ifomatio eceived fom the diagoed object. Thi appoach equie a loge time of eactio, but it povide a moe eliable ad accuate etimatio of the chage-poit []. A poteio etimatio of the time of chage the paamete of tochatic pocee i eeded i may pactical applicatio, uch a the diagoi of ome idutial pocee, the detectio of climate chage [], aalyi of the geetic time eie [], idetificatio of ituio i compute etwok [4], egmetatio of peech igal ad meage of ocial etwok [5]. Fo uch a wide age of tak the developmet of a lage vaiety of mathematical model ad tatitical poceig tool i equied. It hould be oted that mot of the theoetical tudie coected with the etimatio of the chage poit i focued o the cla of adom pocee decibed by the Gauia pobability ditibutio. Howeve, the eal tatitical data ae ofte diffeet fom the Gauia model. The claical method, which ae baed o the pobability deity, ae called the paametic method. The mai poblem i paametic appoache (Bayeia ad maximum-likelihood) ae coected with the equiemet of a a pioi ifomatio about the fom of ditibutio, a well a with the potetially high complexity of thei implemetatio ad the aalyi of the popetie. Thu, a igificat amout of the cotempoay eeach coce the cotuctio of applied tatitical method which would allow to emove o miimize the equied amout of the a pioi ifomatio. Such method ae baed o obut tatitical poceig pocedue that ae ieitive to "o-exacte" of pobabilitic model, o o opaametic citeia, idepedet of pecific type of ditibutio. The pice fo omiio of pobabilitic popetie i hadled tatitical data i the deteioatio of quality chaacteitic i compaio with optimal paametic method [6]. The ue of highe-ode tatitic (decibed by momet o cumulat) i oe of the alteative appoache i olvig poblem elated to poceig of o-gauia igal ad data. Thi mathematical tool ca fid applicatio i vaiou aea, whee the etimatio of the chage-poit i impotat, e.g. i: defiig the momet of aival the acoutic emiio igal [7], the detectio chage of video igal [8], detectio of igal chage i telecommuicatio etwok [9]. I thi pape thee i coideed the applicatio a ew ucovetioal tatitical method, i olvig poblem of a poteio type etimatio of chage poit. The method i called the polyomial maximizatio method (MMP) ad it wa popoed by Kucheko []. It ue tochatic polyomial a the mathematical tool. The method ued i cojuctio with the momet-cumulat deciptio allow to implify ubtatially the poce of ythei of adaptive tatitical algoithm. Moeove ice the ifomatio about pobabilitic popetie of data i ued - a impovemet of the accuacy i obtaied, i.e. eductio of value of etimated vaiace ad deceae of the pobability of eoeou deciio ae achieved. Sice the momet-cumulat deciptio of the pobability ditibutio i oly appoximate, tatitical method. Baed o that deciptio allow to obtai oly aymptotically optimal eult (i.e., it accuacy iceae with the ode of the ued tatitic. Thu, the popoed method ca be claified a emi-paametic. The aim of thi pape ae the followig: - uig the method of polyomial optimizatio to ythei o algoithm of a poteioi etimatio of the of chage-poit of the mea-value of the o-gauia adom equece, - ivetigatio of effectivee of thoe algoithm, uig tatitical modellig. 89

2 th IMEKO TC Wokhop o Techical Diagotic Advaced meauemet tool i techical diagotic fo ytem' eliability ad afety Jue 6-7, 4, Waaw, Polad II. Mathematical fomulatio of the poblem Suppoe thee i a adom ample x = { x, x,... x }. Elemet of thi ample ca be itepeted a a et of idepedet adom vaiable. The pobabilitic atue of thi ample ca be decibed by the mea value θ, vaiace σ ad cumulat coefficiet γ l up to a give ode l =,. Up too ome (a pioi ukow) poit of the dicete time τ, the mea value i equal to θ, ad the, at the time τ + it value jump to θ. Ou pupoe i to etimate the value τ of the chage-poit of the mea value of the equece, o the bai of the aalyi of the ample. Vaiou vaiat of the chage poit etimatio may diffe, depedig o the availability of a a pioy ifomatio about value of a vaiable paamete (befoe ad/o afte the chage), a well a o the kowledge of the pobabilitic chaacte of the othe paamete of the adom equece model. III. A poteioi etimatio of the chage-poit of mea value by maximum likelihood method Oe of the baic diectio i ivetigatio of a poteioi poblem of the chage poit tudy i baed o the idea of the maximizatio of the likelihood. It wa elaboated i detail by Hickley []. He popoed a geeal aymptotic appoach to obtai ditibutio of a pioi chage-poit etimate by method of maximum likelihood (MML). Applicatio of thi appoach equie a a pioi ifomatio about the ditibutio law of tatitical data, befoe ad afte the chage. Fo a Gauia ditibutio it i kow that etimatio of the mea value by MML method i the ame a a liea etimatio by method of momet (MM), i.e. ˆ θ = () x v Etimate of the fom () i coitet ad ot hifted. So o-paametic MM etimato ca be ued fo etimatio of the mea value of adom vaiable of ay abitay ditibutio. Howeve, thi aemet i effective oly fo the Gauia model. Fo thi pobabilitic model the logaithm of the maximum likelihood fuctio (MML) with kow vaiace σ i tafomed [] ito tatitic of the fom: ( ) T (, θ ) = ( xv θ ) + ( ) ( xv θ ) θ. () + T θ,θ ha a maximum i a eighbouhood of the tue value of the chage-poit τ. Thu, the deied chage-poit etimate ca be fidig by algoithm: ( θ, θ ) ˆ τ = ag max T (a) Hickley coideed alo the cae whe paamete θ ad θ of the Gau ditibutio ae ukow. I thi cae, the MML etimatio fo the chage-poit of the mea value take o the fom: whee, x v ˆ τ = ag max ˆ θ =, ˆ, = x v + ( ˆ θ ˆ θ ) + ( )( ˆ θ ˆ θ ),,, () θ. (4) Sice tatitic () ad () do ot deped o ay othe pobabilitic paamete, they ca be ued fo opaametic etimatio of the chage-poit of the aveage of adom equece with a abitay ditibutio. Howeve, i uch ituatio (imilaly, i the cae whee the aveage i evaluatig accodig (), i geeal the opaametic algoithm loe thei optimality. To ovecome thi difficulty, ew oliea etimatio algoithm baed o the of the miimizatio of the polyomial ae decibed below. They allow to take ito accout, i a imple way, the degee of o-gauia chaacte of the tatitical data IV. Two-paamete meauemet of tai ad tempeatue chage Let x be equally ditibuted ampled elemet. Coide the algoithm peeted i [] ad deoted by MMP. It i how i that pape that the etimate of a abitay paamete ϑ ca be foud by olvig the followig tochatic equatio with epect to ϑ: 9

3 th IMEKO TC Wokhop o Techical Diagotic Advaced meauemet tool i techical diagotic fo ytem' eliability ad afety Jue 6-7, 4, Waaw, Polad i hi ( ϑ ) x v αi ( ϑ) =, i= ϑ = ˆ ϑ whee: i the ode of the polyomial fo paamete etimatio, α i ( ϑ) - i the theoetical iitial momet of the i-th ode. Coefficiet h i ( ϑ) (fo i =, ) ca be foud by olvig the ytem of liea algebaic equatio, give by coditio of miimizatio of vaiace (with the appopiate ode ) of the etimate of the paamete ϑ, amely: d h i ( ϑ) Fi, j ( ϑ) = α j ( ϑ), j =,, (5) dϑ i= whee Fi, j ( ϑ) = αi+ j ( ϑ) αi ( ϑ) α j ( ϑ) - ceteed coelat of ( j) i, dimeio. Equatio (5) ca be olved aalytically uig the Kame method, i.e. h i i ( ϑ ) =, i =,, whee = det F i, j ; i, j =, - volume of the tochatic polyomial body of dimeio, i - i the detemiat obtaied fom by eplacig the i-th colum by the colum of fee tem of eq. (5). A ew appoach fo fidig the poteioi etimate of chage-poit, popoed i thi pape, i baed o applicatio of MMP method. I thi appoach thee i ued a popety of the followig tochatic polyomial: i l x ϑ = k ϑ + k x, (6) ϑ whee k ( ϑ) = [ hi ( ϑ) αi ( ϑ) ] dϑ, ki ( ϑ) hi ( ϑ) dϑ a i= The mathematical expectatio { l ( x ϑ) } value poit of thi paamete. E ( ) ( ) i ( ϑ) Value of the paamete ϑ belog to ome iteval ( b) ϑ a v i= =, i =, (7a,b), teated a a fuctio of ϑ aume the maximum at the tue a,. If the tochatic polyomial of the fom (6) will be maximized with ue a paamete ϑ which ha a chage-poit (tep chage fom value ϑ to value ϑ ), the we ca build a polyomial fom tatitic: i xv i= + ( ) i (, ϑ ) = k( ϑ ) + ki ( ϑ ) xv + ( ) k( ϑ ) + ki ( ϑ ) P ϑ, (8) i= which will have a maximum i a eighbohood of the tue valueτ of the chage-poit. Thu the geeal algoithm of applyig MMP method fo fidig the etimatio of the chage-poit τ ca be fomulated a follow ˆ τ = ag max P ϑ, ϑ. (9) ( ) ( ) V. Polyomial etimatio of the chage-poit of mea value It i kow fom [] that the etimate of the aveage θ obtaied by MMPl method uig a polyomial of ode = coicide with the fom () of the liea etimate MM. Hece the ythei of polyomial algoithm fo etimatig the chage-poit of thi paamete i jutified oly fo ode. At a ode = polyomial etimate of the mea value ca be foud by olvig the followig quadatic equatio: γ θ γ xv σ = θ = ˆ θ ( + γ 4 ) θ σ ( + γ 4 ) xv + γ ( xv ) σ. () The aalyi of eq. () how that the etimated value of ˆ θ = deped o coefficiet of kewe γ ad γ kutoi 4. If the value of thee paamete ae equal to zeo, the ditibutio i the omal (Gauia) oe. I thi cae the polyomial etimate () educe to the claical etimate of the fom (). It i how i [] that the ue of eq. () eue highe accuacy (deceae the vaiace) of etimate compaed with the etimate (). The aymptotic value of thi etimate (fo ) i give by the followig fomula: 9

4 th IMEKO TC Wokhop o Techical Diagotic Advaced meauemet tool i techical diagotic fo ytem' eliability ad afety Jue 6-7, 4, Waaw, Polad γ g ( γ, γ 4 ) =. (a) + γ 4 Uig the aalytical expeio (5) ad (7) oe ca eaily fid that, fo ode =, the coefficiet maximizig the elected tochatic polyomial of the fom (6) i a eighbohood of the tue value of the paamete θ ae the followig: σ [ 4 6 ] 6 6 whee = σ ( + γ ). k ( θ ) = γ θ + ( + γ ) σθ γ σ θ, k ( ) = γ θ + ( γ ) σθ, ( ) = γ θ 4 γ [ ] θ 4 σ + σ k θ (a-c) I the peece of a pioi ifomatio about the the mea value of θ befoe ad θ afte of chage-poit ad ude the coditio θ > θ, fo the ode of the polyomial = the tatitic (8) ca be expeed a follow: P ( ) ( θ, θ ) = ( ) γ ( θ θ ) + σ ( + γ )( θ θ ) σ γ ( θ θ ) + [ γ ( θ θ ) + σ ( + γ )( θ θ )] x γ ( θ θ ) x. 4 4 v + v + I the cae, whe a pioi ifomatio about the mea value θ ad θ ae ukow, polyomial evaluatio of chage-poit momet (a i the claical cae) ca be foud by eplacig the ukow value of thee paamete by thei poteio etimate of the fom (4). Thee etimate ae fomed fo each potetial chagepoit. Thu, fo = a adaptive algoithm fo fo etimatig the time of chage-poit τˆ baed at MMPl method ca be fomulated a follow: 4 ˆ ( ) ˆ ( ) ˆ ˆ τ = ag max γ θ, + σ + γ 4 θ, σ + xv γ θ, + () 4 ( ) ˆ ( ) ˆ ( ) ˆ + γ θ, + σ + γ 4 θ, σ + xv γ θ,. + The aalyi of the tuctue of polyomial tatitic () ad () cofim agai the fact that, fo =, the ue MMPl i jutified oly i the cae of a aymmety ( γ ) of the ditibutio of the tatitical data. VI. Statitical modelig of a poteioi etimate of chage-poit Baed o eult of above coideatio, a oftwae package i a oftwae eviomet MATLAB, ha bee developed. It allow to pefom the tatitical modelig of the popoed emi-paametic etimatio pocedue, applied to the etimatio of the mea value ad vaiace of the chage-poit of o-gauia adom equece. Both, igle ad multi- expeimet (i the ee of the Mote Calo method) ca be imulated. The accuacy obtaied by claical ad popoed polyomial algoithm fo expeimetal data ca be alo compaed. I Fig. thee ae peeted the eult fo a umeical example obtaied by etimatio pocedue fo the mea value of the chage-poit of aveage value θ ad θ of the o-gauia equece (Fig. a), = whee σ =, γ = ad γ 4 = 5. The calculatio wee pefomed uig the claical veio () of the algoithm of a poteioi etimatio by MMP method (coicidig with MMPl if = ) a well by polyomial algoithm () of MMP fo =. The eult peeted i Fig.,b clealy cofim the potetially highe peciio obtaied by polyomial tatitic fo =, ice the maximum of the coepodig fuctio i togly maked, a compaed with the moothed fom of the tatitic fo =. Reult of the igle expeimet do ot allow to compae adequately the accuacy of the tatitical etimatio algoithm. A a compaative citeio of efficiecy, the atio of vaiace of the etimate of the chage-poit i ued. That ca be obtaied by a eie of expeimet with the ame iitial value of the model paamete. It hould be oted that theoetically the eult of tatitical algoithm of a poteioi etimatio of the chagepoit ca deped o a vaiou facto, icludig e.g.: the elative value of the mea jump at the chage-poit, the pobabilitic atue (value of coefficiet of highe ode cumulat) of o-gauia adom equece, the peece of a a pioi ifomatio about the value vaiable of paamete. Futhemoe, the accuacy of etimatio of the chage-poit deped o the choe umbe of the ample ad o the accuacy of the vaiace etimate, i.e., o the umbe of expeimet m pefomed ude the ame iitial coditio. = () 9

5 th IMEKO TC Wokhop o Techical Diagotic Advaced meauemet tool i techical diagotic fo ytem' eliability ad afety Jue 6-7, 4, Waaw, Polad а) b) Figue. Example of a poteioi etimatio of the chage-poit of the mea value. The eult of tatitical modelig fo = ad m = ae how i Figue. G i the atio of vaiace of the chage-poit etimate obtaied by MMPl method with the polyomial ode: = ad = (Nomal tatitic) epectively. The value of G chaacteize the elative iceae of accuacy. Figue a how the depedece of G o the elative value of the jump at the chage-poit q = ( θ θ ) σ, obtaied with diffeet coefficiet of kewe γ ad kutoi γ 4. Figue b peet the depedece of G o γ (if γ 4 = ad q =. 5 ), obtaied ude diffeet a pioi ifomatio about the mea value of adom equece befoe ad afte the chage. a) b) Figue - The expeimetal value of G coefficiet, which how the eductio of vaiace of etimate of the chage-poit of mea value, obtaied whe MMPl method i ued. 9

6 th IMEKO TC Wokhop o Techical Diagotic Advaced meauemet tool i techical diagotic fo ytem' eliability ad afety Jue 6-7, 4, Waaw, Polad Aalyi of thee ad may othe expeimetal eult cofim the theoetical eult coceig the effectivee of the polyomial method i the chage-poit etimatio. It tu out that the elative gowth of accuacy i oughly the ame fo diffeet fomulatio of the poblem, elated to the peece o abece of a a pioi ifomatio about value of the vaiable paamete. The impovemet doe ot igificatly deped o the elative magitude of the jump at the chage-poit. It i detemied pimaily by the ode of the "o-gauia" of the poce, which umeically i expeed a abolute value of the of highe-ode cumulat coefficiet. VII. Cocluio Reult of the eeach lead to the geeal cocluio about the potetially high efficiecy of the implemetatio of the polyomial maximizatio method to the ythei of imple adaptive algoithm fo etimatig of chage-poit of paamete of tochatic pocee of o-gauia chaacte of tatitical data. Obtaied theoetical eult allowed to develop a fudametally ew appoach to the cotuctio of emipaametic algoithm fo a poteio etimatio of the chage-poit. Thi appoach i baed o the applicatio of tochatic polyomial. Amog may poible diectio of futhe eeach oe hould metio the followig: iceae of the degee of the tochatic polyomial, which i eceay to get moe effective olutio, epecially fo o-gauia equece with ymmetical ditibutio; aalyi of the depedece of the accuacy of detemiig the paamete of o-gauia model (highe-ode tatitic) o the tability of polyomial algoithm fo a poteioi etimatio of the chage-poit; ythei of polyomial algoithm fo etimatig the chage-poit with epect to othe paamete (e.g., dipeio o coelatio ad egeio coefficiet), o i cae whee the value of eveal paamete ae chaged imultaeouly (e.g., the mea value ad the vaiace, etc.). Refeece [] Che J., Gupta A. K. (). Paametic tatitical chage poit aalyi. Bikhaeue, p.7 [] Reeve J., Che J., Wag X. L., Lud R., ad Lu Q. (7). A eview ad compaio of chage-poit detectio techique fo climate data. Joual of Applied Meteoology ad Climatology, 46 (6), 9-95 [] Wag Y., Wu C., Ji Z., Wag B., ad Liag Y. (). No-paametic chage-poit method fo diffeetial gee expeio detectio. PLoS ONE, 6 (5), e.6. [4] Yamaihi K., Takeuchi J., William G., ad Mile P. (). O-lie uupevied outlie detectio uig fiite mixtue with dicoutig leaig algoithm. Poceedig of the Sixth ACM SIGKDD Iteatioal Cofeece o Kowledge Dicovey ad Data Miig, pp.-4,. [5] Liu S., Yamada M., Collie N., & Sugiyama M. (). Chage-poit detectio i time-eie data by elative deity-atio etimatio. Neual Netwok, vol.4, p.7-8. [6] Bodky B. ad Dakhovky B. (99) Nopaametic Method i Chage-Poit Poblem. Kluwe Academic Publihe, Dodecht, the Nethelad. [7] Lokajicek T, Klima K. A. (), Fit aival idetificatio ytem of acoutic emiio (ae) igal by mea of a highe-ode tatitic appoach. Meauemet Sciece ad Techology. Vol. 7, [8] Yih-Ru Wag. (8). The igal chage-poit detectio uig the high-ode tatitic of log-likelihood diffeece fuctio. Iteatioal Acoutic, Speech ad Sigal Poceig,. ICASSP 8. IEEE Iteatioal Cofeece, [9] Cotatio S. Hila, Ioai T. Rekao, Pai At. Matoocota. (). Chage poit detectio i time eie uig highe-ode tatitic: a heuitic appoach, Mathematical Poblem i Egieeig, vol., Aticle ID 76, page. [] Kucheko Y. (), Polyomial Paamete Etimatio of Cloe to Gauia Radom vaiable. Shake Velag, Aache Gemay. [] Hikley D. (97). Ifeece about the chage-poit i a equece of adom vaiable. Biometika. vol. 57. No.. p