WSEAS RASACS o MAEMAICS Guochao Zhag Qigmig Gui A Bayesia Approach for Deecig Ouliers i ARMA ime Series GUOC ZAG Isiue of Sciece Iformaio Egieerig Uiversiy 45 Zhegzhou CIA 94587@qqcom QIGMIG GUI Isiue of Sciece Iformaio Egieerig Uiversiy 45 Zhegzhou CIA guiqigmig@6com Absrac: he presece of ouliers i ime series ca seriously affec he model specificaio ad parameer esimaio o avoid hese adverse effecs i is esseial o deec hese ouliers ad remove hem from ime series By he Bayesia saisical heory his aricle proposes a mehod for simulaeously deecig he addiive oulier ad iovaive oulier i a auoregressive movig-average ARMA ime series Firsly a approximae calculaio mehod of he oi probabiliy desiy fucio of he ARMA ime series is give he cosiderig he siuaio ha ad may prese a he same ime i a ARMA ime series a model for deecig ouliers wih he classificaio variables is cosruced By his model his aricle rasforms he problem of deecig ouliers io a muliple hypohesis esig hirdly he poserior probabiliies of he muliple hypoheses are calculaed wih a Gibbs samplig ad based o he priciple of Bayesia saisical iferece he locaios ad ypes of ouliers ca be obaied Wha s more he abormal magiude of every oulier also ca be calculaed by he Gibbs samples A las he ew mehod is esed by some experimes ad compared wih oher mehods exisig I has bee proved ha he ew approach ca simulaeously deec he ad successfully ad performs beer i erms of deecig he oulier which is boh ad ad bu cao be recogized by oher mehods exisig Key-words: ARMA model; Addiive oulier ; Iovaive oulier ; Classificaio variable; Bayesia hypohesis es; Gibbs samplig Iroducio ime series aalysis is a very impora saisical mehod of dyamic daa processig i sciece ad egieerig [-3] A ime series ofe coais all kids of ouliers such as addiive oulier iovaive oulier emporary chage C level shif LS ec [4 5] As [6] poied he presece of hese ouliers could easily mislead he coveioal ime series aalysis procedure resulig i erroeous coclusios So i is impora o have procedures ha deec ad remove such ouliers effecs [7] E-ISS: 4-88 3 Volume 6 7
WSEAS RASACS o MAEMAICS Guochao Zhag Qigmig Gui he bayesia mehod for deecig ouliers i a ime series had bee cosidered by [8] i he earlies ime [9] used he Gibbs sampler i he Bayesia aalysis of auoregressive AR ime series ad solved he problem of deecig he i he AR model by he Gibbs samplig [] illusraed he reaso of maskig ad swampig ad proposed a soluio o he problem based o he sadard Gibbs samplig [] developed a procedure for deecig he s i he ARMA model by model selecio sraegies ad Bayesia iformaio crierio BIC owever here are some disadvaages i he exisig Bayesia approaches for deecig ouliers i a ime series a As we all kow he ARMA model is widely applied i pracice ha he AR model ad ye he mos of exisig mehods aim a deecio of ouliers i he AR model oly a few procedures focus o he ARMA model b he oi probabiliy desiy fucio of he ARMA ime series is esseial o Bayesia iferece owever i is complex o be o calculaed accuraely whe he umber of observaios is large because of he correlaio amog he observaios of he ARMA ime series So here is o way o Bayesia iferece o he ARMA model c I is commo ha all kids of ouliers i he ARMA ime series may appeared a he same ime Bu he exisig Bayesia mehods cao deec hem simulaeously herefore his aricle proposes a mehod for deecig all kids of ouliers simulaeously especially for deecig ad he mos commo ouliers i he ARMA ime series by he Bayesia saisical heory he res of he paper is orgaized as follows I secio a model of deecig he ad i he ARMA ime series simulaeously is cosruced based o he classificaio variables of ouliers ad a rule of deecig ouliers is proposed by applyig he priciple of Bayesia hypohesis esig Secio 3 develops a mehod of esimaig he oi probabiliy desiy fucio of he observaios of he ARMA ime series ad he codiioal poserior disribuios of ukow parameers are deduced I secio 4 a procedure of deecig all kids of ouliers simulaeously based o he Gibbs samplig is proposed Secio 5 shows he beer performaces of he approach proposed i his aricle comparig wih oher exisig mehods by some simulaig experimes Fially some coclusios are give i secio 6 he model ad rule for oulier deecio wih he classificaio variables Assume ha { z } be a ime series followig a geeral ARMA pq process where B z B i i d p B I B B pb B I p q B is a backshif q B B qb k operaor such ha B z z k { } is a k sequece of idepede radom errors ideically disribued o esure he ARMA pq model beig saioary ad iverible assume ha all of he zeros of B ad B are o or ouside he uie circle [-3] O he basis of he defiiios of ad [4-7] he observaio y ha is affeced by a or a or by boh of hem simulaeously ca be wrie wih he classificaio variables [83] as follows: y z B B E-ISS: 4-88 4 Volume 6 7
WSEAS RASACS o MAEMAICS Guochao Zhag Qigmig Gui From above a model of deecig he ad i he ARMA ime series { y } cosruced as follows: simulaeously is y x x x x a a i i d p p q q Where x is he observaio which may affeced by is a classificaio variable of ad follows a Beroulli disribuio If he observaio magiude is is o a y is a ha he abormal ; if he observaio y is a classificaio variable of ad i is also follows a Beroulli disribuio If is acceped y is a bu o a ha 3 is acceped meas ha y is boh ad Wha s more if 4 is acceped y is a bu o a Based o he priciple of Bayesia hypohesis esig[4] we eed o choose a appropriae prior disribuio for every ukow parameer ad calculae he poserior probabiliy of every hypohesis as follows: Y Y Y Y 3 4 where Y y p y p y If i Y max { Y Y Y Y } he 3 4 i will be acceped which meas ha y ca be ideified o be a ormal observaio or some kid of oulier he observaio y is a ha he abormal magiude is ; if he observaio y is o a Suppose ha here are observaios of he ime series{ y } say y y y ad he fro p observaios y y y p p are o ouliers[8] If we wa o udge wheher y p is a or or o i is ecessary o es a muliple hypohesis: : : : : 3 4 ere if he hypohesis coclude ha 3 is acceped we ca y is eiher a or a If 3 Esimaig he oi probabiliy desiy fucio of observaios ad calculaig he codiioal poserior disribuio of ukow parameers 3 Esimaig he oi probabiliy desiy fucio of observaios Whe he parameers p q ad i he ARMA pq model are kow assume ha he observaios z z z are go ad he mea of every observaio is zero so Z z z z follows E-ISS: 4-88 5 Volume 6 7
WSEAS RASACS o MAEMAICS Guochao Zhag Qigmig Gui a -variae ormal disribuio ad ca be expressed as Z by he defiiio of he ARMA model Due o he correlaio amog he observaios of he ARMA ime series he covariace marix of Z is a -order marix relaed o he parameers p q ad From above i is difficul o calculae accuraely he oi probabiliy desiy fucio of Z whe he umber is very large Ad furher i is more difficul o calculae accuraely he oi probabiliy desiy fucio of Y which may iclude all kids of ouliers eve hough he firs p observaios y y y p are ormal o calculae he codiioal poserior disribuios of ukow parameers which will be used i he followig Gibbs samplig a approach for approximaely esimaig he oi probabiliy desiy fucio of Y is cosidered as follows: Whe Y y y y p q p p p probabiliy desiy fucio of y py exp{ y X } p A las he oi probabiliy desiy fucio of Y is calculaed by p Y x p p py p p exp{ y p 4 } X 3 Calculaig he codiioal poserior disribuios of ukow parameers Firsly based o he selecio priciples of prior disribuios [5] suppose ha here is a small prior probabiliy ha every observaio y p is a or a which is p ad are kow based o he model we ca esimae he by he equaios p meas ha p p [8] So he prior disribuios of all ukow parameers are seleced as follows: ha is ~ p V p V p exp y X where X x x x y m p m m m m p ad q By he model { V } ha is ~ q W { W } q W p exp y X ~ u ~ u p so he ~ b ha is p E-ISS: 4-88 6 Volume 6 7
WSEAS RASACS o MAEMAICS Guochao Zhag Qigmig Gui ~ b ha is p v v ~ IG where V W u u ad are he hyper parameers ha are v kow he codiioal poserior disribuios of ukow parameers are acquired by he Bayesia formulas [6] as follows where ad he oi poserior probabiliy desiy fucio p Y x p of Y are esimaed as described before he codiioal poserior disribuio of is Y ~ ˆ Vˆ 5 Y exp{ y X 8 } Y exp{ y X } 9 Y 3 y X - exp{ } where Vˆ [ X X V ] ˆ ˆ p V [ ] X y V p he codiioal poserior disribuio of is Y ~ IG a b 6 p v where a b [ y p X ] v 3 he codiioal poserior disribuio of is Y ~ ˆ Wˆ 7 where ˆ W ˆ Wˆ W p y X W p 4 he codiioal poserior disribuio of ca be calculaed as follows: Y 4 exp{ y X } where mi p is 5 he codiioal poserior disribuio of Y ~ ˆ ˆ where ˆ [ ] ˆ ˆ [ y X m ym m m m m xm m u y x ] m p m p m E-ISS: 4-88 7 Volume 6 7
WSEAS RASACS o MAEMAICS Guochao Zhag Qigmig Gui 6 he codiioal poserior disribuio of is Y ~ ˆ ˆ 3 where ˆ [ ] ˆ ˆ [ y u X ] samplig Sep 3: Impleme he Gibbs samplig as follows ad ge he Gibbs samples Suppose ha he k--h k sample k k k k k k has bee acquired he he k-h sample ca be obaied by he followig procedure: k a Esimae he by he equaios a y X p k k k p k 4 he implemeaio for oulier deecio k b Obai from k k Y k k k k k Whe he orders p ad q of he ARMA model are kow bu he parameers k c Obai from k k Y p q ad are k k k k k ukow he poserior probabiliy i Y k d Obai from k k Y cao be calculaed direcly by he Bayesia formula hus we ca use he Gibbs samplig based o he codiioal poserior disribuios of ukow parameers o esimae i Y Above all a procedure for deecig he ad i a ARMA ime series simulaeously is proposed based o he Gibbs samplig [5 7] as follows: Sep : Choose he hyper parameers V W u u v ad for he prior disribuios of all ukow parameers i he ARMA model k k k k k e Calculae k k k ad k ad obai k k 3 4 k k from Y k k k k k k k k k k k k where p ad k k k k k k k k p Sep : Choose he iiial values f Obai k from Y k k ad for he Gibbs k k k k k k E-ISS: 4-88 8 Volume 6 7
WSEAS RASACS o MAEMAICS Guochao Zhag Qigmig Gui g k k k k k where k k Obai p k from k k k Y k k k k k k where k i k k k k 3 4 i 34 is he codiioal poserior probabiliy of i i 34 a he k-h Gibbs samplig Y max{ Y Y Y If i 3 where k k k k p 4 Y } he hypohesis i is acceped ad y k k Impleme ad ed he ieraive procedure afer he Gibbs samplig is coverge Sep 4: Make he Bayesia iferece Supposig ha samples are acquired ad he Gibbs samplig is coverge afer acquirig he M-h sample we use he las -M samples o make he followig Bayesia iferece a Ge he locaios ad ypes of ouliers By he above Gibbs samplig he poserior probabiliies of four hypoheses ca be calculaed approximaely by Y Y 4 M k k k k k km 3 4 Y Y 5 M k k k k k km 3 4 Y Y 3 6 M k 3 k k k k km 3 4 Y Y 4 7 M k 4 k k k k km 3 4 ca be ideified o be a ormal observaio or some kid of oulier b Esimae he abormal magiudes of ouliers If y p is recogized as a is abormal magiude may esimaed by k km 8 ˆ M If y p is recogized as a is abormal magiude may esimaed by k km 9 ˆ M 5 Examples ad aalysis I order o illusrae he performace of he approach for deecig ouliers i he ARMA model proposed by his aricle hree examples are desiged based o observaios from a ARMA model of orders x 5x 3x 5 i i d Example : Add a of he abormal magiude equals o 5 a =3; add a of he abormal magiude equals o a =8 Calculae he poserior probabiliies of four hypoheses ad hey are show i Fig E-ISS: 4-88 9 Volume 6 7
WSEAS RASACS o MAEMAICS Guochao Zhag Qigmig Gui ime series ad he abormal magiudes of hese ouliers ca be esimaed accuraely owever he erroeous ideificaio for he locaios ad ypes of ouliers is preseed i he deecio resuls of he ierae procedure proposed by [8] Example : Add a of he abormal magiude equals o 6 a =3 8 respecively he poserior probabiliies of four hypoheses are show i Fig he Fig shows ha 43 Y max { 3 Y 3 Y 33 Y 43 Y } y max{ Y Y Y 8 8 8 38 48 Y } ad Y max{ Y Y Y Y} 38 3 4 Wha s more he abormal magiudes of ouliers ca be calculaed by formula 8 ad 9 A las he resuls of deecig ouliers are ha he 3 h observaio is a ad he 8 h observaio is a ad he esimaes of heir abormal magiudes are ˆ 3 46494 ad ˆ 8 89637 respecively We also use he ieraive procedure proposed by [8] o oulier deecio ad he resuls are ha boh 3 h ad 8 h observaios are s ad heir abormal magiudes are ˆ 3 375 ad ˆ 866 respecively he 35 h observaio is 8 a ad is abormal magiude is ˆ 35 3744 Compared wo resuls above we ca see clearly ha he approach proposed i his aricle ca locae ad recogize precisely he ouliers i a ARMA he Fig shows ha 3 Y max{ 3 Y 3 Y 33 Y 43 Y } 8 Y max{ 8 Y 8 Y 38 Y 48 Y} Y max{ Y Y Y ad 3 Y } 38 he abormal magiudes 4 of ouliers also ca be calculaed by formula 8 A las he resuls of deecig ouliers are ha boh 3h ad 8h observaios are s ad he esimaes of heir abormal magiudes are ˆ 5475 3 ad 8 ˆ 877 respecively Sice he mehod proposed by [] oly ca deec he i he ARMA ime series we use i o deec he ouliers ad ge he resuls ha he 3 h E-ISS: 4-88 Volume 6 7
WSEAS RASACS o MAEMAICS Guochao Zhag Qigmig Gui 53 h ad 8 h all are s ad heir abormal magiudes are ˆ 3 6 ˆ 53 36966 ad ˆ 598 respecively 8 Compared he resuls above i is obvious ha he mehod proposed i his aricle ca deec precisely he muliple ouliers i a ARMA ime series Bu he mehod proposed by [] cao locae he ouliers accuraely ad may cause misudgme of Wha s more he abormal magiudes of s sill iclude a seriously bias Example 3: Add a of he abormal magiude equals o ad a of he abormal magiude equals o 8 a =5 simulaeously; add a of he abormal magiude equals o a =8he poserior probabiliies of four hypoheses are show i Fig3 Similarly he 5 h observaio is recogized as boh a of he abormal magiude equals o 974ad a of he abormal magiude equals o 7354 ad he 8 h observaio is recogized as a of he abormal magiude equals o 9487 From he hree examples above we ca see ha he Bayesia approach proposed i his aricle ca ge he locaios ypes abormal magiudes of ad i a ARMA ime series precisely Especially he approach has a good performace of deecig he observaio which is boh ad I addiio by he compariso of example ad he pheomeo ca be foud ha he Bayesia approach proposed i his aricle ca more accurae deec he ad i a ARMA ime series ha he previous mehods i he lieraures 6 Coclusios I view of some difficulies exised i he deecio of ouliers i he ARMA ime series his paper suggess may soluios Firsly if he umber of he observaios from he ARMA ime series is very large calculaig he oi probabiliy desiy fucio of he observaios is very difficul o solve his problem a mehod of esimaig he fucio is cosidered Ad his mehod also be embed io he Gibbs samplig laer i order o realize he oulier deecio for he ARMA ime series Secodly i order o simulaeously deec he ad i a ARMA ime series his paper cosrucs a model which ca be used o reflec he observaio affeced by ad a he same ime ad coclude he problem of simulaeously deecig ad o a muliple hypohesis esig exly he Gibbs samplig is suggesed o calculae he poserior probabiliy of every hypohesis ad he he muliple hypohesis is esed based o he priciple of Bayesia hypohesis esig so ha he kids of ouliers ca be deeced I addiio his aricle shows a compleely procedure from Bayesia approach for deecig ouliers i a ARMA ime series afer solvig he problems meioed above Fially i order o show he effec of approach proposed i his aricle hree simulaio experimes are desiged he resuls of his procedure are compared wih oher exisig mehods which shows ha he approach for simulaeously deecig ad i a ARMA ime series ca ge he locaios wih ypes of ouliers accuraely E-ISS: 4-88 Volume 6 7
WSEAS RASACS o MAEMAICS Guochao Zhag Qigmig Gui ad has a beer deecio resuls ha oher mehods Especially he observaio ha is boh ad ca be recogized easily by his procedure proposed i his aricle Ackowledgemes: his research was suppored by aioal Sciece Foudaio of Chia o44749 Refereces: [] R Shumway D S Soffer ime Series Aalysis ad Is Applicaios wih R Examples Spriger 6 [] R S say Aalysis of Fiacial ime Series Wiley [3] G E Box G M Jekis G C Reisel ime Series Aalysis Forecasig ad Corol reice all 994 [4] Y Z Cai Davies A Simple Diagosic Mehod of Oulier Deecio for Saioary Gaussia ime Series Joural of Applied Saisics Vol3 o 3 pp 5-3 [5] C Che L M Liu Joi Esimaio of Model arameers ad Oulier Effecs i ime Series Joural of he America Saisical Associaio Vol88 o4 993 pp 84-97 [6] R S say Ouliers Level Shifs ad Variace Chages i ime Series Joural of Forecasig Vol7 o 988 pp [7] K Choy Oulier Deecio for Saioary ime Series Joural of Saisical laig ad Iferece Vol99 o pp 7 [8] B Abraham G E Box Bayesia Aalysis of Some Oulier roblems i ime Series Biomerika Vol66 o 979 pp 9-36 [9] R E Mcculloch R S say Bayesia Aalysis of Auoregressive ime Series via he Gibbs Sampler Joural of ime Series Aalysis Vol5 o 994 pp 35-5 [] A Jusel D eña R S say Deecio of Oulier aches i Auoregressive ime Series Saisica Siica Vol o3 998 pp 65-673 [] D eña Galeao Ecoomic ime Series: Modelig ad Seasoaliy Chapma & all [] J Lu L Shi F Che Oulier Deecio i ime Series Models Usig Local Ifluece Mehod Commuicaio i Saisics- heory ad Mehods Vol4 o pp - [3] Loui Oulier Deecio i ARMA Models Joural of ime series aalysis Vol9 o6 8 pp 57-65 [4] G E Box G C iao Bayesia Iferece i Saisical Aalysis Models Addiso-Wesley 973 [5] J Lud M Rudemo Geeralised Gibbs Sampler ad Muligrid Moe Carlo for Bayesia Compuaio Biomerika Vol87 o pp 353-369 [6] J O Berger Saisical Decisio heory ad Bayesia Aalysis Wiley 985 [7] A E Gelfad Gibbs Samplig Joural of he America Saisical Associaio Vol95 o45 pp 3-34 [8] R S say ime Series Model Specificaio i he resece of Oulier Joural of he America Saisical Associaio Vol8 o393 986 pp 3-4 E-ISS: 4-88 Volume 6 7