Methods for the estimation of missing values in time series

Transcription

1 Edh Cowan Unversy Research Onlne Theses: Docoraes and Masers Theses 6 Mehods for he esmaon of mssng values n me seres Davd S. Fung Edh Cowan Unversy Recommended Caon Fung, D. S. (6). Mehods for he esmaon of mssng values n me seres. Rereved from hp://ro.ecu.edu.au/heses/63 Ths Thess s posed a Research Onlne. hp://ro.ecu.edu.au/heses/63

2 Edh Cowan Unversy Copyrgh Warnng You may prn or download ONE copy of hs documen for he purpose of your own research or sudy. The Unversy does no auhorze you o copy, communcae or oherwse make avalable elecroncally o any oher person any copyrgh maeral conaned on hs se. You are remnded of he followng: Copyrgh owners are enled o ake legal acon agans persons who nfrnge her copyrgh. A reproducon of maeral ha s proeced by copyrgh may be a copyrgh nfrngemen. Where he reproducon of such maeral s done whou arbuon of auhorshp, wh false arbuon of auhorshp or he auhorshp s reaed n a derogaory manner, hs may be a breach of he auhor s moral rghs conaned n Par IX of he Copyrgh Ac 968 (Ch). Cours have he power o mpose a wde range of cvl and crmnal sancons for nfrngemen of copyrgh, nfrngemen of moral rghs and oher offences under he Copyrgh Ac 968 (Ch). gher penales may apply, and hgher damages may be awarded, for offences and nfrngemens nvolvng he converson of maeral no dgal or elecronc form.

3 USE OF TESIS The Use of Thess saemen s no ncluded n hs verson of he hess.

4 Mehods for he Esmaon of Mssng Values n Tme Seres A hess Submed o he Faculy of Communcaons, ealh and Scence Edh Cowan Unversy Perh, Wesern Ausrala By Davd Sheung Ch Fung In Fulfllmen of he Requremens For he Degree Of Maser of Scence (Mahemacs and Plannng) 6

5 Absrac Tme Seres s a sequenal se of daa measured over me. Examples of me seres arse n a varey of areas, rangng from engneerng o economcs. The analyss of me seres daa consues an mporan area of sascs. Snce, he daa are records aken hrough me, mssng observaons n me seres daa are very common. Ths occurs because an observaon may no be made a a parcular me owng o fauly equpmen, los records, or a msake, whch canno be recfed unl laer. When one or more observaons are mssng may be necessary o esmae he model and also o oban esmaes of he mssng values. By ncludng esmaes of mssng values, a beer undersandng of he naure of he daa s possble wh more accurae forecasng. Dfferen seres may requre dfferen sraeges o esmae hese mssng values. I s necessary o use hese sraeges effecvely n order o oban he bes possble esmaes. The objecve of hs hess s o examne and compare he effecveness of varous echnques for he esmaon of mssng values n me seres daa models. The process of esmang mssng values n me seres daa for unvarae daa nvolves analyss and modellng. Tradonal me seres analyss s commonly dreced oward scalar-valued daa, and can be represened by radonal Box-Jenkns auoregressve, movng average, or auoregressve-movng average models. In some cases hese models can be used o oban mssng values usng an nerpolaon approach. In a recen developmen of me seres analyss, he sraegy has been o rea several varables smulaneously as vecor-valued varables, and o nroduce an alernave way of represenng he model, called sae space modellng. Examples of popular models ha

6 can be represened n sae space form are srucured me seres models where daa s consdered as a combnaon of level, rend and seasonal componens. Ths approach, whch uses Kalman flerng, can also be used o oban smoohed esmaes of mssng values. There are oher common approaches avalable for modellng me seres daa wh mssng values such as: me seres decomposon; leas squares approxmaon; and numercal nerpolaon mehods. Some of hese approaches are quck and he model buldng procedures are easy o carry ou. Ofen hese approaches are popular choces for professonals who work whn he me seres analyss ndusres. Ths hess wll examne each approach by usng a varey of compuer sofware and smulaed daa ses for esmaon and forecasng of daa wh mssng values. In addon hs hess wll also explore he advanages and dsadvanages of srucural sae space modellng for dfferen mssng value daa paerns by examnng he mean absolue devaon of he resduals for he mssng values. 3

7 Declaraon I cerfy ha hs hess does no ncorporae whou acknowledgmen any maeral prevously submed for a degree or dploma n any nsuon of hgher educaon; and ha o he bes of my knowledge and belef does no conan any maeral prevously publshed or wren by anoher person where due reference s made n he ex. Sgnaure.. Dae. 4

8 Acknowledgmens I would lke o express my specal hanks and graude o Assocae Professor James Cross, my supervsor, from he School of Engneerng and Mahemacs a Joondalup Campus a Edh Cowan Unversy, for hs experse and asssance n clarfyng quesons I had regardng Tme Seres Analyss and he advse he gave whch was nvaluable n enablng me o compue hs hess. I am especally apprecave of my frend Grace Guadagnno, who provded me wh advce and suppor hroughou he wrng of hs dsseraon. Las, bu no leas, I would lke o hank my mum and dad for her ongong suppor and encouragemen hroughou my sudes, parcularly n he compleon of hs hess. 5

9 Table of Conens Page Absrac Declaraon Acknowledgmens Table of Conens Ls of Fgures Ls of Tables Chaper Inroducon. Abou hs Chaper 5. Background Deermnsc Modellng Sochasc Modellng Sae Space Modellng..8.3 Am of Research Sgnfcance of Research..9.5 Daa Research Mehodology..7 Srucure of he Dsseraon. Leraure Revew Deermnsc Models for Mssng Values 3. Abou hs Chaper 3 6

10 3. Leas Squares Approxmaons Inerpolang wh a Cubc Splne.37 4 Sochasc Models for Mssng Values 4. Abou hs Chaper Saonary Models Non-saonary Models Box-Jenkns Models Auoregressve Models Movng Average Models Auoregressve Movng Average Models Auoregressve Inegraed Movng Average Models Box-Jenkns Models and Mssng Values Leas Squares Prncple Inerpolaon Weghng for general ARIMA models 65 5 Sae Space Models for Mssng Values 5. Abou hs Chaper 8 5. Sae Space Models Srucural Tme Seres Models The Kalman Fler Predcon Sage Updang Sage Esmaon of Parameers The Expecaon-Maxmsaon (EM) Algorhm Mssng Values.96 7

11 6. Analyss and Comparson of Tme Seres Model 6. Abou hs Chaper.. 6. Applyng Polynomal Curve fng o Box-Jenkns Models Applyng Cubc Splne o Box-Jenkns Models ARIMA Inerpolaon Applyng Srucural Tme Seres Analyss o Box-Jenkns Models Concluson 7. Abou hs chaper Effecveness of varous approaches Furher comparson beween varous approaches Fuure research drecon Concluson.. 6 References 63 Appendx A...65 Appendx B..87 8

12 Ls of Fgures Page Fgure 3. Fgure 3. Fgure 4. Fgure 4. Fgure 5. Fgure 5. Fgure 5.3 Fgure 5.4 Fgure 5.5 Fgure 5.6 Fgure 6. Fgure 6. Fgure 6.3 Fgure 6.4 Fgure 6.5 Fgure 6.6 Fgure 6.7 Fgure 6.8 Fgure 6.9 Fgure 6. Fgure 6. Esmae mssng values usng leas square approxmaons..36 Esmae mssng values usng cubc splne 45 Esmae mssng value n AR() process by usng Leas Square Prncple...6 Esmae wo mssng values n AR() process by usng Leas Square Prncple...6 Plo of 7 observaons on purse snachng n Chcago...98 Acual and Fed Values on purse snachng n Chcago 99 Normalsed Resdual on purse snachng n Chcago..99 Plo on purse snachng n Chcago wh a run of mssng observaons... Resuls of esmang mssng values on purse snachng n Chcago.. Acual values and fed values on purse snachng n Chcago. AR mssng a 49 S.D..4.7 AR mssng a 49 S.D AR mssng a 7 S.D AR mssng a 7 S.D..4 AR mssng a 9 S.D AR mssng a 9 S.D MA mssng a 49 S.D..4 7 MA mssng a 49 S.D MA mssng a 7 S.D..4.. MA mssng a 7 S.D..4 MA mssng a 9 S.D..4 9

13 Ls of Fgures Page Fgure 6. Fgure 6.3 Fgure 6.4 Fgure 6.5 Fgure 6.6 Fgure 6.7 Fgure 6.8 Fgure 6.9 Fgure 6. Fgure 6. Fgure 6. Fgure 6.3 Fgure 6.4 Fgure 6.5 Fgure 6.6 Fgure 6.7 Fgure 6.8 Fgure 6.9 Fgure 6.3 Fgure 6.3 Fgure 6.3 Fgure 6.33 Fgure 6.34 MA mssng a 9 S.D Splne-AR Mssng 49 S.D..4.5 Splne-AR Mssng 49 S.D Splne-AR Mssng 7 S.D Splne-AR Mssng 7 S.D..4.6 Splne-AR Mssng 9 S.D..4.7 Splne-AR Mssng 9 S.D Splne-MA Mssng 49 S.D..4 8 Splne-MA Mssng 49 S.D Splne-MA Mssng 7 S.D Splne-MA Mssng 7 S.D..4 3 Splne-MA Mssng 9 S.D..4 3 Splne-MA Mssng 9 S.D Inerpolaon -AR Mssng 49 S.D Inerpolaon -AR Mssng 49 S.D Inerpolaon -AR Mssng 7 S.D Inerpolaon -AR Mssng 7 S.D Inerpolaon -AR Mssng 9 S.D Inerpolaon -AR Mssng 9 S.D Inerpolaon -MA Mssng 49 S.D Inerpolaon -MA Mssng 49 S.D Inerpolaon -MA Mssng 7 S.D Inerpolaon -MA Mssng 7 S.D..4.39

14 Ls of Fgures Page Fgure 6.35 Fgure 6.36 Fgure 6.37 Fgure 6.38 Fgure 6.39 Fgure 6.4 Fgure 6.4 Fgure 6.4 Fgure 7. Fgure 7. Fgure 7.3 Fgure 7.4 Fgure 7.5 Fgure 7.6 Fgure 7.7 Fgure 7.8 Fgure 7.9 Fgure 7. Fgure 7. Fgure 7. Inerpolaon -MA Mssng 9 S.D Inerpolaon -MA Mssng 9 S.D STSA -AR Mssng 49 S.D STSA -AR Mssng 7 S.D STSA -AR Mssng 9 S.D STSA -MA Mssng 49 S.D STSA -MA Mssng 7 S.D STSA -MA Mssng 9 S.D Varous Mehods -AR Mssng 49 S.D Varous Mehods -AR Mssng 7 S.D Varous Mehods -AR Mssng 9 S.D Varous Mehods -MA Mssng 49 S.D Varous Mehods -MA Mssng 7 S.D..4 5 Varous Mehods -MA Mssng 9 S.D STSA vs Varous Mehods -AR Mssng 49 S.D STSA vs Varous Mehods -AR Mssng 7 S.D STSA vs Varous Mehods -AR Mssng 9 S.D STSA vs Varous Mehods -MA Mssng 49 S.D STSA vs Varous Mehods -MA Mssng 7 S.D STSA vs Varous Mehods -MA Mssng 9 S.D

15 Ls of Tables Page Table 3.. Table 3.. Table 3.3. Table 4.6. Table 4.6. Seven daa pons wh hree consecuve mssng values...35 Esmae mssng values usng leas square approxmaons...36 Esmae mssng values usng cubc splne.43 AR() me seres model wh mssng value when AR() me seres model wh mssng value when 6 and 7 6 Table 4.7. Weghngs w l and w l for mssng value n ARIMA model...78 Table 4.7. Weghngs w l and w l for mssng value n ARIMA model...79 Table 5.5. Table 6.. Table 6.. Table 6.. Table 6..3 Table 6..4 Table 6..5 Table 6..6 Table 6..7 Table 6..8 Table 6..9 Acual values and fed values on purse snachng n Chcago. Varous me seres models for smulaon 3 AR me seres model wh mssng value a poson Mssng value a poson 49 SD.4.7 Mssng value a poson 49 SD AR me seres model wh mssng value a poson 7. 9 Mssng value a poson 7 SD.4... Mssng value a poson 7 SD.4. AR me seres model wh mssng value a poson 9... Mssng value a poson 9 SD Mssng value a poson 9 SD Table 6.. MA me seres model wh mssng value a poson Table 6.. Table 6.. Mssng value a poson 49 SD Mssng value a poson 49 SD.4.7 Table 6..3 MA me seres model wh mssng value a poson Table 6..4 Mssng value a poson 7 SD.4.

16 Ls of Tables Page Table 6..5 Table 6..6 Table 6..7 Table 6..8 Table 6.3. Table 6.3. Table Table Table Table Table Table Table Mssng value a poson 7 SD MA me seres model wh mssng value a poson 9. - Mssng value a poson 9 SD.4. Mssng value a poson 9 SD AR Mssng a poson 49 S.D AR Mssng a poson 49 S.D AR Mssng a poson 7 S.D AR Mssng a poson 7 S.D AR Mssng a poson 9 S.D AR Mssng a poson 9 S.D MA Mssng a poson 49 S.D MA Mssng a poson 49 S.D MA Mssng a poson 7 S.D Table 6.3. MA Mssng a poson 7 S.D Table 6.3. MA Mssng a poson 9 S.D Table 6.3. MA Mssng a poson 9 S.D Table 6.4. Table 6.4. Table Table Table Table Table AR Mssng a poson 49 S.D AR Mssng a poson 49 S.D AR Mssng a poson 7 S.D AR Mssng a poson 7 S.D AR Mssng a poson 9 S.D AR Mssng a poson 9 S.D MA Mssng a poson 49 S.D

17 Ls of Tables Page Table Table MA Mssng a poson 49 S.D MA Mssng a poson 7 S.D Table 6.4. MA Mssng a poson 7 S.D Table 6.4. MA Mssng a poson 9 S.D Table 6.4. MA Mssng a poson 9 S.D Table 6.5. Table 6.5. Table Table Table Table Table 7.. Table 7.. Table 7..3 Table 7..4 Table 7..5 Table 7..6 Table 7.3. Table 7.3. Table Table Table Table AR Mssng a poson 49 S.D AR Mssng a poson 7 S.D AR Mssng a poson 9 S.D MA Mssng a poson 49 S.D MA Mssng a poson 7 S.D MA Mssng a poson 9 S.D AR Mssng a poson 49 S.D AR Mssng a poson 7 S.D AR Mssng a poson 9 S.D MA Mssng a poson 49 S.D..4.5 MA Mssng a poson 7 S.D MA Mssng a poson 9 S.D AR Mssng a poson 49 S.D AR Mssng a poson 7 S.D AR Mssng a poson 9 S.D MA Mssng a poson 49 S.D MA Mssng a poson 7 S.D MA Mssng a poson 9 S.D

18 CAPTER INTRODUCTION. Abou hs Chaper Ths chaper provdes an overvew of he research underaken n hs hess. Secon. wll brefly dscuss he background o hs sudy whle secons..,.. and..3 look a varous approaches for modellng me seres daa. The am and sgnfcance of hs research are dealed n secons.3 and.4. The daa used n hs hess s descrbed n secon.5. Fnally, he research mehodology and bref overvew of each chaper are saed n secons.6 and.7.. Background In our socey, we ofen have o analyse and make nferences usng real daa ha s avalable for collecon. Ideally, we would lke o hnk ha he daa s carefully colleced and has regular paerns wh no oulers or mssng value. In realy, hs does no always happen, so ha an mporan par of he nal examnaon of he daa s o assess he qualy of he daa and o consder modfcaons where necessary. A common problem ha s frequenly encounered s mssng observaons for me seres daa. Also daa ha s known or suspeced o have been observed erroneously may be regarded as havng mssng values. In addon, poor record keepng, los records and 5

19 uncooperave responses durng daa collecon wll also lead o mssng observaons n he seres. One of he key seps n me seres analyss s o ry o denfy and correc obvous errors and fll n any mssng observaons enablng comprehensve analyss and forecasng. Ths can somemes be acheved usng smple mehods such as eyeballng, or calculang approprae mean value ec. owever, more complex mehods may be needed and hey may also requre a deeper undersandng of he me seres daa. We may have o aemp o dscover he underlyng paerns and seasonaly. Once we undersand he naure of he daa, we can ackle he problem usng common sense combned wh varous mahemacal approaches. Ths s boh an ar and a scence. Somemes, we are requred o forecas values beyond, or pror o, he range of known values. To complee hs ask successfully we need a model whch sasfacorly fs he avalable daa even when mssng values are presen. More complex mehods for analysng me seres daa wll depend on he ype of daa ha we are handlng. Bu mos of he me, we would use eher a deermnsc or sochasc approach... Deermnsc Modellng (also called Numercal Analyss Modellng) Ths mehod assumes he me seres daa corresponds o an unknown funcon and we ry o f he funcon n an approprae way. The mssng observaon can be esmaed by usng he approprae value of he funcon a he mssng observaon. Unlke radonal me seres approaches, hs mehod dscards any relaonshp beween he varables over me. The approach s based on obanng he bes f for he me seres 6

20 daa and s usually easy o follow compuaonally. In hs approach here s a requremen for bes f process o be clearly defned. There are a varey of curves ha can be used o f he daa. Ths wll be consdered n more deal n a laer chaper... Sochasc Modellng (also called Tme Seres Modellng) Anoher common me seres approach for modellng daa s o use Box-Jenkns Auoregressve Inegraed Movng Average (ARIMA) models. The ARIMA models are based on sascal conceps and prncples and are able o model a wde range of me seres paerns. These models use a sysemac approach o denfy he known daa paerns and hen selec he approprae formulas ha can generae he knd of paerns denfed. Once he approprae model has been obaned, he known me seres daa can be used o deermne approprae values for he parameers n he model. The Box-Jenkns ARIMA models can provde many sascal ess for verfyng he valdy of he chosen model. In addon he sascal heory behnd Box-Jenkns ARIMA models allows for sascal measuremens of he uncerany n a forecas o be made. owever, a dsadvanage of he Box-Jenkns ARIMA models s ha assumes ha daa s recorded for every me perod. Ofen me seres daa wh mssng values requre us o apply some nuve mehod or approprae nerpolave echnque o esmae hose mssng values pror o Box-Jenkns s ARIMA approach. 7

21 ..3 Sae Space Modellng A new approach o me seres analyss s he use of sae space modellng. Ths modellng approach can ncorporae varous me seres models such as Box-Jenkns ARIMA models and srucural me seres. Sae space modellng emphasses he noon ha a me seres s a se of dsnc componens. Thus, we may assume ha observaons relae o he mean level of he process hrough an observaon equaon, whereas one or more sae equaons descrbe how he ndvdual componens change hrough me. A relaed advanage of hs approach s ha observaons can be added one a a me, and he esmang equaons are hen updaed o produce new esmaes. (p.44 Kendall and Ord) Essenally, Kalman Flerng and Maxmum Lkelhood Esmaon mehods are mporan procedures for handlng sae space models. The approach connually performs esmang and smoohng calculaons ha depend only on oupu from forward and backward recursons. Wh modfcaons on he maxmum lkelhood procedure, enables he approach o esmae and forecas for daa wh mssng values. Ths research wll nvesgae he above models o deermne he mos approprae echnque for modellng me seres daa wh mssng values. These models can hen be used for esmang he mssng values and enable forecasng. In parcular, dfferen paerns and frequences of mssng values wll be consdered usng a large number of smulaed daa ses. 8

22 .3 Am of Research The research objecves are as follows: a) o compare he applcaon of deermnsc and sochasc approaches o modellng me seres daa wh mssng value; b) o compare varous sochasc approaches and assocaed models for dfferen mssng daa paerns; and c) o compare radonal Box-Jenkns ARIMA and sae space models o oban esmaes of mssng values and for forecasng..4 Sgnfcance of he Research The objecve of hs research s o provde a revew of varous modellng approaches for me seres daa wh mssng values. In addon, new nsghs wll be provded regardng he mos approprae modellng echnques for dfferen mssng daa paerns..5 Daa To es he effecveness of each esmaon mehod, we requre many daa ses ha represen dfferen me seres models. In hs hess, we have chosen a popular spreadshee program creaed by Mcrosof called Excel and a mul-purpose mahemacs package called Mnab o creae smulaons on he compuer. We chose hese packages because of her populary and easy access. By creang macros, boh 9

23 packages can generae specfc me seres daa ses whou any dffculy. The specfc me seres models we are gong o generae are Auoregressve (AR) and Movng Average (MA) me seres wh varous parameers. All smulaed daa ses are gong o be saonary me seres wh mssng values based on he assumpon ha decomposon or ransformaon echnques can be used o conver non-saonary daa o saonary daa. (Chafeld, 3 p.4)..6 Research Mehodology To conduc our research, we wll se a sngle mssng value a varous posons for each daa se. The am s o compare he performance of each mehod for dfferen me seres models wh mssng values n dfferen posons. For dfferen esmaon mehods, he mssng value posons for a pon daa se are a follows: Polynomal Curve Fng: Mssng value a poson 7, 49 and 9. Cubc Splne: Mssng value a poson 7, 49 and 9. ARIMA Inerpolaon: Mssng value a poson 7, 4,, 8, 35, 4, 49, 56, 63, 7,77, 84 and 9. Sae Space Modellng: Mssng value a poson 7, 49 and 9. In each of he esmaon processes, we calculae he absolue devaon beween he esmaed value and he orgnal value. Afer repeang he process one hundred mes, we deermne he Mean Absolue Devaon (MAD) and assocaed sandard devaon.

24 In hs hess, our objecve s o examne he accuracy of varous esmaon approaches o calculae mssng values a dfferen posons whn a me seres daa se. Wh mnor adjusmen, mos of he approaches are able o be adaped o daa ses wh mulple mssng values. owever, f a daa se consss of consecuve mssng values, our esmaes become less accurae due o lack of nformaon. In hs hess, we are gong o focus our analyss on a sngle mssng value a varous posons whn he daa se..7 Srucure of he Dsseraon Chaper Inroducon Chaper Leraure revew Ths chaper provdes a summary of he arcles and publcaons relaed o hs hess. Chaper 3 Deermnsc Approach Deermnsc approach usually refers o he use of numercal analyss echnques on he me seres daa. The prncple of numercal analyss s o consder he me seres daa paern as an unknown behavour of a funcon. We ry o denfy he mos approprae funcon for ha behavour n order o esmae he mssng values.

25 Chaper 4 Sochasc Approach In me seres we ofen consder fuure values as realsaons from a probably dsrbuon whch s condoned by knowledge of pas values. ence, he experse and experences of he analys play an mporan role when analysng daa. Two mporan ypes of sochasc models used are saonary and non-saonary models. Chaper 5 Sae Space Approach Sae Space Modellng s a popular approach n me seres analyss because of s ably o adap o dfferen me seres models such as he Box-Jenkns ARIMA and Srucural Tme Seres. Observaons can be added one a a me and he esmang equaons are hen updaed o produce new esmaes. Chaper 6 Analyss and Comparson Snce we have examned varous me seres modellng approaches, we are gong o apply hose approaches o smulaed daa ses derved from dfferen me seres models. From hs exercse, we should be able o gan an nsgh on how each mehod performs for dfferen me seres suaons.

26 Chaper 7 Concluson In concluson, we wll make approprae comparsons beween mehods and hghlgh her performance on esmang mssng value for ARIMA models. 3

27 CAPTER Leraure Revew Ths secon of he hess covers he avalable research leraure relaed o me seres wh mssng values. Durng he research, we have focused on lnear me seres models as here s lle reference n he leraure o nonlnear me seres daa and o spaal daa wh mssng values. One approach relang o spaal daa wh mssng values was oulned by Gomez e al (995). I uses he boosrap mehod o npu mssng naural resource nvenory daa. In addon, wo arcles on non-lnear me seres model were denfed. The laes arcle s by Volker Tresp and Remar ofmann (998) ha develops echnques for nonlnear me seres predcon wh mssng and nosy daa. Ths s an exenson of her prevous arcle enled Mssng and Nosy Daa n Nonlnear Tme Seres Predcon publshed n 995. Tradonally he approach o obanng mssng values for lnear me seres has nvolved he use of curve fng. Deals of hese approaches can be found n many books such as: Appled Numercal Analyss by Curs F. Gerald & Parck O. Whealey (994), The Analyss of Tme Seres An Inroducon by Chrs Chafeld (3), Tme Seres Forecasng Smulaon & Applcaon by Gareh Janacek and Louse Swf (993), Forecasng, Srucural Tme Seres Models and he Kalman Fler by Andrew 4

28 C. arvey (), Tme Seres: Theory and Mehods by Peer J. Brockwell & Rchard A. Davs (99) and Tme Seres Analyss by James D. amlon (994). In more recen mes he focus has shfed owards more sophscaed approaches usng Box Jenkns ARIMA and Sae Space Modellng. A summary of relevan arcles relang o lnear me seres wh mssng values s provded below. One of he earles arcles whch relaes o mssng values n me seres usng sae space modellng was wren by Kalman (96). In hs arcle he oulned a new approach o lnear flerng and predcon problem. The auhor nvesgaed he classcal flerng and predcon problem by usng he Bode-Shannon represenaon of random processes and he sae-ranson mehod of analyss of dynamc sysems. As a resul of hs nvesgaon, he auhor dscovered he followng: a) The formulaon and mehod of soluon of he problem apply whou modfcaon o saonary and nonsaonary sascs and o growng-memory and nfne-memory flers. b) A nonlnear dfference (or dfferenal) equaon s derved for he covarance marx of he opmal esmaon error. From he soluon of hs equaon he coeffcens of he dfference (or dfferenal) equaon of he opmal lnear fler are obaned whou furher calculaons. 5

29 c) The flerng problem s shown o be he dual of he nose-free regulaor problem. The new mehod developed here s appled o wo well-known problems, confrmng and exendng earler resuls. (R. E. Kalman, 96, p.35) The approach developed by Kalman n hs arcle has been exended by Rchard. Jones (98) who derved a mehod of calculang he exac lkelhood funcon of a saonary auoregressve movng average (ARMA) me seres based on Akake s Markovan represenaon combned wh Kalman recursve esmaon. Ths sae space approach nvolves marces and vecors wh dmensons equal o Max (p,q) where p s he order of he auoregresson and q s he order of he movng average, raher han marces wh dmensons equal o he number of observaons. A key o he calculaon of he exac lkelhood funcon s he proper calculaon of he nal sae covarance marx. The paper also ncluded some dscusson on observaonal error n he model and he exenson o mssng observaons. The use of a nonlnear opmzaon program gves he maxmum lkelhood esmaes of he parameers and allows for model denfcaon based on Akake s Informaon Creron (AIC). G. Gardner A.C. arvey and G.D.A. Phllps (98) presened an algorhm ha enables he exac lkelhood funcon of a saonary auoregressve-movng average (ARMA) process o be calculaed by means of he Kalman fler. I consss of wo basc subrounes. The frs, subroune STARMA, cass he ARMA model no he sae space form necessary for Kalman flerng, and compues he covarance marx assocaed wh he nal value of he sae vecor. The second subroune, KARMA, 6

30 carres ou he recursons and produces a se of sandardzed predcon errors, ogeher wh he deermnan of he covarance marx of he observaons. These wo quanes ogeher yeld he exac lkelhood, hs may be maxmzed by an erave procedure based on a numercal opmzaon algorhm whch does no requre analyc dervaves. In parcular, he second subroune KARMA conans a devce whereby he lkelhood may be approxmaed o a level of accuracy whch s under he conrol of he user. Ths enables a consderable amoun of compung me o be saved, wh very lle aendan loss n precson. Fnally, anoher subroune, KALFOR, may be used o compue predcons of fuure values of he seres, ogeher wh he assocaed condonal mean square errors. Four years laer, A. C. arvey and R. G. Perse (984) dscussed wo relaed problems nvolvng me seres wh mssng daa. The frs concerns he maxmum lkelhood esmaon of he parameers n an ARIMA model when some of he observaons are mssng or subjec o emporal aggregaon. The second concerns he esmaon of he mssng observaons. They also poned ou boh problems can be solved by seng up he model n sae space form and applyng he Kalman fler. Durng 986, Rober Kohn and Crag F. Ansley showed how o defne and hen compue effcenly he margnal lkelhood of an ARIMA model wh mssng observaons. The compuaon s carred ou by usng he unvarae verson of he modfed Kalman fler nroduced by Ansley and Khon (985), whch allows a parally dffuse nal sae vecor. They also show how o predc and nerpolae mssng observaons and oban he mean squared error of he esmae. Wh he help of modern compuers, sae space modellng has became a much smpler process han before and mahemacans have ye o examne ways o furher develop hs useful process. 7

31 There are many arcles whch focus on he use of ARIMA models as well as oher echnques o deermne mssng values n me seres. Evnd Damsleh (979) developed a mehod o fnd he opmal lnear combnaon of he forecas and backforecas for mssng values n a me seres whch can be represened by an ARIMA model. Durng he followng year, W. Dunsmur and P.M. Robnson (98) derved a mehod for he esmaon of models for dscree me seres n he presence of mssng daa. They also saed he advanages for he use of hs mehod over alernaves. A he begnnng hey assessed he performance of he mehod used n esmang smple models by usng smulaons, and hen hey appled o a me seres of polluon levels conanng some mssng observaons. In 98, B. Abraham dscussed a mehod based on forecasng echnques o esmae mssng observaons n me seres. e also compared hs mehod usng mnmum mean square esmae as a measure of effcency. Anoher popular esmaon procedure s called exac maxmum lkelhood esmaon. Mchael A. Wncek and Gregory C. Rensel (986) developed an explc procedure o oban he exac maxmum lkelhood esmaes of he parameers n a regresson model wh ARMA me seres errors wh possbly non-consecuve daa. The mehod s based on an nnovaon ransformaon approach from whch an explc recursve procedure s derved for he effcen calculaon of he exac lkelhood funcon and assocaed dervaves. The nnovaons and assocaed dervaves are used o develop a modfed Newon-Raphson procedure for compuaon of he esmaes. A weghed nonlnear leas squares nerpreaon of he esmaor s also gven. A numercal example s provded o llusrae he mehod. A he same me, P. M. Robnson (985) demonsraed how o use he score prncple o es for seral correlaon n resduals for sac me seres regresson n he presence of mssng daa. I s appled boh o he 8

32 lkelhood condonal on he observaon mes, and o an uncondonal form of lkelhood. Also, asympoc dsrbuons of he es sascs are esablshed, under boh he null hypohess of no seral correlaon, and sequences of local, correlaed, alernaves, enablng analyc comparson of effcency. Osvaldo Ferrero (987) dscussed dfferen alernaves for he esmaon of mssng observaon n saonary me seres for auoregressve movng average models. e ndcaed ha he occurrence of mssng observaons s que common n me seres and n many cases s necessary o esmae hem. The arcle offers a seres of esmaon alernaves o help esmae mssng observaons. Two years laer, Yonna Rosen, Boaz Pora (989) consdered he esmaon of he covarances of saonary me seres wh mssng observaons. The esmaon provded general formulas for he asympoc second-order momens of he sample covarances, for eher random or deermnsc paern of mssng values. The auhors derved closed-form expressons for he random Bernoull paern and for he deermnsc perodc of mssng observaons. These expressons are explcly evaluaed for auoregressve movng average me seres and he resuls are useful for consrucng and analysng parameer or specrum esmaon algorhm based on he sample covarances for saonary me seres wh mssng observaons. Boh auhors also consder he problem of specral esmaon hrough he auoregressve movng average modellng of saonary processes wh mssng observaons. They presen a class of esmaors based on he sample covarances and propose an asympocally opmal esmaor n hs class. The proposed algorhm s based on a nonlnear leas squares f of he sample covarances compued from he daa o he rue 9

33 covarances of he assumed ARMA model. The sascal properes of he algorhm are explored and used o show ha s asympocally opmal, n he sense of achevng he smalles possble asympoc varance. The performance of he algorhm s llusraed by some numercal examples. Durng he same year, Grea M Ljung (989) derved an expresson for he lkelhood funcon of he parameers n an auoregressve movng average model when here are mssng values whn he me seres daa. Also, he mehod o esmae he mssng values for saonary as well as nonsaonary models relaed o he mean squared errors are consdered. Danel Pena and George C. Tao (99) demonsraed ha mssng values n me seres can be reaed as unknown parameers and esmaed by maxmum lkelhood or as random varables and predced by he expecaon of he unknown values gven he daa. They provded examples o llusrae he dfference beween hese wo procedures. I s argued ha he second procedure s, n general, more relevan for esmang mssng values n me seres. Thomas S. Shvely (99) consruced some ess for auoregressve dsurbances n a me seres regresson wh mssng observaons, where he dsurbance erms are generaed by () an AR() process and () an AR(p) process wh a possble seasonal componen. Also, a pon opmal nvaran (POI) es s consruced for each problem o check for effcency. In addon, he paper shows how o compue exac small sample p-values for he ess n O(n) operaons, and gves a compuaonally effcen procedure for choosng he specfc alernave agans whch o make he POI es mos powerful nvaran. Seve Beverdge (99) also exended he concep of usng mnmum mean square error lnear nerpolaor for mssng values n me seres o handle any paern of non-consecuve observaons. The paper refers o he applcaon of smple ARMA models o dscuss he usefulness of eher he 3

34 nonparamerc or he paramerc form of he leas squares nerpolaor. In more recen years, Fabo. Neo and Jorge Marnez (996) demonsraed a lnear recursve echnque ha does no use he Kalman fler o esmae mssng observaons n unvarae me seres. I s assumed ha he seres follows an nverble ARIMA model. Ths procedure s based on he resrced forecasng approach, and he recursve lnear esmaors are obaned when he mnmum mean-square error are opmal. In 997, Albero Luceno (997) exended Ljung s (989) mehod for esmang mssng values and evaluang he correspondng lkelhood funcon n scalar me seres o he vecor cases. The seres s assumed o be generaed by a possbly parally nonsaonary and nonnverble vecor auoregressve movng average process. I s assumed no parcular paern of mssng daa exsed. In order o avod nalsaon problems, he auhor does no use Kalman fler eraons. Also, does no requre he seres o be dfferenced and hus avods complcaons caused by over-dfferencng. The esmaors of he mssng daa are provded by he normal equaons of an approprae regresson echnque. These equaons are adaped o cope wh emporally aggregaed daa; he procedure parallels a marx reamen of conour condons n he analyss of varance. I can be seen from he leraure ha here are a varey of mehods avalable o esmang mssng values for me seres daa. Wha s however lackng n he leraure s a comparson of dfferen mehods for dfferen ypes of daa ses and dfferen posons for he mssng daa. Ths hess ams o provde such a comparson by usng a varey of smulaed daa ses wh mssng values n dfferen locaons. 3

35 CAPTER 3 Deermnsc Models for Mssng Values 3. Abou hs Chaper In general, deermnsc models for me seres refers o he use of numercal analyss echnques for modelng me seres daa. A major advanage of numercal analyss s ha a numercal answer can be obaned even when a problem has no analycal soluon. (p. Gerald and Whealy 994) The prncple of numercal analyss s o assume he me seres daa paern s a realsaon of an unknown funcon. The am s o denfy he mos approprae funcon o represen he daa n order o esmae he mssng values. We assume he behavour of he me seres daa follows a polynomal funcon or combnaon of polynomal funcons and examne he me nerval ha nvolved he mssng values. Somemes hs s he mos dffcul par of he analyss process. We have o examne all he facors nvolved and decde he approprae lengh of me nerval o be consdered. We wll hen fnd a polynomal ha fs he seleced se of pons and assume ha he polynomal and he funcon behave nearly he same over he nerval n queson. Values of he polynomal should be reasonable esmaes of he values of he unknown funcon (p.3 Gerald and Whealy 994). owever, when he daa appears o have local rregulares, hen we are requred o f sub-regons of he daa wh dfferen polynomals. Ths ncludes specal polynomals called splnes. 3

36 For mos of he me seres daa, we do no wan o fnd a polynomal ha fs exacly o he daa. Ofen funcons used o f a se of real values wll creae dscrepances or he daa se may come from a se of expermenal measuremens ha are subjec o error. A echnque called leas squares s normally used n such cases. Based on sascal heory, hs mehod fnds a polynomal ha s more lkely o approxmae he rue values (p.3 Gerald and Whealy 994). 3. Leas Squares Approxmaons For any curve fng exercse, we are ryng o mnmze he devaons of he daa pons from he esmaed curve. Our goal s o make he magnude of he maxmum error a mnmum, bu for mos problems hs creron s rarely underaken because he absolue-value funcon has no dervave a he orgn. The usual approach s o mnmze he sum of he squares of he errors for a polynomal of gven degree, he leas-squares prncple. In addon o gvng a unque resul for a gven se of daa, he leas-squares mehod s also n accord wh he maxmum-lkelhood prncple of sascs. If he measuremen errors are normally dsrbued and f he sandard devaon s consan for all he daa, he lne deermned by mnmzng he sum of squares can be shown o have values of slope and nercep whch have maxmum lkelhood of occurrence. I s hghly unlkely ha me seres daa s lnear, so we need o f he daa se wh funcons oher han a frs-degree polynomal. As we use hgher-degree polynomals, 33

37 34 we wll reduce he devaons of he pons from he curve unl he degree of he polynomal, n, equals o one less han he number of daa pons, where here s an exac mach (assumng no duplcae daa a he same x-value). We call hs funcon an nerpolang polynomal whch s llusraed below: Consder fng a polynomal of fxed degree m m a m x x a x a a y L () o n daa pons ( ) ( ) ( ) x n y n y x y x,,,,,, L. By subsung hese n values no equaon (), we should oban a sysem of equaons m a m x x a a y L m a m x x a a y L M M M M M M m n m n n x a x a a y L. () In order o solve he sysem of equaons (), we would use marces o represen he sysem Mv y where y n y y y M, m n n n m m x x x x x x x x x M L M M M M L L, a m a a v M. ence, he coeffcens of he polynomal can be deermned by he followng: Mv y Mv M y M ( ) ( ) ( )v M M M M y M M M y M M M v ) (

38 Example 3. Esmae mssng values usng leas square approxmaons. Gven he followng daa pons: T X() Acual Value X() (3.78) mssng 4 (6.) mssng 5 (7.569) mssng Table 3.. Seven daa pons wh hree consecuve mssng values. For mssng values a 3, 4, and 5 we would use he daa pons, x().7;, x().85; 6, x().7; 7, x() ence,.7 a a a a 3.85 a a a a a a 6a 36a a a 7a 49a y, M, a a v, a a3 M v ( M M ) M y 35

39 v.4. ence, he polynomal of 3 rd degree s: y x.4x.x X() Acual Value X() Esmae (3.78) Mssng 3. 4 (6.) Mssng (7.569) Mssng Table 3.. Esmae mssng values usng leas square approxmaons. x() Esmae mssng values usng leas square approxmaons TRUE Esmae Fgure 3. Esmae mssng values usng leas square approxmaons. Example 3. shows ha nerpolang polynomal has several advanages. Frsly he mehod s easy o follow and quck o produce esmang values; s no resrced o esmang a sngle mssng value; and provdes reasonable esmaes compared o he 36

40 orgnal mssng values. owever, hs approach does no examne any naural facors concernng he me seres ha could lead o unrealsc esmaons. 3.3 Inerpolang wh a Cubc Splne A cubc splne s a common numercal curve fng sraegy ha fs a "smooh curve" o he known daa, usng cross-valdaon beween each par of adjacen pons o se he degree of smoohng and esmae he mssng observaon by he value of he splne. Whle splnes can be of any degree, cubc splnes are he mos popular. We wre he equaon for a cubc n he h nerval as follows: 3 F ( x) A ( x x ) B ( x x ) C ( x x ) D (3) Thus he cubc splne funcon we wan s of he form F( x) F ( x) on he nerval x, ], for,,..., ( n ) [ x and mees hese condons: F ( x) Y,,,..., n and F n ( xn ) Yn, (4) F x ) F ( x ),,,..., n ; (5) ( ' ( ' F x ) F ( x ),,,..., n ; (6) '' ( '' F x ) F ( x ),,,..., n ; (7) 37

41 Equaons (4), (5), (6) and (7) ndcae ha he cubc splne fs o each of he pons, s connuous, and has connuous slope and curvaure (6) and (7) hroughou he regon. If here are ( n ) pons n equaon (3), he number of nervals and he number of F ( x)' s are n. There are hus n mes four unknowns whch are he A, B, C, D } { for,,..., ( n ). We know ha: Y for,,...,( n ) D 3 A ( x x ) B ( x x ) C ( x x ) Y Y 3 A B C Y for,,...,( n ) (8) where x x ), he wdh of he h nerval. ( To relae he shapes and curvaures of he jonng splnes, we dfferenae he funcon (8) and he resul s below: F '( x) 3A B C, (9) F ''( x) 6A B, for,,...,( n ) () If we le S F ''( x ) for,,...,( n ) S 6A ( x x ) B, B () herefore equaon () becomes S B () S 6A ( x 6A x ) B, B (3) 38

42 39 Afer re-arrangng equaons (3) and subsue () no (3): S S A 6 (4) Subsue equaons () and (4) no (8) Y C S S S Y 3 6 As a resul 6 S S Y Y C (5) Invokng he condon ha he slopes of he wo cubcs ha jon a ), ( Y X are he same, we oban he followng from equaon (9): C C x x B x x A Y, ) ( ) ( 3 ' x x where 6 S S Y Y whch s equvalen o equaon (5). In he prevous nerval, from x o x, from equaon (9), he slope a s rgh end wll be: 3, ) ( ) ( 3 ' C B A C x x B x x A Y When we subsue equaon (), (4) and (5) we have: S S Y Y S S S (6) When we smplfy equaon (6) we ge: ) 6( ) ( Y Y Y Y S S S (7)

43 4 If we wre equaon (7) n marx form, we wll have he followng: n S n S * * 3 S S S S n ) n n ( n * * 3 ) 3 ( ) ( ) ( ], [ ], [ * * * * ] 3, [ ] 4, 3 [ ], [ ] 3, [ ], [ ], [ 6 n x n x F n x n x F x x F x x F x x F x x F x x F x x F (8) As we can see from he marx (8) we ge wo addonal equaons nvolvng S and n S when we specfy condons peranng o he end nervals of he whole curve. There are four alernave condons whch we would ofen use and each of hese condons wll make slgh changes o our coeffcen marx. Ths s when we: (a) Make he end cubcs approach lneary a her exremes. ence, S and n S. Ths condon s called a naural splne. Coeffcen Marx : ) ( * * ) ( ) ( ) ( n n n 3 3

44 4 (b) Force he slope a each end o assume specfed values. If A x F ) ( ' and B x F n ) ( ' we use he followng relaons: A he lef end : ) ] [ (. A x x F 6 S S A he rgh end : ) ], [ ( n n n n n n x x F B 6 S S Coeffcen Marx : n n * * ) ( ) ( (c) Assume he end cubcs approach parabolas a her exremes. ence, S S, and n n S S. Coeffcen Marx : ) ( * * ) ( ) ( n n n

45 4 (d) Take S as a lnear exrapolaon from S and S, and n S as a lnear exrapolaon from n S and n S. Only hs condon gves cubc splne curves ha mach exacly o ) (x F when ) (x F s self a cubc. We use he followng relaons: A he lef end: S S S S S S S ) (, A he rgh end: n n n n n n n n n n n n n S S S S S S S ) (, (Noe: Afer solvng he se of equaons, we have o use hese relaons agan o calculae he values S and n S.) Coeffcen Marx : n n n n n n n n 3 3 ) )( ( * * ) ( ) ( ) )( (

46 Afer he S values are obaned, we ge he coeffcens A, B, C and cubcs n each nerval and we can calculae he pons on he nerpolang curve. (Curs F. Gerald & Parck O. Whealey 994) D for he Example 3. Esmae mssng values usng cubc splne: We are gong o esmae he me seres 5 f ( ) e 3 E for several 5 mssng values where { E } s a sequence of ndependen dencally dsrbued normal random varables wh mean zero and consan varance. f() True Value f() (3.78) mssng 4 (6.) mssng 5 (7.569) mssng (.733) mssng (6.88) mssng (3.88) mssng Table 3.3. Esmae mssng values usng cubc splne. 43

47 Frsly, we have o calculae he wdh of h nerval : For a naural cubc splne, we used end condon and solve S S S S 4 S S S S S S 4 S S , gvng S, S.35769, S , S , S , S 5.36, S and S 7. Usng hese S s, we compue he coeffcens of he ndvdual cubc splnes. For nerval [,6], we have A -.797, B.7885, C.379 and D.85. f 3 ( ).797( ).7885( ).379( ).85 For nerval [9,3], we have A 5.88, B 5.686, C and D f 3 ( ).88( 9).686( 9).6695( 9) 8.68 Fgure 3. show he esmaed mssng values usng cubc splne curve. 44

48 Curve Fng (cubc splne curve) 5 4 f() 3 True Values Esmaed Values Fgure 3. Esmae mssng values usng cubc splne. In example 3., we use f ( ) o fnd he followng: f (3).656 (rue ) f (4) 5.3 (rue 6.7) f (5) 8.73 (rue ) Also, we use f 5 ( ) o fnd: f 5 (). (rue.7359) f 5 () 4.83 (rue 6.879) f 5 () 3.78 (rue ) Once agan wh he help of modern echnology, hs approach s easy o follow and very quck o produce resuls. I s a popular approach as cross-valdaes beween each par of adjacen pons o se he degree of smoohng. In some cases, hs s more accurae han oher nerpolang polynomal approaches. As wh all numercal analyss mehods, does no examne he naure as well as he purpose of he daa se. Whle hs s no a problem for he less complex daa se, unrealsc esmaons could occur n some cases. When usng he mehods above o analyse a se of daa, s no requred ha he user have a full background undersandng of he me seres analyss and herefore hose esmaons could be less convncng a mes. 45

49 CAPTER 4 Box-Jenkns Models for Mssng Values 4. Abou hs Chaper In our socey, daa analyss s an exremely mporan process. Through daa analyss, s possble o make decsons based on facs. Tme seres analyss s a specfc ype of daa analyss; we mus realze ha successve observaons are usually no ndependen and ha he analyss mus ake no accoun he me order of he observaons. In chaper 3 we have menoned deermnsc models, a me seres ha can be predced exacly o s behavour. owever, mos of he me seres are realsaons of sochasc models. Fuure values are only parly deermned by he pas values, so ha exac predcons are mpossble. We mus herefore consder fuure values as realsaons from a probably dsrbuon whch s condoned by knowledge of pas values. ence, he experse and experences of he analys play an mporan role when analysng daa. Two mporan ypes of sochasc models used are saonary and nonsaonary models and hese are dscussed below. 46

50 4. Saonary Models Frsly, le us look a saonary me seres from an nuve pon of vew. If he propery of one secon of he daa s much lke hose of any oher secon hen we can call he seres saonary. In oher word, he seres has no sysemac change n mean and varance. In addon, any perodc varaons mus also be removed. The saonary me seres model shows ha values flucuae unformly over a se perod of me a a fxed level. The fxed level over whch flucuaes s generally he mean of he seres. Mos of he probably heory of me seres s concerned wh saonary me seres, and for hs reason me seres analyss ofen requres one o urn a nonsaonary seres no a saonary one so as o use hs heory. (Chafeld, 989 p.) Mahemacally, a me seres ARMA process of order (p,q) can be defned as follows: X X... θ E... p X p E θe q q where E (whe nose) s a sequence of ndependen and dencally dsrbued normal random varable wh mean zero and varance σ z, and he auoregressve and movng average parameers respecvely. ' s and s θ are called he j ' A me seres s sad o be srcly saonary f he jon dsrbuon of X ),..., X ( ) s ( n he same as he jon dsrbuon of X ( τ ),..., X ( τ ) for all, τ. In oher n,... n words, shfng he me orgn by an amoun τ has no effec on he jon dsrbuons, 47

51 whch mus herefore depend only on he nerval beween..... The above, n defnon holds for any value of n. (Chafeld, 3 p.34). More nformaon on saonary models can be found n Chafeld 3 p.34 and 35. The followng are wo useful ools avalable o help denfy dfferen me seres models: The Auocorrelaon funcon (ACF) whch s one of he mos mporan ools n he denfcaon sage for buldng me seres models. I measures how srongly me seres values a a specfed number of perods apar are correlaed o each oher over me. The number of perods apar s usually called he lag. Thus an auocorrelaon for lag s a measure of how successve values are correlaed o each oher hroughou he seres. An auocorrelaon for lag measures how seres values ha are wo perods away from each oher are correlaed hroughou he seres. The auocorrelaon value may range from negave one o posve one, where a value close o posve one ndcaes a srong posve correlaon, a value close o negave one ndcaes a srong negave correlaon, and a value close o ndcaes no correlaon. Usually, a se of auocorrelaon values would be compued for a gven me seres correspondng o a range of lags values.e.,, 3 ec. The auocorrelaon beween X and X s defned as: m ρ Corr( X, X ) m cov( X var( X, X m ) var( X ) k ) 48

52 The Paral-auocorrelaon funcon (PACF) s anoher mporan ool, smlar o auocorrelaon funcon. Wh PACF a se of paralauocorrelaon values would be compued for a gven me seres correspondng o a range of lag values whch are used o evaluae relaonshps beween me seres values. Paral-auocorrelaons values also range from negave one o posve one, and are very useful n some suaons where he auocorrelaon paerns are hard o deermne. Boh auocorrelaons and paral-auocorrelaons are usually dsplayed eher as a able of values or as a plo or graph of correlaon values, called a correlogram. Correlograms are specal graphs n whch auocorrelaons or paral-auocorrelaons for a gven me seres s ploed as a seres of bars or spkes. I s probably he mos wdely used mehod of vsually demonsrang any paerns ha may exs n he auocorrelaons or paralauocorrelaons of a saonary me seres. As a resul, plays an mporan role n denfyng he mos approprae model for me seres daa. 4.3 Non-saonary Models A me seres s non-saonary f appears o have no fxed level. The me seres may also dsplay some perodc flucuaons. Mos of he me seres ools only apply o saonary me seres. In oher words, we have o examne he daa and ry o ransform any non-saonary daa se no a saonary model. There are wo man ypes of ransformaons o make he seres saonary. Frsly, he daa can be ransformed hrough dfferencng f he sochasc process has an unsable mean. Ths ype of 49

53 ransformaon s used for he purpose of removng he polynomal rend ha s exhbed by he daa. The logarhmc and square roo ransformaons are specal cases of he class of ransformaons called he Box-Cox ransformaon whch can be used o nduce saonary. These ransformaons are used f he seres beng examned has a non-consan mean and varance and resuls n a sragher curve plo (Box e al.,976, pp.7-8, 85). 4.4 Box-Jenkns Models In general, he Box-Jenkns models provde a common framework for me seres forecasng. I emphaszes he mporance of denfyng an approprae model n an neracve approach. In addon, he framework can cope wh non-saonary seres by he use of dfferencng. Box-Jenkns mehod derves forecass of a me seres solely on he bass of he hsorcal behavour of he seres self. I s a unvarae mehod whch means only one varable s forecas. The deas are based on sascal conceps and prncples ha are able o model a wde specrum of me seres behavour. The basc Box-Jenkns models can be represened by a lnear combnaon of pas daa and random varables. The sequence of random varables s called a whe nose process. These random varables are uncorrelaed and normal wh mean zero and consan varance. 5

54 4.4. Auoregressve Models (AR models) The mos basc AR model s he model ha conans only one AR parameer as follows: X E X where { E } s a sequence of ndependen dencally dsrbued normal random varable wh mean zero and varance σ z. I means ha any gven value X n he me seres s drecly proporonal o he pervous value X plus some random error E. As he number of AR parameers ncrease, X becomes drecly relaed o addonal pas values. The general auoregressve model wh p AR parameers can be wren as: X E X X... p X p where,... are he AR parameers. The subscrps on he ' s are called he orders of, he AR parameers. The hghes order p s referred o as he order of he model. ence, s called an auoregressve model of order p and s usually abbrevaed AR( p ). (Box e al.,976, pp.7-8, 85). The values of whch make he process saonary are such ha he roos of ( B) le ousde he un crcle n he complex plane where B s he backward shf operaor such ha: j B x x and j ( B). p B... pb (Chafeld, 989, p.4) 5

55 4.4. Movng Average Models (MA models) The second ype of basc Box-Jenkns model s called he Movng Average Model. Unlke he auoregressve model, he Movng Average Model parameers relaes wha happens n perod only o he random errors ha occurred n pas me perods,.e., o E, E,... ec. Basc MA model wh one parameer can be wren as follow: X θ E E I means ha any gven X n he me seres s drecly proporonal only o he random error E from he prevous perod plus some curren random error E. General movng average wh q MA parameers can be wren as: X E θ E θ E... θ E p p where θ,... are he MA parameers. Lke he auoregressve model, he subscrps on, θ he θ s are called he orders of he MA parameers. The hghes order q s referred o as he order of he model. ence, s called a movng-average model of order q and s usually abbrevaed MA( q ) Auoregressve Movng Average Models (ARMA models) In mos case, s bes o develop a mxed auoregressve movng average model when buldng a sochasc model o represen a saonary me seres. The order of an ARMA model s expressed n erms of boh p and q. The model parameers relaes o wha 5

56 happens n perod o boh he pas values and he random errors ha occurred n pas me perods. A general ARMA model can be wren as follow: X E X... X ) ( θ E... θ E ) (9) ( p p q q Equaon (9) of he me seres model wll be smplfed by a backward shf operaor B o oban ( B ) X θ ( B) E where: B s he backward shf operaor such ha: j B x x and j ( B) and p B... pb θ ( B) θ B... θ. B q q Auoregressve Inegraed Movng Average (ARIMA) Models Generally, mos of he me seres are non-saonary and Box-Jenkns mehod recommends he user o remove any non-saonary sources of varaon hen f a saonary model o he me seres daa. In pracce, we can acheve saonary by applyng regular dfferences o he orgnal me seres. These models are wren n he same way as he basc models, excep ha he dfferenced (saonary) seres W s subsued for he orgnal seres X. In order o express ARIMA models, we have o undersand he use of dfference operaor. For example, he frs dfference of a seres can be expressed as: X X W () 53

57 owever, he use of a symbol s used o smplfy equaon (). The frs dfferences of he seres W X X could hen be wren as: If we ake he second consecuve dfference of he orgnal seres X, he expresson would be defned as: W X ( X X ) ( X ) In general, he d-h consecuve dfferencng would be expressed as 983, pp.5-53). d X (Vandaele, General form of ARIMA can be wren as follow: W... θ W... pw p E q Eq () By usng he dfference operaor, we may wre he ARIMA model () as follow: ( B ) W θ ( B) E or ( θ E. d B )( B) X ( B) In ARIMA models, he erm negraed, whch s a synonym for summed, s used because he dfferencng process can be reversed o oban he orgnal me seres values by summng he successve values of he dfferenced seres. (off, 983, pp.6) 54

58 4.5 Box-Jenkns Models and Mssng Values Box-Jenkns mehod s a popular choce for analys o use on me seres modellng. I provdes he followng advanages: ) The generalzed Box-Jenkns models can model a wde varey of me seres paerns. ) ) v) There s a sysemac approach for denfyng he correc model form. I provdes many sascal ess for verfyng model valdy. The sascal heory behnd Box-Jenkns mehod also allows he mehod o use sascal measuremens for measurng he accuracy of he forecas. All he basc Box-Jenkns models have recognzable heorecal auocorrelaon and paral-auocorrelaon paerns. To denfy a Box-Jenkns model for a gven me seres model, we compue he auocorrelaon and paral-auocorrelaon (usually by he use of correlogram) and compare hem wh he known heorecal ACF and PACF paerns. A summary of he heorecal ACF and PACF paerns assocaed wh AR, MA, and ARMA models can be found n off p.7. owever, f mssng values occurred whn he me seres daa hen s mpossble o compue any of hese values. For hs reason, Box-Jenkns mehods may no be he bes choce and hey canno be appled drecly o me seres whch ncludes he mssng values. To apply Box-Jenkns mehod o me seres daa wh mssng values, we have o consder he followng: 55

59 ) ow ofen do he mssng values occur? ) ) Where are he mssng values locaed n he me seres? Do we have suffcen daa before, afer or beween he mssng values o apply Box-Jenkns mehod o he remanng daa? I s possble o ndrecly apply Box-Jenkns mehod o me seres wh mssng values. In chaper 3, numercal approaches have been used o esmae mssng values whn a me seres. The accuracy of resuls s manly dependen on he ype of me seres. Once mssng values have been flled wh esmaes, Box-Jenkns mehod can hen be appled. Alernavely, f suffcen daa exss before, beween or afer mssng values, Box- Jenkns mehod can be appled o secons of daa ha do no conan mssng values. In hs case, mssng values can be obaned as forward or backward forecas values are used or combnaons of each are appled. 4.6 Leas Square Prncple Applyng he leas square prncple o me seres wh mssng values s a basc approach whch can be ncorporaed no ARIMA modellng. As oulned n Ferrero 987 hs mehod s nended o fnd mssng values for saonary me seres. By he use of dfferencng, can be appled o me seres wh one gap or more provded hey are well separaed and have suffcen daa o oban an ARIMA model. 56

60 Consder ARMA process of order (p,q), he process can be wren n more compac form as follow : X... θ X... p X p E θe q Eq () Then we have o rearrange equaon (8), o oban X... X... p X p E θe θ q Eq (3) Subsung he backshf operaor no equaon (9) p ( B ) X θ ( B) where B) B..., (4) E ( p B θ B) θ B... θ ( B q q and B s he backward shf operaor BX, B X X X Rearrangng equaon (), we have E ( B) θ ( B) X Furhermore, we can smplfy he equaon o E ( B) X where j ( B ) B B... j B (5) j From equaon (), we can express he equaon as follow: E X X X... 57

61 To calculae he sum of squares of errors over a L pons he followng formulas are used: SS L L E SS L ( L X ) In order o mnmze he sum of squares wh respec o he mssng values he followng seps are requred: Le X s be mssng SS Z s L s s ( X ) (6) If we le L and subsue j s no equaon (6), hen j ( X s j ) j - j Es j ( B ) Es and j E s j ( B) X s j s j E E (7) s j From equaon (7), s possble o oban he leas squares equaon for approxmang he mssng observaons. The equaon becomes: ( B ) ( B) X s (Ferrero, O. 987, pp.66) 58

62 Example 4.6. Esmae mssng value n AR() process by usng Leas Square Prncple X. E 7 X (. 7B) X E (. 7B) ( B) ; (. 7B ) ( B ) ( B ) ( B) X s (. 7B)(. 7B ) X (. 7B. 7B. 49) X ( B. 7B ) X (8) From equaon (8), we can express he equaon as follow... 49X 7 X 7 X 7 7 ence, Xˆ.. X X where Xˆ s he esmae for he mssng value. The daa below s obaned by applyng he AR() process wh E beng a purely random process where he mean equals zero and he varance equals one. X Esmaed 6 (-.89) Mssng X ˆ X X Table 4.6. AR() me seres model wh mssng value when 6.

63 Leas Square Esmaon Esmae daa Orgnal daa Fgure 4. Esmae mssng value n AR() process by usng Leas Square Prncple. Example 4.6. Esmae wo mssng values n AR() process by usng Leas Square Prncple X Esmaed 6 (-.89) Mssng (-.845) Mssng X ˆ Xˆ.7.49 ˆ.7 (.49 X ).7.49 ( ) X Table 4.6. AR() me seres model wh mssng value when 6 and 7. ˆ Leas Square Esmaon Esmae daa Orgnal Daa Fgure 4. Esmae wo mssng values n AR() process by usng Leas Square Prncple. 6

64 The examples above show how he Box-Jenkns mehods are used o esmae mssng values. owever, hs mehod s only suable for lower order of ARMA models when he calculaons are carred ou manually. For hgher order ARMA (p.q) models s more complcaed o calculae he mssng values and consequenly mahemacal processes are usually carred ou by usng approprae sofware packages. 4.7 Inerpolaon Inerpolaon s a concep whch can be appled o me seres daa o deermne mssng values. Frs of all, we have o deermne an approprae ARIMA me seres model for he known daa. Accordng o he poson of he mssng observaon we can hen calculae he approprae weghngs for boh he forward and backward forecas. Fnally, usng a lnear combnaon of he forward and backward forecas wh he weghngs already calculaed, we are able o predc he values of he mssng observaons under he gven condons. Le us consder he ARIMA (p, d, q) (auoregressve negraed movng average) model ( θ d B )( B) X ( B) E where: B s he backward shf operaor such ha B j x x, j ( B)..., p B p B θ ( B) θ..., such ha ( B ) θ ( B). q B θ q B { E } s a sequence of ndependen dencally dsrbued normal random varable wh mean zero and varance σ z. 6

65 Gven he observaons x,... he mnmum mean square error for forward forecas of x q l ( l ) a me q s q, x q eˆ l q ( l) ψ jeq l j, ψ j where ψ j are defned by: θ ( B ) ( ψ ). d B ψ B...) ( B)( B In addon, he backward represenaon of our ARIMA (p,d,q) model can be expressed as: ( θ d F )( F) x ( F) c where: F s he forward shf operaor such ha j F x x j, { c } s a sequence of ndependen dencally dsrbued normal random varable wh mean zero and varance σ ( σ σ ). c z c Now, gven xq m j ( j ) he mnmum mean square error for backward forecas of x a me q m s ql m l s ( m l) jcq l j s q m j ~ e ψ, For saonary models, an opmal esmae Xˆ l of x q l may be obaned by fndng he lnear combnaon of xˆ ( l) and ~ ( m l). The esmae based on he lnear q x s combnaon of xˆ ( l) and ~ x s ( m l) may be gven as ˆ ' ( xˆ q,..., xˆ ) wh q X q m 6

66 xˆ xˆ ( l) w ~ x ( m q l w l q l s where l) w l represen he weghs ye o be calculaed for he forward forecas, w l represen he weghs ye o be calculaed for he backward forecas, j ( l) xˆ q ( l) π x, j q j ~ ( m ) xs ( m ) l j j l π x, q m j l ( l ) ( lh) j π j l π hπ j h ( ) π and π π, j,,... j j In order o calculae he weghs ( w and l w l ), we use he followng equaon: E [ x w xˆ ( l) w ~ x ( m l)] q l l q l s w lσ wlσ w lwlσ w l ( w l wl ) σ (9) w l ( w l wl ) σ ( w l wl ) σ x where σ x v xq ) ( j ( l ψ ) σ, j z l ( j x l j σ v eˆ) ( ψ ) σ z σ cov(ˆ, e q ), ml ~ ) σ ( ( ) cov( ~ v e ψ j σ z σ e, xq l ), e ˆ ˆ ( l), e q j e~ e~ s ( m l), ' ' σ ψ D ψ cov(ˆ, e ~ ), e 63

67 ' ( l and ψ ψ... ψ ), ' ( ml ψ ψ... ψ ), D ( d j ) s an l ( m l) marx wh d j E( zq l cq l j ) ( π h ) ψ h j, π, h (,,..., l ; j,,..., m l; l,,..., m) By akng paral dervaves on equaon (9) wh respec o w and w l can be l shown ha he mnmzng values of w and w l are gven by l w l L ( L L ) /, w l L ( L L ) / where L ; (3) L L L ; (3) σ Z σ L ; and (3) σ Z σ L ; (33) σ Z σ σ σ (Abraham B., 98, pp.646) As a resul, we have he followng equaons: w ( σ σ )[( σ σ ) ( σ σ σ σ x x x l (34) ( σ x σ )( σ x σ ) ( σ x σ σ σ ) )] w ( σ σ )[( σ σ ) ( σ σ σ σ x x x l (35) ( σ x σ )( σ x σ ) ( σ x σ σ σ ) )] 64

68 owever n he non-saonary cases he varance s no fne and creaes a problem n he mean square. In Abraham (98) he proposed soluon o hs problem s: ( σ σ ) w l, (36) ( σ σ σ ) w l wl To gan furher undersandng of he prncples dealed n hs arcle, he above approach s appled o some smple me seres models. When only one observaon s mssng, he forward and backward forecass receve he same weghng. In addon, f he model s non-saonary wh dfference equal o (d), he esmae of he mssng value s an average of he forward and backward forecass of he mssng observaons. When a me seres has one observaon mssng (m, l ), we wll have he followng varance and sandard devaons. ence he weghng of he equaon would become equaon (38). σ, σ σ z σ ( πhψ h ) σ z (37) h w w (38) 4.7. Weghng for general ARIMA models : For hs me seres model, we have he followng sandards equaon. ( B)X θ(b) (39) E 65

69 Rearrange equaon (39), we can have he followng equaons: X θ(b) ( B) E ( ψ B ψ B...) E Also, equaon (39) can be rearranged as E ( B) θ ( B) X ( π B π B...) X When mssng observaon equal o (m), we can oban he varance as follow: σ l v(ê) ( ψ j ) j σ z (4) j ( ψ σ ) z ( ) σ z σ z σ ~ ml v( e) ( ψ j ) j σ z (4) j ( ψ σ ) z ( ) σ z σ z 66

70 Example 4.7.: Calculae weghngs for ARIMA (,,) model The model s expressed as ( B)X E (4) In addon, when we rearrange equaon (4) no he followng equaon: X B E ( B B...) E and now, le k ; ψ ;... ; ψ k (43) ψ Also, equaon (4) can be expressed as E B X Leng π π ;... ; ( k ) ; π k > ence, we can subsue from (43) o (4) and oban he followng: σ x v(x ql ) ( ψ j ) j σ z σ x v(x ql ) ( j ) j σ z 4 (...) σ Z σ Z (44) 67

71 68 From equaon (4) and (4) we have z σ σ σ As we have saed from equaon (3) ha X L σ σ We subsue equaon (44) no equaon (3) and obaned he followng: L Z σ Z σ L Z σ (45) From equaon (3) x L σ σ Subsue equaon (44) no equaon (3) and obaned he followng: L Z σ Z σ L Z σ (46) Also, from equaon (33) x L σ σ σ σ Now, subsue equaons (37), (4), (4) and (44) no (33) and we oban ) ( L Z Z h z h h Z σ σ σ ψ π σ Z z h h h z Z σ σ ψ π σ σ

72 69 Z z z Z σ σ σ σ ) ( ) ( σ Z 4 σ Z (47) Snce, L L and equaon (3) saed he followng: L L L We can subsue equaon (45), (46) and (47) no (3) 4 ) ( ) ( Z Z σ σ ( ) 8 4 ) ( Z σ Now, equaon (37) can be rewren as : ( ) ) ( Z Z Z Z w l σ σ σ σ ( ) ) ( Z Z Z σ σ σ

73 4 ( ) ( ) ( ) ( )( ) In order o calculae w, we have o refer o equaon (35) l w l ( σ ( σ x x σ )[( σ σ )( σ x x σ ) ( σ σ ) ( σ x x σ σ σ σ σ σ )] ) As we already know ha σ σ σ z Therefore: w l w l Example 4.7.: Calculae weghngs for ARIMA (,,) model In hs example, he me seres model can be expressed as ( ) E X θb (48) and from he me seres model, we can esablsh he followng : ψ θ ; ψ ;... ; ψ k (k > ) 7

74 Also, equaon (48) can also be expressed as E θb X ence, X ( θb θ B...) E and π θ π θ ;... ; ; π θ k k From equaon (4), we have he followng: σ x v(x ql ) ( ψ j ) j σ z x ( ) σ σ θ (49) Z From he prevous example, we already esablshed ha σ σ σ z and equaon (3) s L σ X σ If we subsue equaon (49) no (3) hen he equaon would become L θ ( θ ) σ Z σ σ Z Z Also, equaon (3) saed ha L σ x σ 7

75 To fnd L we have o subsue equaon (49) no (3) and oban L ( θ ) σ Z σ θ σ Z Z From equaon (33), L σ x σ σ σ Subsue equaon (4) and (49) no (36) L ( θ ) σ ( π ψ σ σ σ Z h h ) h ( θ ) σ σ π ψ σ σ Z z h h z Z h ( θ ) σ σ θ σ σ Z z z Z ( θ θ ) σ Z and from equaon (3) we can esablsh he followng: ( θ ) ( σ ) Z z Z Z Equaon (34) s rewren as w l ( σ ( σ x x σ )[( σ σ )( σ x x σ ) ( σ σ ) ( σ x x σ σ σ σ σ σ )] ) L ( L L ) ( θ ) σ Z ( θ ) σ Z ) ( θ ) ( σ ) Z 7

76 From equaon (35) we know w l ( σ ( σ x x σ )[( σ σ )( σ x x σ ) ( σ σ ) ( σ x x σ σ σ σ σ σ )] ) and σ σ σ z Therefore w l w l Example 4.7.3: Calculae weghngs for ARIMA (,,) model For hs me seres model, we have he followng equaon: ( B) X ( θb) E (5) and equaon (5) can be rearranged as X θb B E ( B B...)( θb) E (... B θb B θb B θb...) E 3 3 (... ( θ ) B ( θ ) B ( θ ) B...) E ence, k k ( θ ) ; ψ ( θ ) ;.. ; ψ θ ( k ) ψ (5) k > Equaon (5) can also be rewren as E B θb X ( θb θ B...)( B) X 73

77 74 ( ) X B B B B B B θ θ θ θ θ ( ) ( ) ( ) ( ) X B B B θ θ θ θ θ ence, ) (k ;.. ; ; k k k > θ θ π θ θ π θ π Subsue equaon (5) no equaon (4), we have he followng z j j q x ) ( ) (x v σ ψ σ l ( ) ( ) ( ) ( ) 3... Z σ θ θ θ ( ) ( ) ( ) ( ) 4... Z σ θ θ θ ( ) ( ) ( ) [ ] { } 4... Z σ θ θ θ ( ) ( ) [ ] { } 4... Z σ θ ( ) Z σ θ ) ( Z σ θ (5) As z σ σ σ and X L σ σ Equaon (3) would become he followng: ) ( L Z σ Z σ θ ) ( σ Z θ

78 75 Also, equaon (3) x L σ σ would become ) ( L Z σ Z σ θ ) ( σ Z θ Consder : ( )( ) ( )( ) ( )( ) h h h θ θ θ θ θ θ θ θ ψ π ( ) ( ) ( ) ( ) ( )... θ θ θ θ θ θ θ ( ) ( ) ( )... θ θ θ θ θ ( ) ( )... θ θ θ ( ) θ θ ( ) θ θ As we know equaon (33) s x L σ σ σ σ, we can subsue equaon (37) and (5) no (33) ( ) ) ( L Z Z h z h h Z σ σ σ ψ π σ θ ( ) Z z h h h z Z σ σ ψ π σ σ θ ( ) ( ) Z z z Z σ σ θ θ σ σ θ ( ) ( ) Z σ θ θ θ

79 76 ( ) ( ) ( ) ( ) ( )( ) σ Z θ θ θ θ ( ) ( ) ( )( ) σ Z θ θ θ ( ) ( ) ( )( ) σ Z θ θ θ and from equaon (3) ( ) ( ) ( ) ( )( ) ) ( ) ( Z Z σ θ θ θ σ θ ( ) ( ) ( ) ( ) ( ) ( ) 4 4 ) ( ) ( Z σ Z θ θ θ σ θ ( ) ( ) ( ) ( ) ( ) ( ) 4 4 ) ( σ Z θ θ θ θ θ Usng equaon (34) o fnd l w, we have ( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( ) ( ) 4 4 z Z Z Z w σ θ θ θ θ θ σ θ θ θ σ θ σ θ l ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) θ θ θ θ θ θ θ θ θ θ ( ) ( ) ( ) ( ) ( ) θ θ θ θ θ θ θ θ θ θ

80 ( σ x σ )[( σ x σ ) ( σ x σ σ σ )] As we know ha w l ( σ σ )( σ σ ) ( σ σ σ σ ) x x x and Therefore: σ σ σ z w l w l Example 4.7.4: Calculae weghngs for ARIMA (p,,q) model: For hs class of model ( B)( B) X θ( B) E In any non-saonary case w ( σ σ ) ( σ σ σ l, ) when mssng observaon equal o (m), and we already know σ σ σ z Therefore: w l ( σ z σ ) ( σ σ σ z z ) w l Also w l w l Therefore: w l 77

81 The weghngs for he above cases are summarzed n he followng able: ARIMA model Weghngs w l and w l can be deermne by (,,) ( ) (,,) (,,) (p,,r) ( θ) ( θ) Table 4.7. Weghngs w l and w l for mssng value n ARIMA model. When wo observaons are mssng, he wo values of forward forecas should be unequally weghed and hs also should apply o he wo values of backward forecas. owever, he frs value of he forward forecas and he second value of he backward forecas (also he second value of he forward forecas and he frs value of he backward forecas) should be weghed equally. (Abraham B., 98, pp.647) For he suaon where wo observaons are mssng (m), we have l, σ σ z σ ψ ) ( σ z σ h h h h )} h h {( ψ π ) ψ ( ψ π ψ σ z l, σ ψ ) ( σ z σ σ z σ h h h h )} h h {( ψ π ) ψ ( ψ π ψ σ z 78

82 The weghngs for hese cases are summarzed n he followng able: ARIMA model l Weghngs w l and l w can be deermne by w l w l (,,) ( ) ( 4 ) ( 4 ) w w (,,) ( θ)( θ) ( θ)( θ) ( ( θ) θ)( θ) w w ( )( 3 ) - w (,,) ( )( 3 ) - w w θ - w 3 θ (,,) - w w Table 4.7. Weghngs w l and w l for mssng value n ARIMA model. In 979, an alernave approach whch produced he same resuls as Abraham was developed by Evnd Damsleh o fnd he opmal lnear combnaon of he forward and backward forecas for mssng values n an ARIMA me seres. Abraham was unaware of Damsleh resuls a he me of wrng hs paper. The mehod s based on he dea of usng cross covarance o generae parameers for based lnear combnaon for forward and backward forecas wh a mnmum error. The auhor saes he mehod requres he daa o have suffcen lengh for parameers and he model o esmae correcly. Wh shorer daa lengh he problem s more dffcul and s no dscussed 79

83 whn hs paper. The advanage of hs mehod s ha provdes a sep by sep algorhm for he reader o follow. Usng he noaon of hs hess, he algorhm as presened by Damsleh 979 s expressed as follows: d ) Calculae he opmal forecas e ˆq ( l) usng ( B )( B) X θ ( B) E and he opmal backforecas ~ ( l d e s m ) usng ( F)( F) x θ(f)c. ) Calculae he coeffcens π j of j B n he polynomal expanson of θ ( B) ( B) for j,, K, max( l, m l). ) Calculae he cross covarance funcon γ ( j) for j,, K, m l, usng p ( ) j θ F B γ ae ( j s) ± α jσ, p j, q j B θ ( ) θ F γ ae ( j s) ± α jσ, j q, Ths gves p q lnear equaons n p q unknowns γ ( p s), γ ( p s ), K γ ( q s ), γ ( q s), whch can be solved. The ae ae ae soluons wll provde sarng values for he dfference equaon θ ( F) ( B) γ ae ( j s) ± α jσ, j K,, K where α j s he coeffcen of ae ae j B n ( B) ( B ), θ gven by he followng: j α j θ j mn( p j, q) mn( p, q) mn( p, q j) θ j θ θ j j < p p j j j q j > q 8

84 v) Calculae v (eˆ) from v (e~ ) from l ( ψ j ) σ z j m l ( ψ j ) σ z j l ml ψ ψ jγ ae ( j) k,, and σ where γ (k) s he cross covarance funcon beween { a } and { } e. v) If { X } s saonary hen calculae c and d as he soluons o ae Ex c Ex Ex ( σ ) ( σ σ σ ) σ d Ex c Ex d Ex ( σ σ σ ) ( σ ) σ In he non-saonary case, we calculae c from σ σ c and d c. σ σ σ v) The opmal beween-forecas of x r l s hen gven by c ( eˆ ( l ) Ex) d ( e ~ ( m ) Ex) q q m l 8

85 CAPTER 5 Sae Space Modellng for Mssng Values 5. Abou hs chaper Recenly, sae space modellng has become a popular approach n me seres analyss. The man advanage of hs approach s ha can adap o dfferen me seres models such as he Box-Jenkns ARIMA and Srucural Tme Seres. Ths approach emphasses he noon ha a me seres s a se of dsnc componens. Thus, we may assume ha observaons relae o he mean level of he process hrough an observaon equaon, whereas one or more sae equaons descrbe how he ndvdual componens change hrough me. In sae space modellng, observaons can be added one a a me and he esmang equaons are hen updaed o produce new esmaes. In hs chaper, we wll nvesgae he above approach and deermne he mos suable me seres model(s) for he use of hs echnque. 5. Sae Space Models Sae space modellng s a general approach whch orgnally developed by conrol engneers for applcaons concernng mul-sensor sysems such as rackng devces. I assumes ha he sae of he process summarzes all he nformaon from he pas ha s necessary o predc he fuure. 8

86 Suppose ha he observed value of a me seres a me, X, s a funcon of one or more random varables d θ θ,..., whch also occur a me bu whch are no observed. These varables are called he sae varables and we can represen hem by d T ) vecor θ ( θ... θ. The smples model s o assume ha X s a lnear funcon of θ whch s shown as: X d d h θ hθ... h θ ε (53) In hs case h s a consan parameer and ε represens he observaon error. The equaon (53) above s called measuremen or observaon equaon and can be wren as a marx noaon whch s shown below: X T θ ε In hs case T s a d-dmensonal row vecor of parameer. Alhough he sae vecor θ s no drecly observable, s assumed ha we know how θ changes hrough me and we are able o use he observaon on X o make nferences abou θ. The updaed equaon s a θ Gθ Kη. In he equaon, G s d d marx of parameers and η s a whe nose vecor wh covarance marx W. When we consder he sae varables a me whch s represened by d T ( θ... θ ) θ, ha s a recursve process dependen on he prevous sae of he varable θ θ,., Consequenly a smple model for he sae vecor could be θ G θ where G s a d d marx of parameer. If he sae dmenson s he model s θ (54) Gθ Gθ 83

87 θ (55) Gθ Gθ From equaons (54) & (55), we can creae he followng coeffcen marx G G G G G As usual, we have o buld random nose erms no each equaon o make he model more flexble. To acheve hs flexbly, we nroduce he random nose vecor η no our model. For nsance, where he sae dmenson s and here are ndependen random varables of nose wh zero mean ogeher wh he nose vecor of T ( η, η ) η hen our model becomes: θ K η K η Gθ Gθ θ Gθ Gθ where K j are addonal parameers. In general, he equaon θ Gθ Kη s known as he sysem or ranson equaon. As G descrbes he evoluon from one sae o he nex s called he ranson marx, and K s called he sae nose coeffcen marx for obvous reasons. (Gareh Janacek and Louse Swf 993) The advanages of usng sae space modellng s ha s possble o pu many ypes of me-seres models no he sae space formulaon. Ths s llusraed by he followng hree examples. 84

88 Example 5.. A smple sae space model X (.5.5)θ ε And θ.7.8 θ η.5 wh var( ε ).5 and var ( η ) 4. Noe : The marx, marx G, marx K all conan parameers of he model. If hese parameers are unknown hey mus be esmaed. I s mporan o remember ha, n general, he sae varables are no observed. Noce also ha all he nformaon n he pas of he seres { X } whch can nfluence X mus be conaned only n he sae varables θ. The sae space modellng s assumed o have he followng assumpons (arvey, 989, pp.5-6, pp. -) : a) E( X ) and he auocorrelaons of and b) ε s a zero mean observaon error wh varance nose wh varance marx X are ndependen of for weak saonary; Q. Ths s shown as: Z and η s a vecor whe ε ( ss) Z ~ NID, η Q Two furher assumpons are specfed by (arvey, 98, pp. -) for he sae space sysem: 85

89 c) The nal sae vecor, θ, has a mean of a and a covarance marx P, ha s, E ( θ ) a and Var ( θ ) P. d) The observaon errors ε (ss) and η are uncorrelaed wh each oher n all me perods, and uncorrelaed wh he nal sae, ha s, E ε η ), for all s, for,..., N ( ( ss) and E ( α ), E ( η α ) for,..., N ε ( ss ) Example 5.. : Usng sae space modellng for he ARMA (p,p-) model. The ARMA (p,p-) model represenaon can be obaned by changng he elemens of marx K wh he MA(p-) coeffcens. In hs case, (, β β ) K, and T (,,). Ths sae space ARMA model represenaon can be verfed as above by repeaedly subsung he elemens n he sae vecor. When ARMA s ncorporaed no sae space model framework consderaon mus be gven o ARMA process of order (3,) and s shown as: X α X α X α 3 X 3 Z βz β Z Snce has only one equaon and one nose seres, does no seem compable wh he sae space model. 86

90 87 owever, le us assume he followng:, ) ( X θ X η β β α α α θ 3 (56) We can represen he sae vecor by T X X X X ),, ( 3. When we begn a he boom row of (56), he sae equaon s X X η β α 3 3 The second row (56), he sae equaon s X X X η β α 3 If we subsue 3 X no X gves: X X X η β η β α α 3 Fnally, when we look a he frs row of (56), he sae equaon s X X X η α Subsue X no X whch gves X X X X η β η η β α α α 3 3 whch s an ARMA model equaon bu n he frs componen of he sae X wh nose erm Z. ARIMA models may also be represened n sae space form. arvey and Phllps(979) derved an exac maxmum lkelhood esmaon procedure for ARIMA models. One

91 advanage of hs approach s he ably o handle mssng observaons snce we may smply updae he esmaes; ha s, we may replace he mssng observaons by s one-sep ahead forecas. Oher han regresson ARMA models, sae space modellng can also represen a rendand-seasonal model for whch exponenal smoohng mehods are hough o be approprae. In order o apply sae space modellng for ARIMA models, we need o know and G parameers n he model equaons and also o know he varances and covarances of he dsurbance erms. The choce of suable values for and G parameers may be accomplshed usng a varey of ads ncludng exernal knowledge and prelmnary examnaon of he daa. In oher words, he use of sae space modellng does no help o ake away he usual problem of fndng a suable ype of model. (C.Chafed 994) Inally, we assume ha such parameers are known; esmaon procedures for hese parameers such as log-lkelhood funcon, maxmum lkelhood funcon and expeced maxmum algorhm can be used accordngly. 5.3 Srucural Tme Seres Models Srucural me seres models are a specfc ype of sae space models. These are modelled as a sum of meanngful and separae componens and are well sued o sock assessmen. 88

92 89 A Basc Srucural Model s represened as he observed value n erms of one or more unobserved componens, called he sae vecor. These componens can be dvded no separae groups. Thus, a s seasonal s S j j η γ γ The srucural model can be represened n a sae space form. Tha s, he onedmensonal sae would be ],...,,,,, [ s γ γ γ γ β μ θ and he sae nose vecor, conssng of uncorrelaed whe nose, would be ] [ (3) () () η η η η. For example, assumng 4 s, he basc srucural model has he followng observaon equaon and ranson equaon. Observaon Equaon [ ] X ε γ γ γ β μ Transon Equaon (3) () () 3 η η η γ γ γ β μ γ γ γ β μ Please noe covarance marx s ),,, ( (3) () () dag η η η. (Janaceck and Swf, 99, pp ).

93 The sae space modellng can hen be appled o deermne he local level, rend and seasonal componens. Whou he seasonal componen, srucural models wll be: X μ ε (57) () μ β η μ (58) () η β β (59) The equaons (57), (58) and (59) above can be referred o as a lnear growh model. The frs equaon s he observaon equaon. The sae vecor [ μ β ] θ, where μ s he local level, whch changes hrough me and β s he local rend whch may evolve hrough me. In sae-space form, he observaon equaon s X μ β [ ] ε and he ranson equaon s μ β μ β η η () (), or θ θ η (Chafeld, 989, p.84). For all he man srucural models, he observaon equaon nvolves a lnear funcon of he sae varables and ye does no resrc he model o be consan hrough me. Raher allows local feaures such as rend and seasonaly o be updaed hrough me usng he ranson equaon. 9

94 5.4 The Kalman Fler In sae space modellng, he prme objecve s o esmae he sae vecor θ. When we are ryng o produce fuure esmaors, he problem s called forecasng or predcon, f we are ryng o updae curren esmaors hen he process s called flerng and f we are ryng o fnd pas esmaors, he process s called smoohng. The Kalman Fler provdes us wh a se of equaons whch allows us o updae he esmae of θ when new observaons become avalable. There are wo sages nvolved n hs process whch are called he predcon sage and he updang sage Predcon Sage Kalman Fler has provded us wh wo predcon equaons. Frsly les consder he updae equaon for sae space model θ Gθ Kη, where η s sll unknown a me, herefore he esmaor for θ can be wren as G θ θ where θ s he bes (mnmum mean square) esmaor for θ and he resulng esmaor as θ. 9

95 Secondly he varance-covarance marx P of he esmaor error of θ s gven by G P T P G KW K T (Janaceck and Swf, 99, pp. 97) 5.4. Updang Sage When he new observaon becomes avalable, he esmaor of θ can be modfed o make adjusmens for he addonal nformaon. Consder he sae space model X T θ ε. Now, he predcon error s gven by ε X θ and can be shown ha he updang equaons are gven by T θ θ K ε and P T P K P where K T P n /[ P σ ] (also called he Kalman gan) (Janaceck and Swf, 99, pp. 97) A major advanage of he Kalman Fler s ha he calculaons are recursve, herefore he curren esmaes are based on he whole pas. Also, he Kalman Fler converges 9

96 farly quckly when here s a consan underlyng model, bu can also follow he movemen of a sysem where he underlyng model s evolvng hrough me Esmaon of Parameers In 965 Schweppe developed an nnovave form of lkelhood funcon whch s based on ndependen Gaussan random vecors wh zero means. The lkelhood funcon s defned as n ' ( θ ) ε ( θ ) Σ ( θ ) ε ( ) ln ( θ ) log Σ θ L Y n The funcon s hghly non-lnear and complcaed wh unknown parameers. In hs case, he mos convenen way s o use Kalman fler recursons. The Kalman fler recursons s a se of recursve equaons ha are used o esmae parameer values n a sae space model. The usual procedure s o se an nal value x and hen use Kalman fler recursons for he log lkelhood funcon and s frs wo dervaves. Then, a Newon-Raphson algorhm can be used successvely o updae he parameer values unl he log lkelhood s maxmzed. 93

97 5.4.4 The Expecaon-Maxmsaon (EM) Algorhm In addon o Newon-Raphson, Shumway and Soffer (98) presened a concepually smpler esmaon procedure based on EM (expecaon maxmzaon) algorhm (Dempser e al 977). I s an algorhm for nonlnear opmsaon algorhm ha s approprae for me seres applcaons nvolvng unknown componens. When we consder he unobserved sgnal process θ and an unobserved nose process ε (ss). Boh processes form he funcon X whch s an ncomplee daa se. Log lkelhood log L ( θ, Ψ) may be based on he complee daa se, or an ncomplee daa se, where he parameers denoed by he marx Ψ are o be esmaed. For he ncomplee daa lkelhood s requred o maxmse a funcon usng one of he convenonal non-lnear opmsaon echnques. In comparson, for he complee daa lkelhood, he maxmsaon echnque s usually very easy, excep for he unobserved values of θ and (ss) ε (Shumway, 988, p. -). The Expecaon-Maxmsaon algorhm was used for esmang he unknown parameers n an unobserved componen model. Consder a general model ha s menvaran as follows: X T θ ε, (ss) θ T θ R η, 94

98 wh a and P are known, and Var ) (η Q s unresrced. If he elemens n he sae vecor are observed for,..., N, he log-lkelhood funcon for he X s and θ s would be: log L( X N N, θ ) log π log h h N ( X θ ) Nn N log π log Q N ( θ T θ * )' Q ( θ Nθ ) n log π log P ( θ a ) P ( θ a ). I follows ha he eraon procedure of he EM algorhm proceeds by evaluang log L E Ψ X N. whch s condonal on he laes esmae of Ψ. The expresson s hen se o a vecor of zeros and solved o yeld a new se of esmaes of Ψ. The lkelhood wll reman he same or ncrease a each eraon under suable condons. I wll also converge o a local maxmum (arvey, 989, p.88). In general, f we have he complee daa se, we could hen use he resuls from mulvarae normal heory o easly oban he maxmum lkelhood esmaons of θ. In he case of ncomplee daa se, EM algorhm gves us an erave mehod for 95

99 fndng he maxmum lkelhood esmaons of θ, by successvely maxmzng he condonal expecaon of he complee daa lkelhood. The overall procedure can be regarded as smply alernang beween he Kalman flerng and smoohng recursons and he mulvarae normal maxmum lkelhood esmaors. 5.5 Mssng Values In pracce, he Kalman fler equaons are more easly o cope wh mssng values. When a mssng observaon s encounered a me he predcon equaons are processed as usual, he updang equaons canno be processed bu can be replaced by P and θ θ. P ^ ^ If here s a second consecuve mssng value, he predcon equaons can be processed agan o gve ^ ^ ^ Gθ G θ θ, P GP G KW K T T T GP G KW K T ( GP T G G KW K ) G KW K T T T 96

100 Ths concep can be exended o cases wh any number of consecuve mssng values. In addon, we can use he resul of ^ θ 's and P 's as parameers n smoohng algorhms and o effcenly oban he esmaon of he mssng daa hemselves. (C.Chafed 994) Example 5.5. Sae Space Modellng The PURSE daa seres consss of 7 observaons on purse snachng n Chcago. I s very complcaed o esmae he model manually and I have used he compuer sofware "STAMP" (Srucural Tme seres Analyser, Modeller and Predcor) o esmae he me seres model. The compuer program, based on he work of Professor Andrew arvey and wren prmarly by Smon Peers, carres ou he esmaon and esng of srucural me seres models. The Kalman fler plays a key role n handlng he model. 97

101 Fgure 5. Plo of 7 observaons on purse snachng n Chcago From Fgure 5., appears ha he daa seres have he followng properes: a) The model consss of a level and evolve slowly over me accordng o a sochasc (random) mechansm. b) The model consss of no slope. c) The model consss of an rregular erm (random walk nose) 98

102 Usng "STAMP" o esmae o he model and he resuls are as follow : Fg 5. Acual and Fed Values on purse snachng n Chcago Fg 5.3 Normalsed Resdual on purse snachng n Chcago 99

103 The esmaed model seems o f he daa que well. The PURMISS daa seres s he purse seres, wh a run of mssng observaons. I have used a smlar echnque n an aemp o esmae he model. Fgure 5.4 Plo on purse snachng n Chcago wh a run of mssng observaons. From Fgure 5.4, appears ha he daa seres has he followng properes : a) The model consss of a level and evolves slowly over me accordng o a sochasc (random) mechansm. b) The model consss of sochasc slope. c) The model consss of an rregular erm (random walk nose) Usng "STAMP" o esmae o he model and he resuls are as follow : Fgure 5.5 Resuls of esmang mssng values on purse snachng n Chcago.

104 Fgure 5.6 Acual values and fed values on purse snachng n Chcago. Noe: he fed value on he graph have been shfed me nerval forward by he sofware. Observaon Acual Fed Resdual mssng mssng mssng mssng mssng Table 5.5. Acual values and fed values on purse snachng n Chcago By usng STAMP, we have obaned opmal esmaes of he mssng observaons usng srucural me seres analyss.

105 CAPTER 6 Analyss and Comparson of Tme Seres Model 6. Abou hs Chaper In hs chaper, we apply varous esmaon mehods o smulaed daa ses derved from dfferen me seres models. From hs exercse, we can gan an nsgh on how each mehod performs for dfferen me seres suaons and make approprae comparsons beween he mehods. In order o es he effecveness of each esmaon mehod, we requre many daa ses represenng dfferen me seres models. One way o oban such a large amoun of daa s by smulaon usng he compuer. In hs hess, we have chosen o use Mcrosof Excel and Mnab for our daa generaon process. Mcrosof Excel s a popular spreadshee program creaed by Mcrosof. We chose hs sofware because of s populary and easy access. The oher package we used for daa generaon s Mnab. I s anoher popular mul-purpose mahemacs package whch we can easly access. By creang macros, boh packages can generae specfc me seres daa ses whou any dffculy. The specfc me seres models we are gong o generae are AR and MA for varous parameers. We chose hese specfc Tme Seres Models because hey are smple saonary Box- Jenkns models ha faclae easy comparson for dfferen parameers values. The

106 analyss wll also apply for nonsaonary daa ses where ransformaons can be appled o oban saonary daa. Where he sandard devaon (SD) relaes o he purely random process; he daa ses for specfc me seres models are as follows: AR Ph SD MA Thea SD MA Thea SD Table 6.. Varous me seres models for smulaon. We wll se a sngle mssng value a varous posons for each daa se. By usng hs seup, we should be able o assess he performance of each mehod for dfferen me seres models wh mssng values n dfferen posons. For dfferen esmaon mehods, he mssng value posons are as follows: 3

107 Polynomal Curve Fng: Mssng value a poson 7, 49 and 9. Cubc Splne: Mssng value a poson 7, 49 and 9. ARIMA nerpolaon: Mssng value a poson 7, 4,, 8, 35, 4, 49, 56, 63, 7,77, 84 and 9. Sae Space Modellng: Mssng value a poson 7, 49 and 9. To sar our esng, we wll frs consder numercal analyss mehods such as leas square approxmaon (polynomal curve fng) and cubc splne curve fng. Then we apply ARIMA modellng usng forecasng and weghed means. Fnally we wll examne sae space modellng. Afer we have appled each of he above mehods, we wll examne our resuls o compare he effcences of each mehod for dfferen me seres models. 6. Applyng Polynomal Curve Fng (Leas Square Approxmaon) o Box-Jenkns Models The objecve of leas square approxmaon s o f a daa se wh varous non-lnear polynomal funcons. The am s o denfy he approprae polynomal degree wh he mnmum MAD as defned prevously. I could be predced ha hs mehod s bes sued for me seres wh a hghly rregular paern. As here s no requremen for users o defne a me seres model for he daa se, we beleve hs mehod s suable for general curve fng suaons. 4

108 For hs mehod, we have used an excel plug-n wren by Advanced Sysems Desgn and Developmen. We appled he plug-n and creaed a spreadshee ha wll f polynomals o a se of daa wh auomac ncremenaon of he degree on each polynomal. The smulaon was carred ou by followng a number of seps. These were:. Generae specfc me seres model daa ses for esng.. Take ou a specfc value from he daa se and sore a anoher cell for comparson. 3. Apply he plug-n and ry o f he daa se wh dfferen degrees of polynomals. 4. Calculae he absolue devaon for he mssng value of he absolue devaons. 5. Repea he process one hundred mes, deermne he MAD and sandard devaon. 5

109 For each model he polynomal degree wh lowes MAD s hghlghed. The resuls are as follows for AR me seres wh mssng value a poson 49: Table 6.. AR me seres model wh mssng value a poson 49. Summary Table deg 3 deg 4 deg 5 deg 6 deg 7 deg MAD MAD MAD MAD MAD MAD No. Case Ph SD SD SD SD SD SD SD AR AR AR AR AR AR AR AR AR AR AR AR AR AR AR AR

110 For he AR model, as he Ph values approaches he daa has less flucuaon. The able for Ph values greaer han.4 shows ha a hgher degree of polynomal provdes he mnmum MAD. Where he Ph values are negave he daa flucuaes more. In hs case a degree polynomal wll produce opmum resuls. Accordng o our resul, he relaonshp beween mean absolue devaon and sandard devaon of random nose s drecly proporonal. I ndcaed ha f we ncrease he sandard devaon of he random nose by scale of, hen he mean absolue devaon s also ncreased approxmaely by mes. When Ph values are posve we noce ha here s a posve relaonshp beween Ph values and he mean absolue devaon. As he Ph values ncrease he mean absolue devaon ncreases. Bes MAD vs Ph Values Table 6.. Mssng value a poson 49, SD.4. AR mssng value Fgure 6. AR mssng a 49, S.D..4. 7

111 Table 6..3 Mssng value a poson 49, SD.4. AR mssng value Fgure 6. AR mssng a 49, S.D..4. In order o show he effecveness of polynomnal curve fng for me seres daa, he bes MAD value s ploed agans dfferen Ph values and hen graphed as seen n ables 6.. and These resuls show ha when he mssng value s a poson 49 hs mehod s bes sued when he Ph values are closer o. I should also be noed ha he sandard devaon of he MAD decreases as Ph approaches. The above process and analyss s repeaed for mssng values a begnnng and end of daa. 8

112 Table 6..4 summarzes he resuls for mssng daa a poson 7. Table 6..4 AR me seres model wh mssng value a poson 7. Summary Table deg 3 deg 4 deg 5 deg 6 deg 7 deg MAD MAD MAD MAD MAD MAD No. Case Ph SD SD SD SD SD SD SD AR AR AR AR AR AR AR AR AR AR AR AR AR AR AR AR

113 The smulaon for mssng value a 7 ndcaed ha f he mssng value s a he begnnng of he AR daa se, hen we have o ncrease he degree of polynomals n order o acheve a smaller MAD. When he Ph value s negave hen we do no requre a hgh degree polynomnal. In fac, a degree polynomnal s suffcen o acheve a small MAD. Tables also shows he bes MAD value correspondng o obaned Ph values for dfferen sandard devaons. Bes MAD vs Ph Values Table 6..5 Mssng value a poson 7, SD.4. AR mssng value Fgure 6.3 AR mssng a 7, S.D..4.

114 Table 6..6 Mssng value a poson 7, SD.4. AR mssng value Fgure 6.4 AR mssng a 7, S.D..4 The graph of bes MAD values ploed for mssng value a poson 7 as gven n Fgures 6.3 and 6.4 s smlar o mssng value a poson 49 excep ha he MAD values are hgher for posve Ph values less han.6. The resul for an AR smulaon wh mssng value 9 s provded n able 6..7.

115 Table 6..7 AR me seres model wh mssng value a poson 9. Summary Table deg 3 deg 4 deg 5 deg 6 deg 7 deg MAD MAD MAD MAD MAD MAD No. Case Ph SD SD SD SD SD SD SD AR AR AR AR AR AR AR AR AR AR AR AR AR AR AR AR

116 The above smulaon ndcaed ha f he mssng value s a he end of he AR daa se, hen wo degrees polynomal should be used o make a reasonable esmaon for posve Ph values. Ths s because s dffcul o f a hgh degree polynomnal curve whn he daa gven. The bes MAD values correspondng o dfferen Ph values are provded n ables for dfferen sandard devaons. Bes MAD vs Ph Values Table 6..8 Mssng value a poson 9, SD.4. AR mssng value Fgure 6.5 AR mssng a 9, S.D..4. 3

117 Table 6..9 Mssng value a poson 9, SD.4. AR mssng value Fgure 6.6 AR mssng a 9, S.D..4. For any AR me seres model wh los of daa pons before he mssng value, he graph ndcaes ha as he Ph value ncreases, he MAD value wll become smaller han for he daa wh a mssng value a he begnnng or mddle. As a resul, hs mehod s suable for AR model wh large Ph value. I can be concluded ha for any AR process where here s a suffcen amoun of pas daa avalable, polynomals of wo or hree degrees should gve a reasonable esmaon. If he mssng value s a he begnnng of he daa se, hen we should reverse he daa se and calculae he mssng value wh wo degree polynomal or jus use polynomal of fve or sx degrees o esmae he mssng value. The resuls are as follows for Movng Average me seres wh mssng values a poson 49. 4

118 Table 6.. MA me seres model wh mssng value a poson 49. Summary Table deg 3 deg 4 deg 5 deg 6 deg 7 deg MAD MAD MAD MAD MAD MAD No. Case Ph SD SD SD SD SD SD SD MA MA MA MA MA MA MA MA MA MA MA MA MA MA MA MA MA

119 Summary Table deg 3 deg 4 deg 5 deg 6 deg 7 deg MAD MAD MAD MAD MAD MAD No. Case Ph SD SD SD SD SD SD SD 8 MA MA MA The MA me seres model has more random nose whn he daa and for all posve Thea values he smulaon ndcaed ha f he mssng value s n he mddle of he daa se hen a hgh degree polynomal s requred o gve a reasonable esmaon. Whereas, he negave Thea values conss of more random nose so herefore he behavour of he me seres model shows an errac paern whch suggess ha a lower degree polynomnal s suffcen o acheve a small MAD. I can be noed ha he sandard devaon of he MAD values are smaller when he Thea values are closer o. In addon, smlar o he AR daa, he MAD values and he sandard devaon of he random error are drecly proporonal o each oher. 6

120 Bes MAD vs Thea Values Table 6.. Mssng value a poson 49, SD.4. MA mssng value Fgure 6.7 MA mssng a 49, S.D..4. Table 6.. Mssng value a poson 49, SD.4. MA mssng value Fgure 6.8 MA mssng a 49, S.D..4. 7

121 The graph shows he bes MAD values ploed for mssng value a poson 49 ndcaes ha mnmum MAD s acheved when Thea value s zero.e. purely random process. For he MA process he sandard devaon of he MAD also decreases as Thea approaches. For larger posve and negave Thea values he esmaon s unrelable ndcang a need for cauon n usng leas squares polynomal. The followng able summarzes he resuls for mssng daa a poson 7. Table 6..3 MA me seres model wh mssng value a poson 7. Summary Table deg 3 deg 4 deg 5 deg 6 deg 7 deg MAD MAD MAD MAD MAD MAD No. Case Ph SD SD SD SD SD SD SD MA MA MA MA MA MA MA MA MA MA MA MA

122 Summary Table deg 3 deg 4 deg 5 deg 6 deg 7 deg MAD MAD MAD MAD MAD MAD No. Case Ph SD SD SD SD SD SD SD 3 MA MA MA MA MA MA MA MA In he case of mssng value a poson 7, s dffcul o predc he approprae degree of polynomal o be used for dfferen posve Thea values. One possble explanaon s ha here s nsuffcen daa before he mssng value. Where, he negave Thea values conss of more random nose so herefore he behavour of he me seres model shows an errac paern whch suggess ha a lower degree polynomal s suffcen o acheve a small MAD. 9

123 Bes MAD vs Thea Values Table 6..4 Mssng value a poson 7, SD.4. MA mssng value Fgure 6.9 MA mssng a 7, S.D..4. Table 6..5 Mssng value a poson 7, SD. 4. MA mssng value Fgure 6. MA mssng a 7, S.D..4. The graph of bes MAD values ploed for mssng value a poson 7 as gven n Fgures 6.9 and 6. s smlar o mssng value a poson 49.

124 The resul for an MA smulaon wh mssng value 9 s provded n able Table 6..6 MA me seres model wh mssng value a poson 9. Summary Table deg 3 deg 4 deg 5 deg 6 deg 7 deg MAD MAD MAD MAD MAD MAD No. Case Ph SD SD SD SD SD SD SD MA MA MA MA MA MA MA MA MA MA MA MA MA MA MA MA

125 Summary Table deg 3 deg 4 deg 5 deg 6 deg 7 deg MAD MAD MAD MAD MAD MAD No. Case Ph SD SD SD SD SD SD SD 7 MA MA MA MA Where here are los of daa pons gven before he mssng value here wll be los of random nose whn hs ype of me seres model, wheher s a posve or negave Thea model. As a resul s dffcul o mpose a hgh degree polynomnal o f he daa se. The wo degree polynomnal should be used because wll provde he mos accurae resuls producng he smalles MAD. Bes MAD vs Thea Values Table 6..7 Mssng value a poson 9, SD.4. MA mssng value Fgure 6. MA mssng a 9, S.D..4.

126 Table 6..8 Mssng value a poson 9, SD.4. MA mssng value Fgure 6. MA mssng a 9, S.D..4. From he resul above, can be concluded ha for any MA process wh suffcen amoun of pas daa avalable, polynomals of wo degrees should gve a reasonable esmaon. If he mssng value s a he begnnng of he daa se, hen we should reverse he daa se and calculae he mssng value wh hree degrees polynomals or jus use polynomal of sx degrees o esmae he mssng value. In concluson, he resuls show ha when he mssng value s a poson 49 hs mehod s bes sued when he Thea values are closer o and a mnmum MAD s acheved. The same concluson can be appled o he case when he mssng values appear a poson 9. Whereas, poson 7 shows ha he value of MAD sll reans a mnmum value when he Thea value s close o. owever, when he Thea value s posve he resuls show ha here s a sgnfcan ncrease n he MAD value. In concluson hs mehod s no suable for use when here s a large posve value and he mssng value appears a he begnnng of he daa. 3