Using GLS to generate forecasts in regression models with auto-correlated disturbances with simulation and Palestinian market index data

Transcription

1 America Joural of Theoreical ad Applied Saisics 04; 3(: 6-7 Published olie December 30, 03 (hp:// doi: 0.648/.aas Usig o geerae forecass i regressio models wih auo-correlaed disurbaces wih simulaio ad Palesiia marke idex daa Samir K. Safi, Ehab A. Abu Saif Dep. of Ecoomics ad Saisics, Faculy of Commerce, he Islamic Uiversiy of Gaza, Gaza, Palesie Saisicia Researcher, Gaza, Palesie address: samirsafi@gmail.com (S. K. Safi, cesa@homail.com (E. A. Abu Saif To cie his aricle: Samir K. Safi, Ehab. A. Abu Saif. Usig o Geerae Forecass i Regressio Models wih Auo-correlaed Disurbaces wih simulaio ad Palesiia Marke Idex Daa. America Joural of Theoreical ad Applied Saisics. Vol. 3, No., 04, pp doi: 0.648/.aas Absrac: This paper ivolves a impora saisical problem cocerig forecasig i regressio models i ime series processes. I is well kow ha he mos famous mehod of esimaig ad forecasig is he Ordiary Leas Squares (OLS. OLS may be o he opimal i his coex. So over he years may specialized esimaio echiques have bee developed, for example Geeralized Leas Squares (. We are comparig he forecasig based o some esimaors wih he predicio usig he esimae. This compariso will be used by wha is kow as measures of forecas accuracy. We coduc a exesive compuer simulaio ime series daa, o make compariso amog hese mehods. The similar forecasig crieria were developed ad evaluaed for he real daa se o daily closig price i he Palesiia marke idex (Alquds Idex. The daa cosiss of 64 mohly observaios ad obaied from he websie of he Palesie Sock Exchage. The mai fidig is ha, for forecasig purposes here is o much gaied i ryig o ideifyig he exac order ad form of he auo-correlaed disurbaces by usig esimaio mehod. I addiio, we oiced ha he accuracy of forecasig usig mehod does o differ subsaially ha he oher mehods as Maximum Likelihood Esimaio (E, Miimize Codiioal Sum of Squares ( ad he combiaio of hese wo mehods. Moreover, for parameer esimaio, he is early as efficie as he exac parameer esimaio. O he oher had, he Ordiary Leas Squares (OLS mehod performs much less efficie ha he oher esimaio mehods ad producig poor forecasig accuracy. Keywords: Geeralized Leas Squares, Ordiary Leas Squares, Maximum Likelihood, Forecasig Accuracy, Simulaio. Iroducio Compariso of esimaors ad forecasig i liear regressio models wih auocorrelaed disurbaces is ispired by problems, which arise i meeorology ad ecoomics. I is well kow ha he mos famous mehod of esimaig ad forecasig is he Ordiary Leas Squares (OLS, i is maybe o he opimal i his coex. So over he years may specialized esimaio echiques have bee developed, for example Geeralized Leas Squares (. These mehods are more complicaed ha OLS ad are less udersood. We are comparig he predicio based o some esimaors wih he predicio usig he esimae. This compariso will be used by wha is kow as measures of forecas accuracy. This paper aims o sudy he mehod for parameer esimaio i he regressio models wih auocorrelaed disurbaces. Compariso of for esimaio wih oher well kow mehods based o forecass crierio is discussed. I his secio we iroduce some relaed sudies ad recall he mos impora fidigs. Shiu ad Asemoa (009 were compared he performace of model order deermiaio crieria i erms of selecig he correc order of a auoregressive model usig he simulaio mehod i small ad large samples. The crieria cosidered are he Akaike iformaio crierio (AIC; Bayesia iformaio crierio (BIC, ad he Haa Qui crierio (HQ. The resuls shows ha BIC performs bes i erms of selecig he correc order of a Auoregressive model for small

2 America Joural of Theoreical ad Applied Saisics 04; 3(: samples irrespecive of he AR srucure, HQ crieria ca be said o perform bes i large sample. Eve hough he AIC has he leas performace amog he crieria cosidered, i appears o be he bes i erms of he closeess of he seleced order o he rue value. Oo ad Olaayo (009 were compared subse Auoregressive Iegraed Movig Average (ARIMA models, wih full ARIMA models. They used residual variace, AIC ad BIC, o deermie he performace of he models. Resuls revealed ha he residual variace aached o he subse auoregressive iegraed movig average models is smaller ha he residual variace aached o he full auoregressive iegraed movig average models. Subse auoregressive iegraed movig average models performed beer ha he full auoregressive iegraed movig average models. Lee ad Lud (004 were proposed he properies of OLS ad esimaors i a simple liear regressio wih saioary auocorrelaed errors. Explici expressios for he variaces of he regressio parameer esimaors are derived for some commo ime series auocorrelaio srucures, icludig a firs-order auoregressio ad geeral movig averages. Applicaios of he resuls iclude cofidece iervals ad a example where he variace of he red slope esimaor does o icrease wih icreasig auocorrelaio. Koreisha ad Fag (004 were sudied ad described a ew procedure for geeraig forecass for regressio models wih serial correlaio based o OLS. From a large simulaio sudy hey foud ha for fiie samples he predicive efficiecy of heir wo-sep liear approach is higher ha ha of OLS for shor ad medium horizo, ad very comparable o ha of based o AR ( pɶ correcios wih pɶ = T, where T is umber of observaio, which is also kow o be very similar o he esimaio procedure whe he error covariace marix Ω will be esimaed from daa. For loger horizos OLS yields forecass ha are as efficie as hose geeraed by approaches ad he wo-sep procedure for geeraig forecass for regressio models wih serial correlaio based exclusively o ordiary leas squares (SOLS esimaio. Safi (004 discussed he compariso of efficiecy of he OLS esimaio o aleraive procedures such as ad esimaed (E esimaors i he presece of firs ad secod order auoregressive disurbaces. The mos impora fidigs ha he relaive efficiecy of he OLS esimaor as compared o he esimaor decreases wih icreasig values of ρ, he foud ha he efficiecy of he OLS esimaor for esimaig a iercep appears o be early as efficie as he esimaor for ρ.7 for relaively small ad moderae sample sizes. However for large sample size, OLS appears o be early as efficie as he esimaor for he addiioal values of ρ = ±.9. Ad The OLS esimaor may ofe be beer ha assumig aoher icorrec rucaio of he acual process. I addiio, i is someimes beer o igore he problem alogeher ad use OLS raher ha o icorrecly assume he process is AR (. Fidley (003 sudded properies of forecas errors ad esimaes of misspecified ARIMA ad iermediae memory models ad he Opimaliy of for Oe-Sep- Ahead Forecasig. Boh OLS ad esimaes of he mea fucio are cosidered. He showed ha has a opimal oe-sep-ahead forecasig propery relaive o OLS whe he model omis a regressio variable of he rue mea fucio ha is asympoically correlaed wih a modeled regressio variable. Some ihere ambiguiy i he cocep of bias for regressio coefficie esimaors i his siuaio is discussed. The goals of his paper ca be spli io hree mai issues. Firsly, sudy he forecasig behavior usig mehod i regressio models wih auo-correlaed disurbaces, ad compare he forecas accuracy wih oher esimaio mehods. Secodly, evaluae he forecasig for he real daa se o Alquds Idex for illusraive purposes, ad fially coducig exhausive simulaio sudy seup for examiig he accuracy of our fidigs. This paper is orgaized as follows: Secio iroduces model esimaio by E, OLS, ad. I secio 3, a case sudy o Palesiia Al-Quds idex sock daa is aalyzed. Secio 4 focuses o forecasig evaluaio ad prese he comparisos of esimaio mehods for he real ad simulaed daa. Secio 5 summaries he resuls ad offers suggesios for fuure research for usig o geerae forecass i regressio models wih auocorrelaed disurbaces.. Model Esimaio Buildig ime series models ivolves hree basic seps, model ideificaio, model esimaio ad model diagosics. I his secio, we iroduce he model esimaio which is releva o he purpose of his paper. We cosider, Maximum Likelihood esimaio, Leas Squares Esimaio, ad Esimaio... Maximum Likelihood Esimaio Basically, a ARMA model combies he ideas of auoregressive AR ad movig average MA models io a compac form. A mixed auoregressive movig average model wih p auoregressive erms ad q movig average erms is abbreviaed ARMA ( p, q ad may be wrie as, Cryer ad Cha (008. X = φx + φx + + φ X + ε + θ ε + θ ε + + θ ε p p q q Whe he orders p ad q of ARMA(p,q model are kow, esimaes of he s i φ ad θ s ca be foud whe he daa

3 8 Samir K. Safi ad Ehab. A. Abu Saif: Usig o Geerae Forecass i Regressio Models wih Auo-correlaed Disurbaces wih simulaio ad Palesiia Marke Idex Daa beig observaios from a Gaussia ARMA model. Eve X is o Gaussia, he Gaussia likelihood sill is a if { } reasoable measure of goodess of fi of he model, so maximizig i is sesible. Also, he asympoic disribuio of maximum likelihood esimaors is he same wheher he whie oise iovaios ε (ad so he process iself are Normal or o. Suppose our observed daa x,..., x are placed i he daa vecor ( x,..., x (afer appropriae differecig ad mea-correcio are modeled by a saioary zero-mea Gaussia ARMA ( p, q process { } φ X wih parameers = ( φ,..., φ p, = ( θ,... θq ad σ. Recall ha σ is he variace of he WN variables i he ARMA process. The covariace marix of X = ( x,..., x θ ( = Γ ( φ, θ, σ E XX say, is a symmeric posiive defiie marix, Cosiss γ =, hemselves from he covariaces i, i,,..., fucios of he model parameers. Because he X have a oi -variae ormal disribuio he likelihood based o daa x is L ( φ, θ, σ ; x ( π exp x x = Γ Γ ad he log-likelihood up o a cosa erm l ( φ θ σ x = Γ Γ,, ; x x log I priciple he maximum likelihood esimaes ca be obaied from his by umerical maximizaio. I pracice however, if is large, direc calculaio of Γ ad Γ could be a problem. Foruaely algorihms are available o avoid he difficuly. Wha is eeded for a umerical maximizaio procedure is he abiliy o evaluae he loglikelihood l quickly a specified values of he parameers. Give his abiliy a good maximizaio rouie should be able o ierae efficiely owards he parameer values a which l is larges. Thus maximizaio depeds o he abiliy o evaluae l easily a ay give se of parameer values (See for example Abraham ad Ledoler (005. Oe mehod of evaluaio is o build up l, sarig as hough here were very few observaios, ad successively calculaig he chages as ew observaios are ake io accou. To see how his works we eed some furher oaio. For each =,,... Le X ˆ deoe he miimum variace esimae of X based o liear combiaios of observaios made before ime, X, X,..., X, These X ˆ 's are called oe-sep-ahead predicors. They are liear combiaios of earlier observaios, wih coefficies which are fucios of he covariaces for he paricular model we are cosiderig. These coefficies ca be expressed i erms of he model parameers, bu we will o eed hem explicily for he curre discussio. Also le U deoe he differece X ˆ X, called he iovaio a ime.a propery of he U 's is ha hey have zero expecaio ad are ucorrelaed wih each oher. (This follows from he easily-proved fac ha X ˆ is he codiioal expecaio of X give all values observed before. Fially le X ˆ ad U deoe he vecors of X ˆ ˆ,..., X ad iovaios ( U,..., U predicors ( respecively. Sice X ˆ is a fucio oly of earlier i hem, we ca wrie U = X Xˆ = A X X 's ad is liear where A is a marix wih 's alog is diagoal ad zeroes above he diagoal, he values below he diagoal beig he coefficies deermiig he X ˆ. Beig riagular wih posiive diagoal eries, A is o-sigular, ad so, wriig A = C, we have ( ˆ X = C X X Like A, he iverse C is also riagular wih 's alog is diagoal ad zeroes above i. Also, sice he compoes of X X ˆ are ucorrelaed, he variacecovariace marix of his vecor, E D say, ( ˆ ( ˆ X X X X = D mus be diagoal, D =diag ( υ0,..., υ, where ( υ = Var U is he iovaio variace a ime. ad From he above ( E ( ( ˆ ( ˆ Γ = E X X = C X X X X C = C D C ( ˆ X ( ˆ X D X X ( X ˆ X = = υ = Γ = CDC = C D = υ0 υ i..., (. sice C because of C 's ui-diagoal/riagular = srucure, Cochrae (005.. Thus he log-likelihood becomes

4 America Joural of Theoreical ad Applied Saisics 04; 3(: ˆ l ( x X (. ( φ, θ, σ ; x = logυ = = υ The grea advaage of his expressio is ha he ad X ˆ 's ca be calculaed very efficiely by recursio usig he Iovaios Algorihm The procedure herefore gives a highly effecive roue o he maximum likelihood esimaors. I fac he Iovaios Algorihm works for ay process, wheher of ARMA form or o, saioary or o. For a ARMA model i paricular i is foud ha he iovaio variaces υ all have he form where he υ = σw υ w deped o φ ad θ bu o o σ. The coefficies defiig he oe-sep-ahead predicors X ˆ i erms of earlier observaios also do o deped o σ. The log-likelihood herefore becomes ( ˆ l x X (.3 ( φ, θ, σ ; x = logσ logw = σ = w ad if we maximize his wih respec o σ (by differeiaig wih respec o σ, seig he derivaive equal o 0 ad solvig he resulig equaio, we fid ha = ( ˆ φ ˆ θ ˆ S, ( x X ˆ σ = = (.4 w say, givig he maximum likelihood esimae of 's σ i erms of a sum of squares S depedig o he esimaes of he oher parameers. If his expressio for ˆ σ is subsiued back io he log-likelihood, hereby elimiaig σ, we fid ha ˆ φ ad ˆ θ mus be he values of φ ad θ maximizig l ha is, miimizig ( φ, θ, σ ; x = logw log = S ( φ, θ S ( φ, θ logw + log (.5 = Miimizaio of S ( φ, θ aloe would be a form of leas squares esimaio. From (.5 we see ha if logw is small or varies lile wih φ ad θ he = resulig esimaes are likely o be close o he maximum likelihood esimaes. I pracice (.4 is lile used as a esimae of σ. Isead, as for he esimaio of variace i regressio, he ( ˆ φ, ˆ θ S esimae ɶ σ = is preferred. (See for example, p q Everi ad Hohor, 00 ad Fox, Leas Squares Esimaio I his secio, we prese he leas square esimaio i ime series ad regressio model.... The Leas Squares i Time Series The uderlyig idea i he fiig of a ime series model by leas squares, by aalogy wih regressio, is ha we should choose parameer values which miimize he sum of squared differeces bewee he observed daa ad heir expeced values accordig o he model. We prese leas squares esimaio i hree models, amely: AR, MA, ad ARMA. For AR(, X = φx + ε : Give he observaios up o ime, he expeced value of X is φx, so i is reasoable o use S ( φ = ( x φx = as he sum of squares o be miimized. Noe ha he = erm is omied sice x 0 is o available. Miimizaio of S wih respec o φ leads o he esimae ɶ φ = = = x x x The same idea works i he same way for AR ( p models, leavig ou differeces from x 's wih p. For MA (, X = ε + θε : if he MA model is iverible he X = ( θ i X + ε i By aalogy wih he previous example ad wih regressio his suggess basig a sum of squares o he differeces ε = X ( θ i X i However we cao use hese direcly (ifiiely may erms i he sums. Isead ake = S ( θ ε ad oe ha from he defiig relaio of he process we could fid he erms here successively from.

5 0 Samir K. Safi ad Ehab. A. Abu Saif: Usig o Geerae Forecass i Regressio Models wih Auo-correlaed Disurbaces wih simulaio ad Palesiia Marke Idex Daa ε = X θε 0 ε = X θε ε = X θε provided we kew ε 0. No kowig ε 0 i's aural o replace i by is expecaio, zero. S calculaed i his way is referred o as a codiioal sum of squares, ad he esimae obaied by miimizig he codiioal S is called he codiioal leas-squares esimae. For a geeral ARMA model: he codiioal sum of squares ca be defied much as i he MA case, q ε = X θ ε φ X i i i = = p o calculae ε from he previous oes ad pas daa. As i he MA case we ca ake he ε 's o he righ had side o be zero for o-posiive idices. As i he AR case we may coe ourselves wih calculaig he ε 's oly for > p so ha we oly use observed X 's, or we migh ake X 's wih egaive suffices o be zero oo. Ucodiioal Leas Squares meas miimizig he sum of squares S φ, θ i (.5. As oed his is o quie he same as ( maximizig he likelihood, bu will ofe give similar resuls.... The Leas Squares i Regressio Model The regressio model ca be wrie as y = f ( x ; β + ε (.6 where f ( x ; β is a mahemaical fucio of he p idepede variables x = ( x,..., x p ad ukow parameers β = ( β,..., β m. Due o he radom aure of he error erms ε, he depede variable y iself is a radom variable. The model i (.6 ca herefore also be expressed i erms of he codiioal disribuio of y give x = ( x,..., x p. The regressio assumpios ca be wrie as: E ( y ( ;. The codiioal mea, x = f x β, depeds o he idepede variables ad he parameers β, ad he variace Var ( y x = σ is x idepede of ad ime.. The depede variables y ad y k for differe ime periods (or subecs are ucorrelaed x [ x β ][ x β ] Cov ( y, y = E y f ( ; y f ( ; = 0 k k k 3. Codiioal o x, y follows a ormal disribuio wih mea f ( x ; β ad variace σ, his is deoed by N ( f ( ;, σ x β. Leas squares esimaes are miimize he sum of he squared deviaios S ( = [ y ( ; ] f β x β ad deoed = by β ˆ. The geeral liear regressio model Liear, regressio models ca be wrie as y = Xβ + ε (.7 where y is a vecor of observaios o a depede variable, X is a k marix of idepede variables of full colum rak, β is a k vecor of parameers o be esimaed, ad ε is a vecor of disurbaces. I marix oaio he leas squares crierio ca be expressed as miimizig S ( β = ( y x β = ( y Xβ ( y Xβ = The miimizaio of S ( β leads o he leas squares esimaor β ˆ, which saisfies he p + equaios ( XX βˆ = Xy (.8 These are referred o as he ormal equaios. Sice we have assumed ha he desig marix X is of full colum rak, he iverse of X X ca be calculaed. The soluio of he ormal equaios is he give by βˆ = ( X X Xy (.9 The ( p + ( p + covariace marix of β ˆ is give by V ( ˆ ( β = σ X X (.0 Cosider a sadard liear model (.7 wih all he assumpios of he classical liear model excep he assumpio of cosa variace, o-auocorrelaed error erms. Replace his assumpio wih [ ] σ Var ( ε = E εε = Ω, where Ω is a symmeric ad iverible marix. Each eleme w of Ω is proporioal o he covariace i bewee he errors ε i ad ε i of observaios i ad. Each diagoal eleme w ii is proporioal o he variace of ε i. Whe he variace ad covariace of he uobserved facors akes his form he he formula for esimaig he variace covariace marix of β ˆ is - ( ˆ = ( ( Var β σ X X X ΩX X X (.

6 America Joural of Theoreical ad Applied Saisics 04; 3(: 6-7 Uforuaely we do o usually kow Ω uless we pu a specific srucure o he process ha deermie he uobservable facors i he model ε. I geeral, I is impossible o esimae Ω wihou resricios o is ( + srucure. Ω has uique parameers, bu we oly have observaios ad he umber of parameers o be esimaed, icludig hose i β ˆ, mus be less ha he umber of observaios. For Cosise esimaio of he variace covariace marix, here are ways o esimae XεεX = σ XΩX which is a k k marix of all he cross producs of he error erms ad he regressors. There are various forms desiged o deal wih differe siuaios. I geeral, wha s required is a esimae of he form ˆ ε ˆ iε xx i. Bu i his usually resriced i some way, e.g. he The Huber- Whie esimae is ˆi ε x i x ad deals wih i heeroskedasiciy oly. The Newey-Wes esimaor exeds his o serial correlaio. Aoher popular choice is for clusered sadard errors, which may be useful approximaios wih spaial daa..3. Geeralized Leas Squares Esimaio Regressors are assumed o be o sochasic, i.e. fixed i repeaed samplig, idepede ad ucorrelaed wih he error erms i he classical liear regressio model. These assumpios are o always saisfied especially i ime series. OLS is o he mos efficie esimaor here. We ca gai precisio i leas-squares esimaes by weighig observaios wih lower variace more heavily ha hose wih higher variace, so ha he weighed error variace covariace marix is of he sadard form. The iuiio is ha we weigh he esimaor so ha i places greaer emphasis o observaios for which he observable explaaory variables do a beer ob of explaiig he depede variable. This meas we eed o devise a weighig marix C such ha: [ ] Var ( Cε = E CεεC = σ I A marix ha does his, is a marix C such ha C C = Ω, which implies CΩC = I. I fac, several such marices C exis, so ha, for coveiece, we ca assume C = C. To derive he form of he Bes Liear Ubiased Esimaor (BLUE of β for he geeralized regressio model uder he assumpio ha is Ω kow, weighig all he variables i he model gives Cy = CXβ + Cε. The, he OLS esimaor applied o his gives he esimaor β ˆ = ( X Ω X X Ω y If he Ω are kow he he esimaor β ˆ is BLUE, wih variace covariace marix Var ( ˆ β = σ ( X Ω X. Noe ha his assumes ha he parameers are homogeous across he sample, i.e. hey do o chage for differe groups i he daa. Weighig he daa will he chage he parameer esimae accordig o which groups are more heavily weighed. I pracice, Ω is ypically ukow so ha he esimaor is maybe o available. meaig ha β ˆ is ooperaioal, ad a esimaed or feasible geeralized leas squares (F esimaor is used. If he marix Ω ukow, we ca parameerize he marix Ω = Ω ( θ i erms of a fiie-dimesioal parameer vecor θ, ad use he classical OLS residuals o obai cosise esimaors θˆ ad Ω ˆ = Ω( θ ˆ of θ ad Ω. The replace he ukow Ω wih he esimaed Ω ˆ i he formula for, yieldig he feasible esimaor, ( ˆF β = ˆ ˆ XΩ X XΩ y 3. Case Sudy: Palesiia Al-Quds Idex Sock Daa I his secio, we cosider a real daa se called Palesiia Al-Quds idex sock Daa. The daa is obaied from Palesie Exchage (PEX web page ( We cosider he closig price values for he Al-Quds Idex. R-saisical sofware is used for fiig ARIMA model for he ime series. 3.. Daa Descripio We cosider he mohly closig price values for he Al- Quds Idex i Palesie, from Sepember 997 o April 0. The daa is ake a he ed of las radig every moh, hus we have (64 observaio. The closig price for he Al-Quds Idex rages bewee 97.0 ad wih mea ad sadard deviaio Daa Processig The Lug-Box es saisic equals This is referred o a chi-square disribuio wih oe degree of freedom. This leads o a p-value <.e-6, so we reec he ull hypohesis ha he error erms are ucorrelaed. I oher words, here is srog evidece of auocorrelaio i he residuals of his daa. Figure 3. displays he ime series plo. The series displays cosiderable flucuaios over ime, especially sice 005, ad a saioary model does o seem o be reasoable. The higher values display cosiderably more variaio ha he lower values.

7 Samir K. Safi ad Ehab. A. Abu Saif: Usig o Geerae Forecass i Regressio Models wih Auo-correlaed Disurbaces wih simulaio ad Palesiia Marke Idex Daa Close value Plo of Orgial daa Figure 3.. Mohly series of closig value for al-quds idex: Sep. 997 o April 0 Time The sample EACF compued o he firs differeces of he Mohly Closig Values for Al-Quds Idex is show i Table 3.. I his able, a ARMA(p,q process will have a heoreical paer of a riagle of zeroes, wih he upper lef-had verex correspodig o he ARMA orders. Table 3. displays he schemaic paers for possibiliy of IMA(,, ARIMA(,,, or ARIMA(,,. Table 3.. EACF for Differece of Mohly Closig Values for Al-Quds Idex Series KPSS es for level saioariy is applied o he origial series leads o a es saisic of ad a p-value of 0.0. Wih saioariy as he ull hypohesis, his provides srog evidece supporig he osaioariy ad he appropriaeess of akig a differece of he origial series. The differeces of he closig values for al-quds idex are displayed i Figure 4.. The differeced series looks much more saioary whe compared wih he origial ime series show i Figure 4.. O he basis of his plo, we migh well cosider a saioary model as appropriae. A R MA x x x x x x x x x x x x x 0 0 x x s diff.closevalue Figure 3.. The Differece Series of he Mohly Closig Values for Al- Quds Idex KPSS es is applied o he differeced series leads o a es saisic of ad a p-value of 0.0. Tha is, we do o reec he ull hypohesis of Saioariy Model Specificaio Boh he sample ACF ad PACF displayed i Figure 3.3, cu off afer lag (, srogly suggesed a MA( or AR( appropriae model for he differeced series, respecively. Therefore, i is quie difficul o ideify he MA, AR, or mixed model from his figure. ACF Parial ACF plo of firs differeces for closevalue Figure 3.3. Sample ACF ad PACF for Differece of he Mohly Closig Values for Al-Quds Idex Time cvalue Lag Series diffsdaa Lag 3.4. Model Selecio The esimae of he series mea is o sigificaly differe from zero (P-value = The hree seleced ARIMA models ad heir correspodig crieria are show i Table 3.. These crieria cofirm he selecio suggesio -IMA(,- based o he smalles values of AIC, AICc, ad BIC amog he oher ARIMA choices. Table 3.. Differe crieria for suggesed ARIMA models Models AIC AICc BIC IMA(, ARIMA(,, ARIMA(,, Model Diagosic Figure 3.4 displays he ime series plo of he sadardized residuals from he IMA(, model esimaed for he Al-Quds idex series ime series. The model was fied usig maximum likelihood esimaio. There are few residuals wih magiude larger ha. The sadardized residuals do show clusers of volailiy ad seem o be fairly radom wih o paricular paers. Sadardized Residuals Figure 3.4. Sadardized Residuals of he Fied Model from Al-Quds idex IMA (, Model Time

8 America Joural of Theoreical ad Applied Saisics 04; 3(: ACF Series residuals(model Figure 3.5. Sample ACF of Residuals of he Fied Model IMA(, Model To check o he idepedece of he error erms i he model, we cosider he sample auocorrelaio fucio of he residuals. Figure 3.5 displays he sample ACF of he residuals from he IMA (, model of he from Al-Quds idex daa. The dashed horizoal lies ploed are based o he large lag sadard error of ± = ± 0.56 (=64. The graph does o show saisically sigifica evidece Lag of ozero auocorrelaio i he residuals. I oher words, here is o evidece of auocorrelaio i he residuals of his model. These residual auocorrelaios look excelle. I addiio o lookig a residual correlaios a idividual lags, i is useful o have a es ha akes io accou heir magiudes as a group. Figure 3.6 shows he p-values for he Lug-Box es saisic for a whole rage of values of K from o 0. The horizoal dashed lie a 5% helps udge he size of he p-values. The Lug-Box es saisic wih K = is equal o.798. This is referred o a chi-square disribuio wih 0 degrees of freedom. This leads o a p-value of , so we have o evidece o reec he ull hypohesis ha he error erms are ucorrelaed. The suggesed model looks o fi he modelig ime series very well. p.value Figure 3.6. P-values for he Lug-Box Tes for he Fied Model lag Sample Quailes Normal Q-Q Plo Theoreical Quailes series is show i Figure 3.7. The pois seem o follow he sraigh lie fairly closely. This graph would o lead us o reec ormaliy of he error erms i his model. I addiio, wih a few mior excepios i he lower ad upper ails, he hisogram of he sadardized residuals seems o be ormal. Therefore he esimaed IMA(, model seems o be capurig he depedece srucure of he differece of Al- Quds idex. Hisogram of rsadard(model 3.5. Esimaio Mehods Frequecy rsadard(model Figure 3.7. Quaile-Quaile Plo ad hisogram of he Residuals of he Fied Model from Al-Quds idex IMA (, Model A quaile-quaile plos are a effecive ool for assessig ormaliy. Here we apply hem o he residuals of he fied model. A quaile-quaile plo of he residuals from he IMA(, model esimaed for he Al-Quds idex Usig forecas package i R program, here are hree esimaio mehods.. "-" miimize codiioal sum-of-squares o fid sarig values he maximum likelihood (he defaul mehod.. "" maximum likelihood. 3. "" miimize codiioal sum-of-squares. I addiio will be used for compariso purposes. Table (3.3 shows he resuls for he four meioed mehods for he hree seleced ARIMA models, amely: IMA(,, ARIMA(,, ad ARIMA(,,.

9 4 Samir K. Safi ad Ehab. A. Abu Saif: Usig o Geerae Forecass i Regressio Models wih Auo-correlaed Disurbaces wih simulaio ad Palesiia Marke Idex Daa Table 3.3. resul of mehods wih measures of esimae ad measures of forecas accuracy Mehod φ φ θ θ IMA(, ARIMA(,, ARIMA(,, AIC AIC C BIC Based o AIC, AICc, ad BIC, he resuls cofirm IMA (, is he bes model amog he ohers. Here we see ha ˆ θ = Noig he P-values for he esimae of he movig average coefficie, ˆ θ (0.005 is sigificaly differe from zero saisically, ad isigifica for he iercep erm iercep (0.6, cosequely, i is o icluded i he esimaed model. Therefore, The fied model ca be wrie as: w = ε 0.84 ε 4. Forecasig Evaluaio The crucial obec i measurig forecas accuracy is he loss fucio. I his secio we prese he mos widely saisical loss fucios. Accuracy measures are usually defied o he forecas errors e ˆ + k, = X + k X + k,. Defiiio 4.. Mea Squared Error (MSE elimiaes he posiive-egaive problem by squarig he errors. The resul eds o place more emphasis o he larger errors ad herefore gives a more coservaive measure ha he MAE. This approach pealizes large forecasig errors. The MSE is give by T e + k, = MSE = (4. T where, T is umber of periods used i he calculaio. T k The roo mea squared error, RMSE = e +,, is T = easy o ierpre i is oe of he mos commoly used measures of forecas accuracy. 4.. Comparisos of Esimaio Mehods for Real Daa We cosider five esimaio mehods -,,,, ad classical OLS for he hree seleced ARIMA models, amely: IMA(,, ARIMA(,, ad ARIMA(,,. We compare he forecasig performace for esimaio mehod wih he oher four mehods. Table (4. shows he complee resuls for he forecas accuracy crierio RMSE. We ca deduce he followig: For usig he bes esimaio model, IMA (,, esimaio mehod performs early as efficie as he oher esimaio mehods. For example, he RMSE for IMA (, usig, -,, ad equal , , , ad , respecively. For oher over esimaio models, he measures of forecas accuracy crieria usig he esimaio mehod do o differ subsaially comparig o he oher esimaio mehods. For example, he RMSE for ARIMA (,, usig, -,, ad equal 49.80, 49.80, , ad , respecively. For he rasformed daa usig he firs differece, OLS performs as early as he oher esimaio mehods. The RMSE equals However, for he origial daa, OLS performs much less efficie ha he oher esimaio mehods, resulig poor forecasig accuracy. The RMSE equals Mehod - OLS * OLS ** IMA(, * OLS for he rasformed daa * OLS for he origial daa Table 4.. RMSE for Real Daa ARIMA(,, ARIMA(,, Comparisos of Esimaio Mehods for Simulaed Daa I his secio, we cosider he robusess of he four

10 America Joural of Theoreical ad Applied Saisics 04; 3(: esimaio mehods. We compare he bes forecasig amog some of he esimaio mehods such as -,,,, ad classical OLS. This simulaio will be coduced o examie he sesiiviy of he seleced esimaio mehods o model forecasig. I paricular, wha is he appropriae esimaio mehods for selecig he mos adequae forecasig model. This secio displays he resuls of simulaio sudy. Three fiie sample sizes (50, 64, ad 500 are geeraed from hree differe ARIMA models, amely: IMA(,, ARIMA(,, ad ARIMA(,, wih esimaio parameers as meioed i Table (4.. I each case 000 of simulaios wih 000 replicaios were geeraed by R saisical sofware package ad he value of RMSE were compued for each seleced model, ad sample size, ad esimaio mehod. The complee simulaio resuls are preseed i Table 4.. For usig he mos appropriae esimaio model, IMA (,, esimaio mehod performs early as efficie as he oher esimaio mehods. For example, he RMSE for IMA (, usig, -,, ad for N=50 equal , , , ad , respecively. For oher over esimaio models, he measures of forecas accuracy crieria usig he esimaio mehod does o differ subsaially comparig o he oher esimaio mehods. For example, he RMSE for ARIMA (,, usig, -,, ad for N=500 equal , , , ad.07795, respecively. For he origial daa, OLS performs much less efficie ha he oher esimaio mehods, resulig poor forecasig accuracy. For example, whe N=50 for IMA(,, he RMSE equals However, For he rasformed daa usig he firs differece, OLS performs as early as he oher esimaio mehods. The RMSE i his case equals Model IMA (, ARIMA (,, Mehod - OLS * OLS ** - OLS * Table 4.. RMSE for Simulaed Daa N= N= N= Model ARIMA (,, Mehod OLS ** - OLS * OLS ** N=50 * OLS for he origial daa * OLS for he rasformed daa N= N= Defiiio 4.. The efficiecy of esimaes relaive o ha of E i erms of he MSE of he daa, ˆ ζ, is give by ˆ ζ = k i= k i= ( X ˆ + k X + k, ( X ˆ + k X + k, E (4. where k is he umber of simulaios. A raio less ha oe idicaes ha he esimaes is more efficie ha E, ad if is close o oe, he he esimae is early as efficie as E esimaes. Table 4.3 shows he complee simulaio resuls of he raios of he esimaio mehod relaive o he oher mehods i erms of he MSE of he daa, ˆ ζ i (4.. The able preses he resuls for he hree sample sizes cosidered, as well as all five seleced esimaio mehod for each of he seleced model. Firs, we see ha regardless of he sample size, if he model is correcly specified, i.e. IMA(,, esimaio mehod performs early as efficiely as, -, ad. For example whe N=500, he relaive efficiecy of o ha of, ˆ ζ = For oher over esimaio models, esimaio mehod does o differ sigificaly comparig o he oher esimaio mehods. For example whe N=50 for ARIMA (,,, he relaive efficiecy of o ha of, ˆ ζ =. I addiio, as he sample size icreases, he efficiecy of mimics o ha of he oher esimaio mehods. For example for ARIMA (,,, he relaive efficiecy of o ha of, ˆ.0905,.0408, ad.0357 ζ = whe N=50, 63, ad 500, respecively. To furher demosrae he efficiecy of, cosider OLS for he origial daa for all seleced sample sizes ad all seleced models. is much more efficie i forecasig ha he OLS esimaio mehod. I oher words, OLS for he origial daa performs poorly as show i Table (4.3. For example, he relaive efficiecy of o ha of OLS whe T = 50, 63, ad 500 for IMA(,

11 6 Samir K. Safi ad Ehab. A. Abu Saif: Usig o Geerae Forecass i Regressio Models wih Auo-correlaed Disurbaces wih simulaio ad Palesiia Marke Idex Daa equals ˆ ζ = , , ad , respecively. The superioriy of over OLS is due o he fac ha has a smaller variace ad he auocorrelaed aure of disurbaces is accoued for i he. Accordig o he Geeralized Gauss Markov Theorem, he esimaor provides he Bes Liear Ubiased Esimaor (BLUE of he regressio coefficie. However, OLS for he rasformed daa performs early as efficiely as for all seleced sample sizes ad all seleced models. This resul is o surprisig sice he auocorrelaed aure of disurbaces is accoued for i he rasformed daa. For example, he relaive efficiecy of o ha of OLS for rasformed daa whe T = 50, 63, ad 500 for IMA(, equals ˆ ζ =.0066,.0077, ad.0084, respecively. Table 4.3. Efficiecy for RMSEs of he Esimaors Relaive o oher Mehods Model IMA (, ARIMA (,, ARIMA (,, Mehod - N= N= N= OLS * OLS ** OLS * OLS ** OLS * OLS ** * OLS for he origial daa * OLS for he rasformed daa Table 4.4 shows he acual ad forecasig resuls wih lower ad upper 95% cofidece ierval for he daily closig price Alquds Idex usig IMA (, model. Table 4.4. Acual ad Forecasig wih lower ad upper 95% cofidece ierval of IMA (, model for daily closig price Alquds Idex Moh Lower Forecas Upper Acual May Ju Jul Aug Sep Oc Nov Dec Figure 4. shows he daa ad forecasig resuls wih lower ad upper 95% cofidece ierval of IMA (, model for he daily closig price Alquds Idex. x Figure 4.. Daa ad Forecasig wih lower ad upper 95% cofidece ierval of IMA (, model for daily closig price Alquds Idex The acual values from May 0 o Sep. 0 are observed ad added o Figure 4.. o see if hese pois fall wihi he cofidece ierval. Figure 4. illusraes his ew daa. I is clear ha he ew acual values locaed wihi he cofidece ierval idicaig a excelle forecasig for IMA (, model based o esimaio mehod. x Figure 4.. Full Daa ad Forecasig wih lower ad upper 95% cofidece ierval of IMA (, model for daily closig price Alquds Idex 5. Coclusio ad Fuure Research I his secio, we iroduce coclusio of he mai fidigs ad offer suggesios for fuure research for usig o geerae forecass i regressio models wih auocorrelaed disurbaces. 5.. Coclusio Plo of observed ad prediced i addiio o he forecass wih 95% C.I.s Observed Forecas 95% C.I Plo of observed,prediced,forecass wih 95% C.I.s,ew acual value,ad zoom i he plo Observ ed Forecas 95% C.I. Acual fuure v alue This paper has proposed five differe esimaio mehods, amely:, -,,, ad OLS. We iroduced he accuracy of he forecasig resuls based o RMSE usig ARIMA model o real daa for daily closig price Alquds idex i Palesie ad simulaio echique. The mai fidigs of his paper are as follows: The resuls of boh real daa ad simulaio reveal ha esimaio mehod is comparable o oher complicaed esimaio mehods such as E procedures which ofe require iversio of large marices ad hece preferable as a robus esimaio ad forecasig mehod.

12 America Joural of Theoreical ad Applied Saisics 04; 3(: If he model is correcly specified, i.e. IMA (,, esimaio mehod performs early as efficiely as he oher esimaio mehods as, -, ad. For oher over esimaio models, he esimaio mehod does o differ sigificaly comparig o he oher esimaio mehods. I addiio, as he sample size icreases, he efficiecy of mimics o ha of he oher esimaio mehods. For he origial daa, is much more efficie i forecasig ha he OLS esimaio mehod. However, for he rasformed daa, OLS performs as early as he oher esimaio mehods. Fially, for forecasig purposes here is o much gaied i ryig o ideifyig he exac order ad form of he auo-correlaed disurbaces by usig esimaio mehod. 5.. Fuure Research The plae for fuure research ca be spli io he followig: Firs: Examie he effec o forecasig performace for oher differe models such as ARCH, ad GARCH models ad coduc applicaios i ecoomic ad fiacial forecasig A secod impora cosideraio is he esimaio of he sadard errors of he esimaors. I is uclear, however, how he variace esimaors for esimaio behave for complicaed ime series models. Sudy he impac ha he variace esimaors may have o iferece based o he esimaor. Ackowledgme We would deeply like o hak he referees for heir valuable commes ad suggesios o earlier draf of his paper. Refereces [] Abraham, B. ad Ledoler, J. (005. Saisical mehods for forecasig. Joh Wiley & Sos, Ic. [] Cochrae, J. H. (005. Time Series for Macroecoomics ad Fiace. Uiversiy of Chicago. [3] Cryer, J. ad Cha, K. (008. Time Series Aalysis Wih Applicaios i R, d Ed. Spriger Sciece + Busiess Media, LLC. [4] Everi, B. S. ad Hohor, I. (00. A Hadbook of Saisical Aalyses Usig R, d Ed. Taylor ad Fracis Group, LLC. [5] Fidley, D. (003. Properies of Forecas Errors ad Esimaes of Misspecified RegARIMA ad Iermediae Memory Models ad he Opimaliy of for Oe-Sep- Ahead Forecasig. Mehodology ad Sadards Direcorae U.S. Bureau of he Cesus Washigo D.C [6] Fox, J. (00. Time-Series Regressio ad Geeralized Leas Squares. Appedix o A R ad S-PLUS Compaio o Applied Regressio. [7] Koreisha, G. ad Fag, Y. (004. Forecasig wih serially correlaed regressio models. Joural of Saisical Compuaio ad Simulaio Vol. 74(9, [8] Lee, J. ad Lud, R. (004. Revisiig simple liear regressio wih auocorrelaed errors, Biomerika, Vol. 9 (: doi: 0.093/biome/9..40 [9] Oo, J. ad Olaayo, T. (009. O he Esimaio ad Performace of Subse Auoregressive Iegraed Movig Average Models. Europea Joural of Scieific Research ISSN 450-6X Vol.8 (, [0] Safi, S. (004. The efficiecy of OLS i he presece of auo-correlaed disurbaces i regressio models. PhD disseraio, America Uiversiy Washigo, D.C [] Shiu, O.I. ad Asemoa, M.J. (009. Compariso of Crieria for Esimaig he Order of Auoregressive Process: A Moe Carlo Approach. Europea Joural of Scieific Research ISSN 450-6X Vol. 30(3,