USING ITERATIVE LINEAR REGRESSION MODEL TO TIME SERIES MODELS

Transcription

1 Elecronic Journl of Applied Sisicl Anlysis EJASA (202), Elecron. J. App. S. Anl., Vol. 5, Issue 2, e-issn , DOI 0.285/i v5n2p Universià del Sleno hp://sib-ese.unile.i/index.php/ejs/index USING ITERATIVE LINEAR REGRESSION MODEL TO TIME SERIES MODELS Zououi Chikr-el-Mezour *(), Mohmed Aouch (2) () Deprmen of Exc Science, Bechr Universiy, Algeri (2) Deprmen of Mhemics, Djillli Libès Universiy, Algeri Received 6 Sepember 200; Acceped 07 Sepember 20 Avilble online 26 April 202 Absrc: This pper presens n iered liner regression model nd compres is forecsing performnce wih he rdiionl liner regression (LR) nd Box- Jenkins ARIMA models using wo well-known ime series dses: irline d nd sunspo d. The difference beween iered LR nd rdiionl LR is he former considers he error nd uses i s dependen vribles gin o reduce he error re unil error re is very smll. The resuls show h he performnce of iered LR is slighly beer hn Box-J model nd much beer hn rdiionl LR models. Keywords: Liner regression, iered liner regression, ime series nd biliner models.. Inroducion Liner Regression (LR) hs been used o model ime series in vrious fields of pplicions including idenificion nd clssificion nd dynmicl sysems. The complexiy observed nd encounered in ime series suggess he use of LR which hve been proven o be cpble of modeling liner relionship wihou priori ssumpion of he nure of he relionship. Frwy nd Chfield (998) fis vriey of neurl nework models o he well-known irline d nd compred he resuling forecss wih hose obined from he Box-Jenkins, hol- Winers mehods nd liner regression. Mny commercil pckges re vilble for fiing LR models. Here we hve used he MATLAB pckge nd he MINITAB pckge (relese 4). From our experience, we found h he finl resuls of he fiing nd forecsing depend on he * E-mil: chikrime@yhoo.fr 37

2 Using ierive liner regression model o ime series models choice of hese explnory vribles. In his work he explnory vribles is deermined by deermining he bes subse uoregressive (AR) model o he d using he Box-Jenkins (BJ) echnique nd he AIC nd BIC crierions for he model selecion. Assuming h he resuling model is: p + = φ + + k φ φ k φ p k. 2 2 Then we choose explnory vribles s k,,,. k 2 k Secion 2 defined Iered Liner p Regression mehod. In he following secion we discuses cse sudies. Finlly conclusions re drwn. 2. Iered Liner Regression Mehod (ILR) Suppose h he ime series being genered hrough MA(q), or ARMA(p,q) or biliner BL(p,o,m,k). Now, we propos new mehod for fiing ARMA or biliner models o observed d {, =,2,,N} by enering he error erms s he explnory vribles nd cll his mehod s iered liner regression or repeed residul liner regression. The mehod for fiing n ARMA(p,q) model is described in he following seps:. Fi n LR model using he AR erms k,,,. k 2 k s he explnory vribles, p hen obin he resuling residuls (errors), he error sum of squres nd denoe hem by ( ) ˆ ( ˆ ) ( ) = nd SSE respecively. We propose h hese errors re iniil explnory errors in ARMA. ( ) ( ) ( ) 2. Fi n LR model using k,,,. k 2 k, ˆ ˆ ˆ p, 2,, s explnory vribles, p hen obin he resuling he new residuls (errors) nd he new error sum of squres nd ( 2) ( 2) denoe hem by ˆ nd SSE respecively. 3. Repe he fiing sep [2] unil he men squre errors converge (fer i ierions) if: ( i) ( i ) ( i) SSE SSE SSE < 0.. ( ) For exmple, n ARMA(2,) model: φ φ2 2 = θ0 + θ, is fied, given relizion {, =,2,,N} s follows:. Fi n LR AR(2) model choosing s dependen vrible nd, 2 s he ( ) explnory vribles. Clcule he corresponding residuls (errors), he error sum ( ) of squres SSE. 2. Fi n ARMA (2,) model by choosing s he dependen vrible nd, 2 nd ( ) ( 2) s he explnory vribles. Clcule he corresponding residuls (errors) ˆ, he ( 2) error sum of squres SSE. 3. Repe sep (2) unil he convergence occurs. We sy Th he procedure converges 38

3 Zououi C., Aouch M. (202). Elecron. J. App. S. Anl., Vol. 5, Issue 2, sep i (fer i ierions) if: ( i) ( i ) ( i) ( SSE SSE ) < SSE. The sme procedure is pplied for he biliner ime series models. For exmple o fi he biliner model BL(2,0,,): φ φ2 2 = θ0 + + β, given relizion {, =,2,,N}, we proceed s follows:. Fi n AR (2) model by choosing s he dependen vribles,, 2 s he ( ) explnory vribles. Clcule he corresponding residuls (errors) nd he corresponding residul sum of squres SSE. 2. Fi n BL(2,0,,) model by choosing s he dependen vribles,, 2,nd ( ) ( 2) s he explnory vribles. Clcule he corresponding residuls (errors) ˆ, ( 2) he error sum of squres SSE ). 3. Repe sep (2) unil he convergence occurs. We sy h he procedure converges sep ( i) ( i ) ( i) i (fer i ierions) if: SSE SSE SSE < 0.. ( ) ( ) 3. Cse Sudies Recenly, we hve been winessing lmos exponenil growh in he pplicions of LR fiing o rel ime series d. Nurlly, some of hese pplicions re more successful hn ohers. In ll cses, he experiences repored re very vluble. Some of hese pplicions hve rced he mos enion mong liner ime series nlyss. Choosing he bes model will involve obining some imporn vlues in fiing nd forecsing phses like SSE (Sum Squred Errors), AIC (Akike Informion Crierion) which will be compred o heir corresponding vlues when pplying he new LR mehod. For our illusrion nd comprisons, we consider wo ses of rel d, irline d nd sunspo numbers. For ech se of d, he ol number of observions is denoed by T, he firs N re used for fiing he models nd he remining observions T-N (=M) re used for predicions. The effecive number of observions is denoed by n (n=n-mximum lg). For ech model fied, we compue he following sisics [2]: ( ) 2 ) SSE, he sum of squred residuls: SSE = ˆ, where, b) σ = SSE ( n ν ) nd ˆ re he rue nd prediced oupu. ˆ, where ν is he number of prmeers used. SSE c) he Akike informion crierion (AIC): AIC = nln + 2ν. n N = 39

4 Using ierive liner regression model o ime series models SSE d) The Byesin informion crierion (BIC): BIC = nln +ν + ν ln( n). n N ˆ e) ARE, he verge relive error (used in fiing phse): ARE = = + 00, n γ where γ, is he mximum lg. f) ARE f, he verge relive error (used in forecsing phse): M ˆ ARE f = = 00. M n g) MSE, he men sum of squred residuls (used in fiing phse): MSE ( ˆ = i i). n i = h) MSE f, he men sum of squred residuls (used in forecsing phse): M MSE ( ˆ f = i i). M i = 3. The Airline D The irline d comprises monhly ols of inernionl irline pssengers from Jnury 949 o December 960 (see [] nd [2]). Figure () shows h he d hve n upwrd rend ogeher wih sesonl vriion whose size is roughly proporionl o he locl men level (clled muliplicive sesonliy). The presence of sesonliy ws one reson for choosing ([2]) his d se. A common pproch o deling wih his ype of sesonliy is o choose rnsformion, usully logrihms, o mke he sesonliy ddiive (figure (b)). We denoe he originl d by {x } nd he rnsformed d by {y } (y =ln(x )). 600 number of pssengers x Monh Fig (). Airline d monhly ols (in housnd) of inernionl irline pssengers from Jnury 949 o December 960: nurl logrihms. 40

5 Zououi C., Aouch M. (202). Elecron. J. App. S. Anl., Vol. 5, Issue 2, log(no. of pssengers) y Monh Fig (). Airline d monhly ols (in housnd) of inernionl irline pssengers from Jnury 949 o December 960: nurl logrihms. i. Box-Jenkins Model: The sndrd Box-Jenkins nlysis ([] nd [3]) involves king nurl logrihms of he d following by sesonl nd non-sesonl differencing o mke he series sionry. A specil ype of sesonl uoregressive inegred moving verge (SARIMA) model, of order 0,, 0,, in he usul noion ([]), is hen fied which hs he form: ( ) ( ) ( B)( B ) y = ( 0.64B)( B ), in which we hve ˆ θ 0.396, ˆ = Θ = This model is ofen clled he irline model nd is used s he yrdsick for fuure comprisons, hough oher SARIMA models could be found wih similr fi nd forecs ccurcy. For he irline model fied o he irline d wih N=32 nd M=2, he MINITAB pckge (relese 4) gve he following vlues (fer bckrnsforming ll forecss from he model for he logged d ino he originl unis): SSE = 0789, SS IS = 4328, ˆ σ = 9.522, ARE f = ARE = , AIC = , BIC = 547.9, ii. Clssicl LR Model Using liner regression o fi model wih y -, y -2, y -3 s he explnory vribles nd y s dependen vrible. The liner regression equion (for he d fer scling by dividing by 00 which chnges he consn bu no he oher coefficiens) is: y = y y y y 3 iii. Iered LR Model Using liner regression o fi model wih y -, y -2, y -3, e -, e -9 s he explnory vribles nd y s dependen vrible. The liner regression equion (for he d fer scling by dividing by 00 which chnges he consn bu no he oher coefficiens) is: 4

6 Using ierive liner regression model o ime series models y = y ˆ y y ˆ e e 9 Tble. Conins he resuls of he clssicl LR nd iered LR models for he irline d N=32, n=9, M=2 monhs. Mesures of fi Forecs ccurcy n-p SSE σ ARE AIC BIC ARE SS IS y, y 2, y y e, y, e 2 9, y n-p: number of prmeers. From he ble bove we cn see h ILR model is beer hn he LR in rining nd forecsing using o irline d mse n Fig 2. Convergence of MSE. From he figure bove we cn see h he convergence of MSE ccure fer he fourh ierion. 42

7 Zououi C., Aouch M. (202). Elecron. J. App. S. Anl., Vol. 5, Issue 2, originl d ARMA AR 5.5 irline d monh Fig 3. Forecsing of irline d. From he figure bove we cn see h he ILR model is beer hn he LR model in forecsing. Probbiliy Plo of eir Norml Percen Men SDev N 9 AD P-Vlue eir Fig 4 Normliy es of he errors obined from LR model. From he figure bove we cn see h he errors obined from LR model re normlly disribued. 43

8 Using ierive liner regression model o ime series models Probbiliy Plo of e2ir Norml Percen Men SDev N 9 AD 0.97 P-Vlue e2ir Fig 5. Normliy es of he errors obined from Iered LR model. From he figure bove we cn see h he errors obined from ILR model re normlly disribued. From he bove resuls, we see h he ierive LR pproch is beer hn boh he clssicl LR models nd he Box-Jenkins model in boh fiing nd forecsing. 3.2 Sunspo d The d we consider is he clssic series of he Wolf yerly sunspo numbers for he yers This series hs cerin hisoric ineres for sisicins, see, [4] nd he references in i nd [5]. Scieniss believe h he sunspo numbers ffec he weher of he erh nd hence humn civiies such s griculure, elecommunicions, nd ohers. I is believed by mny scieniss h his series hs n eleven yer cycle. The plo of he series, Fig.6, nd he uocorrelion funcion indice h he series is sionry in he men. However, squre roo rnsformion is suggesed o be pplied for he series o be sionry lso in he vrince. The squre roo of he d is shown in Fig.7. Subb Ro nd Gbr (984) fied subse AR nd subse BL models o he originl nnul sunspo numbers of , N=22, using he dbnk vilble he Universiy of Mncheser Insiue of Science & Technology, UK, he ime which conined he d only for he period They used he nex 35 observions (92-955) for predicions. The fied subse AR model is : = wih he men squred error of fiing, MSE=203.2 nd men squred error of predicion 24.. The fied subse BL model is 44

9 Zououi C., Aouch M. (202). Elecron. J. App. S. Anl., Vol. 5, Issue 2, = wih he men squred error of fiing, MSE=24.33 nd men squred error of predicion sunspo numbers Yer Fig 6. Wlfer sunspo numbers for he yers Squre roo of sunspo numbers Y Yer Fig 7. Squre roo of he sunspo numbers for he yers Wei (990) fied he following Box-Jenkins model o he squre roo of nnul sunspo numbers of (N=285); 45

10 Using ierive liner regression model o ime series models 2 9 (.7B B + 0.2B )( Y 6. 3) = wih he men squred error of fiing, MSE=.362. Now we use he nnul sunspo numbers wih sme fiing period, , he sme predicion period nd he sme squre roo rnsformion. i. Box-Jenkins Model Using he BJ mehodology on he squre roo of sunspo d for he period , N=280 for fiing, we found h he bes ARMA model ccording o boh, he AIC nd BIC crierion s, is given by: Y.233Y = wih men squred error, MSE= vr( ) =.4 nd men squred error of predicion (fer bckrnsforming ll forecss ccurcy from he model for he squre roo d ino he originl unis) is ii. Clssicl LR Model The liner Regression model is fied o he squre roo of sunspo d for he period , N=280 nd hen predicors re clculed for he nex 8 (M) observions. The equion obined is: Y = Y Y Y 9 iii. Iered LR Model The equion obined is: Y = Y Y e Y 0.080e Y Y e 4 Tble 2 conins he resuls of he clssicl LR nd iered LR models sunspo d for he period , N=280 nd hen predicors re clculed for he nex 8 (M) observions. Mesures of fi Forecs ccurcy n-p* y,y 2,y 9 y,y,y,y e 2 9 4,y e,e *n-p: number of prmeers. MSE σ AIC BIC SS IS MSS From he ble bove we cn see h ILR model is beer hn he LR in rining nd forecsing using o sunspo d. IS 46

11 Zououi C., Aouch M. (202). Elecron. J. App. S. Anl., Vol. 5, Issue 2, Tble 3 Conins one sep hed predicions of sunspo numbers (980-7) wih 979 s he bse yer using he differen models. Subse AR BL Yer Observion Predicion Errors Predicion Errors MSE The ble bove shows h he MSE obined by using ILR model is smule hn he obined by using LR mse n Fig 8. Convergence of MSE. From he figure bove we cn see h he convergence of MSE ccure fer he fourh ierion. 47

12 Using ierive liner regression model o ime series models originl d BL AR 20 sunspo d yer Fig 9. Forecsing sunspo d. From he figure bove we cn see h he ILR model is beer hn he LR model in forecsing. Probbiliy Plo of erors-lr Norml Percen Men SDev.063 N 22 AD 0.39 P-Vlue erors-lr Fig 0. Normliy es of he errors obined from LR model. From he figure bove we cn see h he errors obined from LR model re normlly disribued. 48

13 Zououi C., Aouch M. (202). Elecron. J. App. S. Anl., Vol. 5, Issue 2, Probbiliy Plo of erors-ilr Norml Percen Men SDev.052 N 22 AD P-Vlue erors-ilr Fig. Normliy es of he errors obined from Iered LR model. From he figure bove we cn see h he errors obined from ILR model re normlly disribued. From he bove resuls, we see h he ierive LR pproch is beer hn boh he clssicl LR models nd he Box-Jenkins model in boh fiing nd forecsing. 4. Conclusions. Fiing nd forecsing using iered LR mehod were beer compred o he clssicl LR mehod. 2. The resuls obined by he pplicion of he iered LR mehod o irline d ws found o be beer hn h obined by Frwy nd Chfiled (998) 3. The resuls obined by he pplicion of he iered LR mehod o he sunspo numbers were found o be beer compred o h obined by he subse AR nd subse BL (Subb Ro nd Gbr (984)). 4. The errors obined from LR nd ILR re normlly disribued (see Fig0 nd Fig). 49

14 Using ierive liner regression model o ime series models References []. Box, G.E.P., Jenkins, G.M. nd Reinsel, G.C. (994). Time Series Anlysis: Forecsing nd Conrol 3 rd Ed., Englewood Cliffs, New Jersey: Prenice Hll. [2]. Frwy, J. nd Chfield, C. (998). Time series Forecsing wih Neurl Neworks: A Comprive Sudy Using he Airline D. Appl. Sis., 47 (2), [3]. Hrvey, A.C. (993). Time Series Models 2 nd Ed. Hemel Hempsed: Hrveser Wheshef. [4]. Subb Ro, T., Gbr, M.M. (984). An Inroducion o Bispecrl Anlysis nd Biliner Time Series Models. Lecure Noes in Sisics, 24, Berlin: Springer-Verlg. [5]. Wei, W.W.S. (990). Time series nlysis: Univrie nd mulivrie mehods. Addison- Wesley Publishing Compny, Inc This pper is n open ccess ricle disribued under he erms nd condiions of he Creive Commons Aribuzione - Non commercile - Non opere derive 3.0 Ili License. 50