USING ITERATIVE LINEAR REGRESSION MODEL TO TIME SERIES MODELS

Elecronic Journl of Applied Sisicl Anlysis EJASA (202), Elecron. J. App. S. Anl., Vol. 5, Issue 2, 37 50 e-issn 2070-5948, DOI 0.285/i20705948v5n2p37 202 Universià del Sleno hp://sib-ese.unile.i/index.php/ejs/index USING ITERATIVE LINEAR REGRESSION MODEL TO TIME SERIES MODELS Zououi Chikr-el-Mezour *(), Mohmed Aouch (2) () Deprmen of Exc Science, Bechr Universiy, Algeri (2) Deprmen of Mhemics, Djillli Libès Universiy, Algeri Received 6 Sepember 200; Acceped 07 Sepember 20 Avilble online 26 April 202 Absrc: This pper presens n iered liner regression model nd compres is forecsing performnce wih he rdiionl liner regression (LR) nd Box- Jenkins ARIMA models using wo well-known ime series dses: irline d nd sunspo d. The difference beween iered LR nd rdiionl LR is he former considers he error nd uses i s dependen vribles gin o reduce he error re unil error re is very smll. The resuls show h he performnce of iered LR is slighly beer hn Box-J model nd much beer hn rdiionl LR models. Keywords: Liner regression, iered liner regression, ime series nd biliner models.. Inroducion Liner Regression (LR) hs been used o model ime series in vrious fields of pplicions including idenificion nd clssificion nd dynmicl sysems. The complexiy observed nd encounered in ime series suggess he use of LR which hve been proven o be cpble of modeling liner relionship wihou priori ssumpion of he nure of he relionship. Frwy nd Chfield (998) fis vriey of neurl nework models o he well-known irline d nd compred he resuling forecss wih hose obined from he Box-Jenkins, hol- Winers mehods nd liner regression. Mny commercil pckges re vilble for fiing LR models. Here we hve used he MATLAB pckge nd he MINITAB pckge (relese 4). From our experience, we found h he finl resuls of he fiing nd forecsing depend on he * E-mil: chikrime@yhoo.fr 37

Using ierive liner regression model o ime series models choice of hese explnory vribles. In his work he explnory vribles is deermined by deermining he bes subse uoregressive (AR) model o he d using he Box-Jenkins (BJ) echnique nd he AIC nd BIC crierions for he model selecion. Assuming h he resuling model is: p + = φ + + k + + + 0 φ φ k φ p k. 2 2 Then we choose explnory vribles s k,,,. k 2 k Secion 2 defined Iered Liner p Regression mehod. In he following secion we discuses cse sudies. Finlly conclusions re drwn. 2. Iered Liner Regression Mehod (ILR) Suppose h he ime series being genered hrough MA(q), or ARMA(p,q) or biliner BL(p,o,m,k). Now, we propos new mehod for fiing ARMA or biliner models o observed d {, =,2,,N} by enering he error erms s he explnory vribles nd cll his mehod s iered liner regression or repeed residul liner regression. The mehod for fiing n ARMA(p,q) model is described in he following seps:. Fi n LR model using he AR erms k,,,. k 2 k s he explnory vribles, p hen obin he resuling residuls (errors), he error sum of squres nd denoe hem by ( ) ˆ ( ˆ ) ( ) = nd SSE respecively. We propose h hese errors re iniil explnory errors in ARMA. ( ) ( ) ( ) 2. Fi n LR model using k,,,. k 2 k, ˆ ˆ ˆ p, 2,, s explnory vribles, p hen obin he resuling he new residuls (errors) nd he new error sum of squres nd ( 2) ( 2) denoe hem by ˆ nd SSE respecively. 3. Repe he fiing sep [2] unil he men squre errors converge (fer i ierions) if: ( i) ( i ) ( i) SSE SSE SSE < 0.. ( ) 00000 For exmple, n ARMA(2,) model: φ φ2 2 = θ0 + θ, is fied, given relizion {, =,2,,N} s follows:. Fi n LR AR(2) model choosing s dependen vrible nd, 2 s he ( ) explnory vribles. Clcule he corresponding residuls (errors), he error sum ( ) of squres SSE. 2. Fi n ARMA (2,) model by choosing s he dependen vrible nd, 2 nd ( ) ( 2) s he explnory vribles. Clcule he corresponding residuls (errors) ˆ, he ( 2) error sum of squres SSE. 3. Repe sep (2) unil he convergence occurs. We sy Th he procedure converges 38

Zououi C., Aouch M. (202). Elecron. J. App. S. Anl., Vol. 5, Issue 2, 37 50. sep i (fer i ierions) if: ( i) ( i ) ( i) ( SSE SSE ) < 0. 00000 SSE. The sme procedure is pplied for he biliner ime series models. For exmple o fi he biliner model BL(2,0,,): φ φ2 2 = θ0 + + β, given relizion {, =,2,,N}, we proceed s follows:. Fi n AR (2) model by choosing s he dependen vribles,, 2 s he ( ) explnory vribles. Clcule he corresponding residuls (errors) nd he corresponding residul sum of squres SSE. 2. Fi n BL(2,0,,) model by choosing s he dependen vribles,, 2,nd ( ) ( 2) s he explnory vribles. Clcule he corresponding residuls (errors) ˆ, ( 2) he error sum of squres SSE ). 3. Repe sep (2) unil he convergence occurs. We sy h he procedure converges sep ( i) ( i ) ( i) i (fer i ierions) if: SSE SSE SSE < 0.. ( ) 00000 ( ) 3. Cse Sudies Recenly, we hve been winessing lmos exponenil growh in he pplicions of LR fiing o rel ime series d. Nurlly, some of hese pplicions re more successful hn ohers. In ll cses, he experiences repored re very vluble. Some of hese pplicions hve rced he mos enion mong liner ime series nlyss. Choosing he bes model will involve obining some imporn vlues in fiing nd forecsing phses like SSE (Sum Squred Errors), AIC (Akike Informion Crierion) which will be compred o heir corresponding vlues when pplying he new LR mehod. For our illusrion nd comprisons, we consider wo ses of rel d, irline d nd sunspo numbers. For ech se of d, he ol number of observions is denoed by T, he firs N re used for fiing he models nd he remining observions T-N (=M) re used for predicions. The effecive number of observions is denoed by n (n=n-mximum lg). For ech model fied, we compue he following sisics [2]: ( ) 2 ) SSE, he sum of squred residuls: SSE = ˆ, where, b) σ = SSE ( n ν ) nd ˆ re he rue nd prediced oupu. ˆ, where ν is he number of prmeers used. SSE c) he Akike informion crierion (AIC): AIC = nln + 2ν. n N = 39

Using ierive liner regression model o ime series models SSE d) The Byesin informion crierion (BIC): BIC = nln +ν + ν ln( n). n N ˆ e) ARE, he verge relive error (used in fiing phse): ARE = = + 00, n γ where γ, is he mximum lg. f) ARE f, he verge relive error (used in forecsing phse): M ˆ ARE f = = 00. M n g) MSE, he men sum of squred residuls (used in fiing phse): MSE ( ˆ = i i). n i = h) MSE f, he men sum of squred residuls (used in forecsing phse): M MSE ( ˆ f = i i). M i = 3. The Airline D The irline d comprises monhly ols of inernionl irline pssengers from Jnury 949 o December 960 (see [] nd [2]). Figure () shows h he d hve n upwrd rend ogeher wih sesonl vriion whose size is roughly proporionl o he locl men level (clled muliplicive sesonliy). The presence of sesonliy ws one reson for choosing ([2]) his d se. A common pproch o deling wih his ype of sesonliy is o choose rnsformion, usully logrihms, o mke he sesonliy ddiive (figure (b)). We denoe he originl d by {x } nd he rnsformed d by {y } (y =ln(x )). 600 number of pssengers x 500 400 300 200 00 24 48 72 96 20 44 Monh Fig (). Airline d monhly ols (in housnd) of inernionl irline pssengers from Jnury 949 o December 960: nurl logrihms. 40

Zououi C., Aouch M. (202). Elecron. J. App. S. Anl., Vol. 5, Issue 2, 37 50. 6.5 log(no. of pssengers) y 6.0 5.5 5.0 4.5 24 48 72 96 20 44 Monh Fig (). Airline d monhly ols (in housnd) of inernionl irline pssengers from Jnury 949 o December 960: nurl logrihms. i. Box-Jenkins Model: The sndrd Box-Jenkins nlysis ([] nd [3]) involves king nurl logrihms of he d following by sesonl nd non-sesonl differencing o mke he series sionry. A specil ype of sesonl uoregressive inegred moving verge (SARIMA) model, of order 0,, 0,, in he usul noion ([]), is hen fied which hs he form: ( ) ( ) 2 2 2 ( B)( B ) y = ( 0.64B)( + 0.370B ), in which we hve ˆ θ 0.396, ˆ = Θ = 0. 64. This model is ofen clled he irline model nd is used s he yrdsick for fuure comprisons, hough oher SARIMA models could be found wih similr fi nd forecs ccurcy. For he irline model fied o he irline d wih N=32 nd M=2, he MINITAB pckge (relese 4) gve he following vlues (fer bckrnsforming ll forecss from he model for he logged d ino he originl unis): SSE = 0789, SS IS = 4328, ˆ σ = 9.522, ARE f = 2.656 ARE = 2.90278, AIC = 540.35, BIC = 547.9, ii. Clssicl LR Model Using liner regression o fi model wih y -, y -2, y -3 s he explnory vribles nd y s dependen vrible. The liner regression equion (for he d fer scling by dividing by 00 which chnges he consn bu no he oher coefficiens) is: y = 0.0322 + 0.7824 y y +.0720 y 2 0. 8394 y 3 iii. Iered LR Model Using liner regression o fi model wih y -, y -2, y -3, e -, e -9 s he explnory vribles nd y s dependen vrible. The liner regression equion (for he d fer scling by dividing by 00 which chnges he consn bu no he oher coefficiens) is: 4

Using ierive liner regression model o ime series models y = 0.087 + 0.8384y ˆ +.0804y 2 0.949y 3 0.983ˆ e + 0.826e 9 Tble. Conins he resuls of he clssicl LR nd iered LR models for he irline d N=32, n=9, M=2 monhs. Mesures of fi Forecs ccurcy n-p SSE σ ARE AIC BIC ARE SS IS y, y 2, y 3 4.807 0.03 3.0765-540.94-525.83 3.3828 0.5073 y e, y, e 2 9, y 3 6.50 0.0993 3.0244-543.76-52.09 3.094 0.40 n-p: number of prmeers. From he ble bove we cn see h ILR model is beer hn he LR in rining nd forecsing using o irline d..2.8.6 mse.4.2..08 0 2 4 6 8 0 2 4 6 8 20 n Fig 2. Convergence of MSE. From he figure bove we cn see h he convergence of MSE ccure fer he fourh ierion. 42

Zououi C., Aouch M. (202). Elecron. J. App. S. Anl., Vol. 5, Issue 2, 37 50. 6.5 6 originl d ARMA AR 5.5 irline d 5 4.5 4 3.5 0 2 4 6 8 0 2 monh Fig 3. Forecsing of irline d. From he figure bove we cn see h he ILR model is beer hn he LR model in forecsing. Probbiliy Plo of eir Norml Percen 99.9 99 95 90 80 70 60 50 40 30 20 0 5 Men -0.00000336 SDev 0.000 N 9 AD 0.304 P-Vlue 0.566 0. -0.4-0.3-0.2-0. 0.0 eir 0. 0.2 0.3 Fig 4 Normliy es of he errors obined from LR model. From he figure bove we cn see h he errors obined from LR model re normlly disribued. 43

Using ierive liner regression model o ime series models Probbiliy Plo of e2ir Norml Percen 99.9 99 95 90 80 70 60 50 40 30 20 0 5 Men -0.0000068 SDev 0.0972 N 9 AD 0.97 P-Vlue 0.886 0. -0.4-0.3-0.2-0. 0.0 e2ir 0. 0.2 0.3 Fig 5. Normliy es of he errors obined from Iered LR model. From he figure bove we cn see h he errors obined from ILR model re normlly disribued. From he bove resuls, we see h he ierive LR pproch is beer hn boh he clssicl LR models nd he Box-Jenkins model in boh fiing nd forecsing. 3.2 Sunspo d The d we consider is he clssic series of he Wolf yerly sunspo numbers for he yers 700-988. This series hs cerin hisoric ineres for sisicins, see, [4] nd he references in i nd [5]. Scieniss believe h he sunspo numbers ffec he weher of he erh nd hence humn civiies such s griculure, elecommunicions, nd ohers. I is believed by mny scieniss h his series hs n eleven yer cycle. The plo of he series, Fig.6, nd he uocorrelion funcion indice h he series is sionry in he men. However, squre roo rnsformion is suggesed o be pplied for he series o be sionry lso in he vrince. The squre roo of he d is shown in Fig.7. Subb Ro nd Gbr (984) fied subse AR nd subse BL models o he originl nnul sunspo numbers of 700-920, N=22, using he dbnk vilble he Universiy of Mncheser Insiue of Science & Technology, UK, he ime which conined he d only for he period 700-955. They used he nex 35 observions (92-955) for predicions. The fied subse AR model is :.2496 + 0.55 2 0. 45 9 = wih he men squred error of fiing, MSE=203.2 nd men squred error of predicion 24.. The fied subse BL model is 44

Zououi C., Aouch M. (202). Elecron. J. App. S. Anl., Vol. 5, Issue 2, 37 50..502 + 0.767 2 0.52 9 0.00752 + 0.004334 = 6.886 0.458 2 3 4 0.006047 + 0.00782 2 4 3 3 2 + 0.00632 + 0.00369 + 8 6 wih he men squred error of fiing, MSE=24.33 nd men squred error of predicion 23.77. 200 50 sunspo numbers 00 50 0 700 750 800 850 900 950 Yer Fig 6. Wlfer sunspo numbers for he yers 700-988. 4 Squre roo of sunspo numbers Y 2 0 8 6 4 2 0 700 750 800 850 900 950 Yer Fig 7. Squre roo of he sunspo numbers for he yers 700-988. Wei (990) fied he following Box-Jenkins model o he squre roo of nnul sunspo numbers of 700-984 (N=285); 45

Using ierive liner regression model o ime series models 2 9 (.7B + 0.46B + 0.2B )( Y 6. 3) = wih he men squred error of fiing, MSE=.362. Now we use he nnul sunspo numbers wih sme fiing period, 700-979, he sme predicion period 980-987 nd he sme squre roo rnsformion. i. Box-Jenkins Model Using he BJ mehodology on he squre roo of sunspo d for he period 700-979, N=280 for fiing, we found h he bes ARMA model ccording o boh, he AIC nd BIC crierion s, is given by: Y.233Y + 0.4609 2 0.2283 9 = 0. 726 wih men squred error, MSE= vr( ) =.4 nd men squred error of predicion (fer bckrnsforming ll forecss ccurcy from he model for he squre roo d ino he originl unis) is 37.36. ii. Clssicl LR Model The liner Regression model is fied o he squre roo of sunspo d for he period 700-979, N=280 nd hen predicors re clculed for he nex 8 (M) observions. The equion obined is: Y = 0.6733 +.2752Y 0.5433Y 2 + 0. 590Y 9 iii. Iered LR Model The equion obined is: Y = 0.7267 +.2724Y 0.0069Y e 4 5 0.5464Y 0.080e 6 2 + 0.575Y 9 + 0.0Y e 4 Tble 2 conins he resuls of he clssicl LR nd iered LR models sunspo d for he period 700-979, N=280 nd hen predicors re clculed for he nex 8 (M) observions. Mesures of fi Forecs ccurcy n-p* y,y 2,y 9 y,y,y,y e 2 9 4,y e,e 4 5 6 *n-p: number of prmeers. MSE σ AIC BIC SS IS MSS 4.245.0706 3.87 50.49 093.6 36.6992 7.07.0674 34.54 65.04 787.0 98.37 From he ble bove we cn see h ILR model is beer hn he LR in rining nd forecsing using o sunspo d. IS 46

Zououi C., Aouch M. (202). Elecron. J. App. S. Anl., Vol. 5, Issue 2, 37 50. Tble 3 Conins one sep hed predicions of sunspo numbers (980-7) wih 979 s he bse yer using he differen models. Subse AR BL Yer Observion Predicion Errors Predicion Errors 980 54.7 59.8028-5.028 55.2547-0.5547 98 40.5 22.7683 7.737 28.7530.7470 982 5.9 00.2049 5.695 06.6250 9.2750 983 66.6 79.74-2.574 75.2996-8.6996 984 45.9 34.2955.6045 30.0708 5.8292 985 7.9 29.6227 -.7227 29.5248 -.6248 986 3.4 0.3749 3.025 0.466 2.9834 987 29.2 20.9005 8.2995 9.5876 9.624 MSE 36.6992 98.3758 The ble bove shows h he MSE obined by using ILR model is smule hn he obined by using LR..3.25.2.5 mse..05..095.5 2 2.5 3 3.5 4 4.5 5 5.5 6 n Fig 8. Convergence of MSE. From he figure bove we cn see h he convergence of MSE ccure fer he fourh ierion. 47

Using ierive liner regression model o ime series models 60 40 originl d BL AR 20 sunspo d 00 80 60 40 20 0 2 3 4 5 6 7 8 yer Fig 9. Forecsing sunspo d. From he figure bove we cn see h he ILR model is beer hn he LR model in forecsing. Probbiliy Plo of erors-lr Norml Percen 99.9 99 95 90 80 70 60 50 40 30 20 0 5 Men 0.000003302 SDev.063 N 22 AD 0.39 P-Vlue 0.378 0. -4-3 -2-0 erors-lr 2 3 4 Fig 0. Normliy es of he errors obined from LR model. From he figure bove we cn see h he errors obined from LR model re normlly disribued. 48

Zououi C., Aouch M. (202). Elecron. J. App. S. Anl., Vol. 5, Issue 2, 37 50. Probbiliy Plo of erors-ilr Norml Percen 99.9 99 95 90 80 70 60 50 40 30 20 0 5 Men 0.000002358 SDev.052 N 22 AD 0.563 P-Vlue 0.44 0. -4-3 -2-0 erors-ilr 2 3 4 Fig. Normliy es of he errors obined from Iered LR model. From he figure bove we cn see h he errors obined from ILR model re normlly disribued. From he bove resuls, we see h he ierive LR pproch is beer hn boh he clssicl LR models nd he Box-Jenkins model in boh fiing nd forecsing. 4. Conclusions. Fiing nd forecsing using iered LR mehod were beer compred o he clssicl LR mehod. 2. The resuls obined by he pplicion of he iered LR mehod o irline d ws found o be beer hn h obined by Frwy nd Chfiled (998) 3. The resuls obined by he pplicion of he iered LR mehod o he sunspo numbers were found o be beer compred o h obined by he subse AR nd subse BL (Subb Ro nd Gbr (984)). 4. The errors obined from LR nd ILR re normlly disribued (see Fig0 nd Fig). 49

Using ierive liner regression model o ime series models References []. Box, G.E.P., Jenkins, G.M. nd Reinsel, G.C. (994). Time Series Anlysis: Forecsing nd Conrol 3 rd Ed., Englewood Cliffs, New Jersey: Prenice Hll. [2]. Frwy, J. nd Chfield, C. (998). Time series Forecsing wih Neurl Neworks: A Comprive Sudy Using he Airline D. Appl. Sis., 47 (2), 23-250. [3]. Hrvey, A.C. (993). Time Series Models 2 nd Ed. Hemel Hempsed: Hrveser Wheshef. [4]. Subb Ro, T., Gbr, M.M. (984). An Inroducion o Bispecrl Anlysis nd Biliner Time Series Models. Lecure Noes in Sisics, 24, Berlin: Springer-Verlg. [5]. Wei, W.W.S. (990). Time series nlysis: Univrie nd mulivrie mehods. Addison- Wesley Publishing Compny, Inc This pper is n open ccess ricle disribued under he erms nd condiions of he Creive Commons Aribuzione - Non commercile - Non opere derive 3.0 Ili License. 50