Multivariate Regression: A Very Powerful Forecasting Method

Transcription

1 Archves of Busess Reserch Vol., No. Pulco De: Jue. 5, 8 DOI:.78/r..7. Vslooulos. (8). Mulvre Regresso: A Very Powerful Forecsg Mehod. Archves of Busess Reserch, (), 8. Mulvre Regresso: A Very Powerful Forecsg Mehod Vslooulos, Ph.D. ST. JON S UNIVERSITY The Peer J. To College of Busess CIS/DS derme ABSTRACT Regresso Alyss s he ceer of lmos every Forecsg echque, ye few eole re comforle wh he Regresso mehodology. We hoe o grely mrove he level of comfor wh hs rcle. ere we refly dscuss he heory ehd he mehodology d he oule seyse rocedure, whch wll llow lmos everyoe o cosruc Regresso Forecsg fuco for oh he ler d Mulvre cse. The Ler Regresso s show o e secl cse of he mulvre rolem. Also dscussed, ddo o model formo d esmo, s model esg (o eslsh sscl sgfcce of fcors) d he Procedure y whch he fl regresso equo s oed from he esmed equo. The Fl Regresso Equo s reed d used s he forecsg equo. A hd soluo s derved for relvely smll smle rolem, d hs soluo s comred o he MINITABderved soluo o eslsh cofdece he sscl ool, whch he c e used exclusvely for lrger rolems. Key Words: Mulvre Regresso, Mrx Alger, Ler Regresso, Esmed Equo, Fl Equo INTRODUCTION AND MODEL ESTIMATION FOR TE MULTIVARIATE PROBLEM Mulvre Regresso lyss, whch equo s derved h coecs he vlue of oe deede vrle (Y) o he vlues of deede vrles,,, srs wh gve mulvre d se d uses he Les Squres mehod o ssg he es ossle vlues o he ukow mullers foud he model we wsh o esme. The mulvre d se used o esme he mulvre model cosss of ules of vlues: ( x x,, x, y ), ( x, x,, x, y ),( x, x x, y ),, ) Esmo of he Model The mulvre model s gve y: Y () or Y ( ) () Noe h he frs erms of he mulvre model gve y equo () re decl o he ler model Y + d equo () we roduced vrle, whose vlue s lwys equl o (f we wsh he model o hve cos erm), o mke he hdlg of he mulvre model eser, usg mrx oeros. Noe lso h he equo () s se equl o equo (). To esme he Mulvre model, we use he Les Squres Mehodology, whch clls for he formo of he Qudrc fuco:

2 Archves of Busess Reserch (ABR) Q(,,, [ y y ] ) cul Mul vr le fuco "# y $ % '() Vol., Issue, Jue8 () To derve he Norml Equos for he Mulvre model, from whch he vlues of:,,,, re derved, we ke rl dervves of he Q(,,,, d ) fuco wh resec o,,,, d resecvely, d se ech equl o ;.e. we o, d se equl o zero: Q Q Q () owever, whe emg o solve he se of equos () lgerclly, he resuls re very comlced, d s dvsle o se he resulg Norml equos mrx form, y whch hey re sed s: ( ) ( Y) (5) where: Mrx formed from he vlues of he deede vrles ( hs rows d colums, or s x mrx), Trsosed mrx ( hs rows d colums, or s x mrx) Y Colum Vecor (or x mrx) of he gve Y vlues Colum Vecor (or x mrx) of he ukow mullers,,,, The Mulvre d se, from whch he mrces:,, Y, d re defed, hs he srucure show elow: Y Y Y Y Y () The vlues uder vrle + (.e. ++, +, +.,, + ) c ech e se equl o o mke sure he mulvre equo hs cos erm. The, from equo () we defe he mrces,,, d, d form he Mrx Producs d eeded equo (5). We o: Coyrgh Socey for Scece d Educo, Ued Kgdom 5

3 Vslooulos. (8). Mulvre Regresso: A Very Powerful Forecsg Mehod. Archves of Busess Reserch, (), 8.. (7) (8) where s x mrx, The Trsosed of mrx, hs he rows d colums erchged such h f s α x mrx. +. ;.e. Y s colum vecor or x mrx (9) +. ;.e. s colum vecor or x mrx () The mrx roducs erg equo (5) re ll defed d hve he dmesoles: s x mrx ( ) s x mrx () s x mrx d where Y y + y y x y x y x + y + y x + y x + y x + + y + y x + y x + y x + + y x + + y x y x () URL: h://dx.do.org/.78/r..7.

4 Archves of Busess Reserch (ABR) Vol., Issue, Jue8 ' ' '.' '.' + ' '7 ' ' ' ' The mrx soluo o equo (5) s gve y: ( ) ( Y ) () where ( ) s he Iverse of Mrx (see equo () ove) whch c e foud usg eher he GussElmo mehod or he Adjo Mrx mehod. Noe: If he mrx s Dgol,.e. hs ozero elemes oly log he m dgol, fdg he Iverse mrx s rvl. d d For exmle, f d, ( ). (5) d d d To comlee he esmo of he mulvre model we eed o frs fd he vrces V( ), V( ), V( ),, V( ) from whch he we c o: ) V( ),, ( ) V( ). The vrce of he vecor ( V() ( ) where ˆ d Y Y y y + y + + y vecor, or. ) s gve y: ( ˆ () Y Y Y Q, (7) ( )., Y ws derved equo () d s he rsosed of Afer equo (7) s susued o equo () d he mullco of he mrx Coyrgh Socey for Scece d Educo, Ued Kgdom 7

5 Vslooulos. (8). Mulvre Regresso: A Very Powerful Forecsg Mehod. Archves of Busess Reserch, (), 8. ( ) y ˆ kes lce, V() ssumes he form: V ( ) V ( V() Co vrce Terms ) V ( ) Co vrce Terms.. V ( ) Therefore, he vrces V( ), V( ),, V( ) re he vlues log he m dgol of he V() mrx, whle he offhemdgol erms re Covrce erms. Noe: A hs o, we hve, for he gve d se:,,,, d σ ( ), σ ( ), σ ( ),, σ ( ). II) MODEL TESTING EAMPLE Now h our model of eres hs ee esmed, we eed o es for he sgfcce of he erms foud he esmed model. Ths s very mor ecuse he resuls of hs esg wll deerme he fl equo whch wll e reed d used for Forecsg uroses. Tesg TE MULTIVARIATE MODEL ŷ ) ( Tesg of hs model cosss of he followg ses: A) To es for he sgfcce of ech fcor serely The vlues of,,,, re oed from equo () d he vlues of σ ( ), σ ( ), σ ( of ech fcor serely y eher: ),, σ( ) from equos () d (7). The, we es for he sgfcce ) Tesg he hyoheses: : vs. : ¹ y clculg Z or σ () σ ( ), for 9 :. The, : s rejeced f Z > Z / (or f Z < Z / ), whe ³ or f > (or f ( / ) < ), f < ( / ) or ) By cosrucg he cofdece ervls P[ Z / ( ) + Z / ( )], f ³ or P [ ) + ( )], f <. ( / ) ( ( / ) If he vlue s ousde of hese Cofdece ervls, : s rejeced. B) To es for he Sgfcce of he ere Regresso (cludg he cos) The hyoheses eg esed re: : vs. : The re o ll equl o or : The ere regresso (cludg he cos) s o sgfc vs. : The ere regresso (cludg he cos) s sgfc. URL: h://dx.do.org/.78/r..7. 8

6 Archves of Busess Reserch (ABR) Vol., Issue, Jue8 I s crred ou y clculg: RSS / DOF FTol ESS / DOF ( Y Y Y / Y) / d comrg o F ( ). If F Tol > F ( ), s rejeced d we coclude h he ere regresso (cludg he cos) s sgfc (o he clculo of he Y vlue). C) To es for he Sgfcce of he ere Regresso (excludg he cos) The hyoheses eg esed re: : vs. :,,, re o ll equl o or : The ere regresso (excludg he cos) s o sgfc vs. : The ere regresso (excludg he cos) s sgfc. I s crred ou y clculg: ( RSS SS ) / ( Y SS ) / FTol, where SS < > ESS / ( Y Y Y) / d comrg o F ( ). If F > Tol F ( ), s rejeced d we coclude h he ere regresso (excludg he cos) s sgfc. D) Deermo of he Fl equo Ay vrle, for whch he hyohess: : (vs. : ¹ ) s o rejeced, s o e droed from he regresso equo. The remg erms re used o form he Fl Equo whch s he reed d used for Predco/Forecsg uroses. E) A Mulvre Exmle The sles mger of cer frm eleves h Sles Aly deeds o slesm s Verl Resog Aly d Vocol Ieres. e s eresed cosrucg regresso equo o use fuure hrg, o redc cdde s success s slesm,.e. he ws o derve he regresso equo. Y Sles Aly ( ) + ( Verl Re sog) + ( Vocol Ieres ) To verfy hs elef, slesme re seleced rdom from hs sff d gve ess: Oe for Verl Resog Aly, he oher for Vocol Ieres. The resuls re show elow: Slesm YAverge sles moh 5 Verl Resog 5 5 Aly Vocol Ieres 5 ) Fd he esmed regresso equo for hs d se. ) Fd TSS, RSS, Q ESS, d SS Coyrgh Socey for Scece d Educo, Ued Kgdom 9

7 Vslooulos. (8). Mulvre Regresso: A Very Powerful Forecsg Mehod. Archves of Busess Reserch, (), 8. ) Fd σ ( ), σ ( ), σ ( ) ) Tes he hyoheses: ) : vs. : ¹ ) : vs. : ¹ c) : vs. : ¹ 5) Cosruc 95% Cofdece ervls for:,, ) Tes he hyohess: : The ere regresso equo (cludg he cos) s o sgfc (.e. : ) 7) Tes he hyohess: : The ere regresso equo (excludg he cos) s o sgfc (.e. : ) 8) Deerme he fl regresso equo Soluos: The mrces d her roduc re gve elow. Noe h he vlues of vrle re ll equl o ecuse we w our regresso equo o hve cos erm. Sce s o dgol mrx ( A / s dgol mrx d he A / y seco) we wll use he Adjo Mrx mehod o fd he verse of mrx ( / ). The resul s: ( ) URL: h://dx.do.org/.78/r..7.

8 Archves of Busess Reserch (ABR) Vol., Issue, Jue8 Also, Y The: ( ) ( Y ) d , 9., 8. 7, Therefore, he regresso equo s: ) y ˆ ( ) ) TSS Y Y y y + y + + y. [ ] RSS Y 8.77 ESS Q Y Y Y y ( y + y + + y ) () 9 SS 9. ESS Q ) From equo (7) ˆ The: V ( ˆ ) (.79).8, d ( ) V ( ˆ ) (.79).9, d ( ). 7 7 V ( ˆ ) (.79).77, d ( ) ) Tes: : vs :, : vs :, : vs :, Coyrgh Socey for Scece d Educo, Ued Kgdom

9 Vslooulos. (8). Mulvre Regresso: A Very Powerful Forecsg Mehod. Archves of Busess Reserch, (), Clcule:.8 ( )..9.9 ( )..7.9 ( ).9 d comre ech gs: Sce.5 <.8 <.5, do o rejec : ; Therefore,.9 >.5, rejec : ; Therefore,.9 >.5, rejec : ; Therefore, 5) Sce < he smlg dsruos of,, re 7. Therefore, he Cofdece Iervls re o e oed from: P[ ) + ( )] 7( / ) ( 7 ( / ) d hey re (whe he rore vlues re susued ): ) P[.99.].95; sce flls sde hs Cofdece Iervl, we do o rejec : ; Therefore, ) P[.8.].95; sce flls ousde hs Cofdece Iervl we rejec : ; Therefore, ¹ c) P[. < <.8].95; sce flls ousde hs Cofdece Iervl we rejec : ; Therefore, ¹ ) We clcule: ( Y) /.77 /.58 FTol.87 ( Y Y Y) / (.95) / 7.79 d comre o: F ( ) F 7 ( ).5, 8.5, f f.5. Sce F Tol > F 7 ( ), for oh vlues, : (The ere equo cludg he cos s o sgfc) s rejeced, d we coclude h he ere equo s sgfc. 7) We clcule:.7 ( Y SS ) / (.77 9.) / F Tol ( Y Y Y ) / (.95) / , f.5 d comre o: F ( ) F7 ( ) 9.55, f. Sce F > F ( ) Tol 7, for oh vlues, we rejec ( : The ere equo (excludg he cos) s o sgfc) d coclude h he ere regresso equo, excludg he cos, s sgfc. URL: h://dx.do.org/.78/r..7.

10 Archves of Busess Reserch (ABR) Vol., Issue, Jue8 8) Becuse : s o rejeced u : d : re rejeced, he fl regresso equo s: y ˆ whch should e reed d used o redc he sles ly of fuure cddes. F) MINITAB SOLUTION o he Prolem We eer he gve d d ssue he regresso commd s show: MTB > Se C DATA > 5 DATA > ed MTB > Se C DATA > 5 5 DATA> ed MTB > Se C DATA> 5 DATA> ed MTB > r C C C D Dsly Row C C C MTB > REGRESS c c c; SUBC > CONSTANT; SUBC > BRIEF. Regresso Alyss: C versus C, C The regresso equo s C C +. C Predcor Coef SE Coef T P Cos C C S.57 RSq 9.8% RSq(dj) 9.% Alyss of Vrce Coyrgh Socey for Scece d Educo, Ued Kgdom

11 Vslooulos. (8). Mulvre Regresso: A Very Powerful Forecsg Mehod. Archves of Busess Reserch, (), 8. Source DF SS MS F P Regresso Resdul Error Tol 9.9 Source DF Seq SS C.5 C.5 Uusul Oservos Os C C F SE F Resdul S Resd R R deoes oservo wh lrge sdrdzed resdul Whe we comre he MINITAB d hd soluos, hey re decl, wh he roudg of frcol umers. From he vlues clculed y MINITAB we see, oce g, h : s o rejeced ecuse.7 >.5 u : d : re rejeced ecuse.5 <.5. The F vlue of 5.9, show he MINITAB Alyss of Vrce s lmos decl o he hd vlue F 5.77 oed o es he hyohess Tol : The ere regresso (excludg he cos) s o sgfc. Also, he vlues of ˆ re lmos decl. SUMMARY OF MULTIVARIATE PROCEDURE PROCEDURE FOR SOLVING MULTIVARIATE/BIVARIATE PROBLEMS A) Model Esmo Gve Mulvre (or vre) d Se: ) Idefy he mrces: Y,,, Z whch form he mrx equo Y+Z ) Clcule d Y ) Clcule The Iverse of Mrx ( Y) ( ). If s dgol Mrx, ( ) s esy o fd. If s o Dgol, Fdg ( ) s more dffcul. Use he Guss Elmo Mehod. Use he Adjo Mrx Mehod ) Clcule:.... ( ) Y 5) Clcule: Y Y, Y, QY Y Y, SS ( Y) / ) Clcule: σ Q/ 7) Clcule: V() ( ) σ ßThe Vrces V(), V(), V() re log he M Dgol Noe: The oher erms of he V() mrx re covrces Noe: A hs o we hve:,,, d : σ(), σ(), σ() Also vlle re ll he sums of squres URL: h://dx.do.org/.78/r..7.

12 Archves of Busess Reserch (ABR) Vol., Issue, Jue8 B) Model Tesg ) To Tes for he Sgfcce of Ech Fcor Serely ) From he kowledge of d σ() Use Eher Z ( f ) or ( f < ) o es : o: β vs. β ) From he kowledge of d σ() Use eher Z or o cosruc cofdece ervls P[ Zα/ σ() β + Zα/ σ()] α or P[ (α/) σ() β + (α/) σ()] α ) To Tes for he Sgfcce of he Ere Equo (Icludg he Cos) ) Cosruc ANOVA wh SS ) Tes he yohess : o : β β β β vs. : The β re o ll I. Clcule F [( Y)/] / [ Q/] II. Comre F o : F (α) III. Rejec o f : F > F (α) c) To Tes For he Sgfcce of he Ere Equo (excludg he Cos). Cosruc ANOVA whou SS. Tes he hyohess: o: β β β vs. : The β re o All. Clcule F ( Y SS)/ Q/. Comre F o : F (α). Rejec o If : F > F (α) DETERMINE TE FINAL EQUATION Ay Vrle,, for whch he yohess: o: β vs. : β s o rejeced, s o e droed from he regresso equo. The remg erms re used o form he Fl Equo whch s he reed d used for Predco/Forecsg uroses. TE LINEAR REGRESSION MODEL AS A SPECIAL CASE OF TE MULTIVARIATE MODEL For he Ler model, he vlue of oe deede vrle (Y) deeds o he vlues of oe deede vrle (). I hs cse, he gve d se s Bvre, d gve y: D + D D. D (8) From whch we c o: ' ' (9) ' () ' Y E + E E. E () () Coyrgh Socey for Scece d Educo, Ued Kgdom 5

13 Vslooulos. (8). Mulvre Regresso: A Very Powerful Forecsg Mehod. Archves of Busess Reserch, (), 8. ' ' () umer of ordered rs ( ', ' ) () The del ler model wll e rereseed y: E +GD (5) whle he model of oservos (esmed model) wll e wre s: E ' + + I ' () The, lyg he mrx mehodology, summrzed ove, we eed o defe he mrces: Y, β,, d where: E + E E. ; G J G ; ; E D + D D. D These vecors/mrces re oed whe he gve d se s susued he model of oservos (equos ove) whch ecomes he lgerc sysem of equos : E + + D + + I + E + D + I E. + D. + I. (8) E + D + I whch c e rewre he mrx form s: E + E E D + D D + (7) I + I I (9) or: Y + Z () To o he Norml equos for he Ler model ( ), we eed o frs o d. D + D Sce D. s x mrx, s x, d gve y: D + D D. d D ecomes x mrx: D D + D D. D D + D D ' ' ' () d D + D D. D E + E E. E E + E E. D + E + D E D. E. E D E E ' E ' () URL: h://dx.do.org/.78/r..7.

14 Archves of Busess Reserch (ABR) Vol., Issue, Jue8 The, he orml equos Y ecome: (whch re decl o he lgerc orml equos) d her soluo s: ( ) 7+ E ' E ' E ' E ', () Noe: The verse mrx () s oed y usg eher he Guss Elmo Mehod or he Adjo Mrx Mehod. The, from equo (5) we o: E ' E ' E ' () (5) E ' () E ' E ' 7 D ' P (Noe: P S> 9T D ) D Q ' P D Q ' Σ( D) Q ecuse D ' ( D) Q (8) ( D) Σ( D) Q (9) () Also, Covrce, 7 c ` (`7`) >, () d (E +, E, E.,, E ) E ' d Q h 7 E ' E + E E. E ( ) E ' E ' E ' () fgg () Noe: The exressos for he orml equos of he Ler model, V(), V(), Cov(,), Q x ESS, d σ re decl o hose oed drecly for he Ler model. CONCLUSIONS. Regresso Alyss, wheher Ler, NoLer, or Mulvre, s exremely mor s Forecsg Techque.. Ler Regresso s relvely esy o erform usg urely lgerc mehods.. Bu, Ler Regresso c lso e cosdered s secl cse of he more geerl Mulvre Regresso Model, whch c e lyzed effcely y usg mrx () Coyrgh Socey for Scece d Educo, Ued Kgdom 7

15 Vslooulos. (8). Mulvre Regresso: A Very Powerful Forecsg Mehod. Archves of Busess Reserch, (), 8. mehods.. A seyse rocedure o how o solve he mulvre regresso rolem s cluded hs er. 5. The Alco of he Mehod (whch cosss of: Model Esmo, Model Tesg, d he dervo of he FINAL Regresso equo whch s reed d used for forecsg uroses) requres elemery kowledge of Mrx Alger, cludg he clculo of he Iverse Mrx.. Sscl ools, such s he MINITAB, c lso e used o solve he Mulvre rolem y comuer, d he comre he hd d MINITAB resuls. 7. Usg MINITAB o lso solve he smle rolem, roduces soluos whch re decl o he hd soluos. 8. The MINITAB ouu o oly esmes he model, u lso geeres vlues for ll he mor model rmeers, whch llows he esg of her sgfcce. 9. The vlue, clled he Oserved level of sgfcce, cors o he ror α vlue, hs he followg relosh o α: ) If >α, do o Rejec o. ) If <α, Rejec o.. The MINITAB ouu lso rovdes vlues for R (coeffce for Mulle Deermo) d R djused whch ells us how well he model fs he gve d Refereces Bereso, Mrc, L.; Leve, Dvd, M.; Krehel, Tmohy, C.; Bsc Busess Blck, Ke; Busess Sscs, Wley Cvos, George, C.; Aled Proly d Sscl Mehods; Lle, Brow; 98 Chldress, Roer, L.; Gorsky, Ro, D.; W, Rchrd, M.; Mhemcs for Mgerl Decsos, Prece ll, 989 Crlso, Wllm, L.; Thore, Bey; Aled Sscl Mehods, Prece ll, 997 Chldress, Roer, L.; Gorsky, Ro, D.; W, Rchrd, M.; Mhemcs for Mgerl Chou, Ylu; Sscl Alyss for Busess d Ecoomcs ; Elsever, 99 Drer, Norm; Smh, rry; Aled Regresso Alyss Joh Wley Sos, 9 Johso, J. Ecoomerc Mehods, McGrwll, 9 Pdyck, Roer S; Rufeld, Del L.; Ecoomerc Models d Ecoomc Forecss, d Edo McGrwll, 98 Vslooulos, A. Busess Sscs A Logcl Aroch. Theory, Models, Procedures, d Alcos Icludg Comuer (MINITAB) Soluos, Boso, MA; Perso Cusom Pulshg, 7 Vslooulos, A. Lu, F. Vcor. Quve Mehods for Busess wh Comuer Alcos, Boso, MA; Perso Cusom Pulshg, Vslooulos, A. Regresso Alyss Revsed, Revew of Busess, S. Joh s Uversy, Jmc, NY; 5 URL: h://dx.do.org/.78/r..7. 8