Linear and Non-Linear Regression: Powerful and Very Important Forecasting Methods

Transcription

1 Epert Joural of Busess ad Maagemet, Volume, Issue, pp. 0-8, 0 0 The Author. Publshed b Sprt Ivestf. ISSN Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods Athaasos VASILOPOULOS * St. Joh s Uverst, Uted States Regresso Aalss s at the ceter of almost ever Forecastg techque, et few people are comfortable wth the Regresso methodolog. We hope to mprove the level of comfort wth ths artcle. I ths artcle we brefl dscuss the theor behd the methodolog ad the outle a step-b-step procedure, whch wll allow almost everoe to costruct a Regresso Forecastg fucto for both the lear ad some o-lear cases. Also dscussed, addto to the model costructo metoed above, s model testg (to establsh sgfcace ad the procedure b whch the Fal Regresso equato s derved ad retaed to be used as the Forecastg equato. Had solutos are derved for some small-sample problems (for both the lear ad o-lear cases ad ther solutos are compared to the MINITAB-derved solutos to establsh cofdece the statstcal tool, whch ca be used eclusvel for larger problems. Kewords: Lear Regresso, No-Lear Regresso, Best-Fttg Model, Forecastg JEL Classfcato: M0. Itroducto ad Model Estmato for the Lear Model Regresso aalss, whch a equato s derved that coects the value of oe depedet varable (Y to the values of oe depedet varable X (lear model ad some o-lear models, starts wth a gve bvarate data set ad uses the Least Squares Method to assg the best possble values to the ukow multplers foud the models we wsh to estmate. The bvarate data, used to estmate the lear model ad some o-lear models, cossts of ordered pars of values: (,,(,,..., (, The lear model we wsh to estmate, usg the gve data, s: + ( * Correspodg Author: Athaasos Vaslopoulos, Ph.D., St. Joh s Uverst, The Peter J. Tob College of Busess, CIS/DS Departmet, 8000 Utopa Parkwa Jamaca, N.Y. 49 Artcle Hstor: Receved November 0 Accepted November 0 Avalable Ole 9 December 0 Cte Referece: Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 0

2 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 whle the o-lear models of terest are gve b ad (Epoetal Model ( (Power Model ( + + (Quadratc Model (4 To estmate model ( we use the Least Squares Methodolog, whch calls for the formato of the quadratc fucto:,!"!"!"!" +!" +!" ( To derve the ormal equatos for the lear model from whch the values of a ad b of the lear model are obtaed, we take the partal dervatve of Q(a,b of equato ( wth respect to a ad b, set each equal to zero, ad the smplf: The result s: ad Q(a, b a Q(a, b b +b +a X +a +b (6 (7 Whe (6 ad (7 are set equal to zero ad smplfed, we obta the Normal equatos for the lear model: a+b a +b (8 (9 The ol ukows equato (8 ad (9 are a ad b ad the should be solved for them smultaeousl, thus dervg (or estmatg the lear model. Ths s so because all the other values of equatos (8 ad (9 come from the gve data, where: umber of ordered pars (, sum of the values sum of the values sum of the gve values, whch are frst squared sum of the products of the ad values each ordered par. Note: The values of (a ad (b obtaed from the Normal equatos correspod to a mmum value for the Quadratc fucto Q(a,b gve b equato (, as ca be easl demostrated b usg the 06

3 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 Optmzato methodolog of Dfferetal Calculus for fuctos of depedet varables. To complete the Estmato of the Lear model we eed to fd the stadard devato for a, σ(a, ad b, σ(b, whch are eeded for testg of the sgfcace of the model. The stadard devatos, σ(a, ad σ(b, are gve b: ad where: σ (a ˆ σ σ (b / ( σˆ ( a b ˆ σ / ˆ σ / (0, ( / The a ad b equato ( come from the soluto of equatos (8 ad (9 whle come drectl from the gve bvarate data. (,, ad. Model Testg Now that our model of terest has bee estmated, we eed to test for the sgfcace of the terms foud the estmated model. Ths s ver mportat because the results of ths testg wll determe the fal equato whch wll be retaed ad used for Forecastg purposes. Testg of the lear model cossts of the followg steps:.. Testg for the sgfcace of each term separatel Here we test the hpotheses:. H 0: β 0 vs H : β 0, ad. H 0: α 0 vs H : α 0, based o our kowledge of b, σ(b, a, ad σ(a. If 0, we calculate Z * b b σ (b ad Z * a a σ (a ad compare each to Z α/ (where Z α/ s a value obtaed from the stadard Normal Table whe α, or - α, s specfed. 07

4 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 For eample f α 0.0, Z α/ Z ; f α 0.0, Z α/ Z ; f α 0.0, Z α/ Z 0.0. ad f α 0.0, Z α/ Z If Z * b > Z α/ (or Z * b < -Z α/, the hpothess H 0: β 0 s rejected ad we coclude that β 0 ad the term b ( the estmated model ŷa+b s mportat for the calculato of the value of. Smlarl, f Z * a > Z α/ (or Z * a < -Z α/, H 0: α 0 s rejected, ad we coclude that the lear equato ŷa+b does ot go through the org. If < 0, we calculate t * b b σ (b ad t * a a σ (b ad compare each to t -(α/, for a gve α value, where t -(α/ s obtaed from the t-dstrbuto table, wth the same terpretato for H 0: β 0 ad H 0: α 0 as above. But, stead of hpothess testg, we ca costruct Cofdece Itervals for β ad α usg the equatos: ad, f 0, or ad, f < 0, $ % ( * + % ( ( $ % ( + % ( (4 $, - ( * +, - ( ( $, - ( +, - ( (6 If the hpotheszed values: β 0 falls sde the Cofdece Itervals gve b equatos ( or (, or α 0 falls sde the Cofdece Itervals gve b equatos (4 or (6, the correspodg hpotheses H 0: β 0 ad H 0: α 0 are ot rejected ad we coclude that β 0 (ad b 0 ad the term b s ot mportat for the calculato of ad α 0 (.e. a 0 ad the le goes through zero. If for a gve data set, we performed the above-dscussed tests, we wll obta oe of 4 possble coclusos: A H 0: β 0 ad H 0: α 0 are both rejected; Therefore β 0, ad α 0, ad both the terms a ad b are mportat to the calculato of. I ths case the fal equato s ŷa+b, wth both terms stag the equato. B H 0: β 0 s rejected, but H 0: α 0 s ot rejected. Therefore β 0 but α 0 ad the term a s ot mportat to the calculato of. I ths case the fal equato s ŷa+b, wth the term a droppg out of the equato. C H 0: β 0 s ot rejected but H 0: α 0 s rejected. Therefore β 0 ad the term b s ot mportat for the calculato of, whle a 0 ad s mportat to the calculato of. I ths case the fal equato s ŷ a, wth the term b droppg out of the equato D H 0: β 0 ad H 0: α 0 are both ot rejected; Therefore β 0, ad α 0, ad both terms a ad b are ot mportat to the calculato of. I ths case the fal equato wll be ŷ 0, wth both terms a ad b droppg out of the equato. 08

5 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp Testg for the Sgfcace of the Etre Lear Equato Ths test cossts of testg the hpothess:. H 0: α β 0 vs H 0: α ad β are ot both equal to 0, or. H 0: The Etre Regresso equato s ot sgfcat vs H : The Etre Regresso equato s sgfcat For a gve bvarate data set ad a gve α value, we eed to frst calculate: Total Sum of Squares ( Regresso Sum of Squares RSSb ( ˆ b ( b (8 Error Sum of Squares ESS ( ˆ Q* a b (9 Sum of Squares Due to the Costat SS The we calculate: a * F Total (RSS b + SS a / ESS / ( ad compare F * Total to F -(α, whch s a tabulated value, for a specfed α value. If F * Total > F -(α, we reject H 0 ad coclude that the etre regresso equato (.e. ŷa+b or that both the costat term a, ad the factor (ad term b are sgfcat to the calculato of the value, smultaeousl. Note : Whe TSS, RSS b, ad ESS are kow, we ca also defe the coeffcet of determato R², where: (7 (0 R RSS b TSS ESS TSS ( where 0 R, whch tells us how well the regresso equato ŷ a + b fts the gve bvarate data. A value of R close to mples a good ft. Note :. /.. 0,/ /, (.. A Bvarate Eample A sample of adult me for whom heghts ad weghts are measured gves the followg results (Table : Table. Gve bvarate data set ( H W H W HW ² 0² ² 4² ² 0² ² 6² ² 70²

6 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 For ths Bvarate Data set we have: ² + 6² + 66² + 67² + 68², ² +4² +0² +6² +70² 6,0 ( (6 4 + ( ( ( ,60 To obta the lear equato ŷ a + b, we substtute the values of, to equatos (8 ad (9 ad obta: ,790 0,60,, Whe these equatos are solved smultaeousl we obta: a -08 ad b 0, ad the regresso equato s ŷ a + b The, usg the values of a -08, b0, ad, 6, 0 ( 08(760 (0(0, 60 ˆ σ ad from equatos (0 ad (: / ad we obta from equato (: 0 / 0.68 / / 0,790,790,790 σ ( a (0 0 0,790 0 σ (b 0 0 / / [, 790 (0 0] 0 / Sce <0, a ad b are dstrbuted as t t varables ad whe α 0.0, t (α/ t (0.0 ±.84. The the hpotheses H 0: β 0 vs. H : β 0, ad H 0: α 0 vs. H 0: α 0 are both rejected because: ad, ( <.84 66,0, ( 0 0 >.84 0

7 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 Therefore, the fal equato s ŷ a + b To test for the sgfcace of the etre equato, ad to calculate the coeffcet of determato, we frst evaluate, TSS, RSS b, ESS, SS a usg equatos (7 (0 ad obta: (760 TSS 6,0-00 (0 RSS 0,790 0 (0 000 b ESS 6,0 - (-08(760-0(0,60 0 SS a (760,0 From equato (, we obta R 000/ , whch tells us that 97% of the varato the values of Y ca be eplaed (or are accouted for b the varable X cluded the regresso equato ad ol % s due to other factors. Sce R s close to, the ft of the equato to the data s ver good. Note: The correlato coeffcet r, whch measures the stregth of the lear relatoshp betwee Y ad X s related to the coeffcet of determato b: r R for ths eample. Clearl X ad Y are ver strogl learl related. Usg equato ( we obta: F * Total ( RSSb + SSa / (000 +,0 / 8,60,86 ESS / 0 / 0 whe F * Total s compared to 0. f α 0.0 (α F (α 4. f α 0.0 F H 0 (The etre regresso equato s ot sgfcat s rejected, ad we coclude that the etre regresso equato s sgfcat.. MINITAB Soluto to the Lear Regresso Problem We eter the gve data ad ssue the regresso commad as show Table. Table. Data set MINITAB MTB > Set C DATA> DATA> ed MTB > set C DATA> DATA> ed MTB > Name C 'X' C 'Y' MTB > REGRESS 'Y' 'X' ad obta the MINITAB output preseted Table, Table 4, ad Fgure.

8 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 Table. Regresso Aalss: Y versus X Regresso equato: Y X Predctor Coef SE Coef T p Costat X Regresso ft: S R-Sq R-Sq (adj.6 97.% 96.% Aalss of Varace: Source DF SS MS F p Regresso Resdual Error Total Table 4. Correlatos: Y, X Pearso correlato of Y ad X 0.98 P-Value Y X Fgure. Plot Y * X Whe we compare the MINITAB ad had solutos, the are detcal. We obta the same equato ŷ , the same stadard devatos for a ad b (uder SE Coeffcet ad the same t values, the same R, the same s σ ad σ 0. Notce also that a Aalss of Varace table provdes the values for RSS b, ESS, ad TSS. The ol value mssg s SS a, whch ca be easl calculated from SS a. The MINITAB soluto also gves a p-value for each coeffcet. The p-value s called the Observed Level of Sgfcace ad represets the probablt of obtag a value more etreme tha the value of the test statstc. For eample the p-value for the predctor X s calculated as p 0.00, ad t s gve b: p- value P(t > t*0 0 The p-value has the followg coecto to the selected α-value. If p α, do ot reject H 0 f ( t dt 0.00 (4

9 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 If p < α, reject H 0 Sce p 0.00 < α 0.0, H 0: β 0 wll be rejected. 4. Itroducto ad Model Estmato for Some No-Lear Models of Iterest Sometmes two varables are related but ther relatoshp s ot lear ad trg to ft a lear equato to a data set that s heretl o-lear wll result a bad-ft. But, because o-lear regresso s, geeral, much more dffcult tha lear regresso, we eplore ths part of the paper estmato methods that wll allow us to ft o-lear equatos to a data set b usg the results of lear regresso whch s much easer to uderstad ad aalze. Ths becomes possble b frst performg logarthmc trasformatos of the o-lear equatos, whch chage the o-lear to lear equatos, ad the usg the ormal equatos of the lear model to geerate the ormal equatos of the learzed o-lear equatos, from whch the values of the ukow model parameters ca be obtaed. I ths paper we show how the epoetal model, ŷ ke c, ad the power model, ŷ a b (for b ca be easl estmated b usg logarthmc trasformatos to frst derve the learzed verso of the above o-lear equatos, amel: l ˆ l k + c ad l ˆ l a + b l, ad the comparg these to the orgal lear equato, ŷ a + b, ad ts ormal equatos (see equatos (8 ad (9. Also dscussed s the quadratc model, ŷ a + b + c whch, eve though s a o-lear model, ca be dscussed drectl usg the lear methodolog. But ow we have to solve smultaeousl a sstem of equatos ukows, because the ormal equatos for the quadratc model become: a + b a ( a + b + b + c + c 4 + c A procedure s also dscussed whch allows us to ft these four models (.e. lear, epoetal, power, quadratc, ad possbl others, to the same data set, ad the select the equato whch fts the data set best. These four models are used etesvel forecastg ad, because of ths, t s mportat to uderstad how these models are costructed ad how MINITAB ca be used to estmate such models effcetl. 4.. The Lear Model ad ts Normal Equatos The lear model ad the ormal equatos assocated wth t as eplaed above, are gve b: Lear Model + ( Normal Equatos a + b a + b (8 (9

10 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp The Epoetal Model The epoetal model s defed b the equato: c ˆ ke (6 Our objectve s to use the gve data to fd the best possble values for k ad c, just as our objectve equato ( was to use the data to fd the best ( the least-square sese values for a ad b. Takg atural logarthms (.e. logarthms to the base e of both sdes of equato (6 we obta c l( ˆ ke or c l ˆ l( ke ( Logarthmc Laws To smplf equato (7, we have to use some of the followg laws of logarthms: log (A B log A + log B (8 log (A/B log A log B (9 log (A log A (0 The, usg equato (8 we ca re-wrte equato (7 as: l + c l k l e ( ad, b applg equato (0 to the secod term of the rght had sde of equato (, equato ( ca be wrtte fall as: or l l k + c(l e l l k + c (because l e log e e ( Eve though equato (6 s o-lear, as ca be verfed b plottg agast, equato ( s lear (.e. the logarthmc trasformato chaged equato (6 from o-lear to lear as ca be verfed b plottg: l agast. But, f equato ( s lear, t should be smlar to equato (, ad must have a set of ormal equatos smlar to the ormal equatos of the lear model (see equatos (8 ad (9. Questo: How are these ormal equatos gog to be derved? Aswer: We wll compare the trasformed lear model,.e. equato (, to the actual lear model (equato (, ote the dffereces betwee these two models, ad the make the approprate chages to the ormal equatos of the lear model to obta the ormal equatos of the trasformed lear model Comparso of the Logarthmc Trasformed Epoetal Model to the Lear Model To make the comparso easer, we lst below the models uder cosderato, amel: a Orgal Lear Model: + ( b Trasformed Lear Model: l l k + c ( Comparg equatos ( ad (, we ote the followg three dffereces betwee the two models:. equato ( has bee replaced b l equato (. a equato ( has bee replaced b l k equato (. b equato ( has bee replaced b c equato ( 4

11 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp Normal Equatos of Epoetal Model Whe the three chages lsted above are appled to the ormal equatos of the actual lear model (equatos (8 ad (9, we wll obta the ormal equatos of the trasformed model. The ormal equatos of the trasformed lear model are: (l k + c l ( (l k + c (l (4 I equatos ( ad (4 all the quattes are kow umbers, derved from the gve data as wll be show later, ecept for: l k ad c, ad equatos ( ad (4 must be solved smultaeousl for l k ad c. Suppose that for a gve data set, the soluto to equatos ( ad (4 produced the values: l k 0. ad c. ( If we eame the epoetal model (equato (6, we observe that the value of c. ca be substtuted drectl to equato (6, but we do ot et have the value of k; stead we have the value of l k 0.! Questo: If we kow: l k 0., how do we fd the value of k? Aswer: If l k 0., the: k e 0. ( Therefore, ow that we have both the k ad c values, the o-lear model, gve b equato (6, has bee completel estmated. 4.. The Power Model Aother o-lear model, whch ca be aalzed a smlar maer, s the Power Model defed b the equato: b ˆ a (6 whch s o-lear f b ad, as before, we must obta the best possble values for a ad b ( the least-square sese usg the gve data Logarthmc Trasformato of Power Model A logarthmc trasformato of equato (6 produces the trasformed lear model l l a + b l (7 Whe equato (7 s compared to equato (, we ote the followg chages:. equato ( has bee replaced b l equato (7. a equato ( has bee replaced b l a equato (7 (8. equato ( has bee replaced b l equato (7 Whe the chages lsted (8 are substtuted to equatos (8 ad (9, we obta the ormal equatos for ths trasformed lear model whch are gve b equatos (9 ad (40 below: 4... Normal Equatos of Power Model (l a + b l l (9 (l a l + b (l (l (l (40

12 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 Equatos (9 ad (40 must be solved smultaeousl for (l a ad b. If l a 0.4, the a e 0.4 ( ad, sce we have umercal values for both a ad b, the o-lear model defed b equato (6 has bee completel estmated Dervato of the ormal equatos for the Quadratc model, a + b + c To derve the ormal equatos of the quadratc model, frst form the fucto [ a b c ] Q(a, b,c (4 Q Q Q The take the partal dervatves:,,, ad set each equal to 0, to obta the equatos a b c eeded to solve for a, b, c. We obta: or: Q + a a + b [ a b c ] ( + c 0, (4 Q b + [ a b c ] ( 0, or: or: a + c + b Q + c a [ a b c ] ( 4 + c + b 0, (4 (44 Equatos (4, (4, ad (44 are detcal to equato (. 4.. Data Utlzato Estmatg the 4 Models To geerate the quattes eeded to estmate the 4 models: a. The Lear Model b. The Epoetal Model c. The Power Model, d. The Quadratc Model, the gve (, bvarate data must be mapulated as show Tables:, 6, 7, ad 8, respectvel Gve Data to Evaluate the Lear Model Table. Mapulato of Gve Data to Evaluate the Lear Model 6

13 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 N N N N 4 To evaluate a + b, substtute: N, N, N, N 4 to equatos (8 ad (9 ad solve for a ad b smultaeousl Gve Data to Evaluate the Epoetal Model To evaluate smultaeousl. Table 6. Mapulato of Gve Data to Evaluate the Epoetal Model l l l l l l l l l l l N N 6 N 7 N 8 N 9 l c ke, substtute N, N 7, N 8, N 9 to equatos ( ad (4 ad solve for l k ad c 4... Gve Data to Evaluate the Power Model Table 7. Mapulato of Gve Data to Evaluate the Power Model l (l (l (l l l (l l (l (l (l (l (l (l (l l (l l l l l (l (l (l l l (l (l (l l N 0 N N N N 4 N b ˆ a To evaluate, substtute N, N, N 4, N to equatos (9 ad (40 ad solve smultaeousl for (l a ad b. 7

14 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp Gve Data to Evaluate the Quadratc Model Table 8. Mapulato of Gve Data to Evaluate the Quadratc Model N 6 N 7 N 8 N 9 N 0 N N To evaluate + +, substtute N 6, N 7, N 8, N 9, N 0, N, N to equatos (4, (4, ad (44, ad solve smultaeousl for a, b, ad c.. Selectg the Best-Fttg Model.. The Four Models Cosdered Gve a data set (,, we have show how to ft to such a data set four dfferet models, amel: a. Lear: ˆ (4 a + b b. Epoetal: c ˆ (46 ke c. Power: ˆ b (47 a d. Quadratc: ˆ a + b + c (48 We mght decde to ft all four models to the same data set f, after eamg the scatter dagram of the gve data set, we are uable to decde whch of the 4 models appears to ft the data BEST. But, after we ft the 4 models, how ca we tell whch model fts the data best? To aswer ths questo, we calculate the varace of the resdual values for each of the models, ad the select as the best model the oe wth the smallest varace of the resdual values... Calculatg the Resdual Values of Each Model ad Ther Varace Use each value, of the gve data set (,, to calculate the ŷ value, from the approprate model, ad the for each, form the resdual: Resdual of observato s ( ˆ, (49 for each. The the varace of the resdual values s defed b: V (Resdual DOF where DOF Degrees of Freedom. ( ˆ, (0 Note: The DOF are DOF for the frst three models (Lear, Epoetal, Power due to the fact that each of these models has ukow quattes that eed to be evaluated from the data (a ad b, k ad c, ad a ad b, respectvel ad, as a cosequece, degrees of freedom are lost. For the Quadratc model, DOF because the model has ukow quattes that eed to be estmated ad, as a cosequece, degrees of freedom are lost. 8

15 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 Usg equato (0 to calculate the varace of the resduals for each of the 4 models, we obta: V (Resdual Lear ( a b [( a b + ( a b + + ( a ]... b ( c V (Resdual Epoeta l ( ke ( c c c [( ke + ( ke + + ( ke ] L (4 b V (Resdual Power ( a ( b b ( a + ( a + + ( b a [ ] L (6 V (Resdual Quadratc ( a b c (7 [( a b c + ( a b c ( a b c ] (8 After the calculato of the 4 varaces from equatos: (, (4, (6, ad (8, the model wth the smallest varace s the model whch fts the gve data set best. We wll ow llustrate, through a eample, how the 4 models we dscussed above ca be ftted to a gve bvarate data set, ad the how the best model from amog them s selected... A Cosdered Eample A sample of adult me for whom heghts ad weghts are measured gves the followg results (Table 9. Table 9. Sample of adult me # X Heght Y Weght Problem: Ft the lear, epoetal, power, ad quadratc models to ths bvarate data set ad the select as the best the model wth the smallest varace of the resdual values.... Fttg the Lear Model ŷ a + b To ft the lear model, we must eted the gve bvarate data so that we ca also calculate ad, as show below, Table 0: Table 0. Calculatos for bvarate data of adults for the lear model X ( 9

16 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8, ,60 We the substtute the geerated data to the ormal equatos for the lear model, amel equatos (8 ad (9: a + b a + b, ad obta the equatos: ,790 0,60 Whe these equatos are solved smultaeousl for a ad b we obta: G 08,H 0 Therefore, the lear model s: ŷ a + b The varace of the resdual values for the lear model s calculated as show below, Table : Table. Varace of the resdual values for the lear model Gve X Gve Y Calculated Y Resdual (Resdual ŷ ŷ ( - ŷ (64 - ( (6 4 + ( (66 - ( ( ( ( (- 4 Therefore, the varace of the resdual values, for the lear model s: 0 ( ˆ 0 V(Resdual Lear ( ˆ ( Fttg the Epoetal Model ŷ ke c To ft the epoetal model we eed to eted the gve bvarate data so that we ca calculate, addto to 0 ad,790, l ad ( l as show below, Table : Table. Calculatos for bvarate data of adults for the epoetal model l l

17 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp , l l We the substtute the geerated data to the ormal equatos for the epoetal model (.e. equatos ( ad (4: (l k + c l (l k + c (l, ad obta the equatos: l l +, Whe these equatos are solved smultaeousl for l k ad c, we obta: c ad lk 0.6, or: k e Therefore, the epoetal model s: c ˆ ke.8684e or l l k + c The, the varace of the resdual values, for the epoetal model, s calculated as show below, Table : Table. Varace of the resdual values for the epoetal model ŷ ke c.8684 e ŷ ( - ŷ e ( e ( e ( e ( e ( ( ˆ Therefore, the varace of the resdual values, for the epoetal model s: c V(Resdual l ( ˆ ( ke Epoeta [ ] e 8.40

18 4: l, Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp Fttg the Power Model, ŷ a b To ft the power model we eed to eted the gve bvarate data set to geerate the quattes: (l, l ad (l (l, ad ths s accomplshed as show below, Table Table 4. Calculatos for bvarate data of adults for the power model l (l l (l (l l (l l.0967 (l (l 0.0 We the substtute the geerated data to the ormal equatos of the power model, amel equatos (9 ad (40: (l a + b (l a l + b ad obta the equatos: l l (l l G 0.947l (l (l Whe these equatos are solved smultaeousl for b ad l a we obta: G 4.766,H l.6 Therefore, the learzed power model becomes: lŷ l + l The the varace of the resdual values for the power model s obtaed as show below: Table. Varace of the resdual values for the power model l l ŷ l a + b l ŷ - ŷ ( - ŷ l ŷ l ŷ l ŷ l ŷ l ŷ ( - ŷ

19 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 Therefore, the varace of the resduals values for the power model s: V(Resdual b Power ( a Fttg the Quadratc Model, ŷ a + b +c To ft the quadratc model, we eed to use the gve bvarate data set ad eted t to geerate the quattes: X 0; X,790 ; X,49,460; X 4 9,,074; Y 760; X Y 0,60; X Y,,70 We the substtute the geerated data to the ormal equatos of the quadratc model (see equato (, ad obta: a + 0b +,790c 760 K 0a +,790b +,49,460c 0,60,790a +,49,460b + 9,,074c,,70 Solvg these equatos smultaeousl, we obta a -,6/7, b 70/7, c -/7. Therefore, the quadratc fucto ŷ f( s gve b: ŷ a + b + c 7 [, ] 6: The varace of the resdual values for the quadratc model s calculated as show below, Table Table 6. Varace of the resdual values for the quadratc model ŷ [, ] - ŷ ( - ŷ ŷ ŷ ŷ ŷ ŷ Therefore, the varace of the resdual values for the quadratc model s: V(Resdual Quadratc ( - ˆ ( - ŷ Summar of Results ad Selecto of the Best Model We have ftted the 4 models: lear, epoetal, power, ad quadratc models, calculated the respectve resdual varaces, ad have obtaed the followg results:

20 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 a The lear model s: ˆ a + b wth V(Resdual Lear 0 b The epoetal model s: ˆ c The power model s: c ke.8684e wth V(Resdual Epoetal.807 l ˆ l a + b l wth V(Resdual Power.0 l d The quadratc model s [, ] ˆ a + b + c 7 wth V(Resdual Quadratc.486 Sce the lear model has the smallest varace of the resdual values of the 4 models ftted to the same bvarate data set, the lear model s the best model (but the other values are ver close. The lear model, therefore, wll be selected as the best model ad used for forecastg purposes. 6. MINITAB Solutos To obta the MINITAB solutos of the four models we dscussed ths paper we do the followg: 6.. Fdg the MINITAB Soluto for the Lear Model The data set used to fd the MINITAB soluto for the lear model s preseted Table 7. Table 7. Data set MINITAB for the lear model MTB > Set C DATA> DATA> ed MTB > set C DATA> DATA> ed MTB > Name C 'X' C 'Y' MTB > REGRESS 'Y' 'X' The results of the regresso aalss for the lear model s preseted Table 8. Table 8. Regresso aalss: Y versus X for the lear model Regresso equato: Y X Predctor Coef SE Coef T p Costat X Regresso ft: S R-Sq R-Sq (adj.6 97.% 96.% 4

21 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 Aalss of Varace: Source DF SS MS F p Regresso Resdual Error Total Fdg the MINITAB Soluto for the Epoetal Model The data set used to fd the MINITAB soluto for the epoetal model s preseted Table 9. Table 9. Data set MINITAB for the epoetal model MTB > Set C DATA> DATA> ed MTB > set C DATA> DATA> ed MTB > Name C 'X' C 'Y' MTB > REGRESS 'Y' 'X' The results of the regresso aalss for the epoetal model s preseted Table 0. Table 0. Regresso aalss: Y versus X for the epoetal model Regresso equato: Y X Predctor Coef SE Coef T p Costat X Regresso ft: S R-Sq R-Sq (adj % 9.% Aalss of Varace: Source DF SS MS F p Regresso Resdual Error Total Fdg the MINITAB Soluto for the Power Model The data set used to fd the MINITAB soluto for the power model s preseted Table. Table. Data set MINITAB for the power model MTB > Set C DATA> 4.888; ; ; ; DATA> ed MTB > set C DATA> 4.867; ;.006;.094;.798 DATA> ed MTB > Name C 'X' C 'Y' MTB > REGRESS 'Y' 'X' The results of the regresso aalss for the power model s preseted Table.

22 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 Table. Regresso aalss: Y versus X for the power model Regresso equato: Y X Predctor Coef SE Coef T p Costat X Regresso ft: S R-Sq R-Sq (adj % 9.% Aalss of Varace: Source DF SS MS F p Regresso Resdual Error Total Fdg the MINITAB Soluto for the Wuadratc Model The data set used to fd the MINITAB soluto for the quadratc model s preseted Table. Table. Data set MINITAB for the quadratc model MTB > Set C DATA> DATA> ed MTB > set C DATA> DATA> ed MTB > SET C DATA> DATA> END MTB > NAME C 'X' C 'X' C 'Y' MTB > REGRESS 'Y' 'X' 'X' The results of the regresso aalss for the quadratc model s preseted Table 4. Table 4. Regresso aalss: Y versus X, X for the quadratc model Regresso equato: Y X X Predctor Coef SE Coef T p Costat X X Regresso ft: S R-Sq R-Sq (adj % 9.6% Aalss of Varace: Source DF SS MS F p Regresso Resdual Error.86.4 Total Source DF Seg SS X X 7.4 6

23 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp Coclusos Revewg our prevous dscusso we come to the followg coclusos: The Lear Regresso problem s relatvel eas to solve ad ca be hadled usg algebrac methods. The problem ca also be solved easl usg avalable statstcal software, lke MINITAB. Eve though the soluto to Regresso problems ca be obtaed easl usg MINITAB (or other statstcal software t s mportat to kow what the had methodolog s ad how t solves these problems before ou ca properl terpret ad uderstad MINITAB s output. I geeral, o-lear regresso s much more dffcult to perform tha lear regresso. There are, however, some smple o-lear models that ca be evaluated relatvel easl b utlzg the results of lear regresso. The o-lear models aalzed ths paper are: Epoetal Model, Power Model, Quadratc Model. A procedure s also dscussed whch allows us to ft to the same bvarate data set ma models (such as: lear, epoetal, power, quadratc ad select as the best fttg model the model wth the smallest varace of the resduals. I a umercal eample, whch all 4 of these models were ftted to the same bvarate data set, we foud that the Lear model was the best ft, wth the Quadratc model secod best. The Power ad Epoetal models are thrd best ad fourth best respectvel, but are ver close to each other. The evaluato of these models s facltated cosderabl b usg the statstcal software package MINITAB whch, addto to estmatg the ukow parameters of the correspodg models, also geerates addtoal formato (such as the p-value, stadard devatos of the parameter estmators, ad R. Ths addtoal formato allows us to perform hpothess testg ad costruct cofdece tervals o the parameters, ad also to get a measure of the goodess of the equato, b usg the value of R. A value of R close to s a dcato of a good ft. The MINITAB soluto for the lear model shows that both a ad b (of ŷ a + b are sgfcat because the correspodg p-values are smaller tha α 0.0, whle the value of R 97.%, dcatg that the regresso equato eplas 97.% of the varato the -values ad ol.9% s due to other factors. The MINITAB soluto for the quadratc model shows that a, b, ad c (of ŷ a + b + c -, are dvduall ot sgfcat (because of the correspodg hgh p-values, but b ad c jotl are sgfcat because of the correspodg p-value of p 0.0 < α 0.0. The value of R s: R 97.8%. The MINITAB soluto for the power model shows that both a ad b (of ŷ a b or l l a + b l l are sgfcat because the correspodg p-values are smaller tha α 0.0, whle the value of R 96.%. The MINITAB soluto for the epoetal model shows that the k ( ŷ ke c.8684e or l ŷ l k + c s ot sgfcat because of the correspodg hgh p-value, whle the c s sgfcat because of the correspodg p-value beg smaller tha α 0.0. The value of R 96.4%. Refereces Adamowsk, J., H., Fug Cha, S.O., Prasher, B., Ozga-Zelsk, ad Slusareva. A., 0. Comparso of multple lear ad olear regresso, autoregressve tegrated movg average, artfcal eural etwork, ad wavelet artfcal eural etwork methods for urba water demad forecastg. Water Resources, 48, W08, Motreal: Caada Bereso, M.L., Leve, D.M. ad Krehbel, T.C., 004. Basc Busess Statstcs (9th Edto. Upper Saddle Rver, N.J.: Pretce-Hall Bhata, N., 009. Lear Regresso: A Approach for Forecastg Black, K., 004. Busess statstcs (4th Edto. Hoboke, NJ: Wle Caavos, G.C., 984. Appled Probablt ad Statstcal Methods. Bosto: Lttle Brow Carlso, W.L. ad Thore, B., 997. Appled Statstcal Methods. Upper Saddle Rver, N.J.: Pretce-Hall Che, Kua-Yu, 0. Combg lear ad olear model forecastg toursm demad. Epert Sstems wth Applcatos, 8(8, pp

24 Vaslopoulos, A., 0. Lear ad No-Lear Regresso: Powerful ad Ver Importat Forecastg Methods. Epert Joural of Busess ad Maagemet, (, pp.0-8 Chldress, R.L., Gorsk, R.D. ad Wtt, R.M., 989. Mathematcs for Maageral Decsos. Upper Saddle Rver, N.J.: Pretce-Hall Chou, Ya-lu, 99. Statstcal Aalss for Busess ad Ecoomcs. New York: Elsever Freud, J.E. ad Wllams, F.J., 98. Elemetar Busess Statstcs: The Moder Approach. Upper Saddle Rver, N.J.: Pretce-Hall McClave, J.T., Beso, G.P. ad Scch, T., 00. Statstcs for Busess ad Ecoomcs (8th Edto. Upper Saddle Rver, N.J.: Pretce-Hall Pdck, R. ad Rubfeld, D.L., 98. Ecoometrc Models ad Ecoomc Forecasts (d Edto. New York: McGraw-Hll Vaslopoulos, A. ad Lu, F.V., 006. Quattatve Methods for Busess wth Computer Applcatos. Bosto, MA: Pearso Custom Publshg Vaslopoulos, A., 00. Regresso Aalss Revsted. Revew of Busess, 6 (, pp.6-46 Vaslopoulos, A., 007. Busess Statstcs A Logcal Approach. Theor, Models, Procedures, ad Applcatos Icludg Computer (MINITAB Solutos. Bosto, MA: Pearso Custom Publshg 8