Multiple Regressions and Correlation Analysis

Transcription

1 Mulple Regreon and Correlaon Analy Chaper 4 McGraw-Hll/Irwn Copyrgh 2 y The McGraw-Hll Compane, Inc. All rgh reerved.

2 GOALS. Decre he relaonhp eween everal ndependen varale and a dependen varale ung mulple regreon analy. 2. Se up, nerpre, and apply an ANOVA ale 3. Compue and nerpre he mulple andard error of emae, he coeffcen of mulple deermnaon, and he adjued coeffcen of mulple deermnaon. 4. Conduc a e of hypohe o deermne wheher regreon coeffcen dffer from zero. 5. Conduc a e of hypohe on each of he regreon coeffcen. 6. Ue redual analy o evaluae he aumpon of mulple regreon analy. 7. Evaluae he effec of correlaed ndependen varale. 8. Ue and underand qualave ndependen varale. 9. Underand and nerpre he epwe regreon mehod.. Underand and nerpre pole neracon among ndependen varale. 4-2

3 Mulple Regreon Analy The general mulple regreon wh k ndependen varale gven y: X X k are he ndependen varale. a he Y-nercep he ne change n Y for each un change n X holdng X 2 X k conan. I called a paral regreon coeffcen or ju a regreon coeffcen. The lea quare creron ued o develop h equaon. Deermnng, 2, ec. very edou, a ofware package uch a Excel or MINITAB recommended. 4-3

4 Mulple Lnear Regreon - Example ^ Y X X 2 X 3 Salerry Realy ell home along he ea coa of he Uned Sae. One of he queon mo frequenly aked y propecve uyer : If we purchae h home, how much can we expec o pay o hea durng he wner? The reearch deparmen a Salerry ha een aked o develop ome gudelne regardng heang co for nglefamly home. Three varale are hough o relae o he heang co: () he mean daly oude emperaure, (2) he numer of nche of nulaon n he ac, and (3) he age n year of he furnace. To nvegae, Salerry reearch deparmen eleced a random ample of 2 recenly old home. I deermned he co o hea each home la January, a well 4-4

5 Mulple Lnear Regreon Mna Example Regreon oupu produced y Mna and Excel a

6 The Mulple Regreon Equaon Inerpreng he Regreon Coeffcen and Applyng he Model for Emaon Inerpreng he Regreon Coeffcen The regreon coeffcen for mean oude emperaure, X, The coeffcen negave a he oude emperaure ncreae, he co o hea he home decreae. For every un ncreae n emperaure, holdng he oher wo ndependen varale conan, monhly heang co expeced o decreae y $ Applyng he Model for Emaon Wha he emaed heang co for a home f he mean oude emperaure 3 degree, here are 5 nche of nulaon n he ac, and he furnace year old? The ac nulaon varale, X 2, alo how an nvere relaonhp (negave coeffcen). The more nulaon n he ac, he le he co o hea he home. For each addonal nch of nulaon, he co o hea he home expeced o declne y $4.83 per monh. 4-6 The age of he furnace varale how a drec relaonhp. Wh an older furnace, he co o hea he home ncreae. For each addonal year older he furnace, he co expeced o ncreae y $6. per monh.

7 Mulple Sandard Error of Emae The mulple andard error of emae a meaure of he effecvene of he regreon equaon. I meaured n he ame un a he dependen varale. I dffcul o deermne wha a large value and wha a mall value of he andard error. 4-7

8 Mulple Regreon and Correlaon Aumpon The ndependen varale and he dependen varale have a lnear relaonhp. The dependen varale mu e connuou and a lea nervalcale. The redual mu e he ame for all value of Y. When h he cae, we ay he dfference exh homocedacy. The redual hould follow he normal drued wh mean. Succeve value of he dependen varale mu e uncorrelaed. 4-8

9 Coeffcen of Mulple Deermnaon (r 2 ) Coeffcen of Mulple Deermnaon:. Symolzed y R Range from o. 3. Canno aume negave value. 4. Eay o nerpre. The Adjued R 2. The numer of ndependen varale n a mulple regreon equaon make he coeffcen of deermnaon larger. 2. If he numer of varale, k, and he ample ze, n, are equal, he coeffcen of deermnaon.. 3. To alance he effec ha he numer of ndependen varale ha on he coeffcen of mulple deermnaon, adjued R 2 ued nead. 4-9

10 Gloal Te: Teng he Mulple Regreon Model The gloal e ued o nvegae wheher any of he ndependen varale have gnfcan coeffcen. The hypohee are: H H : 2... : No all equal k Decon Rule: Rejec H f F > F,k,n-k- 4-

11 Fndng he Compued and Crcal F F,k,n-k- F.5,3,6 4-

12 Inerpreaon The compued value of F 2.9, whch n he rejecon regon. The null hypohe ha all he mulple regreon coeffcen are zero herefore rejeced. Inerpreaon: ome of he ndependen varale (amoun of nulaon, ec.) do have he aly o explan he varaon n he dependen varale (heang co). Logcal queon whch one?

13 Evaluang Indvdual Regreon Coeffcen (β = ) Th e ued o deermne whch ndependen varale have nonzero regreon coeffcen. The varale ha have zero regreon coeffcen are uually dropped from he analy. The e ac he druon wh n-(k+) degree of freedom. The hypohe e a follow: H : β = H : β Rejec H f > /2,n-k- or < - /2,n-k- 4-3

14 4-4 Crcal for he Slope : f Rejec H.25,6.25,6 3 2,2.5/ 3 2,2.5/ 2, / 2, / 2, / 2, / k n k n k n k n

15 4-5 Compued for he Slope

16 Concluon on Sgnfcance of Slope (Temp) -3.9 (Inulaon).52 (Age) 4-6

17 4-7 New Regreon Model whou Varale Age Mna

18 4-8 New Regreon Model whou Varale Age Mna

19 Teng he New Model for Sgnfcance d.f. (2,7) 3.59 Compued F =

20 4-2 Crcal -a for he New Slope : f Rejec H.25,7.25,7 2 2,2.5/ 2 2,2.5/ 2, / 2, / 2, / 2, / k n k n k n k n

21 Concluon on Sgnfcance of New Slope (Temp) Inulaon

22 Evaluang he Aumpon of Mulple Regreon. There a lnear relaonhp. Tha, here a ragh-lne relaonhp eween he dependen varale and he e of ndependen varale. 2. The varaon n he redual he ame for oh large and mall value of he emaed Y To pu anoher way, he redual unrelaed wheher he emaed Y large or mall. 3. The redual follow he normal proaly druon. 4. The ndependen varale hould no e correlaed. Tha, we would lke o elec a e of ndependen varale ha are no hemelve correlaed. 5. The redual are ndependen. Th mean ha ucceve oervaon of he dependen varale are no correlaed. Th aumpon ofen volaed when me nvolved wh he ampled oervaon. A redual he dfference eween he acual value of Y and he predced value of Y. 4-22

23 Scaer and Redual Plo A plo of he redual and her correpondng Y value ued for howng ha here are no rend or paern n he redual. 4-23

24 Druon of Redual Boh MINITAB and Excel offer anoher graph ha help o evaluae he aumpon of normally drued redual. I a called a normal proaly plo and hown o he rgh of he hogram. 4-24

25 Mulcollneary Mulcollneary ex when ndependen varale (X ) are correlaed. Effec of Mulcollneary on he Model:. An ndependen varale known o e an mporan predcor end up havng a regreon coeffcen ha no gnfcan. 2. A regreon coeffcen ha hould have a pove gn urn ou o e negave, or vce vera. 3. When an ndependen varale added or removed, here a drac change n he value of he remanng regreon coeffcen. However, correlaed ndependen varale do no affec a mulple regreon equaon aly o predc he dependen varale (Y). 4-25

26 Varance Inflaon Facor A general rule f he correlaon eween wo ndependen varale eween -.7 and.7 here lkely no a prolem ung oh of he ndependen varale. A more prece e o ue he varance nflaon facor (VIF). A VIF > unafacory. Remove ha ndependen varale from he analy. The value of VIF found a follow: VIF 2 R j The erm R 2 j refer o he coeffcen of deermnaon, where he eleced ndependen varale ued a a dependen varale and he remanng ndependen varale are ued a ndependen varale. 4-26

27 Mulcollneary Example Refer o he daa n he ale, whch relae he heang co o he ndependen varale oude emperaure, amoun of nulaon, and age of furnace. Develop a correlaon marx for all he ndependen varale. Doe appear here a prolem wh mulcollneary? Correlaon Marx of he Varale Fnd and nerpre he varance nflaon facor for each of he ndependen varale. 4-27

28 VIF Mna Example Coeffcen of Deermnaon The VIF value of.32 le han he upper lm of. Th ndcae ha he ndependen varale emperaure no rongly correlaed wh he oher ndependen varale. 4-28

29 Independence Aumpon The ffh aumpon aou regreon and correlaon analy ha ucceve redual hould e ndependen. When ucceve redual are correlaed we refer o h condon a auocorrelaon. Auocorrelaon frequenly occur when he daa are colleced over a perod of me. 4-29

30 Redual Plo veru Fed Value: Teng he Independence Aumpon When ucceve redual are correlaed we refer o h condon a auocorrelaon, whch frequenly occur when he daa are colleced over a perod of me. Noe he run of redual aove he mean of he redual, followed y a run elow he mean. A caer plo uch a h would ndcae pole auocorrelaon. 4-3

31 Qualave Varale - Example Frequenly we wh o ue nomnal-cale varale uch a gender, wheher he home ha a wmmng pool, or wheher he por eam wa he home or he vng eam n our analy. Thee are called qualave varale. To ue a qualave varale n regreon analy, we ue a cheme of dummy varale n whch one of he wo pole condon coded and he oher. EXAMPLE Suppoe n he Salerry Realy example ha he ndependen varale garage added. For hoe home whou an aached garage, ued; for home wh an aached garage, a ued. We wll refer o he garage varale a The daa from Tale 4 2 are enered no he MINITAB yem. 4-3

32 Qualave Varale - Mna Garage a dummy varale 4-32

33 Ung he Model for Emaon Wha he effec of he garage varale? Suppoe we have wo houe exacly alke nex o each oher n Buffalo, New York; one ha an aached garage, and he oher doe no. Boh home have 3 nche of nulaon, and he mean January emperaure n Buffalo 2 degree. For he houe whou an aached garage, a uued for n he regreon equaon. The emaed heang co $28.9, found y: Whou garage For he houe wh an aached garage, a uued for n he regreon equaon. The emaed heang co $358.3, found y: Wh garage 4-33

34 Evaluang Indvdual Regreon Coeffcen (β = ) Th e ued o deermne whch ndependen varale have nonzero regreon coeffcen. The varale ha have zero regreon coeffcen are uually dropped from he analy. The e ac he druon wh n-(k+) or n-k-degree of freedom. The hypohe e a follow: H : β = H : β Rejec H f > /2,n-k- or < - /2,n-k- 4-34

35 4-35 Teng Varale Garage for Sgnfcance : f Rejec H.25,6.25,6 3 2,2.5/ 3 2,2.5/ 2, / 2, / 2, / 2, / k n k n k n k n Concluon: The regreon coeffcen no zero. The ndependen varale garage hould e ncluded n he analy.

36 Sepwe Regreon The advanage o he epwe mehod are:. Only ndependen varale wh gnfcan regreon coeffcen are enered no he equaon. 2. The ep nvolved n uldng he regreon equaon are clear. 3. I effcen n fndng he regreon equaon wh only gnfcan regreon coeffcen. 4. The change n he mulple andard error of emae and he coeffcen of deermnaon are hown. 4-36

37 Sepwe Regreon Mna Example The epwe MINITAB oupu for he heang co prolem follow. Temperaure eleced fr. Th varale explan more of he varaon n heang co han any of he oher hree propoed ndependen varale. Garage eleced nex, followed y Inulaon. 4-37

38 Regreon Model wh Ineracon In Chaper 2 neracon among ndependen varale wa covered. Suppoe we are udyng wegh lo and aume, a he curren leraure ugge, ha de and exerce are relaed. So he dependen varale amoun of change n wegh and he ndependen varale are: de (ye or no) and exerce (none, moderae, gnfcan). We are nereed n eeng f hoe uded who mananed her de and exerced gnfcanly ncreaed he mean amoun of wegh lo? In regreon analy, neracon can e examned a a eparae ndependen varale. An neracon predcon varale can e developed y mulplyng he daa value n one ndependen varale y he value n anoher ndependen varale, herey creang a new ndependen varale. A wo-varale model ha nclude an neracon erm : Refer o he heang co example. I here an neracon eween he oude emperaure and he amoun of nulaon? If oh varale are ncreaed, he effec on heang co greaer han he um of avng from warmer emperaure and he avng from ncreaed nulaon eparaely? 4-38

39 Regreon Model wh Ineracon - Example 4-39 Creang he Ineracon Varale Ung he nformaon from he ale n he prevou lde, an neracon varale creaed y mulplyng he emperaure varale y he nulaon. For he fr ampled home he value emperaure 35 degree and nulaon 3 nche o he value of he neracon varale 35 X 3 = 5. The value of he oher neracon produc are found n a mlar fahon.

40 Regreon Model wh Ineracon - Example The regreon equaon : I he neracon varale gnfcan a.5 gnfcance level? 4-4