Transcription

1 Regresso

2

3

4

5

6

7

8

9 What s a Model? 1. Ofte Descrbe Relatoshp betwee Varables 2. Types - Determstc Models (o radomess) - Probablstc Models (wth radomess) EPI 809/Sprg

10 Determstc Models 1. Hypothesze Eact Relatoshps 2. Sutable Whe Predcto Error s Neglgble 3. Eample: Body mass de (BMI) s measure of body fat based BMI = Weght Klograms (Heght Meters) 2 EPI 809/Sprg

11 Probablstc Models 1. Hypothesze 2 Compoets Determstc Radom Error 2. Eample: Systolc blood pressure of ewbors Is 6 Tmes the Age days + Radom Error SBP = 6 age(d) + Radom Error May Be Due to Factors Other Tha age days (e.g. Brthweght) EPI 809/Sprg

12 Smple Regresso Smple regresso aalyss s a statstcal tool that gves us the ablty to estmate the mathematcal relatoshp betwee a depedet varable (usually called y) ad a depedet varable (usually called ). The depedet varable s the varable for whch we wat to make a predcto. Whle varous o-lear forms may be used, smple lear regresso models are the most commo.

13 Itroducto The prmary goal of quattatve aalyss s to use curret formato about a pheomeo to predct ts future behavor. Curret formato s usually the form of a set of data. I a smple case, whe the data form a set of pars of umbers, we may terpret them as represetg the observed values of a depedet (or predctor or eplaatory) varable X ad a depedet ( or respose or outcome) varable Y. lot sze Ma-hours

14 Ma-Hour Itroducto The goal of the aalyst who studes the data s to fd a fuctoal relato betwee the respose varable y ad the predctor varable Statstcal relato betwee Lot sze ad Ma-Hour y f () Lot sze

15 Pctoral Presetato of Lear Regresso Model

16 Lear Regresso Model EPI 809/Sprg

17

18 Assumptos Lear regresso assumes that 1. The relatoshp betwee X ad Y s lear 2. Y s dstrbuted ormally at each value of X 3. The varace of Y at every value of X s the same (homogeety of varaces) 4. The observatos are depedet

19 Lear Equatos Y Y = mx + b m = Slope Chage X b = Y-tercept Chage Y X T/Maker Co. EPI 809/Sprg

20 Lear Regresso Model 1. Relatoshp Betwee Varables Is a Lear Fucto Populato Y-Itercept Populato Slope Radom Error Y X 0 1 Depedet (Respose) Varable (e.g., CD+ c.) Idepedet (Eplaatory) Varable (e.g., Years s. seroco.)

21 Meag of Regresso Coeffcets Geeral regresso model 1. 0, ad 1 are parameters 2. X s a kow costat 3. Devatos are depedet N(o, 2 ) The values of the regresso parameters 0, ad 1 are ot kow. We estmate them from data. 1 dcates the chage the mea respose per ut crease X.

22 Populato Lear Regresso Model Y Y X 0 1 = Radom error Observed value Y E 0 1 X Observed value X EPI 809/Sprg

23 Estmatg Parameters: Least Squares Method EPI 809/Sprg

24 Scatter plot Plot of All (X, Y ) Pars Suggests How Well Model Wll Ft Y X EPI 809/Sprg

25 Thkg Challege How would you draw a le through the pots? How do you determe whch le fts best? Y X EPI 809/Sprg

26 Thkg Challege How would you draw a le through the pots? How do you determe whch le fts best? Y Itercept uchaged Slope chaged EPI 809/Sprg X

27 Thkg Challege How would you draw a le through the pots? How do you determe whch le fts best? Y Itercept chaged Slope uchaged EPI 809/Sprg X

28 Thkg Challege How would you draw a le through the pots? How do you determe whch le fts best? Y Itercept chaged Slope chaged EPI 809/Sprg X

29 What s the best fttg le

30 Predcto Error

31 Least Squares 1. Best Ft Meas Dfferece Betwee Actual Y Values & Predcted Y Values Are a Mmum. But Postve Dffereces Off-Set Negatve. So square errors! Y Yˆ LS Mmzes the Sum of the Squared Dffereces (errors) (SSE) 1 ˆ 2 EPI 809/Sprg

32 Least Squares Graphcally Y 1 LS mmzes Y ^ 2 X ^ 1 ^ X EPI 809/Sprg ^ Y X

33 How to estmate parameters

34 Estmatg the tercept ad slope: least squares estmato ** Least Squares Estmato A lttle calculus. What are we tryg to estmate? β, the slope, from What s the costrat? We are tryg to mmze the squared dstace (hece the least squares ) betwee the observatos themselves ad the predcted values, or (also called the resduals, or left-over ueplaed varablty) Dfferece = y (β + α) Dfferece 2 = (y (β + α)) 2 Fd the β that gves the mmum sum of the squared dffereces. How do you mamze a fucto? Take the dervatve; set t equal to zero; ad solve. Typcal ma/m problem from calculus. d 2 ( y ( )) 2( ( y )( )) d 1 1 2( 1 ( y )) 0... From here takes a lttle math trckery to solve for β 2

35 The stadard error of Y gve X s the average varablty aroud the regresso le at ay gve value of X. It s assumed to be equal at all values of X. Sy/ Sy/ Sy/ Sy/ Sy/ Sy/

36

37 Regresso Pcture y C A ŷ y A B B y C y 2 ( y y) 1 1 A 2 B 2 C 2 SS total Total squared dstace of observatos from aïve mea of y Total varato ( yˆ y) 1 SS reg Dstace from regresso le to aïve mea of y Varablty due to (regresso) 2 ( yˆ y ) *Least squares estmato gave us the le (β) that mmzed C 2 2 R 2 =SSreg/SStotal SS resdual Varace aroud the regresso le Addtoal varablty ot eplaed by what least squares method ams to mmze

38 Regresso Le If the scatter plot of our sample data suggests a lear relatoshp betwee two varables.e. y 1 0 we ca summarze the relatoshp by drawg a straght le o the plot. Least squares method gve us the best estmated le for our set of sample data.

39 Regresso Le We wll wrte a estmated regresso le based o sample data as yˆ b0 b1 The method of least squares chooses the values for b 0, ad b 1 to mmze the sum of squared errors SSE 2 ( y ˆ y ) y b b 0 1

40 Regresso Le Usg calculus, we obta estmatg formulas: or y y y y b ) ( ) ( ) )( ( b y b 1 0 y S S b r 1

41 Estmato of Mea Respose Ftted regresso le ca be used to estmate the mea value of y for a gve value of. Eample The weekly advertsg epedture () ad weekly sales (y) are preseted the followg table. y

42 Pot Estmato of Mea Respose From prevous table we have: y y The least squares estmates of the regresso coeffcets are: b y ) ( y 10(818755) (564)(14365) 2 10(32604) (564) b (56.4)

43 Pot Estmato of Mea Respose The estmated regresso fucto s: ŷ Sales Epedtur e Ths meas that f the weekly advertsg epedture s creased by $1 we would epect the weekly sales to crease by $10.8.

44 Pot Estmato of Mea Respose Ftted values for the sample data are obtaed by substtutg the value to the estmated regresso fucto. For eample f the advertsg epedture s $50, the the estmated Sales s: Sales (50) 1368 Ths s called the pot estmate (forecast) of the mea respose (sales).

45 Lear correlato ad lear regresso

46 Covarace cov(, y) 1 ( X )( y Y ) 1

47 Iterpretg Covarace cov(x,y) > 0 cov(x,y) < 0 cov(x,y) = 0 X ad Y are postvely correlated X ad Y are versely correlated X ad Y are depedet

48 Correlato coeffcet Pearso s Correlato Coeffcet s stadardzed covarace (utless): covarace(, y) r var var y

49 Correlato Measures the relatve stregth of the lear relatoshp betwee two varables Ut-less Rages betwee 1 ad 1 The closer to 1, the stroger the egatve lear relatoshp The closer to 1, the stroger the postve lear relatoshp The closer to 0, the weaker ay postve lear relatoshp

50 Scatter Plots of Data wth Varous Correlato Coeffcets Y Y Y Y X X r = -1 r = -.6 r = 0 Y Y X X r = +1 r = +.3 Slde from: Statstcs for Maagers Usg Mcrosoft Ecel 4th Edto, 2004 Pretce-Hall X r = 0 X

51 Lear Correlato Lear relatoshps Curvlear relatoshps Y Y X X Y Y X Slde from: Statstcs for Maagers Usg Mcrosoft Ecel 4th Edto, 2004 Pretce-Hall X

52 Lear Correlato Strog relatoshps Weak relatoshps Y Y X X Y Y X Slde from: Statstcs for Maagers Usg Mcrosoft Ecel 4th Edto, 2004 Pretce-Hall X

53 Lear Correlato No relatoshp Y X Y Slde from: Statstcs for Maagers Usg Mcrosoft Ecel 4th Edto, 2004 Pretce-Hall X

54 Calculatg by had 1 ) ( 1 ) ( 1 ) )( ( var var ), ( cov ˆ y y y y y y arace r

55 Smpler calculato formula y y SS SS SS y y y y y y y y r ) ( ) ( ) )( ( 1 ) ( 1 ) ( 1 ) )( ( ˆ y y SS SS SS r ˆ Numerator of covarace Numerators of varace

56 Least Square estmato Slope (beta coeffcet) = ˆ Cov(, y) Var( ) Itercept= Calculate : ˆ y - ˆ Regresso le always goes through the pot: (, y)

57 Relatoshp wth correlato rˆ ˆ SD SD y I correlato, the two varables are treated as equals. I regresso, oe varable s cosdered depedet (=predctor) varable (X) ad the other the depedet (=outcome) varable Y.

58 Resdual Aalyss: check assumptos e Y Yˆ The resdual for observato, e, s the dfferece betwee ts observed ad predcted value Resduals are hghly useful for studyg whether a gve regresso model s approprate for the data at had. Check the assumptos of regresso by eamg the resduals Eame for learty assumpto Eame for costat varace for all levels of X (homoscedastcty) Evaluate ormal dstrbuto assumpto Evaluate depedece assumpto Graphcal Aalyss of Resduals

59 Resdual = observed - predcted X=95 mol/l y yˆ y 34 yˆ 14

60 resduals resduals Resdual Aalyss for Learty Y Y Not Lear Lear Slde from: Statstcs for Maagers Usg Mcrosoft Ecel 4th Edto, 2004 Pretce-Hall

61 resduals resduals Resdual Aalyss for Homoscedastcty Y Y No-costat varace Costat varace Slde from: Statstcs for Maagers Usg Mcrosoft Ecel 4th Edto, 2004 Pretce-Hall

62 resduals resduals resduals Resdual Aalyss for Idepedece Not Idepedet Idepedet X X X Slde from: Statstcs for Maagers Usg Mcrosoft Ecel 4th Edto, 2004 Pretce-Hall

63 Eample: weekly advertsg epedture y y-hat Resdual (e)

64 Estmato of the varace of the error terms, 2 The varace 2 of the error terms the regresso model eeds to be estmated for a varety of purposes. It gves a dcato of the varablty of the probablty dstrbutos of y. It s eeded for makg ferece cocerg regresso fucto ad the predcto of y.

65 Regresso Stadard Error To estmate we work wth the varace ad take the square root to obta the stadard devato. For smple lear regresso the estmate of 2 s the average squared resdual s ˆ y. e ( y y ) 2 2 To estmate, use s estmates the stadard devato of the error term the statstcal 2 model for smple lear regresso. s s y. y.

66 Regresso Stadard Error y y-hat Resdual (e) square(e) y-hat = X total S y

67 Resdual plots The pots ths resdual plot have a curve patter, so a straght le fts poorly

68 Resdual plots The pots ths plot show more spread for larger values of the eplaatory varable, so predcto wll be less accurate whe s large.

69 Varable trasformatos If the resdual plot suggests that the varace s ot costat, a trasformato ca be used to stablze the varace. If the resdual plot suggests a o lear relatoshp betwee ad y, a trasformato may reduce t to oe that s appromately lear. Commo learzg trasformatos are: Varace stablzg trasformatos are: 1, log( ) 1, y log( y), y, y 2

70 2 predctors: age ad vt D

71 Dfferet 3D vew

72 Ft a plae rather tha a le O the plae, the slope for vtam D s the same at every age; thus, the slope for vtam D represets the effect of vtam D whe age s held costat.