Department of Quantitative Methods & Information Systems. Time Series and Their Components QMIS 320. Chapter 6

Transcription

1 Department of Quanttatve Methods & Informaton Systems Tme Seres and Ther Components QMIS 30 Chapter 6 Fall 00 Dr. Mohammad Zanal These sldes were modfed from ther orgnal source for educatonal purpose only. The copyrghted materals belong to Busness Statstcs: A Decson Makng Approach, 7e 008 Prentce Hall, Inc. Chapter Goals After completng ths chapter, you should be able to: Explan the smple lnear regresson model Obtan and nterpret the smple lnear regresson equaton for a set of data Descrbe R as a measure of explanatory power of the regresson model Understand the assumptons behnd regresson analyss Explan measures of varaton and determne whether the ndependent varable s sgnfcant

2 3 Chapter Goals (contnued) After completng ths chapter, you should be able to: Calculate and nterpret confdence ntervals for the regresson coeffcents Use a regresson equaton for predcton Form forecast ntervals around an estmated Y value for a gven X Use graphcal analyss to recognze potental problems n regresson analyss Explan the correlaton coeffcent and perform a hypothess test for zero populaton correlaton 4 Overvew of Lnear Models An equaton can be ft to show the best lnear relatonshp between two varables: Y = β 0 + β X Where Y s the dependent varable and X s the ndependent varable β 0 s the Y-ntercept β s the slope

3 5 Least Squares Regresson Estmates for coeffcents β 0 and β are found usng a Least Squares Regresson technque The least-squares regresson lne, based on sample data, s yˆ = b0 + bx Where b s the slope of the lne and b 0 s the y- ntercept: Cov(x,y) b = s x b0 = y bx 6 Introducton to Regresson Analyss Regresson analyss s used to: Predct the value of a dependent varable based on the value of at least one ndependent varable Explan the mpact of changes n an ndependent varable on the dependent varable Dependent varable: the varable we wsh to explan (also called the endogenous varable) Independent varable: the varable used to explan the dependent varable (also called the exogenous varable)

4 7 Lnear Regresson Model The relatonshp between X and Y s descrbed by a lnear functon Changes n Y are assumed to be caused by changes n X Lnear regresson populaton equaton model Y = β + β x + ε 0 Where β 0 and β are the populaton model coeffcents and ε s a random error term. 8 Smple Lnear Regresson Model The populaton regresson model: Dependent Varable Populaton Y ntercept Y = β + β X + 0 Populaton Slope Coeffcent Independent Varable ε Random Error term Lnear component Random Error component

5 9 Smple Lnear Regresson Model (contnued) Y Y = β + β X + ε 0 Observed Value of Y for X Predcted Value of Y for X ε Random Error for ths X value Slope = β Intercept = β 0 X X 0 Smple Lnear Regresson Equaton The smple lnear regresson equaton provdes an estmate of the populaton regresson lne Estmated (or predcted) y value for observaton y ˆ = b + b Estmate of the regresson ntercept 0 Estmate of the regresson slope x Value of x for observaton The ndvdual random error terms e have a mean of zero e = y - yˆ ) = y -(b + b x ) ( 0

6 Least Squares Estmators b 0 and b are obtaned by fndng the values of b 0 and b that mnmze the sum of the squared dfferences between y and ŷ : mn SSE = mn = mn = mn e (y yˆ ) [y (b 0 + b x )] Dfferental calculus s used to obtan the coeffcent estmators b 0 and b that mnmze SSE Least Squares Estmators (contnued) The slope coeffcent estmator s n (x x)(y y) Cov(x, y) = b = = = n sx (x x) = And the constant or y-ntercept s b0 = y bx The regresson lne always goes through the mean x, y r xy s s y x

7 3 Fndng the Least Squares Equaton The coeffcents b 0 and b, and other regresson results n ths chapter, wll be found usng a computer Hand calculatons are tedous Statstcal routnes are bult nto Mntab Other statstcal analyss software can be used 4 Lnear Regresson Model Assumptons The true relatonshp form s lnear (Y s a lnear functon of X, plus random error) The error terms, ε are ndependent of the x values The error terms are random varables wth mean 0 and constant varance, σ (the constant varance property s called homoscedastcty) E[ε ] = 0 and E[ε ] = σ for ( =, K,n) The random error terms, ε, are not correlated wth one another, so that E[ε ε ] = 0 j for all j

8 5 Interpretaton of the Slope and the Intercept s the estmated average value of y when the value of x s zero (f x = 0 s n the range of observed x values) b 0 s the estmated change n the average value of y as a result of a one-unt change n x b 6 Smple Lnear Regresson Example A real estate agent wshes to examne the relatonshp between the sellng prce of a home and ts sze (measured n square feet) A random sample of 0 houses s selected Dependent varable (Y) = house prce n $000s Independent varable (X) = square feet

9 7 Sample Data for House Prce Model House Prce n $000s (Y) Square Feet (X) Graphcal Presentaton House Prce ($000s) House prce model: scatter plot Square Feet

10 9 Regresson Usng Mntab Mntab wll be used to generate the coeffcents and measures of goodness of ft for regresson 0 Mntab Output

11 Mntab Output (contnued) Regresson Statstcs R Square 0.58 Adjusted R Square 0.58 Standard Error Observatons 0 The regresson equaton s: house prce = (square feet) ANOVA df SS MS F Sgnfcance F Regresson Resdual Total Coeffcents Standard Error t Stat P-value Lower 95% Upper 95% Intercept Square Feet Graphcal Presentaton House prce model: scatter plot and regresson lne Intercept = House Prce ($000s) Square Feet Slope = house prce = (square feet)

12 3 Interpretaton of the Intercept, b 0 house prce = (square feet) b 0 s the estmated average value of Y when the value of X s zero (f X = 0 s n the range of observed X values) Here, no houses had 0 square feet, so b 0 = just ndcates that, for houses wthn the range of szes observed, $98,48.33 s the porton of the house prce not explaned by square feet 4 Interpretaton of the Slope Coeffcent, b house prce = (square feet) b measures the estmated change n the average value of Y as a result of a oneunt change n X Here, b =.0977 tells us that the average value of a house ncreases by.0977($000) = $09.77, on average, for each addtonal one square foot of sze

13 5 Measures of Varaton Total varaton s made up of two parts: SST = SSR + SSE Total Sum of Squares Regresson Sum of Squares Error Sum of Squares SST = (y y SSR = (yˆ y SSE = (y yˆ ) where: y ) = Average value of the dependent varable y = Observed values of the dependent varable ŷ = Predcted value of y for the gven x value ) 6 Measures of Varaton (contnued) SST = total sum of squares Measures the varaton of the y values around ther mean, y SSR = regresson sum of squares Explaned varaton attrbutable to the lnear relatonshp between x and y SSE = error sum of squares Varaton attrbutable to factors other than the lnear relatonshp between x and y

14 7 y Y _ y y _ SST = (y - y) Measures of Varaton SSE = (y - y ) _ SSR = (y - y) (contnued) y _ y x X 8 Coeffcent of Determnaton, R The coeffcent of determnaton s the porton of the total varaton n the dependent varable that s explaned by varaton n the ndependent varable The coeffcent of determnaton s also called R-squared and s denoted as R SSR regresson sum of squares R = = SST total sum of squares note: 0 R

15 9 Examples of Approxmate r Values Y r = Y r = X Perfect lnear relatonshp between X and Y: 00% of the varaton n Y s explaned by varaton n X r = X 30 Examples of Approxmate r Values Y Y X 0 < r < Weaker lnear relatonshps between X and Y: Some but not all of the varaton n Y s explaned by varaton n X X

16 3 Examples of Approxmate r Values Y r = 0 No lnear relatonshp between X and Y: r = 0 X The value of Y does not depend on X. (None of the varaton n Y s explaned by varaton n X) 3 Regresson Statstcs R Square 0.58 Adjusted R Square 0.58 Standard Error Observatons 0 Mntab Output SSR R = = = SST % of the varaton n house prces s explaned by varaton n square feet ANOVA df SS MS F Sgnfcance F Regresson Resdual Total Coeffcents Standard Error t Stat P-value Lower 95% Upper 95% Intercept Square Feet

17 33 Correlaton and R The coeffcent of determnaton, R, for a smple regresson s equal to the smple correlaton squared R = rxy 34 Estmaton of Model Error Varance An estmator for the varance of the populaton model error s σˆ = s e n e = SSE = = n n Dvson by n nstead of n s because the smple regresson model uses two estmated parameters, b 0 and b, nstead of one s e = s e s called the standard error of the estmate

18 35 Regresson Statstcs R Square 0.58 Adjusted R Square 0.58 Standard Error Observatons 0 Mntab Output s e = ANOVA df SS MS F Sgnfcance F Regresson Resdual Total Coeffcents Standard Error t Stat P-value Lower 95% Upper 95% Intercept Square Feet Comparng Standard Errors s e s a measure of the varaton of observed y values from the regresson lne Y Y small s e X large se X The magntude of s e should always be judged relatve to the sze of the y values n the sample data.e., s e = $4.33K s moderately small relatve to house prces n the $00 - $300K range

19 .5 37 Inferences About the Regresson Model The varance of the regresson slope coeffcent (b ) s estmated by s b se = (x x) = se (n )s x where: s b = Estmate of the standard error of the least squares slope s e = SSE = Standard error of the estmate n 38 Mntab Output Regresson Statstcs R Square 0.58 Adjusted R Square 0.58 Standard Error Observatons 0 s = b ANOVA df SS MS F Sgnfcance F Regresson Resdual Total Coeffcents Standard Error t Stat P-value Lower 95% Upper 95% Intercept Square Feet

20 39 Comparng Standard Errors of the Slope S b s a measure of the varaton n the slope of regresson lnes from dfferent possble samples Y Y small S b X large S b X 40 Inference about the Slope: t Test t test for a populaton slope Is there a lnear relatonshp between X and Y? Null and alternatve hypotheses H 0 : β = 0 H : β 0 Test statstc t = (no lnear relatonshp) (lnear relatonshp does exst) b β s b d.f. = n where: b = regresson slope coeffcent β = hypotheszed slope s b = standard error of the slope

21 4 Inference about the Slope: t Test (contnued) House Prce n $000s (y) Square Feet (x) Estmated Regresson Equaton: house prce = (sq.ft.) The slope of ths model s Does square footage of the house affect ts sales prce? 4 Inferences about the Slope: t Test Example H 0 : β = 0 H : β 0 From Mntab output: s b Coeffcents Standard Error t Stat P-value Intercept Square Feet b b β t = s b = t =

22 43 H 0 : β = 0 H : β 0 d.f. = 0- = 8 t 8,.05 =.3060 α/=.05 Inferences about the Slope: t Test Example Test Statstc: t = 3.39 Reject H 0 Do not reject H 0 Reject H 0 -t n-,α/ 0 t n-,α/ From Mntab output: α/= b sb (contnued) Coeffcents Standard Error t Stat P-value Intercept Square Feet Decson: Reject H 0 Concluson: There s suffcent evdence that square footage affects house prce t 44 H 0 : β = 0 H : β 0 Inferences about the Slope: t Test Example Ths s a two-tal test, so the p-value s P(t > 3.39)+P(t < -3.39) = (for 8 d.f.) P-value = From Mntab output: Decson: P-value < α so (contnued) P-value Coeffcents Standard Error t Stat P-value Intercept Square Feet Reject H 0 Concluson: There s suffcent evdence that square footage affects house prce

23 45 Confdence Interval Estmate for the Slope Confdence Interval Estmate of the Slope: b tn,α/sb < β < b + tn,α/s Mntab Prntout for House Prces: b d.f. = n - Coeffcents Standard Error t Stat P-value Lower 95% Upper 95% Intercept Square Feet At 95% level of confdence, the confdence nterval for the slope s (0.0337, 0.858) 46 Confdence Interval Estmate for the Slope (contnued) Coeffcents Standard Error t Stat P-value Lower 95% Upper 95% Intercept Square Feet Snce the unts of the house prce varable s $000s, we are 95% confdent that the average mpact on sales prce s between $33.70 and $85.80 per square foot of house sze Ths 95% confdence nterval does not nclude 0. Concluson: There s a sgnfcant relatonshp between house prce and square feet at the.05 level of sgnfcance

24 47 F-Test for Sgnfcance F Test statstc: F = MSR MSE where MSR = SSR k SSE MSE = n k where F follows an F dstrbuton wth k numerator and (n k - ) denomnator degrees of freedom (k = the number of ndependent varables n the regresson model) 48 Mntab Output Regresson Statstcs R Square 0.58 Adjusted R Square 0.58 Standard Error Observatons 0 MSR F = = =.0848 MSE Wth and 8 degrees of freedom ANOVA df SS MS F Sgnfcance F Regresson Resdual Total P-value for the F-Test Coeffcents Standard Error t Stat P-value Lower 95% Upper 95% Intercept Square Feet

25 0 49 H 0 : β = 0 H : β 0 α =.05 df = df = 8 Do not reject H 0 Crtcal Value: F α = 5.3 F-Test for Sgnfcance α =.05 Reject H 0 F.05 = 5.3 F Test Statstc: MSR F = =.08 MSE Decson: Reject H 0 at α = 0.05 (contnued) Concluson: There s suffcent evdence that house sze affects sellng prce 50 Predcton The regresson equaton can be used to predct a value for y, gven a partcular x For a specfed value, x n+, the predcted value s ˆ y n+ = b0 + bxn+

26 5 Predctons Usng Regresson Analyss Predct the prce for a house wth 000 square feet: house prce = = (sq.ft.) (000) = The predcted prce for a house wth 000 square feet s 37.85($,000s) = $37,850 5 Relevant Data Range When usng a regresson model for predcton, only predct wthn the relevant range of data Relevant data range House Prce ($000s) Square Feet Rsky to try to extrapolate far beyond the range of observed X s

27 53 Estmatng Mean Values and Predctng Indvdual Values Goal: Form ntervals around y to express uncertanty about the value of y for a gven x Confdence Interval for Y the expected value of y, gven x y y = b 0 +b x Predcton Interval for an sngle observed y, gven x x X 54 Confdence Interval for the Average Y, Gven X Confdence nterval estmate for the expected value of y gven a partcular x Confdence nterval for E(Y n+ X n+ ): yˆ n+ ± t n,α/ s e + n (xn x) (x x) + Notce that the formula nvolves the term (x x n + ) so the sze of nterval vares accordng to the dstance x n+ s from the mean, x

28 55 Predcton Interval for an Indvdual Y, Gven X Confdence nterval estmate for an actual observed value of y gven a partcular x Confdence nterval for yˆ n+ : yˆ n+ ± t n,α/ s e + n (xn x) + (x x) + Ths extra term adds to the nterval wdth to reflect the added uncertanty for an ndvdual case 56 Estmaton of Mean Values: Example Confdence Interval Estmate for E(Y n+ X n+ ) Fnd the 95% confdence nterval for the mean prce of,000 square-foot houses Predcted Prce y = ($,000s) (x x) yˆ n + ± tn-,α/se + = ± 37. n (x x) The confdence nterval endponts are and , or from $80,660 to $354,900

29 57 Estmaton of Indvdual Values: Example Confdence Interval Estmate for y n+ Fnd the 95% confdence nterval for an ndvdual house wth,000 square feet Predcted Prce y = ($,000s) (X X) yˆ n + ± tn-,α/se + + = ± 0.8 n (X X) The confdence nterval endponts are 5.50 and 40.07, or from $5,500 to $40, Correlaton Analyss Correlaton analyss s used to measure strength of the assocaton (lnear relatonshp) between two varables Correlaton s only concerned wth strength of the relatonshp No causal effect s mpled wth correlaton Correlaton was frst presented n Chapter 3

30 59 Correlaton Analyss The populaton correlaton coeffcent s denoted ρ (the Greek letter rho) The sample correlaton coeffcent s where s xy r = (x = s s x xy s y x)(y n y) 60 Hypothess Test for Correlaton To test the null hypothess of no lnear assocaton, H 0 : ρ = 0 the test statstc follows the Student s t dstrbuton wth (n ) degrees of freedom: t = r (n ) ( r )

31 6 Decson Rules Hypothess Test for Correlaton Lower-tal test: H 0 : ρ 0 H : ρ < 0 Upper-tal test: H 0 : ρ 0 H : ρ >0 Two-tal test: H 0 : ρ = 0 H : ρ 0 α α α/ α/ -t α t α -t α/ t α/ Reject H 0 f t < -t n-, α Reject H 0 f t > t n-, α Reject H 0 f t < -t n-, α/ or t > t n-, α/ Where r (n ) t = ( r ) has n - d.f. 6 Graphcal Analyss The lnear regresson model s based on mnmzng the sum of squared errors If outlers exst, ther potentally large squared errors may have a strong nfluence on the ftted regresson lne Be sure to examne your data graphcally for outlers and extreme ponts Decde, based on your model and logc, whether the extreme ponts should reman or be removed

32 63 Chapter Summary Introduced the lnear regresson model Revewed correlaton and the assumptons of lnear regresson Dscussed estmatng the smple lnear regresson coeffcents Descrbed measures of varaton Descrbed nference about the slope Addressed estmaton of mean values and predcton of ndvdual values