CHAPTER6 LINEAR REGRESSION

Transcription

1 CHAPTER6 LINEAR REGRESSION YI-TING HWANG DEPARTMENT OF STATISTICS NATIONAL TAIPEI UNIVERSITY

2 EXAMPLE 1 Suppose that a real-estate developer is interested in determining the relationship between family income (X in thousands of dollars) of local residents and the square footage of their homes (Y, in hundreds of square feet). Income (X) : Footage (Y) :

3 SELECT PLOT

4 SCATTER PLOT

5 SELECT VARIABLES

6 OUTPUT

7 SYNTAX GRAPH /SCATTERPLOT(BIVAR)=income WITH footage /MISSING=LISTWISE.

8 CORRELATIONS To determine whether a significant linear relationship exists between X and Y. We use a sample coefficient of correlation defined as ( X i X )( Yi Y ) r = 2 2 ( X X ) ( Y Y ) i i where X = n 1 X, Y = n 1 i Y i

9 PROPERTIES The measure r ranges from 1 to 1. The larger r is, the stronger is the linear relationship. The measure r near zero indicates there is no linear relationship between X and Y. The sign of r tells us whether the relationship between X and Y is positive or negative.

10 CORRELATIONS

11 SELECT VARIABLES

12 OUTPUT

13 SYNTAX CORRELATIONS /VARIABLES=income footage /PRINT=TWOTAIL NOSIG /MISSING=PAIRWISE.

14 SIMPLE LINEAR REGRESSION Objective: to study the (linear) relationship between two variables X and Y. Model: Y 0 1 = β + β X + ε where (1) β is the deterministic portion of 0 + β 1 X the model. (2) ε is the random error portion.

15 ASSUMPTIONS The mean of each error component is zero. Each error component (random variable) follows an approximate normal distribution. The variance of the error component is the same for each value of X. The errors are independent of each other.

16 DISTRIBUTION OF Y N(β 0 + β 1 X,σ 2 ) Y Y 2 3

17 ESTIMATIONS Estimate the beta coefficients using least square method or maximum likelihood. SS b 1 X = 2 = ( X X ) and SCP = ( X SCP SS Estimate the error variance s 2 XY X = ˆ σ i and = Y b ( Yi Yˆ) n 2 b 0 X XY SSE n e = = = 1 i X )( Y MSE i Y )

18 SAMPLING DISTRIBUTION OF b 1 The point estimator follows a normal distribution with mean and variance b 1 Eb ( X i X )( Yi Y ) ( X X ) = 2 i = β 2 σ ( X 1 2 σ { b 1 } 1 = i X ) 2

19 HYPOTHESIS Two-tailed test Test statistic where 0 : 0 : = β β H H } { = n t b s b t = ) ( / } { X X s b s i

20 MEASURE OF THE STRENGTH OF THE MODEL Coefficient of determination 2 SSE SSE R = 1 = 1 2 ( Y Y ) SSTO i Percentage of explained variation in the dependent variable using the simple linear regression model.

21 R 2 VALUE Since 0< SSE< SSTO, it follows 0< R 2 <1 The larger R 2 is, the more the total variation of Y is reduced by introducing X. Limitations The prediction interval may be wider than desired. R 2 value only measures linear relationship but not curvilinear.

22 REGRESSION

23 SELECT VARIABLES

24 OUTPUT

25 SYNTAX REGRESSION /MISSING LISTWISE /STATISTICS COEFF OUTS R ANOVA /CRITERIA=PIN(.05) POUT(.10) /NOORIGIN /DEPENDENT footage /METHOD=ENTER income.

26 EXAMINE THE RESIDUAL I Assumption 1: the errors are normally distributed. Use histogram Use a probability plot (QQ plot) Assumption 2: The variance of the errors remains constant. Plot residuals against the independent variable Should observe no pattern in the graphs Use weighted least square to resort the violation

27 EXAMINE RESIDUAL II Assumption 3: The error are independent. Plot residuals against time Should observe no pattern in the graphs Use the Durbin-Watson statistics to detect if there is a correlation between residuals and time

28 GRAPHICS

29 OUTPUT -- HISTROGRAM 直方圖 依變數 : FOOTAGE 次數 標準差 =.94 平均數 = N = 迴歸標準化殘差

30 OUTPUT QQ PLOT 迴歸標準化殘差的常態 P-P 圖 依變數 : FOOTAGE 預期累積機率1.00 觀察累積機率

31 CHECK OUTLIERS Outliers are those points which are very fairly obvious in a scatter diagram, since such points do not seem to fit with the remaining observations. An outlier can have a dramatic effect on the least squares line, because the regression line will be pulled in the direction of the outliers, reducing the effectiveness of the regression line as a predictor.

32 2 n p IDENTIFY OUTLIERS OF INDEPENDENT VARIABLES Define the hat matrix as H=X(X X) -1 X. Sample leverage h ii is defined as the diagonal element of the hat matrix. p 0 < h ii < 1 and hii = p i=1 Accept a sample observation is an outlier if h ii >

33 IDENTIFY OUTLIERS OF DEPENDENT VARIABLES Standardized residuals standardiz ed residual = Accept a sample observation is an outlier if its standardized residual is larger than 2 or less than 2. s Y -Yˆ i i 1-h ii

34 IDENTIFY INFLUENTIAL OBSERVATIONS Observations that have a very large impact on the sample regression line are called the influential observations. Cook s distance measure 1 hii 2 Di = (standardized residual) 2 1 h ii The cutoff point for the Cook s distance is 0.8

35 STATISTICS

36 SAVE STATISTICS

37 OUTPUT

38 EXAMPLE 2 The editor of a monthly automotive magazine is interested in determining how well automotive manufacturers are meeting federal mandates concerning the average fuel economy that a manufacturer s fleet of cars must reach. The editor suspects that a linear relationship exists between X=engine capacity (in liters) and Y=miles per gallon (mpg).

39 EXAMPLE 3 Observational studies have suggested that low dietary intake or low plasma concentrations of retinol, betacarotene, or other carotenoids might be associated with increased risk of developing certain types of cancer. However, relatively few studies have investigated the determinants of plasma concentrations of these micronutrients. A cross-sectional study is designed to investigate the relationship between personal characteristics and dietary factors, and plasma concentrations of retinol, betacarotene and other carotenoids. Study subjects (N = 315) were patients who had an elective surgical procedure during a three-year period to biopsy or remove a lesion of the lung, colon, breast, skin, ovary or uterus that was found to be non-cancerous.

40 VARIABLE NAMES Variable Names in order from left to right: AGE: Age (years) SEX: Sex (1=Male, 2=Female). SMOKSTAT: Smoking status (1=Never, 2=Former, 3=Current Smoker) QUETELET: Quetelet (weight/(height^2)) VITUSE: Vitamin Use (1=Yes, fairly often, 2=Yes, not often, 3=No) CALORIES: Number of calories consumed per day. FAT: Grams of fat consumed per day. FIBER: Grams of fiber consumed per day. ALCOHOL: Number of alcoholic drinks consumed per week. CHOLESTEROL: Cholesterol consumed (mg per day). BETADIET: Dietary beta-carotene consumed (mcg per day). RETDIET: Dietary retinol consumed (mcg per day) BETAPLAS: Plasma beta-carotene (ng/ml) RETPLAS: Plasma Retinol (ng/ml)