STAT420 Midterm Exam. University of Illinois Urbana-Champaign October 19 (Friday), :00 4:15p. SOLUTIONS (Yellow)

Transcription

1 STAT40 Midterm Exam University of Illinois Urbana-Champaign October 19 (Friday), 018 3:00 4:15p SOLUTIONS (Yellow) Question 1 (15 points) (10 points) 3 (50 points) extra ( points) Total (77 points) Points Question 1: Facts [15 pts] There are 5 problems with 3 points each. Choose the most ONE from the statements for each problem. [ ] X (1) Suppose that N Y following statements: ([ ] µx, µ y (a) X Y N (µ x µ y, σ x + σ y) [ σ x ρσ x σ y ρσ x σ y σ y ]). Choose the most appropriate ONE from the (b) If ρ 0, any linear combination of X, Y cannot be a Gaussian random variable (c) Cor(X, Y ) = ρσ x σ y (d) If ρ = 0, X (Y =y) = X () In linear regression, we denote the fitted values as ŷ = X β, and 1 n as a column vector with all elements being 1. Choose the most appropriate ONE from the following statements: (a) 1 T n (y ŷ) = 0. (b) 1 T n (y ȳ) = 0. (c) 1 T n (ŷ ȳ) = 0. (d) All the above. (3) Judge the statement: We reject the null hypothesis H 0 : β 0 = β 1 = 0 if we can reject any of the two hypotheses H 0 : β i = 0 for i = 0, 1. (a) TRUE. (b) FALSE. 1

2 (b). (4) Judge the statement: Given a data set X, y, the 95% confidence interval of β 0 is calculated as (l(x, y), u(x, y)). This means that such interval (l(x, y), u(x, y)) has the probability 95% to contain the estimates β(x r, y r ) for repeated experiments r = 1,, R. (a) TRUE. (b) FALSE. (b). (5) Choose the factor(s) that affect the length of confidence interval for β i from the following statements (only ONE answer): (a) The normality assumption of the noise. (b) The constant variance assumption of the noise. (c) The value of β i. (d) Both (a) and (b). Question : Proof of understanding [10 pts] There are problems with 5 points each. (1) In the simple linear regression, let ŷ i = ˆβ 0 + ˆβ 1 x i be the fitted values, and r xy := be the sample correlation between x and y. Prove the following R := SSR SST = r xy Hint: Show SSR = (ŷ i ȳ) = ˆβ 1s x(n 1) first. (xi x)(yi ȳ) n (xi x) (yi ȳ) Proof. First, we show SSR = (ŷ i ȳ) = ˆβ 1s x(n 1). Substituting ŷ i into SSR, we have SSR = ( ˆβ 0 + ˆβ 1 x i ȳ) With the property that regression line passes through the centroid of data, we have ȳ = ˆβ 0 + ˆβ 1 x. Substituting ȳ in yields SSR = ( ˆβ 1 x i ˆβ 1 x) = β1 Note that SST = s y(n 1). With the solution of ˆβ 1 = r xy s y s x (x i x) = ˆβ 1s x(n 1) R = SSR SST = ˆβ 1s x(n 1) s = rxy y(n 1) we finally have

3 Thus it completes the proof. () What is the geometric interpretation of multiple linear regression (MLR)? The geometric interpretation of MLR means that the optimal solution (OLS) ŷ = X β can be obtained by projecting (by a hat matrix H) the original data vector y to the column space of the design matrix X of predictors, that is, ŷ = Hy. The projection requires the error e := y ŷ = (I H)y to the orthogonal to the column space col(x), i.e. X T e = 0. Such orthogonality condition corresponds the normal equation for deriving OLS by optimization: X T y = X T ŷ = X T X β The SSE can be viewed as the squared -norm of the error vector: SSE = e = y T (I H)y, which measures the discrepancy between the data vector and its fitted value in the linear model. Question 3: Calculation [50 pts] There are 10 sub-problems with 5 points each. Suppose a researcher wants to perform a simple linear regression on the samples he collected. However, the original data was lost, and he has only the access to some summary statistics n = 30, x = , ȳ = , x i y i = , x i = , yi = C o e f f i c i e n t s : Based on the provided information, fill in the blanks in the following report of linear regression. Estimate Std. Error t value Pr( > t ) 3 ( I n t e r c e p t ) (a) x (a) (d) (e) (f) **(or 0.00) 5 6 S i g n i f. codes : R e s i d u a l standard e r r o r : (c) 13.8 on (c) 8 d e g r e e s o f freedom 9 M u l t i p l e R squared : (b) 0.30, Adjusted R squared : F s t a t i s t i c : (g) 1.11 on (h) 1 and (h) 8 DF, p value : (a) (h) 5 points each. Write down steps for partial credits. (i) (5 points) For a new predictor x new =, predict the new response y new. (j) (5 points) Calculate the 95% CI for y new. ˆβ 1 = (x i x)(y i ȳ) n (x = x iy i n xȳ i x) n x i = n x = ˆβ 0 = ȳ ˆβ 1 x = = (b) Based on the what proved in -(1) we have SSR = ˆβ 1( x i n x ) = ( ) =, By the definition of coefficient determination R = SSR SST = SSR y i =, nȳ =

4 (c) We need to calculate the square root of MSE. We calculate SSE based on R and therefore ˆσ = SSE SST(1 MSE = n = R ) ( ) ( ) = = n 30 And the degree of freedom of residuals is n = 8. (d) The standard error of ˆβ 1 can be calculated based on the definition se( ˆβ 1 ) = ˆσ (n 1)sx = / = (e) Then the t-statistic for individual test β 1 = 0 is t = ˆβ 1 se( ˆβ 1 ) = = (f) And we know that the t-statistic follows a t(n ) distribution. Therefore based on the t-table p = P[ t > t ] = P[ t > ] < P[ t > 3.408] = 0.00 (g) We calculate F -statistic for the joint test β 1 = = β p = 0 using R based on the formula F = R (n ) = (30 ) = R (h) The degrees of freedom of the F -statistics are 1 and n = 8 respectively. (i) We predict the new response y new with µ new = ˆβ 0 + ˆβ 1 x new = ( ) = (j) The 95% CI for y new can be calculated with the formula 1 (xnew x) l(ynew) = µnew + t 0.05/,n ˆσ n (n 1)s = ( ) 1 + 1/30 + x = (xnew x) u(ynew) = µnew t 0.05/,n ˆσ n (n 1)s = ( ) 1 + 1/30 + x = Therefore the 95% CI for y new is ( , ). (Extra-credit points) Do you have any comments (1 point) and suggestions (1 point) about this course? Any comments and suggestions are welcome! 4

5 5