Solution to Series 3

Transcription

1 Prof. Nicolai Meinshausen Regression FS 2016 Solution to Series 3 1. a) The general least-squares regression estimator is given as Using the model equation, we get in this case ( ) X T x X (1)T x (1) x (1)T x (2) x (2)T x (1) x (2)T x (2) Hence we have Plugging this into (1), we get i.e. β (X T X) 1 X T y. (1) (X T X) ρ 2 β 1 1 ρ 2 ( ) 1 ρ. ρ 1 ( ) x (1)T y ρx (2)T y x (2)T y ρx (1)T, y b) Pluggin in the model equation y Xβ + ɛ into (2) gives ( ) 1 ρ. ρ 1 β ρ 2 ( x (1) ρx (2)) T y. (2) β ρ 2 ( x (1) ρx (2)) T (Xβ + ɛ) The only random part is ɛ, so ] [ 1 ( Var [ β1 Var 1 ρ 2 x (1) ρx (2)) ] T ɛ [ n ] 1 (1 ρ 2 ) 2 Var ( ) x (1) i ρx (2) i ɛ i 1 (1 ρ 2 ) 2 n i1 i1 ( ) 2 x (1) i ρx (2) i Var[ɛi ] (since ɛ i are indep.) σ 2 ( (1 ρ 2 ) 2 x (1)T x (1) 2ρx (1)T x (2) + ρ 2 x (2)T x (2)) σ 2 (1 ρ 2 ) 2 (1 2ρ2 + ρ 2 ) σ 2 (1 ρ 2 ). Hence, for ρ close to 1 (high correlation), the variance of the least-squares estimator is large. 2. a) In the script, Chapter 1.3.3, we find In Chapter we then find n ˆσ 2 i1 (y i ŷ i ) 2. n p ˆσ 2 α ˆσ 2 ((X t X) 1 ) 11 and ˆσ 2 β ˆσ 2 ((X t X) 1 ) 22. These expressions are evaluated with the code below. The results are ˆσ , ˆσ α and ˆσ β

2 2 ## R Code library(car) data(sahlins) str(sahlins) y <- Sahlins$acres x <- Sahlins$consumers alpha.hat < beta.hat < yhat <- alpha.hat+x*beta.hat resid <- y-yhat #a) sigmahat <- sqrt(sum(resid^2)/18) xdesign <- cbind(1,x) mat <- solve(t(xdesign)%*%xdesign) sigmalpha <- sigmahat*sqrt(mat[1,1]) sigmabeta <- sigmahat*sqrt(mat[2,2]) b) The confidence intervals are calculate according to Chapter So we need ˆσ α, ˆσ β and the 97.5%-quantile of the t n p -distribution. The latter can be obtained with qua <- qt(0.975,18). We then get V I 0.95 (α) [ˆα qua ˆσ α, ˆα + qua ˆσ α ] [0.3915, ] and similarly for ˆβ V I 0.95 (β) [ ˆβ qua ˆσ β, ˆβ + qua ˆσ β ] [ , ]. For the t-statistics of the nullhypotheses α 0 and β 0 we obtain ˆα/ˆσ α and ˆβ/ˆσ β The distribution function of the t-distribution is obtained with pt(). The p-values are then calculated according to the following: ## R Code pvalalpha <- 2*(1-pt(2.9368,18)) # gives pvalbeta <- 2*(1-pt(1.7197,18)) # gives We want to check now these results with the functions lm() and confint. We obtain > lmobj <- lm(y~x) > summary(lmobj) Call: lm(formula y ~ x) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) ** x Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: on 18 degrees of freedom Multiple R-squared: ,Adjusted R-squared: F-statistic: on 1 and 18 DF, p-value: > confint(lmobj,level0.95)

3 3 2.5 % 97.5 % (Intercept) x c) We cannot conclude that the slope is different from zero. The corresponding p-value is The p-value of the intercept is meaning that we can reject the nullhypothesis α 0 on a 1% level and conclude that the intercept is different from zero. 3. a) The scatterplot is generated with the code below. The desired contries are also obtained with this code. Scatterplot LifeExp People.per.TV People.per.Dr ## R Code data <- read.table(" str(data) 'data.frame': 40 obs. of 5 variables: $ LifeExp : num $ People.per.TV : num $ People.per.Dr : int $ LifeExp.Male : int $ LifeExp.Female: int plot(data[,c(1,2,3)]) # we see above that variables 1,2,3 # are relevant countries <- row.names(data) # vector with all countries indizele <- order(data[,1],decreasingtrue) # gives indices of expectation of life # ordered by magnitude countries[indizele[1:3]] # 3 contries with the highes exp. of life "Japan" "Italy" "Spain" Analogeous, all the other desired countries can be obtained (Burma, Ethopia, Bangladesh and Ethopia, Tanzania, Zaire). b) We see that the second column of the data set has missing values. The rows can be determined with is.na(), which() and which(is.na(data[,2])). They can then be removed with datanew <- data[-c(32,40),]. The remaining R code is given in the following tv <- datanew$people.per.tv le <- datanew$lifeexp dr <- datanew$people.per.dr # to simplify lmobj <- lm(le~log2(tv)+log2(dr)) # gives fit summary(lmobj) Call: lm(formula le ~ log2(tv) + log2(dr)) Residuals:

4 4 Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) < 2e-16 *** log2(tv) e-05 *** log2(dr) ** Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: on 35 degrees of freedom Multiple R-squared: ,Adjusted R-squared: F-statistic: 64.6 on 2 and 35 DF, p-value: 1.788e-12 So we obtain for the intercept 90.62, for β 1 and for β 2 for the model le α + β 1 log 2 (tv) + β 2 log 2 (dr) + ɛ. So doubling the number of people per TV leads to a decrease in the expectation of life of 2.02 years. c) No. A correlation is not necessarily a causal dependence. It is possible that the correlation comes from confounding variables. In our case, this means that there is a variable with influence on the two other variables under consideration. Correlation doesn t imply causation! However, we can predict the expectation of life with the number of people per TV. Perhaps, there is no causal connection, but we can use this for prediction anyway. 4. a) R result: > pairs(basisch) We see that there is a negative linear correlation between h.quad and ph. The ph-values are all above 7 and thus indicate basic soils. The influence of l.sar is not clear. There are two outliers. b) R result: > summary(lm(h.quad ~ ph + l.sar, databasisch)) Call: lm(formula h.quad ~ ph + l.sar, data basisch) Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e-14 *** ph e-11 *** l.sar Residual standard error: on 120 degrees of freedom Multiple R-Squared: , Adjusted R-squared: F-statistic: on 2 and 120 degrees of freedom, p-value: 0 c) On the 5%-level we cannot reject H 0 : β 2 0 (p-value0.1153). So the variable l.sar is not useful.

5 ph l.sar height h.quad d) R result: > new.pt <- data.frame(ph8, l.sar1) > conf.interval <- predict(basisch.lm,new.pt,int"confidence",level.95) fit lwr upr [1,] > pred.interval <- predict(basisch.lm,new.pt,int"prediction",level.95) fit lwr upr [1,] fitfitted value, lwrlower bound of intervall, uprupper bound of interval. The prediction interval for the height can be obtained by solving for y 0 and taking the square root (see Chapter (f)). For the confidence interval of the height we cannot do that (see Chapter (e)). A regression with height cannot be done since some model assumptions are not fulfilled (hints: Q-Q-plot, Tukey-Anscombe-plot). In practice, sometimes the square-roots are taken anyway since E[ y 0 ] and E[y 0 ] are equal to first order. Or the interval is found with simulations. 5. a) R results: > wdi.select <- wdi2005[,c(780, 515, 1196, 455)] > wdi.select[,1] <- log(wdi.select[,1]) > fit <- lm(wdi.select[,1] ~wdi.select[,2] + wdi.select[,3] + wdi.select[,4] ) > summary(fit) Call: lm(formula wdi.select[, 1] ~ wdi.select[, 2] + wdi.select[, 3] + wdi.select[, 4]) Residuals: Min 1Q Median 3Q Max

6 6 Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) 5.002e e < 2e-16 *** wdi.select[, 2] e e e-05 *** wdi.select[, 3] e e ** wdi.select[, 4] e e e-08 *** Signif. codes: 0 *** ** 0.01 * Residual standard error: on 63 degrees of freedom (181 observations deleted due to missingness) Multiple R-squared: ,Adjusted R-squared: F-statistic: on 3 and 63 DF, p-value: < 2.2e-16 b) R code and output: > confint(fit) 2.5 % 97.5 % (Intercept) e e+00 wdi.select[, 2] e e-02 wdi.select[, 3] e e-03 wdi.select[, 4] e e-05 The confidence interval is ( , ) The confidence interval does not contain zero, so the effect of Social contributions is significant at the 5% level. c) We can calulate the bootstrap confidence interval using the function boot and boot.ci as follows. require(boot) conf.int <- function(u,i){ bs <- u[i,] fit <- lm(bs[,1] ~bs[,2]+ bs[,3] + bs[,4] ) fit$coefficients[3] } set.seed(2) bs <- boot(data wdi.select, statistic conf.int, R1000) boot.ci(bs,type c("basic"),replacet) R output: BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS Based on 1000 bootstrap replicates CALL : boot.ci(boot.out bs, type c("basic"), replace T) Intervals : Level Basic 95% ( , ) Calculations and Intervals on Original Scale