Stat 4510/7510 Homework 7

Transcription

1 Stat 4510/7510 Due: 1/10. Stat 4510/7510 Homework 7 1. Instructions: Please list your name and student number clearly. In order to receive credit for a problem, your solution must show sufficient details so that the grader can determine how you obtained your answer. Concrete is the most important material in civil engineering. Concrete compressive strength is a highly nonlinear function of age and ingredients. The dataset concrete.csv contains the following information: Cement (component 1) (kg per cubic meter) Blast Furnace Slag (component 2) (kg per cubic meter) Fly Ash (component 3) (kg per cubic meter) Water (component 4) (kg per cubic meter) Superplasticizer (component 5) (kg per cubic meter) Coarse Aggregate (component 6) (kg per cubic meter) Fine Aggregate (component 7) (kg per cubic meter) Age (day) Concrete compressive strength (MPa, megapascals) Use these data to answer the following questions. (a). Use the poly() function fit polynomial regressions for predicting Compressive Strength using Age. Plot the data and add these polynomial fits ranging from degree 1 to 7. Be sure to include a legend (see?legend). Additionally, display a table of the RSS for each degree. 1

2 Stat 4510/7510 Due: 2/10 concrete=read.csv("concrete.csv") concrete=concrete[,-1] plot(concrete$age,concrete$concretecompressivestrength, xlab="age",ylab="compressive Strength") for(i in 1:7){ fit=lm(concretecompressivestrength~poly(age,degree=i,raw=true),data=concrete) points(seq(0,400,length.out=1000), predict.lm(fit,newdata=list(age=seq(0,400,length.out=1000))), col=i+1,type="l")} legend("bottomright",col=2:8, legend=c("degree 1", "degree 2","degree 3","degree 4", "degree 5","degree 6","degree 7"),lty=1) Compressive Strength degree 1 degree 2 degree 3 degree 4 degree 5 degree 6 degree Age 2

3 Stat 4510/7510 Due: 3/10 RSS=NULL for(i in 1:7){ fit=lm(concretecompressivestrength~poly(age,degree=i,raw=true),data=concrete) RSS[i]=sum(fit$resid^2)} Polynomial RSS (b). Based on your plot in (a), which polynomial degree do you think fits the trends in the data best? The degree 3 polynomial appears to be the best fitted trend. The higher order polynomials appear to overfit the data. (c). Now use 10-fold cross validation to select the best degree polynomial. Which degree was chosen? Does it match the conclusion you made in (b)? You might consider making a plot to justify your decision. K=10 cv.error=matrix(ncol=7,nrow=k) set.seed(1) folds = sample(1:k,nrow(concrete),replace=t) for(k in 1:K){ CV.train = concrete[folds!= k,] CV.test = concrete[folds == k,] for(i in 1:7){ cv.fit=lm(concretecompressivestrength~poly(age,degree=i,raw=true), data=cv.train) cv.pred=predict.lm(cv.fit,newdata=cv.test) cv.error[k,i]=mean((cv.pred-cv.test$concretecompressivestrength)^2) }} apply(cv.error,2,mean) [1] In terms of out-of-sample prediction, the model with polynomial degree of 4 or more have nearly the same predictive error. Therefore, we suggest the model with degree 4 as the models with degree 5 or more are likely overfitting. 3

4 Stat 4510/7510 Due: 4/10 (d). Use the bs() function to fit a regression spline to predict Compressive Strength using Age. Report the output for the fit using 5 degrees of freedom (which results in 1 knot!). Plot the resulting fit. Where is the knot placed? library(splines) fit=lm(concretecompressivestrength~bs(age,df=5),data=concrete) summary(fit) Call: lm(formula = Concretecompressivestrength ~ bs(age, df = 5), data = concrete) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) < 2e-16 *** bs(age, df = 5) ** bs(age, df = 5) < 2e-16 *** bs(age, df = 5) < 2e-16 *** bs(age, df = 5) bs(age, df = 5) e-15 *** --- Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: 13.7 on 1024 degrees of freedom Multiple R-squared: ,Adjusted R-squared: F-statistic: on 5 and 1024 DF, p-value: < 2.2e-16 plot(concrete$age,concrete$concretecompressivestrength, xlab="age",ylab="compressive Strength") points(seq(0,400,length.out=1000), predict.lm(fit,newdata=list(age=seq(1,365,length.out=1000))), col=2,type="l") 4

5 Stat 4510/7510 Due: 5/10 Compressive Strength Age #attr(bs(concrete Age,df=5,intercept=T),"knots") For 5 degrees of freedom, there is only 1 knot, which is at the median value of Age, 28. (e). (7510*) Perform 10-fold cross-validation in order to select the best single-knot model. Note that the knot locations can be anywhere between (but not equal to) the miniumum and maximum Age values in the data. Make a plot of the errors verus knot location and describe your results. K=10 cv.error=matrix(ncol=365,nrow=k) set.seed(1) folds = sample(1:k,nrow(concrete),replace=t) 5

6 Stat 4510/7510 Due: 6/10 for(k in 1:K){ CV.train = concrete[folds!= k,] CV.test = concrete[folds == k,] for(i in 2:364){ cv.fit=lm(concretecompressivestrength~bs(age,knots=i),data=cv.train) cv.pred=predict.lm(cv.fit,newdata=cv.test) cv.error[k,i]=mean((cv.pred-cv.test$concretecompressivestrength)^2) }} plot(1:365,apply(cv.error,2,mean)) apply(cv.error, 2, mean) :365 The best single-knot location chosen by 10-fold cross validation is Age=159. (f). Fit a smoothing spline and use cross validation to select λ. What is the chosen degrees of freedom? Plot the fit along with the fit from part (d). How do they compare? 6

7 Stat 4510/7510 Due: 7/10 fit.ss=smooth.spline(x=concrete$age,y=concrete$concretecompressivestrength, cv=true) fit.ss$df [1] fit.bs=lm(concretecompressivestrength~bs(age,knots=159),data=concrete) plot(concrete$age,concrete$concretecompressivestrength) lines(fit.ss,col=2) points(seq(0,400,length.out=1000), predict.lm(fit.bs,newdata=list(age=seq(1,365,length.out=1000))), col=4,type="l") legend("bottomright",col=c(2,4), legend=c("smoothing spline","regression spline"),lty=1) 7

8 Stat 4510/7510 Due: 8/10 concrete$concretecompressivestrength smoothing spline regression spline concrete$age (g). Split the data into a 90% training set and a 10% test set. Be sure to set a seed of 1 for consistency of results. set.seed(1) train.set = sample(1:nrow(concrete),.9*nrow(concrete),replace=false) concrete.train=concrete[train.set,] concrete.test=concrete[-train.set,] (h). Fit a linear regression on the training data using Compressive Strength as the response and all other variables as predictors. Which variables are significant? 8

9 Stat 4510/7510 Due: 9/10 fit.lm=lm(concretecompressivestrength~.,data=concrete.train) summary(fit.lm) Call: lm(formula = Concretecompressivestrength ~., data = concrete.train) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) Cement < 2e-16 *** BlastFurnaceSlag < 2e-16 *** FlyAsh e-10 *** Water *** Superplasticizer ** CoarseAggregate FineAggregate Age < 2e-16 *** --- Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: on 918 degrees of freedom Multiple R-squared: ,Adjusted R-squared: F-statistic: on 8 and 918 DF, p-value: < 2.2e-16 All the of the variables are significant at the α = 0.1 level. (i). Predict Compressive Strength on the test set, what is the test MSE? pred.lm=predict.lm(fit.lm,newdata=concrete.test) mean((concrete.test$concretecompressivestrength-pred.lm)^2) [1] The test MSE is (j). Fit a GAM on the training data using Compressive Strength as the response and all variables except Age as linear predictors. For the Age variable, investigate different degrees of freedom for the smoothing splines. Is there evidence of a non-linear relationship? 9

10 Stat 4510/7510 Due: 10/10 library(gam) Loading required package: foreach Loaded gam 1.16 fit.gam1=gam(concretecompressivestrength~.,data=concrete.train) fit.gam5=gam(concretecompressivestrength~cement+blastfurnaceslag+ FlyAsh+Water+Superplasticizer+CoarseAggregate+ FineAggregate+s(Age,5),data=concrete.train) anova(fit.gam1,fit.gam5) Analysis of Deviance Table Model 1: Concretecompressivestrength ~ Cement + BlastFurnaceSlag + FlyAsh + Water + Superplasticizer + CoarseAggregate + FineAggregate + Age Model 2: Concretecompressivestrength ~ Cement + BlastFurnaceSlag + FlyAsh + Water + Superplasticizer + CoarseAggregate + FineAggregate + s(age, 5) Resid. Df Resid. Dev Df Deviance Pr(>Chi) < 2.2e-16 *** --- Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Yes. The p-value comparing the linear model with the non-linear GAM model with 5 degrees of freedom for Age is significant. Therefore, there is strong evidence of a non-linear relationship. (k). Predict Compressive Strength on the test set for different degrees of freedom of the smoothing spline for Age. What degrees of freedom has the lowest test MSE? Is it better than that of the linear regression from part (i)? ss.error=na for(i in 1:20){ fit.gam=gam(concretecompressivestrength~cement+blastfurnaceslag+ FlyAsh+Water+Superplasticizer+CoarseAggregate+ FineAggregate+s(Age,i),data=concrete.train) pred.gam=predict(fit.gam,newdata=concrete.test) ss.error[i]=mean((concrete.test$concretecompressivestrength-pred.gam)^2) } The test MSE from the GAM model with degrees of freedom 7 on the Age variable is 42.3, which is much lower than that from the linear model. Therefore, we conclude that the GAM is better than the linear model. 10