Stat 342 Homework Fall 2014

Transcription

1 Stat 34 Homework Fall 014 Assigmet 1 Due 9/5/14 Sectio 1.1 of the Course Outlie 1. Cosider a probability model for the radom pair yx, with joit desity 1 y exp for 0 x 1 ad y 0 f y, x x x 0 otherwise a) Fid the margial distributio of x ad the coditioal desity of y x. b) Fid both the optimal squared error loss ad optimal absolute error loss predictors of y based o x. c) Evaluate the risk of the optimal squared error loss predictor.. Suppose that the radom pair yx, has a joit distributio o 0,1 0, specified as follows: P y that may be P y 1.7 ad 0.3 coditioal o the value of y, the variable x is Expoetial with mea y1 A "desity" for yx, (that oe adds over x ad itegrates over y ) is the f 1 x.7i y 0 exp x.3i y 1 exp if y, x 0,1 0, y, x 0 otherwise a) Evaluate Px 1. b) Fid the coditioal probability that 1 c) Fid the 0-1 loss optimal predictor of y based o x. d) Evaluate the risk of your predictor i c). y give the value of x, P y 1 x. 1

2 3. Below is a table givig a joit pmf for the radom pair yx., x 1 x x 3 y y y a) Fid the SEL optimal predictor of y based o x. (You eed to evaluate all of 1,, ad 3 ˆ ˆ ˆ y opt y opt y opt.) Evaluate the risk of this predictor. b) Fid the 0-1 loss predictor of y based o x. (Agai, you eed to evaluate all of 1,, ad 3 ˆ ˆ ˆ y opt y opt y opt.) Evaluate the risk of this predictor. Sectio 1. of the Course Outlie 4. Cosider the toy statistical model where the etries of x x x x are iid N,1 a) Write out the likelihood fuctio for iferece about.,,, 1. b) Fid the maximum likelihood estimator of (the fuctio of x that maximizes the MLE likelihood fuctio, ˆ x ). (Operatioally, you will probably fid it easiest to maximize the logarithm of the likelihood fuctio.) c) Fid the SEL risk fuctio R for the maximum likelihood estimator of. MLE d) A secod possible estimator of is.5ˆ x. Fid the risk fuctio of this secod estimator of ad compare it to your aswer to c). Is oe of the two estimators always (for all ) better tha the other? If ot, for which values of (the ukow) is this secod estimator preferable? 5. Suppose that x Bi 5, p. a) Plot o the same set of axes, the 6 differet possible log-likelihood fuctios

3 x l p 1 5 x p (the 5 x is icosequetial). b) I light of the plot i a) for the x 0 ad x 5 cases ad some calculus, what is the maximum likelihood estimator of p, MLE ˆp c) Two more possible estimators of p are x? What is the risk fuctio of this estimator? 1 x 1 pˆx 5 ad pˆ3x Fid the SEL risk fuctios for these estimators ad plot them together with the risk fuctio from b) o the same set of axes. Is ay oe of these uiformly/always smaller tha the others? 6. Cosider the two biomial pmf's defied o 0,1,,5 decisio fuctio 5 5 f x f x x x ad ad decisio about 0,1 x 5 x x 5 x Ix.5 a x (You must fid the two values R0 ad R 1.). Fid the 0-1 loss risk fuctio for the Sectio 1.3 of the Course Outlie 7. For x N,1 ad a N 0, 10 prior distributio for, what is the SEL Bayes optimal estimator of. Hit: What is the form of the coditioal distributio of give x? 8. For x Bi 5, p ad a Uiform 0,1 prior distributio for p, what is the SEL Bayes optimal estimator of p? Hit: What is the form of the coditioal distributio of p give x? 3

4 Assigmet Due 9/19/14 Sectio.1 of the Course Outlie 9. I class we derived the distributio of U U U 1 1 for U1 ad U iid with pmf u 0 1 f u a) Fid the distributio of U U U U U U U U U radom variables U1, U, U3, U 4 with this margial distributio. for iid b) Compare the mea ad variace for U 1 to the meas ad variaces for both the ad 4 versios of U. 10. Suppose that U1 ad U are iid discrete radom variables, each uiformly distributed o the set 0,1,,3, 4,5,6. Fid the pmf of the radom variable V UU 1. (Determie the set of possible values ad correspodig probabilities ad put them ito a table specifyig this pmf.) 11. Suppose that UV, is uiformly distributed o the uit square. That is, suppose that the pair has joit pdf Cosider the radom variable T U V., 0 1 ad 0 1 f u v I u v a) Fid the cdf of T. (You ca fid differet "formulas" for Ft depedig upo how t compares to the values 0,1, ad. Oce you idetify the set of, equal to t, simple geometry ca be used to fid the value of Ft.) b) Use a) ad fid the pdf of T. uv with total less tha or 1. Suppose that U ad V are iid Exp1, i.e. have joit pdf 4

5 f uv, I0 u1 ad 0 v1 expu v Fid the cdf ad pdf for the product of U ad V. (The first will require computatio of a double itegral over a appropriate uv, regio.) 13. Fid a fuctio h so that for U U0,1 the radom variable hu has pdf 3 f v v I v Cosider the discrete distributio of Problem 9. Fid a fuctio h so that for V U0,1 the radom variable hv has that simple distributio. (There are may solutios here. h should have oly 3 possible values, ad it suffices to break its domai ito 3 itervals correspodig to those possible values.) 15. Fid the momet geeratig fuctio of the U0, distributio. 16. Argue usig momet geeratig fuctios that the sum of 5 iid Exp 1 radom variables has a gamma distributio. x 1,,, are iid N x, 17. Suppose that the etries of x x x y y1, y,, ym are iid N y,, ad x ad sample mea ad variace of the Defie x i 's ad, the etries of y are idepedet. Let x ad s x are the y ad s y are the sample mea ad variace of the y 's. i s pooled 1 x 1 1m1 s m s y a) Argue carefully that there is a multiple of multiplier ad what are the degrees of freedom?) s pooled that has a distributio. (What is the 5

6 b) Idetify a fuctio of the radom variables x y s has a t distributio. What are appropriate degrees of freedom? ad pooled ad the differece x y that c) Suppose that x 7, s 1, 10, y 5, s.8, ad m 1. Use your aswer to b) ad fid 90% two-sided cofidece limits for x y. x y Assigmet 3 Due 9/6/ I Example 18 the claim is made that the sum of two idepedet Poisso radom variables is Poisso. Prove this. Sectio. of the Course Outlie 19. Use the fact that for large a Poisso distributio with mea is approximately ormal ad fid limits based o a Poisso variable X that (at least for large ) fuctio as approximate 90% cofidece limits for. What iterval is produced if a Poisso observatio is X 500? 0. Suppose that x1, x, x3, are iid U0,. Cosider the radom variable m max x, x,, x 1 the largest of the first of these observatios. Notice that m t if ad oly if all of the first observatios are less tha or equal to t. (So, for t the Pm t Px t.) Make from m the radom variable a) Fid for y 0 the limit as of What is the approximate distributio of y? y m P y y 1 y b) What is a approximate distributio for y /? (What is the limit of P t?) t for positive 6

7 c) Use your aswer to b) ad fid a large approximately 95% upper cofidece boud for that is a multiple of m. 1. As i the previous problem, suppose that x1, x, x3, are iid U0,. For x the sample mea of the first of these variables, fid (usig the CLT ad a delta method argumet) a approximate distributio for x. Sectio.3 of the Course Outlie. Use R simulatios to a) Fid approximate aswers for all of Problem 11. b) Fid approximate aswers for all of Problem 1. (I both cases use at least 10,000 simulated values of the variables i questio.) Assigmet 4 Due 10/17/14 3. Below is some BUGS code that ca be used to do the simulatios i Problem a). You ca read about BUGS sytax i the user maual available through the help meu i either WiBUGS or OpeBUGS. Ru this code i either WiBUGS or OpeBUGS with at least 100,000 iteratios ad get estimated desities for UV,, ad T ad estimated values for the cdf of T at the values t 0,.,.4,,1.8,.0. model { U~duif(0,1) V~duif(0,1) T<-U+V I.<-step(.-T) I.4<-step(.4-T) I.6<-step(.6-T) I.8<-step(.8-T) I1.0<-step(1.0-T) I1.<-step(1.-T) I1.4<-step(1.4-T) I1.6<-step(1.6-T) I1.8<-step(1.8-T) I.0<-step(.0-T) } 7

8 4. Modify the code i Problem 3 ad get estimated desities for UV,, ad b) usig either WiBUGS or OpeBUGS with at least 100,000 iteratios. 5. Suppose that x Bi 5, p ad a priori U0,1 a) What is the posterior distributio of p x? p. T for Problem b) Use WiBUGS or OpeBUGS with at least 100,000 iteratios to approximate the posterior desity ad the posterior mea of p for the observatio x. Also fid a correspodig 95% credible iterval for p. Here is some relevat BUGS code: model { X~dbi(p,5) p~duif(0,1) } list(x=) Sectio.4 of the Course Outlie 6. Below is some R code for implemetig a o-parametric bootstrap for studyig the distributio of T the sample media of observatios (supposig that oe is makig iid draws from some distributio). Apply this code to the sample of 0 umbers 0.18,0.15,0.14,0.44,.89,1.3,0.54,0.96,0.15,1.39,0.76,1.4,4.4,1.05,1.04,1.88,0.65,0.34,0.59,.36 (that actually comprise a rouded sample of size 0 geerated from a Exp 1 distributio. Use that code to estimate E T0, VarT 0, ad the first ad 9 th deciles of the distributio of T 0. #Set the seed o the radom umber geerator to get the same results for repeat rus set.seed(0) #Create some data iput "data" <-0 observed<-c(roud(rexp(),digits=)) observed #Ready a matrix with B rows ad colums for B bootstrap samples of size B< Boot<-matrix(c(rep(0,B*)),row=B,byrow=TRUE) #Create the matrix of bootstrapped samples 8

9 for (i i 1:B) { Boot[i,]<-sample(observed,replace=TRUE,) } #Ready a vector for bootstrapped values of T Tstar<-c(rep(0,B)) #Make a vector of bootstrapped values of T for (i i 1:B) { Tstar[i]<-media(Boot[i,]) } #Get some summaries of the bootstrap distrbutio of Tstar summary(tstar) hist(tstar) quatile(tstar,probs=seq(0,1,.05)) 7. The mea of the observatios listed i Problem 6 is x 1.1. If oe assumes that the values i Problem 6 are a rouded sample from some expoetial distributio, the expoetial distributio with mea 1.1 is a plausible choice. (If oe ackowledges the roudig, 1.1 is ot ecessarily quite the MLE of the expoetial mea, but we'll igore this fie poit.) Thus a parametric bootstrap replaces samplig with replacemet from the observed values with samplig from a rouded versio of the expoetial distributio with mea 1.1 (ad thus "rate" 1/1.1 ). I the code above, oe ca simply replace sample(observed,replace=true,) with roud(rexp(,rate=1/1.1),) a) Redo Problem 6 usig this parametric bootstrap. b) Replace 1.1 above with 1.00 ad thereby use simulatio to fid the true values of the characteristics of the distributio of the sample media of 0 rouded expoetial variables that are beig estimated i Problem 6 ad above i part b). Sectio 3.1 of the Course Outlie 8. Suppose that x1, x,, x for the parameter vector,. are iid Beta,. Idetify a two-dimesioal sufficiet statistic 9

10 9. Suppose that x1, x,, x are iid N,. a) Argue carefully that the statistic T is sufficiet for the parameter vector,. b) The show that x, s sufficiet. x, i i1 i1 x i S is a 1-1 oto fuctio of T (is equivalet to T ) ad is thus also 30. For x1, x,, x Poisso argue carefully that T iid x xi is miimal sufficiet for. (Showig sufficiecy is easy. Showig miimal sufficiecy is somewhat harder. You ca argue that if Tx Tx the,1,1 x x as fuctios of. That meas that you have to show that for some particular the values of the likelihood ratios differ.) i1 31. Cosider the statistical model for x with 1,, 3 ad pmfs f x give i the table below. Idetify a miimal sufficiet statistic for this model, say T. (Give values of T for each possible value of x.) x Sectio 3. of the Course Outlie 3. Fid the Fisher iformatio i a sigle Beroulli p radom variable about the parameter p i two differet ways. First use the defiitio of Fisher iformatio, the use Propositio of the typed outlie. 10

11 33. Fid the Fisher iformatio i a sigle Expoetial variable with mea about the parameter. First use the defiitio of Fisher iformatio, the use Propositio of the typed outlie. 34. Suppose that ad with mea. x y are idepedet radom variables, x Poisso a) What is the Fisher iformatio i the pair x, y about? ad y Expoetial b) If Tx, y is ubiased for, what is the miimum possible value for Var Tx, y For a fixed umber 0,1 let ˆ, 1 c) Show that each ˆ is ubiased for. x y x y.? d) Plot the SEL risk fuctios for both ˆ ˆ 1 ad over the rage 0 4. O the same set of 3 axes plot, the Cramèr-Rao lower boud for the variace of a ubiased estimator of (this is a fuctio of ). For which values of do the two liear combiatios of x ad y achieve this lower boud? Assigmet 5 (Not to be Collected, but to be Covered o Exam ) 35. Argue carefully that for x1, x,, x Beroulli p variables, pˆ x is ubiased for p. The show that the variace of this estimator achieves the Cramèr-Rao lower boud for the variace of a ubiased estimator of p for all values of p. iid 36. Argue carefully that for x1, x,, x iid Expoetial variables with mea, x is ubiased for. The show that the variace of this estimator achieves the Cramèr-Rao lower boud for the variace of a ubiased estimator of for all values of. Sectio 3.3 of the Course Outlie 11

12 37. Cosider the cotext of Problem 35. a) Argue that pˆ x is the maximum likelihood estimator of p. b) Argue two ways that for ay 0 P ˆ p 0 p p0 0 as First use Propositio 7 ad the use Theorem 13 from the typed outlie. c) Argue two ways that for large, valid but uusable approximately 95% cofidece limits for p are pˆ 1.96 p 1 p First use Propositio 8 ad the use Theorem 14 from the typed outlie. d) Use Corollary 9 ad fid a valid usable replacemet for the limits of part c). (Use the "expected iformatio" fix.) e) Use Corollary 30 ad fid a valid usable replacemet for the limits of part c). (Use the "observed iformatio" fix.) 38. Suppose that x1, x,, x Poisso. are iid a) Argue that ˆ x is the maximum likelihood estimator of. b) Argue two ways that for ay 0 P ˆ as First use Propositio 7 ad the use Theorem 13 from the typed outlie. c) Argue two ways that for large, valid but uusable approximately 95% cofidece limits for are ˆ

13 First use Propositio 8 ad the use Theorem 14 from the typed outlie. d) Use Corollary 9 ad fid a valid usable replacemet for the limits of part c). (Use the "expected iformatio" fix.) e) Use Corollary 30 ad fid a valid usable replacemet for the limits of part c). (Use the "observed iformatio" fix.) 39. Suppose that x1, x,,, x are iid with margial pmf f x p specified below. x f x p 1 p 31 p p 31 p p 3 p Let 0 3 the umber of xi takig the value 0 the umber of xi takig the value the umber of x takig the value 3 i a) Give a formula for the log-likelihood fuctio based o the x i, L p. b) A sample of size 100 produces 0 64, 9, ad 3 7 ad a log-likelihood that is plotted below. Further, some umerical aalysis ca be doe to show that L , L ad L Use Corollary 30 (the "observed iformatio" fix) ad give approximate 95% cofidece limits for p based o this sample. 13

14 Sectio 3.4 of the Course Outlie 40. Cosider a statistical model where a sigle discrete observatio x has probability mass fuctio f x idicated i the table below. x

15 a) Iitially cosider oly the possibilities that 4 ad 1. Idetify a miimum Bayes risk test of H 0: 4 vs H a : 1 for a prior distributio with g4.6 ad g1.4. What are the size ad power for this test? b) Now cosider testig H 0: 4 vs H a : 4. Fid values for the geeralized log likelihood ratio statistic x l max, 4 potetially useful i this simple versus composite testig problem. Cosider the test that decides i favor of H a : 4 exactly whe x l 3 /. For which x values does this test reject H 0? What is the size of this test? x 41. For x1, x,, x iid Expoetial variables with mea, cosider testig H : 1 vs H : 1. 0 a a) Fid the geeralized log likelihood ratio statistic x l max,1 ad ote that it is a fuctio of the sample mea. Argue that as a fuctio of x it is decreasig to the left of x 1 ad icreasig to the right of x 1. b) O the basis part a), for 100 give two equatios i x that ca be solved umerically to L U L U produce d100 1 ad d100 1 so that a test rejectig H 0 if x d100 or x d100 is a likelihood ratio test of size approximately.05. c) Show how you could use the Cetral Limit Theorem to evaluate the (Type II) error probability for this test for the possibility that. (Give a formula for this i terms of d ad d.) L U d) Suppose that 100 ad x Plot the log-likelihood ad show graphically how you ca make a approximately 95% cofidece iterval for. What is this iterval? x 15

16 4. Retur to Problem 39 ad the plot of the likelihood provided for a particular sample of size 100. Read off from that plot a approximate 95% cofidece iterval for p as the set of all p 's with log-likelihood withi.5 times the upper 5% poit of the maximum log-likelihood. 1 distributio of the Assigmet 6 Due 11/14/14 Sectios 4.1 ad 4. of the Course Outlie 43. Vardema will sed you a R file providig code that will allow you to study simple -class classificatio problems with x 0,1. As provided, the code is set up to study a case where x x x x f 0 I 0,1, g 0.5, f 1 x x I 0,1, ad g You ca make simple modificatios to chage all of these, but for this problem, begi with this set-up. (You should thoroughly uderstad the code ad be able to chage parameters it uses ad modify it to hadle other problems.) a) Fid the form of the Bayes optimal classifier i this problem ad use -variable calculus or geometry to fid the error rate for this classifier. (Note that NO data-based approximatio to the Bayes optimal classifier ca have a real error rate lower tha this value.) b) Geerate (usig the code) a traiig set of size N 100 for this problem. Make a plot of the regios i 0,1 where the 5 earest eighbor classifier classifies to 0 ad to 1. How does this classifier compare to the optimal classifier? What is the K 5 fold cross validatio error rate for this classifier? Now make a similar plot ad evaluate the correspodig cross-validatio error rate for the 9 earest eighbor classifier. c) Redo part b) for a traiig set of size N 400. d) For the size N 400 traiig set, fid the k for a earest eighbor classifier with the best K 10 cross-validatio error rate. e) Redo part d) for the bootstrap estimate of error rate. f) Commo practice i predictive aalytics is to look for the least complex classifier with predicted error rate "fairly close" to the miimum. What does this heuristic suggest is a good k i the preset problem with N 400? 16

17 44. Redo Problem 43 usig x 1 1 x 0,1 f x1 x I Assigmet 7 Due 11/1/ Retur to the situatio of Problem 43 ad geerate traiig samples of sizes 400, 4000, ad a) For each traiig set size, use cross-validatio to idetify a good k for use i a k classifier. Compare the resultig classifiers to the Bayes/optimal classifier i terms of appearace of a plot of yˆ x1, x (white for 0 ad black for 1) at the poits o the grid used i the first set of classificatio code Vardema distributed. b) Create a test set of 100,000 pairs y, x (separate from those used to make the classifiers). Compute test error rates based o this set for the 3 classifiers of a). How do these compare to the theoretically optimal error rate you computed i 43a)? 46. (Extra Credit, ot required but worth a extra "poit" if doe completely/well.) I Sectio 4.1 of the typed outlie, the suggestio is made that oe might try estimatig f x 0 ad f x 1 ad with g ˆ 0 the fractio of y 0 cases i the traiig set, cosider the classifier ˆ I gˆ0 fˆ 0 1gˆ0 f 1 x x (*) I the cotext of Problem 43 with bivariate cotiuous x x, x for a "badwidth parameter" ad, to use 1 ˆ f x, x 0 exp x x x x N 1 1 1i i 0 i s.t. y i 0, a way to estimate desities is ˆ f x, x 1 exp x x x x N 1 1 1i i 1 i s.t. y i 1 17

18 where N0 ad N 1 are the couts of y 0 ad y 1 cases i the traiig set. For the 400 traiig set of Problem 43, treat as a complexity parameter ad compare several cases of i terms of the traiig error rates for the classifier (*) ad appearace of a plot of ˆ, y x x (white for 0 ad black for 1) at the poits o the grid used i the first set of classificatio code Vardema distributed. (I'm guessig that a reasoable place to start lookig for a good badwidth parameter is aroud.05.) 1 Sectios 4.3 ad 4.4 of the Course Outlie 47. Retur to the situatio of Problem 43. (Uless specifically idicated to the cotrary, use the N 400 traiig set size below.) a) Fit classificatio trees usig the tree() fuctio with both default parameter settigs ad those provided i Vardema's code. Which "full" tree is simpler? Usig the more complex full tree, use cross validatio with the cost-complexity pruig idea to fid a good sub-tree of the full tree. What is the cross-validatio error rate for the chose complexity/weight/umber-of-fialodes? b) Ru the radomforest code provided by Vardema. Does the radom forest predictor seem to behave more sesibly if it is built o a much larger traiig set tha 400? (Also ru the code for 4000.) Compare error rates ad plots of how the classifiers split up 0,1 ito classificatio regios. c) Fit ad compare (based o traiig error rates ad plots of how the classifiers break up 0,1 ito classificatio regios) good logistic regressio-based classifiers produced usig glmet() with 0,.5,1. (Use lambda.1se.) d) How would you modify the classifiers you have idetified i c) if you were give the iformatio that actually, the prevalece of y 0 case i the uiverse is ot as represeted i the traiig set but is actually much more like g (If the traiig set had bee made up to be represetative of the uiverse, oe would have expected oly about 4 istaces of y 0.) (See the outlie discussio of case-cotrol studies.) Show how these classifiers split up 0,1 ito classificatio regios as compared to the oes from which they are derived. 18

19 e) Choosig betwee a umber of possible cost parameters o the basis of cross-validatio, fid a good support vector classifier for the problem. What is its traiig error rate? Make a plot of how it breaks up 0,1 ito classificatio regios. f) Combie the three classifiers you idetify i c) i differet ways. First, with equal weights average fitted probabilities that y 0 ad use a classifier built o the average probability. The simply use equal weight majority votig betwee the classifiers to defie a ew oe. (These are the two ideas of the "Esemble" video.) Compare these two classifiers to each other ad to the three classifiers from which they were made i terms of a test set error rate like that made i Problem 45b). 48. Redo Problem 47 usig x 1 1 x 0,1 f x1 x I I a) look for a choice of parameters that gives a large iitial tree. Assigmet 8 Not to be Collected but Covered o Exam 3 1/5/14 Sectio 5.1 of the Course Outlie 49. (Simple liear regressio i terms of the ormal liear model) Below is a small set of fake x, y data. x y Cosider the (SLR) ormal liear model y where the i are idepedet x i 1 i i otatio as Y Xβ ε for Normal 0, radom variables. This is writte i matrix 19

20 y y Yy 3, X1 3, β, ad ε 3 y y Usig matrix calculatios (either "by had" or aided by the matrix calculatio facility of R) do all of the followig. a) Fid fuctio ˆOLS ˆMLE OLS β β. Based o this fit to the "traiig data," what is y ˆ 1.5 OLS ŷ x?? What is the b) Fid SSE ad the value of the MLE for. c) Make 95% two-sided cofidece limits for i the ormal liear model. d) Give a 90% two-sided cofidece iterval for the icrease i average y that accompaies a uit icrease i x. (See agai the "o-matrix" form of the model.) e) Give a 90% two-sided cofidece iterval for the average value of y whe x 1.5. f) Give a 90% two-sided predictio iterval for the ext value of y whe x 1.5. g) Give a 90% two-sided predictio iterval for the sample mea of the ext 5 values of y whe x 1.5. h) Give a 90% two-sided cofidece iterval for the differece i mea y at x.5 ad mea y at x 1.5. i) Give a 90% two-sided predictio iterval for the differece betwee a ew y at x.5 ad a differet ew y at x (The oe-way ormal model i terms of the geeral ormal liear model) I a ISU egieerig research project, so called "tilt table tests" were doe i order to determie the agles at which certai vehicles experiece lift-off of the "high side" set of wheels ad begi to roll over o their sides. So called "tilt table ratios" (which are the tagets of agles at which liftoff occurred) were measured for 4 differet vas with the followig results. 0

21 Va #1 Va # Va #3 Va #4.96,.970,.967, 1.010, 1.04, 1.01, , , 1.093, 1.090, , 1.001, 1.00, (Notice that Va #3 was tested 5 times while the others were tested 4 times each.) Vas #1 ad # were miivas ad Vas #3 ad #4 were full size vas. We'll cosider aalysis of these data usig a "oe-way ormal model" for y jth tilt table ratio for va i ij of the form yij j ij for ij iid N0,. With y11 11 y y y ,, Y, ad y X β ε y y y this ca be writte i matrix otatio as Y Xβ ε Usig matrix calculatios (either "by had" or aided by the matrix calculatio facility of R) do all of the followig. a) Fid ˆ ˆ OLS MLE β β. b) Fid SSE ad the value of the MLE for. c) Make 95% two-sided cofidece limits for i the ormal liear model. 1

22 d) Give 95% two-sided cofidece limits for 1 (the mea tilt table ratio for Va #1). e) Give 95% two-sided cofidece limits for 3 (the mea tilt table ratio for Va #3). f) Give 95% two-sided predictio limits for the ext tilt table ratio for Va #3. g) Give 95% two-sided cofidece limits for 1 3 (the differece i mea tilt table ratios for Va #1 ad Va #3). h) It might be of iterest to compare the average of the tilt table ratios for the miivas to that of the full size vas. Accordigly, give a 95% two-sided cofidece iterval for the quatity Assigmet 9 Not to be Collected but Covered o the Fial Exam Sectio 5 of the Course Outlie (Practical SEL Predictio) 51. This questio cocers the aalysis of a set of home sale price data obtaied from the Ames City Assessor s Office. Data o sales May 00 through Jue 003 of 1 ad story homes built 1945 ad before, with (above grade) size of 500 sq ft or less ad lot size 0,000 sq ft or less, located i Low- ad Medium-Desity Residetial zoig areas. (The data are i a Excel spreadsheet o the Stat 34 Web page. These eed to be loaded ito R for aalysis.) 88 differet homes fittig this descriptio were sold i Ames durig this period. ( were actually sold twice, but oly the secod sales prices of these were icluded i our data set.) For each home, the value of the respose variable Price recorded sales price of the home ad the values of 14 potetial explaatory variables were obtaied. These variables are 1 Size Lad Bedrooms Cetral Air Fireplace Full Bath the floor area of the home above grade i sq ft, the area of the lot the home occupies i sq ft, a cout of the umber i the home a dummy variable that is 1 if the home has cetral air coditioig ad is 0 if it does ot, a cout of the umber i the home, a cout of the umber of full bathrooms above grade,

23 Half Bath Basemet Fiished Bsmt Bsmt Bath Garage Multiple Car a cout of the umber of half bathrooms above grade, the floor area of the home's basemet (icludig both fiished ad ufiished parts) i sq ft, the area of ay fiished part of the home's basemet i sq ft, a dummy variable that is 1 if there is a bathroom of ay sort (full or half) i the home's basemet ad is 0 otherwise, a dummy variable that is 1 if the home has a garage of ay sort ad is 0 otherwise, a dummy variable that is 1 if the home has a garage that holds more tha oe vehicle ad is 0 otherwise, Style ( Story ) a dummy variable that is 1 if the home is a story (or a home ad is 0 otherwise, ad Zoe ( Tow Ceter ) a dummy variable that is 1 if the home is i a area zoed as "Urba Core Medium Desity" ad 0 otherwise. 1 story) a) I preparatio for aalysis, stadardize all variables that are ot dummy variables (those we'll leave i raw form), makig a data frame with 15 colums. Say clearly how oe goes from a particular ew set of home characteristics to a correspodig set of predictors. The say clearly how a predictio for the stadardized price to a predictio for the dollar price. b) Fid predictors for stadardized price of all the followig forms: OLS Lasso (choose by cross-validatio) Ridge (choose by cross-validatio) Elastic Net with.5 (choose by cross-validatio) Nearest Neighbor (based o the k 4 predictors that have the largest coefficiets i the liear predictors you idetify) Sigle Regressio Tree (choose the tree by cost-complexity pruig of full trees) Radom Forest (use default parameters) N-W Kerel Smoother (based o the k 4 predictors that have the largest coefficiets i the liear predictors you idetify) (look for a good badwidth) Local Liear Regressio Smoother (based o the k 4 predictors that have the largest coefficiets i the liear predictors you idetify) (look for a good badwidth) Neural Network (use oe hidde layer with 8 odes ad cross-validatio i the fittig) Fid the sets of predictios all these methods produce for the traiig set. Compare these sets of predictios by makig scatterplots for all pairs of predictio types ad computig all pairs of correlatios betwee predictios. Which two methods give the least similar predictios? (If these both have good cross-validatio errors, they would become good cadidates for combiig ito a sigle "stacked" predictor.) 3