Session 11. Large-sample intervals. Small-sample intervals. Bootstrap. Confidence Intervals via the bootstrap. Dr. Syring April 2, 2019

Transcription

1 Session 11 Dr. Syring April 2, 2019 Confidence Intervals via the bootstrap Large-sample intervals Recall that we use the Central Limit Theorem to derive approximate confidence intervals for population means of general or non-normal populations, e.g. for a Bernoulli population, the 100(1 α)% approximate CI for p is ( p^ + z α/2 p^ (1 p^ )/n, p^ + z α/2 p^ (1 p^ )/n). And, the textbook claims the approximation is good if np, n(1 p) 8. Here is a short simulation to support that claim: n<-16 p<-0.5 alpha <- 0.1 coverage <- rep(0,1000) for(i in 1:1000){ data<-rbinom(1,n,p) p.hat <- data/n CI <- c(p.hat + qnorm(alpha/2)*sqrt(p.hat*(1-p.hat)/n), p.hat + qnorm(1-alpha/2)*sqrt(p.hat*(1 -p.hat)/n)) coverage[i] <- ifelse(ci[1]<=p & CI[2]>=p,1,0) mean(coverage) ## [1] Small-sample intervals The CLT generally works for n 30 samples. What if we do not have that many? For some experiments, it is very costly and/or time-consuming to collect data, so even n = 30 may be prohibitively large. Suppose a biologist is studying a random mutation. She has the time and funding to grow n = 10 strains of cyanobacteria (photosynthetic microorganisms). Each strain has the same chance of gaining the mutation, independentyl of the other strains. Suppose in her experiment only one strain mutates. Compute the approximate 90% CLT-based confidence interval for the true population proportion of mutating strains. What s problematic about your interval estimate of p? c(.1+qnorm(.05)*sqrt(.1*.9/10),.1+qnorm(.95)*sqrt(.1*.9/10)) ## [1] Bootstrap file:///e:/teaching/washu/math3200sp19/discussion/session11/ds11_solutions.html 1/6

2 We use the CLT to approximate the distribution of a sample mean with a normal distribution. This works well for large samples, but not always for smaller samples. The bootstrap is another way to approximate the probability distribution of a sample mean. The bootstrap is a computational method that uses only the data, no model. Thus, it s sort of like ``pulling oneself up by one s own bootstraps" which never really made sense to me Suppose you have a random sample X 1,..., X n. The bootstrap distribution of the data is the set of all random samples of size n, with replacement, from,...,. X 1 X n For example, suppose n = 3 and X 1 = 1, X 2 = 2, X 3 = 3. The bootstrap distribution of the data is the set of 3 3 = 27 equally-likely sets: {{1, 1, 1, {1, 1, 2, {1, 2, 1,..., {3, 3, 1, {3, 3, 2, {3, 3, 3. In general, there are n n sets (which is a huge number!), though many are equal up to a permutation, e.g. {1, 2, 3 and {3, 2, 1. Suppose we are interested in the population mean. Then, the sample mean based on the sample X 1 = 1, X 2 = 2, X 3 = 3 is 2. But, we also have a bootstrap estimate of the distribution of the sample mean. For each bootstrap data set, compute the sample mean. You will find you get the following distribution: x : 1, 1.33, 1.67, 2, 2.33, 2.67, 3 P ( X = x ) : 1/27, 3/27, 6/27, 7/27, 6/27, 3/27, 1/27 In practice, it is difficult to list all the bootstrap data sets. So, we approximate the bootstrap distribution by simulation. Here s an example for the above data X 1 = 1, X 2 = 2, X 3 = 3. X<-c(1,2,3) means<-rep(0,10000) for(i in 1:10000){ Y <- sample(x,3, replace = T) means[i] = mean(y) hist(means) file:///e:/teaching/washu/math3200sp19/discussion/session11/ds11_solutions.html 2/6

3 sorted <- sort(means) c(sorted[.05*10000], sorted[.95*10000]) ## [1] Confidence intervals using bootstrap There are many ways to compute confidence intervals using the bootstrap. The easiest is the quantile method (or percentile method). Once you have the bootstrap distribution of the sample mean, simply use the α/2 and 1 α/2 quantiles of this distribution as an approximate 100(1 α%) CI. Here s an example for the small sample binomial data: X<-c(1,0,0,0,0,0,0,0,0,0) means<-rep(0,10000) for(i in 1:10000){ Y <- sample(x,10, replace = T) means[i] = mean(y) hist(means) file:///e:/teaching/washu/math3200sp19/discussion/session11/ds11_solutions.html 3/6

4 sorted <- sort(means) c(sorted[.05*10000], sorted[.95*10000]) ## [1] c(quantile(means,.05), quantile(means,.95)) ## 5% 95% ## The interval (0, 0.3) is much more sensible than what we got using the CLT for this small data set. Exercises 1. Poisson. Suppose in n = 5 Poisson trials we observe X 1 = 4, X 2 = 4, X 3 = 6, X 4 = 5, X 5 = 7. Compute approximate 95% confidence intervals using both the CLT and the bootstrap. data <- c(4,4,6,5,7) clt.ci <- c(mean(data)+qnorm(.025)*sqrt(mean(data)/5), mean(data)+qnorm(.975)*sqrt(mean(data)/5 )) clt.ci file:///e:/teaching/washu/math3200sp19/discussion/session11/ds11_solutions.html 4/6

5 ## [1] boot.means<-rep(0,10000) for(i in 1:10000){ boot.data<-sample(data,5,replace = T) boot.means[i] <- mean(boot.data) c(quantile(boot.means,0.025), quantile(boot.means,0.975)) ## 2.5% 97.5% ## Exponential. Suppose in n = 8 Exponential trials we observe $ X_1 = 0.82, X_2 = 1.54, X_3 = 1.26, X_4 = 1.62, X_5 = 0.64, X_6 = 0.84, X_7 = 0.43, X_8 = 1.63$. Assume the exponential is parametrized with E(X) = λ. Compute approximate 80% confidence intervals using both the CLT and the bootstrap. 3. General population with finite variance. Suppose in n = 10 trials we observe 1.06, 1.93, 2.02, 2.17, 0.89, 1.60, 3.07, 1.57, 1.37, Compute approximate 99% confidence intervals using both the CLT and the bootstrap. 4. Simulation study. Design and perform a simulation to compare the coverage probability of CLT intervals and bootstrap intervals for an Exponential population. Assume in each experiment we conduct n = 10 trials and E(X) = λ = 0.1. Simulate this experiment times and compute the corresponding interval estimates for each simulated experiment. Store an indicator value of whether or not each CI contained the true λ = 0.1 and compute the overall coverage proportion of the of each type of CI. Does the bootstrap CI have better coverage than the CLT-based interval in this example? file:///e:/teaching/washu/math3200sp19/discussion/session11/ds11_solutions.html 5/6

6 reps < cover_clt <- rep(0,reps) cover_boot_quant <- rep(0,reps) cover_boot_basic <- rep(0,reps) alpha <- 0.1 lambda <- 0.1 n <- 10 for(i in 1:reps){ data <- rexp(n,1/lambda) xbar <- mean(data) # bootstrap boot.means <- rep(0,1000) for(j in 1:1000){ data.boot <- sample(data,n,replace=t) boot.means[j] <- mean(data.boot) boot.ci <- c(2*xbar-quantile(boot.means,1-alpha/2), 2*xbar+quantile(boot.means,alpha/2)) # a d ifferent way to compute bootstrap intervals... boot.ci.quant <- c(quantile(boot.means,alpha/2), quantile(boot.means,1-alpha/2)) # quantile bo otstrap method # clt clt.ci <- c(xbar+qnorm(alpha/2)*xbar/sqrt(n), xbar+qnorm(1-alpha/2)*xbar/sqrt(n)) # cover or not cover_boot_basic[i] <-ifelse(boot.ci[1]<=lambda & boot.ci[2]>=lambda,1,0) cover_boot_quant[i] <-ifelse(boot.ci.quant[1]<=lambda & boot.ci.quant[2]>=lambda,1,0) cover_clt[i] <-ifelse(clt.ci[1]<=lambda & clt.ci[2]>=lambda,1,0) mean(cover_boot_basic) ## [1] mean(cover_boot_quant) ## [1] mean(cover_clt) ## [1] file:///e:/teaching/washu/math3200sp19/discussion/session11/ds11_solutions.html 6/6