Math 141. Lecture 10: Confidence Intervals. Albyn Jones 1. jones/courses/ Library 304. Albyn Jones Math 141

Size: px

Start display at page:

Download "Math 141. Lecture 10: Confidence Intervals. Albyn Jones 1. jones/courses/ Library 304. Albyn Jones Math 141"

Shauna Mosley
5 years ago
Views:

1 Math 141 Lecture 10: Confidence Intervals Albyn Jones 1 1 Library 304 jones@reed.edu jones/courses/141

2 Inference Suppose X Binomial(n, p). Inference about p includes the topics:

3 Inference Suppose X Binomial(n, p). Inference about p includes the topics: Estimation Example: ˆp = X/n

4 Inference Suppose X Binomial(n, p). Inference about p includes the topics: Estimation Example: ˆp = X/n Hypothesis Testing Example: test H 0 : p = 1/2. Is p = 1/2 plausible given our data?

5 Inference Suppose X Binomial(n, p). Inference about p includes the topics: Estimation Example: ˆp = X/n Hypothesis Testing Example: test H 0 : p = 1/2. Is p = 1/2 plausible given our data? Confidence Intervals If we estimate p with ˆp = X/n, how accurately do we know p? Or: what are the plausible values for p, given our data?

6 Confidence Intervals X Binomial(n, p), we observe X = k. Consider the set of values for p that would not be rejected by a hypothesis test.

7 Confidence Intervals X Binomial(n, p), we observe X = k. Consider the set of values for p that would not be rejected by a hypothesis test. Definition: A 95% Confidence Interval for p is the set I = (P L, P U ) satisfying p = p 0 is not rejected for p 0 I at significance level α =.05.

8 Confidence Intervals X Binomial(n, p), we observe X = k. Consider the set of values for p that would not be rejected by a hypothesis test. Definition: A 95% Confidence Interval for p is the set I = (P L, P U ) satisfying p = p 0 is not rejected for p 0 I at significance level α =.05. In general, a 100(1 α)% CI is the set not rejected at significance level α.

9 Binomial CI s: How to find them For the binomial, iterative search is necessary. Suppose that we observe 12 successes in 20 trials. The lower endpoint of a 95% CI satisfies P(X 12 p = P L ) =.025 while the upper endpoint satisfies P(X 12 p = P U ) =.025 For those interested in algorithms, there are numerous possibilities, including bisection search, and the R uniroot() function.

10 Binomial(20, p), Observed X = 12 Binomial Confidence Interval Probability P_Low: X P_Hi: Outcome

11 Binomial CI s made easy Use the R function binom.test(), which performs the so-called exact test we have been discussing, and constructs the corresponding confidence interval. Example: binom.test(12,20,p=.5) Exact binomial test data: 12 and 20 number of successes = 12, number of trials = 20, p-value = alternative hypothesis: true probability of success is not equal to percent confidence interval:

12 Check the Definition! 95 percent confidence interval: # Pr(X <= 12 p= ) > pbinom(12,20, ) [1] # Pr(X >= 12 p= ) > pbinom(11,20, ,lower.tail=false) [1]

13 More on binom.test() We use the binom.test function for various purposes:

14 More on binom.test() We use the binom.test function for various purposes: Test H 0 : p =.3 with binom.test(12, 20, p =.3)

15 More on binom.test() We use the binom.test function for various purposes: Test H 0 : p =.3 with binom.test(12, 20, p =.3) A one sided test: Test H 0 : p.3 against H 1 : p >.3 with binom.test(12, 20, p =.3, alternative = greater )

16 Margin of Error Polls report an approximate confidence interval they call the margin of error, which they compute in the following way. Recall p(1 p) Var(ˆp) = n Substitute the most conservative choice p = 1/2, and compute the square root, and multiply by 2: 1 1 2SE(ˆp) n = n = 1 n

17 Example Suppose that there were 400 respondents to an electoral poll, and the responses were Candidate N Proportion Romney Santorum Gingrich Paul 32.08

18 Example For n = 400, the MOE calculation is 1/20 =.05. The exact 95% CI for Romney given by binom.test() is: 95 percent confidence interval: While for Ron Paul we get 95 percent confidence interval:

19 Interpretation By construction, a 95% CI is the set of values for p that would not be rejected by a test of size α =.05. Thus the CI answers all questions about specific null hypotheses. For example: suppose the 95% CI for p is [.35,.8],

20 Interpretation By construction, a 95% CI is the set of values for p that would not be rejected by a test of size α =.05. Thus the CI answers all questions about specific null hypotheses. For example: suppose the 95% CI for p is [.35,.8], would we reject H 0 : p =.5?

21 Interpretation By construction, a 95% CI is the set of values for p that would not be rejected by a test of size α =.05. Thus the CI answers all questions about specific null hypotheses. For example: suppose the 95% CI for p is [.35,.8], would we reject H 0 : p =.5? would we reject H 0 : p =.25?

22 Interpretation By construction, a 95% CI is the set of values for p that would not be rejected by a test of size α =.05. Thus the CI answers all questions about specific null hypotheses. For example: suppose the 95% CI for p is [.35,.8], would we reject H 0 : p =.5? would we reject H 0 : p =.25? would we reject H 0 : p =.35?

23 Interpretation: Random Intervals Another definition of a 95% CI is a random interval [L(X), U(X)] which has the property that roughly 95% of the time, p [L(X), U(X)] Note: this is a marginal probability, i.e. before observing the data. Conditional on observing X = k, the interval [L(k), U(k)] is not a random interval! P(p [L(X), U(X)] X = k) is either 0 or 1, and we don t know which! This is related to the fact that p-values are not measuring P(H 0 data).

24 Illustration: 100 Random Intervals X <- rbinom(100,20,p=.5) M <- matrix(0,nrow=100,ncol=2) for(i in 1:100) { M[i,] <- binom.test(x[i],20,p=.5)$conf.int } plot(1:100, X/20, pch=4, ylim=c(0,1), ylab="p",xlab="trial") segments(1:100,m[,1],1:100,m[,2],col="blue",lwd=2) abline(h=.5,col="red")

25 Simulation: Random Intervals One Hundred Confidence Intervals, True p is.5 P Trial

26 Interpretation: Precision A small interval indicates a more accurate estimate, a wide interval a less accurate estimate. Often this is a function of sample size. Suppose X Binomial(n, p) n X 95% CI [.272,.728] [0.429, 0.571] [0.478, 0.522]

27 Confidence Intervals vs Hypothesis Tests In general, a confidence interval conveys much more information than a hypothesis test: the CI tells us the answer for every hypothesis test, and the CI gives a concrete indication of the precision of an estimate.

28 R functions There are two primary R functions for tests and CI s involving a single binomial p:

29 R functions There are two primary R functions for tests and CI s involving a single binomial p: Exact Tests and CI s binom.test(x, n, p 0 )

30 R functions There are two primary R functions for tests and CI s involving a single binomial p: Exact Tests and CI s Approximate Tests and CI s binom.test(x, n, p 0 ) prop.test(x, n, p 0 ) (Also useful for comparing several samples.)

31 The Sign Test A non-parametric test Suppose X 1, X 2,..., X n are IID numeric RV s, like height or weight. We can test H 0 : median(x) = m as follows:

32 The Sign Test A non-parametric test Suppose X 1, X 2,..., X n are IID numeric RV s, like height or weight. We can test H 0 : median(x) = m as follows: 1 Let Y i = 1 if X i > m, else Y i = 0.

33 The Sign Test A non-parametric test Suppose X 1, X 2,..., X n are IID numeric RV s, like height or weight. We can test H 0 : median(x) = m as follows: 1 Let Y i = 1 if X i > m, else Y i = 0. 2 Let S n = Y i, then if P(X m) = p, S n Binomial(n, p)

34 The Sign Test A non-parametric test Suppose X 1, X 2,..., X n are IID numeric RV s, like height or weight. We can test H 0 : median(x) = m as follows: 1 Let Y i = 1 if X i > m, else Y i = 0. 2 Let S n = Y i, then if P(X m) = p, 3 Test H 0 : p = 1/2 S n Binomial(n, p)

35 Sign Test: Example A study comparing 15 pairs of monozygotic twins, where one member of the pair was schizophrenic and the other was not, found the following volumes of the hippocampus, in cm 3 Pair Normal Schiz Diff

36 Sign Test: Example, Continued 14 of the differences were positive, 1 was negative. If there were no systematic differences in the volume, then we would expect each difference to be equally likely to be positive or negative, i.e. the median difference should be 0. binom.test(14,15,p=.5) data: 14 and 15 number of successes = 14, number of trials = 15, p-value =

37 Summary Confidence Intervals answer questions like: How accurate is our estimate? What are the plausible values for our parameter, given our data? In general, you should always report CI s for your estimates, not just p-values for some specific hypothesis test.

Math 141. Quantile-Quantile Plots. Albyn Jones 1. jones/courses/ Library 304. Albyn Jones Math 141

Math 141. Quantile-Quantile Plots. Albyn Jones 1. jones/courses/ Library 304. Albyn Jones Math 141 Math 141 Quantile-Quantile Plots Albyn Jones 1 1 Library 304 jones@reed.edu www.people.reed.edu/ jones/courses/141 Outline: Quantile-Quantile Plots Quantiles and Order Statistics Quantile-Quantile Plots