Lecture 4: Parameter Es/ma/on and Confidence Intervals. GENOME 560, Spring 2015 Doug Fowler, GS

Lecture 4: Parameter Es/ma/on and Confidence Intervals GENOME 560, Spring 2015 Doug Fowler, GS (dfowler@uw.edu) 1

Review: Probability DistribuIons Discrete: Binomial distribuion Hypergeometric distribuion Poisson distribuion 2

Review: Probability DistribuIons Discrete: Binomial distribuion Hypergeometric distribuion Poisson distribuion Con/nuous: Uniform distribuion ExponenIal distribuion Gamma distribuion Normal distribuion 3

MulIvariate Hypergeometric Dist n The HGD can be generalized to picking a sample of size n where there are exactly (k 1, k 2 k c ) items from each of c classes from a populaion of N items of c classes where there are K i items of of class i pmf: Q c i=1 N n K i k i Example: There are 5 black, 10 white and 15 red balls in an urn. If you draw six without replacement, what is the probability that you pick 2 of each color? 5 2 10 2 30 6 15 2 =0.08 5

Goals Basic concepts of parameter esimaion Confidence intervals Intro to hypothesis tesing 6

What Is Parameter? 7

What Is Parameter? Variables vs. Parameters According to Bard & Yonathan (1974) * Usually a probabilisic model is designed to explain the relaionships that exist among quaniies which can be measured independently in an experiment; these are the variables of the model. To formulate these relaionships, however, one frequently introduces "constants" which stand for inherent properies of nature. These are the parameters. We oeen denote by θ * Bard, Yonathan (1974). Nonlinear Parameter Estimation. New York 8

Which are parameters, variables? Binomial distribuion (coin tossing) X: number of Heads aeer n coin tosses P n k n k { X = k} = p (1 p) k variable parameter θ = p Poisson distribuion X: number of e- mails within a week P variable { X = k} = e k λ λ k! parameter θ = λ 9

Parameters Determine DistribuIons When sampling is from a populaion described by a pdf or pmf f(x θ), knowledge of θ yields knowledge of the enire populaion 10

Parameters Determine DistribuIons When sampling is from a populaion described by a pdf or pmf f(x θ), knowledge of θ yields knowledge of the enire populaion This is why parameter esimaion is useful: if we are tossing a coin we would like to esimate the parameter p If we are couning the number of emails per week, we would like to esimate λ 11

Central Dogma of StaIsIcs 12

Parameter EsImaIon Es/mator: StaIsIc whose calculated value is used to esimate a parameter, θ Es/mate: A paricular realizaion of an esimator, θ Types of esimates: Point es/mate: single number that can be regarded as the most plausible value of θ Interval es/mate: a range of numbers, called a confidence interval, that informs us about the quality of our esimate 13

Simple Example EsImators Suppose we take a sample from a binomial distribuion whose parameters are unknown. We get m successes from n samples. How can we esimate the parameter π (the populaion p)? Method 1: We could just use m and n Method 2: Alternately, we could just look in the literature for similar experiments. We could ignore our data and set 16

Good EsImators Are: Consistent: as sample size increases, gets closer to θ Would our example esimators be consistent? 17

Good EsImators Are: Consistent: as sample size increases, gets closer to θ Would our example esimators be consistent? EsImator 1, yes (m/n will approach π, law of large numbers) EsImator 2, no (our data doesn t maner) 18

Good EsImators Are: Consistent: as sample size increases, gets closer to θ Unbiased: 19

What do we mean by unbiased? A biased esimator diverges systemaically from the true parameter value 20

Good EsImators Are: Consistent: as sample size increases, gets closer to θ Unbiased: the expected value of is equal to θ 21

Good EsImators Are: Consistent: as sample size increases, gets closer to θ Unbiased: the expected value of is equal to θ Are our example esimators biased? 22

Good EsImators Are: Consistent: as sample size increases, gets closer to θ Unbiased: the expected value of is equal to θ Are our example esimators biased? EsImator 1 turns out to be unbiased EsImator 2 is has a bias 23

Good EsImators Are: Consistent: as sample size increases, gets closer to θ Unbiased: Precise: 24

What do we mean by precise? An imprecise esimator is subject to random variability 25

Good EsImators Are: Consistent: as sample size increases, gets closer to θ Unbiased: Precise: the variance of EsImator 1 should be minimal EsImator 2? 26

Good EsImators Are: Consistent: as sample size increases, gets closer to θ Unbiased: Precise: the variance of EsImator 1 should be minimal EsImator 2 has zero variance Bias and variance are intertwined, and oeen you will have to chose to minimize one or the other 27

EsImators for normally distributed data Since we know that much experimental data is normally distributed, let s start here General methods for esimaing parameters (MLE, Bayesian) will be covered later. 28

EsImators for normally distributed data F(x) 0.0 0.1 0.2 0.3 0.4 3 2 1 0 1 2 3 x 29

EsImators for normally distributed data F(x) 0.0 0.1 0.2 0.3 0.4 What two parameters define a normal distribuion? 3 2 1 0 1 2 3 x 30

EsImators for normally distributed data F(x) 0.0 0.1 0.2 0.3 0.4 What two parameters define a normal distribuion? mean = μ standard deviaion = σ 3 2 1 0 1 2 3 x 31

EsImators for normally distributed data F(x) 0.0 0.1 0.2 0.3 0.4 μ σ What two parameters define a normal distribuion? mean = μ standard deviaion = σ 3 2 1 0 1 2 3 x 32

EsImators for normally distributed data Given a sample from a normally distributed populaion, what esimators would you use for μ,σ? ˆµ = ˆ = 33

EsImators for normally distributed data Given a sample from a normally distributed populaion, what esimators would you use for μ,σ? ˆµ = ˆ = 34

Confidence intervals: how good is my parameter esimate? 35

Back to our fluorescent yeast Let s say we measure the fluorescence of 25 yeast cells and find x = 89.1; s = 24.25 How good is our esimate ˆµ = 89.1? 36

Back to our fluorescent yeast Let s say we measure the fluorescence of 25 yeast cells and find x = 89.1; s = 24.25 How good is our esimate ˆµ = 89.1? On what will the goodness of the esimate depend? 37

A simple staring point What is the probability that the mean fluorescence of all ~2E8 cells in the culture is within 79.1 and 99.1? 39

A simple staring point What is the probability that the mean fluorescence of all ~2E8 cells in the culture is within 79.1 and 99.1? P (µ is within 10 of x) 40

A simple staring point What is the probability that the mean fluorescence of all ~2E8 cells in the culture is within 79.1 and 99.1? P (µ is within 10 of x) Recall that x is a RV with its own sampling distribuion 0.0 0.1 0.2 0.3 0.4 ˆ = µ x The sampling distribution of the sample mean is: Normal (by central limit theorem) Has µ x = µ = x Has x = p = p s n n 41

Standard Error of the Mean SEM is the standard deviaion of the sampling distribuion of the mean Oeen confused with standard deviaion of a sample in the literature. The standard deviaion is a descripive staisic, but SEM describes the spread of the sampling distribuion. SD of the sample is the degree to which individuals within a sample differ from the sample mean SEM is an esimate of how far the sample mean is likely to be from the populaion mean 42

Standard Error of the Mean are independent obs from a pop. with mean μ and stdev σ 43

Standard Error of the Mean are independent obs from a pop. with mean μ and stdev σ 44

Standard Error of the Mean are independent obs from a pop. with mean μ and stdev σ Is a property of RV 45

Standard Error of the Mean are independent obs from a pop. with mean μ and stdev σ Is a property of RV 46

Standard Error of the Mean are independent obs from a pop. with mean μ and stdev σ Is a property of RV 47

Standard Error of the Mean are independent obs from a pop. with mean μ and stdev σ Is a property of RV 48

Standard Error of the Mean are independent obs from a pop. with mean μ and stdev σ Is a property of RV 49

A simple staring point What is the probability that the mean fluorescence of all ~2E8 cells in the culture is within 79.1 and 99.1? P (µ is within 10 of x) Recall that x is a RV with a sampling distribuion 0.0 0.1 0.2 0.3 0.4 ˆ = µ x The sampling distribution of the sample mean is: Normal Has µ x = µ = x Has x = p = p s n n We just need to find the area under the sampling distribution of the sample mean corresponding to the mean +/- 10 50

A simple staring point What is the probability that the mean fluorescence of all ~2E8 cells in the culture is within 79.1 and 99.1? P (µ is within 10 of x) Recall that x is a RV with a sampling distribuion 0.0 0.1 0.2 0.3 0.4 ˆ = µ x The sampling distribution of the sample mean is: Normal Has Has µ x = µ = x x = p n = s p n = 24.25/5 =4.85 51

A simple staring point What is the probability that the mean fluorescence of all ~2E8 cells in the culture is within 79.1 and 99.1? P (µ is within 10 of x) Recall that x is a RV with a sampling distribuion 0.0 0.1 0.2 0.3 0.4 ˆ = µ x The sampling distribution of the sample mean is: Normal Has Has µ x = µ = x x = p n = s p n = 24.25/5 =4.85 > pnorm(c(79.1, 99.1), mean = 89.1, sd = 4.85) [1] 0.01961074 0.98038926 > 0.9804-0.01961 [1] 0.9607 52

GeneralizaIon We want to set a confidence interval such that 95% of samples from the distribuion are within the interval Given that we can esimate the mean and standard deviaion of the sampling distribuion of the sample mean, how do we do this? 53

GeneralizaIon We find the number of standard deviaions (z) we must move away from the mean to encompass 95% of the sampling distribuion of the sample mean 5% of total area 0.0 0.1 0.2 0.3 0.4 z µ x 54

GeneralizaIon Since the distribuion is symmetric, we can just use the CDF to accomplish this 97.5% of total area 0.0 0.1 0.2 0.3 0.4 z To find z such that CI 95% = µ x ± s p n z We can use the normal cumulative distribution function µ x 55

GeneralizaIon Now we can set a 95% CI for our fluorescence data 97.5% of total area 0.0 0.1 0.2 0.3 0.4 µ x z To find z such that CI 95% = µ x ± s p n z We can use the normal cumulative distribution function > min(which(pnorm(seq(-3,3,0.01))>=0.975)) [1] 497 > seq(-3,3,0.01)[497] [1] 1.96 > 89.1-4.85 * 1.96 [1] 79.594 > 89.1 + 4.85 * 1.96 [1] 98.606 56

A pracical note When sample sizes are greater than ~30, the sampling distribuion of the sample mean is normal and x = p s n is a good esimate When sample sizes are smaller than ~30, is an underesimate x = s p n Thus, in pracice we use the t- distribuion as opposed to the normal distribuion (more on this later) 57

InterpretaIon of confidence intervals If you repeatedly sample the same populaion, the CI (which differs for each sample) would contain the true populaion parameter X% of the Ime NOT the probability that this paricular CI from this paricular sample actually contains the populaion parameter NOT that there is an X% probability of a sample mean from a repeat experiment falling within the interval 58

IntroducIon to hypothesis tesing 59

What is hypothesis tesing? A staisical test examines a set of sample data and, on the basis of an expected distribuion of the data, leads to a decision about whether to accept the hypothesis underlying the expected distribuion or reject that hypothesis and accept an alternaive one. Over the next several weeks, we will come at this basic problem from many angles, but the general idea will remain the same 60

What is hypothesis tesing? A staisical test examines a set of sample data and, on the basis of an expected distribuion of the data, leads to a decision about whether to accept the hypothesis underlying the expected distribuion or reject that hypothesis and accept an alternaive one. StaIsIcal tests rely on compuing a test staisic (i.e. a staisic we will compare between samples, etc) 61

What is hypothesis tesing? A staisical test examines a set of sample data and, on the basis of an expected distribuion of the data, leads to a decision about whether to accept the hypothesis underlying the expected distribuion or reject that hypothesis and accept an alternaive one. To generate the expected distribuion (oeen called the null distribuion) of the test staisic we can use parametric distribuions or, through randomizaion, generate our own based on the data 62

What is hypothesis tesing? A staisical test examines a set of sample data and, on the basis of an expected distribuion of the data, leads to a decision about whether to accept the hypothesis underlying the expected distribuion or reject that hypothesis and accept an alternaive one. Then, we can see what the likelihood of obtaining the test staisic value we calculated from the sample. If it is highly unlikely, we might reject the null hypothesis. 63

What is hypothesis tesing? A staisical test examines a set of sample data and, on the basis of an expected distribuion of the data, leads to a decision about whether to accept the hypothesis underlying the expected distribuion or reject that hypothesis and accept an alternaive one. Then, we can see what the likelihood of obtaining the test staisic value we calculated from the sample. If it is highly unlikely, we might reject the null hypothesis. 0.0 0.1 0.2 0.3 0.4 0.0 0.1 0.2 0.3 0.4 64

Hypothesis TesIng Formally examine two opposing conjectures (hypotheses), H 0 and H 1 These two hypotheses are mutually exclusive and exhausive so that one is true to the exclusion of the other We accumulate evidence collect and analyze sample informaion for the purpose of determining which of the two hypotheses is true and which of the two hypotheses is false 65

Example Consider a genome- wide associaion study (GWAS) for T2D and you measure the blood glucose level of the case/control groups The null hypothesis, H 0 : There is no difference between the case/control groups in the mean blood glucose levels H 0 : μ 1 - μ 2 = 0 The alterna/ve hypothesis, H A : The mean blood glucose levels in the case/control groups are different H A : μ 1 - μ 2 0 66

The Null and AlternaIve Hypothesis The null hypothesis, H 0 : States the assumpion (numerical to be tested) Begin with the assumpion that the null hypothesis is TRUE Always contains the = sign The alterna/ve hypothesis, H A : Is the opposite of the null hypothesis Challenges the status quo Never contains just the = sign Is generally the hypothesis that is believed to be true by the researcher 67

One and Two Sided Tests Hypothesis tests can be one or two sided (tailed) One tailed tests are direcional: H 0 : μ 1 - μ 2 0 H A : μ 1 - μ 2 > 0 Two tailed tests are not direcional: H 0 : μ 1 - μ 2 = 0 H A : μ 1 - μ 2 0 68

P- values Calculate a test sta/s/c from the sample data that is relevant to the hypothesis being tested e.g. In our GWAS example, the test staisic can be determined based on μ 1, μ 2 and σ 1, σ 2 computed from the GWAS data Aeer calculaing a test staisic we convert this to a P- value by comparing its value to distribuion of test staisic s under the null hypothesis Null distribuion 0.0 0.1 0.2 0.3 0.4 Test staisic value Area = P- value (probability of obtaining a test staisic value at least as extreme as the one we observed) 69

P- values Calculate a test sta/s/c in the sample data that is relevant to the hypothesis being tested e.g. In our GWAS example, the test staisic can be determined based on μ 1, μ 2 and σ 1, σ 2 computed from the GWAS data Aeer calculaing a test staisic we convert this to a P- value by comparing its value to distribuion of test staisic s under the null hypothesis Null distribuion 0.0 0.1 0.2 0.3 0.4 Test staisic value Area = P- value (probability of obtaining a test staisic value at least as extreme as the one we observed) 70

When To Reject H 0 Level of significance, α: Specified before an experiment to define rejecion region Rejec/on region: set of all test staisic values for which H 0 will be rejected One sided α = 0.05 Two sided α = 0.05 The test staisic value required to reject is oeen called the criical value 71

When To Reject H 0 Level of significance, α: Specified before an experiment to define rejecion region Rejec/on region: set of all test staisic values for which H 0 will be rejected One sided α = 0.05 When we obtain a test staisic in the rejecion region we can conclude that either the null is true and a highly improbable event has occurred, or the null is false 73

Some NotaIon In general, criical values for an α level test denoted as: One sided test: X α Two sided test: X α/2 where X indicates the distribuion of the test staisic For example, if X ~ N(0,1): One sided test: z α (i.e., z 0.05 = 1.64) Two sided test: z α/2 (i.e., z 0.05/2 = z 0.05/2 = +- 1.96) 74

Errors in Hypothesis TesIng Level of significance, α: Specified before an experiment to define rejecion region Rejec/on region: set of all test staisic values for which H 0 will be rejected One sided α = 0.05 Given α= 0.05, what is the chance we will falsely reject the null hypothesis? 75

Errors in Hypothesis TesIng Actual Situation Truth Decision Do Not Reject H 0 Reject H 0 H 0 True H 0 False Correct Decision 1- α Incorrect Decision Type I Error α Incorrect Decision Type II Error Β Correct Decision 1- β 77

Type I and II Errors Actual Situation Truth Decision Do Not Reject H 0 Reject H 0 H 0 True H 0 False Correct Decision 1- α Incorrect Decision Type I Error α Incorrect Decision Type II Error Β Correct Decision 1- β α = P(Type I Error) β = P(Type II Error) Power = 1 - β 78

Parametric and Non- Parametric Tests Parametric Tests: Relies on theoreical distribuions of the test staisic under the null hypothesis and assumpions about the distribuion of the sample data (i.e., normality) Non- Parametric Tests: Referred to as DistribuIon Free as they do not assume that data are drawn from any paricular distribuion 79

R Session Goals Confidence interval calculaions User- defined funcions 80