Lecture 3. Biostatistics in Veterinary Science. Feb 2, Jung-Jin Lee Drexel University. Biostatistics in Veterinary Science Lecture 3

Transcription

1 Lecture 3 Biostatistics in Veterinary Science Jung-Jin Lee Drexel University Feb 2, 2015

2 Review Let S be the sample space and A, B be events. Then 1 P (S) = 1, P ( ) = 0. 2 If A B, then P (A) P (B). In particular, 0 P (A) 1. 3 P (A B) = P (A) + P (B) P (A B). In particular, if A and B are disjoint, then P (A B) = P (A) + P (B). 4 P (A c ) = 1 P (A).

3 Conditional Probability P (A): the probability that A occurs. Let B be another event. Often whether or not B has already occurred might affect the occurrence of the event A. Example Suppose that we are rolling a fair die, so that the sample space S = {1, 2, 3, 4, 5, 6}. Let A = {1, 2, 4}, then P (A) = 1 2. Roll the die again, and suppose that somehow we know that the outcome is an even number, that is B = {2, 4, 6} has already happenend. Then what is the probability that the outcome was one of A = {1, 2, 4}? Since B has occured, we know that the outcome should be one of 2, 4, 6 and among these three numbers, 2, 4 belong to A. Therefore we conclude that the desired probability is 2 3. Note that 2 3 = A B B = P (A B) P (B).

4 Conditional Probability, continued Definition The probability of an event A occuring when we know that B has already occurred is called the conditional probability of A given B and is denoted by P (A B). In general, Example P (A B) = P (A B). P (B) Let S = {1, 2,..., 10}, A = {1, 2, 3, 4, 5}, and B = {2, 4, 6, 8, 10}. Then P (A) = P (B) = 1 2, P (A B) = 1 P (A B) 5, and P (A B) = P (B) = 1/5 1/2 = 2 5.

5 Independence Definition Let A and B be events. We say that A and B are independent if the occurrence of A does not depend on the occurrence of B. Formally, A and B are independent if P (A) = P (A B) = In other words, A and B are independent if P (A B). P (B) P (A B) = P (A)P (B). Example Let S = {1, 2,..., 10}, A = {1, 2, 3, 4}, and B = {2, 4, 6, 8, 10}. Then P (A B) = 1 5, P (A) = 2 5, and P (B) = 1 2, so P (A B) = P (A)P (B) and we conclude that A and B are independent.

6 Addition and Multiplication Rules Let A and B be events (not necessarily independent). Since P (A B) = P (A B) P (B), it follows that By symmetry, so in general, we get P (A B) = P (B)P (A B). P (A B) = P (B A) = P (A)P (B A), P (A B) = P (B)P (A B) = P (A)P (B A).

7 Addition and Multiplication Rules, continued Let A and B be events. 1 In general, P (A B) = P (A) + P (B) P (A B). In particular, if A and B are disjoint, then P (A B) = P (A) + P (B). 2 In general, P (A B) = P (B)P (A B) = P (A)P (B A). In particular, if A and B are independent, then P (A B) = P (A)P (B).

8 Random Variable Definition A variable which can take different values with given probabilities is called a random variable. It is usually denoted by a capital letter such as X or Y. 1 X is called a discrete random variable if X takes discrete values (e.g. litter size, parity). 2 X is called a continuous random variable if X takes continuous values (e.g. height, weight, temperature).

9 Discrete Random Variable Example Suppose that X can take values 1, 2, 3, and 4 with probabilities 1 12, 1 2, 1 4, and 1 6, respectively. The values of X and the corresponding probabilities can be described using a table: x P (X = x) The function f given by f(x) = P (X = x) (e.g. f(3) = 1 4 in the example above) is called the probability mass function (pmf in short) of X. Note that P (X = x) = 1. x

10 Discrete Random Variable - Bernoulli Distribution Suppose X takes either 1 (with probability p) or 0 (with probability 1 p). We say that X follows Bernoulli distribution with success probability p. x 0 1 P (X = x) 1 p p Bernoulli distribution is useful when the outcome of an experiment is dichotomous (e.g. X = 1 if a subject has a certain disease and X = 0 if a subject has no disease).

11 Discrete Random Variable - Binomial Distribution Suppose that there is an experiment whose outcome is either success (with probability p) or failure (with probability 1 p). Suppose that this experiment is carried out n times and each result is independent of others. Let X be the number of success, then we say that X follows binom(n, p). x n P (X = x) (1 p) n np(1 p) n 1 n(n 1) 2 p 2 (1 p) n 2 p n In general, the pmf is given by P (X = x) = n! x!(n x)! px (1 p) n x, x = 0, 1, 2,..., n. It is important when dealing with proportion (e.g. among n animals, how many animals have a disease, if each animal either has or does not have the disease?)

12 Discrete Random Variable - Binomial Distribution Example Suppose a box has 6 red balls and 4 black balls. A ball is selected from the box 5 times with replacement. Let X denote the total number of red balls selected. What is the probability that 3 X 4? We note that X follows binom(5, 3 5 ), so P (3 X 4) =P (X = 3) + P (X = 4) = 5! 3!2! = ( 3 5 ) 3 ( ) ! 5 4!1! ( 3 5 ) 4 ( ) dbinom(3,size=5,prob=3/5)+dbinom(4,size=5,prob=3/5) ## [1]

13 Discrete Random Variable - Poisson Distribution Let λ > 0. Suppose that X can take values 0, 1, 2, 3,... with the corresponding probabilities described below: x n P (X = x) e λ e λ λ e λ λ 2 2 e λ λ n n! In this case, we say that X follows P oisson(λ). In general, the pmf is given by P (X = x) = e λ λ x x!, x = 0, 1, 2,.... It is important when dealing with the count of the number of events occurring randomly in a time at a constant rate λ (e.g. the number of phone calls received by a call center per day).

14 Continuous Random Variable Continuous random variables take continuous values, so they cannot be described using a table and pmf. Instead, probability density function (pdf in short) is used.

15 Integral Let f be a nonnegative function. Then b the curve y = f(x) between x = a and x = b. a f(x) dx means the area under b a f(x) dx a b x

16 Probability Density Function Let X be a continuous random variable. A function f is called the probability density function of X if for any a and b, the probability P (a X b) can be expressed by P (a X b) = b a f(x) dx. a P (a X b) b x

17 Probability Density Function, continued When X is a continuous random variable, P (X = a) = a a f(x) dx = 0. In particular, f(x) does NOT represent the probability P (X = a). In fact, f(x) could be greater than 1. f(x) dx = 1.

18 Normal Distribution A continuous random variable X is said to follow N(µ, σ 2 ) if its pdf f is given by f(x) = 1 ) (x µ)2 exp ( 2πσ 2σ 2. µ x Note that f is symmetric with respect to x = µ. µ determines the location of the curve y = f(x) and σ determines the shape of the curve (larger σ wider curve).

19 Normal Distribution, continued Example Suppose X follows N(2, 3 2 ). What is P (1 X 5)? Note that the pdf is given by f(x) = 1 ) ( 3 2π exp (x 2) ) 1 ( 1 3 2π exp (x 2)2 dx, the area of the yellow region, represents 18 the desired probability. x

20 Standard Normal Distribution N(0, 1 2 ) (that is, µ = 0 and σ = 1) is called the standard normal distribution. When Z follows N(0, 1 2 ), its pdf is given by P (Z 0) = P (Z 0) = 0 0 f(x) = 1 2π exp 1 2π exp 1 2π exp ( x2 ( x2 2 ) ( x2. 2 ) dx = ) dx = 1 2.

21 Calculuation with Standard Normal Distribution How to compute P (Z z) = example, let s calculate 1 P (Z 2.15) 2 P (Z 1.96) z ) 1 exp ( x2 dx in general. For 2π 2

22 Calculuation with Standard Normal Distribution, continued One can use a standard normal distribution table. P (Z 2.15) = and P (Z 1.96) = =

23 Calculuation with Standard Normal Distribution, continued There are various types of standard normal distribution tables. P (Z 2.15) = 1 P (Z > 2.15) = = and P (Z 1.96) = P (Z > 1.96) =

24 Calculuation with Standard Normal Distribution, continued Another type of standard normal table: Table entry z Standard Normal Probabilities Table entry for z is the area under the standard normal curve to the left of z z P (Z 2.15) = 1 P (Z 2.15) = = and P (Z 1.96) =

25 Calculuation with Standard Normal Distribution, continued Our textbook gives yet another table in pages Another approach: Use R. Note that the R command pnorm(z) gives P (Z z). pnorm(0) ## [1] 0.5 pnorm(2.15) ## [1] pnorm(-1.96) ## [1] 0.025

26 Calculuation with Standard Normal Distribution, continued Problem Suppose that Z follows N(0, 1 2 ). Compute 1 P (Z 1.96) 2 P ( 1.96 Z 2.15) Answer 1 P (Z 1.96) = P (Z 1.96) = P ( 1.96 Z 2.15) = P (Z 2.15) P (Z 1.96) =

27 Standardization of Normal Distribution The previous tables are only for standard normal distribution. How to compute probabilities involving a general normal distribution? Theorem Suppose that X follows N(µ, σ 2 ). Let Z = X µ σ, then Z follows N(0, 1 2 ).

28 Standardization of Normal Distribution, continued Example Suppose that X follows N(2, 3 2 ). What is P (1 X 5)? If we let Z = X 2 3, then Z follows N(0, 12 ) and ( 1 2 P (1 X 5) = P 3 X ) = P ( 13 ) 3 Z 1. So the probability involving a general normal distribution becomes a probability involving standard normal distribution.

29 Standardization of Normal Distribution, continued Example x x pnorm(1)-pnorm(-1/3) ## [1] ## CAN COMPUTE THE PROBABILITY WITHOUT STANDARDIZATION pnorm(5,mean=2,sd=3)-pnorm(1,mean=2,sd=3) ## [1]

30 Critical Value of Normal Distribution Example Suppose that Z follows N(0, 1 2 ). How to find z α such that P (Z z α ) = 0.8? 0.8 One can use qnorm() function in R. Note that z α = qnorm(p) if and only if P (Z z α ) = p. z α x qnorm(0.8) ## [1] pnorm(0.8416) ## [1] 0.8

31 Critical Value of Normal Distribution, continued Problem Suppose that Z follows N(0, 1 2 ). Find z α such that P ( z α Z z α ) = z α 0.95 z α x Answer qnorm( /2) ## [1] 1.96

32 Critical Value of Normal Distribution, continued Problem Suppose that X follows N(2, 3 2 ). Find x 0 such that P (X x 0 ) = x 0 x Answer Using standardization, we get P (Z x0 2 3 ) = 0.8. Solving x = , we get x 0 = = qnorm(0.8,mean=2,sd=3) ## [1] 4.525

33 Normal Approximation Suppose that X follows binom(n, p), so X is a discrete random variable that can take values 0, 1, 2,..., n. Recall that its pmf is given by P (X = x) = n! x!(n x)! px (1 p) n x, x = 0, 1, 2,..., n. By the Central Limit Theorem, if n is large enough, then X behaves like N(np, np(1 p)).

34 Normal Approximation, continued Example Suppose that the probability that a typical cat in a certain town has fleas is 1 3. If there are 100 cats in the town, what is the probability that at most 30 cats have fleas? Let X be the number of cats (among 100) that have fleas, then X follows binom(100, 1 3 ). The desired probability is P (X 30) = P (X = 0) + P (X = 1) + P (X = 2) + + P (X = 30) 30 = P (X = k) = k=0 30 k=0 100! k!(100 k)! ( 1 3 ) k ( ) 100 k 2. 3 sum(dbinom(0:30,size=100,prob=1/3)) ## [1]

35 Normal Approximation, continued Example It is almost impossible to compute 30 k=0 now use normal approximation: binom(100, 1 3 ) N( , ( 100! 1 ) k ( 2 ) 100 k. k!(100 k)! 3 3 We ) = N( 100 3, ) = N( 100 3, ( ) 2 ). pnorm(30,mean=100/3,sd=10*sqrt(2)/3) ## [1] One can further compare: c(sum(dbinom(0:20,size=100,prob=1/3)), sum(dbinom(0:40,size=100,prob=1/3))) ## [1] c(pnorm(20,mean=100/3,sd=10*sqrt(2)/3), pnorm(40,mean=100/3,sd=10*sqrt(2)/3)) ## [1]

36 t-distribution When sample size is small, we need to work with t-distributions. The shape of its pdf depends on its degrees of freedom. pdf of t distributions df=1 df=2 df=3 df= x

37 t-distribution, continued

38 χ 2 -Distribution χ 2 -distribution is used, for example, to test independence in various setting. The shape of its pdf depends on its degrees of freedom. pdf of chi square distributions df=1 df=2 df=3 df= x

39 χ 2 -Distribution, continued

40 F -Distribution F -distribution is used, for example, in ANOVA. The shape of its pdf depends on a pair of degrees of freedom. pdf of F distributions df1=1, df2=1 df1=2, df2=1 df1=5, df2=2 df1=30, df2= x

41 F -Distribution, continued

42 Population and Sample Suppose that we are interested in the average weight of all cats in Philadelphia. Instead of studying the whole population (which is often impossible due to economic and practical considerations), we use a sample. Then our inference on the whole population is based on what we get from the sample. Samples are used for parameter estimation and hypothesis testing. We start with estimation.

43 Sampling Distribution of Normal Population Assume that a population follows N(µ, σ 2 ), where µ and σ 2 are unknown numbers to be estimated. Suppose that we took a sample of size n, say X 1, X 2,..., X n, from the population, then each X i is a random variable. Define the sample mean X by X = X 1 + X X n. n We also define the the sample variance S 2 by S 2 = 1 n 1 n (X i X) 2. It is known that X (respectively, S 2 ) is a good estimator for µ (respectively, σ 2 ). i=1

44 Sampling Distribution of Normal Population, continued Problem Suppose that the following sample of size 25 was obtained from a weight data that follows a normal population: 0.5, 1.2, 4.3, 4.1, 5.4, 4.3, 2.1, 4.0, 0.2, 7.3, 3.2, 6.0, 5.4, 3.2, 2.7, 7.5, 4.3, 3.3, 5.9, 4.2, 1.7, 1.8, 3.8, 1.5, 4.2 Estimate the population mean µ and the population variance σ 2. Answer weight=c(0.5, 1.2, 4.3, 4.1, 5.4, 4.3, 2.1, 4.0, 0.2, 7.3, 3.2, 6.0, 5.4, 3.2, 2.7, 7.5, 4. mean(weight) ## [1] var(weight) ## [1] 3.773

45 Sampling Distribution of Normal Population, continued It follows that ˆµ = and ˆσ 2 = 3.773, so we infer that the normal population follows N(3.684, 3.773) based on our sample above. Of course, these estimates would have been different if we took a different sample. So a natural question follows: How accurate (or stable) is ˆµ = 3.684? The answer relies on the following theorem.

46 Sampling Distribution of Mean Theorem Assume that a population follows N(µ, σ 2 ), where µ is unknown and σ 2 is known. Suppose that we took a sample of size n, say X 1, X 2,..., X n, from ( the population, then X is a random variable that follows ( ) 2 σ N µ, n ). In particular, ( P X 1.96 σ µ X σ ) = n n This is, however, rarely used in practice, because we usually do not know σ. Instead, we use the following theorem.

47 Sampling Distribution of Mean, continued Theorem Assume that a population follows N(µ, σ 2 ), where µ and σ 2 are unknown. Suppose that we took a sample of size n, say X 1, X 2,..., X n, from the population, then X µ S/ is a random variable that follows a n t-distribution with n 1 degrees of freedom. In particular, ( ) S P X t α/2 n µ X S + t α/2 n = 1 α. Here t α/2 is determined so that P (T t α/2 ) = 1 α/2, where T follows a t-distribution with n 1 degrees of freedom. The interval ( X tα/2 S n, X + t α/2 S n ) is called the (1 α) 100 % confidence interval of µ.

48 Sampling Distribution of Mean, continued Example Consider the weight data again. We are interested in finding the 95% confidence interval of µ. So α = 0.05 and t α/2 = Therefore, the 95% confidence interval of µ is given by ( ) , = (2.882, 4.486) n=length(weight) alpha=0.05 t=qt(1-alpha/2,df=n-1) t ## [1] CI95=c(mean(weight)-t*sqrt(var(weight)/n),mean(weight)+t*sqrt(var(weight)/n)) CI95 ## [1]

49 Sampling Distribution of Mean, continued When the sample size n is large enough, the result above remains valid (approximately) even when the population does not necessarily follow a normal distribution (Central Limit Theorem).