Practice Problem - Skewness of Bernoulli Random Variable. Lecture 7: Joint Distributions and the Law of Large Numbers. Joint Distributions - Example

Transcription

1 A little more E(X Practice Problem - Skewness of Bernoulli Random Variable Lecture 7: and the Law of Large Numbers Sta30/Mth30 Colin Rundel February 7, 014 Let X Bern(p We have shown that E(X = p Var(X = p(1 p Find the Skewness of X where skewness is defined as ( (X E(X 3 E = E ( (X µ 3 SD(X σ 3 Sta30/Mth30 (Colin Rundel Lecture 7 February 7, / - Example Draw two socks at random, without replacement, from a drawer full of twelve colored socks: black, 4 white, purple Let B be the number of Black socks, W the number of White socks drawn, then the distributions of B and W are given by: P(B=k P(W=k 8 Note - B HyperGeo(1,, = = 1 11 = = = 3 ( ( k k = = ( 1 and W HyperGeo(1, 4, = ( ( 4 8 k k ( 1 - Example, cont. Let B be the number of Black socks, W the number of White socks drawn, then the distributions of B and W are given by: 0 1 B 1 1 W P(B = b, W = w = 3 ( ( 4 b w( ( 1 b w Sta30/Mth30 (Colin Rundel Lecture 7 February 7, 014 / Sta30/Mth30 (Colin Rundel Lecture 7 February 7, /

2 Marginal Distribution Conditional Distribution Note that the column and row sums are the distributions of B and W respectively. P(B = b = P(B = b, W = 0 + P(B = b, W = 1 + P(B = b, W = P(W = w = P(B = 0, W = w + P(B = 1, W = w + P(B =, W = w Conditional distributions are defined as we have seen previously with P(X = x Y = y = P(X = x, Y = y P(Y = y = joint marginal Therefore the pmf for white socks given no black socks were drawn is These are the marginal distributions of B and W. In general, P(X = x = P(X = x, Y = y = P(X = x Y = yp(y = y P(W = w B = 0 = P(W = w, B = 0 P(B = 0 = / 1 = 1 / 8 = 8 if W = 0 if W = 1 / = if W = = P(X = x, Y = y dy = P(X = x Y = yp(y = y dy Sta30/Mth30 (Colin Rundel Lecture 7 February 7, / Sta30/Mth30 (Colin Rundel Lecture 7 February 7, / Expectation of Discrete E ( g(x, Y = x g(x, yp(x = x, Y = y For example we can define g(x, y = x y then y E(BW =(0 0 1/ + (0 1 8/ + (0 / + (1 0 1/ + (1 1 4/ + (1 0/ + ( 0 / + ( 1 0/ + (1 0/ =4/ = 4/11 Expectation of Discrete Conditional Distribution Works like any other discrete distribution E(X Y = y = x P(X = x Y = y x Therefore we can calculating things like conditional means and variances, E(W B = 0 = 0 1/ + 1 8/ + / = 0/ = Note that E(BW E(BE(W since E(BE(W = (0 / + 1 3/ + / (0 8/ + 1 3/ + / = / 44/ = /3 This implies that B and W are not independent and Cov(X, Y 0. Sta30/Mth30 (Colin Rundel Lecture 7 February 7, 014 / E(W B = 0 = 0 1/ + 1 8/ + / = 3/ =.1333 Var(W B = 0 = E(W B = 0 E(W B = 0 = 3/ (4/3 = 1/45 = Sta30/Mth30 (Colin Rundel Lecture 7 February 7, /

3 Joint Distribution - Example Joint Distribution - Example Suppose that X and Y have a discrete joint distribution for which the joint pmf is defined as follows: { c x + y for x, y {, 1, 0, 1, } f (x, y = 0 otherwise Suppose that X and Y have a discrete joint distribution for which the joint pmf is defined as follows { 1 f (x, y = 30 (x + y for x = 0, 1, and y = 0, 1,, 3 0 otherwise a What is the value of the constant c b P(X = 0 and Y = c P(X = 1 a Determine the marginal pmf s of X and Y. b Are X and Y independent? d P(X = 1 Y = 0 e P( X Y 1 From De Groot and Schervish (011 From De Groot and Schervish (011 Sta30/Mth30 (Colin Rundel Lecture 7 February 7, / Sta30/Mth30 (Colin Rundel Lecture 7 February 7, / Multinomial Distribution Multinomial Example Let X 1, X,, X k be the k random variables that reflect the number of outcomes belonging to category k in n trials with the probability of success for category k being p k, X 1,, X k Multinom(n, p 1,, p k P(X 1 = x 1,, X k = x k = f (x 1,, x k n, p 1,, p k where n! = x 1! x k! px 1 1 px k k k k x i = n and p i = 1 Some regions of DNA have an elevated amount of GC relative to AT base pairs. If in a normal region of DNA we expect equal amounts of ACGT vs a GC rich region which has twice as much GC as AT. If we observe the following sequence ACTGACTTGGACCCGACGGA what is the probability that it is from a normal region or a GC rich region. E(X i = np i Var(X i = np i (1 p i Cov(X i, X j = np i p j Sta30/Mth30 (Colin Rundel Lecture 7 February 7, / Sta30/Mth30 (Colin Rundel Lecture 7 February 7, /

4 Markov s Inequality For any random variable X 0 and constant a > 0, then Derivation of Markov s Inequality Let X be a random variable such that X 0 then P(X a E(X a Corollary - Chebyshev s Inequality: P( X E(X a Var(X a The inequality says that the probability that X is far away from its mean is bounded by a quantity that increases as Var(X increases. Sta30/Mth30 (Colin Rundel Lecture 7 February 7, / Sta30/Mth30 (Colin Rundel Lecture 7 February 7, / Derivation of Chebyshev s Inequality Proposition - if f (x is a non-decreasing function then ( P(X a = P f (X f (a E( f (X f (a If we define the positive valued random variable to be X E(X and f (x = x then Chebyshev s Inequality - Example Use Chebyshev s inequality to make a statement about the bounds for the probability of being with in 1,, or 3 standard deviations of the mean for all random variables. If we define a = kσ where σ = Var(X then P( X E(X kσ Var(X k σ = 1 k Sta30/Mth30 (Colin Rundel Lecture 7 February 7, / Sta30/Mth30 (Colin Rundel Lecture 7 February 7, 014 /

5 Independent and Identically Distributed (iid A collection of random variables that share the same probability distribution and all are mutually independent. Example If X Binom(n, p then X = n Y i where Y 1,, Y n iid Bern(p Sums of iid Random Variables Let X 1, X,, X n iid D where D is some probability distribution with E(X i = µ and Var(X i = σ. We defined S n = X 1 + X + + X n E(S n = E(X 1 + X + + X n = E(X 1 + E(X + + E(X n = µ + µ + + µ = nµ Var(S n = E[((X 1 + X + + X n (µ + µ + + µ ] = E[((X 1 mu + (X µ + + (X n µ ] n n n = E[(X i µ ] + E[(X i µ(x j µ] j=1 i j n n n = Var(X i + Cov(X i, X j = nσ j=1 i j Sta30/Mth30 (Colin Rundel Lecture 7 February 7, / Sta30/Mth30 (Colin Rundel Lecture 7 February 7, / Average of iid Random Variables Let X 1, X,, X n iid D where D is some probability distribution with E(X i = µ and Var(X i = σ. Weak Based on these results and Markov s Inequality we can show the following: We defined X n = (X 1 + X + + X n /n then E( X n = E(S n /n = E(S n /n = µ Var( X n = Var(S n /n = 1 n Var(S n Therefore, as long as σ < = nσ n = σ n lim P( X n µ ɛ = 0 lim P( X n µ < ɛ = 1 n n Sta30/Mth30 (Colin Rundel Lecture 7 February 7, / Sta30/Mth30 (Colin Rundel Lecture 7 February 7, /

6 Weak ( X n converges in probability to µ: lim P( X n µ > ɛ = 0 n Strong ( X n converges almost surely to µ: ( P X n = µ = 1 lim n LLN - Example How large a random sample must be taken from a given distribution in order for the probability to be at least 0.99 that the sample mean will be within standard deviations of the mean of the distribution? What about 0.95 probability to be within 1 standard deviations of the mean? Strong LLN is a more powerful result (Strong LLN implies Weak LLN, but its proof is more complicated. Sta30/Mth30 (Colin Rundel Lecture 7 February 7, / Sta30/Mth30 (Colin Rundel Lecture 7 February 7, / LLN and CLT Law of large numbers shows us that S n nµ lim = lim n n ( X n µ 0 n which shows that for large n, n S n nµ. What happens if we divide by something that grows slower than n like n? S n nµ lim = lim n( Xn µ d N(0, σ n n n This is the Central Limit Theorem, of which the DeMoivre-Laplace theorem for the normal approximation to the binomial is a special case. Hopefully by the end of this class we will have the tools to prove this. Sta30/Mth30 (Colin Rundel Lecture 7 February 7, 014 /