Intro to Probability Instructor: Alexandre Bouchard

Transcription

1 Intro to Probability Instructor: Alexandre Bouchard

2 Announcements Graded midterm available after lecture Webwork due tonight

3 Regrading policy IF you would like a partial regrading, you should, BEFORE or ON Friday March 15 Tuesday 20th, hand in to me at the beginning of a lecture: your exam a clean piece of paper stapled to it that clearly (i) explains the question(s) you would like us to regrade AND (ii) the issue(s) you would like to raise NOTE: for fairness, the new grade for the question could stay the same, increase, or, in certain cases, decrease (except if the request is limited to the points mentioned in last Friday mass )

4 Plan for today Sum of continuous random variables Conditional densities

5 Ex 53 Review: transformations Suppose I tell you that is the distribution of Richter scales What is the distribution of the amplitudes? For simplicity: Assume Richter scale X ~ Uniform(0, 1) What is the distribution of Y = exp(x)?

6 Review: recipe for transformations Suppose I tell you that is the distribution of Richter scales What is the distribution of the amplitudes? For simplicity: Assume Richter scale X ~ Uniform(0,1) What is the distribution of exp(x)? Recipe for finding the distribution of transforms of r.v. s 1 Find the CDF Density fx Richter: Amplitude: Differentiate to find the density

7 Review: recipe for transformations Suppose I tell you that is the distribution of Richter scales What is the distribution of the amplitudes? For simplicity: Assume Richter scale X ~ Uniform(0,1) What is the distribution of exp(x)? 1 Find the CDF F Y (y) =P (exp(x) apple y) = P (X apple log(y)) Why? = F X (log(y)) = 1 [1,e] (y) log(y)

8 Why P(exp(X) y) = P(X log(y)) Because (exp(x) y) = (X log(y)), which is true because: log is increasing, i.e. x 1 x 2 iff log(x 1 ) log(x 2 ) this means I can take log on both sides of the inequality: (exp(x) y) = (log(exp(x)) log(y)) log/exp are invertible: log(exp(z)) = z, so (log(exp(x)) log(y)) = (X log(y))

9 Review: recipe for transformations Suppose I tell you that is the distribution of Richter scales What is the distribution of the amplitudes? For simplicity: Assume Richter scale X ~ Uniform(0,1) What is the distribution of exp(x)? 2 Differentiate to find the density f Y (y) = df Y (y) dy at points where FY is differentiable =1 [1,e] (y) 1 y

10 Sums of independent discrete random variables (exact method)

11 Sum of independent r.v.s: summary Approximations: Central limit theorem (Normal approximation) Use software/ppl Exact methods: Binomial distribution (works only for sum of Bernoullis) Today: general, exact method CONVOLUTIONS

12 Ex 68 Simple example X: outcome of white dice Y: outcome of black dice Example: computing P(X + Y = 4)

13 Simple example

14 Application Not convinced? Play this game: Settler of Catan

15 Prop 16 General formula for discrete r.v.s If: Z = X + Y Then: p Z (z) = y = p X (z y) p Y (y).

16 Sums of independent continuous random variables

17 Sum of continuous r.v.s X: a continuous r.v. with density fx Y: a continuous r.v. with density fy Assume they are indep: f(x, y) = fx(x) fy(y) What is the density fz of the sum Z = X + Y? Recipe for finding the distribution of transforms of r.v. s 1 Find the CDF Density fx Richter: Amplitude: Differentiate to find the density

18 Ex 69 Example Let X and Y be independent and both uniform on [0, 1] y What is the density fz of the sum Z = X + Y? x

19 Example Let X and Y be independent and both uniform on [0, 1] y 1 Find the CDF What is the density fz of the sum Z = X + Y? x FZ(z) = P(Z z) example: z = 1 P( Z 1 ) = P( X + Y 1 ) =?

20 Example Let X and Y be independent and both uniform on [0, 1] y 1 Find the CDF P(Z z) for all z What is the density fz of the sum Z = X + Y? x example: z = 1 P( Z 1 ) = P( X + Y 1 ) y A = {(x,y) : x + y 1} = P( (X, Y) A ) = = Z A Z 1 = 1/2 f(x, y) dx dy Z x f(x, y) dy dx x

21 Example Let X and Y be independent and both uniform on [0, 1] y 1 Find the CDF P(Z z) for all z x What is the density fz of the sum Z = X + Y? P( Z z ) = P( X + Y z ) = Definition of the = CDF F(y) = Z 1 Z z 1 1 Z 1 1 Z 1 1 f X (x) x f X (x)f Y (y) dy Z z 1 f X (x)(f Y (z x f Y (y) dy x)) dx dx dx

22 Example Let X and Y be independent and both uniform on [0, 1] y x What is the density fz of the sum Z = X + Y? 1 Find the CDF Differentiate to find the 2 density FZ(z) = P( Z z ) f Z (z) = df Z(z) = dz Under regularity conditions, you = can interchange integrals and derivatives = = Z 1 1 Z 1 1 Z 1 1 Z 1 1 f X (x)f Y (z x)dx Chain rule of f X (x) df Y (z x) calculus dx dz d f X (x)f Y (z x) (z x) dx dz f X (x)f Y (z x)dx

23 Prop 16b Sum of continuous r.v.s X: a continuous r.v. with density fx Y: a continuous r.v. with density fy What is the density fz of the sum Z = X + Y? f Z (z) = = f X (z y) f Y (y) dy f X (x) f Y (z x) dx Terminology: convolution

24 Ex 69 Let X and Y be independent and both uniform on [0, 1] y What is the density fz of the sum Z = X + Y? x Note: Not equal to the sum of the densities!!!

25 Conditional densities

26 Def 26* Conditional PMF and density Conditional PMF given y p X Y (x y) = joint PMF marginal PMF = p(x, y) p Y (y) Conditional density given y f X Y (x y) = joint density marginal density = f(x, y) f Y (y) * if denominator is non-zero

27 Prop 17a Rewriting chain rule For any events A, B, with P(A) > 0: P( A, B) = P(A) P(B A) p(x, y) =p X (x)p Y X (y x) Correspondence? A = (X = x), B = (Y = y)

28 Prop 17b Rewriting Bayes rule P (H E) = P (H)P (E H) P (E) H: hypothesis (unknown), E: evidence/observation p Z X (z x) = p Z(z)p X Z (x z) p Y (y) Correspondence? H = (Z = z), E = (X = x)

29 Prop 17c Density versions Chain rule Bayes rule densities PMFs Events P (A, B) =P (A)P (B A) p(x, y) =p X (x)p Y X (y x) f(x, y) =f X (x)f Y X (y x) P (H E) = P (H)P (E H) P (E) p Z X (z x) = p Z(z)p X Z (x z) p X (x) f Z X (z x) = f Z(z)f X Z (x z) f X (x)

30 Usual warning f and p behave similarly in formulas (replacing sums by integrals) BUT: as always, f(x, y), fx(x), fy(y) and fx Y(x y) are NOT probabilities. We integrate over a region to get probabilities For fx(x), fy(y) and fx Y(x y), use a single integral For f(x, y), use a double integral

31 Example: Using conditioning to predict the number of future members of the human species

32 Ex 72 Simple problem I have a measuring tape, but you do not know how long is it. Length of tape: Z I go in a separate room, unroll it fully, and pick a number at random from the tape. Random point on tape: Y If I tell you Y, how should we optimally guess Z?

33 Ex 72 Model I have a measuring tape, but you do not know how long is it. Z ~ Unif(0, 5) Length of tape: Z Let s say we think it s less than 5m I go in a separate room, unroll it fully, and pick a number at random from the tape. Y Z ~ Unif(0, Z) Random point on tape: Y

34 Ex 72 More dramatic version: how to predict the number of future members of the human species? I have a measuring tape, but you do not know how long is it (Z). Total number of humans to ever live, future and past (in trillion) I go in a separate room, unroll it fully, and pick a number (Y) at random from the tape. Number of humans that were born before present (from archeological records, ~0.06 trillion) If I tell you Y, how should we optimally guess Z? Can we guess (probabilistically) how many more human there will be?

35 Conditional probability: continuous case Beliefs before new info (prior) fz(z) Conditioning Updated beliefs fz Y(z y) New information (observation): a fixed point y

36 Ex 72 Exercises: see handout Write fz(z) and fy Z(y z) Write f(z,y) Compute fy(y) Z ~ Unif(0, 5) Y Z ~ Unif(0, Z) Observed: Y 0.06 Compute fz Y(z y) Compute the conditional expectation: Z 1 E[Z Y ]= 1 zf Z Y (z Y )dz

37 Useful formulas for continuous random variables Marginalization Z +1 Uniform density U Unif(a, b) f X (x) = 1 f(x, y) dy f U (u) = 1 (a,b)(u) b a Conditional density given y f X Y (x y) = joint density marginal density = f(x, y) f Y (y)

38 Ex 72a Z ~ Unif(0, 5) Joint density? Y Z ~ Unif(0, Z) Observed: Y 0.06 A. 1 [y,5] (z) y 5 1 [0,z] (y) z Hint: B. 1 [0,5] (y) 5 1 [0,y] (z) y C. D. 1 [0,5] (z) 5 1 [0,5] (z) 5 1 [0,z] (y) z 1 [0,5] (y) 5

39 Ex 72a Z ~ Unif(0, 5) Joint density? Y Z ~ Unif(0, Z) Observed: Y 0.06 A. 1 [y,5] (z) y 5 1 [0,z] (y) z Hint: B. 1 [0,5] (y) 5 1 [0,y] (z) y C. D. 1 [0,5] (z) 5 1 [0,5] (z) 5 1 [0,z] (y) z 1 [0,5] (y) 5

40 Z ~ Unif(0, 5) Joint density Y Z ~ Unif(0, Z) Observed: Y (0,5) (z) 5 1 (0,z) (y) z f(z,y){ { { fz(z) fy Z(y z)

41 Ex 72b Marginal of Y, fy(y) For 0 < y < 5: Z ~ Unif(0, 5) Y Z ~ Unif(0, Z) Observed: Y 0.06 A. 1 5 (log 5 log y) B. log 5 log y C. 2 2 y 2 25 D y 2 25

42 Ex 72b Marginal of Y, fy(y) For 0 < y < 5: Z ~ Unif(0, 5) Y Z ~ Unif(0, Z) Observed: Y 0.06 A. 1 5 (log 5 log y) B. log 5 log y C. 2 2 y 2 25 D y 2 25

43 Posterior density, f Z Y (z y) f Z Y (z y) = 1 (0,5)(z)1 (0,z) (y) z(log(5) log y) At y = 0.06, get: 1.5 Density of Z Approximation of f(x) x

44 Carter catastrophe Density of Z At y = 0.06, get: 1.5 Approximation of f(x) x - Does not mean humanity will come to an end! Why? Assumptions (e.g. that our birth rank should be viewed uniform among all human births) still hotly debated - Choice of prior on Z: are we over-pessimistic/optimistic by assuming a uniform prior density on [0,5]? - However, note that the math is solid (think about the measuring tape example if uncomfortable with Carter s assumptions) Brandon Carter; McCrea, W. H. (1983). "The anthropic principle and its implications for biological evolution".philosophical Transactions of the Royal Society of London. A310 (1512): doi: /rsta

45 Ex 72c Conditional expectation Z ~ Unif(0, 5) Y Z ~ Unif(0, Z) Observed: Y 0.06 f Z Y (z y) = 1 (0,5)(z)1 (0,z) (y) z(log(5) log y) At y = E[Z Y ]= Z 1 1 zf Z Y (z Y )dz... A. 1.5 B C D

46 Ex 72c Conditional expectation Z ~ Unif(0, 5) Y Z ~ Unif(0, Z) Observed: Y 0.06 f Z Y (z y) = 1 (0,5)(z)1 (0,z) (y) z(log(5) log y) At y = E[Z Y ]= Z 1 1 zf Z Y (z Y )dz... A. 1.5 B C D