N.I. Lobachevsky State Uiversity of Nizhi Novgorod Probability theory ad mathematical statistics: Law of Total Probability. Associate Professor A.V. Zorie Law of Total Probability. 1 / 14
Theorem Let H 1, H 2,..., H,... be a fiite or deumerable sequece of mutualually exclusive evets with positive probability each, ad A H 1 H 2... H.... The P(A) = P(H 1 )P(A H 1 ) + P(H 2 )P(A H 2 ) +... + P(H )P(A H ) +... Proof. Evets A H 1, A H 2,..., A H,... are mutually exclusive ad A = (A H 1 ) (A H 2 )... (A H ).... The P(A) = P((A H 1 ) (A H 2 )... (A H )...) = P(A H 1 ) + P(A H 2 ) +... + P(A H ) +... = P(H 1 )P(A H 1 ) + P(H 2 )P(A H 2 ) +... P(H )P(A H ) +... Law of Total Probability. 2 / 14
Example. There are m red balls ad blue balls i a ur. Two balls are take out oe by oe without replacemet. What s the probability the secod ball is blue (evet A)? Law of Total Probability. 3 / 14
Example. There are m red balls ad blue balls i a ur. Two balls are take out oe by oe without replacemet. What s the probability the secod ball is blue (evet A)? Solutio by classical formula We ca use classical formula here. A outcome is a arragemet of two balls without repetitios. Favorable outcome are those with the secod ball of the blue color. The first ball ca be either blue or red. N(Ω) = A 2 m+ = (m + )(m + 1), N(A) = ( 1) + m, So, P(A) = ( 1) + m (m + )(m + 1) = m + Law of Total Probability. 3 / 14
Example. There are m red balls ad blue balls i a ur. Two balls are take out oe by oe without replacemet. What s the probability the secod ball is blue (evet A)? Law of Total Probability. 4 / 14
Example. There are m red balls ad blue balls i a ur. Two balls are take out oe by oe without replacemet. What s the probability the secod ball is blue (evet A)? Solutio by law of total probability. Let evet H 1 occur whe the first ball is red ad H 2 occur whe the first ball is blue. P(H 1 ) = m m +, Law of Total Probability. 4 / 14
Example. There are m red balls ad blue balls i a ur. Two balls are take out oe by oe without replacemet. What s the probability the secod ball is blue (evet A)? Solutio by law of total probability. Let evet H 1 occur whe the first ball is red ad H 2 occur whe the first ball is blue. P(H 1 ) = m m +, P(A H 1) = m + 1 Law of Total Probability. 4 / 14
Example. There are m red balls ad blue balls i a ur. Two balls are take out oe by oe without replacemet. What s the probability the secod ball is blue (evet A)? Solutio by law of total probability. Let evet H 1 occur whe the first ball is red ad H 2 occur whe the first ball is blue. P(H 1 ) = P(H 2 ) = m m +, P(A H 1) = m +, m + 1 Law of Total Probability. 4 / 14
Example. There are m red balls ad blue balls i a ur. Two balls are take out oe by oe without replacemet. What s the probability the secod ball is blue (evet A)? Solutio by law of total probability. Let evet H 1 occur whe the first ball is red ad H 2 occur whe the first ball is blue. P(H 1 ) = P(H 2 ) = m m +, P(A H 1) = m + 1 m +, P(A H 2) = 1 m + 1 Law of Total Probability. 4 / 14
Example. There are m red balls ad blue balls i a ur. Two balls are take out oe by oe without replacemet. What s the probability the secod ball is blue (evet A)? Solutio by law of total probability. Let evet H 1 occur whe the first ball is red ad H 2 occur whe the first ball is blue. The P(H 1 ) = P(H 2 ) = m m +, P(A H 1) = m + 1 m +, P(A H 2) = 1 m + 1 P(A) = P(H 1 )P(A H 1 ) + P(H 2 )P(A H 2 ) = m + m m + 1 + m + 1 m + 1 = m + Law of Total Probability. 4 / 14
Example. There 5 urs, two of them cotai 2 red balls ad 1 blue ball, aother two cotai 3 red balls ad 1 blue ball, ad the last oe cotais 10 blue balls. A ur is picked at radom ad oe ball is take out. What s the probability that the ball is red? Law of Total Probability. 5 / 14
Example. There 5 urs, two of them cotai 2 red balls ad 1 blue ball, aother two cotai 3 red balls ad 1 blue ball, ad the last oe cotais 10 blue balls. A ur is picked at radom ad oe ball is take out. What s the probability that the ball is red? Solutio. Let evet H 1 occur whe oe of the first two urs is chose, H 2 occur whe the third or the fourth urs is chose, H 3 occur whe the fifth ur is chose. P(H 1 ) = 2 5, P(H 2) = 2 5, P(H 3) = 1 5, P(A H 1) = 2 3, P(A H 2 ) = 3 4, P(A H 3) = 0. The P(A) = 2 5 2 3 + 2 5 3 4 + 1 5 0 = 17 30 Law of Total Probability. 5 / 14
There are m red balls ad blue balls i a ur. r balls are take out oe by oe without replacemet, r m +. Prove that the probability for the last ball to be blue is m+. Law of Total Probability. 6 / 14
There are m red balls ad blue balls i a ur. r balls are take out oe by oe without replacemet, r m +. Prove that the probability for the last ball to be blue is m+. Deote by π(r; m, ) the probability i questio. It s obvious that Now assume that π(1; m, ) = m +. π(2; m, ) = π(3; m, ) =... = π(r 1; m, ) = We have to prove that π(r; m, ) = m + m + Law of Total Probability. 6 / 14
Let evet H 1 occur whe the first ball is red ad H 2 occur whe the first ball is blue. P(H 1 ) = m + m, P(H 2) = m + Law of Total Probability. 7 / 14
Let evet H 1 occur whe the first ball is red ad H 2 occur whe the first ball is blue. P(H 1 ) = m + m, P(H 2) = m + If H 1 occurs, the oly + m 1 balls left, of them are blue, ad we still have to take r 1 balls out, so the coditioal probability that the last ball is blue give that the first is red equals π(r 1; m 1, ). Law of Total Probability. 7 / 14
Let evet H 1 occur whe the first ball is red ad H 2 occur whe the first ball is blue. P(H 1 ) = m + m, P(H 2) = m + If H 1 occurs, the oly + m 1 balls left, of them are blue, ad we still have to take r 1 balls out, so the coditioal probability that the last ball is blue give that the first is red equals π(r 1; m 1, ). If H 2 occurs, the oly + m 1 balls left, 1 of them are blue, ad we are to take r 1 balls out, so the coditioal probability that the last ball is blue give that the first is blue equals π(r 1; m, 1). Law of Total Probability. 7 / 14
By assumptio, π(r 1; m 1, ) = m + 1, π(r 1; m, 1) = 1 m + 1. Accordig to the law of total probability, π(r; m, ) = P(H 1 )π(r 1; m 1, ) + P(H 2 )π(r 1; m, 1) = m + m m + 1 + m + 1 m + 1 = m + Law of Total Probability. 8 / 14
Rui problem Law of total probability Assume A ad Bart play Heads ad Tails. If A wis, she pays oe rouble to Bart. If Bart wis, he pays oe rouble to A. I the begiig A has a roubles, Bart has b roubles. What s the probability A loses all her moey? A s rui meas that A has o more roubles. The same holds for Bart s rui. Law of Total Probability. 9 / 14
Rui problem Law of total probability Assume A ad Bart play Heads ad Tails. If A wis, she pays oe rouble to Bart. If Bart wis, he pays oe rouble to A. I the begiig A has a roubles, Bart has b roubles. What s the probability A loses all her moey? A s rui meas that A has o more roubles. The same holds for Bart s rui. The total sum of roubles is a + b. Deote by π(x) probabilty of A s rui havig x roubles iitially. If she wis a toss, her fortue becomes x + 1 roubles, otherwise x 1 roubles. She wis a toss with probability 1 2 ad loses with probability 1 2. If has x roubles ad she wis a toss, whether she gets ruied or ot is defied oly by the future tosses which are idepedet of the past tosses, so the rui probability after that is π(x + 1). Law of Total Probability. 9 / 14
By the total probability formula, besides that, From (1), for a costat c, ad Usig (2), obtai π(a) = 1 2 π(x + 1) + 1 π(x 1) (1) 2 π(0) = 1 ad π(a + b) = 0. (2) π(x) π(x 1) = π(x + 1) π(x) = c π(x) = π(0) + xc. π(x) = 1 x a + b Thus, A gets ruied with probability π(a) = b a+b. Law of Total Probability. 10 / 14
A problem i rus I ay ordered sequece of elemets of two kids, each maximal subsequece of elemets of like kid is called a ru. For example, the sequece xxxyxxyyyx opes with a x ru of legth 3; it is followed by rus of legth 1, 2, 3, 1, respectively. Law of Total Probability. 11 / 14
A problem i rus I ay ordered sequece of elemets of two kids, each maximal subsequece of elemets of like kid is called a ru. For example, the sequece xxxyxxyyyx opes with a x ru of legth 3; it is followed by rus of legth 1, 2, 3, 1, respectively. Let α ad β be positive itegers, ad cosider a potetially ulimited sequece i idepedet trials, such as tossig a coi or throwig dice. Symbol x occurs i a sigle trial with probability p, symbol y occurs i a sigle trial with probability q = 1 p. What s the probability that a ru of α cosecutive x s will occur before a ru of β cosecutive y s? Law of Total Probability. 11 / 14
Sice the trials are idepedet, the probability of the sequece xxxyxxyyyx equals p p p q p p q q q p = p 6 q 4. Law of Total Probability. 12 / 14
Sice the trials are idepedet, the probability of the sequece xxxyxxyyyx equals p p p q p p q q q p = p 6 q 4. Let A be the evet that a ru of α cosecutive x s occurs before a ru of β cosecutive y s. Deote x = P(A). Let The u = P(A the first trial results i x ), v = P(A the first trial results i y ). x = pu + qv. Law of Total Probability. 12 / 14
Suppose that the first trial results i x. I this case the evet A ca occur i α mutually exclusive ways: 1 The followig α 1 trials result i x s; the probability for this is p α 1. Law of Total Probability. 13 / 14
Suppose that the first trial results i x. I this case the evet A ca occur i α mutually exclusive ways: 1 The followig α 1 trials result i x s; the probability for this is p α 1. 2 The first y occurs at the rth trial where 2 r α. Let this evet be H r. The P(H r ) = p r 2 q, ad P(A H r ) = v. Hece u = p α 1 + qv(1 + p +... + p α 2 ) Law of Total Probability. 13 / 14
Suppose that the first trial results i x. I this case the evet A ca occur i α mutually exclusive ways: 1 The followig α 1 trials result i x s; the probability for this is p α 1. 2 The first y occurs at the rth trial where 2 r α. Let this evet be H r. The P(H r ) = p r 2 q, ad P(A H r ) = v. Hece u = p α 1 + qv(1 + p +... + p α 2 ) = p α 1 + v(1 p α ). Law of Total Probability. 13 / 14
Suppose that the first trial results i x. I this case the evet A ca occur i α mutually exclusive ways: 1 The followig α 1 trials result i x s; the probability for this is p α 1. 2 The first y occurs at the rth trial where 2 r α. Let this evet be H r. The P(H r ) = p r 2 q, ad P(A H r ) = v. Hece u = p α 1 + qv(1 + p +... + p α 2 ) = p α 1 + v(1 p α ). If the first trial results i y, a similar argumet leads to v = pu(1 + q +... + q β 2 ) Law of Total Probability. 13 / 14
Suppose that the first trial results i x. I this case the evet A ca occur i α mutually exclusive ways: 1 The followig α 1 trials result i x s; the probability for this is p α 1. 2 The first y occurs at the rth trial where 2 r α. Let this evet be H r. The P(H r ) = p r 2 q, ad P(A H r ) = v. Hece u = p α 1 + qv(1 + p +... + p α 2 ) = p α 1 + v(1 p α ). If the first trial results i y, a similar argumet leads to v = pu(1 + q +... + q β 2 ) = q(1 q β 1 ). Law of Total Probability. 13 / 14
We have two equatios for the two ukow u ad v: u = p α 1 + v(1 p α ), v = qu(1 q β 1 ). Law of Total Probability. 14 / 14
We have two equatios for the two ukow u ad v: Solutio is u = p α 1 + v(1 p α ), u = v = v = qu(1 q β 1 ). p α q pq β + p α (q q β ), p α (q q β ) pq β + p α (q q β ). Law of Total Probability. 14 / 14
We have two equatios for the two ukow u ad v: u = p α 1 + v(1 p α ), v = qu(1 q β 1 ). Solutio is u = v = p α q pq β + p α (q q β ), p α (q q β ) pq β + p α (q q β ). Fially, x = pu + qv = p α 1 1 p α p α 1 + q β 1 p α 1 q β 1 Law of Total Probability. 14 / 14
We have two equatios for the two ukow u ad v: u = p α 1 + v(1 p α ), v = qu(1 q β 1 ). Solutio is u = v = p α q pq β + p α (q q β ), p α (q q β ) pq β + p α (q q β ). Fially, x = pu + qv = p α 1 1 p α p α 1 + q β 1 p α 1 q β 1 For example, i tossig a coi (p = 1 2 ) the probability that a ru of two heads appears before a ru of three tails is 0,7. Law of Total Probability. 14 / 14