f X (12) = Pr(X = 12) = Pr({(6, 6)}) = 1/36

Size: px

Start display at page:

Download "f X (12) = Pr(X = 12) = Pr({(6, 6)}) = 1/36"

Bryan Wright
5 years ago
Views:

1 Probability Distributios A Example With Dice If X is a radom variable o sample space S, the the probablity that X takes o the value c is Similarly, Pr(X = c) = Pr({s S X(s) = c} Pr(X c) = Pr({s S X(s) c} This makes sese sice the rage of X is the real umbers Example: I the coi example, Pr(#H = 2) = 4/9 ad Pr(#H ) = 5/9 Give a probability measure Pr o a sample space S ad a radom variable X, the probability distributio associated with X is f X (x) = Pr(X = x) f X is a probability measure o the real umbers The cumulative distributio associated with X is F X (x) = Pr(X x) Suppose S is the sample space correspodig to tossig a pair of fair dice: {(i, j) i,j 6} Let X be the radom variable that gives the sum: X(i,j) = i + j f X (2) = Pr(X = 2) = Pr({(, )}) = /36 f X (3) = Pr(X = 3) = Pr({(, 2), (2, )}) = 2/36 f X (7) = Pr(X = 7) = Pr({(, 6), (2, 5),,(6, )}) = 6/36 f X (2) = Pr(X = 2) = Pr({(6, 6)}) = /36 Ca similarly compute the cumulative distributio: F X (2) = f X (2) = /36 F X (3) = f X (2) + f X (3) = 3/36 F X (2) = 2 The Fiite Uiform Distributio The Biomial Distributio The fiite uiform distributio is a equiprobable distributio If S = {x,,x }, where x < x 2 < < x, the: f(x k ) = / F(x k ) = k/ Suppose there is a experimet with probability p of success ad thus probability q = p of failure For example, cosider tossig a biased coi, where Pr(h) = p Gettig heads is success, ad gettig tails is failure Suppose the experimet is repeated idepedetly times For example, the coi is tossed times This is called a sequece of Beroulli trials Key features: Oly two possibilities: success or failure Probability of success does ot chage from trial to trial The trials are idepedet 3 4

2 What is the probability of k successes i trials? Suppose = 5 ad k = 3 How may sequeces of 5 coi tosses have exactly three heads? hhhtt hhtht hhtth C(5, 3) such sequeces! What is the probability of each oe? p 3 ( p) 2 Therefore, probability is C(5, 3)p 3 ( p) 2 Let B,p (k) be the probability of gettig k successes i Beroulli trials with probability p of success B,p (k) = C(,k)p k ( p) k Not surprisigly, B,p is called the Biomal Distributio The Poisso Distributio A large call ceter receives, o average, λ calls/miute What is the probability that exactly k calls come durig a give miute? Uderstadig this probability is critical for staffig! Similar issues arise if a priter receives, o average λ jobs/miute, a site gets λ hits/miute, This is modelled well by the Poisso distributio with parameter λ: f λ (k) = e λλk f λ (0) = e λ f λ () = e λ λ f λ (2) = e λ λ 2 /2 e λ is a ormalizatio costat, sice + λ + λ 2 /2 + λ 3 /3! + = e λ 5 6 Derivig the Poisso New Distributios from Old Poisso distributio = limit of biomial distributios Suppose at most oe call arrives i each secod Sice λ calls come each miute, expect about λ/60 each secod The probability that k calls come is B 60,λ/60 (k) This model does t allow more tha oe call/secod What s so special about 60? Suppose we divide oe miute ito time segmets Probability of gettig a call i each segmet is λ/ Probability of gettig k calls i a miute is B,λ/ (k) = C(, k)(λ/) k ( λ ) k = C(, k) = λk! ( k)! Now let : ( ) lim λ = e λ (! lim ) k ( k)! λ = λ/ λ ( λ k ( λ ) k ( λ Coclusio: lim B,λ/ (k) = e λλk 7 ) ) If X ad Y are radom variables o a sample space S, so is X + Y, X + 2Y, XY, si(x), etc For example, (X + Y )(s) = X(s) + Y (s) si(x)(s) = si(x(s)) Note si(x) is a radom variable: a fuctio from the sample space to the reals 8

3 Some Examples Example : A fair die is rolled Let X deote the umber that shows up What is the probability distributio of Y = X 2? {s : Y (s) = k} = {s : X 2 (s) = k} = {s : X(s) = k} {s : X(s) = k} Coclusio: f Y (k) = f X ( k) + f X ( k) So f Y () = f Y (4) = f Y (9) = f Y (36) = /6 f Y (k) = 0 if k / {, 4, 9, 6, 25, 36} Example 2: A coi is flipped Let X be if the coi shows H ad - if T Let Y = X 2 I this case Y, so Pr(Y = ) = Example 3: If two dice are rolled, let X be the umber that comes up o the first dice, ad Y the umber that comes up o the secod Formally, X((i,j)) = i, Y ((i,j)) = j The radom variable X +Y is the total umber showig Example 4: Suppose we toss a biased coi times (more geerally, we perform Beroulli trials) Let X k describe the outcome of the kth coi toss: X k = if the kth coi toss is heads, ad 0 otherwise How do we formalize this? What s the sample space? Notice that Σ k=x k describes the umber of successes of Beroulli trials If the probability of a sigle success is p, the Σ k=x k has distributio B,p The biomial distributio is the sum of Beroullis 9 0 Idepedet radom variables Pairwise vs mutual idepedece I a roll of two dice, let X ad Y record the umbers o the first ad secod die respectively What ca you say about the evets X = 3, Y = 2? What about X = i ad Y = j? Defiitio: The radom variables X ad Y are idepedet if for every x ad y the evets X = x ad Y = y are idepedet Example: X ad Y above are idepedet Defiitio: The radom variables X,X 2,,X are mutually idepedet if, for every x, x 2,x Pr(X = x,,x = x ) = Pr(X = x ) Pr(X = x ) Example: X k, the success idicators i Beroulli trials, are idepedet Mutual idepedece implies pairwise idepedece; the coverse may ot be true: Example : A ball is radomly draw from a ur cotaiig 4 balls: oe blue, oe red, oe gree ad oe multicolored (red + blue + gree)x Let X, X 2 ad X 3 deote the idicators of the evets the ball has (some) blue, red ad gree respectively Pr(X i = ) = /2, for i =, 2, 3 X = 0 X = X ad X 2 idepedet: X 2 = 0 /4 /4 X 2 = /4 /4 Similarly, X ad X 3 are idepedet; so are X 2 ad X 3 Are X, X 2 ad X 3 idepedet? No! Pr(X = X 2 = X 3 = ) = /4 Pr(X = ) Pr(X 2 = ) Pr(X 3 = ) = /8 Example 2: Suppose X ad X 2 are bits (0 or ) chose uiformly at radom; X 3 = X X 2 X, X 2 are idepedet, as are X, X 3 ad X 2, X 3 But X, X 2, ad X 3 are ot mutually idepedet X ad X 2 together determie X 3! 2

4 The distributio of X + Y Example: The Sum of Biomials Suppose X ad Y are idepedet radom variables whose rage is icluded i {0,,,} For k {0,,,2}, X + Y = k = k j=0 ((X = j) (Y = k j)) Note that some of the evets might be empty Eg, X = k is boud to be empty if k > This is a disjoit uio so Pr(X + Y = k) = Σ k j=0 Pr(X = j Y = k j) = Σ k j=0 Pr(X = j) Pr(Y = k j) [by idepedece] Suppose X has distributio B,p, Y has distributio B m,p, ad X ad Y are idepedet Pr(X + Y = k) = Σ k j=0 Pr(X = j Y = k j) [sum rule] = Σ k j=0 Pr(X = j) Pr(Y = k j) [idepedece] = Σ k ( ) j=0 j p j ( p) j( ) m k j p k j ( p) m k+j = Σ k ( )( ) m j=0 j k j p k ( p) +m k ( )( ) m )p k ( p) +m k = (Σ k j=0) j k j p k ( p) +m k = ( +m k Thus, X + Y has distributio B +m,p A easier argumet: Perform + m Beroulli trials Let X be the umber of successes i the first ad let Y be the umber of successes i the last m X has distributio B,p, Y has distributio B m,p, X ad Y are idepedet, ad X +Y is the umber of successes i all +m trials, ad so has distributio B +m,p 3 4 Expected Value Suppose we toss a biased coi, with Pr(h) = 2/3 If the coi lads heads, you get $; if the coi lads tails, you get $3 What are your expected wiigs? 2/3 of the time you get $; /3 of the time you get $3 (2/3 ) + (/3 3) = 5/3 What s a good way to thik about this? We have a radom variable W (for wiigs): W(h) = W(t) = 3 The expectatio of W is E(W) = Pr(h)W(h) + Pr(t)W(t) = Pr(W = ) + Pr(W = 3) 3 Example: What is the expected cout whe two dice are rolled? Let X be the cout: E(X) = Σ 2 i=2i Pr(X = i) = = = 7 More geerally, the expected value of radom variable X o sample space S is A equivalet defiitio: E(X) = Σ x x Pr(X = x) E(X) = Σ s S X(s) Pr(s) 5 6

5 Expectatio of Biomials Expectatio is Liear What is E(B,p ), the expectatio for the biomial distributio B,p How may heads do you expect to get after tosses of a biased coi with Pr(h) = p? Method : Use the defiitio ad crak it out: E(B,p ) = Σ k=0k k p k ( p) k This looks awful, but it ca be calculated Method 2: Use Iductio; break it up ito what happes o the first toss ad o the later tosses O the first toss you get heads with probability p ad tails with probability p O the last tosses, you expect E(B,p ) heads Thus, the expected umber of heads is: E(B,p ) = p( + E(B,p )) + ( p)(e(b,p )) = p + E(B,p ) E(B,p ) = p Now a easy iductio shows that E(B,p ) = p There s a eve easier way 7 Theorem: E(X + Y ) = E(X) + E(Y ) Proof: Recall that Thus, E(X) = Σ s S Pr(s)X(s) E(X + Y ) = Σ s S Pr(s)(X + Y )(s) = Σ s S Pr(s)X(s) + Σ s S Pr(s)Y (s) = E(X) + E(Y ) Theorem: E(aX) = ae(x) Proof: E(aX) = Σ s S Pr(s)(aX)(s) = aσ s S X(s) = ae(x) 8 Example : Back to the expected value of tossig two dice: Let X be the cout o the first die, X 2 the cout o the secod die, ad let X be the total cout Notice that E(X ) = E(X 2 ) = ( )/6 = 35 E(X) = E(X +X 2 ) = E(X ) +E(X 2 ) = = 7 Example 2: Back to the expected value of B,p Let X be the total umber of successes ad let X k be the outcome of the kth experimet, k =,,: E(X k ) = p + ( p) 0 = p Expectatio of Poisso Distributio Let X be Poisso with parameter λ: f X (k) = e λλk for k N E(X) = Σ k=0k e λλk = λσ k=e λ λk (k )! = λσ j=0e λλj j! = λ Does this make sese? Recall that, for example, X models the umber of icomig calls for a tech support ceter whose average rate per miute is λ Therefore X = X + + X E(X) = E(X ) + + E(X ) = p 9 20

6 Expectatio of geometric distributio Cosider a sequece of Beroulli trials Let X deote the umber of the first successful trial Eg, the first time you see heads X has a geometric distributio f X (k) = ( p) k p k N + The probability of seeig heads for the first time o the kth toss is the probability of gettig k tails followed by heads This is also called a egative biomial distributio of order The egative biomial of order gives the probability that it will take k trials to have successes What is the probability that X is fiite? Ca ow compute E(X): Σ k=f X (k) = Σ k=( p) k p = pσ j=0( p) j = p ( p) = E(X) = Σ k=k ( p) k p = p [ Σ k=( p) k + Σ k=2( p) k + Σ k=3( p) k + ] = p[(/p) + ( p)/p + ( p) 2 /p + ] = + ( p) + ( p) 2 + = /p So, for example, if the success probability p is /3, it will take o average 3 trials to get a success All this computatio for a result that was ituitively clear all alog 2 22 Coditioal Expectatio Variace ad Stadard Deviatio E(X A) is the coditioal expectatio of X give A E(X A) = Σ x x Pr(X = x A) = Σ x x Pr(X = x A)/ Pr(A) Theorem: For all evets A such that Pr(A), Pr(A) > 0: E(X) = E(X A) Pr(A) + E(X A) Pr(A) Proof: E(X) = Σ x x Pr(X = x) = Σ x x(pr((x = x) A) + Pr((X = x) A)) = Σ x x(pr(x = x A) Pr(A) + Pr(X = x A) Pr(A)) = Σ x (x Pr(X = x A) Pr(A)) + (x Pr(X = x A) Pr(A)) = E(X A) Pr(A) + E(X A) Pr(A) Example: I toss a fair die If it lads with 3 or more, I toss a coi with bias p (towards heads) If it lads with less tha 3, I toss a coi with bias p 2 What is the expected umber of heads? Let A be the evet that the die lads with 3 or more Pr(A) = 2/3 E(#H) = E(#H A) Pr(A) + E(#H A) Pr(A) = p p Expectatio summarizes a lot of iformatio about a radom variable as a sigle umber But o sigle umber ca tell it all Compare these two distributios: Distributio : Pr(49) = Pr(5) = /4; Pr(50) = /2 Distributio 2: Pr(0) = Pr(50) = Pr(00) = /3 Both have the same expectatio: 50 But the first is much less dispersed tha the secod We wat a measure of dispersio Oe measure of dispersio is how far thigs are from the mea, o average Give a radom variable X, (X(s) E(X)) 2 measures how far the value of s is from the mea value (the expectatio) of X Defie the variace of X to be Var(X) = E((X E(X)) 2 ) = Σ s S Pr(s)(X(s) E(X)) 2 The stadard deviatio of X is σ X = Var(X) = Σ s S Pr(s)(X(s) E(X)) 2 24

7 Why ot use X(s) E(X) as the measure of distace istead of variace? (X(s) E(X)) 2 turs out to have icer mathematical properties I R, the distace betwee (x,,x ) ad (y,,y ) is (x y ) (x y ) 2 Example: The variace of distributio is 4 (5 50)2 + 2 (50 50)2 + 4 (49 50)2 = 2 The variace of distributio 2 is 3 (00 50)2 + 3 (50 50)2 + 3 (0 50)2 = Variace: Examples Let X be Beroulli, with probability p of success Recall that E(X) = p Var(X) = (0 p) 2 ( p) + ( p) 2 p = p( p)[p + ( p)] = p( p) Theorem: Var(X) = E(X 2 ) E(X) 2 Proof: E(X E(X)) 2 = E(X 2 2E(X)X + E(X) 2 ) = E(X 2 ) 2E(X)E(X) + E(E(X) 2 ) = E(X 2 ) 2E(X) 2 + E(X) 2 = E(X 2 ) E(X) 2 Example: Suppose X is the outcome of a roll of a fair die Recall E(X) = 7/2 E(X 2 ) = = 9 6 So Var(X) = 9 6 (7 2 )2 = Var(X + Y) Defiitio: The covariace of X ad Y is Cov(X, Y) = E(XY) E(X) E(Y) Cov(X, X) = Var(X) Cov(X, X) = Var(X) If X ad Y are idepedet, Cov(X, Y) = 0 [proved soo] covariace provides a measure of correlatio: Cor(X, Y) = Cov(X, Y)/σ X σ Y Cor(X, X) = Cor(X, X) = Cor(X, Y) = 0 if X ad Y are idepedet Claim: Var(X + Y) = Var(X) + Var(Y) + 2 Cov(X, Y) Proof: E(X + Y ) = E(X) + E(Y ), so Var(X + Y) = E[(X + Y ) 2 ] (E(X) + E(Y )) 2 = E(X 2 + 2XY + Y 2 ) (E(X) 2 + 2E(X)E(Y ) + E(Y ) 2 ) = [E(X 2 ) E(X) 2 ] + [E(Y 2 ) E(Y ) 2 ] +2[E(XY ) E(X)E(Y )] = Var(X) + Var(Y) + 2Cov(X, Y) 27

f X (12) = Pr(X = 12) = Pr({(6, 6)}) = 1/36

f X (12) = Pr(X = 12) = Pr({(6, 6)}) = 1/36 Probability Distributios A Example With Dice If X is a radom variable o sample space S, the the probability that X takes o the value c is Similarly, Pr(X = c) = Pr({s S X(s) = c}) Pr(X c) = Pr({s S X(s)