Probability Distributios A Example With Dice If X is a radom variable o sample space S, the the probability that X takes o the value c is Similarly, Pr(X = c) = Pr({s S X(s) = c}) Pr(X c) = Pr({s S X(s) c} This makes sese sice the rage of X is the real umbers Example: I the coi example, Pr(#H = 2) = 4/9 ad Pr(#H 1) = 5/9 Give a probability measure Pr o a sample space S ad a radom variable X, the probability distributio associated with X is f X (x) = Pr(X = x) f X is a probability measure o the real umbers The cumulative distributio associated with X is F X (x) = Pr(X x) Suppose S is the sample space correspodig to tossig a pair of fair dice: {(i, j) 1 i,j 6} Let X be the radom variable that gives the sum: X(i,j) = i + j f X (2) = Pr(X = 2) = Pr({(1, 1)}) = 1/36 f X (3) = Pr(X = 3) = Pr({(1, 2), (2, 1)}) = 2/36 f X (7) = Pr(X = 7) = Pr({(1, 6), (2, 5),,(6, 1)}) = 6/36 f X (12) = Pr(X = 12) = Pr({(6, 6)}) = 1/36 Ca similarly compute the cumulative distributio: F X (2) = f X (2) = 1/36 F X (3) = f X (2) + f X (3) = 3/36 F X (12) = 1 1 2 The Fiite Uiform Distributio The Biomial Distributio The fiite uiform distributio is a equiprobable distributio If S = {x 1,,x }, where x 1 < x 2 < < x, the: f(x k ) = 1/ F(x k ) = k/ Suppose there is a experimet with probability p of success ad thus probability q = 1 p of failure For example, cosider tossig a biased coi, where Pr(h) = p Gettig heads is success, ad gettig tails is failure Suppose the experimet is repeated idepedetly times For example, the coi is tossed times This is called a sequece of Beroulli trials Key features: Oly two possibilities: success or failure Probability of success does ot chage from trial to trial The trials are idepedet 3 4
What is the probability of k successes i trials? Suppose = 5 ad k = 3 How may sequeces of 5 coi tosses have exactly three heads? hhhtt hhtht hhtth C(5, 3) such sequeces! What is the probability of each oe? p 3 (1 p) 2 Therefore, probability is C(5, 3)p 3 (1 p) 2 Let B,p (k) be the probability of gettig k successes i Beroulli trials with probability p of success B,p (k) = C(,k)p k (1 p) k Not surprisigly, B,p is called the Biomal Distributio The Poisso Distributio A large call ceter receives, o average, λ calls/miute What is the probability that exactly k calls come durig a give miute? Uderstadig this probability is critical for staffig! Similar issues arise if a priter receives, o average λ jobs/miute, a site gets λ hits/miute, This is modelled well by the Poisso distributio with parameter λ: f λ (k) = e λλk f λ (0) = e λ f λ (1) = e λ λ f λ (2) = e λ λ 2 /2 e λ is a ormalizatio costat, sice 1 + λ + λ 2 /2 + λ 3 /3! + = e λ 5 6 Derivig the Poisso New Distributios from Old Poisso distributio = limit of biomial distributios Suppose at most oe call arrives i each secod Sice λ calls come each miute, expect about λ/60 each secod The probability that k calls come is B 60,λ/60 (k) This model does t allow more tha oe call/secod What s so special about 60? Suppose we divide oe miute ito time segmets Probability of gettig a call i each segmet is λ/ Probability of gettig k calls i a miute is B,λ/ (k) = C(, k)(λ/) k (1 λ ) k = C(, k) = λk! ( k)! Now let : ( ) lim 1 λ = e λ (! lim ) 1 k ( k)! λ = 1 λ/ 1 λ ( 1 λ k ( 1 λ ) k ( 1 λ Coclusio: lim B,λ/ (k) = e λλk 7 ) ) If X ad Y are radom variables o a sample space S, so is X + Y, X + 2Y, XY, si(x), etc For example, (X + Y )(s) = X(s) + Y (s) si(x)(s) = si(x(s)) Note si(x) is a radom variable: a fuctio from the sample space to the reals 8
Some Examples Example 1: A fair die is rolled Let X deote the umber that shows up What is the probability distributio of Y = X 2? {s : Y (s) = k} = {s : X 2 (s) = k} = {s : X(s) = k} {s : X(s) = k} Coclusio: f Y (k) = f X ( k) + f X ( k) So f Y (1) = f Y (4) = f Y (9) = f Y (36) = 1/6 f Y (k) = 0 if k / {1, 4, 9, 16, 25, 36} Example 2: A coi is flipped Let X be 1 if the coi shows H ad -1 if T Let Y = X 2 I this case Y 1, so Pr(Y = 1) = 1 Example 3: If two dice are rolled, let X be the umber that comes up o the first dice, ad Y the umber that comes up o the secod Formally, X((i,j)) = i, Y ((i,j)) = j The radom variable X +Y is the total umber showig Example 4: Suppose we toss a biased coi times (more geerally, we perform Beroulli trials) Let X k describe the outcome of the kth coi toss: X k = 1 if the kth coi toss is heads, ad 0 otherwise How do we formalize this? What s the sample space? Notice that Σ k=1x k describes the umber of successes of Beroulli trials If the probability of a sigle success is p, the Σ k=1x k has distributio B,p The biomial distributio is the sum of Beroullis 9 10 Idepedet radom variables Pairwise vs mutual idepedece I a roll of two dice, let X ad Y record the umbers o the first ad secod die respectively What ca you say about the evets X = 3, Y = 2? What about X = i ad Y = j? Defiitio: The radom variables X ad Y are idepedet if for every x ad y the evets X = x ad Y = y are idepedet Example: X ad Y above are idepedet Defiitio: The radom variables X 1,X 2,,X are mutually idepedet if, for every x 1, x 2,x Pr(X 1 = x 1 X = x ) = Pr(X 1 = x 1 ) Pr(X = x ) Example: X k, the success idicators i Beroulli trials, are idepedet Mutual idepedece implies pairwise idepedece; the coverse may ot be true: Example 1: A ball is radomly draw from a ur cotaiig 4 balls: oe blue, oe red, oe gree ad oe multicolored (red + blue + gree) Let X 1, X 2 ad X 3 deote the idicators of the evets the ball has (some) blue, red ad gree respectively Pr(X i = 1) = 1/2, for i = 1, 2, 3 X 1 = 0 X 1 = 1 X 1 ad X 2 idepedet: X 2 = 0 1/4 1/4 X 2 = 1 1/4 1/4 Similarly, X 1 ad X 3 are idepedet; so are X 2 ad X 3 Are X 1, X 2 ad X 3 idepedet? No! Pr(X 1 = 1 X 2 = 1 X 3 = 1) = 1/4 Pr(X 1 = 1) Pr(X 2 = 1) Pr(X 3 = 1) = 1/8 Example 2: Suppose X 1 ad X 2 are bits (0 or 1) chose uiformly at radom; X 3 = X 1 X 2 X 1, X 2 are idepedet, as are X 1, X 3 ad X 2, X 3 But X 1, X 2, ad X 3 are ot mutually idepedet X 1 ad X 2 together determie X 3! 11 12
The distributio of X + Y Example: The Sum of Biomials Suppose X ad Y are idepedet radom variables whose rage is icluded i {0, 1,,} For k {0, 1,,2}, (X + Y = k) = k j=0 ((X = j) (Y = k j)) Note that some of the evets might be empty Eg, X = k is boud to be empty if k > This is a disjoit uio so Pr(X + Y = k) = Σ k j=0 Pr(X = j Y = k j) = Σ k j=0 Pr(X = j) Pr(Y = k j) [by idepedece] Suppose X has distributio B,p, Y has distributio B m,p, ad X ad Y are idepedet Pr(X + Y = k) = Σ k j=0 Pr(X = j Y = k j) [sum rule] = Σ k j=0 Pr(X = j) Pr(Y = k j) [idepedece] = Σ k ( ) j=0 j p j (1 p) j( ) m k j p k j (1 p) m k+j = Σ k ( )( ) m j=0 j k j p k (1 p) +m k = (Σ k ( )( ) m j=0 j k j )p k (1 p) +m k ) p k (1 p) +m k = ( +m k = B +m,p (k) Thus, X + Y has distributio B +m,p A easier argumet: Perform + m Beroulli trials Let X be the umber of successes i the first ad let Y be the umber of successes i the last m X has distributio B,p, Y has distributio B m,p, X ad Y are idepedet, ad X +Y is the umber of successes i all +m trials, ad so has distributio B +m,p 13 14 Expected Value Suppose we toss a biased coi, with Pr(h) = 2/3 If the coi lads heads, you get $1; if the coi lads tails, you get $3 What are your expected wiigs? 2/3 of the time you get $1; 1/3 of the time you get $3 (2/3 1) + (1/3 3) = 5/3 What s a good way to thik about this? We have a radom variable W (for wiigs): W(h) = 1 W(t) = 3 The expectatio of W is E(W) = Pr(h)W(h) + Pr(t)W(t) = Pr(W = 1) 1 + Pr(W = 3) 3 Example: What is the expected cout whe two dice are rolled? Let X be the cout (the sum of the values o the two dice): E(X) = Σ 12 i=2i Pr(X = i) = 2 1 36 + 3 2 36 + 4 3 36 + + 7 6 36 + + 12 1 36 = 252 36 = 7 More geerally, the expected value of radom variable X o sample space S is A equivalet defiitio: E(X) = Σ x x Pr(X = x) E(X) = Σ s S X(s) Pr(s) 15 16
Expectatio of Biomials Expectatio is Liear What is E(B,p ), the expectatio for the biomial distributio B,p How may heads do you expect to get after tosses of a biased coi with Pr(h) = p? Method 1: Use the defiitio ad crak it out: E(B,p ) = Σ k=0k k p k (1 p) k This looks awful, but it ca be calculated Method 2: Use Iductio; break it up ito what happes o the first toss ad o the later tosses O the first toss you get heads with probability p ad tails with probability 1 p O the last 1 tosses, you expect E(B 1,p ) heads Thus, the expected umber of heads is: E(B,p ) = p(1 + E(B 1,p )) + (1 p)(e(b 1,p )) = p + E(B 1,p ) E(B 1,p ) = p Now a easy iductio shows that E(B,p ) = p There s a eve easier way 17 Theorem: E(X + Y ) = E(X) + E(Y ) Proof: Recall that Thus, E(X) = Σ s S Pr(s)X(s) E(X + Y ) = Σ s S Pr(s)(X + Y )(s) = Σ s S Pr(s)X(s) + Σ s S Pr(s)Y (s) = E(X) + E(Y ) Theorem: E(aX) = ae(x) Proof: E(aX) = Σ s S Pr(s)(aX)(s) = aσ s S X(s) = ae(x) 18 Example 1: Back to the expected value of tossig two dice: Let X 1 be the cout o the first die, X 2 the cout o the secod die, ad let X be the total cout Notice that E(X 1 ) = E(X 2 ) = (1 + 2 + 3 + 4 + 5 + 6)/6 = 35 E(X) = E(X 1 +X 2 ) = E(X 1 ) +E(X 2 ) = 35 + 35 = 7 Example 2: Back to the expected value of B,p Let X be the total umber of successes ad let X k be the outcome of the kth experimet, k = 1,,: Therefore E(X k ) = p 1 + (1 p) 0 = p X = X 1 + + X E(X) = E(X 1 ) + + E(X ) = p Expectatio of Poisso Distributio Let X be Poisso with parameter λ: f X (k) = e λλk for k N E(X) = Σ k=0k e λλk λ λk = Σ k=1e (k 1)! = λe λ Σ k=1 λk 1 (k 1)! = λe λ Σ j=0 λj j! = λe λ e λ [Taylor series!] = λ Does this make sese? Recall that, for example, X models the umber of icomig calls for a tech support ceter whose average rate per miute is λ 19 20
Geometric Distributio Cosider a sequece of Beroulli trials Let X deote the umber of the first successful trial Eg, the first time you see heads X has a geometric distributio f X (k) = (1 p) k 1 p k N + The probability of seeig heads for the first time o the kth toss is the probability of gettig k 1 tails followed by heads This is also called a egative biomial distributio of order 1 The egative biomial of order gives the probability that it will take k trials to have successes 21