skipping section 6.6 / 5.6 (generating permutations and combinations) concludes basic counting in Chapter 6 / 5

Transcription

1 kiing ection 6.6 / 5.6 generating ermutation and combination conclude baic counting in Chater 6 / 5 on to Chater 7 / 6: Dicrete robability before we go to trickier counting in Chater 8 / 7

2 age / Goal of Chater 7 / 6 undertanding baic robabilitie, a they o u all over the lace: am filter: i am when it contain rolex? drug tet: are you ick when you tet oitive? evaluation of loy channel: wa on bit ent when on bit received? laying roulette/lotterie/game how

3 age / Introduction to dicrete robability baic definition: amle ace: a et of oible outcome hand of card, number on dice given a amle ace S, an exeriment reult in an outcome S - dealing card no re. hand of card - rolling dice with re. number event: a ubet of the amle ace three of a kind, um i ix if S finite and each S i equally likely to be the reult of an exeriment, then robability of event E i E = E / S

4 age / Comlement and union of event event E S amle ace: the robability that E doe not occur i E 1 E ue E S E E i comlementary event of E wrt S event E 1, E 2 S, then E 1 E 2 = E 1 + E 2 E 1 E 2 roof immediate from E 1 E 2 = E 1 + E 2 E 1 E 2 counting i crucial for elementary dicrete robabilitie unfortunately it i not enough

5 age / age / Monty & exercie Examle of dicrete robabilitie urn, door, coin, dice, card: three door, rice behind only one door: robability 1/3 to win the rize elect one card from tandard deck: robability 4/52 = 1/13 it an ace roll two dice: robability 5/36 that um=6, for event = {1,5,2,4,3,3,4,2,5,1} get comlicated very eaily how to better model um of two dice with amle ace {2,3,,12} and event with different robabilitie? how to model unfair coin, loaded dice,?

6 age / Probability theory, odd and end more flexible aroach to robability needed, to deal with unfair coin, um of dice, more contrived combination of event, etc. aigning robabilitie: not jut E = E / S conditional robability, indeendence Bernoulli trial: reeating exeriment random variable: from outcome to value birthday aradox: colliion unavoidable and, later, oibly: robabilitic alg: wrt time & outcome the robabilitic method: noncontructive exitence roof baed on robability theory

7 age / Aigning robabilitie to lift the E = E / S retriction: let S be a countable et of outcome robability ditribution on S i a function : S [0,1] = R with 1 thu: each S i aigned a robability for each S: 0 1 together S robabilitie um to 1: each exeriment reult in ome outcome define S E 1, ince E S E 0, 1 S

8 age / Aigning robabilitie, imle remark robability ditribution aroach cover earlier dicrete robabilitie: uniform ditribution on S with S = n: S = 1/n E = E / S electing an element from a amle ace with uniform ditribution i amling at random unfair coin or dice, um of dice, etc: eay to model jut make ure 1 comlement E 1 E and union E 1 E 2 = E 1 + E 2 E 1 E 2 follow a before S

9 age / Conditional robability and indeendence often robabilitie exit in ome context, or when a certain condition i atified: what chance to tet oitive what chance to tet oitive if ick what chance i am, if rolex... we need to be able to figure out if context or condition influence robability: what the chance of head if the lat five toe were tail? generally eaking: intuition cannot be truted

10 age 442/404 Conditional robability: definition let E and F be event with F > 0 thu E, F S, for ome amle ace S the conditional robability of E given F i denoted by E F een thi in 1 t emeter already E F i defined a E F F and hould be interreted a the robability that E occur given the fact that F occur intuition: univere S relaced by F, event E by EF E= E / S by E F= EF / F = EF / S / F / S = EF/F

11 age / Conditional robability, examle roll a die, what robability outcome i even? 3/6 = ½ but given that outcome i robability become 1/3, ince F = {1,2,3}, F = ½, E = {2,4,6}, EF={2}, EF=1/6, E F = EF/F = 1/6/1/2 = 1/3 to coin 6; robability lat to i head? ½ but given that firt five are tail? robability remain ½: F={tttttt,ttttth}, EF={ttttth}, E F=½ condition may or may not affect robability

12 age / Indeendence if E F = E, then aarently occurrence of F doe not influence E E and F are called indeendent: event E and F are defined to be indeendent if EF =EF E F=EF/F=EF/F=E note that F E=F follow too if E0 how doe one decide indeendence? calculate EF, E, and F, declare indeendence if EF =EF in articular: don t trut your intuition

13 age /406 Indeendence examle conider familie with k 2 children, aume all 2 k boy/girl configuration equally likely E event that family ha boy and girl F event that family ha at mot one boy are E and F indeendent? my intuition uele: anwer deend on k k=2: E={bg,gb}=1/2, F={bg,gb,gg}=3/4 EF={bg,gb}=1/2; EF EF: no k=3:e={bbg,bgb,bgg,gbb,gbg,ggb}=6/8, F={bgg,gbg,ggb,ggg}=4/8, EF={bgg,gbg,ggb}=3/8 = EF: ye k=4: : not indeendent

14 age / Brief reca robability ditribution on countable et of outcome S i a function E : S [0,1] = R with S S 1 : E, E 1 E E 0, 1 if S = n and S = 1/n then: uniform ditribution, election at random E, F S EF = E + F EF conditional robability if F0 E F = EF/F if EF =EF then E and F are indeendent E F = E

15 age 456/419 Conditional robabilitie, examle D: event that omeone ha dieae d : event that omeone tet oitive for d event: 1. given D : true oitive 2. given D : fale oitive 3. given D : fale negative 4. given D : true negative event robabilitie given by lab exeriment note that 1&3 and 2&4 are comlementary what can we ay about the robability of D given hould one worry when teting oitive? D given can one be relieved when teting negative?

16 age 456/419 Examle continued uoe D = and D 0.999: i.e., tet i 99.9% accurate what can we ay about D and D? generally eaking, almot nothing: it deend on frequency of dieae if common dieae, ay D = 0.01: be concerned if tet oitive: D > 0.9 if rare dieae, ay D = : be only lightly concerned if tet oitive: D < quite a bit maller than 0.999

17 age / Baed on: Baye theorem turning D into D, etc: definition: D = D/D D 0 D = DD imilarly, if 0: D = D D = DD D = DD/ with D: have dieae, : tet oitive D D if chance of having dieae and teting oitive are comarable D << D if dieae unlikely comared to teting oitive for it

18 Baye theorem, more ueful&common form een that: D = DD/ with a dijoint union: Baye thm follow where 0, D 0: age / D D D D D D D D D D D D D D D

19 age 456/419 Baye theorem, detail of earlier examle D: event to have dieae d : event to tet oitive for d D = and D 0.999, thu D and D If D = : If D = 0.01: D D D D D D D D

20 age 456/419 Baye theorem, examle detail continued if D = 0.01: D D D D D D D if D = : D

21 age 458/421 Baye theorem, recognizing am S: event that an meage i am S: comlementary event that it i not am in book: S S : ame robabilitie for am and non-am more general: for intance aume twice a much am a non-am S 2/3, S 1/ 3 aume you ve oberved that oortunity S = 1/10 oortunity S = 1/100 what i robability that an meage containing oortunity i am?

22 Baye theorem, recognizing am, continued age 458/421 S 2/3, S 1/ 3 we have and have oberved that oortunity S = 1/10 oortunity S = 1/100 need the robability that an containing oortunity i am, i.e., S oortunity according to Baye thm w = oortunity : w S S S w w S S w S S 0.1 2/ / /

23 conclude ection 7.3 / 6.3 on to exected value and variance etc, ection 7.4 / 6.4

24 age / Exected value and variance given ome robability ditribution: what can we exect to haen? how much fluctuation i reaonable? intuitively: exect average over all outcome, each outcome weighted by it robability roll one die: outcome are 1,2,, 6, each with robability 1/6, average outcome: 1/6+2/6+ +6/6 = 3½ roll two dice: outcome are air 1,1, 1,2, 1,3,,6,6, each with robability 1/36, average outcome: 1,1+ +6,6/36 =? toing a coin, what the average?

25 age / Tranforming outcome into real value: random variable a random variable i not random not a variable but: a function S R, where S i a amle ace a random variable aign a real value to each oible outcome in S ditribution of random variable on S i the et of air r, =r for r S, where r S: r note that thi i a robability ditribution

26 age / Random variable, examle tranform uniform ditribution into a more general robability ditribution: let S = {1,1,1,2,,6,6}, et of outcome of rolling two dice define on S by i,j = i+j, then: S = {2,3,,12} = S uniform ditribution on S generate non-uniform robability ditribution on S 2 3 S : 3 2 S : 2 1,2 1,1 2,1 1/ 36 1/18, etc.

27 Exected value of a random variable given random variable on amle ace S, intuitive definition of exected value E become with it follow that thu two different way to comute E: um over value of : 2/36+3/ /18+12/36 = 7 um over amle ace: /36 = 7 which one to ue deend on circumtance age / S r r r E r S r : S r S S r r E :

28 age / Examle: Bernoulli trial an exeriment with two oible outcome ucce or failure i called a Bernoulli trial: if i ucce robability, then q = 1 i the failure robability three relevant quetion: if ame Bernoulli trial i reeated n time, what i robability of a total of k uccee? how many uccee exected after n trial? exect how many trial before ucce? aumtion: n trial mutually indeendent, i.e., conditional robability of ucce of any trial i, conditioned on outcome of other

29 age / n indeendent Bernoulli trial: if ucce robability of each trial i, then the robability of reciely k uccee in n indeendent trial i Cn,k k q nk : each articular equence of k uccee and thu nk failure occur with robability k q nk due to indeendence there are Cn,k different equence of k uccee um robabilitie of dijoint event a a function of k: the binomial ditribution becaue anity check n k C n, k k q n 1 k 0 relie on binomial theorem

30 age 465/428 n Bernoulli trial, exected # uccee : random variable counting the number of uccee after n Bernoulli trial, = k = Cn,k k q nk E uing k S k k where S={0,1,,n} n k nk E kc n, k q k1 ick k from n firt, then leader among k, or ick leader firt, then k 1from n 1 n... k nc n 1 n 1, k 1 E : inconvenient S k q nk

31 n Bernoulli trial, exected # uccee, eaier if and are random variable on S, then E+ = E + E roof: inconvenient alication: i,j reult of two dice, 1 i,j=i, 2 i,j=j, then E =E 1 +E 2 = 3½+3½ = 7 age 466/429 two function um of ue definition of E E E S S S S S t t t E

32 age 470/429 Remaining quetion on Bernoulli trial how many trial can we exect before ucce? exeriment: erform trial until ucce outcome:, N, NN,., NN N, infinite amle ace S = {,N,NN, } random variable on S: = number of trial needed for S, thu =1, N=2, NN=3, =1=, =2=q,, =k=q k1, note: k1 q q /1 q 1, k1 0 thu called geometric ditribution we need E ue Tr, lecture 8, lide 5; >0: k1 2 E kq... /1 q 1/ k1

33 age / More on exectation een that E+ = E + E for any random variable and on amle ace S i it alo true that E = EE? to coin twice, outcome {HH,HT,TH,TT} = total number head, o E = 1 = total number tail, o E = 1 E = 20/4+11/4+11/4+02/4 = ½ E not equal to EE in general E not equal to EE

34 Indeendence of random variable if x,yr: = x and = y equal = x = y then and are indeendent if and are indeendent: E = EE =0 and =0 = 0 1/16 = =0=0 age / and,, E E y y x x y x xy y x xy r r E S y S x S y S x S y S x S r

35 age / variance and tandard deviation if S: 0 then E can be ued to bound robability that deviate from E: xr >0 : x E/x Markov inequality f: E / x r / x r r S r S, rx tronger reult ue variance V of, V and the tandard deviation note: if in unit, then V in unit 2 S r E 2, x V

36 Variance of a random variable age V / E S V = E 2 E 2 roof: ue definition variance ingle Bernoulli trial i q, indeendent: V+= V + V f.: ue V = E 2 E 2, E+ = E + E alway true and E = EE due to indeendence variance n inde. Bernoulli trial i nq

37 age 476/439 Chebyhev inequality: E x V/x 2 f.: A={S E x} V/x 2 A Chebyhev tronger than Markov S 0: xr >0 x E/x Chebyhev uele for x exonential and non-trivial etimate: ue Chernoff bound not here

38 Variance examle: coure evaluation E = /176 = 4.55 E 2 = /176 = V = E 2 E 2 = = 1.11 uing Chebyhev E x V/x 2, how many extremely oor can we exect? / = o: at mot = 15.50

39 Baic robability, fact to remember Baye theorem: D random variable, a function from a amle ace to the real number exected value E i additive: for all random variable and : E+ = E + E variance V = E 2 E 2 D D D D D D for indeendent random variable and : E = EE V+ = V + V after about n drawing from n: colliion

40 age / Final remark on Ch.7/6: birthday roblem S amle ace with S = n, draw k element at random with relacement how likely i a colliion, i.e., that an element i drawn twice or more? alication: building hah table, digital fingerrinting, crytanalyi, etc. if k 1: dulicate with robability 0 if k > n: dulicate with robability 1 colliion robability increae with growing k, for what k i colliion robability ½? uroe: how that thi k i not about n/2

41 age / Birthday roblem, rough analyi drawing k random element with relacement from amle ace S with S = n, for what k i colliion robability ½? look at comlementary roblem: analye robability to ick k ditinct element

42 age / Probability to ick k ditinct element 1. if k = 1: robability 1 that element i unique n 1 2. if k = 2: robability 1 to have two ditinct element n n 1 n 2 3. if k = 3: robability 1 to have n n three ditinct element n 1 n 2 n k 1 4. for general k: 1 n n n i robability to have k ditinct element thi become zero for k > n, which i right

43 age / Birthday roblem, rough analyi continued S amle ace with S = n, draw k element at random with relacement all-ditinct robability after k drawing i n 1 n 2 n k 1 1 n n n clearly decreaing: for what k doe it get ½ and thu colliion robability ½?

44 age / Birthday roblem, rough analyi continued for what k colliion robability ½, i.e.: n 1 n 2 n k 1 1 1/ 2 n n n thi i equivalent to k1 n 1 n 2... n k 1 n / 2 hand-wavy argument: n1n2 nk+1 = n k1 kk1/2n k2 + n k1 kk1/2n k2 + n k1 /2 n k1 /2 kk1/2n k2 uffice to take k a little bigger than n

45 age / Birthday roblem, concluion S amle ace with S = n, draw k element at random with relacement: after only k n / 2 drawing, robability of a colliion i larger than 11/e = k i lower than what intuition ugget, commonly called birthday aradox nothing aradoxical about it, jut a conequence of k = kk+1/2 lead to lot of algorithm, and trouble