Chapter 1: Introduction to Probability

Transcription

1 Chapter : Introducton to obablty Sectons Engneerng pplcatons of obablty Random Experments and Events Defntons of obablty The Relatve Frequency pproach Elementary Set Theory The xomatc pproach Condtonal obablty Independence Combned Experments Bernoull Trals pplcatons of Bernoull Trals Important Concepts and pplcatons: Random Experments and Events Defntons of obablty obablstc Experments Compound Events Condtonal obablty Bayes Theorem Dgtal Communcatons Systems Random Varables Bernoull Trals Relablty Smulatng obablstc Experments Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng 205 of 25 ECE 3800

2 Defntons of obablty Experment n experment s some acton that results n an outcome. random experment s one n whch the outcome s uncertan before the experment s performed. Possble Outcomes descrpton of all possble expermental outcomes. The set of possble outcomes may be dscrete or form a contnuum. Trals Event The sngle performance of a well-defned experment. n elementary event s one for whch there s only one outcome. composte event s one for whch the desred result can be acheved n multple ways. Multple outcomes result n the event descrbed. Equally Lkely Events/Outcomes When the set of events or each of the possble outcomes s equally lkely to occur. term that s used synonymously to equally lkely outcomes s random. Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

3 Defntons of obablty, Performng an Experment Objects The physcal tem nvolved n the experment. ttrbute The characterstc of the object that defnes an outcome. Sample Space descrpton of all possble tral outcomes for the experment. For dscrete outcomes, the sample space descrbes a set that contans all possble expermental results. For contnuous outcomes, the sample space descrbes a regon that s physcally (axomatc or mathematcally) descrbed. Wth Replacement and Wthout Replacement When trals are performed wth replacement, the ntal condtons of the experment are restored pror to each tral. When trals are performed wthout replacement, each successve tral s performed based on the expermental condtons remanng at the concluson of the prevous tral. Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

4 Experment : bag of marbles, draw (Marble_Example.m) bag of marbles: 3-blue, 2-red, one-yellow Objects: Marbles ttrbutes: Color (Blue, Red, Yellow) Experment: Draw one marble, wth replacement Sample Space: {B, R, Y} obablty (relatve frequency method) The probablty for each possble event n the sample space s. Event obablty Blue 3/ Red 2/ Yellow / Total / Ths experment would be easy to run and verfy after lots of trals. see Matlab Sec_Marble.m ntrals = vs. 00 vs. 000 (repeat executon a few tmes) (nother problem: f we ran trals, what s the probablty that we get events that exactly match the probablty? 3-Blue, 2-Red, Yellow - much harder problem) Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

5 Mathematcal Descrptons and More Defntons obablty, the relatve frequency method: The number of trals and the number of tmes an event occurs can be descrbed as N N N B N C the relatve frequency s then r N N note that N N N N B N N C r rb rc When expermental results appear wth statstcal regularty, the relatve frequency tends to approach the probablty of the event. and Where lm r N B C s defned as the probablty of event. Mathematcal defnton of probablty: B C, for mutually exclusve events 3. n mpossble event,, can be represented as 0 4. certan event,, can be represented as.. Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

6 Experment 2: bag of marbles, draw 2 Experment: Draw one marble, replace, draw a second marble, wth replacement Sample Space: {BB, BR, BY, RR, RB, RY, YB, YR, YY} Defne the probablty of each event n the sample space. Jont obablty When a desred outcome conssts of multple events. (Read the probablty of events and B)., B Statstcally Independent When the probablty of an event does not depend upon any other pror events. If trals are performed wth replacement and/or the ntal condtons are restored, you expect tral outcomes to be ndependent. Therefore, B B, B The margnal probablty of each event s not affected by pror/other events. The probablty of event gven event B occurred s the same as the probablty of event and vce versa. B and B B pplcable for multple objects wth sngle attrbutes and wth replacement. st-rows\2 nd -col Blue Red Yellow Blue Red Yellow Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng 205 of 25 ECE 3800

7 Next Concept Condtonal obablty When the probablty of an event depends upon pror events. If trals are performed wthout replacement and/or the ntal condtons are not restored, you expect tral outcomes to be dependent on pror results or condtons. The jont probablty s. B when follows B, B B, B B B pplcable for objects that have multple attrbutes and/or for trals performed wthout replacement. Experment 3: bag of marbles, draw 2 wthout replacement Experment: Draw two marbles, wthout replacement Sample Space: {BB, BR, BY, RR, RB, RY, YB, YR} Note: no YY! Therefore strows\2 nd Sum st - Blue Red Yellow Marble col Blue Red Yellow Sum 2 nd Marble Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

8 Resstor Example: Jont and Condtonal obablty Smlar to textbook problems (more realstc resstor values) 50 ohms 00 ohms 200 ohms Subtotal ¼ watt ½ watt watt Subtotal Each object has two attrbutes: mpedance (ohms) and power ratng (watts) Margnal obabltes: (uses subtotals) (¼ watt) = 70/50 (½ watt) = 55/50 ( watt) = 25/50 (50 ohms) = 80/50 (00 ohms) = 50/50 (200 ohms) = 20/50 These are called the margnal probabltes when fewer than all the attrbutes are consdered (or don t matter). Jont obabltes: dvded each member of the table by 50! 50 ohms 00 ohms 200 ohms Subtotal ¼ watt 40/50=0.2 20/50=0.33 0/50=0.0 70/50=0.4 ½ watt 30/50= /50=0.33 5/50= /50=0.3 watt 0/50=0.0 0/50=0.0 5/50= /50=0. Subtotal 80/50= /50= /50= /50=.0 These are called the jont probabltes when all unque attrbutes must be consdered. (Concept of total probablty thngs that sum to.0) Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

9 Condtonal obabltes: When one attrbutes probablty s determned based on the exstence (or non-exstence) of another attrbute. Therefore, The probablty of a ¼ watt resstor gven that the mpedance s 50 ohm. (¼ watt gven that the mpedance s 50 ohms) = (¼ watt 50 ohms) = 40/80 = ohms ¼ watt 40/80=0.50 ½ watt 30/80=0.375 watt 0/80=0.25 Total 80/80=.0 Smple math that does not work to fnd the soluton: (¼ watt) = 70/50 and (50 ohms) = 80/50 (¼ watt) x (50 ohms) = 70/50 x 80/50 = 5/225 = What about (50 ohms gven the power s ¼ watt) 50 ohms 00 ohms 200 ohms Total ¼ watt 40/70= /70=0.28 0/70= /70=.0 (50 ohms ¼ watt) = (50 ¼) = 40/70 = 0.57 Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

10 Can you determne? (00, ½) = (00) = (50, ½) = (½ 50) = (50 ½) = ( ) = 50 ohms 00 ohms 200 ohms Subtotal ¼ watt ½ watt watt Subtotal Jont obabltes (00, ½) = (00) = (50, ½) = Condtonal obabltes (½ 00) = (200 ½) = Margnal obablty ( ) = re there multple ways to conceptually defne such problems Yes Relatve Frequency pproach (statstcs) Set Theory pproach (formal math) Venn Dagrams (pctures based on set theory) Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

11 Set Theory Defntons Set collecton of objects known as elements a, a, 2, a n Subset The set whose elements are all members of another set (usually larger but possble the same sze). B a, a, 2, a n k therefore B Space The set contanng the largest number of elements or all elements from all the subsets of nterest. For probablty, the set contanng the event descrpton of all possble expermental outcomes. S, for all subsets Null Set or Empty Set The set contanng no elements Venn Dagram graphcal (geometrc) representaton of sets that can provde a way to vsualze set theory and probablty concepts and can lead to an understandng of the related mathematcal concepts. from: Robert M. Gray and Lee D. Davsson, n Introducton to Statstcal Sgnal ocessng, Cambrdge Unversty ess, Pdf fle verson found at Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng 205 of 25 ECE 3800

12 More Set Theory Defntons Equalty Set equals set B f and only f (ff) every element of s an element of B ND every element of B s an element of. B ff B and B Sum or Unon The sum or unon of sets results n a set that contans all of the elements that are elements of every set beng summed. S 2 3 N Laws for Unons B B S S B, f B oducts or Intersecton The product or ntersecton of sets results n a set that contans all of the elements that are present n every one of the sets. S Laws for Intersectons B B S B B, f B Mutually Exclusve or Dsjont Sets Mutually exclusve or dsjont sets of no elements n common. B NOTE: The ntersecton of two dsjont sets s a set the null set. Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

13 Venn Dagram from: Robert M. Gray and Lee D. Davsson, n Introducton to Statstcal Sgnal ocessng, Cambrdge Unversty ess, Pdf fle verson found at (a) The space (b) Subset G (c) Subset F (d) The Complement of F (e) Intersecton of F and G (f) Unon of F and G Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

14 Complement The complement of a set s the set contanng all elements n the space that are not elements of the set. Laws for Complement and S S S B, f B B, f B DeMorgan s Law B B B B Dfferences The dfference of two sets, -B, s the set contanng the elements of that are not elements of B. Laws for Dfferences B B B B B S S B Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

15 Venn Dagram from: Robert M. Gray and Lee D. Davsson, n Introducton to Statstcal Sgnal ocessng, Cambrdge Unversty ess, Pdf fle verson found at (a) Dfference F-G (b) Dfference F-G Unon wth Dfference G-F F G G F What can be sad about F G? F G F G F F G Takng the probablty F G F G G F F G G F F G F G F G G F F G G F G F G F G F G F G F G Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

16 oofs of Set lgebra from: Robert M. Gray and Lee D. Davsson, n Introducton to Statstcal Sgnal ocessng, Cambrdge Unversty ess, ppendx, Set Theory. Pdf fle verson found at Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng 205 of 25 ECE 3800

17 xomatc Defntons Usng Sets For event 0 Dsjont Sets If S 0 B B B, then Complement If S, then Manpulaton () B B, the unon of dsjont sets B B B Manpulaton (2) B B B, the unon of dsjont sets B B B B B Manpulaton (3) B B B, rearrangng from (2) Substtuton for () B B B Note that: B B B B equalty holds for and B beng dsjont sets! Inequaltes can be used to bound expected reults! Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

18 More Examples: -sded de The probablty of rollng a or a 3, event, The probablty of rollng a 3 or 5, event B 3,5 3 5 B The probablty of event or event B, event C B C B B B Note: C B,3,5 C C 3 5 Conceptual Example: Poker Texas Holdem obablty of gettng a sut (to make a flush),, or an card to make a par, B. B B B B In any card game, every card dealt changes the odds for the cards remanng n the deck, but, n many games, the player does not know the value of cards dealt to other players. sde: Therefore, there would seem to be a sgnfcant advantage to face-up black-jack, unless you are uncomfortable about lettng people see your decsons?! 2 Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

19 Condtonal obablty Defnng the condtonal probablty of event gven that event B has occurred. Usng a Venn dagram, we know that B has occurred then the probablty that has occurred gven B must relate to the area of the ntersecton of and B B B B, for B 0 or B B, for B 0 B For elementary events, B B, B, for B 0 B B If s a subset of B, then the condtonal probablty must be B Therefore, t can be sad that B B B B, for B B B B, for B If B s a subset of, then the condtonal probablty becomes If and B are mutually exclusve, B B B B B, for B B B 0 B 0, for B B Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

20 Resstor Example: Jont and Condtonal obablty 50 ohms 00 ohms 200 ohms Subtotal ¼ watt 40/50=0.2 20/50=0.33 0/50=0.0 70/50=0.4 ½ watt 30/50= /50=0.33 5/50= /50=0.3 watt 0/50=0.0 0/50=0.0 5/50= /50=0. Subtotal 80/50= /50= /50= /50=.0 Condtonal obabltes (¼ watt gven that the mpedance s 50 ohms) = (¼ watt 50 ohms) = 40/80 = 0.50 B B (50 ohms ¼ watt) = (50 ¼) = 40/70 = 0.57 B B 40 B (50 ohms ¼ watt) = (50 ¼) = 40/70 = 0.57 What about (½ 00) = (200 ½) = Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

21 Total obablty For a space, S, that conssts of multple mutually exclusve events, the probablty of a random event, B, occurrng n space S, can be descrbed based on the condtonal probabltes assocated wth each of the possble events. (see Venn dagram Fg. -7) oof: S 2 3 n and, for j j B B S B B B B B 2 3 n 2 3 n B B B B B 2 3 n But B B, for 0 Therefore B B B B 2 2 n n Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

22 or and Posteror obablty The probabltes defned for the expected outcomes,, are referred to as a pror probabltes (before the event). They descrbe the probablty before the actual experment or expermental results are known. fter an event has occurred, the outcome B s known. Then, the probablty of the event belongng to one of the expected outcomes can be defned as B Usng mathematcs from before the probablty of the ntersecton can be wrtten two ways: B B B B B B, for B 0 B and the fnal form B B B B B 2 2 n n Ths probablty s referred to as the a posteror probablty (after the event). The probablty that event occurred gven that outcome B was observed. It s also referred to as Bayes Theorem. In communcatons, I beleve I receved a nstead of a 0 n my dgtal recever. What s the probablty that a was sent? What s the probablty that a 0 was sent? Ths can be thought of as a correct or ncorrect bt detecton problem, leadng to bt-errorrate based on sgnal-to-nose ratos. Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

23 Examples More Resstors Bn Bn 2 Bn 3 Bn 4 Bn 5 Bn Subtotal 0 ohm ohm ohm Subtotal What s the probablty of selectng a 0 ohm resstor from a random bn? (B) obablty of selectng a bn Bn# Bn 0 Bn 2 0 Bn Bn 4 0 Bn 5 0 Bn Form the total probablty B B B B 2 2 B n n B Note that B the resstors are not unformly (evenly) dstrbuted n the bns! Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

24 Same Resstors Bn Bn 2 Bn 3 Bn 4 Bn 5 Bn Subtotal 0 ohm ohm ohm Subtotal ssumng a 0 ohm resstor s selected, what s the probablty t came from bn 3? From Bayes Theorem (an a-posteror problem) B Bn3 0 B B B B 2 2 n 0 Bn3 Bn3 0 Bn Bn 0 Bn Bn Bn n The condtonal probabltes of gettng a 0 ohm resstor from each of the bns Bn Bn Bn Bn Bn Bn The a-posteror probabltes of selectng a 0 ohm resstor from each bn. Bn Bn Bn Bn Bn Bn Note that the probablty that a 0 ohm resstor came from one of the bn (total prob.) s. Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800

25 Independence Two events, and B, are ndependent f and only f B B Independence s typcally assumed when there s no apparent physcal mechansm by whch the two events could depend on each other. For events derved from ndependent elemental events, ther ndependence may not be obvous but may be able to be derved. Independence can be extended to more than two events, for example three,, B, and C. The condtons for ndependence of three events s B B B C B C C C B C B C Note that t s not suffcent to establsh par-wse ndependence; the entre set of equatons s requred. For multple events, every set of events from n down must be verfed. Ths mples that n 2 n equatons must be verfed for n ndependent events. Important opertes of Independence Unons B B B B B B Intersecton wth a Unon B C B C Notes and fgures are based on or taken from materals n the course textbook: obablstc Methods of Sgnal and System nalyss (3rd ed.) by George R. Cooper and Clare D. McGllem; Oxford ess, 999. ISBN: B.J. Bazun, Sprng of 25 ECE 3800