Chapter 2: Probability 2-1 Sample Spaces & Events 2-1.1 Random Experiments 2-1.2 Sample Spaces 2-1.3 Events 2-1 1.4 Counting Techniques 2-2 Interpretations & Axioms of Probability 2-3 Addition Rules 2-4 Conditional Probability 2-5 Multiplication & Total Probability Rules 2-6 Independence 2-7 Bayes Theorem 2-8 Random Variables 1 Chapter Learning Objectives After careful study of this chapter you should be able to do the following: 1. Understand and describe sample spaces and events for random experiments with graphs, tables, lists, or tree diagrams 2. Interpret probabilities and use probabilities of outcomes to calculate probabilities of events in discrete sample spaces 3. Use permutation and combinations to count the number of outcomes in both an event and the sample space 4. Calculate the probabilities of joint events such as unions and intersections from the probabilities of individuals events 5. Interpret and calculate conditional probabilities of events 6. Determine the independence of events and use independence to calculate probabilities 7. Use Bayes theorem to calculate conditional probabilities 8. Understand random variables 2 Random Experiments Dfiii Definition: Dfiii Definition: Sample Spaces 3 E.g. if we are to flip a (typical) coin once, the possible outcomes are heads or tails (that we can denote ` H and T respectively) to give: ` S={H, T} ` If our experiment ``````` involves flipping a single coin ` twice we get: ` S = { HH, HT, TH, TT } 4 `
More Sample Space Examples More Sample Space Examples Example 2-1 Example 2-1 (continued) ` ` ` ``````` ` ` ` ``````` 5 ` ` ` ``````` 6 Discrete v. Continuous Sample Spaces Discrete: The ` number of members of the sample space is finite it or countably ` infinite ` ti ``````` The number ` of members of the sample space is infinite ` and uncountable ` ``````` A countably infinite set has the same number of members as the set of ` Continuous: Notes: positive integers (i.e. you can count them but it ll take you forever) An uncountable set has the same number of members as a portion of the real line (e.g. there s an uncountable number of real numbers between 0 and 1) 7 More Sample Space Examples Example 2-2 ` ` ` ``````` ` ` ` 8
More Sample Space Examples Example 2-2 (extended) Where n means a failed connector and y an acceptable one In this case, the sample space is discrete (as the number of outcomes is countably infinite) 9 The Outcomes in a Sample Space Are Not Necessarily Equally Likely In general, the outcomes in a sample space are not equally likely: If the coin used in our coin-flipping examples is fair, f each outcome is equally likely In all of our other examples so far, the outcomes are not equally likely Another example with unequally likely outcomes: Suppose we have a population of 100 items, n of which are defective, and we draw `````` a sample of two items Denoting a good item by g and a defective item by d, the sample `````` space is as follows (provided that n>=2): `````` S = {gg, gd, dg, dd}... if the ordering of the samples is important `````` or `````` S = {gg, gd, dd}... if the ordering of the samples is unimportant ` Q. what s the sample space if there is only a single defective item in the population? 10 Sampling With (and Without) Replacement Visualizing a Sample Space via a Tree Digram In experiments where samples are taken from a population, it is significant whether or not this is done with or without replacement: When sampling with replacement, the sampled item is replaced into the population before the next sample is drawn When sampling without replacement, sampled items are not returned dto the population E.g. if two items are sampled (one at a time) from the population {a, b, c} the sample space would be one of: S without = `````` {ab, ac, ba, bc, ca, cb} S with = {aa, ab, ac, bb, ba, bc, cc, ca, cb} `````` ```````` 11 When a sample space can be constructed in several steps (or stages), we can represent it using a tree diagram Construction procedure for a tree diagram: Start at the root of the tree (though this is typically at the top of the diagram, not the bottom) Draw a branch (from the root) to represent each of the n 1 outcomes at the first stage From the end of each branch, draw a branch to represent each of the n 2 outcomes at the second stage Continue in a similar fashion for all subsequence stages 12
A Tree Diagram Example Example 2-3 Dfiii Definition: Events An event typically represents a collection of related outcomes that may be of signifcance Since an event is a set (remember, the sample space is a set), we can manipulate events using set operators to form other events of significance 13 14 ` Event (Set) Operators ` 15 Example 2-6 An Event Example 16
Mutually Exclusive Events Dfiii Definition: `````` This says that the intersection of two mutually exclusive events is the empty set I.e. these events have no outcomes in common Visualizing Sample Spaces and Events Using Venn Diagrams 17 Figure 2-8 Venn diagrams. 18 Determining the Size of a Sample Space The term counting techniques is used to refer to formulae for computing the size of a sample space (or event) One such technique is the multiplication rule: More Counting Techniques: Permutations Permutations: 19 Permutations of subsets: ` ` ` ` ` ` ` ` 20
A Permutations Example Permutations of subsets: Example 2-10 Further Permutations Permutations of Similar Objects 21 22 Another Permutations Example Permutations of Similar Objects: Example 2-11 Another Permutations Example Permutations of Similar Objects: Example 2-12 `````` 23 24
More Counting Techniques: Combinations Combinations A Combinations Example Combinations: Example 2-13 `````` 25 26 Interpretations of Probability A Simple Frequentist Example Two ````` philosophical approaches are ````` commonly taken: ````` ````` ````` ```` The relative frequency of occurrence, in a long run ````` ```` of trials, of some type of event (e.g. g a coin turns up heads ) ` ```` ```` j ( y ) ```` The degree of belief in a statement, or the extent to ```` which it is supported by the available evidence (e.g. the ``````` Calgary Flames will win the 2010 home opener) Frequentist (aka objective): Subjective (aka Bayesian): 27 2-2.1 Introduction Figure 2-10 Relative frequency of corrupted pulses sent over a communications channel. 28
Dealing With Discrete Sample Spaces For the special case where each member of a sample space is equally likely to occur: A Simple Example on Event Probability (Example 2-15) Let the event E be that t one of the 30 diodes d meeting power requirements is chosen, then P(E) is determined as follows: To determine the probability of an event: 29 30 Another Example on Event Probability Example 2-16 The Axioms of Mathematical Probability 31 32
Addition Rules The probability of the union of two events: The probability of the union of three events: ents: A More General Definition of Mutually Exclusive Events Slide #18 defined mutual exclusivity for two events The concept generalizes to the case of k events as follows (where 1 i k; 1 j k; i j): `````` 33 34 An Example of Four Mutually Exclusive Events Figure 2-12 Venn diagram of four mutually exclusive events 35 Conditional Probability This concept deals with how the probability of some event A should be reevaluated if we know that some other event B has occurred Consider a manufacturing process example: 10% of items produced contain a visible surface flaw and 25% of these are functionally defective 5% of the parts without a visible surface flaw are functionally defective Let D denote the event that a part is defective and let F denote the event that a part has a surface flaw Then, we let P(D F) denote the conditional probability of D given F, i.e. the probability that a part is defective, given that the part has a surface flaw 36
Visualizing This Example Using a Venn Diagram Conditional Probability Computations The defining computation: ````` ````` ````` ````` ``` The multiplication rule: Figure 2-13 Conditional probabilities for parts with surface flaws 37 38 Example 2-26 A Conditional Probability Example Using a Tree Diagram to Display Conditional Probabilities The data on 400 parts in the table below can be visualized in a tree diagram: `````` 39 40
Random Samples and Conditional Probability To select from a batch randomly implies that at each step of the sample, the items that remain in the batch are equally likely to be selected Example: 50 parts in total, 10 from machine 1 and 40 from machine 2 If 2 parts are selected randomly y( (without replacement) what s the probability that the 1 st is from m/c 1 and the 2 nd from m/c 2? The Total Probability Rule This gives a way of determining the probability of an event if enough conditional probabilities of the event are known ```` Let E nd 1 =first part selected is from machine 1; E 2 =2 part selected is from machine 2 ```` Since the sampling is ```` random, P(E 1 )=10/50 and P(E 2 E 1 )=40/49 Thus P(E 1 E 2 ) = P(E 2 E 1 ). P(E 1 ) = 40/49. 10/50 = 8/49 ```` ```` ```` 41 42 The Total Probability Rule(s) Total Probability Rule (for two events): A Total Probability Rule Example Example 2-27: semiconductor failure (and contamination level): Total Probability Rule (for multiple events): 43 44
Independence A Simple Independence Example The two-event case: `````` The multiple-event case: 45 46 Bayes Theorem Preamble: rearrange eqn. 2-10 from earlier: A Bayes Theorem Example (Ex. 2-37) The theorem: 47 48
Definition: Random Variables Types and Examples of Random Variable Variable Discrete v. continuous random variables: Distinguishing random variables from real experimental outcomes: Examples: 49 50