Chapter 1 (Basic Probability) What is probability? Consider the following experiments: 1. Count the number of arrival requests to a web server in a day. 2. Determine the execution time of a program. 3. No of calls received at the IR help desk. What is common to the above experiments? Random phenomena in computer environments: Arrival of jobs. Execution time. Memory requirement. Failure of components. Exposure to viruses. Errors in codes. Etc. Random phenomena elsewhere: Economy: Stock prices, number of jobs, price of oil, etc. Environment: Temperature, earthquakes, rainfall, etc. Going to UTD: # of green lights, available parking, etc. This class: Scores, time spent on each topic, etc. This course: Model uncertainty, quantify uncertainty and make decisions under uncertainty. Probability means Chance (common language) Odds (gambling) Long-term proportion (relative frequency) Likelihood (forecasting) Finite measure (mathematics) Probability = function of an event = P(E) What is an event? Consider a random experiment. Its results are outcomes. Sample space S = {all possible outcomes} Event E = a combination of outcomes = a subset of S Event E occurs when the experiment results in an outcome that is contained in E. 1
Range of probability: Ex: Find S for the experiments 1-3 mentioned at the beginning. Ex: Consider the execution of an if statement twice: if then T else E. What is S? Ex: Consider a system with two components. The experiment consists of observing the status of the components: W (working) or F (failed). Describe the following events. (a) At least one component has failed. (b) Exactly one component has failed. Finding probability of an event E: If E = {O 1, O 2,, O k } = {outcomes}, then P(E) = An empty event φ = {} has P(φ) = 0. P(S) = Some set operations: E 1, E 2,, E n events (sets of outcomes) Union of events E 1, E 2,, E n is an event that consists of all outcomes of E 1, E 2,, E n occurs if any of E 1, E 2,, E n occurs (or at least one of E 1, E 2,, E n occurs). is denoted as E 1 E 2 E n = {E 1 or E 2 or or E n } Venn Diagram: 2
Intersection of events E 1, E 2,, E n is an event that consists of common outcomes of E 1, E 2,, E n occurs if each E 1, E 2,, E n occurs is denoted as E 1 E 2 E n = {E 1 and E 2 and and E n } Venn Diagram: Complement of an event E is an event that consists of outcomes that are not in E occurs if E does not occur is denoted as E C = {not E}. Venn Diagram: What is the Venn Diagram for the event A C B? Disjoint or mutually exclusive events are those that cannot occur together their intersection = {} Venn diagram: Exhaustive events are those whose union = Sample space at least occurs for sure. Example =? 3
Basic rules of probability For any event E, P({}) = and P(S) = For disjoint events E 1, E 2,, E n : P(E 1 E 2 E n ) = For any two events E 1 and E 2 : P(E 1 E 2 ) = Complement rule. For event E and its complement E C : P(E E C ) = Which one should we compute, P(E) or P(E C )? Independent events: P(E 1 E 2 ) = P(E 1 ) P(E 2 ) P(E 1 E 2 E n ) = P(E 1 ) P(E 2 ) P(E n ) Basic idea: One event has information about the other event. Q. Can disjoint events be independent? Q. Can exhaustive events be independent? (More on independence later.) How to find probability of an event? Recall: If E = {O 1, O 2,, O k } = {outcomes}, then P(E) = Q: How to assign probabilities to the outcomes? 4
Classical approach for assigning probability: Assume a mathematical model for the outcomes in S. Popular model: When S = finite = {O 1, O 2,, O N }, assume that the outcomes {O i } are equally likely, i.e., P(O 1 ) = P(O 2 ) = = P(O N ) = Then, for any event E, P(E) = So, the problem reduces to that of just counting. We will talk more about counting techniques later. Equally likely outcomes Random sampling Tossing a fair coin Not equally likely outcomes Errors in different modules of a program Tossing an unfair coin Market ups and downs Ex: Suppose a dice is such that the even outcomes are twice as likely to occur as the odd outcomes; all the even outcomes are equally likely and all the odd outcomes are equally likely. O i = Outcome i occurs. (a) Find P(O i ) for i=1,2,3,4,5,6. (c) A = outcome is odd. Find P(A). (d) B = 3 outcome 5. Find P(B). 5
Relative frequency approach for assigning probability: Repeat the experiment a large number of times under identical conditions. # E = # times the event E occurs in n repetitions. P(E) = So, approximately, P(E) = long-term proportion of times E occurs. Ex: How to find P(It will rain tomorrow)? Subjective approach: If the experiment is either theoretically or practically unrepeatable, then use your personal degree of belief to assign probability. Note: Irrespective of which approach is used to assign probabilities, we generally use the relative frequency approach to interpret probabilities. Thus, P(E) is interpreted as the proportion of times the event A will occur in a large number of identical replications of the experiment. Ex: The experience of a computer manufacturer shows that 30% of the customers buy a flat screen monitor, 60% buy a printer and 20% buy both. (a) What is the probability that a customer will buy at least one of the two options? (b) What is the probability that a customer buys none of the two options? 6