Sample Space: Specify all possible outcomes from an experiment. Event: Specify a particular outcome or combination of outcomes.
|
|
- Deirdre Reeves
- 5 years ago
- Views:
Transcription
1 Chapter 2 Introduction to Probability 2.1 Probability Model Probability concerns about the chance of observing certain outcome resulting from an experiment. However, since chance is an abstraction of something not physically measurable, we need a precise mathematical definition. In this chapter, we will use the follow definition of the probability model, which consists of three components: Sample Space: Specify all possible outcomes from an experiment. Event: Specify a particular outcome or combination of outcomes. Probability Law: Specify how likely should an event occur. A pictorial summary of these concepts are illustrated in Figure 2.1. Figure 2.1: Illustration of sample space, event and probability law. 1
2 Sample Space Definition 1. A sample space Ω is the set of all possible outcomes from an experiment. We denote ω as an element in Ω. Examples: Coin flip: Ω = {H, T }. Throw a dice: Ω = {1, 2, 3, 4, 5, 6}. Waiting time for a bus in West Lafayette: Ω = {t 0 t 30minutes} In the last example, we see that a sample space can be continuous. Counter Examples: Throw a dice: Ω = {1, 2, 3} is not a sample space because it is not exhaustive. Throw a dice: Ω = {1, 1, 2, 3, 4, 5, 6} is not a sample space because it is not exclusive. Therefore, in order to make a valid sample space, we have to make sure Ω contains all possible outcomes and there is no repetition in the outcomes. Event Definition 2. An event F is a subset in the sample space Ω. Note that an outcome ω is an element in Ω but an event F is a subset contained in Ω, i.e., F Ω. Thus, an event can contain one single outcome but it can also contain many elements. Example: Throw a dice. Let Ω = {1, 2, 3, 4, 5, 6}. F 1 = {even numbers} = {2, 4, 6}. F 2 = {less than 3} = {1, 2}. Example: Wait a bus. Let Ω = {0 t 30}. F 1 = {wait less than 10 minutes} = {0 t < 10} F 2 = {wait less than 5 or more than 20 minutes} = {0 t < 5} {20 < t 30}. c 2017 Stanley Chan. All Rights Reserved. 2
3 In this example, we see that we can create a new event by operating on existing events through set operations, e.g., union, intersection, etc. Formally, we define the collection of all possible events as the event space. Definition 3. The collection of all possible events is called the event space or the σ-field, denoted as F. An event space satisfies the following two properties: If F F, then also F c F. If F 1, F 2,... F, then the union F i F. The two properties of the event space are essential to ensure that every possible subsets in Ω are included, because any other set operations can be derived from complement and union. Example. In a coin flip experiment where Ω = {H, T }, the event space is F = {, H, T, Ω}. Probability Law Definition 4. A probability law is a function P : F [0, 1] that maps an event A to a real number in [0, 1]. The function must satisfy three axioms known as the axioms of probability: I. Non-negativity: P[A] 0, for any A Ω. II. Normalization: P[Ω] = 1. III. Additivity: For any disjoint sets {A 1, A 2,...}, it holds that [ ] P A i = P[A i ]. The non-negativity axiom ensures that a probability value cannot be negative. The normalization axioms ensures that the probability of observing all possible outcomes is 1. The additivity axiom defines how set operations can be translated into probability operations. The infinite number of sets in the axiom makes sure that it is applicable to both discrete and continuous sample spaces. Finite Additivity. The countable additivity stated in the axiom III involves an infinite number of sets. As a special case we can simplify the infinite number into finite number, which states that for any two disjoint sets A and B, we have P[A B] = P[A] +. c 2017 Stanley Chan. All Rights Reserved. 3
4 In words, if A and B are disjoint, then the probability of observing either A or B is the sum of the two individual probabilities. The union of A and B is equivalent to the logical OR. Once this OR operation is defined, all other logical operations can be subsequently defined. The following corollaries are some examples. Corollary 1. P[A c ] = 1 P[A]. Proof. Since Ω = A A c, by finite additivity we have P[Ω] = P[A A c ] = P[A] + P[A c ]. By normalization axiom, we have P[Ω] = 1. Therefore, P[A c ] = 1 P[A]. Corollary 2. For any A Ω, P[A] 1. Proof. We prove by contradiction. Assume P[A] > 1. Consider the complement A c where A A c = Ω. Since P[A c ] = 1 P[A], we must have P[A c ] < 0 because by hypothesis P[A] > 1. But P[A c ] < 0 violates the non-negativity axiom. So we must have P[A] 1. Corollary 3. P[ ] = 0. Proof. Since Ω = Ω, by the first corollary we have P[ ] = 1 P[Ω] = 0. Corollary 4. For any A and B, P[A B] = P[A] + P[A B]. Note that this statement is different from axiom III because A and B are not necessarily disjoint. Proof. First, observe that A B can be partitioned into three disjoint subsets as A B = (A\B) (A B) (B\A). Since A\B = A B c and B\A = B A c, by finite additivity we have that P[A B] = P[A\B] + P[A B] + P[B\A] = P[A B c ] + P[A B] + P[B A c ] (a) = P[A B c ] + P[A B] + P[B A c ] + P[A B] P[A B] (b) = P[A (B c B)] + P[(A c A) B] P[A B] = P[A Ω] + P[Ω B] P[A B] = P[A] + P[A B], where in (a) we added and subtracted a term P[A B], and in (b) we used finite additivity so that P[A B c ] + P[A B] = P[(A B c ) (A B)] = P[A (B c B)]. The above proof is a rigorous way of deriving the result. A simpler way to visualize the result is to draw a Venn diagram and show that A B contains an overlapping part A B which needs to be subtracted. c 2017 Stanley Chan. All Rights Reserved. 4
5 Corollary 5 (Union Bound). For any A and B, P[A B] P[A] +. Proof. Since P[A B] = P[A] + P[A B] and by non-negativity axiom P[A B] 0, we must have P[A B] P[A] +. Union bound is a common tool we use to analyze probabilities when the intersection A B is difficult to evaluate. Corollary 6. If A B, then P[A]. Proof. If A B, then there exists a set B\A such that B = A (B\A). Therefore, by finite additivity we have = P[A] + P[B\A] P[A]. This corollary is useful when considering two events of different sizes. For example, in the bus waiting example, if we let A = {t 5}, and B = {t 10}, then P[A] because we have to wait for the first 5 minutes in order to go into the remaining 5 minutes. 2.2 Conditional Probability Definition 5. Assume 0. The conditional probability of A given B is P[A B] def = P[A B]. (2.1) Pictorially, a conditional probability is the proportion of P[A B] over. It is the probability that A happens when we know that B has already happened. The difference between P[A B] and P[A B] is the denominator they carry: P[A B] = P[A B] and P[A B] = P[A B]. P[Ω] Since P[Ω], P[A B] is always larger than or equal to P[A B]. Conditional probabilities are ubiquitous in this course and beyond. They concern about the likelihood that one event happens subject to another event has happened. This notion of causality is common in our daily lives. The followings are some examples. Example. Throw a dice. Let A = {Get 3} and B = {odd numbers}. c 2017 Stanley Chan. All Rights Reserved. 5
6 Figure 2.2: Illustration of conditional probability and its comparison with P[A B]. Clearly, P[A] = 1/6 and = 1/2. It is also not difficult to see that P[A B] = P[A] = 1/6 because A B and so A B = A. The conditional probability of A given B is P[A B] = P[A B] = 1 3. In words, if we know that we have an odd number, then the probability of obtaining a 3 has to be computed over {1, 3, 5}, which give us a probability 1. If we do not know that we have 3 an odd number, then the probability of obtaining a 3 has to be computed from the sample space {1, 2, 3, 4, 5, 6} which will give us 1. 6 Example. Let In this example, A = {Eat 2 burgers} and B = {Finish a football game}. P[A] = Probability that you eat 2 burgers = Probability that you just finish a football game P[A B] = Probability that you just finish a football game and you eat 2 burgers P[A B] = Probability that you eat 2 burgers given that you just finish a football game. Without knowing that you just finish a football game, you may not be hungry and so the probability of eating 2 burgers (i.e. P[A]) could be low. However, if we know that you finish a football game, then it is quite likely you are hungry and want to eat 2 burgers. This is the conditional probability P[A B]. Example. Let A = {Purdue gets Big Ten champion} and B = {Purdue wins 15 games consecutively}. c 2017 Stanley Chan. All Rights Reserved. 6
7 In this example, P[A] = Probability that Purdue gets chamion = Probability that Purdue wins 15 games consecutively P[A B] = Probability that Purdue gets champion and wins 15 games consecutively P[A B] = Probability that Purdue gets champion given that we win 15 games consecutively. If we do not know if Purdue has won 15 games consecutively, then it is unlikely that we will get the champion because the sample space of all possible competition reults is large. However, if we have already won 15 games consecutively, then the denominator of the probability becomes much smaller. In this case, the conditional probability is high. Proposition 1. Let > 0. The conditional probability P[A B] satisfies Axiom I to Axiom III. Proof. Let s check the axioms: Axiom I: P[A B] = P[A B]. Since > 0 and Axiom II requires P[A B] 0, we therefore have P[A B] 0. Axiom II: P[Ω B] = P[Ω B] = = 1. Axiom III: Consider two disjoint sets A and C. Then, P[(A C) B] P[(A B) (C B)] P[A C B] = = (a) P[A B] P[C B] = + = P[A B] + P[C B], where (a) holds because if A and C are disjoint then A B and C B are also disjoint. The implication of Proposition 1 is that conditional probabilities are legitimate probabilities. In proving the proposition, we note that the set B is present and is fixed for all three axioms. 2.3 Independence Definition 6. Two events A and B are statistically independent if P[A B] = P[A]. Disjoint VS Independent. It should be cautioned that disjoint and independent are two different concepts, i.e., Disjoint Independence. c 2017 Stanley Chan. All Rights Reserved. 7
8 If A and B are disjoint, then A B =. This only implies that P[A B] = 0. However, it says nothing about if P[A B] can be factorized into P[A]. If A and B are independent, then we have P[A B] = P[A]. But this does not imply that P[A B] = 0. The only possibility that Disjoint Independence is when P[A] = 0 or = 0. Example. Throw a dice twice. Let A = {1st dice is 3} and B = {2nd dice is 4}. Are A and B independent? We can show that P[A B] = P[(3, 4)] = 1 36 P[A] = 1 6, and = 1 6. So P[A B] = P[A]. Thus, A and B are independent. Example. Throw a dice twice. Let A = {1st dice is 1} and B = {sum is 7}. Are A and B independent? Note that P[A B] = P[(1, 6)] = 1 36 P[A] = 1 6 = P[(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)] = 1 6. So P[A B] = P[A]. Thus, A and B are independent. Example. Throw a dice twice. Let A = {max is 2} and B = {min is 2}. Are A and B independent? Let us first list out A and B: A = {(1, 2), (2, 1), (2, 2)} B = {(2, 2), (2, 3), (2, 4), (2, 5), (2, 6), (3, 2), (4, 2), (5, 2), (6, 2)}. c 2017 Stanley Chan. All Rights Reserved. 8
9 Therefore, the probabilities are P[A] = 3 36 and = 9 36 P[A B] = P[(2, 2)] = Clearly, P[A B] P[A] and so A and B are dependent. Independence Via Conditional Probability. Recall that P[A B] = P[A B]. If A and B are independent, then P[A B] = P[A] and so P[A B] = P[A B] = P[A] = P[A]. This suggests an interpretation of independence: If the occurrence of B provides no additional information about the occurrence of A, then A and B are independent. However, we do not define independence via conditional probability because P[A B] = P[A] holds for 0 but conditional probability P[A B] requires > Bayes Theorem and Law of Total Probability Theorem 1. (Bayes Theorem) For any two events A and B such that P[A] > 0 and > 0, it holds that P[B A] P[A] P[A B] =. Proof. By definition of conditional probabilities, we have P[A B] = P[A B] and P[B A] = P[B A]. P[A] Rearranging the terms yields P[A B] = P[B A]P[A], which gives the desired result by dividing both sides by. Bayes Theorem provides two views of the intersection P[A B] using two different conditional probabilities. See Figure 2.3 for a pictorial illustration. We call P[B A] the conditional probability of B given A, and P[A B] the posterior probability of B given A. c 2017 Stanley Chan. All Rights Reserved. 9
10 Figure 2.3: Bayes theorem provides two views of P[A B] using P[A B] and P[B A]. Theorem 2. (Law of Total Probability) Let {A 1, A 2,..., A n } be a partition of Ω, i.e., A 1,..., A n are disjoint and Ω = A 1 A 2... A n. Then, for any B Ω, = n P[B A i ] P[A i ]. Proof. We start from the right hand side. [ n n n ] P[B A i ] P[A i ] (a) = P[B A i ] (b) = P (B A i ) (c) = P [ ( n )] B A i (d) = P[B Ω] =, where (a) follows from the definition of conditional probability, (b) is due to Axiom III, (c) holds because of the distributive property of sets, and (d) is resulted from the partition property of {A 1, A 2,..., A n }. Interpretation. Law of total probability can be understood as follows. If the sample space Ω consists of disjoint subset A 1,..., A n, then we can compute the probability by summing over its portion P[B A 1 ],..., P[B A n ]. However, the probability of having A 1,..., A n is determined by P[A 1 ],..., P[A n ]. Therefore, when performing the sum we need to weight each P[B A i ] by P[A i ]. See Figure 2.4 for illustration. Corollary 7. Let {A 1, A 2,..., A n } be a partition of Ω, i.e., A 1,..., A n Ω = A 1 A 2... A n. Then, for any B Ω, are disjoint and P[A j B] = P[B A j] P[A j ] n P[B A i] P[A i ]. c 2017 Stanley Chan. All Rights Reserved. 10
11 Figure 2.4: Law of total probability decomposes the probability into multiple conditional probabilities P[B A i ]. The probability of obtaining each P[B A i ] is P[A i ]. Proof. We just need to apply Bayes Theorem and Law of Total Probability: P[A j B] = P[B A j] P[A j ] = P[B A j ] P[A j ] n P[B A i] P[A i ]. Example. Consider a communication channel shown in Figure 2.5. The probability of sending a 1 is p and the probability of sending a 0 is 1 p. Given that 1 is sent, the probability of receiving 1 is 1 η. Given that 0 is sent, the probability of receiving 0 is 1 ε. We want to find the probability that a 1 has been correctly received. Define the events S 0 = 0 is sent, and R 0 = 0 is received. S 1 = 1 is sent, and R 1 = 1 is received. Then, the probability that 1 is received is P[R 1 ]. However, P[R 1 ] 1 η because 1 η is the conditional probability that 1 is received given 1 is sent. It is possible that we receive 1 as a result of an error when 0 is sent. Therefore, we need to consider the probabilities of having S 0 and S 1. Using Law of total probability we have P[R 1 ] = P[R 1 S 1 ] P[S 1 ] + P[R 1 S 0 ] P[S 0 ] = (1 η)p + ε(1 p). Now, suppose that we have received 1. What is the probability that 1 was originally sent? This is asking the posterior probability P[S 1 R 1 ], which can be found using Bayes Theorem P[S 1 R 1 ] = P[R 1 S 1 ] P[S 1 ] P[R 1 ] = (1 η)p (1 η)p + ε(1 p). c 2017 Stanley Chan. All Rights Reserved. 11
12 Figure 2.5: A two-channel communication system. Example. Consider a tennis tournament. Your probability of winning the game is 0.3 against against against 1 4 of the players (Event A). of the players (Event B). of the players (Event C). What is the probability of winning the game? Let W be the event that you win the game. Then, by Law of Total Probability, we have P[W ] = P[W A] P[A] + P[W B] + P[W C] P[C] = (0.3)(0.5) + (0.4)(0.25) + (0.5)(0.25) = c 2017 Stanley Chan. All Rights Reserved. 12
ECE 302: Chapter 02 Probability Model
ECE 302: Chapter 02 Probability Model Fall 2018 Prof Stanley Chan School of Electrical and Computer Engineering Purdue University 1 / 35 1. Probability Model 2 / 35 What is Probability? It is a number.
More informationLecture Notes 1 Basic Probability. Elements of Probability. Conditional probability. Sequential Calculation of Probability
Lecture Notes 1 Basic Probability Set Theory Elements of Probability Conditional probability Sequential Calculation of Probability Total Probability and Bayes Rule Independence Counting EE 178/278A: Basic
More informationCMPSCI 240: Reasoning about Uncertainty
CMPSCI 240: Reasoning about Uncertainty Lecture 2: Sets and Events Andrew McGregor University of Massachusetts Last Compiled: January 27, 2017 Outline 1 Recap 2 Experiments and Events 3 Probabilistic Models
More informationLECTURE 1. 1 Introduction. 1.1 Sample spaces and events
LECTURE 1 1 Introduction The first part of our adventure is a highly selective review of probability theory, focusing especially on things that are most useful in statistics. 1.1 Sample spaces and events
More informationStatistical Inference
Statistical Inference Lecture 1: Probability Theory MING GAO DASE @ ECNU (for course related communications) mgao@dase.ecnu.edu.cn Sep. 11, 2018 Outline Introduction Set Theory Basics of Probability Theory
More informationDiscrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14
CS 70 Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14 Introduction One of the key properties of coin flips is independence: if you flip a fair coin ten times and get ten
More informationVenn Diagrams; Probability Laws. Notes. Set Operations and Relations. Venn Diagram 2.1. Venn Diagrams; Probability Laws. Notes
Lecture 2 s; Text: A Course in Probability by Weiss 2.4 STAT 225 Introduction to Probability Models January 8, 2014 s; Whitney Huang Purdue University 2.1 Agenda s; 1 2 2.2 Intersection: the intersection
More informationFundamentals of Probability CE 311S
Fundamentals of Probability CE 311S OUTLINE Review Elementary set theory Probability fundamentals: outcomes, sample spaces, events Outline ELEMENTARY SET THEORY Basic probability concepts can be cast in
More informationDept. of Linguistics, Indiana University Fall 2015
L645 Dept. of Linguistics, Indiana University Fall 2015 1 / 34 To start out the course, we need to know something about statistics and This is only an introduction; for a fuller understanding, you would
More information2011 Pearson Education, Inc
Statistics for Business and Economics Chapter 3 Probability Contents 1. Events, Sample Spaces, and Probability 2. Unions and Intersections 3. Complementary Events 4. The Additive Rule and Mutually Exclusive
More informationOrigins of Probability Theory
1 16.584: INTRODUCTION Theory and Tools of Probability required to analyze and design systems subject to uncertain outcomes/unpredictability/randomness. Such systems more generally referred to as Experiments.
More informationProbability- describes the pattern of chance outcomes
Chapter 6 Probability the study of randomness Probability- describes the pattern of chance outcomes Chance behavior is unpredictable in the short run, but has a regular and predictable pattern in the long
More informationEE 178 Lecture Notes 0 Course Introduction. About EE178. About Probability. Course Goals. Course Topics. Lecture Notes EE 178
EE 178 Lecture Notes 0 Course Introduction About EE178 About Probability Course Goals Course Topics Lecture Notes EE 178: Course Introduction Page 0 1 EE 178 EE 178 provides an introduction to probabilistic
More informationP (A B) P ((B C) A) P (B A) = P (B A) + P (C A) P (A) = P (B A) + P (C A) = Q(A) + Q(B).
Lectures 7-8 jacques@ucsdedu 41 Conditional Probability Let (Ω, F, P ) be a probability space Suppose that we have prior information which leads us to conclude that an event A F occurs Based on this information,
More informationRecitation 2: Probability
Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions
More informationProbability 1 (MATH 11300) lecture slides
Probability 1 (MATH 11300) lecture slides Márton Balázs School of Mathematics University of Bristol Autumn, 2015 December 16, 2015 To know... http://www.maths.bris.ac.uk/ mb13434/prob1/ m.balazs@bristol.ac.uk
More informationNotes 1 Autumn Sample space, events. S is the number of elements in the set S.)
MAS 108 Probability I Notes 1 Autumn 2005 Sample space, events The general setting is: We perform an experiment which can have a number of different outcomes. The sample space is the set of all possible
More informationthe time it takes until a radioactive substance undergoes a decay
1 Probabilities 1.1 Experiments with randomness Wewillusethetermexperimentinaverygeneralwaytorefertosomeprocess that produces a random outcome. Examples: (Ask class for some first) Here are some discrete
More informationProbability (Devore Chapter Two)
Probability (Devore Chapter Two) 1016-345-01: Probability and Statistics for Engineers Fall 2012 Contents 0 Administrata 2 0.1 Outline....................................... 3 1 Axiomatic Probability 3
More informationInformation Science 2
Information Science 2 Probability Theory: An Overview Week 12 College of Information Science and Engineering Ritsumeikan University Agenda Terms and concepts from Week 11 Basic concepts of probability
More informationLecture 1. ABC of Probability
Math 408 - Mathematical Statistics Lecture 1. ABC of Probability January 16, 2013 Konstantin Zuev (USC) Math 408, Lecture 1 January 16, 2013 1 / 9 Agenda Sample Spaces Realizations, Events Axioms of Probability
More informationLecture notes for probability. Math 124
Lecture notes for probability Math 124 What is probability? Probabilities are ratios, expressed as fractions, decimals, or percents, determined by considering results or outcomes of experiments whose result
More informationWhy study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables
ECE 6010 Lecture 1 Introduction; Review of Random Variables Readings from G&S: Chapter 1. Section 2.1, Section 2.3, Section 2.4, Section 3.1, Section 3.2, Section 3.5, Section 4.1, Section 4.2, Section
More informationConditional Probability and Bayes
Conditional Probability and Bayes Chapter 2 Lecture 5 Yiren Ding Shanghai Qibao Dwight High School March 9, 2016 Yiren Ding Conditional Probability and Bayes 1 / 13 Outline 1 Independent Events Definition
More informationIntroduction to Probability
Introduction to Probability Gambling at its core 16th century Cardano: Books on Games of Chance First systematic treatment of probability 17th century Chevalier de Mere posed a problem to his friend Pascal.
More informationLecture 3 Probability Basics
Lecture 3 Probability Basics Thais Paiva STA 111 - Summer 2013 Term II July 3, 2013 Lecture Plan 1 Definitions of probability 2 Rules of probability 3 Conditional probability What is Probability? Probability
More informationChap 1: Experiments, Models, and Probabilities. Random Processes. Chap 1 : Experiments, Models, and Probabilities
EE8103 Random Processes Chap 1 : Experiments, Models, and Probabilities Introduction Real world word exhibits randomness Today s temperature; Flip a coin, head or tail? WalkAt to a bus station, how long
More informationConditional Probability
Chapter 3 Conditional Probability 3.1 Definition of conditional probability In spite of our misgivings, let us persist with the frequency definition of probability. Consider an experiment conducted N times
More informationI - Probability. What is Probability? the chance of an event occuring. 1classical probability. 2empirical probability. 3subjective probability
What is Probability? the chance of an event occuring eg 1classical probability 2empirical probability 3subjective probability Section 2 - Probability (1) Probability - Terminology random (probability)
More informationAxioms of Probability
Sample Space (denoted by S) The set of all possible outcomes of a random experiment is called the Sample Space of the experiment, and is denoted by S. Example 1.10 If the experiment consists of tossing
More informationChapter 3 : Conditional Probability and Independence
STAT/MATH 394 A - PROBABILITY I UW Autumn Quarter 2016 Néhémy Lim Chapter 3 : Conditional Probability and Independence 1 Conditional Probabilities How should we modify the probability of an event when
More informationAxiomatic Foundations of Probability. Definition: Probability Function
Chapter 1 sections We will SKIP a number of sections Set theory SKIP Real number uncountability Definition of probability Finite sample spaces Counting methods Combinatorial methods SKIP tennis tournament
More informationProbability and Independence Terri Bittner, Ph.D.
Probability and Independence Terri Bittner, Ph.D. The concept of independence is often confusing for students. This brief paper will cover the basics, and will explain the difference between independent
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 3 9/10/2008 CONDITIONING AND INDEPENDENCE
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 3 9/10/2008 CONDITIONING AND INDEPENDENCE Most of the material in this lecture is covered in [Bertsekas & Tsitsiklis] Sections 1.3-1.5
More informationECE353: Probability and Random Processes. Lecture 2 - Set Theory
ECE353: Probability and Random Processes Lecture 2 - Set Theory Xiao Fu School of Electrical Engineering and Computer Science Oregon State University E-mail: xiao.fu@oregonstate.edu January 10, 2018 Set
More informationtossing a coin selecting a card from a deck measuring the commuting time on a particular morning
2 Probability Experiment An experiment or random variable is any activity whose outcome is unknown or random upfront: tossing a coin selecting a card from a deck measuring the commuting time on a particular
More informationIntroduction and basic definitions
Chapter 1 Introduction and basic definitions 1.1 Sample space, events, elementary probability Exercise 1.1 Prove that P( ) = 0. Solution of Exercise 1.1 : Events S (where S is the sample space) and are
More informationWith Question/Answer Animations. Chapter 7
With Question/Answer Animations Chapter 7 Chapter Summary Introduction to Discrete Probability Probability Theory Bayes Theorem Section 7.1 Section Summary Finite Probability Probabilities of Complements
More informationPreliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com
1 School of Oriental and African Studies September 2015 Department of Economics Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com Gujarati D. Basic Econometrics, Appendix
More informationTopic 5 Basics of Probability
Topic 5 Basics of Probability Equally Likely Outcomes and the Axioms of Probability 1 / 13 Outline Equally Likely Outcomes Axioms of Probability Consequences of the Axioms 2 / 13 Introduction A probability
More informationModule 1. Probability
Module 1 Probability 1. Introduction In our daily life we come across many processes whose nature cannot be predicted in advance. Such processes are referred to as random processes. The only way to derive
More informationChapter 1: Introduction to Probability Theory
ECE5: Stochastic Signals and Systems Fall 8 Lecture - September 6, 8 Prof. Salim El Rouayheb Scribe: Peiwen Tian, Lu Liu, Ghadir Ayache Chapter : Introduction to Probability Theory Axioms of Probability
More informationMath Treibergs. Solutions to Fourth Homework February 13, 2009
Math 5010 1. Treibergs Solutions to Fourth Homework February 13, 2009 76[12] Two roads join Ayton to Beaton, and two further roads join Beaton to the City. Ayton is directly connected to the City by railway.
More informationGame Theory Lecture 10+11: Knowledge
Game Theory Lecture 10+11: Knowledge Christoph Schottmüller University of Copenhagen November 13 and 20, 2014 1 / 36 Outline 1 (Common) Knowledge The hat game A model of knowledge Common knowledge Agree
More informationPROBABILITY THEORY 1. Basics
PROILITY THEORY. asics Probability theory deals with the study of random phenomena, which under repeated experiments yield different outcomes that have certain underlying patterns about them. The notion
More informationThe probability of an event is viewed as a numerical measure of the chance that the event will occur.
Chapter 5 This chapter introduces probability to quantify randomness. Section 5.1: How Can Probability Quantify Randomness? The probability of an event is viewed as a numerical measure of the chance that
More informationECE 340 Probabilistic Methods in Engineering M/W 3-4:15. Lecture 2: Random Experiments. Prof. Vince Calhoun
ECE 340 Probabilistic Methods in Engineering M/W 3-4:15 Lecture 2: Random Experiments Prof. Vince Calhoun Reading This class: Section 2.1-2.2 Next class: Section 2.3-2.4 Homework: Assignment 1: From the
More informationMidterm #1 - Solutions
Midterm # - olutions Math/tat 94 Quizzes. Let A be the event Andrea and Bill are both in class. The complementary event is (choose one): A c = Neither Andrea nor Bill are in class A c = Bill is not in
More informationCMPSCI 240: Reasoning Under Uncertainty
CMPSCI 240: Reasoning Under Uncertainty Lecture 2 Prof. Hanna Wallach wallach@cs.umass.edu January 26, 2012 Reminders Pick up a copy of B&T Check the course website: http://www.cs.umass.edu/ ~wallach/courses/s12/cmpsci240/
More informationProbability Theory and Applications
Probability Theory and Applications Videos of the topics covered in this manual are available at the following links: Lesson 4 Probability I http://faculty.citadel.edu/silver/ba205/online course/lesson
More informationSTA Module 4 Probability Concepts. Rev.F08 1
STA 2023 Module 4 Probability Concepts Rev.F08 1 Learning Objectives Upon completing this module, you should be able to: 1. Compute probabilities for experiments having equally likely outcomes. 2. Interpret
More informationDynamic Programming Lecture #4
Dynamic Programming Lecture #4 Outline: Probability Review Probability space Conditional probability Total probability Bayes rule Independent events Conditional independence Mutual independence Probability
More informationProbability - Lecture 4
1 Introduction Probability - Lecture 4 Many methods of computation physics and the comparison of data to a mathematical representation, apply stochastic methods. These ideas were first introduced in the
More informationThe enumeration of all possible outcomes of an experiment is called the sample space, denoted S. E.g.: S={head, tail}
Random Experiment In random experiments, the result is unpredictable, unknown prior to its conduct, and can be one of several choices. Examples: The Experiment of tossing a coin (head, tail) The Experiment
More information2. AXIOMATIC PROBABILITY
IA Probability Lent Term 2. AXIOMATIC PROBABILITY 2. The axioms The formulation for classical probability in which all outcomes or points in the sample space are equally likely is too restrictive to develop
More informationChapter 2 Classical Probability Theories
Chapter 2 Classical Probability Theories In principle, those who are not interested in mathematical foundations of probability theory might jump directly to Part II. One should just know that, besides
More informationF71SM STATISTICAL METHODS
F71SM STATISTICAL METHODS RJG SUMMARY NOTES 2 PROBABILITY 2.1 Introduction A random experiment is an experiment which is repeatable under identical conditions, and for which, at each repetition, the outcome
More informationChapter Learning Objectives. Random Experiments Dfiii Definition: Dfiii Definition:
Chapter 2: Probability 2-1 Sample Spaces & Events 2-1.1 Random Experiments 2-1.2 Sample Spaces 2-1.3 Events 2-1 1.4 Counting Techniques 2-2 Interpretations & Axioms of Probability 2-3 Addition Rules 2-4
More informationProbabilistic models
Probabilistic models Kolmogorov (Andrei Nikolaevich, 1903 1987) put forward an axiomatic system for probability theory. Foundations of the Calculus of Probabilities, published in 1933, immediately became
More informationExample: Suppose we toss a quarter and observe whether it falls heads or tails, recording the result as 1 for heads and 0 for tails.
Example: Suppose we toss a quarter and observe whether it falls heads or tails, recording the result as 1 for heads and 0 for tails. (In Mathematical language, the result of our toss is a random variable,
More informationChapter 1 (Basic Probability)
Chapter 1 (Basic Probability) What is probability? Consider the following experiments: 1. Count the number of arrival requests to a web server in a day. 2. Determine the execution time of a program. 3.
More informationProbability: Axioms, Properties, Interpretations
Probability: Axioms, Properties, Interpretations Engineering Statistics Section 2.2 Josh Engwer TTU 03 February 2016 Josh Engwer (TTU) Probability: Axioms, Properties, Interpretations 03 February 2016
More informationSTAT:5100 (22S:193) Statistical Inference I
STAT:5100 (22S:193) Statistical Inference I Week 3 Luke Tierney University of Iowa Fall 2015 Luke Tierney (U Iowa) STAT:5100 (22S:193) Statistical Inference I Fall 2015 1 Recap Matching problem Generalized
More informationProbability the chance that an uncertain event will occur (always between 0 and 1)
Quantitative Methods 2013 1 Probability as a Numerical Measure of the Likelihood of Occurrence Probability the chance that an uncertain event will occur (always between 0 and 1) Increasing Likelihood of
More informationLECTURE NOTES by DR. J.S.V.R. KRISHNA PRASAD
.0 Introduction: The theory of probability has its origin in the games of chance related to gambling such as tossing of a coin, throwing of a die, drawing cards from a pack of cards etc. Jerame Cardon,
More informationProbability deals with modeling of random phenomena (phenomena or experiments whose outcomes may vary)
Chapter 14 From Randomness to Probability How to measure a likelihood of an event? How likely is it to answer correctly one out of two true-false questions on a quiz? Is it more, less, or equally likely
More information7.1 What is it and why should we care?
Chapter 7 Probability In this section, we go over some simple concepts from probability theory. We integrate these with ideas from formal language theory in the next chapter. 7.1 What is it and why should
More informationDiscrete Probability. Mark Huiskes, LIACS Probability and Statistics, Mark Huiskes, LIACS, Lecture 2
Discrete Probability Mark Huiskes, LIACS mark.huiskes@liacs.nl Probability: Basic Definitions In probability theory we consider experiments whose outcome depends on chance or are uncertain. How do we model
More informationLecture 1. Chapter 1. (Part I) Material Covered in This Lecture: Chapter 1, Chapter 2 ( ). 1. What is Statistics?
Lecture 1 (Part I) Material Covered in This Lecture: Chapter 1, Chapter 2 (2.1 --- 2.6). Chapter 1 1. What is Statistics? 2. Two definitions. (1). Population (2). Sample 3. The objective of statistics.
More informationCMPSCI 240: Reasoning about Uncertainty
CMPSCI 240: Reasoning about Uncertainty Lecture 4: Sequential experiments Andrew McGregor University of Massachusetts Last Compiled: February 2, 2017 Outline 1 Recap 2 Sequential Experiments 3 Total Probability
More informationECE 450 Lecture 2. Recall: Pr(A B) = Pr(A) + Pr(B) Pr(A B) in general = Pr(A) + Pr(B) if A and B are m.e. Lecture Overview
ECE 450 Lecture 2 Recall: Pr(A B) = Pr(A) + Pr(B) Pr(A B) in general = Pr(A) + Pr(B) if A and B are m.e. Lecture Overview Conditional Probability, Pr(A B) Total Probability Bayes Theorem Independent Events
More informationINDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. NPTEL National Programme on Technology Enhanced Learning. Probability Methods in Civil Engineering
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR NPTEL National Programme on Technology Enhanced Learning Probability Methods in Civil Engineering Prof. Rajib Maity Department of Civil Engineering IIT Kharagpur
More informationPROBABILITY CHAPTER LEARNING OBJECTIVES UNIT OVERVIEW
CHAPTER 16 PROBABILITY LEARNING OBJECTIVES Concept of probability is used in accounting and finance to understand the likelihood of occurrence or non-occurrence of a variable. It helps in developing financial
More informationMATH2206 Prob Stat/20.Jan Weekly Review 1-2
MATH2206 Prob Stat/20.Jan.2017 Weekly Review 1-2 This week I explained the idea behind the formula of the well-known statistic standard deviation so that it is clear now why it is a measure of dispersion
More informationLecture 1 : The Mathematical Theory of Probability
Lecture 1 : The Mathematical Theory of Probability 0/ 30 1. Introduction Today we will do 2.1 and 2.2. We will skip Chapter 1. We all have an intuitive notion of probability. Let s see. What is the probability
More informationELEG 3143 Probability & Stochastic Process Ch. 1 Probability
Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 1 Probability Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Applications Elementary Set Theory Random
More informationUNIT 5 ~ Probability: What Are the Chances? 1
UNIT 5 ~ Probability: What Are the Chances? 1 6.1: Simulation Simulation: The of chance behavior, based on a that accurately reflects the phenomenon under consideration. (ex 1) Suppose we are interested
More information1 Probability Theory. 1.1 Introduction
1 Probability Theory Probability theory is used as a tool in statistics. It helps to evaluate the reliability of our conclusions about the population when we have only information about a sample. Probability
More informationProbability & Random Variables
& Random Variables Probability Probability theory is the branch of math that deals with random events, processes, and variables What does randomness mean to you? How would you define probability in your
More informationChapter 14. From Randomness to Probability. Copyright 2012, 2008, 2005 Pearson Education, Inc.
Chapter 14 From Randomness to Probability Copyright 2012, 2008, 2005 Pearson Education, Inc. Dealing with Random Phenomena A random phenomenon is a situation in which we know what outcomes could happen,
More informationChapter 2. Conditional Probability and Independence. 2.1 Conditional Probability
Chapter 2 Conditional Probability and Independence 2.1 Conditional Probability Example: Two dice are tossed. What is the probability that the sum is 8? This is an easy exercise: we have a sample space
More informationBrief Review of Probability
Brief Review of Probability Nuno Vasconcelos (Ken Kreutz-Delgado) ECE Department, UCSD Probability Probability theory is a mathematical language to deal with processes or experiments that are non-deterministic
More informationConditional Probability
Conditional Probability When we obtain additional information about a probability experiment, we want to use the additional information to reassess the probabilities of events given the new information.
More informationIndependent Events. The multiplication rule for independent events says that if A and B are independent, P (A and B) = P (A) P (B).
Independent Events Two events are said to be independent if the outcome of one of them does not influence the other. For example, in sporting events, the outcomes of different games are usually considered
More informationProperties of Probability
Econ 325 Notes on Probability 1 By Hiro Kasahara Properties of Probability In statistics, we consider random experiments, experiments for which the outcome is random, i.e., cannot be predicted with certainty.
More informationLecture Slides. Elementary Statistics Eleventh Edition. by Mario F. Triola. and the Triola Statistics Series 4.1-1
Lecture Slides Elementary Statistics Eleventh Edition and the Triola Statistics Series by Mario F. Triola 4.1-1 4-1 Review and Preview Chapter 4 Probability 4-2 Basic Concepts of Probability 4-3 Addition
More informationNuevo examen - 02 de Febrero de 2017 [280 marks]
Nuevo examen - 0 de Febrero de 0 [0 marks] Jar A contains three red marbles and five green marbles. Two marbles are drawn from the jar, one after the other, without replacement. a. Find the probability
More informationProbability. 25 th September lecture based on Hogg Tanis Zimmerman: Probability and Statistical Inference (9th ed.)
Probability 25 th September 2017 lecture based on Hogg Tanis Zimmerman: Probability and Statistical Inference (9th ed.) Properties of Probability Methods of Enumeration Conditional Probability Independent
More informationIf S = {O 1, O 2,, O n }, where O i is the i th elementary outcome, and p i is the probability of the i th elementary outcome, then
1.1 Probabilities Def n: A random experiment is a process that, when performed, results in one and only one of many observations (or outcomes). The sample space S is the set of all elementary outcomes
More informationProving simple set properties...
Proving simple set properties... Part 1: Some examples of proofs over sets Fall 2013 Proving simple set properties... Fall 2013 1 / 17 Introduction Overview: Learning outcomes In this session we will...
More informationLecture 2. Conditional Probability
Math 408 - Mathematical Statistics Lecture 2. Conditional Probability January 18, 2013 Konstantin Zuev (USC) Math 408, Lecture 2 January 18, 2013 1 / 9 Agenda Motivation and Definition Properties of Conditional
More informationCS626 Data Analysis and Simulation
CS626 Data Analysis and Simulation Instructor: Peter Kemper R 104A, phone 221-3462, email:kemper@cs.wm.edu Today: Probability Primer Quick Reference: Sheldon Ross: Introduction to Probability Models 9th
More information4. Probability of an event A for equally likely outcomes:
University of California, Los Angeles Department of Statistics Statistics 110A Instructor: Nicolas Christou Probability Probability: A measure of the chance that something will occur. 1. Random experiment:
More information3.2 Probability Rules
3.2 Probability Rules The idea of probability rests on the fact that chance behavior is predictable in the long run. In the last section, we used simulation to imitate chance behavior. Do we always need
More informationImportant Concepts Read Chapter 2. Experiments. Phenomena. Probability Models. Unpredictable in detail. Examples
Probability Models Important Concepts Read Chapter 2 Probability Models Examples - The Classical Model - Discrete Spaces Elementary Consequences of the Axioms The Inclusion Exclusion Formulas Some Indiscrete
More informationConditional Probability & Independence. Conditional Probabilities
Conditional Probability & Independence Conditional Probabilities Question: How should we modify P(E) if we learn that event F has occurred? Definition: the conditional probability of E given F is P(E F
More informationStatistical Theory 1
Statistical Theory 1 Set Theory and Probability Paolo Bautista September 12, 2017 Set Theory We start by defining terms in Set Theory which will be used in the following sections. Definition 1 A set is
More informationMean, Median and Mode. Lecture 3 - Axioms of Probability. Where do they come from? Graphically. We start with a set of 21 numbers, Sta102 / BME102
Mean, Median and Mode Lecture 3 - Axioms of Probability Sta102 / BME102 Colin Rundel September 1, 2014 We start with a set of 21 numbers, ## [1] -2.2-1.6-1.0-0.5-0.4-0.3-0.2 0.1 0.1 0.2 0.4 ## [12] 0.4
More informationIntroduction to Probability Theory, Algebra, and Set Theory
Summer School on Mathematical Philosophy for Female Students Introduction to Probability Theory, Algebra, and Set Theory Catrin Campbell-Moore and Sebastian Lutz July 28, 2014 Question 1. Draw Venn diagrams
More informationWeek 2: Probability: Counting, Sets, and Bayes
Statistical Methods APPM 4570/5570, STAT 4000/5000 21 Probability Introduction to EDA Week 2: Probability: Counting, Sets, and Bayes Random variable Random variable is a measurable quantity whose outcome
More information