Probability Theory Review

Size: px
Start display at page:

Download "Probability Theory Review"

Transcription

1 Cogsci 118A: Natural Computation I Lecture 2 (01/07/10) Lecturer: Angela Yu Probability Theory Review Scribe: Joseph Schilz Lecture Summary 1. Set theory: terms and operators In this section, we provide definitions for the set terms theory operations used in our review of probability. 2. Probability spaces We define and explain the components of a probability space. 3. Joint probability, conditional probability, and marginal probability We define and explain concepts in probability involving multiple random variables. 1 Set theory: terms and operators To avoid inconsistencies and ambiguities, we will pursue a formal understanding of probability. This formal understanding will be based on the theory of sets, and this lecture requires familiarity with the following set theory terms and operators: 1. Naming conventions: We generally denote by lower case letters a,b,c,d,...,x,y,z singular, non-set items. We denote by upper case letters A,B,C,D,... sets of items. Calligraphic letters A, B, C... are often used to denote sets which contain other sets. 2. Set membership: We denote that an element x is a member of the set A by: x A. It is also possible for one set to be a member of another set. If the set E is a member of F then we write: E F. 3. Subset\superset relation:,,, If all elements of the set A can also be found in B, we say that A is a subset of B and write: A B. Then it s also true that B is a superset of A and we may write: B A. Formally we write A B iff x A(x B). Note: Some authors use and interchangeably with and. Other authors may write A B only in cases where A B and A B. In this course and will be used in the former sense, though we may use and to emphasize the possible equality of the sets in question. 4. The empty set: The empty set is the set which contains no elements. It is denoted by or by set brackets enclosing no elements: {}. For any set A, it is true that A. 2 (01/07/10)-1

2 5. Union: For sets A and B, the union of A and B is the set containing exactly those elements in either A or B. This is denoted A B. Formally, A B = {x : x A x B}. 6. Intersection: For sets A and B, the intersection of A and B is the set containing exactly those elements in both A and B. This is denoted A B. Formally, A B = {x : x A x B}. 7. Mutually exclusive: Two sets A and B are said to be mutually exclusive if they contain no elements in common. More than two sets, for example A 1,A 2,A 3,..., are said to be mutually exclusive if no two sets contain any elements in common. Formally, A 1,A 2,A 3,... are mutually exclusive iff i j(i j A i A j = ). 8. Complementation: A C In the context of a set U, where A U, the complement of A is the set of all those elements in U that are not in A. This is denoted A C. Throughout this lecture, the set which fulfills the role of U will generally be Ω. 2 Probability Spaces This section discusses probability spaces. A probability space is a set of three elements: a sample space Ω, an event space F, and a probability measure P, each obeying special properties. Section 2.1, Section 2.2, and Section 2.3 discuss these elements respectively. Together, these elements define a mathematical experiment: they list its possible outcomes, the events to which we can assign probability, and the actual probabilities that we assign to those events. Example: We take the single roll of a numbered, six-sided die as an example of a mathematical experiment. We expand upon this example as this section progresses. 2.1 Sample Space The sample space is the set of all possible outcomes of our mathematical experiment. It is denoted Ω. The sample space may be finite or it may be infinite, depending upon the experiment. It cannot be empty. That is: Ω. The sample space may also be called the outcome space. The variable ω is often used to denote the elements of a sample space. Example: If Ω is the set of all possible outcomes of rolling a six-sided die, then Ω = {1,2,3,4,5,6}. 2.2 Event Space The event space is the set of all events to which we can assign probabilities. In other words, it is the set of all events of which we are able to measure the probability. The event space is denoted by F. An event is a subset of Ω. An event is often denoted by the letter E, with some identifying subscript. The event which contains every element of Ω may be denoted E Ω, and by definition then E Ω = Ω. The event which contains no elements may be denoted E 0 and is equal to. An event which contains a single outcome from Ω is called an atomic event. 2 (01/07/10)-2

3 Example: In our example Ω = {1,2,3,4,5,6}. Consider the atomic event in which a four is rolled: we might call this E 4 where E 4 = {4}. Let E o = {1,3,5} be the event in which some odd number is rolled and E e = {2,4,6} be the event in which some even number is rolled. Then we might have E 4,E o,e e,e Ω F. (HW) Including the null event {} and the all inclusive event of the sample space itself Ω, how many total events could there be in our event space F? An event space must also satisfy the three additional properties of a sigma algebra. These properties are: 1. F The empty set must be a member of F. This ensures that F will not be empty. 2. E F E C F For any event we find in F, we must also find its complement E C in F. This requirement is called closure under complements. 3. E 1,E 2,E 3,... F i E i F For any finite or countably infinite group of events in F, we must also find the set which is the union of those events in F. This requirement is called closure under countable unions. (HW) Demonstrate that closure under complementation and closure under countable union together imply closure under countable intersection. Example: Consider F = {E 4,E o,e e,e Ω }. F does not meet condition (1) of a sigma algebra as / F. We can add, which we denote E 0, to F to produce F = {E 0,E 4,E o,e e,e Ω }. Though F satisfies condition (1), it does not satisfy closure under complements. For example, E C 4 = {4} C = {1,2,3,5,6} is not in F. Consider F = {E 0,E o,e e,e Ω }. F satisfies condition two, as E C 0 = {} C = {1,2,3,4,5,6} = E Ω is in F, E C o = {1,3,5} C = {2,4,6} = E e is in F, and so on. F also satisfies closure under countable unions. For example, E 0 E o = {} {1,3,5} = {1,3,5} = E o is in F, as is any other arbitrary union. Thus, F is a sigma algebra with respect to Ω. Though in common speech we might use event and outcome interchangeably, they have distinct definitions in probability theory. Example: Suppose our die is fair. Having no number come up is not a possible outcome of rolling a die, thus it does not belong in our sample space Ω. However, the event in which no number comes up when rolling a die is an event to which we can assign probability. Thus, E 0 = {} = is a valid element of F. Later we will see that the probability we assign to E 0 is zero. 2.3 Probability Measure A probability measure P is a function from F into the real numbers. It is the function that assigns to each event in F a numerical probability. Furthermore, P is subject to the following conditions: 2 (01/07/10)-3

4 1. E F(0 P(E) 1) For any event in our event space, the probability of that event occurring must be between zero and one, inclusive. We can then define our probability measure as P : F [0,1]. 2. P(Ω) = 1 The probability of the event Ω, also denoted E Ω occurring must be equal to one. Recall, E Ω is the event in which any one of the possible outcomes of the experiment occurs. Thus, (2) states that when the experiment is performed we are certain to see an outcome from the sample space. 3. For mutually exclusive events E 1,E 2,...,E n, P( i E i ) = i P(E i ) By (3), we can easily determine the probability of an event E if we know the probabilities of a mutually exclusive group of simpler events which together compose E. Example: Given P(E e ) = 1/2 and P(E 5 ) = 1/6 we may determine the probability that the die lands on an even number or five by (3). E e and E 5 are mutually exclusive, so P(E e E 5 ) = P(E e ) + P(E 5 ) = 1/2 + 1/6 = 4/6. Probability measures have additional properties, which can be derived from the three stated above. 1. P( ) = 0 The probability of no outcome occurring as a result of the experiment is equal to zero. Proof By (2) we have P(Ω) = 1. We also find that P(Ω) = P(Ω ) (3) = P(Ω) + P( ) (2) = 1 + P( ). Both the left and right sides of the equation are equal to one, and so P( ) must equal zero. 2. Monotonicity: A B P(A) P(B) Example: Suppose our die may be unfair. Even so, under no parameters would it be more probable to roll a one than to roll either a one or a two. Formally, we can write this as: {1} {1,2} P({1}) P({1,2}). (HW) Prove that monotonicity follows from the definition of a probability measure. 3. Sub-additivity: A i A i P(A) i P(A i ) Example: Even if our die is unfair, we may be confident that P({6}) P({4,5,6}+P({5,6}) because {6} {4,5,6} {5,6}. (HW) Prove that sub-additivity follows from the definition of a probability measure. Note: A probability measure may sometimes be denoted by a regular upper-case P. Though in one context this upper-case P may refer to a probability measure, in a different context it may refer to a cumulative density function or CDF. The symbol P unambiguously refers to a probability measure. 2 (01/07/10)-4

5 3 Joint Probability, Conditional Probability, and Marginal Probability 3.1 Random Variables For this course, we will consider a random variable (r.v.) to be a variable representing experimental outcomes which are numbers. That is, we assume that Ω R, or R n. For an arbitrary sample space Ω, we construct an Ω R by defining x : Ω R. Then let Ω be the image of Ω under x. We define a new probability measure P by P (x A) = P({ω : x(ω) A}). In practice, we then refer to Ω and P as Ω and P respectively. Example: Consider the toss of a coin. Let Ω = { heads, tails }. Then we may define x by x( heads ) = 1 and x( tails ) = 0. Then Ω = {1,0}. If the coin is fair then P (1) = P( heads ) = 1/2 and P (0) = P( tails ) = 1/2. We generally consider P(x E), the probability that the outcome of our experiment x is a member of the event E. As done above, we may sometimes write probabilities of the form P(E) or P(x). Example: Consider the toss of a single fair coin. We might write that P(x {0}) = 1/2, however it is often equally clear to write that P(0) = 1/2. Now consider tossing two fair coins, the outcomes of which are given by random variables x 1 and x 2. If the coins are independent (see Section 3.2.1), then it is true that P(x 1 A,x 2 B) = P(x 1 A)P(x 2 B) no matter our choices of A and B. If we simply wished to emphasize this attribute, we might write: P(x 1,x 2 ) = P(x 1 )P(x 2 ). 3.2 Joint Probability Joint probability is the probability of two events both occurring as a result of performing an experiment. Using random variables, we write the probability of our outcome x being both a member of event E 1 and event E 2 as P(x E 1,x E 2 ). We define this as P(x E 1,x E 2 ) = P(x E 1 E 2 ) Independence We say that events E 1, E 2 are independent if P(x E 1,x E 2 ) = P(x E 1 )P(x E 2 ). Example: Consider drawing a single card from a fair deck of 52. We find the probability that the card is both a heart and a two, written P( hearts, two ). We find that P( heart )P( two ) = (1/4)(1/13) = 1/52. We reason that drawing a card which is both a heart and a two will happen iff we have drawn the two of hearts, which will occur with probability 1/52. Thus we confirm the independence of the two events. Consider the probability of drawing a card which is both a heart and red, written P( hearts, red ). We find that P( heart )P( red ) = (1/4)(1/2) = 1/8. However, we reason that drawing a card which is both a heart and red will happen iff we have drawn a heart. So P( hearts, red ) = P( hearts ) = 1/4. Thus we confirm that the two events are not independent. (HW) Show that if events A and B are independent, then P(A B) = P(A) and P(B A) = P(B). 2 (01/07/10)-5

6 3.3 Conditional Probability Conditional probability is the probability of an event occurring given the assurance that a separate event occurs. We denote the probability of A given B as P(A B). We define conditional probability as: P(A B) = P(A,B) P(B). Example: Suppose we ve tossed a fair die, and we know only that an even number has come up. We find the probability that a two has come up using conditional probability: P({2} {2,4,6}) = P({2}, {2,4,6}) P({2,4,6}) = P({2} {2,4,6}) P({2,4,6}) = P({2}) P({2,4,6}) = 1/6 1/2 = 1/ Marginal Probability Marginal probability is a property relating the unconditional probability of a single event to a sum of joint probabilities. Specifically, for a group of mutually exclusive events E 1,E 2,E 3,..., such that E i = Ω, P(x) = P(x,y E i ). i Example: Consider tossing two fair coins, the outcomes of which are given by random variables x and y respectively. Knowing that P(x = 1,y = 1) = P(x = 1,y = 0) = 1/4, we can find P(x = 1) as P(x = 1,y = 1) + P(x = 1,y = 0) = 1/4 + 1/4 = 1/2. 6

LECTURE 1. 1 Introduction. 1.1 Sample spaces and events

LECTURE 1. 1 Introduction. 1.1 Sample spaces and events LECTURE 1 1 Introduction The first part of our adventure is a highly selective review of probability theory, focusing especially on things that are most useful in statistics. 1.1 Sample spaces and events

More information

CMPSCI 240: Reasoning about Uncertainty

CMPSCI 240: Reasoning about Uncertainty CMPSCI 240: Reasoning about Uncertainty Lecture 2: Sets and Events Andrew McGregor University of Massachusetts Last Compiled: January 27, 2017 Outline 1 Recap 2 Experiments and Events 3 Probabilistic Models

More information

Lecture Notes 1 Basic Probability. Elements of Probability. Conditional probability. Sequential Calculation of Probability

Lecture Notes 1 Basic Probability. Elements of Probability. Conditional probability. Sequential Calculation of Probability Lecture Notes 1 Basic Probability Set Theory Elements of Probability Conditional probability Sequential Calculation of Probability Total Probability and Bayes Rule Independence Counting EE 178/278A: Basic

More information

Lecture 11: Random Variables

Lecture 11: Random Variables EE5110: Probability Foundations for Electrical Engineers July-November 2015 Lecture 11: Random Variables Lecturer: Dr. Krishna Jagannathan Scribe: Sudharsan, Gopal, Arjun B, Debayani The study of random

More information

Origins of Probability Theory

Origins of Probability Theory 1 16.584: INTRODUCTION Theory and Tools of Probability required to analyze and design systems subject to uncertain outcomes/unpredictability/randomness. Such systems more generally referred to as Experiments.

More information

Lecture Lecture 5

Lecture Lecture 5 Lecture 4 --- Lecture 5 A. Basic Concepts (4.1-4.2) 1. Experiment: A process of observing a phenomenon that has variation in its outcome. Examples: (E1). Rolling a die, (E2). Drawing a card form a shuffled

More information

STAT Chapter 3: Probability

STAT Chapter 3: Probability Basic Definitions STAT 515 --- Chapter 3: Probability Experiment: A process which leads to a single outcome (called a sample point) that cannot be predicted with certainty. Sample Space (of an experiment):

More information

Recitation 2: Probability

Recitation 2: Probability Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions

More information

Deep Learning for Computer Vision

Deep Learning for Computer Vision Deep Learning for Computer Vision Lecture 3: Probability, Bayes Theorem, and Bayes Classification Peter Belhumeur Computer Science Columbia University Probability Should you play this game? Game: A fair

More information

Review of Basic Probability Theory

Review of Basic Probability Theory Review of Basic Probability Theory James H. Steiger Department of Psychology and Human Development Vanderbilt University James H. Steiger (Vanderbilt University) 1 / 35 Review of Basic Probability Theory

More information

Lecture 1 Introduction to Probability and Set Theory Text: A Course in Probability by Weiss

Lecture 1 Introduction to Probability and Set Theory Text: A Course in Probability by Weiss Lecture 1 to and Set Theory Text: A Course in by Weiss 1.2 2.3 STAT 225 to Models January 13, 2014 to and Whitney Huang Purdue University 1.1 Agenda to and 1 2 3 1.2 Motivation Uncertainty/Randomness in

More information

Venn Diagrams; Probability Laws. Notes. Set Operations and Relations. Venn Diagram 2.1. Venn Diagrams; Probability Laws. Notes

Venn Diagrams; Probability Laws. Notes. Set Operations and Relations. Venn Diagram 2.1. Venn Diagrams; Probability Laws. Notes Lecture 2 s; Text: A Course in Probability by Weiss 2.4 STAT 225 Introduction to Probability Models January 8, 2014 s; Whitney Huang Purdue University 2.1 Agenda s; 1 2 2.2 Intersection: the intersection

More information

Axioms of Probability

Axioms of Probability Sample Space (denoted by S) The set of all possible outcomes of a random experiment is called the Sample Space of the experiment, and is denoted by S. Example 1.10 If the experiment consists of tossing

More information

Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable

Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed

More information

Module 1. Probability

Module 1. Probability Module 1 Probability 1. Introduction In our daily life we come across many processes whose nature cannot be predicted in advance. Such processes are referred to as random processes. The only way to derive

More information

Lecture 10: Probability distributions TUESDAY, FEBRUARY 19, 2019

Lecture 10: Probability distributions TUESDAY, FEBRUARY 19, 2019 Lecture 10: Probability distributions DANIEL WELLER TUESDAY, FEBRUARY 19, 2019 Agenda What is probability? (again) Describing probabilities (distributions) Understanding probabilities (expectation) Partial

More information

Econ 325: Introduction to Empirical Economics

Econ 325: Introduction to Empirical Economics Econ 325: Introduction to Empirical Economics Lecture 2 Probability Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall Ch. 3-1 3.1 Definition Random Experiment a process leading to an uncertain

More information

Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable

Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed

More information

Math 1313 Experiments, Events and Sample Spaces

Math 1313 Experiments, Events and Sample Spaces Math 1313 Experiments, Events and Sample Spaces At the end of this recording, you should be able to define and use the basic terminology used in defining experiments. Terminology The next main topic in

More information

Properties of Probability

Properties of Probability Econ 325 Notes on Probability 1 By Hiro Kasahara Properties of Probability In statistics, we consider random experiments, experiments for which the outcome is random, i.e., cannot be predicted with certainty.

More information

Lecture 9: Conditional Probability and Independence

Lecture 9: Conditional Probability and Independence EE5110: Probability Foundations July-November 2015 Lecture 9: Conditional Probability and Independence Lecturer: Dr. Krishna Jagannathan Scribe: Vishakh Hegde 9.1 Conditional Probability Definition 9.1

More information

Fundamentals of Probability CE 311S

Fundamentals of Probability CE 311S Fundamentals of Probability CE 311S OUTLINE Review Elementary set theory Probability fundamentals: outcomes, sample spaces, events Outline ELEMENTARY SET THEORY Basic probability concepts can be cast in

More information

Chapter 6: Probability The Study of Randomness

Chapter 6: Probability The Study of Randomness Chapter 6: Probability The Study of Randomness 6.1 The Idea of Probability 6.2 Probability Models 6.3 General Probability Rules 1 Simple Question: If tossing a coin, what is the probability of the coin

More information

Statistics for Financial Engineering Session 2: Basic Set Theory March 19 th, 2006

Statistics for Financial Engineering Session 2: Basic Set Theory March 19 th, 2006 Statistics for Financial Engineering Session 2: Basic Set Theory March 19 th, 2006 Topics What is a set? Notations for sets Empty set Inclusion/containment and subsets Sample spaces and events Operations

More information

The enumeration of all possible outcomes of an experiment is called the sample space, denoted S. E.g.: S={head, tail}

The enumeration of all possible outcomes of an experiment is called the sample space, denoted S. E.g.: S={head, tail} Random Experiment In random experiments, the result is unpredictable, unknown prior to its conduct, and can be one of several choices. Examples: The Experiment of tossing a coin (head, tail) The Experiment

More information

Discrete Probability. Mark Huiskes, LIACS Probability and Statistics, Mark Huiskes, LIACS, Lecture 2

Discrete Probability. Mark Huiskes, LIACS Probability and Statistics, Mark Huiskes, LIACS, Lecture 2 Discrete Probability Mark Huiskes, LIACS mark.huiskes@liacs.nl Probability: Basic Definitions In probability theory we consider experiments whose outcome depends on chance or are uncertain. How do we model

More information

Appendix A : Introduction to Probability and stochastic processes

Appendix A : Introduction to Probability and stochastic processes A-1 Mathematical methods in communication July 5th, 2009 Appendix A : Introduction to Probability and stochastic processes Lecturer: Haim Permuter Scribe: Shai Shapira and Uri Livnat The probability of

More information

ELEG 3143 Probability & Stochastic Process Ch. 1 Probability

ELEG 3143 Probability & Stochastic Process Ch. 1 Probability Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 1 Probability Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Applications Elementary Set Theory Random

More information

HW MATH425/525 Lecture Notes 1

HW MATH425/525 Lecture Notes 1 HW MATH425/525 Lecture Notes 1 Definition 4.1 If an experiment can be repeated under the same condition, its outcome cannot be predicted with certainty, and the collection of its every possible outcome

More information

I - Probability. What is Probability? the chance of an event occuring. 1classical probability. 2empirical probability. 3subjective probability

I - Probability. What is Probability? the chance of an event occuring. 1classical probability. 2empirical probability. 3subjective probability What is Probability? the chance of an event occuring eg 1classical probability 2empirical probability 3subjective probability Section 2 - Probability (1) Probability - Terminology random (probability)

More information

Chapter 2.5 Random Variables and Probability The Modern View (cont.)

Chapter 2.5 Random Variables and Probability The Modern View (cont.) Chapter 2.5 Random Variables and Probability The Modern View (cont.) I. Statistical Independence A crucially important idea in probability and statistics is the concept of statistical independence. Suppose

More information

Statistical Inference

Statistical Inference Statistical Inference Lecture 1: Probability Theory MING GAO DASE @ ECNU (for course related communications) mgao@dase.ecnu.edu.cn Sep. 11, 2018 Outline Introduction Set Theory Basics of Probability Theory

More information

Lecture notes for probability. Math 124

Lecture notes for probability. Math 124 Lecture notes for probability Math 124 What is probability? Probabilities are ratios, expressed as fractions, decimals, or percents, determined by considering results or outcomes of experiments whose result

More information

Lecture 1 : The Mathematical Theory of Probability

Lecture 1 : The Mathematical Theory of Probability Lecture 1 : The Mathematical Theory of Probability 0/ 30 1. Introduction Today we will do 2.1 and 2.2. We will skip Chapter 1. We all have an intuitive notion of probability. Let s see. What is the probability

More information

Introduction to Probability. Ariel Yadin. Lecture 1. We begin with an example [this is known as Bertrand s paradox]. *** Nov.

Introduction to Probability. Ariel Yadin. Lecture 1. We begin with an example [this is known as Bertrand s paradox]. *** Nov. Introduction to Probability Ariel Yadin Lecture 1 1. Example: Bertrand s Paradox We begin with an example [this is known as Bertrand s paradox]. *** Nov. 1 *** Question 1.1. Consider a circle of radius

More information

Probability Theory. Probability and Statistics for Data Science CSE594 - Spring 2016

Probability Theory. Probability and Statistics for Data Science CSE594 - Spring 2016 Probability Theory Probability and Statistics for Data Science CSE594 - Spring 2016 What is Probability? 2 What is Probability? Examples outcome of flipping a coin (seminal example) amount of snowfall

More information

Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com

Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com 1 School of Oriental and African Studies September 2015 Department of Economics Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com Gujarati D. Basic Econometrics, Appendix

More information

Math-Stat-491-Fall2014-Notes-I

Math-Stat-491-Fall2014-Notes-I Math-Stat-491-Fall2014-Notes-I Hariharan Narayanan October 2, 2014 1 Introduction This writeup is intended to supplement material in the prescribed texts: Introduction to Probability Models, 10th Edition,

More information

CHAPTER 1 SETS AND EVENTS

CHAPTER 1 SETS AND EVENTS CHPTER 1 SETS ND EVENTS 1.1 Universal Set and Subsets DEFINITION: set is a well-defined collection of distinct elements in the universal set. This is denoted by capital latin letters, B, C, If an element

More information

STAT 430/510 Probability

STAT 430/510 Probability STAT 430/510 Probability Hui Nie Lecture 3 May 28th, 2009 Review We have discussed counting techniques in Chapter 1. Introduce the concept of the probability of an event. Compute probabilities in certain

More information

Mathematical Foundations of Computer Science Lecture Outline October 18, 2018

Mathematical Foundations of Computer Science Lecture Outline October 18, 2018 Mathematical Foundations of Computer Science Lecture Outline October 18, 2018 The Total Probability Theorem. Consider events E and F. Consider a sample point ω E. Observe that ω belongs to either F or

More information

What is Probability? Probability. Sample Spaces and Events. Simple Event

What is Probability? Probability. Sample Spaces and Events. Simple Event What is Probability? Probability Peter Lo Probability is the numerical measure of likelihood that the event will occur. Simple Event Joint Event Compound Event Lies between 0 & 1 Sum of events is 1 1.5

More information

Stats Probability Theory

Stats Probability Theory Stats 241.3 Probability Theory Instructor: Office: W.H.Laverty 235 McLean Hall Phone: 966-6096 Lectures: Evaluation: M T W Th F 1:30pm - 2:50pm Thorv 105 Lab: T W Th 3:00-3:50 Thorv 105 Assignments, Labs,

More information

EE 178 Lecture Notes 0 Course Introduction. About EE178. About Probability. Course Goals. Course Topics. Lecture Notes EE 178

EE 178 Lecture Notes 0 Course Introduction. About EE178. About Probability. Course Goals. Course Topics. Lecture Notes EE 178 EE 178 Lecture Notes 0 Course Introduction About EE178 About Probability Course Goals Course Topics Lecture Notes EE 178: Course Introduction Page 0 1 EE 178 EE 178 provides an introduction to probabilistic

More information

Review of Probability. CS1538: Introduction to Simulations

Review of Probability. CS1538: Introduction to Simulations Review of Probability CS1538: Introduction to Simulations Probability and Statistics in Simulation Why do we need probability and statistics in simulation? Needed to validate the simulation model Needed

More information

The probability of an event is viewed as a numerical measure of the chance that the event will occur.

The probability of an event is viewed as a numerical measure of the chance that the event will occur. Chapter 5 This chapter introduces probability to quantify randomness. Section 5.1: How Can Probability Quantify Randomness? The probability of an event is viewed as a numerical measure of the chance that

More information

Chapter 3 : Conditional Probability and Independence

Chapter 3 : Conditional Probability and Independence STAT/MATH 394 A - PROBABILITY I UW Autumn Quarter 2016 Néhémy Lim Chapter 3 : Conditional Probability and Independence 1 Conditional Probabilities How should we modify the probability of an event when

More information

Dept. of Linguistics, Indiana University Fall 2015

Dept. of Linguistics, Indiana University Fall 2015 L645 Dept. of Linguistics, Indiana University Fall 2015 1 / 34 To start out the course, we need to know something about statistics and This is only an introduction; for a fuller understanding, you would

More information

The Theory behind PageRank

The Theory behind PageRank The Theory behind PageRank Mauro Sozio Telecom ParisTech May 21, 2014 Mauro Sozio (LTCI TPT) The Theory behind PageRank May 21, 2014 1 / 19 A Crash Course on Discrete Probability Events and Probability

More information

5. Conditional Distributions

5. Conditional Distributions 1 of 12 7/16/2009 5:36 AM Virtual Laboratories > 3. Distributions > 1 2 3 4 5 6 7 8 5. Conditional Distributions Basic Theory As usual, we start with a random experiment with probability measure P on an

More information

UNIT 5 ~ Probability: What Are the Chances? 1

UNIT 5 ~ Probability: What Are the Chances? 1 UNIT 5 ~ Probability: What Are the Chances? 1 6.1: Simulation Simulation: The of chance behavior, based on a that accurately reflects the phenomenon under consideration. (ex 1) Suppose we are interested

More information

the time it takes until a radioactive substance undergoes a decay

the time it takes until a radioactive substance undergoes a decay 1 Probabilities 1.1 Experiments with randomness Wewillusethetermexperimentinaverygeneralwaytorefertosomeprocess that produces a random outcome. Examples: (Ask class for some first) Here are some discrete

More information

Compound Events. The event E = E c (the complement of E) is the event consisting of those outcomes which are not in E.

Compound Events. The event E = E c (the complement of E) is the event consisting of those outcomes which are not in E. Compound Events Because we are using the framework of set theory to analyze probability, we can use unions, intersections and complements to break complex events into compositions of events for which it

More information

Why study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables

Why study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables ECE 6010 Lecture 1 Introduction; Review of Random Variables Readings from G&S: Chapter 1. Section 2.1, Section 2.3, Section 2.4, Section 3.1, Section 3.2, Section 3.5, Section 4.1, Section 4.2, Section

More information

If S = {O 1, O 2,, O n }, where O i is the i th elementary outcome, and p i is the probability of the i th elementary outcome, then

If S = {O 1, O 2,, O n }, where O i is the i th elementary outcome, and p i is the probability of the i th elementary outcome, then 1.1 Probabilities Def n: A random experiment is a process that, when performed, results in one and only one of many observations (or outcomes). The sample space S is the set of all elementary outcomes

More information

Topic 5 Basics of Probability

Topic 5 Basics of Probability Topic 5 Basics of Probability Equally Likely Outcomes and the Axioms of Probability 1 / 13 Outline Equally Likely Outcomes Axioms of Probability Consequences of the Axioms 2 / 13 Introduction A probability

More information

Lecture 2. October 21, Department of Biostatistics Johns Hopkins Bloomberg School of Public Health Johns Hopkins University.

Lecture 2. October 21, Department of Biostatistics Johns Hopkins Bloomberg School of Public Health Johns Hopkins University. Lecture 2 Department of Biostatistics Johns Hopkins Bloomberg School of Public Health Johns Hopkins University October 21, 2007 1 2 3 4 5 6 Define probability calculus Basic axioms of probability Define

More information

Probability COMP 245 STATISTICS. Dr N A Heard. 1 Sample Spaces and Events Sample Spaces Events Combinations of Events...

Probability COMP 245 STATISTICS. Dr N A Heard. 1 Sample Spaces and Events Sample Spaces Events Combinations of Events... Probability COMP 245 STATISTICS Dr N A Heard Contents Sample Spaces and Events. Sample Spaces........................................2 Events........................................... 2.3 Combinations

More information

Probability Theory for Machine Learning. Chris Cremer September 2015

Probability Theory for Machine Learning. Chris Cremer September 2015 Probability Theory for Machine Learning Chris Cremer September 2015 Outline Motivation Probability Definitions and Rules Probability Distributions MLE for Gaussian Parameter Estimation MLE and Least Squares

More information

2) There should be uncertainty as to which outcome will occur before the procedure takes place.

2) There should be uncertainty as to which outcome will occur before the procedure takes place. robability Numbers For many statisticians the concept of the probability that an event occurs is ultimately rooted in the interpretation of an event as an outcome of an experiment, others would interpret

More information

Review of Basic Probability

Review of Basic Probability Review of Basic Probability Erik G. Learned-Miller Department of Computer Science University of Massachusetts, Amherst Amherst, MA 01003 September 16, 2009 Abstract This document reviews basic discrete

More information

Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14

Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14 CS 70 Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14 Introduction One of the key properties of coin flips is independence: if you flip a fair coin ten times and get ten

More information

Lecture 1: Probability Fundamentals

Lecture 1: Probability Fundamentals Lecture 1: Probability Fundamentals IB Paper 7: Probability and Statistics Carl Edward Rasmussen Department of Engineering, University of Cambridge January 22nd, 2008 Rasmussen (CUED) Lecture 1: Probability

More information

RS Chapter 1 Random Variables 6/5/2017. Chapter 1. Probability Theory: Introduction

RS Chapter 1 Random Variables 6/5/2017. Chapter 1. Probability Theory: Introduction Chapter 1 Probability Theory: Introduction Basic Probability General In a probability space (Ω, Σ, P), the set Ω is the set of all possible outcomes of a probability experiment. Mathematically, Ω is just

More information

Math 416 Lecture 3. The average or mean or expected value of x 1, x 2, x 3,..., x n is

Math 416 Lecture 3. The average or mean or expected value of x 1, x 2, x 3,..., x n is Math 416 Lecture 3 Expected values The average or mean or expected value of x 1, x 2, x 3,..., x n is x 1 x 2... x n n x 1 1 n x 2 1 n... x n 1 n 1 n x i p x i where p x i 1 n is the probability of x i

More information

Independence. P(A) = P(B) = 3 6 = 1 2, and P(C) = 4 6 = 2 3.

Independence. P(A) = P(B) = 3 6 = 1 2, and P(C) = 4 6 = 2 3. Example: A fair die is tossed and we want to guess the outcome. The outcomes will be 1, 2, 3, 4, 5, 6 with equal probability 1 6 each. If we are interested in getting the following results: A = {1, 3,

More information

PROBABILITY THEORY. Prof. S. J. Soni. Assistant Professor Computer Engg. Department SPCE, Visnagar

PROBABILITY THEORY. Prof. S. J. Soni. Assistant Professor Computer Engg. Department SPCE, Visnagar PROBABILITY THEORY By Prof. S. J. Soni Assistant Professor Computer Engg. Department SPCE, Visnagar Introduction Signals whose values at any instant t are determined by their analytical or graphical description

More information

Sample Spaces, Random Variables

Sample Spaces, Random Variables Sample Spaces, Random Variables Moulinath Banerjee University of Michigan August 3, 22 Probabilities In talking about probabilities, the fundamental object is Ω, the sample space. (elements) in Ω are denoted

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 3 9/10/2008 CONDITIONING AND INDEPENDENCE

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 3 9/10/2008 CONDITIONING AND INDEPENDENCE MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 3 9/10/2008 CONDITIONING AND INDEPENDENCE Most of the material in this lecture is covered in [Bertsekas & Tsitsiklis] Sections 1.3-1.5

More information

Review Basic Probability Concept

Review Basic Probability Concept Economic Risk and Decision Analysis for Oil and Gas Industry CE81.9008 School of Engineering and Technology Asian Institute of Technology January Semester Presented by Dr. Thitisak Boonpramote Department

More information

Probability theory basics

Probability theory basics Probability theory basics Michael Franke Basics of probability theory: axiomatic definition, interpretation, joint distributions, marginalization, conditional probability & Bayes rule. Random variables:

More information

4/17/2012. NE ( ) # of ways an event can happen NS ( ) # of events in the sample space

4/17/2012. NE ( ) # of ways an event can happen NS ( ) # of events in the sample space I. Vocabulary: A. Outcomes: the things that can happen in a probability experiment B. Sample Space (S): all possible outcomes C. Event (E): one outcome D. Probability of an Event (P(E)): the likelihood

More information

Computing Probability

Computing Probability Computing Probability James H. Steiger October 22, 2003 1 Goals for this Module In this module, we will 1. Develop a general rule for computing probability, and a special case rule applicable when elementary

More information

STAT 111 Recitation 1

STAT 111 Recitation 1 STAT 111 Recitation 1 Linjun Zhang January 20, 2017 What s in the recitation This class, and the exam of this class, is a mix of statistical concepts and calculations. We are going to do a little bit of

More information

RVs and their probability distributions

RVs and their probability distributions RVs and their probability distributions RVs and their probability distributions In these notes, I will use the following notation: The probability distribution (function) on a sample space will be denoted

More information

Probability Year 10. Terminology

Probability Year 10. Terminology Probability Year 10 Terminology Probability measures the chance something happens. Formally, we say it measures how likely is the outcome of an event. We write P(result) as a shorthand. An event is some

More information

Lecture 2: Probability. Readings: Sections Statistical Inference: drawing conclusions about the population based on a sample

Lecture 2: Probability. Readings: Sections Statistical Inference: drawing conclusions about the population based on a sample Lecture 2: Probability Readings: Sections 5.1-5.3 1 Introduction Statistical Inference: drawing conclusions about the population based on a sample Parameter: a number that describes the population a fixed

More information

If the objects are replaced there are n choices each time yielding n r ways. n C r and in the textbook by g(n, r).

If the objects are replaced there are n choices each time yielding n r ways. n C r and in the textbook by g(n, r). Caveat: Not proof read. Corrections appreciated. Combinatorics In the following, n, n 1, r, etc. will denote non-negative integers. Rule 1 The number of ways of ordering n distinguishable objects (also

More information

MATH 556: PROBABILITY PRIMER

MATH 556: PROBABILITY PRIMER MATH 6: PROBABILITY PRIMER 1 DEFINITIONS, TERMINOLOGY, NOTATION 1.1 EVENTS AND THE SAMPLE SPACE Definition 1.1 An experiment is a one-off or repeatable process or procedure for which (a there is a well-defined

More information

M378K In-Class Assignment #1

M378K In-Class Assignment #1 The following problems are a review of M6K. M7K In-Class Assignment # Problem.. Complete the definition of mutual exclusivity of events below: Events A, B Ω are said to be mutually exclusive if A B =.

More information

Probability Pearson Education, Inc. Slide

Probability Pearson Education, Inc. Slide Probability The study of probability is concerned with random phenomena. Even though we cannot be certain whether a given result will occur, we often can obtain a good measure of its likelihood, or probability.

More information

Part IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015

Part IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015 Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.

More information

1 Sequences of events and their limits

1 Sequences of events and their limits O.H. Probability II (MATH 2647 M15 1 Sequences of events and their limits 1.1 Monotone sequences of events Sequences of events arise naturally when a probabilistic experiment is repeated many times. For

More information

2011 Pearson Education, Inc

2011 Pearson Education, Inc Statistics for Business and Economics Chapter 3 Probability Contents 1. Events, Sample Spaces, and Probability 2. Unions and Intersections 3. Complementary Events 4. The Additive Rule and Mutually Exclusive

More information

ECE 340 Probabilistic Methods in Engineering M/W 3-4:15. Lecture 2: Random Experiments. Prof. Vince Calhoun

ECE 340 Probabilistic Methods in Engineering M/W 3-4:15. Lecture 2: Random Experiments. Prof. Vince Calhoun ECE 340 Probabilistic Methods in Engineering M/W 3-4:15 Lecture 2: Random Experiments Prof. Vince Calhoun Reading This class: Section 2.1-2.2 Next class: Section 2.3-2.4 Homework: Assignment 1: From the

More information

Probability. Lecture Notes. Adolfo J. Rumbos

Probability. Lecture Notes. Adolfo J. Rumbos Probability Lecture Notes Adolfo J. Rumbos October 20, 204 2 Contents Introduction 5. An example from statistical inference................ 5 2 Probability Spaces 9 2. Sample Spaces and σ fields.....................

More information

Lecture 2: Repetition of probability theory and statistics

Lecture 2: Repetition of probability theory and statistics Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites:

More information

4 Lecture 4 Notes: Introduction to Probability. Probability Rules. Independence and Conditional Probability. Bayes Theorem. Risk and Odds Ratio

4 Lecture 4 Notes: Introduction to Probability. Probability Rules. Independence and Conditional Probability. Bayes Theorem. Risk and Odds Ratio 4 Lecture 4 Notes: Introduction to Probability. Probability Rules. Independence and Conditional Probability. Bayes Theorem. Risk and Odds Ratio Wrong is right. Thelonious Monk 4.1 Three Definitions of

More information

Statistics for Managers Using Microsoft Excel/SPSS Chapter 4 Basic Probability And Discrete Probability Distributions

Statistics for Managers Using Microsoft Excel/SPSS Chapter 4 Basic Probability And Discrete Probability Distributions Statistics for Managers Using Microsoft Excel/SPSS Chapter 4 Basic Probability And Discrete Probability Distributions 1999 Prentice-Hall, Inc. Chap. 4-1 Chapter Topics Basic Probability Concepts: Sample

More information

Chapter 2. Probability

Chapter 2. Probability 2-1 Chapter 2 Probability 2-2 Section 2.1: Basic Ideas Definition: An experiment is a process that results in an outcome that cannot be predicted in advance with certainty. Examples: rolling a die tossing

More information

Fundamentals. CS 281A: Statistical Learning Theory. Yangqing Jia. August, Based on tutorial slides by Lester Mackey and Ariel Kleiner

Fundamentals. CS 281A: Statistical Learning Theory. Yangqing Jia. August, Based on tutorial slides by Lester Mackey and Ariel Kleiner Fundamentals CS 281A: Statistical Learning Theory Yangqing Jia Based on tutorial slides by Lester Mackey and Ariel Kleiner August, 2011 Outline 1 Probability 2 Statistics 3 Linear Algebra 4 Optimization

More information

Probability Theory and Applications

Probability Theory and Applications Probability Theory and Applications Videos of the topics covered in this manual are available at the following links: Lesson 4 Probability I http://faculty.citadel.edu/silver/ba205/online course/lesson

More information

Announcements. Topics: To Do:

Announcements. Topics: To Do: Announcements Topics: In the Probability and Statistics module: - Sections 1 + 2: Introduction to Stochastic Models - Section 3: Basics of Probability Theory - Section 4: Conditional Probability; Law of

More information

Random Variables. Definition: A random variable (r.v.) X on the probability space (Ω, F, P) is a mapping

Random Variables. Definition: A random variable (r.v.) X on the probability space (Ω, F, P) is a mapping Random Variables Example: We roll a fair die 6 times. Suppose we are interested in the number of 5 s in the 6 rolls. Let X = number of 5 s. Then X could be 0, 1, 2, 3, 4, 5, 6. X = 0 corresponds to the

More information

ECE353: Probability and Random Processes. Lecture 2 - Set Theory

ECE353: Probability and Random Processes. Lecture 2 - Set Theory ECE353: Probability and Random Processes Lecture 2 - Set Theory Xiao Fu School of Electrical Engineering and Computer Science Oregon State University E-mail: xiao.fu@oregonstate.edu January 10, 2018 Set

More information

A PECULIAR COIN-TOSSING MODEL

A PECULIAR COIN-TOSSING MODEL A PECULIAR COIN-TOSSING MODEL EDWARD J. GREEN 1. Coin tossing according to de Finetti A coin is drawn at random from a finite set of coins. Each coin generates an i.i.d. sequence of outcomes (heads or

More information

Stat 225 Week 1, 8/20/12-8/24/12, Notes: Set Theory

Stat 225 Week 1, 8/20/12-8/24/12, Notes: Set Theory Stat 225 Week 1, 8/20/12-8/24/12, Notes: Set Theory The Fall 2012 Stat 225 T.A.s September 7, 2012 The material in this handout is intended to cover general set theory topics. Information includes (but

More information

Statistics for Managers Using Microsoft Excel (3 rd Edition)

Statistics for Managers Using Microsoft Excel (3 rd Edition) Statistics for Managers Using Microsoft Excel (3 rd Edition) Chapter 4 Basic Probability and Discrete Probability Distributions 2002 Prentice-Hall, Inc. Chap 4-1 Chapter Topics Basic probability concepts

More information

Probability Year 9. Terminology

Probability Year 9. Terminology Probability Year 9 Terminology Probability measures the chance something happens. Formally, we say it measures how likely is the outcome of an event. We write P(result) as a shorthand. An event is some

More information

Week 2. Review of Probability, Random Variables and Univariate Distributions

Week 2. Review of Probability, Random Variables and Univariate Distributions Week 2 Review of Probability, Random Variables and Univariate Distributions Probability Probability Probability Motivation What use is Probability Theory? Probability models Basis for statistical inference

More information