18.600: Lecture 7 Bayes formula and independence
|
|
|
- Diane Armstrong
- 5 years ago
- Views:
Transcription
1 Lecture : Lecture 7 Bayes formula and independence Scott Sheffield MIT
2 Lecture 7 Outline Bayes formula Independence
3 Lecture 7 Outline Bayes formula Independence
4 Recall definition: conditional probability Definition: P(E F ) = P(EF )/P(F ).
5 Recall definition: conditional probability Definition: P(E F ) = P(EF )/P(F ). Equivalent statement: P(EF ) = P(F )P(E F ).
6 Recall definition: conditional probability Definition: P(E F ) = P(EF )/P(F ). Equivalent statement: P(EF ) = P(F )P(E F ). Call P(E F ) the conditional probability of E given F or probability of E conditioned on F.
7 Lecture 7 Dividing probability into two cases P(E) = P(EF ) + P(EF c ) = P(E F )P(F ) + P(E F c )P(F c )
8 Lecture 7 Dividing probability into two cases P(E) = P(EF ) + P(EF c ) = P(E F )P(F ) + P(E F c )P(F c ) In words: want to know the probability of E. There are two scenarios F and F c. If I know the probabilities of the two scenarios and the probability of E conditioned on each scenario, I can work out the probability of E.
9 Lecture 7 Dividing probability into two cases P(E) = P(EF ) + P(EF c ) = P(E F )P(F ) + P(E F c )P(F c ) In words: want to know the probability of E. There are two scenarios F and F c. If I know the probabilities of the two scenarios and the probability of E conditioned on each scenario, I can work out the probability of E. Example: D = have disease, T = positive test.
10 Lecture 7 Dividing probability into two cases P(E) = P(EF ) + P(EF c ) = P(E F )P(F ) + P(E F c )P(F c ) In words: want to know the probability of E. There are two scenarios F and F c. If I know the probabilities of the two scenarios and the probability of E conditioned on each scenario, I can work out the probability of E. Example: D = have disease, T = positive test. If P(D) = p, P(T D) =.9, and P(T D c ) =.1, then P(T ) =.9p +.1(1 p).
11 Lecture 7 Dividing probability into two cases P(E) = P(EF ) + P(EF c ) = P(E F )P(F ) + P(E F c )P(F c ) In words: want to know the probability of E. There are two scenarios F and F c. If I know the probabilities of the two scenarios and the probability of E conditioned on each scenario, I can work out the probability of E. Example: D = have disease, T = positive test. If P(D) = p, P(T D) =.9, and P(T D c ) =.1, then P(T ) =.9p +.1(1 p). What is P(D T )?
12 Lecture 7 Bayes theorem Bayes theorem/law/rule states the following: P(A B) = P(B A)P(A) P(B).
13 Lecture 7 Bayes theorem Bayes theorem/law/rule states the following: P(A B) = P(B A)P(A) P(B). Follows from definition of conditional probability: P(AB) = P(B)P(A B) = P(A)P(B A).
14 Lecture 7 Bayes theorem Bayes theorem/law/rule states the following: P(A B) = P(B A)P(A) P(B). Follows from definition of conditional probability: P(AB) = P(B)P(A B) = P(A)P(B A). Tells how to update estimate of probability of A when new evidence restricts your sample space to B.
15 Lecture 7 Bayes theorem Bayes theorem/law/rule states the following: P(A B) = P(B A)P(A) P(B). Follows from definition of conditional probability: P(AB) = P(B)P(A B) = P(A)P(B A). Tells how to update estimate of probability of A when new evidence restricts your sample space to B. So P(A B) is P(B A) P(B) times P(A).
16 Lecture 7 Bayes theorem Bayes theorem/law/rule states the following: P(A B) = P(B A)P(A) P(B). Follows from definition of conditional probability: P(AB) = P(B)P(A B) = P(A)P(B A). Tells how to update estimate of probability of A when new evidence restricts your sample space to B. So P(A B) is P(B A) P(B) times P(A). Ratio P(B A) P(B) determines how compelling new evidence is.
17 Lecture 7 Bayes theorem Bayes theorem/law/rule states the following: P(A B) = P(B A)P(A) P(B). Follows from definition of conditional probability: P(AB) = P(B)P(A B) = P(A)P(B A). Tells how to update estimate of probability of A when new evidence restricts your sample space to B. So P(A B) is P(B A) P(B) times P(A). Ratio P(B A) P(B) determines how compelling new evidence is. What does it mean if ratio is zero?
18 Lecture 7 Bayes theorem Bayes theorem/law/rule states the following: P(A B) = P(B A)P(A) P(B). Follows from definition of conditional probability: P(AB) = P(B)P(A B) = P(A)P(B A). Tells how to update estimate of probability of A when new evidence restricts your sample space to B. So P(A B) is P(B A) P(B) times P(A). Ratio P(B A) P(B) determines how compelling new evidence is. What does it mean if ratio is zero? What if ratio is 1/P(A)?
19 Bayes theorem Bayes formula P(A B) = P(B A)P(A) P(B) to guide intuition. is often invoked as tool
20 Bayes theorem Bayes formula P(A B) = P(B A)P(A) P(B) is often invoked as tool to guide intuition. Example: A is event that suspect stole the 10, 000 under my mattress, B is event that suspect deposited several thousand dollars in cash in bank last week.
21 Bayes theorem Bayes formula P(A B) = P(B A)P(A) P(B) is often invoked as tool to guide intuition. Example: A is event that suspect stole the 10, 000 under my mattress, B is event that suspect deposited several thousand dollars in cash in bank last week. Begin with subjective estimates of P(A), P(B A), and P(B A c ). Compute P(B). Check whether B occurred. Update estimate.
22 Bayes theorem Bayes formula P(A B) = P(B A)P(A) P(B) is often invoked as tool to guide intuition. Example: A is event that suspect stole the 10, 000 under my mattress, B is event that suspect deposited several thousand dollars in cash in bank last week. Begin with subjective estimates of P(A), P(B A), and P(B A c ). Compute P(B). Check whether B occurred. Update estimate. Repeat procedure as new evidence emerges.
23 Bayes theorem Bayes formula P(A B) = P(B A)P(A) P(B) is often invoked as tool to guide intuition. Example: A is event that suspect stole the 10, 000 under my mattress, B is event that suspect deposited several thousand dollars in cash in bank last week. Begin with subjective estimates of P(A), P(B A), and P(B A c ). Compute P(B). Check whether B occurred. Update estimate. Repeat procedure as new evidence emerges. Caution required. My idea to check whether B occurred, or is a lawyer selecting the provable events B 1, B 2, B 3,... that maximize P(A B 1 B 2 B 3...)? Where did my probability estimates come from? What is my state space? What assumptions am I making?
24 Bayesian sometimes used to describe philosophical view Philosophical idea: we assign subjective probabilities to questions we can t answer. Will Obama win election? Will Red Sox win world series? Will stock prices go up this year?
25 Bayesian sometimes used to describe philosophical view Philosophical idea: we assign subjective probabilities to questions we can t answer. Will Obama win election? Will Red Sox win world series? Will stock prices go up this year? Bayes essentially described probability of event as value of right to get some thing if event occurs. value of thing
26 Bayesian sometimes used to describe philosophical view Philosophical idea: we assign subjective probabilities to questions we can t answer. Will Obama win election? Will Red Sox win world series? Will stock prices go up this year? Bayes essentially described probability of event as value of right to get some thing if event occurs. value of thing Philosophical questions: do we have subjective probabilities/hunches for questions we can t base enforceable contracts on? Do there exist other universes? Are there other intelligent beings? Are there beings smart enough to simulate universes like ours? Are we part of such a simulation?...
27 Bayesian sometimes used to describe philosophical view Philosophical idea: we assign subjective probabilities to questions we can t answer. Will Obama win election? Will Red Sox win world series? Will stock prices go up this year? Bayes essentially described probability of event as value of right to get some thing if event occurs. value of thing Philosophical questions: do we have subjective probabilities/hunches for questions we can t base enforceable contracts on? Do there exist other universes? Are there other intelligent beings? Are there beings smart enough to simulate universes like ours? Are we part of such a simulation?... Do we use Bayes subconsciously to update hunches?
28 Bayesian sometimes used to describe philosophical view Philosophical idea: we assign subjective probabilities to questions we can t answer. Will Obama win election? Will Red Sox win world series? Will stock prices go up this year? Bayes essentially described probability of event as value of right to get some thing if event occurs. value of thing Philosophical questions: do we have subjective probabilities/hunches for questions we can t base enforceable contracts on? Do there exist other universes? Are there other intelligent beings? Are there beings smart enough to simulate universes like ours? Are we part of such a simulation?... Do we use Bayes subconsciously to update hunches? Should we think of Bayesian priors and updates as part of the epistemological foundation of science and statistics?
29 Updated odds Define odds of A to be P(A)/P(A c ).
30 Updated odds Define odds of A to be P(A)/P(A c ). Define conditional odds of A given B to be P(A B)/P(A c B).
31 Updated odds Define odds of A to be P(A)/P(A c ). Define conditional odds of A given B to be P(A B)/P(A c B). Is there nice way to describe ratio between odds and conditional odds?
32 Updated odds Define odds of A to be P(A)/P(A c ). Define conditional odds of A given B to be P(A B)/P(A c B). Is there nice way to describe ratio between odds and conditional odds? P(A B)/P(Ac B) P(A)/P(A c ) =?
33 Lecture 7 Updated odds Define odds of A to be P(A)/P(A c ). Define conditional odds of A given B to be P(A B)/P(A c B). Is there nice way to describe ratio between odds and conditional odds? P(A B)/P(Ac B) P(A)/P(A c ) =? By Bayes P(A B)/P(A) = P(B A)/P(B).
34 Lecture 7 Updated odds Define odds of A to be P(A)/P(A c ). Define conditional odds of A given B to be P(A B)/P(A c B). Is there nice way to describe ratio between odds and conditional odds? P(A B)/P(Ac B) P(A)/P(A c ) =? By Bayes P(A B)/P(A) = P(B A)/P(B). After some algebra, P(A B)/P(Ac B) P(A)/P(A c ) = P(B A)/P(B A c )
35 Lecture 7 Updated odds Define odds of A to be P(A)/P(A c ). Define conditional odds of A given B to be P(A B)/P(A c B). Is there nice way to describe ratio between odds and conditional odds? P(A B)/P(Ac B) P(A)/P(A c ) =? By Bayes P(A B)/P(A) = P(B A)/P(B). After some algebra, P(A B)/P(Ac B) P(A)/P(A c ) = P(B A)/P(B A c ) Say I think A is 5 times as likely as A c, and P(B A) = 3P(B A c ). Given B, I think A is 15 times as likely as A c.
36 P( F ) is a probability measure We can check the probability axioms: 0 P(E F ) 1, P(S F ) = 1, and P( E i F ) = P(E i F ), if i ranges over a countable set and the E i are disjoint.
37 P( F ) is a probability measure We can check the probability axioms: 0 P(E F ) 1, P(S F ) = 1, and P( E i F ) = P(E i F ), if i ranges over a countable set and the E i are disjoint. The probability measure P( F ) is related to P( ).
38 P( F ) is a probability measure We can check the probability axioms: 0 P(E F ) 1, P(S F ) = 1, and P( E i F ) = P(E i F ), if i ranges over a countable set and the E i are disjoint. The probability measure P( F ) is related to P( ). To get former from latter, we set probabilities of elements outside of F to zero and multiply probabilities of events inside of F by 1/P(F ).
39 P( F ) is a probability measure We can check the probability axioms: 0 P(E F ) 1, P(S F ) = 1, and P( E i F ) = P(E i F ), if i ranges over a countable set and the E i are disjoint. The probability measure P( F ) is related to P( ). To get former from latter, we set probabilities of elements outside of F to zero and multiply probabilities of events inside of F by 1/P(F ). It P( ) is the prior probability measure and P( F ) is the posterior measure (revised after discovering that F occurs).
40 Lecture 7 Outline Bayes formula Independence
41 Lecture 7 Outline Bayes formula Independence
42 Independence Say E and F are independent if P(EF ) = P(E)P(F ).
43 Independence Say E and F are independent if P(EF ) = P(E)P(F ). Equivalent statement: P(E F ) = P(E). Also equivalent: P(F E) = P(F ).
44 Independence Say E and F are independent if P(EF ) = P(E)P(F ). Equivalent statement: P(E F ) = P(E). Also equivalent: P(F E) = P(F ). Example: toss two coins. Sample space contains four equally likely elements (H, H), (H, T ), (T, H), (T, T ).
45 Independence Say E and F are independent if P(EF ) = P(E)P(F ). Equivalent statement: P(E F ) = P(E). Also equivalent: P(F E) = P(F ). Example: toss two coins. Sample space contains four equally likely elements (H, H), (H, T ), (T, H), (T, T ). Is event that first coin is heads independent of event that second coin heads.
46 Independence Say E and F are independent if P(EF ) = P(E)P(F ). Equivalent statement: P(E F ) = P(E). Also equivalent: P(F E) = P(F ). Example: toss two coins. Sample space contains four equally likely elements (H, H), (H, T ), (T, H), (T, T ). Is event that first coin is heads independent of event that second coin heads. Yes: probability of each event is 1/2 and probability of both is 1/4.
47 Independence Say E and F are independent if P(EF ) = P(E)P(F ). Equivalent statement: P(E F ) = P(E). Also equivalent: P(F E) = P(F ). Example: toss two coins. Sample space contains four equally likely elements (H, H), (H, T ), (T, H), (T, T ). Is event that first coin is heads independent of event that second coin heads. Yes: probability of each event is 1/2 and probability of both is 1/4. Is event that first coin is heads independent of event that number of heads is odd?
48 Independence Say E and F are independent if P(EF ) = P(E)P(F ). Equivalent statement: P(E F ) = P(E). Also equivalent: P(F E) = P(F ). Example: toss two coins. Sample space contains four equally likely elements (H, H), (H, T ), (T, H), (T, T ). Is event that first coin is heads independent of event that second coin heads. Yes: probability of each event is 1/2 and probability of both is 1/4. Is event that first coin is heads independent of event that number of heads is odd? Yes: probability of each event is 1/2 and probability of both is 1/4...
49 Independence Say E and F are independent if P(EF ) = P(E)P(F ). Equivalent statement: P(E F ) = P(E). Also equivalent: P(F E) = P(F ). Example: toss two coins. Sample space contains four equally likely elements (H, H), (H, T ), (T, H), (T, T ). Is event that first coin is heads independent of event that second coin heads. Yes: probability of each event is 1/2 and probability of both is 1/4. Is event that first coin is heads independent of event that number of heads is odd? Yes: probability of each event is 1/2 and probability of both is 1/4... despite intuitive sense that oddness of the number of heads depends on the first coin.
50 Lecture 7 Independence of multiple events Say E 1... E n are independent if for each {i 1, i 2,..., i k } {1, 2,... n} we have P(E i1 E i2... E ik ) = P(E i1 )P(E i2 )... P(E ik ).
51 Lecture 7 Independence of multiple events Say E 1... E n are independent if for each {i 1, i 2,..., i k } {1, 2,... n} we have P(E i1 E i2... E ik ) = P(E i1 )P(E i2 )... P(E ik ). In other words, the product rule works.
52 Lecture 7 Independence of multiple events Say E 1... E n are independent if for each {i 1, i 2,..., i k } {1, 2,... n} we have P(E i1 E i2... E ik ) = P(E i1 )P(E i2 )... P(E ik ). In other words, the product rule works. Independence implies P(E 1 E 2 E 3 E 4 E 5 E 6 ) = P(E 1 )P(E 2 )P(E 3 )P(E 4 )P(E 5 )P(E 6 ) P(E 4 )P(E 5 )P(E 6 ) = P(E 1 E 2 E 3 ), and other similar statements.
53 Lecture 7 Independence of multiple events Say E 1... E n are independent if for each {i 1, i 2,..., i k } {1, 2,... n} we have P(E i1 E i2... E ik ) = P(E i1 )P(E i2 )... P(E ik ). In other words, the product rule works. Independence implies P(E 1 E 2 E 3 E 4 E 5 E 6 ) = P(E 1 )P(E 2 )P(E 3 )P(E 4 )P(E 5 )P(E 6 ) P(E 4 )P(E 5 )P(E 6 ) = P(E 1 E 2 E 3 ), and other similar statements. Does pairwise independence imply independence?
54 Lecture 7 Independence of multiple events Say E 1... E n are independent if for each {i 1, i 2,..., i k } {1, 2,... n} we have P(E i1 E i2... E ik ) = P(E i1 )P(E i2 )... P(E ik ). In other words, the product rule works. Independence implies P(E 1 E 2 E 3 E 4 E 5 E 6 ) = P(E 1 )P(E 2 )P(E 3 )P(E 4 )P(E 5 )P(E 6 ) P(E 4 )P(E 5 )P(E 6 ) = P(E 1 E 2 E 3 ), and other similar statements. Does pairwise independence imply independence? No. Consider these three events: first coin heads, second coin heads, odd number heads. Pairwise independent, not independent.
55 Lecture 7 Independence: another example Shuffle 4 cards with labels 1 through 4. Let E j,k be event that card j comes before card k. Is E 1,2 independent of E 3,4?
56 Lecture 7 Independence: another example Shuffle 4 cards with labels 1 through 4. Let E j,k be event that card j comes before card k. Is E 1,2 independent of E 3,4? Is E 1,2 independent of E 1,3?
57 Lecture 7 Independence: another example Shuffle 4 cards with labels 1 through 4. Let E j,k be event that card j comes before card k. Is E 1,2 independent of E 3,4? Is E 1,2 independent of E 1,3? No. In fact, what is P(E 1,2 E 1,3 )?
58 Lecture 7 Independence: another example Shuffle 4 cards with labels 1 through 4. Let E j,k be event that card j comes before card k. Is E 1,2 independent of E 3,4? Is E 1,2 independent of E 1,3? No. In fact, what is P(E 1,2 E 1,3 )? Generalize to n > 7 cards. What is P(E 1,7 E 1,2 E 1,3 E 1,4 E 1,5 E 1,6 )?
Lecture 7. Bayes formula and independence
18.440: Lecture 7 Bayes formula and independence Scott Sheffield MIT 1 Outline Bayes formula Independence 2 Outline Bayes formula Independence 3 Recall definition: conditional probability Definition: P(E
18.600: Lecture 7 Bayes formula and independence
18.600: Lecture 7 Bayes formula and independence Scott Sheffield MIT Outline Bayes formula Independence Outline Bayes formula Independence Recall definition: conditional probability Definition: P(E F )
Lecture 16. Lectures 1-15 Review
18.440: Lecture 16 Lectures 1-15 Review Scott Sheffield MIT 1 Outline Counting tricks and basic principles of probability Discrete random variables 2 Outline Counting tricks and basic principles of probability
STAT:5100 (22S:193) Statistical Inference I
STAT:5100 (22S:193) Statistical Inference I Week 3 Luke Tierney University of Iowa Fall 2015 Luke Tierney (U Iowa) STAT:5100 (22S:193) Statistical Inference I Fall 2015 1 Recap Matching problem Generalized
MA : Introductory Probability
MA 320-001: Introductory Probability David Murrugarra Department of Mathematics, University of Kentucky http://www.math.uky.edu/~dmu228/ma320/ Spring 2017 David Murrugarra (University of Kentucky) MA 320:
LECTURE 1. 1 Introduction. 1.1 Sample spaces and events
LECTURE 1 1 Introduction The first part of our adventure is a highly selective review of probability theory, focusing especially on things that are most useful in statistics. 1.1 Sample spaces and events
Chapter 3 : Conditional Probability and Independence
STAT/MATH 394 A - PROBABILITY I UW Autumn Quarter 2016 Néhémy Lim Chapter 3 : Conditional Probability and Independence 1 Conditional Probabilities How should we modify the probability of an event when
Lecture 9. Expectations of discrete random variables
18.440: Lecture 9 Expectations of discrete random variables Scott Sheffield MIT 1 Outline Defining expectation Functions of random variables Motivation 2 Outline Defining expectation Functions of random
18.600: Lecture 3 What is probability?
18.600: Lecture 3 What is probability? Scott Sheffield MIT Outline Formalizing probability Sample space DeMorgan s laws Axioms of probability Outline Formalizing probability Sample space DeMorgan s laws
Example: Suppose we toss a quarter and observe whether it falls heads or tails, recording the result as 1 for heads and 0 for tails.
Example: Suppose we toss a quarter and observe whether it falls heads or tails, recording the result as 1 for heads and 0 for tails. (In Mathematical language, the result of our toss is a random variable,
Conditional Probability and Bayes
Conditional Probability and Bayes Chapter 2 Lecture 5 Yiren Ding Shanghai Qibao Dwight High School March 9, 2016 Yiren Ding Conditional Probability and Bayes 1 / 13 Outline 1 Independent Events Definition
18.600: Lecture 4 Axioms of probability and inclusion-exclusion
18.600: Lecture 4 Axioms of probability and inclusion-exclusion Scott Sheffield MIT Outline Axioms of probability Consequences of axioms Inclusion exclusion Outline Axioms of probability Consequences of
Independence. CS 109 Lecture 5 April 6th, 2016
Independence CS 109 Lecture 5 April 6th, 2016 Today s Topics 2 Last time: Conditional Probability Bayes Theorem Today: Independence Conditional Independence Next time: Random Variables The Tragedy of Conditional
Lecture 10. Variance and standard deviation
18.440: Lecture 10 Variance and standard deviation Scott Sheffield MIT 1 Outline Defining variance Examples Properties Decomposition trick 2 Outline Defining variance Examples Properties Decomposition
Mutually Exclusive Events
172 CHAPTER 3 PROBABILITY TOPICS c. QS, 7D, 6D, KS Mutually Exclusive Events A and B are mutually exclusive events if they cannot occur at the same time. This means that A and B do not share any outcomes
Lecture 2: Probability, conditional probability, and independence
Lecture 2: Probability, conditional probability, and independence Theorem 1.2.6. Let S = {s 1,s 2,...} and F be all subsets of S. Let p 1,p 2,... be nonnegative numbers that sum to 1. The following defines
4 Lecture 4 Notes: Introduction to Probability. Probability Rules. Independence and Conditional Probability. Bayes Theorem. Risk and Odds Ratio
4 Lecture 4 Notes: Introduction to Probability. Probability Rules. Independence and Conditional Probability. Bayes Theorem. Risk and Odds Ratio Wrong is right. Thelonious Monk 4.1 Three Definitions of
CMPSCI 240: Reasoning about Uncertainty
CMPSCI 240: Reasoning about Uncertainty Lecture 4: Sequential experiments Andrew McGregor University of Massachusetts Last Compiled: February 2, 2017 Outline 1 Recap 2 Sequential Experiments 3 Total Probability
Denker FALL Probability- Assignment 6
Denker FALL 2010 418 Probability- Assignment 6 Due Date: Thursday, Oct. 7, 2010 Write the final answer to the problems on this assignment attach the worked out solutions! Problem 1: A box contains n +
Lecture Notes 1 Basic Probability. Elements of Probability. Conditional probability. Sequential Calculation of Probability
Lecture Notes 1 Basic Probability Set Theory Elements of Probability Conditional probability Sequential Calculation of Probability Total Probability and Bayes Rule Independence Counting EE 178/278A: Basic
Dynamic Programming Lecture #4
Dynamic Programming Lecture #4 Outline: Probability Review Probability space Conditional probability Total probability Bayes rule Independent events Conditional independence Mutual independence Probability
CS4705. Probability Review and Naïve Bayes. Slides from Dragomir Radev
CS4705 Probability Review and Naïve Bayes Slides from Dragomir Radev Classification using a Generative Approach Previously on NLP discriminative models P C D here is a line with all the social media posts
Axiomatic Foundations of Probability. Definition: Probability Function
Chapter 1 sections We will SKIP a number of sections Set theory SKIP Real number uncountability Definition of probability Finite sample spaces Counting methods Combinatorial methods SKIP tennis tournament
Chapter 2. Conditional Probability and Independence. 2.1 Conditional Probability
Chapter 2 Conditional Probability and Independence 2.1 Conditional Probability Example: Two dice are tossed. What is the probability that the sum is 8? This is an easy exercise: we have a sample space
Outline Conditional Probability The Law of Total Probability and Bayes Theorem Independent Events. Week 4 Classical Probability, Part II
Week 4 Classical Probability, Part II Week 4 Objectives This week we continue covering topics from classical probability. The notion of conditional probability is presented first. Important results/tools
Introduction and basic definitions
Chapter 1 Introduction and basic definitions 1.1 Sample space, events, elementary probability Exercise 1.1 Prove that P( ) = 0. Solution of Exercise 1.1 : Events S (where S is the sample space) and are
Chapter 2. Conditional Probability and Independence. 2.1 Conditional Probability
Chapter 2 Conditional Probability and Independence 2.1 Conditional Probability Probability assigns a likelihood to results of experiments that have not yet been conducted. Suppose that the experiment has
Lecture Lecture 5
Lecture 4 --- Lecture 5 A. Basic Concepts (4.1-4.2) 1. Experiment: A process of observing a phenomenon that has variation in its outcome. Examples: (E1). Rolling a die, (E2). Drawing a card form a shuffled
Bayesian statistics, simulation and software
Module 1: Course intro and probability brush-up Department of Mathematical Sciences Aalborg University 1/22 Bayesian Statistics, Simulations and Software Course outline Course consists of 12 half-days
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 3 9/10/2008 CONDITIONING AND INDEPENDENCE
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 3 9/10/2008 CONDITIONING AND INDEPENDENCE Most of the material in this lecture is covered in [Bertsekas & Tsitsiklis] Sections 1.3-1.5
the time it takes until a radioactive substance undergoes a decay
1 Probabilities 1.1 Experiments with randomness Wewillusethetermexperimentinaverygeneralwaytorefertosomeprocess that produces a random outcome. Examples: (Ask class for some first) Here are some discrete
With Question/Answer Animations. Chapter 7
With Question/Answer Animations Chapter 7 Chapter Summary Introduction to Discrete Probability Probability Theory Bayes Theorem Section 7.1 Section Summary Finite Probability Probabilities of Complements
Statistical Inference
Statistical Inference Lecture 1: Probability Theory MING GAO DASE @ ECNU (for course related communications) mgao@dase.ecnu.edu.cn Sep. 11, 2018 Outline Introduction Set Theory Basics of Probability Theory
Conditional Probability & Independence. Conditional Probabilities
Conditional Probability & Independence Conditional Probabilities Question: How should we modify P(E) if we learn that event F has occurred? Definition: the conditional probability of E given F is P(E F
18.175: Lecture 2 Extension theorems, random variables, distributions
18.175: Lecture 2 Extension theorems, random variables, distributions Scott Sheffield MIT Outline Extension theorems Characterizing measures on R d Random variables Outline Extension theorems Characterizing
Statistical Theory 1
Statistical Theory 1 Set Theory and Probability Paolo Bautista September 12, 2017 Set Theory We start by defining terms in Set Theory which will be used in the following sections. Definition 1 A set is
Lecture 23. Random walks
18.175: Lecture 23 Random walks Scott Sheffield MIT 1 Outline Random walks Stopping times Arcsin law, other SRW stories 2 Outline Random walks Stopping times Arcsin law, other SRW stories 3 Exchangeable
Probability Theory for Machine Learning. Chris Cremer September 2015
Probability Theory for Machine Learning Chris Cremer September 2015 Outline Motivation Probability Definitions and Rules Probability Distributions MLE for Gaussian Parameter Estimation MLE and Least Squares
Chapter 2.5 Random Variables and Probability The Modern View (cont.)
Chapter 2.5 Random Variables and Probability The Modern View (cont.) I. Statistical Independence A crucially important idea in probability and statistics is the concept of statistical independence. Suppose
Lecture 1. ABC of Probability
Math 408 - Mathematical Statistics Lecture 1. ABC of Probability January 16, 2013 Konstantin Zuev (USC) Math 408, Lecture 1 January 16, 2013 1 / 9 Agenda Sample Spaces Realizations, Events Axioms of Probability
Independence. P(A) = P(B) = 3 6 = 1 2, and P(C) = 4 6 = 2 3.
Example: A fair die is tossed and we want to guess the outcome. The outcomes will be 1, 2, 3, 4, 5, 6 with equal probability 1 6 each. If we are interested in getting the following results: A = {1, 3,
Chapter 3 Conditional Probability and Independence. Wen-Guey Tzeng Computer Science Department National Chiao Tung University
Chapter 3 Conditional Probability and Independence Wen-Guey Tzeng Computer Science Department National Chiao Tung University Conditional probability P(A B) = the probability of event A given the occurrence
Introduction to Probability
Introduction to Probability Content Experiments, Counting Rules, and Assigning Probabilities Events and Their Probability Some Basic Relationships of Probability Conditional Probability Bayes Theorem 2
Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14
CS 70 Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14 Introduction One of the key properties of coin flips is independence: if you flip a fair coin ten times and get ten
Probability Theory and Applications
Probability Theory and Applications Videos of the topics covered in this manual are available at the following links: Lesson 4 Probability I http://faculty.citadel.edu/silver/ba205/online course/lesson
CS626 Data Analysis and Simulation
CS626 Data Analysis and Simulation Instructor: Peter Kemper R 104A, phone 221-3462, email:kemper@cs.wm.edu Today: Probability Primer Quick Reference: Sheldon Ross: Introduction to Probability Models 9th
P (A B) P ((B C) A) P (B A) = P (B A) + P (C A) P (A) = P (B A) + P (C A) = Q(A) + Q(B).
Lectures 7-8 jacques@ucsdedu 41 Conditional Probability Let (Ω, F, P ) be a probability space Suppose that we have prior information which leads us to conclude that an event A F occurs Based on this information,
Consider an experiment that may have different outcomes. We are interested to know what is the probability of a particular set of outcomes.
CMSC 310 Artificial Intelligence Probabilistic Reasoning and Bayesian Belief Networks Probabilities, Random Variables, Probability Distribution, Conditional Probability, Joint Distributions, Bayes Theorem
Conditional Probability. CS231 Dianna Xu
Conditional Probability CS231 Dianna Xu 1 Boy or Girl? A couple has two children, one of them is a girl. What is the probability that the other one is also a girl? Assuming 50/50 chances of conceiving
Lecture 2. Conditional Probability
Math 408 - Mathematical Statistics Lecture 2. Conditional Probability January 18, 2013 Konstantin Zuev (USC) Math 408, Lecture 2 January 18, 2013 1 / 9 Agenda Motivation and Definition Properties of Conditional
AMS7: WEEK 2. CLASS 2
AMS7: WEEK 2. CLASS 2 Introduction to Probability. Probability Rules. Independence and Conditional Probability. Bayes Theorem. Risk and Odds Ratio Friday April 10, 2015 Probability: Introduction Probability:
4. Conditional Probability P( ) CSE 312 Autumn 2012 W.L. Ruzzo
4. Conditional Probability P( ) CSE 312 Autumn 2012 W.L. Ruzzo 1 conditional probability Conditional probability of E given F: probability that E occurs given that F has occurred. Conditioning on F S Written
Lecture 4. Selected material from: Ch. 6 Probability
Lecture 4 Selected material from: Ch. 6 Probability Example: Music preferences F M Suppose you want to know what types of CD s males and females are more likely to buy. The CD s are classified as Classical,
Today. Statistical Learning. Coin Flip. Coin Flip. Experiment 1: Heads. Experiment 1: Heads. Which coin will I use? Which coin will I use?
Today Statistical Learning Parameter Estimation: Maximum Likelihood (ML) Maximum A Posteriori (MAP) Bayesian Continuous case Learning Parameters for a Bayesian Network Naive Bayes Maximum Likelihood estimates
Dept. of Linguistics, Indiana University Fall 2015
L645 Dept. of Linguistics, Indiana University Fall 2015 1 / 34 To start out the course, we need to know something about statistics and This is only an introduction; for a fuller understanding, you would
Determining Probabilities. Product Rule for Ordered Pairs/k-Tuples:
Determining Probabilities Product Rule for Ordered Pairs/k-Tuples: Determining Probabilities Product Rule for Ordered Pairs/k-Tuples: Proposition If the first element of object of an ordered pair can be
Lecture 4: Probability, Proof Techniques, Method of Induction Lecturer: Lale Özkahya
BBM 205 Discrete Mathematics Hacettepe University http://web.cs.hacettepe.edu.tr/ bbm205 Lecture 4: Probability, Proof Techniques, Method of Induction Lecturer: Lale Özkahya Resources: Kenneth Rosen, Discrete
SDS 321: Introduction to Probability and Statistics
SDS 321: Introduction to Probability and Statistics Lecture 2: Conditional probability Purnamrita Sarkar Department of Statistics and Data Science The University of Texas at Austin www.cs.cmu.edu/ psarkar/teaching
Chapter 3 Conditional Probability and Independence. Wen-Guey Tzeng Computer Science Department National Chiao Tung University
Chapter 3 Conditional Probability and Independence Wen-Guey Tzeng Computer Science Department National Chiao Tung University Conditional probability P(A B) = the probability of event A given the occurrence
Mean, Median and Mode. Lecture 3 - Axioms of Probability. Where do they come from? Graphically. We start with a set of 21 numbers, Sta102 / BME102
Mean, Median and Mode Lecture 3 - Axioms of Probability Sta102 / BME102 Colin Rundel September 1, 2014 We start with a set of 21 numbers, ## [1] -2.2-1.6-1.0-0.5-0.4-0.3-0.2 0.1 0.1 0.2 0.4 ## [12] 0.4
Lecture 9: Conditional Probability and Independence
EE5110: Probability Foundations July-November 2015 Lecture 9: Conditional Probability and Independence Lecturer: Dr. Krishna Jagannathan Scribe: Vishakh Hegde 9.1 Conditional Probability Definition 9.1
Chapter 2 Probability
Basic Axioms and properties Chapter 2 Probability 1. 0 Pr(A) 1 2. Pr(S) = 1 3. A B Pr(A) Pr(B) 4. Pr(φ ) = 0 5. If A and B are disjoint i.e. A B φ P( A B) = P(A) + P(B). 6. For any two events A and B,
Discrete Structures for Computer Science
Discrete Structures for Computer Science William Garrison bill@cs.pitt.edu 6311 Sennott Square Lecture #24: Probability Theory Based on materials developed by Dr. Adam Lee Not all events are equally likely
Lecture 4 Bayes Theorem
Lecture 4 Bayes Theorem Thais Paiva STA 111 - Summer 2013 Term II July 5, 2013 Lecture Plan 1 Probability Review 2 Bayes Theorem 3 More worked problems Why Study Probability? A probability model describes
2. AXIOMATIC PROBABILITY
IA Probability Lent Term 2. AXIOMATIC PROBABILITY 2. The axioms The formulation for classical probability in which all outcomes or points in the sample space are equally likely is too restrictive to develop
Lecture 7. Sums of random variables
18.175: Lecture 7 Sums of random variables Scott Sheffield MIT 18.175 Lecture 7 1 Outline Definitions Sums of random variables 18.175 Lecture 7 2 Outline Definitions Sums of random variables 18.175 Lecture
Recitation 2: Probability
Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions
Let us think of the situation as having a 50 sided fair die; any one number is equally likely to appear.
Probability_Homework Answers. Let the sample space consist of the integers through. {, 2, 3,, }. Consider the following events from that Sample Space. Event A: {a number is a multiple of 5 5, 0, 5,, }
Lecture 10: Bayes' Theorem, Expected Value and Variance Lecturer: Lale Özkahya
BBM 205 Discrete Mathematics Hacettepe University http://web.cs.hacettepe.edu.tr/ bbm205 Lecture 10: Bayes' Theorem, Expected Value and Variance Lecturer: Lale Özkahya Resources: Kenneth Rosen, Discrete
Conditional Probability & Independence. Conditional Probabilities
Conditional Probability & Independence Conditional Probabilities Question: How should we modify P(E) if we learn that event F has occurred? Definition: the conditional probability of E given F is P(E F
EE 178 Lecture Notes 0 Course Introduction. About EE178. About Probability. Course Goals. Course Topics. Lecture Notes EE 178
EE 178 Lecture Notes 0 Course Introduction About EE178 About Probability Course Goals Course Topics Lecture Notes EE 178: Course Introduction Page 0 1 EE 178 EE 178 provides an introduction to probabilistic
Econ 325: Introduction to Empirical Economics
Econ 325: Introduction to Empirical Economics Lecture 2 Probability Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall Ch. 3-1 3.1 Definition Random Experiment a process leading to an uncertain
CS 361: Probability & Statistics
February 12, 2018 CS 361: Probability & Statistics Random Variables Monty hall problem Recall the setup, there are 3 doors, behind two of them are indistinguishable goats, behind one is a car. You pick
Compound Events. The event E = E c (the complement of E) is the event consisting of those outcomes which are not in E.
Compound Events Because we are using the framework of set theory to analyze probability, we can use unions, intersections and complements to break complex events into compositions of events for which it
Probability Review Lecturer: Ji Liu Thank Jerry Zhu for sharing his slides
Probability Review Lecturer: Ji Liu Thank Jerry Zhu for sharing his slides slide 1 Inference with Bayes rule: Example In a bag there are two envelopes one has a red ball (worth $100) and a black ball one
MATH 556: PROBABILITY PRIMER
MATH 6: PROBABILITY PRIMER 1 DEFINITIONS, TERMINOLOGY, NOTATION 1.1 EVENTS AND THE SAMPLE SPACE Definition 1.1 An experiment is a one-off or repeatable process or procedure for which (a there is a well-defined
Origins of Probability Theory
1 16.584: INTRODUCTION Theory and Tools of Probability required to analyze and design systems subject to uncertain outcomes/unpredictability/randomness. Such systems more generally referred to as Experiments.
Lecture 1: Probability Fundamentals
Lecture 1: Probability Fundamentals IB Paper 7: Probability and Statistics Carl Edward Rasmussen Department of Engineering, University of Cambridge January 22nd, 2008 Rasmussen (CUED) Lecture 1: Probability
Lecture 4 : Conditional Probability and Bayes Theorem 0/ 26
0/ 26 The conditional sample space Motivating examples 1. Roll a fair die once 1 2 3 S = 4 5 6 Let A = 6 appears B = an even number appears So P(A) = 1 6 P(B) = 1 2 1/ 26 Now what about P ( 6 appears given
P (E) = P (A 1 )P (A 2 )... P (A n ).
Lecture 9: Conditional probability II: breaking complex events into smaller events, methods to solve probability problems, Bayes rule, law of total probability, Bayes theorem Discrete Structures II (Summer
18.440: Lecture 26 Conditional expectation
18.440: Lecture 26 Conditional expectation Scott Sheffield MIT 1 Outline Conditional probability distributions Conditional expectation Interpretation and examples 2 Outline Conditional probability distributions
Announcements. Topics: To Do:
Announcements Topics: In the Probability and Statistics module: - Sections 1 + 2: Introduction to Stochastic Models - Section 3: Basics of Probability Theory - Section 4: Conditional Probability; Law of
Lecture 3 - Axioms of Probability
Lecture 3 - Axioms of Probability Sta102 / BME102 January 25, 2016 Colin Rundel Axioms of Probability What does it mean to say that: The probability of flipping a coin and getting heads is 1/2? 3 What
CSCI 5832 Natural Language Processing. Today 1/31. Probability Basics. Lecture 6. Probability. Language Modeling (N-grams)
CSCI 5832 Natural Language Processing Jim Martin Lecture 6 1 Today 1/31 Probability Basic probability Conditional probability Bayes Rule Language Modeling (N-grams) N-gram Intro The Chain Rule Smoothing:
STAT 285 Fall Assignment 1 Solutions
STAT 285 Fall 2014 Assignment 1 Solutions 1. An environmental agency sets a standard of 200 ppb for the concentration of cadmium in a lake. The concentration of cadmium in one lake is measured 17 times.
Introduction to Probability 2017/18 Supplementary Problems
Introduction to Probability 2017/18 Supplementary Problems Problem 1: Let A and B denote two events with P(A B) 0. Show that P(A) 0 and P(B) 0. A A B implies P(A) P(A B) 0, hence P(A) 0. Similarly B A
LECTURE NOTES by DR. J.S.V.R. KRISHNA PRASAD
.0 Introduction: The theory of probability has its origin in the games of chance related to gambling such as tossing of a coin, throwing of a die, drawing cards from a pack of cards etc. Jerame Cardon,
18.175: Lecture 14 Infinite divisibility and so forth
18.175 Lecture 14 18.175: Lecture 14 Infinite divisibility and so forth Scott Sheffield MIT 18.175 Lecture 14 Outline Infinite divisibility Higher dimensional CFs and CLTs Random walks Stopping times Arcsin
Probability COMP 245 STATISTICS. Dr N A Heard. 1 Sample Spaces and Events Sample Spaces Events Combinations of Events...
Probability COMP 245 STATISTICS Dr N A Heard Contents Sample Spaces and Events. Sample Spaces........................................2 Events........................................... 2.3 Combinations
Lecture 4 Bayes Theorem
1 / 24 Lecture 4 Bayes Theorem September 09, 2010 2 / 24 Lesson Plan 1. Bayes Theorem 2. Simpson s Paradox 3. More worked problems 3 / 24 Why Study Probability? A probability model describes the random
CSE 312, 2017 Winter, W.L.Ruzzo. 5. independence [ ]
![CSE 312, 2017 Winter, W.L.Ruzzo. 5. independence [ ]](/thumbs/95/123242418.jpg "CSE 312, 2017 Winter, W.L.Ruzzo. 5. independence [ ]")
CSE 312, 2017 Winter, W.L.Ruzzo 5. independence [ ] independence Defn: Two events E and F are independent if P(EF) = P(E) P(F) If P(F)>0, this is equivalent to: P(E F) = P(E) (proof below) Otherwise, they
The enumeration of all possible outcomes of an experiment is called the sample space, denoted S. E.g.: S={head, tail}
Random Experiment In random experiments, the result is unpredictable, unknown prior to its conduct, and can be one of several choices. Examples: The Experiment of tossing a coin (head, tail) The Experiment
STAT 430/510 Probability
STAT 430/510 Probability Hui Nie Lecture 3 May 28th, 2009 Review We have discussed counting techniques in Chapter 1. Introduce the concept of the probability of an event. Compute probabilities in certain
Introduction to Machine Learning
Introduction to Machine Learning CS4375 --- Fall 2018 Bayesian a Learning Reading: Sections 13.1-13.6, 20.1-20.2, R&N Sections 6.1-6.3, 6.7, 6.9, Mitchell 1 Uncertainty Most real-world problems deal with
LEARNING WITH BAYESIAN NETWORKS
LEARNING WITH BAYESIAN NETWORKS Author: David Heckerman Presented by: Dilan Kiley Adapted from slides by: Yan Zhang - 2006, Jeremy Gould 2013, Chip Galusha -2014 Jeremy Gould 2013Chip Galus May 6th, 2016
Why study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables
ECE 6010 Lecture 1 Introduction; Review of Random Variables Readings from G&S: Chapter 1. Section 2.1, Section 2.3, Section 2.4, Section 3.1, Section 3.2, Section 3.5, Section 4.1, Section 4.2, Section
Lecture 1. Chapter 1. (Part I) Material Covered in This Lecture: Chapter 1, Chapter 2 ( ). 1. What is Statistics?
Lecture 1 (Part I) Material Covered in This Lecture: Chapter 1, Chapter 2 (2.1 --- 2.6). Chapter 1 1. What is Statistics? 2. Two definitions. (1). Population (2). Sample 3. The objective of statistics.
Week 2: Probability: Counting, Sets, and Bayes
Statistical Methods APPM 4570/5570, STAT 4000/5000 21 Probability Introduction to EDA Week 2: Probability: Counting, Sets, and Bayes Random variable Random variable is a measurable quantity whose outcome
Introduction to Machine Learning
Uncertainty Introduction to Machine Learning CS4375 --- Fall 2018 a Bayesian Learning Reading: Sections 13.1-13.6, 20.1-20.2, R&N Sections 6.1-6.3, 6.7, 6.9, Mitchell Most real-world problems deal with
What is Probability? Probability. Sample Spaces and Events. Simple Event
What is Probability? Probability Peter Lo Probability is the numerical measure of likelihood that the event will occur. Simple Event Joint Event Compound Event Lies between 0 & 1 Sum of events is 1 1.5