CSE 21 Math for Algorithms and Systems Analysis. Lecture 10 Condi<onal Probability

Transcription

1 CSE 21 Math for Algorithms and Systems Analysis Lecture 10 Condi<onal Probability

2 Outline Review of defini<ons of probability Condi<onal Probability Decision Trees and Probability Intro to Bayes Rule

3 Probability Defini<on A probability space is given by two things: A set, U, called the sample space (think of this as all possible things of interest that can occur) A probability func<on, f, that specifies how likely each of the elements in U is to occur. In order to be a valid probability space we require that: x U f(x) = 1 and 0 f(x) 1 x U

4 Probability of an Event We define an event as a subset of the elements in the set U Suppose the set E is an event ( E U ) The probability of the event E occurring is: P (E) = x E f(x)

5 Probability Sample Problems Consider drawing a 5- card hand from a deck of cards Compute the probability of: GeVng at least 2 queens GeVng a full house GeVng a royal flush

6 Probability and Coun<ng the Complement P (E) =1 P (E) Example Birthday Problem What is the probability that out of a class of n people at least two people share the same birthday (my apologies to anyone born on February 29 th!) P (E) =1 365 n n! 365 n The probability that at least two people in this class share a birthday is:

7 Another Component of the Birthday Problem Suppose we have a class of 62 people. Suppose I start from the first person and ask them to state their birthday Then I ask if anyone in the class shares the same birthday What is the probability that we iden<fy a common birthday within n people

8 Workspace

9 Probabili<es for some Values of n Probability a^er 1 selec<on Probability a^er 2 selec<ons Probability a^er 3 selec<ons Probability a^er 4 selec<ons Probability a^er 5 selec<ons Probability a^er 6 selec<ons Probability a^er 7 selec<ons Probability a^er 8 selec<ons Probability a^er 20 selec<ons

10 Binomial Probability Distribu<on Suppose we perform a sequence of n trials. The probability of a success on each trial is q. Each trial is independent. The probability of achieving exactly k successes in n trials is: P (k successes) = n q k (1 q) n k k

11 Probability and Decision Trees Suppose we observe San Diego weather over a period of 3 days. What is the probability that it rains exactly two of the days? Assume (for now) that the probability of it raining on any par<cular day is 1/10 and that the event of it raining on any par<cular day is independent of it raining on any other day Compute the probability of it raining exactly two of the days

12 Workspace

13 Condi<onal Probability Here is another model of San Diego weather. Let R i be the event that it rains on day i and S i be the event that it is sunny on day i. P(R i ) = 1/10 P(R i R i- 1 ) = 7/10 Read as Given P(S i S i- 1 ) = 95/100 Read as Given

14 Defini<on of Condi<onal Probability U P (A B) = P (A B) P (B) A B Given f(x) = 1/15 for all x. What is P(A B)?

15 Condi<onal Probability and Condi<onal Independence Recall our defini<on of the independence of two events A and B. We say A and B are independent events if and only if Two events A and B are condi<onally independent given a third event C if and only if Equivalently: P (A B) =P (A)P (B) P (A B C) =P (A C)P (B C) P (A B C) =P (A C)

16 Back to our Example About The Weather in San Diego Suppose we want to know the probability of it raining on 2 out of three days in San Diego Recall our model of the weather P(R i ) = 1/10 P(R i R i- 1 ) = 7/10 P(S i S i- 1 ) = 95/100 Addi<onally suppose R i is condi<onally independent of R 1 R i- 2 given R i- 1

17 Represen<ng Condi<onal Probabili<es Using Decision Trees P(S 1 ) =.9 P(R 1 ) =.1 Day 1 P(S 2 S 1 ) =.95 S P(R 2 S 1 ) =.05 P(S 2 R 1 ) = 3/10 R P(R 2 R 1 ) = 7/10 Day 2 S R S R P (S 3 S 2 S 1 )= Day 3 S R S R S R S R

18 Decision Trees and The Rule of Product The rule of product states that P (A 1 A 2... A n )=P (A 1 ) P (A 2 A 1 ) P (A 3 A 1 A 2 )... P (A n A 1... A n 1 ) Remember the commuta<ve property of set intersec<on. What does this tell us about the preceding rule?

19 Decision Trees and the Rule of Product P(S 1 ) =.9 P(R 1 ) =.1 Day 1 P(S 2 S 1 ) =.95 S P(S 2 R 1 ) = 3/10 R Day 2 S P (R 3 S 1 S 2 )=.05 P (R 3 R 1 S 2 )=.05 S Day 3 R R P (S 1 R 2 R 3 )= P (R 1 S 2 R 3 )=

20 Sample Problem: What is the Probability that it rains exactly 2 out of 3 days?

21 Sample Problem: What is the Probability that it rains on day 3?

22 More Examples of Probability and Decision Trees Suppose an Urn is filled with 3 green marbles, 2 blue marbles, and 1 red marble A marble is selected If it is red, it is not put back in the urn If it is blue, it is put back in the urn along with 3 addi<onal blue marbles If it is green, it is put back in the urn along with 5 addi<onal blue marbles A second marble is selected What is the probability that both marbles are green?

23 Workspace

24 Workspace

25 Intro to Bayes Rule Bayes rule can be easily derived from the product rule P (A B) =P (A)P (B A) P (A B) =P (B A) =P (B)P (A B) P (A B) = P (A)P (B A) P (B)

26 More Condi<onal Probability Problems Suppose there are two type of animals in the world cats and dogs P(dog) =.7 P(cat) =.3 P(animal weighs over 40 pounds cat) =.001 P(animal weighs over 40 pounds dog) =.5 P(animal has a tail over 6 inches cat) =.99 P(animal has a tail over 6 inches dog) =.8 Suppose the tail length and weight are condi<onally independent given the iden<ty of the animal Compute P(dog animal weighs over 40 pounds and has a tail over 6 inches)

27 Workspace

28 Example from Machine Learning (not officially part of the class) Consider the problem of dis<nguishing smiles from non smiles

29 What is an image? An image is a collec<on of pixels. Each number specifies the brightness of the pixel at a par<cular loca<on E.g

30 Goal Our goal will be predict whether a person is smiling or not given an image of their face P(smile pixels) By Bayes rule: P (smile pixels) = P (pixels smile)p (smile) P (pixels) Our method of predic<ng will be to always predict the most likely category. So will predict a smile if P (smile pixels) P (nosmile pixels) > 1

31 Predic<on Rule A^er Applying Bayes rule we predict smile when: P (smile pixels)p (smile) P (pixels) P (nosmile pixels)p (nosmile) P (pixels) = P (smile pixels)p (smile) P (nosmile pixels)p (nosmile) > 1

32 How do we apply this rule We need to build a model of the pixels given each of the categories Without going into too much detail about how this is done, the simplest method is to create a Naïve Bayes classifier. Here we assume that the probability of the pixel brightness at each any two loca<ons are condi<onally independent given the category (smile / not smile)

33 An Example of Probability of a Pixel

34 An Example of Probability of a Pixel

35 Now I can visualize how good each pixel is at predic<ng the expression