Probability theory review

Transcription

1 130A Section February 21, 2018 Probability theory review 1 Probability distributions This is a relatively informal introduction to probability, for the purposes of this class. For a more formal introduction, I recommend A probability distribution is a set (often called the support of the distribution), along with a probability (a real number) for each element of the set, which fulfill the following requirements: All probabilities must be in the interval [0,1]. All the probabilities must together sum to one. For instance, we could express the distribution connected to a (fair) coin flip as follows: P(Coin) = {Heads: 0.5, Tails: 0.5}. Then P(Coin=Heads) = 0.5, P(Coin=Tails) = 0.5, P(Coin=Tails or Coin=Heads) = 1.0, P(Coin=Tails and Coin=Heads) = 0.0. Which of the following is a distribution: {A: 0.7, B: 0.3} {A: 0.6, 0.5} {A: 1.1, B: -0.1} Confusing Terminology People write P(A) to mean a distribution, but P(a) for a A to mean the probability value assigned to a in a distribution. 2 Joint and conditional probabilities If P(D 1 ) is a distribution and P(D 2 ) is a distribution, the joint distribution P(D 1, D 2 ) is a distribution whose support is the Cartesian product of the support of D 1 and the support of D 2. For example: P(Coin 1 ) = {Heads 1 : 0.5, Tails 1 : 0.5} 1

2 P(Coin 2 ) = {Heads 2 : 0.7, Tails 2 : 0.3} P(Coin 1, Coin 2 ) = {<Heads 1,Heads 2 >: 0.35, <Heads 1,Tails 2 >: 0.15, <Tails 1,Heads 2 >: 0.35, <Tails 1,Tails 2 >: 0.15} In this case, I m assuming that the outcome of flipping Coin 1 has no effect on the outcome of flipping Coin 2. In the language of probability, Coin 1 and Coin 2 are independent. This means that P(Coin 1, Coin 2 ) = P(Coin 1 ) P(Coin 2 ). So for instance P(<Coin 1,Coin 2 >=<Heads 1,Heads 2 >) = P(Coin 1 =Heads 1 ) P(Coin 2 =Heads 2 ). A conditional probability distribution P (A B) is a distribution over a set A given some element b B. For instance, let support(a)={heads,tails}, support(b)={q, Q}, for Q= The coin flipper likes to cheat and wants the coin to come up heads.. We could then imagine a conditional distribution P (A B) as follows, which expresses the fact that if Q is true, the coin is more likely to come up heads than tails. Note that each row of this table is a distribution over A: Heads Tails Q Q Note that if we have a joint probability distribution P (A, B), then we can also compute the conditional probability using the following formula: P (B A) = P (A,B) P (A) 3 Bayes rule In many cases we only know the probability distribution P (A B) but still want to determine the probability of P (B A). This can be done using Bayes rule: P (B A) = P (A B) P (B) P (A) For example, a patient might have a disease D with probability P (D) = 0.1 (therefore the probability of not having the disease P ( D) = = 0.9. A doctor uses a blood sample to test for D. The procedure is 95% accurate, i.e., if the patient has the disease, the test will be positive in 95% of the cases, so P (test positive D) =

3 Similarly, P (test negative notd) = Because probabilities always sum up to 1, P (test negative D) = 0.05 and P (test positive D) = What is the probability that a patient has the disease given that the test is negative? P (D test negative) = P (D test negative) = P (test negative D) P (D) P (test negative) P (test negative) 4 RSA example Consider a reference game where a speaker wants to communicate to a listener which one of the following three shapes (R = {r 1, r 2, r 3 }) she wants to refer to. The speaker may use one of the following messages: M = {circle, square, black, white} Question: If we assume that a listener is equally likely to choose one of the three shapes before hearing the utterance, what is the probability of choosing each of the shapes (i.e, the prior probabilities over shapes)? P (r 1 ) = 1 3 P (r 2 ) = 1 3 P (r 3 ) = 1 3 Question: If we assume that a speaker is a priori equally likely to choose one of the four messages, what is the probability of choosing each of the messages (i.e, the prior probabilities over messages)? 3

4 P (circle) = 1 4 P (square) = 1 4 P (black) = 1 4 P (white) = 1 4 Let s assume we computed our speaker probabilities using the RSA model and we ended up having the following table of conditional probabilities P S (message referent): black white circle square r r r Question: Compute the following pragmatic listener probabilities (assuming that α = 1 and the cost for all utterances is 0): P L (r 1 black) = P L (r 2 black) = P L (r 3 black) = P L (r 1 circle) = P L (r 2 square) = P S (black r 1 ) P (r 1 ) r R P S (black r ) P (r ) = = 0.4 P S (black r 2 ) P (r 2 ) r R P S (black r ) P (r ) = = 0.6 P S (black r 3 ) P (r 3 ) r R P S (black r ) P (r ) = = 0 P S (circle r 1 ) P (r 1 ) r R P S (circle r ) P (r ) = = 1 P S (square r 2 ) P (r 2 ) r R P S (square r ) P (r ) = = 0.6 4

5 Alternatively, instead of computing the individual probabilities, we could have also computed all these probabilities by transposing the matrix for P S and then re-normalizing it: Transposed matrix: r 1 r 2 r 3 black white circle square Re-normalized matrix P L : r 1 r 2 r 3 black white circle square