Probability Calculus. p.1

Transcription

1 Probability Calculus p.1

2 Joint probability distribution A state of belief : assign a degree of belief or probability to each world 0 Pr ω 1 ω Ω sentence > event world > elementary event Pr ω ω α 1 Pr ω p.2

3 Joint probability distribution world Earthquake Burglary Alarm Pr ω 1 true true true.019 ω 2 true true false.001 ω 3 true false true.056 ω 4 true false false.024 ω 5 false true true.162 ω 6 false true false.018 ω 7 false false true.0072 ω 8 false false false.7128 p.3

4 Properties of Beliefs 0 1 for any sentence α 0 when α is inconsistent 1 when α is valid mutually exclusive 1 Pr Pr when α and are p.4

5 Updating Beliefs We need to update our beliefs given new piece of information Evidence: we observed some event (sentence) Update the state of belief Pr denote by Pr, called conditioning on into a new state of belief Every world that contradicts with gets 0 belief The beliefs in other worlds are normalized so that they sum to one Pr ω 0 Pr ω Pr if ω if ω p.5

6 Independence Bayes conditioning, Conditional probability Pr Two events α and are independent iff Equivalently Pr α Pr or Pr Pr Earthquake Pr Earthquake Burglary 1 p.6

7 Conditional Independence Independent events may become dependent given new evidence, and dependent events may become independent given new evidence. Pr Burglary Alarm 741 Pr Burglary Alarm Earthquake 253 Two events α and are conditionally independent given event γ iff γ γ Equivalently Pr α γ Pr γ or γ γ Pr γ p.7

8 Notation The domain/space D X of X: a random variable X can take a set of values D X Denote variables by uppercase letters (A) and their values by lowercase letters (a) Pr a Pr A Pr a b Pr A a B a b Pr A a B b p.8

9 Variable Independence Let X, Y and Z be three disjoint sets of variables. X is independent of Y given Z, denoted I Pr X Z Y, iff Pr x y z Pr x z x D X y D Y z D Z Marginal independence I Pr X / 0 Y p.9

10 Properties of probability Chain rule 1 αn 1 α 2 αn 2 α 3 αn n Law of total probability or Case Analysis n i 1 i where the events 1 n i 1 i Pr i n are mutually exclusive and exhaustive p.10

11 Properties of probability Special cases Pr Pr Marginalization Pr x y D Y Pr x y p.11

12 Bayes Rule/Theorem Pr α Pr A patient was just tested for a particular disease and the test came out positive. We know that one in every thousand people has this disease. We also know that the test is not reliable: it has a false positive rate of 2% and a false negative rate of 5%. Assess our belief in the patient having the disease given that the test came out positive Pr D 001 Pr T D 02 Pr T D 05 Pr D T Pr T Pr T D Pr D D Pr D Pr T D Pr D 4 5% p.12

13 Soft Evidence Soft evidence is not conclusive: we may get an unreliable testimony that event occurred, but not to the point where we would consider it certain. One method for specifying a soft evidence on event is by stating the new belief in after the evidence has been accommodated All things considered method given this soft evidence on, my belief in becomes.85 constraints on the new state of belief Pr : Pr q, Pr 1 q p.13

14 Jeffrey s Rule If we insist on preserving the relative beliefs in worlds that satisfy ( ) Pr ω q Pr 1 q Pr Pr ω Pr ω if ω if ω which is equivalent to Pr ω Pr ω Pr ω Pr ω Jeffrey s Rule q 1 q p.14

15 Jeffrey s Rule Jeffrey s Rule in the case where the evidence concerns a set of mutually exclusive and exhaustive event 1 n n i 1 q i i p.15

16 Nothing else considered method Define the odds of event as follows O Pr Pr We can specify soft evidence on event by declaring the relative change it induces on the odds of, that is, by specifying the ratio O O, known as the Bayes factor A Bayes factor of 1 indicates a neutral evidence, while a Bayes factor of 2 indicates an evidence on which is strong enough to double the odds of p.16

17 Nothing else considered method Suppose now that we obtain soft evidence on whose strength is given by a Bayes factor of k Translate this evidence into a constraint on Pr Pr kpr kpr Pr Using Jeffrey s Rule k kpr Pr p.17

18 Nothing else considered method An example due to Pearl concerns the alarm of Mr. Holmes house and the potential of a burglary. One day, Mr. Holmes receives a call from his neighbor, Mrs. Gibbons, saying that she may have heard the alarm of his house going off. Since Mrs. Gibbons suffers from a hearing problem, Mr. Holmes concludes that Mrs. Gibbons testimony increases the odds of the alarm going off by a factor of 4: O Alarm O Alarm 4. p.18

19 Nothing else considered method The case where the evidence concerns a set of mutually exclusive and exhaustive event 1 Define the odds of event i to event j n O i j Pr i Pr j A soft evidence bearing on a set of mutually exclusive and exhaustive events 1 n can then be specified using a set of numbers λ 1 λn with the following interpretation O i j O i j λ i λ j p.19

20 Nothing else considered method Each ratio λ i λ j is known as the Bayes factor for events i, j. Translate this evidence into a constraint on Pr using Jeffrey s Rule and n i n i 1 λ i 1 λ ipr i i p.20

21 Virtual Evidence Suppose that we have a soft evidence bearing on a set of mutually exclusive and exhaustive events 1 n We can model this evidence explicity by augmenting our language with a new propositional variable V, which represents the event of receiving this soft evidence We quantify the strength of this evidence by specifying the probability that we will receive it given each of the events Pr V i λ i p.21

22 Virtual Evidence The new state of belief Pr, after the soft evidence has been accommodated, is now given by Pr V Model the soft evidence in terms of hard evidence on a new virtual variable V > the ratios λ i λ j can be interpreted as bayes factors The method of virtual evidence is quite important practically, as it allows us to integrate soft evidence using the tools developed for hard evidence p.22