On the Revision of Probabilistic Beliefs using Uncertain Evidence

Transcription

1 On the Revson of Probablstc Belefs usng Uncertan Evdence He Chan and Adnan Darwche Computer Scence Department Unversty of Calforna, Los Angeles Los Angeles, CA Abstract We revst the problem of revsng probablstc belefs usng uncertan evdence, and report results on four major ssues relatng to ths problem: How to specfy uncertan evdence? How to revse a dstrbuton? Should, and do, terated belef revsons commute? And how to provde guarantees on the amount of belef change nduced by a revson? Our dscusson s focused on two man methods for probablstc revson: Jeffrey s rule of probablty knematcs and Pearl s method of vrtual evdence, where we analyze and unfy these methods from the perspectve of the questons posed above. 1 Introducton We consder n ths paper the problem of revsng belefs gven uncertan evdence, where belefs are represented usng a probablty dstrbuton. There are two man methods for revsng probablstc belefs n ths case. The frst method s known as Jeffrey s rule and s based on the prncple of probablty knematcs, whch can be vewed as a prncple for mnmzng belef change [Jeffrey, 1965]. The second method s called vrtual evdence and s proposed by Pearl n the context of belef networks even though t can be easly generalzed to arbtrary probablty dstrbutons and s based on reducng uncertan evdence nto certan evdence on some vrtual event [Pearl, 1988]. We analyze both of these methods n ths paper wth respect to the followng four questons: 1. How should one specfy uncertan evdence? 2. How should one revse a probablty dstrbuton? 3. Should, and do, terated belef revsons commute? 4. What guarantees can be offered on the amount of belef change nduced by a partcular revson? Our man fndngs can be summarzed as follows. Frst, we show that Jeffrey s rule and Pearl s method both revse belefs usng the prncple of probablty knematcs; Jeffrey s rule explctly commts to ths prncple, whle Pearl s method s based on a dfferent prncple, yet we show that Pearl s method mples the prncple of probablty knematcs, leadng to the same revson method as that of Jeffrey s. The dfference between Jeffrey s rule and Pearl s method s n the way uncertan evdence s specfed. Jeffrey requres uncertan evdence to be specfed n terms of the effect t has on belefs once accepted, whch s a functon of both evdence strength and belefs held before the evdence s obtaned. Pearl, on the other hand, requres uncertan evdence to be specfed n terms of ts strength only. Despte ths dfference, we show that one can easly translate between the two methods of specfyng evdence and provde the method for carryng out ths translaton. The multplcty of methods for specfyng evdence also rases an mportant queston: How should nformal statements about evdence be captured formally usng avalable methods? For example, what should the followng statement translate to: Seeng these clouds, I beleve there s an 80% chance that t wll ran? We wll dscuss ths problem of nterpretng nformal evdental statements n a separate secton. As to the queston of terated belef revson: It s well known that Jeffrey s rule does not commute; hence, the order n whch evdence s ncorporated matters [Dacons & Zabell, 1982]. Ths has long been perceved as a problem, untl clarfed recently by the work of Wagner who observed that Jeffrey s method of specfyng evdence s dependent on what s beleved before the evdence s obtaned and, hence, should not be commutatve to start wth [Wagner, 2002]. Wagner proposed a method for specfyng evdence, based on the noton of Bayes factor, and argued that ths method specfes only the strength of evdence, and s ndependent of the belefs held when attanng evdence. Wagner argued that when evdence s specfed n that partcular way, terated revsons should commute. He even showed that combnng ths method for specfyng evdence wth the prncple of probablty knematcs leads to a revson rule that commutes. We actually show that Pearl s method of vrtual evdence s specfyng evdence accordng to Bayes factor, exactly as proposed by Wagner and, hence, corresponds exactly to the proposal he calls for. Therefore, the results we dscuss n ths paper unfy the two man methods for probablstc belef revson proposed by Jeffrey and Pearl, and show that dfferences between them amount to a dfference n the protocol for specfyng uncertan evdence. Our last set of results relate to the problem of provdng guarantees on the amount of belef change nduced by a revson. We have recently proposed a dstance measure for

2 boundng belef changes, and showed how one can use t to provde such guarantees [Chan & Darwche, 2002]. We show n ths paper how ths dstance measure can be computed when one dstrbuton s obtaned from another usng the prncple of probablty knematcs. We then show how the guarantees provded by ths measure can be realzed when applyng ether Jeffrey s rule or Pearl s method, snce they both are performng revson based on the prncple of probablty knematcs. 2 Probablty Knematcs and Jeffrey s Rule Suppose that we have two probablty dstrbutons Pr and Pr whch dsagree on the probabltes they assgn to a set of mutually exclusve and exhaustve events γ 1,..., γ n, yet: Pr(α γ ) Pr (α γ ), (1) for 1,..., n, and for every event α n the probablty space. We say here that Pr s obtaned from Pr by probablty knematcs on γ 1,..., γ n. Ths concept was proposed by Jeffrey [1965] to capture the noton that even though Pr and Pr dsagree on the probabltes of events γ, they agree on ther relevance to every other event α. Consder now the problem of revsng a probablty dstrbuton Pr gven uncertan evdence relatng to a set of mutually exclusve and exhaustve events γ 1,..., γ n. One method of specfyng uncertan evdence s through the effect that t would have on belefs once accepted. That s, we can say that the evdence s such that the probablty of γ becomes once the evdence s accepted. If we adopt ths method of evdence specfcaton, we conclude that there s only one dstrbuton Pr such that: Pr (γ ) for 1,..., n. Pr s obtaned from Pr by probablty knematcs on γ 1,..., γ n. Moreover, ths specfc dstrbuton s gven by: Pr (ω) def Pr(ω), f ω γ, (2) where ω s an atomc event, also known as a world, and s the logcal entalment relatonshp. Ths s exactly the dstrbuton that Jeffrey suggests and, hence, ths method of revson s known as Jeffrey s rule. We stress here that we are drawng a dstncton between the prncple of probablty knematcs and Jeffrey s rule, whch are often consdered synonymous. Specfcally, Jeffrey s rule arses from a combnaton of two proposals: (1) the prncple of probablty knematcs, and (2) the specfcaton of uncertan evdence usng a posteror dstrbuton. It s possble for one to combne the prncple of probablty knematcs wth other methods for specfyng evdence as we dscuss later. It s not hard to show that the above dstrbuton Pr s ndeed obtaned from Pr by probablty knematcs on γ 1,..., γ n, as t satsfes Equaton 1: Pr (α γ ) Pr (α, γ ) Pr (γ ) ω α,γ Pr (ω) q ω α,γ Pr(ω) ω α,γ Pr(ω) Pr(α, γ ) Pr(α γ ). Therefore, for any event α, ts probablty under the new dstrbuton Pr s: Pr (α) Pr (α γ )Pr (γ ) Pr(α γ ) Pr(α, γ ), whch s the closed form for Jeffrey s rule. We now show an example of usng Jeffrey s rule. Example 1 (Due to Jeffrey) Assume that we are gven a pece of cloth, where ts color can be one of: green (c g ), blue (c b ), or volet (c v ). We want to know whether, n the next day, the cloth wll be sold (s), or not sold (s). Our orgnal state of belef s gven by the dstrbuton Pr: Pr(s, c g ).12, Pr(s, c b ).12, Pr(s, c v ).32, Pr(s, c g ).18, Pr(s, c b ).18, Pr(s, c v ).08. Therefore, our orgnal state of belef on the color of the cloth (c g, c b, c v ) s gven by the dstrbuton (.3,.3,.4). Assume that we now nspect the cloth by candlelght, and we want to revse our state of belef on the color of the cloth to the new dstrbuton (.7,.25,.05) usng Jeffrey s rule. If we apply Jeffrey s rule (Equaton 2), we get the new dstrbuton Pr : Pr (s, c g ).28, Pr (s, c b ).10, Pr (s, c v ).04, Pr (s, c g ).42, Pr (s, c b ).15, Pr (s, c v ) Vrtual Evdence and Pearl s Method The problem of revsng a probablty dstrbuton under uncertan evdence can be approached from a dfferent perspectve than that of probablty knematcs. Specfcally, when we have uncertan evdence about some mutually exclusve and exhaustve events γ 1,..., γ n, we can nterpret that evdence as hard evdence on some vrtual event η, where the relevance of γ 1,..., γ n to η s uncertan. It s assumed that the vrtual event η depends only on the events γ 1,..., γ n and, therefore, s ndependent of any other event α gven γ : Pr(η γ, α) Pr(η γ ). (3) Accordng to ths approach, the uncertanty regardng evdence on γ 1,..., γ n s recast as uncertanty n the relevance of γ 1,..., γ n to the vrtual event η. Specfcally, the uncertanty s recast as the lkelhood of γ gven vrtual evdence η: Pr(η γ ), for 1,..., n.

3 We next show that the new dstrbuton obtaned after acceptng the uncertan evdence on γ 1,..., γ n, Pr( η), s: Pr(ω η) Pr(ω) j Pr(γ, f ω γ, (4) where λ 1,..., λ n are ratos chosen such that: λ 1 :... : λ n Pr(η γ 1 ) :... : Pr(η γ n ). Hence, the specfc lkelhoods Pr(η γ ) are not mportant here, but ther ratos are. Ths s why ths method usually specfes uncertan evdence usng a set of lkelhood ratos λ 1,..., λ n [Pearl, 1988]. The dervaton of Equaton 4 s based on the assumptons underlyng the method of vrtual evdence gven by Equaton 3: Pr(ω η) Pr(ω, η) Pr(η) Pr(η ω)pr(ω) j Pr(η γ j)pr(γ j ) Pr(η γ, ω)pr(ω) j Pr(η γ j)pr(γ j ) Pr(η γ )Pr(ω) j Pr(η γ j)pr(γ j ) Pr(ω) j Pr(γ, f ω γ. The last step s based on Pr(η γ ) k, where k s a constant. For any event α, the new probablty after accommodatng the vrtual evdence s: Pr(α η) Pr(α, γ η) Pr(α, γ ) j λ jpr(γ j ) Pr(α, γ ) j λ jpr(γ j ), whch s the closed form for Pearl s method. The above revson method s a generalzaton of the method of vrtual evdence proposed by Pearl [1988] n the context of belef networks. A belef network s a graphcal probablstc model, composed of two parts: a drected acyclc graph where nodes represent varables, and a set of condtonal probablty tables (CPTs), one for each varable [Pearl, 1988; Jensen, 2001]. The CPT for varable X wth parents U defnes a set of condtonal probabltes of the form Pr(x u), where x s a value of varable X, and u s an nstantaton of parents U. Suppose now that we have some vrtual evdence bearng on varable Y, whch has values y 1,..., y n. Ths vrtual evdence s represented n the belef network by addng a dummy node Z and a drected edge Y Z, where one value of Z, say z, corresponds to the vrtual event η. Ths ensures the assumpton of Equaton 3, that vrtual event z s ndependent of every other event α gven every y,.e., Pr(z y, α) Pr(z y ), whch follows from the ndependence semantcs of belef networks [Pearl, 1988]. The uncertanty of evdence s quantfed by the lkelhood ratos: Pr(z y 1 ) :... : Pr(z y n ) λ 1 :... : λ n, whch are specfed n the CPT of varable Z. Fnally, we ncorporate the presence of the vrtual event z by addng the observaton Z z to the rest of evdence n the belef network. We now show a smple example. Example 2 (Due to Pearl) We are gven a belef network wth two varables: A represents whether the alarm of Mr. Holmes house goes off, and B represents whether there s a burglary. To represent the nfluence between the two varables, there s a drected edge B A. The CPTs of A and B are gven by: Pr(a b).95, meanng the alarm goes off f there s a burglary wth probablty.95; Pr(a b).01, meanng the alarm goes off f there s no burglary wth probablty.01; and Pr(b) 10 4, meanng on any gven day, there s a burglary on any gven house wth probablty One day, Mr. Holmes receves a call from hs neghbor, Mrs. Gbbons, sayng she may have heard the alarm of hs house gong off. Snce Mrs. Gbbons suffers from a hearng problem, Mr. Holmes concludes that there s an 80% chance that Mrs. Gbbons dd hear the alarm gong off. Accordng to the method of vrtual evdence, ths uncertan evdence can be specfed by the vrtual event η and the lkelhood rato: Pr(η a) : Pr(η a) 4 : 1. To ncorporate the vrtual evdence nto the belef network, we add the varable Z, and the drected edge A Z, and specfy the CPT of Z such that Pr(z a) : Pr(z a) 4 : 1. For example, we can assgn Pr(z a).4 and Pr(z a).1. After ncorporatng the vrtual evdence by addng the observaton Z z to the evdence, we can easly compute the answers to queres n the belef network. For example, the probablty that there s a burglary at Mr. Holmes house s now Pr(b z) Comparng the Revson Methods From the llustratons of the two belef revson methods, Jeffrey s rule and Pearl s method of vrtual evdence, we can see that a belef revson method can be broken nto two parts: a formal method of specfyng uncertan evdence, and a prncple of belef revson that commts to a unque dstrbuton among many whch satsfy the uncertan evdence. 4.1 Pearl s method and Probablty Knematcs We now show that Pearl s method, lke Jeffrey s rule, also obeys the prncple of probablty knematcs; what they dffer n s how uncertan evdence s specfed. Suppose that a probablty dstrbuton Pr was revsed usng the method of vrtual evdence, wth lkelhood ratos λ 1,..., λ n bearng on events γ 1,..., γ n, obtanng the new dstrbuton Pr( η). We can easly see that the revson satsfes the prncple of probablty knematcs,.e. Equaton 1: Pr(α γ, η) Pr(α, γ η) Pr(γ η) Pr(α, γ ) j Pr(γj)λj j Pr(γj)λj

4 Pr(α, γ ) Pr(α γ ). Therefore, both Jeffrey s rule and Pearl s method uses the prncple of probablty knematcs for belef revson. 4.2 From Pearl s Method to Jeffrey s Rule Wth the prevous result, we now show how we can easly translate between the two methods of specfyng uncertan evdence. For example, to translate from Pearl s method to Jeffrey s rule, we note that the new probabltes of events γ 1,..., γ n after applyng vrtual evdence, wth lkelhood ratos λ 1,..., λ n, are gven by: Pr(γ η) ω γ Pr(ω η) ω γ Pr(ω) j Pr(γ j Pr(γ. Suppose nstead that we want to revse the dstrbuton Pr usng Jeffrey s rule, assumng that after acceptng uncertan evdence, the probablty of γ becomes: j Pr(γ. Substtutng the above probablty n Jeffrey s rule (Equaton 2), we get: Pr (ω) Pr(ω) j Pr(γj)λj Pr(ω) j Pr(γ, f ω γ, whch s exactly the dstrbuton obtaned by the method of vrtual evdence (Equaton 4). We now llustrate ths translaton by revstng Example 2. Example 3 In Example 2, we appled the method of vrtual evdence on Pr, by specfyng the vrtual evdence: Pr(η a) : Pr(η a) λ a : λ a 4 : 1. The orgnal probabltes of a and a are gven by: Pr(a) , and Pr(a) After applyng the vrtual evdence, the new probabltes of a and a are gven by: Pr(a η) Pr(a) ; Pr(a)λ a + Pr(a)λ a Pr(a η) Pr(a) Pr(a)λ a + Pr(a)λ a Alternatvely, we can apply Jeffrey s rule to obtan the new dstrbuton Pr such that Pr (a) Pr(a η) and Pr (a) Pr(a η) , and Pr wll be the same to the dstrbuton Pr( η) obtaned by vrtual evdence. λ a λ a 4.3 From Jeffrey s Rule to Pearl s Method We can also easly translate from Jeffrey s rule to Pearl s method. The new probabltes of events γ 1,..., γ n after applyng Jeffrey s rule are gven by: Pr (γ ). Suppose nstead that we want to revse the dstrbuton Pr usng Pearl s method. We can do ths by applyng the method of vrtual evdence, wth lkelhood ratos λ 1,..., λ n, such that: λ 1 :... : λ n q 1 Pr(γ 1 ) :... : q n Pr(γ n ). Therefore, k /, where k s a constant. Substtutng n Pearl s method (Equaton 4), we get: Pr(ω η) Pr(ω) k Pr(γ ) j Pr(γ j) kq j Pr(γ j) Pr(ω), f ω γ, whch s exactly the dstrbuton obtaned by Jeffrey s rule (Equaton 2). We now llustrate ths translaton by revstng Example 1. Example 4 In Example 1, we appled Jeffrey s rule on the orgnal dstrbuton Pr to obtan new dstrbuton Pr. We can also do ths by applyng the method of vrtual evdence. We can nterpret the nspecton of the cloth by candlelght as vrtual evdence η, where Pr(η c g ) : Pr(η c b ) : Pr(η c v ) λ g : λ b : λ v 7 : 2.5 : The probablty dstrbuton of the color of the cloth after applyng vrtual evdence s therefore gven by: Pr(c g η) Pr(c b η) Pr(c v η) Pr(c g )λ g.7; Pr(c g )λ g + Pr(c b )λ b + Pr(c v )λ v Pr(c b )λ b.25; Pr(c g )λ g + Pr(c b )λ b + Pr(c v )λ v Pr(c v )λ v.05. Pr(c g )λ g + Pr(c b )λ b + Pr(c v )λ v We can easly verfy that ths probablty dstrbuton Pr( η) obtaned by vrtual evdence s the same as the dstrbuton Pr obtaned by Jeffrey s rule. 5 Interpretng Evdental Statements The evdence specfcaton protocols adopted by Jeffrey s rule and Pearl s method have been dscussed by Pearl [2001], n relaton to the problem of formally nterpretng evdental statements. Consder the followng statement as an example: Lookng at ths evdence, I am wllng to bet 2:1 that Davd s not the kller. Ths statement can be formally nterpreted usng ether protocol. For example, f α denotes the event Davd s not the kller, ths statement can be nterpreted n two ways: 1 The λ s are chosen such that kpr (c )/Pr(c ), where k s a constant.

5 1. After acceptng the evdence, the probablty that Davd s not the kller becomes twce the probablty that Davd s the kller: Pr (α) 2/3 and Pr (α) 1/3. 2. The probablty that I wll see ths evdence η gven that Davd s not the kller s twce the probablty that I wll see t gven that Davd s the kller: Pr(η α) : Pr(η α) 2 : 1. The frst nterpretaton translates drectly nto a formal pece of evdence, Jeffrey s style, and can be characterzed as an All thngs consdered nterpretaton snce t s a statement about the agent s fnal belefs, whch are a functon of both hs pror belefs and the evdence [Pearl, 2001]. On the other hand, the second nterpretaton translates drectly nto a formal pece of evdence, Pearl s style, and can be characterzed as a Nothng else consdered nterpretaton snce t s a statement about the evdence only [Pearl, 2001]. The two nterpretatons can lead to contradctory conclusons about the evdence. For example, f we use the Nothng else consdered approach to nterpret our statement, we wll conclude that the evdence s aganst Davd beng the kller. However, f we use the All thngs consdered nterpretaton, t s not clear whether the evdence s for or aganst, unless we know the orgnal probablty that Davd s the kller. If, for example, Davd s one of four suspects who are equally lkely to be the kller, then orgnally we have: Pr(α) 3/4. Therefore, ths evdence has actually ncreased the probablty that Davd s the kller! Because of ths, Pearl argues for the Nothng else consdered nterpretaton, as t provdes a summary of the evdence and the evdence alone, and dscusses how people tend to use bettng odds to quantfy ther belefs even when they are based on the evdence only [Pearl, 2001]. Example 2 provdes another opportunty to llustrate the subtlety nvolved n nterpretng evdental statements. The evdental statement n ths case s Mr. Holmes concludes that there s an 80% chance that Mrs. Gbbons dd hear the alarm gong off. Interpretng ths statement usng the All thngs consdered approach gves us the concluson that Pr (a) : Pr (a) 4 : 1, where a denotes the event that the alarm has gone off. Ths nterpretaton assumes that the 4 : 1 rato apples to the posteror belef n a, after Mr. Holmes has accommodated the evdence provded by Mrs. Gbson. However, n Example 2, ths statement was gven a Nothng else consdered nterpretaton, as by Pearl [1988, Page 44-47], where the 4 : 1 rato s taken as a quantfcaton of the evdence strength. That s, the statement s nterpreted as Pr(η a) : Pr(η a) 4 : 1, where η stands for the evdence. In fact, the two nterpretatons wll lead to two dfferent probablty dstrbutons and, hence, gve us dfferent answers to further probablstc queres. For example, f we use the All thngs consdered approach n nterpretng ths evdental statement, the probablty of havng a burglary wll be Pr (b) , whch s much larger than the probablty we get usng the Nothng else consdered approach n Example 2, whch s From the dscussons above, the formal nterpretaton of evdental statements appears to be a non trval task, whch can be senstve to context and communcaton protocols. Regardless of how ths s accomplshed though, we need to stress that the process of mappng an nformal evdental statement nto a revsed probablty dstrbuton nvolves three dstnct elements: 1. One must adopt a formal method for specfyng evdence, such as Jeffrey s rule or Pearl s method. 2. One must nterpret the nformal evdental statement, by translatng t nto a formal pece of evdence, usng ether the All thngs consdered or Nothng else consdered nterpretaton. 3. One must apply a revson, by mappng the orgnal probablty dstrbuton and formal pece of evdence nto a new dstrbuton. Our man pont here s that Jeffrey s rule and Pearl s method employs the same belef revson prncple,.e. probablty knematcs. Moreover, although they adopt dfferent, formal methods for specfyng evdence, one can translate between the two methods of specfcaton. Fnally, one may take the vew that Jeffrey s rule and Pearl s method consttute commtments to how nformal evdental statements should be nterpreted, makng the frst and second elements the same, but we do not take ths vew here. Instead, we regard the ssue of nterpretaton as a thrd, orthogonal dmenson whch s best addressed ndependently. 6 Commutatvty of Iterated Revsons We now dscuss the problem of commutatvty of terated revsons, that s, whether the order n whch we ncorporate uncertan evdence matters. 2 It s well known that terated revsons by Jeffrey s rule are not commutatve [Dacons & Zabell, 1982]. As a smple example, assume that we are gven a pece of uncertan evdence whch suggests that the probablty of event α s.7, followed by another pece of uncertan evdence whch suggests that the probablty of α s.8. After ncorporatng both peces n that order, we wll beleve that the probablty of α s.8. However, f the opposte order of revson s employed, we wll beleve that ths probablty s.7. In general, even f we are gven peces of uncertan evdence on dfferent events, terated revsons by Jeffrey s rule are not commutatve. Ths was vewed as a problematc aspect of Jeffrey s rule for a long tme, untl clarfed recently by Wagner [2002]. Frst, Wagner observed and stressed that the evdence specfcaton method adopted by Jeffrey s sutable for the All thngs consdered nterpretaton of evdental statements. Moreover, he argued convncngly that when evdental statements carry ths nterpretaton, they must not be commutatve to start wth. So the lack of commutatvty s not a problem of the revson method, but a property of the method used to specfy evdence. In fact, Wagner suggested a thrd method for specfyng evdence based on Bayes factors [Good, 1950; 2 There s a key dstncton between terated revsons usng certan evdence versus uncertan evdence. In the former case, peces of evdence may be logcally nconsstent, whch adds another dmenson of complexty to the problem [Darwche & Pearl, 1997], leadng to dfferent propertes and treatments.

6 1983; Jeffrey, 1992], whch leads to commutatvty. Specfcally, f Pr and Pr are two probablty dstrbutons, and γ and γ j are two events, the Bayes factor (or odds factor), F Pr,Pr (γ : γ j ), s defned as the rato of new-to-old odds: F Pr,Pr (γ : γ j ) def Pr (γ )/Pr (γ j ) /Pr(γ j ). Gven ths noton, one can specfy uncertan evdence on a set of mutually exclusve and exhaustve events γ 1,..., γ n by specfyng the Bayes factor for every par of events γ and γ j. One nterestng property of ths method of specfcaton s that Bayes factors do not constran the probablty dstrbuton Pr,.e., any evdence specfed n ths way s compatble wth every dstrbuton Pr. 3 Hence, they are sutable for a Nothng else consdered nterpretaton of evdental statements. Interestngly enough, Wagner [2002] showed that when evdence s specfed usng Bayes factors and revsons are accomplshed by probablty knematcs, belef revson becomes commutatve. 4 Ths was an llustraton that Jeffrey s rule s ndeed composed of two ndependent elements: an evdence specfcaton method, and a revson method. In fact, we now show that the evdence specfcaton method adopted by Pearl s method corresponds to the method of specfyng evdence by Bayes factors. Ths has a number of mplcatons. Frst, t shows that revsons by the vrtual evdence method are commutatve. Second, t provdes an alternate, more classcal, semantcs for the vrtual evdence method. Fnally, t shows agan that Jeffrey s rule and Pearl s method both revse dstrbutons usng probablty knematcs. Specfcally, suppose that we revse a probablty dstrbuton Pr usng the method of vrtual evdence, wth lkelhood ratos λ 1,..., λ n bearng on events γ 1,..., γ n, and suppose that we get dstrbuton Pr as a result. We then have: F Pr,Pr (γ : γ j ) Pr (γ )/Pr (γ j ) /Pr(γ j ) λ j. Pr(γ )/( k Pr(γ k)λ k ) Pr(γ /( k Pr(γ k)λ k ) Pr(γ j) That s, f we specfy evdence by Bayes factor /λ j for every par of events γ and γ j, and then revse dstrbuton Pr usng probablty knematcs, we obtan the same dstrbuton that arses from applyng the vrtual evdence method. 3 Ths s not true f we use ratos of probabltes nstead of ratos of odds. For example, f Pr (α) 2Pr(α), we must have Pr(α).5 because Pr (α) 1 [Wagner, 2002]. 4 Wagner shows not only that the representaton of uncertan evdence usng Bayes factors s suffcent for commutatvty, but n a large number of cases, necessary. 7 Boundng Belef Change Induced by Probablty Knematcs One mportant queston relatng to belef revson s that of measurng the extent to whch a revson dsturbs exstng belefs. We have recently proposed a dstance measure defned between two probablty dstrbutons whch can be used to bound the amount of belef change nduced by a revson [Chan & Darwche, 2002]. We revew ths measure next and then use t to provde some guarantees on any revson whch s based on probablty knematcs. 5 Defnton 1 [Chan & Darwche, 2002] Let Pr and Pr be two probablty dstrbutons over the same set of worlds ω. We defne a measure D(Pr, Pr ) as follows: D(Pr, Pr ) def ln max ω where 0/0 s defned as 1. Pr (ω) Pr(ω) ln mn ω Pr (ω) Pr(ω), Ths dstance measure can also be expressed usng the Bayes factor: D(Pr, Pr ) ln max ω,ω j F Pr,Pr (ω : ω j ). Ths measure satsfes the three propertes of dstance: postveness, symmetry, and the trangle nequalty. It s useful to compute ths dstance measure between two probablty dstrbutons as t allows us to bound the dfference n belefs captured by them. Theorem 1 [Chan & Darwche, 2002] Let Pr and Pr be two probablty dstrbutons over the same set of worlds. Let α and β be two events. We then have: e D(Pr,Pr ) O (α β) O(α β) ) ed(pr,pr, where O(α β) Pr(α β)/pr(α β) s the odds of event α gven β wth respect to Pr, and O (α β) Pr (α β)/pr (α β) s the odds of event α gven β wth respect to Pr. 6 The bound s tght n the sense that for every par of dstrbutons Pr and Pr, there are events α and β such that: O (α β) O(α β) ) ed(pr,pr ; O (α β) O(α β) ) e D(Pr,Pr. Accordng to Theorem 1, f we are able to compute the dstance measure between the orgnal and revsed dstrbutons, we can get a tght bound on the new belef n any condtonal event gven our orgnal belef n that event. The followng theorem computes ths dstance measure for belef revson methods based on probablty knematcs. 5 The results n ths secton are reformulatons of prevous results [Chan & Darwche, 2002], and are nspred by a new understandng of Jeffrey s rule and Pearl s method as two specfc nstances of revson based on probablty knematcs, and the understandng of Pearl s method n terms of Bayes factors. 6 Of course, we must have Pr(β) 0 and Pr (β) 0 for the odds to be defned.

7 Theorem 2 If Pr comes from Pr by probablty knematcs on γ 1,..., γ n, the dstance measure between Pr and Pr s gven by: D(Pr, Pr Pr (γ ) Pr (γ ) ) ln max ln mn. Theorem 2 allows us to compute the dstance measure for revsons based on Jeffrey s rule and vrtual evdence. Corollary 1 If Pr comes from Pr by applyng Jeffrey s rule on γ 1,..., γ n, wth posteror probabltes q 1,..., q n, the dstance measure between Pr and Pr s gven by: D(Pr, Pr ) ln max ln mn. Corollary 2 If Pr comes from Pr by applyng the method of vrtual evdence, wth lkelhood ratos λ 1,..., λ n bearng on γ 1,..., γ n, the dstance measure between Pr and Pr s gven by: D(Pr, Pr ) ln max ln mn. The mportance of Corollares 1 and 2 s that we can compute the dstance measure easly n both cases. For Jeffrey s rule, we can compute the dstance measure by knowng only the pror and posteror probabltes of events γ 1,..., γ n. For Pearl s method, we can compute the dstance measure by knowng only the lkelhood ratos λ 1,..., λ n. For both cases, the dstance measure can be computed n constant tme from the uncertan evdence, and we can guarantee a bound on the belef change due to the revson prncple of probablty knematcs, wthout explctly knowng the revsed probablty dstrbuton. We close ths secton by showng that the prncple of probablty knematcs s optmal n a very precse sense: t commts to a probablty dstrbuton that mnmzes our dstance measure. Theorem 3 If Pr comes from Pr by probablty knematcs on γ 1,..., γ n, t s optmal n the followng sense. Among all possble dstrbutons that agree wth Pr on the probabltes of events γ 1,..., γ n, Pr s the closest to Pr accordng to the dstance measure gven by Defnton 1. 8 Concluson In ths paper, we analyzed two man methods for revsng probablty dstrbutons gven uncertan evdence: Jeffrey s rule and Pearl s method of vrtual evdence. We showed that the two methods use the same belef revson prncple,.e. probablty knematcs, wth ther dfference beng only n the manner n whch they specfy uncertan evdence, and showed how to translate between the two methods for specfyng evdence. We also dscussed the much debated problem of nterpretng evdental statements. Moreover, we showed that the method of vrtual evdence can be reformulated n terms of Bayes factors, whch mples a number of results, ncludng the commutatvty of revsons based on ths method. Fnally, we showed that revsons based on probablty knematcs are optmal n a very specfc way, and ponted to a dstance measure for boundng belef change trggered by any revson based on probablty knematcs. Our bounds ncluded Jeffrey s rule and Pearl s method as specal cases. Acknowledgments Ths work has been partally supported by NSF grant IIS , MURI grant N , and DMI grant We also wsh to thank Carl Wagner, Judea Pearl, and the late Rchard Jeffrey, for ther valuable comments and dscussons. References [Chan & Darwche, 2002] He Chan and Adnan Darwche. A dstance measure for boundng probablstc belef change. In Proceedngs of the Eghteenth Natonal Conference on Artfcal Intellgence, pages , Edmonton, Canada, July [Darwche & Pearl, 1997] Adnan Darwche and Judea Pearl. On the logc of terated belef revson. Artfcal Intellgence 87(1 2):1 29, January [Dacons & Zabell, 1982] Pers Dacons and Sandy L. Zabell. Updatng subjectve probablty. Journal of the Amercan Statstcal Assocaton 77: , [Good, 1950] Irvng J. Good. Probablty and the Weghng of Evdence. Charles Grffn, London, [Good, 1983] Irvng J. Good. Good Thnkng: the Foundatons of Probablty and Its Applcatons. Unversty of Mnnesota Press, Mnneapols, [Jeffrey, 1965] Rchard C. Jeffrey. The Logc of Decson. McGraw-Hll, New York, (2nd edton) Unversty of Chcago Press, Chcago, (Paperback correcton) [Jeffrey, 1992] Rchard C. Jeffrey. Probablty and the Art of Judgement. Cambrdge Unversty Press, Cambrdge, [Jensen, 2001] Fnn V. Jensen. Bayesan Networks and Decson Graphs. Sprnger Verglag, New York, [Pearl, 1988] Judea Pearl. Probablstc Reasonng n Intellgent Systems: Networks of Plausble Inference. Morgan Kaufmann, San Francsco, [Pearl, 2001] Judea Pearl. On two pseudo-paradoxes n Bayesan analyss. Annals of Mathematcs and Artfcal Intellgence 32: , [Wagner, 2002] Carl Wagner. Probablty knematcs and commutatvty. Phlosophy of Scence 69: , 2002.