Utility Estimation and Preference Aggregation under Uncertainty by Maximum Entropy Inference

Transcription

1 Utility Estimation and Pefeence Aggegation unde Uncetainty by Maximum Entopy Infeence Andé Ahua FenUnivesität in Hagen D-5884 Hagen ABSTRACT. This pape deals with the poblem how to estimate multiciteia utility functions of odinal attibutes unde incomplete infomation. We investigate a pobabilistic epesentation of a pefeence system. By this it is possible to communicate infomation about a decisionmae s pefeences in the language of pobabilistic conditionals and futhe it enables to apply the concept of maximum entopy (ME) infeence to pocess incomplete infomation about single multiciteia pefeences to complement and to aggegate the pefeences; both is ustified by infomation theoy. The cucial point is to undestand a single utility as a special ind of infomation and with this ME infeence povides utility estimates that maximize the expected value of utility. Also the hee suggested access allows to handle multi-attibute utility estimates as well. In addition it is outlined how to nom the evaluated utility functions how to measue the mutual dependency of two sets of andom vaiables and how to pefom an accoding analysis of the estimates eliability. Fo MEpocessing and eliability analysis the pobabilistic expet system shell SPIRIT will be used within an example. 1. Utility definition by infomation Let V = { V... 1 Vn} denote a finite set of discete odinal obective vaiables with space of configuations V = { v= ( v1... v n )} = V. Fo a distinct decisionmae = 1... n we want to estimate his maginal as well as total utility functions uj : V { 1... } R J n afte he J povided infomation in tems of a finite set of single utility estimates R i i i i : { : 1... i i R = R uj v }. J x i = T = v J V x R J Please note that assuming a utility function of an odinal vaiable lifts to a cadinal scale: v v... v 1 2 m ( 1) ( 2)... u v u v u v m But in geneal the povided infomation R may be not sufficient to detemine unique utility functions. Now this completion shall be designed as linea as possible while especting the odinality as well as the set R. Respecting the set R and heading fo lineaity will be pefomed popely by the ME-infeence as shown in the next chapte. The advantage of this access is a modelling of pefeence aggegation which is fee of extensional assumptions of the pefeence stuctue as will be seen in chapte 2; such an open access is hadly investigated [2]. Also in such a model it will be possible to estimate maginal utility functions in accodance with the povided incomplete infomation on each obective. And the odinality can be obseved by the following way of modelling. Fist fo each obective V with values v... 1 v define m a set of vitual incements D = { d 1... d m }. Now suppose a maginal distibution on D is given. With this a distibutive function P on v can be defined by P v : = p d (1) l l= 1... So in paticula P ( v ) 1 m = always holds. Though the incements ae needed fo technical easons only to be able to use pobability functions of nominal vaiables instead of distibutive functions of odinal ones to be pecise mostly it will be possible to popely intepet the incements in the eal wold. Fo example a vey good esult is good plus something significantly moe and in the case of cadinality an intepetation of the incements is given natually. Anyway now maginal utility functions uj J { 1... n} on obectives V can be defined as follows. J u : ld J v ( ) J PJ v J Hee ld denotes the dual logaithm. As PJ ( vj ) = (2) 1 this definition obviously povides utility functions by means of v v u v u v J J l J J J J l and u v. (3) J J

2 If the utility function u shall be limited to ( 1) α u v = α e.g. α=1 because of (2) P( v1 ): = 2 has to be defined α additionally which is equivalent to p( d 1 ) : = 2 accoding to (1) (e.g. p( d 1) = 5 fo α=1). This limits the pobability mass of the incemental events in (1) to 2 α fo any esult v what is equal to limit the utility to α (c.f. (2)). Also (2) popely lins the maginal independence of andom vaiables with the pefeence independence of attibutes. The basic popeties of the logaithm veify that P VV = P V P V u VV = u V + u V (4) s s s s s s s s whee P s and u s denote the oint distibution and the oint utility function espectively. Thei constuction will be an easy esult of the following chapte. Popety (4) is an obvious advantage of the utility definition by (2) which might have seemed to be athe abitay befoe. In addition infomation theoy defines the infomation of an event ω to be inf ω : = ld p ω (5) when a coesponding pobability function p is given [3]; hee again ld denotes the dual logaithm. So the utility defined by (2) allows the intepetation as a special ind of infomation because of (5). Also such a utility function possesses that shape which was ecommended by DANIEL BERNOULLI the founde of utility theoy. In ode to eceive the desiable popety (4) all maginal and oint distibutions have to be nown and ME infeence is an excellent tool to ovecome this tas in the case of incomplete infomation. 2. ME infeence and measuing dependency Given (Ω A P) whee Ω denotes a discete domain with algeba of events A and pobability distibution P the entopy H of the distibution P is defined to be H( P) : = p( ω) ld p( ω) [bit]. (6) ω Ω Hee ld denotes the dual logaithm and p the pobability function of P. The entopy measues the aveage uncetainty of the distibution [6] and in addition (5) allows the entopy to be intepeted as the expectation value of dispatching infomation. Because of definition (2) in this pape we also may intepet the entopy as the expectation value of utility of the espective citeia vaiables. If P is not nown but Ω is the coss poduct of a finite set of finite andom vaiables value spaces and cetain infomation is given in tems of a set of pobabilistic ules the entopy can be employed to estimate the distibution. Hee a pobabilistic ule is an expession of the type A B [ x] ead A implies B with pobability x whee A and B ae popositional sentences built by liteals <vaiable>=<value> and lined by the opeatos (and) (o) (not) (implies) and espective paentheses. Thus infomation is consideed as a statement about a possible impact and modeled as a conditional expession with a espective pobability [6]. If the pemise B is a tautology such a pobabilistic ule is called a pobabilistic fact. The set of such pobabilistic ules is the ule basis R. The pobabilistic extension W(R) denotes all distibutions on Ω which espect R. Obeying to the pinciple of infomation fidelity by means of choosing a distibution without additional not intended dependency between the vaiables involved [7] we ty to find that one with maximum entopy; especting all the esticting ules the distibution peseves as much uncetainty as possible. Mathematically speaing find P* = ag max H( P) P W( R ) (7) ( { }) Solving (7) yields the distibution P* which contains ust the infomation given by the ule basis (c.f. [7]) and which now is eady to give answes in tems of evaluating any expession in the language of ules by calculating its pobability. As P* contains the fomely fomulated nowledge it is called a nowledge base. And because the coectness of an assetion can be evaluated as its possibility in a distibution which was woed out by maximizing the entopy this evaluation is called maximum entopy infeence (ME infeence). Please note that the language of conditionals needs a thee-value logic [3]. But this is ignoed on the level of pobabilities o if all conditionals at last ae facts. Futhemoe ME infeence is shown to pefom a cautious stuctual analysis by means of peseving as much independence as possible [7] and it enables to measue the eduction of complexity of identifying the hidden stuctue by the amount of H( P ) H( P*) whee P denotes the initial unifom distibution i.e. solving (7) with total ignoance R = [1]. And thee is anothe cucial esult povided by the entopy namely the synentopy o tansinfomation I of (two sets) of vaiables. = + I VV HV HV HVV. s s s The synentopy symmetically measues the eduction of uncetainty if the dependencies of both (sets of) vaiables involved ae espected [4] and thus it can be intepeted as a measue fo the mutual dependencies. This intepetation will be impotant fo a test of dependent maginal pefeences and such a test will be done in the next chapte within an example. (8)

3 3. Example Let V1 (4-value) and V2 (5-value) be two discete odinal vaiables and it is supposed that all utilities ae in [1]. The decisionmae now supplies with the following infomative facts. R = { u1( v13) = 5 u2( v22) = 85 u ( v ) = 3 u( v v ) = Because of definition (2) the single facts of R ae infomation values and as (5) is equivalent to p inf ω ( ω) 2 = (9) it emains to use (1) in ode to eceive a ule basis which then can be pocessed in SPIRIT i.e. the set R has to be tanslated into the language of pobabilistic conditionals. These two steps povide the following. Fist R equals to { Pv = Pv = Pv = Pvv = } 71; 55; 81; Second thee is the incemental epesentation = + + P v p d p d p d = + P v p d p d = P v p d p d p d p d = + P v v p d d p d d Now Fig. 1 shows the esulting ule basis aleady as an input in SPIRIT s ule window. SPIRIT [8] is a pobabilistic expet system shell designed fo ME infeence on discete nominal vaiables; it allows communication in the language of pobabilistic conditionals and suppots the whole ME infeence pocessing. Please note that the ules have been a little bit simplified with espective coected pobabilities. Because of limiting the maginal utility functions (by the fist and the second ule in Fig.1) one may leave out the espective events in the othe ules if the pobability is deceased (by 5 fo α=1 in this example); this has been done in the emaining fou ules. In addition SPIRIT expects notation in the conunctive standad fom e.g. p d d + p d d p d d d d ( ) Fig.1 shows the initial unifom distibutions befoe ME pocessing. } Figue 1. Rule basis of the example in SPIRIT. By a simple use of a button SPIRIT now solves the nonlinea poblem (7) iteatively see Fig.2. Figue 2. Stating ME pocessing in SPIRIT. Afte the vey quic pocessing SPIRIT povides the entopy optimal maginal distibutions (see the left side of Fig.3). Figue 3. Distibutions afte ME pocessing in SPIRIT. SPIRIT also identifies local event goups LEGs i.e. subsets of vaiables with mutual dependencies as fomulated explicit o implicit by the ule basis. On the othe hand the vaiables of two LEGs ae conditionally independent i.e. independent when the vaiables of thei intesection ae fixed valued. In addition fo each LEG SPIRIT calculates its entopy measue (369 [bit] fo the example see Fig.3) and povides its maginal distibution. With this and the definition of conditional pobability [3] the oint distibution can be econstucted which as a whole nomally is too big to be stoed. Anyway this is beyond the scope of this pape. In the example thee is only one single LEG consisting of ust two vaiables. We use it to get the maginal distibution of the LEG which fo the example aleady is the oint distibution (see the ight side of Fig.3). The maginal distibutions allow to complete the maginal utility functions by using (1) (2) and (5). Exemplaily this is demonstated fo the fomely unspecified utility u2( v 23). ( ) ( ) u ( v ) = ld P v = ld p d + p d + p d = ld = ld 68 = Futhe the oint distibution enables to aggegate pefeences by estimating the oint utility function. Fo example it is

4 uv ( 14v24) = ld = ld = ld ( p ( dv11dv21) p( dv12dv21 ) p( dv13dv24 ) p( dv14dv24 )) ( p( dv11dv25 ) p( dv12dv25 ) p( dv13dv25 ) p( dv14dv25 )) The synentopy (8) is manually calculated to be I( V1 V2) = H( V1) + H( V2) H( V1 V2) = = 29 [bit]. Thus the both vaiables ae not totally independent fom each othe. 4. Additional emas 4.1. Reliability Although the distibution P* povided by (7) is unique nomally this is not the only element of W(R) thus additional infomation may change the pobability values estimated by the ME infeence pocess. REUCHER [5] suggests the functional Θ ( R): = max R QP (1) Q W( R) to measue the emaining uncetainty of selecting some distibution out of W(R). In (1) again P denotes the unifom distibution and R is the elative entopy defined by ( ω) ( ω) RQP ( ): = q( ω) ld q (11) p ω Ω fo two distibution Q P on the same space Ω [4]. In addition fo all pobabilities of P* detemined by (7) REUCHER defines single intevals of uncetainty i.e. the maximum individual inteval fo any pobability in accodance with the ule basis [5]. The detemination of an uncetainty inteval is pefomed by an iteative pocess based on the elative entopy. Fo futhe explanation of this ind of postoptimal analysis the boo of REUCHER is ecommended. Anyway thee is a pilot outine fo the detemination of uncetainty intevals implemented in SPIRIT (vesion 3.2 and up) ME utility and the Benoulli pinciple In chapte 2 it was stated that the entopy can be intepeted as the expectation value of dispatching infomation. As in this pape utility was ust defined to be infomation (2)(5) solving (7) povides a distibution P* with a maximum expectation value of utility. But this is quite diffeent fom the BERNOULLI pinciple fo decision unde is max ϕ( a ) = u a s p s ai A s S i i i (12) whee ai ae actions and s ae andom states with distibutive function p. This latte pinciple gives the advice to choose that altenative which yields the maximum expectation value of utility in the case that the distibution is nown and fix. This is the diffeence to the ME utility estimation whee the maximization is pefomed by vaiation of pobabilities. Thus ME utility estimation is suitable to lift fom an uncetainty up to a is situation within a decisionmaing poblem and then fo example the BERNOULLI pinciple may be used to choose an altenative. Howeve ME utility estimation is cautious and will not exclude any altenative because of undeestimation of the decision citeion given by the ight side of (12). 5. Summay and outloo In this pape utilities of citeia at last have been defined as infomation values. In the situation of incomplete infomation the ME infeence is able to detemine maginal distibutions especting infomation fidelity on cetain sets of vaiables. With this and by using a special way of modelling ME infeence yields the oint as well as the maginal utility functions in accodance with the povided espective infomation. Futhe the elative entopy enables to wo out the eliability of these utility functions by means of intevals of uncetainty and the synentopy allows to measue mutual dependencies. Thee ae two essential aspects to be studied next. Fist ME utility estimation is a vey pomising method fo an inteactive pocess of utility estimates acquisition and theoetically this is aleady coveed up by the pocess of gadually building a nowledge base. Second it has to be investigated how to effectively ecognize the stuctue of a distinct pefeence aggegation in ode to deive a epesentative system of obectives. REFERENCES [1] Ahua A.: How to ovecome the poect is complexity by maximum enttopy infeence Poc. PMS 24 9 th Int. Woshop on Poect Management and Scheduling. Nancy Fance 24. [2] Fishbun P.C.: Nonlinea Pefeence and Utility Theoy. Wheatsheaf Bighton [3] Ken-Isbene G.: Conditionals in Nonmonotonic Reasoning and Belief Revision. Spinge Belin [et al.] 21. [4] Matha R.: Infomationstheoie Disete Modelle und Vefahen. Teubne Stuttgat [5] Reuche E.: Modellbildung bei Unsicheheit und Ungewißheit in onditionalen Stutuen. Logos Belin 22. [6] Rödde W.: Conditional Logic and the Pinciple of Entopy. Atificial Intelligence [7] Rödde W.: Knowledge Pocessing unde Infomation Fidelity Poc. IJCAI th Int. Joint Confeence on Atificial Intelligence. Seattle Washington

5 [8] SPIRIT in the wold wide web: Fee download of the open souce ava shell afte egistation.