Constructive Decision Theory

Transcription

1 Constructive Decision Theory p. 1/27 Constructive Decision Theory Joe Halpern Cornell University Joint work with Larry Blume and David Easley, Economics, Cornell

2 Constructive Decision Theory p. 2/27 Savage s Approach Savage s approach to decision making has dominated decision theory since the 1950 s. It assumes that a decision maker (DM) is given/has a sets of states a seto of outcomes A (Savage) act is a function from states to outcomes.

3 Constructive Decision Theory p. 2/27 Savage s Approach Savage s approach to decision making has dominated decision theory since the 1950 s. It assumes that a decision maker (DM) is given/has a sets of states a seto of outcomes A (Savage) act is a function from states to outcomes. Example: Betting on a horse race. S = possible orders of finish O = how much you win act = bet

4 Constructive Decision Theory p. 3/27 Savage s Theorem Savage assumes that a DM has a preference order on acts satisfying certain postulates: E.g. transitivity: ifa 1 a 2 anda 2 a 3, thena 1 a 3. He proves that if a DM s preference order satisfies these postulates, then the DM is acting as if he has a probabilitypr on states he has a utility functionuon outcomes he is maximizing expected utility: a b iffe Pr [u a ] E Pr [u b ]. u a (s) = u(a(s)): the utility of actain states

5 Constructive Decision Theory p. 4/27 Are Savage Acts Reasonable? Many problems have been pointed out with Savage s framework. We focus on one: People don t think of acts as function from states to outcomes In a complex environment, it s hard to specify the state space and outcome space before even contemplating the acts What are the states/outcomes if we re trying to decide whether to attack Iraq? What are the acts if we can t specify the state/outcome space?

6 Constructive Decision Theory p. 5/27 Acts as Programs An alternative: instead of taking acts to be functions from states to outcomes, we take acts to be syntactic objects essentially, acts are programs that the DM can run. Consider the act Buy 100 shares of IBM : Call the stock broker, place the order,...

7 Constructive Decision Theory p. 5/27 Acts as Programs An alternative: instead of taking acts to be functions from states to outcomes, we take acts to be syntactic objects essentially, acts are programs that the DM can run. Consider the act Buy 100 shares of IBM : Call the stock broker, place the order,... Program can also have tests if the Democrats win then buy 100 shares of IBM To specify tests, we need a language

8 The Setting Savage assumes that a DM is given a state space and an outcome space. We assume that the DM has a seta 0 of primitive programs Buy 100 shares of IBM Attack Iraq a sett 0 of primitive tests (i.e., formulas) The price/earnings ratio is at least 7 The moon is in the seventh house a theoryax Some axioms that describe relations between tests E.g.,t 1 t 2 t 3 Constructive Decision Theory p. 6/27

9 Constructive Decision Theory p. 7/27 The Programming Language In this talk, we consider only one programming construct: if... then... else Ifa 1 anda 2 are programs, andtis a test, then iftthena 1 elsea 2 is a program if moon in seventh house then buy 100 shares IBM tests formed by closing offt 0 under conjunction and negation: tests are just propositional formulas LetAdenote this set of programs (acts). In the full paper we also consider randomization. With probabilityr performa 1 ; with probability1 r, performa 2

10 Constructive Decision Theory p. 8/27 Programming Language Semantics What should a program mean? In this paper, we consider input-output semantics: A program defines a function from states to outcomes once we are given a state space and an outcome space, a program determines a Savage act The state and outcome spaces are now subjective. Different agents can model them differently The agent s theory AX affects the semantics: interpretation of tests must respect the axioms

11 Constructive Decision Theory p. 9/27 Semantics: Formal Details I Given a state spaces and an outcome spaceo, we want to view a program as a function froms too, that respects AX. We first need a program interpretation ρ SO that associates with each primitive program ina 0 a function froms too a test interpretation π S that associates with each primitive proposition int 0 an event (a subset ofs) extend tot in the obvious way require thatπ S (t) = S for each axiomt AX axioms are necessarily true

12 Constructive Decision Theory p. 10/27 Can extendρ SO to a function that associates with each program inaa function froms too: ρ SO (iftthena 1 elsea 2 )(s) = ρ SO (a 1 )(s) ifs π S (t) ρ SO (a 2 )(s) ifs / π S (t)

13 Constructive Decision Theory p. 11/27 Where We re Headed We prove the following type of theorem: If a DM has a preference order on programs satisfying appropriate postulates, then there exist a state spaces, a probabilitypr ons, an outcome space O, a utility functionuono, a program interpretationρ SO, a test interpretationπ S such thata b iffe Pr [u ρso (a)] E Pr [u ρso (b)].

14 Constructive Decision Theory p. 12/27 This is a Savage-like result The postulates are variants of standard postulates The DM has to put a preference order only on reasonable acts But nows andoare subjective, just likepr andu! S,O,Pr,u,ρ SO, andπ S are all in the DM s head S ando are not part of the description of the problem

15 Constructive Decision Theory p. 13/27 The Benefits of the Approach We have replaced Savage acts by programs and prove Savage-type theorems. So what have we gained?

16 Constructive Decision Theory p. 13/27 The Benefits of the Approach We have replaced Savage acts by programs and prove Savage-type theorems. So what have we gained? Acts are easier for a DM to contemplate No need to construct a state space/outcome space Just think about what you can do

17 Constructive Decision Theory p. 13/27 The Benefits of the Approach We have replaced Savage acts by programs and prove Savage-type theorems. So what have we gained? Acts are easier for a DM to contemplate No need to construct a state space/outcome space Just think about what you can do Different agents can have different conceptions of the world You might make decision on stock trading based on price/earnings ratio I might use astrology (and might not even understand the notion of p/e ratio)

18 Constructive Decision Theory p. 14/27 Agreeing to disagree results [Aumann] (which assume a common state space) disappear

19 Constructive Decision Theory p. 14/27 Agreeing to disagree results [Aumann] (which assume a common state space) disappear (Un)awareness becomes particularly important

20 Constructive Decision Theory p. 14/27 Agreeing to disagree results [Aumann] (which assume a common state space) disappear (Un)awareness becomes particularly important Can deal with unanticipated events, novel concepts: Updating conditioning

21 Constructive Decision Theory p. 14/27 Agreeing to disagree results [Aumann] (which assume a common state space) disappear (Un)awareness becomes particularly important Can deal with unanticipated events, novel concepts: Updating conditioning We do not have to identify two acts that act the same as functions Can capture resource-bounded reasoning (agent can t tell two acts are equivalent) allow nonstandard truth assignments t 1 t 2 may not be equivalent tot 2 t 1 Can capture framing effects

22 Constructive Decision Theory p. 15/27 Framing Effects Example: [McNeill et al.] DMs are asked to choose between surgery or radiation therapy as a treatment for lung cancer. They are told that, Version 1: of 100 people having surgery, 90 alive after operation, 68 alive after 1 year, 34 alive after 5 years; with radiation, all live through the treatment, 77 alive after 1 year, 22 alive after 5 years Version 2: with surgery, 10 die after operation, 32 dead after one year, 66 dead after 5 years; with radiation, all live through the treatment, 23 dead after one year, 78 dead after 5 years. Both versions equivalent, but In Version 1, 18% of DMs prefer radiation; in Version 2, 44% do

23 Constructive Decision Theory p. 16/27 Framing in our Framework Primitive propositions: RT : 100 people have radiation therapy; S: 100 people have surgery; L 0 (k): k/100 people live through operation (i = 0) L 1 (k): k/100 are alive after one year L 5 (k): k/100 are alive after five years D 0 (k),d 1 (k),d 5 (k) similar, with death Primitive programs a S : perform surgery (primitive program) a R : perform radiation therapy

24 Constructive Decision Theory p. 17/27 Version 1: Which program does the DM prefer: a 1 = ift 1 thena S elsea, or a 2 = ift 1 thena R elsea, whereais an arbitary program and t 1 = (S L 0 (90) L 1 (68) L 5 (34)) (RT L 0 (100) L 1 (77) L 5 (22)) Can similarly capture Version 2, with analogous testt 2 and programsb 1 andb 2 Perfectly consistent to havea 1 a 2 andb 2 b 1 A DM does not have to identifyt 1 andt 2 Preferences should change oncet 1 t 2 is added to theory

25 Constructive Decision Theory p. 18/27 The Cancellation Postulate Back to the Savage framework: Cancellation Postulate: Given two sequences a 1,...,a n and b 1,...,b n of acts, suppose that for each states S {{a 1 (s),...,a n (s)}} = {{b 1 (s),...,b n (s)}}. {{o,o,o,o,o }} is a multiset Ifa i b i fori = 1,...,n 1, thenb n a n.

26 Constructive Decision Theory p. 19/27 Cancellation is surprising powerful. It implies Reflexivity Transitivity: Supposea b andb c. Take a 1,a 2,a 3 = a,b,c and b 1,b 2,b 3 = b,c,a. Event independence: Suppose thatt S andf T g f T g f T g is the act that agrees withf ont andg ons T. Take a 1,a 2 = f T g,f T h and b 1,b 2 = f T g,f Th. Conclusion: f T h f T h

27 Constructive Decision Theory p. 20/27 Cancellation in Our Framework A program maps truth assignments to primitive programs: E.g., consider iftthena 1 else (ift thena 2 elsea 3 ): t t a 1 t t a 1 t t a 2 t t a 3 Similarly for every program.

28 Constructive Decision Theory p. 20/27 Cancellation in Our Framework A program maps truth assignments to primitive programs: E.g., consider iftthena 1 else (ift thena 2 elsea 3 ): t t a 1 t t a 1 t t a 2 t t a 3 Similarly for every program. Can rewrite the cancellation postulate using programs: replace outcomes by primitive programs replace states by truth assignments

29 Constructive Decision Theory p. 21/27 The Main Result Theorem: Given a preference order on acts satisfying Cancellation, there exist a sets of states and a setp of probability measures ons, a seto of outcomes and a utility functionuono, a program interpretationρ SO, a test interpretationπ S such that a b iffe Pr [u a ] E Pr [u b ] for allpr P. Moreover, if is totally ordered, thenp can be taken to be a singleton.

30 Constructive Decision Theory p. 22/27 Updating In the representation, can always take the state space to have the form AT AX TOT( ): AT AX = all truth assignments to tests compatible with the axioms AX TOT( ) = total orders extending Updating proceeds by conditioning: Learnt representation isp t Learna b: representation isp ( (a,b))

31 Constructive Decision Theory p. 23/27 Uniqueness Savage gets uniqueness; we don t: We do have a canonical representationat AX TOT( ) In the totally ordered case,s = AT. Cannot takes = AT AX in the partially-ordered case Even with no primitive propositions, if primitive programs a and b are incomparable, need two states, two outcomes, and two probability measures to represent this. Can t hope to have a unique probability measure ons, even in the totally ordered case: there aren t enough acts. Savage s postulates force uncountably many acts

32 Constructive Decision Theory p. 24/27 Program Equivalence When are two programs equivalent? That depends on the choice of semantics With input-output semantics, two programs are equivalent if they determine the same functions no matter what S,O,π S,ρ SO are.

33 Constructive Decision Theory p. 24/27 Program Equivalence When are two programs equivalent? That depends on the choice of semantics With input-output semantics, two programs are equivalent if they determine the same functions no matter what S,O,π S,ρ SO are. Example 1: (iftthenaelseb) (if t thenbelsea). These programs determine the same functions, no matter how t, a, and b are interpreted.

34 Constructive Decision Theory p. 24/27 Program Equivalence When are two programs equivalent? That depends on the choice of semantics With input-output semantics, two programs are equivalent if they determine the same functions no matter what S,O,π S,ρ SO are. Example 1: (iftthenaelseb) (if t thenbelsea). These programs determine the same functions, no matter how t, a, and b are interpreted. Example 2: Ift t, then (iftthenaelseb) (ift thenaelseb).

35 Constructive Decision Theory p. 25/27 Cancellation and Equivalence Testing equivalence of propositional formulas is hard co-np complete, even for this simple programming language Have to check propositional equivalence Cancellation implies a DM is indifferent beteween equivalent programs. Lemma: Cancellation ifa b, thena b.

36 Constructive Decision Theory p. 25/27 Cancellation and Equivalence Testing equivalence of propositional formulas is hard co-np complete, even for this simple programming language Have to check propositional equivalence Cancellation implies a DM is indifferent beteween equivalent programs. Lemma: Cancellation ifa b, thena b. Cancellation requires smart decision makers! We don t have to require cancellation Can consider more resource-bounded DM s... by changing the axioms

37 Constructive Decision Theory p. 26/27 Non-classical DMs We have assumed that DMs obey all the axioms of propositional logic π S ( t) = S π S (t) andπ S (t 1 t 2 ) = π S (t 1 ) π S (t 2 ). But we don t have to assume this! Instead, write down explicitly what propositional properties hold We still get that Cancellation, and thata b impliesa b But now this isn t so bad: intuitively, the logic is restricted so that if a b, then the DM can tell thataandbare equivalent, and so we should havea b

38 Constructive Decision Theory p. 27/27 Conclusions The theorems we have proved show only that this approach generalizes the classic Savage approach. The really interesting steps are now to use the approach to deal with issues that the classical approach can t deal with conditioning on unanticipated events (un)awareness papers with Rêgo learning concepts...