Fun with Utilities. 1 Introduction. 2 Preference Patterns. Michael A. Goodrich. September 1, 2005

Transcription

1 Fun with Utilities Michael A. Goodrich September 1, Introduction In these notes, we will introduce the axioms underlying utility theory, and then prove one half of what I call the Utility Theorem. The description of the Utility Theorem is taken from [3, 1], but the proofs were generated by me. If I make changes to these notes after they are posted and if these changes are important (beyond cosmetic), the changes will highlighted in bold red font. This will allow you to quickly compare this version of the notes to an old version. 2 Preference Patterns Consider some abstract set C with elements c i. Thus, C = {c i : i I} where I is some index set. For example, C can be the set of consequences that can arise from taking action from a particular state. A preference pattern is a binary relation over C. The following notation is used to describe the preference relation between various elements of C [3]: c i c j : c i is preferred to c j. c i c j : the agent is indifferent 1 between c i and c j ; the two elements are equally preferred. c i c j : c i is at least as preferred as c j. A preference pattern is a linear ordering [2]. As such, it has the following properties: For all c C, c c. For all c i, c j C, if c i c j and c j c i then c i c j. For all c i, c j, c k C, if c i c j and c j c k then c i c k. For all c i, c j C, either c i c j or c j c i. These appear to be fairly benign properties, and for many problems they give us no problem. However, the fourth property seems especially strong to me. This property says, in the context of making a choice, that every consequence can be compared to every other consequence. This, in turn, requires that every possible consequence of an action be known before a comparison of various actions can be undertaken. Can you think of a situation where this might be problematic? 1 One web definition of indifference is involving no preference, concern, or attention. 1

2 3 The Utility Theorem 3.1 What Is It? The Utility Theorem simply says that if an agent has a preference relation that satisfy the axioms of preference then a real-valued utility function can be constructed that reflects this preference relation. (This utility function also expresses how strong the preference is.) The following are called the axioms of preference. (In case you cannot remember, means OR, and means AND.) They include the requirements of a linear ordering, and add some extra constraints that allow preference strength to be determined. I choose to use A, B, and C notation rather than c i, c j, and c k, notation simply because it is easier to type. The A s, B s, and C s are simply elements of some set (such as the set of consequences) that are to be compared. Orderability: (A B) (B A) (A B). Transitivity: (A B) (B C) (A C). Continuity: A B C p (0, 1) : [p, A; 1 p, C] B. Substitutability: A B p [0, 1][p, A; 1 p, C] [p, B; 1 p, C]. ( ) Monotonicity: A B p, q [0, 1] : p q [p, A; 1 p, B] [q, A; 1 q, B] with indifference if and only if p = q. [ ] Decomposability: p, A; 1 p, [q, B; 1 q, C] [p, A; (1 p)q, B; (1 p)(1 q), C]. The notation [p, A; 1 p, B] denotes a lottery ticket where, with probability p, the option A is won and with probability 1 p the option B is won. If the preference pattern follows the axioms of utility, then there exists a (not necessarily unique) real-valued function U such that the following hold: 3.2 Constructing the Utility U(A) > U(B) A B, U(A) = U(B) A B. In this section, we will discuss a procedure for constructing the real-valued utility function U that satisfies the utility theorem. In essence, the utility theorem states that well-behaved preferences can be represented by a real-valued function; the procedure we discuss allows us to create such a function. To demonstrate the procedure, we will use an example. I call this technique the preferences among lotteries method. Example. You are graduating from college soon, and you have four job offers: one from Microsoft (as a programmer), one from McDonald s (as a hamburger maker), one from Walmart (as a checkout clerk), and one from Sun (as a tester). Suppose that your preferences are as follows: Microsoft Sun Walmart McDonald s. Construct a utility that represents this preference pattern. The first step in creating U is to assign a real number to the most and least preferred options. Any real number will do, provided that the number assigned to the most preferred option is higher than the number 2

3 assigned to the least preferred option. By the axioms of preference, we know that a most-preferred and least-preferred option exist 2. Example continued. In the set of possible jobs, Microsoft is most preferred and McDonald s is least preferred. Suppose that I choose the following values for each option: U(Microsoft) = 100.0, U(McDonald s) = 1.0. By the continuity property, we know that there exists a p such that the agent is indifferent between an option and the lottery where the most preferred option is received with probability p and the least preferred option is received with probability 1 p. For all options, identify this p. For each option, A, assign the utility of A as follows: U(A) = pu(most Preferred) + (1 p)u(least Preferred). When you have done this for all options, the utility function is complete. Before finishing the example, note that by finding the value p you are essentially identifying how strongly you feel about the option. Note also that by creating U(A) using probabilities, you have created a utility function that is compatible with taking expectations. Example concluded. We need to find the p for Sun such that Sun [p,microsoft; 1-p, McDonald s]. Suppose this value is p = 0.9. We also need to find the p for Walmart such that Walmart [p,microsoft;1- p,mcdonald s]. Suppose that this value is p = 0.2. Applying our formula, we have U(Sun) = = 90.1, and U(Walmart) = = Note that since Sun is preferred to Walmart, the value of p is higher. This is a general property that occurs because of the monotonicity property. 3.3 Proving the Utility Theorem For this proof, we will use the procedure presented above and discussed in class for constructing a utility function. If we assume that procedure, then we can show: U(A) > U(B) A B Utility Theorem. For a finite set of options, O = {A, B,...}, if a preference relations satisfies the axioms of preference, then the procedure we just described can be used to create a real-valued function U : O R such that U(A) > U(B) A B Let A be the most preferred option according to the preference pattern, and let A be the least preferred. Choose two real numbers U(A ) and U(A ) such that U(A ) > U(A ). For all options B such that A B A, define U(B) using continuity: p B = p (0, 1) : [p, A ; 1 p, A ] B U(B) = p B U(A ) + (1 p B )U(A ) Now, choose two options B 1 and B 2 such that B 1 B 2. We will consider three cases. These three cases address how B 1 and B 2 can be related to A and A. 2 Orderability says that every option is comparable to every other option. A simple O(n) algorithm can be used to find a most preferred option provided that the number of options is finite 3

4 Case 1: B 1 A If B 1 A then we know that, by construction, U(B 1 ) = U(A ); we also know that p B1 = 1. By continuity, we know that p B2 < 1. Putting these pieces together gives: U(B 1 ) = U(A ) = p B1 U(A ) + (1 p B1 )U(A ) > p B2 U(A ) + (1 p B2 )U(A ) = U(B 2 ). Thus, when B 1 B 2 and B 1 A it follows that U(B 1 ) > U(B 2 ). Point of clarification: When you read the above sequence of mixed equalities and inequalities, be careful. The proper way to read this is that the left part of the top equation equals the right part, the right part equals the next line which is greater than the next (second to last) line which equals the last line. Stringing these together gives that the left part of the top equation is greater than the bottom right of the bottom equation. Case 2: B 2 A We can use a similar procedure as the one above to show that, when B 2 = A, then U(B 1 ) > U(B 2 ). Case 3: A B 1 B 2 A By continuity, for the given B 1 and B 2, p B1, p B2 (0, 1) such that U(B 1 ) = p B1 U(A ) + (1 p B1 )U(A ) U(B 2 ) = p B2 U(A ) + (1 p B2 )U(A ) Since we know that B 1 B 2, and [p B1, A ; 1 p B1, A ] B 1, and B 2 [p B2, A ; 1 p B2, A ], it follows that [p B1, A ; 1 p B1, A ] [p B2, A ; 1 p B2, A ]. Since A A it follows from monotonicity that p B1 > p B2, but this means that U(B 1 ) = p B1 U(A ) + (1 p B1 )U(A ) (1) > p B2 U(A ) + (1 p B2 )U(A ) (2) = U(B 2 ) (3) which is what we set out to prove. Note that moving from Equation (1) to Equation (2) follows from the observation that a both equations are convex blends of a big number, U(A ) and a small number, U(A ). If the weighting on the big number goes down, then the weighted sum decreases. 3.4 The Rest of the Proof Can you show the other direction U(A) > U(B) A B? Can you complete the proof for equality/indifference U(A) = U(B) A B? 4

5 References [1] Jr. J. F. Nash. The bargaining problem. Econometrica, 18: , Reprinted in Classics in Game Theory, H. W. Kuhn, ed. [2] B. Khoussainov and A. Nerode. Automata Theory and Its Applications. Progress in Computer Science and Applied Logic. Birkhäuser, Boton, Massachusetts, USA, [3] S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall,