Engineering Decisions

Transcription

1 GSOE9

2 Outline Decision problem classes Decision problems can be classiied based on an agent s epistemic state: Decisions under certainty: the agent knows the actual state Decisions under uncertainty: Decisions under ignorance (ull uncertainty): the agent believes multiple states/outcomes are possible; likelihoods unknown : the agent believes multiple states/outcomes are possible; likelihood inormation available

3 River example X C Example (River logistics) lice s warehouse is located at X on a river that lows down-stream rom C to. She delivers goods to a client at C via motor boats. On some days a (ree) goods erry travels up the river, stopping at then and C, but not at X. The uel required (litres) to reach C rom each starting point: X C To C rom: lice wants to minimise uel consumption (in litres). Decisions under incomplete inormation: risk Example (Ferry likelihood) In the river logistics problem, suppose lice has received an order requiring a package to be delivered every day or the next eight days. Her records show that out o the last days, the erry was operating on 75. dditional inormation (lice s records) can be used to estimate likelihood o erry being operational on any given day Maximum likelihood estimation assumption: erry operates in three out o every our days Is the Maximin strategy () still the most rational choice?

4 River example X C C lice considers three possible ways to get to C (rom starting point X): : via, by loating down the river : via, by travelling up-stream to C : by travelling all the way to C Outcomes are measured in litres let in a our-litre tank. Exercise Let w : Ω R denote uel consumption in litres. What transormation : R R is responsible or the values v : Ω R in the decision table? Single decision; multiple trials Fuel savings or delivering one package per day over eight days when erry operates on six o those days: vg min 8.5 where, e.g., = 6 +, = 6 +, etc. Can we assume the erry will operate in six o the eight days? Maximin chooses based on least avourable state () Given inormation about likelihood o, is Maximin suitable?

5 Single decision; multiple trials lternatively: In how many o the next eight days will erry operate: Six? Five? Eight? None? ssume long sequence o days... or maximum likelihood estimate (six out o eight) Proportion o days in which erry operational: p = 6 8 = E.5 min Is p probability that erry will operate on any given day? Frequency interpretation o probability Outcomes: t, h, h, t, t, h, t, h, t, h,... }{{} n Deinition (Frequency interpretation o probability) The probability o an event, E, in an experiment o chance, is the limit o the average occurrences o E over any sequence o indeinitely many trials; i.e., P (E) = lim n n E n where n E is the number o occurrences o event E in the irst n trials. e.g., For event H:,,,, 5, 6, 7,..., n H n,... What is P (H) or this experiment?

6 Expected values Deinition (Expected value) The expected value o a random variable X : Ω R with probability distribution P : Ω R is given by: E(X) = ω Ω P (ω)x(ω) Deinition The event corresponding to value x R, denoted X x, is deined as: More generally, or R: X x = X [x] = {ω Ω X(ω) = x} X = X [] = {ω Ω X(ω) } Expected values For a random variable (real-valued unction rom Ω to R) X: E(X) is also called the limiting (or long run) average o X E(X) may not be any actual value in ran X E(X) is a measure o the centre, or centroid, o the values o the outcomes Natural correspondence with the centre o gravity/mass o a distribution o point masses on a line, where P (X = x i ) corresponds to the proportion o the total mass positioned at x i

7 Multiple random trials In this situation there are multiple trials (days) o some random process: days In each trial (day) dierent states may occur: erry () or no erry () Inormation exists about the likelihood o occurrence o states: 75% erry to 5% no erry Maximin assumes worst case or each action even when the worst case (no erry) is unlikely; i.e., it ignores likelihood inormation Over working days, lice s total value is greater via than decision rule which takes likelihood inormation into account would be preerable Probabilistic lotteries Deinition (Probabilistic lottery) probabilistic lottery over a inite set o outcomes, or prizes, Ω, is a pair l = (Ω, P ), where P : Ω R is a probability unction. The lottery l is written: l = [p : c p : c... p n : c n ] where or each s i S P(Ω), p i = P (s i ) = P (c i ). Example (To C via ) lice s decision to travel via corresponds to: l = [ : : ] where outcomes have been replaced by their values. : :

8 Value o a lottery Deinition (Value o a lottery) The value o a probabilistic lottery (Ω, P, v) is the expected value over its outcomes: V v (l) = E(v) = ω Ω P (ω)v(ω) For strategy : V (l ) = () + () = + = Note: not value o any outcome o strategy :, Frequency interpretation: V (l ) is the average value o over many days Outline

9 Under risk, each strategy in a decision problem corresponds to a probabilistic lottery. Deinition (ayes value) Given a probability distribution over states, the ayes value, V, o a strategy is the expected value o its outcomes. Deinition (ayes strategy) ayes strategy is a strategy with maximal ayes value. Deinition (ayes decision rule) The ayes decision rule is the rule which selects all the ayes strategies. ayes strategies For the river problem, assume the proportion o days in which the erry operates is p = p: V p p 5 V p p + ayes values or each strategy plotted or all values o p [, ]. Exercise For what values o p will the ayes decision rule preer to? p

10 Indierence curves: Maximin For the pure actions below: s s C D 5 E 5 Consider curves o all points representing strategies with same Maximin value; i.e., Maximin indierence curves. s 5 E C I() D 5 s Indierence curves: ayes ayes decision rule indierence curves are linear: p p V p p + a v v pv + ( p)v Indierence curves: V (a) = pv + ( p)v = u p = p = In gradient-intercept orm, v = u p p p v, where m = p p ; e.g., or p =, m = / = ecause v u; i.e., higher lines receive greater ayes values

11 Indierence curves: ayes In general, or two actions: p p s s a a b b y = a b x = a b y x p = y x+ y y p = x + y = m m where m is the gradient o line. For example: i is (, ) and is (, ) then: p = ( )+( ) = + = Indierence classes and Exercises Prove the expression or p For the river problem, what is the slope o the line joining the two actions? For what probability are the two actions o equal ayes value? What is the ayes value associated with this line? Repeat the above exercises or regret

12 ayes strategies For the pure actions below with P (s ) = p: s s s V p 5 + p C 5 5 p 5 C p = Slope o C: m = 5 5 =. p = + =. Note: p m. 5 s ayes strategies For the pure actions below with P (s ) = p: s s V p 5 + p C 5 5 p For p =, the value o the ayes action(s) is least. V 5 C p Deinition The least avourable probability distribution on the states/outcomes is the probability distribution or which ayes strategies have minimal values.

13 ayes solutions For the pure actions below with P (s ) = p: s s s V 5 5 p + p C p Slope o C: m = =. p =. Slope o C: m =. p =. 5 CMp = p = 5 p = s ayes strategies s V 5 a p = C M p = p = 5 C a M 5 s p Note that the Maximin action is a ayes action or p = Note that the internal mixed strategy a.5..c is not ayes

14 ayes summary Theorem Results about ayes decision rule: Mixing can improve upon the Maximin value o pure strategies, but it does not improve upon the ayes value o pure strategies ayes strategies are invariant/preserved under regret; i.e., the same strategy is chosen under regret as otherwise Exercise Prove the theorems above. dmissible mixed strategies s s C D C D s Exercises Which mixed strategies above are admissible? re Maximin mixed strategies always admissible? re ayes mixed strategies always admissible? re Maximin mixed strategies always ayes or some p? re admissible mixed strategies ayes or some p? s

15 ayes summary Partial inormation situations (risk) Inormation can aect degree o likelihood/belie (ayesian probability) ayes rule more appropriate when partial inormation present ayes values, ayes decision rule, ayes strategies Graphical representation o ayes values ayes indierence curves