Monte Carlo Simulation I

Transcription

1 Monte Carlo Simulation I Anders Ringgaard Kristensen

2 What is simulation? Simulation is an attempt to model a real world system in order to: Obtain a better understanding of the system (including interactions) Fysiological models Herd models Study the effects of various (complex) decision strategies Herd models Slide 2

3 Different kinds of simulation Randomness (cf. Chapter 14.3 in textbook) Deterministic forget it in herd management! All calculations based on average values Same input Same output System comprehension Differential equations, physiology (NorFor). Probabilistic models All calculations based on distributions Same input Same output System comprehension, decision strategies Markov chains (earlier this course) Stochastic (Monte Carlo) models All random events are simulated by random number generation Same input Different output i.e. we need many replications System comprehension, decision strategies Herd constraints, complexity Slide 3

4 Different kinds of simulation Hierarchy (levels) Mechanistic: A system is modeled by its elements (sub-systems) A herd is modeled by its individual animals and their interactions. Empirical Only one level modeled Output direcly modeled from output Time Dynamic Static Slide 4

5 Our main example: SimFlock A stochastic, mechanistic, dynamic simulation model of a scavenging chicken production system. Smallholder production most often women. A very direct modeling of the real system Slide 5

6 SimFlock: An object oriented model Breeding animals Hens & Cocks Eggs Chicks Growers Infertile Dead Household consumption Pullets Cockerels Slide 6 Market

7 User interface visible objects All birds and eggs present in the flock shown. States of the birds can be investigated Demo

8 Other (Danish) herd simulation models SimHerd (dairy herds case example later) Jan Tind Sørensen Søren Østergaard Anne Braad Kudahl Jehan Ettema The Dina Pig simulation model Erik Jørgensen Slide 8

9 The TV Show once again Earlier we solved the decision problem by use of Bayesian networks and/or decision graphs. Are there any other methods we could apply? Experiment many replications needed Simulation create a simulation model

10 How to model the problem Identify the variables: True placement, True {1, 2, 3} First choice, Choice1 {1, 2, 3} Door opened, Opened {1, 2, 3} Second choice, Choice 2 {Keep, Change} Reward, Gain {0, 1000} Define a decision strategy (2 options): Choice2 = Keep Choice2 = Change Slide 10

11 What is needed for simulation Coin: Probabilities A random number generator, e.g. A coin A dice A computer 1 0,8 0,6 0,4 0, Dice: Probabilities Computer: Density 1 1,2 0,8 0,6 0,4 0,2 1 0,8 0,6 0,4 0, ,2 0,4 0,6 0,8 1

12 Simulation procedure: Use dice Host must place the reward behind an arbitrarily selected door: Roll the dice: 1 or 2: Door 1 3 or 4: Door 2 5 or 6: Door 3 Value of variable True determined as 1, 2 or 3! Participant must choose a door at random Roll the dice: 1 or 2: Door 1 3 or 4: Door 2 5 or 6: Door 3 Value of variable Choice 1 determined as 1, 2 or 3! Slide 12

13 Simulation procedure II Check whether True = Choice 1 If yes (two options): Roll the dice (or toss a coin) 1, 2 or 3: Open the lowest door number i True 4, 5 or 6: Open the highest door number i True If no (only 1 option): Open door i True and i Choice 1 Value of Opened determined Slide 13

14 Simulation procedure III Value of Choice 2 determined in accordance with the decision strategy Define new variable Final guess {1, 2, 3} If Choice 2 = Keep: Final guess = Choice 1 If Choice 2 = Change: Final guess = i, where i Choice 1 and i Opened Slide 14

15 Simulation procedure IV Check whether Final guess = True If yes: Gain = 1000 If no: Gain = 0 Simulation completed! Slide 15

16 Evaluation of strategies Define the strategy as Choice 2 = Keep Repeat the simulation many times (e.g. 1000) and calculate the average gain under the strategy. Define the strategy as Choice 2 = Change Repeat the simulation many times (e.g. 1000) and calculate the average gain under the strategy. Compare the average gain under the two strategies and select the best. Slide 16

17 Simulation procedure: Use computer Exactly as before, except: Instead of rolling the dice, we let the computer draw a random number r. Converting to variable value for True is done as follows: If r < : Door 1 If < r < : Door 2 If r > : Door 3 (Similar for other variable values) Let s try! Slide 17

18 Formalizing the simulation We have (refer to Chapter 14.2 of the textbook) A decision rule Θ: Very simple, just Keep or Change A set of parameters Φ = (Φ 0, Φ s. ) State of nature Φ 0 we have 4, p t1, p t2, p c1, p c2 : p t1 = P(True = 1), p t2 = P(True = 2), but P(True = 3) = 1 -p t1 -p t2 p c1 = P(Ch1= 1), p c2 = P(Ch1 = 2), but P(Ch1= 3) = 1 -p c1 -p c2 State variables, describing the system during simulation: Φ s. : True, Choice 1, Opened, Gain An output variable Ω: Gain. Notice that Ω Φ s. Model input is (Φ 0, Θ). Slide 18

19 Purpose of simulation The purpose of simulation usually is to calculate the expected utility E(U(Θ, Φ)) under a certain decision rule, Θ. In the example it is the expected gain under a specified decision rule (Keep or Change) This may be regarded as solving the numerical integration problem: Slide 19

20 The integral expression Horrible that s why we choose to build a simulation model instead! Let us take a look at the elements: Φ = (p t1, p t2, p c1, p c2, True,Choice 1, Opened, Gain) is a complete set of parameters. A value could for instance be φ = (1/3, 1/3, 1/3, 1/3, 2, 1, 3, 1000). Notice that we can easily specify the utility for a given outcome: U(Θ, Φ = φ) = Gain If we wanted to solve the integral directly we should furthermore be able to calculate the combined probability p(φ = φ) for any legal outcome φ of Φ. Almost impossible! Slide 20

21 The numerical approximation Simulation: Generate (through simulation) e.g. k outcomes φ 1, φ 2,, φ k under a chosen decision rule Θ: φ 1 = (1/3, 1/3, 1/3, 1/3, 2, 1, 3, 1000), U(Θ, φ 1 ) = 1000 φ 2 = (1/3, 1/3, 1/3, 1/3, 3, 3, 1, 0), U(Θ, φ 2 ) = 0 φ k = (1/3, 1/3, 1/3, 1/3, 1, 3, 2, 1000), U(Θ, φ k ) = 1000 Approximate the integral with the average utility: Slide 21

22 State of nature, I In the example the state of nature has been regarded as fixed and known Φ 0 = (1/3, 1/3, 1/3, 1/3). Assume that the host has a favorite door, e.g. door 3. He places the reward behind Door 3 with probability 0.8 and behind each of the others with probability 0.1. State of nature under those circumstances would be Φ 0 = (0.1, 0.1, 1/3, 1/3) if the participant doesn t have a favorite door. Does it change anything? Let s try! Slide 22

23 State of nature, II Assume further that also the participant has a favorite door, e.g. door 3. He also selects (first choice) Door 3 with probability 0.8 and each of the others with probability 0.1. State of nature under those circumstances would be Φ 0 = (0.1, 0.1, 0.1, 0.1) when both have Door 3 as their favorite door. Does it change anything? Let s try! Slide 23

24 State of nature, III In general, the optimal decision rule as well as the expected result depend on the state of nature. What is the state of nature in a livestock simulation model: Average growth rate Herd mortality rate Average milk yield Is the true state of nature known (with certainty)? Does it matter? Slide 24

25 State of nature, livestock models In livestock models we never know the true state of nature. We need to represent the uncertainty of the state of nature. We typically have some ideas a belief in the true values. The belief may be represented as a statistical distribution. Slide 25

26 Simple example, I Slide 26

27 Simple example, II Evaluation of expected value of production Deterministic: Stochastic, known state of nature Simulate 100 replications with µ = 10 Average: 435,27; Std. Dev Stochastic, unknown state of nature: Draw 100 random values of µ from N(10, 1) For each µ, simulate 100 replications Advanced Quantitative Average: Methods in Herd Management 430, Std. Dev Slide 27

28 Simple example, III 0,4 Frequency 0,3 0,2 0, Value of production Slide 28 Know n state of nature Unknow n state of nature

29 Representation of uncertainty of SON Essential, but Most simulation models ignore it Since the parameters Φ = (Φ 0, Φ s. ) consist of state of nature parameters and state variables (changing during simulation) we may split the integration from before: Slide 29

30 Interpretation Innermost integral: The expected utility for a given state of nature (Φ 0 = φ 0 ). Outermost integral: The expected utility across states of nature. Slide 30

31 Simulation procedure Select a number of alternative decision rules Θ to be tested. Draw n states of nature φ 01, φ 02,, φ 0 n from the underlying distribution of state of nature p 0 (Φ 0 =φ 0 ) For each decision rule: For each state of nature, i: Run the simulation model m times. Calculate the average result U i over the m simulation results (numerical evaluation of the innermost integral). Calculate the average value of U 1, U 2,, U n (numerical evaluation of the outermost integral). Select the best performing decision rule. Slide 31

32 Main problem It is difficult to specify the distribution of the state of nature. For a systematic description of the approach used in the SimFlock model, reference is made to Kristensen & Pedersen (2003) link at the homepage. Basic principle: Each parameter of the state of nature is specified through a distribution instead of a value. Such a distribution is called a hyper distribution. The parameters of a hyper distribution are called hyper parameters. Estimated from production data from 30 flocks in Zimbabwe. Easy, if parameters are independent Difficult if they interact Wednesday we shall take a closer look! Slide 32