Monte Carlo Methods. Jesús Fernández-Villaverde University of Pennsylvania

Transcription

1 Monte Carlo Methods Jesús Fernández-Villaverde University of Pennsylvania 1

2 Why Monte Carlo? From previous chapter, we want to compute: 1. Posterior distribution: π ³ θ Y T,i = f(y T θ,i)π (θ i) R Θ i f(y T θ,i)π (θ i) dθ 2. Marginal likelihood: P ³ Y T i = Z Θ i f(y T θ,i)π (θ i) dθ Difficult to asses analytically or even to approximate (Phillips, 1996). Resort to simulation. 2

3 A Bit of Historical Background and Intuition Metropolis and Ulam (1949) and Von Neuman (1951). Why the name Monte Carlo? Two silly examples: 1. Probability of getting a total of six points when rolling two (fair) dices. 2. Throwing darts at a graph. 3

4 Classical Monte Carlo Integration Assume we know how to generate draws from π ³ θ Y T,i. What does it mean to draw from π ³ θ Y T,i? Two Basic Questions: 1. Whydowewanttodoit? 2. Howdowedoit? 4

5 WhyDoWeDoIt? Basic intuition: Glivenko-Cantelli s Theorem. Let X 1,...,X n be iid as X with distribution function F.Letω be the outcome and F n (x, ω) be the empirical distribution function based on observations X 1 (ω),...,x n (ω). Then, as n, sup F n (x, ω) F (x) a.s. 0, <x< It can be generalized to include dependence: A.W. Van der Vaart and J.A. Wellner, Weak Convergence and Empirical Processes, Springer- Verlag,

6 Basic Result Let h (θ) be a function of interest: indicator, moment, etc... By the Law of Large Numbers: Z E π( Y T,i) [h (θ)] = Θ i h (θ) π ³ θ Y T,i dθ ' h m = 1 m mx j=1 h ³ θ j If Var π( Y T,i) [h (θ)] <, bythecentrallimittheorem: Var π( Y T,i) [h m] ' 1 m mx j=1 ³ h ³ θj hm 2 6

7 How Do We Do It? Large literature. Two good surveys: 1. Luc Devroye: Non-Uniform Random Variate Generation, Springer- Verlag, Available for free at 2. Christian Robert and George Casella, Monte Carlo Statistical Methods, 2nd ed, Springer-Verlag,

8 Random Draws? Natural sources of randomness. Difficult to use. Acomputer......but a computer is a deterministic machine! Von Neumann (1951): Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin. 8

9 Was Von Neumann Right? Let s us do a simple experiment. Let susstartmatlab,typeformat long, typerand. Did we get ? This does not look terribly random. Why is this number appearing? 9

10 Basic Building Block MATLAB uses highly non-linear iterative algorithms that look like random. That is why sometimes we talk of pseudo-random number generators. We will concentrate on draws from a uniform distribution. Other (standard and nonstandard) distributions come from manipulations of the uniform. 10

11 Goal Derive algorithms that, starting from some initial value and applying iterative methods, produce a sequence that: (Lehmer, 1951): 1. It is unpredictable for the uninitiated (relation with Chaotic dynamical systems). 2. It passes a battery of standard statistical tests of randomness (like Kolmogorov-Smirnov test, ARMA(p,q), etc). 11

12 Basic Component Multiplicative Congruential Generator: x i =(ax i 1 + b)mod(m +1) x i takesvalueson{0, 1,...,M}. Transformation into a generator on [0, 1] with: x i = ax i 1 + b M +1 x 0 is called the seed. 12

13 Choices of Parameters Period and performance depends crucially on a, b, and M. Pick a =13,c=0,M=31, and x 0 =1. Let us run badrandom.m. Historical bad examples: IBM RND from the 1960 s. 13

14 AGoodChoice A traditional choice: a =7 5 = 16807, c=0,m= Period bounded by M. 32 bits versus 64 bits hardware. You may want to be aware that there is something called IEEE standard for floating point arithmetic. Problems and alternatives. 14

15 Real Life You do not want to code your own random number generator. Matlab implements the state of the art: KISS by Marsaglia and Zaman (1991). What about a compiler in Fortran or C++? 15

16 Nonuniform Distributions In general we need something different from the uniform distribution. How do we move from a uniform draw to other distributions? Simple transformations for standard distributions. Foundations of commercial software. 16

17 Two Basic Approaches 1. Use transformations. 2. Use inverse method. Why are those approaches attractive? 17

18 An Example of Transformations I: Normal Distributions Box and Muller (1958). Let U 1 and U 2 two uniform variables, then: x = cos2πu 1 ( 2logU 2 ) 0.5 y = sin2πu 1 ( 2logU 2 ) 0.5 are independent normal variables. Problem: x and y fall in a spiral. 18

19 An Example of Transformations II: Multivariate Normal Distributions If x N (0,I), then is distributed as N µ, Σ 0 Σ. y = µ + Σx Σ can be the Cholesky decomposition of the matrix of variancescovariances. 19

20 An Example of Transformations III: Discrete Uniform We want to draw from x U{1,N}. Find 1/N. Draw from U U [0.1]. If u [0, 1/N ] x =1, if u [1/N, 2/N ] x =3,andsoon. 20

21 Inverse Method Conceptually general procedure for random variables on <. For a non-decreasionf function F on <, the generalized inverse of F, F, is the function F (u) =inf{x : F (x) u} Lemma: If U U [0.1], then the random variable F (U) hasthe distribution F. 21

22 Proof: For u [0.1] and x F ([0.1]), we satisfy: F ³ F (u) u and F (F (x)) x Therefore n (u, x) :F (u) x o = {(u, x) :F (x) u} and P ³ F (U) x = P (U F (x)) = F (x) 22

23 An Example of the Inverse Method Exponential distribution: x Exp(1). F (x) =1 e x. x = log (1 u). Thus, X = log U is exponential if U is uniform. 23

24 Problems Algebraic tricks and the inverse method are difficult to generalize. Why? Complicated, multivariate distributions. Often, we only have a numerical procedure to evaluate the distribution we want to sample from. We need more general methods. 24

25 Acceptance Sampling is called the target den- θ π ³ θ Y T,i sity. with support C. π ³ θ Y T,i z g (z) withsupportc 0 C. g is called the source density. We require: 1. Weknowhowtodrawfromg. 2. Condition: sup θ C π ³ θ Y T,i g (θ) = a< 25

26 Acceptance Sampling Algorithm Steps: 1. u U(0, 1). 2. θ g. 3. If u> π θ Y T,i ag(θ ) return to step Set θ m = θ. 26

27 Why Does Acceptance Sampling Work? Unconditional probability of moving from step 3 to 4: ³ θ Y T ³,i θ Y T,i ZC0 π g (θ) dθ = ag(θ) ZC0 π dθ = 1 a a Unconditional probability of moving from step 3 to 4 when θ A: Z π ³ θ Y T,i Z π ³ θ Y T,i g (θ) dθ = dθ = 1 A ag(θ) A a a ZA π ³ θ Y T,i dθ Dividing both expressions: RA π ³ θ Y T,i dθ 1 a 1 a 27 = ZA π ³ θ Y T,i dθ

28 An Example Target density: π ³ θ Y T,i min " exp à θ2 2! ³sin (6θ) 2 +3cos(θ) 2 sin (4θ) 2 +1, 0 # Source density: g (θ) 1 Ã! (2π) 0.5 exp θ2 2 Let s take a look: acceptance.m. 28

29 Problems of Acceptance Sampling Two issues: 1. We disregard a lot of draws. We want to minimize a. How? 2. We need to check π/g is bounded. Necessary condition: g has thicker tails than those of f. 3. We need to evaluate bound a. Difficult to do. Can we do better? Yes, through importance sampling. 29

30 Importance Sampling I Similar framework than in acceptance sampling: 1. θ π ³ θ Y T,i with support C. π ³ θ Y T,i density. is called the target 2. z g (z) withsupportc 0 C. g is called the source density. Note that we can write: Z E π( Y T,i) [h (θ)] = h (θ) π ³ θ Y T,i dθ = Θ i ZΘi h (θ) π ³ θ Y T,i g (θ) g (θ) dθ 30

31 Importance Sampling II If E π( Y T,i) [h (θ)] exists, a Law of Large Numbers holds: h (θ) ZΘi π ³ θ Y T,i g (θ) dθ ' h I m = 1 mx h ³ π ³ θ j Y θ j T,i g (θ) m g ³ θ j j=1 and E π( Y T,i) [h (θ)] ' hi m where n θ j o m j=1 are draws from g (θ) andπ θ j Y T,i sampling weights. g(θ j ) are the important 31

32 Importance Sampling III π θ Y T,i If E π(θ Y T,i) Geweke, 1989) and: g(θ) exists, a Central Limit Theorem applies (see µ m 1/2 h I m E π( Y T,i) [h (θ)] N ³ 0, σ 2 σ 2 ' 1 m mx j=1 ³ h ³ θj h I m 2 π ³ θ j Y T,i g ³ θ j 2 Where, again, n θ j o m j=1 are draws from g (θ). 32

33 Importance Sampling IV Notice that: σ 2 ' 1 m mx j=1 ³ h ³ θj h I m 2 π ³ θ j Y T,i g ³ θ j 2 Therefore, we want π θ Y T,i g(θ) to be almost flat. 33

34 Importance Sampling V Intuition: σ 2 is minimized when π ³ θ Y T,i drawing from π ³ θ j Y T,i. = g (θ).,i.e. we are Hint: we can use as g (θ) thefirst terms of a Taylor approximation to π ³ θ Y T,i. How do we compute the Taylor approximation? 34

35 Conditions for the existence of E π(θ Y T,i) π θ Y T,i g(θ) This has to be checked analytically. Asimplecondition: π θ Y T,i g(θ) has to be bounded. Some times, we label ω ³ θ Y T,i = π θ Y T,i. g(θ) 35

36 Normalizing Factor I Assume we do not know the normalizing constant for π ³ θ Y T,i g (θ). and Let s call the unnormalized densities: eπ ³ θ Y T,i and eg (θ). Then: E π( Y T,i) [h (θ)] = R Θ i h (θ) eπ ³ θ Y T,i dθ RΘ i eπ ³ θ Y T,i dθ = R Θ i h (θ) eπ θ Y T,i eg (θ) dθ eg(θ) R eπ θ Y T,i Θ i eg(θ) eg (θ) dθ 36

37 Normalizing Factor II Consequently: h I m = 1 m P mj=1 h ³ θ j π θj Y T,i 1 m g(θ j ) P mj=1 π θ j Y T,i g(θ j ) = P mj=1 h ³ θ j ω ³ θj Y T,i P mj=1 ω ³ θ j Y T,i and: σ 2 ' m P ³ ³ ³ ³ m j=1 h θj h I m 2 ω θj Y T,i 2 ³ Pmj=1 ω ³ θ j Y T,i 2 37

38 The Importance of the Behavior of ω ³ θ j Y T,i : ExampleI Assume that we know π ³ θ j Y T,i = t ν. Butwedonotknowhowtodrawfromit. InsteadwedrawfromN (0, 1). Why? Let s run normalt.m 38

39 Evaluate the mean of t v. Draw n θ j o m j=1 from N (0, 1). Let t v(θ j ) φ(θ j ) = ω ³ θ j. Evaluate mean = P mj=1 θ j ω ³ θ j m 39

40 Evaluate the variance of the estimated mean of t v. Compute: var est mean = P mj=1 ³ θj mean 2 ω ³ θj 2 m Note: difference between: 1. The variance of a function of interest. 2. The variance of the computed mean of the function of interest. 40

41 Estimation of the Mean of t v : importancenormal.m υ Est. Mean Est.ofVar.ofEst.Mean

42 The Importance of the Behavior of ω ³ θ j Y T,i : ExampleII Opposite case than before. Assume that we know π ³ θ j Y T,i = N (0, 1). Butwedonotknowhowtodrawfromit. Insteadwedrawfromt v. 42

43 Estimation of the Mean of N (0, 1): importancet.m t ν Est. Mean Est. of Var. of Est. Mean

44 A Procedure to Check How Good is the Important Sampling Function This procedure is due to Geweke. It is called Relative Numerical Efficiency (RNE). Firstnoticethatifg (θ) =π ³ θ Y T,i, we have that: σ 2 ' 1 m = 1 m mx j=1 mx j=1 ³ h ³ θj h I m 2 π ³ θ j Y T,i g ³ θ j 2 ³ ³ h θj h I m 2 ' Varπ( Y T,i) [h (θ)] = 44

45 A Procedure of Checking how Good is the important Sampling Function II Therefore, for a given g (θ), the RNE: RNE = Var π( Y T,i) σ 2 [h (θ)] If RNE closed to 1 the important sampling procedure is working properly. If RNE is very low, closed to 0, the procedure is not working as properly. 45

46 Estimation of the Mean of t v t ν RNE Estimation of the Mean of N (0, 1) t ν RNE

47 Important Sampling and Robustness of Priors Priors are researcher specific. Imagine researchers 1 and 2 are working with the same model, i.e. with the same likelihood function, f(y T θ, 1) = f(y T θ, 2). (Now 1 and 2 do not imply different models but different researchers) But they have different priors π (θ 1) 6= π (θ 2). Imagine that researcher 1 has draws from the her posterior distribution n θj o N j=1 π ³ θ Y T, 1. 47

48 A Simple Manipulation If researcher 2 wants to compute ZΘi h (θ) π ³ θ Y T, 2 dθ for any ` (θ), he does not need to recompute everything. Note that h (θ) π ZΘi ³ θ Y T, 2 dθ = h (θ) ZΘi π ³ θ Y T, 2 π ³ θ Y T, 1 π ³ θ Y T, 1 dθ = RΘ i h (θ) f(yt θ,2)π(θ 2) f(y T θ,1)π(θ 1) π ³ θ Y T, 1 dθ f(y RΘ T θ,2)π(θ 1) i f(y T θ,1)π(θ 1) π ³ θ Y T, 1 dθ = RΘ i h (θ) π(θ 2) π(θ 1) π ³ θ Y T, 1 dθ π(θ 2) RΘ i π(θ 1) π ³ θ Y T, 1 dθ 48

49 Importance Sampling Then: 1 m P mj=1 h ³ θ j π(θj 2) 1 m π(θ j 1) P mj=1 π(θ j 2) π(θ j 1) RΘ i h (θ) π(θ 2) π(θ 1) π ³ θ Y T, 1 dθ π(θ 2) RΘ i π(θ 1) π ³ θ Y T, 1 dθ = = P mj=1 h ³ θ j π(θj 2) π(θ j 1) P mj=1 π(θ j 2) π(θ j 1) h (θ) π ZΘi ³ θ Y T, 2 dθ Simple computation. Increased variance. 49