1 Monte Carlo Solution of Integral Equations of the second kind Adam M. Johansen a.m.johansen@warwick.ac.uk www2.warwick.ac.uk/fac/sci/statistics/staff/academic/johansen/talks/ March 7th, 2011 MIRaW Monte Carlo Methods Day
2 Outline Background Importance Sampling IS Fredholm Equations of the 2nd Kind Sequential IS and 2nd Kind Fredholm Equations Methodology A Path-space Interpretation MCMC for 2nd Kind Fredholm Equations Results A Toy Example An Asset-Pricing Example
3 The Monte Carlo Method Consider estimating: I ϕ = E f [ϕx] = fxϕxdx Given X 1,..., X N iid f: lim N 1 N N ϕx i I ϕ i=1 Alteratively, approximate: and use fx = 1 N N δx x i 1=1 E f [ϕ X 1,..., X N ] E f [ϕ].
4 The Monte Carlo Method Consider estimating: I ϕ = E f [ϕx] = fxϕxdx Given X 1,..., X N iid f: lim N 1 N N ϕx i I ϕ i=1 Alteratively, approximate: and use fx = 1 N N δx x i 1=1 E f [ϕ X 1,..., X N ] E f [ϕ].
5 Importance Sampling If f g i.e. gx = 0 fx = 0: Iϕ = gx fx gx ϕxdx [ ] fx =E g ϕx gx }{{} wx Given X 1,..., X N iid g: lim N 1 N N wx i ϕx i I ϕ i=1 Alteratively, approximate: fx = 1 N N wx i δx X i 1=1 E f [ϕ X 1... X N ] E f [ϕ].
6 Importance Sampling If f g i.e. gx = 0 fx = 0: Iϕ = gx fx gx ϕxdx [ ] fx =E g ϕx gx }{{} wx Given X 1,..., X N iid g: lim N 1 N N wx i ϕx i I ϕ i=1 Alteratively, approximate: fx = 1 N N wx i δx X i 1=1 E f [ϕ X 1... X N ] E f [ϕ].
7 Fredholm Equations of the Second Kind Consider the equation: fx = E fykx, ydy + gx in which source gx and transition Kx, y are known, but fx is unknown. This is a Fredholm equation of the second kind. Finding fx is an inverse problem.
The Von Neumann Expansion We have the expansion: fx = fykx, ydy + gx E [ ] = fzky, zdx + gy Kx, ydy + gx E E. =gx + K n x, yfydy n=1 where: K 1 x, y =Kx, y K n x, y = E K n 1 x, zkz, ydz For convergence, it is sufficient that: gx + K n x, ygydy < n=1 8
9 A Sampling Approach Choose µ a pdf on E. Define E := E { } Let Mx, y be a Markov transition kernel on E such that: is absorbing Mx, = Pd and Mx, y > 0 whenever Kx, y > 0. The pair µ, M defines a Markov chain on E.
10 Sequential Importance Sampling SIS Algorithm Simulate N paths Calculate weights: W X i 0:k i = { } X i N until 0:k i Xi +1 i=1 k i +1 =. k i K X i 1 k 1,Xi k µ X i 0 k=1 M X i k 1,Xi k g X i 0 µ X i 0 g X i k i P d if k i 1, P d if k i = 0. Approximate fx 0 with: f x 0 = 1 N N W i=1 X i δ 0:k i x 0 X i 0
11 Von Neumann Representation Revisited Starting from fx 0 = gx 0 + K n x 0, x n fx n dx n n=1 E We have directly: n fx 0 =gx 0 + Kx p 1, x p gx n dx 1... dx n =f 0 x 0 + n=1 E n n=1 p=1 E n f n x 0:n dx 1... dx n with f n x 0:n := gx n n Kx p 1, x p. p=1
12 A Path-Space Interpretation of SIS Define: where πn, x 0:n =p n π n x 0 : n on F = k=0 {k} E k+1 p n =PX 0:n E n+1, X n+1 = = 1 P d n P d π n x 0:n = µx 0 n k=1 Mx k 1, x k 1 P d n Define also: Approximate with: fk, x 0:k =f 0 x 0 δ 0,k + f n x 0:n δ n,k n=1 ˆfx 0 = 1 N N i=1 fk i, X i 0:k i πk i, X i δx 0 X i 0 0:k i
13 Limitations of SIS Arbitary geometric distribution for k. Arbitary initial distribution. Importance weights 1 k i K µ M X i 0 k=1 X i k 1, Xi k X i k 1, Xi k g X i k i P d Resampling will not help.
14 Limitations of SIS Arbitary geometric distribution for k. Arbitary initial distribution. Importance weights 1 k i K µ M X i 0 k=1 X i k 1, Xi k X i k 1, Xi k g X i k i P d Resampling will not help.
15 Optimal Importance Distribution Minimize variance of absolute value of weights. Consider choosing πn, x 0:n = p k δ k,n π k x 0:k i=k π n x 0:n =c 1 n f n x 0:n c n = f n x 0:n dx 0:n E n+1 p n =c n /c c = c n. We have: n=0 f x 0 = c sgn f 0 x 0 π 0, x 0 + c sgn f n x 0:n π n, x 0:n dx 1:n. E n n=1
16 That s where the MCMC comes in Obtain a sample-approximation of π... using your favourite sampler, Reversible Jump MCMC 2, for instance Use the samples to approximate the optimal proposal. Combine with importance sampling. Estimate normalising constant, c.
17 A Simple RJMCMC Algorithm Initialization. Set k 1, X 1 randomly or deterministically. 0:k 1 Iteration i 2. Sample U U[0, 1] If U u k i 1 k i, x i Update Move. 0:k i Else If U u k i 1 + b k i 1: Else k i, x i 0:k i Birth Move. k i, x i 0:k i Death Move.
18 Very Simple Update Move Set k i = k i 1, sample J U { 0, 1,..., k i} and XJ q u X i 1 J,. With probability π k i, X i 1 min 1, 0:J 1, X J, Xi 1 π k i, X i 1 q 0:k i u set X i = X i 1 0:k i 0:J 1, X J, Xi 1, J+1:k i otherwise set X i 0:k i = X i 1 0:k i 1. Acceptance probabilities involve: π l, x 0:l π k, x 0:k = c lπ l x 0:l c k π k x 0:k = J+1:k i X i 1 J q u XJ, Xi 1 J, XJ f l x 0:l f k x 0:k.
19 Very Simple Update Move Set k i = k i 1, sample J U { 0, 1,..., k i} and XJ q u X i 1 J,. With probability π k i, X i 1 min 1, 0:J 1, X J, Xi 1 π k i, X i 1 q 0:k i u set X i = X i 1 0:k i 0:J 1, X J, Xi 1, J+1:k i otherwise set X i 0:k i = X i 1 0:k i 1. Acceptance probabilities involve: π l, x 0:l π k, x 0:k = c lπ l x 0:l c k π k x 0:k = J+1:k i X i 1 J q u XJ, Xi 1 J, XJ f l x 0:l f k x 0:k.
20 Very Simple Birth Move Sample J U { 0, 1,..., k i 1}, sample X J q b. With probability π k i 1 + 1, min 1, π X i 1 0:J 1, X J, Xi 1 d J:k i 1 k i 1 +1 k i 1, X i 1 0:k i 1 q b X J bk i 1 set k i = k i 1 + 1, X i 0:k = X i 1 0:J 1, X J, Xi 1, J:k i 1 otherwise set k i = k i 1, X i 0:k i = X i 1 0:k i 1.
21 Very Simple Death Move Sample J U { 0, 1,..., k i 1}. With probability π k i 1 1, min 1, π X i 1 0:J 1, Xi 1 J+1:k i 1 set k i = k i 1 1, X i 0:k i = q b k i 1, X i 1 d 0:k i 1 k i 1 X i 1 J X i 1 0:J 1, Xi 1, J+1:k i 1 otherwise set k i = k i 1, X i 0:k i = X i 1 0:k i 1. b k i 1 1
22 Estimating the Normalising Constant Estimate p n as: p n = 1 N Estimate directly, also, c 0 and hence n i=1 δ n,k i gxdx ĉ = c 0 / p 0 ĉ n =ĉ p n
23 Toy Example The simple algorithm was applied to fx = 1 The solution fx = expx. 0 1 expx y fydy + 2 } 3 {{} 3 expx. }{{} Kx,y gx Birth, death and update probabilities were set to 1/3. A uniform distribution over the unit interval was used for all proposals.
24 Point Estimation for the Toy Example 3.5 Point Estimates and their Standard Deviations 3 2.5 Truth N=100 N=1,000 N=10,000 fx 0 2 1.5 1 0.5 0.2 0 0.2 0.4 0.6 0.8 1 1.2 x 0 Fix x 0 and simulate everything else.
25 Estimating fx from Samples Given ˆfx = 1 N such that [ E N i=1 W i δx X i 0, ] ˆfxϕx = fxϕxdx Simple option: histogram ϕx = I A x. More subtle option: fx = Kx, yˆfydy + gx =gx + 1 N N i=1 W i Kx, X i 0.
26 Estimating fx from Samples Given ˆfx = 1 N such that [ E N i=1 W i δx X i 0, ] ˆfxϕx = fxϕxdx Simple option: histogram ϕx = I A x. More subtle option: fx = Kx, yˆfydy + gx =gx + 1 N N i=1 W i Kx, X i 0.
Estimated fx for the Toy Problem 3 Estimate 1 Estimate 2 Truth 2.5 2 1.5 1 0.5 0 0 0.2 0.4 0.6 0.8 1 27
28 An Asset-Pricing Problem The rational expectation pricing model cf. 4 requires: the price, V s of an asset in some state s E satisfies V s = πs + β V tpt sdt. E Where: πs denotes the return on investment, β is a discount factor and pt s is a Markov kernel which models the state evolution. Simple example: E =[0, 1] β =0.85 pt s = with a = 0.05, b = 0.85 and λ = 100. φ t as+b λ Φ 1 [as+b] λ Φ as+b λ
29 Estimated fx for asset pricing case. 3.5 3 2.5 2 1.5 1 0.5 0 0 0.2 0.4 0.6 0.8 1
30 Simulated Chain Lengths for Different Starting Points 0.6 X00 X11 0.5 0.4 0.3 0.2 0.1 0 0 2 4 6 8 10 12 14 16 18
31 Conclusions Many Monte Carlo methods can be viewed as importance sampling. Many problems can be recast as calculation of expectations. Approximate solution of Fredholm equations. Also applicable to Volterra Equations 3. Any Monte Carlo method could be used. Some room for further investigation of this method.
32 References [1] A. Doucet, A. M. Johansen, and V. B. Tadić. On solving integral equations using Markov Chain Monte Carlo. Applied Mathematics and Computation, 216:2869 2889, 2010. [2] P. J. Green. Reversible jump Markov Chain Monte Carlo computation and Bayesian model determination. Biometrika, 82:711 732, 1995. [3] G. W. Peters, A. M. Johansen, and A. Doucet. Simulation of the annual loss distribution in operational risk via Panjer recursions and Volterra integral equations for value at risk and expected shortfall estimation. Journal of Operational Risk, 23:29 58, Fall 2007. [4] J. Rust, J. F. Traub, and H. Wózniakowski. Is there a curse of dimensionality for contraction fixed points in the worst case? Econometrica, 701:285 329, 2002.