Rare Event Simulation using Importance Sampling and Cross-Entropy p. 1

Size: px

Start display at page:

Download "Rare Event Simulation using Importance Sampling and Cross-Entropy p. 1"

Silas Nichols
5 years ago
Views:

1 Rare Event Simulation using Importance Sampling and Cross-Entropy Caitlin James Supervisor: Phil Pollett Rare Event Simulation using Importance Sampling and Cross-Entropy p. 1

2 500 Simulation of a population Population size X(t) Time t (years) Rare Event Simulation using Importance Sampling and Cross-Entropy p. 2

3 Outline A rare event problem Stochastic SIS Logistic Epidemic (SIS) Crude Monte Carlo Method (CMCM) Importance Sampling (IS) Cross-Entropy Method (CE) Comparison of simulation estimates Rare Event Simulation using Importance Sampling and Cross-Entropy p. 3

4 A rare event problem Denote X(t) as the size of the population at time t. Let N denote the maximum population size. We wish to estimate α = P(X(t) hits 0 before N). Rare Event Simulation using Importance Sampling and Cross-Entropy p. 4

5 Notation Suppose (X(t) : t 0) is a birth-death process on finite space X = {0, 1,..., N}. λ 0 λ 1 λ N 2 λ N N-1 N µ 1 µ 2 µ N 1 µ N Figure 1: Transition diagram of a finite-state birth-death process Rare Event Simulation using Importance Sampling and Cross-Entropy p. 5

6 Notation Suppose (X(t) : t 0) is a birth-death process on finite space X = {0, 1,..., N}. Let (X m, m = 0, 1,... ) denote the corresponding jump chain. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 5

7 Notation Suppose (X(t) : t 0) is a birth-death process on finite space X = {0, 1,..., N}. Let (X m, m = 0, 1,... ) denote the corresponding jump chain. Let A be the collection of all the sample paths of (X m ) & let A 0 be the collection of all the paths that hit 0 before N. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 5

8 Notation Suppose (X(t) : t 0) is a birth-death process on finite space X = {0, 1,..., N}. Let (X m, m = 0, 1,... ) denote the corresponding jump chain. Let A be the collection of all the sample paths of (X m ) & let A 0 be the collection of all the paths that hit 0 before N. Let X = (X 0, X 1,... ) be a random sample path of A and let A be the event {X A 0 }. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 5

9 Equation Then we can write α = P(A) = E f H(X) = where A H(x)f(x; u, P )µ(dx). H(X) is the indicator function of the rare event A defined by H(x) = { 1 if x A0 0 if x A 0. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 6

10 Equation Then we can write and α = P(A) = E f H(X) = A H(x)f(x; u, P )µ(dx). f(x; u, P ) = u(x 0 ) m 1 k=0 P (x k, x k+1 ). Rare Event Simulation using Importance Sampling and Cross-Entropy p. 7

11 Equation Then we can write and α = P(A) = E f H(X) = A H(x)f(x; u, P )µ(dx). f(x; u, P ) = u(x 0 ) m 1 k=0 P (x k, x k+1 ). u = (u(i) : i X ) is the initial distribution. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 7

12 Equation Then we can write and α = P(A) = E f H(X) = A H(x)f(x; u, P )µ(dx). f(x; u, P ) = u(x 0 ) m 1 k=0 P (x k, x k+1 ). u = (u(i) : i X ) is the initial distribution. P = (P (i, j) : i, j X ) is the one-step transition matrix of (X m ). Rare Event Simulation using Importance Sampling and Cross-Entropy p. 7

13 Stochastic SIS Logistic Epidemic The SIS model is a finite-state birth and death process for which the origin is an absorbing state. λ 1 λ N 2 λ N N-1 N µ 1 µ 2 µ N 1 µ N Figure 2: Transition diagram of SIS model Rare Event Simulation using Importance Sampling and Cross-Entropy p. 8

14 Stochastic SIS Logistic Epidemic The SIS model is a finite-state birth and death process for which the origin is an absorbing state. The rate of infection per contact is denoted by λ and µ denotes the per-capita death rate. λ 1 λ N 2 λ N N-1 N µ 1 µ 2 µ N 1 µ N Figure 2: Transition diagram of SIS model Rare Event Simulation using Importance Sampling and Cross-Entropy p. 8

15 Stochastic SIS Logistic Epidemic The non-zero transition rates are defined as λ i = λ 1 N i(n i) and µ i = µi. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 9

16 Stochastic SIS Logistic Epidemic The non-zero transition rates are defined as λ i = λ 1 N i(n i) and µ i = µi. The jump chain (X m ) has jump probabilities p i = and q i = (1 p i ) = λ(n i) µn + λ(n i) µn µn + λ(n i). Rare Event Simulation using Importance Sampling and Cross-Entropy p. 9

17 Crude Monte Carlo Method We can estimate our probability using CMCM. This involves simulating n replicates, X 1,..., X n, from f( ; u, P ) and setting ˆα = 1 n n k=1 H(X k ). We simulate our model until the process hits N or 0. We count 1 if the process hits 0. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 10

18 Crude Monte Carlo Method We can estimate our probability using CMCM. This involves simulating n replicates, X 1,..., X n, from f( ; u, P ) and setting ˆα = 1 n n k=1 H(X k ). The number of trials needed to get one successful trial has a geometric distribution with the expected value being 1/p, where p is the probability of a successful trial. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 10

19 CMCM example If we start in State 8 and we wish to estimate the probability of reaching 0 before 10, then we would require 1/α runs to see one successful path hit 0 before 10. (Failure is hitting 10 before 0.) Rare Event Simulation using Importance Sampling and Cross-Entropy p. 11

20 CMCM example If we start in State 8 and we wish to estimate the probability of reaching 0 before 10, then we would require 1/α runs to see one successful path hit 0 before 10. (Failure is hitting 10 before 0.) The exact value of α starting in state 8 is 1.18E-05. Thus we need around 1/1.18E-05 85, 000 runs to see our first path hit 0 before 10. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 11

21 CMCM example If we start in State 8 and we wish to estimate the probability of reaching 0 before 10, then we would require 1/α runs to see one successful path hit 0 before 10. (Failure is hitting 10 before 0.) The exact value of α starting in state 8 is 1.18E-05. Thus we need around 1/1.18E-05 85, 000 runs to see our first path hit 0 before 10. If we use CMCM, then we would require many more than 85, 000 runs to obtain a reasonable estimate for α. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 11

22 Importance Sampling ũ to be an alternative initial distribution Rare Event Simulation using Importance Sampling and Cross-Entropy p. 12

23 Importance Sampling ũ to be an alternative initial distribution P to be an alternative transition matrix Rare Event Simulation using Importance Sampling and Cross-Entropy p. 12

24 Importance Sampling ũ to be an alternative initial distribution P to be an alternative transition matrix g(x; ũ, P ) = ũ(x 0 ) m 1 k=0 P (x k, x k+1 ) Rare Event Simulation using Importance Sampling and Cross-Entropy p. 12

25 Importance Sampling ũ to be an alternative initial distribution P to be an alternative transition matrix g(x; ũ, P ) = ũ(x 0 ) m 1 k=0 P (x k, x k+1 ) The following conditions must also hold: u(i) > 0 implies ũ(i) > 0 and P (i, j) > 0 implies P (i, j) > 0. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 12

26 Importance Sampling Under the alternative measure g we can estimate α by f(x; u, P ) α = E f H(X) = H(x) A g(x; ũ, P ) g(x; ũ, P )µ(dx) ) = E g (H(X)L T (X; u, P, ũ, P ); T <, where Rare Event Simulation using Importance Sampling and Cross-Entropy p. 13

27 Importance Sampling Under the alternative measure g we can estimate α by f(x; u, P ) α = E f H(X) = H(x) A g(x; ũ, P ) g(x; ũ, P )µ(dx) ) = E g (H(X)L T (X; u, P, ũ, P ); T <, where L m (x; u, P, ũ, P ) = f(x; u, P ) g(x; ũ, P ) = u(x m 1 0) k=0 P (x k, x k+1 ) ũ(x 0 ) m 1 P k=0 (x k, x k+1 ). Rare Event Simulation using Importance Sampling and Cross-Entropy p. 13

28 Importance Sampling Under the alternative measure g we can estimate α by f(x; u, P ) α = E f H(X) = H(x) A g(x; ũ, P ) g(x; ũ, P )µ(dx) ) = E g (H(X)L T (X; u, P, ũ, P ); T <, where L m (x; u, P, ũ, P ) = with f(x; u, P ) g(x; ũ, P ) = u(x m 1 0) k=0 P (x k, x k+1 ) ũ(x 0 ) m 1 P k=0 (x k, x k+1 ). T = inf{m : X m = 0 or X m = N} ("Stopping Time") Rare Event Simulation using Importance Sampling and Cross-Entropy p. 13

29 Importance Sampling estimator We can estimate α using the importance sampling (IS) estimator, defined by ˆα = 1 n n i=1 H(X)L m (X; u, P, ũ, P ), where (recall) L m (x; u, P, ũ, P ) = f(x; u, P ) g(x; ũ, P ) = u(x m 1 0) k=0 P (x k, x k+1 ) ũ(x 0 ) m 1 P k=0 (x k, x k+1 ). Rare Event Simulation using Importance Sampling and Cross-Entropy p. 14

30 Optimal IS density Which change of measure gives the IS estimator with smallest variance? Rare Event Simulation using Importance Sampling and Cross-Entropy p. 15

31 Optimal IS density Which change of measure gives the IS estimator with smallest variance? The smallest variance is obtained when g = g, the optimal importance sampling density, given by g (x) = A H(x) f(x; u, P ) H(x) f(x; u, P )µ(dx). Rare Event Simulation using Importance Sampling and Cross-Entropy p. 15

32 Optimal IS density Which change of measure gives the IS estimator with smallest variance? The smallest variance is obtained when g = g, the optimal importance sampling density, given by g (x) = A H(x) f(x; u, P ) H(x) f(x; u, P )µ(dx). If H(x) 0, then g (x) = H(x)f(x; u, P ) α. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 15

33 Cross-Entropy Method The Kullback-Leibler Cross-Entropy (CE) measure defines a "distance" between two densities g and h D(g, h) = = g(x) ln g(x) h(x) µ(dx) g(x) ln g(x)µ(dx) g(x) ln h(x)µ(dx). The purpose of CE is to choose the IS density h such that the "distance" between the optimal IS density g and density h is as small as possible. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 16

34 Properties of CE 1. D(, ) is non-symmetric ie: D(g, h) D(h, g), thus D(g, h) is not a true distance between g and h in a formal sense, although 2. D(g, h) D(g, g) = 0. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 17

35 Restrict the IS density If we restrict the density to belong to some family F which contains the original density f(x; u, P ) = u(x 0 ) and the alternative density f(x; ũ, P ) = ũ(x 0 ) m 1 k=0 m 1 k=0 P (x k, x k+1 ) P (x k, x k+1 ) Rare Event Simulation using Importance Sampling and Cross-Entropy p. 18

36 CE optimisation problem Then the CE method aims to solve the parametric optimisation problem min (ũ, P ) D(g, f( ; ũ, P )), where (recall) g (x) = H(x)f( ; u, P ) α. Rare Event Simulation using Importance Sampling and Cross-Entropy p. 19

37 CE reference parameter Since f( ; u, P ) does not depend on (ũ, P ), minimising the CE distance between g and f( ; ũ, P ) is equivalent to maximising, with respect to (ũ, P ), H(x) f(x; u, P ) ln f(x; ũ, P )µ(dx) = E (u,p ) H(X) ln f(x; ũ, P ). Rare Event Simulation using Importance Sampling and Cross-Entropy p. 20

38 CE reference parameter Since f( ; u, P ) does not depend on (ũ, P ), minimising the CE distance between g and f( ; ũ, P ) is equivalent to maximising, with respect to (ũ, P ), H(x) f(x; u, P ) ln f(x; ũ, P )µ(dx) = E (u,p ) H(X) ln f(x; ũ, P ). Assuming H(X) 0, the optimal (ũ, P ) (with respect to CE) is the solution to (ũ, P ) = arg max (ũ, P ) E (u,p ) H(X) ln f(x; ũ, P ). Rare Event Simulation using Importance Sampling and Cross-Entropy p. 20

39 CE reference parameter Since f( ; u, P ) does not depend on (ũ, P ), minimising the CE distance between g and f( ; ũ, P ) is equivalent to maximising, with respect to (ũ, P ), H(x) f(x; u, P ) ln f(x; ũ, P )µ(dx) = E (u,p ) H(X) ln f(x; ũ, P ). Assuming H(X) 0, the optimal P (with respect to CE) is the solution to P = arg max P E (u,p ) H(X) ln f(x; u, P ). Rare Event Simulation using Importance Sampling and Cross-Entropy p. 20

40 CE reference parameter The optimal transition probability matrix is given by P (i, j) = E (u,p )H(X) k:x k =i 1 (X k+1 =j) E (u,p ) H(X) k:x k =i 1. We can approximate this optimal transition probability matrix by implementing IS to obtain: Rare Event Simulation using Importance Sampling and Cross-Entropy p. 21

41 CE reference parameter The optimal transition probability matrix is given by P (i, j) = E (u,p )H(X) k:x k =i 1 (X k+1 =j) E (u,p ) H(X) k:x k =i 1. We can approximate this optimal transition probability matrix by implementing IS to obtain: P l+1(i, j) = E (u, P l ) H(X)L m(x; u, P, u, P l ) k:x k =i 1 (X k+1 =j) E (u, Pl ) H(X)L m(x; u, P, u, P l ) k:x k =i 1 Rare Event Simulation using Importance Sampling and Cross-Entropy p. 21

42 CE reference parameter The optimal transition probability matrix is given by P (i, j) = E (u,p )H(X) k:x k =i 1 (X k+1 =j) E (u,p ) H(X) k:x k =i 1. We can approximate this optimal transition probability matrix by implementing IS to obtain: P l+1(i, j) X n X=X 1 H(X)L m(x; u, P, u, P l ) k:x k =i 1 (X k+1 =j) X n X=X 1 H(X)L m(x; u, P, u, P l ) k:x k =i 1, Rare Event Simulation using Importance Sampling and Cross-Entropy p. 21

43 Initial CE parameter Original parameters: p i = λ(n i) µn + λ(n i) and q i = (1 p i ) = µn µn + λ(n i). Initial change of measures: p i = µ(n i) λn + µ(n i) and q i = (1 p i ) = λn λn + µ(n i). Rare Event Simulation using Importance Sampling and Cross-Entropy p. 22

44 Estimation of the probability of reaching 0 before N exact CMC Importance Sampling estimation IS 95% CI Probability Initial state Rare Event Simulation using Importance Sampling and Cross-Entropy p. 23

45 Estimation of the probability of reaching 0 before N exact CMC Importance Sampling estimation IS 95% CI Probability Initial state Rare Event Simulation using Importance Sampling and Cross-Entropy p. 24

46 Comparison of simulation estimates N = 10, λ = 0.9, µ = 0.1, ρ = 0.11, X 0 = 8, No. of CE runs= 4 Sample Size Exact IS CMCM E E E E E E E E-05 0 Rare Event Simulation using Importance Sampling and Cross-Entropy p. 25

47 Comparison of CE runs N = 10, λ = 0.9, µ = 0.1, ρ = 0.11, X 0 = 8, Sample Size = 1000, Exact = 1.18E-05 CE run IS 2 StD p 1 p 5 p E E E E E E E E E E E E Rare Event Simulation using Importance Sampling and Cross-Entropy p. 26

48 Acknowledgements Phil Pollett Joshua Ross David Sirl Ben Gladwin Rare Event Simulation using Importance Sampling and Cross-Entropy p. 27

Cross Entropy. Owen Jones, Cardiff University. Cross Entropy. CE for rare events. CE for disc opt. TSP example. A new setting. Necessary.

Cross Entropy. Owen Jones, Cardiff University. Cross Entropy. CE for rare events. CE for disc opt. TSP example. A new setting. Necessary. Owen Jones, Cardiff University The Cross-Entropy (CE) method started as an adaptive importance sampling scheme for estimating rare event probabilities (Rubinstein, 1997), before being successfully applied