A Guaranteed Automatic Integration Library for Monte Carlo Simulation

Transcription

1 A Guaranteed Automatic Integration Library for Monte Carlo Simulation Lan Jiang Department of Applied Mathematics Illinois Institute of Technology Joint work with Prof. Fred J. Hickernell, Dr. Sou-cheng Choi, Yuhan Ding, Lluis Antoni Jimenez Rugama, Xin Tong, Yizhi Zhang and Xuan Zhou July 8, 2014

2 Guaranteed Automatic Integration Library(GAIL)

3 Guaranteed Automatic Integration Library(GAIL) Guaranteed Automatic Integration Library(GAIL): is a suite of algorithms for integration problems in one and many dimensions, whose answers are guaranteed to be correct. It contains four algorithms in its latest release[choi, Ding, Hickernell, Jiang, Zhang, MATLAB software, 2014]: meanmc g: Monte Carlo method for evaluating the mean of a random variable. cubmc g: Monte Carlo method for numerical integration. funappx g: univariate function recovery. integral g: trapezoidal rule approximation of one-dimensional integrals. The next release is scheduled in Sep. 2014, more algorithms will be added. My collaborators would also give talks during SIAM AM14, Fred Hickernell(Wed. 4:00) Yuhan Ding(Wed. 5:30) Llus Antoni Jimnez Rugama(Tue. 5:40) Yizhi Zhang(TR 9:30) Xuan Zhou(Fr. 8:30)

4 Guaranteed Automatic Integration Library(GAIL) Guaranteed Automatic Integration Library(GAIL): is a suite of algorithms for integration problems in one and many dimensions, whose answers are guaranteed to be correct. It contains four algorithms in its latest release[choi, Ding, Hickernell, Jiang, Zhang, MATLAB software, 2014]: meanmc g: Monte Carlo method for evaluating the mean of a random variable. cubmc g: Monte Carlo method for numerical integration. funappx g: univariate function recovery. integral g: trapezoidal rule approximation of one-dimensional integrals. The next release is scheduled in Sep. 2014, more algorithms will be added. My collaborators would also give talks during SIAM AM14, Fred Hickernell(Wed. 4:00) Yuhan Ding(Wed. 5:30) Llus Antoni Jimnez Rugama(Tue. 5:40) Yizhi Zhang(TR 9:30) Xuan Zhou(Fr. 8:30)

5 Guaranteed Automatic Integration Library(GAIL) Guaranteed Automatic Integration Library(GAIL): is a suite of algorithms for integration problems in one and many dimensions, whose answers are guaranteed to be correct. It contains four algorithms in its latest release[choi, Ding, Hickernell, Jiang, Zhang, MATLAB software, 2014]: meanmc g: Monte Carlo method for evaluating the mean of a random variable. cubmc g: Monte Carlo method for numerical integration. funappx g: univariate function recovery. integral g: trapezoidal rule approximation of one-dimensional integrals. The next release is scheduled in Sep. 2014, more algorithms will be added. My collaborators would also give talks during SIAM AM14, Fred Hickernell(Wed. 4:00) Yuhan Ding(Wed. 5:30) Llus Antoni Jimnez Rugama(Tue. 5:40) Yizhi Zhang(TR 9:30) Xuan Zhou(Fr. 8:30)

6 Guaranteed Automatic Integration Library(GAIL) Guaranteed Automatic Integration Library(GAIL): is a suite of algorithms for integration problems in one and many dimensions, whose answers are guaranteed to be correct. It contains four algorithms in its latest release[choi, Ding, Hickernell, Jiang, Zhang, MATLAB software, 2014]: meanmc g: Monte Carlo method for evaluating the mean of a random variable. cubmc g: Monte Carlo method for numerical integration. funappx g: univariate function recovery. integral g: trapezoidal rule approximation of one-dimensional integrals. The next release is scheduled in Sep. 2014, more algorithms will be added. My collaborators would also give talks during SIAM AM14, Fred Hickernell(Wed. 4:00) Yuhan Ding(Wed. 5:30) Llus Antoni Jimnez Rugama(Tue. 5:40) Yizhi Zhang(TR 9:30) Xuan Zhou(Fr. 8:30)

7 Guaranteed Automatic Integration Library(GAIL) Guaranteed Automatic Integration Library(GAIL): is a suite of algorithms for integration problems in one and many dimensions, whose answers are guaranteed to be correct. It contains four algorithms in its latest release[choi, Ding, Hickernell, Jiang, Zhang, MATLAB software, 2014]: meanmc g: Monte Carlo method for evaluating the mean of a random variable. cubmc g: Monte Carlo method for numerical integration. funappx g: univariate function recovery. integral g: trapezoidal rule approximation of one-dimensional integrals. The next release is scheduled in Sep. 2014, more algorithms will be added. My collaborators would also give talks during SIAM AM14, Fred Hickernell(Wed. 4:00) Yuhan Ding(Wed. 5:30) Llus Antoni Jimnez Rugama(Tue. 5:40) Yizhi Zhang(TR 9:30) Xuan Zhou(Fr. 8:30)

8 Guaranteed Automatic Integration Library(GAIL) Guaranteed Automatic Integration Library(GAIL): is a suite of algorithms for integration problems in one and many dimensions, whose answers are guaranteed to be correct. It contains four algorithms in its latest release[choi, Ding, Hickernell, Jiang, Zhang, MATLAB software, 2014]: meanmc g: Monte Carlo method for evaluating the mean of a random variable. cubmc g: Monte Carlo method for numerical integration. funappx g: univariate function recovery. integral g: trapezoidal rule approximation of one-dimensional integrals. The next release is scheduled in Sep. 2014, more algorithms will be added. My collaborators would also give talks during SIAM AM14, Fred Hickernell(Wed. 4:00) Yuhan Ding(Wed. 5:30) Llus Antoni Jimnez Rugama(Tue. 5:40) Yizhi Zhang(TR 9:30) Xuan Zhou(Fr. 8:30)

9 What is in GAIL Algorithms Unittests meanmc g cubmc g funappx g GAIL matlab Documentations Workouts OutputFiles Papers Utilities Third party integral g funmin g cubsobol g cublattice g and more

10 Absolute error criterion Relative error criterion Algorithms to get the reliable error estimation of Bernoulli random variables How to estimate ˆp in order to satisfy the following inequalities? Pr( ˆp p ε a ) 99% Absolute error criterion ( ˆp p Pr p ε r ) 99% Relative error criterion

11 Absolute error criterion Relative error criterion Why not using Central Limit Theorem? Central Limit Theorem is an asymptotic result and carries no guarantee. Pr( p ˆp n ɛ) 1 α as n N Φ 1 (1 α) 4ε By CLT 2 Our method, using Hoeffding s Inequality, carries probabilistic guarantees with finite sample sizes. Pr( p ˆp n ɛ) 1 α N log(2/α) 2ε By Hoeffding s Inequality 2 ratio of N the ratio of N for Hoeffdings and CLT Hoeffding/CLT alpha Take NCLT = N Hoeff = Φ 1 (1 α) 4ε 2 log(2/α) 2ε 2 The ratio is: r = N Hoeff /N CLT 2 log(2/α) Φ 1 (1 α).

12 Absolute error criterion Relative error criterion Absolute error criterion: Pr( ˆp p ε a ) 1 α User specifies: the absolute error tolerance ε a 0 the uncertainty α (0, 1) Using n = log(2/α) IID Bernoulli random samples to compute sample mean: ˆp = 1 n n i=1 Y i. Theorem 2ε 2 a Let Y be a Bernoulli random variable with mean p, it follows that using sample size n yields an estimate ˆp which satisfies the fixed width confidence interval condition: Proof. Follow directly by Hoeffding s Inequality. Pr( ˆp p ε a ) 1 α.

13 Absolute error criterion Relative error criterion Relative error criterion: Pr ( ˆp p /p ε r ) 1 α User specifies: The relative error tolerance, ε r > 0, The uncertainty, α (0, 1), 1 Set i = 1 and compute the sample average ˆp ni using n i = 4i log(α i) 2ε IID 2 r Bernoulli random samples that are independent of those used to compute ˆp n1,, ˆp ni 1, where α i = 1 (1 α/2) 2 i. 2 Compute ˆp L = (ˆp ni ε r 2 i ) + a highly probable lower bound on p. If ˆp L 2ε r2 i then ˆp L is large enough, go to step 3. Else, i = i+1, and go back to step 1. 3 Compute ˆp using n = log(α/4) 2(ˆp L ε r) IID Bernoulli random sample that is 2 independent of those used to compute ˆp n1,, ˆp ni. Terminate the algorithm. If the algorithm terminates, then the relative error criterion is satisfied.

14 Absolute error criterion Relative error criterion Theorem to guarantee the success of Let Y be a Bernoulli random variable with mean p, it follows that using sample size n yields an estimate ˆp which satisfies the fixed width confidence interval condition: ( ) ˆp p Pr ε r 1 α. p Proof. ( ) ˆp p Pr ε r Pr ( ˆp p ˆp L ε r & ˆp L p) p Pr ( ˆp p ˆp L ε r & ˆp ni ε r 2 i p i ) Pr ( ˆp p ˆp L ε r ) + Pr (ˆp ni ε r 2 i p i ) 1 by Pr(A B) Pr(A) + Pr(B) 1 (1 α/2) + (1 α/2) 2 i 1 By Hoeffding s Ineq and independance = 1 α i=1

15 Using 500 random samples with true p: log 10 (p) U[ 3, 1], we calculate the estimated ˆp to some specified, random chosen, absolute error tolerance and relative tolerance. The figures below shows the results: 10 0 The error / tolerance comparison for different p 10 0 The error / tolerance comparison for different p abserr/abstol relerr/reltol budget exceeded guaranteed true p abs budget exceeded guaranteed true p rel

16 Guarantee the upper bound on cost of. Theoretical research of meanmc g with relative error criterion. Algorithm implementation and GAIL development.

17 Acknowledgements I would like to express my special appreciation and thanks to my advisor Professor Fred J. Hickernel, and all other collaborators. I would like to thank my husband Xuan, my two kids Audrey and Alvin, you support and encourage me unconditionally. I would also like to express my thank to SIAM AM 2014 organizers for hosting this wonderful event. Thanks for your attention!

18 Relative error criterion Proof. As for each step n i = log(αi) 2(ε r2 i ) log(αi) 2 2(ε r2 i ) 1 e 2ni(εr2 i ) 2 1 α 2 i = (1 α/2) 2 i, apply one side Hoeffding s Inequality: Pr(p ˆp ni ε r 2 i ) 1 e 2ni(εr2 i ) 2 (1 α/2) 2 i Similarly, as n = log(α/4) 2(ˆp L ε r) log(α/4) 2 2(ˆp L ε r) 1 2e 2n(ˆp Lε r) 2 1 α/2, apply 2 two side Hoeffding s inequality again, Pr( p ˆp n ˆp L ε r ) 1 2e 2n(ˆp Lε r) 2 1 α/2

19 Tools needed to solve the problem With the help of Bernoulli and Hoeffding, I found a way to solve this problem and win the prize. A Bernoulli random variable Y is one that takes on the values 0 or 1 according to P (Y = 1) = p P (Y = 0) = 1 p Thus, E(Y ) = p and var(y ) = p(1 p). By Hoeffding s Inequality, if the random variable Y is bounded in [0, 1], and let Y 1,, Y n be IID observations such that E(Y i ) = p, then the following inequalities hold: P (p ˆp n + ε) e 2nε2 P (p ˆp n ε) e 2nε2 P ( ˆp n p ε) 2e 2nε2