Sequential Monte Carlo and adaptive numerical integration
|
|
- Samson Daniel
- 5 years ago
- Views:
Transcription
1 Sequetial Mote Carlo ad adaptive umerical itegratio VLADIMIR M. IVANOV, MAXIM L. KORENEVSKY Departmet of Computer Sciece Sait-Petersburg State Polytechical Uiversity , Politehichesaya, 29, Sait-Petersburg RUSSIA Abstract: Paper presets theory of the Sequetial Mote Carlo method ad its applicatio for developmet of adaptive statistical multidimetioal itegratio algorithms. Theorem of Sequetial Mote Carlo covergece is give. Several covetioal variace reductio methods are used to develop adaptive itegratio methods. These methods accumulate data about itegrad durig executio ad use it to accelerate covergece of itegral estimates. Such a approach provides optimal covergece rates for may importat fuctioal classes while retais mai merits of covetioal Mote Carlo itegratio methods. Keywords: Sequetial Mote Carlo, adaptive itegratio, successive bisectio. 1 Itroductio Mote Carlo is oe of the most popular methods for multidimetioal itegratio. It is very simple for implemetatio, aturally successive ad allows to chec accuracy durig computatios. The other importat merit of Mote Carlo method is that its rate of covergece does ot deped o itegral dimesio. I fact, itegratio error is of the order O( 0.5 ), where is the umber of sample poits i which itegrad is evaluated. This is very differet from traditioal cubature rules, whose rate of covergece expoetially decreases with the icrease of dimesio. Itegratio accuracy ca be substatially icreased by use of various variace reductio methods, e.g. importace samplig, correlated ad cotrol variate samplig, atithetic variates, stratified samplig etc. [1], [2], [3]. May of them use available a priori data about itegrad to attai more rapid covergece. However, data of such id ca be also accumulated durig computatios ad effectively applied to accelerate covergece of itegratio process, i.e. algorithm ca adapt to the features of itegrad. The Sequetial Mote Carlo method [4] provides a coveiet framewor for desig ad ivestigatio of such adaptive itegratio methods. The rest of paper is orgaized as follows. Sectio 2 gives a brief descriptio of Mote Carlo itegratio ad some variace reductio techiques. Sequetial Mote Carlo method is stated ad ivestigated i sectio 3. Some geeral approaches to develop adaptive itegratio methods are outlied i sectio 4. Sectio 5 describes successive bisectio algorithm ad adaptive methods for itegratio of smooth fuctios. Discussio of some umerical experimets is give i sectio 6. 2 Mote Carlo itegratio The problem is to evaluate itegral J = f(x)dx, (1) over closed bouded domai of s-dimesioal euclidea space R s. Simplest (crude) Mote Carlo estimate of J is J = µ() f(x ), (2) where x are idepedet radom variables uiformly distributed over ad µ() is the measure of. Estimate J is ubiased ad its variace is V ar{j }= 1 µ() f 2 (x)dx J 2 = σ2. (3) This implyies (due to Chebyshev iequality) that error of itegratio for ay predefied cofidece level decreases as O( 0.5 ) ad this rate does ot deped o dimesio s. Itegratio process ca be orgaized i successful maer, ad simple accuracy chec based o sample variace ca be doe 1
2 to stop computatio as soo as required accuracy is attaied. Numerous variace reductio techiques are developed to reduce multiplier σ 2 i (3). The idea of cotrol variate samplig is to itegrate pricipal part of f(x) aalitycally ad apply Mote Carlo estimate to remaider oly. Ideed, let g(x) be a easily itegrable approximatio of f(x) (so-called easy approximatio ). The J ca be estimated as follows J = g(x)dx + µ() (f(x ) g(x )). (4) Evidetly, V ar{j } is defied by (3) with f(x) substituted by (f(x) g(x)), ad it teds to zero whe g(x) f(x). Therefore all the a priori data about f(x) ca be used to costruct easy approximatio g(x) carefully. Importace samplig is based o the observatio that oe eed to sample more poits ito those parts of where f(x) is greater i absolute value, i.e. samplig should be cotrolled by some probability desity fuctio (pdf) p(x) > 0. This iduce more geeral form of Mote Carlo estimate: J = 1 f(x ) p(x ) (5) where x are idepedet radom variables with pdf p(x). Variace of this estimate is as follows V ar{j } = 1 f 2 (x) p(x) dx J 2 = σ2. (6) Well-ow [2], that the least value of σ 2 is attaied whe p(x) is proportioal to f(x). Provided that f(x) is of costat sig, this p(x) eve reduces σ 2 to zero. Although exact choice of p(x) i such a maer is ot possible, approximate choice is ofte quite acceptable. So, let g(x) be a easily itegrable approximatio of f(x). The p(x) = g(x) g(x)dx = g(x) gives importace samplig pdf ad J = J g f(x ) g(x ) J g (7) gives importace samplig itegral estimate. Agai a priori data about itegrad ca be used to costruct importace samplig pdf. Both cotrol variate ad importace samplig ca be cosidered as special cases of geeral approach, offered by J.H.Halto [4] ad called Sequetial Mote Carlo. 3 Sequetial Mote Carlo Sequetial Mote Carlo method operates o two types of ubiased itegral estimates primary S ad secodary J related as follows: J = β S + (1 β )J 1, (8) where 0 β 1, β 1 = 1 are some umerical factors. Both primary ad secodary estimates deped o radom poits x sampled over accordig to some pdfs (may be depedet o ). Secodary estimates J ca be also expressed as J = α () S, α () = β j=+1 (1 β j ). Easy to see that crude Mote Carlo is the special case of Sequetial Mote Carlo correspodig to S = µ()f(x ), (9) while cotrol variate samplig correspods to S = g(x)dx + µ()(f(x ) g(x )) (10) ad importace samplig correspods to S = f(x ) p(x ). (11) I all cases S deped oly o x ad therefore idepedet from each other ad α () = β = 1. Oe ca assume that use of depedet primary estimates may provide some beefits ad it is really true. The mose useful results ca be obtaied for the case of depedet but ucorrelated primary estimates, i.e. E{(S i J)(S j J)} = 0 for i j. I this case the variace of secodary estimates is as follows: V ar{j } = α () D, D = E{ S J 2 }. 2
3 Here D are the variaces of S (estimatio is tae over radom variables x 1,..., x which S depeds o). Followig theorem provides some results about covergece of Sequetial Mote Carlo. Theorem 1 [5, 6] Let S are ucorrelated, their variaces ca be estimated as D = O( γ l δ ) for some costats γ, δ 0, ad let coefficiets β are chose as follows: The β = 1 + γ + γ for ay γ > γ 1. (12) 2 V ar{j } = O( γ 1 l δ ) ad J coverg to J with probability 1 for ay γ > 0. Theorem 1 shows that suitable choice of coefficiets β allows for secodary estimates to coverg oe order more rapidly that for primary oes. Moreover, it ca be show that ay other choice of coefficiets ca t implove order of V ar{j } decrease. Empirical estimatio of V ar{j } ca be orgaized i parallel with estimatio of itegral J that allows to chec accuracy of J durig computatios ad stop them as soo as required accuracy is attaied. 4 Adaptive itegratio methods The uderlyig idea of adaptive methods offered below is as follows: algorithm ca accumulate its owledge about itegrad to successively mae curret easy approximatio more precise ad successively decrease variaces of itegral estimate. To mae this more formal assume that alog with samplig radom poits x 1,..., x,... the sequece of easy itegrad approximatios is costructed, f 1 (x),..., f (x),... such that f (x) = f (x; x 1,..., x 1 ). (Thus, each approximatio depeds o the values of f(x) at all poits sampled earlier.) The primary estimates S = f (x)dx + µ()(f(x ) f (x )) (13) are direct aalogy to (10) used for covetioal corol variate samplig method. But as f (x) ted to f(x) variaces of S ted to zero, ad oe ca expect (due to theorem 1) that variaces of secodary estimates J will decrease more rapidly tha O( 1 ). Equatio (13) defies adaptive cotrol variate samplig method. Similarly, adaptive importace samplig method ca be itroduced. Now assume that itegrad f(x) ad all approximatios f (x) are strictly positive 1. The primary estimates S = f(x ) p (x ), p (x) = f (x) f (x)dx, (14) where x is sampled over accordig to pdf p (x) are direct aalogy to (11) used for covetioal importace samplig method. Agai, as f (x) ted to f(x), p (x) ted to the optimal oe ad variaces of S ted to zero. Metio of theorem 1 few lies above is valid oly if we state that estimates S from (13) or (14) are ucorrelated. Fortuately, i fact they are. It follows directly from the relatio E x {S x 1,..., x 1 } = J, where estimatio is coditioal over x uder fixed x 1,..., x 1 ad from the depedece of S oly o x i, i. Thus, theorem 1 is completely applicable. It ca easily be show [6], that D E x1,...,x 1 µ() (f(x) f (x)) 2 dx for adaptive cotrol variate samplig ad (f(x) f (x)) 2 D E x1,...,x 1 dx p (x) for adaptive importace samplig, i.e. variaces D ted to zero whe f (x) ted to f(x) i L 2 (). To use proposed adaptive methods oe should be able to costruct the sequece of approximatios f (x) o the base of values f(x i ) i the poits already sampled. There are may ways to do it. The first ad simplest oe was offered by Kulchitsy ad Srobotov [7] for oe-dimesioal problem ad is as follows. f (x) is chose as piecewisecostat fuctio o the, f (x i ) = f(x i ) for i < ad f (x) is costat betwee x i. It was show for adaptive importace samplig, that i this case V ar{j } = O( 3 ), i.e. method is much 1 This assumptio ca usually be easily satisfied by additio of suffitiely large positive costat to the itegrad. 3
4 more rapid tha covetioal o-adaptive importace samplig. Further [?], this result was geeralized to piecewisepolyomial approximatios. For oe-dimesioal itegrads of class C m (a) havig cotiuous derivatives up to order m all bouded by costat a, adaptive itegratio methods ca be costructed for which V ar{j } = O( 2m 1 ). 5 Successive bisectio Described approach to costruct approximatios i oe-dimesioal case ca be geeralized for multidimetioal itegratio. For simplicity we will ow cosider oly itegratio over hyperparallelepiped, more geeral case is addressed at the ed of this sectio. Let s cosider -th itegratio step, whe poits x 1,..., x 1 are sampled ad itegratio domai is divided ito N ooverlappig subdomais j, j = 1,..., N ad assume that f (x) is the piecewise approximatio of f(x) over this partitio of. Let s say that sequece of f (x) approximates f(x) with order l > 0 if there exists C > 0 such that f(x) f (x) C [ µ( j ) ] l x j, (15) for all > 0 ad j = 1,..., N. Approximatios that satisfy (15) ca be costructed for wide variety of fuctios. I particular, fuctios of s- dimesioal class C m (a) ca be approximated with the order l up to m/s. Provided (15), variaces D are estimated through the quatity 2 M 2l+1 N = E x1,...,x 1 [µ()] 2l+1 j=1 that ca be called as partitio momet of order (2l + 1) relative to x 1,..., x 1. Now, good geeralizatio of oe-dimesioal algorithm should provide both rapid decrease of M 2l+1 ad relatively slow icrease of N (otherwise computatioal load would be too large). Oe of possible ways to solve this miimax problem is successive bisectio method. Its idea is very simple: ew partitio is obtaied from the curret oe by bisectio of subdomai 2 For adaptive importace samplig oe should additioally assume, that p (x) are uiformly boded from below by some positive costat where ew sampled poit x falls ito. Bisectio is carried out alog the directio where subdomai to be divided is the most legthy. Clearly this way provides very moderate icrease of N (amely N = ). Moreover it ca be show [6] that for both adaptive cotrol variate samplig ad adaptive importace samplig M 2l+1 = O( 2l ). Thus, for fuctios of class C m (a) adaptive methods ca be costructed for which D{J } = O( 1 2m/s ). Bahvalov [9] showed that this order is the best possible for ay odetermiistic itegratio method that uses oly O() values of itegrad. Therefore, proposed adaptive methods are optimal o C m (a). Now what if itegratio domai is differet from hyperparallelepiped. Covetioal approach is to immerse it ito larger hyperparallelepiped ad set f(x) to be zero outside. O the oe had, i this case f(x) loses its smoothess i the boudary poits ad it caot be approximated with high order over all subdomais of partitio. But o the aother had, it is ot a big trouble, because boudary is the maifold of dimesio (s 1) ad therefore fractio of subdomais where approximatio is bad asymptotically decreases whe icreases. The most coveiet implemetatio of successive bisectio is to use biary tree of subdomais, each ode of which cotais subdomai coordiates ad itegral of curret approximatio over it. 6 Discussio Extesive umerical experimets were carried out [6] which approved theretical estimates of adaptive methods covergece rate. The mai coclusio of experimets is as follows. For small dimesios cubature rules are the most effective for itegratio, while for large dimesios the simplest Mote Carlo ad quasi-mote Carlo itegratios are most effective (maily due to their simplicity). But whe the dimesio is moderate (5-15) ad especially whe there are strog accuracy requiremets adaptive methods are the most useful ad coveiet. They are sequetial, asymptotically more rapid the simplest Mote-Carlo ad allows to chec accuracy easily durig computatio i cotrast to quasi-mote Carlo methods ad cubature rules. 4
5 Methods of itegratio domai divisio differet from successive bisectio ca be used. For example, divisio ca be fully determiistic if oe always bisects the largest subdomai. The asyptotic behaviour of partitio momets obtaied for successive bisectio is still valid. Adaptive sequetial Mote Carlo methods have also bee developed for sequeces of global itegrad approximatio [6]. I these methods f(x) was approximated by trucated Fourier series over some orthogoal basis. For the classes of fuctios expadable ito rapidly coverget trigoometric Fourier series ad Fourier-Haar series optimal or almost optimal rates of covergece were obtaied. [9] N.S.Bahvalov O the approximate calculatio of multiple itegrals, Bulleti of Moscow State Uiversity, No.4, 1959, pp.3 18 (i Russia). Refereces [1] S.M.Ermaov, G.A.Mihailov, Course of Statistical Modellig, Moscow, Naua, 1976 (i Russia). [2] I.M.Sobol, Numerical Mote Carlo Methods, Moscow, Naua, 1973 (i Russia). [3] W.H.Press, S.A.Teuolsy, W.T.Vettrlig, B.P.Flaery Numerical Recipes i C, 2-d editio, Cambridge Uiversity Press, [4] J.H.Halto, Sequetial Mote Carlo, Proc. Cambridge Philos. Soc., vol.58, No.1, 1962, pp [5] M.L.Koreevsy, Developmet of Adaptve- Statictical Methods for Defiite Itegrals Evaluatio, Ph.D. thesis, Sait-Petersburg State Techical Uiversity, 2000 (i Russia). [6] V.M.Ivaov, M.L.Koreevsy, Adaptive- Statistical Methods of Numerical Itegratio, Sait-Petersburg State Polytechical Uiversity, 2003 (i Russia). [7] O.Yu.Kulchitsy, S.V.Srobotov, Adaptive algorithm of Mote Carlo method for computig itegral characteristics of complex systems, Automatics ad telemechaics, No.6, 1986, pp (i Russia). [8] V.M.Ivaov, M.L.Koreevsii, O.Yu.Kul chitsii, Adaptive Schemes for the Mote Carlo Method of a Ehaced Accuracy, Dolady Mathematics (Proceedigs of the Russia Academy of Scieces), vol. 60, No.1, 1999, pp
Fall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More informationLecture 2: Monte Carlo Simulation
STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?
More informationMonte Carlo Integration
Mote Carlo Itegratio I these otes we first review basic umerical itegratio methods (usig Riema approximatio ad the trapezoidal rule) ad their limitatios for evaluatig multidimesioal itegrals. Next we itroduce
More information62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +
62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of
More informationStochastic Simulation
Stochastic Simulatio 1 Itroductio Readig Assigmet: Read Chapter 1 of text. We shall itroduce may of the key issues to be discussed i this course via a couple of model problems. Model Problem 1 (Jackso
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationA collocation method for singular integral equations with cosecant kernel via Semi-trigonometric interpolation
Iteratioal Joural of Mathematics Research. ISSN 0976-5840 Volume 9 Number 1 (017) pp. 45-51 Iteratioal Research Publicatio House http://www.irphouse.com A collocatio method for sigular itegral equatios
More informationLecture 7: Density Estimation: k-nearest Neighbor and Basis Approach
STAT 425: Itroductio to Noparametric Statistics Witer 28 Lecture 7: Desity Estimatio: k-nearest Neighbor ad Basis Approach Istructor: Ye-Chi Che Referece: Sectio 8.4 of All of Noparametric Statistics.
More informationBasics of Probability Theory (for Theory of Computation courses)
Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.
More informationComparison Study of Series Approximation. and Convergence between Chebyshev. and Legendre Series
Applied Mathematical Scieces, Vol. 7, 03, o. 6, 3-337 HIKARI Ltd, www.m-hikari.com http://d.doi.org/0.988/ams.03.3430 Compariso Study of Series Approimatio ad Covergece betwee Chebyshev ad Legedre Series
More informationRandom Walks on Discrete and Continuous Circles. by Jeffrey S. Rosenthal School of Mathematics, University of Minnesota, Minneapolis, MN, U.S.A.
Radom Walks o Discrete ad Cotiuous Circles by Jeffrey S. Rosethal School of Mathematics, Uiversity of Miesota, Mieapolis, MN, U.S.A. 55455 (Appeared i Joural of Applied Probability 30 (1993), 780 789.)
More informationSequential Monte Carlo Methods - A Review. Arnaud Doucet. Engineering Department, Cambridge University, UK
Sequetial Mote Carlo Methods - A Review Araud Doucet Egieerig Departmet, Cambridge Uiversity, UK http://www-sigproc.eg.cam.ac.uk/ ad2/araud doucet.html ad2@eg.cam.ac.uk Istitut Heri Poicaré - Paris - 2
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationClases 7-8: Métodos de reducción de varianza en Monte Carlo *
Clases 7-8: Métodos de reducció de variaza e Mote Carlo * 9 de septiembre de 27 Ídice. Variace reductio 2. Atithetic variates 2 2.. Example: Uiform radom variables................ 3 2.2. Example: Tail
More informationOutput Analysis and Run-Length Control
IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationMathematical Methods for Physics and Engineering
Mathematical Methods for Physics ad Egieerig Lecture otes Sergei V. Shabaov Departmet of Mathematics, Uiversity of Florida, Gaiesville, FL 326 USA CHAPTER The theory of covergece. Numerical sequeces..
More information6 Integers Modulo n. integer k can be written as k = qn + r, with q,r, 0 r b. So any integer.
6 Itegers Modulo I Example 2.3(e), we have defied the cogruece of two itegers a,b with respect to a modulus. Let us recall that a b (mod ) meas a b. We have proved that cogruece is a equivalece relatio
More informationLecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More informationMonte Carlo method and application to random processes
Mote Carlo method ad applicatio to radom processes Lecture 3: Variace reductio techiques (8/3/2017) 1 Lecturer: Eresto Mordecki, Facultad de Ciecias, Uiversidad de la República, Motevideo, Uruguay Graduate
More informationChapter 4. Fourier Series
Chapter 4. Fourier Series At this poit we are ready to ow cosider the caoical equatios. Cosider, for eample the heat equatio u t = u, < (4.) subject to u(, ) = si, u(, t) = u(, t) =. (4.) Here,
More informationSurveying the Variance Reduction Methods
Iteratioal Research Joural of Applied ad Basic Scieces 2013 Available olie at www.irjabs.com ISSN 2251-838X / Vol, 7 (7): 427-432 Sciece Explorer Publicatios Surveyig the Variace Reductio Methods Arash
More informationStudy the bias (due to the nite dimensional approximation) and variance of the estimators
2 Series Methods 2. Geeral Approach A model has parameters (; ) where is ite-dimesioal ad is oparametric. (Sometimes, there is o :) We will focus o regressio. The fuctio is approximated by a series a ite
More informationLast time: Moments of the Poisson distribution from its generating function. Example: Using telescope to measure intensity of an object
6.3 Stochastic Estimatio ad Cotrol, Fall 004 Lecture 7 Last time: Momets of the Poisso distributio from its geeratig fuctio. Gs () e dg µ e ds dg µ ( s) µ ( s) µ ( s) µ e ds dg X µ ds X s dg dg + ds ds
More informationLecture 8: Solving the Heat, Laplace and Wave equations using finite difference methods
Itroductory lecture otes o Partial Differetial Equatios - c Athoy Peirce. Not to be copied, used, or revised without explicit writte permissio from the copyright ower. 1 Lecture 8: Solvig the Heat, Laplace
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationNEW FAST CONVERGENT SEQUENCES OF EULER-MASCHERONI TYPE
UPB Sci Bull, Series A, Vol 79, Iss, 207 ISSN 22-7027 NEW FAST CONVERGENT SEQUENCES OF EULER-MASCHERONI TYPE Gabriel Bercu We itroduce two ew sequeces of Euler-Mascheroi type which have fast covergece
More informationCSE 527, Additional notes on MLE & EM
CSE 57 Lecture Notes: MLE & EM CSE 57, Additioal otes o MLE & EM Based o earlier otes by C. Grat & M. Narasimha Itroductio Last lecture we bega a examiatio of model based clusterig. This lecture will be
More informationBernoulli numbers and the Euler-Maclaurin summation formula
Physics 6A Witer 006 Beroulli umbers ad the Euler-Maclauri summatio formula I this ote, I shall motivate the origi of the Euler-Maclauri summatio formula. I will also explai why the coefficiets o the right
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationOn Random Line Segments in the Unit Square
O Radom Lie Segmets i the Uit Square Thomas A. Courtade Departmet of Electrical Egieerig Uiversity of Califoria Los Ageles, Califoria 90095 Email: tacourta@ee.ucla.edu I. INTRODUCTION Let Q = [0, 1] [0,
More informationCHAPTER 10 INFINITE SEQUENCES AND SERIES
CHAPTER 10 INFINITE SEQUENCES AND SERIES 10.1 Sequeces 10.2 Ifiite Series 10.3 The Itegral Tests 10.4 Compariso Tests 10.5 The Ratio ad Root Tests 10.6 Alteratig Series: Absolute ad Coditioal Covergece
More informationDiscrete Orthogonal Moment Features Using Chebyshev Polynomials
Discrete Orthogoal Momet Features Usig Chebyshev Polyomials R. Mukuda, 1 S.H.Og ad P.A. Lee 3 1 Faculty of Iformatio Sciece ad Techology, Multimedia Uiversity 75450 Malacca, Malaysia. Istitute of Mathematical
More informationSequences of Definite Integrals, Factorials and Double Factorials
47 6 Joural of Iteger Sequeces, Vol. 8 (5), Article 5.4.6 Sequeces of Defiite Itegrals, Factorials ad Double Factorials Thierry Daa-Picard Departmet of Applied Mathematics Jerusalem College of Techology
More informationLecture 3 The Lebesgue Integral
Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified
More informationThe standard deviation of the mean
Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider
More informationChapter 9: Numerical Differentiation
178 Chapter 9: Numerical Differetiatio Numerical Differetiatio Formulatio of equatios for physical problems ofte ivolve derivatives (rate-of-chage quatities, such as velocity ad acceleratio). Numerical
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationL = n i, i=1. dp p n 1
Exchageable sequeces ad probabilities for probabilities 1996; modified 98 5 21 to add material o mutual iformatio; modified 98 7 21 to add Heath-Sudderth proof of de Fietti represetatio; modified 99 11
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationMONTE CARLO VARIANCE REDUCTION METHODS
MONTE CARLO VARIANCE REDUCTION METHODS M. Ragheb /4/3. INTRODUCTION The questio arises of whether oe ca reduce the variace associated with the samplig of a radom variable? Ideed we ca, but we eed to be
More informationA statistical method to determine sample size to estimate characteristic value of soil parameters
A statistical method to determie sample size to estimate characteristic value of soil parameters Y. Hojo, B. Setiawa 2 ad M. Suzuki 3 Abstract Sample size is a importat factor to be cosidered i determiig
More informationMath 2784 (or 2794W) University of Connecticut
ORDERS OF GROWTH PAT SMITH Math 2784 (or 2794W) Uiversity of Coecticut Date: Mar. 2, 22. ORDERS OF GROWTH. Itroductio Gaiig a ituitive feel for the relative growth of fuctios is importat if you really
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More informationProduct measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.
Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the
More informationChapter 2 The Monte Carlo Method
Chapter 2 The Mote Carlo Method The Mote Carlo Method stads for a broad class of computatioal algorithms that rely o radom sampligs. It is ofte used i physical ad mathematical problems ad is most useful
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationENGI Series Page 6-01
ENGI 3425 6 Series Page 6-01 6. Series Cotets: 6.01 Sequeces; geeral term, limits, covergece 6.02 Series; summatio otatio, covergece, divergece test 6.03 Stadard Series; telescopig series, geometric series,
More informationMath 155 (Lecture 3)
Math 55 (Lecture 3) September 8, I this lecture, we ll cosider the aswer to oe of the most basic coutig problems i combiatorics Questio How may ways are there to choose a -elemet subset of the set {,,,
More informationWeighted Approximation by Videnskii and Lupas Operators
Weighted Approximatio by Videsii ad Lupas Operators Aif Barbaros Dime İstabul Uiversity Departmet of Egieerig Sciece April 5, 013 Aif Barbaros Dime İstabul Uiversity Departmet Weightedof Approximatio Egieerig
More informationON POINTWISE BINOMIAL APPROXIMATION
Iteratioal Joural of Pure ad Applied Mathematics Volume 71 No. 1 2011, 57-66 ON POINTWISE BINOMIAL APPROXIMATION BY w-functions K. Teerapabolar 1, P. Wogkasem 2 Departmet of Mathematics Faculty of Sciece
More informationSelf-normalized deviation inequalities with application to t-statistic
Self-ormalized deviatio iequalities with applicatio to t-statistic Xiequa Fa Ceter for Applied Mathematics, Tiaji Uiversity, 30007 Tiaji, Chia Abstract Let ξ i i 1 be a sequece of idepedet ad symmetric
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5
CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationSequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet
More informationTR/46 OCTOBER THE ZEROS OF PARTIAL SUMS OF A MACLAURIN EXPANSION A. TALBOT
TR/46 OCTOBER 974 THE ZEROS OF PARTIAL SUMS OF A MACLAURIN EXPANSION by A. TALBOT .. Itroductio. A problem i approximatio theory o which I have recetly worked [] required for its solutio a proof that the
More informationMath 113 Exam 3 Practice
Math Exam Practice Exam 4 will cover.-., 0. ad 0.. Note that eve though. was tested i exam, questios from that sectios may also be o this exam. For practice problems o., refer to the last review. This
More informationExponential Families and Bayesian Inference
Computer Visio Expoetial Families ad Bayesia Iferece Lecture Expoetial Families A expoetial family of distributios is a d-parameter family f(x; havig the followig form: f(x; = h(xe g(t T (x B(, (. where
More informationECE 901 Lecture 12: Complexity Regularization and the Squared Loss
ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality
More informationMa 4121: Introduction to Lebesgue Integration Solutions to Homework Assignment 5
Ma 42: Itroductio to Lebesgue Itegratio Solutios to Homework Assigmet 5 Prof. Wickerhauser Due Thursday, April th, 23 Please retur your solutios to the istructor by the ed of class o the due date. You
More informationAdvanced Analysis. Min Yan Department of Mathematics Hong Kong University of Science and Technology
Advaced Aalysis Mi Ya Departmet of Mathematics Hog Kog Uiversity of Sciece ad Techology September 3, 009 Cotets Limit ad Cotiuity 7 Limit of Sequece 8 Defiitio 8 Property 3 3 Ifiity ad Ifiitesimal 8 4
More informationNumerical Method for Blasius Equation on an infinite Interval
Numerical Method for Blasius Equatio o a ifiite Iterval Alexader I. Zadori Omsk departmet of Sobolev Mathematics Istitute of Siberia Brach of Russia Academy of Scieces, Russia zadori@iitam.omsk.et.ru 1
More informationMATH301 Real Analysis (2008 Fall) Tutorial Note #7. k=1 f k (x) converges pointwise to S(x) on E if and
MATH01 Real Aalysis (2008 Fall) Tutorial Note #7 Sequece ad Series of fuctio 1: Poitwise Covergece ad Uiform Covergece Part I: Poitwise Covergece Defiitio of poitwise covergece: A sequece of fuctios f
More informationSection A assesses the Units Numerical Analysis 1 and 2 Section B assesses the Unit Mathematics for Applied Mathematics
X0/70 NATIONAL QUALIFICATIONS 005 MONDAY, MAY.00 PM 4.00 PM APPLIED MATHEMATICS ADVANCED HIGHER Numerical Aalysis Read carefully. Calculators may be used i this paper.. Cadidates should aswer all questios.
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More information1 Duality revisited. AM 221: Advanced Optimization Spring 2016
AM 22: Advaced Optimizatio Sprig 206 Prof. Yaro Siger Sectio 7 Wedesday, Mar. 9th Duality revisited I this sectio, we will give a slightly differet perspective o duality. optimizatio program: f(x) x R
More informationA gentle introduction to Measure Theory
A getle itroductio to Measure Theory Gaurav Chadalia Departmet of Computer ciece ad Egieerig UNY - Uiversity at Buffalo, Buffalo, NY gsc4@buffalo.edu March 12, 2007 Abstract This ote itroduces the basic
More informationBull. Korean Math. Soc. 36 (1999), No. 3, pp. 451{457 THE STRONG CONSISTENCY OF NONLINEAR REGRESSION QUANTILES ESTIMATORS Seung Hoe Choi and Hae Kyung
Bull. Korea Math. Soc. 36 (999), No. 3, pp. 45{457 THE STRONG CONSISTENCY OF NONLINEAR REGRESSION QUANTILES ESTIMATORS Abstract. This paper provides suciet coditios which esure the strog cosistecy of regressio
More informationAccess to the published version may require journal subscription. Published with permission from: Elsevier.
This is a author produced versio of a paper published i Statistics ad Probability Letters. This paper has bee peer-reviewed, it does ot iclude the joural pagiatio. Citatio for the published paper: Forkma,
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationDetailed proofs of Propositions 3.1 and 3.2
Detailed proofs of Propositios 3. ad 3. Proof of Propositio 3. NB: itegratio sets are geerally omitted for itegrals defied over a uit hypercube [0, s with ay s d. We first give four lemmas. The proof of
More informationThe Growth of Functions. Theoretical Supplement
The Growth of Fuctios Theoretical Supplemet The Triagle Iequality The triagle iequality is a algebraic tool that is ofte useful i maipulatig absolute values of fuctios. The triagle iequality says that
More informationMath 312 Lecture Notes One Dimensional Maps
Math 312 Lecture Notes Oe Dimesioal Maps Warre Weckesser Departmet of Mathematics Colgate Uiversity 21-23 February 25 A Example We begi with the simplest model of populatio growth. Suppose, for example,
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More information( a) ( ) 1 ( ) 2 ( ) ( ) 3 3 ( ) =!
.8,.9: Taylor ad Maclauri Series.8. Although we were able to fid power series represetatios for a limited group of fuctios i the previous sectio, it is ot immediately obvious whether ay give fuctio has
More informationn n i=1 Often we also need to estimate the variance. Below are three estimators each of which is optimal in some sense: n 1 i=1 k=1 i=1 k=1 i=1 k=1
MATH88T Maria Camero Cotets Basic cocepts of statistics Estimators, estimates ad samplig distributios 2 Ordiary least squares estimate 3 3 Maximum lielihood estimator 3 4 Bayesia estimatio Refereces 9
More informationTHE KALMAN FILTER RAUL ROJAS
THE KALMAN FILTER RAUL ROJAS Abstract. This paper provides a getle itroductio to the Kalma filter, a umerical method that ca be used for sesor fusio or for calculatio of trajectories. First, we cosider
More information1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable
More informationEconomics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator
Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters
More informationCOMPUTING THE EULER S CONSTANT: A HISTORICAL OVERVIEW OF ALGORITHMS AND RESULTS
COMPUTING THE EULER S CONSTANT: A HISTORICAL OVERVIEW OF ALGORITHMS AND RESULTS GONÇALO MORAIS Abstract. We preted to give a broad overview of the algorithms used to compute the Euler s costat. Four type
More informationInformation-based Feature Selection
Iformatio-based Feature Selectio Farza Faria, Abbas Kazeroui, Afshi Babveyh Email: {faria,abbask,afshib}@staford.edu 1 Itroductio Feature selectio is a topic of great iterest i applicatios dealig with
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationUnbiased Estimation. February 7-12, 2008
Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
More informationCentral limit theorem and almost sure central limit theorem for the product of some partial sums
Proc. Idia Acad. Sci. Math. Sci. Vol. 8, No. 2, May 2008, pp. 289 294. Prited i Idia Cetral it theorem ad almost sure cetral it theorem for the product of some partial sums YU MIAO College of Mathematics
More information4.3 Growth Rates of Solutions to Recurrences
4.3. GROWTH RATES OF SOLUTIONS TO RECURRENCES 81 4.3 Growth Rates of Solutios to Recurreces 4.3.1 Divide ad Coquer Algorithms Oe of the most basic ad powerful algorithmic techiques is divide ad coquer.
More informationENGI 4421 Confidence Intervals (Two Samples) Page 12-01
ENGI 44 Cofidece Itervals (Two Samples) Page -0 Two Sample Cofidece Iterval for a Differece i Populatio Meas [Navidi sectios 5.4-5.7; Devore chapter 9] From the cetral limit theorem, we kow that, for sufficietly
More informationChapter 10: Power Series
Chapter : Power Series 57 Chapter Overview: Power Series The reaso series are part of a Calculus course is that there are fuctios which caot be itegrated. All power series, though, ca be itegrated because
More informationG. R. Pasha Department of Statistics Bahauddin Zakariya University Multan, Pakistan
Deviatio of the Variaces of Classical Estimators ad Negative Iteger Momet Estimator from Miimum Variace Boud with Referece to Maxwell Distributio G. R. Pasha Departmet of Statistics Bahauddi Zakariya Uiversity
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationx x x Using a second Taylor polynomial with remainder, find the best constant C so that for x 0,
Math Activity 9( Due with Fial Eam) Usig first ad secod Taylor polyomials with remaider, show that for, 8 Usig a secod Taylor polyomial with remaider, fid the best costat C so that for, C 9 The th Derivative
More informationGoodness-Of-Fit For The Generalized Exponential Distribution. Abstract
Goodess-Of-Fit For The Geeralized Expoetial Distributio By Amal S. Hassa stitute of Statistical Studies & Research Cairo Uiversity Abstract Recetly a ew distributio called geeralized expoetial or expoetiated
More informationNANYANG TECHNOLOGICAL UNIVERSITY SYLLABUS FOR ENTRANCE EXAMINATION FOR INTERNATIONAL STUDENTS AO-LEVEL MATHEMATICS
NANYANG TECHNOLOGICAL UNIVERSITY SYLLABUS FOR ENTRANCE EXAMINATION FOR INTERNATIONAL STUDENTS AO-LEVEL MATHEMATICS STRUCTURE OF EXAMINATION PAPER. There will be oe 2-hour paper cosistig of 4 questios.
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More information