Introduction. A Dirichlet Form approach to MCMC Optimal Scaling. MCMC idea
|
|
- Francine Price
- 5 years ago
- Views:
Transcription
1 Introuction A Dirichlet Form approach to MCMC Optimal Scaling Markov chain Monte Carlo (MCMC quotes: Metropolis et al. (1953, running coe on the Los Alamos MANIAC: a feasible approach to statistical mechanics problems which are as yet not analytically soluble. Giacomo Zanella, Wilfri S. Kenall, an Mylène Béar. g.zanella@warwick.ac.uk, w.s.kenall@warwick.ac.uk, mylene.bear@umontreal.ca Supporte by EPSRC Research Grants EP/D002060, EP/K Arian Smith (circa 1990, you set the MCMC sampler going, then you go off to the pub. Charlie Geyer (circa 1996 on the uses of MCMC, If you can write own a moel, I can o likelihoo inference for it,..., whatever. Lonon Probability Seminar King s College Lonon Elizabeth Thompson (circa 2004, You shoul only use MCMC when all else has faile. 7th October 2016 If you have to use MCMC, how to make it work as well as possible? 1 3 Example: MCMC for Anglo-Saxon statistics Some historians conjecture that Anglo-Saxon placenames cluster by issimilar names. Zanella(2016,2015 uses MCMC: ata provies some support, resulting in useful clustering. MCMC iea Goal: estimate E = E π [h(x]. Metho: simulate ergoic Markov chain with stationary istribution π: use empirical estimate Ê n = 1 n0 +n n n=n 0 h(x n. (Much easier to apply theory if chain is reversible. Theory: Ê n E almost surely. 4 6
2 Varieties of MH MCMC Here is the famous Metropolis-Hastings recipe for rawing from a istribution with ensity f : Propose Y using conitional ensity q(y x; Accept/Reject move from X to Y, base on ratio f(y q(x Y / f(x q(y X Gaussian MHRW MCMC Simple Python coe for Gaussian MHRW MCMC, using normal an exponential from Numpy: Propose multivariate Gaussian step; Options: 1. Inepenence sampler: proposal q(y x = q(y oesn t epen on x; 2. Ranom walk (MHRW MCMC: proposal q(y x = q(y x behaves as a ranom walk; 3. Langevin MH MCMC (or MALA: proposal q(y x = q(y x λ gra log f rifts towars high target ensity f. Test whether to accept proposal by comparing exponential ranom variable with log MH ratio; Implement step if accepte (vector aition while not mcmc.stoppe(: z = normal(0, tau, size=mcmc.im if exponential( > mcmc.phi(mcmc.x + z-mcmc.phi(mcmc.x: mcmc.x += z mcmc.recor_result( We shall focus on MHRW MCMC with Gaussian proposals. 7 8 MHRW MCMC with Gaussian proposals (I MHRW MCMC with Gaussian proposals (II (smooth target, marginal exp( x 4 (smooth target, marginal exp( x 4 Target is given by 10 i.i.. coorinates. Target is given by 20 i.i.. coorinates: Scale parameter for proposal: τ = 0.01 is too small. Scale parameter for proposal: τ = 0.01 is clearly too small. 9 Acceptance ratio 98.5% 10 Acceptance ratio 96.7%
3 MCMC Optimal Scaling: classic result (I MHRW MCMC on (R, π π(x i = e φ(x i x i ; MH acceptance rule A ( = 0 or 1. X ( 0 = ( X 1,..., X X i ii π X ( 1 = (X 1 + A ( W 1,..., X + A ( W W i ii N(0, σ 2 Questions: (1 complexity as? (2 optimal σ? Theorem (Roberts, Gelman an Gilks, 1997 Given σ 2 = σ 2, Lipschitz φ, an finite E π [(φ 8 ], E π [(φ 4 ] {X ( t,1 } t Z where Z t = s(σ 1 2 Bt + s(σ φ (Z t /2 t. Answers: (1 mix in O( steps; (2 σ max = arg max σ s(σ. 11 MCMC Optimal Scaling: classic result (II How to maximize s(σ? Given I = E π [φ (X 2 ] an normal CDF Φ, s(σ = σ 2 2Φ( σ I 2 = σ 2 A(σ = 4 I ( Φ 1 ( A(σ 2 2 A(σ So σ max given by maximizing { asymptotic acceptance rate (Φ } A(σ max = arg max 1 A [0,1] ( A 2 2 A} Strengths: Establish complexity as ; Practical information on how to tune proposal; Does not epen on φ (CLT-type universality. Some weaknesses that we aress: (there are others Convergence of marginal rather than joint istribution Strong regularity assumptions: Lipschitz g, finite E [ (g 8], E [ (g 4] MCMC Optimal Scaling: classic result (III There is a wie range of extensions: for example, Langevin / MALA, for which the magic acceptance probability is (Roberts an Rosenthal 1998; Non-ientically istribute inepenent target coorinates (Béar 2007; Gibbs ranom fiels (Breyer an Roberts 2000; Infinite imensional ranom fiels (Mattingly, Pillai, an Stuart 2012; Markov chains on a hypercube (Roberts 1998; Aaptive MCMC; ajust online to optimize acceptance probability (Anrieu an Thoms 2008; Rosenthal All these buil on the s..e. approach of Roberts, Gelman, an Gilks (1997; hence regularity conitions ten to be severe (though see Durmus, Le Corff, Moulines, an Roberts Dirichlet forms an MCMC 1 Definition of Dirichlet form A (symmetric Dirichlet form E on a Hilbert space H is a bilinear function E(u, v, efine for any u, v D H, which satisfies: 1. D is a ense linear subspace of H; 2. E(u, v = E(v, u for u, v D, so E is symmetric; 3. E(u = E(u, u 0 for u D; 4. D is a Hilbert space uner the ( Sobolev inner prouct u, v + E(u, v; 5. If u D then u = (u 1 0 D, moreover E(u, u E(u, u. Relate to Markov processes if (quasi-regular.
4 Dirichlet forms an MCMC 2 Two examples 1. Dirichlet form obtaine from (re-scale MHRW MCMC: E (h = [ ( ] 2 E h(x ( 2 1 h(x( 0. (E can be viewe as the Dirichlet form arising from speeing up the MHRW MCMC by rate. 2. Heuristic infinite-imensional iffusion limit of this form uner scaling: E (h = s(σ 2 E π [ h 2]. Can we euce that the MHRW MCMC scales to look like the infinite-imensional iffusion, by showing that E converges to E? 16 Useful moes of convergence for Dirichlet forms 1. Gamma-convergence; E n Γ -converges to E if (Γ 1 E (h lim inf n E n (h n whenever h n h H; (Γ 2 For every h H there are h n h H such that E (h lim sup n E n (h n. 2. Mosco (1994 introuces stronger conitions; (M1 E (h lim inf n E n (h n whenever h n h weakly in H; (M2 For every h H there are h n h strongly in H such that E (h lim sup n E n (h n. 3. Mosco (1994, Theorem 2.4.1, Corollary 2.6.1: conitions (M1 an (M2 imply convergence of associate resolvent operators, an inee of associate semigroups. 4. Sun (1998 gives further conitions which imply weak convergence of the associate processes: these conitions are implie by existence of a finite constant C such that E n (h C( h 2 + E(h for all h H Results Theorem (Zanella, Kenall an Béar, 2016 Consier the Gaussian MHRW MCMC base on proposal variance σ 2 / with target π, where π = f x = e φ x. Suppose I = φ 2 f x < (finite Fisher information, an φ (x + v φ (x < κ max{ v γ, v α } for some κ > 0, 0 < γ < 1, an α > 1. Let E be the corresponing [ Dirichlet form scale as] above. E Mosco-converges to E 1 exp(n ( 1 2 σ 2 I, σ 2 I E, so corresponing L 2 semigroups also converge. Corollary Suppose in the above that φ is globally Lipschitz. The corresponingly scale processes exhibit weak convergence. 20 Lemma Methos of proof 1: a CLT result Uner the conitions of the Theorem, almost surely (in x with invariant measure π the log Metropolis-Hastings ratio converges weakly (in W as follows as : f (x i + σ W i log f (x i = ( φ(x i + σ W i φ(x i N ( 1 2 σ 2 I, σ 2 I. We may use this to euce the asymptotic acceptance rate of the MHRW MCMC sampler.
5 Key iea for CLT Methos of proof 2: establishing conition (M2 ( φ(x i + σ W i φ(x i φ (x i σ W i + = 1 ( σ W i φ (x i + σ W i 0 u φ (x i u. Conition implicitly on x for first 2.5 steps. 1. First summan converges weakly to N (0, σ 2 I. 2. Decompose [ variance of secon summan to euce σ W Var i ( ] 1 0 φ (x i + σ W i u φ (x i u Use [ Hoeffing s inequality then absolute expectations: ( ] E φ (x i + σ W i u φ (x i u 1 2 σ 2 I. σ W i 1 0 For every h L 2 (π, fin h n h (strongly in L 2 (π such that E (h lim sup n E n (h n. 1. Sufficient to consier case E (h <. 2. Fin sequence of smooth cyliner functions h n with compact cylinrical support, such that E (h E (h n 1/n. 3. Using smoothness etc, E m (h n E (h n as m. 4. Subsequences Methos of proof 3: establishing conition (M1 Conclusion If h n h weakly in L 2 (π, show E (h lim inf n E n (h n. 1. Set Ψ (h = 2 (h(x( 0 h(x( Integrate against test function ξ(x 1:N, W 1:N I(U < a(x 1:N, W 1:N for ξ smooth, compact support, U a Uniform(0, 1 ranom variable. Apply Cauchy-Schwarz. 3. Use integration by parts, careful analysis an conitions on φ. The Dirichlet form approach allows significant relaxation of conitions require for optimal scaling results; Nee to explore whether further relaxation can be obtaine (almost surely possible; Nee to explore evelopment beyon i.i.. targets; e.g. can regularity be similarly relaxe in more general ranom fiel settings? Can this be applie in iscrete Markov chain cases (c.f. Roberts 1998; Investigate applications to Aaptive MCMC
6 Anrieu, C. an J. Thoms (2008. A tutorial on aaptive MCMC. Statistics an Computing 18(4, Béar, M. (2007, January. Weak convergence of Metropolis algorithms for non-i.i.d. target istributions. Annals of Applie Probability 17(4, Breyer, L. A. an G. O. Roberts (2000. From Metropolis to iffusions : Gibbs states an optimal scaling. Stochastic Processes an their Applications 90(2, Durmus, A., S. Le Corff, E. Moulines, an G. O. Roberts (2016. Optimal scaling of the Ranom Walk Metropolis algorithm uner L p mean ifferentiability. arxiv Geyer, C. (1999. Likelihoo inference for spatial point processes. In O. E. Barnorff-Nielsen, WSK, an M. N. M. van Lieshout (Es., Stochastic Geometry: likelihoo an computation, Chapter 4, pp Boca Raton: Chapman & Hall/CRC. Hastings, W. K. (1970. Monte Carlo sampling methos using Markov chains an their applications. Biometrika 57, Mattingly, J. C., N. S. Pillai, an A. M. Stuart (2012. Diffusion limits of the ranom walk metropolis algorithm in high imensions. Annals of Applie Probability 22(3, Metropolis, N., A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, an E. Teller (1953, June. Equation of state calculations by fast computing machines. Journal Chemical Physics 21(6, Mosco, U. (1994. Composite meia an asymptotic Dirichlet forms. Journal of Functional Analysis 123(2, Roberts, G. O. (1998. Optimal Metropolis algorithms for prouct measures on the vertices of a hypercube. Stochastics an Stochastic Reports (June 2013, Roberts, G. O., A. Gelman, an W. Gilks (1997. Weak Convergence an Optimal Scaling of Ranom Walk Algorithms. The Annals of Applie Probability 7(1, Roberts, G. O. an J. S. Rosenthal (1998. Optimal scaling of iscrete approximations to Langevin iffusions. J. R. Statist. Soc. B 60(1, Rosenthal, J. S. (2011. Optimal Proposal Distributions an Aaptive MCMC. Hanbook of Markov Chain Monte Carlo (1,
7 Sun, W. (1998. Weak convergence of Dirichlet processes. Science in China Series A: Mathematics 41(1, Thompson, E. A. (2005. MCMC in the analysis of genetic ata on peigree. In WSK, F. Liang, an J.-S. Wang (Es., Markov chain Monte Carlo: Innovations an Applications, Chapter 5, pp Singapore: Worl Scientific. Zanella, G. (2016. Ranom Partition Moels an Complementary Clustering of Anglo-Saxon Placenames. Annals of Applie Statistics 9(4, Zanella, G., WSK, an M. Béar (2016. A Dirichlet Form approach to MCMC Optimal Scaling. arxiv , 22pp. Zanella, G. (2015. Bayesian Complementary Clustering, MCMC, an Anglo-Saxon Placenames. Ph thesis, University of Warwick
A Dirichlet Form approach to MCMC Optimal Scaling
A Dirichlet Form approach to MCMC Optimal Scaling Giacomo Zanella, Wilfrid S. Kendall, and Mylène Bédard. g.zanella@warwick.ac.uk, w.s.kendall@warwick.ac.uk, mylene.bedard@umontreal.ca Supported by EPSRC
More informationConvergence of Random Walks
Chapter 16 Convergence of Ranom Walks This lecture examines the convergence of ranom walks to the Wiener process. This is very important both physically an statistically, an illustrates the utility of
More informationLecture Introduction. 2 Examples of Measure Concentration. 3 The Johnson-Lindenstrauss Lemma. CS-621 Theory Gems November 28, 2012
CS-6 Theory Gems November 8, 0 Lecture Lecturer: Alesaner Mąry Scribes: Alhussein Fawzi, Dorina Thanou Introuction Toay, we will briefly iscuss an important technique in probability theory measure concentration
More informationMarkov Chain Monte Carlo (MCMC)
Markov Chain Monte Carlo (MCMC Dependent Sampling Suppose we wish to sample from a density π, and we can evaluate π as a function but have no means to directly generate a sample. Rejection sampling can
More informationYVES F. ATCHADÉ, GARETH O. ROBERTS, AND JEFFREY S. ROSENTHAL
TOWARDS OPTIMAL SCALING OF METROPOLIS-COUPLED MARKOV CHAIN MONTE CARLO YVES F. ATCHADÉ, GARETH O. ROBERTS, AND JEFFREY S. ROSENTHAL Abstract. We consier optimal temperature spacings for Metropolis-couple
More informationA Review of Multiple Try MCMC algorithms for Signal Processing
A Review of Multiple Try MCMC algorithms for Signal Processing Luca Martino Image Processing Lab., Universitat e València (Spain) Universia Carlos III e Mari, Leganes (Spain) Abstract Many applications
More informationExpected Value of Partial Perfect Information
Expecte Value of Partial Perfect Information Mike Giles 1, Takashi Goa 2, Howar Thom 3 Wei Fang 1, Zhenru Wang 1 1 Mathematical Institute, University of Oxfor 2 School of Engineering, University of Tokyo
More informationSINGULAR PERTURBATION AND STATIONARY SOLUTIONS OF PARABOLIC EQUATIONS IN GAUSS-SOBOLEV SPACES
Communications on Stochastic Analysis Vol. 2, No. 2 (28) 289-36 Serials Publications www.serialspublications.com SINGULAR PERTURBATION AND STATIONARY SOLUTIONS OF PARABOLIC EQUATIONS IN GAUSS-SOBOLEV SPACES
More informationA quick introduction to Markov chains and Markov chain Monte Carlo (revised version)
A quick introduction to Markov chains and Markov chain Monte Carlo (revised version) Rasmus Waagepetersen Institute of Mathematical Sciences Aalborg University 1 Introduction These notes are intended to
More informationarxiv:math/ v1 [math.pr] 3 Jul 2006
The Annals of Applie Probability 006, Vol. 6, No., 475 55 DOI: 0.4/050560500000079 c Institute of Mathematical Statistics, 006 arxiv:math/0607054v [math.pr] 3 Jul 006 OPTIMAL SCALING FOR PARTIALLY UPDATING
More informationTopic 7: Convergence of Random Variables
Topic 7: Convergence of Ranom Variables Course 003, 2016 Page 0 The Inference Problem So far, our starting point has been a given probability space (S, F, P). We now look at how to generate information
More informationLogarithmic spurious regressions
Logarithmic spurious regressions Robert M. e Jong Michigan State University February 5, 22 Abstract Spurious regressions, i.e. regressions in which an integrate process is regresse on another integrate
More informationSelf-normalized Martingale Tail Inequality
Online-to-Confience-Set Conversions an Application to Sparse Stochastic Banits A Self-normalize Martingale Tail Inequality The self-normalize martingale tail inequality that we present here is the scalar-value
More informationFunction Spaces. 1 Hilbert Spaces
Function Spaces A function space is a set of functions F that has some structure. Often a nonparametric regression function or classifier is chosen to lie in some function space, where the assume structure
More informationeqr094: Hierarchical MCMC for Bayesian System Reliability
eqr094: Hierarchical MCMC for Bayesian System Reliability Alyson G. Wilson Statistical Sciences Group, Los Alamos National Laboratory P.O. Box 1663, MS F600 Los Alamos, NM 87545 USA Phone: 505-667-9167
More informationAgmon Kolmogorov Inequalities on l 2 (Z d )
Journal of Mathematics Research; Vol. 6, No. ; 04 ISSN 96-9795 E-ISSN 96-9809 Publishe by Canaian Center of Science an Eucation Agmon Kolmogorov Inequalities on l (Z ) Arman Sahovic Mathematics Department,
More information1. Aufgabenblatt zur Vorlesung Probability Theory
24.10.17 1. Aufgabenblatt zur Vorlesung By (Ω, A, P ) we always enote the unerlying probability space, unless state otherwise. 1. Let r > 0, an efine f(x) = 1 [0, [ (x) exp( r x), x R. a) Show that p f
More informationMonte Carlo Methods with Reduced Error
Monte Carlo Methos with Reuce Error As has been shown, the probable error in Monte Carlo algorithms when no information about the smoothness of the function is use is Dξ r N = c N. It is important for
More informationBayesian Complementary Clustering, MCMC and Anglo-Saxon placenames
Bayesian Complementary Clustering, MCMC and Anglo-Saxon placenames Giacomo Zanella g.zanella@warwick.ac.uk Department of Statistics University of Warwick, Coventry, UK 20 May 2014 Overview 1. Motivation:
More informationRobust Forward Algorithms via PAC-Bayes and Laplace Distributions. ω Q. Pr (y(ω x) < 0) = Pr A k
A Proof of Lemma 2 B Proof of Lemma 3 Proof: Since the support of LL istributions is R, two such istributions are equivalent absolutely continuous with respect to each other an the ivergence is well-efine
More informationWEIGHTING A RESAMPLED PARTICLE IN SEQUENTIAL MONTE CARLO. L. Martino, V. Elvira, F. Louzada
WEIGHTIG A RESAMPLED PARTICLE I SEQUETIAL MOTE CARLO L. Martino, V. Elvira, F. Louzaa Dep. of Signal Theory an Communic., Universia Carlos III e Mari, Leganés (Spain). Institute of Mathematical Sciences
More informationHomework 2 Solutions EM, Mixture Models, PCA, Dualitys
Homewor Solutions EM, Mixture Moels, PCA, Dualitys CMU 0-75: Machine Learning Fall 05 http://www.cs.cmu.eu/~bapoczos/classes/ml075_05fall/ OUT: Oct 5, 05 DUE: Oct 9, 05, 0:0 AM An EM algorithm for a Mixture
More informationMarkov chain Monte Carlo
Markov chain Monte Carlo Karl Oskar Ekvall Galin L. Jones University of Minnesota March 12, 2019 Abstract Practically relevant statistical models often give rise to probability distributions that are analytically
More informationBrief introduction to Markov Chain Monte Carlo
Brief introduction to Department of Probability and Mathematical Statistics seminar Stochastic modeling in economics and finance November 7, 2011 Brief introduction to Content 1 and motivation Classical
More informationGeneralization of the persistent random walk to dimensions greater than 1
PHYSICAL REVIEW E VOLUME 58, NUMBER 6 DECEMBER 1998 Generalization of the persistent ranom walk to imensions greater than 1 Marián Boguñá, Josep M. Porrà, an Jaume Masoliver Departament e Física Fonamental,
More information3 The variational formulation of elliptic PDEs
Chapter 3 The variational formulation of elliptic PDEs We now begin the theoretical stuy of elliptic partial ifferential equations an bounary value problems. We will focus on one approach, which is calle
More informationKernel Sequential Monte Carlo
Kernel Sequential Monte Carlo Ingmar Schuster (Paris Dauphine) Heiko Strathmann (University College London) Brooks Paige (Oxford) Dino Sejdinovic (Oxford) * equal contribution April 25, 2016 1 / 37 Section
More informationComputing Exact Confidence Coefficients of Simultaneous Confidence Intervals for Multinomial Proportions and their Functions
Working Paper 2013:5 Department of Statistics Computing Exact Confience Coefficients of Simultaneous Confience Intervals for Multinomial Proportions an their Functions Shaobo Jin Working Paper 2013:5
More informationBayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
More informationA Search and Jump Algorithm for Markov Chain Monte Carlo Sampling. Christopher Jennison. Adriana Ibrahim. Seminar at University of Kuwait
A Search and Jump Algorithm for Markov Chain Monte Carlo Sampling Christopher Jennison Department of Mathematical Sciences, University of Bath, UK http://people.bath.ac.uk/mascj Adriana Ibrahim Institute
More informationKernel adaptive Sequential Monte Carlo
Kernel adaptive Sequential Monte Carlo Ingmar Schuster (Paris Dauphine) Heiko Strathmann (University College London) Brooks Paige (Oxford) Dino Sejdinovic (Oxford) December 7, 2015 1 / 36 Section 1 Outline
More informationRandom products and product auto-regression
Filomat 7:7 (013), 1197 103 DOI 10.98/FIL1307197B Publishe by Faculty of Sciences an Mathematics, University of Niš, Serbia Available at: http://www.pmf.ni.ac.rs/filomat Ranom proucts an prouct auto-regression
More informationDarboux s theorem and symplectic geometry
Darboux s theorem an symplectic geometry Liang, Feng May 9, 2014 Abstract Symplectic geometry is a very important branch of ifferential geometry, it is a special case of poisson geometry, an coul also
More informationIntroduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos & Aarti Singh Contents Markov Chain Monte Carlo Methods Goal & Motivation Sampling Rejection Importance Markov
More informationMonotonicity for excited random walk in high dimensions
Monotonicity for excite ranom walk in high imensions Remco van er Hofsta Mark Holmes March, 2009 Abstract We prove that the rift θ, β) for excite ranom walk in imension is monotone in the excitement parameter
More informationModelling and simulation of dependence structures in nonlife insurance with Bernstein copulas
Moelling an simulation of epenence structures in nonlife insurance with Bernstein copulas Prof. Dr. Dietmar Pfeifer Dept. of Mathematics, University of Olenburg an AON Benfiel, Hamburg Dr. Doreen Straßburger
More informationHastings-within-Gibbs Algorithm: Introduction and Application on Hierarchical Model
UNIVERSITY OF TEXAS AT SAN ANTONIO Hastings-within-Gibbs Algorithm: Introduction and Application on Hierarchical Model Liang Jing April 2010 1 1 ABSTRACT In this paper, common MCMC algorithms are introduced
More informationUniversity of Toronto Department of Statistics
Norm Comparisons for Data Augmentation by James P. Hobert Department of Statistics University of Florida and Jeffrey S. Rosenthal Department of Statistics University of Toronto Technical Report No. 0704
More informationAcute sets in Euclidean spaces
Acute sets in Eucliean spaces Viktor Harangi April, 011 Abstract A finite set H in R is calle an acute set if any angle etermine by three points of H is acute. We examine the maximal carinality α() of
More information7.1 Support Vector Machine
67577 Intro. to Machine Learning Fall semester, 006/7 Lecture 7: Support Vector Machines an Kernel Functions II Lecturer: Amnon Shashua Scribe: Amnon Shashua 7. Support Vector Machine We return now to
More informationA new proof of the sharpness of the phase transition for Bernoulli percolation on Z d
A new proof of the sharpness of the phase transition for Bernoulli percolation on Z Hugo Duminil-Copin an Vincent Tassion October 8, 205 Abstract We provie a new proof of the sharpness of the phase transition
More informationImproving Estimation Accuracy in Nonrandomized Response Questioning Methods by Multiple Answers
International Journal of Statistics an Probability; Vol 6, No 5; September 207 ISSN 927-7032 E-ISSN 927-7040 Publishe by Canaian Center of Science an Eucation Improving Estimation Accuracy in Nonranomize
More information05 The Continuum Limit and the Wave Equation
Utah State University DigitalCommons@USU Founations of Wave Phenomena Physics, Department of 1-1-2004 05 The Continuum Limit an the Wave Equation Charles G. Torre Department of Physics, Utah State University,
More informationarxiv: v4 [math.pr] 27 Jul 2016
The Asymptotic Distribution of the Determinant of a Ranom Correlation Matrix arxiv:309768v4 mathpr] 7 Jul 06 AM Hanea a, & GF Nane b a Centre of xcellence for Biosecurity Risk Analysis, University of Melbourne,
More informationExponential asymptotic property of a parallel repairable system with warm standby under common-cause failure
J. Math. Anal. Appl. 341 (28) 457 466 www.elsevier.com/locate/jmaa Exponential asymptotic property of a parallel repairable system with warm stanby uner common-cause failure Zifei Shen, Xiaoxiao Hu, Weifeng
More informationA simple tranformation of copulas
A simple tranformation of copulas V. Durrleman, A. Nikeghbali & T. Roncalli Groupe e Recherche Opérationnelle Créit Lyonnais France July 31, 2000 Abstract We stuy how copulas properties are moifie after
More informationREVERSIBILITY FOR DIFFUSIONS VIA QUASI-INVARIANCE. 1. Introduction We look at the problem of reversibility for operators of the form
REVERSIBILITY FOR DIFFUSIONS VIA QUASI-INVARIANCE OMAR RIVASPLATA, JAN RYCHTÁŘ, AND BYRON SCHMULAND Abstract. Why is the rift coefficient b associate with a reversible iffusion on R given by a graient?
More informationTensors, Fields Pt. 1 and the Lie Bracket Pt. 1
Tensors, Fiels Pt. 1 an the Lie Bracket Pt. 1 PHYS 500 - Southern Illinois University September 8, 2016 PHYS 500 - Southern Illinois University Tensors, Fiels Pt. 1 an the Lie Bracket Pt. 1 September 8,
More information6 Markov Chain Monte Carlo (MCMC)
6 Markov Chain Monte Carlo (MCMC) The underlying idea in MCMC is to replace the iid samples of basic MC methods, with dependent samples from an ergodic Markov chain, whose limiting (stationary) distribution
More information17 : Markov Chain Monte Carlo
10-708: Probabilistic Graphical Models, Spring 2015 17 : Markov Chain Monte Carlo Lecturer: Eric P. Xing Scribes: Heran Lin, Bin Deng, Yun Huang 1 Review of Monte Carlo Methods 1.1 Overview Monte Carlo
More informationESTIMATION OF INTEGRALS WITH RESPECT TO INFINITE MEASURES USING REGENERATIVE SEQUENCES
Applie Probability Trust (9 October 2014) ESTIMATION OF INTEGRALS WITH RESPECT TO INFINITE MEASURES USING REGENERATIVE SEQUENCES KRISHNA B. ATHREYA, Iowa State University VIVEKANANDA ROY, Iowa State University
More informationLower bounds on Locality Sensitive Hashing
Lower bouns on Locality Sensitive Hashing Rajeev Motwani Assaf Naor Rina Panigrahy Abstract Given a metric space (X, X ), c 1, r > 0, an p, q [0, 1], a istribution over mappings H : X N is calle a (r,
More informationQuantile function expansion using regularly varying functions
Quantile function expansion using regularly varying functions arxiv:705.09494v [math.st] 9 Aug 07 Thomas Fung a, an Eugene Seneta b a Department of Statistics, Macquarie University, NSW 09, Australia b
More informationSurvey Sampling. 1 Design-based Inference. Kosuke Imai Department of Politics, Princeton University. February 19, 2013
Survey Sampling Kosuke Imai Department of Politics, Princeton University February 19, 2013 Survey sampling is one of the most commonly use ata collection methos for social scientists. We begin by escribing
More informationWinter 2019 Math 106 Topics in Applied Mathematics. Lecture 9: Markov Chain Monte Carlo
Winter 2019 Math 106 Topics in Applied Mathematics Data-driven Uncertainty Quantification Yoonsang Lee (yoonsang.lee@dartmouth.edu) Lecture 9: Markov Chain Monte Carlo 9.1 Markov Chain A Markov Chain Monte
More informationTutorial on Maximum Likelyhood Estimation: Parametric Density Estimation
Tutorial on Maximum Likelyhoo Estimation: Parametric Density Estimation Suhir B Kylasa 03/13/2014 1 Motivation Suppose one wishes to etermine just how biase an unfair coin is. Call the probability of tossing
More informationLecture 1: Review of Basic Asymptotic Theory
Lecture 1: Instructor: Department of Economics Stanfor University Prepare by Wenbo Zhou, Renmin University Basic Probability Theory Takeshi Amemiya, Avance Econometrics, 1985, Harvar University Press.
More informationConvergence of Langevin MCMC in KL-divergence
Convergence of Langevin MCMC in KL-ivergence Xiang Cheng x.cheng@berkeley.eu an Peter Bartlett peter@berkeley.eu Eitor: Abstract Langevin iffusion is a commonly use tool for sampling from a given istribution.
More informationControlled sequential Monte Carlo
Controlled sequential Monte Carlo Jeremy Heng, Department of Statistics, Harvard University Joint work with Adrian Bishop (UTS, CSIRO), George Deligiannidis & Arnaud Doucet (Oxford) Bayesian Computation
More informationGLOBAL SOLUTIONS FOR 2D COUPLED BURGERS-COMPLEX-GINZBURG-LANDAU EQUATIONS
Electronic Journal of Differential Equations, Vol. 015 015), No. 99, pp. 1 14. ISSN: 107-6691. URL: http://eje.math.txstate.eu or http://eje.math.unt.eu ftp eje.math.txstate.eu GLOBAL SOLUTIONS FOR D COUPLED
More informationarxiv: v1 [math.pr] 4 Feb 2016
Mittag-Leffler Lévy Processes Arun Kumar an N. S. Upahye *Inian Statistical Institute, Chennai Center, Taramani, Chennai-636, Inia an **Department of Mathematics, Inian Institute of Technology Maras, Chennai
More informationSlice Sampling with Adaptive Multivariate Steps: The Shrinking-Rank Method
Slice Sampling with Adaptive Multivariate Steps: The Shrinking-Rank Method Madeleine B. Thompson Radford M. Neal Abstract The shrinking rank method is a variation of slice sampling that is efficient at
More informationVariance Bounding Markov Chains
Variance Bounding Markov Chains by Gareth O. Roberts * and Jeffrey S. Rosenthal ** (September 2006; revised April 2007.) Abstract. We introduce a new property of Markov chains, called variance bounding.
More informationA PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks
A PAC-Bayesian Approach to Spectrally-Normalize Margin Bouns for Neural Networks Behnam Neyshabur, Srinah Bhojanapalli, Davi McAllester, Nathan Srebro Toyota Technological Institute at Chicago {bneyshabur,
More informationON MULTIPLE TRY SCHEMES AND THE PARTICLE METROPOLIS-HASTINGS ALGORITHM
ON MULTIPLE TRY SCHEMES AN THE PARTICLE METROPOLIS-HASTINGS ALGORITHM L. Martino, F. Leisen, J. Coraner University of Helsinki, Helsinki (Finlan). University of Kent, Canterbury (UK). ABSTRACT Markov Chain
More informationMarkov chain Monte Carlo
Markov chain Monte Carlo Peter Beerli October 10, 2005 [this chapter is highly influenced by chapter 1 in Markov chain Monte Carlo in Practice, eds Gilks W. R. et al. Chapman and Hall/CRC, 1996] 1 Short
More informationModeling of Dependence Structures in Risk Management and Solvency
Moeling of Depenence Structures in Risk Management an Solvency University of California, Santa Barbara 0. August 007 Doreen Straßburger Structure. Risk Measurement uner Solvency II. Copulas 3. Depenent
More informationA Short Note on Self-Similar Solution to Unconfined Flow in an Aquifer with Accretion
Open Journal o Flui Dynamics, 5, 5, 5-57 Publishe Online March 5 in SciRes. http://www.scirp.org/journal/oj http://x.oi.org/.46/oj.5.57 A Short Note on Sel-Similar Solution to Unconine Flow in an Aquier
More informationTractability results for weighted Banach spaces of smooth functions
Tractability results for weighte Banach spaces of smooth functions Markus Weimar Mathematisches Institut, Universität Jena Ernst-Abbe-Platz 2, 07740 Jena, Germany email: markus.weimar@uni-jena.e March
More informationSome Examples. Uniform motion. Poisson processes on the real line
Some Examples Our immeiate goal is to see some examples of Lévy processes, an/or infinitely-ivisible laws on. Uniform motion Choose an fix a nonranom an efine X := for all (1) Then, {X } is a [nonranom]
More informationMath 456: Mathematical Modeling. Tuesday, April 9th, 2018
Math 456: Mathematical Modeling Tuesday, April 9th, 2018 The Ergodic theorem Tuesday, April 9th, 2018 Today 1. Asymptotic frequency (or: How to use the stationary distribution to estimate the average amount
More informationMonte Carlo methods for sampling-based Stochastic Optimization
Monte Carlo methods for sampling-based Stochastic Optimization Gersende FORT LTCI CNRS & Telecom ParisTech Paris, France Joint works with B. Jourdain, T. Lelièvre, G. Stoltz from ENPC and E. Kuhn from
More informationPROBABILITY THEORY EXPONENTIAL FUNCTIONAL OF LÉVY PROCESSES: GENERALIZED WEIERSTRASS PRODUCTS AND WIENER-HOPF FACTORIZATION P. PATIE AND M.
PROBABILITY THEORY EXPONENTIAL FUNCTIONAL OF LÉVY PROCESSES: GENERALIZED WEIERSTRASS PRODUCTS AND WIENER-HOPF FACTORIZATION P. PATIE AND M. SAVOV Abstract. In this note, we state a representation of the
More informationarxiv: v1 [math-ph] 2 May 2016
NONLINEAR HEAT CONDUCTION EQUATIONS WITH MEMORY: PHYSICAL MEANING AND ANALYTICAL RESULTS PIETRO ARTALE HARRIS 1 AND ROBERTO GARRA arxiv:165.576v1 math-ph] May 16 Abstract. We stuy nonlinear heat conuction
More informationSequential Monte Carlo Methods in High Dimensions
Sequential Monte Carlo Methods in High Dimensions Alexandros Beskos Statistical Science, UCL Oxford, 24th September 2012 Joint work with: Dan Crisan, Ajay Jasra, Nik Kantas, Andrew Stuart Imperial College,
More informationChaos, Solitons and Fractals Nonlinear Science, and Nonequilibrium and Complex Phenomena
Chaos, Solitons an Fractals (7 64 73 Contents lists available at ScienceDirect Chaos, Solitons an Fractals onlinear Science, an onequilibrium an Complex Phenomena journal homepage: www.elsevier.com/locate/chaos
More informationSome Results on the Ergodicity of Adaptive MCMC Algorithms
Some Results on the Ergodicity of Adaptive MCMC Algorithms Omar Khalil Supervisor: Jeffrey Rosenthal September 2, 2011 1 Contents 1 Andrieu-Moulines 4 2 Roberts-Rosenthal 7 3 Atchadé and Fort 8 4 Relationship
More informationWUCHEN LI AND STANLEY OSHER
CONSTRAINED DYNAMICAL OPTIMAL TRANSPORT AND ITS LAGRANGIAN FORMULATION WUCHEN LI AND STANLEY OSHER Abstract. We propose ynamical optimal transport (OT) problems constraine in a parameterize probability
More informationSession 3A: Markov chain Monte Carlo (MCMC)
Session 3A: Markov chain Monte Carlo (MCMC) John Geweke Bayesian Econometrics and its Applications August 15, 2012 ohn Geweke Bayesian Econometrics and its Session Applications 3A: Markov () chain Monte
More informationIntroduction to the Vlasov-Poisson system
Introuction to the Vlasov-Poisson system Simone Calogero 1 The Vlasov equation Consier a particle with mass m > 0. Let x(t) R 3 enote the position of the particle at time t R an v(t) = ẋ(t) = x(t)/t its
More informationSolution to the exam in TFY4230 STATISTICAL PHYSICS Wednesday december 1, 2010
NTNU Page of 6 Institutt for fysikk Fakultet for fysikk, informatikk og matematikk This solution consists of 6 pages. Solution to the exam in TFY423 STATISTICAL PHYSICS Wenesay ecember, 2 Problem. Particles
More informationHigh-Dimensional p-norms
High-Dimensional p-norms Gérar Biau an Davi M. Mason Abstract Let X = X 1,...,X be a R -value ranom vector with i.i.. components, an let X p = j=1 X j p 1/p be its p-norm, for p > 0. The impact of letting
More informationConvergence rates of moment-sum-of-squares hierarchies for optimal control problems
Convergence rates of moment-sum-of-squares hierarchies for optimal control problems Milan Kora 1, Diier Henrion 2,3,4, Colin N. Jones 1 Draft of September 8, 2016 Abstract We stuy the convergence rate
More informationAnalysis IV, Assignment 4
Analysis IV, Assignment 4 Prof. John Toth Winter 23 Exercise Let f C () an perioic with f(x+2) f(x). Let a n f(t)e int t an (S N f)(x) N n N then f(x ) lim (S Nf)(x ). N a n e inx. If f is continuously
More informationWELL-POSEDNESS OF A POROUS MEDIUM FLOW WITH FRACTIONAL PRESSURE IN SOBOLEV SPACES
Electronic Journal of Differential Equations, Vol. 017 (017), No. 38, pp. 1 7. ISSN: 107-6691. URL: http://eje.math.txstate.eu or http://eje.math.unt.eu WELL-POSEDNESS OF A POROUS MEDIUM FLOW WITH FRACTIONAL
More informationSYMPLECTIC GEOMETRY: LECTURE 3
SYMPLECTIC GEOMETRY: LECTURE 3 LIAT KESSLER 1. Local forms Vector fiels an the Lie erivative. A vector fiel on a manifol M is a smooth assignment of a vector tangent to M at each point. We think of M as
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
More informationSYSTEMS OF DIFFERENTIAL EQUATIONS, EULER S FORMULA. where L is some constant, usually called the Lipschitz constant. An example is
SYSTEMS OF DIFFERENTIAL EQUATIONS, EULER S FORMULA. Uniqueness for solutions of ifferential equations. We consier the system of ifferential equations given by x = v( x), () t with a given initial conition
More informationChapter 2 Lagrangian Modeling
Chapter 2 Lagrangian Moeling The basic laws of physics are use to moel every system whether it is electrical, mechanical, hyraulic, or any other energy omain. In mechanics, Newton s laws of motion provie
More informationComputational Complexity of Metropolis-Hastings Methods in High Dimensions
Computational Complexity of Metropolis-Hastings Methods in High Dimensions Alexandros Beskos and Andrew Stuart Abstract This article contains an overview of the literature concerning the computational
More informationA Spectral Method for the Biharmonic Equation
A Spectral Metho for the Biharmonic Equation Kenall Atkinson, Davi Chien, an Olaf Hansen Abstract Let Ω be an open, simply connecte, an boune region in Ê,, with a smooth bounary Ω that is homeomorphic
More informationSolutions to Math 41 Second Exam November 4, 2010
Solutions to Math 41 Secon Exam November 4, 2010 1. (13 points) Differentiate, using the metho of your choice. (a) p(t) = ln(sec t + tan t) + log 2 (2 + t) (4 points) Using the rule for the erivative of
More informationRobust Low Rank Kernel Embeddings of Multivariate Distributions
Robust Low Rank Kernel Embeings of Multivariate Distributions Le Song, Bo Dai College of Computing, Georgia Institute of Technology lsong@cc.gatech.eu, boai@gatech.eu Abstract Kernel embeing of istributions
More informationThe simple slice sampler is a specialised type of MCMC auxiliary variable method (Swendsen and Wang, 1987; Edwards and Sokal, 1988; Besag and Green, 1
Recent progress on computable bounds and the simple slice sampler by Gareth O. Roberts* and Jerey S. Rosenthal** (May, 1999.) This paper discusses general quantitative bounds on the convergence rates of
More informationOptimal scaling for partially updating MCMC algorithms. Neal, Peter and Roberts, Gareth. MIMS EPrint:
Optimal scaling for partially upating MCMC algorithms Neal, Peter an Roberts, Gareth 006 MIMS EPrint: 007.93 Manchester Institute for Mathematical Sciences School of Mathematics The University of Manchester
More informationEnergy behaviour of the Boris method for charged-particle dynamics
Version of 25 April 218 Energy behaviour of the Boris metho for charge-particle ynamics Ernst Hairer 1, Christian Lubich 2 Abstract The Boris algorithm is a wiely use numerical integrator for the motion
More informationLECTURE NOTES ON DVORETZKY S THEOREM
LECTURE NOTES ON DVORETZKY S THEOREM STEVEN HEILMAN Abstract. We present the first half of the paper [S]. In particular, the results below, unless otherwise state, shoul be attribute to G. Schechtman.
More information16 : Markov Chain Monte Carlo (MCMC)
10-708: Probabilistic Graphical Models 10-708, Spring 2014 16 : Markov Chain Monte Carlo MCMC Lecturer: Matthew Gormley Scribes: Yining Wang, Renato Negrinho 1 Sampling from low-dimensional distributions
More informationDiscrete Mathematics
Discrete Mathematics 309 (009) 86 869 Contents lists available at ScienceDirect Discrete Mathematics journal homepage: wwwelseviercom/locate/isc Profile vectors in the lattice of subspaces Dániel Gerbner
More information