Monte Carlo Basics. Aiichiro Nakano
|
|
- Lucy Hines
- 6 years ago
- Views:
Transcription
1 onte Carlo Basics Aiichiro Nakano Collaboratory for Advanced Computing & Simulations Department of Computer Science Department of Physics & Astronomy Department of Chemical Engineering & aterials Science Department of Biological Sciences University of Southern California Goals: C simulation in practice (understand & use) inimal math = probability arkov-chain (etropolis) C
2 onte Carlo ethod onte Carlo (C) method: A computational method that utilizes random numbers ajor applications of the C method: 1. ultidimensional integrations (e.g., statistical mechanics in physics) 2. Simulation of stochastic natural phenomena (e.g., stock price) A({! r i }) = d r! 1 " d r! N exp( V ({! r i }) k B T )A({! r i }) d r! 1 " d r! N exp( V ({! r i }) k B T )
3 Numerical integration onte Carlo Integration b a dxf (x) Δxf (x n ) = onte Carlo integration (r n [a,b]: random #): Error ~ 1/ 1/2 b a dxf (x) ultidimensional integration: O(C N 1 1 ) dx 1 f (x 1,, x N ) 0! dx N 0 b a f (r n ) b a f (x n )
4 Stochastic odel of Stock Prices Fluctuation in stock price Computational stock portfolio trading Basis of Black-Scholes analysis of option prices ds = µsdt +σsε dt (1997 Nobel Economy Prize to yron Scholes) Andrey Omeltchenko (Quantlab)
5 Goals: onte Carlo Basics 1. Sample-mean C ( ) 2 dxf (x) p(x) f ± f 2 f 1 f = 1 f (r n ) 2. Visualize probability density by a set of points: nonuniform randomnumber generation by coordinate transformation P(r P ʹ (ζ 1,,ζ N ) = 1,,r N ) (ζ 1,,ζ N ) / (r 1,,r N ) 3. etropolis (arkov-chain) C algorithm for importance sampling Random attempt conditional acceptance with probability exp(-δv/k B T)
6 onte Carlo Integration: Hit-&-iss Visualize probability density function Program hit.c: Calculate π #include <stdio.h>! #include <stdlib.h>! main() {! double x, y, pi;! int hit = 0, try, ntry;! printf("input the number of C trials\n");! scanf("%d",&ntry);! srand((unsigned)time((long *)0));! for (try=0; try<ntry; try++) {! x = rand()/(double)rand_ax;! y = rand()/(double)rand_ax;! if (x*x+y*y < 1.0) ++hit;! }! pi = 4.0*hit/ntry;! printf("c estimate for PI = %f\n", pi);! }! hpc-login3.usc.edu> more /usr/include/stdlib.h!...! #define RAND_AX ! = rand() [1,RAND_AX-1] N try P(x,y)dxdy points dy dx
7 Single-Electron Double-Slit Experiment Akira Tonomura (Hitachi, Ltd.)
8 onte Carlo Integration: Sample ean b a Bottom line: C simulation as integration dxf (x) 1 b a f (r n ) x 2 = dx 1+ x 2 = 0 = (b a) 1 π / 4 f (r n ) = (b a) f (r n ) dθ 4 cos 2 θ 1+ tan 2 = 4dθ = π θ 0 π / 4 0 average
9 Sample ean C Program Program mean.c: Calculate π #include <stdio.h>! #include <stdlib.h>! main() {! double x, pi, sum = 0.0;! int try, ntry;! printf("input the number of C trials\n");! scanf("%d",&ntry);! srand((unsigned)time((long *)0));! for (try=0; try<ntry; try++) {! x = rand()/(double)rand_ax;! sum += 4.0/(1.0 + x*x);! }! pi = sum/ntry;! printf("c estimate for PI = %f\n", pi);! }!
10 ultidimensional Integration Statistical mechanics k B = erg/k: Boltzmann constant S = k B log W Need onte Carlo!
11 Probability: Foundation of C inimal probability theory enough to understand C simulation Random variable: A variable that takes different values at different measurements. Probability distribution function p(x) = (# of samples with result x)/(total # of samples) Normalization Expectation (weighted sum) Variance (spread) Var[ x] = Standard deviation p(x k ) = 1 k E[ x] = x = x k p(x k ) k ( x x ) 2 = x 2 2x x + x 2 = x 2 x 2 Std[ x] = Var[ x]
12 Continuous Random Variables Probability density function (note the unit) p(x)dx = (# of samples in the range [x, x+dx])/(total # of samples) Visualize! Normalization dxp(x) = 1 Expectation E[ x] = x = dxxp(x) N try p(x)dx points ( lim Δxp(x k ) = 1) Δx 0 k Histogram Limit:
13 C Estimation Distinguish infinite vs. sample (~ poll) means vs. f (x) = 1 f (x) = dxf (x) p(x) takes random values f (r n ) Unbiased estimator: A statistical quantity, using samples, which converges to the underlying population value for large The mean of sampled random variables, f (x) converges to its expected value f (x) : f (x) is an unbiased estimator of f (x) The error of an C estimation decreases as 1/ 1/2 Central Theorem of C Sampling random variable Bottom line ( ) 2 dxf (x) p(x) f ± f 2 f 1 Exact Finite sampling
14 Proof of the Central Theorem Unbiased estimator of the mean E{ f (x)} = 1 E{ f (r n )} = 1 f (x) = Error estimate variance of the sample mean 1 Var{ f (x)} = Var f (r n ) = 1 2 m=1 For uncorrelated random variables, AB = A B, and thus [ f (r m ) f (x) ][ f (r n ) f (x) ] = 1 f (x) = f (x) [ f (r m ) f (x) ][ f (r n ) f (x) ]. Var{ f (x)} (m = n) f (r m ) f (x) f (r n ) f (x) = 0 (else) or Var{ f (x)} = 1 2 m=1 Var{ f (x)} = Var{ f (x)} Std{ f (x)} Std{ f (x)} = Error decreases as 1/ 1/2, but what is the prefactor Var{f(x)}?
15 An Example of Correlated Random Variables A spin variable, s, takes the value of +1 (spin up, ) with probability 0.5, & -1 (spin down, ) with probability 0.5 s = ( 1) 1 2 = 0 (Uncorrelated spins) s 1 s 2 = s 1 s 2 = 0 far don t see each other atom 1 atom 2 (Correlated spins) Ferromagnetic spins: they always align with each other (spin configurations with probability 0.5 and with probability 0.5) s 1 s 2 = ( 1 ) = 1 s 1 s 2
16 Estimate of the Variance of Sample ean Unbiased estimator of the variance s = 1 f 2 (r n ) 1 s = 1 f 2 (x) 1 = f 2 (x) m=1 f (r n ) For uncorrelated random variables f (r m ) f (r n ) = f (x) f (x) s = 1 1 f 2 ( 1) (x) 2 f (x) f (x) = 1 Var{ f (x)} f (r m ) f (r n ) f 2 (x) 1 m=1 2 m=1 2 f (r m ) f (r n ). (n m ) dxf (x) p(x) f ± Std{ f } f ± Std{ f (x)} f ± f 2 f ( ) 2 1
17 Running mean.c C estimation of π: unbiased estimators for the mean & its standard deviation Use the estimators! 1 dxf (x) f ± f 2 f 1 0 ( ) 2 f = 1 (N try ) f (r n ) f 2 = 1 f 2 (r n ) f (r) = 4 1+ r 2
18 Error Bar & the Exact Value Q: Should the exact value always fall within the error bar? A: No, only ~68% of the time in the case of the Gaussian distribution
19 Standard Deviation of mean.c C estimation of π: unbiased estimators for the mean & its standard deviation 1 dxf (x) f ± f 2 f 1 0 f = 1 f (r n ) f 2 = 1 f (x) = 4 /(1+ x 2 ) ( ) 2 f 2 (r n ) σ = N 1 seed N 2 1 seed π i π i N seed i=1 N seed i=1 Unbiased estimates & measured values of the standard deviation See lecture on least square fit 2 σ = C p! logσ "# = logc + plog!" # y! x! log 10
20 Random Number Generation Uniform random number generator: A routine to generate a sequence of random numbers within a specified range (typically [0,1]) Linear congruential generator: Generates a sequence X i of positive integers based on recursion X i+1 = ax i mod in the range [1,-1] & generates a random number in the range [0,1] as r i = X i / (Example) = 7 Good: a = 3 X i+1 = 3X i mod 7 Bad: a = 2 X i+1 = 2X i mod 7 i X i Period -1 = 6 i X i Period 3 < -1
21 Linear Congruential Generator ost common generator: cycle ~ 2 billion (, a) = ( = , 7 5 = 16807) Overflow avoidance algorithm (using 32-bit integer arithmetic) Let = aq + r, i.e., q = [/a], r = mod a then a(z mod q) r[z /q] if 0 az mod = a(z mod q) r[z /q]+ otherwise #define IA 16807! #define I ! #define A (1.0/I)! #define IQ ! #define IR 2836! float ran0(int *idum) {! long k;! float ans;! k = (*idum)/iq;! *idum = IA*(*idum - k*iq) - IR*k;! if (*idum < 0) *idum += I;! ans = A*(*idum);! return ans;! }!
22 Gaussian (Normal) Distribution ρ(ζ ) = 1 2π 2 exp ζ 2 dζρ(ζ ) = 1 Var{ζ } = dζζ 2 ρ(ζ ) = 1 Let s use coordinate transformation to generate nonuniform random numbers
23 Greenland Phenomenon Spherical coordinate system (latitude, θ & longitude, ϕ) is curved
24 Transformation ethod Uniformly generate population in these distorted maps to generate nonuniform distributions GDP! CO 2 Emission!
25 Box-uller Transformation Uniform r 1,r 2 [0,1] Non-uniform ζ 1 = 2 ln r 1 cos(2πr 2 ) ζ 2 = 2 ln r 1 sin(2πr 2 ) P(r 1,r 2 ) = 1 P(ζ 1/ 2 ) = 1 2π exp 2 ζ 1/ 2 2
26 Nonuniform Random Numbers Uniform random number P(r 1,r 2 ) = 1! Coordinate transformation method r! θ Number of samples: N P(r 1,r 2 )!# "# $ dr 1 dr 2 = N P ʹ (ζ 1,ζ 2 ) S(r 1,r 2,dr 1,dr 2 ) =1 density area Areal transformation: P ʹ (ζ 1,ζ 2 ) = ζ 1,ζ 2 r 1,r 2 total sample size ( ) ( ) ( ) ( ) dr 1 dr 2 S = ζ 1,ζ 2 r 1,r 2 1 P(r 1,r 2 )!# "# $ = r 1,r 2 ζ 1,ζ 2 =1 ( ) ( ) Jacobian = areal scaling
27 Areal Transformation S S = ξ x ζ x ξ y dxdy = ξ ζ ζ x y ξ ζ dxdy y x y Parallelogram! S =! k! l sinθ = a 2 + b 2 c 2 + d 2 sinθ = ad bc
28 ultidimensional Integration Statistical mechanics k B = erg/k: Boltzmann constant S = k B log W Need importance sampling! (Example: harmonic oscillators) = for 100 oscillators
29 Importance Sampling (Problem) How to generate 3N-dimensional sample points according to the Boltzmann probability distribution? A = d! r N A( r! N ) exp V ( r! ( N ) /k B T ) A(! N! r n ) exp V ( N ( r n ) /kb T ) d r! N exp V ( r! N ) /k B T ( ) exp( V ( r! N n ) /kb T ) Importance sampling: Chooses a sequence of random numbers from a p(! N r n ) probability density function, p(x) arkov chain: A sequence of trials that satisfies (a) The outcome of each trial belongs to a finite set of outcomes, {Γ 1, Γ 2,..., Γ N }, called the state space (b) The outcome of each trial depends only on the outcome of the trial that immediately precedes it Transition probability matrix: Its element π mn is the conditional probability that the next state is Γ given that the current state is Γ n, i.e., the probability of state transition, n m N p m (τ ) = π mn p n (τ 1)
30 arkov Chain Normalization: Given that the system is in the state, Γ n, then the next state must be one of the N possible states atrix-vector notation π 11 π 12! π 1N π Π = 21 π 22 " # (Perron-Frobenius theorem) Eigenvalues: 1. Fixed point: π N1 π NN N π mn m=1 = 1 ρ (τ ) = p 1 (τ ) p 2 (τ ) " p N (τ ) ρ (τ ) = Πρ (τ 1) = Π 2 ρ (τ 2) =! = Π τ ρ (0) Πρ 1 = ρ 1 Andrey arkov ( ) Πρ ν = ε ν ρ ν ( ε 1 = 1, ε 2,, ε N < 1) 2. Filter: Π τ N ν =1 c ν ρ ν = N ν =1 c ν ε ν ρ ν c 1 ρ 1 τ Oskar Perron ( ) Ferdinand G. Frobenius ( )
31 Example: Two-Level System Π = a 1 b = a = 0.6,b = a b ( ) p (τ ) p (τ ) τ 1 = τ
32 Eigenvalue Problem Eigensystem Πw = λw w = u v a 1 b u or 1 a b v = λ u v λ a b 1 u = 0 a 1 λ b v 0 For a nontrivial (other than u = v = 0) solution, (Π-λI) -1 should not exist Eigenvalues: λ + = 1 and -1 < λ - = a+b-1 < 1 Eigenvectors: Π λi = (λ a)(λ b) (a 1)(b 1) = (λ a b +1)(λ 1) = 0 for a positive matrix: a, b > 0 w + = u + = v a b 1 b and 1 a Perron-Frobenius! w = u = 1 1 v
33 Telescopic Technique Π u + u = λ + v + + v + Π u u = λ v v Π 11 Π 12 or u U + Π 21 Π 22 v + u v ΠU = UΛ or Π = UΛU 1 = u + v + u v λ Λ λ p 1 (n) p 2 (n) = p Πn 1 = UΛU 1 UΛU 1!UΛU 1 UΛU 1 p 1 = UΛ n U 1 p 1 p 2 p 2 p 2 p 1 (n) p 2 (n) = p Πn 1 p 2 n u + v + λ + n 0 0 λ n 1 0 n b = 2 a b 0.3/0.7 = a = 0.4 /0.7 = a b Perron-Frobenius theorem!
34 Detailed Balance Condition Perron-Frobenius theorem repeated application of Π converges to its eigenvector, ρ, with eigenvalue 1 where Πρ = ρ or N Π mn ρ (τ ) = Π τ (τ =0) ρ τ ρ n = ρ m ρ (Problem = importance sampling) Find a transition probability matrix Π whose 1st eigenvector equals a given ρ, e.g., ρ n = exp V n /k B T N π mn ρ n = N π nm ( ) 1 n ( ) exp( V n /k B T ) (Solution = detailed balance condition) Let Π satisfy, for any pair of states, π mn ρ n = π nm ρ m N then π Steady mn ρ n = ρ m. population ρ m ρ CA =34 π NV CA π CA NV ρ NV =2 Balanced flux
35 etropolis Algorithm (Problem) Find a transition probability matrix Π that satisfies the detailed balance condition for a given ρ: π mn ρ n = π nm ρ m (Solution) Let α mn = α nm be a symmetric attempt matrix, then α mn ρ m ρ n m n π mn = ( ρ m ρ n )α mn ρ m < ρ n m n 1 m = n π m'n m' n (Example) Statistical mechanics: no need for the absolute weight! next { r i } ( ) ( current { r i }) = P! P! exp V! next { r i } ( ( ) k B T ) ( current { r i }) k B T ( ) exp V!! next! current = exp δ V { r i },{ r i } ( ( ) k B T )
36 etropolis Algorithm Let X be a state variable (e.g. positions of atoms R 3N ) and calculate the statistical average of function A(X) with probability density P(X) e -V(X)/k BT Sum_A = 0! for n = 1 to Ntrial! X X + dx /* Random state-transition attempt */! Compute dv = V(X ) - V(X)! if dv < 0 then /* Greedy */! Accept X X! else if rand() < exp(-dv/k B T) then /* Occasional uphill move */ Accept X X! /* else the state remains X */! endif! Sum_A = Sum_A + A(X) /* Count X more than once, if it remains */! endfor! Avg_A = Sum_A/Ntrial! 0 1 exp(-dv/k B T) Note: rand() here is a uniform random number in the range [0, 1]
37 onte Carlo Simulation Random trial acceptance by a cost criterion
38 Summary: onte Carlo Basics 1. Sample-mean C ( ) 2 dxf (x) p(x) f ± f 2 f 1 2. Visualize probability density by a set of points: nonuniform randomnumber generation by coordinate transformation P(r P ʹ (ζ 1,,ζ N ) = 1,,r N ) (ζ 1,,ζ N ) / (r 1,,r N ) Areal magnification factor 3. etropolis (arkov-chain) C algorithm for importance sampling Random attempt conditional acceptance with probability exp(-δv/k B T)
dxf (x) Δxf (x n M dxf (x) M f (r ) ,, x N f (x 1 dx N
I Monte Carlo Basics Introduction WHAT IS THE MONTE CARLO METHOD? Monte Carlo (MC) method: A computational method that utilizes random numbers. Two major applications of the MC method:. Multidimensional
More informationMonte Carlo Simulation of Spins
Monte Carlo Simulation of Spins Aiichiro Nakano Collaboratory for Advanced Computing & Simulations Department of Computer Science Department of Physics & Astronomy Department of Chemical Engineering &
More informationRandom processes and probability distributions. Phys 420/580 Lecture 20
Random processes and probability distributions Phys 420/580 Lecture 20 Random processes Many physical processes are random in character: e.g., nuclear decay (Poisson distributed event count) P (k, τ) =
More information1 Introduction. 2 Binary shift map
Introduction This notes are meant to provide a conceptual background for the numerical construction of random variables. They can be regarded as a mathematical complement to the article [] (please follow
More informationfunctions Poisson distribution Normal distribution Arbitrary functions
Physics 433: Computational Physics Lecture 6 Random number distributions Generation of random numbers of various distribuition functions Normal distribution Poisson distribution Arbitrary functions Random
More informationFrom Random Numbers to Monte Carlo. Random Numbers, Random Walks, Diffusion, Monte Carlo integration, and all that
From Random Numbers to Monte Carlo Random Numbers, Random Walks, Diffusion, Monte Carlo integration, and all that Random Walk Through Life Random Walk Through Life If you flip the coin 5 times you will
More informationand in each case give the range of values of x for which the expansion is valid.
α β γ δ ε ζ η θ ι κ λ µ ν ξ ο π ρ σ τ υ ϕ χ ψ ω Mathematics is indeed dangerous in that it absorbs students to such a degree that it dulls their senses to everything else P Kraft Further Maths A (MFPD)
More informationSampling Distributions Allen & Tildesley pgs and Numerical Recipes on random numbers
Sampling Distributions Allen & Tildesley pgs. 345-351 and Numerical Recipes on random numbers Today I explain how to generate a non-uniform probability distributions. These are used in importance sampling
More informationLecture 15 Random variables
Lecture 15 Random variables Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical Sciences, Peking University, tieli@pku.edu.cn No.1
More informationSwitching Regime Estimation
Switching Regime Estimation Series de Tiempo BIrkbeck March 2013 Martin Sola (FE) Markov Switching models 01/13 1 / 52 The economy (the time series) often behaves very different in periods such as booms
More informationToday: Fundamentals of Monte Carlo
Today: Fundamentals of Monte Carlo What is Monte Carlo? Named at Los Alamos in 1940 s after the casino. Any method which uses (pseudo)random numbers as an essential part of the algorithm. Stochastic -
More informationMonte Carlo. Lecture 15 4/9/18. Harvard SEAS AP 275 Atomistic Modeling of Materials Boris Kozinsky
Monte Carlo Lecture 15 4/9/18 1 Sampling with dynamics In Molecular Dynamics we simulate evolution of a system over time according to Newton s equations, conserving energy Averages (thermodynamic properties)
More informationToday: Fundamentals of Monte Carlo
Today: Fundamentals of Monte Carlo What is Monte Carlo? Named at Los Alamos in 940 s after the casino. Any method which uses (pseudo)random numbers as an essential part of the algorithm. Stochastic - not
More informationMath 46, Applied Math (Spring 2009): Final
Math 46, Applied Math (Spring 2009): Final 3 hours, 80 points total, 9 questions worth varying numbers of points 1. [8 points] Find an approximate solution to the following initial-value problem which
More informationMarkov Processes. Stochastic process. Markov process
Markov Processes Stochastic process movement through a series of well-defined states in a way that involves some element of randomness for our purposes, states are microstates in the governing ensemble
More informationPhysics 403. Segev BenZvi. Parameter Estimation, Correlations, and Error Bars. Department of Physics and Astronomy University of Rochester
Physics 403 Parameter Estimation, Correlations, and Error Bars Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Review of Last Class Best Estimates and Reliability
More informationLecture 20. Randomness and Monte Carlo. J. Chaudhry. Department of Mathematics and Statistics University of New Mexico
Lecture 20 Randomness and Monte Carlo J. Chaudhry Department of Mathematics and Statistics University of New Mexico J. Chaudhry (UNM) CS 357 1 / 40 What we ll do: Random number generators Monte-Carlo integration
More informationChapter 4: Monte-Carlo Methods
Chapter 4: Monte-Carlo Methods A Monte-Carlo method is a technique for the numerical realization of a stochastic process by means of normally distributed random variables. In financial mathematics, it
More informationMultiple Random Variables
Multiple Random Variables Joint Probability Density Let X and Y be two random variables. Their joint distribution function is F ( XY x, y) P X x Y y. F XY ( ) 1, < x
More informationProbability Density Functions
Statistical Methods in Particle Physics / WS 13 Lecture II Probability Density Functions Niklaus Berger Physics Institute, University of Heidelberg Recap of Lecture I: Kolmogorov Axioms Ingredients: Set
More informationDescriptive Statistics
Descriptive Statistics DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Descriptive statistics Techniques to visualize
More informationReduction of Variance. Importance Sampling
Reduction of Variance As we discussed earlier, the statistical error goes as: error = sqrt(variance/computer time). DEFINE: Efficiency = = 1/vT v = error of mean and T = total CPU time How can you make
More informationMonte Carlo integration (naive Monte Carlo)
Monte Carlo integration (naive Monte Carlo) Example: Calculate π by MC integration [xpi] 1/21 INTEGER n total # of points INTEGER i INTEGER nu # of points in a circle REAL x,y coordinates of a point in
More informationMonte Carlo Methods in Statistical Mechanics
Monte Carlo Methods in Statistical Mechanics Mario G. Del Pópolo Atomistic Simulation Centre School of Mathematics and Physics Queen s University Belfast Belfast Mario G. Del Pópolo Statistical Mechanics
More informationWeek 1 Quantitative Analysis of Financial Markets Distributions A
Week 1 Quantitative Analysis of Financial Markets Distributions A Christopher Ting http://www.mysmu.edu/faculty/christophert/ Christopher Ting : christopherting@smu.edu.sg : 6828 0364 : LKCSB 5036 October
More informationExpectation. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda
Expectation DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean,
More informationIntroduction and Vectors Lecture 1
1 Introduction Introduction and Vectors Lecture 1 This is a course on classical Electromagnetism. It is the foundation for more advanced courses in modern physics. All physics of the modern era, from quantum
More informationScientific Computing: Monte Carlo
Scientific Computing: Monte Carlo Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 April 5th and 12th, 2012 A. Donev (Courant Institute)
More informationBrandon C. Kelly (Harvard Smithsonian Center for Astrophysics)
Brandon C. Kelly (Harvard Smithsonian Center for Astrophysics) Probability quantifies randomness and uncertainty How do I estimate the normalization and logarithmic slope of a X ray continuum, assuming
More information1 of 7 7/16/2009 6:12 AM Virtual Laboratories > 7. Point Estimation > 1 2 3 4 5 6 1. Estimators The Basic Statistical Model As usual, our starting point is a random experiment with an underlying sample
More informationSampling Distributions Allen & Tildesley pgs and Numerical Recipes on random numbers
Sampling Distributions Allen & Tildesley pgs. 345-351 and Numerical Recipes on random numbers Today I explain how to generate a non-uniform probability distributions. These are used in importance sampling
More informationStatistical signal processing
Statistical signal processing Short overview of the fundamentals Outline Random variables Random processes Stationarity Ergodicity Spectral analysis Random variable and processes Intuition: A random variable
More information16. Working with the Langevin and Fokker-Planck equations
16. Working with the Langevin and Fokker-Planck equations In the preceding Lecture, we have shown that given a Langevin equation (LE), it is possible to write down an equivalent Fokker-Planck equation
More informationMarkov Chain Monte Carlo Simulations and Their Statistical Analysis An Overview
Markov Chain Monte Carlo Simulations and Their Statistical Analysis An Overview Bernd Berg FSU, August 30, 2005 Content 1. Statistics as needed 2. Markov Chain Monte Carlo (MC) 3. Statistical Analysis
More informationMonte Carlo and cold gases. Lode Pollet.
Monte Carlo and cold gases Lode Pollet lpollet@physics.harvard.edu 1 Outline Classical Monte Carlo The Monte Carlo trick Markov chains Metropolis algorithm Ising model critical slowing down Quantum Monte
More informationMonte Carlo methods for kinetic equations
Monte Carlo methods for kinetic equations Lecture 2: Monte Carlo simulation methods Lorenzo Pareschi Department of Mathematics & CMCS University of Ferrara Italy http://utenti.unife.it/lorenzo.pareschi/
More informationExpectation. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Expectation DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean, variance,
More informationMore General Functions Is this technique limited to the monomials {1, x, x 2, x 3,...}?
More General Functions Is this technique limited to the monomials {1, x, x 2, x 3,...}? Interpolation with General Sets of Functions For a general set of functions {ϕ 1,..., ϕ n }, solve the linear system
More informationPhysics 403 Probability Distributions II: More Properties of PDFs and PMFs
Physics 403 Probability Distributions II: More Properties of PDFs and PMFs Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Last Time: Common Probability Distributions
More informationE X A M. Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours. Number of pages incl.
E X A M Course code: Course name: Number of pages incl. front page: 6 MA430-G Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours Resources allowed: Notes: Pocket calculator,
More informationNegative Association, Ordering and Convergence of Resampling Methods
Negative Association, Ordering and Convergence of Resampling Methods Nicolas Chopin ENSAE, Paristech (Joint work with Mathieu Gerber and Nick Whiteley, University of Bristol) Resampling schemes: Informal
More informationMTH4101 CALCULUS II REVISION NOTES. 1. COMPLEX NUMBERS (Thomas Appendix 7 + lecture notes) ax 2 + bx + c = 0. x = b ± b 2 4ac 2a. i = 1.
MTH4101 CALCULUS II REVISION NOTES 1. COMPLEX NUMBERS (Thomas Appendix 7 + lecture notes) 1.1 Introduction Types of numbers (natural, integers, rationals, reals) The need to solve quadratic equations:
More information( x) ( ) F ( ) ( ) ( ) Prob( ) ( ) ( ) X x F x f s ds
Applied Numerical Analysis Pseudo Random Number Generator Lecturer: Emad Fatemizadeh What is random number: A sequence in which each term is unpredictable 29, 95, 11, 60, 22 Application: Monte Carlo Simulations
More information2t t dt.. So the distance is (t2 +6) 3/2
Math 8, Solutions to Review for the Final Exam Question : The distance is 5 t t + dt To work that out, integrate by parts with u t +, so that t dt du The integral is t t + dt u du u 3/ (t +) 3/ So the
More informationCHEM-UA 652: Thermodynamics and Kinetics
1 CHEM-UA 652: Thermodynamics and Kinetics Notes for Lecture 4 I. THE ISOTHERMAL-ISOBARIC ENSEMBLE The isothermal-isobaric ensemble is the closest mimic to the conditions under which most experiments are
More informationA simple algorithm that will generate a sequence of integers between 0 and m is:
Using the Pseudo-Random Number generator Generating random numbers is a useful technique in many numerical applications in Physics. This is because many phenomena in physics are random, and algorithms
More informationMonte Carlo Simulations
Monte Carlo Simulations What are Monte Carlo Simulations and why ones them? Pseudo Random Number generators Creating a realization of a general PDF The Bootstrap approach A real life example: LOFAR simulations
More informationTEST CODE: MMA (Objective type) 2015 SYLLABUS
TEST CODE: MMA (Objective type) 2015 SYLLABUS Analytical Reasoning Algebra Arithmetic, geometric and harmonic progression. Continued fractions. Elementary combinatorics: Permutations and combinations,
More informationStatistics, Probability Distributions & Error Propagation. James R. Graham
Statistics, Probability Distributions & Error Propagation James R. Graham Sample & Parent Populations Make measurements x x In general do not expect x = x But as you take more and more measurements a pattern
More informationFrom Determinism to Stochasticity
From Determinism to Stochasticity Reading for this lecture: (These) Lecture Notes. Outline of next few lectures: Probability theory Stochastic processes Measurement theory You Are Here A B C... α γ β δ
More informationSTA 294: Stochastic Processes & Bayesian Nonparametrics
MARKOV CHAINS AND CONVERGENCE CONCEPTS Markov chains are among the simplest stochastic processes, just one step beyond iid sequences of random variables. Traditionally they ve been used in modelling a
More informationBrownian Motion. An Undergraduate Introduction to Financial Mathematics. J. Robert Buchanan. J. Robert Buchanan Brownian Motion
Brownian Motion An Undergraduate Introduction to Financial Mathematics J. Robert Buchanan 2010 Background We have already seen that the limiting behavior of a discrete random walk yields a derivation of
More information2.3 Estimating PDFs and PDF Parameters
.3 Estimating PDFs and PDF Parameters estimating means - discrete and continuous estimating variance using a known mean estimating variance with an estimated mean estimating a discrete pdf estimating a
More informationThe Monte Carlo Algorithm
The Monte Carlo Algorithm This algorithm was invented by N. Metropolis, A.W. Rosenbluth, M.N. Rosenbluth, A.H. Teller, and E. Teller, J. Chem. Phys., 21, 1087 (1953). It was listed as one of the The Top
More informationModern Navigation. Thomas Herring
12.215 Modern Navigation Thomas Herring Estimation methods Review of last class Restrict to basically linear estimation problems (also non-linear problems that are nearly linear) Restrict to parametric,
More informationNumerical integration and importance sampling
2 Numerical integration and importance sampling 2.1 Quadrature Consider the numerical evaluation of the integral I(a, b) = b a dx f(x) Rectangle rule: on small interval, construct interpolating function
More informationInfinite Series. 1 Introduction. 2 General discussion on convergence
Infinite Series 1 Introduction I will only cover a few topics in this lecture, choosing to discuss those which I have used over the years. The text covers substantially more material and is available for
More informationxy 2 e 2z dx dy dz = 8 3 (1 e 4 ) = 2.62 mc. 12 x2 y 3 e 2z 2 m 2 m 2 m Figure P4.1: Cube of Problem 4.1.
Problem 4.1 A cube m on a side is located in the first octant in a Cartesian coordinate system, with one of its corners at the origin. Find the total charge contained in the cube if the charge density
More informationRandom number generators and random processes. Statistics and probability intro. Peg board example. Peg board example. Notes. Eugeniy E.
Random number generators and random processes Eugeniy E. Mikhailov The College of William & Mary Lecture 11 Eugeniy Mikhailov (W&M) Practical Computing Lecture 11 1 / 11 Statistics and probability intro
More informationChapter 3: Random Variables 1
Chapter 3: Random Variables 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.
More informationIntroduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos & Aarti Singh Contents Markov Chain Monte Carlo Methods Goal & Motivation Sampling Rejection Importance Markov
More informationMean Intensity. Same units as I ν : J/m 2 /s/hz/sr (ergs/cm 2 /s/hz/sr) Function of position (and time), but not direction
MCRT: L5 MCRT estimators for mean intensity, absorbed radiation, radiation pressure, etc Path length sampling: mean intensity, fluence rate, number of absorptions Random number generators J Mean Intensity
More information8.1 Sequences. Example: A sequence is a function f(n) whose domain is a subset of the integers. Notation: *Note: n = 0 vs. n = 1.
8. Sequences Example: A sequence is a function f(n) whose domain is a subset of the integers. Notation: *Note: n = 0 vs. n = Examples: 6. Find a formula for the general term a n of the sequence, assuming
More informationRambo s Math GRE Practice Test. Charles Rambo
Rambo s Math GRE Practice Test Charles Rambo Preface Thank you for checking out my practice exam! This was heavily influenced by the GR1268, GR0568, and GR8767 exams. I also used Rudin s Principles of
More informationMath 180A. Lecture 16 Friday May 7 th. Expectation. Recall the three main probability density functions so far (1) Uniform (2) Exponential.
Math 8A Lecture 6 Friday May 7 th Epectation Recall the three main probability density functions so far () Uniform () Eponential (3) Power Law e, ( ), Math 8A Lecture 6 Friday May 7 th Epectation Eample
More informationLaplace s Equation. Chapter Mean Value Formulas
Chapter 1 Laplace s Equation Let be an open set in R n. A function u C 2 () is called harmonic in if it satisfies Laplace s equation n (1.1) u := D ii u = 0 in. i=1 A function u C 2 () is called subharmonic
More informationNumerical Methods I Monte Carlo Methods
Numerical Methods I Monte Carlo Methods Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course G63.2010.001 / G22.2420-001, Fall 2010 Dec. 9th, 2010 A. Donev (Courant Institute) Lecture
More informationHands-on Generating Random
CVIP Laboratory Hands-on Generating Random Variables Shireen Elhabian Aly Farag October 2007 The heart of Monte Carlo simulation for statistical inference. Generate synthetic data to test our algorithms,
More informationEELE 3331 Electromagnetic I Chapter 3. Vector Calculus. Islamic University of Gaza Electrical Engineering Department Dr.
EELE 3331 Electromagnetic I Chapter 3 Vector Calculus Islamic University of Gaza Electrical Engineering Department Dr. Talal Skaik 2012 1 Differential Length, Area, and Volume This chapter deals with integration
More informationLecture Notes 7 Random Processes. Markov Processes Markov Chains. Random Processes
Lecture Notes 7 Random Processes Definition IID Processes Bernoulli Process Binomial Counting Process Interarrival Time Process Markov Processes Markov Chains Classification of States Steady State Probabilities
More informationRandom Number Generation. Stephen Booth David Henty
Random Number Generation Stephen Booth David Henty Introduction Random numbers are frequently used in many types of computer simulation Frequently as part of a sampling process: Generate a representative
More informationBivariate distributions
Bivariate distributions 3 th October 017 lecture based on Hogg Tanis Zimmerman: Probability and Statistical Inference (9th ed.) Bivariate Distributions of the Discrete Type The Correlation Coefficient
More information14 : Approximate Inference Monte Carlo Methods
10-708: Probabilistic Graphical Models 10-708, Spring 2018 14 : Approximate Inference Monte Carlo Methods Lecturer: Kayhan Batmanghelich Scribes: Biswajit Paria, Prerna Chiersal 1 Introduction We have
More informationData Analysis I. Dr Martin Hendry, Dept of Physics and Astronomy University of Glasgow, UK. 10 lectures, beginning October 2006
Astronomical p( y x, I) p( x, I) p ( x y, I) = p( y, I) Data Analysis I Dr Martin Hendry, Dept of Physics and Astronomy University of Glasgow, UK 10 lectures, beginning October 2006 4. Monte Carlo Methods
More informationENGI 9420 Lecture Notes 1 - ODEs Page 1.01
ENGI 940 Lecture Notes - ODEs Page.0. Ordinary Differential Equations An equation involving a function of one independent variable and the derivative(s) of that function is an ordinary differential equation
More informationSection 8.2 : Homogeneous Linear Systems
Section 8.2 : Homogeneous Linear Systems Review: Eigenvalues and Eigenvectors Let A be an n n matrix with constant real components a ij. An eigenvector of A is a nonzero n 1 column vector v such that Av
More informationStatistical Data Analysis
DS-GA 0 Lecture notes 8 Fall 016 1 Descriptive statistics Statistical Data Analysis In this section we consider the problem of analyzing a set of data. We describe several techniques for visualizing the
More informationConditional Distributions
Conditional Distributions The goal is to provide a general definition of the conditional distribution of Y given X, when (X, Y ) are jointly distributed. Let F be a distribution function on R. Let G(,
More informationPhysics 127b: Statistical Mechanics. Renormalization Group: 1d Ising Model. Perturbation expansion
Physics 17b: Statistical Mechanics Renormalization Group: 1d Ising Model The ReNormalization Group (RNG) gives an understanding of scaling and universality, and provides various approximation schemes to
More informationMath Spring Practice for the Second Exam.
Math 4 - Spring 27 - Practice for the Second Exam.. Let X be a random variable and let F X be the distribution function of X: t < t 2 t < 4 F X (t) : + t t < 2 2 2 2 t < 4 t. Find P(X ), P(X ), P(X 2),
More informationMATH3383. Quantum Mechanics. Appendix D: Hermite Equation; Orthogonal Polynomials
MATH3383. Quantum Mechanics. Appendix D: Hermite Equation; Orthogonal Polynomials. Hermite Equation In the study of the eigenvalue problem of the Hamiltonian for the quantum harmonic oscillator we have
More informationGRE Math Subject Test #5 Solutions.
GRE Math Subject Test #5 Solutions. 1. E (Calculus) Apply L Hôpital s Rule two times: cos(3x) 1 3 sin(3x) 9 cos(3x) lim x 0 = lim x 2 x 0 = lim 2x x 0 = 9. 2 2 2. C (Geometry) Note that a line segment
More informationAdaptive Filter Theory
0 Adaptive Filter heory Sung Ho Cho Hanyang University Seoul, Korea (Office) +8--0-0390 (Mobile) +8-10-541-5178 dragon@hanyang.ac.kr able of Contents 1 Wiener Filters Gradient Search by Steepest Descent
More informationy = x 3 and y = 2x 2 x. 2x 2 x = x 3 x 3 2x 2 + x = 0 x(x 2 2x + 1) = 0 x(x 1) 2 = 0 x = 0 and x = (x 3 (2x 2 x)) dx
Millersville University Name Answer Key Mathematics Department MATH 2, Calculus II, Final Examination May 4, 2, 8:AM-:AM Please answer the following questions. Your answers will be evaluated on their correctness,
More informationBohr & Wheeler Fission Theory Calculation 4 March 2009
Bohr & Wheeler Fission Theory Calculation 4 March 9 () Introduction The goal here is to reproduce the calculation of the limiting Z /A against spontaneous fission Z A lim a S. (.) a C as first done by
More informationModeling of Micro-Fluidics by a Dissipative Particle Dynamics Method. Justyna Czerwinska
Modeling of Micro-Fluidics by a Dissipative Particle Dynamics Method Justyna Czerwinska Scales and Physical Models years Time hours Engineering Design Limit Process Design minutes Continious Mechanics
More informationQuantum Physics III (8.06) Spring 2007 FINAL EXAMINATION Monday May 21, 9:00 am You have 3 hours.
Quantum Physics III (8.06) Spring 2007 FINAL EXAMINATION Monday May 21, 9:00 am You have 3 hours. There are 10 problems, totalling 180 points. Do all problems. Answer all problems in the white books provided.
More informationMAE294B/SIOC203B: Methods in Applied Mechanics Winter Quarter sgls/mae294b Solution IV
MAE9B/SIOC3B: Methods in Applied Mechanics Winter Quarter 8 http://webengucsdedu/ sgls/mae9b 8 Solution IV (i The equation becomes in T Applying standard WKB gives ɛ y TT ɛte T y T + y = φ T Te T φ T +
More informationWalsh Diffusions. Andrey Sarantsev. March 27, University of California, Santa Barbara. Andrey Sarantsev University of Washington, Seattle 1 / 1
Walsh Diffusions Andrey Sarantsev University of California, Santa Barbara March 27, 2017 Andrey Sarantsev University of Washington, Seattle 1 / 1 Walsh Brownian Motion on R d Spinning measure µ: probability
More informationCalculus II Practice Test Problems for Chapter 7 Page 1 of 6
Calculus II Practice Test Problems for Chapter 7 Page of 6 This is a set of practice test problems for Chapter 7. This is in no way an inclusive set of problems there can be other types of problems on
More informationIntroduction to Error Analysis
Introduction to Error Analysis Part 1: the Basics Andrei Gritsan based on lectures by Petar Maksimović February 1, 2010 Overview Definitions Reporting results and rounding Accuracy vs precision systematic
More informationHILL S EQUATION WITH RANDOM FORCING TERMS. Fred C. Adams and Anthony M. Bloch Also: Suzie Butler, Jeff Druce and Jake Ketchum
1 HILL S EQUATION WITH RANDOM FORCING TERMS Fred C. Adams and Anthony M. Bloch Also: Suzie Butler, Jeff Druce and Jake Ketchum 2 Orbit problems in astrophysics Solutions to Hill s equation with forcing
More informationPrinciple of Mathematical Induction
Advanced Calculus I. Math 451, Fall 2016, Prof. Vershynin Principle of Mathematical Induction 1. Prove that 1 + 2 + + n = 1 n(n + 1) for all n N. 2 2. Prove that 1 2 + 2 2 + + n 2 = 1 n(n + 1)(2n + 1)
More informationMultifractality in simple systems. Eugene Bogomolny
Multifractality in simple systems Eugene Bogomolny Univ. Paris-Sud, Laboratoire de Physique Théorique et Modèles Statistiques, Orsay, France In collaboration with Yasar Atas Outlook 1 Multifractality Critical
More informationECE 636: Systems identification
ECE 636: Systems identification Lectures 3 4 Random variables/signals (continued) Random/stochastic vectors Random signals and linear systems Random signals in the frequency domain υ ε x S z + y Experimental
More informationMATH325 - QUANTUM MECHANICS - SOLUTION SHEET 11
MATH35 - QUANTUM MECHANICS - SOLUTION SHEET. The Hamiltonian for a particle of mass m moving in three dimensions under the influence of a three-dimensional harmonic oscillator potential is Ĥ = h m + mω
More informationVariance Reduction Techniques for Monte Carlo Simulations with Stochastic Volatility Models
Variance Reduction Techniques for Monte Carlo Simulations with Stochastic Volatility Models Jean-Pierre Fouque North Carolina State University SAMSI November 3, 5 1 References: Variance Reduction for Monte
More informationNotes on Special Functions
Spring 25 1 Notes on Special Functions Francis J. Narcowich Department of Mathematics Texas A&M University College Station, TX 77843-3368 Introduction These notes are for our classes on special functions.
More informationSummer 2017 MATH Solution to Exercise 5
Summer 07 MATH00 Solution to Exercise 5. Find the partial derivatives of the following functions: (a (xy 5z/( + x, (b x/ x + y, (c arctan y/x, (d log((t + 3 + ts, (e sin(xy z 3, (f x α, x = (x,, x n. (a
More information2.20 Fall 2018 Math Review
2.20 Fall 2018 Math Review September 10, 2018 These notes are to help you through the math used in this class. This is just a refresher, so if you never learned one of these topics you should look more
More information