Today: Fundamentals of Monte Carlo
|
|
- Bathsheba West
- 5 years ago
- Views:
Transcription
1 Today: Fundamentals of Monte Carlo What is Monte Carlo? Named at Los Alamos in 940 s after the casino. Any method which uses (pseudo)random numbers as an essential part of the algorithm. Stochastic - not deterministic! A method for doing highly dimensional integrals by sampling the integrand. Often a Markov chain, called Metropolis MC. Simple example: Buffon s needle Monte Carlo determination of Consider a square inscribed by circle. Consider one quadrant. By geometry: area of /4 circle = r 2 /4 = /4. area of square r 2 Simple MC like throwing darts at unit square target: Using RNG (0,), pick a pair of coordinates (x,y). Count # of points (darts) in shaded section versus total. Pts in shaded circle/pts in square ~ /4. hits = 0 DO n=,n{ x = (random #) y = (random #) distance2 = (x 2 + y 2 ) If (distance2 ) hits = hits + } pi= 4 * hits/n 2
2 MC is advantageous for high dimensional integrals -the best general method Consider an integral in the unit D-dimensional hypercube: I dx...dx D f (x,...,x D ) By conventional deterministic methods: Lay out a grid with L points in each direction with h=/l Number of points is N =L D =h D CPU time. How does error scale with CPU time or Dimensionality? Error in trapezoidal rule goes as =f (x) h 2 since f(x)=f(x 0 ) + f (x 0 ) h + (/2)f (x 0 )h 2 + direct CPU time -D/2. (~h 2 and CPU~h D ) But by sampling we find -2. (~M -/2 and CPU~M) To get another decimal place takes 00 times longer! But MC is advantageous for D>4! 3 Improved Numerical Integration Integration methods in D-dimensions: Trapeziodal Rule: ~ f (2) (x) h 2 Simpson s Rule: ~ f (4) (x) h 4 generally: ~ f () (x) h And CPU time scales with sample points (grid size) CPU time ~ h -D (e.g., -D like /L and 2-D like /L 2 ) Time to do integral: T int ~ -D/ By conventional deterministic methods: Lay out a grid with L points in each direction with h=/l Number of points is N=h D =L D CPU time. Stochastic Integration (Monte Carlo) Monte Carlo: ~ M -/2 (sqrt of sampled points) CPU time: T ~ M -2. > In the limit of small, MC wins if D > 2! 4 2
3 4D 6D 8D Log(CPU Time) 2D MC Good Log(error) Bad Other reasons to do Monte Carlo: Conceptually and practically simple. Comes with built in error bars. Many methods of integration have been tried, and will be tried in this world of sin and woe. No one pretends that Monte Carlo is perfect or all-wise. Indeed, it has been said that Monte Carlo is the worst method except all those other methods that have been tried from time to time. Churchill Probability Distributions P(x)dx= probability of observing a value in (x,x+dx) is a probability distribution function (p.d.f.) dx P(x) P(x) 0 x can be either a continuous or discrete variable. Cumulative distribution: Probability of x<y. y c( y) dx Useful for sampling P(x) 0 c( y) Average or expectation Moments: Zero th moment I 0 = Mean <x>=i Variance <(x-<x>) 2 >=I 2 -(I ) 2 g(x) g I n x n dx P(x)g(x) dx P(x)x n 6 3
4 Mappings of random variables Let p x (x)dx be a probability distribution Let y=g(x) be a new variable y=g(x) -- e.g., y=g(x) = ln(x) with 0 < x soy, 0 What is the pdf of y? With p y (y)dy=p x (x)dx p y (y) p x (x) dx dy p x(x) dx dg(x) p x(x) dg(x) dx x What happens when g is not monotonic? x Example: y=g(x) = ln(x) p y (y)dy p x (x) dg(x) dx dy e y dy *Distributed exponentially, like in Poisson event, e.g. y/ has PDF of e y 7 What is Mapping Doing? Generate random deviate p(x)dx dx 0 x 0 otherswise p(x) dx PDF is normalized Let p(y)=f(y), (y), then y(x) = F (x) )(functional inverse), dx/dy = f(y) or x=f(y) F(y) = 0y dy f(y ) Uniform deviate in x 0 y transformed deviate out p(y) = f(y) Allows a random deviate y from a known probability distribution p(y). The indefinite integral of p(y) must be known and invertible. A uniform deviate x is chosen from (0,) such that its corresponding y on the definite-integral curve is desired deviate, i.e. x = F(y). 8 4
5 Interpreting the Mapping Let p(y)=f(y), then y(x) = F (x), dx/dy = f(y) and x=f(y) Uniform deviate in x 0 y transformed deviate out F(y) = 0y dy f(y ) p(y) = f(y) Since F(y) is area under p(y)=f(y), y(x) = F (x) prescribes that Choose x=(0,], then find value y that has that fraction x of area to left of y, or x=f(y) Return that value of y. Example: Drawing from Poisson Distribution y= ln(x)/, p(y)= e y and x=f(y) = e y Note: x= e y or x = - x = e y (which is still x =(0,]). So, indeed, y(x) = ln(x )/ 9 Example: Drawing from Normal Gaussian p(y, y 2,...)dy dy 2... p(x, x 2,...) (x, x 2,...) (y, y 2,...) dy dy 2... Box-Muller method: get random deviates with normal distr. p(y)dy 2 e y 2 /2 dy Consider: uniform deviates (0,), x and x 2, and two quantities y and y 2. where y ln x cos2 x 2 y 2 ln x sin 2 x 2 Or, equivalently, x exp([y 2 y 2 2 ]/2) x 2 2 arctan(y 2 / y ) (x, x 2,...) 2 2 (y, y 2,...) 2 e y /2 2 e y 2 /2 So, each y is independently normal distributed. Better: Pick R 2 v 2 v 2 2 so x R 2 and (v,v 2 ) 2 x 2 Advantage: no sine and cosine by using v / R 2 and v 2 / R 2 and And get two RNG per calculation ( for now, 2 for next time) 0 5
6 Reminder: Gauss Central Limit Theorem Sample N values from p(x)dx, i.e. (X, X 2, X 3, X N ). Estimate mean from y = (/N) x i. What is the pdf of mean? Solve by fourier transforms. If you add together two random variables, you multiply together their characteristic functions: c x (k) e ikx dx P(x)e ikx so c x y (k) c x (k)c y (k) Then c x (k) c (k) N and c (k) c (k / N) N x... x N ) x y x Taylor expand ln[c y (k)] n (ik) n n! n cumulants Cumulants: n Mean = =<x> = x Variance= 2 =<(x-x) 2 > = 2 Skewness = 3 / 3 =<((x-x)/) 3 > Kurtosis= 4 / =<((x-x)/) 4 >-3 n What happens to the reduced d moments? n N n The n= moment remains invariant. The rest get reduced by higher powers of N. lim N c y (k) e ik k 2 2 /2N ik 3 3 /6N 2... P( y) (N /2 2 ) /2 exp N( y )2 2 2 Given enough averaging almost anything becomes a Gaussian distribution. 2 6
7 Approach to normality Gauss Central Limit Thm For any population distribution, the distribution of the mean will approach Gaussian. 3 Conditions on Central Limit Theorem I n x n dx P(x)x n We need the first three moments to exist. If I 0 is not defined not a pdf If I does not exist not mathematically well-posed. If I 2 does not exist infinite variance. Important to know if variance is finite for Monte Carlo. Divergence could happen because of tails of distribution I 2 x 2 dx P(x)x 2 We need: lim x x 3 P(x) 0 Divergence because of singular behavior of P at finite x: lim x0 xp(x) 0 4 7
8 Multidimensional Generalization Suppose r is an m dimensional vector from a multidimensional pdf: p(r)d m r. The mean is defined as before. The variance becomes the covariance, a positive symmetric m x m matrix : V i,j = <(x i -< x i >)(x j -< x j >)> For sufficiently large N, the estimated mean (y) will approach the distribution: P(y)dy 2 /2 N det(v) exp[(y i y i ) Nv 2 (y y j j )] 5 2d histogram of occurrences of means x 2 Off-diagonal components of ij are called the co-variance. Data can be uncorrelated, positively or negatively correlated depending on sign of ij Like a moment of inertia tensor 2 principal axes with variances Find axes with diagonalization or singular value decomposition x Individual error bars on x and x 2 can be misleading if correlated. 6 8
Today: Fundamentals of Monte Carlo
Today: Fundamentals of Monte Carlo What is Monte Carlo? Named at Los Alamos in 1940 s after the casino. Any method which uses (pseudo)random numbers as an essential part of the algorithm. Stochastic -
More informationToday: Fundamentals of Monte Carlo
Today: Fundamentals of Monte Carlo What is Monte Carlo? Named at Los Alamos in 1940 s after the casino. Any method which uses (pseudo)random numbers as an essential part of the algorithm. Stochastic -
More informationSampling Distributions Allen & Tildesley pgs and Numerical Recipes on random numbers
Sampling Distributions Allen & Tildesley pgs. 345-351 and Numerical Recipes on random numbers Today I explain how to generate a non-uniform probability distributions. These are used in importance sampling
More informationData Analysis I. Dr Martin Hendry, Dept of Physics and Astronomy University of Glasgow, UK. 10 lectures, beginning October 2006
Astronomical p( y x, I) p( x, I) p ( x y, I) = p( y, I) Data Analysis I Dr Martin Hendry, Dept of Physics and Astronomy University of Glasgow, UK 10 lectures, beginning October 2006 4. Monte Carlo Methods
More informationNumerical integration - I. M. Peressi - UniTS - Laurea Magistrale in Physics Laboratory of Computational Physics - Unit V
Numerical integration - I M. Peressi - UniTS - Laurea Magistrale in Physics Laboratory of Computational Physics - Unit V deterministic methods in 1D equispaced points (trapezoidal, Simpson...), others...
More information[POLS 8500] Review of Linear Algebra, Probability and Information Theory
[POLS 8500] Review of Linear Algebra, Probability and Information Theory Professor Jason Anastasopoulos ljanastas@uga.edu January 12, 2017 For today... Basic linear algebra. Basic probability. Programming
More informationMonte Carlo Simulations
Monte Carlo Simulations What are Monte Carlo Simulations and why ones them? Pseudo Random Number generators Creating a realization of a general PDF The Bootstrap approach A real life example: LOFAR simulations
More information16 : Markov Chain Monte Carlo (MCMC)
10-708: Probabilistic Graphical Models 10-708, Spring 2014 16 : Markov Chain Monte Carlo MCMC Lecturer: Matthew Gormley Scribes: Yining Wang, Renato Negrinho 1 Sampling from low-dimensional distributions
More informationStructural Reliability
Structural Reliability Thuong Van DANG May 28, 2018 1 / 41 2 / 41 Introduction to Structural Reliability Concept of Limit State and Reliability Review of Probability Theory First Order Second Moment Method
More information2 Functions of random variables
2 Functions of random variables A basic statistical model for sample data is a collection of random variables X 1,..., X n. The data are summarised in terms of certain sample statistics, calculated as
More informationChapter 2. Some Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables
Chapter 2 Some Basic Probability Concepts 2.1 Experiments, Outcomes and Random Variables A random variable is a variable whose value is unknown until it is observed. The value of a random variable results
More informationconditional cdf, conditional pdf, total probability theorem?
6 Multiple Random Variables 6.0 INTRODUCTION scalar vs. random variable cdf, pdf transformation of a random variable conditional cdf, conditional pdf, total probability theorem expectation of a random
More informationNote: Final Exam is at 10:45 on Tuesday, 5/3/11 (This is the Final Exam time reserved for our labs). From Practice Test I
MA Practice Final Answers in Red 4/8/ and 4/9/ Name Note: Final Exam is at :45 on Tuesday, 5// (This is the Final Exam time reserved for our labs). From Practice Test I Consider the integral 5 x dx. Sketch
More informationPhysics 116C The Distribution of the Sum of Random Variables
Physics 116C The Distribution of the Sum of Random Variables Peter Young (Dated: December 2, 2013) Consider a random variable with a probability distribution P(x). The distribution is normalized, i.e.
More informationStochastic Processes for Physicists
Stochastic Processes for Physicists Understanding Noisy Systems Chapter 1: A review of probability theory Paul Kirk Division of Molecular Biosciences, Imperial College London 19/03/2013 1.1 Random variables
More informationAndré Schleife Department of Materials Science and Engineering
André Schleife Department of Materials Science and Engineering Length Scales (c) ICAMS: http://www.icams.de/cms/upload/01_home/01_research_at_icams/length_scales_1024x780.png Goals for today: Background
More informationFrom Random Numbers to Monte Carlo. Random Numbers, Random Walks, Diffusion, Monte Carlo integration, and all that
From Random Numbers to Monte Carlo Random Numbers, Random Walks, Diffusion, Monte Carlo integration, and all that Random Walk Through Life Random Walk Through Life If you flip the coin 5 times you will
More informationP (x). all other X j =x j. If X is a continuous random vector (see p.172), then the marginal distributions of X i are: f(x)dx 1 dx n
JOINT DENSITIES - RANDOM VECTORS - REVIEW Joint densities describe probability distributions of a random vector X: an n-dimensional vector of random variables, ie, X = (X 1,, X n ), where all X is are
More informationUNIVERSITY OF HOUSTON HIGH SCHOOL MATHEMATICS CONTEST Spring 2018 Calculus Test
UNIVERSITY OF HOUSTON HIGH SCHOOL MATHEMATICS CONTEST Spring 2018 Calculus Test NAME: SCHOOL: 1. Let f be some function for which you know only that if 0 < x < 1, then f(x) 5 < 0.1. Which of the following
More informationMonte Carlo Integration. Computer Graphics CMU /15-662, Fall 2016
Monte Carlo Integration Computer Graphics CMU 15-462/15-662, Fall 2016 Talk Announcement Jovan Popovic, Senior Principal Scientist at Adobe Research will be giving a seminar on Character Animator -- Monday
More informationSampling Distributions Allen & Tildesley pgs and Numerical Recipes on random numbers
Sampling Distributions Allen & Tildesley pgs. 345-351 and Numerical Recipes on random numbers Today I explain how to generate a non-uniform probability distributions. These are used in importance sampling
More informationMultivariate Distribution Models
Multivariate Distribution Models Model Description While the probability distribution for an individual random variable is called marginal, the probability distribution for multiple random variables is
More informationGrade: The remainder of this page has been left blank for your workings. VERSION D. Midterm D: Page 1 of 12
First Name: Student-No: Last Name: Section: Grade: The remainder of this page has been left blank for your workings. Midterm D: Page of 2 Indefinite Integrals. 9 marks Each part is worth marks. Please
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationThe Gaussian distribution
The Gaussian distribution Probability density function: A continuous probability density function, px), satisfies the following properties:. The probability that x is between two points a and b b P a
More informationStatistics, Data Analysis, and Simulation SS 2015
Statistics, Data Analysis, and Simulation SS 2015 08.128.730 Statistik, Datenanalyse und Simulation Dr. Michael O. Distler Mainz, 27. April 2015 Dr. Michael O. Distler
More informationPreliminary statistics
1 Preliminary statistics The solution of a geophysical inverse problem can be obtained by a combination of information from observed data, the theoretical relation between data and earth parameters (models),
More informationBayesian Methods with Monte Carlo Markov Chains II
Bayesian Methods with Monte Carlo Markov Chains II Henry Horng-Shing Lu Institute of Statistics National Chiao Tung University hslu@stat.nctu.edu.tw http://tigpbp.iis.sinica.edu.tw/courses.htm 1 Part 3
More information18.440: Lecture 28 Lectures Review
18.440: Lecture 28 Lectures 18-27 Review Scott Sheffield MIT Outline Outline It s the coins, stupid Much of what we have done in this course can be motivated by the i.i.d. sequence X i where each X i is
More informationLecture 2 : CS6205 Advanced Modeling and Simulation
Lecture 2 : CS6205 Advanced Modeling and Simulation Lee Hwee Kuan 21 Aug. 2013 For the purpose of learning stochastic simulations for the first time. We shall only consider probabilities on finite discrete
More informationTransformation of Probability Densities
Transformation of Probability Densities This Wikibook shows how to transform the probability density of a continuous random variable in both the one-dimensional and multidimensional case. In other words,
More informationMonte Carlo and cold gases. Lode Pollet.
Monte Carlo and cold gases Lode Pollet lpollet@physics.harvard.edu 1 Outline Classical Monte Carlo The Monte Carlo trick Markov chains Metropolis algorithm Ising model critical slowing down Quantum Monte
More informationA6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2011
A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2011 Reading Chapter 5 (continued) Lecture 8 Key points in probability CLT CLT examples Prior vs Likelihood Box & Tiao
More informationReduction of Variance. Importance Sampling
Reduction of Variance As we discussed earlier, the statistical error goes as: error = sqrt(variance/computer time). DEFINE: Efficiency = = 1/vT v = error of mean and T = total CPU time How can you make
More informationMultiple Random Variables
Multiple Random Variables Joint Probability Density Let X and Y be two random variables. Their joint distribution function is F ( XY x, y) P X x Y y. F XY ( ) 1, < x
More informationMonte Carlo Methods in High Energy Physics I
Helmholtz International Workshop -- CALC 2009, July 10--20, Dubna Monte Carlo Methods in High Energy Physics CALC2009 - July 20 10, Dubna 2 Contents 3 Introduction Simple definition: A Monte Carlo technique
More informationRandom processes and probability distributions. Phys 420/580 Lecture 20
Random processes and probability distributions Phys 420/580 Lecture 20 Random processes Many physical processes are random in character: e.g., nuclear decay (Poisson distributed event count) P (k, τ) =
More informationCheng Soon Ong & Christian Walder. Canberra February June 2018
Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2018 Outlines Overview Introduction Linear Algebra Probability Linear Regression
More information8 STOCHASTIC SIMULATION
8 STOCHASTIC SIMULATIO 59 8 STOCHASTIC SIMULATIO Whereas in optimization we seek a set of parameters x to minimize a cost, or to maximize a reward function J( x), here we pose a related but different question.
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationNumerical integration and importance sampling
2 Numerical integration and importance sampling 2.1 Quadrature Consider the numerical evaluation of the integral I(a, b) = b a dx f(x) Rectangle rule: on small interval, construct interpolating function
More informationSeries Solution of Linear Ordinary Differential Equations
Series Solution of Linear Ordinary Differential Equations Department of Mathematics IIT Guwahati Aim: To study methods for determining series expansions for solutions to linear ODE with variable coefficients.
More informationEE4601 Communication Systems
EE4601 Communication Systems Week 2 Review of Probability, Important Distributions 0 c 2011, Georgia Institute of Technology (lect2 1) Conditional Probability Consider a sample space that consists of two
More informationWinter 2019 Math 106 Topics in Applied Mathematics. Lecture 9: Markov Chain Monte Carlo
Winter 2019 Math 106 Topics in Applied Mathematics Data-driven Uncertainty Quantification Yoonsang Lee (yoonsang.lee@dartmouth.edu) Lecture 9: Markov Chain Monte Carlo 9.1 Markov Chain A Markov Chain Monte
More informationMonte Carlo integration (naive Monte Carlo)
Monte Carlo integration (naive Monte Carlo) Example: Calculate π by MC integration [xpi] 1/21 INTEGER n total # of points INTEGER i INTEGER nu # of points in a circle REAL x,y coordinates of a point in
More informationMonte Carlo. Lecture 15 4/9/18. Harvard SEAS AP 275 Atomistic Modeling of Materials Boris Kozinsky
Monte Carlo Lecture 15 4/9/18 1 Sampling with dynamics In Molecular Dynamics we simulate evolution of a system over time according to Newton s equations, conserving energy Averages (thermodynamic properties)
More informationE[X n ]= dn dt n M X(t). ). What is the mgf? Solution. Found this the other day in the Kernel matching exercise: 1 M X (t) =
Chapter 7 Generating functions Definition 7.. Let X be a random variable. The moment generating function is given by M X (t) =E[e tx ], provided that the expectation exists for t in some neighborhood of
More informationAP Calculus Chapter 9: Infinite Series
AP Calculus Chapter 9: Infinite Series 9. Sequences a, a 2, a 3, a 4, a 5,... Sequence: A function whose domain is the set of positive integers n = 2 3 4 a n = a a 2 a 3 a 4 terms of the sequence Begin
More informationSUMMARY OF PROBABILITY CONCEPTS SO FAR (SUPPLEMENT FOR MA416)
SUMMARY OF PROBABILITY CONCEPTS SO FAR (SUPPLEMENT FOR MA416) D. ARAPURA This is a summary of the essential material covered so far. The final will be cumulative. I ve also included some review problems
More informationNumerical Methods I Monte Carlo Methods
Numerical Methods I Monte Carlo Methods Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course G63.2010.001 / G22.2420-001, Fall 2010 Dec. 9th, 2010 A. Donev (Courant Institute) Lecture
More informationIntroduction. Microscopic Theories of Nuclear Structure, Dynamics and Electroweak Currents June 12-30, 2017, ECT*, Trento, Italy
Introduction Stefano Gandolfi Los Alamos National Laboratory (LANL) Microscopic Theories of Nuclear Structure, Dynamics and Electroweak Currents June 12-30, 2017, ECT*, Trento, Italy Stefano Gandolfi (LANL)
More informationMath Exam III - Spring
Math 3 - Exam III - Spring 8 This exam contains 5 multiple choice questions and hand graded questions. The multiple choice questions are worth 5 points each and the hand graded questions are worth a total
More informationA Probability Review
A Probability Review Outline: A probability review Shorthand notation: RV stands for random variable EE 527, Detection and Estimation Theory, # 0b 1 A Probability Review Reading: Go over handouts 2 5 in
More informationfunctions Poisson distribution Normal distribution Arbitrary functions
Physics 433: Computational Physics Lecture 6 Random number distributions Generation of random numbers of various distribuition functions Normal distribution Poisson distribution Arbitrary functions Random
More informationSTA 256: Statistics and Probability I
Al Nosedal. University of Toronto. Fall 2017 My momma always said: Life was like a box of chocolates. You never know what you re gonna get. Forrest Gump. There are situations where one might be interested
More informationcomponent risk analysis
273: Urban Systems Modeling Lec. 3 component risk analysis instructor: Matteo Pozzi 273: Urban Systems Modeling Lec. 3 component reliability outline risk analysis for components uncertain demand and uncertain
More informationLet X and Y denote two random variables. The joint distribution of these random
EE385 Class Notes 9/7/0 John Stensby Chapter 3: Multiple Random Variables Let X and Y denote two random variables. The joint distribution of these random variables is defined as F XY(x,y) = [X x,y y] P.
More informationECE295, Data Assimila0on and Inverse Problems, Spring 2015
ECE295, Data Assimila0on and Inverse Problems, Spring 2015 1 April, Intro; Linear discrete Inverse problems (Aster Ch 1 and 2) Slides 8 April, SVD (Aster ch 2 and 3) Slides 15 April, RegularizaFon (ch
More information( x) ( ) F ( ) ( ) ( ) Prob( ) ( ) ( ) X x F x f s ds
Applied Numerical Analysis Pseudo Random Number Generator Lecturer: Emad Fatemizadeh What is random number: A sequence in which each term is unpredictable 29, 95, 11, 60, 22 Application: Monte Carlo Simulations
More informationStochastic Processes. M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno
Stochastic Processes M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno 1 Outline Stochastic (random) processes. Autocorrelation. Crosscorrelation. Spectral density function.
More informationLecture 6: Monte-Carlo methods
Miranda Holmes-Cerfon Applied Stochastic Analysis, Spring 2015 Lecture 6: Monte-Carlo methods Readings Recommended: handout on Classes site, from notes by Weinan E, Tiejun Li, and Eric Vanden-Eijnden Optional:
More informationp(z)
Chapter Statistics. Introduction This lecture is a quick review of basic statistical concepts; probabilities, mean, variance, covariance, correlation, linear regression, probability density functions and
More informationMA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
More information2 (Statistics) Random variables
2 (Statistics) Random variables References: DeGroot and Schervish, chapters 3, 4 and 5; Stirzaker, chapters 4, 5 and 6 We will now study the main tools use for modeling experiments with unknown outcomes
More informationELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process
Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Definition of stochastic process (random
More information3 Operations on One R.V. - Expectation
0402344 Engineering Dept.-JUST. EE360 Signal Analysis-Electrical - Electrical Engineering Department. EE360-Random Signal Analysis-Electrical Engineering Dept.-JUST. EE360-Random Signal Analysis-Electrical
More informationExponential Family and Maximum Likelihood, Gaussian Mixture Models and the EM Algorithm. by Korbinian Schwinger
Exponential Family and Maximum Likelihood, Gaussian Mixture Models and the EM Algorithm by Korbinian Schwinger Overview Exponential Family Maximum Likelihood The EM Algorithm Gaussian Mixture Models Exponential
More informationReview: mostly probability and some statistics
Review: mostly probability and some statistics C2 1 Content robability (should know already) Axioms and properties Conditional probability and independence Law of Total probability and Bayes theorem Random
More informationMarkov Chains CK eqns Classes Hitting times Rec./trans. Strong Markov Stat. distr. Reversibility * Markov Chains
Markov Chains A random process X is a family {X t : t T } of random variables indexed by some set T. When T = {0, 1, 2,... } one speaks about a discrete-time process, for T = R or T = [0, ) one has a continuous-time
More informationRandom Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R
In probabilistic models, a random variable is a variable whose possible values are numerical outcomes of a random phenomenon. As a function or a map, it maps from an element (or an outcome) of a sample
More informationProbability and Statistics
Probability and Statistics 1 Contents some stochastic processes Stationary Stochastic Processes 2 4. Some Stochastic Processes 4.1 Bernoulli process 4.2 Binomial process 4.3 Sine wave process 4.4 Random-telegraph
More informationMath 122L. Additional Homework Problems. Prepared by Sarah Schott
Math 22L Additional Homework Problems Prepared by Sarah Schott Contents Review of AP AB Differentiation Topics 4 L Hopital s Rule and Relative Rates of Growth 6 Riemann Sums 7 Definition of the Definite
More informationDennis Bricker Dept of Mechanical & Industrial Engineering The University of Iowa
Dennis Bricker Dept of Mechanical & Industrial Engineering The University of Iowa dennis-bricker@uiowa.edu Probability Theory Results page 1 D.Bricker, U. of Iowa, 2002 Probability of simultaneous occurrence
More informationChapter 2. Random Variable. Define single random variables in terms of their PDF and CDF, and calculate moments such as the mean and variance.
Chapter 2 Random Variable CLO2 Define single random variables in terms of their PDF and CDF, and calculate moments such as the mean and variance. 1 1. Introduction In Chapter 1, we introduced the concept
More informationExpectation. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda
Expectation DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean,
More informationPhysics 403. Segev BenZvi. Parameter Estimation, Correlations, and Error Bars. Department of Physics and Astronomy University of Rochester
Physics 403 Parameter Estimation, Correlations, and Error Bars Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Review of Last Class Best Estimates and Reliability
More informationMonte Carlo Integration I
Monte Carlo Integration I Digital Image Synthesis Yung-Yu Chuang with slides by Pat Hanrahan and Torsten Moller Introduction L o p,ωo L e p,ωo s f p,ωo,ω i L p,ω i cosθ i i d The integral equations generally
More informationContinuous Random Variables
Continuous Random Variables Recall: For discrete random variables, only a finite or countably infinite number of possible values with positive probability. Often, there is interest in random variables
More informationQuiz Solutions, Math 136, Sec 6, 7, 16. Spring, Warm-up Problems - Review
Quiz Solutions, Math 36, Sec 6, 7, 6. Spring, Warm-up Problems - Review. Find appropriate substitutions to evaluate the following indefinite integrals: a) b) t + 4) 3 dt. Set u t + 4. We have du dt t.
More informationChapter 5,6 Multiple RandomVariables
Chapter 5,6 Multiple RandomVariables ENCS66 - Probabilityand Stochastic Processes Concordia University Vector RandomVariables A vector r.v. is a function where is the sample space of a random experiment.
More informationDimensionality Reduction and Principle Components
Dimensionality Reduction and Principle Components Ken Kreutz-Delgado (Nuno Vasconcelos) UCSD ECE Department Winter 2012 Motivation Recall, in Bayesian decision theory we have: World: States Y in {1,...,
More informationMATH 307: Problem Set #3 Solutions
: Problem Set #3 Solutions Due on: May 3, 2015 Problem 1 Autonomous Equations Recall that an equilibrium solution of an autonomous equation is called stable if solutions lying on both sides of it tend
More information1 Review of di erential calculus
Review of di erential calculus This chapter presents the main elements of di erential calculus needed in probability theory. Often, students taking a course on probability theory have problems with concepts
More informationChapter 3, 4 Random Variables ENCS Probability and Stochastic Processes. Concordia University
Chapter 3, 4 Random Variables ENCS6161 - Probability and Stochastic Processes Concordia University ENCS6161 p.1/47 The Notion of a Random Variable A random variable X is a function that assigns a real
More informationECE Lecture #10 Overview
ECE 450 - Lecture #0 Overview Introduction to Random Vectors CDF, PDF Mean Vector, Covariance Matrix Jointly Gaussian RV s: vector form of pdf Introduction to Random (or Stochastic) Processes Definitions
More informationInfinite series, improper integrals, and Taylor series
Chapter 2 Infinite series, improper integrals, and Taylor series 2. Introduction to series In studying calculus, we have explored a variety of functions. Among the most basic are polynomials, i.e. functions
More informationL2: Review of probability and statistics
Probability L2: Review of probability and statistics Definition of probability Axioms and properties Conditional probability Bayes theorem Random variables Definition of a random variable Cumulative distribution
More informationLecture 1: August 28
36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 1: August 28 Our broad goal for the first few lectures is to try to understand the behaviour of sums of independent random
More informationPart IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationChapter 4. Continuous Random Variables 4.1 PDF
Chapter 4 Continuous Random Variables In this chapter we study continuous random variables. The linkage between continuous and discrete random variables is the cumulative distribution (CDF) which we will
More informationReview of Probability Theory
Review of Probability Theory Arian Maleki and Tom Do Stanford University Probability theory is the study of uncertainty Through this class, we will be relying on concepts from probability theory for deriving
More informationIf the objects are replaced there are n choices each time yielding n r ways. n C r and in the textbook by g(n, r).
Caveat: Not proof read. Corrections appreciated. Combinatorics In the following, n, n 1, r, etc. will denote non-negative integers. Rule 1 The number of ways of ordering n distinguishable objects (also
More information3. Probability and Statistics
FE661 - Statistical Methods for Financial Engineering 3. Probability and Statistics Jitkomut Songsiri definitions, probability measures conditional expectations correlation and covariance some important
More informationChapter 3: Random Variables 1
Chapter 3: Random Variables 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.
More informationBrandon C. Kelly (Harvard Smithsonian Center for Astrophysics)
Brandon C. Kelly (Harvard Smithsonian Center for Astrophysics) Probability quantifies randomness and uncertainty How do I estimate the normalization and logarithmic slope of a X ray continuum, assuming
More informationName Class. (a) (b) (c) 4 t4 3 C
Chapter 4 Test Bank 77 Test Form A Chapter 4 Name Class Date Section. Evaluate the integral: t dt. t C (a) (b) 4 t4 C t C C t. Evaluate the integral: 5 sec x tan x dx. (a) 5 sec x tan x C (b) 5 sec x C
More informationAlgorithms for Uncertainty Quantification
Algorithms for Uncertainty Quantification Tobias Neckel, Ionuț-Gabriel Farcaș Lehrstuhl Informatik V Summer Semester 2017 Lecture 2: Repetition of probability theory and statistics Example: coin flip Example
More informationExpectation. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Expectation DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean, variance,
More informationBrief introduction to Markov Chain Monte Carlo
Brief introduction to Department of Probability and Mathematical Statistics seminar Stochastic modeling in economics and finance November 7, 2011 Brief introduction to Content 1 and motivation Classical
More informationLECTURE NOTES ELEMENTARY NUMERICAL METHODS. Eusebius Doedel
LECTURE NOTES on ELEMENTARY NUMERICAL METHODS Eusebius Doedel TABLE OF CONTENTS Vector and Matrix Norms 1 Banach Lemma 20 The Numerical Solution of Linear Systems 25 Gauss Elimination 25 Operation Count
More information