Exercise 5 Release: Due:
|
|
- Erika Harper
- 5 years ago
- Views:
Transcription
1 Stochastic Modeling and Simulation Winter 28 Prof. Dr. I. F. Sbalzarini, Dr. Christoph Zechner (MPI-CBG/CSBD TU Dresden, 87 Dresden, Germany Exercise 5 Release: Due: Question : Variance of MC Integration vs Importance Sampling a umerically compute the mean and variance of (x2 dx using simple MC integration with a standard uniform PDF proposal p(x and importance sampling with the proposal PDF q(x = 2x, ( x. Use trials with = samples for each method. Compare the mean and variances with the analytical solution (see lecture notes. For simple Monte Carlo integration, the estimate is: x 2 dx Θ = x 2 i Analytically, the solution is: x 2 dx = x = The variance of Θ is: { V ar{θ = V ar x 2 i = V ar{x2 = ( E[x 4 ] E[x 2 ] 2 ( = ( x 4 p(xdx x 2 p(xdx = ( x5 x 2 5 ( = ( 5 = ( 4 45
2 import numpy a s np 2 import s c i p y. s t a t s as s s import m a t p l o t l i b. pyplot as p l t 4 from s c i p y. s p e c i a l import e r f 5 from s c i p y. s p e c i a l import e r f i n v 6 7 # simple mc i n t e g r a t i o n 8 def mci ( a, b, n : 9 x = np. random. uniform ( a, b, n f x = x 2 i n t e g r a l = (b a /n np. sum( f x 2 return i n t e g r a l 4 r e s u l t s = np. array ( [ ] 5 f o r i in range ( : 6 r e s u l t s = np. append ( r e s u l t s, mci (,, 7 r e s u l t s. std ( 2 # v a r i a n c e 8 r e s u l t s. mean ( # mean Using simple Monte Carlo integration, the mean and variance are.697 and.8655, respectively. The variance roughly matches the analytical solution Θ = ( 4 45 = for =. For importance sampling with a proposal distribution of q(x = 2x: x 2 dx Θ 2 = x 2 i q(x i The variance of Θ 2 is: { { V ar{θ 2 x 2 i = V ar = V ar 2x i = { x V ar = 2 4 V ar{x = ( E[x 2 ] E[x] 2 4 = 4 = 4 = 4 = 4 ( ( x 2 q(xdx ( ( x 2 (2xdx 2x4 4 ( 2 ( 2x 2 ( 2 = x i 2 xq(xdx x(2xdx ( 72 2
3 # importance sampling 2 def impsam ( a, b, n : x = np. random. uniform (,, n 4 y = np. s q r t ( x 5 f y = ( y 2 /(2 y 6 i n t e g r a l = /n np. sum( f y 7 return i n t e g r a l 8 9 r e s u l t s = np. array ( [ ] f o r i in range ( : r e s u l t s = np. append ( r e s u l t s, impsam (,, 2 r e s u l t s. std ( 2 # v a r i a n c e r e s u l t s. mean ( # mean With importance sampling, the mean and variance are.2826 and , respectively. The variance from importance sampling is lower than from simple Monte Carlo integration. b Compute the variance from using simple MC integration with a uniform proposal distribution from X U(, and importance sampling with a normal proposal distribution from X (5, 4 to estimate (x x2 dx. Solve this problem analytically and numerically using trials with = samples for each method. You may use numerical solvers to evaluate definite integrals in the analytical solution. The analytical solution of (x x2 dx is: ( x (x x 2 2 dx = 2 x = = 5 2 For simple MC integration, the variance is: { V ar{θ = V ar (x i x 2 i = ( E[(x x 2 ] (E[(x x 2 ] = = = ( (x x 2 p(xdx ( (x x 2 dx ( ( 5 = V ar{x x2 ( ( = 5 9 (x x 2 p(xdx x x 2 dx
4 # simple mc i n t e g r a t i o n 2 def mci ( a, b, n : x = np. random. uniform ( a, b, n 4 f x = x x 2 5 i n t e g r a l = (b a /n np. sum( f x 6 return i n t e g r a l 7 8 r e s u l t s = np. array ( [ ] 9 f o r i in range ( : r e s u l t s = np. append ( r e s u l t s, mci (,, r e s u l t s. std ( 2 # v a r i a n c e 2 r e s u l t s. mean ( # mean Using simple Monte Carlo integration with = samples, the mean and variance are and , respectively. The analytical solution of variance is 5/9 = For importance sampling, the variance is: { V ar{θ 2 (x i x 2 i = V ar q(x i { (x i x 2 i = V ar q(x i = { (x x 2 V ar q(x ( [ ((x = ] x 2 2 [ ] (x x 2 2 E E q(x q(x ( = ( (x x 2 2 ( (x x 2 q(xdx q(x q(x ( = ( (x x 2 dx (x x 2 dx q(x = ( (x x 2 dx (x x 2 dx (x µ2 e 2σ 2 2πσ 2 ( = ( = 2.52 q(xdx 4
5 # importance sampling 2 def impsam ( a, b, n : x = np. random. normal ( 5, 2, n 4 # x only within bounds 5 x = x [ np. where ( ( x >= & ( x <= ] 6 # q ( x i s normalized to account f o r bounds 7 q = s s. norm. pdf ( x, l o c =5, s c a l e =2/ e r f (5/(2 np. s q r t ( 2 8 f x = ( x x 2 /q 9 i n t e g r a l = /( l e n ( x np. sum( f x return i n t e g r a l 2 r e s u l t s = np. array ( [ ] f o r i in range ( : 4 r e s u l t s = np. append ( r e s u l t s, impsam (,, 5 r e s u l t s. std ( 2 # v a r i a n c e 6 r e s u l t s. mean ( # mean Using importance sampling with = samples, the mean and variance are and.525, respectively. The analytical solution of variance is 2.52/ =.25, which is smaller than the variance from simple Monte Carlo integration. Question 2: Optimum Proposal Distribution Find the optimum σ 2 for the proposal distribution q(x (.5, σ 2 to estimate the CDF of X Beta(2, 2 by importance sampling. Solve this problem numerically by scanning through σ 2 values from. to. in increments of (../2. Use trials with = samples to obtain the variances. The normal distribution (.5, σ 2 that most closely matches the Beta(2, 2 distribution will be the best proposal distribution to use. # p l o t s normal d i s t r i b u t i o n s and a beta d i s t r i b u t i o n 2 x = np. l i n s p a c e (,, 2 f o r i in norm vars : 4 y = s s. norm. pdf ( x, l o c =. 5, s c a l e = np. s q r t ( i 5 p l t. p l o t ( x, y 6 y = s s. beta. pdf ( x, a = 2, b = 2 7 p l t. p l o t ( x, y 8 p l t. show ( 5
6 # importance sampling 2 def importance sampling (n, var : x = np. random. normal ( l o c =. 5, s c a l e = np. s q r t ( var, s i z e = n 4 # x only within bounds 5 x = x [ np. where ( ( x >= & ( x <= ] 6 # n o r m a l i z a t i o n f a c t o r k 7 k =. 5 ( e r f ((.5 /( np. s q r t (2 var +e r f (. 5 / np. s q r t (2 var 8 # normalized q ( x 9 q = s s. norm. pdf ( x, l o c =. 5, s c a l e = np. s q r t ( var /k f x = s s. beta. pdf ( x, a = 2, b = 2 /q i n t e g r a l = / l e n ( x np. sum( f x 2 return i n t e g r a l 4 # run importance sampling at d i f f e r e n t v a r i a n c e s 5 norm vars = np. l i n s p a c e (.,., 2 6 mc vars = np. array ( [ ] 7 f o r i in norm vars : 8 r e s u l t s = np. array ( [ ] 9 f o r j in range ( : 2 r e s u l t s = np. append ( r e s u l t s, importance sampling (, i 2 mc vars = np. append ( mc vars, r e s u l t s. std ( p l t. p l o t ( norm vars, mc vars 24 p l t. show ( The importance sampling estimate variance was lowest using a proposal distribution of (.5,
7 Question : Variance Reduction by Antithetic Variates a Estimate (/(x+dx using MC integration with 2 = samples from a uniform proposal distribution X U(, and with = 5 samples from X together with its antithetic variates X. Compare the mean and variance between the two approaches from trials. # simple MC i n t e g r a t i o n 2 def mci ( a, b, n : x = np. random. uniform ( a, b, n 4 f x = /( x+ 5 i n t e g r a l = (b a /n np. sum( f x 6 return i n t e g r a l 7 8 r e s u l t s = np. array ( [ ] 9 f o r i in range ( : r e s u l t s = np. append ( r e s u l t s, mci (,, p r i n t ( r e s u l t s. mean ( 2 p r i n t ( r e s u l t s. std ( 2 Mean and variance from simple Monte Carlo integration are and.989, respectively. # using a n t i t h e t i c v a r i a t e s 2 def m c i a n t i ( a, b, n : x = np. random. uniform ( a, b, i n t ( n/2 4 a n t i x = x # a n t i t h e t i c v a r i a t e s 5 a l l x = np. append ( x, a n t i x # combine a l l x 6 f x = /( a l l x + 7 i n t e g r a l = (b a /n np. sum( f x 8 return i n t e g r a l 9 r e s u l t s = np. array ( [ ] f o r i in range ( : 2 r e s u l t s = np. append ( r e s u l t s, m c i a n t i (,, p r i n t ( r e s u l t s. mean ( 4 p r i n t ( r e s u l t s. std ( 2 Mean and variance from simple Monte Carlo integration using antithetic variates are and.88, respectively. Variance is lower with antithetic variates. 7
8 b Estimate (x x2 dx by importance sampling with a proposal distribution p(x (5, 4 using antithetic variates from a uniform distribution. Use trials and a total of 2 = samples to obtain the importance sampling variance. How different is it from the variance obtained in question b? Hint: Generate random numbers and their antithetic variates from a uniform distribution and do a transformation to a normal distribution for importance sampling OR generate antithetic variates of random numbers from a normal distribution by getting their mirror image values from the mean. # importance sampling with a n t i t h e t i c v a r i a t e s 2 def impsam anti ( a, b, n : x = np. random. uniform (,, i n t ( n/2 4 a n t i x = x 5 a l l x = np. append ( x, a n t i x 6 y = 2 np. s q r t ( 2 e r f i n v (2 a l l x +5 7 y = y [ np. where ( ( y >= & ( y <= ] 8 q = s s. norm. pdf ( y, l o c =5, s c a l e =2/ e r f (5/(2 np. s q r t ( 2 9 f y = ( y y 2 /q i n t e g r a l = /( l e n ( y np. sum( f y return i n t e g r a l 2 r e s u l t s = np. array ( [ ] 4 f o r i in range ( : 5 r e s u l t s = np. append ( r e s u l t s, impsam anti (,, 6 p r i n t ( r e s u l t s. mean ( 7 p r i n t ( r e s u l t s. std ( 2 Mean and variance from importance sampling using antithetic variates are and 59.97, respectively. The variance is higher as compared to the importance sampling in b. Antithetic variates don t reduce the variance of Monte Carlo integration estimates when integrating non-monotonic functions. 8
A Probability Primer. A random walk down a probabilistic path leading to some stochastic thoughts on chance events and uncertain outcomes.
A Probability Primer A random walk down a probabilistic path leading to some stochastic thoughts on chance events and uncertain outcomes. Are you holding all the cards?? Random Events A random event, E,
More informationLecture 17: The Exponential and Some Related Distributions
Lecture 7: The Exponential and Some Related Distributions. Definition Definition: A continuous random variable X is said to have the exponential distribution with parameter if the density of X is e x if
More information1 Expectation of a continuously distributed random variable
OCTOBER 3, 204 LECTURE 9 EXPECTATION OF A CONTINUOUSLY DISTRIBUTED RANDOM VARIABLE, DISTRIBUTION FUNCTION AND CHANGE-OF-VARIABLE TECHNIQUES Expectation of a continuously distributed random variable Recall
More information14 : Approximate Inference Monte Carlo Methods
10-708: Probabilistic Graphical Models 10-708, Spring 2018 14 : Approximate Inference Monte Carlo Methods Lecturer: Kayhan Batmanghelich Scribes: Biswajit Paria, Prerna Chiersal 1 Introduction We have
More informationStochastic processes and Data mining
Stochastic processes and Data mining Meher Krishna Patel Created on : Octorber, 2017 Last updated : May, 2018 More documents are freely available at PythonDSP Table of contents Table of contents i 1 The
More informationChapter 4 Multiple Random Variables
Review for the previous lecture Theorems and Examples: How to obtain the pmf (pdf) of U = g ( X Y 1 ) and V = g ( X Y) Chapter 4 Multiple Random Variables Chapter 43 Bivariate Transformations Continuous
More informationProbability and Distributions
Probability and Distributions What is a statistical model? A statistical model is a set of assumptions by which the hypothetical population distribution of data is inferred. It is typically postulated
More informationSystem Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models
System Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models Fatih Cavdur fatihcavdur@uludag.edu.tr March 20, 2012 Introduction Introduction The world of the model-builder
More informationp. 6-1 Continuous Random Variables p. 6-2
Continuous Random Variables Recall: For discrete random variables, only a finite or countably infinite number of possible values with positive probability (>). Often, there is interest in random variables
More informationChapter 4 - Lecture 3 The Normal Distribution
Chapter 4 - Lecture 3 The October 28th, 2009 Chapter 4 - Lecture 3 The Standard Chapter 4 - Lecture 3 The Standard Normal distribution is a statistical unicorn It is the most important distribution in
More informationLecture 3. Probability - Part 2. Luigi Freda. ALCOR Lab DIAG University of Rome La Sapienza. October 19, 2016
Lecture 3 Probability - Part 2 Luigi Freda ALCOR Lab DIAG University of Rome La Sapienza October 19, 2016 Luigi Freda ( La Sapienza University) Lecture 3 October 19, 2016 1 / 46 Outline 1 Common Continuous
More informationSimulation. Alberto Ceselli MSc in Computer Science Univ. of Milan. Part 4 - Statistical Analysis of Simulated Data
Simulation Alberto Ceselli MSc in Computer Science Univ. of Milan Part 4 - Statistical Analysis of Simulated Data A. Ceselli Simulation P.4 Analysis of Sim. data 1 / 15 Statistical analysis of simulated
More informationDistributions of Functions of Random Variables. 5.1 Functions of One Random Variable
Distributions of Functions of Random Variables 5.1 Functions of One Random Variable 5.2 Transformations of Two Random Variables 5.3 Several Random Variables 5.4 The Moment-Generating Function Technique
More informationLecture 4. Continuous Random Variables and Transformations of Random Variables
Math 408 - Mathematical Statistics Lecture 4. Continuous Random Variables and Transformations of Random Variables January 25, 2013 Konstantin Zuev (USC) Math 408, Lecture 4 January 25, 2013 1 / 13 Agenda
More informationF X (x) = P [X x] = x f X (t)dt. 42 Lebesgue-a.e, to be exact 43 More specifically, if g = f Lebesgue-a.e., then g is also a pdf for X.
10.2 Properties of PDF and CDF for Continuous Random Variables 10.18. The pdf f X is determined only almost everywhere 42. That is, given a pdf f for a random variable X, if we construct a function g by
More informationReview for the previous lecture
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 Review for the previous lecture Definition: Several discrete distributions, including discrete uniform, hypergeometric, Bernoulli,
More informationChapter 4: Continuous Random Variables and Probability Distributions
Chapter 4: and Probability Distributions Walid Sharabati Purdue University February 14, 2014 Professor Sharabati (Purdue University) Spring 2014 (Slide 1 of 37) Chapter Overview Continuous random variables
More informationChapter 3: Random Variables 1
Chapter 3: Random Variables 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.
More informationStochastic Simulation Variance reduction methods Bo Friis Nielsen
Stochastic Simulation Variance reduction methods Bo Friis Nielsen Applied Mathematics and Computer Science Technical University of Denmark 2800 Kgs. Lyngby Denmark Email: bfni@dtu.dk Variance reduction
More informationContinuous Random Variables. and Probability Distributions. Continuous Random Variables and Probability Distributions ( ) ( ) Chapter 4 4.
UCLA STAT 11 A Applied Probability & Statistics for Engineers Instructor: Ivo Dinov, Asst. Prof. In Statistics and Neurology Teaching Assistant: Christopher Barr University of California, Los Angeles,
More informationHW Solution 12 Due: Dec 2, 9:19 AM
ECS 315: Probability and Random Processes 2015/1 HW Solution 12 Due: Dec 2, 9:19 AM Lecturer: Prapun Suksompong, Ph.D. Problem 1. Let X E(3). (a) For each of the following function g(x). Indicate whether
More informationBayesian Inference and MCMC
Bayesian Inference and MCMC Aryan Arbabi Partly based on MCMC slides from CSC412 Fall 2018 1 / 18 Bayesian Inference - Motivation Consider we have a data set D = {x 1,..., x n }. E.g each x i can be the
More information16 : Markov Chain Monte Carlo (MCMC)
10-708: Probabilistic Graphical Models 10-708, Spring 2014 16 : Markov Chain Monte Carlo MCMC Lecturer: Matthew Gormley Scribes: Yining Wang, Renato Negrinho 1 Sampling from low-dimensional distributions
More informationfunctions Poisson distribution Normal distribution Arbitrary functions
Physics 433: Computational Physics Lecture 6 Random number distributions Generation of random numbers of various distribuition functions Normal distribution Poisson distribution Arbitrary functions Random
More informationContinuous Random Variables. and Probability Distributions. Continuous Random Variables and Probability Distributions ( ) ( )
UCLA STAT 35 Applied Computational and Interactive Probability Instructor: Ivo Dinov, Asst. Prof. In Statistics and Neurology Teaching Assistant: Chris Barr Continuous Random Variables and Probability
More informationMonte Carlo Methods. Leon Gu CSD, CMU
Monte Carlo Methods Leon Gu CSD, CMU Approximate Inference EM: y-observed variables; x-hidden variables; θ-parameters; E-step: q(x) = p(x y, θ t 1 ) M-step: θ t = arg max E q(x) [log p(y, x θ)] θ Monte
More informationS6880 #13. Variance Reduction Methods
S6880 #13 Variance Reduction Methods 1 Variance Reduction Methods Variance Reduction Methods 2 Importance Sampling Importance Sampling 3 Control Variates Control Variates Cauchy Example Revisited 4 Antithetic
More informationECE 302 Division 2 Exam 2 Solutions, 11/4/2009.
NAME: ECE 32 Division 2 Exam 2 Solutions, /4/29. You will be required to show your student ID during the exam. This is a closed-book exam. A formula sheet is provided. No calculators are allowed. Total
More informationSTAT 430/510: Lecture 15
STAT 430/510: Lecture 15 James Piette June 23, 2010 Updates HW4 is up on my website. It is due next Mon. (June 28th). Starting today back at section 6.4... Conditional Distribution: Discrete Def: The conditional
More informationQuick Tour of Basic Probability Theory and Linear Algebra
Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra CS224w: Social and Information Network Analysis Fall 2011 Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra Outline Definitions
More informationContinuous distributions
CHAPTER 7 Continuous distributions 7.. Introduction A r.v. X is said to have a continuous distribution if there exists a nonnegative function f such that P(a X b) = ˆ b a f(x)dx for every a and b. distribution.)
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Introduction to Probability and Statistics Lecture 14: Continuous random variables Purnamrita Sarkar Department of Statistics and Data Science The University of Texas at Austin www.cs.cmu.edu/
More information2 Continuous Random Variables and their Distributions
Name: Discussion-5 1 Introduction - Continuous random variables have a range in the form of Interval on the real number line. Union of non-overlapping intervals on real line. - We also know that for any
More informationon t0 t T, how can one compute the value E[g(X(T ))]? The Monte-Carlo method is based on the
SPRING 2008, CSC KTH - Numerical methods for SDEs, Szepessy, Tempone 203 Monte Carlo Euler for SDEs Consider the stochastic differential equation dx(t) = a(t, X(t))dt + b(t, X(t))dW (t) on t0 t T, how
More informationProbability and Estimation. Alan Moses
Probability and Estimation Alan Moses Random variables and probability A random variable is like a variable in algebra (e.g., y=e x ), but where at least part of the variability is taken to be stochastic.
More informationLecture 1: August 28
36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 1: August 28 Our broad goal for the first few lectures is to try to understand the behaviour of sums of independent random
More informationComputer Intensive Methods in Mathematical Statistics
Computer Intensive Methods in Mathematical Statistics Department of mathematics johawes@kth.se Lecture 5 Sequential Monte Carlo methods I 31 March 2017 Computer Intensive Methods (1) Plan of today s lecture
More informationThe Binomial distribution. Probability theory 2. Example. The Binomial distribution
Probability theory Tron Anders Moger September th 7 The Binomial distribution Bernoulli distribution: One experiment X i with two possible outcomes, probability of success P. If the experiment is repeated
More informationIEOR 3106: Introduction to Operations Research: Stochastic Models. Professor Whitt. SOLUTIONS to Homework Assignment 2
IEOR 316: Introduction to Operations Research: Stochastic Models Professor Whitt SOLUTIONS to Homework Assignment 2 More Probability Review: In the Ross textbook, Introduction to Probability Models, read
More informationLecture 7 and 8: Markov Chain Monte Carlo
Lecture 7 and 8: Markov Chain Monte Carlo 4F13: Machine Learning Zoubin Ghahramani and Carl Edward Rasmussen Department of Engineering University of Cambridge http://mlg.eng.cam.ac.uk/teaching/4f13/ Ghahramani
More informationVariance reduction. Michel Bierlaire. Transport and Mobility Laboratory. Variance reduction p. 1/18
Variance reduction p. 1/18 Variance reduction Michel Bierlaire michel.bierlaire@epfl.ch Transport and Mobility Laboratory Variance reduction p. 2/18 Example Use simulation to compute I = 1 0 e x dx We
More informationAnswers and expectations
Answers and expectations For a function f(x) and distribution P(x), the expectation of f with respect to P is The expectation is the average of f, when x is drawn from the probability distribution P E
More informationChapter 4. Continuous Random Variables 4.1 PDF
Chapter 4 Continuous Random Variables In this chapter we study continuous random variables. The linkage between continuous and discrete random variables is the cumulative distribution (CDF) which we will
More informationStatistical Computing: Exam 1 Chapters 1, 3, 5
Statistical Computing: Exam Chapters, 3, 5 Instructions: Read each question carefully before determining the best answer. Show all work; supporting computer code and output must be attached to the question
More information1 Random Variable: Topics
Note: Handouts DO NOT replace the book. In most cases, they only provide a guideline on topics and an intuitive feel. 1 Random Variable: Topics Chap 2, 2.1-2.4 and Chap 3, 3.1-3.3 What is a random variable?
More informationCVaR and Examples of Deviation Risk Measures
CVaR and Examples of Deviation Risk Measures Jakub Černý Department of Probability and Mathematical Statistics Stochastic Modelling in Economics and Finance November 10, 2014 1 / 25 Contents CVaR - Dual
More informationMath 180A. Lecture 16 Friday May 7 th. Expectation. Recall the three main probability density functions so far (1) Uniform (2) Exponential.
Math 8A Lecture 6 Friday May 7 th Epectation Recall the three main probability density functions so far () Uniform () Eponential (3) Power Law e, ( ), Math 8A Lecture 6 Friday May 7 th Epectation Eample
More informationProbability. Paul Schrimpf. January 23, UBC Economics 326. Probability. Paul Schrimpf. Definitions. Properties. Random variables.
Probability UBC Economics 326 January 23, 2018 1 2 3 Wooldridge (2013) appendix B Stock and Watson (2009) chapter 2 Linton (2017) chapters 1-5 Abbring (2001) sections 2.1-2.3 Diez, Barr, and Cetinkaya-Rundel
More information4. Distributions of Functions of Random Variables
4. Distributions of Functions of Random Variables Setup: Consider as given the joint distribution of X 1,..., X n (i.e. consider as given f X1,...,X n and F X1,...,X n ) Consider k functions g 1 : R n
More informationS n = x + X 1 + X X n.
0 Lecture 0 0. Gambler Ruin Problem Let X be a payoff if a coin toss game such that P(X = ) = P(X = ) = /2. Suppose you start with x dollars and play the game n times. Let X,X 2,...,X n be payoffs in each
More informationProbability reminders
CS246 Winter 204 Mining Massive Data Sets Probability reminders Sammy El Ghazzal selghazz@stanfordedu Disclaimer These notes may contain typos, mistakes or confusing points Please contact the author so
More informationIntroduction to Machine Learning
What does this mean? Outline Contents Introduction to Machine Learning Introduction to Probabilistic Methods Varun Chandola December 26, 2017 1 Introduction to Probability 1 2 Random Variables 3 3 Bayes
More informationL09. PARTICLE FILTERING. NA568 Mobile Robotics: Methods & Algorithms
L09. PARTICLE FILTERING NA568 Mobile Robotics: Methods & Algorithms Particle Filters Different approach to state estimation Instead of parametric description of state (and uncertainty), use a set of state
More information1 Review of Probability and Distributions
Random variables. A numerically valued function X of an outcome ω from a sample space Ω X : Ω R : ω X(ω) is called a random variable (r.v.), and usually determined by an experiment. We conventionally denote
More informationE X A M. Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours. Number of pages incl.
E X A M Course code: Course name: Number of pages incl. front page: 6 MA430-G Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours Resources allowed: Notes: Pocket calculator,
More informationIntroduction to Probability and Stocastic Processes - Part I
Introduction to Probability and Stocastic Processes - Part I Lecture 2 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark
More informationExponential Distribution and Poisson Process
Exponential Distribution and Poisson Process Stochastic Processes - Lecture Notes Fatih Cavdur to accompany Introduction to Probability Models by Sheldon M. Ross Fall 215 Outline Introduction Exponential
More informationDUBLIN CITY UNIVERSITY
DUBLIN CITY UNIVERSITY SAMPLE EXAMINATIONS 2017/2018 MODULE: QUALIFICATIONS: Simulation for Finance MS455 B.Sc. Actuarial Mathematics ACM B.Sc. Financial Mathematics FIM YEAR OF STUDY: 4 EXAMINERS: Mr
More informationMIT Spring 2015
Assessing Goodness Of Fit MIT 8.443 Dr. Kempthorne Spring 205 Outline 2 Poisson Distribution Counts of events that occur at constant rate Counts in disjoint intervals/regions are independent If intervals/regions
More informationChapter 3: Random Variables 1
Chapter 3: Random Variables 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.
More informationEE 302: Probabilistic Methods in Electrical Engineering
EE : Probabilistic Methods in Electrical Engineering Print Name: Solution (//6 --sk) Test II : Chapters.5 4 //98, : PM Write down your name on each paper. Read every question carefully and solve each problem
More informationThe Gaussian distribution
The Gaussian distribution Probability density function: A continuous probability density function, px), satisfies the following properties:. The probability that x is between two points a and b b P a
More informationContinuous Random Variables and Continuous Distributions
Continuous Random Variables and Continuous Distributions Continuous Random Variables and Continuous Distributions Expectation & Variance of Continuous Random Variables ( 5.2) The Uniform Random Variable
More informationData Analysis I. Dr Martin Hendry, Dept of Physics and Astronomy University of Glasgow, UK. 10 lectures, beginning October 2006
Astronomical p( y x, I) p( x, I) p ( x y, I) = p( y, I) Data Analysis I Dr Martin Hendry, Dept of Physics and Astronomy University of Glasgow, UK 10 lectures, beginning October 2006 4. Monte Carlo Methods
More information1. Point Estimators, Review
AMS571 Prof. Wei Zhu 1. Point Estimators, Review Example 1. Let be a random sample from. Please find a good point estimator for Solutions. There are the typical estimators for and. Both are unbiased estimators.
More informationSolution to Assignment 3
The Chinese University of Hong Kong ENGG3D: Probability and Statistics for Engineers 5-6 Term Solution to Assignment 3 Hongyang Li, Francis Due: 3:pm, March Release Date: March 8, 6 Dear students, The
More informationPure Random process Pure Random Process or White Noise Process: is a random process {X t, t 0} which has: { σ 2 if k = 0 0 if k 0
MODULE 9: STATIONARY PROCESSES 7 Lecture 2 Autoregressive Processes 1 Moving Average Process Pure Random process Pure Random Process or White Noise Process: is a random process X t, t 0} which has: E[X
More informationBMIR Lecture Series on Probability and Statistics Fall, 2015 Uniform Distribution
Lecture #5 BMIR Lecture Series on Probability and Statistics Fall, 2015 Department of Biomedical Engineering and Environmental Sciences National Tsing Hua University s 5.1 Definition ( ) A continuous random
More informationChapter 3 Common Families of Distributions
Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 Review for the previous lecture Definition: Several commonly used discrete distributions, including discrete uniform, hypergeometric,
More informationChapter 4. Chapter 4 sections
Chapter 4 sections 4.1 Expectation 4.2 Properties of Expectations 4.3 Variance 4.4 Moments 4.5 The Mean and the Median 4.6 Covariance and Correlation 4.7 Conditional Expectation SKIP: 4.8 Utility Expectation
More informationNumerical integration - I. M. Peressi - UniTS - Laurea Magistrale in Physics Laboratory of Computational Physics - Unit V
Numerical integration - I M. Peressi - UniTS - Laurea Magistrale in Physics Laboratory of Computational Physics - Unit V deterministic methods in 1D equispaced points (trapezoidal, Simpson...), others...
More informationChapter 3, 4 Random Variables ENCS Probability and Stochastic Processes. Concordia University
Chapter 3, 4 Random Variables ENCS6161 - Probability and Stochastic Processes Concordia University ENCS6161 p.1/47 The Notion of a Random Variable A random variable X is a function that assigns a real
More information3 Operations on One Random Variable - Expectation
3 Operations on One Random Variable - Expectation 3.0 INTRODUCTION operations on a random variable Most of these operations are based on a single concept expectation. Even a probability of an event can
More informationProbability Density Functions
Probability Density Functions Probability Density Functions Definition Let X be a continuous rv. Then a probability distribution or probability density function (pdf) of X is a function f (x) such that
More informationSTAT509: Continuous Random Variable
University of South Carolina September 23, 2014 Continuous Random Variable A continuous random variable is a random variable with an interval (either finite or infinite) of real numbers for its range.
More informationMathematical Statistics 1 Math A 6330
Mathematical Statistics 1 Math A 6330 Chapter 2 Transformations and Expectations Mohamed I. Riffi Department of Mathematics Islamic University of Gaza September 14, 2015 Outline 1 Distributions of Functions
More informationECE353: Probability and Random Processes. Lecture 7 -Continuous Random Variable
ECE353: Probability and Random Processes Lecture 7 -Continuous Random Variable Xiao Fu School of Electrical Engineering and Computer Science Oregon State University E-mail: xiao.fu@oregonstate.edu Continuous
More informationMath Review Sheet, Fall 2008
1 Descriptive Statistics Math 3070-5 Review Sheet, Fall 2008 First we need to know about the relationship among Population Samples Objects The distribution of the population can be given in one of the
More informationLimiting Distributions
Limiting Distributions We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the
More informationChapter 6: Large Random Samples Sections
Chapter 6: Large Random Samples Sections 6.1: Introduction 6.2: The Law of Large Numbers Skip p. 356-358 Skip p. 366-368 Skip 6.4: The correction for continuity Remember: The Midterm is October 25th in
More informationΘ is the stratified sampling estimator of θ. Note: y i s subdivide sampling space into k strata ;
1 STRATIFIED SAMPLING Stratified Sampling Background: suppose θ = E[X]. Simple MC simulation would use θ X. If X simulation depends on some discrete Y with pmf P {Y = y i } = p i, i = 1,..., k, and X can
More informationChapter 5: Monte Carlo Integration and Variance Reduction
Chapter 5: Monte Carlo Integration and Variance Reduction Lecturer: Zhao Jianhua Department of Statistics Yunnan University of Finance and Economics Outline 5.2 Monte Carlo Integration 5.2.1 Simple MC
More informationReduction of Variance. Importance Sampling
Reduction of Variance As we discussed earlier, the statistical error goes as: error = sqrt(variance/computer time). DEFINE: Efficiency = = 1/vT v = error of mean and T = total CPU time How can you make
More informationECE 510 Lecture 7 Goodness of Fit, Maximum Likelihood. Scott Johnson Glenn Shirley
ECE 510 Lecture 7 Goodness of Fit, Maximum Likelihood Scott Johnson Glenn Shirley Confidence Limits 30 Jan 2013 ECE 510 S.C.Johnson, C.G.Shirley 2 Binomial Confidence Limits (Solution 6.2) UCL: Prob of
More information18.440: Lecture 19 Normal random variables
18.440 Lecture 19 18.440: Lecture 19 Normal random variables Scott Sheffield MIT Outline Tossing coins Normal random variables Special case of central limit theorem Outline Tossing coins Normal random
More informationDiscrete Variables and Gradient Estimators
iscrete Variables and Gradient Estimators This assignment is designed to get you comfortable deriving gradient estimators, and optimizing distributions over discrete random variables. For most questions,
More informationCSCI-6971 Lecture Notes: Monte Carlo integration
CSCI-6971 Lecture otes: Monte Carlo integration Kristopher R. Beevers Department of Computer Science Rensselaer Polytechnic Institute beevek@cs.rpi.edu February 21, 2006 1 Overview Consider the following
More informationMonte Carlo Integration. Computer Graphics CMU /15-662, Fall 2016
Monte Carlo Integration Computer Graphics CMU 15-462/15-662, Fall 2016 Talk Announcement Jovan Popovic, Senior Principal Scientist at Adobe Research will be giving a seminar on Character Animator -- Monday
More informationLecture 6: Monte-Carlo methods
Miranda Holmes-Cerfon Applied Stochastic Analysis, Spring 2015 Lecture 6: Monte-Carlo methods Readings Recommended: handout on Classes site, from notes by Weinan E, Tiejun Li, and Eric Vanden-Eijnden Optional:
More informationconditional cdf, conditional pdf, total probability theorem?
6 Multiple Random Variables 6.0 INTRODUCTION scalar vs. random variable cdf, pdf transformation of a random variable conditional cdf, conditional pdf, total probability theorem expectation of a random
More informationSTAT 430/510: Lecture 10
STAT 430/510: Lecture 10 James Piette June 9, 2010 Updates HW2 is due today! Pick up your HW1 s up in stat dept. There is a box located right when you enter that is labeled "Stat 430 HW1". It ll be out
More informationMonte Carlo Studies. The response in a Monte Carlo study is a random variable.
Monte Carlo Studies The response in a Monte Carlo study is a random variable. The response in a Monte Carlo study has a variance that comes from the variance of the stochastic elements in the data-generating
More informationChapter 6: Random Processes 1
Chapter 6: Random Processes 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.
More informationChing-Han Hsu, BMES, National Tsing Hua University c 2015 by Ching-Han Hsu, Ph.D., BMIR Lab. = a + b 2. b a. x a b a = 12
Lecture 5 Continuous Random Variables BMIR Lecture Series in Probability and Statistics Ching-Han Hsu, BMES, National Tsing Hua University c 215 by Ching-Han Hsu, Ph.D., BMIR Lab 5.1 1 Uniform Distribution
More informationMATH : EXAM 2 INFO/LOGISTICS/ADVICE
MATH 3342-004: EXAM 2 INFO/LOGISTICS/ADVICE INFO: WHEN: Friday (03/11) at 10:00am DURATION: 50 mins PROBLEM COUNT: Appropriate for a 50-min exam BONUS COUNT: At least one TOPICS CANDIDATE FOR THE EXAM:
More informationOptimization and Simulation
Optimization and Simulation Variance reduction Michel Bierlaire Transport and Mobility Laboratory School of Architecture, Civil and Environmental Engineering Ecole Polytechnique Fédérale de Lausanne M.
More informationExpectation Propagation for Approximate Bayesian Inference
Expectation Propagation for Approximate Bayesian Inference José Miguel Hernández Lobato Universidad Autónoma de Madrid, Computer Science Department February 5, 2007 1/ 24 Bayesian Inference Inference Given
More informationMonte Carlo and Empirical Methods for Stochastic Inference (MASM11/FMS091)
Monte Carlo and Empirical Methods for Stochastic Inference (MASM11/FMS091) Magnus Wiktorsson Centre for Mathematical Sciences Lund University, Sweden Lecture 4 Variance reduction for MC methods January
More informationSystem Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models
System Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models Fatih Cavdur fatihcavdur@uludag.edu.tr March 29, 2014 Introduction Introduction The world of the model-builder
More information