Convergence of Random Processes

Size: px
Start display at page:

Download "Convergence of Random Processes"

Transcription

1 Convergence of Random Processes DS GA 1002 Probability and Statistics for Data Science Carlos Fernandez-Granda

2 Aim Define convergence for random processes Describe two convergence phenomena: the law of large numbers and the central limit theorem

3 Types of convergence Law of Large Numbers Central Limit Theorem Monte Carlo simulation

4 Convergence of deterministic sequences A deterministic sequence of real numbers x 1, x 2,... converges to x R, lim x i = x i if x i is arbitrarily close to x as i grows For any ɛ > 0 there is an i 0 such that for all i > i 0 x i x < ɛ Problem: Random sequences do not have fixed values

5 Convergence with probability one Consider a discrete random process X and a random variable X defined on the same probability space If we fix the outcome ω, X (i, ω) is a deterministic sequence and X (ω) is a constant We can determine whether for that particular ω lim X (i, ω) = X (ω) i

6 Convergence with probability one X converges with probability one to X if ({ P ω ω Ω, }) lim X (ω, i) = X (ω) = 1 i Deterministic convergence occurs with probability one

7 Puddle Initial amount of water is uniform between 0 and 1 gallon After a time interval i there is i times less water D (ω, i) := ω, i = 1, 2,... i

8 Puddle 0.8 ω = 0.31 ω = 0.89 ω = 0.52 D (ω, i) i

9 Puddle If we fix ω (0, 1) lim D (ω, i) = lim i = 0 i i ω D converges to zero with probability one

10 Puddle 1 D (ω, i) i

11 Alternative idea Idea: Instead of fixing ω and checking deterministic convergence: 1. Measure how close X (i) and X are for a fixed i using a deterministic quantity 2. Check whether the quantity tends to zero

12 Convergence in mean square The mean square of Y X measures how close X and Y are ( If E (X Y ) 2) = 0 then X = Y with probability one Proof: By Markov s inequality for any ɛ > 0 ( ) E ((X Y ) 2) P (Y X ) 2 > ɛ = 0 ɛ

13 Convergence in mean square X converges to X in mean square if ( ( ) lim E X X 2 (i)) = 0 i

14 Convergence in probability Alternative measure: Probability that Y X > ɛ for small ɛ X converges to X in probability if for any ɛ > 0 ( ) X lim P X (i) > ɛ = 0 i

15 Conv. in mean square implies conv. in probability ( ) X lim P X (i) > ɛ i

16 Conv. in mean square implies conv. in probability ( ) ( ( X lim P X (i) > ɛ = lim P X X ) 2 (i) > ɛ 2) i i

17 Conv. in mean square implies conv. in probability ( ) ( ( X lim P X (i) > ɛ = lim P X X ) 2 (i) > ɛ 2) i i ( ( E X X ) ) 2 (i) lim i ɛ 2

18 Conv. in mean square implies conv. in probability ( ) ( ( X lim P X (i) > ɛ = lim P X X ) 2 (i) > ɛ 2) i i ( ( E X X ) ) 2 (i) lim i ɛ 2 = 0

19 Conv. in mean square implies conv. in probability ( ) ( ( X lim P X (i) > ɛ = lim P X X ) 2 (i) > ɛ 2) i i ( ( E X X ) ) 2 (i) lim i ɛ 2 = 0 Convergence with probability one also implies convergence in probability

20 Convergence in distribution The distribution of X (i) converges to the distribution of X X converges in distribution to X if for all x at which F X is continuous lim F i X (i) (x) = F X (x)

21 Convergence in distribution Convergence in distribution does not imply that X (i) and X are close as i! Convergence in probability does imply convergence in distribution

22 Binomial tends to Poisson X (i) is binomial with parameters i and p := λ/i X is a Poisson random variable with parameter λ X (i) converges to X in distribution ( ) i lim p i X (i) (x) = lim p x (1 p) (i x) i x = λx e λ x! = p X (x)

23 Probability mass function of X (40) k

24 Probability mass function of X (80) k

25 Probability mass function of X (400) k

26 Probability mass function of X k

27 Types of convergence Law of Large Numbers Central Limit Theorem Monte Carlo simulation

28 Moving average The moving average à of a discrete random process X is à (i) := 1 i i X (j) j=1

29 Weak law of large numbers Let X be an iid discrete random process with mean µ X := µ and bounded variance σ 2 The average à of X converges in mean square to µ

30 Proof ) E (Ã (i)

31 Proof ) E (Ã (i) = E 1 i i X (j) j=1

32 Proof ) E (Ã (i) = E 1 i = 1 i i j=1 i X (j) j=1 ( ) E X (j)

33 Proof ) E (Ã (i) = E 1 i = 1 i = µ i j=1 i X (j) j=1 ( ) E X (j)

34 Proof ) Var (Ã (i)

35 Proof ) Var (Ã (i) = Var 1 i i X (j) j=1

36 Proof ) Var (Ã (i) = Var 1 i = 1 i 2 i j=1 i X (j) j=1 ( ) Var X (j)

37 Proof ) Var (Ã (i) = Var 1 i = 1 i 2 = σ2 i i j=1 i X (j) j=1 ( ) Var X (j)

38 Proof ( (Ã ) ) 2 lim E (i) µ i

39 Proof ( (Ã ) ) ( 2 (Ã )) ) 2 lim E (i) µ = lim E (i) E (Ã (i) i i

40 Proof ( (Ã ) ) ( 2 (Ã )) ) 2 lim E (i) µ = lim E (i) E (Ã (i) i i ) = lim Var (Ã (i) i

41 Proof ( (Ã ) ) ( 2 (Ã )) ) 2 lim E (i) µ = lim E (i) E (Ã (i) i i ) = lim Var (Ã (i) i = lim i σ 2 i

42 Proof ( (Ã ) ) ( 2 (Ã )) ) 2 lim E (i) µ = lim E (i) E (Ã (i) i i ) = lim Var (Ã (i) i = lim i σ 2 i = 0

43 Strong law of large numbers Let X be an iid discrete random process with mean µ X := µ and bounded variance σ 2 The average à of X converges with probability one to µ

44 iid standard Gaussian Moving average Mean of iid seq i 40 50

45 iid standard Gaussian Moving average Mean of iid seq i

46 iid standard Gaussian Moving average Mean of iid seq i

47 iid geometric with p = Moving average Mean of iid seq i

48 iid geometric with p = Moving average Mean of iid seq i

49 iid geometric with p = Moving average Mean of iid seq i

50 iid Cauchy Moving average Median of iid seq i

51 iid Cauchy 10 5 Moving average Median of iid seq i

52 iid Cauchy Moving average Median of iid seq i

53 Types of convergence Law of Large Numbers Central Limit Theorem Monte Carlo simulation

54 Central Limit Theorem Let X be an iid discrete random process with mean µ X := µ and bounded variance σ 2 ) n (à µ converges in distribution to a Gaussian random variable with mean 0 and variance σ 2 The average à is approximately Gaussian with mean µ and variance σ2 /i

55 Height data Example: Data from a population of people We compare the histogram of the heights and the pdf of a Gaussian random variable fitted to the data

56 Height data Gaussian distribution Real data Height (inches)

57 Sketch of proof Pdf of sum of two independent random variables is the convolution of their pdfs f X +Y (z) = y= f X (z y) f Y (y) dy Repeated convolutions of any pdf with bounded variance result in a Gaussian!

58 Repeated convolutions i = 1 i = 2 i = 3 i = 4 i = 5

59 Repeated convolutions i = 1 i = 2 i = 3 i = 4 i = 5

60 iid exponential λ = 2, i =

61 iid exponential λ = 2, i =

62 iid exponential λ = 2, i =

63 iid geometric p = 0.4, i =

64 iid geometric p = 0.4, i =

65 iid geometric p = 0.4, i =

66 iid Cauchy, i =

67 iid Cauchy, i =

68 iid Cauchy, i =

69 Gaussian approximation to the binomial X is binomial with parameters n and p Computing the probability that X is in a certain interval requires summing its pmf over the interval Central limit theorem provides a quick approximation X = n B i, E (B i ) = p, Var (B i ) = p (1 p) i=1 1 nx is approximately Gaussian with mean p and variance p (1 p) /n X is approximately Gaussian with mean np and variance np (1 p)

70 Gaussian approximation to the binomial Basketball player makes shot with probability p = 0.4 (shots are iid) Probability that she makes more than 420 shots out of 1000? Exact answer: Approximation: P (X 420) = = P (X 420) 1000 x= x=420 p X (x) ( 1000 x ) 0.4 x 0.6 (n x) =

71 Gaussian approximation to the binomial Basketball player makes shot with probability p = 0.4 (shots are iid) Probability that she makes more than 420 shots out of 1000? Exact answer: P (X 420) = = 1000 x= x=420 p X (x) ( 1000 x ) 0.4 x 0.6 (n x) = Approximation: ( ) P (X 420) P np (1 p)u + np 420

72 Gaussian approximation to the binomial Basketball player makes shot with probability p = 0.4 (shots are iid) Probability that she makes more than 420 shots out of 1000? Exact answer: P (X 420) = = 1000 x= x=420 p X (x) ( 1000 x ) 0.4 x 0.6 (n x) = Approximation: ( ) P (X 420) P np (1 p)u + np 420 = P (U 1.29)

73 Gaussian approximation to the binomial Basketball player makes shot with probability p = 0.4 (shots are iid) Probability that she makes more than 420 shots out of 1000? Exact answer: P (X 420) = = 1000 x= x=420 p X (x) ( 1000 x ) 0.4 x 0.6 (n x) = Approximation: ( ) P (X 420) P np (1 p)u + np 420 = P (U 1.29) = 1 Φ (1.29) =

74 Types of convergence Law of Large Numbers Central Limit Theorem Monte Carlo simulation

75 Monte Carlo simulation Simulation is a powerful tool in probability and statistics Models are too complex to derive closed-form solutions (life is not a homework problem!) Example: Game of solitaire

76 Game of solitaire Aim: Compute the probability that you win at solitaire If every permutation of the cards has the same probability P (Win) = Number of permutations that lead to a win Total number Problem: Characterizing permutations that lead to a win is very difficult without playing out the game We can t just check because there are 52! permutations! Solution: Sample many permutations and compute the fraction of wins

77 In the words of Stanislaw Ulam The first thoughts and attempts I made to practice (the Monte Carlo Method) were suggested by a question which occurred to me in 1946 as I was convalescing from an illness and playing solitaires. The question was what are the chances that a Canfield solitaire laid out with 52 cards will come out successfully? After spending a lot of time trying to estimate them by pure combinatorial calculations, I wondered whether a more practical method than "abstract thinking" might not be to lay it out say one hundred times and simply observe and count the number of successful plays.this was already possible to envisage with the beginning of the new era of fast computers.

78 Monte Carlo approximation Main principle: Use simulation to approximate quantities that are challenging to compute exactly To approximate the probability of an event E 1. Generate n independent samples from 1 E : I 1, I 2,..., I n 2. Compute the average of the n samples n à (n) := 1 n i=1 I i By the law of large numbers à converges to P (E) as n since E (1 E ) = P (E)

79 Basketball league Basketball league with m teams In a season every pair of teams plays once Teams are ordered: team 1 is best, team m is worst Model: For 1 i < j m Games are independent P (team j beats team i) := 1 j i + 1

80 Basketball league Aim: Compute probability of team ranks at the end of the season The rank of team i is modeled as a random variable R i Pmf of R 1, R 2,..., R m?

81 m = 3 Game outcomes Rank R 1 R 2 R 3 Probability / / / / / / / /12

82 m = 3 Probability mass function R 1 R 2 R 3 1 7/12 1/2 5/12 2 1/4 1/4 1/4 3 1/6 1/4 1/3

83 Basketball league Problem: Number of possible outcomes is 2 m(m 1)/2! For m = 10 this is larger than Solution: Apply Monte Carlo approximation

84 m = 3 Game outcomes Rank R 1 R 2 R

85 m = 3 Estimated pmf (n = 10) R 1 R 2 R (0.583) 0.7 (0.5) 0.3 (0.417) (0.25) 0.2 (0.25) 0.4 (0.25) (0.167) 0.1 (0.25) 0.3 (0.333)

86 m = 3 Estimated pmf (n = 2, 000) R 1 R 2 R (0.583) (0.5) (0.417) (0.25) (0.25) (0.25) (0.167) (0.25) (0.333)

87 Running times Running time (seconds) Exact computation Monte Carlo approx Number of teams m

88 Error m Average error

89 m = 5

90 m = 20

91 m = 100

Applied Statistics. Monte Carlo Simulations. Troels C. Petersen (NBI) Statistics is merely a quantisation of common sense

Applied Statistics. Monte Carlo Simulations. Troels C. Petersen (NBI) Statistics is merely a quantisation of common sense Applied Statistics Monte Carlo Simulations Troels C. Petersen (NBI) Statistics is merely a quantisation of common sense 1 Random numbers Any one who considers arithmetical methods of producing random digits

More information

Expectation. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda

Expectation. DS GA 1002 Statistical and Mathematical Models.   Carlos Fernandez-Granda Expectation DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean, variance,

More information

Random Processes. DS GA 1002 Probability and Statistics for Data Science.

Random Processes. DS GA 1002 Probability and Statistics for Data Science. Random Processes DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Aim Modeling quantities that evolve in time (or space)

More information

Expectation. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda

Expectation. DS GA 1002 Probability and Statistics for Data Science.   Carlos Fernandez-Granda Expectation DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean,

More information

Computer intensive statistical methods Lecture 1

Computer intensive statistical methods Lecture 1 Computer intensive statistical methods Lecture 1 Jonas Wallin Chalmers, Gothenburg. Jonas Wallin - jonwal@chalmers.se 1/27 People and literature The following people are involved in the course: Function

More information

Review. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda

Review. DS GA 1002 Statistical and Mathematical Models.   Carlos Fernandez-Granda Review DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing with

More information

MATH Notebook 5 Fall 2018/2019

MATH Notebook 5 Fall 2018/2019 MATH442601 2 Notebook 5 Fall 2018/2019 prepared by Professor Jenny Baglivo c Copyright 2004-2019 by Jenny A. Baglivo. All Rights Reserved. 5 MATH442601 2 Notebook 5 3 5.1 Sequences of IID Random Variables.............................

More information

Random variables. DS GA 1002 Probability and Statistics for Data Science.

Random variables. DS GA 1002 Probability and Statistics for Data Science. Random variables DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Motivation Random variables model numerical quantities

More information

Week 9 The Central Limit Theorem and Estimation Concepts

Week 9 The Central Limit Theorem and Estimation Concepts Week 9 and Estimation Concepts Week 9 and Estimation Concepts Week 9 Objectives 1 The Law of Large Numbers and the concept of consistency of averages are introduced. The condition of existence of the population

More information

STAT/MATH 395 PROBABILITY II

STAT/MATH 395 PROBABILITY II STAT/MATH 395 PROBABILITY II Chapter 6 : Moment Functions Néhémy Lim 1 1 Department of Statistics, University of Washington, USA Winter Quarter 2016 of Common Distributions Outline 1 2 3 of Common Distributions

More information

Descriptive Statistics

Descriptive Statistics Descriptive Statistics DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Descriptive statistics Techniques to visualize

More information

Things to remember when learning probability distributions:

Things to remember when learning probability distributions: SPECIAL DISTRIBUTIONS Some distributions are special because they are useful They include: Poisson, exponential, Normal (Gaussian), Gamma, geometric, negative binomial, Binomial and hypergeometric distributions

More information

Probability and Statistics for Data Science. Carlos Fernandez-Granda

Probability and Statistics for Data Science. Carlos Fernandez-Granda Probability and Statistics for Data Science Carlos Fernandez-Granda Preface These notes were developed for the course Probability and Statistics for Data Science at the Center for Data Science in NYU.

More information

Monte Carlo Methods. Part I: Introduction

Monte Carlo Methods. Part I: Introduction Monte Carlo Methods Part I: Introduction Spring Semester 2013/14 Department of Applied Mathematics, University of Crete Instructor: Harmandaris Vagelis, email: vagelis@tem.uoc.gr Course web page: http://www.tem.uoc.gr/~vagelis/courses/em385/

More information

Introduction to Machine Learning CMU-10701

Introduction to Machine Learning CMU-10701 Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos Contents Markov Chain Monte Carlo Methods Sampling Rejection Importance Hastings-Metropolis Gibbs Markov Chains

More information

Chapter 5. Means and Variances

Chapter 5. Means and Variances 1 Chapter 5 Means and Variances Our discussion of probability has taken us from a simple classical view of counting successes relative to total outcomes and has brought us to the idea of a probability

More information

Statistical Data Analysis

Statistical Data Analysis DS-GA 0 Lecture notes 8 Fall 016 1 Descriptive statistics Statistical Data Analysis In this section we consider the problem of analyzing a set of data. We describe several techniques for visualizing the

More information

Probability Distributions Columns (a) through (d)

Probability Distributions Columns (a) through (d) Discrete Probability Distributions Columns (a) through (d) Probability Mass Distribution Description Notes Notation or Density Function --------------------(PMF or PDF)-------------------- (a) (b) (c)

More information

STAT/MATH 395 A - PROBABILITY II UW Winter Quarter Moment functions. x r p X (x) (1) E[X r ] = x r f X (x) dx (2) (x E[X]) r p X (x) (3)

STAT/MATH 395 A - PROBABILITY II UW Winter Quarter Moment functions. x r p X (x) (1) E[X r ] = x r f X (x) dx (2) (x E[X]) r p X (x) (3) STAT/MATH 395 A - PROBABILITY II UW Winter Quarter 07 Néhémy Lim Moment functions Moments of a random variable Definition.. Let X be a rrv on probability space (Ω, A, P). For a given r N, E[X r ], if it

More information

CSE 312: Foundations of Computing II Quiz Section #10: Review Questions for Final Exam (solutions)

CSE 312: Foundations of Computing II Quiz Section #10: Review Questions for Final Exam (solutions) CSE 312: Foundations of Computing II Quiz Section #10: Review Questions for Final Exam (solutions) 1. (Confidence Intervals, CLT) Let X 1,..., X n be iid with unknown mean θ and known variance σ 2. Assume

More information

ECE 302 Division 1 MWF 10:30-11:20 (Prof. Pollak) Final Exam Solutions, 5/3/2004. Please read the instructions carefully before proceeding.

ECE 302 Division 1 MWF 10:30-11:20 (Prof. Pollak) Final Exam Solutions, 5/3/2004. Please read the instructions carefully before proceeding. NAME: ECE 302 Division MWF 0:30-:20 (Prof. Pollak) Final Exam Solutions, 5/3/2004. Please read the instructions carefully before proceeding. If you are not in Prof. Pollak s section, you may not take this

More information

Northwestern University Department of Electrical Engineering and Computer Science

Northwestern University Department of Electrical Engineering and Computer Science Northwestern University Department of Electrical Engineering and Computer Science EECS 454: Modeling and Analysis of Communication Networks Spring 2008 Probability Review As discussed in Lecture 1, probability

More information

Part IA Probability. Theorems. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015

Part IA Probability. Theorems. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015 Part IA Probability Theorems Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.

More information

Lecture Notes 7 Random Processes. Markov Processes Markov Chains. Random Processes

Lecture Notes 7 Random Processes. Markov Processes Markov Chains. Random Processes Lecture Notes 7 Random Processes Definition IID Processes Bernoulli Process Binomial Counting Process Interarrival Time Process Markov Processes Markov Chains Classification of States Steady State Probabilities

More information

STAT 414: Introduction to Probability Theory

STAT 414: Introduction to Probability Theory STAT 414: Introduction to Probability Theory Spring 2016; Homework Assignments Latest updated on April 29, 2016 HW1 (Due on Jan. 21) Chapter 1 Problems 1, 8, 9, 10, 11, 18, 19, 26, 28, 30 Theoretical Exercises

More information

Course: ESO-209 Home Work: 1 Instructor: Debasis Kundu

Course: ESO-209 Home Work: 1 Instructor: Debasis Kundu Home Work: 1 1. Describe the sample space when a coin is tossed (a) once, (b) three times, (c) n times, (d) an infinite number of times. 2. A coin is tossed until for the first time the same result appear

More information

Probability Models. 4. What is the definition of the expectation of a discrete random variable?

Probability Models. 4. What is the definition of the expectation of a discrete random variable? 1 Probability Models The list of questions below is provided in order to help you to prepare for the test and exam. It reflects only the theoretical part of the course. You should expect the questions

More information

Lecture 1. ABC of Probability

Lecture 1. ABC of Probability Math 408 - Mathematical Statistics Lecture 1. ABC of Probability January 16, 2013 Konstantin Zuev (USC) Math 408, Lecture 1 January 16, 2013 1 / 9 Agenda Sample Spaces Realizations, Events Axioms of Probability

More information

Chapter 6: Large Random Samples Sections

Chapter 6: Large Random Samples Sections Chapter 6: Large Random Samples Sections 6.1: Introduction 6.2: The Law of Large Numbers Skip p. 356-358 Skip p. 366-368 Skip 6.4: The correction for continuity Remember: The Midterm is October 25th in

More information

Introduction and Overview STAT 421, SP Course Instructor

Introduction and Overview STAT 421, SP Course Instructor Introduction and Overview STAT 421, SP 212 Prof. Prem K. Goel Mon, Wed, Fri 3:3PM 4:48PM Postle Hall 118 Course Instructor Prof. Goel, Prem E mail: goel.1@osu.edu Office: CH 24C (Cockins Hall) Phone: 614

More information

Discrete Distributions

Discrete Distributions A simplest example of random experiment is a coin-tossing, formally called Bernoulli trial. It happens to be the case that many useful distributions are built upon this simplest form of experiment, whose

More information

STAT 418: Probability and Stochastic Processes

STAT 418: Probability and Stochastic Processes STAT 418: Probability and Stochastic Processes Spring 2016; Homework Assignments Latest updated on April 29, 2016 HW1 (Due on Jan. 21) Chapter 1 Problems 1, 8, 9, 10, 11, 18, 19, 26, 28, 30 Theoretical

More information

Lecture 6. Probability events. Definition 1. The sample space, S, of a. probability experiment is the collection of all

Lecture 6. Probability events. Definition 1. The sample space, S, of a. probability experiment is the collection of all Lecture 6 1 Lecture 6 Probability events Definition 1. The sample space, S, of a probability experiment is the collection of all possible outcomes of an experiment. One such outcome is called a simple

More information

Business Statistics Midterm Exam Fall 2015 Russell. Please sign here to acknowledge

Business Statistics Midterm Exam Fall 2015 Russell. Please sign here to acknowledge Business Statistics Midterm Exam Fall 5 Russell Name Do not turn over this page until you are told to do so. You will have hour and 3 minutes to complete the exam. There are a total of points divided into

More information

(b). What is an expression for the exact value of P(X = 4)? 2. (a). Suppose that the moment generating function for X is M (t) = 2et +1 3

(b). What is an expression for the exact value of P(X = 4)? 2. (a). Suppose that the moment generating function for X is M (t) = 2et +1 3 Math 511 Exam #2 Show All Work 1. A package of 200 seeds contains 40 that are defective and will not grow (the rest are fine). Suppose that you choose a sample of 10 seeds from the box without replacement.

More information

Part IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015

Part IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015 Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.

More information

A Probability Primer. A random walk down a probabilistic path leading to some stochastic thoughts on chance events and uncertain outcomes.

A Probability Primer. A random walk down a probabilistic path leading to some stochastic thoughts on chance events and uncertain outcomes. A Probability Primer A random walk down a probabilistic path leading to some stochastic thoughts on chance events and uncertain outcomes. Are you holding all the cards?? Random Events A random event, E,

More information

Math/Stat 394 Homework 5

Math/Stat 394 Homework 5 Math/Stat 394 Homework 5 1. If we select two black balls then X 4. This happens with probability ( 4 2). If we select two white balls then X 2. This happens with ( 14 probability (8 2). If we select two

More information

Statistics for Economists. Lectures 3 & 4

Statistics for Economists. Lectures 3 & 4 Statistics for Economists Lectures 3 & 4 Asrat Temesgen Stockholm University 1 CHAPTER 2- Discrete Distributions 2.1. Random variables of the Discrete Type Definition 2.1.1: Given a random experiment with

More information

Order Statistics and Distributions

Order Statistics and Distributions Order Statistics and Distributions 1 Some Preliminary Comments and Ideas In this section we consider a random sample X 1, X 2,..., X n common continuous distribution function F and probability density

More information

Math 493 Final Exam December 01

Math 493 Final Exam December 01 Math 493 Final Exam December 01 NAME: ID NUMBER: Return your blue book to my office or the Math Department office by Noon on Tuesday 11 th. On all parts after the first show enough work in your exam booklet

More information

CS145: Probability & Computing

CS145: Probability & Computing CS45: Probability & Computing Lecture 5: Concentration Inequalities, Law of Large Numbers, Central Limit Theorem Instructor: Eli Upfal Brown University Computer Science Figure credits: Bertsekas & Tsitsiklis,

More information

1: PROBABILITY REVIEW

1: PROBABILITY REVIEW 1: PROBABILITY REVIEW Marek Rutkowski School of Mathematics and Statistics University of Sydney Semester 2, 2016 M. Rutkowski (USydney) Slides 1: Probability Review 1 / 56 Outline We will review the following

More information

1. Let X and Y be independent exponential random variables with rate α. Find the densities of the random variables X 3, X Y, min(x, Y 3 )

1. Let X and Y be independent exponential random variables with rate α. Find the densities of the random variables X 3, X Y, min(x, Y 3 ) 1 Introduction These problems are meant to be practice problems for you to see if you have understood the material reasonably well. They are neither exhaustive (e.g. Diffusions, continuous time branching

More information

Lecture 5: The Principle of Deferred Decisions. Chernoff Bounds

Lecture 5: The Principle of Deferred Decisions. Chernoff Bounds Randomized Algorithms Lecture 5: The Principle of Deferred Decisions. Chernoff Bounds Sotiris Nikoletseas Associate Professor CEID - ETY Course 2013-2014 Sotiris Nikoletseas, Associate Professor Randomized

More information

CPSC 320 Sample Final Examination December 2013

CPSC 320 Sample Final Examination December 2013 CPSC 320 Sample Final Examination December 2013 [10] 1. Answer each of the following questions with true or false. Give a short justification for each of your answers. [5] a. 6 n O(5 n ) lim n + This is

More information

EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY (formerly the Examinations of the Institute of Statisticians) HIGHER CERTIFICATE IN STATISTICS, 1996

EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY (formerly the Examinations of the Institute of Statisticians) HIGHER CERTIFICATE IN STATISTICS, 1996 EXAMINATIONS OF THE ROAL STATISTICAL SOCIET (formerly the Examinations of the Institute of Statisticians) HIGHER CERTIFICATE IN STATISTICS, 996 Paper I : Statistical Theory Time Allowed: Three Hours Candidates

More information

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A. 1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n

More information

IE 230 Probability & Statistics in Engineering I. Closed book and notes. 60 minutes.

IE 230 Probability & Statistics in Engineering I. Closed book and notes. 60 minutes. Closed book and notes. 60 minutes. A summary table of some univariate continuous distributions is provided. Four Pages. In this version of the Key, I try to be more complete than necessary to receive full

More information

Bayesian statistics. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda

Bayesian statistics. DS GA 1002 Statistical and Mathematical Models.   Carlos Fernandez-Granda Bayesian statistics DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall15 Carlos Fernandez-Granda Frequentist vs Bayesian statistics In frequentist statistics

More information

Twelfth Problem Assignment

Twelfth Problem Assignment EECS 401 Not Graded PROBLEM 1 Let X 1, X 2,... be a sequence of independent random variables that are uniformly distributed between 0 and 1. Consider a sequence defined by (a) Y n = max(x 1, X 2,..., X

More information

Fundamentals of Applied Probability and Random Processes

Fundamentals of Applied Probability and Random Processes Fundamentals of Applied Probability and Random Processes,nd 2 na Edition Oliver C. Ibe University of Massachusetts, LoweLL, Massachusetts ip^ W >!^ AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS

More information

Massachusetts Institute of Technology

Massachusetts Institute of Technology Problem. (0 points) Massachusetts Institute of Technology Final Solutions: December 15, 009 (a) (5 points) We re given that the joint PDF is constant in the shaded region, and since the PDF must integrate

More information

Chapter 3, 4 Random Variables ENCS Probability and Stochastic Processes. Concordia University

Chapter 3, 4 Random Variables ENCS Probability and Stochastic Processes. Concordia University Chapter 3, 4 Random Variables ENCS6161 - Probability and Stochastic Processes Concordia University ENCS6161 p.1/47 The Notion of a Random Variable A random variable X is a function that assigns a real

More information

Lecture 3. Discrete Random Variables

Lecture 3. Discrete Random Variables Math 408 - Mathematical Statistics Lecture 3. Discrete Random Variables January 23, 2013 Konstantin Zuev (USC) Math 408, Lecture 3 January 23, 2013 1 / 14 Agenda Random Variable: Motivation and Definition

More information

BINOMIAL DISTRIBUTION

BINOMIAL DISTRIBUTION BINOMIAL DISTRIBUTION The binomial distribution is a particular type of discrete pmf. It describes random variables which satisfy the following conditions: 1 You perform n identical experiments (called

More information

Fundamental Tools - Probability Theory IV

Fundamental Tools - Probability Theory IV Fundamental Tools - Probability Theory IV MSc Financial Mathematics The University of Warwick October 1, 2015 MSc Financial Mathematics Fundamental Tools - Probability Theory IV 1 / 14 Model-independent

More information

IE 336 Seat # Name (clearly) Closed book. One page of hand-written notes, front and back. No calculator. 60 minutes.

IE 336 Seat # Name (clearly) Closed book. One page of hand-written notes, front and back. No calculator. 60 minutes. Closed book. One page of hand-written notes, front and back. No calculator. 6 minutes. Cover page and four pages of exam. Fifteen questions. Each question is worth seven points. To receive full credit,

More information

Continuous random variables

Continuous random variables Continuous random variables Continuous r.v. s take an uncountably infinite number of possible values. Examples: Heights of people Weights of apples Diameters of bolts Life lengths of light-bulbs We cannot

More information

Machine Learning. Probability Basics. Marc Toussaint University of Stuttgart Summer 2014

Machine Learning. Probability Basics. Marc Toussaint University of Stuttgart Summer 2014 Machine Learning Probability Basics Basic definitions: Random variables, joint, conditional, marginal distribution, Bayes theorem & examples; Probability distributions: Binomial, Beta, Multinomial, Dirichlet,

More information

Special Discrete RV s. Then X = the number of successes is a binomial RV. X ~ Bin(n,p).

Special Discrete RV s. Then X = the number of successes is a binomial RV. X ~ Bin(n,p). Sect 3.4: Binomial RV Special Discrete RV s 1. Assumptions and definition i. Experiment consists of n repeated trials ii. iii. iv. There are only two possible outcomes on each trial: success (S) or failure

More information

Math Review Sheet, Fall 2008

Math Review Sheet, Fall 2008 1 Descriptive Statistics Math 3070-5 Review Sheet, Fall 2008 First we need to know about the relationship among Population Samples Objects The distribution of the population can be given in one of the

More information

STAT 302 Introduction to Probability Learning Outcomes. Textbook: A First Course in Probability by Sheldon Ross, 8 th ed.

STAT 302 Introduction to Probability Learning Outcomes. Textbook: A First Course in Probability by Sheldon Ross, 8 th ed. STAT 302 Introduction to Probability Learning Outcomes Textbook: A First Course in Probability by Sheldon Ross, 8 th ed. Chapter 1: Combinatorial Analysis Demonstrate the ability to solve combinatorial

More information

Data, Estimation and Inference

Data, Estimation and Inference Data, Estimation and Inference Pedro Piniés ppinies@robots.ox.ac.uk Michaelmas 2016 1 2 p(x) ( = ) = δ 0 ( < < + δ ) δ ( ) =1. x x+dx (, ) = ( ) ( ) = ( ) ( ) 3 ( ) ( ) 0 ( ) =1 ( = ) = ( ) ( < < ) = (

More information

IB Mathematics HL Year 2 Unit 7 (Core Topic 6: Probability and Statistics) Valuable Practice

IB Mathematics HL Year 2 Unit 7 (Core Topic 6: Probability and Statistics) Valuable Practice IB Mathematics HL Year 2 Unit 7 (Core Topic 6: Probability and Statistics) Valuable Practice 1. We have seen that the TI-83 calculator random number generator X = rand defines a uniformly-distributed random

More information

Stat 100a, Introduction to Probability.

Stat 100a, Introduction to Probability. Stat 100a, Introduction to Probability. Outline for the day: 1. Geometric random variables. 2. Negative binomial random variables. 3. Moment generating functions. 4. Poisson random variables. 5. Continuous

More information

ECE302 Spring 2015 HW10 Solutions May 3,

ECE302 Spring 2015 HW10 Solutions May 3, ECE32 Spring 25 HW Solutions May 3, 25 Solutions to HW Note: Most of these solutions were generated by R. D. Yates and D. J. Goodman, the authors of our textbook. I have added comments in italics where

More information

Probability reminders

Probability reminders CS246 Winter 204 Mining Massive Data Sets Probability reminders Sammy El Ghazzal selghazz@stanfordedu Disclaimer These notes may contain typos, mistakes or confusing points Please contact the author so

More information

Lecture 10: Probability distributions TUESDAY, FEBRUARY 19, 2019

Lecture 10: Probability distributions TUESDAY, FEBRUARY 19, 2019 Lecture 10: Probability distributions DANIEL WELLER TUESDAY, FEBRUARY 19, 2019 Agenda What is probability? (again) Describing probabilities (distributions) Understanding probabilities (expectation) Partial

More information

Math 2311 Test 1 Review. 1. State whether each situation is categorical or quantitative. If quantitative, state whether it s discrete or continuous.

Math 2311 Test 1 Review. 1. State whether each situation is categorical or quantitative. If quantitative, state whether it s discrete or continuous. Math 2311 Test 1 Review Know all definitions! 1. State whether each situation is categorical or quantitative. If quantitative, state whether it s discrete or continuous. a. The amount a person grew (in

More information

DS-GA 1002 Lecture notes 2 Fall Random variables

DS-GA 1002 Lecture notes 2 Fall Random variables DS-GA 12 Lecture notes 2 Fall 216 1 Introduction Random variables Random variables are a fundamental tool in probabilistic modeling. They allow us to model numerical quantities that are uncertain: the

More information

MAT 271E Probability and Statistics

MAT 271E Probability and Statistics MAT 271E Probability and Statistics Spring 2011 Instructor : Class Meets : Office Hours : Textbook : Supp. Text : İlker Bayram EEB 1103 ibayram@itu.edu.tr 13.30 16.30, Wednesday EEB? 10.00 12.00, Wednesday

More information

Class 26: review for final exam 18.05, Spring 2014

Class 26: review for final exam 18.05, Spring 2014 Probability Class 26: review for final eam 8.05, Spring 204 Counting Sets Inclusion-eclusion principle Rule of product (multiplication rule) Permutation and combinations Basics Outcome, sample space, event

More information

Lecture 18: Central Limit Theorem. Lisa Yan August 6, 2018

Lecture 18: Central Limit Theorem. Lisa Yan August 6, 2018 Lecture 18: Central Limit Theorem Lisa Yan August 6, 2018 Announcements PS5 due today Pain poll PS6 out today Due next Monday 8/13 (1:30pm) (will not be accepted after Wed 8/15) Programming part: Java,

More information

Overview. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda

Overview. DS GA 1002 Probability and Statistics for Data Science.   Carlos Fernandez-Granda Overview DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing

More information

Lecture Notes 5 Convergence and Limit Theorems. Convergence with Probability 1. Convergence in Mean Square. Convergence in Probability, WLLN

Lecture Notes 5 Convergence and Limit Theorems. Convergence with Probability 1. Convergence in Mean Square. Convergence in Probability, WLLN Lecture Notes 5 Convergence and Limit Theorems Motivation Convergence with Probability Convergence in Mean Square Convergence in Probability, WLLN Convergence in Distribution, CLT EE 278: Convergence and

More information

2 Random Variable Generation

2 Random Variable Generation 2 Random Variable Generation Most Monte Carlo computations require, as a starting point, a sequence of i.i.d. random variables with given marginal distribution. We describe here some of the basic methods

More information

Stat Lecture 20. Last class we introduced the covariance and correlation between two jointly distributed random variables.

Stat Lecture 20. Last class we introduced the covariance and correlation between two jointly distributed random variables. Stat 260 - Lecture 20 Recap of Last Class Last class we introduced the covariance and correlation between two jointly distributed random variables. Today: We will introduce the idea of a statistic and

More information

Learning Objectives for Stat 225

Learning Objectives for Stat 225 Learning Objectives for Stat 225 08/20/12 Introduction to Probability: Get some general ideas about probability, and learn how to use sample space to compute the probability of a specific event. Set Theory:

More information

t x 1 e t dt, and simplify the answer when possible (for example, when r is a positive even number). In particular, confirm that EX 4 = 3.

t x 1 e t dt, and simplify the answer when possible (for example, when r is a positive even number). In particular, confirm that EX 4 = 3. Mathematical Statistics: Homewor problems General guideline. While woring outside the classroom, use any help you want, including people, computer algebra systems, Internet, and solution manuals, but mae

More information

PRACTICE PROBLEMS FOR EXAM 2

PRACTICE PROBLEMS FOR EXAM 2 PRACTICE PROBLEMS FOR EXAM 2 Math 3160Q Fall 2015 Professor Hohn Below is a list of practice questions for Exam 2. Any quiz, homework, or example problem has a chance of being on the exam. For more practice,

More information

3. Review of Probability and Statistics

3. Review of Probability and Statistics 3. Review of Probability and Statistics ECE 830, Spring 2014 Probabilistic models will be used throughout the course to represent noise, errors, and uncertainty in signal processing problems. This lecture

More information

Question 2 We saw in class that if we flip a coin K times, the the distribution in runs of success in the resulting sequence is equal to

Question 2 We saw in class that if we flip a coin K times, the the distribution in runs of success in the resulting sequence is equal to Homework 7, Math070, Spring 205 Question The following table show the split statistics from the ESPN website for Basketball player Stephen Curry for the 204-205 season so far (a) Using relative frequency

More information

EECS 126 Probability and Random Processes University of California, Berkeley: Fall 2014 Kannan Ramchandran September 23, 2014.

EECS 126 Probability and Random Processes University of California, Berkeley: Fall 2014 Kannan Ramchandran September 23, 2014. EECS 126 Probability and Random Processes University of California, Berkeley: Fall 2014 Kannan Ramchandran September 23, 2014 Midterm Exam 1 Last name First name SID Rules. DO NOT open the exam until instructed

More information

Mathematical Statistics 1 Math A 6330

Mathematical Statistics 1 Math A 6330 Mathematical Statistics 1 Math A 6330 Chapter 3 Common Families of Distributions Mohamed I. Riffi Department of Mathematics Islamic University of Gaza September 28, 2015 Outline 1 Subjects of Lecture 04

More information

STAT2201. Analysis of Engineering & Scientific Data. Unit 3

STAT2201. Analysis of Engineering & Scientific Data. Unit 3 STAT2201 Analysis of Engineering & Scientific Data Unit 3 Slava Vaisman The University of Queensland School of Mathematics and Physics What we learned in Unit 2 (1) We defined a sample space of a random

More information

Brief Review of Probability

Brief Review of Probability Maura Department of Economics and Finance Università Tor Vergata Outline 1 Distribution Functions Quantiles and Modes of a Distribution 2 Example 3 Example 4 Distributions Outline Distribution Functions

More information

Midterm Exam 1 Solution

Midterm Exam 1 Solution EECS 126 Probability and Random Processes University of California, Berkeley: Fall 2015 Kannan Ramchandran September 22, 2015 Midterm Exam 1 Solution Last name First name SID Name of student on your left:

More information

Stat 5101 Lecture Slides: Deck 7 Asymptotics, also called Large Sample Theory. Charles J. Geyer School of Statistics University of Minnesota

Stat 5101 Lecture Slides: Deck 7 Asymptotics, also called Large Sample Theory. Charles J. Geyer School of Statistics University of Minnesota Stat 5101 Lecture Slides: Deck 7 Asymptotics, also called Large Sample Theory Charles J. Geyer School of Statistics University of Minnesota 1 Asymptotic Approximation The last big subject in probability

More information

Swarthmore Honors Exam 2012: Statistics

Swarthmore Honors Exam 2012: Statistics Swarthmore Honors Exam 2012: Statistics 1 Swarthmore Honors Exam 2012: Statistics John W. Emerson, Yale University NAME: Instructions: This is a closed-book three-hour exam having six questions. You may

More information

Expectation, inequalities and laws of large numbers

Expectation, inequalities and laws of large numbers Chapter 3 Expectation, inequalities and laws of large numbers 3. Expectation and Variance Indicator random variable Let us suppose that the event A partitions the sample space S, i.e. A A S. The indicator

More information

Each trial has only two possible outcomes success and failure. The possible outcomes are exactly the same for each trial.

Each trial has only two possible outcomes success and failure. The possible outcomes are exactly the same for each trial. Section 8.6: Bernoulli Experiments and Binomial Distribution We have already learned how to solve problems such as if a person randomly guesses the answers to 10 multiple choice questions, what is the

More information

Introduction to Probability

Introduction to Probability Introduction to Probability Math 353, Section 1 Fall 212 Homework 9 Solutions 1. During a season, a basketball team plays 7 home games and 6 away games. The coach estimates at the beginning of the season

More information

Lecture 4: Two-point Sampling, Coupon Collector s problem

Lecture 4: Two-point Sampling, Coupon Collector s problem Randomized Algorithms Lecture 4: Two-point Sampling, Coupon Collector s problem Sotiris Nikoletseas Associate Professor CEID - ETY Course 2013-2014 Sotiris Nikoletseas, Associate Professor Randomized Algorithms

More information

variance of independent variables: sum of variances So chebyshev predicts won t stray beyond stdev.

variance of independent variables: sum of variances So chebyshev predicts won t stray beyond stdev. Announcements No class monday. Metric embedding seminar. Review expectation notion of high probability. Markov. Today: Book 4.1, 3.3, 4.2 Chebyshev. Remind variance, standard deviation. σ 2 = E[(X µ X

More information

Lecture 1: August 28

Lecture 1: August 28 36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 1: August 28 Our broad goal for the first few lectures is to try to understand the behaviour of sums of independent random

More information

Continuous-time Markov Chains

Continuous-time Markov Chains Continuous-time Markov Chains Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ October 23, 2017

More information

It can be shown that if X 1 ;X 2 ;:::;X n are independent r.v. s with

It can be shown that if X 1 ;X 2 ;:::;X n are independent r.v. s with Example: Alternative calculation of mean and variance of binomial distribution A r.v. X has the Bernoulli distribution if it takes the values 1 ( success ) or 0 ( failure ) with probabilities p and (1

More information

1 Presessional Probability

1 Presessional Probability 1 Presessional Probability Probability theory is essential for the development of mathematical models in finance, because of the randomness nature of price fluctuations in the markets. This presessional

More information

MA , Probability (Dr Chernov) Final Exam Tue, March 7, 2000 Student s name Be sure to show all your work. Each problem is 4 points.

MA , Probability (Dr Chernov) Final Exam Tue, March 7, 2000 Student s name Be sure to show all your work. Each problem is 4 points. MA 485-12, Probability (Dr Chernov) Final Exam Tue, March 7, 2000 Student s name Be sure to show all your work. Each problem is 4 points. 1. A basketball player makes 75% of his free throws on the average.

More information