Convergence of Random Processes
|
|
- Oswin Valentine Nicholson
- 5 years ago
- Views:
Transcription
1 Convergence of Random Processes DS GA 1002 Probability and Statistics for Data Science Carlos Fernandez-Granda
2 Aim Define convergence for random processes Describe two convergence phenomena: the law of large numbers and the central limit theorem
3 Types of convergence Law of Large Numbers Central Limit Theorem Monte Carlo simulation
4 Convergence of deterministic sequences A deterministic sequence of real numbers x 1, x 2,... converges to x R, lim x i = x i if x i is arbitrarily close to x as i grows For any ɛ > 0 there is an i 0 such that for all i > i 0 x i x < ɛ Problem: Random sequences do not have fixed values
5 Convergence with probability one Consider a discrete random process X and a random variable X defined on the same probability space If we fix the outcome ω, X (i, ω) is a deterministic sequence and X (ω) is a constant We can determine whether for that particular ω lim X (i, ω) = X (ω) i
6 Convergence with probability one X converges with probability one to X if ({ P ω ω Ω, }) lim X (ω, i) = X (ω) = 1 i Deterministic convergence occurs with probability one
7 Puddle Initial amount of water is uniform between 0 and 1 gallon After a time interval i there is i times less water D (ω, i) := ω, i = 1, 2,... i
8 Puddle 0.8 ω = 0.31 ω = 0.89 ω = 0.52 D (ω, i) i
9 Puddle If we fix ω (0, 1) lim D (ω, i) = lim i = 0 i i ω D converges to zero with probability one
10 Puddle 1 D (ω, i) i
11 Alternative idea Idea: Instead of fixing ω and checking deterministic convergence: 1. Measure how close X (i) and X are for a fixed i using a deterministic quantity 2. Check whether the quantity tends to zero
12 Convergence in mean square The mean square of Y X measures how close X and Y are ( If E (X Y ) 2) = 0 then X = Y with probability one Proof: By Markov s inequality for any ɛ > 0 ( ) E ((X Y ) 2) P (Y X ) 2 > ɛ = 0 ɛ
13 Convergence in mean square X converges to X in mean square if ( ( ) lim E X X 2 (i)) = 0 i
14 Convergence in probability Alternative measure: Probability that Y X > ɛ for small ɛ X converges to X in probability if for any ɛ > 0 ( ) X lim P X (i) > ɛ = 0 i
15 Conv. in mean square implies conv. in probability ( ) X lim P X (i) > ɛ i
16 Conv. in mean square implies conv. in probability ( ) ( ( X lim P X (i) > ɛ = lim P X X ) 2 (i) > ɛ 2) i i
17 Conv. in mean square implies conv. in probability ( ) ( ( X lim P X (i) > ɛ = lim P X X ) 2 (i) > ɛ 2) i i ( ( E X X ) ) 2 (i) lim i ɛ 2
18 Conv. in mean square implies conv. in probability ( ) ( ( X lim P X (i) > ɛ = lim P X X ) 2 (i) > ɛ 2) i i ( ( E X X ) ) 2 (i) lim i ɛ 2 = 0
19 Conv. in mean square implies conv. in probability ( ) ( ( X lim P X (i) > ɛ = lim P X X ) 2 (i) > ɛ 2) i i ( ( E X X ) ) 2 (i) lim i ɛ 2 = 0 Convergence with probability one also implies convergence in probability
20 Convergence in distribution The distribution of X (i) converges to the distribution of X X converges in distribution to X if for all x at which F X is continuous lim F i X (i) (x) = F X (x)
21 Convergence in distribution Convergence in distribution does not imply that X (i) and X are close as i! Convergence in probability does imply convergence in distribution
22 Binomial tends to Poisson X (i) is binomial with parameters i and p := λ/i X is a Poisson random variable with parameter λ X (i) converges to X in distribution ( ) i lim p i X (i) (x) = lim p x (1 p) (i x) i x = λx e λ x! = p X (x)
23 Probability mass function of X (40) k
24 Probability mass function of X (80) k
25 Probability mass function of X (400) k
26 Probability mass function of X k
27 Types of convergence Law of Large Numbers Central Limit Theorem Monte Carlo simulation
28 Moving average The moving average à of a discrete random process X is à (i) := 1 i i X (j) j=1
29 Weak law of large numbers Let X be an iid discrete random process with mean µ X := µ and bounded variance σ 2 The average à of X converges in mean square to µ
30 Proof ) E (Ã (i)
31 Proof ) E (Ã (i) = E 1 i i X (j) j=1
32 Proof ) E (Ã (i) = E 1 i = 1 i i j=1 i X (j) j=1 ( ) E X (j)
33 Proof ) E (Ã (i) = E 1 i = 1 i = µ i j=1 i X (j) j=1 ( ) E X (j)
34 Proof ) Var (Ã (i)
35 Proof ) Var (Ã (i) = Var 1 i i X (j) j=1
36 Proof ) Var (Ã (i) = Var 1 i = 1 i 2 i j=1 i X (j) j=1 ( ) Var X (j)
37 Proof ) Var (Ã (i) = Var 1 i = 1 i 2 = σ2 i i j=1 i X (j) j=1 ( ) Var X (j)
38 Proof ( (Ã ) ) 2 lim E (i) µ i
39 Proof ( (Ã ) ) ( 2 (Ã )) ) 2 lim E (i) µ = lim E (i) E (Ã (i) i i
40 Proof ( (Ã ) ) ( 2 (Ã )) ) 2 lim E (i) µ = lim E (i) E (Ã (i) i i ) = lim Var (Ã (i) i
41 Proof ( (Ã ) ) ( 2 (Ã )) ) 2 lim E (i) µ = lim E (i) E (Ã (i) i i ) = lim Var (Ã (i) i = lim i σ 2 i
42 Proof ( (Ã ) ) ( 2 (Ã )) ) 2 lim E (i) µ = lim E (i) E (Ã (i) i i ) = lim Var (Ã (i) i = lim i σ 2 i = 0
43 Strong law of large numbers Let X be an iid discrete random process with mean µ X := µ and bounded variance σ 2 The average à of X converges with probability one to µ
44 iid standard Gaussian Moving average Mean of iid seq i 40 50
45 iid standard Gaussian Moving average Mean of iid seq i
46 iid standard Gaussian Moving average Mean of iid seq i
47 iid geometric with p = Moving average Mean of iid seq i
48 iid geometric with p = Moving average Mean of iid seq i
49 iid geometric with p = Moving average Mean of iid seq i
50 iid Cauchy Moving average Median of iid seq i
51 iid Cauchy 10 5 Moving average Median of iid seq i
52 iid Cauchy Moving average Median of iid seq i
53 Types of convergence Law of Large Numbers Central Limit Theorem Monte Carlo simulation
54 Central Limit Theorem Let X be an iid discrete random process with mean µ X := µ and bounded variance σ 2 ) n (à µ converges in distribution to a Gaussian random variable with mean 0 and variance σ 2 The average à is approximately Gaussian with mean µ and variance σ2 /i
55 Height data Example: Data from a population of people We compare the histogram of the heights and the pdf of a Gaussian random variable fitted to the data
56 Height data Gaussian distribution Real data Height (inches)
57 Sketch of proof Pdf of sum of two independent random variables is the convolution of their pdfs f X +Y (z) = y= f X (z y) f Y (y) dy Repeated convolutions of any pdf with bounded variance result in a Gaussian!
58 Repeated convolutions i = 1 i = 2 i = 3 i = 4 i = 5
59 Repeated convolutions i = 1 i = 2 i = 3 i = 4 i = 5
60 iid exponential λ = 2, i =
61 iid exponential λ = 2, i =
62 iid exponential λ = 2, i =
63 iid geometric p = 0.4, i =
64 iid geometric p = 0.4, i =
65 iid geometric p = 0.4, i =
66 iid Cauchy, i =
67 iid Cauchy, i =
68 iid Cauchy, i =
69 Gaussian approximation to the binomial X is binomial with parameters n and p Computing the probability that X is in a certain interval requires summing its pmf over the interval Central limit theorem provides a quick approximation X = n B i, E (B i ) = p, Var (B i ) = p (1 p) i=1 1 nx is approximately Gaussian with mean p and variance p (1 p) /n X is approximately Gaussian with mean np and variance np (1 p)
70 Gaussian approximation to the binomial Basketball player makes shot with probability p = 0.4 (shots are iid) Probability that she makes more than 420 shots out of 1000? Exact answer: Approximation: P (X 420) = = P (X 420) 1000 x= x=420 p X (x) ( 1000 x ) 0.4 x 0.6 (n x) =
71 Gaussian approximation to the binomial Basketball player makes shot with probability p = 0.4 (shots are iid) Probability that she makes more than 420 shots out of 1000? Exact answer: P (X 420) = = 1000 x= x=420 p X (x) ( 1000 x ) 0.4 x 0.6 (n x) = Approximation: ( ) P (X 420) P np (1 p)u + np 420
72 Gaussian approximation to the binomial Basketball player makes shot with probability p = 0.4 (shots are iid) Probability that she makes more than 420 shots out of 1000? Exact answer: P (X 420) = = 1000 x= x=420 p X (x) ( 1000 x ) 0.4 x 0.6 (n x) = Approximation: ( ) P (X 420) P np (1 p)u + np 420 = P (U 1.29)
73 Gaussian approximation to the binomial Basketball player makes shot with probability p = 0.4 (shots are iid) Probability that she makes more than 420 shots out of 1000? Exact answer: P (X 420) = = 1000 x= x=420 p X (x) ( 1000 x ) 0.4 x 0.6 (n x) = Approximation: ( ) P (X 420) P np (1 p)u + np 420 = P (U 1.29) = 1 Φ (1.29) =
74 Types of convergence Law of Large Numbers Central Limit Theorem Monte Carlo simulation
75 Monte Carlo simulation Simulation is a powerful tool in probability and statistics Models are too complex to derive closed-form solutions (life is not a homework problem!) Example: Game of solitaire
76 Game of solitaire Aim: Compute the probability that you win at solitaire If every permutation of the cards has the same probability P (Win) = Number of permutations that lead to a win Total number Problem: Characterizing permutations that lead to a win is very difficult without playing out the game We can t just check because there are 52! permutations! Solution: Sample many permutations and compute the fraction of wins
77 In the words of Stanislaw Ulam The first thoughts and attempts I made to practice (the Monte Carlo Method) were suggested by a question which occurred to me in 1946 as I was convalescing from an illness and playing solitaires. The question was what are the chances that a Canfield solitaire laid out with 52 cards will come out successfully? After spending a lot of time trying to estimate them by pure combinatorial calculations, I wondered whether a more practical method than "abstract thinking" might not be to lay it out say one hundred times and simply observe and count the number of successful plays.this was already possible to envisage with the beginning of the new era of fast computers.
78 Monte Carlo approximation Main principle: Use simulation to approximate quantities that are challenging to compute exactly To approximate the probability of an event E 1. Generate n independent samples from 1 E : I 1, I 2,..., I n 2. Compute the average of the n samples n à (n) := 1 n i=1 I i By the law of large numbers à converges to P (E) as n since E (1 E ) = P (E)
79 Basketball league Basketball league with m teams In a season every pair of teams plays once Teams are ordered: team 1 is best, team m is worst Model: For 1 i < j m Games are independent P (team j beats team i) := 1 j i + 1
80 Basketball league Aim: Compute probability of team ranks at the end of the season The rank of team i is modeled as a random variable R i Pmf of R 1, R 2,..., R m?
81 m = 3 Game outcomes Rank R 1 R 2 R 3 Probability / / / / / / / /12
82 m = 3 Probability mass function R 1 R 2 R 3 1 7/12 1/2 5/12 2 1/4 1/4 1/4 3 1/6 1/4 1/3
83 Basketball league Problem: Number of possible outcomes is 2 m(m 1)/2! For m = 10 this is larger than Solution: Apply Monte Carlo approximation
84 m = 3 Game outcomes Rank R 1 R 2 R
85 m = 3 Estimated pmf (n = 10) R 1 R 2 R (0.583) 0.7 (0.5) 0.3 (0.417) (0.25) 0.2 (0.25) 0.4 (0.25) (0.167) 0.1 (0.25) 0.3 (0.333)
86 m = 3 Estimated pmf (n = 2, 000) R 1 R 2 R (0.583) (0.5) (0.417) (0.25) (0.25) (0.25) (0.167) (0.25) (0.333)
87 Running times Running time (seconds) Exact computation Monte Carlo approx Number of teams m
88 Error m Average error
89 m = 5
90 m = 20
91 m = 100
Applied Statistics. Monte Carlo Simulations. Troels C. Petersen (NBI) Statistics is merely a quantisation of common sense
Applied Statistics Monte Carlo Simulations Troels C. Petersen (NBI) Statistics is merely a quantisation of common sense 1 Random numbers Any one who considers arithmetical methods of producing random digits
More informationExpectation. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Expectation DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean, variance,
More informationRandom Processes. DS GA 1002 Probability and Statistics for Data Science.
Random Processes DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Aim Modeling quantities that evolve in time (or space)
More informationExpectation. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda
Expectation DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean,
More informationComputer intensive statistical methods Lecture 1
Computer intensive statistical methods Lecture 1 Jonas Wallin Chalmers, Gothenburg. Jonas Wallin - jonwal@chalmers.se 1/27 People and literature The following people are involved in the course: Function
More informationReview. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Review DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing with
More informationMATH Notebook 5 Fall 2018/2019
MATH442601 2 Notebook 5 Fall 2018/2019 prepared by Professor Jenny Baglivo c Copyright 2004-2019 by Jenny A. Baglivo. All Rights Reserved. 5 MATH442601 2 Notebook 5 3 5.1 Sequences of IID Random Variables.............................
More informationRandom variables. DS GA 1002 Probability and Statistics for Data Science.
Random variables DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Motivation Random variables model numerical quantities
More informationWeek 9 The Central Limit Theorem and Estimation Concepts
Week 9 and Estimation Concepts Week 9 and Estimation Concepts Week 9 Objectives 1 The Law of Large Numbers and the concept of consistency of averages are introduced. The condition of existence of the population
More informationSTAT/MATH 395 PROBABILITY II
STAT/MATH 395 PROBABILITY II Chapter 6 : Moment Functions Néhémy Lim 1 1 Department of Statistics, University of Washington, USA Winter Quarter 2016 of Common Distributions Outline 1 2 3 of Common Distributions
More informationDescriptive Statistics
Descriptive Statistics DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Descriptive statistics Techniques to visualize
More informationThings to remember when learning probability distributions:
SPECIAL DISTRIBUTIONS Some distributions are special because they are useful They include: Poisson, exponential, Normal (Gaussian), Gamma, geometric, negative binomial, Binomial and hypergeometric distributions
More informationProbability and Statistics for Data Science. Carlos Fernandez-Granda
Probability and Statistics for Data Science Carlos Fernandez-Granda Preface These notes were developed for the course Probability and Statistics for Data Science at the Center for Data Science in NYU.
More informationMonte Carlo Methods. Part I: Introduction
Monte Carlo Methods Part I: Introduction Spring Semester 2013/14 Department of Applied Mathematics, University of Crete Instructor: Harmandaris Vagelis, email: vagelis@tem.uoc.gr Course web page: http://www.tem.uoc.gr/~vagelis/courses/em385/
More informationIntroduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos Contents Markov Chain Monte Carlo Methods Sampling Rejection Importance Hastings-Metropolis Gibbs Markov Chains
More informationChapter 5. Means and Variances
1 Chapter 5 Means and Variances Our discussion of probability has taken us from a simple classical view of counting successes relative to total outcomes and has brought us to the idea of a probability
More informationStatistical Data Analysis
DS-GA 0 Lecture notes 8 Fall 016 1 Descriptive statistics Statistical Data Analysis In this section we consider the problem of analyzing a set of data. We describe several techniques for visualizing the
More informationProbability Distributions Columns (a) through (d)
Discrete Probability Distributions Columns (a) through (d) Probability Mass Distribution Description Notes Notation or Density Function --------------------(PMF or PDF)-------------------- (a) (b) (c)
More informationSTAT/MATH 395 A - PROBABILITY II UW Winter Quarter Moment functions. x r p X (x) (1) E[X r ] = x r f X (x) dx (2) (x E[X]) r p X (x) (3)
STAT/MATH 395 A - PROBABILITY II UW Winter Quarter 07 Néhémy Lim Moment functions Moments of a random variable Definition.. Let X be a rrv on probability space (Ω, A, P). For a given r N, E[X r ], if it
More informationCSE 312: Foundations of Computing II Quiz Section #10: Review Questions for Final Exam (solutions)
CSE 312: Foundations of Computing II Quiz Section #10: Review Questions for Final Exam (solutions) 1. (Confidence Intervals, CLT) Let X 1,..., X n be iid with unknown mean θ and known variance σ 2. Assume
More informationECE 302 Division 1 MWF 10:30-11:20 (Prof. Pollak) Final Exam Solutions, 5/3/2004. Please read the instructions carefully before proceeding.
NAME: ECE 302 Division MWF 0:30-:20 (Prof. Pollak) Final Exam Solutions, 5/3/2004. Please read the instructions carefully before proceeding. If you are not in Prof. Pollak s section, you may not take this
More informationNorthwestern University Department of Electrical Engineering and Computer Science
Northwestern University Department of Electrical Engineering and Computer Science EECS 454: Modeling and Analysis of Communication Networks Spring 2008 Probability Review As discussed in Lecture 1, probability
More informationPart IA Probability. Theorems. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Theorems Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationLecture Notes 7 Random Processes. Markov Processes Markov Chains. Random Processes
Lecture Notes 7 Random Processes Definition IID Processes Bernoulli Process Binomial Counting Process Interarrival Time Process Markov Processes Markov Chains Classification of States Steady State Probabilities
More informationSTAT 414: Introduction to Probability Theory
STAT 414: Introduction to Probability Theory Spring 2016; Homework Assignments Latest updated on April 29, 2016 HW1 (Due on Jan. 21) Chapter 1 Problems 1, 8, 9, 10, 11, 18, 19, 26, 28, 30 Theoretical Exercises
More informationCourse: ESO-209 Home Work: 1 Instructor: Debasis Kundu
Home Work: 1 1. Describe the sample space when a coin is tossed (a) once, (b) three times, (c) n times, (d) an infinite number of times. 2. A coin is tossed until for the first time the same result appear
More informationProbability Models. 4. What is the definition of the expectation of a discrete random variable?
1 Probability Models The list of questions below is provided in order to help you to prepare for the test and exam. It reflects only the theoretical part of the course. You should expect the questions
More informationLecture 1. ABC of Probability
Math 408 - Mathematical Statistics Lecture 1. ABC of Probability January 16, 2013 Konstantin Zuev (USC) Math 408, Lecture 1 January 16, 2013 1 / 9 Agenda Sample Spaces Realizations, Events Axioms of Probability
More informationChapter 6: Large Random Samples Sections
Chapter 6: Large Random Samples Sections 6.1: Introduction 6.2: The Law of Large Numbers Skip p. 356-358 Skip p. 366-368 Skip 6.4: The correction for continuity Remember: The Midterm is October 25th in
More informationIntroduction and Overview STAT 421, SP Course Instructor
Introduction and Overview STAT 421, SP 212 Prof. Prem K. Goel Mon, Wed, Fri 3:3PM 4:48PM Postle Hall 118 Course Instructor Prof. Goel, Prem E mail: goel.1@osu.edu Office: CH 24C (Cockins Hall) Phone: 614
More informationDiscrete Distributions
A simplest example of random experiment is a coin-tossing, formally called Bernoulli trial. It happens to be the case that many useful distributions are built upon this simplest form of experiment, whose
More informationSTAT 418: Probability and Stochastic Processes
STAT 418: Probability and Stochastic Processes Spring 2016; Homework Assignments Latest updated on April 29, 2016 HW1 (Due on Jan. 21) Chapter 1 Problems 1, 8, 9, 10, 11, 18, 19, 26, 28, 30 Theoretical
More informationLecture 6. Probability events. Definition 1. The sample space, S, of a. probability experiment is the collection of all
Lecture 6 1 Lecture 6 Probability events Definition 1. The sample space, S, of a probability experiment is the collection of all possible outcomes of an experiment. One such outcome is called a simple
More informationBusiness Statistics Midterm Exam Fall 2015 Russell. Please sign here to acknowledge
Business Statistics Midterm Exam Fall 5 Russell Name Do not turn over this page until you are told to do so. You will have hour and 3 minutes to complete the exam. There are a total of points divided into
More information(b). What is an expression for the exact value of P(X = 4)? 2. (a). Suppose that the moment generating function for X is M (t) = 2et +1 3
Math 511 Exam #2 Show All Work 1. A package of 200 seeds contains 40 that are defective and will not grow (the rest are fine). Suppose that you choose a sample of 10 seeds from the box without replacement.
More informationPart IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationA Probability Primer. A random walk down a probabilistic path leading to some stochastic thoughts on chance events and uncertain outcomes.
A Probability Primer A random walk down a probabilistic path leading to some stochastic thoughts on chance events and uncertain outcomes. Are you holding all the cards?? Random Events A random event, E,
More informationMath/Stat 394 Homework 5
Math/Stat 394 Homework 5 1. If we select two black balls then X 4. This happens with probability ( 4 2). If we select two white balls then X 2. This happens with ( 14 probability (8 2). If we select two
More informationStatistics for Economists. Lectures 3 & 4
Statistics for Economists Lectures 3 & 4 Asrat Temesgen Stockholm University 1 CHAPTER 2- Discrete Distributions 2.1. Random variables of the Discrete Type Definition 2.1.1: Given a random experiment with
More informationOrder Statistics and Distributions
Order Statistics and Distributions 1 Some Preliminary Comments and Ideas In this section we consider a random sample X 1, X 2,..., X n common continuous distribution function F and probability density
More informationMath 493 Final Exam December 01
Math 493 Final Exam December 01 NAME: ID NUMBER: Return your blue book to my office or the Math Department office by Noon on Tuesday 11 th. On all parts after the first show enough work in your exam booklet
More informationCS145: Probability & Computing
CS45: Probability & Computing Lecture 5: Concentration Inequalities, Law of Large Numbers, Central Limit Theorem Instructor: Eli Upfal Brown University Computer Science Figure credits: Bertsekas & Tsitsiklis,
More information1: PROBABILITY REVIEW
1: PROBABILITY REVIEW Marek Rutkowski School of Mathematics and Statistics University of Sydney Semester 2, 2016 M. Rutkowski (USydney) Slides 1: Probability Review 1 / 56 Outline We will review the following
More information1. Let X and Y be independent exponential random variables with rate α. Find the densities of the random variables X 3, X Y, min(x, Y 3 )
1 Introduction These problems are meant to be practice problems for you to see if you have understood the material reasonably well. They are neither exhaustive (e.g. Diffusions, continuous time branching
More informationLecture 5: The Principle of Deferred Decisions. Chernoff Bounds
Randomized Algorithms Lecture 5: The Principle of Deferred Decisions. Chernoff Bounds Sotiris Nikoletseas Associate Professor CEID - ETY Course 2013-2014 Sotiris Nikoletseas, Associate Professor Randomized
More informationCPSC 320 Sample Final Examination December 2013
CPSC 320 Sample Final Examination December 2013 [10] 1. Answer each of the following questions with true or false. Give a short justification for each of your answers. [5] a. 6 n O(5 n ) lim n + This is
More informationEXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY (formerly the Examinations of the Institute of Statisticians) HIGHER CERTIFICATE IN STATISTICS, 1996
EXAMINATIONS OF THE ROAL STATISTICAL SOCIET (formerly the Examinations of the Institute of Statisticians) HIGHER CERTIFICATE IN STATISTICS, 996 Paper I : Statistical Theory Time Allowed: Three Hours Candidates
More informationFall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
More informationIE 230 Probability & Statistics in Engineering I. Closed book and notes. 60 minutes.
Closed book and notes. 60 minutes. A summary table of some univariate continuous distributions is provided. Four Pages. In this version of the Key, I try to be more complete than necessary to receive full
More informationBayesian statistics. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Bayesian statistics DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall15 Carlos Fernandez-Granda Frequentist vs Bayesian statistics In frequentist statistics
More informationTwelfth Problem Assignment
EECS 401 Not Graded PROBLEM 1 Let X 1, X 2,... be a sequence of independent random variables that are uniformly distributed between 0 and 1. Consider a sequence defined by (a) Y n = max(x 1, X 2,..., X
More informationFundamentals of Applied Probability and Random Processes
Fundamentals of Applied Probability and Random Processes,nd 2 na Edition Oliver C. Ibe University of Massachusetts, LoweLL, Massachusetts ip^ W >!^ AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS
More informationMassachusetts Institute of Technology
Problem. (0 points) Massachusetts Institute of Technology Final Solutions: December 15, 009 (a) (5 points) We re given that the joint PDF is constant in the shaded region, and since the PDF must integrate
More informationChapter 3, 4 Random Variables ENCS Probability and Stochastic Processes. Concordia University
Chapter 3, 4 Random Variables ENCS6161 - Probability and Stochastic Processes Concordia University ENCS6161 p.1/47 The Notion of a Random Variable A random variable X is a function that assigns a real
More informationLecture 3. Discrete Random Variables
Math 408 - Mathematical Statistics Lecture 3. Discrete Random Variables January 23, 2013 Konstantin Zuev (USC) Math 408, Lecture 3 January 23, 2013 1 / 14 Agenda Random Variable: Motivation and Definition
More informationBINOMIAL DISTRIBUTION
BINOMIAL DISTRIBUTION The binomial distribution is a particular type of discrete pmf. It describes random variables which satisfy the following conditions: 1 You perform n identical experiments (called
More informationFundamental Tools - Probability Theory IV
Fundamental Tools - Probability Theory IV MSc Financial Mathematics The University of Warwick October 1, 2015 MSc Financial Mathematics Fundamental Tools - Probability Theory IV 1 / 14 Model-independent
More informationIE 336 Seat # Name (clearly) Closed book. One page of hand-written notes, front and back. No calculator. 60 minutes.
Closed book. One page of hand-written notes, front and back. No calculator. 6 minutes. Cover page and four pages of exam. Fifteen questions. Each question is worth seven points. To receive full credit,
More informationContinuous random variables
Continuous random variables Continuous r.v. s take an uncountably infinite number of possible values. Examples: Heights of people Weights of apples Diameters of bolts Life lengths of light-bulbs We cannot
More informationMachine Learning. Probability Basics. Marc Toussaint University of Stuttgart Summer 2014
Machine Learning Probability Basics Basic definitions: Random variables, joint, conditional, marginal distribution, Bayes theorem & examples; Probability distributions: Binomial, Beta, Multinomial, Dirichlet,
More informationSpecial Discrete RV s. Then X = the number of successes is a binomial RV. X ~ Bin(n,p).
Sect 3.4: Binomial RV Special Discrete RV s 1. Assumptions and definition i. Experiment consists of n repeated trials ii. iii. iv. There are only two possible outcomes on each trial: success (S) or failure
More informationMath Review Sheet, Fall 2008
1 Descriptive Statistics Math 3070-5 Review Sheet, Fall 2008 First we need to know about the relationship among Population Samples Objects The distribution of the population can be given in one of the
More informationSTAT 302 Introduction to Probability Learning Outcomes. Textbook: A First Course in Probability by Sheldon Ross, 8 th ed.
STAT 302 Introduction to Probability Learning Outcomes Textbook: A First Course in Probability by Sheldon Ross, 8 th ed. Chapter 1: Combinatorial Analysis Demonstrate the ability to solve combinatorial
More informationData, Estimation and Inference
Data, Estimation and Inference Pedro Piniés ppinies@robots.ox.ac.uk Michaelmas 2016 1 2 p(x) ( = ) = δ 0 ( < < + δ ) δ ( ) =1. x x+dx (, ) = ( ) ( ) = ( ) ( ) 3 ( ) ( ) 0 ( ) =1 ( = ) = ( ) ( < < ) = (
More informationIB Mathematics HL Year 2 Unit 7 (Core Topic 6: Probability and Statistics) Valuable Practice
IB Mathematics HL Year 2 Unit 7 (Core Topic 6: Probability and Statistics) Valuable Practice 1. We have seen that the TI-83 calculator random number generator X = rand defines a uniformly-distributed random
More informationStat 100a, Introduction to Probability.
Stat 100a, Introduction to Probability. Outline for the day: 1. Geometric random variables. 2. Negative binomial random variables. 3. Moment generating functions. 4. Poisson random variables. 5. Continuous
More informationECE302 Spring 2015 HW10 Solutions May 3,
ECE32 Spring 25 HW Solutions May 3, 25 Solutions to HW Note: Most of these solutions were generated by R. D. Yates and D. J. Goodman, the authors of our textbook. I have added comments in italics where
More informationProbability reminders
CS246 Winter 204 Mining Massive Data Sets Probability reminders Sammy El Ghazzal selghazz@stanfordedu Disclaimer These notes may contain typos, mistakes or confusing points Please contact the author so
More informationLecture 10: Probability distributions TUESDAY, FEBRUARY 19, 2019
Lecture 10: Probability distributions DANIEL WELLER TUESDAY, FEBRUARY 19, 2019 Agenda What is probability? (again) Describing probabilities (distributions) Understanding probabilities (expectation) Partial
More informationMath 2311 Test 1 Review. 1. State whether each situation is categorical or quantitative. If quantitative, state whether it s discrete or continuous.
Math 2311 Test 1 Review Know all definitions! 1. State whether each situation is categorical or quantitative. If quantitative, state whether it s discrete or continuous. a. The amount a person grew (in
More informationDS-GA 1002 Lecture notes 2 Fall Random variables
DS-GA 12 Lecture notes 2 Fall 216 1 Introduction Random variables Random variables are a fundamental tool in probabilistic modeling. They allow us to model numerical quantities that are uncertain: the
More informationMAT 271E Probability and Statistics
MAT 271E Probability and Statistics Spring 2011 Instructor : Class Meets : Office Hours : Textbook : Supp. Text : İlker Bayram EEB 1103 ibayram@itu.edu.tr 13.30 16.30, Wednesday EEB? 10.00 12.00, Wednesday
More informationClass 26: review for final exam 18.05, Spring 2014
Probability Class 26: review for final eam 8.05, Spring 204 Counting Sets Inclusion-eclusion principle Rule of product (multiplication rule) Permutation and combinations Basics Outcome, sample space, event
More informationLecture 18: Central Limit Theorem. Lisa Yan August 6, 2018
Lecture 18: Central Limit Theorem Lisa Yan August 6, 2018 Announcements PS5 due today Pain poll PS6 out today Due next Monday 8/13 (1:30pm) (will not be accepted after Wed 8/15) Programming part: Java,
More informationOverview. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda
Overview DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing
More informationLecture Notes 5 Convergence and Limit Theorems. Convergence with Probability 1. Convergence in Mean Square. Convergence in Probability, WLLN
Lecture Notes 5 Convergence and Limit Theorems Motivation Convergence with Probability Convergence in Mean Square Convergence in Probability, WLLN Convergence in Distribution, CLT EE 278: Convergence and
More information2 Random Variable Generation
2 Random Variable Generation Most Monte Carlo computations require, as a starting point, a sequence of i.i.d. random variables with given marginal distribution. We describe here some of the basic methods
More informationStat Lecture 20. Last class we introduced the covariance and correlation between two jointly distributed random variables.
Stat 260 - Lecture 20 Recap of Last Class Last class we introduced the covariance and correlation between two jointly distributed random variables. Today: We will introduce the idea of a statistic and
More informationLearning Objectives for Stat 225
Learning Objectives for Stat 225 08/20/12 Introduction to Probability: Get some general ideas about probability, and learn how to use sample space to compute the probability of a specific event. Set Theory:
More informationt x 1 e t dt, and simplify the answer when possible (for example, when r is a positive even number). In particular, confirm that EX 4 = 3.
Mathematical Statistics: Homewor problems General guideline. While woring outside the classroom, use any help you want, including people, computer algebra systems, Internet, and solution manuals, but mae
More informationPRACTICE PROBLEMS FOR EXAM 2
PRACTICE PROBLEMS FOR EXAM 2 Math 3160Q Fall 2015 Professor Hohn Below is a list of practice questions for Exam 2. Any quiz, homework, or example problem has a chance of being on the exam. For more practice,
More information3. Review of Probability and Statistics
3. Review of Probability and Statistics ECE 830, Spring 2014 Probabilistic models will be used throughout the course to represent noise, errors, and uncertainty in signal processing problems. This lecture
More informationQuestion 2 We saw in class that if we flip a coin K times, the the distribution in runs of success in the resulting sequence is equal to
Homework 7, Math070, Spring 205 Question The following table show the split statistics from the ESPN website for Basketball player Stephen Curry for the 204-205 season so far (a) Using relative frequency
More informationEECS 126 Probability and Random Processes University of California, Berkeley: Fall 2014 Kannan Ramchandran September 23, 2014.
EECS 126 Probability and Random Processes University of California, Berkeley: Fall 2014 Kannan Ramchandran September 23, 2014 Midterm Exam 1 Last name First name SID Rules. DO NOT open the exam until instructed
More informationMathematical Statistics 1 Math A 6330
Mathematical Statistics 1 Math A 6330 Chapter 3 Common Families of Distributions Mohamed I. Riffi Department of Mathematics Islamic University of Gaza September 28, 2015 Outline 1 Subjects of Lecture 04
More informationSTAT2201. Analysis of Engineering & Scientific Data. Unit 3
STAT2201 Analysis of Engineering & Scientific Data Unit 3 Slava Vaisman The University of Queensland School of Mathematics and Physics What we learned in Unit 2 (1) We defined a sample space of a random
More informationBrief Review of Probability
Maura Department of Economics and Finance Università Tor Vergata Outline 1 Distribution Functions Quantiles and Modes of a Distribution 2 Example 3 Example 4 Distributions Outline Distribution Functions
More informationMidterm Exam 1 Solution
EECS 126 Probability and Random Processes University of California, Berkeley: Fall 2015 Kannan Ramchandran September 22, 2015 Midterm Exam 1 Solution Last name First name SID Name of student on your left:
More informationStat 5101 Lecture Slides: Deck 7 Asymptotics, also called Large Sample Theory. Charles J. Geyer School of Statistics University of Minnesota
Stat 5101 Lecture Slides: Deck 7 Asymptotics, also called Large Sample Theory Charles J. Geyer School of Statistics University of Minnesota 1 Asymptotic Approximation The last big subject in probability
More informationSwarthmore Honors Exam 2012: Statistics
Swarthmore Honors Exam 2012: Statistics 1 Swarthmore Honors Exam 2012: Statistics John W. Emerson, Yale University NAME: Instructions: This is a closed-book three-hour exam having six questions. You may
More informationExpectation, inequalities and laws of large numbers
Chapter 3 Expectation, inequalities and laws of large numbers 3. Expectation and Variance Indicator random variable Let us suppose that the event A partitions the sample space S, i.e. A A S. The indicator
More informationEach trial has only two possible outcomes success and failure. The possible outcomes are exactly the same for each trial.
Section 8.6: Bernoulli Experiments and Binomial Distribution We have already learned how to solve problems such as if a person randomly guesses the answers to 10 multiple choice questions, what is the
More informationIntroduction to Probability
Introduction to Probability Math 353, Section 1 Fall 212 Homework 9 Solutions 1. During a season, a basketball team plays 7 home games and 6 away games. The coach estimates at the beginning of the season
More informationLecture 4: Two-point Sampling, Coupon Collector s problem
Randomized Algorithms Lecture 4: Two-point Sampling, Coupon Collector s problem Sotiris Nikoletseas Associate Professor CEID - ETY Course 2013-2014 Sotiris Nikoletseas, Associate Professor Randomized Algorithms
More informationvariance of independent variables: sum of variances So chebyshev predicts won t stray beyond stdev.
Announcements No class monday. Metric embedding seminar. Review expectation notion of high probability. Markov. Today: Book 4.1, 3.3, 4.2 Chebyshev. Remind variance, standard deviation. σ 2 = E[(X µ X
More informationLecture 1: August 28
36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 1: August 28 Our broad goal for the first few lectures is to try to understand the behaviour of sums of independent random
More informationContinuous-time Markov Chains
Continuous-time Markov Chains Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ October 23, 2017
More informationIt can be shown that if X 1 ;X 2 ;:::;X n are independent r.v. s with
Example: Alternative calculation of mean and variance of binomial distribution A r.v. X has the Bernoulli distribution if it takes the values 1 ( success ) or 0 ( failure ) with probabilities p and (1
More information1 Presessional Probability
1 Presessional Probability Probability theory is essential for the development of mathematical models in finance, because of the randomness nature of price fluctuations in the markets. This presessional
More informationMA , Probability (Dr Chernov) Final Exam Tue, March 7, 2000 Student s name Be sure to show all your work. Each problem is 4 points.
MA 485-12, Probability (Dr Chernov) Final Exam Tue, March 7, 2000 Student s name Be sure to show all your work. Each problem is 4 points. 1. A basketball player makes 75% of his free throws on the average.
More information