Advanced Probability Theory (Math541)
|
|
|
- Marjory Scott
- 6 years ago
- Views:
Transcription
1 Advanced Probability Theory (Math541) Instructor: Kani Chen (Classic)/Modern Probability Theory ( ) Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 1 / 17
2 Primitive/Classic Probability: (16th-19th century) Gerolamo Cardano ( ) Liber de Ludo Aleae (games of chance) Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 2 / 17
3 Primitive/Classic Probability: (16th-19th century) Gerolamo Cardano ( ) Liber de Ludo Aleae (games of chance) Galilei Galileo ( ) Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 2 / 17
4 Primitive/Classic Probability: (16th-19th century) Gerolamo Cardano ( ) Liber de Ludo Aleae (games of chance) Galilei Galileo ( ) Pierre de Fermat ( ) and Blaise Pascal ( ) Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 2 / 17
5 Primitive/Classic Probability: (16th-19th century) Gerolamo Cardano ( ) Liber de Ludo Aleae (games of chance) Galilei Galileo ( ) Pierre de Fermat ( ) and Blaise Pascal ( ) Jakob Bernoulli ( ) Bernoulli trial/distribution/r.v/numbers Daniel ( ) (utility function) Johann ( ) (L Hopital rule) Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 2 / 17
6 Abraham de Moivre ( ) Doctrine of Chances Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 3 / 17
7 Abraham de Moivre ( ) Doctrine of Chances Pierre-Simon Laplace ( ) Laplace transform De Moivre-Laplace Theorem (the central limit theorem) Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 3 / 17
8 Abraham de Moivre ( ) Doctrine of Chances Pierre-Simon Laplace ( ) Laplace transform De Moivre-Laplace Theorem (the central limit theorem) Simeon Denis Poisson ( ). Poisson distribution/process, the law of rare events Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 3 / 17
9 Abraham de Moivre ( ) Doctrine of Chances Pierre-Simon Laplace ( ) Laplace transform De Moivre-Laplace Theorem (the central limit theorem) Simeon Denis Poisson ( ). Poisson distribution/process, the law of rare events Emile Borel ( ). Borel sets/measurable, Borel-Cantelli lemma, Borel strong law. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 3 / 17
10 Foundation of modern probability: Keynes, J. M. (1921): A Treatise on Probability. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 4 / 17
11 Foundation of modern probability: Keynes, J. M. (1921): A Treatise on Probability. von Mises, R. (1928): Probability, Statistics and Truth. (1931): Mathematical Theory of Probability and Statistics. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 4 / 17
12 Foundation of modern probability: Keynes, J. M. (1921): A Treatise on Probability. von Mises, R. (1928): Probability, Statistics and Truth. (1931): Mathematical Theory of Probability and Statistics. Kolmogorov, A. (1933): Foundations of the Theory of Probability. Kolmogorov s axioms: Probability space trio: (Ω, F, P). Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 4 / 17
13 Measure-theoretic Probabilities (Chapter 1 begins) Sets, set operations (, and complement), set of sets/subsets, algebra, σ-algebra, measurable space (Ω, F), product space... Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17
14 Measure-theoretic Probabilities (Chapter 1 begins) Sets, set operations (, and complement), set of sets/subsets, algebra, σ-algebra, measurable space (Ω, F), product space... measures, probabilities, product measure, independence of events, conditional probabilities, conditional expectation with respect to σ-algebra. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17
15 Measure-theoretic Probabilities (Chapter 1 begins) Sets, set operations (, and complement), set of sets/subsets, algebra, σ-algebra, measurable space (Ω, F), product space... measures, probabilities, product measure, independence of events, conditional probabilities, conditional expectation with respect to σ-algebra. Transformation/map, measurable transformation/map, random variables/elements defined through mapping X = X(ω). Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17
16 Measure-theoretic Probabilities (Chapter 1 begins) Sets, set operations (, and complement), set of sets/subsets, algebra, σ-algebra, measurable space (Ω, F), product space... measures, probabilities, product measure, independence of events, conditional probabilities, conditional expectation with respect to σ-algebra. Transformation/map, measurable transformation/map, random variables/elements defined through mapping X = X(ω). Expectation/integral. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17
17 Measure-theoretic Probabilities (Chapter 1 begins) Sets, set operations (, and complement), set of sets/subsets, algebra, σ-algebra, measurable space (Ω, F), product space... measures, probabilities, product measure, independence of events, conditional probabilities, conditional expectation with respect to σ-algebra. Transformation/map, measurable transformation/map, random variables/elements defined through mapping X = X(ω). Expectation/integral. Caratheodory s extension theorem, Kolmogorov s extension theorem, Dynkin s π λ theorem, The Radon-Nikodym Theorem. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17
18 Convergence. Convergence modes: convergence almost sure (strong), in probability, in L p (most commonly, L 1 or L 2 ), in distribution/law (weak). The relations of the convergence modes. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 6 / 17
19 Convergence. Convergence modes: convergence almost sure (strong), in probability, in L p (most commonly, L 1 or L 2 ), in distribution/law (weak). The relations of the convergence modes. Dominated convergence theorem, (extension to uniformly integrable r.v.s.) monotone convergence theorem. Fatou s lemma. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 6 / 17
20 Law of Large numbers. X 1,..., X n,... are iid random variables with mean µ. Let S n = n i=1 X i. Weak law of Large numbers: S n /n µ, in probability. i.e., for any ǫ > 0, P( S n /n µ > ǫ) 0. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 7 / 17
21 Law of Large numbers. X 1,..., X n,... are iid random variables with mean µ. Let S n = n i=1 X i. Weak law of Large numbers: S n /n µ, in probability. i.e., for any ǫ > 0, P( S n /n µ > ǫ) 0. Kolmogorov s Strong law of Large numbers: S n /n µ, almost surely. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 7 / 17
22 Law of Large numbers. X 1,..., X n,... are iid random variables with mean µ. Let S n = n i=1 X i. Weak law of Large numbers: S n /n µ, in probability. i.e., for any ǫ > 0, P( S n /n µ > ǫ) 0. Kolmogorov s Strong law of Large numbers: S n /n µ, almost surely. Consistency in statistical estimation. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 7 / 17
23 Large deviation (strengthening weak law) For fixed t > 0, how fast is P(S n /n µ > t) 0? Under regularity conditions, 1 n log P(S n/n µ > t) γ(t), where γ is determined by the commmon distribution of X i. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 8 / 17
24 Large deviation (strengthening weak law) For fixed t > 0, how fast is P(S n /n µ > t) 0? Under regularity conditions, 1 n log P(S n/n µ > t) γ(t), where γ is determined by the commmon distribution of X i. Special case, X i are iid N(0, 1), then γ(t) = t 2 /2. Note that 1 Φ(x) φ(x)/x. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 8 / 17
25 Law of iterated logarithm (strengthening strong law) S n /n µ 0 a.s. Is there a proper a n such that the limit of S n /a n is nonzero finite? Kolmogorov s law of iterated logarithm. lim sup n S n /n µ 2σ 2 n log log(n) 1, a.s. where σ 2 = var(x i ). Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 9 / 17
26 Law of iterated logarithm (strengthening strong law) S n /n µ 0 a.s. Is there a proper a n such that the limit of S n /a n is nonzero finite? Kolmogorov s law of iterated logarithm. lim sup n S n /n µ 2σ 2 n log log(n) 1, a.s. where σ 2 = var(x i ). Marcinkiewicz-Zygmund strong law: for 0 < p < 2, S n nc n 1/p 0, a.s., if and only if E( X i p ) <, where c = µ for 1 p < 2. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 9 / 17
27 Convergence of Series. The convergence of S n for independent X 1,..., X n. Khintchine s convergence theorem: if E(X i ) = 0 and n var(x n) <, then S n converges a.s. as well as in L 2. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 10 / 17
28 Convergence of Series. The convergence of S n for independent X 1,..., X n. Khintchine s convergence theorem: if E(X i ) = 0 and n var(x n) <, then S n converges a.s. as well as in L 2. Kolmogorov s three series theorem: S n converges a.s. if and only if n P( X n > 1) <, n E(X n1 { Xn 1}) <, and n var(x n1 { Xn 1}) <. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 10 / 17
29 Convergence of Series. The convergence of S n for independent X 1,..., X n. Khintchine s convergence theorem: if E(X i ) = 0 and n var(x n) <, then S n converges a.s. as well as in L 2. Kolmogorov s three series theorem: S n converges a.s. if and only if n P( X n > 1) <, n E(X n1 { Xn 1}) <, and n var(x n1 { Xn 1}) <. Kronecker Lemma: If 0 < b n and n (a n/b n ) < then n j=1 a j/b n 0. A technique to show law of large numbers via convergence of series. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 10 / 17
30 The central limit theorem (Chapter 2). De Moivre-Lapalace Theorem For X 1, X 2... independent, under proper conditions, S n E(S n ) var(sn ) N(0, 1) in distribution. i.e., ( Sn E(S n ) ) P var(sn ) < x P(N(0, 1) < x) for all x. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 11 / 17
31 The conditions and extensions Lyapunov. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 12 / 17
32 The conditions and extensions Lyapunov. Lindeberg (1922) condition: X j is mean 0 with variance σj 2, such that n E(Xj 2 1 { Xj >ǫs n}) = o(sn) 2 j=1 for all ǫ > 0, where s 2 n = n j=1 σ2 j. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 12 / 17
33 The conditions and extensions Lyapunov. Lindeberg (1922) condition: X j is mean 0 with variance σj 2, such that n E(Xj 2 1 { Xj >ǫs n}) = o(sn) 2 j=1 for all ǫ > 0, where s 2 n = n j=1 σ2 j. Feller (1935): If CLT holds, σ n /s n 0 and s n, then the above Lindeberg condition holds. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 12 / 17
34 The conditions and extensions Lyapunov. Lindeberg (1922) condition: X j is mean 0 with variance σj 2, such that n E(Xj 2 1 { Xj >ǫs n}) = o(sn) 2 j=1 for all ǫ > 0, where s 2 n = n j=1 σ2 j. Feller (1935): If CLT holds, σ n /s n 0 and s n, then the above Lindeberg condition holds. Extensions to martingales, Markov process... Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 12 / 17
35 The conditions and extensions Lyapunov. Lindeberg (1922) condition: X j is mean 0 with variance σj 2, such that n E(Xj 2 1 { Xj >ǫs n}) = o(sn) 2 j=1 for all ǫ > 0, where s 2 n = n j=1 σ2 j. Feller (1935): If CLT holds, σ n /s n 0 and s n, then the above Lindeberg condition holds. Extensions to martingales, Markov process... Inference in statistics (accuracy justification of estimation: confidence interval, test of hypothesis.) Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 12 / 17
36 Rate of convergence to normality Suppose X 1,..., X n are iid. Berry-Esseen Bound: ( Sn nµ ) P < x nσ 2 P(N(0, 1) < x) cγ/σ3 n 1/2 for all x, where γ = E( X i 3 ) and c is a universal constant. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 13 / 17
37 Rate of convergence to normality Suppose X 1,..., X n are iid. Berry-Esseen Bound: ( Sn nµ ) P < x nσ 2 P(N(0, 1) < x) cγ/σ3 n 1/2 for all x, where γ = E( X i 3 ) and c is a universal constant. Extensions, e.g., to non iid cases, etc.. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 13 / 17
38 Rate of convergence to normality Suppose X 1,..., X n are iid. Berry-Esseen Bound: ( Sn nµ ) P < x nσ 2 P(N(0, 1) < x) cγ/σ3 n 1/2 for all x, where γ = E( X i 3 ) and c is a universal constant. Extensions, e.g., to non iid cases, etc.. Edgeworth expansion. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 13 / 17
39 Random Walk (Chapter 3) Given X 1, X 2,... iid, we study the behavior {S 1, S 2,...} as a sequence of random variables. Stopping times. T is an integer-valued r.v. such that T = n only depends on the values of X 1,..., X n, on, in other words, the values S 1,..., S n. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 14 / 17
40 Random Walk (Chapter 3) Given X 1, X 2,... iid, we study the behavior {S 1, S 2,...} as a sequence of random variables. Stopping times. T is an integer-valued r.v. such that T = n only depends on the values of X 1,..., X n, on, in other words, the values S 1,..., S n. Wald s identity/equation: Under proper conditions, E(S T ) = µe(t). Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 14 / 17
41 Random Walk (Chapter 3) Given X 1, X 2,... iid, we study the behavior {S 1, S 2,...} as a sequence of random variables. Stopping times. T is an integer-valued r.v. such that T = n only depends on the values of X 1,..., X n, on, in other words, the values S 1,..., S n. Wald s identity/equation: Under proper conditions, E(S T ) = µe(t). Interpretation: if X i is the gain/loss of the i-th game, which is fair in the sense that E(X i ) = 0. Then S n is the cumulative gain/loss in the first n games. Under proper conditions, any exit strategy (stopping time) shall still break even. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 14 / 17
42 Tool box: Indicator functions. (e.g.,borel-cantelli Lemma Borel s strong law of large numbers Kolmogorov s 0-1 law.) Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 15 / 17
43 Tool box: Indicator functions. (e.g.,borel-cantelli Lemma Borel s strong law of large numbers Kolmogorov s 0-1 law.) Truncation. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 15 / 17
44 Tool box: Indicator functions. (e.g.,borel-cantelli Lemma Borel s strong law of large numbers Kolmogorov s 0-1 law.) Truncation. Characteristic functions/moment generating functions. (in proving the CLT and its convergence rates) Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 15 / 17
45 Inequalities Jensen, Holder, Cauchy-Schwartz, Lyapunov, Minkowski. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 16 / 17
46 Inequalities Jensen, Holder, Cauchy-Schwartz, Lyapunov, Minkowski. Markov, Chebyshev. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 16 / 17
47 Inequalities Jensen, Holder, Cauchy-Schwartz, Lyapunov, Minkowski. Markov, Chebyshev. Kolmogorov. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 16 / 17
48 Inequalities Jensen, Holder, Cauchy-Schwartz, Lyapunov, Minkowski. Markov, Chebyshev. Kolmogorov. Exponential: Berstein, Bennett, Hoeffding, Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 16 / 17
49 Inequalities Jensen, Holder, Cauchy-Schwartz, Lyapunov, Minkowski. Markov, Chebyshev. Kolmogorov. Exponential: Berstein, Bennett, Hoeffding, Doob (for martingale). Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 16 / 17
50 Inequalities Jensen, Holder, Cauchy-Schwartz, Lyapunov, Minkowski. Markov, Chebyshev. Kolmogorov. Exponential: Berstein, Bennett, Hoeffding, Doob (for martingale). Khintchine, Marcinkiewicz-Zygmund, Burkholder-Gundy, Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 16 / 17
51 Martingales (Chapter 4). 1. Conditional expectation with respect to σ-algebra. 2. Definition of martingales, 3. Inequalities. 4. Optional sampling theorem. 5. Martingale convergence theorem. Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 17 / 17
1. Axioms of probability
1. Axioms of probability 1.1 Introduction Dice were believed to be invented in China during 7~10th AD. Luca Paccioli(1445-1514) Italian (studies of chances of events) Niccolo Tartaglia(1499-1557) Girolamo
STAT 7032 Probability. Wlodek Bryc
STAT 7032 Probability Wlodek Bryc Revised for Spring 2019 Printed: January 14, 2019 File: Grad-Prob-2019.TEX Department of Mathematical Sciences, University of Cincinnati, Cincinnati, OH 45221 E-mail address:
Probability Theory I: Syllabus and Exercise
Probability Theory I: Syllabus and Exercise Narn-Rueih Shieh **Copyright Reserved** This course is suitable for those who have taken Basic Probability; some knowledge of Real Analysis is recommended( will
Useful Probability Theorems
Useful Probability Theorems Shiu-Tang Li Finished: March 23, 2013 Last updated: November 2, 2013 1 Convergence in distribution Theorem 1.1. TFAE: (i) µ n µ, µ n, µ are probability measures. (ii) F n (x)
Theorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension. n=1
Chapter 2 Probability measures 1. Existence Theorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension to the generated σ-field Proof of Theorem 2.1. Let F 0 be
Stochastic Models (Lecture #4)
Stochastic Models (Lecture #4) Thomas Verdebout Université libre de Bruxelles (ULB) Today Today, our goal will be to discuss limits of sequences of rv, and to study famous limiting results. Convergence
Example continued. Math 425 Intro to Probability Lecture 37. Example continued. Example
continued : Coin tossing Math 425 Intro to Probability Lecture 37 Kenneth Harris kaharri@umich.edu Department of Mathematics University of Michigan April 8, 2009 Consider a Bernoulli trials process with
4 Expectation & the Lebesgue Theorems
STA 205: Probability & Measure Theory Robert L. Wolpert 4 Expectation & the Lebesgue Theorems Let X and {X n : n N} be random variables on a probability space (Ω,F,P). If X n (ω) X(ω) for each ω Ω, does
STAT 7032 Probability Spring Wlodek Bryc
STAT 7032 Probability Spring 2018 Wlodek Bryc Created: Friday, Jan 2, 2014 Revised for Spring 2018 Printed: January 9, 2018 File: Grad-Prob-2018.TEX Department of Mathematical Sciences, University of Cincinnati,
Index. Ballot problem, 250 Banach space, 111
d measurable function, 15 Abbott, 163 Abel, 114, 119 Absolutely continuous distribution function, 27, 270 set function, 203, 205 random variables, 27 Additive (set function), 18 a-, 19 see also Subadditive
Probability and Measure
Chapter 4 Probability and Measure 4.1 Introduction In this chapter we will examine probability theory from the measure theoretic perspective. The realisation that measure theory is the foundation of probability
Chapter 1 Axioms of Probability. Wen-Guey Tzeng Computer Science Department National Chiao University
Chapter 1 Axioms of Probability Wen-Guey Tzeng Computer Science Department National Chiao University Introduction Luca Paccioli(1445-1514), Studies of chances of events Niccolo Tartaglia(1499-1557) Girolamo
Part II Probability and Measure
Part II Probability and Measure Theorems Based on lectures by J. Miller Notes taken by Dexter Chua Michaelmas 2016 These notes are not endorsed by the lecturers, and I have modified them (often significantly)
Selected Exercises on Expectations and Some Probability Inequalities
Selected Exercises on Expectations and Some Probability Inequalities # If E(X 2 ) = and E X a > 0, then P( X λa) ( λ) 2 a 2 for 0 < λ
The main results about probability measures are the following two facts:
Chapter 2 Probability measures The main results about probability measures are the following two facts: Theorem 2.1 (extension). If P is a (continuous) probability measure on a field F 0 then it has a
Math 6810 (Probability) Fall Lecture notes
Math 6810 (Probability) Fall 2012 Lecture notes Pieter Allaart University of North Texas September 23, 2012 2 Text: Introduction to Stochastic Calculus with Applications, by Fima C. Klebaner (3rd edition),
Notes 18 : Optional Sampling Theorem
Notes 18 : Optional Sampling Theorem Math 733-734: Theory of Probability Lecturer: Sebastien Roch References: [Wil91, Chapter 14], [Dur10, Section 5.7]. Recall: DEF 18.1 (Uniform Integrability) A collection
Probability Theory. Richard F. Bass
Probability Theory Richard F. Bass ii c Copyright 2014 Richard F. Bass Contents 1 Basic notions 1 1.1 A few definitions from measure theory............. 1 1.2 Definitions............................. 2
4 Sums of Independent Random Variables
4 Sums of Independent Random Variables Standing Assumptions: Assume throughout this section that (,F,P) is a fixed probability space and that X 1, X 2, X 3,... are independent real-valued random variables
Chapter 1 Axioms of Probability. Wen-Guey Tzeng Computer Science Department National Chiao University
Chapter 1 Axioms of Probability Wen-Guey Tzeng Computer Science Department National Chiao University What is probability? A branch of mathematics that deals with calculating the likelihood of a given event
Product measure and Fubini s theorem
Chapter 7 Product measure and Fubini s theorem This is based on [Billingsley, Section 18]. 1. Product spaces Suppose (Ω 1, F 1 ) and (Ω 2, F 2 ) are two probability spaces. In a product space Ω = Ω 1 Ω
MAS113 Introduction to Probability and Statistics
MAS113 Introduction to Probability and Statistics School of Mathematics and Statistics, University of Sheffield 2018 19 Identically distributed Suppose we have n random variables X 1, X 2,..., X n. Identically
8 Laws of large numbers
8 Laws of large numbers 8.1 Introduction We first start with the idea of standardizing a random variable. Let X be a random variable with mean µ and variance σ 2. Then Z = (X µ)/σ will be a random variable
THE LINDEBERG-FELLER CENTRAL LIMIT THEOREM VIA ZERO BIAS TRANSFORMATION
THE LINDEBERG-FELLER CENTRAL LIMIT THEOREM VIA ZERO BIAS TRANSFORMATION JAINUL VAGHASIA Contents. Introduction. Notations 3. Background in Probability Theory 3.. Expectation and Variance 3.. Convergence
CHAPTER 3: LARGE SAMPLE THEORY
CHAPTER 3 LARGE SAMPLE THEORY 1 CHAPTER 3: LARGE SAMPLE THEORY CHAPTER 3 LARGE SAMPLE THEORY 2 Introduction CHAPTER 3 LARGE SAMPLE THEORY 3 Why large sample theory studying small sample property is usually
March 1, Florida State University. Concentration Inequalities: Martingale. Approach and Entropy Method. Lizhe Sun and Boning Yang.
Florida State University March 1, 2018 Framework 1. (Lizhe) Basic inequalities Chernoff bounding Review for STA 6448 2. (Lizhe) Discrete-time martingales inequalities via martingale approach 3. (Boning)
18.175: Lecture 3 Integration
18.175: Lecture 3 Scott Sheffield MIT Outline Outline Recall definitions Probability space is triple (Ω, F, P) where Ω is sample space, F is set of events (the σ-algebra) and P : F [0, 1] is the probability
P (A G) dp G P (A G)
First homework assignment. Due at 12:15 on 22 September 2016. Homework 1. We roll two dices. X is the result of one of them and Z the sum of the results. Find E [X Z. Homework 2. Let X be a r.v.. Assume
CS145: Probability & Computing
CS45: Probability & Computing Lecture 5: Concentration Inequalities, Law of Large Numbers, Central Limit Theorem Instructor: Eli Upfal Brown University Computer Science Figure credits: Bertsekas & Tsitsiklis,
Probability and Measure
Part II Year 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2018 84 Paper 4, Section II 26J Let (X, A) be a measurable space. Let T : X X be a measurable map, and µ a probability
Asymptotic Statistics-III. Changliang Zou
Asymptotic Statistics-III Changliang Zou The multivariate central limit theorem Theorem (Multivariate CLT for iid case) Let X i be iid random p-vectors with mean µ and and covariance matrix Σ. Then n (
Lecture 1: Review on Probability and Statistics
STAT 516: Stochastic Modeling of Scientific Data Autumn 2018 Instructor: Yen-Chi Chen Lecture 1: Review on Probability and Statistics These notes are partially based on those of Mathias Drton. 1.1 Motivating
Lecture Notes 5 Convergence and Limit Theorems. Convergence with Probability 1. Convergence in Mean Square. Convergence in Probability, WLLN
Lecture Notes 5 Convergence and Limit Theorems Motivation Convergence with Probability Convergence in Mean Square Convergence in Probability, WLLN Convergence in Distribution, CLT EE 278: Convergence and
STA205 Probability: Week 8 R. Wolpert
INFINITE COIN-TOSS AND THE LAWS OF LARGE NUMBERS The traditional interpretation of the probability of an event E is its asymptotic frequency: the limit as n of the fraction of n repeated, similar, and
Random Process Lecture 1. Fundamentals of Probability
Random Process Lecture 1. Fundamentals of Probability Husheng Li Min Kao Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville Spring, 2016 1/43 Outline 2/43 1 Syllabus
Martingale Theory for Finance
Martingale Theory for Finance Tusheng Zhang October 27, 2015 1 Introduction 2 Probability spaces and σ-fields 3 Integration with respect to a probability measure. 4 Conditional expectation. 5 Martingales.
18.175: Lecture 14 Infinite divisibility and so forth
18.175 Lecture 14 18.175: Lecture 14 Infinite divisibility and so forth Scott Sheffield MIT 18.175 Lecture 14 Outline Infinite divisibility Higher dimensional CFs and CLTs Random walks Stopping times Arcsin
Convergence of Random Variables
1 / 15 Convergence of Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay March 19, 2014 2 / 15 Motivation Theorem (Weak
LECTURE NOTES S. R. S. VARADHAN. Probability Theory. American Mathematical Society Courant Institute of Mathematical Sciences
C O U R A N T 7 S. R. S. VARADHAN LECTURE NOTES Probability Theory American Mathematical Society Courant Institute of Mathematical Sciences Selected Titles in This Series Volume 7 S. R. S. Varadhan Probability
Lecture 2. We now introduce some fundamental tools in martingale theory, which are useful in controlling the fluctuation of martingales.
Lecture 2 1 Martingales We now introduce some fundamental tools in martingale theory, which are useful in controlling the fluctuation of martingales. 1.1 Doob s inequality We have the following maximal
STOR 635 Notes (S13)
STOR 635 Notes (S13) Jimmy Jin UNC-Chapel Hill Last updated: 1/14/14 Contents 1 Measure theory and probability basics 2 1.1 Algebras and measure.......................... 2 1.2 Integration................................
Mathematics for Informatics 4a
Mathematics for Informatics 4a José Figueroa-O Farrill Lecture 1 18 January 2012 José Figueroa-O Farrill mi4a (Probability) Lecture 1 1 / 23 Introduction Topic: Probability and random processes Lectures:
Lecture 18: Central Limit Theorem. Lisa Yan August 6, 2018
Lecture 18: Central Limit Theorem Lisa Yan August 6, 2018 Announcements PS5 due today Pain poll PS6 out today Due next Monday 8/13 (1:30pm) (will not be accepted after Wed 8/15) Programming part: Java,
Brownian Motion and Conditional Probability
Math 561: Theory of Probability (Spring 2018) Week 10 Brownian Motion and Conditional Probability 10.1 Standard Brownian Motion (SBM) Brownian motion is a stochastic process with both practical and theoretical
Probability and Measure
Probability and Measure Robert L. Wolpert Institute of Statistics and Decision Sciences Duke University, Durham, NC, USA Convergence of Random Variables 1. Convergence Concepts 1.1. Convergence of Real
Limiting Distributions
We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the two fundamental results
On the convergence of sequences of random variables: A primer
BCAM May 2012 1 On the convergence of sequences of random variables: A primer Armand M. Makowski ECE & ISR/HyNet University of Maryland at College Park armand@isr.umd.edu BCAM May 2012 2 A sequence a :
Mod-φ convergence I: examples and probabilistic estimates
Mod-φ convergence I: examples and probabilistic estimates Valentin Féray (joint work with Pierre-Loïc Méliot and Ashkan Nikeghbali) Institut für Mathematik, Universität Zürich Summer school in Villa Volpi,
The Central Limit Theorem: More of the Story
The Central Limit Theorem: More of the Story Steven Janke November 2015 Steven Janke (Seminar) The Central Limit Theorem:More of the Story November 2015 1 / 33 Central Limit Theorem Theorem (Central Limit
1 Contract no: BAP098M-RG-15-Med. Probability theory is a branch of mathematics that has evolved from the investigation
1 Contract no: BAP098M-RG-15-Med Probability Theory J.M. Steele Department of Statistics, Wharton School, University of Pennsylvania Probability theory is a branch of mathematics that has evolved from
I. ANALYSIS; PROBABILITY
ma414l1.tex Lecture 1. 12.1.2012 I. NLYSIS; PROBBILITY 1. Lebesgue Measure and Integral We recall Lebesgue measure (M411 Probability and Measure) λ: defined on intervals (a, b] by λ((a, b]) := b a (so
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 9 10/2/2013. Conditional expectations, filtration and martingales
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 9 10/2/2013 Conditional expectations, filtration and martingales Content. 1. Conditional expectations 2. Martingales, sub-martingales
COMPSCI 240: Reasoning Under Uncertainty
COMPSCI 240: Reasoning Under Uncertainty Andrew Lan and Nic Herndon University of Massachusetts at Amherst Spring 2019 Lecture 20: Central limit theorem & The strong law of large numbers Markov and Chebyshev
Math 832 Fall University of Wisconsin at Madison. Instructor: David F. Anderson
Math 832 Fall 2013 University of Wisconsin at Madison Instructor: David F. Anderson Pertinent information Instructor: David Anderson Office: Van Vleck 617 email: anderson@math.wisc.edu Office hours: Mondays
Part IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
The Lindeberg central limit theorem
The Lindeberg central limit theorem Jordan Bell jordan.bell@gmail.com Department of Mathematics, University of Toronto May 29, 205 Convergence in distribution We denote by P d the collection of Borel probability
3 Integration and Expectation
3 Integration and Expectation 3.1 Construction of the Lebesgue Integral Let (, F, µ) be a measure space (not necessarily a probability space). Our objective will be to define the Lebesgue integral R fdµ
STAT 200C: High-dimensional Statistics
STAT 200C: High-dimensional Statistics Arash A. Amini May 30, 2018 1 / 59 Classical case: n d. Asymptotic assumption: d is fixed and n. Basic tools: LLN and CLT. High-dimensional setting: n d, e.g. n/d
On large deviations for combinatorial sums
arxiv:1901.0444v1 [math.pr] 14 Jan 019 On large deviations for combinatorial sums Andrei N. Frolov Dept. of Mathematics and Mechanics St. Petersburg State University St. Petersburg, Russia E-mail address:
Fundamental Tools - Probability Theory IV
Fundamental Tools - Probability Theory IV MSc Financial Mathematics The University of Warwick October 1, 2015 MSc Financial Mathematics Fundamental Tools - Probability Theory IV 1 / 14 Model-independent
Qualifying Exam in Probability and Statistics. https://www.soa.org/files/edu/edu-exam-p-sample-quest.pdf
Part : Sample Problems for the Elementary Section of Qualifying Exam in Probability and Statistics https://www.soa.org/files/edu/edu-exam-p-sample-quest.pdf Part 2: Sample Problems for the Advanced Section
Spring 2012 Math 541B Exam 1
Spring 2012 Math 541B Exam 1 1. A sample of size n is drawn without replacement from an urn containing N balls, m of which are red and N m are black; the balls are otherwise indistinguishable. Let X denote
Problem Sheet 1. You may assume that both F and F are σ-fields. (a) Show that F F is not a σ-field. (b) Let X : Ω R be defined by 1 if n = 1
Problem Sheet 1 1. Let Ω = {1, 2, 3}. Let F = {, {1}, {2, 3}, {1, 2, 3}}, F = {, {2}, {1, 3}, {1, 2, 3}}. You may assume that both F and F are σ-fields. (a) Show that F F is not a σ-field. (b) Let X :
Ergodic Theorems. Samy Tindel. Purdue University. Probability Theory 2 - MA 539. Taken from Probability: Theory and examples by R.
Ergodic Theorems Samy Tindel Purdue University Probability Theory 2 - MA 539 Taken from Probability: Theory and examples by R. Durrett Samy T. Ergodic theorems Probability Theory 1 / 92 Outline 1 Definitions
ECON 2530b: International Finance
ECON 2530b: International Finance Compiled by Vu T. Chau 1 Non-neutrality of Nominal Exchange Rate 1.1 Mussa s Puzzle(1986) RER can be defined as the relative price of a country s consumption basket in
Limiting Distributions
Limiting Distributions We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the
Notes 15 : UI Martingales
Notes 15 : UI Martingales Math 733 - Fall 2013 Lecturer: Sebastien Roch References: [Wil91, Chapter 13, 14], [Dur10, Section 5.5, 5.6, 5.7]. 1 Uniform Integrability We give a characterization of L 1 convergence.
Brownian motion. Samy Tindel. Purdue University. Probability Theory 2 - MA 539
Brownian motion Samy Tindel Purdue University Probability Theory 2 - MA 539 Mostly taken from Brownian Motion and Stochastic Calculus by I. Karatzas and S. Shreve Samy T. Brownian motion Probability Theory
Inference for Stochastic Processes
Inference for Stochastic Processes Robert L. Wolpert Revised: June 19, 005 Introduction A stochastic process is a family {X t } of real-valued random variables, all defined on the same probability space
Lecture 7: Chapter 7. Sums of Random Variables and Long-Term Averages
Lecture 7: Chapter 7. Sums of Random Variables and Long-Term Averages ELEC206 Probability and Random Processes, Fall 2014 Gil-Jin Jang gjang@knu.ac.kr School of EE, KNU page 1 / 15 Chapter 7. Sums of Random
Proving the central limit theorem
SOR3012: Stochastic Processes Proving the central limit theorem Gareth Tribello March 3, 2019 1 Purpose In the lectures and exercises we have learnt about the law of large numbers and the central limit
7 Convergence in R d and in Metric Spaces
STA 711: Probability & Measure Theory Robert L. Wolpert 7 Convergence in R d and in Metric Spaces A sequence of elements a n of R d converges to a limit a if and only if, for each ǫ > 0, the sequence a
1. Probability Measure and Integration Theory in a Nutshell
1. Probability Measure and Integration Theory in a Nutshell 1.1. Measurable Space and Measurable Functions Definition 1.1. A measurable space is a tuple (Ω, F) where Ω is a set and F a σ-algebra on Ω,
An Introduction to Probability Theory and Its Applications
An Introduction to Probability Theory and Its Applications WILLIAM FELLER (1906-1970) Eugene Higgins Professor of Mathematics Princeton University VOLUME II SECOND EDITION JOHN WILEY & SONS Contents I
HILBERT SPACES AND THE RADON-NIKODYM THEOREM. where the bar in the first equation denotes complex conjugation. In either case, for any x V define
HILBERT SPACES AND THE RADON-NIKODYM THEOREM STEVEN P. LALLEY 1. DEFINITIONS Definition 1. A real inner product space is a real vector space V together with a symmetric, bilinear, positive-definite mapping,
1 Probability space and random variables
1 Probability space and random variables As graduate level, we inevitably need to study probability based on measure theory. It obscures some intuitions in probability, but it also supplements our intuition,
RANDOM MATRICES: LAW OF THE DETERMINANT
RANDOM MATRICES: LAW OF THE DETERMINANT HOI H. NGUYEN AND VAN H. VU Abstract. Let A n be an n by n random matrix whose entries are independent real random variables with mean zero and variance one. We
Notes on stochastic processes
Notes on stochastic processes Paul Keeler March 20, 2018 This work is licensed under a CC BY-SA 3.0 license. Abstract A stochastic process is a type of mathematical object studied in mathematics, particularly
1 Presessional Probability
1 Presessional Probability Probability theory is essential for the development of mathematical models in finance, because of the randomness nature of price fluctuations in the markets. This presessional
Convergence at first and second order of some approximations of stochastic integrals
Convergence at first and second order of some approximations of stochastic integrals Bérard Bergery Blandine, Vallois Pierre IECN, Nancy-Université, CNRS, INRIA, Boulevard des Aiguillettes B.P. 239 F-5456
Midterm Examination. STA 205: Probability and Measure Theory. Wednesday, 2009 Mar 18, 2:50-4:05 pm
Midterm Examination STA 205: Probability and Measure Theory Wednesday, 2009 Mar 18, 2:50-4:05 pm This is a closed-book examination. You may use a single sheet of prepared notes, if you wish, but you may
Course Description - Master in of Mathematics Comprehensive exam& Thesis Tracks
Course Description - Master in of Mathematics Comprehensive exam& Thesis Tracks 1309701 Theory of ordinary differential equations Review of ODEs, existence and uniqueness of solutions for ODEs, existence
Define characteristic function. State its properties. State and prove inversion theorem.
ASSIGNMENT - 1, MAY 013. Paper I PROBABILITY AND DISTRIBUTION THEORY (DMSTT 01) 1. (a) Give the Kolmogorov definition of probability. State and prove Borel cantelli lemma. Define : (i) distribution function
IEOR 6711: Stochastic Models I Fall 2013, Professor Whitt Lecture Notes, Thursday, September 5 Modes of Convergence
IEOR 6711: Stochastic Models I Fall 2013, Professor Whitt Lecture Notes, Thursday, September 5 Modes of Convergence 1 Overview We started by stating the two principal laws of large numbers: the strong
CLASSICAL PROBABILITY MODES OF CONVERGENCE AND INEQUALITIES
CLASSICAL PROBABILITY 2008 2. MODES OF CONVERGENCE AND INEQUALITIES JOHN MORIARTY In many interesting and important situations, the object of interest is influenced by many random factors. If we can construct
Notation. General. Notation Description See. Sets, Functions, and Spaces. a b & a b The minimum and the maximum of a and b
Notation General Notation Description See a b & a b The minimum and the maximum of a and b a + & a f S u The non-negative part, a 0, and non-positive part, (a 0) of a R The restriction of the function
STA 711: Probability & Measure Theory Robert L. Wolpert
STA 711: Probability & Measure Theory Robert L. Wolpert 6 Independence 6.1 Independent Events A collection of events {A i } F in a probability space (Ω,F,P) is called independent if P[ i I A i ] = P[A
Creating Modern Probability, Jan von Plato, Cambridge University Press 1994
1 A Short History of Probability A gambler s dispute in 1654 led to the creation of a mathematical theory of probability by two famous French mathematicians, Blaise Pascal and Pierre de Fermat. Antoine
Martingale Theory and Applications
Martingale Theory and Applications Dr Nic Freeman June 4, 2015 Contents 1 Conditional Expectation 2 1.1 Probability spaces and σ-fields............................ 2 1.2 Random Variables...................................
17. Convergence of Random Variables
7. Convergence of Random Variables In elementary mathematics courses (such as Calculus) one speaks of the convergence of functions: f n : R R, then lim f n = f if lim f n (x) = f(x) for all x in R. This
EE514A Information Theory I Fall 2013
EE514A Information Theory I Fall 2013 K. Mohan, Prof. J. Bilmes University of Washington, Seattle Department of Electrical Engineering Fall Quarter, 2013 http://j.ee.washington.edu/~bilmes/classes/ee514a_fall_2013/
Lecture 3: Expected Value. These integrals are taken over all of Ω. If we wish to integrate over a measurable subset A Ω, we will write
Lecture 3: Expected Value 1.) Definitions. If X 0 is a random variable on (Ω, F, P), then we define its expected value to be EX = XdP. Notice that this quantity may be. For general X, we say that EX exists
Bahadur representations for bootstrap quantiles 1
Bahadur representations for bootstrap quantiles 1 Yijun Zuo Department of Statistics and Probability, Michigan State University East Lansing, MI 48824, USA zuo@msu.edu 1 Research partially supported by
Exercise Exercise Homework #6 Solutions Thursday 6 April 2006
Unless otherwise stated, for the remainder of the solutions, define F m = σy 0,..., Y m We will show EY m = EY 0 using induction. m = 0 is obviously true. For base case m = : EY = EEY Y 0 = EY 0. Now assume
Convergence Concepts of Random Variables and Functions
Convergence Concepts of Random Variables and Functions c 2002 2007, Professor Seppo Pynnonen, Department of Mathematics and Statistics, University of Vaasa Version: January 5, 2007 Convergence Modes Convergence
Section 27. The Central Limit Theorem. Po-Ning Chen, Professor. Institute of Communications Engineering. National Chiao Tung University
Section 27 The Central Limit Theorem Po-Ning Chen, Professor Institute of Communications Engineering National Chiao Tung University Hsin Chu, Taiwan 3000, R.O.C. Identically distributed summands 27- Central
Essentials on the Analysis of Randomized Algorithms
Essentials on the Analysis of Randomized Algorithms Dimitris Diochnos Feb 0, 2009 Abstract These notes were written with Monte Carlo algorithms primarily in mind. Topics covered are basic (discrete) random
Tom Salisbury
MATH 2030 3.00MW Elementary Probability Course Notes Part V: Independence of Random Variables, Law of Large Numbers, Central Limit Theorem, Poisson distribution Geometric & Exponential distributions Tom
1 Sequences of events and their limits
O.H. Probability II (MATH 2647 M15 1 Sequences of events and their limits 1.1 Monotone sequences of events Sequences of events arise naturally when a probabilistic experiment is repeated many times. For
Some basic elements of Probability Theory
Chapter I Some basic elements of Probability Theory 1 Terminology (and elementary observations Probability theory and the material covered in a basic Real Variables course have much in common. However