Lecture 16: Sample quantiles and their asymptotic properties
|
|
- Vivian Miles
- 5 years ago
- Views:
Transcription
1 Lecture 16: Sample quantiles and their asymptotic properties Estimation of quantiles (percentiles Suppose that X 1,...,X n are i.i.d. random variables from an unknown nonparametric F For p (0,1, G 1 (p = inf{x : G(x p} is the pth quantile for any c.d.f. G on R. Quantiles of F are often the parameters of interest. θ p = F 1 (p = pth quantile of F F n = empirical c.d.f. based on X 1,...,X n θ p = Fn 1 (p = the pth sample quantile. θ p = c np X (mp + (1 c np X (mp +1, where X (j is the jth order statistic, m p is the integer part of np, c np = 1 if np is an integer, and c np = 0 if np is not an integer. Thus, θ p is a linear function of order statistics. UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
2 F(θ p = lim x θp,x<θ p F(x F(θ p = lim x θp,x>θ p F(x F(θ p p F(θ p F is not flat in a neighborhood of θ p if and only if p < F(θ p + ε for any ε > 0. Theorem 5.9 Let X 1,...,X n be i.i.d. random variables from a c.d.f. F satisfying p < F(θ p + ε for any ε > 0. Then, for every ε > 0 and n = 1,2,..., P ( θ p θ p > ε 2Ce 2nδ 2 ε, where δ ε is the smaller of F(θ p + ε p and p F(θ p ε and C is the same constant in Lemma 5.1(i. Remarks Theorem 5.9 implies that θ p is strongly consistent for θ p (exercise Theorem 5.9 implies that θ p is n-consistent for θ p if F (θ p and F (θ p + (the left and right derivatives of F at θ p exist (exercise. UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
3 F(θ p = lim x θp,x<θ p F(x F(θ p = lim x θp,x>θ p F(x F(θ p p F(θ p F is not flat in a neighborhood of θ p if and only if p < F(θ p + ε for any ε > 0. Theorem 5.9 Let X 1,...,X n be i.i.d. random variables from a c.d.f. F satisfying p < F(θ p + ε for any ε > 0. Then, for every ε > 0 and n = 1,2,..., P ( θ p θ p > ε 2Ce 2nδ 2 ε, where δ ε is the smaller of F(θ p + ε p and p F(θ p ε and C is the same constant in Lemma 5.1(i. Remarks Theorem 5.9 implies that θ p is strongly consistent for θ p (exercise Theorem 5.9 implies that θ p is n-consistent for θ p if F (θ p and F (θ p + (the left and right derivatives of F at θ p exist (exercise. UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
4 F(θ p = lim x θp,x<θ p F(x F(θ p = lim x θp,x>θ p F(x F(θ p p F(θ p F is not flat in a neighborhood of θ p if and only if p < F(θ p + ε for any ε > 0. Theorem 5.9 Let X 1,...,X n be i.i.d. random variables from a c.d.f. F satisfying p < F(θ p + ε for any ε > 0. Then, for every ε > 0 and n = 1,2,..., P ( θ p θ p > ε 2Ce 2nδ 2 ε, where δ ε is the smaller of F(θ p + ε p and p F(θ p ε and C is the same constant in Lemma 5.1(i. Remarks Theorem 5.9 implies that θ p is strongly consistent for θ p (exercise Theorem 5.9 implies that θ p is n-consistent for θ p if F (θ p and F (θ p + (the left and right derivatives of F at θ p exist (exercise. UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
5 Proof of Theorem 5.9 Let ε > 0 be fixed. Note that, for any c.d.f. G on R, (exercise. Hence G(x t if and only if x G 1 (t P ( θp > θ p + ε = P ( p > F n (θ p + ε = P ( F(θ p + ε F n (θ p + ε > F(θ p + ε p P ( ρ (F n,f > δ ε Ce 2nδ 2 ε, where the last inequality follows from DKW s inequality (Lemma 5.1(i. Similarly, P ( θp < θ p ε Ce 2nδ 2 ε. This completes the proof. UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
6 The distribution of a sample quantile The exact distribution of θ p can be obtained as follows. Since nf n (t has the binomial distribution Bi(F(t,n for any t R, P ( θp t = P ( F n (t p ( n = i n i=l p [F(t] i [1 F(t] n i, where l p = np if np is an integer and l p = 1+ the integer part of np if np is not an integer. If F has a Lebesgue p.d.f. f, then θ p has the Lebesgue p.d.f. ( n 1 ϕ n (t = n [F(t] lp 1 [1 F(t] n l p f (t. l p 1 This can be shown by differentiating P ( F n (t p term by term, which leads to UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
7 ϕ n (t = = n i=l p ( n ( n i n i=l p l p +n i[f(t] i 1 [1 F(t] n i f (t ( n i (n i[f(t] i [1 F(t] n i 1 f (t l p [F(t] l p 1 [1 F(t] n l p f (t n i=l p +1 ( n 1 i 1 [F(t] i 1 [1 F(t] n i f (t n 1( n 1 n [F(t] i [1 F(t] n i 1 f (t i=l p i ( n 1 = n [F(t] lp 1 [1 F(t] n l p f (t. l p 1 The following result provides an asymptotic distribution for n( θ p θ p. UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
8 Theorem 5.10 Let X 1,...,X n be i.i.d. random variables from F. (i If F(θ p = p, then P( n( θ p θ p 0 Φ(0 = 1 2, where Φ is the c.d.f. of the standard normal. (ii If F is continuous at θ p and there exists F (θ p > 0, then P ( n( θ p θ p t Φ(t/σ F, t < 0, where σ F = p(1 p/f (θ p. (iii If F is continuous at θ p and there exists F (θ p + > 0, then P ( n( θ p θ p t Φ(t/σ + F, t > 0, where σ + F = p(1 p/f (θ p +. (iv If F (θ p exists and is positive, then n( θp θ p d N(0,σ 2 F, where σ F = p(1 p/f (θ p. UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
9 Proof The proof of (i is left as an exercise. Part (iv is a direct consequence of (i-(iii and the proofs of (ii and (iii are similar. Thus, we only give a proof for (iii. Let t > 0, p nt = F(θ p + tσ + F n 1/2, c nt = n(p nt p/ p nt (1 p nt, and Z nt = [B n (p nt np nt ]/ np nt (1 p nt, where B n (q denotes a random variable having the binomial distribution Bi(q, n. Then P ( θp θ p + tσ + F n 1/2 = P ( p F n (θ p + tσ + F n 1/2 = P ( Z nt c nt. Under the assumed conditions on F, p nt p and c nt t. Hence, the result follows from P ( Z nt < c nt Φ( cnt 0. But this follows from the CLT (Example 1.33 and Pólya s theorem (Proposition UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
10 If F (θ p and F (θ p + exist and are positive, but F (θ p F (θ p +, then the asymptotic distribution of n( θ p θ p has the c.d.f. Φ(t/σ F I (,0(t + Φ(t/σ + F I [0, (t, a mixture of two normal distributions. An example of such a case when p = 1/2 is F(x = xi [0, 1 2 (x + (2x 1 2 I [ 1 2, 3 4 (x + I [ 3 4, (x. Bahadur s representation When F (θ p = F (θ p + = F (θ p > 0, Theorem 5.9 shows that the asymptotic distribution of n( θ p θ p is the same as that of n[fn (θ p F(θ p ]/F (θ p. The next result reveals a stronger relationship between sample quantiles and the empirical c.d.f. Theorem 5.11 (Bahadur s representation Let X 1,...,X n be i.i.d. random variables from F. If F (θ p exists and is positive, then n( θp θ p = n[f n (θ p F(θ p ]/F (θ p + o p (1. UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
11 If F (θ p and F (θ p + exist and are positive, but F (θ p F (θ p +, then the asymptotic distribution of n( θ p θ p has the c.d.f. Φ(t/σ F I (,0(t + Φ(t/σ + F I [0, (t, a mixture of two normal distributions. An example of such a case when p = 1/2 is F(x = xi [0, 1 2 (x + (2x 1 2 I [ 1 2, 3 4 (x + I [ 3 4, (x. Bahadur s representation When F (θ p = F (θ p + = F (θ p > 0, Theorem 5.9 shows that the asymptotic distribution of n( θ p θ p is the same as that of n[fn (θ p F(θ p ]/F (θ p. The next result reveals a stronger relationship between sample quantiles and the empirical c.d.f. Theorem 5.11 (Bahadur s representation Let X 1,...,X n be i.i.d. random variables from F. If F (θ p exists and is positive, then n( θp θ p = n[f n (θ p F(θ p ]/F (θ p + o p (1. UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
12 If F (θ p and F (θ p + exist and are positive, but F (θ p F (θ p +, then the asymptotic distribution of n( θ p θ p has the c.d.f. Φ(t/σ F I (,0(t + Φ(t/σ + F I [0, (t, a mixture of two normal distributions. An example of such a case when p = 1/2 is F(x = xi [0, 1 2 (x + (2x 1 2 I [ 1 2, 3 4 (x + I [ 3 4, (x. Bahadur s representation When F (θ p = F (θ p + = F (θ p > 0, Theorem 5.9 shows that the asymptotic distribution of n( θ p θ p is the same as that of n[fn (θ p F(θ p ]/F (θ p. The next result reveals a stronger relationship between sample quantiles and the empirical c.d.f. Theorem 5.11 (Bahadur s representation Let X 1,...,X n be i.i.d. random variables from F. If F (θ p exists and is positive, then n( θp θ p = n[f n (θ p F(θ p ]/F (θ p + o p (1. UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
13 Proof Let t R, θ nt = θ p + tn 1/2, Z n (t = n[f(θ nt F n (θ nt ]/F (θ p, and U n (t = n[f(θ nt F n ( θ p ]/F (θ p. It can be shown (exercise that Since F n ( θ p n 1, Z n (t Z n (0 = o p (1. U n (t = n[f(θ nt p + p F n ( θ p ]/F (θ p = n[f(θ nt p]/f (θ p + O(n 1/2 t. Let ξ n = n( θ p θ p. Then, for any t R and ε > 0, P ( ξ n t,z n (0 t + ε = P ( Z n (t U n (t,z n (0 t + ε P ( Z n (t Z n (0 ε/2 +P ( U n (t t ε/2 0 UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
14 Proof (continued Similarly, P ( ξ n t + ε,z n (0 t 0. It follows from the result in Exercise 128 of 1.6 that which is what we need to prove. Corollary 5.1 ξ n Z n (0 = o p (1, Let X 1,...,X n be i.i.d. random variables from F having positive derivatives at θ pj, where 0 < p 1 < < p m < 1 are fixed constants. Then n[( θp1,..., θ pm (θ p1,...,θ pm ] d N m (0,D, where D is the m m symmetric matrix whose (i,jth element is p i (1 p j /[F (θ pi F (θ pj ], i j. UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
15 Proof (continued Similarly, P ( ξ n t + ε,z n (0 t 0. It follows from the result in Exercise 128 of 1.6 that which is what we need to prove. Corollary 5.1 ξ n Z n (0 = o p (1, Let X 1,...,X n be i.i.d. random variables from F having positive derivatives at θ pj, where 0 < p 1 < < p m < 1 are fixed constants. Then n[( θp1,..., θ pm (θ p1,...,θ pm ] d N m (0,D, where D is the m m symmetric matrix whose (i,jth element is p i (1 p j /[F (θ pi F (θ pj ], i j. UW-Madison (Statistics Stat 710, Lecture 16 Jan / 10
Lecture 21: Convergence of transformations and generating a random variable
Lecture 21: Convergence of transformations and generating a random variable If Z n converges to Z in some sense, we often need to check whether h(z n ) converges to h(z ) in the same sense. Continuous
More informationChapter 7. Confidence Sets Lecture 30: Pivotal quantities and confidence sets
Chapter 7. Confidence Sets Lecture 30: Pivotal quantities and confidence sets Confidence sets X: a sample from a population P P. θ = θ(p): a functional from P to Θ R k for a fixed integer k. C(X): a confidence
More informationStat 710: Mathematical Statistics Lecture 31
Stat 710: Mathematical Statistics Lecture 31 Jun Shao Department of Statistics University of Wisconsin Madison, WI 53706, USA Jun Shao (UW-Madison) Stat 710, Lecture 31 April 13, 2009 1 / 13 Lecture 31:
More informationLecture 32: Asymptotic confidence sets and likelihoods
Lecture 32: Asymptotic confidence sets and likelihoods Asymptotic criterion In some problems, especially in nonparametric problems, it is difficult to find a reasonable confidence set with a given confidence
More informationChapter 2: Fundamentals of Statistics Lecture 15: Models and statistics
Chapter 2: Fundamentals of Statistics Lecture 15: Models and statistics Data from one or a series of random experiments are collected. Planning experiments and collecting data (not discussed here). Analysis:
More informationLecture 18: L-estimators and trimmed sample mean
Lecture 18: L-estimators and trimmed sample mean L-functional and L-estimator For a function J(t) on [0,1], define the L-functional as T (G) = xj(g(x))dg(x), G F. If X 1,...,X n are i.i.d. from F and T
More informationChapter 3: Unbiased Estimation Lecture 22: UMVUE and the method of using a sufficient and complete statistic
Chapter 3: Unbiased Estimation Lecture 22: UMVUE and the method of using a sufficient and complete statistic Unbiased estimation Unbiased or asymptotically unbiased estimation plays an important role in
More informationChapter 6. Hypothesis Tests Lecture 20: UMP tests and Neyman-Pearson lemma
Chapter 6. Hypothesis Tests Lecture 20: UMP tests and Neyman-Pearson lemma Theory of testing hypotheses X: a sample from a population P in P, a family of populations. Based on the observed X, we test a
More informationLecture 26: Likelihood ratio tests
Lecture 26: Likelihood ratio tests Likelihood ratio When both H 0 and H 1 are simple (i.e., Θ 0 = {θ 0 } and Θ 1 = {θ 1 }), Theorem 6.1 applies and a UMP test rejects H 0 when f θ1 (X) f θ0 (X) > c 0 for
More information7 Influence Functions
7 Influence Functions The influence function is used to approximate the standard error of a plug-in estimator. The formal definition is as follows. 7.1 Definition. The Gâteaux derivative of T at F in the
More informationLecture 17: Likelihood ratio and asymptotic tests
Lecture 17: Likelihood ratio and asymptotic tests Likelihood ratio When both H 0 and H 1 are simple (i.e., Θ 0 = {θ 0 } and Θ 1 = {θ 1 }), Theorem 6.1 applies and a UMP test rejects H 0 when f θ1 (X) f
More informationLimiting Distributions
Limiting Distributions We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the
More informationLimiting Distributions
We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the two fundamental results
More informationSTAT 512 sp 2018 Summary Sheet
STAT 5 sp 08 Summary Sheet Karl B. Gregory Spring 08. Transformations of a random variable Let X be a rv with support X and let g be a function mapping X to Y with inverse mapping g (A = {x X : g(x A}
More informationLecture 23: UMPU tests in exponential families
Lecture 23: UMPU tests in exponential families Continuity of the power function For a given test T, the power function β T (P) is said to be continuous in θ if and only if for any {θ j : j = 0,1,2,...}
More informationRobust Parameter Estimation in the Weibull and the Birnbaum-Saunders Distribution
Clemson University TigerPrints All Theses Theses 8-2012 Robust Parameter Estimation in the Weibull and the Birnbaum-Saunders Distribution Jing Zhao Clemson University, jzhao2@clemson.edu Follow this and
More informationORDER STATISTICS, QUANTILES, AND SAMPLE QUANTILES
ORDER STATISTICS, QUANTILES, AND SAMPLE QUANTILES 1. Order statistics Let X 1,...,X n be n real-valued observations. One can always arrangetheminordertogettheorder statisticsx (1) X (2) X (n). SinceX (k)
More informationBahadur representations for bootstrap quantiles 1
Bahadur representations for bootstrap quantiles 1 Yijun Zuo Department of Statistics and Probability, Michigan State University East Lansing, MI 48824, USA zuo@msu.edu 1 Research partially supported by
More informationStat 710: Mathematical Statistics Lecture 40
Stat 710: Mathematical Statistics Lecture 40 Jun Shao Department of Statistics University of Wisconsin Madison, WI 53706, USA Jun Shao (UW-Madison) Stat 710, Lecture 40 May 6, 2009 1 / 11 Lecture 40: Simultaneous
More informationLecture 5: Moment generating functions
Lecture 5: Moment generating functions Definition 2.3.6. The moment generating function (mgf) of a random variable X is { x e tx f M X (t) = E(e tx X (x) if X has a pmf ) = etx f X (x)dx if X has a pdf
More informationLecture 17: Minimal sufficiency
Lecture 17: Minimal sufficiency Maximal reduction without loss of information There are many sufficient statistics for a given family P. In fact, X (the whole data set) is sufficient. If T is a sufficient
More informationLecture 3: Random variables, distributions, and transformations
Lecture 3: Random variables, distributions, and transformations Definition 1.4.1. A random variable X is a function from S into a subset of R such that for any Borel set B R {X B} = {ω S : X(ω) B} is an
More informationLecture 28: Asymptotic confidence sets
Lecture 28: Asymptotic confidence sets 1 α asymptotic confidence sets Similar to testing hypotheses, in many situations it is difficult to find a confidence set with a given confidence coefficient or level
More informationST5215: Advanced Statistical Theory
Department of Statistics & Applied Probability Wednesday, October 19, 2011 Lecture 17: UMVUE and the first method of derivation Estimable parameters Let ϑ be a parameter in the family P. If there exists
More informationAsymptotic Statistics-VI. Changliang Zou
Asymptotic Statistics-VI Changliang Zou Kolmogorov-Smirnov distance Example (Kolmogorov-Smirnov confidence intervals) We know given α (0, 1), there is a well-defined d = d α,n such that, for any continuous
More informationOn probabilities of large and moderate deviations for L-statistics: a survey of some recent developments
UDC 519.2 On probabilities of large and moderate deviations for L-statistics: a survey of some recent developments N. V. Gribkova Department of Probability Theory and Mathematical Statistics, St.-Petersburg
More informationH 2 : otherwise. that is simply the proportion of the sample points below level x. For any fixed point x the law of large numbers gives that
Lecture 28 28.1 Kolmogorov-Smirnov test. Suppose that we have an i.i.d. sample X 1,..., X n with some unknown distribution and we would like to test the hypothesis that is equal to a particular distribution
More information3.8 Functions of a Random Variable
STAT 421 Lecture Notes 76 3.8 Functions of a Random Variable This section introduces a new and important topic: determining the distribution of a function of a random variable. We suppose that there is
More informationMATH4427 Notebook 4 Fall Semester 2017/2018
MATH4427 Notebook 4 Fall Semester 2017/2018 prepared by Professor Jenny Baglivo c Copyright 2009-2018 by Jenny A. Baglivo. All Rights Reserved. 4 MATH4427 Notebook 4 3 4.1 K th Order Statistics and Their
More informationSTAT 6385 Survey of Nonparametric Statistics. Order Statistics, EDF and Censoring
STAT 6385 Survey of Nonparametric Statistics Order Statistics, EDF and Censoring Quantile Function A quantile (or a percentile) of a distribution is that value of X such that a specific percentage of the
More informationChapter 5. Measurable Functions
Chapter 5. Measurable Functions 1. Measurable Functions Let X be a nonempty set, and let S be a σ-algebra of subsets of X. Then (X, S) is a measurable space. A subset E of X is said to be measurable if
More informationCS145: Probability & Computing
CS45: Probability & Computing Lecture 5: Concentration Inequalities, Law of Large Numbers, Central Limit Theorem Instructor: Eli Upfal Brown University Computer Science Figure credits: Bertsekas & Tsitsiklis,
More informationConvergence of Multivariate Quantile Surfaces
Convergence of Multivariate Quantile Surfaces Adil Ahidar Institut de Mathématiques de Toulouse - CERFACS August 30, 2013 Adil Ahidar (Institut de Mathématiques de Toulouse Convergence - CERFACS) of Multivariate
More information1 Definition of the Riemann integral
MAT337H1, Introduction to Real Analysis: notes on Riemann integration 1 Definition of the Riemann integral Definition 1.1. Let [a, b] R be a closed interval. A partition P of [a, b] is a finite set of
More informationLecture 15: Density estimation
Lecture 15: Desity estimatio Why do we estimate a desity? Suppose that X 1,...,X are i.i.d. radom variables from F ad that F is ukow but has a Lebesgue p.d.f. f. Estimatio of F ca be doe by estimatig f.
More informationis a Borel subset of S Θ for each c R (Bertsekas and Shreve, 1978, Proposition 7.36) This always holds in practical applications.
Stat 811 Lecture Notes The Wald Consistency Theorem Charles J. Geyer April 9, 01 1 Analyticity Assumptions Let { f θ : θ Θ } be a family of subprobability densities 1 with respect to a measure µ on a measurable
More informationAsymptotic statistics using the Functional Delta Method
Quantiles, Order Statistics and L-Statsitics TU Kaiserslautern 15. Februar 2015 Motivation Functional The delta method introduced in chapter 3 is an useful technique to turn the weak convergence of random
More informationTheoretical Statistics. Lecture 19.
Theoretical Statistics. Lecture 19. Peter Bartlett 1. Functional delta method. [vdv20] 2. Differentiability in normed spaces: Hadamard derivatives. [vdv20] 3. Quantile estimates. [vdv21] 1 Recall: Delta
More informationSmooth nonparametric estimation of a quantile function under right censoring using beta kernels
Smooth nonparametric estimation of a quantile function under right censoring using beta kernels Chanseok Park 1 Department of Mathematical Sciences, Clemson University, Clemson, SC 29634 Short Title: Smooth
More informationSTAT 461/561- Assignments, Year 2015
STAT 461/561- Assignments, Year 2015 This is the second set of assignment problems. When you hand in any problem, include the problem itself and its number. pdf are welcome. If so, use large fonts and
More informationChapter 9: Interval Estimation and Confidence Sets Lecture 16: Confidence sets and credible sets
Chapter 9: Interval Estimation and Confidence Sets Lecture 16: Confidence sets and credible sets Confidence sets We consider a sample X from a population indexed by θ Θ R k. We are interested in ϑ, a vector-valued
More informationStat 710: Mathematical Statistics Lecture 27
Stat 710: Mathematical Statistics Lecture 27 Jun Shao Department of Statistics University of Wisconsin Madison, WI 53706, USA Jun Shao (UW-Madison) Stat 710, Lecture 27 April 3, 2009 1 / 10 Lecture 27:
More informationMathematics for Economists
Mathematics for Economists Victor Filipe Sao Paulo School of Economics FGV Metric Spaces: Basic Definitions Victor Filipe (EESP/FGV) Mathematics for Economists Jan.-Feb. 2017 1 / 34 Definitions and Examples
More informationAPPLICATIONS OF DIFFERENTIABILITY IN R n.
APPLICATIONS OF DIFFERENTIABILITY IN R n. MATANIA BEN-ARTZI April 2015 Functions here are defined on a subset T R n and take values in R m, where m can be smaller, equal or greater than n. The (open) ball
More informationSpring 2012 Math 541B Exam 1
Spring 2012 Math 541B Exam 1 1. A sample of size n is drawn without replacement from an urn containing N balls, m of which are red and N m are black; the balls are otherwise indistinguishable. Let X denote
More informationLecture 14: Multivariate mgf s and chf s
Lecture 14: Multivariate mgf s and chf s Multivariate mgf and chf For an n-dimensional random vector X, its mgf is defined as M X (t) = E(e t X ), t R n and its chf is defined as φ X (t) = E(e ıt X ),
More informationMathematical Statistics 1 Math A 6330
Mathematical Statistics 1 Math A 6330 Chapter 2 Transformations and Expectations Mohamed I. Riffi Department of Mathematics Islamic University of Gaza September 14, 2015 Outline 1 Distributions of Functions
More informationChapter 6: Large Random Samples Sections
Chapter 6: Large Random Samples Sections 6.1: Introduction 6.2: The Law of Large Numbers Skip p. 356-358 Skip p. 366-368 Skip 6.4: The correction for continuity Remember: The Midterm is October 25th in
More informationStatistical Data Analysis
DS-GA 0 Lecture notes 8 Fall 016 1 Descriptive statistics Statistical Data Analysis In this section we consider the problem of analyzing a set of data. We describe several techniques for visualizing the
More informationSTAT 730 Chapter 4: Estimation
STAT 730 Chapter 4: Estimation Timothy Hanson Department of Statistics, University of South Carolina Stat 730: Multivariate Analysis 1 / 23 The likelihood We have iid data, at least initially. Each datum
More informationProbability Models. 4. What is the definition of the expectation of a discrete random variable?
1 Probability Models The list of questions below is provided in order to help you to prepare for the test and exam. It reflects only the theoretical part of the course. You should expect the questions
More informationL p Functions. Given a measure space (X, µ) and a real number p [1, ), recall that the L p -norm of a measurable function f : X R is defined by
L p Functions Given a measure space (, µ) and a real number p [, ), recall that the L p -norm of a measurable function f : R is defined by f p = ( ) /p f p dµ Note that the L p -norm of a function f may
More information5 Introduction to the Theory of Order Statistics and Rank Statistics
5 Introduction to the Theory of Order Statistics and Rank Statistics This section will contain a summary of important definitions and theorems that will be useful for understanding the theory of order
More informationExercises in Extreme value theory
Exercises in Extreme value theory 2016 spring semester 1. Show that L(t) = logt is a slowly varying function but t ǫ is not if ǫ 0. 2. If the random variable X has distribution F with finite variance,
More information40.530: Statistics. Professor Chen Zehua. Singapore University of Design and Technology
Singapore University of Design and Technology Lecture 9: Hypothesis testing, uniformly most powerful tests. The Neyman-Pearson framework Let P be the family of distributions of concern. The Neyman-Pearson
More informationMonte Carlo Studies. The response in a Monte Carlo study is a random variable.
Monte Carlo Studies The response in a Monte Carlo study is a random variable. The response in a Monte Carlo study has a variance that comes from the variance of the stochastic elements in the data-generating
More informationSTAT Sample Problem: General Asymptotic Results
STAT331 1-Sample Problem: General Asymptotic Results In this unit we will consider the 1-sample problem and prove the consistency and asymptotic normality of the Nelson-Aalen estimator of the cumulative
More informationSTATISTICAL INFERENCE UNDER ORDER RESTRICTIONS LIMIT THEORY FOR THE GRENANDER ESTIMATOR UNDER ALTERNATIVE HYPOTHESES
STATISTICAL INFERENCE UNDER ORDER RESTRICTIONS LIMIT THEORY FOR THE GRENANDER ESTIMATOR UNDER ALTERNATIVE HYPOTHESES By Jon A. Wellner University of Washington 1. Limit theory for the Grenander estimator.
More informationFundamentals of Statistics
Chapter 2 Fundamentals of Statistics This chapter discusses some fundamental concepts of mathematical statistics. These concepts are essential for the material in later chapters. 2.1 Populations, Samples,
More informationAsymptotic Statistics-III. Changliang Zou
Asymptotic Statistics-III Changliang Zou The multivariate central limit theorem Theorem (Multivariate CLT for iid case) Let X i be iid random p-vectors with mean µ and and covariance matrix Σ. Then n (
More information1 General problem. 2 Terminalogy. Estimation. Estimate θ. (Pick a plausible distribution from family. ) Or estimate τ = τ(θ).
Estimation February 3, 206 Debdeep Pati General problem Model: {P θ : θ Θ}. Observe X P θ, θ Θ unknown. Estimate θ. (Pick a plausible distribution from family. ) Or estimate τ = τ(θ). Examples: θ = (µ,
More informationWeak convergence and Brownian Motion. (telegram style notes) P.J.C. Spreij
Weak convergence and Brownian Motion (telegram style notes) P.J.C. Spreij this version: December 8, 2006 1 The space C[0, ) In this section we summarize some facts concerning the space C[0, ) of real
More informationExercises Measure Theoretic Probability
Exercises Measure Theoretic Probability 2002-2003 Week 1 1. Prove the folloing statements. (a) The intersection of an arbitrary family of d-systems is again a d- system. (b) The intersection of an arbitrary
More informationTHEOREMS, ETC., FOR MATH 515
THEOREMS, ETC., FOR MATH 515 Proposition 1 (=comment on page 17). If A is an algebra, then any finite union or finite intersection of sets in A is also in A. Proposition 2 (=Proposition 1.1). For every
More informationA Kullback-Leibler Divergence for Bayesian Model Comparison with Applications to Diabetes Studies. Chen-Pin Wang, UTHSCSA Malay Ghosh, U.
A Kullback-Leibler Divergence for Bayesian Model Comparison with Applications to Diabetes Studies Chen-Pin Wang, UTHSCSA Malay Ghosh, U. Florida Lehmann Symposium, May 9, 2011 1 Background KLD: the expected
More informationON THE UNIFORM ASYMPTOTIC VALIDITY OF SUBSAMPLING AND THE BOOTSTRAP. Joseph P. Romano Azeem M. Shaikh
ON THE UNIFORM ASYMPTOTIC VALIDITY OF SUBSAMPLING AND THE BOOTSTRAP By Joseph P. Romano Azeem M. Shaikh Technical Report No. 2010-03 April 2010 Department of Statistics STANFORD UNIVERSITY Stanford, California
More informationMATH 409 Advanced Calculus I Lecture 12: Uniform continuity. Exponential functions.
MATH 409 Advanced Calculus I Lecture 12: Uniform continuity. Exponential functions. Uniform continuity Definition. A function f : E R defined on a set E R is called uniformly continuous on E if for every
More informationlarge number of i.i.d. observations from P. For concreteness, suppose
1 Subsampling Suppose X i, i = 1,..., n is an i.i.d. sequence of random variables with distribution P. Let θ(p ) be some real-valued parameter of interest, and let ˆθ n = ˆθ n (X 1,..., X n ) be some estimate
More informationλ(x + 1)f g (x) > θ 0
Stat 8111 Final Exam December 16 Eleven students took the exam, the scores were 92, 78, 4 in the 5 s, 1 in the 4 s, 1 in the 3 s and 3 in the 2 s. 1. i) Let X 1, X 2,..., X n be iid each Bernoulli(θ) where
More informationThe main results about probability measures are the following two facts:
Chapter 2 Probability measures The main results about probability measures are the following two facts: Theorem 2.1 (extension). If P is a (continuous) probability measure on a field F 0 then it has a
More informationChapter 2: Resampling Maarten Jansen
Chapter 2: Resampling Maarten Jansen Randomization tests Randomized experiment random assignment of sample subjects to groups Example: medical experiment with control group n 1 subjects for true medicine,
More informationSTAT 830 Hypothesis Testing
STAT 830 Hypothesis Testing Richard Lockhart Simon Fraser University STAT 830 Fall 2018 Richard Lockhart (Simon Fraser University) STAT 830 Hypothesis Testing STAT 830 Fall 2018 1 / 30 Purposes of These
More informationUses of Asymptotic Distributions: In order to get distribution theory, we need to norm the random variable; we usually look at n 1=2 ( X n ).
1 Economics 620, Lecture 8a: Asymptotics II Uses of Asymptotic Distributions: Suppose X n! 0 in probability. (What can be said about the distribution of X n?) In order to get distribution theory, we need
More informationLecture 13: p-values and union intersection tests
Lecture 13: p-values and union intersection tests p-values After a hypothesis test is done, one method of reporting the result is to report the size α of the test used to reject H 0 or accept H 0. If α
More informationTheorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension. n=1
Chapter 2 Probability measures 1. Existence Theorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension to the generated σ-field Proof of Theorem 2.1. Let F 0 be
More informationCan we do statistical inference in a non-asymptotic way? 1
Can we do statistical inference in a non-asymptotic way? 1 Guang Cheng 2 Statistics@Purdue www.science.purdue.edu/bigdata/ ONR Review Meeting@Duke Oct 11, 2017 1 Acknowledge NSF, ONR and Simons Foundation.
More informationLecture Notes 5 Convergence and Limit Theorems. Convergence with Probability 1. Convergence in Mean Square. Convergence in Probability, WLLN
Lecture Notes 5 Convergence and Limit Theorems Motivation Convergence with Probability Convergence in Mean Square Convergence in Probability, WLLN Convergence in Distribution, CLT EE 278: Convergence and
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random
More informationOn the Uniform Asymptotic Validity of Subsampling and the Bootstrap
On the Uniform Asymptotic Validity of Subsampling and the Bootstrap Joseph P. Romano Departments of Economics and Statistics Stanford University romano@stanford.edu Azeem M. Shaikh Department of Economics
More informationStatistics (1): Estimation
Statistics (1): Estimation Marco Banterlé, Christian Robert and Judith Rousseau Practicals 2014-2015 L3, MIDO, Université Paris Dauphine 1 Table des matières 1 Random variables, probability, expectation
More informationChapter 8.8.1: A factorization theorem
LECTURE 14 Chapter 8.8.1: A factorization theorem The characterization of a sufficient statistic in terms of the conditional distribution of the data given the statistic can be difficult to work with.
More informationStrong approximations for resample quantile processes and application to ROC methodology
Strong approximations for resample quantile processes and application to ROC methodology Jiezhun Gu 1, Subhashis Ghosal 2 3 Abstract The receiver operating characteristic (ROC) curve is defined as true
More informationWeek 9 The Central Limit Theorem and Estimation Concepts
Week 9 and Estimation Concepts Week 9 and Estimation Concepts Week 9 Objectives 1 The Law of Large Numbers and the concept of consistency of averages are introduced. The condition of existence of the population
More informationThus f is continuous at x 0. Matthew Straughn Math 402 Homework 6
Matthew Straughn Math 402 Homework 6 Homework 6 (p. 452) 14.3.3, 14.3.4, 14.3.5, 14.3.8 (p. 455) 14.4.3* (p. 458) 14.5.3 (p. 460) 14.6.1 (p. 472) 14.7.2* Lemma 1. If (f (n) ) converges uniformly to some
More informationThe reference [Ho17] refers to the course lecture notes by Ilkka Holopainen.
Department of Mathematics and Statistics Real Analysis I, Fall 207 Solutions to Exercise 6 (6 pages) riikka.schroderus at helsinki.fi Note. The course can be passed by an exam. The first possible exam
More informationFall f(x)g(x) dx. The starting place for the theory of Fourier series is that the family of functions {e inx } n= is orthonormal, that is
18.103 Fall 2013 1. Fourier Series, Part 1. We will consider several function spaces during our study of Fourier series. When we talk about L p ((, π)), it will be convenient to include the factor 1/ in
More information1 Exercises for lecture 1
1 Exercises for lecture 1 Exercise 1 a) Show that if F is symmetric with respect to µ, and E( X )
More informationLecture 20: Linear model, the LSE, and UMVUE
Lecture 20: Linear model, the LSE, and UMVUE Linear Models One of the most useful statistical models is X i = β τ Z i + ε i, i = 1,...,n, where X i is the ith observation and is often called the ith response;
More informationX n D X lim n F n (x) = F (x) for all x C F. lim n F n(u) = F (u) for all u C F. (2)
14:17 11/16/2 TOPIC. Convergence in distribution and related notions. This section studies the notion of the so-called convergence in distribution of real random variables. This is the kind of convergence
More informationContinuous r.v. s: cdf s, Expected Values
Continuous r.v. s: cdf s, Expected Values Engineering Statistics Section 4.2 Josh Engwer TTU 29 February 2016 Josh Engwer (TTU) Continuous r.v. s: cdf s, Expected Values 29 February 2016 1 / 17 PART I
More informationSTAT 830 Hypothesis Testing
STAT 830 Hypothesis Testing Hypothesis testing is a statistical problem where you must choose, on the basis of data X, between two alternatives. We formalize this as the problem of choosing between two
More informationRecall that in order to prove Theorem 8.8, we argued that under certain regularity conditions, the following facts are true under H 0 : 1 n
Chapter 9 Hypothesis Testing 9.1 Wald, Rao, and Likelihood Ratio Tests Suppose we wish to test H 0 : θ = θ 0 against H 1 : θ θ 0. The likelihood-based results of Chapter 8 give rise to several possible
More informationSimple Iteration, cont d
Jim Lambers MAT 772 Fall Semester 2010-11 Lecture 2 Notes These notes correspond to Section 1.2 in the text. Simple Iteration, cont d In general, nonlinear equations cannot be solved in a finite sequence
More information2 2 + x =
Lecture 30: Power series A Power Series is a series of the form c n = c 0 + c 1 x + c x + c 3 x 3 +... where x is a variable, the c n s are constants called the coefficients of the series. n = 1 + x +
More informationLecture 2: CDF and EDF
STAT 425: Introduction to Nonparametric Statistics Winter 2018 Instructor: Yen-Chi Chen Lecture 2: CDF and EDF 2.1 CDF: Cumulative Distribution Function For a random variable X, its CDF F () contains all
More informationChapter 6. Convergence. Probability Theory. Four different convergence concepts. Four different convergence concepts. Convergence in probability
Probability Theory Chapter 6 Convergence Four different convergence concepts Let X 1, X 2, be a sequence of (usually dependent) random variables Definition 1.1. X n converges almost surely (a.s.), or with
More informationLocal Asymptotic Normality
Chapter 8 Local Asymptotic Normality 8.1 LAN and Gaussian shift families N::efficiency.LAN LAN.defn In Chapter 3, pointwise Taylor series expansion gave quadratic approximations to to criterion functions
More informationHigh Dimensional Empirical Likelihood for Generalized Estimating Equations with Dependent Data
High Dimensional Empirical Likelihood for Generalized Estimating Equations with Dependent Data Song Xi CHEN Guanghua School of Management and Center for Statistical Science, Peking University Department
More information5.6 The Normal Distributions
STAT 41 Lecture Notes 13 5.6 The Normal Distributions Definition 5.6.1. A (continuous) random variable X has a normal distribution with mean µ R and variance < R if the p.d.f. of X is f(x µ, ) ( π ) 1/
More informationNATIONAL UNIVERSITY OF SINGAPORE Department of Mathematics MA4247 Complex Analysis II Lecture Notes Part II
NATIONAL UNIVERSITY OF SINGAPORE Department of Mathematics MA4247 Complex Analysis II Lecture Notes Part II Chapter 2 Further properties of analytic functions 21 Local/Global behavior of analytic functions;
More information