Introduction to Self-normalized Limit Theory
|
|
- Christian Norman
- 6 years ago
- Views:
Transcription
1 Introduction to Self-normalized Limit Theory Qi-Man Shao The Chinese University of Hong Kong
2 Outline What is the self-normalization? Why? Classical limit theorems Self-normalized large deviations Self-normalized moderate deviations Self-normalized Cramér type moderate deviations
3 Main references [1 ]V.H. de la Pena, T.Z. Lai and Q.M. Shao (2009). Self-normalized processes: Limit Theory and Statistical Applications. Spriner, Heidelberg. [2 ] Q.M. Shao and W.X. Zhou (2012). Cramér type moderate deviation theorems for self-normalized processes. 3
4 1. Introduction Let X, X 1, X 2,, X n be independent identically distributed (i.i.d.) random variables and let S n = n X i, V 2 n = n Xi 2. Assume EX = 0 and σ 2 = EX 2 <. Standardized sum: S n /(σ n) Self-normalized sum: S n /V n 4
5 Why are we interested in the self-normalized sum? It is of interest from the probability theory alone It has a close relation with Studentized statistics. Let H n = H n (θ, λ) be a sequence of statistics under consideration, where θ contains parameters of interest and λ is a vector of some unknown nuisance parameters. It is a common practice that one needs to estimate λ first from the data, say, the estimator is ˆλ, and then substitute ˆλ in H n, which naturally brings a studentized statistic Ĥn = H n (θ, ˆλ). Typical examples include: Student s t-statistic Hotelling s T 2 statistics Studentized U-statistics The largest eigenvalue of sample correlation matrices 5
6 2. Classical limit theorems Let X, X 1, X 2,, X n be independent identically distributed (i.i.d.) random variables and let n S n = X i. Law of large numbers: EX = µ S n n µ a.s. Law of the iterated logarithm: EX = 0, 0 < σ 2 = EX 2 < lim sup n S n σ = 1 a.s. n (2 log log n) 1/2 6
7 The central limit theorem: If EX = 0 and σ 2 = EX 2 <, then S n σ n d. N(0, 1) If EX = 0 and EX 2 I{ X x} is slowly varying, then there exist a n and b n such that Uniform Berry-Esseen bounds: sup x P ( S n σ n 1 a n S n b n d. N(0, 1).7975E X 3 x) Φ(x). nσ 3 7
8 Non-uniform Berry-Esseen bounds: P ( S n σ n x) Φ(x) C E X 3 (1 + x 3 ) nσ 3 8
9 Cramér - Chernoff s large deviation: If Ee t 0X < for some t 0 > 0, then x > EX, P ( ) 1/n Sn n x inf t 0 e tx Ee tx. Cramér s moderate deviation: Assume EX = 0 and σ 2 = EX 2 <. If Ee t 0 X 1 1/2 < for t 0 > 0, then ( ) S P n σ n x 1 1 Φ(x) uniformly in 0 x o(n 1/6 ). If Ee t 0 X 1 < for t 0 > 0, then for x 0 and x = o(n 1/2 ) ( ) S P n σ n x { = exp x 2 λ( x } ( ) 1 + O( 1 + x ) ), 1 Φ(x) n n where λ(t) is the Cramér s series. 9
10 Conclusion: Classical limit theorem The normalizing constants are deterministic Moment conditions play a crucial role 10
11 3. Self-normalized limit theorems: a brief review Self-normalized sum: S n /V n, where V 2 n = n Xi 2. Do the classical limit theorems remain valid for S n /V n? 11
12 3.1 The central limit theorem: If EX = 0 and EX 2 <, then S n /V n d. N(0, 1) Gine-Götze-Mason (1995): EX = 0 and EX 2 I{ X x} is slowly varying S n /V n d. N(0, 1) 12
13 Bentkus-Götze (1996): If EX = 0, σ 2 = EX 2 and E X 3 <, then sup x P (S n /V n x) Φ(x) C n 1/2 E X 3 /σ Self-normalized limit distribution for X DASL Logan-Mallows-Rice-Shepp (1973): If X DASL(α), then the limiting density function p(x) of S n /V n exists and satisfies p(x) 1 α ( ) 1/2 2 τ α e x2 τα/2 2 π Conjecture: τ α is the solution of c 1 D α ( τ) + c 2 D α (τ) = 0 if α 1 e τ 2 /2 τ τ 0 e x2 /2 dx = 0 if α = 1 13
14 where D α (x) is the parabolic cylinder function. 14
15 3.3 Self-normalized law of the iterated logarithm (Griffin and Kuelbs (1989)) (a) If EX = 0 and EX 2 I{ X x} is slowly varying, then lim sup n S n V n (2 log log n) 1/2 = 1 a.s. (b) If X is symmetric and P (X x) = l(x) x α, 0 < α < 2, where l(x) is a slowly varying function, then there exists 0 < C α < such that lim sup n However, for any a n S n V n (log log n) 1/2 = C α a.s. lim sup n S n a n = 0 a.s.or a.s. 15
16 4. Self-normalized large deviations Question: Is there a self-normalized large deviation without assuming moment condition? Answer: YES! Observe that for x > 0 P (S n /Vn 2 x) = P (S n xvn 2 ) n = P ( (X i xxi 2 ) x) Assume that EX = 0 or EX 2 =. We have EX xex 2 < 0 and Ee t(x x X2) < for t 0. Therefore, by the Chernoff large deviation P (S n /V 2 n x) 1/n inf t 0 Eet(X x X2 ) 16
17 Question: What if the normalizing constant is V n n? It is well-know that Indeed, we have for a, b > 0. Thus ab (a 2 + b 2 )/2 ab = (1/2) inf c>0 (c a2 + b 2 /c) V n n = (1/2) inf c>0 (V 2 n /c + c n) 17
18 and S n P ( x) V n n = P (S n xv n n) = P (S n x(1/2) inf n /c + c n) ( ) = P sup cs n x(1/2)(vn 2 + c 2 n) 0 c 0 ( ) n = P {cx i x(1/2)(xi 2 + c 2 )} 0. sup c 0 c>0 (V 2 Theorem 1.1 [Shao (1997)] If EX = 0 or EX 2 =, then x > 0 ) 1/n P (S n /V n xn 1/2 sup c 0 inf t 0 Eet(cX x( X 2 +c 2 )/2) 18
19 5. Self-normalized moderate deviations Theorem 1.2 [Shao (1997)] (a) If EX = 0 and EX 2 I{ X x} is slowly varying, then ln P (S n /V n > x n ) x 2 n/2 for x n and x n = o( n), i.e., ε > 0 e (1+ε)x2 n/2 P (S n /V n > x n ) e (1 ε)x2 n/2 (b) Let X be a symmetric random variable satisfying P (X x) = l(x) x α, 0 < α < 2. where l(x) is a slowly varying function. Then x n, x n = o( n) ( ) Sn ln P x n β α x 2 n V n where β α is the solution of 0 2 e 2x x2 /β e 2x x2 /β x α+1 dx = 0. (1.1) 19
20 6. Self-normalized Cramér type moderate deviations Theorem 1.3 (Shao (1999)) If EX = 0 and E X 3 <, then uniformly in 0 x o(n 1/6 ). P (S n /V n x) 1 Φ(x) 1 Theorem 1.4 [Jing-Shao-Wang (2003)]. If EX = 0 and E X 3 <, then for x 0, and P (S n /V n x) 1 Φ(x) = 1 + O(1) (1 + x)3 E X 3 nσ 3 P (S n /V n x) (1 Φ(x)) for 0 x n 1/6 σ/(e X 3 ) 1/3. A(1 + x) 3 e x2 /2 n 1/2 E X 3 /σ 3 20
21 7. Self-normalized Cramér moderate deviations for independent random variables Let X 1, X 2,, X n be independent random variables with EX i = 0 and EXi 2 <. Put n S n = and for x > 0. n,x = X i, V 2 n = (1 + x)2 B 2 n (1 + x)3 + Bn 3 n Xi 2, Bn 2 = ESn 2 n EXi 2 I { Xi >B n /(1+x)} n E X i 3 I { Xi B n /(1+x)} Theorem 1.5 [Jing-Shao-Wang (2003)]. 21
22 There is an absolute constant A (> 1) such that for all x 0 satisfying P (S n xv n ) 1 Φ(x) = e O(1) n,x and x 2 max 1 i n EX2 i B 2 n n,x (1 + x) 2 /A, where O(1) A. 22
23 Theorem 1.6 [Jing-Shao-Wang (2003)]. for P (S n /V n x) (1 Φ(x)) n A(1 + x) 3 e x2 /2 E X i 3 0 x B n B 3 n ( n E X i 3 ) 1/3 23
24 8. Cramér moderate deviations for self-normalized processes 8.1 A general result Let ξ 1,..., ξ n be independent random variables satisfying n Eξ i = 0, 1 i n, Eξi 2 = 1. (1.2) Assume the nonlinear process of interest can be decomposed as a standardized partial sum of {ξ i } n, say, W n, plus a remainder, say, D n,1, while its self-normalized version can be written as where W n = T n = W n + D n,1 V n (1 + D n,2 ) 1/2, (1.3) n ( n ) 1/2 ξ i, V n = ξi 2, and D n,1, D n,2 are measurable functions of {ξ i } n. Examples satisfying (1.3) include t-statistic, Studentized U-statistics and L-statistics. In this section, we establish a general Cramér type moderation theorem for self-normalized process T n in the form of (1.3). For 1 24
25 i n and x 0, let δ i,x = (1 + x) 2 E[ξi 2 I {(1+x) ξi >1}] + (1 + x) 3 E[ ξ i 3 I {(1+x) ξi 1}], n n n,x = δ i,x and I n,x = E[e xw n x 2 Vn 2/2 ] = e[e xξ i x 2 ξi 2/2 ]. Let D (i) n,1 and D(i) n,2, for each 1 i n, be arbitrary measurable functions of {ξ j } n j=1,j i, such that {D(i) n,1, D(i) n,2 } and ξ i are independent. Set also for x > 0 that { R n,x := In,x 1 E[(x D n,1 + x 2 D n,2 )e n j=1 (xξ j x 2 ξj 2/2) ] n } + E[min(x ξ i, 1)( D n,1 D (i) n,1 + x D n,2 D (i) n,2 )e j i (xξ j x 2 ξj 2/2) ], where j i = n j=1,j i. Theorem 1.7 (Shao and Zhou (2012)) Let T n be defined in (1.3). Then there is an absolute constant A (> 1) such that exp{o(1) n,x } ( 1 A R n,x ) P (T n x) 1 Φ(x) 25 (1.4)
26 and P (T n x) ( 1 Φ(x) ) exp{o(1) n,x } ( 1 + A R n,x ) (1.5) +P { D n,1 /V n > 1/(4x) } + P { D n,2 > 1/(4x 2 ) } for all x > 1 satisfying and max δ i,x 1 (1.6) 1 i n n,x (1 + x) 2 /A, (1.7) where O(1) A. 26
27 8.2 Studentized U-statistics Let X 1, X 2,, X n be a sequence of i.i.d. random variables and let h : R m R be a Borel measurable symmetric function of m variables, where 2 m < n/2. Consider Hoeffding s U-statistic with a kernel h of degree m given by ( ) 1 n U n = m 1 i 1 <...<i m n h(x i1,..., X im ), (1.8) which is an unbiased estimate of θ = eh(x 1,..., X m ). The U-statistic is a basic statistic and its asymptotic properties have been extensively studied in literature. Let g(x) = Eh(x, X 2,..., X m ), x R and σ 2 = Var(g(X 1 )). For standardized (non-degenerate) U-statistic n Z n = mσ (U n θ), (1.9) where σ > 0 and m is fixed. 27
28 Since σ is usually unknown, consider the following Studentized U-statistic n T n = (U n θ), (1.10) ms 1 where s 2 1 is the leave-one-out Jackknife estimator of σ 2 given by s 2 (n 1) n 1 = (q (n m) 2 i U n ) 2 with (1.11) ( ) 1 n 1 q i = h(x i, X l1,..., X lm 1 ) m 1 (l 1,...,l m ) C m 1,i and C m 1,i = {(l 1,..., l m 1 ) : 1 l 1 < < l m 1 n, l j i for 1 j m 1}. As a direct but non-trivial consequence of Theorem 1.7, we can establish a sharp Cramér type moderate deviation theorem for Studentized U-statistic T n as follows. Theorem 1.8 (Shao and Zhou (2012)) Let 2 < p 3 and assume 0 < σ p := (E g(x 1 ) θ p ) 1/p <. 28
29 Suppose that there are constants c 0 1 and τ 0 such that m (h(x 1,..., x m ) θ) 2 c 0 (τσ 2 + (g(x i ) θ) ). 2 (1.12) Then there exists a constant A > 1 only depending on m such that { P (T n x) σ p p (1 + x) p } = 1 + O(1) 1 Φ(x) σ p n + c (1 + x)3 0(1 + τ), (1.13) (p 2)/2 n 1/2 for any 0 < x < 1 A min{σn(p 2)/(2p) /σ p, n 1/6 /(c 0 (1 + τ)) 1/6 }, where O(1) A. In particular, we have P (T n x) 1 Φ(x) uniformly in 0 x o(n (p 2)/(2p) ). 1 (1.14) Clearly, condition (1.12) is satisfied for the t-statistic (h(x 1, x 2 ) = (x 1 + x 2 )/2 with c 0 = 2 and τ = 0), sample variance (h(x 1, x 2 ) = (x 1 x 2 ) 2 /2, c 0 = 10, τ = θ 2 /σ 2 ), Gini s mean difference (h(x 1, x 2 ) = x 1 x 2, c 0 = 8, τ = θ 2 /σ 2 ) and one-sample Wilcoxon s statistic (h(x 1, x 2 ) = 1{x 1 + x 2 0}, c 0 = 1, τ = 1/σ 2 ). It would be 29
30 interesting to know if condition (1.12) can be weakened, but it seems impossible to remove condition (1.12) completely. 30
31 9. Miscellanies Horváth-Shao (1996): Large deviation for S n / max 1 i n X i Dembo-Shao (1998): Self-normalized moderate and large deviations in R d Gine-Mason (2001): Sub-gaussian property for X in the Feller class Bercu-Gassiat-Rio (2002): Concentration inequalities, large and moderate deviations for self-normalized empirical processes De la Peña-Klass-Lai (2004): Self-normalized processes: exponential inequalities, moment and limit theorems Jing-Shao-Zhou (2004): Self-normalized saddle point approximations Jing-Shao-Wang (2003): The studentized bootstrap 31
32 He and Shao (1996): Bahadur efficiency of studentized score tests He and Shao (1996, 2000): Bahadur representations for M-estimators Horvath and Shao (1996): Change point analysis Chen and Shao (1997, 2000): Monte Carlo methods in Baysian computations Liu and Shao (2013): Hotelling s T 2 statistics 32
33 Conclusion: Self-normalized limit theorems Require little or no moment assumption Results are more elegant and neater than many classical limit theorems Provide much wider applicability to other fields and to statistics in particular 33
Self-normalized Cramér-Type Large Deviations for Independent Random Variables
Self-normalized Cramér-Type Large Deviations for Independent Random Variables Qi-Man Shao National University of Singapore and University of Oregon qmshao@darkwing.uoregon.edu 1. Introduction Let X, X
More informationCramér-Type Moderate Deviation Theorems for Two-Sample Studentized (Self-normalized) U-Statistics. Wen-Xin Zhou
Cramér-Type Moderate Deviation Theorems for Two-Sample Studentized (Self-normalized) U-Statistics Wen-Xin Zhou Department of Mathematics and Statistics University of Melbourne Joint work with Prof. Qi-Man
More informationIn Memory of Wenbo V Li s Contributions
In Memory of Wenbo V Li s Contributions Qi-Man Shao The Chinese University of Hong Kong qmshao@cuhk.edu.hk The research is partially supported by Hong Kong RGC GRF 403513 Outline Lower tail probabilities
More informationAlmost Sure Central Limit Theorem for Self-Normalized Partial Sums of Negatively Associated Random Variables
Filomat 3:5 (207), 43 422 DOI 0.2298/FIL70543W Published by Faculty of Sciences and Mathematics, University of Niš, Serbia Available at: http://www.pmf.ni.ac.rs/filomat Almost Sure Central Limit Theorem
More informationMathematical Methods for Neurosciences. ENS - Master MVA Paris 6 - Master Maths-Bio ( )
Mathematical Methods for Neurosciences. ENS - Master MVA Paris 6 - Master Maths-Bio (2014-2015) Etienne Tanré - Olivier Faugeras INRIA - Team Tosca October 22nd, 2014 E. Tanré (INRIA - Team Tosca) Mathematical
More informationOn the Breiman conjecture
Péter Kevei 1 David Mason 2 1 TU Munich 2 University of Delaware 12th GPSD Bochum Outline Introduction Motivation Earlier results Partial solution Sketch of the proof Subsequential limits Further remarks
More informationRELATIVE ERRORS IN CENTRAL LIMIT THEOREMS FOR STUDENT S t STATISTIC, WITH APPLICATIONS
Statistica Sinica 19 (2009, 343-354 RELATIVE ERRORS IN CENTRAL LIMIT THEOREMS FOR STUDENT S t STATISTIC, WITH APPLICATIONS Qiying Wang and Peter Hall University of Sydney and University of Melbourne Abstract:
More informationLower Tail Probabilities and Normal Comparison Inequalities. In Memory of Wenbo V. Li s Contributions
Lower Tail Probabilities and Normal Comparison Inequalities In Memory of Wenbo V. Li s Contributions Qi-Man Shao The Chinese University of Hong Kong Lower Tail Probabilities and Normal Comparison Inequalities
More informationEXACT CONVERGENCE RATE AND LEADING TERM IN THE CENTRAL LIMIT THEOREM FOR U-STATISTICS
Statistica Sinica 6006, 409-4 EXACT CONVERGENCE RATE AND LEADING TERM IN THE CENTRAL LIMIT THEOREM FOR U-STATISTICS Qiying Wang and Neville C Weber The University of Sydney Abstract: The leading term in
More informationSelf-normalized Cramér type moderate deviations for the maximum of sums
Bernoulli 193, 2013, 1006 1027 DOI: 10.3150/12-BEJ415 arxiv:1307.6044v1 [math.st] 23 Jul 2013 Self-normalized Cramér type moderate deviations for the maximum of sums WEIDONG LIU 1, QI-MAN SHAO 2 and QIYING
More informationSTA205 Probability: Week 8 R. Wolpert
INFINITE COIN-TOSS AND THE LAWS OF LARGE NUMBERS The traditional interpretation of the probability of an event E is its asymptotic frequency: the limit as n of the fraction of n repeated, similar, and
More informationMatrix concentration inequalities
ELE 538B: Mathematics of High-Dimensional Data Matrix concentration inequalities Yuxin Chen Princeton University, Fall 2018 Recap: matrix Bernstein inequality Consider a sequence of independent random
More informationTail bound inequalities and empirical likelihood for the mean
Tail bound inequalities and empirical likelihood for the mean Sandra Vucane 1 1 University of Latvia, Riga 29 th of September, 2011 Sandra Vucane (LU) Tail bound inequalities and EL for the mean 29.09.2011
More informationConcentration inequalities and the entropy method
Concentration inequalities and the entropy method Gábor Lugosi ICREA and Pompeu Fabra University Barcelona what is concentration? We are interested in bounding random fluctuations of functions of many
More informationAsymptotic Statistics-III. Changliang Zou
Asymptotic Statistics-III Changliang Zou The multivariate central limit theorem Theorem (Multivariate CLT for iid case) Let X i be iid random p-vectors with mean µ and and covariance matrix Σ. Then n (
More informationarxiv: v1 [math.pr] 14 Sep 2007
Probability Surveys Vol. 0 (0000) ISSN: 1549-5787 url: http://www.vtex.lt/ejournals/ps/index.html Pseudo-Maximization and Self-Normalized Processes* arxiv:0709.33v1 [math.pr] 14 Sep 007 Victor H. de la
More informationOn large deviations for combinatorial sums
arxiv:1901.0444v1 [math.pr] 14 Jan 019 On large deviations for combinatorial sums Andrei N. Frolov Dept. of Mathematics and Mechanics St. Petersburg State University St. Petersburg, Russia E-mail address:
More informationLecture 8: Information Theory and Statistics
Lecture 8: Information Theory and Statistics Part II: Hypothesis Testing and I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 23, 2015 1 / 50 I-Hsiang
More informationMarch 1, Florida State University. Concentration Inequalities: Martingale. Approach and Entropy Method. Lizhe Sun and Boning Yang.
Florida State University March 1, 2018 Framework 1. (Lizhe) Basic inequalities Chernoff bounding Review for STA 6448 2. (Lizhe) Discrete-time martingales inequalities via martingale approach 3. (Boning)
More informationLecture 13: Subsampling vs Bootstrap. Dimitris N. Politis, Joseph P. Romano, Michael Wolf
Lecture 13: 2011 Bootstrap ) R n x n, θ P)) = τ n ˆθn θ P) Example: ˆθn = X n, τ n = n, θ = EX = µ P) ˆθ = min X n, τ n = n, θ P) = sup{x : F x) 0} ) Define: J n P), the distribution of τ n ˆθ n θ P) under
More informationPhenomena in high dimensions in geometric analysis, random matrices, and computational geometry Roscoff, France, June 25-29, 2012
Phenomena in high dimensions in geometric analysis, random matrices, and computational geometry Roscoff, France, June 25-29, 202 BOUNDS AND ASYMPTOTICS FOR FISHER INFORMATION IN THE CENTRAL LIMIT THEOREM
More informationSDS : Theoretical Statistics
SDS 384 11: Theoretical Statistics Lecture 1: Introduction Purnamrita Sarkar Department of Statistics and Data Science The University of Texas at Austin https://psarkar.github.io/teaching Manegerial Stuff
More informationThe Convergence Rate for the Normal Approximation of Extreme Sums
The Convergence Rate for the Normal Approximation of Extreme Sums Yongcheng Qi University of Minnesota Duluth WCNA 2008, Orlando, July 2-9, 2008 This talk is based on a joint work with Professor Shihong
More informationTransforms. Convergence of probability generating functions. Convergence of characteristic functions functions
Transforms For non-negative integer value ranom variables, let the probability generating function g X : [0, 1] [0, 1] be efine by g X (t) = E(t X ). The moment generating function ψ X (t) = E(e tx ) is
More informationSelected Exercises on Expectations and Some Probability Inequalities
Selected Exercises on Expectations and Some Probability Inequalities # If E(X 2 ) = and E X a > 0, then P( X λa) ( λ) 2 a 2 for 0 < λ
More informationAn improved result in almost sure central limit theorem for self-normalized products of partial sums
Wu and Chen Journal of Inequalities and Applications 23, 23:29 http://www.ournalofinequalitiesandapplications.com/content/23//29 R E S E A R C H Open Access An improved result in almost sure central it
More informationd(x n, x) d(x n, x nk ) + d(x nk, x) where we chose any fixed k > N
Problem 1. Let f : A R R have the property that for every x A, there exists ɛ > 0 such that f(t) > ɛ if t (x ɛ, x + ɛ) A. If the set A is compact, prove there exists c > 0 such that f(x) > c for all x
More informationCramér type moderate deviations for trimmed L-statistics
arxiv:1608.05015v1 [math.pr] 17 Aug 2016 Cramér type moderate deviations for trimmed L-statistics Nadezhda Gribkova St.Petersburg State University, Mathematics and Mechanics Faculty, 199034, Universitetskaya
More informationVictor H. de la Pena ~. Tze Leung Lai. Qi-Man Shao. Self-Normalized Processes. Limit Theory and Statistical Applications ABC
Victor H. de la Pena ~. Tze Leung Lai. Qi-Man Shao Self-Normalized Processes Limit Theory and Statistical Applications ABC Contents 1 Introduction... 1 Part I Independent Random Variables 2 Classical Limit
More informationComplexity of two and multi-stage stochastic programming problems
Complexity of two and multi-stage stochastic programming problems A. Shapiro School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332-0205, USA The concept
More informationMathematical Statistics
Mathematical Statistics Chapter Three. Point Estimation 3.4 Uniformly Minimum Variance Unbiased Estimator(UMVUE) Criteria for Best Estimators MSE Criterion Let F = {p(x; θ) : θ Θ} be a parametric distribution
More informationA BERRY ESSEEN BOUND FOR STUDENT S STATISTIC IN THE NON-I.I.D. CASE. 1. Introduction and Results
A BERRY ESSEE BOUD FOR STUDET S STATISTIC I THE O-I.I.D. CASE V. Bentkus 1 M. Bloznelis 1 F. Götze 1 Abstract. We establish a Berry Esséen bound for Student s statistic for independent (non-identically)
More informationEconometrics I, Estimation
Econometrics I, Estimation Department of Economics Stanford University September, 2008 Part I Parameter, Estimator, Estimate A parametric is a feature of the population. An estimator is a function of the
More informationAdvanced Statistics II: Non Parametric Tests
Advanced Statistics II: Non Parametric Tests Aurélien Garivier ParisTech February 27, 2011 Outline Fitting a distribution Rank Tests for the comparison of two samples Two unrelated samples: Mann-Whitney
More informationA generalization of Cramér large deviations for martingales
A generalization of Cramér large deviations for martingales Xiequan Fan, Ion Grama, Quansheng Liu To cite this version: Xiequan Fan, Ion Grama, Quansheng Liu. A generalization of Cramér large deviations
More informationSTAT 200C: High-dimensional Statistics
STAT 200C: High-dimensional Statistics Arash A. Amini May 30, 2018 1 / 59 Classical case: n d. Asymptotic assumption: d is fixed and n. Basic tools: LLN and CLT. High-dimensional setting: n d, e.g. n/d
More informationBrief Review on Estimation Theory
Brief Review on Estimation Theory K. Abed-Meraim ENST PARIS, Signal and Image Processing Dept. abed@tsi.enst.fr This presentation is essentially based on the course BASTA by E. Moulines Brief review on
More informationMODERATE DEVIATIONS IN POISSON APPROXIMATION: A FIRST ATTEMPT
Statistica Sinica 23 (2013), 1523-1540 doi:http://dx.doi.org/10.5705/ss.2012.203s MODERATE DEVIATIONS IN POISSON APPROXIMATION: A FIRST ATTEMPT Louis H. Y. Chen 1, Xiao Fang 1,2 and Qi-Man Shao 3 1 National
More informationLower Tail Probabilities and Related Problems
Lower Tail Probabilities and Related Problems Qi-Man Shao National University of Singapore and University of Oregon qmshao@darkwing.uoregon.edu . Lower Tail Probabilities Let {X t, t T } be a real valued
More informationA Law of the Iterated Logarithm. for Grenander s Estimator
A Law of the Iterated Logarithm for Grenander s Estimator Jon A. Wellner University of Washington, Seattle Probability Seminar October 24, 2016 Based on joint work with: Lutz Dümbgen and Malcolm Wolff
More informationAsymptotic results for empirical measures of weighted sums of independent random variables
Asymptotic results for empirical measures of weighted sums of independent random variables B. Bercu and W. Bryc University Bordeaux 1, France Seminario di Probabilità e Statistica Matematica Sapienza Università
More informationExperience Rating in General Insurance by Credibility Estimation
Experience Rating in General Insurance by Credibility Estimation Xian Zhou Department of Applied Finance and Actuarial Studies Macquarie University, Sydney, Australia Abstract This work presents a new
More informationBahadur representations for bootstrap quantiles 1
Bahadur representations for bootstrap quantiles 1 Yijun Zuo Department of Statistics and Probability, Michigan State University East Lansing, MI 48824, USA zuo@msu.edu 1 Research partially supported by
More informationProbability and Measure
Chapter 4 Probability and Measure 4.1 Introduction In this chapter we will examine probability theory from the measure theoretic perspective. The realisation that measure theory is the foundation of probability
More informationDeviation inequalities for martingales with applications
Deviation inequalities for martingales with applications Xiequan Fan,a,b, Ion Grama c, Quansheng Liu,c a Center for Applied Mathematics, Tianjin University, Tianjin 300072, China b Regularity Team, Inria,
More informationA generalization of Strassen s functional LIL
A generalization of Strassen s functional LIL Uwe Einmahl Departement Wiskunde Vrije Universiteit Brussel Pleinlaan 2 B-1050 Brussel, Belgium E-mail: ueinmahl@vub.ac.be Abstract Let X 1, X 2,... be a sequence
More informationNormal Approximation for Non-linear Statistics Using a Concentration Inequality Approach
Normal Approximation for Non-linear Statistics Using a Concentration Inequality Approach Louis H.Y. Chen 1 National University of Singapore and Qi-Man Shao 2 Hong Kong University of Science and Technology,
More informationAsymptotic results for empirical measures of weighted sums of independent random variables
Asymptotic results for empirical measures of weighted sums of independent random variables B. Bercu and W. Bryc University Bordeaux 1, France Workshop on Limit Theorems, University Paris 1 Paris, January
More informationOn probabilities of large and moderate deviations for L-statistics: a survey of some recent developments
UDC 519.2 On probabilities of large and moderate deviations for L-statistics: a survey of some recent developments N. V. Gribkova Department of Probability Theory and Mathematical Statistics, St.-Petersburg
More informationA Gentle Introduction to Stein s Method for Normal Approximation I
A Gentle Introduction to Stein s Method for Normal Approximation I Larry Goldstein University of Southern California Introduction to Stein s Method for Normal Approximation 1. Much activity since Stein
More informationLast Lecture. Biostatistics Statistical Inference Lecture 14 Obtaining Best Unbiased Estimator. Related Theorems. Rao-Blackwell Theorem
Last Lecture Biostatistics 62 - Statistical Inference Lecture 14 Obtaining Best Unbiased Estimator Hyun Min Kang February 28th, 213 For single-parameter exponential family, is Cramer-Rao bound always attainable?
More informationExample continued. Math 425 Intro to Probability Lecture 37. Example continued. Example
continued : Coin tossing Math 425 Intro to Probability Lecture 37 Kenneth Harris kaharri@umich.edu Department of Mathematics University of Michigan April 8, 2009 Consider a Bernoulli trials process with
More informationOrdinal optimization - Empirical large deviations rate estimators, and multi-armed bandit methods
Ordinal optimization - Empirical large deviations rate estimators, and multi-armed bandit methods Sandeep Juneja Tata Institute of Fundamental Research Mumbai, India joint work with Peter Glynn Applied
More informationQualifying Exam CS 661: System Simulation Summer 2013 Prof. Marvin K. Nakayama
Qualifying Exam CS 661: System Simulation Summer 2013 Prof. Marvin K. Nakayama Instructions This exam has 7 pages in total, numbered 1 to 7. Make sure your exam has all the pages. This exam will be 2 hours
More informationWinter 2019 Math 106 Topics in Applied Mathematics. Lecture 8: Importance Sampling
Winter 2019 Math 106 Topics in Applied Mathematics Data-driven Uncertainty Quantification Yoonsang Lee (yoonsang.lee@dartmouth.edu) Lecture 8: Importance Sampling 8.1 Importance Sampling Importance sampling
More informationLecture 7 Introduction to Statistical Decision Theory
Lecture 7 Introduction to Statistical Decision Theory I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 20, 2016 1 / 55 I-Hsiang Wang IT Lecture 7
More informationLecture 5: Importance sampling and Hamilton-Jacobi equations
Lecture 5: Importance sampling and Hamilton-Jacobi equations Henrik Hult Department of Mathematics KTH Royal Institute of Technology Sweden Summer School on Monte Carlo Methods and Rare Events Brown University,
More informationExercises with solutions (Set D)
Exercises with solutions Set D. A fair die is rolled at the same time as a fair coin is tossed. Let A be the number on the upper surface of the die and let B describe the outcome of the coin toss, where
More informationAsymptotics of minimax stochastic programs
Asymptotics of minimax stochastic programs Alexander Shapiro Abstract. We discuss in this paper asymptotics of the sample average approximation (SAA) of the optimal value of a minimax stochastic programming
More informationSharp large deviation results for sums of independent random variables
Sharp large deviation results for sums of independent random variables Xiequan Fan, Ion Grama, Quansheng Liu To cite this version: Xiequan Fan, Ion Grama, Quansheng Liu. Sharp large deviation results for
More informationP (A G) dp G P (A G)
First homework assignment. Due at 12:15 on 22 September 2016. Homework 1. We roll two dices. X is the result of one of them and Z the sum of the results. Find E [X Z. Homework 2. Let X be a r.v.. Assume
More informationLimiting Distributions
Limiting Distributions We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the
More informationNon-uniform Berry Esseen Bounds for Weighted U-Statistics and Generalized L-Statistics
Coun Math Stat 0 :5 67 DOI 0.007/s4004-0-009- Non-unifor Berry Esseen Bounds for Weighted U-Statistics and Generalized L-Statistics Haojun Hu Qi-Man Shao Received: 9 August 0 / Accepted: Septeber 0 / Published
More informationThe largest eigenvalues of the sample covariance matrix. in the heavy-tail case
The largest eigenvalues of the sample covariance matrix 1 in the heavy-tail case Thomas Mikosch University of Copenhagen Joint work with Richard A. Davis (Columbia NY), Johannes Heiny (Aarhus University)
More informationConcentration of Measures by Bounded Size Bias Couplings
Concentration of Measures by Bounded Size Bias Couplings Subhankar Ghosh, Larry Goldstein University of Southern California [arxiv:0906.3886] January 10 th, 2013 Concentration of Measure Distributional
More informationStochastic Processes and Monte-Carlo Methods. University of Massachusetts: Spring 2018 version. Luc Rey-Bellet
Stochastic Processes and Monte-Carlo Methods University of Massachusetts: Spring 2018 version Luc Rey-Bellet April 5, 2018 Contents 1 Simulation and the Monte-Carlo method 3 1.1 Review of probability............................
More informationLimiting Distributions
We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the two fundamental results
More informationLecture 2. We now introduce some fundamental tools in martingale theory, which are useful in controlling the fluctuation of martingales.
Lecture 2 1 Martingales We now introduce some fundamental tools in martingale theory, which are useful in controlling the fluctuation of martingales. 1.1 Doob s inequality We have the following maximal
More informationStat 710: Mathematical Statistics Lecture 31
Stat 710: Mathematical Statistics Lecture 31 Jun Shao Department of Statistics University of Wisconsin Madison, WI 53706, USA Jun Shao (UW-Madison) Stat 710, Lecture 31 April 13, 2009 1 / 13 Lecture 31:
More informationStability of Stochastic Differential Equations
Lyapunov stability theory for ODEs s Stability of Stochastic Differential Equations Part 1: Introduction Department of Mathematics and Statistics University of Strathclyde Glasgow, G1 1XH December 2010
More informationBerry-Esseen bounds for self-normalized martingales
Berry-Essee bouds for self-ormalized martigales Xiequa Fa a, Qi-Ma Shao b a Ceter for Applied Mathematics, Tiaji Uiversity, Tiaji 30007, Chia b Departmet of Statistics, The Chiese Uiversity of Hog Kog,
More informationFormulas for probability theory and linear models SF2941
Formulas for probability theory and linear models SF2941 These pages + Appendix 2 of Gut) are permitted as assistance at the exam. 11 maj 2008 Selected formulae of probability Bivariate probability Transforms
More informationELEG 5633 Detection and Estimation Signal Detection: Deterministic Signals
ELEG 5633 Detection and Estimation Signal Detection: Deterministic Signals Jingxian Wu Department of Electrical Engineering University of Arkansas Outline Matched Filter Generalized Matched Filter Signal
More informationOn corrections of classical multivariate tests for high-dimensional data. Jian-feng. Yao Université de Rennes 1, IRMAR
Introduction a two sample problem Marčenko-Pastur distributions and one-sample problems Random Fisher matrices and two-sample problems Testing cova On corrections of classical multivariate tests for high-dimensional
More informationNegative Association, Ordering and Convergence of Resampling Methods
Negative Association, Ordering and Convergence of Resampling Methods Nicolas Chopin ENSAE, Paristech (Joint work with Mathieu Gerber and Nick Whiteley, University of Bristol) Resampling schemes: Informal
More informationExtreme inference in stationary time series
Extreme inference in stationary time series Moritz Jirak FOR 1735 February 8, 2013 1 / 30 Outline 1 Outline 2 Motivation The multivariate CLT Measuring discrepancies 3 Some theory and problems The problem
More informationMathematical statistics
October 18 th, 2018 Lecture 16: Midterm review Countdown to mid-term exam: 7 days Week 1 Chapter 1: Probability review Week 2 Week 4 Week 7 Chapter 6: Statistics Chapter 7: Point Estimation Chapter 8:
More informationAPPM/MATH 4/5520 Solutions to Exam I Review Problems. f X 1,X 2. 2e x 1 x 2. = x 2
APPM/MATH 4/5520 Solutions to Exam I Review Problems. (a) f X (x ) f X,X 2 (x,x 2 )dx 2 x 2e x x 2 dx 2 2e 2x x was below x 2, but when marginalizing out x 2, we ran it over all values from 0 to and so
More informationEIE6207: Estimation Theory
EIE6207: Estimation Theory Man-Wai MAK Dept. of Electronic and Information Engineering, The Hong Kong Polytechnic University enmwmak@polyu.edu.hk http://www.eie.polyu.edu.hk/ mwmak References: Steven M.
More informationMoments. Raw moment: February 25, 2014 Normalized / Standardized moment:
Moments Lecture 10: Central Limit Theorem and CDFs Sta230 / Mth 230 Colin Rundel Raw moment: Central moment: µ n = EX n ) µ n = E[X µ) 2 ] February 25, 2014 Normalized / Standardized moment: µ n σ n Sta230
More informationCHAPTER 3: LARGE SAMPLE THEORY
CHAPTER 3 LARGE SAMPLE THEORY 1 CHAPTER 3: LARGE SAMPLE THEORY CHAPTER 3 LARGE SAMPLE THEORY 2 Introduction CHAPTER 3 LARGE SAMPLE THEORY 3 Why large sample theory studying small sample property is usually
More informationMOMENT CONVERGENCE RATES OF LIL FOR NEGATIVELY ASSOCIATED SEQUENCES
J. Korean Math. Soc. 47 1, No., pp. 63 75 DOI 1.4134/JKMS.1.47..63 MOMENT CONVERGENCE RATES OF LIL FOR NEGATIVELY ASSOCIATED SEQUENCES Ke-Ang Fu Li-Hua Hu Abstract. Let X n ; n 1 be a strictly stationary
More informationECE 275B Homework # 1 Solutions Version Winter 2015
ECE 275B Homework # 1 Solutions Version Winter 2015 1. (a) Because x i are assumed to be independent realizations of a continuous random variable, it is almost surely (a.s.) 1 the case that x 1 < x 2
More informationChapter 2: Resampling Maarten Jansen
Chapter 2: Resampling Maarten Jansen Randomization tests Randomized experiment random assignment of sample subjects to groups Example: medical experiment with control group n 1 subjects for true medicine,
More informationWeak quenched limiting distributions of a one-dimensional random walk in a random environment
Weak quenched limiting distributions of a one-dimensional random walk in a random environment Jonathon Peterson Cornell University Department of Mathematics Joint work with Gennady Samorodnitsky September
More informationPRECISE ASYMPTOTIC IN THE LAW OF THE ITERATED LOGARITHM AND COMPLETE CONVERGENCE FOR U-STATISTICS
PRECISE ASYMPTOTIC IN THE LAW OF THE ITERATED LOGARITHM AND COMPLETE CONVERGENCE FOR U-STATISTICS REZA HASHEMI and MOLUD ABDOLAHI Department of Statistics, Faculty of Science, Razi University, 67149, Kermanshah,
More informationHan-Ying Liang, Dong-Xia Zhang, and Jong-Il Baek
J. Korean Math. Soc. 41 (2004), No. 5, pp. 883 894 CONVERGENCE OF WEIGHTED SUMS FOR DEPENDENT RANDOM VARIABLES Han-Ying Liang, Dong-Xia Zhang, and Jong-Il Baek Abstract. We discuss in this paper the strong
More informationStein s method and zero bias transformation: Application to CDO pricing
Stein s method and zero bias transformation: Application to CDO pricing ESILV and Ecole Polytechnique Joint work with N. El Karoui Introduction CDO a portfolio credit derivative containing 100 underlying
More informationAhmed Mohammed. Harnack Inequality for Non-divergence Structure Semi-linear Elliptic Equations
Harnack Inequality for Non-divergence Structure Semi-linear Elliptic Equations International Conference on PDE, Complex Analysis, and Related Topics Miami, Florida January 4-7, 2016 An Outline 1 The Krylov-Safonov
More informationEstimation of the quantile function using Bernstein-Durrmeyer polynomials
Estimation of the quantile function using Bernstein-Durrmeyer polynomials Andrey Pepelyshev Ansgar Steland Ewaryst Rafaj lowic The work is supported by the German Federal Ministry of the Environment, Nature
More informationLarge sample covariance matrices and the T 2 statistic
Large sample covariance matrices and the T 2 statistic EURANDOM, the Netherlands Joint work with W. Zhou Outline 1 2 Basic setting Let {X ij }, i, j =, be i.i.d. r.v. Write n s j = (X 1j,, X pj ) T and
More informationSubmitted to the Brazilian Journal of Probability and Statistics
Submitted to the Brazilian Journal of Probability and Statistics Multivariate normal approximation of the maximum likelihood estimator via the delta method Andreas Anastasiou a and Robert E. Gaunt b a
More informationTheory and Applications of Stochastic Systems Lecture Exponential Martingale for Random Walk
Instructor: Victor F. Araman December 4, 2003 Theory and Applications of Stochastic Systems Lecture 0 B60.432.0 Exponential Martingale for Random Walk Let (S n : n 0) be a random walk with i.i.d. increments
More informationExtreme Value Analysis and Spatial Extremes
Extreme Value Analysis and Department of Statistics Purdue University 11/07/2013 Outline Motivation 1 Motivation 2 Extreme Value Theorem and 3 Bayesian Hierarchical Models Copula Models Max-stable Models
More informationChapter 8. Hypothesis Testing. Po-Ning Chen. Department of Communications Engineering. National Chiao-Tung University. Hsin Chu, Taiwan 30050
Chapter 8 Hypothesis Testing Po-Ning Chen Department of Communications Engineering National Chiao-Tung University Hsin Chu, Taiwan 30050 Error exponent and divergence II:8-1 Definition 8.1 (exponent) Arealnumbera
More informationAssessing the dependence of high-dimensional time series via sample autocovariances and correlations
Assessing the dependence of high-dimensional time series via sample autocovariances and correlations Johannes Heiny University of Aarhus Joint work with Thomas Mikosch (Copenhagen), Richard Davis (Columbia),
More informationn! (k 1)!(n k)! = F (X) U(0, 1). (x, y) = n(n 1) ( F (y) F (x) ) n 2
Order statistics Ex. 4. (*. Let independent variables X,..., X n have U(0, distribution. Show that for every x (0,, we have P ( X ( < x and P ( X (n > x as n. Ex. 4.2 (**. By using induction or otherwise,
More informationn! (k 1)!(n k)! = F (X) U(0, 1). (x, y) = n(n 1) ( F (y) F (x) ) n 2
Order statistics Ex. 4.1 (*. Let independent variables X 1,..., X n have U(0, 1 distribution. Show that for every x (0, 1, we have P ( X (1 < x 1 and P ( X (n > x 1 as n. Ex. 4.2 (**. By using induction
More informationChapter 3: Unbiased Estimation Lecture 22: UMVUE and the method of using a sufficient and complete statistic
Chapter 3: Unbiased Estimation Lecture 22: UMVUE and the method of using a sufficient and complete statistic Unbiased estimation Unbiased or asymptotically unbiased estimation plays an important role in
More informationProbability Background
Probability Background Namrata Vaswani, Iowa State University August 24, 2015 Probability recap 1: EE 322 notes Quick test of concepts: Given random variables X 1, X 2,... X n. Compute the PDF of the second
More information