Elements of Asymptotic Theory. James L. Powell Department of Economics University of California, Berkeley
|
|
- Prosper Blankenship
- 6 years ago
- Views:
Transcription
1 Elements of Asymtotic Theory James L. Powell Deartment of Economics University of California, Berkeley Objectives of Asymtotic Theory While exact results are available for, say, the distribution of the classical least squares estimator for the normal linear regression model, and for other leading secial combinations of distributions and statistics, generally distributional results are unavailable when statistics are nonlinear in the underlying data and/or when the joint distribution of those data is not arametrically secified. Fortunately, most statistics of interest can be shown to be smooth functions of samle averages of some observable random variables, in which case, under fairly general conditions, their distributions can be aroximated by a normal distribution, usually with a covariance matrix that shrinks to zero as the number of random variables in the averages increases. Thus, as the samle size increases, the ga between the true distribution of the statistic and its aroximating distribution asymtotes to zero, motivating the term asymtotic theory. There are two main stes in the aroximation of the distribution of a statistic by a normal distribution. The first ste is the demonstration that a samle average converges (in an aroriate sense) to its exectation, and that a standardized version of the statistic has distribution function that converges to a standard normal. Such results are called limit theorems the former (aroximation of a samle average by its exectations) is called a law of large numbers (abbreviated LLN ), and the second (aroximation of the distribution of a standardized samle average by a standard normal distribution) is known as a central limit theorem (abbreviated CLT ). There are a number of LLNs and CLTs in the statistics literature, which imose different combinations of restrictions on the number of moments of the variables being averaged and the deendence between them; some basic versions are given below. The second ste in deriving a normal aroximation of the distribution of a smooth function of samle averages is demonstration that this smooth function is aroximately a linear function of samle averages (which is itself a samle average). Useful results for this ste, which use the usual calculus aroximation of smooth (differentiable) functions by normal functions, are groued under the heading Slutsky theorems, after the author of one of the articularly useful results. So, as long as the conditions of the limit theorems and Slutsky theorems aly, smooth functions of samle averages are aroximately samle averages 1
2 themselves, and thus their distribution functions are aroximately normal. The whole aaratus of asymtotic theory is made u of the regularity conditions for alicability of the limit and Slutsky theorems, along with the rules for mathematical maniulation of the aroximating objects. In order to discuss aroximation of a statistic by a constant, or its distribution by a normal distribution, we first have to extend the usual definitions of limits of deterministic sequences to accomodate sequences of random variables (yielding the concet of convergence in robability ) or their distribution functions ( convergence in distribution ). Convergence in Distribution and Probability For most alications of asymtotic theory to statistical roblems, the objects of interest are sequences of random vectors (or matrices), say or Y N, which are generally statistics indexed by the samle size N; the object is to find simle aroximations for the distribution functions of or Y N which can be made arbitrarily close to their true distribution functions for N large enough. Writing the cumulative distribution function of as F N (x) Pr{ x} (where the argument x has the same dimension as and the inequality holds only if it is true for each comonent), we define convergence in distribution (sometimes called convergence in law) in terms of convergence of the function F N (x) to some fixed distribution function F (x) for any oint x where the F is continuous. That is, the random vector converges in distribution to X, denoted d X as N, if at all oints x where F (x) is continuous. lim N F N(x) =F (x) A few comments on this definition and its imlications are warranted: 1. The limiting random vector X in this definition is really just a laceholder for the distribution function F (x); there needs be no real random vector X which has the limiting distribution function. In fact, it is often customary to relace X with the limiting distribution function F (x) in the 2
3 definition, i.e., to write d F (x) as N ; if, say, F (x) is a multivariate standard normal, then we would write d N(0, I), where the condition N is usually not stated, but imlied. d 2. If F (x), then we can use the c.d.f. F (x) to aroximate robabilies for XN if N is large, i.e., Z Pr{ A} = A df (x), where the symbol = A means that the difference in the left- and right-hand sides goes to zero as N increases. A 3. We would like to construct statistics so that their limiting distributions are of convenient (tabulated) form, e.g., normal or chi-squared distribution. This often involves standardizing the statistics by subtracting off their exectations and dividing by their standard deviations (or the matrix equivalent). 4. If lim N F N (x) is discontinuous, it may not be a distribution function, which is why the qualification at all oints where F (x) is continuous has to be aended to the definition of convergence in distribution. To make this more concrete, suose µ N 0, 1, N so that ³ F N (x) = Φ Nx 0 if x<0 1 2 if x =0 1 if x>0. Obviously here lim N F N (x) is not right-continuous, so we would never use it to aroximate robabilities for (since it isn t a c.d.f.). Instead, a reasonable aroximating function for F N (x) would be F (x) 1{x 0}. 3
4 where 1{A} is the indicator function of the statement A, i.e., it equals one if A is true and is zero otherwise. The function F (x) is the c.d.f. of a degenerate random variable X which equals zero with robability one, which is a retty good aroximation for F N (x) when N is large for this roblem. The last examle convergence of the c.d.f. F N (x) to the c.d.f. of a degenerate ( nonrandom ) random variable is a secial case of the other key convergence concet for asymtotic theory, convergence in robability. Following Ruud s text, we define convergence in robability of the sequence of random vectors to the (nonrandom) vector x, denoted or if i.e., lim = x N x as N d x, F N (x) 1{0 x} whenever all comonents of x are nonzero. (The symbol lim stands for the term robabilty limit.) We can generalize this definition of convergence in robability of to a constant x to ermit to converge in robability to a nondegenerate random vector X by secifying that X means that is that X d 0. An equivalent definition of convergence in robability, which is the definition usually seen in textbooks, x if, for any number ε>0, Pr{ x >ε} 0 4
5 as N. Thus, converges in robability to x if all the robability mass for eventually ends u in an ε-neighborhood of x, no matter how small ε is. Showing the equivalence of these two definitions of robability limit is a routine exercise in robability theory and real analysis. A stronger form of convergence of to x, which can often be used to show convergence in robability, is quadratic mean convergence. We say qm X n x as N if E[ x 2 ] 0. If this condition can be shown, then it follows that converges in robability to x because of Markov s Inequality: If Z is a nonnegative (scalar) random variable, i.e., Pr{Z 0} =1, then for any K>0. Pr{Z >K} E(Z) K The roof of this inequality is worth remembering; it is based uon a decomosition of E(Z) (which may be infinite) as E(Z) = Z 0 Z K 0 zdf Z (z) zdf Z (z)+ Z K (z K)dF Z (z)+ Z K KdF Z (z). Since the first two terms in this sum are nonnegative, it follows that E(Z) Z K KdF Z (z) =K Pr{Z >K}, from which Markov s inequality immediately follows. A better-known variant of this inequality, Tchebysheff s Inequality, takesz =(X E(X)) 2 for a scalar random variable X (with finite second moment) and K = ε 2 for any ε>0, from which it follows that Pr{ X E(X) >ε} =Pr{(X E(X)) 2 >ε 2 } Var(X) ε 2, since Var(X) =E(Z) for this examle. A similar consequence of Markov s inequality is that convergence in quadratic mean imlies convergence in robability: if E[ x 2 ] 0, 5
6 then for any ε>0. Pr{ x >ε} E[ x 2 ] ε 2 0 There is another, stronger version of convergence of a random variable to a limit, known as almost sure convergence (or convergence almost everywhere or convergence with robability one), which is denoted a.s. x. Rather than give the detailed definition of this form of convergence which is best left to a more advanced course it is worth ointing out that it is indeed stronger than convergence in robability (which is sometimes called weak convergence for that reason), but that, for the usual uroses of asymtotic theory, the weaker concet is sufficient. Limit Theorems With the concets of convergence in robability and distribution, we can more recisely state how a samle average is aroximately equal to its exectation, or how its standardized version is aroximately normal. The first result, on convergence in robability, is Weak Law of Large Numbers (WLLN): If 1 N P N i=1 X i is a samle average of scalar, i.i.d. random variables {X i } N i=1 which have E(X i)=μ and Var(X i )=σ 2 finite, then μ = E(Xi ). Proof: By the usual calculations for means and variances of sums of i.i.d. random variables, E( )=μ, Var( )= σ2 N = E( μ) 2, which tends to zero as N. By definition, XN tends to μ in quadratic mean, and thus in robability. Insection of the roof of the WLLN suggests a more general result for averages of random variables that are not i.i.d. namely, that E( ) 0 if 1 N 2 NX i=1 j=1 NX Cov(X i,x j ) 0. 6
7 This general condition can be secialized to yield other LLNs for heterogeneous (non-constant variances) or deendent (nonzero covariances) data. Other laws of large numbers exist which relax conditions on the existence of second moments, and which show almost sure convergence (and are known as strong laws of large numbers ). The next main result, on aroximate normality of samle averages, again alies to i.i.d. data: Central Limit Theorem (Lindeberg-Levy): If XN 1 N P N i=1 X i is a samle average of scalar, i.i.d. random variables {X i } N i=1 which have E(X i)=μ and Var(X i )=σ 2 finite, then Z N E( ) Var( XN ) = µ XN μ N σ d N(0, 1). The roof of this result is a bit more involved, requiring maniulation of characteristic functions, and will not be resented here. The result of this CLT is often rewritten as N( XN μ) d N(0,σ 2 ), which oints out that converges to its mean μ at exactly the rate N increases, so that the roduct eventually balances out to yield a random variable with a normal distribution. Another way to rewrite the consequence of the CLT is µ A N μ, σ2, N where the symbol A means is aroximately distributed as (or is asymtotically distributed as ). It is very imortant here to make the distinction between the limiting distribution of the standardized mean Z N, which is a standard normal, and the asymtotic distribution of the (non-standardized) samle mean, which is actually a sequence of aroximating normal distributions with variances that shrink to zero at seed 1/N. In short, a limiting distribution cannot deend uon N, which has assed to its (infinite) limit, while an asymtotic distribution can involve the samle size N. Like the Weak Law of Large Numbers, the Lindeberg-Levy Central Limit Theorem admits a number of different generalizations (often named after their authors) which relax the assumtion of i.i.d. data while 7
8 imosing various stronger restrictions on the existence of moments or the tail behavior of the true data distributions. To this oint, the WLLN and CLT aly only to scalar random variables {X i }; to extend them to random vectors {X i }, we can use the intriguingly-named Cramér-Wald device, which is really a theorem that states that if the scalar random variable Y N λ 0 converges in distribution to Y λ 0 X for any fixed vector λ having the same dimension as thatis,if λ 0 d λ 0 X for any λ then the vector converges in distribution to X, d X. This result immediately yields a multivariate version of the Lindeberg-Levy CLT: a samle mean of i.i.d. random vectors {X i } with E(X i )=μ and V (X i )=Σ has N( μ) d N (0, Σ). And, since convergence in robability is a secial case of convergence in distribution, the Cramér-Wald device also immediately yields a multivariate version of the WLLN. Slutsky Theorems To extend the limit theorems for samle averages to statistics which are functions of samle averages, asymtotic theory uses smoothness roerties of those functions (i.e., continuity and differentiability) to aroximate those functions by olynomials, usually constant or linear functions. The simlest of these aroximation results is the continuity theorem, which states that robability limits share an imortant roerty of ordinary limits namely, the robability limit of a continuous function is the value of that function evaluated at the robability limit: 8
9 Continuity Theorem: If x0 and the function g(x) is continous at x = x 0, then g( ) g(x 0 ). Proof: For any ε>0, continuity of the function g(x) at x 0 imlies that there is some δ>0 so that x 0 δ imlies that g( ) g(x 0 ) ε. Thus Pr{ g( ) g(x 0 ) ε} Pr{ x 0 δ}. But the robability on the right-hand side converges to one because x0, so the robability on the left-hand side does, too. With the continuity theorem, it should be clear that robability limits are more conveniently maniulated than, say, mathematical exectations, since, for examle, if ˆθ ȲN is an estimator of the ratio of the means of Y i and X i, θ 0 E(Y i) E(X i ) μ Y μ X, then ˆθ N θ0 as long as μ X 6=0, even though E(ˆθ N ) 6= θ 0 in general. Another key aroximation result is Slutsky s Theorem, which states that the sum (or roduct) of a random vector that converges in robability and another that converges in distribution itself converges in distribution: Slutsky s Theorem: If the random vectors and Y N havethesamelength,andif x0 and Y N d Y, then XN + Y N d x0 + Y and X 0 N Y N d x 0 0 Y. The tyical alication of Slutsky s theorem is to estimators that can be reresented in terms of a linear aroximation involving some statistics that converge either in robability or distribution. To be more concrete, suose the (scalar) estimator ˆθ N can be decomosed as N(ˆθN θ 0 )=X 0 NY N + Z N, where x0, d Y N N(0, Σ), and Z N 0. 9
10 This decomosition is often obtained by a first-order Taylor s series exansion of ˆθ N, and the comonents, Y N, and Z N are samle averages to which a LLN or CLT can be alied; an examle is the so-called delta method discussed below. Under these conditions, Slutsky s theorem imlies that N(ˆθN θ 0 ) d N(0, x 0 0Σx 0 ). Both the continuity theorem and Slutsky s theorem are secial cases of the continuous maing theorem, which says that convergence in distribution, like convergence in robability, interchanges with continuous functions, as long as they are continuous everywhere: Continuous Maing Theorem: If d X and the function g(x) is continous for all x, then g(xn ) d g(x). The continuity requirement for g(x) can be relaxed to hold only on the suort of X if needed. The continuous maing theorem can be used to get aroximations for the distributions of test statistics based uon asymtotically-normal estimators; for examle, if Z N N(ˆθ N θ 0 ) d N (0, Σ), then T Z 0 NZ N = N(ˆθ N θ 0 ) 0 (ˆθ N θ 0 ) d χ 2, where dim{ˆθ N }. The final aroximation result discussed here, which is an alication of Slutsky s theorem, is the so-called delta method, which states that continuously-differentiable functions of asymtotically-normal statistics are themselves asymtotically normal: The Delta Method : If ˆθ N is a random vector which is asymtotically normal, with N(ˆθ N θ 0 ) d N (0, Σ) for some θ 0, and if g(θ) is continuously differentiable at θ = θ 0 with Jacobian matrix G 0 g(θ 0) θ 0 10
11 that has full row rank, then N(g(ˆθN ) g(θ 0 )) d N 0, G 0 ΣG 0 0. Proof: First suose that g(θ) is scalar-valued. Since ˆθ N converges in robability to θ 0 (by Slutsky s Theorem), and since g(θ)/ θ 0 is continuous at θ 0, the Mean Value Theorem of differential calculus imlies that, g(ˆθ N )=g(θ 0 )+ g(θ N ) θ 0 (ˆθ N θ 0 ), for some value of θ N between ˆθ N and θ 0 (with arbitrarily high robability). Since ˆθ N that θ N θ 0, so Thus, by Slutsky s Theorem, g(θ N ) θ 0 G 0 g(θ 0) θ 0. θ0,itfollows N(g(ˆθ N ) g(θ 0 )) = g(θ N ) θ 0. N(ˆθ N θ 0 ) d G 0 N(0, Σ) = N(0, G 0 ΣG 0 0). To extend this argument to the case where g(ˆθ N ) is vector-valued, use the same argument to show that any (scalar) linear combination λ 0 g(ˆθ) has the asymtotic distribution N(λ 0 g(ˆθ N ) λ 0 g(θ 0 )) d N(0, λ 0 G 0 ΣG 0 0λ), from which the result follows from the Cramér-Wald device. 11
Elements of Asymptotic Theory. James L. Powell Department of Economics University of California, Berkeley
Elements of Asymtotic Theory James L. Powell Deartment of Economics University of California, Berkeley Objectives of Asymtotic Theory While exact results are available for, say, the distribution of the
More informationEconometrics I. September, Part I. Department of Economics Stanford University
Econometrics I Deartment of Economics Stanfor University Setember, 2008 Part I Samling an Data Poulation an Samle. ineenent an ientical samling. (i.i..) Samling with relacement. aroximates samling without
More informationChapter 7: Special Distributions
This chater first resents some imortant distributions, and then develos the largesamle distribution theory which is crucial in estimation and statistical inference Discrete distributions The Bernoulli
More informationLECTURE 7 NOTES. x n. d x if. E [g(x n )] E [g(x)]
LECTURE 7 NOTES 1. Convergence of random variables. Before delving into the large samle roerties of the MLE, we review some concets from large samle theory. 1. Convergence in robability: x n x if, for
More informationLecture 2: Consistency of M-estimators
Lecture 2: Instructor: Deartment of Economics Stanford University Preared by Wenbo Zhou, Renmin University References Takeshi Amemiya, 1985, Advanced Econometrics, Harvard University Press Newey and McFadden,
More informationEconomics 583: Econometric Theory I A Primer on Asymptotics
Economics 583: Econometric Theory I A Primer on Asymptotics Eric Zivot January 14, 2013 The two main concepts in asymptotic theory that we will use are Consistency Asymptotic Normality Intuition consistency:
More informationMaximum Likelihood Asymptotic Theory. Eduardo Rossi University of Pavia
Maximum Likelihood Asymtotic Theory Eduardo Rossi University of Pavia Slutsky s Theorem, Cramer s Theorem Slutsky s Theorem Let {X N } be a random sequence converging in robability to a constant a, and
More informationConsistency and asymptotic normality
Consistency an asymtotic normality Class notes for Econ 842 Robert e Jong March 2006 1 Stochastic convergence The asymtotic theory of minimization estimators relies on various theorems from mathematical
More informationReview of Classical Least Squares. James L. Powell Department of Economics University of California, Berkeley
Review of Classical Least Squares James L. Powell Department of Economics University of California, Berkeley The Classical Linear Model The object of least squares regression methods is to model and estimate
More information7. Introduction to Large Sample Theory
7. Introuction to Large Samle Theory Hayashi. 88-97/109-133 Avance Econometrics I, Autumn 2010, Large-Samle Theory 1 Introuction We looke at finite-samle roerties of the OLS estimator an its associate
More informationStat 5101 Lecture Slides: Deck 7 Asymptotics, also called Large Sample Theory. Charles J. Geyer School of Statistics University of Minnesota
Stat 5101 Lecture Slides: Deck 7 Asymptotics, also called Large Sample Theory Charles J. Geyer School of Statistics University of Minnesota 1 Asymptotic Approximation The last big subject in probability
More information1 Probability Spaces and Random Variables
1 Probability Saces and Random Variables 1.1 Probability saces Ω: samle sace consisting of elementary events (or samle oints). F : the set of events P: robability 1.2 Kolmogorov s axioms Definition 1.2.1
More informationElementary Analysis in Q p
Elementary Analysis in Q Hannah Hutter, May Szedlák, Phili Wirth November 17, 2011 This reort follows very closely the book of Svetlana Katok 1. 1 Sequences and Series In this section we will see some
More informationAsymptotic theory for linear regression and IV estimation
Asymtotic theory for linear regression and IV estimation Jean-Marie Dufour McGill University First version: November 20 Revised: December 20 his version: December 20 Comiled: December 3, 20, : his work
More informationGeneral Linear Model Introduction, Classes of Linear models and Estimation
Stat 740 General Linear Model Introduction, Classes of Linear models and Estimation An aim of scientific enquiry: To describe or to discover relationshis among events (variables) in the controlled (laboratory)
More informationMATH 2710: NOTES FOR ANALYSIS
MATH 270: NOTES FOR ANALYSIS The main ideas we will learn from analysis center around the idea of a limit. Limits occurs in several settings. We will start with finite limits of sequences, then cover infinite
More information1 Extremum Estimators
FINC 9311-21 Financial Econometrics Handout Jialin Yu 1 Extremum Estimators Let θ 0 be a vector of k 1 unknown arameters. Extremum estimators: estimators obtained by maximizing or minimizing some objective
More informationEC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix)
1 EC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix) Taisuke Otsu London School of Economics Summer 2018 A.1. Summation operator (Wooldridge, App. A.1) 2 3 Summation operator For
More informationLecture 3 Consistency of Extremum Estimators 1
Lecture 3 Consistency of Extremum Estimators 1 This lecture shows how one can obtain consistency of extremum estimators. It also shows how one can find the robability limit of extremum estimators in cases
More informationChapter 6. Convergence. Probability Theory. Four different convergence concepts. Four different convergence concepts. Convergence in probability
Probability Theory Chapter 6 Convergence Four different convergence concepts Let X 1, X 2, be a sequence of (usually dependent) random variables Definition 1.1. X n converges almost surely (a.s.), or with
More informationLecture 2: Review of Probability
Lecture 2: Review of Probability Zheng Tian Contents 1 Random Variables and Probability Distributions 2 1.1 Defining probabilities and random variables..................... 2 1.2 Probability distributions................................
More informationLecture 4: Law of Large Number and Central Limit Theorem
ECE 645: Estimation Theory Sring 2015 Instructor: Prof. Stanley H. Chan Lecture 4: Law of Large Number and Central Limit Theorem (LaTeX reared by Jing Li) March 31, 2015 This lecture note is based on ECE
More informationUse of Transformations and the Repeated Statement in PROC GLM in SAS Ed Stanek
Use of Transformations and the Reeated Statement in PROC GLM in SAS Ed Stanek Introduction We describe how the Reeated Statement in PROC GLM in SAS transforms the data to rovide tests of hyotheses of interest.
More informationMATH 829: Introduction to Data Mining and Analysis Consistency of Linear Regression
1/9 MATH 829: Introduction to Data Mining and Analysis Consistency of Linear Regression Dominique Guillot Deartments of Mathematical Sciences University of Delaware February 15, 2016 Distribution of regression
More informationAsymptotic Theory. L. Magee revised January 21, 2013
Asymptotic Theory L. Magee revised January 21, 2013 1 Convergence 1.1 Definitions Let a n to refer to a random variable that is a function of n random variables. Convergence in Probability The scalar a
More informationAsymptotic Statistics-III. Changliang Zou
Asymptotic Statistics-III Changliang Zou The multivariate central limit theorem Theorem (Multivariate CLT for iid case) Let X i be iid random p-vectors with mean µ and and covariance matrix Σ. Then n (
More informationResearch Note REGRESSION ANALYSIS IN MARKOV CHAIN * A. Y. ALAMUTI AND M. R. MESHKANI **
Iranian Journal of Science & Technology, Transaction A, Vol 3, No A3 Printed in The Islamic Reublic of Iran, 26 Shiraz University Research Note REGRESSION ANALYSIS IN MARKOV HAIN * A Y ALAMUTI AND M R
More informationLecture 6. 2 Recurrence/transience, harmonic functions and martingales
Lecture 6 Classification of states We have shown that all states of an irreducible countable state Markov chain must of the same tye. This gives rise to the following classification. Definition. [Classification
More informationHENSEL S LEMMA KEITH CONRAD
HENSEL S LEMMA KEITH CONRAD 1. Introduction In the -adic integers, congruences are aroximations: for a and b in Z, a b mod n is the same as a b 1/ n. Turning information modulo one ower of into similar
More informationEstimation of the large covariance matrix with two-step monotone missing data
Estimation of the large covariance matrix with two-ste monotone missing data Masashi Hyodo, Nobumichi Shutoh 2, Takashi Seo, and Tatjana Pavlenko 3 Deartment of Mathematical Information Science, Tokyo
More informationSums of independent random variables
3 Sums of indeendent random variables This lecture collects a number of estimates for sums of indeendent random variables with values in a Banach sace E. We concentrate on sums of the form N γ nx n, where
More informationNotes on Instrumental Variables Methods
Notes on Instrumental Variables Methods Michele Pellizzari IGIER-Bocconi, IZA and frdb 1 The Instrumental Variable Estimator Instrumental variable estimation is the classical solution to the roblem of
More informationChapter 3. GMM: Selected Topics
Chater 3. GMM: Selected oics Contents Otimal Instruments. he issue of interest..............................2 Otimal Instruments under the i:i:d: assumtion..............2. he basic result............................2.2
More informationEconomics 241B Review of Limit Theorems for Sequences of Random Variables
Economics 241B Review of Limit Theorems for Sequences of Random Variables Convergence in Distribution The previous de nitions of convergence focus on the outcome sequences of a random variable. Convergence
More informationLecture Notes 5 Convergence and Limit Theorems. Convergence with Probability 1. Convergence in Mean Square. Convergence in Probability, WLLN
Lecture Notes 5 Convergence and Limit Theorems Motivation Convergence with Probability Convergence in Mean Square Convergence in Probability, WLLN Convergence in Distribution, CLT EE 278: Convergence and
More informationAdvanced Econometrics II (Part 1)
Advanced Econometrics II (Part 1) Dr. Mehdi Hosseinkouchack Goethe University Frankfurt Summer 2016 osseinkouchack (Goethe University Frankfurt) Review Slides Summer 2016 1 / 22 Distribution For simlicity,
More informationSDS : Theoretical Statistics
SDS 384 11: Theoretical Statistics Lecture 1: Introduction Purnamrita Sarkar Department of Statistics and Data Science The University of Texas at Austin https://psarkar.github.io/teaching Manegerial Stuff
More informationMoment Generating Function. STAT/MTHE 353: 5 Moment Generating Functions and Multivariate Normal Distribution
Moment Generating Function STAT/MTHE 353: 5 Moment Generating Functions and Multivariate Normal Distribution T. Linder Queen s University Winter 07 Definition Let X (X,...,X n ) T be a random vector and
More information4. Score normalization technical details We now discuss the technical details of the score normalization method.
SMT SCORING SYSTEM This document describes the scoring system for the Stanford Math Tournament We begin by giving an overview of the changes to scoring and a non-technical descrition of the scoring rules
More informationEstimating Time-Series Models
Estimating ime-series Models he Box-Jenkins methodology for tting a model to a scalar time series fx t g consists of ve stes:. Decide on the order of di erencing d that is needed to roduce a stationary
More information8 STOCHASTIC PROCESSES
8 STOCHASTIC PROCESSES The word stochastic is derived from the Greek στoχαστικoς, meaning to aim at a target. Stochastic rocesses involve state which changes in a random way. A Markov rocess is a articular
More informationProblem Set 2 Solution
Problem Set 2 Solution Aril 22nd, 29 by Yang. More Generalized Slutsky heorem [Simle and Abstract] su L n γ, β n Lγ, β = su L n γ, β n Lγ, β n + Lγ, β n Lγ, β su L n γ, β n Lγ, β n + su Lγ, β n Lγ, β su
More informationStat 5101 Notes: Algorithms
Stat 5101 Notes: Algorithms Charles J. Geyer January 22, 2016 Contents 1 Calculating an Expectation or a Probability 3 1.1 From a PMF........................... 3 1.2 From a PDF...........................
More informationECE 534 Information Theory - Midterm 2
ECE 534 Information Theory - Midterm Nov.4, 009. 3:30-4:45 in LH03. You will be given the full class time: 75 minutes. Use it wisely! Many of the roblems have short answers; try to find shortcuts. You
More informationAsymptotically Optimal Simulation Allocation under Dependent Sampling
Asymtotically Otimal Simulation Allocation under Deendent Samling Xiaoing Xiong The Robert H. Smith School of Business, University of Maryland, College Park, MD 20742-1815, USA, xiaoingx@yahoo.com Sandee
More information5601 Notes: The Sandwich Estimator
560 Notes: The Sandwich Estimator Charles J. Geyer December 6, 2003 Contents Maximum Likelihood Estimation 2. Likelihood for One Observation................... 2.2 Likelihood for Many IID Observations...............
More informationAdaptive estimation with change detection for streaming data
Adative estimation with change detection for streaming data A thesis resented for the degree of Doctor of Philosohy of the University of London and the Diloma of Imerial College by Dean Adam Bodenham Deartment
More informationReview of Probability Theory II
Review of Probability Theory II January 9-3, 008 Exectation If the samle sace Ω = {ω, ω,...} is countable and g is a real-valued function, then we define the exected value or the exectation of a function
More informationNumerical Linear Algebra
Numerical Linear Algebra Numerous alications in statistics, articularly in the fitting of linear models. Notation and conventions: Elements of a matrix A are denoted by a ij, where i indexes the rows and
More informationProof: We follow thearoach develoed in [4]. We adot a useful but non-intuitive notion of time; a bin with z balls at time t receives its next ball at
A Scaling Result for Exlosive Processes M. Mitzenmacher Λ J. Sencer We consider the following balls and bins model, as described in [, 4]. Balls are sequentially thrown into bins so that the robability
More informationarxiv: v1 [physics.data-an] 26 Oct 2012
Constraints on Yield Parameters in Extended Maximum Likelihood Fits Till Moritz Karbach a, Maximilian Schlu b a TU Dortmund, Germany, moritz.karbach@cern.ch b TU Dortmund, Germany, maximilian.schlu@cern.ch
More informationColin Cameron: Asymptotic Theory for OLS
Colin Cameron: Asymtotic Theory for OLS. OLS Estimator Proerties an Samling Schemes.. A Roama Consier the OLS moel with just one regressor y i = βx i + u i. The OLS estimator b β = ³ P P i= x iy i canbewrittenas
More informationConvex Analysis and Economic Theory Winter 2018
Division of the Humanities and Social Sciences Ec 181 KC Border Conve Analysis and Economic Theory Winter 2018 Toic 16: Fenchel conjugates 16.1 Conjugate functions Recall from Proosition 14.1.1 that is
More informationEstimation, Inference, and Hypothesis Testing
Chapter 2 Estimation, Inference, and Hypothesis Testing Note: The primary reference for these notes is Ch. 7 and 8 of Casella & Berger 2. This text may be challenging if new to this topic and Ch. 7 of
More informationMATHEMATICAL MODELLING OF THE WIRELESS COMMUNICATION NETWORK
Comuter Modelling and ew Technologies, 5, Vol.9, o., 3-39 Transort and Telecommunication Institute, Lomonosov, LV-9, Riga, Latvia MATHEMATICAL MODELLIG OF THE WIRELESS COMMUICATIO ETWORK M. KOPEETSK Deartment
More informationReal Analysis 1 Fall Homework 3. a n.
eal Analysis Fall 06 Homework 3. Let and consider the measure sace N, P, µ, where µ is counting measure. That is, if N, then µ equals the number of elements in if is finite; µ = otherwise. One usually
More informationMath Camp II. Calculus. Yiqing Xu. August 27, 2014 MIT
Math Camp II Calculus Yiqing Xu MIT August 27, 2014 1 Sequence and Limit 2 Derivatives 3 OLS Asymptotics 4 Integrals Sequence Definition A sequence {y n } = {y 1, y 2, y 3,..., y n } is an ordered set
More informationRobustness of classifiers to uniform l p and Gaussian noise Supplementary material
Robustness of classifiers to uniform l and Gaussian noise Sulementary material Jean-Yves Franceschi Ecole Normale Suérieure de Lyon LIP UMR 5668 Omar Fawzi Ecole Normale Suérieure de Lyon LIP UMR 5668
More informationSystem Reliability Estimation and Confidence Regions from Subsystem and Full System Tests
009 American Control Conference Hyatt Regency Riverfront, St. Louis, MO, USA June 0-, 009 FrB4. System Reliability Estimation and Confidence Regions from Subsystem and Full System Tests James C. Sall Abstract
More informationGraduate Econometrics I: Asymptotic Theory
Graduate Econometrics I: Asymptotic Theory Yves Dominicy Université libre de Bruxelles Solvay Brussels School of Economics and Management ECARES Yves Dominicy Graduate Econometrics I: Asymptotic Theory
More informationON THE LEAST SIGNIFICANT p ADIC DIGITS OF CERTAIN LUCAS NUMBERS
#A13 INTEGERS 14 (014) ON THE LEAST SIGNIFICANT ADIC DIGITS OF CERTAIN LUCAS NUMBERS Tamás Lengyel Deartment of Mathematics, Occidental College, Los Angeles, California lengyel@oxy.edu Received: 6/13/13,
More informationLarge Sample Theory. Consider a sequence of random variables Z 1, Z 2,..., Z n. Convergence in probability: Z n
Large Sample Theory In statistics, we are interested in the properties of particular random variables (or estimators ), which are functions of our data. In ymptotic analysis, we focus on describing the
More informationLecture 21: Convergence of transformations and generating a random variable
Lecture 21: Convergence of transformations and generating a random variable If Z n converges to Z in some sense, we often need to check whether h(z n ) converges to h(z ) in the same sense. Continuous
More informationCS145: Probability & Computing
CS45: Probability & Computing Lecture 5: Concentration Inequalities, Law of Large Numbers, Central Limit Theorem Instructor: Eli Upfal Brown University Computer Science Figure credits: Bertsekas & Tsitsiklis,
More informationIMPROVED BOUNDS IN THE SCALED ENFLO TYPE INEQUALITY FOR BANACH SPACES
IMPROVED BOUNDS IN THE SCALED ENFLO TYPE INEQUALITY FOR BANACH SPACES OHAD GILADI AND ASSAF NAOR Abstract. It is shown that if (, ) is a Banach sace with Rademacher tye 1 then for every n N there exists
More informationLecture 3 January 16
Stats 3b: Theory of Statistics Winter 28 Lecture 3 January 6 Lecturer: Yu Bai/John Duchi Scribe: Shuangning Li, Theodor Misiakiewicz Warning: these notes may contain factual errors Reading: VDV Chater
More informationLimiting Distributions
Limiting Distributions We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the
More informationCHAPTER 2: SMOOTH MAPS. 1. Introduction In this chapter we introduce smooth maps between manifolds, and some important
CHAPTER 2: SMOOTH MAPS DAVID GLICKENSTEIN 1. Introduction In this chater we introduce smooth mas between manifolds, and some imortant concets. De nition 1. A function f : M! R k is a smooth function if
More informationThe non-stochastic multi-armed bandit problem
Submitted for journal ublication. The non-stochastic multi-armed bandit roblem Peter Auer Institute for Theoretical Comuter Science Graz University of Technology A-8010 Graz (Austria) auer@igi.tu-graz.ac.at
More informationOn the asymptotic sizes of subset Anderson-Rubin and Lagrange multiplier tests in linear instrumental variables regression
On the asymtotic sizes of subset Anderson-Rubin and Lagrange multilier tests in linear instrumental variables regression Patrik Guggenberger Frank Kleibergeny Sohocles Mavroeidisz Linchun Chen\ June 22
More informationLimiting Distributions
We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the two fundamental results
More informationCMSC 425: Lecture 4 Geometry and Geometric Programming
CMSC 425: Lecture 4 Geometry and Geometric Programming Geometry for Game Programming and Grahics: For the next few lectures, we will discuss some of the basic elements of geometry. There are many areas
More informationOn split sample and randomized confidence intervals for binomial proportions
On slit samle and randomized confidence intervals for binomial roortions Måns Thulin Deartment of Mathematics, Usala University arxiv:1402.6536v1 [stat.me] 26 Feb 2014 Abstract Slit samle methods have
More informationCONVOLVED SUBSAMPLING ESTIMATION WITH APPLICATIONS TO BLOCK BOOTSTRAP
Submitted to the Annals of Statistics arxiv: arxiv:1706.07237 CONVOLVED SUBSAMPLING ESTIMATION WITH APPLICATIONS TO BLOCK BOOTSTRAP By Johannes Tewes, Dimitris N. Politis and Daniel J. Nordman Ruhr-Universität
More informationIf g is also continuous and strictly increasing on J, we may apply the strictly increasing inverse function g 1 to this inequality to get
18:2 1/24/2 TOPIC. Inequalities; measures of spread. This lecture explores the implications of Jensen s inequality for g-means in general, and for harmonic, geometric, arithmetic, and related means in
More informationElementary theory of L p spaces
CHAPTER 3 Elementary theory of L saces 3.1 Convexity. Jensen, Hölder, Minkowski inequality. We begin with two definitions. A set A R d is said to be convex if, for any x 0, x 1 2 A x = x 0 + (x 1 x 0 )
More informationProbability Lecture III (August, 2006)
robability Lecture III (August, 2006) 1 Some roperties of Random Vectors and Matrices We generalize univariate notions in this section. Definition 1 Let U = U ij k l, a matrix of random variables. Suppose
More informationNonparametric estimation of Exact consumer surplus with endogeneity in price
Nonarametric estimation of Exact consumer surlus with endogeneity in rice Anne Vanhems February 7, 2009 Abstract This aer deals with nonarametric estimation of variation of exact consumer surlus with endogenous
More informationEE 508 Lecture 13. Statistical Characterization of Filter Characteristics
EE 508 Lecture 3 Statistical Characterization of Filter Characteristics Comonents used to build filters are not recisely redictable L C Temerature Variations Manufacturing Variations Aging Model variations
More informationSECTION 5: FIBRATIONS AND HOMOTOPY FIBERS
SECTION 5: FIBRATIONS AND HOMOTOPY FIBERS In this section we will introduce two imortant classes of mas of saces, namely the Hurewicz fibrations and the more general Serre fibrations, which are both obtained
More informationA Primer on Asymptotics
A Primer on Asymptotics Eric Zivot Department of Economics University of Washington September 30, 2003 Revised: October 7, 2009 Introduction The two main concepts in asymptotic theory covered in these
More informationNotes on Random Vectors and Multivariate Normal
MATH 590 Spring 06 Notes on Random Vectors and Multivariate Normal Properties of Random Vectors If X,, X n are random variables, then X = X,, X n ) is a random vector, with the cumulative distribution
More information#A64 INTEGERS 18 (2018) APPLYING MODULAR ARITHMETIC TO DIOPHANTINE EQUATIONS
#A64 INTEGERS 18 (2018) APPLYING MODULAR ARITHMETIC TO DIOPHANTINE EQUATIONS Ramy F. Taki ElDin Physics and Engineering Mathematics Deartment, Faculty of Engineering, Ain Shams University, Cairo, Egyt
More information16.2. Infinite Series. Introduction. Prerequisites. Learning Outcomes
Infinite Series 6. Introduction We extend the concet of a finite series, met in section, to the situation in which the number of terms increase without bound. We define what is meant by an infinite series
More information3.4 Design Methods for Fractional Delay Allpass Filters
Chater 3. Fractional Delay Filters 15 3.4 Design Methods for Fractional Delay Allass Filters Above we have studied the design of FIR filters for fractional delay aroximation. ow we show how recursive or
More informationOn-Line Appendix. Matching on the Estimated Propensity Score (Abadie and Imbens, 2015)
On-Line Aendix Matching on the Estimated Proensity Score Abadie and Imbens, 205 Alberto Abadie and Guido W. Imbens Current version: August 0, 205 The first art of this aendix contains additional roofs.
More information16.2. Infinite Series. Introduction. Prerequisites. Learning Outcomes
Infinite Series 6.2 Introduction We extend the concet of a finite series, met in Section 6., to the situation in which the number of terms increase without bound. We define what is meant by an infinite
More informationProbability Theory and Statistics. Peter Jochumzen
Probability Theory and Statistics Peter Jochumzen April 18, 2016 Contents 1 Probability Theory And Statistics 3 1.1 Experiment, Outcome and Event................................ 3 1.2 Probability............................................
More informationLecture 1: August 28
36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 1: August 28 Our broad goal for the first few lectures is to try to understand the behaviour of sums of independent random
More informationBest approximation by linear combinations of characteristic functions of half-spaces
Best aroximation by linear combinations of characteristic functions of half-saces Paul C. Kainen Deartment of Mathematics Georgetown University Washington, D.C. 20057-1233, USA Věra Kůrková Institute of
More informationChapter 1 Fundamentals
Chater Fundamentals. Overview of Thermodynamics Industrial Revolution brought in large scale automation of many tedious tasks which were earlier being erformed through manual or animal labour. Inventors
More informationAdditive results for the generalized Drazin inverse in a Banach algebra
Additive results for the generalized Drazin inverse in a Banach algebra Dragana S. Cvetković-Ilić Dragan S. Djordjević and Yimin Wei* Abstract In this aer we investigate additive roerties of the generalized
More informationFinal Examination Statistics 200C. T. Ferguson June 11, 2009
Final Examination Statistics 00C T. Ferguson June, 009. (a) Define: X n converges in probability to X. (b) Define: X m converges in quadratic mean to X. (c) Show that if X n converges in quadratic mean
More informationEstimation of Separable Representations in Psychophysical Experiments
Estimation of Searable Reresentations in Psychohysical Exeriments Michele Bernasconi (mbernasconi@eco.uninsubria.it) Christine Choirat (cchoirat@eco.uninsubria.it) Raffaello Seri (rseri@eco.uninsubria.it)
More informationThe Fekete Szegő theorem with splitting conditions: Part I
ACTA ARITHMETICA XCIII.2 (2000) The Fekete Szegő theorem with slitting conditions: Part I by Robert Rumely (Athens, GA) A classical theorem of Fekete and Szegő [4] says that if E is a comact set in the
More informationOn the rate of convergence in the martingale central limit theorem
Bernoulli 192), 2013, 633 645 DOI: 10.3150/12-BEJ417 arxiv:1103.5050v2 [math.pr] 21 Mar 2013 On the rate of convergence in the martingale central limit theorem JEAN-CHRISTOPHE MOURRAT Ecole olytechnique
More informationPositive decomposition of transfer functions with multiple poles
Positive decomosition of transfer functions with multile oles Béla Nagy 1, Máté Matolcsi 2, and Márta Szilvási 1 Deartment of Analysis, Technical University of Budaest (BME), H-1111, Budaest, Egry J. u.
More informationOn Wald-Type Optimal Stopping for Brownian Motion
J Al Probab Vol 34, No 1, 1997, (66-73) Prerint Ser No 1, 1994, Math Inst Aarhus On Wald-Tye Otimal Stoing for Brownian Motion S RAVRSN and PSKIR The solution is resented to all otimal stoing roblems of
More informationPETER J. GRABNER AND ARNOLD KNOPFMACHER
ARITHMETIC AND METRIC PROPERTIES OF -ADIC ENGEL SERIES EXPANSIONS PETER J. GRABNER AND ARNOLD KNOPFMACHER Abstract. We derive a characterization of rational numbers in terms of their unique -adic Engel
More informationBrownian Motion and Random Prime Factorization
Brownian Motion and Random Prime Factorization Kendrick Tang June 4, 202 Contents Introduction 2 2 Brownian Motion 2 2. Develoing Brownian Motion.................... 2 2.. Measure Saces and Borel Sigma-Algebras.........
More information