The Uniform Weak Law of Large Numbers and the Consistency of M-Estimators of Cross-Section and Time Series Models
|
|
- Myron Jacobs
- 5 years ago
- Views:
Transcription
1 The Uniform Weak Law of Large Numbers and the Consistency of M-Estimators of Cross-Section and Time Series Models Herman J. Bierens Pennsylvania State University September 16, The uniform weak law of large numbers In econometrics we often have to deal with sample means of random functions. A random function is a function that is a random variable for each fixed value of its argument. In crosssection econometrics random functions usually take the form of a function g(z,2) of a random vector Z and a non-random vector 2. For example, consider a Logit model: P[Y j ' y X j ] ' y % (1 & y)exp(&α & βt X j ) 1 % exp(&α & β T X j ), y ' 0,1, where Y j 0 {0,1} is the dependent variable and X j 0ú k is a vector of explanatory variables. Denoting Z j ' (Y j,x T j ) T, and given a random sample {Z 1,Z 2,...,Z n }, the log-likelihood function involved takes the form ' n j'1 g(z j,θ), where g(z j,θ) ' ln Y j % (1 & Y j )exp(&α & β T X j ) & ln 1 % exp(&α & β T X j ) ' Y j (α % β T X j ) & ln 1 % exp(α % β T X j ),whereθ ' (α,β T ) T. (1) For such functions we can extend the weak law of large numbers for i.i.d. random variables to a Uniform Weak Law of Large Numbers (UWLLN): Theorem 1: Let Z j, j = 1,..,n, be a random sample from a k-variate distribution. Let g(z,2) be a Borel measurable function on Ζ Θ, where Ζ dú k is a Borel set such that P[ 0 Ζ] ' 1, and Θ is a compact subset of ú m, such that for each z 0 Ζ, g(z,θ) is a continuous function on Θ. Furthermore, let 1
2 E[ *g(z j,θ)*] <4. (2) Then plim n64 *(1/n)' n j'1 g(z j,θ) & E[g(Z 1,θ)]* ' 0. Note that subsets of Euclidean spaces are compact if and only if they are closed and bounded. See, for example, Bierens (2004), Appendix II, Theorem II.2. The original proof of the stronger result *(1/n)' n j'1 g(z j,θ) & E[g(Z 1,θ)]* 60a.s., was given in the seminal paper of Jennrich (1969). This proof is explained in detail in Bierens (2004, Appendix to Chapter 6). The condition that the random vectors Z j are i.i.d. can be relaxed, because the result in Theorem 1 also holds for strictly stationary time series processes with a vanishing memory: Definition 1: A (vector) time series process X t 0ú k is strictly stationary if for arbitrary integers m 1 < m 2 <...< m n the joint distribution of X T t&m 1,...,X T does not depend on the time index t. t&m n T Definition 2: A (vector) time series process X t 0ú k has a vanishing memory if all the sets in the remote F-algebra ö &4 ' _ t σ {X t&j } 4 j'0 have either probability zero or one. Note that if the X t s are independent then by Kolmogorov s zero-one law the time series X t has a vanishing memory. It has been shown in Bierens (2004, Theorem 7.4) that Theorem 2: If X t 0ú k is a strictly stationary time series process with vanishing memory, and E[ X t ] < 4, then plim n64 (1/n)' n t'1 X t ' E[X 1 ]. I will use this result to prove the following more general version of Theorem 1. To be able to generalize the UWLLN to the time series case where the random functions involved depend on 2
3 the entire past of the time series rather than on a finite dimensional vector of variables, I will reformulate and prove Theorem 1 under slightly different moment conditions. Theorem 3: Let 0ú k be a strictly stationary vector time series process with a vanishing memory, 1 defined on a common probability space {Ω,ö,P}. Let g(z,2) be a Borel measurable real function on Ζ Θ 0, where Ζ dú k is a Borel set such that P[ 0 Ζ] ' 1, and Θ 0 is an open subset of ú m, such that for each z 0 Ζ, g(z,θ) is a continuous function on Θ 0. Furthermore, let 1 be a compact subset of Θ 0. Finally, assume that for each θ ( 0 Θ there exists an arbitrary small * > 0, possibly depending on θ (, such that E sup θ&θ( # δ g(z 1,θ) < 4, E inf θ&θ ( # δ g(z 1,θ) > &4. (3) Then plim n64 *(1/n)' n j'1 g(z j,θ) & E[g(Z 1,θ)]* ' 0. Proof: Observe from condition (3) that for each θ 0 Θ, E[g(Z 1,θ)] is well-defined. Actually, due to the compactness of 1, (3) implies (2) [Exercise: Why?], so that the latter is a weaker condition than (3). Moreover, it follows from condition (3), the continuity of g(z,θ) in 2, and the dominated convergence theorem, that lim δ90 E sup θ&θ( # δ g(z 1,θ) & inf θ&θ ( # δ g(z 1,θ) ' 0, (4) pointwise in θ ( 0 Θ. Therefore, for an arbitrary g > 0 and each θ ( 0 Θ we can choose a positive number δ(θ (,g) such that, with we have N(θ ( g) ' {θ 0 Θ 0 : θ & θ ( < δ(θ (,g)}, (5) 0 # E sup θ 0 N(θ( g) g(z 1,θ) & inf θ 0 N(θ ( g) g(z 1,θ) < g. (6) Next, observe that the sets (5) are open, so that is an open covering of Θ. ^θ( 0Θ N(θ ( g) Then by the compactness of Θ there exists a finite sub-covering of Θ: 1 Which includes the case that the s are i.i.d. 3
4 Θ d ^ K i'1 N(θ i g), (7) where K and the vectors θ i 0 Θ depend on g. Using the easy inequality for each θ i 0 Θ, sup x *f(x)* # sup x f(x) % inf x f(x), it is not hard to verify that sup θ0n(θi g) *(1/n)'n t'1 g(,θ) & E[g(Z 1,θ)* # 2 (1/n)' n t'1 sup θ0n(θ i g) g(,θ) & E[sup θ0n(θ i g) g(z 1,θ)] % 2 (1/n)' n t'1 inf θ0n(θ i g) g(,θ) & E[inf θ0n(θ i g) g(z 1,θ)] (8) % 2 E[sup θ0n(θi g) g(z 1,θ)] & E[inf θ0n(θ i g) g(z 1,θ)]. It follows from Theorem 2 that the first two terms at the right-hand side of (8) converge in probability to zero, and from (6) that the last term is less than 2.g. Hence, *(1/n)' n t'1 g(,θ) & E[g(Z 1,θ)* # max 1#i#K sup θ0n(θi g) *(1/n)'n t'1 g(,θ) & E[g(Z 1,θ)* (9) # R n (g) % 2.g, where plim n64 R n (g) ' 0. Theorem 3 follows now straightforwardly from (9). Q.E.D. In time series econometrics there are quite a few cases where we need a UWLLN for functions g(.,θ) depending on &j for all j $ 0. In that case g(.,θ) takes a more general form as a random function: Definition 3: Let {S,ö,P} be the probability space. A random function f(2) on a subset 1 of a Euclidean space is a mapping f(ω,θ): Ω Θ 6úsuch that for each Borel set B in ú and each 2 0 1, {ω 0 Ω: f(ω,θ) 0 B} 0ö. 4
5 Definition 4: A random function f(θ) on a subset Θ of a Euclidean space is almost surely continuous on Θ if there exists a set A with probability one such that for each continuous in θ 0 Θ. ω 0 A, f(ω,θ) is For example, let 0ú MA(1) process]: be a stationary Gaussian moving average process or order 1 [alias an ' U t & α 0 U t&1, α 0 <1,U t - i.i.d. N(0,σ 2 0 ). (10) Then backwards substitution of U t ' α 0 U t&1 % yields U t ' ' 4 j'0 αj 0 &j, hence '&' 4 j'1 αj 0 &1 % U t (11) Thus, denoting ö t ' σ(u t,u t&1,u t&2,...), the distribution of conditional on ö t&1 is normal with conditional expectation &' 4 j'1 αj 0 &1 and conditional variance σ 2 0. If the s were observable for all t # n, a version of the log-likelihood would take the form ' n j'1 g t (θ), where g t (θ) '& 1 2σ 2 '4 j'0 αj 2 1 &j & 2 ln(σ2 ) & ln 2π, θ ' (α,σ 2 ) T, (12) is a random function. In that case we need to reformulate Theorem 3 as follows. Theorem 4: Let ö t ' σ(u t,u t&1,u t&2,...), where U t is a time series process with vanishing memory. Let g t (θ) be a sequence of a.s. continuous random function on an open subset Θ 0 of a Euclidean space, and let Θbe a compact subset of Θ 0. If for each θ ( 0 Θ there exists an arbitrarily small * > 0 such that (a) g t (θ ( ), sup θ&θ( #δ g t (θ) and inf θ&θ are measurable and strictly stationary, ( #δ g t (θ) ö t (b) E[sup θ&θ( #δ g 1 (θ)] < 4, E[inf θ&θ ( #δ g 1 (θ)] > &4, then plim n64 (1/n)' n t'1 g t (θ) & E[g 1 (θ)] ' 0. 5
6 2. Consistency of M-estimators Theorems 3 and 4 are important tools for proving consistency of parameter estimators. A large class of estimators are obtained by maximizing or minimizing an objective function of the form (1/n)' n t'1 g t (θ), for example maximum likelihood estimators or nonlinear least squares estimators. These estimators are called M-estimators (where the M indicates that the estimator is obtained by Maximizing or Minimizing a Mean of random functions). Suppose that the conditions of Theorem 4 are satisfied, and that the parameter vector of interest is θ 0 ' argmax θ0θ E[g 1 (θ)]. (13) Note that "argmax" is a short-hand notation for the argument for which the function involved is maximal. Then it seems a natural choice to use ˆθ ' argmax θ0θ (1/n)' n t'1 g t (θ) (14) as an estimator of θ 0. Indeed, under some mild conditions the estimator involved is consistent: Theorem 5: (Consistency of M-estimators) Let ˆθ = argmax θ0θ ˆQ(θ) and θ 0 = argmax θ0θ Q(θ), where ˆQ(θ) = (1/n)' n t'1 g t (θ) and Q(θ) ' E[ ˆQ(θ)] ' E[g 1 (θ)]. If 2 0 is unique then under the conditions of Theorem 4, plim n64ˆθ ' θ 0. Proof: Since a continuous function on a compact set takes its maximum value in this set [see, for example, Bierens (2004, Appendix II )], it follows that ˆθ 0 Θ and θ 0 0 Θ. Moreover, by the same result it follows from the continuity of every g > 0 for which the set {θ 0 Θ: θ&θ 0 $ g} is non-empty, Q(θ) and the uniqueness of 2 0 that for Q(θ 0 )>, 2θ&θ0 2$g Q(θ) (15) [Exercise: Why?] Now by the definition of 2 0, 6
7 0 # Q(θ 0 ) & Q(ˆθ) ' Q(θ 0 ) & ˆQ(θ 0 ) % ˆQ(θ 0 ) & Q(ˆθ) # Q(θ 0 ) & ˆQ(θ 0 ) % ˆQ(ˆθ) & Q(ˆθ) # 2. * ˆQ(θ) & Q(θ)*, (16) and it follows from Theorem 4 that the right-hand side of (16) converges in probability to zero. Thus: plim n64 Q(ˆθ) ' Q(θ 0 ). (17) Moreover, (15) implies that for arbitrary g > 0 there exists a * > 0 such that Q(θ 0 ) & Q(ˆθ) $ δ if 2ˆθ & θ 0 2$g, hence P 2ˆθ & θ 0 2 > g # P Q(θ 0 ) & Q(ˆθ) $ δ. (18) Combining (17) and (18), the theorem under review follows. Q.E.D. It is easy to verify that Theorem 5 carries over to the "argmin" case. References Bierens, H. J. (2004): Introduction to the Mathematical and Statistical Foundations of Econometrics, Cambridge University Press, Cambridge, U.K. Jennrich, R. I. (1969): Asymptotic Properties of Non-Linear Least Squares Estimators, Annals of Mathematical Statistics 40,
The properties of L p -GMM estimators
The properties of L p -GMM estimators Robert de Jong and Chirok Han Michigan State University February 2000 Abstract This paper considers Generalized Method of Moment-type estimators for which a criterion
More informationIntroduction to Estimation Methods for Time Series models Lecture 2
Introduction to Estimation Methods for Time Series models Lecture 2 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 2 SNS Pisa 1 / 21 Estimators:
More informationClosest Moment Estimation under General Conditions
Closest Moment Estimation under General Conditions Chirok Han and Robert de Jong January 28, 2002 Abstract This paper considers Closest Moment (CM) estimation with a general distance function, and avoids
More informationClosest Moment Estimation under General Conditions
Closest Moment Estimation under General Conditions Chirok Han Victoria University of Wellington New Zealand Robert de Jong Ohio State University U.S.A October, 2003 Abstract This paper considers Closest
More informationf(x θ)dx with respect to θ. Assuming certain smoothness conditions concern differentiating under the integral the integral sign, we first obtain
0.1. INTRODUCTION 1 0.1 Introduction R. A. Fisher, a pioneer in the development of mathematical statistics, introduced a measure of the amount of information contained in an observaton from f(x θ). Fisher
More informationNotes, March 4, 2013, R. Dudley Maximum likelihood estimation: actual or supposed
18.466 Notes, March 4, 2013, R. Dudley Maximum likelihood estimation: actual or supposed 1. MLEs in exponential families Let f(x,θ) for x X and θ Θ be a likelihood function, that is, for present purposes,
More informationInference in non-linear time series
Intro LS MLE Other Erik Lindström Centre for Mathematical Sciences Lund University LU/LTH & DTU Intro LS MLE Other General Properties Popular estimatiors Overview Introduction General Properties Estimators
More informationInformation Criteria and Model Selection
Information Criteria and Model Selection Herman J. Bierens Pennsylvania State University March 12, 2006 1. Introduction Let L n (k) be the maximum likelihood of a model with k parameters based on a sample
More informationGraduate Econometrics I: Maximum Likelihood I
Graduate Econometrics I: Maximum Likelihood I Yves Dominicy Université libre de Bruxelles Solvay Brussels School of Economics and Management ECARES Yves Dominicy Graduate Econometrics I: Maximum Likelihood
More informationFall, 2007 Nonlinear Econometrics. Theory: Consistency for Extremum Estimators. Modeling: Probit, Logit, and Other Links.
14.385 Fall, 2007 Nonlinear Econometrics Lecture 2. Theory: Consistency for Extremum Estimators Modeling: Probit, Logit, and Other Links. 1 Example: Binary Choice Models. The latent outcome is defined
More informationProofs for Large Sample Properties of Generalized Method of Moments Estimators
Proofs for Large Sample Properties of Generalized Method of Moments Estimators Lars Peter Hansen University of Chicago March 8, 2012 1 Introduction Econometrica did not publish many of the proofs in my
More informationEstimation of Dynamic Regression Models
University of Pavia 2007 Estimation of Dynamic Regression Models Eduardo Rossi University of Pavia Factorization of the density DGP: D t (x t χ t 1, d t ; Ψ) x t represent all the variables in the economy.
More informationUniversity of Pavia. M Estimators. Eduardo Rossi
University of Pavia M Estimators Eduardo Rossi Criterion Function A basic unifying notion is that most econometric estimators are defined as the minimizers of certain functions constructed from the sample
More informationEC 521 MATHEMATICAL METHODS FOR ECONOMICS. Lecture 1: Preliminaries
EC 521 MATHEMATICAL METHODS FOR ECONOMICS Lecture 1: Preliminaries Murat YILMAZ Boğaziçi University In this lecture we provide some basic facts from both Linear Algebra and Real Analysis, which are going
More informationis a Borel subset of S Θ for each c R (Bertsekas and Shreve, 1978, Proposition 7.36) This always holds in practical applications.
Stat 811 Lecture Notes The Wald Consistency Theorem Charles J. Geyer April 9, 01 1 Analyticity Assumptions Let { f θ : θ Θ } be a family of subprobability densities 1 with respect to a measure µ on a measurable
More informationAnalogy Principle. Asymptotic Theory Part II. James J. Heckman University of Chicago. Econ 312 This draft, April 5, 2006
Analogy Principle Asymptotic Theory Part II James J. Heckman University of Chicago Econ 312 This draft, April 5, 2006 Consider four methods: 1. Maximum Likelihood Estimation (MLE) 2. (Nonlinear) Least
More informationChapter 3: Maximum Likelihood Theory
Chapter 3: Maximum Likelihood Theory Florian Pelgrin HEC September-December, 2010 Florian Pelgrin (HEC) Maximum Likelihood Theory September-December, 2010 1 / 40 1 Introduction Example 2 Maximum likelihood
More informationChapter 4: Asymptotic Properties of the MLE
Chapter 4: Asymptotic Properties of the MLE Daniel O. Scharfstein 09/19/13 1 / 1 Maximum Likelihood Maximum likelihood is the most powerful tool for estimation. In this part of the course, we will consider
More informationPARAMETER ESTIMATION OF CHIRP SIGNALS IN PRESENCE OF STATIONARY NOISE
Statistica Sinica 8(008), 87-0 PARAMETER ESTIMATION OF CHIRP SIGNALS IN PRESENCE OF STATIONARY NOISE Debasis Kundu and Swagata Nandi Indian Institute of Technology, Kanpur and Indian Statistical Institute
More informationSection 8: Asymptotic Properties of the MLE
2 Section 8: Asymptotic Properties of the MLE In this part of the course, we will consider the asymptotic properties of the maximum likelihood estimator. In particular, we will study issues of consistency,
More informationTheoretical Statistics. Lecture 1.
1. Organizational issues. 2. Overview. 3. Stochastic convergence. Theoretical Statistics. Lecture 1. eter Bartlett 1 Organizational Issues Lectures: Tue/Thu 11am 12:30pm, 332 Evans. eter Bartlett. bartlett@stat.
More informationEstimation theory. Parametric estimation. Properties of estimators. Minimum variance estimator. Cramer-Rao bound. Maximum likelihood estimators
Estimation theory Parametric estimation Properties of estimators Minimum variance estimator Cramer-Rao bound Maximum likelihood estimators Confidence intervals Bayesian estimation 1 Random Variables Let
More information1 Stat 605. Homework I. Due Feb. 1, 2011
The first part is homework which you need to turn in. The second part is exercises that will not be graded, but you need to turn it in together with the take-home final exam. 1 Stat 605. Homework I. Due
More informationFinancial Econometrics and Volatility Models Estimation of Stochastic Volatility Models
Financial Econometrics and Volatility Models Estimation of Stochastic Volatility Models Eric Zivot April 26, 2010 Outline Likehood of SV Models Survey of Estimation Techniques for SV Models GMM Estimation
More informationARMA MODELS Herman J. Bierens Pennsylvania State University February 23, 2009
1. Introduction Given a covariance stationary process µ ' E[ ], the Wold decomposition states that where U t ARMA MODELS Herman J. Bierens Pennsylvania State University February 23, 2009 with vanishing
More informationBerge s Maximum Theorem
Berge s Maximum Theorem References: Acemoglu, Appendix A.6 Stokey-Lucas-Prescott, Section 3.3 Ok, Sections E.1-E.3 Claude Berge, Topological Spaces (1963), Chapter 6 Berge s Maximum Theorem So far, we
More informationSTA205 Probability: Week 8 R. Wolpert
INFINITE COIN-TOSS AND THE LAWS OF LARGE NUMBERS The traditional interpretation of the probability of an event E is its asymptotic frequency: the limit as n of the fraction of n repeated, similar, and
More informationParameter Estimation
Parameter Estimation Consider a sample of observations on a random variable Y. his generates random variables: (y 1, y 2,, y ). A random sample is a sample (y 1, y 2,, y ) where the random variables y
More informationChapter 3: Unbiased Estimation Lecture 22: UMVUE and the method of using a sufficient and complete statistic
Chapter 3: Unbiased Estimation Lecture 22: UMVUE and the method of using a sufficient and complete statistic Unbiased estimation Unbiased or asymptotically unbiased estimation plays an important role in
More informationTesting Algebraic Hypotheses
Testing Algebraic Hypotheses Mathias Drton Department of Statistics University of Chicago 1 / 18 Example: Factor analysis Multivariate normal model based on conditional independence given hidden variable:
More informationChapter 1: A Brief Review of Maximum Likelihood, GMM, and Numerical Tools. Joan Llull. Microeconometrics IDEA PhD Program
Chapter 1: A Brief Review of Maximum Likelihood, GMM, and Numerical Tools Joan Llull Microeconometrics IDEA PhD Program Maximum Likelihood Chapter 1. A Brief Review of Maximum Likelihood, GMM, and Numerical
More informationECE531 Lecture 10b: Maximum Likelihood Estimation
ECE531 Lecture 10b: Maximum Likelihood Estimation D. Richard Brown III Worcester Polytechnic Institute 05-Apr-2011 Worcester Polytechnic Institute D. Richard Brown III 05-Apr-2011 1 / 23 Introduction So
More informationOn Parameter Estimation of Two Dimensional Chirp Signal
On Parameter Estimation of Two Dimensional Chirp Signal Ananya Lahiri & Debasis Kundu, & Amit Mitra Abstract Two dimensional (-D) chirp signals occur in different areas of image processing. In this paper,
More informationThe generalized method of moments
Robert M. Kunst robert.kunst@univie.ac.at University of Vienna and Institute for Advanced Studies Vienna February 2008 Based on the book Generalized Method of Moments by Alastair R. Hall (2005), Oxford
More informationA strong consistency proof for heteroscedasticity and autocorrelation consistent covariance matrix estimators
A strong consistency proof for heteroscedasticity and autocorrelation consistent covariance matrix estimators Robert M. de Jong Department of Economics Michigan State University 215 Marshall Hall East
More informationLocation Multiplicative Error Model. Asymptotic Inference and Empirical Analysis
: Asymptotic Inference and Empirical Analysis Qian Li Department of Mathematics and Statistics University of Missouri-Kansas City ql35d@mail.umkc.edu October 29, 2015 Outline of Topics Introduction GARCH
More informationFundamental Inequalities, Convergence and the Optional Stopping Theorem for Continuous-Time Martingales
Fundamental Inequalities, Convergence and the Optional Stopping Theorem for Continuous-Time Martingales Prakash Balachandran Department of Mathematics Duke University April 2, 2008 1 Review of Discrete-Time
More informationHomework Assignment #2 for Prob-Stats, Fall 2018 Due date: Monday, October 22, 2018
Homework Assignment #2 for Prob-Stats, Fall 2018 Due date: Monday, October 22, 2018 Topics: consistent estimators; sub-σ-fields and partial observations; Doob s theorem about sub-σ-field measurability;
More informationIntroduction to the Mathematical and Statistical Foundations of Econometrics Herman J. Bierens Pennsylvania State University
Introduction to the Mathematical and Statistical Foundations of Econometrics 1 Herman J. Bierens Pennsylvania State University November 13, 2003 Revised: March 15, 2004 2 Contents Preface Chapter 1: Probability
More informationLimiting Distributions
Limiting Distributions We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the
More informationTheoretical Statistics. Lecture 12.
Theoretical Statistics. Lecture 12. Peter Bartlett Uniform laws of large numbers: Bounding Rademacher complexity. 1. Metric entropy. 2. Canonical Rademacher and Gaussian processes 1 Recall: Covering numbers
More information7 Influence Functions
7 Influence Functions The influence function is used to approximate the standard error of a plug-in estimator. The formal definition is as follows. 7.1 Definition. The Gâteaux derivative of T at F in the
More informationESTIMATION OF NONLINEAR BERKSON-TYPE MEASUREMENT ERROR MODELS
Statistica Sinica 13(2003), 1201-1210 ESTIMATION OF NONLINEAR BERKSON-TYPE MEASUREMENT ERROR MODELS Liqun Wang University of Manitoba Abstract: This paper studies a minimum distance moment estimator for
More informationMaximum Likelihood Estimation
Maximum Likelihood Estimation Assume X P θ, θ Θ, with joint pdf (or pmf) f(x θ). Suppose we observe X = x. The Likelihood function is L(θ x) = f(x θ) as a function of θ (with the data x held fixed). The
More informationChapter 3. Point Estimation. 3.1 Introduction
Chapter 3 Point Estimation Let (Ω, A, P θ ), P θ P = {P θ θ Θ}be probability space, X 1, X 2,..., X n : (Ω, A) (IR k, B k ) random variables (X, B X ) sample space γ : Θ IR k measurable function, i.e.
More informationLimiting Distributions
We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the two fundamental results
More informationLecture 21: Convergence of transformations and generating a random variable
Lecture 21: Convergence of transformations and generating a random variable If Z n converges to Z in some sense, we often need to check whether h(z n ) converges to h(z ) in the same sense. Continuous
More informationPARAMETER ESTIMATION OF CHIRP SIGNALS IN PRESENCE OF STATIONARY NOISE
PARAMETER ESTIMATION OF CHIRP SIGNALS IN PRESENCE OF STATIONARY NOISE DEBASIS KUNDU AND SWAGATA NANDI Abstract. The problem of parameter estimation of the chirp signals in presence of stationary noise
More informationST5215: Advanced Statistical Theory
Department of Statistics & Applied Probability Wednesday, October 19, 2011 Lecture 17: UMVUE and the first method of derivation Estimable parameters Let ϑ be a parameter in the family P. If there exists
More informationLarge Sample Theory. Consider a sequence of random variables Z 1, Z 2,..., Z n. Convergence in probability: Z n
Large Sample Theory In statistics, we are interested in the properties of particular random variables (or estimators ), which are functions of our data. In ymptotic analysis, we focus on describing the
More informationComparison of inferential methods in partially identified models in terms of error in coverage probability
Comparison of inferential methods in partially identified models in terms of error in coverage probability Federico A. Bugni Department of Economics Duke University federico.bugni@duke.edu. September 22,
More informationAsymptotics of minimax stochastic programs
Asymptotics of minimax stochastic programs Alexander Shapiro Abstract. We discuss in this paper asymptotics of the sample average approximation (SAA) of the optimal value of a minimax stochastic programming
More informationWorking Paper No Maximum score type estimators
Warsaw School of Economics Institute of Econometrics Department of Applied Econometrics Department of Applied Econometrics Working Papers Warsaw School of Economics Al. iepodleglosci 64 02-554 Warszawa,
More informationPh.D. Qualifying Exam Friday Saturday, January 6 7, 2017
Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017 Put your solution to each problem on a separate sheet of paper. Problem 1. (5106) Let X 1, X 2,, X n be a sequence of i.i.d. observations from a
More informationDA Freedman Notes on the MLE Fall 2003
DA Freedman Notes on the MLE Fall 2003 The object here is to provide a sketch of the theory of the MLE. Rigorous presentations can be found in the references cited below. Calculus. Let f be a smooth, scalar
More informationFollow links for Class Use and other Permissions. For more information send to:
COPYRIGH NOICE: Kenneth J. Singleton: Empirical Dynamic Asset Pricing is published by Princeton University Press and copyrighted, 00, by Princeton University Press. All rights reserved. No part of this
More informationSums of exponentials of random walks
Sums of exponentials of random walks Robert de Jong Ohio State University August 27, 2009 Abstract This paper shows that the sum of the exponential of an oscillating random walk converges in distribution,
More informationA GENERAL THEOREM ON APPROXIMATE MAXIMUM LIKELIHOOD ESTIMATION. Miljenko Huzak University of Zagreb,Croatia
GLASNIK MATEMATIČKI Vol. 36(56)(2001), 139 153 A GENERAL THEOREM ON APPROXIMATE MAXIMUM LIKELIHOOD ESTIMATION Miljenko Huzak University of Zagreb,Croatia Abstract. In this paper a version of the general
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationPropagating terraces and the dynamics of front-like solutions of reaction-diffusion equations on R
Propagating terraces and the dynamics of front-like solutions of reaction-diffusion equations on R P. Poláčik School of Mathematics, University of Minnesota Minneapolis, MN 55455 Abstract We consider semilinear
More informationIntroduction to Empirical Processes and Semiparametric Inference Lecture 12: Glivenko-Cantelli and Donsker Results
Introduction to Empirical Processes and Semiparametric Inference Lecture 12: Glivenko-Cantelli and Donsker Results Michael R. Kosorok, Ph.D. Professor and Chair of Biostatistics Professor of Statistics
More informationSTAT 7032 Probability Spring Wlodek Bryc
STAT 7032 Probability Spring 2018 Wlodek Bryc Created: Friday, Jan 2, 2014 Revised for Spring 2018 Printed: January 9, 2018 File: Grad-Prob-2018.TEX Department of Mathematical Sciences, University of Cincinnati,
More informationStatistics 612: L p spaces, metrics on spaces of probabilites, and connections to estimation
Statistics 62: L p spaces, metrics on spaces of probabilites, and connections to estimation Moulinath Banerjee December 6, 2006 L p spaces and Hilbert spaces We first formally define L p spaces. Consider
More informationProbability and Measure
Chapter 4 Probability and Measure 4.1 Introduction In this chapter we will examine probability theory from the measure theoretic perspective. The realisation that measure theory is the foundation of probability
More informationOptimization. The value x is called a maximizer of f and is written argmax X f. g(λx + (1 λ)y) < λg(x) + (1 λ)g(y) 0 < λ < 1; x, y X.
Optimization Background: Problem: given a function f(x) defined on X, find x such that f(x ) f(x) for all x X. The value x is called a maximizer of f and is written argmax X f. In general, argmax X f may
More informationThe Logit Model: Estimation, Testing and Interpretation
The Logit Model: Estimation, Testing and Interpretation Herman J. Bierens October 25, 2008 1 Introduction to maximum likelihood estimation 1.1 The likelihood function Consider a random sample Y 1,...,
More information5 Birkhoff s Ergodic Theorem
5 Birkhoff s Ergodic Theorem Birkhoff s Ergodic Theorem extends the validity of Kolmogorov s strong law to the class of stationary sequences of random variables. Stationary sequences occur naturally even
More informationLecture I: Asymptotics for large GUE random matrices
Lecture I: Asymptotics for large GUE random matrices Steen Thorbjørnsen, University of Aarhus andom Matrices Definition. Let (Ω, F, P) be a probability space and let n be a positive integer. Then a random
More informationAnalysis Comprehensive Exam, January 2011 Instructions: Do as many problems as you can. You should attempt to answer completely some questions in both
Analysis Comprehensive Exam, January 2011 Instructions: Do as many problems as you can. You should attempt to answer completely some questions in both real and complex analysis. You have 3 hours. Real
More informationλ(x + 1)f g (x) > θ 0
Stat 8111 Final Exam December 16 Eleven students took the exam, the scores were 92, 78, 4 in the 5 s, 1 in the 4 s, 1 in the 3 s and 3 in the 2 s. 1. i) Let X 1, X 2,..., X n be iid each Bernoulli(θ) where
More information8 Laws of large numbers
8 Laws of large numbers 8.1 Introduction We first start with the idea of standardizing a random variable. Let X be a random variable with mean µ and variance σ 2. Then Z = (X µ)/σ will be a random variable
More informationElements of Probability Theory
Chapter 5 Elements of Probability Theory The purpose of this chapter is to summarize some important concepts and results in probability theory. Of particular interest to us are the limit theorems which
More informationOn the Set of Limit Points of Normed Sums of Geometrically Weighted I.I.D. Bounded Random Variables
On the Set of Limit Points of Normed Sums of Geometrically Weighted I.I.D. Bounded Random Variables Deli Li 1, Yongcheng Qi, and Andrew Rosalsky 3 1 Department of Mathematical Sciences, Lakehead University,
More informationLecture 6 Basic Probability
Lecture 6: Basic Probability 1 of 17 Course: Theory of Probability I Term: Fall 2013 Instructor: Gordan Zitkovic Lecture 6 Basic Probability Probability spaces A mathematical setup behind a probabilistic
More informationEstimates for probabilities of independent events and infinite series
Estimates for probabilities of independent events and infinite series Jürgen Grahl and Shahar evo September 9, 06 arxiv:609.0894v [math.pr] 8 Sep 06 Abstract This paper deals with finite or infinite sequences
More informationLecture 2. We now introduce some fundamental tools in martingale theory, which are useful in controlling the fluctuation of martingales.
Lecture 2 1 Martingales We now introduce some fundamental tools in martingale theory, which are useful in controlling the fluctuation of martingales. 1.1 Doob s inequality We have the following maximal
More information4.6 Montel's Theorem. Robert Oeckl CA NOTES 7 17/11/2009 1
Robert Oeckl CA NOTES 7 17/11/2009 1 4.6 Montel's Theorem Let X be a topological space. We denote by C(X) the set of complex valued continuous functions on X. Denition 4.26. A topological space is called
More information17. Convergence of Random Variables
7. Convergence of Random Variables In elementary mathematics courses (such as Calculus) one speaks of the convergence of functions: f n : R R, then lim f n = f if lim f n (x) = f(x) for all x in R. This
More informationOn Least Absolute Deviation Estimators For One Dimensional Chirp Model
On Least Absolute Deviation Estimators For One Dimensional Chirp Model Ananya Lahiri & Debasis Kundu, & Amit Mitra Abstract It is well known that the least absolute deviation (LAD) estimators are more
More informationLecture 2: Consistency of M-estimators
Lecture 2: Instructor: Deartment of Economics Stanford University Preared by Wenbo Zhou, Renmin University References Takeshi Amemiya, 1985, Advanced Econometrics, Harvard University Press Newey and McFadden,
More informationNonlinear GMM. Eric Zivot. Winter, 2013
Nonlinear GMM Eric Zivot Winter, 2013 Nonlinear GMM estimation occurs when the GMM moment conditions g(w θ) arenonlinearfunctionsofthe model parameters θ The moment conditions g(w θ) may be nonlinear functions
More information4 Sums of Independent Random Variables
4 Sums of Independent Random Variables Standing Assumptions: Assume throughout this section that (,F,P) is a fixed probability space and that X 1, X 2, X 3,... are independent real-valued random variables
More informationIntroduction to Stochastic processes
Università di Pavia Introduction to Stochastic processes Eduardo Rossi Stochastic Process Stochastic Process: A stochastic process is an ordered sequence of random variables defined on a probability space
More informationStrong Consistency of Set-Valued Frechet Sample Mean in Metric Spaces
Strong Consistency of Set-Valued Frechet Sample Mean in Metric Spaces Cedric E. Ginestet Department of Mathematics and Statistics Boston University JSM 2013 The Frechet Mean Barycentre as Average Given
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationSection 8.2. Asymptotic normality
30 Section 8.2. Asymptotic normality We assume that X n =(X 1,...,X n ), where the X i s are i.i.d. with common density p(x; θ 0 ) P= {p(x; θ) :θ Θ}. We assume that θ 0 is identified in the sense that
More informationON THE UNIFORM ASYMPTOTIC VALIDITY OF SUBSAMPLING AND THE BOOTSTRAP. Joseph P. Romano Azeem M. Shaikh
ON THE UNIFORM ASYMPTOTIC VALIDITY OF SUBSAMPLING AND THE BOOTSTRAP By Joseph P. Romano Azeem M. Shaikh Technical Report No. 2010-03 April 2010 Department of Statistics STANFORD UNIVERSITY Stanford, California
More informationSTAT 512 sp 2018 Summary Sheet
STAT 5 sp 08 Summary Sheet Karl B. Gregory Spring 08. Transformations of a random variable Let X be a rv with support X and let g be a function mapping X to Y with inverse mapping g (A = {x X : g(x A}
More informationPROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS
PROBABILITY: LIMIT THEOREMS II, SPRING 218. HOMEWORK PROBLEMS PROF. YURI BAKHTIN Instructions. You are allowed to work on solutions in groups, but you are required to write up solutions on your own. Please
More informationSeparate Appendix to: Semi-Nonparametric Competing Risks Analysis of Recidivism
Separate Appendix to: Semi-Nonparametric Competing Risks Analysis of Recidivism Herman J. Bierens a and Jose R. Carvalho b a Department of Economics,Pennsylvania State University, University Park, PA 1682
More informationEconometrics I, Estimation
Econometrics I, Estimation Department of Economics Stanford University September, 2008 Part I Parameter, Estimator, Estimate A parametric is a feature of the population. An estimator is a function of the
More informationGreene, Econometric Analysis (6th ed, 2008)
EC771: Econometrics, Spring 2010 Greene, Econometric Analysis (6th ed, 2008) Chapter 17: Maximum Likelihood Estimation The preferred estimator in a wide variety of econometric settings is that derived
More informationStochastic process for macro
Stochastic process for macro Tianxiao Zheng SAIF 1. Stochastic process The state of a system {X t } evolves probabilistically in time. The joint probability distribution is given by Pr(X t1, t 1 ; X t2,
More informationFall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
More informationEC9A0: Pre-sessional Advanced Mathematics Course. Lecture Notes: Unconstrained Optimisation By Pablo F. Beker 1
EC9A0: Pre-sessional Advanced Mathematics Course Lecture Notes: Unconstrained Optimisation By Pablo F. Beker 1 1 Infimum and Supremum Definition 1. Fix a set Y R. A number α R is an upper bound of Y if
More informationMLE and GMM. Li Zhao, SJTU. Spring, Li Zhao MLE and GMM 1 / 22
MLE and GMM Li Zhao, SJTU Spring, 2017 Li Zhao MLE and GMM 1 / 22 Outline 1 MLE 2 GMM 3 Binary Choice Models Li Zhao MLE and GMM 2 / 22 Maximum Likelihood Estimation - Introduction For a linear model y
More informationExercises from other sources REAL NUMBERS 2,...,
Exercises from other sources REAL NUMBERS 1. Find the supremum and infimum of the following sets: a) {1, b) c) 12, 13, 14, }, { 1 3, 4 9, 13 27, 40 } 81,, { 2, 2 + 2, 2 + 2 + } 2,..., d) {n N : n 2 < 10},
More informationEstimation, Inference, and Hypothesis Testing
Chapter 2 Estimation, Inference, and Hypothesis Testing Note: The primary reference for these notes is Ch. 7 and 8 of Casella & Berger 2. This text may be challenging if new to this topic and Ch. 7 of
More informationHartogs Theorem: separate analyticity implies joint Paul Garrett garrett/
(February 9, 25) Hartogs Theorem: separate analyticity implies joint Paul Garrett garrett@math.umn.edu http://www.math.umn.edu/ garrett/ (The present proof of this old result roughly follows the proof
More informationThe Arzelà-Ascoli Theorem
John Nachbar Washington University March 27, 2016 The Arzelà-Ascoli Theorem The Arzelà-Ascoli Theorem gives sufficient conditions for compactness in certain function spaces. Among other things, it helps
More information