Estimation of Dynamic Regression Models
|
|
- Angela Bradley
- 5 years ago
- Views:
Transcription
1 University of Pavia 2007 Estimation of Dynamic Regression Models Eduardo Rossi University of Pavia
2 Factorization of the density DGP: D t (x t χ t 1, d t ; Ψ) x t represent all the variables in the economy. The econometric analysis will focus on explaining a subset of variables, y t, in terms of the history of the system and of a contemporaneous subset z t, treated as given. Treating z t as given is motivated by the assumption that z t causes y t. What is causality in econometrics? Eduardo Rossi c - Macroeconometria 07 2
3 Factorization of the density y t,z t. y t subset of x t ; z t subset of x t. w t variables in x t that / y t,z t. D = D w,y,z = D w y,z D y z D z (1) ( ) presence (absence) of contemporaneous casuality ( ) presence (absence) of simultaneous relations Eduardo Rossi c - Macroeconometria 07 3
4 Factorization of the density If z t y t w t y t The factor represents D y z completely represents the stochastic mechanism generating y t. Note that y t w t is allowed, and no restriction on the relationships between w t and z t is required, for this to be true. Eduardo Rossi c - Macroeconometria 07 4
5 Factorization of the density Denote by W t 1 = σ (w t 1,w t 2,...) Y t 1 = σ (y t 1,y t 2,...) Z t 1 = σ (z t 1,z t 2,...) Assume there exists a partition of θ into two subvectors θ 1 Θ 1 and θ 2 Θ 2, such that Θ = Θ 1 Θ 2 and D w y,z = D w y,z (w t y t,z t, W t 1, Y t 1, Z t 1 ;d t, θ 2 ) (2) D y z = D y z (y t z t, Y t 1, Z t 1 ;d t, θ 1 ) (3) D z = D z (z t W t 1, Y t 1, Z t 1 ;d t, θ 2 ) (4) D w y,z and D z must not depend on θ 1. D y z must not depend on w t j for j > 0, in the sense that either conditioning or not conditioning on these variables has the same effect. Eduardo Rossi c - Macroeconometria 07 5
6 Factorization of the density Under the condition Θ = Θ 1 Θ 2 the admissible values of θ 1 may not depend on θ 2, so that knowledge of the latter cannot improve influences about the former. In this case θ 1 and θ 2 are said to be variation free. Under those conditions nothing need be known about the forms of D w y,z and D z to analyze D y z since these do not depend on θ 1. The analysis is conducted conditioning on z t and marginalizing on w t. Sequential cut: The separation of θ into two sets. θ 1 parameters of interest for the investigation θ 2 parameters that are not of interest. Eduardo Rossi c - Macroeconometria 07 6
7 Weak exogeneity Suppose that D y z depends on a vector φ of parameter of interest, whose values are the focus of investigation. To make the desired factorization of the DGP, it is only necessary that there exists some parameterization θ such that (2) holds, with θ 1 and θ 2 variation free, and φ = g (θ 1 ). In this analysis, the y t are called endogenous, z t are called weakly exogenous for φ. Weak exogeneity is a relationship between parameters and variables, and is not a property of variables as such. Without the required cut of the parameters, the factorization (1) is not relevant to the investigation. Eduardo Rossi c - Macroeconometria 07 7
8 Other notions of exogeneity Exogeneity is sometimes defined in terms of the independence of the variables in question from the disturbances in a model. In the regression model y t = x tβ + ε t t = 1,...,T x t is independent of ε t+j, j 0, E[x t ε t ] = 0. x t is said to be predetermined. If the independence holds for all j, x t is said to be strictly exogenous. Eduardo Rossi c - Macroeconometria 07 8
9 Setup Variables are related to their own lags in the sequence of observations, it is necessary to introduce conditioning assumptions. I t set of conditioning variables (the smallest σ-field of events containing the σ-fields generated by the conditioning variables). The model y t = x tβ + ǫ t Assumptions: 1. E[ǫ t I t ] = 0 a.s.. 2. E[ǫ 2 t I t ] = σ 2 a.s. Eduardo Rossi c - Macroeconometria 07 9
10 Setup The set I t includes deterministic variables (intercept, seasonal dummies, ecc.) lagged variables, dated t j, j > 0 current dated variables that are weakly exogenous for (β, σ 2 ) Any Borel-measurable function of variable in I t is also in I t : ǫ t j I t. ǫ t j = y t j x t jβ Implication of Assumption 1 is that the disturbances must be serially uncorrelated. Eduardo Rossi c - Macroeconometria 07 10
11 Example Suppose (y t,z t ) is a vector of variables generated by a dynamic DGP represented by the density factorization D t (y t,z t Z t 1, Y t 1 ; φ) = D t (y t z t, Z t 1, Y t 1 ; φ 1 )D t (z t Z t 1, Y t 1 ; φ 2 ) I t = σ(z t ) Z t 1 Y t 1 E[y t z t, Z t 1, Y t 1 ; φ 1 ] = x tβ where x t is composed of elements of z t j j 0 and y t j j > 0, if D t (y t z t, Z t 1, Y t 1 ; φ 1 ) is Gaussian then φ 1 = (β, σ 2 ). Eduardo Rossi c - Macroeconometria 07 11
12 The Method of Maximum Likelihood Notation: Let X 1 n = [x 1,...,x T ] (T m), or simply X, denote a matrix of random variables x t S R m X S T where S T R Tm is the sample space. Supposing the data are continuously distributed, let the joint p.d.f. of these data be denoted by D(X; θ 0 ), a member of a family of functions D( ; θ), θ Θ. D( ; θ) : S T R representing the density associated with each point in S T, for a given θ. Eduardo Rossi c - Macroeconometria 07 12
13 The Method of Maximum Likelihood For a given X S T : D(X; ) : Θ R is called the likelihood function. It is denoted by l(,x). X is to be thought of as a sample that has been observed, and l(θ,x) represents the p.d.f. that would be associated with the sample X had it been generated by the data generation process (DGP) with parameters θ. Eduardo Rossi c - Macroeconometria 07 13
14 The Method of Maximum Likelihood The likelihood function can provide the basis for the inferences from a sample X about the unknown θ. The maximum likelihood estimator is θ = arg max θ Θ L(θ;X) The sample X is representative of the distribution from which it was drawn so that the value of θ for which L is largest is most likely in the sense of attributing the highest probability density to X. Eduardo Rossi c - Macroeconometria 07 14
15 The Method of Maximum Likelihood Economic theory can specify only the first two moments of the distribution, while Gaussian distribution is assumed without any special justification. In this case, the estimator is called quasi-maximum likelihood (QML). Eduardo Rossi c - Macroeconometria 07 15
16 The Classical Gaussian Regression Model When the data are independently sampled from a large population, the joint density is merely the product of the marginal densities of the observations. Considering the partition X 1 T = [y1 T,Z1 T ] (respectively, the first and last m-1 columns) suppose the joint density can be factored so that the parameters of interest are all in the conditional factor D(y 1 T,Z 1 T;θ, ψ) = = D y Z (yt 1 Z 1 T;θ)D Z (Z 1 T;ψ) T D(y t z t ; θ)d(z 1 T; ψ) t=1 under the Gaussianity assumption D(y t z t ; θ) = { 1 exp (y t z tβ) 2 2πσ 2 2σ 2 } Eduardo Rossi c - Macroeconometria 07 16
17 The Classical Gaussian Regression Model The likelihood function is L(β, σ 2 ; X) = ( 1 2πσ 2 ) T exp { S(β) } 2σ 2 where S(β) = T t=1 (y t z tβ) 2. The log-likelihood is L(β, σ 2 ) = T 2 lnσ2 S(β) 2σ 2 The MLE of β is the OLS estimator: β = (Z 1 T Z 1 T) 1 Z 1 T y 1 T The MLE of σ 2 is σ 2 = ǫ ǫ T. Eduardo Rossi c - Macroeconometria 07 17
18 Properties of MLE In general, in case of independent observations the log-likelihood for the t-th observation is: L t (θ) = log D t (x t ; θ) θ Θ it is assumed that, for some θ 0 int(θ), D t ( ; θ 0 ) represents, with probability 1, the true probability function of x t. Eduardo Rossi c - Macroeconometria 07 18
19 MLE of The Dynamic Regression Model The dynamic regression model with the specific conditional Gaussian assumption: { D(y t I t ; β, σ 2 1 ) = exp (y t x tβ) 2 } 2πσ 2 2σ 2 l(β, σ 2 ) = T t=p+1 D(y t I t ; β, σ 2 ) p represents the maximum lag on any variables contained in x t. This an approximation to the likelihood function. It is not a joint density function. Since z t may depend on lagged values of y t (weakly but not strongly exogenous) the marginal factors D(z t Z t 1, Y t 1 ) are needed to describe the joint distribution of (y p+1,...,y T ) Eduardo Rossi c - Macroeconometria 07 19
20 MLE of The Dynamic Regression Model We can regard the maximizers of T t=p+1 D(y t I t ; β, σ 2 ) as ML estimators because the joint density depends on (β, σ 2 ) only through the terms in T t=p+1 D(y t I t ; β, σ 2 ). The OLS estimates are asymptotically equivalent to the MLE when the disturbances are Gaussian. Eduardo Rossi c - Macroeconometria 07 20
21 Properties of MLE Given {x t, I t }, I t = σ(x t,x t 1,...). The loglikelihood of a closed dynamic model, conditioned only on the past, without the factoring-out of weakly exogenous components: L t (θ) = lnd t (x t I t 1 ; θ) θ Θ θ 0 Θ; D t (x t I t 1 ; θ) represents, with prob.1, the true conditional probability function of x t. The parameters of interest θ are confined in D t. Eduardo Rossi c - Macroeconometria 07 21
22 Properties of MLE Under dependent sampling the log-likelihood is the sum of the l t s over the sample plus a term representing the initial conditions. For the asymptotic analysis, we can ignore it. Given the assumptions, it is of smaller order as T. When the probability function is evaluated at x t L t (θ) : Θ Ω R For each fixed ω Ω L t (θ) : Θ R. And for each fixed θ it is a I t -measurable random variable. Eduardo Rossi c - Macroeconometria 07 22
23 Properties of MLE Considering a fixed x t, like x, each l(θ,x) is a mapping from Θ S Ω to R and is a I t -measurable random variable. The same characterization applies to the various partial derivatives w.r.t. the elements of θ. Eduardo Rossi c - Macroeconometria 07 23
24 Information Inequality x continuously distributed with joint density D(x). Let G(x) G(ξ)dξ = 1 S Let S be the support of D: D(x) > 0 : x S. Suppose that D and G have the same support G(x) = 0 if and only if D(x) = 0 They are to be equivalent. G is an equivalent p.d.f. and can be a candidate to approximate D. Given the Jensen s inequality: [ E log G ] ( ) G(ξ) = log D(ξ)dξ log G(ξ)dξ = 0 D D(ξ) S S Eduardo Rossi c - Macroeconometria 07 24
25 Kullback-Leibler information criterion Since log is strictly concave, the inequality holds as an equality only in the case where D(x) = G(x) for almost every x S, the exceptions form a set of measure 0 in S. E [ log G D] measures the the closeness of G to D over the sample space is called the Kullback-Leibler information criterion (KLIC). Obvious choices of G include the others members, with θ θ 0, of the family of densities representing the model. Eduardo Rossi c - Macroeconometria 07 25
26 Kullback-Leibler information criterion The information inequality holds, almost surely, for the case of conditional expectations. With E[ I t 1 ] ( )D t (ξ I t 1 ; θ 0 )dξ E [ log D ] t(x t I t 1 ; θ) D t (x t I t 1 ; θ 0 ) I t 1 log E [ ] Dt (x t I t 1 ; θ) D t (x t I t 1 ; θ 0 ) I t 1 = 0 a.s. Eduardo Rossi c - Macroeconometria 07 26
27 Identification E[L T (θ)] is maximized at θ 0 Given this result, consistency of ML estimator follows from the following theorem Theorem 1. Θ is compact 2. 1 T L T(θ) p E[L T (θ)] (a non-stochastic function of θ) uniformly in Θ 3. θ 0 int(θ) is the unique maximum of E[L T (θ)] then θ T p θ0. Condition 2 can also be stated in the form 1 T L T(θ) E[L T (θ)] p 0 (5) sup θ Θ Eduardo Rossi c - Macroeconometria 07 27
28 Structures θ 1 and θ 2 are said to be observationally equivalent if L T (θ 1,X) = L T (θ 2,X) for almost all X S T, and all T 1. A model is said to be globally (locally) identified if the true structure θ 0 is not observationally equivalent to any other point of Θ (of an open neighborhood of θ 0 ). The KLIC for the complete sample E 0 [L T (θ)] E 0 [L T (θ 0 )] E 0 [ ] denotes the expected value under the true distribution. Underidentification implies that 1 T L T(θ) fails the uniqueness requirement of condition (3). Underidentification means that no consistent estimator exists, and the parameters are simply inaccessible to empirical investigation. Eduardo Rossi c - Macroeconometria 07 28
29 Asymptotic Normality The results hinge on the properties of the gradient of L T (score vector) at θ 0. Define the operator E θ ( I t 1 ) = ( )D t (ξ I t 1 ; θ)dξ (6) representing the conditional expectation of any function of x t when θ is the true parameter. l t () is twice continuously differentiable with respect to θ everywhere on int(θ) S with prob.1, and the derivatives are bounded uniformly in t. Lemma ( E l t θ ) It 1 θ=θ0 = 0 a.s. (7) Eduardo Rossi c - Macroeconometria 07 29
30 Asymptotic Normality Proof Given that l t θ = 1 D t D t θ We can write ( ) l t E θ I t 1 θ=θ0 lt = θ D t(ξ I t 1 ; θ)dξ Dt (ξ I t 1 ; θ) = dξ θ = D t (ξ I t 1 ; θ)dξ θ }{{} =1 = 0 a.s. (8) Eduardo Rossi c - Macroeconometria 07 30
31 Asymptotic Normality Interchanging the order of differentiation and integration. This equality holds for the case θ = θ 0. { } l The adapted sequence t θ 0, I t is a vector m.d. Applying the CLT, we have that 1 L T D T θ N(0, I0 ) (9) θ0 where and I 0 = lim T T 1 I T0 (10) I T0 = E [ L T θ L T θ θ0 ] = [ T E t=1 l t θ l t θ θ0 ] (11) Eduardo Rossi c - Macroeconometria 07 31
32 Asymptotic Normality The matrix I T0 is called the information matrix, being thought of as measuring the amount of information about θ 0 in the sample. I 0 is the limiting information matrix. Theorem I T0 = E [ 2 L T θ θ θ0 ] (12) Eduardo Rossi c - Macroeconometria 07 32
33 Asymptotic Normality Proof t E θ ( lt θ ) l t θ 0 = lt θ θ D t(ξ; θ)dξ ( 2 l t = θ θ + l t l t θ θ ) = E θ ( 2 l t θ θ = t [ I T0 = E ( lt + E θ θ ( 2 ) l t E θ θ θ I t 1 2 L T θ θ ] I t 1 θ0 ) D t (ξ; θ)dξ ) l t θ Finally, T( θ θ0 ) D N(0, I 1 0 ) (13) Eduardo Rossi c - Macroeconometria 07 33
Exogeneity and Causality
Università di Pavia Exogeneity and Causality Eduardo Rossi University of Pavia Factorization of the density DGP: D t (x t χ t 1, d t ; Ψ) x t represent all the variables in the economy. The econometric
More informationUniversity of Pavia. M Estimators. Eduardo Rossi
University of Pavia M Estimators Eduardo Rossi Criterion Function A basic unifying notion is that most econometric estimators are defined as the minimizers of certain functions constructed from the sample
More informationMaximum Likelihood Estimation
University of Pavia Maximum Likelihood Estimation Eduardo Rossi Likelihood function Choosing parameter values that make what one has observed more likely to occur than any other parameter values do. Assumption(Distribution)
More informationEconometrics I, Estimation
Econometrics I, Estimation Department of Economics Stanford University September, 2008 Part I Parameter, Estimator, Estimate A parametric is a feature of the population. An estimator is a function of the
More informationDynamic Regression Models
Università di Pavia 2007 Dynamic Regression Models Eduardo Rossi University of Pavia Data Generating Process & Models Setup y t denote an (n 1) vector of economic variables generated at time t. The collection
More informationGARCH Models Estimation and Inference
Università di Pavia GARCH Models Estimation and Inference Eduardo Rossi Likelihood function The procedure most often used in estimating θ 0 in ARCH models involves the maximization of a likelihood function
More informationGARCH Models Estimation and Inference. Eduardo Rossi University of Pavia
GARCH Models Estimation and Inference Eduardo Rossi University of Pavia Likelihood function The procedure most often used in estimating θ 0 in ARCH models involves the maximization of a likelihood function
More informationGARCH Models Estimation and Inference
GARCH Models Estimation and Inference Eduardo Rossi University of Pavia December 013 Rossi GARCH Financial Econometrics - 013 1 / 1 Likelihood function The procedure most often used in estimating θ 0 in
More informationGraduate Econometrics I: Maximum Likelihood I
Graduate Econometrics I: Maximum Likelihood I Yves Dominicy Université libre de Bruxelles Solvay Brussels School of Economics and Management ECARES Yves Dominicy Graduate Econometrics I: Maximum Likelihood
More informationInstrumental Variables
Università di Pavia 2010 Instrumental Variables Eduardo Rossi Exogeneity Exogeneity Assumption: the explanatory variables which form the columns of X are exogenous. It implies that any randomness in the
More informationLinear models. Linear models are computationally convenient and remain widely used in. applied econometric research
Linear models Linear models are computationally convenient and remain widely used in applied econometric research Our main focus in these lectures will be on single equation linear models of the form y
More informationMaximum Likelihood Asymptotic Theory. Eduardo Rossi University of Pavia
Maximum Likelihood Asymtotic Theory Eduardo Rossi University of Pavia Slutsky s Theorem, Cramer s Theorem Slutsky s Theorem Let {X N } be a random sequence converging in robability to a constant a, and
More informationGeneralized Method of Moments (GMM) Estimation
Econometrics 2 Fall 2004 Generalized Method of Moments (GMM) Estimation Heino Bohn Nielsen of29 Outline of the Lecture () Introduction. (2) Moment conditions and methods of moments (MM) estimation. Ordinary
More informationStatistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach
Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score
More informationIntroduction to Estimation Methods for Time Series models Lecture 2
Introduction to Estimation Methods for Time Series models Lecture 2 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 2 SNS Pisa 1 / 21 Estimators:
More informationWeak convergence. Amsterdam, 13 November Leiden University. Limit theorems. Shota Gugushvili. Generalities. Criteria
Weak Leiden University Amsterdam, 13 November 2013 Outline 1 2 3 4 5 6 7 Definition Definition Let µ, µ 1, µ 2,... be probability measures on (R, B). It is said that µ n converges weakly to µ, and we then
More informationIntroduction to Stochastic processes
Università di Pavia Introduction to Stochastic processes Eduardo Rossi Stochastic Process Stochastic Process: A stochastic process is an ordered sequence of random variables defined on a probability space
More informationState-space Model. Eduardo Rossi University of Pavia. November Rossi State-space Model Fin. Econometrics / 53
State-space Model Eduardo Rossi University of Pavia November 2014 Rossi State-space Model Fin. Econometrics - 2014 1 / 53 Outline 1 Motivation 2 Introduction 3 The Kalman filter 4 Forecast errors 5 State
More informationDiscrete time processes
Discrete time processes Predictions are difficult. Especially about the future Mark Twain. Florian Herzog 2013 Modeling observed data When we model observed (realized) data, we encounter usually the following
More informationTesting for Regime Switching: A Comment
Testing for Regime Switching: A Comment Andrew V. Carter Department of Statistics University of California, Santa Barbara Douglas G. Steigerwald Department of Economics University of California Santa Barbara
More informationEconometrics II - EXAM Outline Solutions All questions have 25pts Answer each question in separate sheets
Econometrics II - EXAM Outline Solutions All questions hae 5pts Answer each question in separate sheets. Consider the two linear simultaneous equations G with two exogeneous ariables K, y γ + y γ + x δ
More informationSystem Identification, Lecture 4
System Identification, Lecture 4 Kristiaan Pelckmans (IT/UU, 2338) Course code: 1RT880, Report code: 61800 - Spring 2016 F, FRI Uppsala University, Information Technology 13 April 2016 SI-2016 K. Pelckmans
More informationEconometrics of Panel Data
Econometrics of Panel Data Jakub Mućk Meeting # 6 Jakub Mućk Econometrics of Panel Data Meeting # 6 1 / 36 Outline 1 The First-Difference (FD) estimator 2 Dynamic panel data models 3 The Anderson and Hsiao
More informationGraduate Econometrics I: Maximum Likelihood II
Graduate Econometrics I: Maximum Likelihood II Yves Dominicy Université libre de Bruxelles Solvay Brussels School of Economics and Management ECARES Yves Dominicy Graduate Econometrics I: Maximum Likelihood
More informationECON 4160, Lecture 11 and 12
ECON 4160, 2016. Lecture 11 and 12 Co-integration Ragnar Nymoen Department of Economics 9 November 2017 1 / 43 Introduction I So far we have considered: Stationary VAR ( no unit roots ) Standard inference
More informationδ -method and M-estimation
Econ 2110, fall 2016, Part IVb Asymptotic Theory: δ -method and M-estimation Maximilian Kasy Department of Economics, Harvard University 1 / 40 Example Suppose we estimate the average effect of class size
More informationEstimating Unnormalised Models by Score Matching
Estimating Unnormalised Models by Score Matching Michael Gutmann Probabilistic Modelling and Reasoning (INFR11134) School of Informatics, University of Edinburgh Spring semester 2018 Program 1. Basics
More informationStochastic Dynamic Programming: The One Sector Growth Model
Stochastic Dynamic Programming: The One Sector Growth Model Esteban Rossi-Hansberg Princeton University March 26, 2012 Esteban Rossi-Hansberg () Stochastic Dynamic Programming March 26, 2012 1 / 31 References
More informationis a Borel subset of S Θ for each c R (Bertsekas and Shreve, 1978, Proposition 7.36) This always holds in practical applications.
Stat 811 Lecture Notes The Wald Consistency Theorem Charles J. Geyer April 9, 01 1 Analyticity Assumptions Let { f θ : θ Θ } be a family of subprobability densities 1 with respect to a measure µ on a measurable
More informationLinear Regression with Time Series Data
Econometrics 2 Linear Regression with Time Series Data Heino Bohn Nielsen 1of21 Outline (1) The linear regression model, identification and estimation. (2) Assumptions and results: (a) Consistency. (b)
More informationInstrumental Variables
Università di Pavia 2010 Instrumental Variables Eduardo Rossi Exogeneity Exogeneity Assumption: the explanatory variables which form the columns of X are exogenous. It implies that any randomness in the
More informationLasso Maximum Likelihood Estimation of Parametric Models with Singular Information Matrices
Article Lasso Maximum Likelihood Estimation of Parametric Models with Singular Information Matrices Fei Jin 1,2 and Lung-fei Lee 3, * 1 School of Economics, Shanghai University of Finance and Economics,
More informationPanel Data Seminar. Discrete Response Models. Crest-Insee. 11 April 2008
Panel Data Seminar Discrete Response Models Romain Aeberhardt Laurent Davezies Crest-Insee 11 April 2008 Aeberhardt and Davezies (Crest-Insee) Panel Data Seminar 11 April 2008 1 / 29 Contents Overview
More informationMore Empirical Process Theory
More Empirical Process heory 4.384 ime Series Analysis, Fall 2008 Recitation by Paul Schrimpf Supplementary to lectures given by Anna Mikusheva October 24, 2008 Recitation 8 More Empirical Process heory
More informationLecture 2: Consistency of M-estimators
Lecture 2: Instructor: Deartment of Economics Stanford University Preared by Wenbo Zhou, Renmin University References Takeshi Amemiya, 1985, Advanced Econometrics, Harvard University Press Newey and McFadden,
More informationChapter 1: A Brief Review of Maximum Likelihood, GMM, and Numerical Tools. Joan Llull. Microeconometrics IDEA PhD Program
Chapter 1: A Brief Review of Maximum Likelihood, GMM, and Numerical Tools Joan Llull Microeconometrics IDEA PhD Program Maximum Likelihood Chapter 1. A Brief Review of Maximum Likelihood, GMM, and Numerical
More informationChapter 4: Asymptotic Properties of the MLE
Chapter 4: Asymptotic Properties of the MLE Daniel O. Scharfstein 09/19/13 1 / 1 Maximum Likelihood Maximum likelihood is the most powerful tool for estimation. In this part of the course, we will consider
More informationLecture 6 Basic Probability
Lecture 6: Basic Probability 1 of 17 Course: Theory of Probability I Term: Fall 2013 Instructor: Gordan Zitkovic Lecture 6 Basic Probability Probability spaces A mathematical setup behind a probabilistic
More informationSystem Identification, Lecture 4
System Identification, Lecture 4 Kristiaan Pelckmans (IT/UU, 2338) Course code: 1RT880, Report code: 61800 - Spring 2012 F, FRI Uppsala University, Information Technology 30 Januari 2012 SI-2012 K. Pelckmans
More informationAnalogy Principle. Asymptotic Theory Part II. James J. Heckman University of Chicago. Econ 312 This draft, April 5, 2006
Analogy Principle Asymptotic Theory Part II James J. Heckman University of Chicago Econ 312 This draft, April 5, 2006 Consider four methods: 1. Maximum Likelihood Estimation (MLE) 2. (Nonlinear) Least
More informationSTAT Financial Time Series
STAT 6104 - Financial Time Series Chapter 4 - Estimation in the time Domain Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 1 / 46 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR
More informationInference in non-linear time series
Intro LS MLE Other Erik Lindström Centre for Mathematical Sciences Lund University LU/LTH & DTU Intro LS MLE Other General Properties Popular estimatiors Overview Introduction General Properties Estimators
More informationCourse: ESO-209 Home Work: 1 Instructor: Debasis Kundu
Home Work: 1 1. Describe the sample space when a coin is tossed (a) once, (b) three times, (c) n times, (d) an infinite number of times. 2. A coin is tossed until for the first time the same result appear
More informationParameter Estimation
Parameter Estimation Consider a sample of observations on a random variable Y. his generates random variables: (y 1, y 2,, y ). A random sample is a sample (y 1, y 2,, y ) where the random variables y
More informationDynamic Discrete Choice Structural Models in Empirical IO
Dynamic Discrete Choice Structural Models in Empirical IO Lecture 4: Euler Equations and Finite Dependence in Dynamic Discrete Choice Models Victor Aguirregabiria (University of Toronto) Carlos III, Madrid
More information2 Statistical Estimation: Basic Concepts
Technion Israel Institute of Technology, Department of Electrical Engineering Estimation and Identification in Dynamical Systems (048825) Lecture Notes, Fall 2009, Prof. N. Shimkin 2 Statistical Estimation:
More informationECON 3150/4150, Spring term Lecture 6
ECON 3150/4150, Spring term 2013. Lecture 6 Review of theoretical statistics for econometric modelling (II) Ragnar Nymoen University of Oslo 31 January 2013 1 / 25 References to Lecture 3 and 6 Lecture
More informationBrief Review on Estimation Theory
Brief Review on Estimation Theory K. Abed-Meraim ENST PARIS, Signal and Image Processing Dept. abed@tsi.enst.fr This presentation is essentially based on the course BASTA by E. Moulines Brief review on
More informationEstimation, Inference, and Hypothesis Testing
Chapter 2 Estimation, Inference, and Hypothesis Testing Note: The primary reference for these notes is Ch. 7 and 8 of Casella & Berger 2. This text may be challenging if new to this topic and Ch. 7 of
More informationStochastic process for macro
Stochastic process for macro Tianxiao Zheng SAIF 1. Stochastic process The state of a system {X t } evolves probabilistically in time. The joint probability distribution is given by Pr(X t1, t 1 ; X t2,
More informationCh.10 Autocorrelated Disturbances (June 15, 2016)
Ch10 Autocorrelated Disturbances (June 15, 2016) In a time-series linear regression model setting, Y t = x tβ + u t, t = 1, 2,, T, (10-1) a common problem is autocorrelation, or serial correlation of the
More informationECON 4160, Autumn term Lecture 1
ECON 4160, Autumn term 2017. Lecture 1 a) Maximum Likelihood based inference. b) The bivariate normal model Ragnar Nymoen University of Oslo 24 August 2017 1 / 54 Principles of inference I Ordinary least
More informationECE531 Lecture 10b: Maximum Likelihood Estimation
ECE531 Lecture 10b: Maximum Likelihood Estimation D. Richard Brown III Worcester Polytechnic Institute 05-Apr-2011 Worcester Polytechnic Institute D. Richard Brown III 05-Apr-2011 1 / 23 Introduction So
More informationNext, we discuss econometric methods that can be used to estimate panel data models.
1 Motivation Next, we discuss econometric methods that can be used to estimate panel data models. Panel data is a repeated observation of the same cross section Panel data is highly desirable when it is
More informationState-space Model. Eduardo Rossi University of Pavia. November Rossi State-space Model Financial Econometrics / 49
State-space Model Eduardo Rossi University of Pavia November 2013 Rossi State-space Model Financial Econometrics - 2013 1 / 49 Outline 1 Introduction 2 The Kalman filter 3 Forecast errors 4 State smoothing
More informationThe Uniform Weak Law of Large Numbers and the Consistency of M-Estimators of Cross-Section and Time Series Models
The Uniform Weak Law of Large Numbers and the Consistency of M-Estimators of Cross-Section and Time Series Models Herman J. Bierens Pennsylvania State University September 16, 2005 1. The uniform weak
More informationStatistics 612: L p spaces, metrics on spaces of probabilites, and connections to estimation
Statistics 62: L p spaces, metrics on spaces of probabilites, and connections to estimation Moulinath Banerjee December 6, 2006 L p spaces and Hilbert spaces We first formally define L p spaces. Consider
More informationEmpirical Risk Minimization
Empirical Risk Minimization Fabrice Rossi SAMM Université Paris 1 Panthéon Sorbonne 2018 Outline Introduction PAC learning ERM in practice 2 General setting Data X the input space and Y the output space
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postponed exam: ECON4160 Econometrics Modeling and systems estimation Date of exam: Wednesday, January 8, 2014 Time for exam: 09:00 a.m. 12:00 noon The problem
More informationARIMA Modelling and Forecasting
ARIMA Modelling and Forecasting Economic time series often appear nonstationary, because of trends, seasonal patterns, cycles, etc. However, the differences may appear stationary. Δx t x t x t 1 (first
More informationShort T Panels - Review
Short T Panels - Review We have looked at methods for estimating parameters on time-varying explanatory variables consistently in panels with many cross-section observation units but a small number of
More informationIf we want to analyze experimental or simulated data we might encounter the following tasks:
Chapter 1 Introduction If we want to analyze experimental or simulated data we might encounter the following tasks: Characterization of the source of the signal and diagnosis Studying dependencies Prediction
More informationCS 540: Machine Learning Lecture 2: Review of Probability & Statistics
CS 540: Machine Learning Lecture 2: Review of Probability & Statistics AD January 2008 AD () January 2008 1 / 35 Outline Probability theory (PRML, Section 1.2) Statistics (PRML, Sections 2.1-2.4) AD ()
More informationADVANCED FINANCIAL ECONOMETRICS PROF. MASSIMO GUIDOLIN
Massimo Guidolin Massimo.Guidolin@unibocconi.it Dept. of Finance ADVANCED FINANCIAL ECONOMETRICS PROF. MASSIMO GUIDOLIN a.a. 14/15 p. 1 LECTURE 3: REVIEW OF BASIC ESTIMATION METHODS: GMM AND OTHER EXTREMUM
More informationFollow links for Class Use and other Permissions. For more information send to:
COPYRIGH NOICE: Kenneth J. Singleton: Empirical Dynamic Asset Pricing is published by Princeton University Press and copyrighted, 00, by Princeton University Press. All rights reserved. No part of this
More informationModel Selection for Geostatistical Models
Model Selection for Geostatistical Models Richard A. Davis Colorado State University http://www.stat.colostate.edu/~rdavis/lectures Joint work with: Jennifer A. Hoeting, Colorado State University Andrew
More information1. Stochastic Processes and Stationarity
Massachusetts Institute of Technology Department of Economics Time Series 14.384 Guido Kuersteiner Lecture Note 1 - Introduction This course provides the basic tools needed to analyze data that is observed
More informationEstimation theory. Parametric estimation. Properties of estimators. Minimum variance estimator. Cramer-Rao bound. Maximum likelihood estimators
Estimation theory Parametric estimation Properties of estimators Minimum variance estimator Cramer-Rao bound Maximum likelihood estimators Confidence intervals Bayesian estimation 1 Random Variables Let
More informationLECTURE 10: NEYMAN-PEARSON LEMMA AND ASYMPTOTIC TESTING. The last equality is provided so this can look like a more familiar parametric test.
Economics 52 Econometrics Professor N.M. Kiefer LECTURE 1: NEYMAN-PEARSON LEMMA AND ASYMPTOTIC TESTING NEYMAN-PEARSON LEMMA: Lesson: Good tests are based on the likelihood ratio. The proof is easy in the
More informationHigh Dimensional Empirical Likelihood for Generalized Estimating Equations with Dependent Data
High Dimensional Empirical Likelihood for Generalized Estimating Equations with Dependent Data Song Xi CHEN Guanghua School of Management and Center for Statistical Science, Peking University Department
More informationLecture 26: Likelihood ratio tests
Lecture 26: Likelihood ratio tests Likelihood ratio When both H 0 and H 1 are simple (i.e., Θ 0 = {θ 0 } and Θ 1 = {θ 1 }), Theorem 6.1 applies and a UMP test rejects H 0 when f θ1 (X) f θ0 (X) > c 0 for
More informationMEI Exam Review. June 7, 2002
MEI Exam Review June 7, 2002 1 Final Exam Revision Notes 1.1 Random Rules and Formulas Linear transformations of random variables. f y (Y ) = f x (X) dx. dg Inverse Proof. (AB)(AB) 1 = I. (B 1 A 1 )(AB)(AB)
More informationTime Series Models and Inference. James L. Powell Department of Economics University of California, Berkeley
Time Series Models and Inference James L. Powell Department of Economics University of California, Berkeley Overview In contrast to the classical linear regression model, in which the components of the
More informationLecture 2: Univariate Time Series
Lecture 2: Univariate Time Series Analysis: Conditional and Unconditional Densities, Stationarity, ARMA Processes Prof. Massimo Guidolin 20192 Financial Econometrics Spring/Winter 2017 Overview Motivation:
More informationChapter 6 Stochastic Regressors
Chapter 6 Stochastic Regressors 6. Stochastic regressors in non-longitudinal settings 6.2 Stochastic regressors in longitudinal settings 6.3 Longitudinal data models with heterogeneity terms and sequentially
More informationLecture 25: Review. Statistics 104. April 23, Colin Rundel
Lecture 25: Review Statistics 104 Colin Rundel April 23, 2012 Joint CDF F (x, y) = P [X x, Y y] = P [(X, Y ) lies south-west of the point (x, y)] Y (x,y) X Statistics 104 (Colin Rundel) Lecture 25 April
More informationPOLI 8501 Introduction to Maximum Likelihood Estimation
POLI 8501 Introduction to Maximum Likelihood Estimation Maximum Likelihood Intuition Consider a model that looks like this: Y i N(µ, σ 2 ) So: E(Y ) = µ V ar(y ) = σ 2 Suppose you have some data on Y,
More informationWooldridge, Introductory Econometrics, 4th ed. Chapter 15: Instrumental variables and two stage least squares
Wooldridge, Introductory Econometrics, 4th ed. Chapter 15: Instrumental variables and two stage least squares Many economic models involve endogeneity: that is, a theoretical relationship does not fit
More informationP n. This is called the law of large numbers but it comes in two forms: Strong and Weak.
Large Sample Theory Large Sample Theory is a name given to the search for approximations to the behaviour of statistical procedures which are derived by computing limits as the sample size, n, tends to
More informationTime Series Analysis Fall 2008
MIT OpenCourseWare http://ocw.mit.edu 4.384 Time Series Analysis Fall 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. Indirect Inference 4.384 Time
More informationModel Specification Testing in Nonparametric and Semiparametric Time Series Econometrics. Jiti Gao
Model Specification Testing in Nonparametric and Semiparametric Time Series Econometrics Jiti Gao Department of Statistics School of Mathematics and Statistics The University of Western Australia Crawley
More informationIntegrating Correlated Bayesian Networks Using Maximum Entropy
Applied Mathematical Sciences, Vol. 5, 2011, no. 48, 2361-2371 Integrating Correlated Bayesian Networks Using Maximum Entropy Kenneth D. Jarman Pacific Northwest National Laboratory PO Box 999, MSIN K7-90
More informationStability of optimization problems with stochastic dominance constraints
Stability of optimization problems with stochastic dominance constraints D. Dentcheva and W. Römisch Stevens Institute of Technology, Hoboken Humboldt-University Berlin www.math.hu-berlin.de/~romisch SIAM
More informationLinear Regression with Time Series Data
u n i v e r s i t y o f c o p e n h a g e n d e p a r t m e n t o f e c o n o m i c s Econometrics II Linear Regression with Time Series Data Morten Nyboe Tabor u n i v e r s i t y o f c o p e n h a g
More informationNishant Gurnani. GAN Reading Group. April 14th, / 107
Nishant Gurnani GAN Reading Group April 14th, 2017 1 / 107 Why are these Papers Important? 2 / 107 Why are these Papers Important? Recently a large number of GAN frameworks have been proposed - BGAN, LSGAN,
More informationOn the convergence of sequences of random variables: A primer
BCAM May 2012 1 On the convergence of sequences of random variables: A primer Armand M. Makowski ECE & ISR/HyNet University of Maryland at College Park armand@isr.umd.edu BCAM May 2012 2 A sequence a :
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationOn the Power of Tests for Regime Switching
On the Power of Tests for Regime Switching joint work with Drew Carter and Ben Hansen Douglas G. Steigerwald UC Santa Barbara May 2015 D. Steigerwald (UCSB) Regime Switching May 2015 1 / 42 Motivating
More informationEcon 623 Econometrics II Topic 2: Stationary Time Series
1 Introduction Econ 623 Econometrics II Topic 2: Stationary Time Series In the regression model we can model the error term as an autoregression AR(1) process. That is, we can use the past value of the
More informationNotes on Measure Theory and Markov Processes
Notes on Measure Theory and Markov Processes Diego Daruich March 28, 2014 1 Preliminaries 1.1 Motivation The objective of these notes will be to develop tools from measure theory and probability to allow
More informationBrandon C. Kelly (Harvard Smithsonian Center for Astrophysics)
Brandon C. Kelly (Harvard Smithsonian Center for Astrophysics) Probability quantifies randomness and uncertainty How do I estimate the normalization and logarithmic slope of a X ray continuum, assuming
More informationLecture 1: Introduction
Principles of Statistics Part II - Michaelmas 208 Lecturer: Quentin Berthet Lecture : Introduction This course is concerned with presenting some of the mathematical principles of statistical theory. One
More informationVerifying Regularity Conditions for Logit-Normal GLMM
Verifying Regularity Conditions for Logit-Normal GLMM Yun Ju Sung Charles J. Geyer January 10, 2006 In this note we verify the conditions of the theorems in Sung and Geyer (submitted) for the Logit-Normal
More informationECON 4160, Spring term Lecture 12
ECON 4160, Spring term 2013. Lecture 12 Non-stationarity and co-integration 2/2 Ragnar Nymoen Department of Economics 13 Nov 2013 1 / 53 Introduction I So far we have considered: Stationary VAR, with deterministic
More informationChapter 2: Fundamentals of Statistics Lecture 15: Models and statistics
Chapter 2: Fundamentals of Statistics Lecture 15: Models and statistics Data from one or a series of random experiments are collected. Planning experiments and collecting data (not discussed here). Analysis:
More informationMultivariate Analysis and Likelihood Inference
Multivariate Analysis and Likelihood Inference Outline 1 Joint Distribution of Random Variables 2 Principal Component Analysis (PCA) 3 Multivariate Normal Distribution 4 Likelihood Inference Joint density
More informationEstimation Theory. as Θ = (Θ 1,Θ 2,...,Θ m ) T. An estimator
Estimation Theory Estimation theory deals with finding numerical values of interesting parameters from given set of data. We start with formulating a family of models that could describe how the data were
More informationModel comparison and selection
BS2 Statistical Inference, Lectures 9 and 10, Hilary Term 2008 March 2, 2008 Hypothesis testing Consider two alternative models M 1 = {f (x; θ), θ Θ 1 } and M 2 = {f (x; θ), θ Θ 2 } for a sample (X = x)
More informationFinal Exam. Economics 835: Econometrics. Fall 2010
Final Exam Economics 835: Econometrics Fall 2010 Please answer the question I ask - no more and no less - and remember that the correct answer is often short and simple. 1 Some short questions a) For each
More informationOptimization. The value x is called a maximizer of f and is written argmax X f. g(λx + (1 λ)y) < λg(x) + (1 λ)g(y) 0 < λ < 1; x, y X.
Optimization Background: Problem: given a function f(x) defined on X, find x such that f(x ) f(x) for all x X. The value x is called a maximizer of f and is written argmax X f. In general, argmax X f may
More information