Introduction to Time Series Analysis. Lecture 12.
|
|
- Melvin Ford
- 6 years ago
- Views:
Transcription
1 Last lecture: Introduction to Time Series Analysis. Lecture 12. Peter Bartlett 1. Parameter estimation 2. Maximum likelihood estimator 3. Yule-Walker estimation 1
2 Introduction to Time Series Analysis. Lecture Review: Yule-Walker estimators 2. Yule-Walker example 3. Efficiency 4. Maximum likelihood estimation 5. Large-sample distribution of MLE 2
3 Review: Yule-Walker estimation Method of moments: We choose parameters for which the moments are equal to the empirical moments. In this case, we choose φ so that γ(h) = ˆγ(h) for h = 0,...,p. Yule-Walker equations for ˆφ: ˆΓ pˆφ = ˆγp, ˆσ 2 = ˆγ(0) ˆφ ˆγ p. These are the forecasting equations. We can use the Durbin-Levinson algorithm. 3
4 Review: Confidence intervals for Yule-Walker estimation If {X t } is an AR(p) process, ˆφ AN (φ, σ2 ˆφ hh AN ( 0, 1 n n Γ 1 p ) ), ˆσ 2 P σ 2. for h > p. Thus, we can use the sample PACF to test for AR order, and we can calculate approximate confidence intervals for the parameters φ. 4
5 Review: Confidence intervals for Yule-Walker estimation If {X t } is an AR(p) process, and n is large, n(ˆφ p φ p ) is approximately N(0,ˆσ 2ˆΓ 1 p ), with probability 1 α,φ pj is in the interval ˆσ ) 1/2 ˆφ pj ±Φ 1 α/2 n (ˆΓ 1 p, jj whereφ 1 α/2 is the 1 α/2 quantile of the standard normal. 5
6 Review: Confidence intervals for Yule-Walker estimation with probability 1 α,φ p is in the ellipsoid { ) } φ R p : (ˆφp φ) ˆΓp (ˆφp φ ˆσ2 n χ2 1 α(p), whereχ 2 1 α(p) is the(1 α) quantile of the chi-squared withpdegrees of freedom. 6
7 Yule Walker estimation: Example 1.2 Sample ACF: Depth of Lake Huron,
8 Yule Walker estimation: Example 1 Sample PACF: Depth of Lake Huron,
9 Yule Walker estimation: Example ˆΓ 2 = ˆφ 2 = ˆγ 2 = ˆΓ 2 ˆγ 2 = ˆσ 2 w = ˆγ(0) ˆφ 2ˆγ 2 =
10 Yule Walker estimation: Example Confidence intervals: ˆφ 1 ±Φ 1 α/2 (ˆσ 2 wˆγ 1 2 /n ) 1/2 11 = ± ˆφ 2 ±Φ 1 α/2 (ˆσ 2 wˆγ 1 2 /n ) 1/2 22 = ±
11 Yule-Walker estimation It is also possible to define analogous estimators for ARMA(p,q) models withq > 0: q ˆγ(j) φ 1ˆγ(j 1) φ pˆγ(j p) = σ 2 θ i ψ i j, where ψ(b) = θ(b)/φ(b). Because of the dependence on theψ i, these equations are nonlinear inφ i,θ i. There might be no solution, or nonunique solutions. Also, the asymptotic efficiency of this estimator is poor: it has unnecessarily high variance. i=j 11
12 Efficiency of estimators Let ˆφ (1) and ˆφ (2) be two estimators. Suppose that ˆφ (1) AN(φ,σ 2 1), ˆφ(2) AN(φ,σ 2 2). The asymptotic efficiency of ˆφ (1) relative to ˆφ (2) is ( e φ, ˆφ (1), ˆφ (2)) = σ2 2 σ1 2. If e (φ, ˆφ (1), ˆφ ) (2) 1 for all φ, we say that ˆφ (2) is a more efficient estimator ofφthan ˆφ (1). For example, for an AR(p) process, the moment estimator and the maximum likelihood estimator are as efficient as each other. For an MA(q) process, the moment estimator is less efficient than the innovations estimator, which is less efficient than the MLE. 12
13 Yule Walker estimation: Example AR(1): AR(2): and ˆφ 1 ˆφ 2 γ(0) = σ2 1 φ 2 1 ˆφ 1 AN (φ 1, σ2 AN σ 2 n Γ 1 2 = 1 n φ 1 n Γ 1 1 φ 2 ) ( ) = AN φ 1, 1 φ2 1. n, σ2 n Γ φ 2 2 φ 1 (1+φ 2 ) φ 1 (1+φ 2 ) 1 φ
14 Yule Walker estimation: Example Suppose{X t } is an AR(1) process and the sample size n is large. If we estimate φ, we have Var(ˆφ 1 ) 1 φ2 1 n If we fit a larger model, say an AR(2), to this AR(1) process,. We have lost efficiency. Var(ˆφ 1 ) 1 φ2 2 n = 1 n > 1 φ2 1 n. 14
15 Introduction to Time Series Analysis. Lecture Review: Yule-Walker estimators 2. Yule-Walker example 3. Efficiency 4. Maximum likelihood estimation 5. Large-sample distribution of MLE 15
16 Parameter estimation: Maximum likelihood estimator One approach: Assume that {X t } is Gaussian, that is,φ(b)x t = θ(b)w t, where W t is i.i.d. Gaussian. Chooseφ i,θ j to maximize the likelihood: L(φ,θ,σ 2 ) = f(x 1,...,X n ), where f is the joint (Gaussian) density for the given ARMA model. (c.f. choosing the parameters that maximize the probability of the data.) 16
17 Maximum likelihood estimation Suppose that X 1,X 2,...,X n is drawn from a zero mean Gaussian ARMA(p,q) process. The likelihood of parameters φ R p,θ R q, σ 2 w R + is defined as the density of X = (X 1,X 2,...,X n ) under the Gaussian model with those parameters: L(φ,θ,σ 2 w) = 1 (2π) n/2 Γ n 1/2 exp ( 1 ) 2 X Γ 1 n X, where A denotes the determinant of a matrix A, and Γ n is the variance/covariance matrix of X with the given parameter values. The maximum likelihood estimator (MLE) ofφ,θ,σ 2 w maximizes this quantity. 17
18 Parameter estimation: Maximum likelihood estimator Advantages of MLE: Efficient (low variance estimates). Often the Gaussian assumption is reasonable. Even if{x t } is not Gaussian, the asymptotic distribution of the estimates (ˆφ, ˆθ,ˆσ 2 ) is the same as the Gaussian case. Disadvantages of MLE: Difficult optimization problem. Need to choose a good starting point (often use other estimators for this). 18
19 Preliminary parameter estimates Yule-Walker for AR(p): Regress X t ontox t 1,...,X t p. Durbin-Levinson algorithm withγ replaced by ˆγ. Yule-Walker for ARMA(p,q): Method of moments. Not efficient. Innovations algorithm for MA(q): withγ replaced by ˆγ. Hannan-Rissanen algorithm for ARMA(p,q): 1. Estimate high-order AR. 2. Use to estimate (unobserved) noisew t. 3. Regress X t ontox t 1,...,X t p,ŵt 1,...,Ŵt q. 4. Regress again with improved estimates ofw t. 19
20 Recall: Maximum likelihood estimation Suppose that X 1,X 2,...,X n is drawn from a zero mean Gaussian ARMA(p,q) process. The likelihood of parameters φ R p,θ R q, σ 2 w R + is defined as the density of X = (X 1,X 2,...,X n ) under the Gaussian model with those parameters: L(φ,θ,σ 2 w) = 1 (2π) n/2 Γ n 1/2 exp ( 1 ) 2 X Γ 1 n X, where A denotes the determinant of a matrix A, and Γ n is the variance/covariance matrix of X with the given parameter values. The maximum likelihood estimator (MLE) ofφ,θ,σ 2 w maximizes this quantity. 20
21 Maximum likelihood estimation We can simplify the likelihood by expressing it in terms of the innovations. Since the innovations are linear in previous and current values, we can write X 1 X 1 X1 0. = C. X n }{{} X X n Xn n 1 }{{} U where C is a lower triangular matrix with ones on the diagonal. Take the variance/covariance of both sides to see that Γ n = CDC where D = diag(p 0 1,...,P n 1 n ). 21
22 Thus, Γ n = C 2 P 0 1 P n 1 n Maximum likelihood estimation = P 0 1 P n 1 n and X Γ 1 n X = U C Γ 1 n CU = U C C T D 1 C 1 CU = U D 1 U. So we can rewrite the likelihood as L(φ,θ,σ 2 w) = = 1 ( (2π)n P 0 1 Pn 1 n 1 ( (2πσ 2 w ) n r 0 1 rn 1 n ) 1/2 exp ( ) 1/2 exp n 1 2 i=1 ( S(φ,θ) 2σw 2 (X i X i 1 i ), ) 2 /P i 1 i ) where r i 1 i = P i 1 i /σw 2 and S(φ,θ) = n i=1 ( Xi X i 1 ) 2 i r i 1. i 22
23 Maximum likelihood estimation The log likelihood of φ,θ,σ 2 w is l(φ,θ,σ 2 w) = log(l(φ,θ,σ 2 w)) = n 2 log(2πσ2 w) 1 2 n i=1 logr i 1 i S(φ,θ). 2σw 2 Differentiating with respect toσ 2 w shows that the MLE(ˆφ,ˆθ,ˆσ 2 w) satisfies n 2ˆσ 2 w = and ˆφ,ˆθ minimize S(ˆφ, ˆθ) 2ˆσ 4 w log ( ) S(ˆφ,ˆθ) n ˆσ w 2 = S(ˆφ,ˆθ), n + 1 n n i=1 logr i 1 i. 23
24 Summary: Maximum likelihood estimation The MLE(ˆφ,ˆθ,ˆσ 2 w) satisfies and ˆφ,ˆθ minimize ˆσ w 2 S(ˆφ, ˆθ) =, n ( ) S(ˆφ,ˆθ) log n + 1 n n i=1 logr i 1 i, where r i 1 i = P i 1 i /σw 2 and S(φ,θ) = n i=1 ( Xi X i 1 ) 2 i r i 1. i 24
25 Maximum likelihood estimation Minimization is done numerically (e.g., Newton-Raphson). Computational simplifications: Unconditional least squares. Drop thelogr i 1 i terms. Conditional least squares. Also approximate the computation of x i 1 i dropping initial terms ins. e.g., for AR(2), all but the first two terms ins depend linearly onφ 1,φ 2, so we have a least squares problem. The differences diminish as sample size increases. For example, Pt t 1 σw 2 sort t 1 1, and thusn 1 i logri 1 i 0. by 25
26 Maximum likelihood estimation: Confidence intervals For an ARMA(p,q) process, the MLE and un/conditional least squares estimators satisfy ˆφ 1 φ AN 0, σ2 w Γ φφ Γ φθ, ˆθ θ n Γ θφ Γ θθ, where Γ φθ Γ φφ Γ θφ Γ θθ, = Cov((X,Y),(X,Y)), X = (X 1,...,X p ) φ(b)x t = W t, Y = (Y 1,...,Y p ) θ(b)y t = W t. 26
27 Introduction to Time Series Analysis. Lecture Review: Yule-Walker estimators 2. Yule-Walker example 3. Efficiency 4. Maximum likelihood estimation 5. Large-sample distribution of MLE 27
Introduction to Time Series Analysis. Lecture 11.
Introduction to Time Series Analysis. Lecture 11. Peter Bartlett 1. Review: Time series modelling and forecasting 2. Parameter estimation 3. Maximum likelihood estimator 4. Yule-Walker estimation 5. Yule-Walker
More informationParameter estimation: ACVF of AR processes
Parameter estimation: ACVF of AR processes Yule-Walker s for AR processes: a method of moments, i.e. µ = x and choose parameters so that γ(h) = ˆγ(h) (for h small ). 12 novembre 2013 1 / 8 Parameter estimation:
More informationSTAT 720 sp 2019 Lec 06 Karl Gregory 2/15/2019
STAT 720 sp 2019 Lec 06 Karl Gregory 2/15/2019 This lecture will make use of the tscourse package, which is installed with the following R code: library(devtools) devtools::install_github("gregorkb/tscourse")
More information6.3 Forecasting ARMA processes
6.3. FORECASTING ARMA PROCESSES 123 6.3 Forecasting ARMA processes The purpose of forecasting is to predict future values of a TS based on the data collected to the present. In this section we will discuss
More informationContents. 1 Time Series Analysis Introduction Stationary Processes State Space Modesl Stationary Processes 8
A N D R E W T U L L O C H T I M E S E R I E S A N D M O N T E C A R L O I N F E R E N C E T R I N I T Y C O L L E G E T H E U N I V E R S I T Y O F C A M B R I D G E Contents 1 Time Series Analysis 5
More informationSTAT Financial Time Series
STAT 6104 - Financial Time Series Chapter 4 - Estimation in the time Domain Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 1 / 46 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR
More informationMidterm Suggested Solutions
CUHK Dept. of Economics Spring 2011 ECON 4120 Sung Y. Park Midterm Suggested Solutions Q1 (a) In time series, autocorrelation measures the correlation between y t and its lag y t τ. It is defined as. ρ(τ)
More informationMAT3379 (Winter 2016)
MAT3379 (Winter 2016) Assignment 4 - SOLUTIONS The following questions will be marked: 1a), 2, 4, 6, 7a Total number of points for Assignment 4: 20 Q1. (Theoretical Question, 2 points). Yule-Walker estimation
More informationUniversity of Oxford. Statistical Methods Autocorrelation. Identification and Estimation
University of Oxford Statistical Methods Autocorrelation Identification and Estimation Dr. Órlaith Burke Michaelmas Term, 2011 Department of Statistics, 1 South Parks Road, Oxford OX1 3TG Contents 1 Model
More informationAutoregressive Moving Average (ARMA) Models and their Practical Applications
Autoregressive Moving Average (ARMA) Models and their Practical Applications Massimo Guidolin February 2018 1 Essential Concepts in Time Series Analysis 1.1 Time Series and Their Properties Time series:
More informationSTAT 443 Final Exam Review. 1 Basic Definitions. 2 Statistical Tests. L A TEXer: W. Kong
STAT 443 Final Exam Review L A TEXer: W Kong 1 Basic Definitions Definition 11 The time series {X t } with E[X 2 t ] < is said to be weakly stationary if: 1 µ X (t) = E[X t ] is independent of t 2 γ X
More informationAkaike criterion: Kullback-Leibler discrepancy
Model choice. Akaike s criterion Akaike criterion: Kullback-Leibler discrepancy Given a family of probability densities {f ( ; ψ), ψ Ψ}, Kullback-Leibler s index of f ( ; ψ) relative to f ( ; θ) is (ψ
More informationITSM-R Reference Manual
ITSM-R Reference Manual George Weigt February 11, 2018 1 Contents 1 Introduction 3 1.1 Time series analysis in a nutshell............................... 3 1.2 White Noise Variance.....................................
More informationLevinson Durbin Recursions: I
Levinson Durbin Recursions: I note: B&D and S&S say Durbin Levinson but Levinson Durbin is more commonly used (Levinson, 1947, and Durbin, 1960, are source articles sometimes just Levinson is used) recursions
More informationTime Series I Time Domain Methods
Astrostatistics Summer School Penn State University University Park, PA 16802 May 21, 2007 Overview Filtering and the Likelihood Function Time series is the study of data consisting of a sequence of DEPENDENT
More informationLevinson Durbin Recursions: I
Levinson Durbin Recursions: I note: B&D and S&S say Durbin Levinson but Levinson Durbin is more commonly used (Levinson, 1947, and Durbin, 1960, are source articles sometimes just Levinson is used) recursions
More informationCalculation of ACVF for ARMA Process: I consider causal ARMA(p, q) defined by
Calculation of ACVF for ARMA Process: I consider causal ARMA(p, q) defined by φ(b)x t = θ(b)z t, {Z t } WN(0, σ 2 ) want to determine ACVF {γ(h)} for this process, which can be done using four complementary
More informationLecture 7: Model Building Bus 41910, Time Series Analysis, Mr. R. Tsay
Lecture 7: Model Building Bus 41910, Time Series Analysis, Mr R Tsay An effective procedure for building empirical time series models is the Box-Jenkins approach, which consists of three stages: model
More informationMAT 3379 (Winter 2016) FINAL EXAM (SOLUTIONS)
MAT 3379 (Winter 2016) FINAL EXAM (SOLUTIONS) 15 April 2016 (180 minutes) Professor: R. Kulik Student Number: Name: This is closed book exam. You are allowed to use one double-sided A4 sheet of notes.
More informationIntroduction to Estimation Methods for Time Series models Lecture 2
Introduction to Estimation Methods for Time Series models Lecture 2 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 2 SNS Pisa 1 / 21 Estimators:
More informationLecture 16: State Space Model and Kalman Filter Bus 41910, Time Series Analysis, Mr. R. Tsay
Lecture 6: State Space Model and Kalman Filter Bus 490, Time Series Analysis, Mr R Tsay A state space model consists of two equations: S t+ F S t + Ge t+, () Z t HS t + ɛ t (2) where S t is a state vector
More informationMAT 3379 (Winter 2016) FINAL EXAM (PRACTICE)
MAT 3379 (Winter 2016) FINAL EXAM (PRACTICE) 15 April 2016 (180 minutes) Professor: R. Kulik Student Number: Name: This is closed book exam. You are allowed to use one double-sided A4 sheet of notes. Only
More informationSTAT 443 (Winter ) Forecasting
Winter 2014 TABLE OF CONTENTS STAT 443 (Winter 2014-1141) Forecasting Prof R Ramezan University of Waterloo L A TEXer: W KONG http://wwkonggithubio Last Revision: September 3, 2014 Table of Contents 1
More informationMultivariate Time Series
Multivariate Time Series Notation: I do not use boldface (or anything else) to distinguish vectors from scalars. Tsay (and many other writers) do. I denote a multivariate stochastic process in the form
More informationSTOR 356: Summary Course Notes
STOR 356: Summary Course Notes Richard L. Smith Department of Statistics and Operations Research University of North Carolina Chapel Hill, NC 7599-360 rls@email.unc.edu February 19, 008 Course text: Introduction
More informationIntroduction to Time Series Analysis. Lecture 7.
Last lecture: Introduction to Time Series Analysis. Lecture 7. Peter Bartlett 1. ARMA(p,q) models: stationarity, causality, invertibility 2. The linear process representation of ARMA processes: ψ. 3. Autocovariance
More informationChapter 4: Models for Stationary Time Series
Chapter 4: Models for Stationary Time Series Now we will introduce some useful parametric models for time series that are stationary processes. We begin by defining the General Linear Process. Let {Y t
More informationStatistics 3858 : Maximum Likelihood Estimators
Statistics 3858 : Maximum Likelihood Estimators 1 Method of Maximum Likelihood In this method we construct the so called likelihood function, that is L(θ) = L(θ; X 1, X 2,..., X n ) = f n (X 1, X 2,...,
More informationSTA 6857 Estimation ( 3.6)
STA 6857 Estimation ( 3.6) Outline 1 Yule-Walker 2 Least Squares 3 Maximum Likelihood Arthur Berg STA 6857 Estimation ( 3.6) 2/ 19 Outline 1 Yule-Walker 2 Least Squares 3 Maximum Likelihood Arthur Berg
More information2. An Introduction to Moving Average Models and ARMA Models
. An Introduction to Moving Average Models and ARMA Models.1 White Noise. The MA(1) model.3 The MA(q) model..4 Estimation and forecasting of MA models..5 ARMA(p,q) models. The Moving Average (MA) models
More informationCh 4. Models For Stationary Time Series. Time Series Analysis
This chapter discusses the basic concept of a broad class of stationary parametric time series models the autoregressive moving average (ARMA) models. Let {Y t } denote the observed time series, and {e
More informationUnivariate Time Series Analysis; ARIMA Models
Econometrics 2 Fall 24 Univariate Time Series Analysis; ARIMA Models Heino Bohn Nielsen of4 Outline of the Lecture () Introduction to univariate time series analysis. (2) Stationarity. (3) Characterizing
More informationGaussian processes. Basic Properties VAG002-
Gaussian processes The class of Gaussian processes is one of the most widely used families of stochastic processes for modeling dependent data observed over time, or space, or time and space. The popularity
More informationLecture 3: Autoregressive Moving Average (ARMA) Models and their Practical Applications
Lecture 3: Autoregressive Moving Average (ARMA) Models and their Practical Applications Prof. Massimo Guidolin 20192 Financial Econometrics Winter/Spring 2018 Overview Moving average processes Autoregressive
More informationModel selection using penalty function criteria
Model selection using penalty function criteria Laimonis Kavalieris University of Otago Dunedin, New Zealand Econometrics, Time Series Analysis, and Systems Theory Wien, June 18 20 Outline Classes of models.
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random
More informationPh.D. Qualifying Exam Friday Saturday, January 6 7, 2017
Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017 Put your solution to each problem on a separate sheet of paper. Problem 1. (5106) Let X 1, X 2,, X n be a sequence of i.i.d. observations from a
More informationModule 3. Descriptive Time Series Statistics and Introduction to Time Series Models
Module 3 Descriptive Time Series Statistics and Introduction to Time Series Models Class notes for Statistics 451: Applied Time Series Iowa State University Copyright 2015 W Q Meeker November 11, 2015
More informationCovariances of ARMA Processes
Statistics 910, #10 1 Overview Covariances of ARMA Processes 1. Review ARMA models: causality and invertibility 2. AR covariance functions 3. MA and ARMA covariance functions 4. Partial autocorrelation
More informationECON/FIN 250: Forecasting in Finance and Economics: Section 6: Standard Univariate Models
ECON/FIN 250: Forecasting in Finance and Economics: Section 6: Standard Univariate Models Patrick Herb Brandeis University Spring 2016 Patrick Herb (Brandeis University) Standard Univariate Models ECON/FIN
More informationAssociation studies and regression
Association studies and regression CM226: Machine Learning for Bioinformatics. Fall 2016 Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar Association studies and regression 1 / 104 Administration
More informationFE570 Financial Markets and Trading. Stevens Institute of Technology
FE570 Financial Markets and Trading Lecture 5. Linear Time Series Analysis and Its Applications (Ref. Joel Hasbrouck - Empirical Market Microstructure ) Steve Yang Stevens Institute of Technology 9/25/2012
More informationBIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation
BIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation Yujin Chung November 29th, 2016 Fall 2016 Yujin Chung Lec13: MLE Fall 2016 1/24 Previous Parametric tests Mean comparisons (normality assumption)
More informationLecture 3: Autoregressive Moving Average (ARMA) Models and their Practical Applications
Lecture 3: Autoregressive Moving Average (ARMA) Models and their Practical Applications Prof. Massimo Guidolin 20192 Financial Econometrics Winter/Spring 2018 Overview Moving average processes Autoregressive
More informationLecture 1: Fundamental concepts in Time Series Analysis (part 2)
Lecture 1: Fundamental concepts in Time Series Analysis (part 2) Florian Pelgrin University of Lausanne, École des HEC Department of mathematics (IMEA-Nice) Sept. 2011 - Jan. 2012 Florian Pelgrin (HEC)
More informationEconometrics II Heij et al. Chapter 7.1
Chapter 7.1 p. 1/2 Econometrics II Heij et al. Chapter 7.1 Linear Time Series Models for Stationary data Marius Ooms Tinbergen Institute Amsterdam Chapter 7.1 p. 2/2 Program Introduction Modelling philosophy
More informationCh 6. Model Specification. Time Series Analysis
We start to build ARIMA(p,d,q) models. The subjects include: 1 how to determine p, d, q for a given series (Chapter 6); 2 how to estimate the parameters (φ s and θ s) of a specific ARIMA(p,d,q) model (Chapter
More informationCOS513 LECTURE 8 STATISTICAL CONCEPTS
COS513 LECTURE 8 STATISTICAL CONCEPTS NIKOLAI SLAVOV AND ANKUR PARIKH 1. MAKING MEANINGFUL STATEMENTS FROM JOINT PROBABILITY DISTRIBUTIONS. A graphical model (GM) represents a family of probability distributions
More information1 One-way analysis of variance
LIST OF FORMULAS (Version from 21. November 2014) STK2120 1 One-way analysis of variance Assume X ij = µ+α i +ɛ ij ; j = 1, 2,..., J i ; i = 1, 2,..., I ; where ɛ ij -s are independent and N(0, σ 2 ) distributed.
More informationSystem Identification
System Identification Arun K. Tangirala Department of Chemical Engineering IIT Madras July 26, 2013 Module 6 Lecture 1 Arun K. Tangirala System Identification July 26, 2013 1 Objectives of this Module
More informationIntroduction to ARMA and GARCH processes
Introduction to ARMA and GARCH processes Fulvio Corsi SNS Pisa 3 March 2010 Fulvio Corsi Introduction to ARMA () and GARCH processes SNS Pisa 3 March 2010 1 / 24 Stationarity Strict stationarity: (X 1,
More information2014/2015 Smester II ST5224 Final Exam Solution
014/015 Smester II ST54 Final Exam Solution 1 Suppose that (X 1,, X n ) is a random sample from a distribution with probability density function f(x; θ) = e (x θ) I [θ, ) (x) (i) Show that the family of
More informationWe will only present the general ideas on how to obtain. follow closely the AR(1) and AR(2) cases presented before.
ACF and PACF of an AR(p) We will only present the general ideas on how to obtain the ACF and PACF of an AR(p) model since the details follow closely the AR(1) and AR(2) cases presented before. Recall that
More information{ } Stochastic processes. Models for time series. Specification of a process. Specification of a process. , X t3. ,...X tn }
Stochastic processes Time series are an example of a stochastic or random process Models for time series A stochastic process is 'a statistical phenomenon that evolves in time according to probabilistic
More informationARMA Estimation Recipes
Econ. 1B D. McFadden, Fall 000 1. Preliminaries ARMA Estimation Recipes hese notes summarize procedures for estimating the lag coefficients in the stationary ARMA(p,q) model (1) y t = µ +a 1 (y t-1 -µ)
More informationLecture 2: Univariate Time Series
Lecture 2: Univariate Time Series Analysis: Conditional and Unconditional Densities, Stationarity, ARMA Processes Prof. Massimo Guidolin 20192 Financial Econometrics Spring/Winter 2017 Overview Motivation:
More informationStatistics - Lecture One. Outline. Charlotte Wickham 1. Basic ideas about estimation
Statistics - Lecture One Charlotte Wickham wickham@stat.berkeley.edu http://www.stat.berkeley.edu/~wickham/ Outline 1. Basic ideas about estimation 2. Method of Moments 3. Maximum Likelihood 4. Confidence
More informationSTAD57 Time Series Analysis. Lecture 8
STAD57 Time Series Analysis Lecture 8 1 ARMA Model Will be using ARMA models to describe times series dynamics: ( B) X ( B) W X X X W W W t 1 t1 p t p t 1 t1 q tq Model must be causal (i.e. stationary)
More informationCHANGE DETECTION IN TIME SERIES
CHANGE DETECTION IN TIME SERIES Edit Gombay TIES - 2008 University of British Columbia, Kelowna June 8-13, 2008 Outline Introduction Results Examples References Introduction sunspot.year 0 50 100 150 1700
More informationCentral Limit Theorem ( 5.3)
Central Limit Theorem ( 5.3) Let X 1, X 2,... be a sequence of independent random variables, each having n mean µ and variance σ 2. Then the distribution of the partial sum S n = X i i=1 becomes approximately
More informationAR, MA and ARMA models
AR, MA and AR by Hedibert Lopes P Based on Tsay s Analysis of Financial Time Series (3rd edition) P 1 Stationarity 2 3 4 5 6 7 P 8 9 10 11 Outline P Linear Time Series Analysis and Its Applications For
More informationARMA MODELS Herman J. Bierens Pennsylvania State University February 23, 2009
1. Introduction Given a covariance stationary process µ ' E[ ], the Wold decomposition states that where U t ARMA MODELS Herman J. Bierens Pennsylvania State University February 23, 2009 with vanishing
More informationFigure 29: AR model fit into speech sample ah (top), the residual, and the random sample of the model (bottom).
Original 0.4 0.0 0.4 ACF 0.5 0.0 0.5 1.0 0 500 1000 1500 2000 0 50 100 150 200 Residual 0.05 0.05 ACF 0 500 1000 1500 2000 0 50 100 150 200 Generated 0.4 0.0 0.4 ACF 0.5 0.0 0.5 1.0 0 500 1000 1500 2000
More informationLecture 32: Asymptotic confidence sets and likelihoods
Lecture 32: Asymptotic confidence sets and likelihoods Asymptotic criterion In some problems, especially in nonparametric problems, it is difficult to find a reasonable confidence set with a given confidence
More informationStat 5100 Handout #12.e Notes: ARIMA Models (Unit 7) Key here: after stationary, identify dependence structure (and use for forecasting)
Stat 5100 Handout #12.e Notes: ARIMA Models (Unit 7) Key here: after stationary, identify dependence structure (and use for forecasting) (overshort example) White noise H 0 : Let Z t be the stationary
More informationPh.D. Qualifying Exam Friday Saturday, January 3 4, 2014
Ph.D. Qualifying Exam Friday Saturday, January 3 4, 2014 Put your solution to each problem on a separate sheet of paper. Problem 1. (5166) Assume that two random samples {x i } and {y i } are independently
More informationFor iid Y i the stronger conclusion holds; for our heuristics ignore differences between these notions.
Large Sample Theory Study approximate behaviour of ˆθ by studying the function U. Notice U is sum of independent random variables. Theorem: If Y 1, Y 2,... are iid with mean µ then Yi n µ Called law of
More informationProblem Selected Scores
Statistics Ph.D. Qualifying Exam: Part II November 20, 2010 Student Name: 1. Answer 8 out of 12 problems. Mark the problems you selected in the following table. Problem 1 2 3 4 5 6 7 8 9 10 11 12 Selected
More informationSTOR 356: Summary Course Notes Part III
STOR 356: Summary Course Notes Part III Richard L. Smith Department of Statistics and Operations Research University of North Carolina Chapel Hill, NC 27599-3260 rls@email.unc.edu April 23, 2008 1 ESTIMATION
More informationREGRESSION WITH SPATIALLY MISALIGNED DATA. Lisa Madsen Oregon State University David Ruppert Cornell University
REGRESSION ITH SPATIALL MISALIGNED DATA Lisa Madsen Oregon State University David Ruppert Cornell University SPATIALL MISALIGNED DATA 10 X X X X X X X X 5 X X X X X 0 X 0 5 10 OUTLINE 1. Introduction 2.
More information1 Class Organization. 2 Introduction
Time Series Analysis, Lecture 1, 2018 1 1 Class Organization Course Description Prerequisite Homework and Grading Readings and Lecture Notes Course Website: http://www.nanlifinance.org/teaching.html wechat
More informationEconometría 2: Análisis de series de Tiempo
Econometría 2: Análisis de series de Tiempo Karoll GOMEZ kgomezp@unal.edu.co http://karollgomez.wordpress.com Segundo semestre 2016 III. Stationary models 1 Purely random process 2 Random walk (non-stationary)
More informationLecture on ARMA model
Lecture on ARMA model Robert M. de Jong Ohio State University Columbus, OH 43210 USA Chien-Ho Wang National Taipei University Taipei City, 104 Taiwan ROC October 19, 2006 (Very Preliminary edition, Comment
More informationChapter 6: Model Specification for Time Series
Chapter 6: Model Specification for Time Series The ARIMA(p, d, q) class of models as a broad class can describe many real time series. Model specification for ARIMA(p, d, q) models involves 1. Choosing
More informationDiscrete time processes
Discrete time processes Predictions are difficult. Especially about the future Mark Twain. Florian Herzog 2013 Modeling observed data When we model observed (realized) data, we encounter usually the following
More informationEstimation Theory. as Θ = (Θ 1,Θ 2,...,Θ m ) T. An estimator
Estimation Theory Estimation theory deals with finding numerical values of interesting parameters from given set of data. We start with formulating a family of models that could describe how the data were
More informationSTAD57 Time Series Analysis. Lecture 14
STAD57 Time Series Analysis Lecure 14 1 Maximum Likelihood AR(p) Esimaion Insead of Yule-Walker (MM) for AR(p) model, can use Maximum Likelihood (ML) esimaion Likelihood is join densiy of daa {x 1,,x n
More informationECON/FIN 250: Forecasting in Finance and Economics: Section 8: Forecast Examples: Part 1
ECON/FIN 250: Forecasting in Finance and Economics: Section 8: Forecast Examples: Part 1 Patrick Herb Brandeis University Spring 2016 Patrick Herb (Brandeis University) Forecast Examples: Part 1 ECON/FIN
More informationStatistical Inference
Statistical Inference Classical and Bayesian Methods Revision Class for Midterm Exam AMS-UCSC Th Feb 9, 2012 Winter 2012. Session 1 (Revision Class) AMS-132/206 Th Feb 9, 2012 1 / 23 Topics Topics We will
More informationTime Series: Theory and Methods
Peter J. Brockwell Richard A. Davis Time Series: Theory and Methods Second Edition With 124 Illustrations Springer Contents Preface to the Second Edition Preface to the First Edition vn ix CHAPTER 1 Stationary
More informationParametric Techniques Lecture 3
Parametric Techniques Lecture 3 Jason Corso SUNY at Buffalo 22 January 2009 J. Corso (SUNY at Buffalo) Parametric Techniques Lecture 3 22 January 2009 1 / 39 Introduction In Lecture 2, we learned how to
More informationModule 4. Stationary Time Series Models Part 1 MA Models and Their Properties
Module 4 Stationary Time Series Models Part 1 MA Models and Their Properties Class notes for Statistics 451: Applied Time Series Iowa State University Copyright 2015 W. Q. Meeker. February 14, 2016 20h
More informationComputing the MLE and the EM Algorithm
ECE 830 Fall 0 Statistical Signal Processing instructor: R. Nowak Computing the MLE and the EM Algorithm If X p(x θ), θ Θ, then the MLE is the solution to the equations logp(x θ) θ 0. Sometimes these equations
More informationReview Session: Econometrics - CLEFIN (20192)
Review Session: Econometrics - CLEFIN (20192) Part II: Univariate time series analysis Daniele Bianchi March 20, 2013 Fundamentals Stationarity A time series is a sequence of random variables x t, t =
More informationMS&E 226: Small Data. Lecture 11: Maximum likelihood (v2) Ramesh Johari
MS&E 226: Small Data Lecture 11: Maximum likelihood (v2) Ramesh Johari ramesh.johari@stanford.edu 1 / 18 The likelihood function 2 / 18 Estimating the parameter This lecture develops the methodology behind
More informationLecture 1: Stationary Time Series Analysis
Syllabus Stationarity ARMA AR MA Model Selection Estimation Lecture 1: Stationary Time Series Analysis 222061-1617: Time Series Econometrics Spring 2018 Jacek Suda Syllabus Stationarity ARMA AR MA Model
More informationModel Identification for Time Series with Dependent Innovations
Model Identification for Time Series with Dependent Innovations Shuhao Chen 1, Wanli Min 2 and Rong Chen 1,3 1 Rutgers University, 2 IBM Singapore and 3 Peking University Abstract: This paper investigates
More informationLecture 3. Inference about multivariate normal distribution
Lecture 3. Inference about multivariate normal distribution 3.1 Point and Interval Estimation Let X 1,..., X n be i.i.d. N p (µ, Σ). We are interested in evaluation of the maximum likelihood estimates
More informationLecture 2: ARMA(p,q) models (part 2)
Lecture 2: ARMA(p,q) models (part 2) Florian Pelgrin University of Lausanne, École des HEC Department of mathematics (IMEA-Nice) Sept. 2011 - Jan. 2012 Florian Pelgrin (HEC) Univariate time series Sept.
More informationBTRY 4090: Spring 2009 Theory of Statistics
BTRY 4090: Spring 2009 Theory of Statistics Guozhang Wang September 25, 2010 1 Review of Probability We begin with a real example of using probability to solve computationally intensive (or infeasible)
More informationParametric Techniques
Parametric Techniques Jason J. Corso SUNY at Buffalo J. Corso (SUNY at Buffalo) Parametric Techniques 1 / 39 Introduction When covering Bayesian Decision Theory, we assumed the full probabilistic structure
More informationCh 8. MODEL DIAGNOSTICS. Time Series Analysis
Model diagnostics is concerned with testing the goodness of fit of a model and, if the fit is poor, suggesting appropriate modifications. We shall present two complementary approaches: analysis of residuals
More informationEcon 623 Econometrics II Topic 2: Stationary Time Series
1 Introduction Econ 623 Econometrics II Topic 2: Stationary Time Series In the regression model we can model the error term as an autoregression AR(1) process. That is, we can use the past value of the
More informationLecture 15. Hypothesis testing in the linear model
14. Lecture 15. Hypothesis testing in the linear model Lecture 15. Hypothesis testing in the linear model 1 (1 1) Preliminary lemma 15. Hypothesis testing in the linear model 15.1. Preliminary lemma Lemma
More informationMultivariate Time Series: VAR(p) Processes and Models
Multivariate Time Series: VAR(p) Processes and Models A VAR(p) model, for p > 0 is X t = φ 0 + Φ 1 X t 1 + + Φ p X t p + A t, where X t, φ 0, and X t i are k-vectors, Φ 1,..., Φ p are k k matrices, with
More informationResidual Bootstrap for estimation in autoregressive processes
Chapter 7 Residual Bootstrap for estimation in autoregressive processes In Chapter 6 we consider the asymptotic sampling properties of the several estimators including the least squares estimator of the
More informationTIME SERIES ANALYSIS. Forecasting and Control. Wiley. Fifth Edition GWILYM M. JENKINS GEORGE E. P. BOX GREGORY C. REINSEL GRETA M.
TIME SERIES ANALYSIS Forecasting and Control Fifth Edition GEORGE E. P. BOX GWILYM M. JENKINS GREGORY C. REINSEL GRETA M. LJUNG Wiley CONTENTS PREFACE TO THE FIFTH EDITION PREFACE TO THE FOURTH EDITION
More informationForecasting with ARMA
Forecasting with ARMA Eduardo Rossi University of Pavia October 2013 Rossi Forecasting Financial Econometrics - 2013 1 / 32 Mean Squared Error Linear Projection Forecast of Y t+1 based on a set of variables
More informationApplied time-series analysis
Robert M. Kunst robert.kunst@univie.ac.at University of Vienna and Institute for Advanced Studies Vienna October 18, 2011 Outline Introduction and overview Econometric Time-Series Analysis In principle,
More informationEstimation and Model Selection in Mixed Effects Models Part I. Adeline Samson 1
Estimation and Model Selection in Mixed Effects Models Part I Adeline Samson 1 1 University Paris Descartes Summer school 2009 - Lipari, Italy These slides are based on Marc Lavielle s slides Outline 1
More information