Teletrac modeling and estimation

Size: px
Start display at page:

Download "Teletrac modeling and estimation"

Transcription

1 Teletrac modeling and estimation File 1 José Roberto Amazonas Alexandre Barbosa de Lima jra,ablima@lcs.poli.usp.br Telecommunications and Control Engineering Dept. - PTC Escola Politécnica University of São Paulo - USP São Paulo 2009 JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

2 Outline 1 Introduction Motivation Objectives and contributions 2 Time series basic concepts Time series examples Operators notation Stochastic processes Time Series Modeling Auto-regressive models Non-stationary series modeling JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

3 Outline 1 Introduction Motivation Objectives and contributions 2 Time series basic concepts Time series examples Operators notation Stochastic processes Time Series Modeling Auto-regressive models Non-stationary series modeling JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

4 Introduction Motivation Outline 1 Introduction Motivation Objectives and contributions 2 Time series basic concepts Time series examples Operators notation Stochastic processes Time Series Modeling Auto-regressive models Non-stationary series modeling JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

5 Introduction Motivation Telecommunications carriers objectives Integration of voice, data, video, image and data over a single packet switched convergent network. This network is also known as: integrated services, multi-service or next generation network. Requirements: acceptable performance for each kind of service; operational costs reduction; exibility to support current and future services; dynamic bandwidth allocation; integrated transport of all kinds of information; ecient usage of network resources by means of statistical multiplexing. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

6 Introduction Motivation Quality of service - QoS The convergent network's core consists of a single IP infrastructure supporting Quality of Service (QoS), virtual private networks and IP protocols v4 and v6; QoS = f(pdt, PDV, THRU, PLR); IETF QoS implementation proposals: MPLS constraint-based routing trac engineering IntServ DiServ JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

7 Introduction Motivation Trac control The implementation of mechanisms to monitor and to control the teletrac is a must for an ecient operation of convergent networks. Without trac control, the unbounded demand of shared resources (buers, band and processors) may seriously degrade the network performance. The trac control is necessary to protect the QoS perceived by the users and to ensure the ecient usage of network resources. Trac control functions: a) CAC - Call Admission Control b) UPC - Usage Parameter Control c) Priority control d) Congestion control Functions a), c) and d) require on-line trac prediction. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

8 Introduction Motivation Trac characteristics Measurements have shown that the data network's aggregate trac has statistical properties quite dierent from the existing trac in the PSTN. Data trac is the time series of the bytes count per unit of time. A. K. Erlang showed in 1909 that the number of generated telephone calls in a certain time interval can be modeled by the Poisson process. By contrast, the data trac traces (series) have fractal properties as long range dependence (LRD) or self-similarity, and impulsiveness (great variability), in several time aggregation scales, that are not captured by the Poisson process. The trac in some LANs and WANs is extremely impulsive because it has a heavy tail marginal distribution, i. e., non-gaussian distributions. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

9 Introduction Motivation Trac characteristics - 2 It is a known fact that the buer overow probability, with just one server working at a constant service rate, under Markovian trac is an exponential function of the buer size. an increase of the buer size produces a signicative decrease of the PLR. On the other hand, data trac long memory and impulsiveness degrade the network's performance (PLR increases) because the the buer overow probability in systems under fractal trac is a power function of the buer size, i. e., a buer queue hyperbolic decay much slower than the Markovian case. Self-similar trac produces buer overow much more frequently than the Markovian trac. Such phenomena have been observed in the beginning of the 1990 decade in ATM networks implemented with small buers (10 thru 100 cells). JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

10 Introduction Objectives and contributions Outline 1 Introduction Motivation Objectives and contributions 2 Time series basic concepts Time series examples Operators notation Stochastic processes Time Series Modeling Auto-regressive models Non-stationary series modeling JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

11 Introduction Objectives and contributions Questions to be answered How can we make prediction of signals that present LRD? How can we apply the theory of time series prediction to the telecommunications network real trac? Is teletrac a linear or non-linear signal? Is it Gaussian or non-gaussian? How should we make the model's parameters estimation? How can we identify the adequate model? If the prediction can be made using a linear estimation theory, what are the available techniques? JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

12 Introduction Objectives and contributions Contributions Development of a new fractal teletrac model in the state space. Proposal of a LRD teletrac statistical analysis methodology. Demonstration that long memory signals modeling by means of high-order AR processes may be unfeasible in practice. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

13 Time series examples Outline 1 Introduction Motivation Objectives and contributions 2 Time series basic concepts Time series examples Operators notation Stochastic processes Time Series Modeling Auto-regressive models Non-stationary series modeling JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

14 Time series examples Johnson & Johnson quarterly earnings per share Quarterly earnings per share Time JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

15 Time series examples Global warming Yearly average global temperature deviations Time JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

16 Time series examples Speech signal: aaa...hhh Speech signal Time JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

17 Time series examples New York Stock Exchange Returns of the NYSE Time JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

18 Time series examples New York Stock Exchange Earthquake EQ Time Explosion EXP Time JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

19 Operators notation Outline 1 Introduction Motivation Objectives and contributions 2 Time series basic concepts Time series examples Operators notation Stochastic processes Time Series Modeling Auto-regressive models Non-stationary series modeling JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

20 Operators notation Notation (a) 1 sample delay operator, denoted by B: Bx t = x t 1. (1) (b) m samples delay operator, denoted by B m, m Z: B m x t = x t m. (2) The impulse response h t of the m samples delay system is: h t = δ t m, (3) and its transfer function, dened as the z transform(h(z) = t= h t z t ) of the impulse response is: H(z) = z m. (4) JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

21 Operators notation Notation 2 (c) dierence or backshift operator x t = x t x t 1 = (1 B)x t, (5) B j (x t ) = x t j and j (x t ) = ( j 1 (x t )), j 1, with 0 (x t ) = x t 2 x t = ( x t ) = (1 B)(1 B)x t = (1 2B + B 2 )x t = x t 2x t 1 + x t 2. The dierence operator has impulse response h t = δ t δ t 1, (6) and transfer function H(z) = (1 z 1 ). (7) JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

22 Operators notation Notation 3 (d) summation operator or integrator lter, denoted by S: Sx t = x t i = x t + x t 1 + x t = i=0 = (1 + B + B )x t = (1 B) 1 x t = 1 x t. (8) The integrator lter's transfer function corresponds to the inverse of the system's function dened by (7), i. e., H(z) = (1 z 1 ) 1. (9) JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

23 Stochastic processes Outline 1 Introduction Motivation Objectives and contributions 2 Time series basic concepts Time series examples Operators notation Stochastic processes Time Series Modeling Auto-regressive models Non-stationary series modeling JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

24 Stochastic processes Denitions Denition (Stochastic Process) Let T be an arbitrary set. A stochastic process is a family {x t, t T }, such that, for each t T, x t is a random variable. When the set T is the set of integer numbers Z, then {x t } is a discrete time stochastic process (or random sequence); {x t } is a continuous time stochastic process if T is taken as the set of real numbers R. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

25 Stochastic processes Denitions 2 The random variable x t is, in fact, a function of two arguments x(t, ζ), t T, ζ Ω, given that it is dened over the sample space Ω. For each ζ Ω we have a realization, trajectory or time series x t. The set of all realizations is called ensemble. Each trajectory is a function or a non-random sequence and for each xed t, x t is a number. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

26 Stochastic processes Probability distribution functions A process x t is completely specied by its nite-dimensional distributions or n-order probability distribution functions. F x (x 1, x 2,..., x n ; t 1, t 2,..., t n ) = P{x(t 1 ) x 1, x(t 2 ) x 2,..., x(t n ) x n } (10) in which t 1, t 2,..., t n are any elements of T and n 1. The rst order probability distribution function is also known as Cumulative Distribution Function - CDF. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

27 Stochastic processes Probability density function The probability density function - PDF is given by: f x (x 1, x 2,..., x n ; t 1, t 2,..., t n ) = n F x (x 1, x 2,..., x n ; t 1, t 2,..., t n ) x 1 x 2... x n. Applying the conditional probability density formula, (11) f x (x k x k 1,..., x 1 ) = f x(x 1,..., x k 1, x k ), (12) f x (x 1,..., x k 1 ) in which f x (x 1,..., x k 1, x k ) denotes f x (x 1,..., x k 1, x k ; t 1,..., t k 1, t k ), repeatedly over f x (x 1,..., x n 1, x n ) we get the probability chain rule f x (x 1, x 2,..., x n ) = f x (x 1 )f x (x 2 x 1 )f x (x 3 x 2, x 1 )... f x (x n x n 1,..., x 1 ). (13) JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

28 Stochastic processes Independent and IID process When x t is a sequence of mutually independent random variables, (13) can be rewritten as f x (x 1, x 2,..., x n ) = f x (x 1 )f x (x 2 )... f x (x n ). (14) Denition (Purely Stochastic Process) A purely stochastic process {x t, t Z} is a sequence of mutually independent random variables. Denition (Independent and Identically Distributed Process) An Independent and Identically Distributed (IID) process {x t, t Z}, denoted by x t IID, is a purely stochastic and identically distributed process. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

29 Stochastic processes Stationarity Denition (Strict Sense Stationarity) A random process x t is stationary in the strict sense if F x (x 1, x 2,..., x n ; t 1, t 2,..., t n ) = F x (x 1, x 2,..., x n ; t 1 +c, t 2 +c,..., t n +c), for any c. (15) JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

30 Stochastic processes Low order moments The mean µ x (t) of x t is the expected value of the random variable x t : µ x (t) = E[x t ] = xf x (x; t)dx, (16) in which f x (x; t) is the rst order probability density function of x t. The autocorrelation R x (t 1, t 2 ) of x t is the expected value of the product x t1 x t2 : R x (t 1, t 2 ) = E[x t1 x t2 ] = x 1 x 2 f x (x 1, x 2 ; t 1, t 2 )dx 1 dx 2, (17) in which f x (x 1, x 2 ; t 1, t 2 ) is the second order probability density function of x t. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

31 Stochastic processes Low order moments 2 The autocovariance C x (t 1, t 2 ) of x t is the covariance of the random variables x t1 and x t2 : C x (t 1, t 2 ) = R x (t 1, t 2 ) µ t1 µ t2. (18) The correlation coecient ρ x (t 1, t 2 ) of x t is the ratio: ρ x (t 1, t 2 ) = C x (t 1, t 2 ) Cx (t 1, t 1 )C x (t 2, t 2 ). (19) Many authors refer to ρ x (t 1, t 2 ) as Autocorrelation Function (ACF). JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

32 Stochastic processes Wide Sense Stationarity Denition (Wide Sense Stationarity) A random process x t is wide sense stationary if its mean is constant E[x t ] = µ x, (20) and if its autocorrelation depends only on the lag τ = t 2 t 1 : R x (t 1, t 2 ) = R x (t 1, t 1 + τ) = R x (τ). (21) Observe that (21) implies an autocovariance that depends on the lag, i. e., C x (τ). In this case, the stationary process' variance is constant and given by σ 2 x = C x (0). (22) JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

33 Stochastic processes Non-stationarity Real series generally present the following types of non-stationarity: (a) level-based non-stationarity: the series oscillate around and average level during a certain time and then jump to another level. The rst dierence makes these series stationary. (b) slope-based non-stationarity: the series oat around a straight line, with positive or negative slope. The second dierence makes these series stationary. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

34 Stochastic processes White noise Denition (White Independent Noise) A process w t IID is a White Independent Noise WIN when it has mean µ w and variance σ 2 w, w t WIN(µ w, σ 2 w). Denition (White Noise) A sequence {w t, t Z} of non-correlated with mean µ w and variance σ 2 w is called White Noise WN, w t WN(µ w, σ 2 w). Observation (White Gaussian Noise) A White Gaussian Noise (WGN) w t is a WIN and denoted by w t N (µ w, σ 2 w), in which N stands for the probabilities normal (Gaussian) distribution. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

35 Stochastic processes Sample mean and autocovariance Consider a stationary process x t. The sample mean of a realization x t with N points is given by x = 1 N N x t, (23) t=1 the sample autocovariance of lag τ by Ĉ τ = 1 N (x t x)(x t τ x). (24) N t=τ+1 JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

36 Stochastic processes Sample autocorrelation and variance The sample autocorrelation (SACF) of lag τ by in which Ĉ 0 (also denoted as s 2 x ) ˆρ τ = Ĉτ Ĉ 0, (25) Ĉ 0 = s 2 x = 1 N N (x t x) 2 (26) t=1 is the sample variance of x t. The sample variance can also be dened as s 2 x = 1 N 1 N t=1 (x t x) 2. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

37 Stochastic processes Ergodicity Denition (Ergodicidade) A stationary process x t is called ergodic if its main moments converge in probability a to the population's moments, i. e., if x p p µ, Ĉ τ C τ e p ˆρ τ ρ τ. a We say that the sequence {x1, x2,..., xn,...} converges in probability to x if lim P( xn x ɛ) = 0 for all ɛ > 0. n JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

38 Time Series Modeling Outline 1 Introduction Motivation Objectives and contributions 2 Time series basic concepts Time series examples Operators notation Stochastic processes Time Series Modeling Auto-regressive models Non-stationary series modeling JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

39 Time Series Modeling Introduction A time seriesx t modeling consists on estimating an invertible function h(.), called model of x t, such that in which w t IID and x t = h(..., w t 2, w t 1, w t, w t+1, w t+2,...), (27) g(..., x t 2, x t 1, x t, x t+1, x t+2,...) = w t, (28) in which g(.) = h 1 (.). The process w t is the innovation at instant t and represents the new information about the series that is obtained at instant t. In practice, the adjusted model is causal, i. e., x t = h(w t, w t 1, w t 2,...). (29) JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

40 Time Series Modeling Model construction methodology The model construction methodology is based on the iterative cycle illustrated by the following steps: (a) a models general class is considered for analysis (specication); (b) there is the identication of a model, based on statistical criteria; (c) it follows the estimation phase, in which the model's parameters are obtained. In practice, it is important that the model is parsimonious 1 and (d) at last, there is the diagnostic of the adjusted model by means of a statistical analysis of the series of residues w t (is w t compatible with a WN?) 1 We say that a model is parsimonious when it uses few parameters. The use of an excessive number of parameters is undesirable because the uncertainty degree of the statistical inference procedure increases with the number of parameters. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

41 Time Series Modeling Model construction methodology 2 Postulate general class of the model Model Identification Estimation of parameters Diagnosis No Yes Figure: Box-Jenkins' iterative cycle. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

42 Time Series Modeling Linear process The process x t of (29) is linear when it corresponds to the convolution of a process w t IID and a deterministic sequence h t x t = h t w t = h k w t k k=0 = w t + h 1 w t 1 + h 2 w t (30) = (1 + h 1 B + h 2 B )w t = H(B)w t in which the symbol denotes the convolution operation and h 0 = 1. Eq. (30) is also known as the innite order moving average (MA( )) representation. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

43 Time Series Modeling ARMA model The linear lter general format of (30) is x(t) = p φ k x(t k) + w(t) k=1 q θ k w(t k). (31) k=1 The sequence h t is called impulse response of (31), also known as ARMA model of orders p and q (ARMA(p, q)). In a more compact format, we have φ(b)x t = θ(b)w t, (32) in which φ(b) is the order p auto-regressive operator φ(b) = 1 φ 1 B φ 2 B 2... φ p B p (33) and θ(b) is the q order moving average operator θ(b) = 1 θ 1 B θ 2 B 2... θ q B q. (34) JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

44 Time Series Modeling ARMA model 2 If all poles and zeros of the transfer function H(z) = h k z k = θ(z) φ(z) k=0 (35) of the lter (32) are inside the unity radius circle φ(z) = 0, z < 1 (36) θ(z) = 0, z < 1 (37) then the process x t given by (30) is stationary (or non-explosive) and invertible, respectively. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

45 Time Series Modeling α-stable distribution A random variable X with α-stable distribution has a heavy tail and is dened by the following characteristic function: Z Φ X (w) = E[e jwx ] = f X (x)e jwx dx = exp{jµw σw α [1 jη sign(w)ϕ(w, α)]}, (38) in which, 8 >< tan (απ/2) se α 1 ϕ(w, α) = (39) >: 2 π ln w se α = 1, and sign(.) is the sign function, α (0 < α 2) is the characteristic exponent, µ (µ R) is the localization parameter, η ( 1 η 1) is the asymmetry parameter and σ 0 is the dispersion parameter or scale. The variance of X is innite for 0 < α < 2. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

46 Time Series Modeling Non-Gaussian process When the innovations in (30) are random variables with α-stable distribution, (30) denes an innite variance non-gaussian process. This fact is justied by the generalized central limit theorem, which states that, if the limit of an IID random variables sum converges, then this limit can only be a random variable with a stable distribution (the normal distribution is a particular case of the stable distribution 2 ). 2 When α = 2. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

47 Time Series Modeling Non-linearity and non-gaussianity If the innovations in (30) have nite variance, i. e., w t IWN(µ w, σ 2 w), then x t is Gaussian when h t has innite duration (central limit theorem). Any non-linearity in the function h(.) of (29) implies a non-linear process x t. In this case, x t has statistics that are necessarily non-gaussian. On the other hand, Gaussian processes are necessarily linear. The rest of this Section assumes that the innovations in (30) are of the kind w t IWN(µ w, σ 2 w). JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

48 Time Series Modeling IWN's autocorrelation and SDF The IWN's w t autocorrelation in (30) is in which δ τ is the discrete time unit pulse 3. Then, its spectral density function or SDP is R w(τ) = σ 2 wδ τ + µ 2 w, (40) S w (f ) = µ 2 wδ(f ) + σ 2 w, 1/2 f 1/2, (41) in which f is the normalized frequency and δ(f ) is the Impulse generalized function (or Dirac's Delta). The w t 's SDF is dened as the Discrete Time Fourier Transform (DFT) of its autocorrelation R w(τ), i. e., S w (f ) = 3 δ τ = 1 for τ = 0, δ τ = 0 para τ 0, τ Z. m= R w(m) e j2πfm JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

49 Time Series Modeling The linear process x t 's SDF and auto-covariance The linear process x t 's SDF is S x (f ) = H(f ) 2 S w (f ) = µ 2 w H(0) 2 δ(f ) + σ 2 w H(f ) 2, 1/2 f 1/2, (42) in which H(f ) = H(z) z=e j2πf is the lter's frequency response. The x t 's auto-covariance is given by C x(τ) = σ 2 w h t h t+τ, (43) t=0 and its variance by σ 2 x = σ 2 w h 2 t. (44) t=0 JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

50 Time Series Modeling AR( ) model As, in practice, the estimated models are invertible (i. e., (37) is valid), we can dene the inverse operator G(B) = H 1 (B) and rewrite (30) in the innite order auto-regressive format (AR( )) x t = g 1 x t 1 + g 2 x t w t = g k x t k + w t. k=1 (45) So, x t may be interpreted as a weighted sum of its past values x t 1, x t 2,... plus an innovation w t. The equivalent model AR( ) suggests that we can compute the probability of a future value x t+k being between two specied bounds, i. e., (45) states that it is possible to make inferences or predictions of the series' future values. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

51 Auto-regressive models Outline 1 Introduction Motivation Objectives and contributions 2 Time series basic concepts Time series examples Operators notation Stochastic processes Time Series Modeling Auto-regressive models Non-stationary series modeling JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

52 Auto-regressive models AR(p) model An order p auto-regressive model satises the equation in which φ(b) is an order p polynomial. φ(b)x t = w t. (46) JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

53 Auto-regressive models Auto-correlation function Multiplying both sides of (46) and x t k and taking the expectation we get E[x t x t k ] = φ 1 E[x t 1 x t k ] + φ 2 E[x t 2 x t k ] φ pe[x t p x t k ] + E[w t x t k ], as x t k does not depend on w t, but only on the noise up to instant t k, that are not correlated with w t, then E[w t x t k] = 0, k > 0, and C x (k) = φ 1 C x (k 1) + φ 2 C x (k 2) φ p C x (k p), k > 0. (47) Dividing (47) by C x (0) = σ 2, we get x ρ x (k) = φ 1 ρ x (k 1) + φ 2 ρ x (k 2) φ p ρ x (k p), k > 0, (48) or φ(b)ρ x (k) = 0. (49) JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

54 Auto-regressive models Auto-correlation function 2 Let G 1, i = 1,..., p, be the roots of φ(b) = 0 (model's characteristic i equation). Then, we can write p φ(b) = (1 G i B), i=1 and it can be shown that the general solution of (49) is ρ x (k) = A 1 G k 1 + A 2 G k A p G k, (50) p in which the constants A i, i = 1, 2,..., p, are determined by initial conditions over ρ x (0), ρ x (1),..., ρ x (p 1). As the roots of φ(b) = 0 must be out of unit radius circle, we must have G k < 1, k = 1,..., p. So, the ACF plot of an AR(p) process will normally show a mixture of damping sine and cosine patterns and exponential decays. As B = z 1, then Eqs. 36 and 37 may be written as φ(b) = 0 for B > 1 and θ(b) = 0 for B > 1. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

55 Auto-regressive models Auto-correlation function 3 For example, consider an AR(2) model x t = φ 1 x t 1 + φ 2 x t 2 + w t and its ACF, which satises the second order dierence equation ρ k = φ 1 ρ k 1 + φ 2 ρ k 2, k > 0 with initial values ρ 0 = 1 e ρ 1 = φ 1 1 φ 2. From (50), the general solution of this equation is ρ k = G 1(1 G 2 2 )G k 1 G 2(1 G 2 1 )G k 2 (G 1 G 2 )(1 + G 1 G 2 ). JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

56 Auto-regressive models Identication In practice, the order p of an AR series is unknown and must be empirically specied. There are two approaches: i) use of the Partial Auto-correlation Function (FACP); ii) use of some model's selection (identication) criterium. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

57 Auto-regressive models PACF of AR(p) models Let φ mi be the i-th coecient of an AR(m) process, so the last coecient is φ mm. Making k = 1,..., m in (48) (in the following, we adopt the simplied notation ρ x (k) = ρ k ) and considering that ρ k = ρ k (ACF's even symmetry), we get the Yule-Walker equations ρ 1 = φ m1 + φ m2 ρ φ mm ρ m 1, ρ 2 = φ m1 ρ 1 + φ m φ mm ρ m 2,. ρ m = φ m1 ρ m 1 + φ m2 ρ m φ mm, (51) JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

58 Auto-regressive models PACF of AR(p) models 2 that may be written in matrix format 1 ρ 1... ρ m 1 ρ ρ m ρ m 1 ρ m φ m1 φ m2. φ mm ρ 1 ρ 2 ρ m =. (52) or in its compact format R m φ m = ρ m, (53) in which R m is the order m autocorrelations matrix, φ m the model's parameters vector and ρ m is the autocorrelations vector. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

59 Auto-regressive models PACF of AR(p) models 3 Solving (53) for m = 1, 2,..., we get φ 11 = ρ 1 1 ρ 1 ρ 1 ρ 2 φ 22 = 1 ρ 1 ρ ρ 1 ρ 1 ρ 1 1 ρ 2 ρ 2 ρ 1 ρ 3 φ 33 = 1 ρ 1 ρ 2 ρ 1 1 ρ 1 ρ 2 ρ 1 1 (54) JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

60 Auto-regressive models PACF of AR(p) models 4 And, in general, φ mm = R m R m, (55) in which R m is the m order autocorrelations matrix' determinant and R m is matrix R m with the last column replaced by the autocorrelations vector. The sequence {φ mm, m = 1, 2,...} is the PACF. It can be shown that an AR(p) model has φ mm 0 for m p and φ mm = 0 for m > p. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

61 Auto-regressive models PACF of AR(p) models 5 The PACF may be estimated by adjusting the AR(m), m = 1, 2,... models sequence x t = φ 11 x t 1 + w 1t x t = φ 21 x t 1 + φ 22 x t 2 + w 2t. x t = φ m1 x t 1 + φ m2 x t φ mm x t m + w mt, (56). by the least square method. The sequence { ˆφ mm, m = 1, 2,...} is the Sample Partial Autocorrelation Function (SPACF). JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

62 Auto-regressive models Model identication The basic idea of an ARMA model selection criterium (or information criterion) is to choose the orders k and l that minimize the quantity P(k, l) = ln ˆσ 2 k,l + (k + l)c(n) N, (57) in which ˆσ 2 k,l is a residual variance estimate obtained by adjusting an ARMA(k, l) model to the N series observations. C(N) is a functions of the series size. The quantity (k + l) C(N) is called penalty term and it increases when N the number of parameters increases, while ˆσ 2 decreases. k,l JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

63 Auto-regressive models Akaike's criterium Akaike proposed the information criterium AIC(k, l) = ln ˆσ 2 k,l 2(k + l) +, (58) N known as AIC, in which ˆσ 2 k,l is the maximum likelihood estimator of σ 2 w for an ARMA(k, l) model. Upper bounds K and L for k and l must be specied. (58) has to evaluated for all possible (k, l) combinations with 0 k K 0 l L. In general, K and L are functions of N, for example, K = L = ln N. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

64 Auto-regressive models AIC and BIC For the case of AR(p) models, (58) reduces to AIC(k) = ln ˆσ 2 + 2k k, k K. (59) N Another criterium that is much used is the (Schwarz) Bayesian Information Criteria (BIC) BIC(k, l) = ln ˆσ 2 + ln N k,l (k + l). (60) N For the case of AR(p) models, (60) reduces to BIC(k) = ln ˆσ 2 k + k ln N N. (61) JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

65 Auto-regressive models AR models estimation Having identied the AR model's order p, we can go to the parameters estimation phase. The methods of moments, Least Squares and Maximum Likelihood may be used. As, in general, the moments estimators are not good, statistical packages as S-PLUS, E-VIEWS, etc., use some Least Squares or Maximum Likelihood estimator. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

66 Non-stationary series modeling Outline 1 Introduction Motivation Objectives and contributions 2 Time series basic concepts Time series examples Operators notation Stochastic processes Time Series Modeling Auto-regressive models Non-stationary series modeling JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

67 Non-stationary series modeling Introduction A non-stationary process has time dependent moments. Common types of non-stationarity: time dependent mean time dependent variance JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

68 Non-stationary series modeling Stationary series with respect to trends A process y t is called stationary with respect to trends if it is of the type y t = TD t + x t, (62) in which TD t denotes the term of deterministic trends (constant, trend, season) that depends on t and x t is a stationary process. For example, the process y t given by y t = µ + δt + x t, x t = φx t 1 + w t y t µ δt = φ(y t 1 µ δ(t 1)) + w t y t = c + βt + φy t 1 + w t (63) in which φ < 1, c = µ(1 φ) + δ, β = δ(1 φ)t and w t is a WGN with null mean and power σ 2, is a stationary AR(1) process with respect to trends. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

69 Non-stationary series modeling ARIMA model If a process corresponds to the dierence of order d = 1, 2,... of x t y t = (1 B) d x t = d x t (64) is stationary, then y t can be represented by an ARMA(p, q) model φ(b)y t = θ(b)w t. (65) In this case, φ(b) d x t = θ(b)w t (66) is an ARIMA(p, d, q) model and we say that x t is an integral of y t because x t = S d y t. (67) JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

70 Non-stationary series modeling ARIMA model 2 As the ARIMA(p, d, q) model H(z) = θ(z) φ(z)(1 z 1 ) d (68) is marginally stable, as it has d roots on the unit circle, x t of (66) is a homogeneous non-stationary process (meaning non-explosive) or having unit roots JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

71 Non-stationary series modeling ARIMA model 3 Observe that: (a) d = 1 corresponds to homogeneous non-stationary series with respect to the level (they oscillate around a mean level during a certain time and then jump to another temporary level); (b) d = 2 corresponds to homogeneous non-stationary series with respect to the trend (they oscillate along a direction for a certain time and then change to another temporary direction). JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

72 Non-stationary series modeling ARIMA model 3 The ARIMA model (66) may be represented in three ways: (a) ARMA(p + d, q) (similar to Eq. (31)) p+d q x(t) = ϕ k x(t k) + w(t) θ k w(t k); (69) k=1 k=1 (b) AR( ) (inverted format), given by (45) or (c) MA( ), according to (30). JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

73 Non-stationary series modeling Random walk Consider the model y t I (1) y t = y t 1 + x t, (70) in which x t is a stationary process. If we assume the initial condition y 0, (70) can be rewritten as an integrated sum y t = y 0 + t x j. (71) The integrated sum t j=1 x j is called stochastic trend and it is denoted by TS t. Observe that TS t = TS t 1 + x t, (72) in which TS 0 = 0. j=1 JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

74 Non-stationary series modeling Random walk 2 If x t N (0, σ 2 x) in (70), then y t is known as random walk. Including a constant in the right side of (70), we have a random walk with drift, y t = θ 0 + y t 1 + x t. (73) Given the initial condition y 0, we can write y t = y 0 + θ 0 t + t x j (74) j=1 = TD t + TS t JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

75 Non-stationary series modeling Random walk 3 The mean, variance, autocovariance and ACF of y t are given by µ t = y 0 + tθ 0 (75) σ 2 (t) = tσ 2 x (76) C k (t) = (t k)σ 2 x (77) ρ k (t) = t k. (78) t Observe that ρ k (t) 1 when t >> k and the literature states that the random walk has strong memory. The random walk's SACF decays linearly for large lags. JRAA-ABL (Escola Politécnica da USP) Teletrac modeling and estimation São Paulo / 74

Teletrac modeling and estimation

Teletrac modeling and estimation Teletrac modeling and estimation File 2 José Roberto Amazonas jra@lcs.poli.usp.br Telecommunications and Control Engineering Dept. - PTC Escola Politécnica University of São Paulo - USP São Paulo 11/2008

More information

Teletrac modeling and estimation

Teletrac modeling and estimation Teletrac modeling and estimation File 4 José Roberto Amazonas jra@lcs.poli.usp.br Telecommunications and Control Engineering Dept. - PTC Escola Politécnica University of São Paulo - USP São Paulo 11/2008

More information

Applied time-series analysis

Applied time-series analysis Robert M. Kunst robert.kunst@univie.ac.at University of Vienna and Institute for Advanced Studies Vienna October 18, 2011 Outline Introduction and overview Econometric Time-Series Analysis In principle,

More information

FE570 Financial Markets and Trading. Stevens Institute of Technology

FE570 Financial Markets and Trading. Stevens Institute of Technology FE570 Financial Markets and Trading Lecture 5. Linear Time Series Analysis and Its Applications (Ref. Joel Hasbrouck - Empirical Market Microstructure ) Steve Yang Stevens Institute of Technology 9/25/2012

More information

{ } Stochastic processes. Models for time series. Specification of a process. Specification of a process. , X t3. ,...X tn }

{ } Stochastic processes. Models for time series. Specification of a process. Specification of a process. , X t3. ,...X tn } Stochastic processes Time series are an example of a stochastic or random process Models for time series A stochastic process is 'a statistical phenomenon that evolves in time according to probabilistic

More information

3. ARMA Modeling. Now: Important class of stationary processes

3. ARMA Modeling. Now: Important class of stationary processes 3. ARMA Modeling Now: Important class of stationary processes Definition 3.1: (ARMA(p, q) process) Let {ɛ t } t Z WN(0, σ 2 ) be a white noise process. The process {X t } t Z is called AutoRegressive-Moving-Average

More information

Time Series I Time Domain Methods

Time Series I Time Domain Methods Astrostatistics Summer School Penn State University University Park, PA 16802 May 21, 2007 Overview Filtering and the Likelihood Function Time series is the study of data consisting of a sequence of DEPENDENT

More information

STAT 443 Final Exam Review. 1 Basic Definitions. 2 Statistical Tests. L A TEXer: W. Kong

STAT 443 Final Exam Review. 1 Basic Definitions. 2 Statistical Tests. L A TEXer: W. Kong STAT 443 Final Exam Review L A TEXer: W Kong 1 Basic Definitions Definition 11 The time series {X t } with E[X 2 t ] < is said to be weakly stationary if: 1 µ X (t) = E[X t ] is independent of t 2 γ X

More information

Introduction to ARMA and GARCH processes

Introduction to ARMA and GARCH processes Introduction to ARMA and GARCH processes Fulvio Corsi SNS Pisa 3 March 2010 Fulvio Corsi Introduction to ARMA () and GARCH processes SNS Pisa 3 March 2010 1 / 24 Stationarity Strict stationarity: (X 1,

More information

Advanced Econometrics

Advanced Econometrics Advanced Econometrics Marco Sunder Nov 04 2010 Marco Sunder Advanced Econometrics 1/ 25 Contents 1 2 3 Marco Sunder Advanced Econometrics 2/ 25 Music Marco Sunder Advanced Econometrics 3/ 25 Music Marco

More information

Time Series 2. Robert Almgren. Sept. 21, 2009

Time Series 2. Robert Almgren. Sept. 21, 2009 Time Series 2 Robert Almgren Sept. 21, 2009 This week we will talk about linear time series models: AR, MA, ARMA, ARIMA, etc. First we will talk about theory and after we will talk about fitting the models

More information

Review Session: Econometrics - CLEFIN (20192)

Review Session: Econometrics - CLEFIN (20192) Review Session: Econometrics - CLEFIN (20192) Part II: Univariate time series analysis Daniele Bianchi March 20, 2013 Fundamentals Stationarity A time series is a sequence of random variables x t, t =

More information

Time Series Outlier Detection

Time Series Outlier Detection Time Series Outlier Detection Tingyi Zhu July 28, 2016 Tingyi Zhu Time Series Outlier Detection July 28, 2016 1 / 42 Outline Time Series Basics Outliers Detection in Single Time Series Outlier Series Detection

More information

ECON 616: Lecture 1: Time Series Basics

ECON 616: Lecture 1: Time Series Basics ECON 616: Lecture 1: Time Series Basics ED HERBST August 30, 2017 References Overview: Chapters 1-3 from Hamilton (1994). Technical Details: Chapters 2-3 from Brockwell and Davis (1987). Intuition: Chapters

More information

University of Oxford. Statistical Methods Autocorrelation. Identification and Estimation

University of Oxford. Statistical Methods Autocorrelation. Identification and Estimation University of Oxford Statistical Methods Autocorrelation Identification and Estimation Dr. Órlaith Burke Michaelmas Term, 2011 Department of Statistics, 1 South Parks Road, Oxford OX1 3TG Contents 1 Model

More information

STAT Financial Time Series

STAT Financial Time Series STAT 6104 - Financial Time Series Chapter 4 - Estimation in the time Domain Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 1 / 46 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR

More information

Discrete time processes

Discrete time processes Discrete time processes Predictions are difficult. Especially about the future Mark Twain. Florian Herzog 2013 Modeling observed data When we model observed (realized) data, we encounter usually the following

More information

Time Series Analysis. Solutions to problems in Chapter 5 IMM

Time Series Analysis. Solutions to problems in Chapter 5 IMM Time Series Analysis Solutions to problems in Chapter 5 IMM Solution 5.1 Question 1. [ ] V [X t ] = V [ǫ t + c(ǫ t 1 + ǫ t + )] = 1 + c 1 σǫ = The variance of {X t } is not limited and therefore {X t }

More information

Chapter 12: An introduction to Time Series Analysis. Chapter 12: An introduction to Time Series Analysis

Chapter 12: An introduction to Time Series Analysis. Chapter 12: An introduction to Time Series Analysis Chapter 12: An introduction to Time Series Analysis Introduction In this chapter, we will discuss forecasting with single-series (univariate) Box-Jenkins models. The common name of the models is Auto-Regressive

More information

Marcel Dettling. Applied Time Series Analysis SS 2013 Week 05. ETH Zürich, March 18, Institute for Data Analysis and Process Design

Marcel Dettling. Applied Time Series Analysis SS 2013 Week 05. ETH Zürich, March 18, Institute for Data Analysis and Process Design Marcel Dettling Institute for Data Analysis and Process Design Zurich University of Applied Sciences marcel.dettling@zhaw.ch http://stat.ethz.ch/~dettling ETH Zürich, March 18, 2013 1 Basics of Modeling

More information

Some Time-Series Models

Some Time-Series Models Some Time-Series Models Outline 1. Stochastic processes and their properties 2. Stationary processes 3. Some properties of the autocorrelation function 4. Some useful models Purely random processes, random

More information

Autoregressive Moving Average (ARMA) Models and their Practical Applications

Autoregressive Moving Average (ARMA) Models and their Practical Applications Autoregressive Moving Average (ARMA) Models and their Practical Applications Massimo Guidolin February 2018 1 Essential Concepts in Time Series Analysis 1.1 Time Series and Their Properties Time series:

More information

Ch 6. Model Specification. Time Series Analysis

Ch 6. Model Specification. Time Series Analysis We start to build ARIMA(p,d,q) models. The subjects include: 1 how to determine p, d, q for a given series (Chapter 6); 2 how to estimate the parameters (φ s and θ s) of a specific ARIMA(p,d,q) model (Chapter

More information

Exercises - Time series analysis

Exercises - Time series analysis Descriptive analysis of a time series (1) Estimate the trend of the series of gasoline consumption in Spain using a straight line in the period from 1945 to 1995 and generate forecasts for 24 months. Compare

More information

Multiplicative Multifractal Modeling of. Long-Range-Dependent (LRD) Trac in. Computer Communications Networks. Jianbo Gao and Izhak Rubin

Multiplicative Multifractal Modeling of. Long-Range-Dependent (LRD) Trac in. Computer Communications Networks. Jianbo Gao and Izhak Rubin Multiplicative Multifractal Modeling of Long-Range-Dependent (LRD) Trac in Computer Communications Networks Jianbo Gao and Izhak Rubin Electrical Engineering Department, University of California, Los Angeles

More information

Lecture 1: Fundamental concepts in Time Series Analysis (part 2)

Lecture 1: Fundamental concepts in Time Series Analysis (part 2) Lecture 1: Fundamental concepts in Time Series Analysis (part 2) Florian Pelgrin University of Lausanne, École des HEC Department of mathematics (IMEA-Nice) Sept. 2011 - Jan. 2012 Florian Pelgrin (HEC)

More information

Lecture 3: Autoregressive Moving Average (ARMA) Models and their Practical Applications

Lecture 3: Autoregressive Moving Average (ARMA) Models and their Practical Applications Lecture 3: Autoregressive Moving Average (ARMA) Models and their Practical Applications Prof. Massimo Guidolin 20192 Financial Econometrics Winter/Spring 2018 Overview Moving average processes Autoregressive

More information

Lesson 13: Box-Jenkins Modeling Strategy for building ARMA models

Lesson 13: Box-Jenkins Modeling Strategy for building ARMA models Lesson 13: Box-Jenkins Modeling Strategy for building ARMA models Facoltà di Economia Università dell Aquila umberto.triacca@gmail.com Introduction In this lesson we present a method to construct an ARMA(p,

More information

TIME SERIES ANALYSIS AND FORECASTING USING THE STATISTICAL MODEL ARIMA

TIME SERIES ANALYSIS AND FORECASTING USING THE STATISTICAL MODEL ARIMA CHAPTER 6 TIME SERIES ANALYSIS AND FORECASTING USING THE STATISTICAL MODEL ARIMA 6.1. Introduction A time series is a sequence of observations ordered in time. A basic assumption in the time series analysis

More information

Covariances of ARMA Processes

Covariances of ARMA Processes Statistics 910, #10 1 Overview Covariances of ARMA Processes 1. Review ARMA models: causality and invertibility 2. AR covariance functions 3. MA and ARMA covariance functions 4. Partial autocorrelation

More information

Part III Example Sheet 1 - Solutions YC/Lent 2015 Comments and corrections should be ed to

Part III Example Sheet 1 - Solutions YC/Lent 2015 Comments and corrections should be  ed to TIME SERIES Part III Example Sheet 1 - Solutions YC/Lent 2015 Comments and corrections should be emailed to Y.Chen@statslab.cam.ac.uk. 1. Let {X t } be a weakly stationary process with mean zero and let

More information

1 Linear Difference Equations

1 Linear Difference Equations ARMA Handout Jialin Yu 1 Linear Difference Equations First order systems Let {ε t } t=1 denote an input sequence and {y t} t=1 sequence generated by denote an output y t = φy t 1 + ε t t = 1, 2,... with

More information

ESSE Mid-Term Test 2017 Tuesday 17 October :30-09:45

ESSE Mid-Term Test 2017 Tuesday 17 October :30-09:45 ESSE 4020 3.0 - Mid-Term Test 207 Tuesday 7 October 207. 08:30-09:45 Symbols have their usual meanings. All questions are worth 0 marks, although some are more difficult than others. Answer as many questions

More information

Time Series Analysis

Time Series Analysis Time Series Analysis hm@imm.dtu.dk Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby 1 Outline of the lecture Chapter 9 Multivariate time series 2 Transfer function

More information

A SARIMAX coupled modelling applied to individual load curves intraday forecasting

A SARIMAX coupled modelling applied to individual load curves intraday forecasting A SARIMAX coupled modelling applied to individual load curves intraday forecasting Frédéric Proïa Workshop EDF Institut Henri Poincaré - Paris 05 avril 2012 INRIA Bordeaux Sud-Ouest Institut de Mathématiques

More information

Covariance Stationary Time Series. Example: Independent White Noise (IWN(0,σ 2 )) Y t = ε t, ε t iid N(0,σ 2 )

Covariance Stationary Time Series. Example: Independent White Noise (IWN(0,σ 2 )) Y t = ε t, ε t iid N(0,σ 2 ) Covariance Stationary Time Series Stochastic Process: sequence of rv s ordered by time {Y t } {...,Y 1,Y 0,Y 1,...} Defn: {Y t } is covariance stationary if E[Y t ]μ for all t cov(y t,y t j )E[(Y t μ)(y

More information

γ 0 = Var(X i ) = Var(φ 1 X i 1 +W i ) = φ 2 1γ 0 +σ 2, which implies that we must have φ 1 < 1, and γ 0 = σ2 . 1 φ 2 1 We may also calculate for j 1

γ 0 = Var(X i ) = Var(φ 1 X i 1 +W i ) = φ 2 1γ 0 +σ 2, which implies that we must have φ 1 < 1, and γ 0 = σ2 . 1 φ 2 1 We may also calculate for j 1 4.2 Autoregressive (AR) Moving average models are causal linear processes by definition. There is another class of models, based on a recursive formulation similar to the exponentially weighted moving

More information

Econometrics II Heij et al. Chapter 7.1

Econometrics II Heij et al. Chapter 7.1 Chapter 7.1 p. 1/2 Econometrics II Heij et al. Chapter 7.1 Linear Time Series Models for Stationary data Marius Ooms Tinbergen Institute Amsterdam Chapter 7.1 p. 2/2 Program Introduction Modelling philosophy

More information

AR, MA and ARMA models

AR, MA and ARMA models AR, MA and AR by Hedibert Lopes P Based on Tsay s Analysis of Financial Time Series (3rd edition) P 1 Stationarity 2 3 4 5 6 7 P 8 9 10 11 Outline P Linear Time Series Analysis and Its Applications For

More information

Lecture 7: Model Building Bus 41910, Time Series Analysis, Mr. R. Tsay

Lecture 7: Model Building Bus 41910, Time Series Analysis, Mr. R. Tsay Lecture 7: Model Building Bus 41910, Time Series Analysis, Mr R Tsay An effective procedure for building empirical time series models is the Box-Jenkins approach, which consists of three stages: model

More information

Univariate ARIMA Models

Univariate ARIMA Models Univariate ARIMA Models ARIMA Model Building Steps: Identification: Using graphs, statistics, ACFs and PACFs, transformations, etc. to achieve stationary and tentatively identify patterns and model components.

More information

Forecasting using R. Rob J Hyndman. 2.4 Non-seasonal ARIMA models. Forecasting using R 1

Forecasting using R. Rob J Hyndman. 2.4 Non-seasonal ARIMA models. Forecasting using R 1 Forecasting using R Rob J Hyndman 2.4 Non-seasonal ARIMA models Forecasting using R 1 Outline 1 Autoregressive models 2 Moving average models 3 Non-seasonal ARIMA models 4 Partial autocorrelations 5 Estimation

More information

Stochastic Processes

Stochastic Processes Stochastic Processes Stochastic Process Non Formal Definition: Non formal: A stochastic process (random process) is the opposite of a deterministic process such as one defined by a differential equation.

More information

Econometrics I: Univariate Time Series Econometrics (1)

Econometrics I: Univariate Time Series Econometrics (1) Econometrics I: Dipartimento di Economia Politica e Metodi Quantitativi University of Pavia Overview of the Lecture 1 st EViews Session VI: Some Theoretical Premises 2 Overview of the Lecture 1 st EViews

More information

Stat 248 Lab 2: Stationarity, More EDA, Basic TS Models

Stat 248 Lab 2: Stationarity, More EDA, Basic TS Models Stat 248 Lab 2: Stationarity, More EDA, Basic TS Models Tessa L. Childers-Day February 8, 2013 1 Introduction Today s section will deal with topics such as: the mean function, the auto- and cross-covariance

More information

Class 1: Stationary Time Series Analysis

Class 1: Stationary Time Series Analysis Class 1: Stationary Time Series Analysis Macroeconometrics - Fall 2009 Jacek Suda, BdF and PSE February 28, 2011 Outline Outline: 1 Covariance-Stationary Processes 2 Wold Decomposition Theorem 3 ARMA Models

More information

Circle a single answer for each multiple choice question. Your choice should be made clearly.

Circle a single answer for each multiple choice question. Your choice should be made clearly. TEST #1 STA 4853 March 4, 215 Name: Please read the following directions. DO NOT TURN THE PAGE UNTIL INSTRUCTED TO DO SO Directions This exam is closed book and closed notes. There are 31 questions. Circle

More information

Moving Average (MA) representations

Moving Average (MA) representations Moving Average (MA) representations The moving average representation of order M has the following form v[k] = MX c n e[k n]+e[k] (16) n=1 whose transfer function operator form is MX v[k] =H(q 1 )e[k],

More information

Lecture 3: Autoregressive Moving Average (ARMA) Models and their Practical Applications

Lecture 3: Autoregressive Moving Average (ARMA) Models and their Practical Applications Lecture 3: Autoregressive Moving Average (ARMA) Models and their Practical Applications Prof. Massimo Guidolin 20192 Financial Econometrics Winter/Spring 2018 Overview Moving average processes Autoregressive

More information

The Identification of ARIMA Models

The Identification of ARIMA Models APPENDIX 4 The Identification of ARIMA Models As we have established in a previous lecture, there is a one-to-one correspondence between the parameters of an ARMA(p, q) model, including the variance of

More information

Empirical Market Microstructure Analysis (EMMA)

Empirical Market Microstructure Analysis (EMMA) Empirical Market Microstructure Analysis (EMMA) Lecture 3: Statistical Building Blocks and Econometric Basics Prof. Dr. Michael Stein michael.stein@vwl.uni-freiburg.de Albert-Ludwigs-University of Freiburg

More information

Ross Bettinger, Analytical Consultant, Seattle, WA

Ross Bettinger, Analytical Consultant, Seattle, WA ABSTRACT DYNAMIC REGRESSION IN ARIMA MODELING Ross Bettinger, Analytical Consultant, Seattle, WA Box-Jenkins time series models that contain exogenous predictor variables are called dynamic regression

More information

16.584: Random (Stochastic) Processes

16.584: Random (Stochastic) Processes 1 16.584: Random (Stochastic) Processes X(t): X : RV : Continuous function of the independent variable t (time, space etc.) Random process : Collection of X(t, ζ) : Indexed on another independent variable

More information

Prof. Dr. Roland Füss Lecture Series in Applied Econometrics Summer Term Introduction to Time Series Analysis

Prof. Dr. Roland Füss Lecture Series in Applied Econometrics Summer Term Introduction to Time Series Analysis Introduction to Time Series Analysis 1 Contents: I. Basics of Time Series Analysis... 4 I.1 Stationarity... 5 I.2 Autocorrelation Function... 9 I.3 Partial Autocorrelation Function (PACF)... 14 I.4 Transformation

More information

MODELING INFLATION RATES IN NIGERIA: BOX-JENKINS APPROACH. I. U. Moffat and A. E. David Department of Mathematics & Statistics, University of Uyo, Uyo

MODELING INFLATION RATES IN NIGERIA: BOX-JENKINS APPROACH. I. U. Moffat and A. E. David Department of Mathematics & Statistics, University of Uyo, Uyo Vol.4, No.2, pp.2-27, April 216 MODELING INFLATION RATES IN NIGERIA: BOX-JENKINS APPROACH I. U. Moffat and A. E. David Department of Mathematics & Statistics, University of Uyo, Uyo ABSTRACT: This study

More information

Time Series Models and Inference. James L. Powell Department of Economics University of California, Berkeley

Time Series Models and Inference. James L. Powell Department of Economics University of California, Berkeley Time Series Models and Inference James L. Powell Department of Economics University of California, Berkeley Overview In contrast to the classical linear regression model, in which the components of the

More information

Chapter 4: Models for Stationary Time Series

Chapter 4: Models for Stationary Time Series Chapter 4: Models for Stationary Time Series Now we will introduce some useful parametric models for time series that are stationary processes. We begin by defining the General Linear Process. Let {Y t

More information

Stat 5100 Handout #12.e Notes: ARIMA Models (Unit 7) Key here: after stationary, identify dependence structure (and use for forecasting)

Stat 5100 Handout #12.e Notes: ARIMA Models (Unit 7) Key here: after stationary, identify dependence structure (and use for forecasting) Stat 5100 Handout #12.e Notes: ARIMA Models (Unit 7) Key here: after stationary, identify dependence structure (and use for forecasting) (overshort example) White noise H 0 : Let Z t be the stationary

More information

Time Series Examples Sheet

Time Series Examples Sheet Lent Term 2001 Richard Weber Time Series Examples Sheet This is the examples sheet for the M. Phil. course in Time Series. A copy can be found at: http://www.statslab.cam.ac.uk/~rrw1/timeseries/ Throughout,

More information

ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process

ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Definition of stochastic process (random

More information

Design of IP networks with Quality of Service

Design of IP networks with Quality of Service Course of Multimedia Internet (Sub-course Reti Internet Multimediali ), AA 2010-2011 Prof. Pag. 1 Design of IP networks with Quality of Service 1 Course of Multimedia Internet (Sub-course Reti Internet

More information

Lecture 2: Univariate Time Series

Lecture 2: Univariate Time Series Lecture 2: Univariate Time Series Analysis: Conditional and Unconditional Densities, Stationarity, ARMA Processes Prof. Massimo Guidolin 20192 Financial Econometrics Spring/Winter 2017 Overview Motivation:

More information

Econ 623 Econometrics II Topic 2: Stationary Time Series

Econ 623 Econometrics II Topic 2: Stationary Time Series 1 Introduction Econ 623 Econometrics II Topic 2: Stationary Time Series In the regression model we can model the error term as an autoregression AR(1) process. That is, we can use the past value of the

More information

Lesson 2: Analysis of time series

Lesson 2: Analysis of time series Lesson 2: Analysis of time series Time series Main aims of time series analysis choosing right model statistical testing forecast driving and optimalisation Problems in analysis of time series time problems

More information

Midterm Suggested Solutions

Midterm Suggested Solutions CUHK Dept. of Economics Spring 2011 ECON 4120 Sung Y. Park Midterm Suggested Solutions Q1 (a) In time series, autocorrelation measures the correlation between y t and its lag y t τ. It is defined as. ρ(τ)

More information

Nonlinear time series

Nonlinear time series Based on the book by Fan/Yao: Nonlinear Time Series Robert M. Kunst robert.kunst@univie.ac.at University of Vienna and Institute for Advanced Studies Vienna October 27, 2009 Outline Characteristics of

More information

7. Forecasting with ARIMA models

7. Forecasting with ARIMA models 7. Forecasting with ARIMA models 309 Outline: Introduction The prediction equation of an ARIMA model Interpreting the predictions Variance of the predictions Forecast updating Measuring predictability

More information

Lecture 4a: ARMA Model

Lecture 4a: ARMA Model Lecture 4a: ARMA Model 1 2 Big Picture Most often our goal is to find a statistical model to describe real time series (estimation), and then predict the future (forecasting) One particularly popular model

More information

Chapter 6: Model Specification for Time Series

Chapter 6: Model Specification for Time Series Chapter 6: Model Specification for Time Series The ARIMA(p, d, q) class of models as a broad class can describe many real time series. Model specification for ARIMA(p, d, q) models involves 1. Choosing

More information

3 Theory of stationary random processes

3 Theory of stationary random processes 3 Theory of stationary random processes 3.1 Linear filters and the General linear process A filter is a transformation of one random sequence {U t } into another, {Y t }. A linear filter is a transformation

More information

Financial Time Series Analysis: Part II

Financial Time Series Analysis: Part II Department of Mathematics and Statistics, University of Vaasa, Finland Spring 2017 1 Unit root Deterministic trend Stochastic trend Testing for unit root ADF-test (Augmented Dickey-Fuller test) Testing

More information

Time Series: Theory and Methods

Time Series: Theory and Methods Peter J. Brockwell Richard A. Davis Time Series: Theory and Methods Second Edition With 124 Illustrations Springer Contents Preface to the Second Edition Preface to the First Edition vn ix CHAPTER 1 Stationary

More information

Reliability and Risk Analysis. Time Series, Types of Trend Functions and Estimates of Trends

Reliability and Risk Analysis. Time Series, Types of Trend Functions and Estimates of Trends Reliability and Risk Analysis Stochastic process The sequence of random variables {Y t, t = 0, ±1, ±2 } is called the stochastic process The mean function of a stochastic process {Y t} is the function

More information

Lecture 1: Stationary Time Series Analysis

Lecture 1: Stationary Time Series Analysis Syllabus Stationarity ARMA AR MA Model Selection Estimation Lecture 1: Stationary Time Series Analysis 222061-1617: Time Series Econometrics Spring 2018 Jacek Suda Syllabus Stationarity ARMA AR MA Model

More information

Lecture 2: ARMA(p,q) models (part 2)

Lecture 2: ARMA(p,q) models (part 2) Lecture 2: ARMA(p,q) models (part 2) Florian Pelgrin University of Lausanne, École des HEC Department of mathematics (IMEA-Nice) Sept. 2011 - Jan. 2012 Florian Pelgrin (HEC) Univariate time series Sept.

More information

MGR-815. Notes for the MGR-815 course. 12 June School of Superior Technology. Professor Zbigniew Dziong

MGR-815. Notes for the MGR-815 course. 12 June School of Superior Technology. Professor Zbigniew Dziong Modeling, Estimation and Control, for Telecommunication Networks Notes for the MGR-815 course 12 June 2010 School of Superior Technology Professor Zbigniew Dziong 1 Table of Contents Preface 5 1. Example

More information

A Data-Driven Model for Software Reliability Prediction

A Data-Driven Model for Software Reliability Prediction A Data-Driven Model for Software Reliability Prediction Author: Jung-Hua Lo IEEE International Conference on Granular Computing (2012) Young Taek Kim KAIST SE Lab. 9/4/2013 Contents Introduction Background

More information

STAT STOCHASTIC PROCESSES. Contents

STAT STOCHASTIC PROCESSES. Contents STAT 3911 - STOCHASTIC PROCESSES ANDREW TULLOCH Contents 1. Stochastic Processes 2 2. Classification of states 2 3. Limit theorems for Markov chains 4 4. First step analysis 5 5. Branching processes 5

More information

Ch 4. Models For Stationary Time Series. Time Series Analysis

Ch 4. Models For Stationary Time Series. Time Series Analysis This chapter discusses the basic concept of a broad class of stationary parametric time series models the autoregressive moving average (ARMA) models. Let {Y t } denote the observed time series, and {e

More information

Stochastic Processes. M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno

Stochastic Processes. M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno Stochastic Processes M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno 1 Outline Stochastic (random) processes. Autocorrelation. Crosscorrelation. Spectral density function.

More information

Time Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY

Time Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY Time Series Analysis James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY PREFACE xiii 1 Difference Equations 1.1. First-Order Difference Equations 1 1.2. pth-order Difference Equations 7

More information

Linear models. Chapter Overview. Linear process: A process {X n } is a linear process if it has the representation.

Linear models. Chapter Overview. Linear process: A process {X n } is a linear process if it has the representation. Chapter 2 Linear models 2.1 Overview Linear process: A process {X n } is a linear process if it has the representation X n = b j ɛ n j j=0 for all n, where ɛ n N(0, σ 2 ) (Gaussian distributed with zero

More information

IDENTIFICATION OF ARMA MODELS

IDENTIFICATION OF ARMA MODELS IDENTIFICATION OF ARMA MODELS A stationary stochastic process can be characterised, equivalently, by its autocovariance function or its partial autocovariance function. It can also be characterised by

More information

Quantitative Finance I

Quantitative Finance I Quantitative Finance I Linear AR and MA Models (Lecture 4) Winter Semester 01/013 by Lukas Vacha * If viewed in.pdf format - for full functionality use Mathematica 7 (or higher) notebook (.nb) version

More information

Econometría 2: Análisis de series de Tiempo

Econometría 2: Análisis de series de Tiempo Econometría 2: Análisis de series de Tiempo Karoll GOMEZ kgomezp@unal.edu.co http://karollgomez.wordpress.com Segundo semestre 2016 II. Basic definitions A time series is a set of observations X t, each

More information

Time Series Solutions HT 2009

Time Series Solutions HT 2009 Time Series Solutions HT 2009 1. Let {X t } be the ARMA(1, 1) process, X t φx t 1 = ɛ t + θɛ t 1, {ɛ t } WN(0, σ 2 ), where φ < 1 and θ < 1. Show that the autocorrelation function of {X t } is given by

More information

Introduction to Signal Processing

Introduction to Signal Processing to Signal Processing Davide Bacciu Dipartimento di Informatica Università di Pisa bacciu@di.unipi.it Intelligent Systems for Pattern Recognition Signals = Time series Definitions Motivations A sequence

More information

Note: The primary reference for these notes is Enders (2004). An alternative and more technical treatment can be found in Hamilton (1994).

Note: The primary reference for these notes is Enders (2004). An alternative and more technical treatment can be found in Hamilton (1994). Chapter 4 Analysis of a Single Time Series Note: The primary reference for these notes is Enders (4). An alternative and more technical treatment can be found in Hamilton (994). Most data used in financial

More information

Fourier Analysis Linear transformations and lters. 3. Fourier Analysis. Alex Sheremet. April 11, 2007

Fourier Analysis Linear transformations and lters. 3. Fourier Analysis. Alex Sheremet. April 11, 2007 Stochastic processes review 3. Data Analysis Techniques in Oceanography OCP668 April, 27 Stochastic processes review Denition Fixed ζ = ζ : Function X (t) = X (t, ζ). Fixed t = t: Random Variable X (ζ)

More information

TIME SERIES ANALYSIS. Forecasting and Control. Wiley. Fifth Edition GWILYM M. JENKINS GEORGE E. P. BOX GREGORY C. REINSEL GRETA M.

TIME SERIES ANALYSIS. Forecasting and Control. Wiley. Fifth Edition GWILYM M. JENKINS GEORGE E. P. BOX GREGORY C. REINSEL GRETA M. TIME SERIES ANALYSIS Forecasting and Control Fifth Edition GEORGE E. P. BOX GWILYM M. JENKINS GREGORY C. REINSEL GRETA M. LJUNG Wiley CONTENTS PREFACE TO THE FIFTH EDITION PREFACE TO THE FOURTH EDITION

More information

ECONOMICS 7200 MODERN TIME SERIES ANALYSIS Econometric Theory and Applications

ECONOMICS 7200 MODERN TIME SERIES ANALYSIS Econometric Theory and Applications ECONOMICS 7200 MODERN TIME SERIES ANALYSIS Econometric Theory and Applications Yongmiao Hong Department of Economics & Department of Statistical Sciences Cornell University Spring 2019 Time and uncertainty

More information

Circle the single best answer for each multiple choice question. Your choice should be made clearly.

Circle the single best answer for each multiple choice question. Your choice should be made clearly. TEST #1 STA 4853 March 6, 2017 Name: Please read the following directions. DO NOT TURN THE PAGE UNTIL INSTRUCTED TO DO SO Directions This exam is closed book and closed notes. There are 32 multiple choice

More information

at least 50 and preferably 100 observations should be available to build a proper model

at least 50 and preferably 100 observations should be available to build a proper model III Box-Jenkins Methods 1. Pros and Cons of ARIMA Forecasting a) need for data at least 50 and preferably 100 observations should be available to build a proper model used most frequently for hourly or

More information

Non-Stationary Time Series and Unit Root Testing

Non-Stationary Time Series and Unit Root Testing Econometrics II Non-Stationary Time Series and Unit Root Testing Morten Nyboe Tabor Course Outline: Non-Stationary Time Series and Unit Root Testing 1 Stationarity and Deviation from Stationarity Trend-Stationarity

More information

Part 1. Multiple Choice (50 questions, 1 point each) Part 2. Problems/Short Answer (10 questions, 5 points each)

Part 1. Multiple Choice (50 questions, 1 point each) Part 2. Problems/Short Answer (10 questions, 5 points each) GROUND RULES: This exam contains two parts: Part 1. Multiple Choice (50 questions, 1 point each) Part 2. Problems/Short Answer (10 questions, 5 points each) The maximum number of points on this exam is

More information

Statistical signal processing

Statistical signal processing Statistical signal processing Short overview of the fundamentals Outline Random variables Random processes Stationarity Ergodicity Spectral analysis Random variable and processes Intuition: A random variable

More information

We will only present the general ideas on how to obtain. follow closely the AR(1) and AR(2) cases presented before.

We will only present the general ideas on how to obtain. follow closely the AR(1) and AR(2) cases presented before. ACF and PACF of an AR(p) We will only present the general ideas on how to obtain the ACF and PACF of an AR(p) model since the details follow closely the AR(1) and AR(2) cases presented before. Recall that

More information

Time Series Analysis

Time Series Analysis Time Series Analysis hm@imm.dtu.dk Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby 1 Outline of the lecture Input-Output systems The z-transform important issues

More information

Time Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY

Time Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY Time Series Analysis James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY & Contents PREFACE xiii 1 1.1. 1.2. Difference Equations First-Order Difference Equations 1 /?th-order Difference

More information

4. MA(2) +drift: y t = µ + ɛ t + θ 1 ɛ t 1 + θ 2 ɛ t 2. Mean: where θ(l) = 1 + θ 1 L + θ 2 L 2. Therefore,

4. MA(2) +drift: y t = µ + ɛ t + θ 1 ɛ t 1 + θ 2 ɛ t 2. Mean: where θ(l) = 1 + θ 1 L + θ 2 L 2. Therefore, 61 4. MA(2) +drift: y t = µ + ɛ t + θ 1 ɛ t 1 + θ 2 ɛ t 2 Mean: y t = µ + θ(l)ɛ t, where θ(l) = 1 + θ 1 L + θ 2 L 2. Therefore, E(y t ) = µ + θ(l)e(ɛ t ) = µ 62 Example: MA(q) Model: y t = ɛ t + θ 1 ɛ

More information