Lesson 2: Analysis of time series

Lesson 2: Analysis of time series

Time series Main aims of time series analysis choosing right model statistical testing forecast driving and optimalisation

Problems in analysis of time series time problems problems in methodology forecast problems

Problems with time choosing time points of observation - not too short intervals, but not too little points calendar - convention: 30 days (21 of them working) per month 360 days (252 of them working) standardization real number of days conventional number of days different scale during time (rising costs etc.)

Methodology Decomposition trend T t cyclic item C t season item S t residual (random) item E t Box-Jenkins method based on dependences among the observations (correlation analysis) basic models: moving averages MA(k): Y t = ɛ t + a 1 ɛ t 1 +... + a k ɛ t k auto-regression models AR(k): Y t = a 1 Y t 1 +... + a k Y t k + ɛ t

Forecast - point and interval forecast point forecast - one concrete value representing the best forecast (e.g. price of a given stock in 10 days will be 103.60 EUR) interval forecast - confidence interval meaning that the future value will lie in this interval with given probability (e.g. 95%-interval for the price of a given stock in 10 days is (99.50,107.70))

Forecast in structural model or in the time series model? This problem occur when the values of the time series are given by two or more random values (e.g. financial portfolio consisting of more stocks) Forecast constructed from the driving values may differ from the forecast obtained by the series analysis our decision

Forecast - statical and dynamical forecast statical forecast - estimate Ŷt is constructed from the real values (e.g. Ŷ t = 0.78Y t 1 ) dynamical forecast - estimate Ŷt is constructed from the estimated values (e.g. Ŷ t = 0.78Ŷt 1) less precise used only in special cases, e.g. in the case of outlying or missing value Y t 1

Forecast - precision measures Definition Forecast error is defined as e t = Y t Ŷt. The most often used measures: sum of squared errors SSE = n+h (Y t Ŷt) 2 = n+1 n+h t=n+1 e 2 t mean squared error MSE = 1 h n+h t=n+1 (Y t Ŷt) 2 = 1 h n+h t=n+1 e 2 t

Forecast - precision measures root mean squared error RMSE = 1 h n+h (Y t Ŷt) 2 = 1 h n+h e 2 t t=n+1 t=n+1 mean absolute error MAE = 1 h n+h t=n+1 mean absolute percentage error MAP E = 100 h Y t Ŷt = 1 h n+h t=n+1 n+h t=n+1 Y t Ŷt Y t e t

I. Decomposition methods

items: Decomposition of the time series trend T t cyclic item C t season item S t residual (random) item E t usage: additive: Y t = T t + C t + S t + E t multiplicative: Y t = T t C t S t E t smoothed series: Y t = T t + C t + S t and Y t = T t C t S t, respectively

Trend in time series Subjective methods - graphical smoothing Mathematical curves possibility of mathematical forecast assumption: Y t = T t + E t

Constant trend Trend in time series T t = β 0, t = 1, 2,... estimation b 0 of β 0 : n b 0 = Ȳ = 1 n t=1 Y t forecast of future value: Ŷ T = b 0

Linear trend Trend in time series T t = β 0 + β 1 t, t = 1, 2,... estimations b 0, b 1 of β 0 and β 1 : least squares method (MLS) estimated values n t=1 b 1 = ty t n+1 n Y 2 t=1 t n(n 2 1) 12 forecast of future value: Ŷ T = b 0 + b 1 T, b 0 = Ȳ n + 1 b 1 2

Trend in time series Quadratic trend T t = β 0 + β 1 t + β 2 t 2, t = 1, 2,... estimation b 0, b 1 and b 2 of β 0, β 1 and β 2 : least squares forecast of future value: Ŷ T = b 0 + b 1 t + b 2 t 2

Trend in time series Exponential trend T t = αβ t, β > 0, t = 1, 2,..., i.e. T t+1 /T t = β if β > 1...rise, if β < 1...fall estimation a, b of α and β: 1. ln T t = ln α + t ln β linear trend 2. calculate estimation a and b of ln α and ln β 3. a = e a, b = e b forecast of future value: Ŷ T = ab T Remark Sometimes, we used weighted LSE with weights w t, i.e. a, b = argmin n t=1 w t(ln Y t ln α t ln β) 2.

Gompertz trend Trend in time series T t = exp{γ + αβ t }, β > 0, t = 1, 2,... model with α < 1 and 0 < β < 1: bounded above by γ in the time t = ln( α)/ ln β change inflection (from convex to concave) used for booms

Trend in time series - moving averages assumption: each part time series can be smoothed by a polynomial function y t is replaced by the smoothed value ŷ t given by a weighted average of 2m + 1 values around Y t Example: We want to smooth the series {Y t } by a polynomial function of the third order used 5 values, i.e. for all t, we look for the estimates b 0, b 1, b 2, b 3 = argmin 2 (Y τ= 2 t+τ β 0 β 1 τ β 2 τ 2 β 3 τ 3 ) 2. Solution: Ŷ t = b 0 = 1 35 ( 3Y t 2 + 12Y t 1 + 17Y t + 12Y t+1 + ( 3)Y t+2 ) = 1 35 ( 3, 12, 17, 12, 3)Y t

Moving averages - forecast 1. calculate also b 1, b 2,... 2. forecast in future time T n m+k = b 0 + b 1 k + b 2 k 2 +... Example: In our example, we obtain ( b 1 = 1 2 65 τy t+τ 17 72 τ= 2 ) 2 τ 3 Y t+τ τ= 2 ( b 2 = 1 2 ) 2 τ 2 Y t+τ 2 Y t+τ 14 τ= 2 τ= 2 ( ) b 3 = 1 2 2 5 τ 3 Y t+τ 17 τy t+τ 72 τ= 2 τ= 2

Moving averages - forecast Example: For k = 1, 2,... we have Ŷ n 2+k = b 0 + b 1 k + b 2 k 2 + b 3 k 3 = = 1 35 ( 3, 12, 17, 12, 3)Y n 2 + k 12 (1, 8, 0, 8, 1)Y n 2+ thus e.g. + k2 14 (2, 1, 2, 1, 2)Y n 2 + k3 12 ( 1, 2, 0, 2, 1)Y n 2, etc. Ŷ n+1 = Ŷn 2+3 = 1 5 ( 4, 11, 4, 14, 16)Y n 2

Moving averages Other types Simple averages: Ŷ t = 1 2m+1 m i= m Y t+i linear or constant trend Central averages: used when we need average from an even number of observation (12 month, 4 seasons etc.) Ȳ (2m) t = 1 4m (Y t m + 2Y t m+1 +... + 2Y t+m 1 + Y t+m )

Season item in time series Assumption: Y t = T t + S t + E t In order to do the previous analyses, we need to eliminate the season item S t. When having k seasons, the item S t consists of the season factors I 1,..., I k : for additive method we suppose I 1 +... + I k = 0, for multiplicative method we suppose I 1... I k = 1.

Additive method Season item in time series 1. Construct center averages Ȳ (2m) t (e.g. Ȳ (12) t ). 2. Set Yt = Y t Ȳ (2m) t (removing trend). 3. Calculate (non-centered) season factors Ii, i = 1,..., 2m as classical average of the values Yt corresponding to the i-th season. 4. Center the values I1,..., I2m by 5. Remove season item as when t is the i-th season. I i = I i Ī = I i I 1 +... + I2m. 2m Ŷ t = Y t I i,

Season item in time series Multiplicative method 1. Construct center averages Ȳ (2m) t (e.g. Ȳ (12) t ). 2. Set Yt = Y t /Ȳ (2m) t (removing trend). 3. Calculate (non-centered) season factors Ii, i = 1,..., 2m as classical average of the values Yt corresponding to the i-th season. 4. Center the values I1,..., I2m by 5. Remove season item as I i = I i Î = when t is the i-th season. I i 12 I 1... I 2m. Ŷ t = Y t I i,

II. Autocorrelation methods: Box-Jenkins methodology

Stationarity strict stationarity - the behavior of the time series is time-invariant, i.e. the joint distribution of (Y t1,..., Y tk ) is the same as the distribution of (Y t1 +h,..., Y tk +h) for arbitrary h. (weak) stationarity - for arbitrary s, t and h, we have EY t = µ and cov(y s, Y t ) = E(Y s µ)(y t µ) = cov(y s+h, Y t+h ), especially vary t = σ 2 Y = const. Remark We will assume weak stationary if it is not said otherwise.

Autocovariance and autocorrelation function Definition Autocovariance function (for a delay k) is defined as γ k = cov(y t, Y t k ), k =..., 1, 0, 1,.... Definition Autocorrelation function (for a delay k) is defined as ρ k = γ k γ 0, k =..., 1, 0, 1,.... Remark From weak stationarity, it holds that γ k = γ k.

Autocovariance and autocorrelation function - estimation Let n Ȳ = 1 n be the estimation of the mean value µ. Then estimation of the autocovariance function (for a delay k) is defined as c k = 1 n (Y t n k Ȳ )(Y t k Ȳ ), k = 0, 1,..., n 1, t=k+1 estimation of the autocorrelation function (for a delay k) is defined as r k = c k c 0, k = 0, 1,..., n 1. t=1 Y t

Breakpoint of autocorrelation function Definiton By the breakpoint of autocorrelation function we mean a point k 0 such that ρ k = 0 for all k > k 0. Bartlett approximation: If ρ k = 0 for all k > k 0, then r k N 0, 1 n ( 1 + 2 k 0 j=1 r 2 j).

Partial autocorrelation function Definiton Partial autocorrelation function is defined as ρ kk = corr(y t, Y t k ) conditionally on the fixed known values Y t k+1,...,y t 1. Its estimation is given by r kk = r k k 1 r j=1 k 1,j r k j 1 k 1 r, j=1 k 1,j r j where r 11 = r 1 and r kj = r k 1,j r kk r k 1,k j for j = 1,..., k 1. 1 Quenouille approximation: Breakpoint k 0 is given by r kk N(0, ) n for k > k 0.

Process of moving averages MA(q) Warning! Different method than the moving averages from decomposition!!! Process MA(q) is defined as Y t = ɛ t + θ 1 ɛ t 1 +... + θ q ɛ t q, where θ 1,..., θ q are parameters and {ɛ t, t = 0, 1,..., n} is a white noise, i.e. a sequence of independent random variables with identical distribution N(0, σ 2 ).

Process of moving averages MA(q) Its main properties are: stationary µ = 0 and σ 2 Y = (1 + θ 2 1 +... + θ 2 q)σ 2 autocorrelation function ρ k = θ k + θ 1 θ k+1 +... + θ q k θ q 1 + θ 2 1 +... + θ 2 q for k = 1,..., q and ρ k = 0 for k > q.

Autoregression process AR(p) Process AR(p) is defined as Y t = φ 1 Y t 1 +... + φ p Y t p + ɛ t, where φ 1,..., φ p are parameters and {ɛ t, t = 0, 1,..., n} is a white noise.

Autoregression process AR(p) Its main properties are: stationary if for all roots x 1,..., x p of the equation 1 φ 1 x... φ p x p = 0, it holds that x 1 > 1,..., x p > 1 (in complex space) when stationary then µ = 0 and σy 2 = σ 2 /(1 φ 1 ρ 1... φ p ρ p ) autocorrelation function satisfies ρ k = φ 1 ρ k 1 +... + φ p ρ k p for k = 1,..., q and ρ k = 0 for k > q.

Mixed process ARMA(p,q) Process ARMA(p, q) is defined as Y t = φ 1 Y t 1 +... + φ p Y t p + ɛ t + θ 1 ɛ t 1 +... + θ q ɛ t q, where φ 1,..., φ p, θ 1,..., θ q are parameters and {ɛ t, t = 0, 1,..., n} is a white noise. When it is stationary then µ = 0 and autocorrelation function satisfies ρ k = φ 1 ρ k 1 +... + φ p ρ k p for k = 1,..., q and ρ k = 0 for k > q.

Model identification Shape of correlation functions AR(p) MA(q) ARMA(p,q) ρ k k 0 does not exist, k 0 = q k 0 does not exist, ρ k has U-shape ρ k has U-shape after q p values ρ kk k 0 = p k 0 does not exist, k 0 does not exist, ρ kk has U-shape ρ kk bounded by U-curve after q p values U-curve...curve of sinus type with decreasing amplitude, curve of decreasing geometrical series or their combination

Model identification Information criteria 1. Suppose ARMA(p, q) model. 2. Estimate the parameters for some grid of p and q. 3. Calculate (ˆp, ˆq) = argmina(k, l), where A(k, l) is a suitable criterium, e.g. A(k, l) = ˆσ k,l A(k, l) = ln ˆσ k,l + 2(k+l+1) A(k, l) = ln ˆσ k,l + n 2(k+l+1) ln n n

Estimating parameters from the relations among the parameters and the correlation functions, e.g. Yule-Walker equations: iterations ρ k = φ 1 ρ k 1 +... + φ p ρ k p, k = 1,..., p

stationarity control correlation functions properties of white noise Model control

Thank you for your attention! :-)