Time Series Modeling of Financial Data. Prof. Daniel P. Palomar

Transcription

1 Time Series Modeling of Financial Data Prof. Daniel P. Palomar The Hong Kong University of Science and Technology (HKUST) MAFS6010R- Portfolio Optimization with R MSc in Financial Mathematics Fall , HKUST, Hong Kong

2 Outline 1 Financial Data and Stylized Facts 2 i.i.d. Models 3 Mean Models 4 Covariance Models - Volatility Clustering 5 Fitting of Models - Estimation or Calibration 6 Summary

3 Outline 1 Financial Data and Stylized Facts 2 i.i.d. Models 3 Mean Models 4 Covariance Models - Volatility Clustering 5 Fitting of Models - Estimation or Calibration 6 Summary

4 Asset log-prices Let p t be the price of an asset at (discrete) time index t. The fundamental model is based on modeling the log-prices y t log p t as a random walk: y t = µ + y t 1 + ɛ t SP500 index log price Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan D. Palomar Time Series Modeling 4 / 64

5 Asset returns Simple return (a.k.a. linear or net return) is R t p t p t 1 p t 1 = p t p t 1 1. Log-return (a.k.a. continuously compounded return) is r t y t y t 1 = log p t p t 1 = log (1 + R t ). Note r t = log (1 + R t ) R t when R t is small. D. Palomar Time Series Modeling 5 / 64

6 S&P 500 index - log-returns SP500 index log return Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan D. Palomar Time Series Modeling 6 / 64

7 Stylized facts of financial data A set of properties, common across many instruments, markets, and time periods, has been observed by independent studies and classified as stylized facts. 1 Lack of stationarity: past returns do not necessarily reflect future performance (watch out fund s brochures) Absence of autocorrelations: autocorrelations of returns are often insignificant (efficient market hypothesis) Heavy tails: Gaussian distributions generally do not hold in financial data (even after correcting for volatility clustering) Gain/loss asymmetry: basically asymmetry of the pdf Aggregational Gaussianity: for lower frequencies, the distribution tends to become more Gaussian. Volatility clustering: high-volatility evens tend to cluster in time 1 R. Cont, Empirical properties of assets returns: Stylised facts and statistical issues, Quantitative Finance, vol. 1, pp , D. Palomar Time Series Modeling 7 / 64

8 Autocorrelation ACF of S&P 500 log-returns: S&P 500 index ACF of log returns Lag D. Palomar Time Series Modeling 8 / 64

9 Non-Gaussianity and asymmetry Histograms of S&P 500 log-returns: Histogram of daily log returns Histogram of weekly log returns Histogram of biweekly log returns Density Density Density return return return Histogram of monthly log returns Histogram of bimonthly log returns Histogram of quarterly log returns Density Density Density return return return D. Palomar Time Series Modeling 9 / 64

10 Volatility clustering S&P 500 log-returns: SP500 index log return Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan D. Palomar Time Series Modeling 10 / 64

11 Volatility clustering removed Standardized S&P 500 log-returns: Standardized SP500 index log return Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan D. Palomar Time Series Modeling 11 / 64

12 Outline 1 Financial Data and Stylized Facts 2 i.i.d. Models 3 Mean Models 4 Covariance Models - Volatility Clustering 5 Fitting of Models - Estimation or Calibration 6 Summary

13 General structure of a model Denote log-return of N assets as r t R N. Denote F t 1 as the previous historical data. Financial modeling aims at modeling r t conditional on F t 1. Conditional on F t 1, we can decompose r t R N as follows: where µ t is the conditional mean r t = µ t + w t µ t = E[r t F t 1 ] w t is a white noise with zero mean and conditional covariance Σ t = E[(r t µ t )(r t µ t ) T F t 1 ]. D. Palomar Time Series Modeling 13 / 64

14 i.i.d. model It assumes r t follows an i.i.d. distribution. That is, both the conditional mean and conditional covariance are constant µ t = µ, Σ t = Σ w. Very simple model, however, it is one of the most fundamental assumptions for many important works, e.g., the Nobel prize-winning Markowitz portfolio theory 2. 2 H. Markowitz, Portfolio selection, J. Financ., vol. 7, no. 1, pp , D. Palomar Time Series Modeling 14 / 64

15 Factor model The factor model is r t = α + Bf t + w t, where α denotes a constant vector f t R K with K N is a vector of a few factors that are responsible for most of the randomness in the market, B R N K denotes how the low dimensional factors affect the higher dimensional market; w t is a white noise residual vector that has only a marginal effect. The factors can be explicit or implicit. Widely used by practitioners (they buy factors at a high premium). Connections with Principal Component Analysis (PCA) 3. 3 I. Jolliffe, Principal Component Analysis. Springer-Verlag, D. Palomar Time Series Modeling 15 / 64

16 i.i.d. vs factor models Factor models are special cases of the i.i.d. model with the variation being decomposed into two parts: low dimensional factors and marginal noise. The explicit factor model explains the log-returns with a smaller number of fundamental or macroeconomic variables, however, in general there is no systematic method to choose the right factors. The hidden factor model explores the structure of the covariance matrix, is a more systematical approach and thus it may provide a better explanatory power, however, does not have explicit econometric interpretations. D. Palomar Time Series Modeling 16 / 64

17 Outline 1 Financial Data and Stylized Facts 2 i.i.d. Models 3 Mean Models 4 Covariance Models - Volatility Clustering 5 Fitting of Models - Estimation or Calibration 6 Summary

18 Time-correlation The previous i.i.d. and factor models cannot characterize any time-correlation in the log-returns: S&P 500 index ACF of log returns Lag D. Palomar Time Series Modeling 18 / 64

19 VAR(1) model Recall that we model the log-returns r t = y t = y t y t 1, where y t denotes the log-prices. The VAR (Vector Auto-Regressive) model or order 1 is r t = φ 0 + Φ 1 r t 1 + w t, where the vector φ 0 R N and the matrix Φ 1 R N N are parameters, w t is a white noise series with zero mean and constant covariance matrix Σ w. The conditional mean and covariance matrix are µ t = φ 0 + Φ 1 r t 1, Σ t = Σ w. D. Palomar Time Series Modeling 19 / 64

20 AR(1) example A univariate AR(1) path looks like 0.04 AR(1) path D. Palomar Time Series Modeling 20 / 64

21 VAR(p) model The VAR (Vector Auto-Regressive) model of order p is r t = φ 0 + p Φ i r t i + w t, i=1 where p is a nonnegative integer and the vector φ 0 R N and the matrices Φ i R N N are parameters, w t is a white noise series with zero mean and constant covariance matrix Σ w. The conditional mean and covariance matrix are µ t = φ 0 + Σ t = Σ w. p Φ i r t i, i=1 D. Palomar Time Series Modeling 21 / 64

22 VARMA(p,q) model The VARMA (Vector Auto-Regressive and Moving Average) model is r t = φ 0 + p Φ i r t i + w t i=1 where p and q are nonnegative integers and q Θ j w t j, the vector φ 0 R N and the matrices Φ i, Θ j R N N are parameters, w t is a white noise series with zero mean and constant covariance matrix Σ w. j=1 The conditional mean and covariance matrix are µ t = φ 0 + Σ t = Σ w. p Φ i r t i i=1 q Θ j w t j, j=1 D. Palomar Time Series Modeling 22 / 64

23 Order selection of models All time series models have an order that is typically assumed to be known and given, e.g., the orders p and q in a VARMA(p,q) model. In practice, the order of a model is unknown and also has to be determined from the observed data. Observe that the higher the order, the more parameters the model has to fit the data and, thus, the better the fit. So it seems the best model will be the one with higher order. However, this is completely wrong, because it will be doomed to overfit the data: one thing is to fit better the training data, a very different one is to fit better the future coming data. In practice, there are two common approaches: cross-validation: splitting the data into a training part and a cross-validation part, the latter being used to test the model trained with the training data for different combinations of orders. penalized estimation methods: penalizing the number of parameters of the model with a penalty term like: AIC, BIC, SIC, HQIC, etc. 4 4 H. Lütkepohl, New Introduction to Multiple Time Series Analysis. Springer Science & Business Media, D. Palomar Time Series Modeling 23 / 64

24 VARIMA(p,d,q) model A multivariate time series y t is said to be a VARIMA(p,1,q) process if it is nonstationary but after differencing the series times x t = y t y t 1 then x t follows a stationary VARMA(p,q) model. More generally, a VARIMA(p,d,q) process has to be differenced d times to obtain a stationary VARMA(p,q) process. In finance, price series p t (or log-prices y t = log (p t )) are believed to be nonstationary, but the log-return series r t = y t y t 1 = log (p t ) log (p t 1 ) is stationary. Thus, it is the same to talk about a VARIMA(p,1,q) log-price series and about a VARMA(p,q) log-return series. D. Palomar Time Series Modeling 24 / 64

25 Vector Error Correction Model (VECM) Until now we have focused on modeling directly the log-returns r t = y t = y t y t 1, where y t denotes the log-prices. The reason is that in general the log-price time series y t is not weakly stationary (first and second-order moments are not constant). Example: think of Apple stock whose log-prices keep increasing. On the other hand, the log-return time series r t is weakly stationary (at least over some time horizon), which is good. However, it turns out that differencing may destroy part of the structure in the relationship among the log-prices of the stocks which may be invaluable for forecasting. So it makes sense to analyze the original (probably non-stationary, be careful!) time series in y t directly: y t = φ 0 + p Φ i y t i + w t. i=1 D. Palomar Time Series Modeling 25 / 64

26 VECM The VECM 5 is better written as p 1 r t = φ 0 + Πy t 1 + Φ i r t i + w t, i=1 where the term Πy t 1 is called error correction term and p Φ j = i=j+1 Φ i Π = (I Φ 1 Φ p ). The conditional mean and covariance matrix are p 1 µ t = φ 0 + Πy t 1 + Φ i r t i, Σ t = Σ w. 5 R. F. Engle and C. W. J. Granger, Co-integration and error correction: Representation, estimation, and testing, Econometrica: Journal of the Econometric Society, pp , D. Palomar Time Series Modeling 26 / 64 i=1

27 VECM - Matrix Π The matrix Π is of extreme importance. Notice that from the model r t = φ 0 + Πy t 1 + p 1 Φ i=1 i r t i + w t one can conclude that Πy t must be stationary even though y t is not!!! If that happens, it is said that y t is cointegrated. There are three possibilities for Π: rank (Π) = 0: This implies Π = 0, thus y t is not cointegrated (so no mystery here) and the VECM reduces to a VAR model on the log-returns. rank (Π) = N: This implies Π is invertible and thus y t must be stationary already 0 < rank (Π) < N: This is the interestinc case and Π can be decomposed as Π = αβ T with α, β R N r with full column rank. This means that y t has r linearly independent cointegrated components, i.e., β T y t, which can be used to design mean-reversion statistical arbitrage investment strategies (e.g., pairs trading). D. Palomar Time Series Modeling 27 / 64

28 Outline 1 Financial Data and Stylized Facts 2 i.i.d. Models 3 Mean Models 4 Covariance Models - Volatility Clustering 5 Fitting of Models - Estimation or Calibration 6 Summary

29 Volatility clustering The previously models focus on modeling the conditional mean but assume that the conditional covariance is constant: Σ t = Σ w. In practice, the volatility (i.e., the square root of conditional variance) is clustered. If we take the sample volatility computed from the past month as the estimate of the conditional volatility, the real data shows clearly volatility clustering patterns. D. Palomar Time Series Modeling 29 / 64

30 Volatility clustering Synthetic Normal White Noise White noise Conditional sample volatility Jun 10 Jun 11 Jun 12 Jun 13 Jun 14 Jun 15 volatility clusters SP Log returns Conditional sample volatility Jun 10 Jun 11 Jun 12 Jun 13 Jun 14 Jun 15

31 Univariate ARCH model An autoregressive conditional heteroskedasticity (ARCH) Model is one of the earliest model to deal with the volatility clustering effect. The ARCH(m) model 6 is w t = σ t z t, where z t is a white noise series with zero mean and constant unit variance, and the conditional variance σ 2 t is modeled by σ 2 t = ω + m α i wt i 2 Here, m is a nonnegative integer, ω > 0, α i 0 for all i > 0. i=1 6 R. F. Engle, Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation, Econometrica: Journal of the Econometric Society, pp , D. Palomar Time Series Modeling 31 / 64

32 Univariate ARCH model Even though the ARCH model can model the conditional heteroskedasticity, it has several disadvantages: positive and negative noise have the same effects on volatility, but in practice they have different impact too restrictive to capture some patterns, e.g., excess kurtosis doesn t provide any new insight, just a mechanical way to describe the behavior of conditional variance tend to overpredict the volatility because it responds slowly to large isolated noise clusters. D. Palomar Time Series Modeling 32 / 64

33 Univariate ARCH model example 0.6 ARCH Noise ARCH σ t Synthetic ARCH Noise D. Palomar Time Series Modeling 33 / 64

34 Univariate GARCH model A limitation of the ARCH model is that the high volatility is not persistent enough. This can be overcame by a Generalized ARCH (GARCH) model. 7 The GARCH(m, s) model is w t = σ t z t, where z t is a white noise series with zero mean and constant unit variance, and the conditional variance σ 2 t is modeled by σ 2 t = ω + m α i wt i 2 + i=1 s β j σt j. 2 Here, m and s are nonnegative integers, ω > 0, α i 0, β j 0 for all i > 0 and j > 0 and m i=1 α i + s j=1 β j 1. 7 T. Bollerslev, Generalized autoregressive conditional heteroskedasticity, Journal of Econometrics, vol. 31, no. 3, pp , D. Palomar Time Series Modeling 34 / 64 j=1

35 Univariate GARCH example 1.5 GARCH Noise GARCH σ t Synthetic GARCH Noise D. Palomar Time Series Modeling 35 / 64

36 Univariate ARCH vs GARCH example 0.5 ARCH(1) Noise ARCH(1) σ t Synthetic ARCH(1) Noise ARCH(9) Noise ARCH(9) σ t Synthetic ARCH(9) Noise GARCH(1,1) Noise GARCH(1,1) σ t Synthetic GARCH(1,1) Noise D. Palomar Time Series Modeling 36 / 64

37 Multivariate GARCH model The multivariate noise (a vector) is modeled as w t = Σ 1/2 t z t, where z t R N is an i.i.d. white noise series with zero mean and constant covariance matrix I. The key is to model the conditional covariance matrix Σ t. But watch out as the number of parameters may quickly explode... and that will inevitably produce overfitting. D. Palomar Time Series Modeling 37 / 64

38 VEC GARCH model One of the first extensions to the vector case is the vector (VEC) GARCH model: vech (Σ t ) = a 0 + m à i vech(w t i wt i) T + i=1 s B j vech (Σ t j ), where m and s are nonnegative integers, vech ( ) is the half-vectorization operator that keeps the N(N + 1)/2 lower triangular part of its N N matrix argument, a 0 is an N(N + 1)/2 dimensional vector, and à i, B j are N(N + 1)/2 by N(N + 1)/2 matrices. Advantage: This model is very flexible. Disadvantages: Does not guarantee Σ t to be a positive definite covariance matrix and the number of parameters grows quickly as O ( (m + s) N 4). j=1 D. Palomar Time Series Modeling 38 / 64

39 Diagonal VEC (DVEC) model The DVEC model 8 is more parsimonious model assuming that à i, B j are diagonal and can be simplified as Σ t = A 0 + m A i (w t i wt i) T + i=1 s B j Σ t j, where A i, B j are symmetric N N matrix parameters. Here, the operator denotes the Hadamard (elementwise) product can be interpreted as moving weight matrices. Advantage: This is an element-wise GARCH model, so very simple. Disadvantages: Still Σ t is not guaranteed to be positive-definite and the number of parameters grows more slowly but still fast as O ( (m + s) N 2). j=1 8 T. Bollerslev, R. F. Engle, and J. M. Wooldridge, A capital asset pricing model with time-varying covariances, The Journal of Political Economy, pp , D. Palomar Time Series Modeling 39 / 64

40 BEKK model To guarantee a positive-definite Σ t, the Baba-Engle-Kraft-Kroner (BEKK) model 9 was proposed as Σ t = A 0 A T 0 + m A i (w t i wt i)a T T i + i=1 s B j Σ t j B T j, where A i, B j are N N matrix parameters and A 0 is lower triangular. j=1 Advantage: Guarantees positive definiteness of Σ t. Disadvantages: The parameters A i and B j do not have direct interpretations. Number of parameters still increases as O ( (m + s) N 2) (although now roughly the number of parameters is twice as that in DVEC). 9 R. F. Engle and K. F. Kroner, Multivariate simultaneous generalized ARCH, Econometric Theory, vol. 11, no. 01, pp , D. Palomar Time Series Modeling 40 / 64

41 CCC model The constant conditional correlation (CCC) model 10 restricts the number of parameters while still guaranteeing the positive definite covariance. The idea is to model the conditional heteroskedasticity in each asset while having a constant correlation. Mathematically, the model is Σ t = D t CD t, where D t = Diag (σ 1,t,..., σ N,t ) is the time-varying conditional volatilities of each stock and C is the CCC matrix of the standardized noise vector η t = D 1 t w t. Advantages: Guarantees positive definiteness of Σ t and small number of parameters that grows as O ( (m + s) N + N 2). Disadvantages: Not too flexible due to constant asset correlations. 10 T. Bollerslev, Modelling the coherence in short-run nominal exchange rates: A multivariate generalized arch model, The Review of Economics and Statistics, pp , D. Palomar Time Series Modeling 41 / 64

42 DCC model The main limitation of the CCC model is that the correlation is constant. To overcome this drawback, the dynamic conditional correlation (DCC) was proposed 11 as Σ t = D t C t D t, where C t contains diagonal elements equal to 1. In particular, Engle modeled it as follows: q ij,t C ij,t = qii,t q jj,t with each q ij,t modeled by a simple GARCH(1,1) model: q ij,t = αη i,t 1 η j,t 1 + (1 α) q ij,t 1 11 R. F. Engle, Dynamic conditional correlation: A simple class of multivariate generalized autoregressive conditional heteroskedasticity models, Journal of Business & Economic Statistics, vol. 20, no. 3, pp , D. Palomar Time Series Modeling 42 / 64

43 DCC model More compactly, in matrix form: Q t = αη t η T t + (1 α) Q t 1. and C t = Diag 1/2 (Q t ) Q t Diag 1/2 (Q t ). Advantages: Guarantees positive definiteness of Σ t and small number of parameters that grows as O ((m + s) N). Disadvantages: Good flexibility, although it forces all the correlation coefficients to have the same memory via the same α. D. Palomar Time Series Modeling 43 / 64

44 Beyond The previous models for the conditional mean and covariance matrix can be jointly combined to fit the real financial data better. Limitations: High-frequency data: when the sampling period becomes very small, say minutes, seconds, or even smaller, the previous models become invalid and one reaches a quantum regime where things are not fluid anymore but quantized into the limit order book. Not only the models have to be properly modified, but also the computer and internet communication speed matter (e.g., co-location of computers). Heavy tails: most models assume a Gaussian distribution for simplicity, but they can be easily extended to deal with heavy-tailed distributions. Lack of stationarity: financial data is only stationary for some time horizon, this produces a tradeoff between having enough data to properly estimate the parameters of the model but still within the stationarity time horizon. Other practical details: different stocks may have a different historical length and some days the prices may be missing due to no trading or bad quality of data. D. Palomar Time Series Modeling 44 / 64

45 Great References on Time Series Models H. Lütkepohl, New Introduction to Multiple Time Series Analysis. Springer Science & Business Media, 2007 R. S. Tsay, Analysis of Financial Time Series, 3rd. John Wiley & Sons, 2010 R. S. Tsay, Multivariate Time Series Analysis: With R and Financial Applications. John Wiley & Sons, 2014 D. Palomar Time Series Modeling 45 / 64

46 Outline 1 Financial Data and Stylized Facts 2 i.i.d. Models 3 Mean Models 4 Covariance Models - Volatility Clustering 5 Fitting of Models - Estimation or Calibration 6 Summary

47 General estimation methodologies: LS Least squares (LS) estimator: The idea is to define an error between the observed financial data and the model under consideration, and then minimize the l 2 -norm of the error. For example, for a VAR(1) model r t = φ 0 + Φ 1 r t 1 + w t, where we have T observations, the problem to solve would be minimize φ 0,Φ 1 T t=2 r t φ 0 Φ 1 r t 1 2. D. Palomar Time Series Modeling 47 / 64

48 General estimation methodologies: MLE Maximum likelihood estimator (MLE): The idea is to assume some distribution for the residual of the model w t, typically Gaussian for mathematical simplicity and tractability: f (r) = 1 (2π) N Σ e 1 2 (r µ)t Σ 1 (r µ) Then, given the T samples, the negative log-likelihood function is formed l(µ, Σ) = T 2 log Σ T (r t µ) T Σ 1 (r t µ) + const. t=1 Finally, one can minimize the negative log-likelihood with respect to the parameters to be estimated, like µ and Σ. D. Palomar Time Series Modeling 48 / 64

49 Estimation of i.i.d. model: Sample estimators i.i.d. model: r t = µ + w t, where µ R N is the mean and w t R N is a white noise series with zero mean and constant covariance matrix Σ. Good old sample estimators (sample mean and sample covariance matrix): ˆµ = 1 T T r t, t=1 ˆΣ = 1 T 1 T (r t ˆµ)(r t ˆµ) T. t=1 In practice: they are horrible! They can be improved with heavy-tail estimators and shrinkage. However, they still leave something to be desired. D. Palomar Time Series Modeling 49 / 64

50 Estimation of i.i.d. model: LS estimator Minimize the least-square error in the T observed i.i.d. samples: minimize µ 1 T T t=1 r t µ 2 2. The optimal solution is the sample mean: T ˆµ = 1 T t=1 r t The sample covariance of the residuals is the sample covariance estimator: ˆΣ = 1 T T (r t ˆµ)(r t ˆµ) T. t=1 D. Palomar Time Series Modeling 50 / 64

51 Estimation of i.i.d. model: Gaussian Maximum Likelihood Estimator (MLE) Assume r t are i.i.d. Gaussian distributed: 1 f (r) = (2π) N Σ e 1 2 (r µ)t Σ 1 (r µ) Given the T i.i.d. samples, the negative log-likelihood function is l(µ, Σ) = T 2 log Σ + 1 T (r t µ) T Σ 1 (r t µ) + const. 2 t=1 Setting the derivative of l(µ, Σ) w.r.t. µ and Σ 1 to zero and solving the equations yield: µ = 1 T r t, T Σ = 1 T t=1 T (r t µ) (r t µ) T. t=1 D. Palomar Time Series Modeling 51 / 64

52 Estimation of Factor Model: MLE Likelihood of the factor model: The log-likelihood of the parameters (α, Σ) given T i.i.d. observations r t = α + Bf t + w t is L (α, Σ) = log p (x 1,..., x T α, Σ) = TN 2 log (2π) T 2 log Σ 1 2 Maximum likelihood estimation (MLE): minimize α,σ,b,ψ subject to T (x t α) T Σ 1 (x t α) t=1 T 2 log Σ T t=1 (x t α) T Σ 1 (x t α) Σ = BB T + Ψ Note that without the constraint, the solution would be ˆα = 1 T x t and ˆΣ = 1 T (x t ˆα) (x t ˆα) T. T T t=1 D. Palomar Time Series Modeling 52 / 64 t=1

53 which is asymptotically chi-square with 1 2 ((N K)2 N K) degrees of freedom. D. Palomar Time Series Modeling 53 / 64 Estimation of Factor Model: MLE MLE solution: minimize α,σ,b,ψ subject to T 2 log Σ T t=1 (x t α) T Σ 1 (x t α) Σ = BB T + Ψ Employ the Expectation-Maximization (EM) algorithm to compute ˆα, ˆB, and ˆΨ. Estimate factor realization {f t } using, for example, the GLS estimator: ˆft = (ˆB T ˆΨ 1 ˆB) 1 ˆB T ˆΨ 1 x t, t = 1,..., T. The number of factors K can be estimated with a variety of methods such as the likelihood ratio (LR) test, Akaike information criterion (AIC), etc. For example: LR(K) = (T (2N + 5) 2 ) (log 3 K) ˆΣ SCM log ˆBˆB T + ˆΨ

54 Principal Component Analysis PCA: is a dimension reduction technique used to explain the majority of the information in the covariance matrix. N-variate random variable x with E [x] = α and Cov [x] = Σ. Spectral/eigen decomposition Σ = ΓΛΓ T where Λ = diag (λ 1,..., λ K ) with λ 1 λ 2 λ K Γ orthogonal K K matrix: Γ T Γ = I N Principal component variables: p = Γ T (x α) with [ ] E [p] = E Cov [p] = Cov Γ T (x α) [ Γ T (x α) = Γ T (E [x] α) = 0 ] = Γ T Cov [x] Γ = Γ T ΣΓ = Λ. p is a vector of zero-mean, uncorrelated random variables, in order of importance (i.e., the first components explain the largest portion of the sample covariance matrix) In terms of multifactor model, the K most important principal components are the factor realizations. D. Palomar Time Series Modeling 54 / 64

55 Estimation of Factor Model: PCA Factor model from PCA: From PCA, we can write the random vector x as x = α + Γp where E [p] = 0 and Cov [p] = Λ = diag (Λ 1, Λ 2 ). Partition Γ = [ ] Γ 1 Γ 2 where Γ1 corresponds to the K largest eigenvalues of [ Σ. ] p1 Partition p = where p 1 contains the firs K elements p 2 Then we can write where x = α + Γ 1 p 1 + Γ 2 p 2 = α + Bf + ɛ B = Γ 1, f = p 1 and ɛ = Γ 2 p 2. This is like a factor model except that Cov [ɛ] = Γ 2 Λ 2 Γ T 2, where Λ 2 is a diagonal matrix of last N K eigenvalues but Cov [ɛ] is not diagonal... D. Palomar Time Series Modeling 55 / 64

56 Estimation of Factor Model: Why PCA? When the factors are unknown (implicit factor modeling), the idea is to obtain the factors from the returns themselves: or, more compactly, x t = α + Bf t + ɛ t, t = 1,..., T x t = α + B f t = C T x t + d ( ) C T x t + d + ɛ t, t = 1,..., T The problem formulation for the parameter estimation is minimize α,b,c,d 1 T T t=1 x t α + B ( C T x t + d ) 2 Solution is involved to derive and is given by: f t = Γ T 1 (x t α) or, if normalized factors are desired, f t = Λ Γ T 1 (x t α). D. Palomar Time Series Modeling 56 / 64

57 Estimation of Factor Model via PCA 1 Compute sample estimates ˆα = 1 T x t and ˆΣ = 1 T T t=1 T (x t ˆα) (x t ˆα) T. t=1 2 Compute spectral decomposition: ˆΣ = ˆΓ ˆΛˆΓ T, ˆΓ = [ ˆΓ 1 ˆΓ 2 ] 3 Form the factor loadings, factor realizations, and residuals (so that x t = ˆα + ˆBˆf t + ˆɛ t ): ˆB = ˆΓ 1 ˆΛ 1 2 1, ˆft = ˆΛ Covariance matrices: ˆΣ f = 1 T ˆftˆfT T t = I K, ˆΨ = 1 T t=1 ˆΓ T 1 (x t ˆα), T ˆɛ t ˆɛ T t t=1 ˆɛ t = x t ˆα ˆBˆf t ˆΣ = ˆΓ 1 ˆΛ 1 ˆΓ 1 + ˆΓ 2 ˆΛ 2 ˆΓ 2 = ˆB ˆΣ f ˆB T + ˆΨ. = ˆΓ 2 ˆΛ 2 ˆΓ 2 (not diagonal!) D. Palomar Time Series Modeling 57 / 64

58 Estimation of Factor Model: Principal Factor Method Since the previous method does not lead to a diagonal covariance matrix for the residual, let s refine the method with the following iterative approach 1 PCA: sample mean: ˆα = x = 1 T XT 1 T demeaned matrix: X T = X T x1 T T sample covariance matrix: ˆΣ = 1 T X T X eigen-decomposition: ˆΣ = ˆΓ ˆΛˆΓ T set index s = 0 2 Estimates: ˆB (s) = ˆΓ 1 ˆΛ ˆΨ (s) = diag( ˆΣ ˆB (s) ˆB T (s) ) ˆΣ (s) = ˆB (s) ˆB T (s) + ˆΨ (s) 3 Update the eigen-decomposition as ˆΣ ˆΨ (s) = ˆΓ ˆΛˆΓ T 4 Repeat Steps 2-3 generating a sequence of estimates (ˆB (s), ˆΨ (s), ˆΣ (s) ) until convergence. D. Palomar Time Series Modeling 58 / 64

59 Estimation of VAR model with sparsity Sparsity refers to parameters having zero entries, so effectively reducing the number of parameters and the danger of overfitting. Mathematically, the number of nonzero entries of a vector or matrix is expressed via the l 0 -pseudo norm 0. In practice, the l 0 -pseudo norm is tough to manage and optimize and it is commonly approximated with the l 1 -norm 1. Countless of examples where sparsity naturally arises in finance: Sparse PCA for factor modeling: computation of sparse eigenvectors is key in the high-dimensional setting for automatic feature selection. 12,13 Sparse parameters in all the multivariate models are required for parameter reduction (feature selection) such as VAR: minimize φ 0,Φ 1 T t=1 r t φ 0 Φ 1 r t 1 2 subject to Φ 1 0 P 12 J. Song, P. Babu, and D. P. Palomar, Sparse generalized eigenvalue problem via smooth optimization, IEEE Trans. Signal Process., vol. 63, no. 7, pp , K. Benidis, Y. Sun, P. Babu, and D. P. Palomar, Orthogonal sparse PCA and covariance estimation via procrustes reformulation, IEEE Trans. Signal Process., vol. 64, no. 23, pp , D. Palomar Time Series Modeling 59 / 64

60 Estimation of models with low rank Low-rank matrices are also useful to effectively reduce the number of parameters to be estimated and the danger of overfitting. For example, a VAR(1) model has parameters φ 0 and Φ 1, which amounts to N + N 2 parameters. If the matrix has, say, rank r N, then the number of parameters becomes N + 2Nr, which can be much smaller. If N = 100 and r = 5, then we go from 10, 100 parameters to 1, 100, which is one order of magnitude smaller. Low-rank naturally arises in finance: Low-rank matrices are required to discover the low-dimensional structure in models like VAR: minimize T φ t=1 r t φ 0 Φ 1 r t 1 2 0,Φ 1 subject to rank (Φ 1 ) K Low-rank matrices are necessary in multivariate GARCH models for dimensionality reduction: minimize T {Σ t},{b i } 2 log Σ + 1 T 2 t=1 w t T Σ 1 w t subject to Σ t = B 0 B T 0 + s j=1 B jσ t j B T j rank (B j ) K. D. Palomar Time Series Modeling 60 / 64

61 Estimation of VECM with low rank Π Another example where a low-rank matrix is required is in VECM modeling, in particular for matrix Π: minimize φ 0,Φ 1,Π subject to T t=1 r t φ 0 Πy t 1 Φ 1 r t 1 2 Φ 1 0 P rank (Π) K. D. Palomar Time Series Modeling 61 / 64

62 Outline 1 Financial Data and Stylized Facts 2 i.i.d. Models 3 Mean Models 4 Covariance Models - Volatility Clustering 5 Fitting of Models - Estimation or Calibration 6 Summary

63 Summary The returns can be expressed as r t = µ t + w t where µ t = E[r t F t 1 ] is the conditional mean on the history F t 1 and w t is the residual with conditional covariance matrix Σ t = E[(r t µ t )(r t µ t ) T F t 1 ]. We have overviewed many models for the conditional mean: i.i.d. model, factor model, VAR models, VMA models, VARMA models, VECM, etc. We have overviewed the two basic models for the univariate conditional volatility that attemps to model the volatility clustering: ARCH and GARCH. The volatility clustering models can be extended to the multivariate case: VEC, DVEC, BEKK, CCC, DCC. The estimation of these models can be simple in some cases but also very difficult in other cases like the volatility clustering models. Many packages available in R for the fitting of these models. D. Palomar Time Series Modeling 63 / 64

64 Thanks For more information visit: