ABSTRACT. Robust GARCH methods and analysis of partial least squares regression. Joseph Egbulefu

Transcription

1

2 ABSTRACT Robust GARCH methods and analysis of partial least squares regression by Joseph Egbulefu New approaches to modeling volatility are evaluated and properties of partial least squares (PLS) regression are investigated. Common methods for modeling volatility, the standard deviation of price changes over a period, that account for the heavy tails of asset returns rely on maximum likelihood estimation using a heavy-tailed distribution. A fractional power GARCH model is developed for robust volatility modeling of heavy tailed returns using a fractional power transform and Gaussian quasi maximum likelihood estimation. Furthermore, a smooth periodic GARCH model, incorporating seasonal trends by wavelet analysis, is developed and shown to outperform existing approaches in long-horizon volatility forecasting. PLS is a latent variable method for regression with correlated predictors. Previous approaches to derive the asymptotic covariance of PLS regression coefficients rely on restrictive assumptions. The asymptotic covariance of PLS coefficients are derived under general conditions. PLS regression is applied to variable selection in the context of index tracking.

3 iii Acknowledgments I wish to thank my advisor Dennis Cox, and my committee members, Katherine Ensor, and Mahmoud El-Gamal for their support, time, and advice. I also which to thank the Rice Department of Statistics and the National Science Foundation for supporting this research.

4 Contents Abstract List of Illustrations List of Tables ii vii xi 1 Introduction 1 2 Fractional power GARCH for heavy tailed heteroskedastic time series Introduction Background Robust GARCH estimators Fractional power GARCH(p,q) model Parameter estimation Simulation Simulation results Data analysis Conclusion Appendix Smooth periodic GARCH model for long-horizon volatil-

5 ity Introduction Current approaches to seasonal volatility Smooth time varying models and almost periodicity The DWT and the maximal overlap DWT The discrete wavelet transform Maximal overlap DWT Wavelet smoothing Smooth periodic GARCH model Model fitting and parameter estimation Parameter estimation Data analysis Results Conclusion Asymptotic variance of PLS Regression coefficients Introduction Statistical framework and PLS Asymptotic distribution of β Monte-Carlo simulation Computational complexity Conclusion Index tracking with PLS regression Introduction PLS Regression Index tracking Selection of constituents Elastic Net v

6 vi Forward stepwise regression Weight estimation Estimation of dynamic weights Data analysis and results Results Conclusion Conclusion Multivariate FP-GARCH

7 Illustrations 2.1 The power transformation function for various values of α Autocorrelation function of squared returns for S&P 500, Crude oil futures, and Euro spot Fitted and simulated ARCH(2) - 1 st row, and GARCH(1,1) - 2 nd row - innovations used to conduct bootstrap hypothesis tests. Solid lines are the simulated densities while dotted lines represent the densities of the fitted innovation series SPX, Crude oil, and Euro spot, daily log returns from January 2008 through November 2011: the data used for parameter estimation and fitting SPX, Crude oil, and Euro spot, daily log returns from November 15, 2011 through May 2013: data used for out of sample forecasts In-sample fitted SPX daily volatility for ARCH(2, 0) and GARCH(1, 1) models In-sample fitted Crude futures daily volatility for ARCH(2, 0) and GARCH(1, 1) models In-sample fitted Euro daily volatility for ARCH(2, 0) and GARCH(1, 1) models Out-of-sample forecasted conditional variance for daily S&P

8 viii 2.10 Out-of-sample forecasted conditional variance for daily Crude oil futures Out-of-sample forecasted conditional variance for daily Euro spot The Coiflet 18 wavelet h and wavelet smoothing coefficients g Mulit-resolution analysis of log of squared log-returns for corn futures. The wavelet details for level i are labeled as di while the level 4 smooth s4 is shown in the last row Plot of wavelet coefficients, the w i s for level i = 1,..., 8 are displayed. The estimated standard deviation of the log squared return series is shown as the horizontal (red) line Fitted conditional variance of log returns for select DJIA index member stocks Fitted conditional variance of log returns for select commodity futures The first plot shows the absolute percentage error between the standard deviation and average of the h-step ahead forecasts for the 6 month forecast period. The second plot displays the standard deviation of forecast innovations. Note that the SP-GARCH forecast innovations have a median value close to one with less spread than the GARCH forecasts Forecasted conditional variance for select DJIA index constituents over the 6 month forecast horizon

9 ix 4.1 QQNorm and density plots of the iterates of ɛ s = T Σ 1 2 β,s ( β s β s ), which are the black dots, against standard normal iterates, which are gray. The data are generated from the normal distribution. The first two columns plot the iterates for normal distributed innovations for sample sizes T = 300, with k, the number of PLS factors, fixed at 2, when the number of predictors, p = 8 and p = QQNorm and density plots of the iterates of ɛ s = T Σ 1 2 β,s ( β s β s ), which are the black dots, against standard normal iterates, which are gray. The data are generated from the normal distribution. The first two columns plots the iterates for normal distributed innovations for sample sizes T = 1000, with k fixed at 2, when p = 8 and p = QQNorm and density plots of the approximately iterates of ɛ s = T Σ 1 2 β,s ( β s β s ), which are the black dots, against standard normal iterates, which are gray; the innovations are from the heavy tailed t distribution with 5 degrees of freedom. The first two columns plot the iterates for sample sizes T = 300, with k fixed at 2, when p = 8 and p = QQNorm and density plots of the approximately standard normal iterates of ɛ s = T Σ 1 2 β,s ( β s β s ); the innovations are from the heavy tailed t distribution with 5 degrees of freedom. The first two columns plot the iterates for sample sizes T = 1000, with k fixed at 2, when p = 8 and p = HC Cor and HC PCor Gap statistics (5.26) for cluster size r = 1,..., 30; one observes HC PCor to have higher separation, for each r number of clusters, between expected average dissimilarity (under uniform clustering) and observed average similarity

10 x 5.2 Groups resulting from clustering by HC Cor and HC PCor of S&P 500 constituents. For HC Cor the optimal number of clusters is 1. For HC PCor, the Gap statistic yielded 2, 5, and 10 as candidate for number of groups

11 Tables 2.1 Simulation results comparing model fit and one-step volatility forecast performance of FP-GARCH(1, 1), NL-GARCH(1, 1) and GARCH(1, 1). The simulated process y t is the standard GARCH when α = 1 ν =, where ν is the degrees of freedom of the Student t distribution. The simulated process is the heavy tailed Fractional power GARCH process when α < 1 and ν = Summary statistics of squared mean centered log returns. p-val DB is the p-value of Ljung-Box test for significance of the first 10 lags of autocorrelation of squared log returns S&P 500, Crude futures, and Euro estimated ARCH(2) and GARCH(1, 1) parameters in addition to statistics on fitted predicted innovations for each of the GARCH methods. σ ε 2 is the variance of in-sample fitted innovations. γ ε (1) is the lag-1 autocorrelation of fitted innovations Results of bootstrap hypothesis test to evaluate wether GARCH or FP-GARCH is more suitable for modeling conditional variance of the observed squared log returns

12 xii 2.5 Results of out of sample forecasts of daily conditional variance for 360 days sing parameters estimated from sample consisting of first 1,010 observations. Conditional variance forecasts were computed from past returns and prior conditional variance forecasts Results from model fitting of GARCH(1,1), P-GARCH(1,1), and SP-GARCH(1,1) models on DJIA constituents Results from model fitting GARCH(1,1), P-GARCH(1,1), and SP-GARCH(1,1) models to commodity futures Corn, Soybeans, Wheat, Heating Oil, Natural gas, and Crude oil Six month GARCH(1,1), P-GARCH(1,1), and SP-GARCH(1,1) volatility forecasts for DJIA constituents. σ y is the standard deviation of daily log returns over six month forecast horizon. The absolute error for each method measure the deviations of the average h step volatility forecast Six month GARCH(1,1), P-GARCH(1,1), and SP-GARCH(1,1) volatility forecasts for select commodities Worst case computational complexity of components of the Jacobian of V ar( β) from (4.46). p is the number of predictors, which, in our formulation, is also the number of columns of the X data matrix, r = p(p + 1)/ Possible number of clusters for each partial correlation matrix. The ideal number of clusters, r, as determined by (5.27) is given in the first column. Note that s = s r /n Model fitting results for HC PCor for each particular partial correlation matrix

13 xiii constituents selected using forward stepwise regression (FSR), Elastic Net (E Net), Hierarchical clustering with correlation (HC Cor), and Hierarchical clustering with partial correlation (HC PCor). The Weight column is the S&P 500 percentage weight as of the end of the in-sample period. Industry group classification is Model fitting summary results. Tracking portfolio based on HC PCor selection resulted in lowest in-sample tracking error and RMSE with the index Out of sample results: Tracking portfolio based on FSR and HC Cor selections outperformed HC PCor and Elastic Net based tracking portfolios

14 1 1. Introduction In the following chapters, two extensions of the generalized autoregressive conditional heteroskedasticity (GARCH) framework for modeling conditional variance / volatility are developed, the asymptotic covariance of the partial least squares (PLS) regression coefficients is derived, and the PLS method is applied to the index tracking problem. Modeling and forecasting volatility, the standard deviation of price changes over a period, is important for pricing and risk management. For options and other financial contracts, the expected volatility of an asset is a key variable for pricing. In quantifying risk on a portfolio, the volatility of individual positions and their correlations are relevant. Furthermore, in asset allocation, the variance/covariance of assets under consideration features prominently in identification of optimal weights for a minimum variance portfolio, for example. When modeling volatility it is important to take into consideration, the properties of heavy tailedness and conditional heteroskedasticity, which have been observed in asset returns [3], [6]. Conditional heteroskedasticity is the property of time varying conditional variance, which stems from the observation that asset returns experience periods of high and low variability. The heavy-tailed property is the observation that extreme returns appear more frequently than would be expected under a Gaussian

15 2 distribution. Technically, a density f(y) is heavy tailed if y has polynomial rate of decay lim y f(y) = 1 (1.1) y κ+1 equivalently notated f(y) y κ+1, for κ > 0. The tail index κ determines the degree of haevy tailedness of the distribution. It has been observed that tail index of financial returns typically range between two and five [3]. Inference and forecasting from heavy tailed distribution present challenges. Since, in general, moments of y E(y λ ) for λ > κ do not exist, asymptotically normal convergence of volatility parameter estimates are not guaranteed when κ < 4. A second problem with heavy tailed returns concerns the influence of extreme returns in estimating parameters of a volatility model. The class of generalized autoregressive conditional heterosckedactic (GARCH) models, [1], generalizes the the ARCH models of Engle [4] by incorporating an autoregressive model of order p for the variance σt 2. The GARCH(p, q) process for y t is notated: y t = σ t ε t, ε t N(0, 1) (1.2) q p σt 2 = ψ 0 + ψ i yt i 2 + β i σt i 2 (1.3) i=1 assumes the return y t to be conditionally normally distributed, where ψ 0 > 0, ψ i 0, β i 0, and ψ q and β p non zero. Assuming y t has finite second moments, then {y t } t=1 is a stationary sequence with E(y t ) = 0 and variance: V (y t ) = ψ 0 /(1 q i=1 ψ i p i=1 β i) if q p ψ i + β i < 1 i=0 i=1 Under assumption that ε t N(0, 1), the conditional distribution of y t is N(0, σ 2 t ). In practice, however, the distribution of ε t is unknown. Other researchers have considered GARCH modeling under assumption of heavy tailed distributions; refer to [8] i=1

16 3 for example. In Chapter 2 we consider a fractional power GARCH model for heavy tailed univariate returns that applies a fractional power transformation on the return series {y t } t=1,2,... such that influence of extreme returns on the conditional variance σt 2 is lessened relative to influence of near median returns. Our results show the fractional power GARCH method to yield more accurate forecast of daily short horizon volatility, volatility forecasts of one to two days ahead when compared to the standard GARCH and other robust GARCH methods. In 6.1 of the conclusion, I provide a multivariate GARCH extension to the univariate fractional power GARCH model. In many volatility forecast scenarios such as in the pricing of financial derivatives; however, it is the long horizon volatility, the volatility of an asset over a period spanning several months or years in the future, that is desired. It can be shown, see [5] that the GARCH(1,1) forecast converges to the long-run unconditional variance rate σ 2 = ψ 0 /(1 ψ 1 β 1 ) at a rate determined by (ψ 1 + β 1 ). Asset returns alternate between long periods of high and low volatility [2]. This heteroskedasticity can be attributed in part to market participants reaction to reoccurring events and seasonal variations in the business cycle. GARCH(1,1) long horizon volatility forecasts, which converge to the mean long run variance, do not account for the expected seasonal trends over the future period. In Chapter 3 we use the discrete wavelet transform to develop a novel smooth periodic GARCH(1,1) model for long horizon volatility forecasts. Our model is shown to outperform the standard GARCH and periodic GARCH(1,1) model of [2] in 6 month volatility forecasts. In portfolio selection, asset allocation, covariance estimation, and related problems requiring multivariate methods, one inevitably encounters the problem of collinearity of asset prices and/or returns. When multicollinearity exists for two or more variables then an approximate linear relationship can be found that relates the variables. When multicollinearity exists, there is a moderate to strong correlation between the variables, which engen-

17 4 ders problems in application of certain techniques and interpretability of results. For example, the covariance matrix of a set of collinear variables may be singular. The factor model assumes an unobserved set of variables explain the variation in the observed, presumable larger set of variables. Factor analysis methods aim to identify the set of factors under varying assumptions on the factors f t, factor loadings Λ, and errors ε t. The factor model for a set of variables x t takes the form x t = Λf t + ɛ t (1.4) Some classical assumptions are that the errors ɛ t are independent and identically distributed (i.i.d) with E[ɛ t ] = 0 and the factors are uncorrelated. However, the application setting may warrant the relaxing of to allow for serial and cross sectional correlation in {ε t }, for example. Refer to [15] for additional assumptions. Since the factors f t are not observed as data, several multivariate analysis methods have been used to determine variables f t, which we refer to as empirical factors, from observed data to approximate the unobserved factors. By far, the most popular is the method of principal components PC, where {f t } T t=1 and loadings matrix Λ are chosen to explain the variance of {x t } T t=1 by selecting Λ as the principal k eigenvectors of the sample covariance matrix T 1 T t=1 x tx t (assuming the x variables have mean zero) or the sample correlation matrix. The corresponding k factors for observation t are then determined by f t = Λ x t. PC factors are limited in their ability to predict a response variable [11]. When a large set of predictors x t is employed such that individual predictors x ti, i = 1,..., p have varying correlations with a response y, the dominant predictive factors are not neces-

18 5 sarily the dominant explanatory factors, i.e those factors constructed by maximizing the variation explained in the data observed on the p variables. This consideration has motivated the use of Partial Least Squares (PLS) to construct the empirical factors. PLS is an iterative method, attributed to Herman Wold [16], that identifies common factors as linear combinations of possibly correlated predictor variables that maximize the covariance between the predictor and response variable(s). As of the early nineties, statistical analysis of the PLS method in regression was being investigated, [7, 9]. In Chapter 4 I address some short comings of prior work [10, 12, 13, 14] on the asymptotic covariance matrix of PLS regression coefficients by developing the full asymptotic covariance matrix of PLS regression coefficients using the Central Limit Theorem and the Delta method and show our asymptotic covariance calculation to be computationally more efficient when the number of predictor variables p is larger than the sample size. Unlike PC PLS is able to identify k p empirical factors f t that maximize the covariance with a response variable y t. In Chapter 5 I consider tracking an index with replicating portfolios constructed from four selection methods. I evaluate a method that selects constituents for a tracking portfolio using hierarchical clustering with the PLS partial correlation matrix to selection from clusters based on the correlation matrix, the forward stepwise regression selection, and the elastic net selection method. I find that the tracking portfolio based on the PLS partial correlation matrix yield the best in-sample performance.

19 6 Bibliography [1] T. Bollerslev. Generalized autoregressive conditional heteroskedasticity. Journal of econometrics, 31(3): , [2] Tim Bollerslev and Eric Ghysels. Periodic autoregressive conditional heteroscedasticity. Journal of Business & Economic Statistics, 14(2): , [3] Rama Cont. Empirical properties of asset returns: stylized facts and statistical issues [4] R.F. Engle. Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation. Econometrica: Journal of the Econometric Society, pages , [5] Robert F Engle and Andrew J Patton. What good is a volatility model. Quantitative finance, 1(2): , [6] Eugene F Fama. Mandelbrot and the stable paretian hypothesis. The journal of business, 36(4): , [7] LLdiko E Frank and Jerome H Friedman. A statistical view of some chemometrics regression tools. Technometrics, 35(2): , 1993.

20 7 [8] Peter Hall and Qiwei Yao. Inference in arch and garch models with heavy tailed errors. Econometrica, 71(1): , [9] Inge S Helland. Partial least squares regression and statistical models. Scandinavian Journal of Statistics, pages , [10] Weilu Lin and Elaine Martin. Prediction intervals of an alternative formulation of partial least squares algorithm. In Advanced Control of Industrial Processes (ADCONIP), 2011 International Symposium on, pages IEEE, [11] Kantilal Varichand Mardia, John T Kent, and John M Bibby. Multivariate analysis [12] A Phatak, PM Reilly, and A Penlidis. An approach to interval estimation in partial least squares regression. Analytica chimica acta, 277(2): , [13] A Phatak, PM Reilly, and A Penlidis. The asymptotic variance of the univariate pls estimator. Linear Algebra and its Applications, 354(1): , [14] Rosario Romera. Prediction intervals in partial least squares regression via a new local linearization approach. Chemometrics and Intelligent Laboratory Systems, 103(2): , [15] James H Stock and Mark W Watson. Forecasting using principal components from a large number of predictors. Journal of the American statistical association, 97(460): , [16] Herman Wold. Soft modeling by latent variables: The nonlinear iterative partial least squares approach. Perspectives in probability and statistics, papers in honour of MS Bartlett, pages , 1975.

21 8 2. Fractional power GARCH for heavy tailed heteroskedastic time series 2.1 Introduction In this chapter we introduce the fractional power transformed GARCH (FP-GARCH) model for minimizing the influence of extremes on conditional variance estimation. The conditional variance of the standard GARCH is an ARMA model with past scaled squared innovations as moving average (MA) or ARCH terms and past conditional variances as autoregressive (AR) terms. Large innovations can lead to poor conditional variance forecasts by receiving relatively more weight in parameter estimation. Furthermore, influence of extreme innovations are magnified in the presence of volatility clustering when extremes returns are followed by returns of similar magnitude over time. If y t follows the standard GARCH(p, q) [4] then y t = σ t ε t (2.1) σt 2 = q p ψ 0 + ψ i yt i 2 + β i σt i 2 (2.2) i=1 i=1

22 9 where ψ 0 > 0 and ψ i, β i 0, but ψ q and β p non-zero. Additionally, ε t IID(0, 1) is independent of {y t i } i 1. From Theorem 1 of [4], the GARCH(p,q) process to be stationary if q i=1 ψ i + p i=1 β i < 1. From stationarity of y t, V ar(y t ) = ψ 0 1 q i=1 ψ i p i=1 β i if and only if q ψ i + i=1 p β i < 1 i=1 Let Θ = R p+q+1 such that for all θ = (ψ 0, ψ 1,..., ψ q, β 1,..., β p ) Θ, ψ 0 0, and ψ i > 0, i = 1,..., q, and β i > 0, i = 1,..., p. Estimation of θ is usually carried out under quasi maximum likelihood estimation, where y t, conditional on observing its previous q realizations and p conditional variances, is assumed to be Gaussian with mean zero and variance σt 2. The Gaussian quasi-maximum likelihood estimate θ can be obtained as the θ minimizing the negative log-likelihood, l(θ). For the initial values {σ 2 t, y t } for t = max(p, q) + 1,..., 0, σ 2 t can be initialized to σ 2 t = y 2 t. Then given observations y 1,..., y T, L(θ) = C 1 2 l(θ) = 1 T T i=1 T log(σi 2 (θ)) 1 2 t=1 ( log(σt 2 (θ)) + y2 t σt 2 (θ) T t=1 ) y 2 t σ 2 t (θ) (2.3) θ = argmin(l(θ)) (2.4) θ If the distribution of innovations {ε t } is heavy tailed but with finite fourth order moments, then under some regularity conditions if θ lies in the interior of Θ then θ is asymptotically normal [16]. The fractional power transformed GARCH(p, q) (which we refer as FP-GARCH(p, q))

23 10 is a generalization of the standard GARCH(p, q) model that transforms the predictors in the ARCH terms of the conditional variance equation (2.2). q σt 2 = ψ 0 + ψ i g(y t i ) + g(z) = ( z 2) α i=1 p β i σt i 2 (2.5) i=1 (2.6) where 0 < α Background Various generalizations of the GARCH(p,q) model have been introduced. The nonlinear GARCH (NL-GARCH) model of [14]: σ 2 t = ( ψ 0 (σ 2 ) δ + q ψ i (yt i) 2 δ + i=1 p i=1 β i (σ 2 t i) δ ) 1 δ (2.7) encompasses the standard GARCH(p, q) model when δ = 1, and the absolute value GARCH model σ t = ψ 0 + q ψ 1 y t i + i=1 p σ t i (2.8) i=1 when δ = 1. As δ 0 (2.8) yields the log-garch(p, q) process, 2 log(σ 2 t ) = ψ 0 + q ψ 1 log(yt i) 2 + i=1 p β i log(σt i) 2 (2.9) i=1 of which parameters can more easily be estimated since it avoids the non-negativity restrictions on θ. This relationship between the NL-GARCH and log-garch can be

24 11 seen by applying the variance stabilizing Box-Cox transformation of [6]: f(z) = z (δ) f(σ 2 t ) = ψ 0 + q f(yt i) 2 + i=1 p i=1 f(σ 2 t i) where (z α 1)/α if α 0; z (δ) = log(z) if α = 0. (2.10) for z > 0. The FP-GARCH(p, q) conditional variance (2.5) differs from (2.7) by transforming only the lagged observed variable; therefore, the conditional variance equation of (2.5) is linear unlike (2.7). The NL-GARCH model subsequent inverse transformation is required to obtain conditional variance for (2.7). The GARCH processes discussed thus far are capable of modeling the volatility clustering and heavy tailed properties of asset returns. However asset returns, on equities, for example, have been characterized by the leverage effect: conditional variance of an asset being negatively correlated with changes in price [8]. [9] suggested the asymmetric power ARCH (APARCH) model: σ δ t = ψ 0 + q ψ i ( y t i γ i y t i ) δ + i=1 p β j σt i δ (2.11) i=1 where δ 0, ψ i > 0 and β i > 0 and 1 γ i 1. When γ i = 0 then σ t is symmetric with respect to observation y t i : a positive y t i has the same effect on conditional variance as a negative y t i, and when γ i > 0, a positive y t i results in a relative decrease in conditional variance σ 2 t whereas a negative y t i increases σ 2 t. The standard GARCH with power parameter δ = 2 assumes the best model for observed data is linear in second moments. The APARCH model is more flexible as it estimates the parameter δ 0 to maximize the lagged correlation cor(σ δ t, y δ t i), i = 1,... q,

25 12 potentially yielding a better fit; however, when δ 2 the APARCH requires subsequent transformation to obtain the conditional variance. The FP-GARCH model, in contrast, is linear in lagged conditional variance and non-linear in y 2 t i, when α < Robust GARCH estimators Many of the models discussed thus far also have beneficial robustness properties, which are now explored. By robustness of GARCH estimators, we mean robustness to extremes: large magnitude innovations, which may arise from a heavy-tailed innovation distribution. As the conditional variance equation of the standard GARCH(p, q) model (2.2) is linear in yt i, 2 large innovations may lead to large observations which, when squared, influence the conditional variance. Additionally, an extreme innovation ε t is problematic because it results in large residuals which can adversely affect parameter estimation. From (2.3) one obtains the fitted innovation: ε 2 t (θ) = y2 t σ 2 t (θ) (2.12) where ε 2 t (θ) is the squared residual. When estimating θ by a method such as the quasi-maximum likelihood, the residuals should have equal weight in the optimization criterion. Expressing (2.3) in terms of ε 2 t (θ), l(θ) = 1 T T [ log(σ 2 t (θ)) + ρ ( log ( ε 2 t (θ) ))] i=1 where in this case ρ( ) = exp( ). GARCH M-estimators bound the influence of fitted innovations ε 2 t (θ) by selecting ρ, and/or its derivative ρ, to be a bounded function. Bounding ε 2 t (θ) decreases its influence on the parameter estimate ψ i, the estimate of the ARCH coefficient for yt i. 2 The MLE of ψ i θ is found at the point where

26 13 l(θ) θ = 0. At the MLE, ρ ψ i = T ( ) ρ ( ε 2 t (θ)) y σt 2 t i 2 = 0. (2.13) (θ) t=1 Therefore a bounded ρ caps the influence of large residuals on the estimate ψ i, i = 1,..., q. However, from (2.13) one can observe that even with ρ bounded, a large y 2 t i can act as a leverage point that shifts the resulting ψ i away from the optimum that would have been attained were y 2 t i is not to be extreme. [16] suggests the bounded M (BM) estimators with influence function of the form ρ(w) = m(log(f(e w/2 )e w/2 )) where f( ) is the density of ε t and m is a smooth version of the function m c (u) = u if u c; c if u > c. They address potential leverage points by censoring y 2 t i to q η : the 1 η quantile of y 2 t by using the conditional variance equation: σ 2 t = ψ 0 + q i=1 ( ) y ψ i σt im 2 2 t i qη + σt i 2 p β i σt i 2 i=1 so that large y 2 t i are thresholded to the 1 η quantile. [17] recommend the least absolute deviation GARCH (LAD-GARCH) estimator: θ = argmin θ T logyt 2 log{σ t (θ) 2 } (2.14) t=1 which uses ρ = the absolute value function which, though unbounded, the resulting θ is more robust to outliers in the conditional variance equation than the QMLE. Recent research in GARCH methods has focused on estimation under heavy tails

27 14 and non-stationarity. The standard GARCH model assumption of constant unconditional variance and fixed parameters is untenable for financial returns over long periods that may include shifts in regime due to change in economic activity. One of the earlier time-varying parametric GARCH models is the Markov switching model of [13]. Markov switching GARCH models assume a fixed set of states; each state has a set parameters, and transitions between states are governed by a Markov process. The Markov switching GARCH model has seen recent improvements in estimation [12], [3], [11]. Newer approaches include smooth transition models with time varying parameters. Stationary smooth transition models [18] assume a long run and short run volatility with the short-run fluctuating about the long-run level. Non-stationary smooth transition models specify time varying unconditional variance specified by a smooth transition or step function [1, 10]. 2.3 Fractional power GARCH(p,q) model My contribution is to introduce the FP-GARCH(p, q) process for mitigating the influence of extremes on estimated parameters under a heavy tailed process. y t = σ t ε t, ε t IID(0, 1) (2.15) q p σt 2 = ψ 0 + ψ i (yt i) 2 α + β i σt i 2 (2.16) i=1 i=1 where 0 < α 1, ψ 0 0, ψ i, β j 0 for i = 1,..., q 1, j = 1,..., p 1, and ψ q > 0, β p > 0. It is a generalization of the GARCH(p, q) when the power parameter α 1. When α = 0 then the conditional variance σ 2 t is independent of lagged observations y t i, i = 1,..., q, and (2.16) resolves to an autoregressive model for σ 2 t. Though the technical conditions for y t to be stationary when α < 1 have not been shown, one can conjecture y t with a fractional α to be stationary since g(z) = (z 2 ) α effects a slower

28 15 varying conditional variance function relative to the GARCH(p, q) process with the same coefficients. We observe from Figure 2.1 that g(y) is smooth everywhere except Figure 2.1 : The power transformation function for various values of α. FP transformation 2.0 g(y) variable alpha = 1 alpha = 0.7 alpha = 0.5 alpha = 0.3 alpha = y at zero. When α < 1/2, g(y) stretches y for y < 1 while shrinking y if y > 1. Thus, depending on α the FP-GARCH exhibits varying degrees of robustness to leverage points that appear as lagged values in the conditional volatility equation. 2.4 Parameter estimation The goal is to estimate θ = (ψ, β, α) for the FP-GARCH(p, q) model under assumption that y t = σ t ε t with σ t and ε t unobserved and {ε t } drawn from a symmetric but unknown, possibly heavy tailed distribution. Further, assume θ lies in the interior of the parameter space Θ R = R + 0 R q+p + (0, 1].

29 16 Under these assumptions, the QML method is suitable for finding θ; however, optimality of the QMLE θ needs to be checked against corner solutions: θ such that α = 1. Let σt 2 (θ) be defined by (2.16) and g(z) be as defined by (2.6). The problem of finding θ = (ψ, β, α) can be solved by minimizing the negative log-likelihood (2.3), subject to constraints on θ. θ = argmin θ subject to 1 T T t=1 ( ) log(σt 2 (θ)) + y2 t σt 2 (θ) (2.17) ψ i 0 i = 0,... q (2.18) β i 0 i = 1,..., p (2.19) q p 1 + ψ i + β i < 0 (2.20) i=1 i=1 α < 0 (2.21) α 1 0 (2.22) The objective function l(θ) is convex in θ Θ. To see this let u = y 2 t and first consider f(α) = (u) α for α (0, 1]. Then convexity f holds with respect to α since the second derivative is positive everywhere. 2 f α 2 = (u)α log 2 (u) > 0 (2.23) Since σ 2 t (θ) is linear combination of remaining parameters ψ and β, σ 2 t (θ) is a convex functions of θ. Next for u = σ 2 t, f(u) = log(u) + c u, d 2 u (2c u) = > 0 du2 u 3

30 17 the second derivative implies f(u) to be convex for 0 < u < 2c, therefore if u t = σ 2 t and c t = y 2 t then the objective (2.17) is convex when 1 T T σt 2 < 2 1 T t=1 T t=1 y 2 t which holds since σ 2 t = E(y 2 t ). With (2.17) being twice continuously differentiable and convex in θ, one can focus on the constraints (3.68) - (6.22) and observe they are convex in θ and twice continuously differentiable. An approach to solving inequality constrained convex optimization problems like (2.17) is by the interior point method, an optimization technique that solves inequality constrained problems by using Newton s method to solve a sequence of equalityconstrained problems such that solution of the ith equality-constrained problem is used as input to equality-constrained problem i+1; the sequence of solutions converges to the solution of the inequality constrained problem. The logarithmic barrier interior point method [7] first converts (2.17) with the s = q + p + 3 inequality constraints into the unconstrained problem θ = argmin I (x) = θ f 0 (θ) + 0 if x 0; if x > 0. s I (f i (θ)) (2.24) i=1 where f 0 (θ) is the objective (2.17), f i (θ) i = 1,..., s are the left side of the inequality constraints. For a particular θ, if f i (θ) > 0, constraint i is violated and infinite cost is incurred for θ in the minimization. However, the modified objective function (2.24) is not differentiable because of the indicator functions. The indicator function is

31 18 approximated by the continuously differentiable function Î (x, w) = (1/w)log( x). Î (x, w) is such that as x approaches zero, Î (x, w) approaches infinity and Î (x, w) is defined to be infinity when x > 0. The variable w determines the rate of convergence of Î (x, w) to I (x) with lim w + Î (x, w) = I (x). Then for some w, the approximate problem to (2.17) is θ(w) = argmin θ wf 0 (θ) + s log( f i (θ)) (2.25) i=1 Since (2.25) is equivalent to an equality constrained problem, it can be solved by Newton s method. Note the minimizer θ(w) of wf 0 (θ) also minimizes f 0 (θ). The minimum of Lagrangian function L(θ(w), λ(w)) = f 0 (θ(w)) + s λ i (w)f i (θ(w)) i=1 is a lower bound for f 0 (θ(w)). This minimum is attained at (θ (w), λ (w)). For some w > 0, let θ(w) be the minimizer of (2.25), and f 0 (θ) the gradient of f 0 at θ. Since, at θ (w), w f 0 (θ (w)) + m 1 f i (θ (w)) f i(θ (w)) = 0 i=1 therefore at λ i = 1 wf i (θ (w)), i = 1,..., s, and θ (w) the Lagrangian is minimized L(θ (w), λ (w)) = f 0 (θ (w)) s/w Therefore for any w > 0 the optimality of θ(w) for the original problem (2.17) can be evaluated by the gap f 0 (θ(w)) f 0 (θ ) s/w; so that the optimum solution is

32 19 approached with increasing w. Newton s method is used sequentially to solve (2.17) by first selecting constants tolerance ξ > 0, w 0 = s/ξ, and λ > 1. On step i 1 θ i is obtained by solving (2.25) with w = λw i 1 and step i + 1 is carried out if s/w ξ, with Newton s method for step i + 1 is initialized with θ i. Alternatively one may stop the sequence when f 0 ( θ i+1 ) f 0 ( θ i ) is less than some tolerance. 2.5 Simulation Monte-Carlo simulation is conducted to evaluate robustness of the fractionally transformed GARCH models under heavy tailed innovations. The following FP-GARCH(1,1) process y t was generated as follows. y t = σ t ε t, ε t t ν σ 2 t = ψ 0 + ψ 1 ( y 2 t ) α + β1 σ 2 t 1 A sample path of length T = 1000 of the innovation {ε t } was generated from the t-distribution: ε t t ν, ν = 3,. For each path ψ and β were generated from Uniform(0.1, 0.8) distribution, such that ψ 0 + ψ 1 + β < 1. For each path, α = {0.4, 0.7, 1}. Initializing y1 2 = σ1 2 = ε 2 1, y t, t, = 2,..., T were obtained following (2.26) and (2.26). The F P GARCH(1, 1) estimates for θ = (ψ, β, α) were obtained by Gaussian QMLE. The FP-GARCH(1, 1) fit and forecast is evaluated against standard GARCH(1, 1) and the non-linear GARCH (NL-GARCH) model of [14] discussed earlier (2.7). The NL-GARCH, because of its generality to encompass many non-linear robust GARCH models, is suitable for evaluating the relative performance of the FP-GARCH model under heavy tails. Parameter estimates for GARCH(1,1) and NL-GARCH(1,1) were also obtained by QMLE. Performance of the

33 20 methods were assessed by root mean squared error (RMSE) of GARCH coefficients and one step ahead conditional variance forecasts. Additionally, variance σ 2 ε and lag-1 autocorrelation γ ε (1) of residuals ε t = yt σ t were also evaluated Simulation results Of the three methods, the standard GARCH, NL-GARCH, and FP-GARCH, all for which θ was estimated by QMLE, the NL-GARCH method yielded the lowest RMSE for one step conditional variance forecasts. For the heavy tailed cases, when ν = 3, the FP-GARCH method yielded RMSE of conditional variance forecast that was 14%, 3%, and 18% lower than the RMSE of corresponding GARCH(1,1) forecasts. Also, the NL-GARCH(1,1) yielded a slight improvement over the FP-GARCH(1,1) in RMSE of estimated parameter α. The standard GARCH(1,1) method yielded the lowest residual variance, σ ε, 2 followed by the FP-GARCH method. Therefore, based on the lower RMSE for conditional volatility forecasts under heavy tailed innovations, one can conclude the FP-GARCH to be a stronger robust conditional variance/volatility forecast method than the standard GARCH method; and based on lower residual variance σ 2 ε of FP-GARCH relative to the NL-GARCH, one can conclude the FP- GARCH provides a better fit to the simulated data than the NL-GARCH.

34 Table 2.1 : Simulation results comparing model fit and one-step volatility forecast performance of FP-GARCH(1, 1), NL-GARCH(1, 1) and GARCH(1, 1). The simulated process y t is the standard GARCH when α = 1 ν =, where ν is the degrees of freedom of the Student t distribution. The simulated process is the heavy tailed Fractional power GARCH process when α < 1 and ν = 3 RMSE Method ν α α σ T +1 σ ε 2 γ ε (1) GARCH NL-GARCH FP-GARCH GARCH NL-GARCH FP-GARCH GARCH NL-GARCH FP-GARCH GARCH NL-GARCH FP-GARCH GARCH NL-GARCH FP-GARCH GARCH NL-GARCH FP-GARCH Data analysis Conditional variance of daily log returns for two market variables: S&P 500 index, the Euro-Dollar exchange rate, and the Crude oil (West Texas Intermediate) futures were forecasted from historic data spanning the period from January 2008 through May 2013, excluding days of market closure; the sample used for parameter estimation spanned the first four years and consisted of 1010 observations on each of the three variables. The remaining data, 360 observations, was used for out-of-sample forecasting. For all methods evaluated, the ARCH parameter q {1, 2} while the autorregressive parameter p = 1. Evaluated models include the standard GARCH(q, p),

35 22 NL-GARCH(q, p), FP-GARCH(q, p), and the LAD-GARCH(p, q). For the standard GARCH, NL-GARCH, and FP-GARCH models, parameters were estimated by Gaussian quasi-maximum likelihood. Sample summary statistics are presented in Table 2.2. The first 10 lags of the autocorrelation function, Figure 2.2, of squared returns for all three variables are statistically significant based on resulting p-values of Ljung-Box test [15]. Figures 2.4 and 2.5 of the Appendix display the sample data for parameter estimation and out of sample forecasting, respectively. Table 2.2 : Summary statistics of squared mean centered log returns. p-val DB is the p-value of Ljung-Box test for significance of the first 10 lags of autocorrelation of squared log returns. Mean Median Max Std. Dev Skew Kurt p-val DB S&P Crude Euro Figure 2.2 : Autocorrelation function of squared returns for S&P 500, Crude oil futures, and Euro spot. value variable SP500 Crude Euro Lag Parameter estimates and innovations resulting from sample fit are presented in Table 2.3. The fitted conditional variances are displayed in figures: 2.6, 2.7, 2.8 in the Appendix. Overall, variance of fitted innovations was lowest for the FP-ARCH and FP-GARCH fits. One can observe the effect of the nonlinear transformation (y 2 t i) α on resulting coefficients. For both ARCH(2) and GARCH(1,1) estimates, NL- GARCH and FP-GARCH coefficients for transformed variables, have less magnitude

36 23 than LAD-GARCH and standard GARCH estimates. The mean conditional variance ψ 0 tends to be higher for the ARCH(2) than GARCH(1, 1) as well. With the exception of Crude oil, fitted FP-ARCH(2) and FP-GARCH(1,1) result in the same estimates for power parameters α. Considering α < 1 as the degree of non linearity between conditional variance and squared returns, one can reason SPX and Crude oil conditional variance to exhibit such nonlinear relationship with squared return.

37 Table 2.3 : S&P 500, Crude futures, and Euro estimated ARCH(2) and GARCH(1, 1) parameters in addition to statistics on fitted predicted innovations for each of the GARCH methods. σ ε 2 is the variance of in-sample fitted innovations. γ ε (1) is the lag-1 autocorrelation of fitted innovations. S&P 500 ψ 0 ψ1 ψ2 α σ ε 2 γ ε (1) ARCH(2) NA LAD-ARCH(2) NA NL-ARCH(2) FP-ARCH(2) ψ 0 ψ1 β1 α σ ε 2 γ ε (1) GARCH(1, 1) NA LAD-GARCH(1, 1) NA NL-GARCH(1, 1) FP-GARCH(1, 1) Crude oil futures ψ 0 ψ1 ψ2 α σ ε 2 γ ε (1) ARCH(2) NA LAD-ARCH(2) NA NL-ARCH(2) FP-ARCH(2) ψ 0 ψ1 β1 α σ ε 2 γ ε (1) GARCH(1, 1) NA LAD-GARCH(1, 1) NA NL-GARCH(1, 1) FP-GARCH(1, 1) Euro spot ψ 0 ψ1 ψ2 α σ ε 2 γ ε (1) ARCH(2) NA LAD-ARCH(2) NA NL-ARCH(2) FP-ARCH(2) ψ 0 ψ1 β1 α σ ε 2 γ ε (1) GARCH(1, 1) NA LAD-GARCH(1, 1) NA NL-GARCH(1, 1) FP-GARCH(1, 1) This conclusion was formally tested by carrying out a bootstrap likelihood ratio

38 25 test (LRT) to evaluate the appropriateness of the best FP-GARCH model relative to the the standard ARCH(2) and GARCH(1,1) models (with α = 1). The LRT tests the fitness of the Null model to an alternative. Assume θ 0 is the set of parameters under the Null, θ 0 Θ 0, and θ 1 is the set of parameters under the alternative, θ 1 Θ 1, and Θ 1 = Θ c 0. When the distribution function F (ε t ) is known and regularity conditions such as requiring that θ 0 lie in the interior of Θ hold, then the likelihood ratio test statistic: λ = 2log L( θ 0 ) L( θ 1 ) with θ 0 and θ 1 being the MLEs under the null and alternative models, respectively, has asymptotically Chi-squared distribution. However since F (ε t ) is unknown and θ 0 lies on the boundary of Θ since α 0 = 1, a simulated null distribution is used. For the bootstrap hypothesis test, the null distribution F is the empirical distribution of innovations simulated from the Student-t ν distribution of best fit (based on first two moments) to the fitted innovations ε t = y t σ t, t = max(p, q),..., T Figure 2.3 displays the densities of the fitted and sample simulated innovations for the S&P 500, Crude oil, and Euro. From the ith sample path of innovations, the statistic, λ i, i = 1, is calculated to obtain F λ for the test statistic. The Null hypothesis that the observed {yt 2 } belongs to the standard GARCH(1, 1) model, is rejected if the observed statistics λ is large relative to the distribution F λ. The p-value of such a test is given by 1 F λ ( λ). Table 2.4 below presents the test results. Based on the p-value of the LRT test with Null hypothesis that observed Crude oil returns follow the GARCH(1,1) process, I reject the Null hypothesis.

39 26 Figure 2.3 : Fitted and simulated ARCH(2) - 1 st row, and GARCH(1,1) - 2 nd row - innovations used to conduct bootstrap hypothesis tests. Solid lines are the simulated densities while dotted lines represent the densities of the fitted innovation series. S&P500 Crude Euro density density density innovation innovation innovation density density density innovation innovation innovation Table 2.4 : Results of bootstrap hypothesis test to evaluate wether GARCH or FP- GARCH is more suitable for modeling conditional variance of the observed squared log returns. Null model Variable λ p-value SP ARCH(2) Crude Euro SP GARCH(1,1) Crude Euro The out-of-sample conditional variance forecasts are displayed in Figures 2.9, 2.10, and 2.11 of the Appendix. The forecasts were evaluated based on variance of predicted innovations ε t, lag-1 autocorrelation of predicted innovations γ ε (1), and the RMSE, RMSE M, resulting from the regression of excess returns on conditional vari-

40 27 ance forecasts. The RMSE M measures the degree the conditional variance forecast explains the observed daily market risk premium, the return less the risk-free rate. An accepted tenet of mean-variance optimization and the capital asset pricing model is that the level of the risk premium is proportional to the non-diversifiable risk, refer to [2], for example. Using the forecasted conditional variance as the measure of expected market risk, a GARCH in mean (GARCH-M) regression model [5] x t,m = µ M + δσ 2 t,m + ɛ t (2.26) can be used to evaluate the volatility forecasts. x t,m is the daily market return above the risk-free rate and σ t,m is the daily market volatility. We assume the better volatility forecast will proved a lower root mean square error, RMSE M, for the GARCH-M model. Daily risk premiums were calculated by subtracting each day s annualized yield on the 90-day US Treasury bill from the observed return; while the one step ahead daily conditional variance forecast, calculated for each GARCH method, was used for the conditional variance σt,m 2. Out of sample results are presented in Table 2.5. One can observe that while conditional variance forecasts of the various GARCH methods performed equally well for the RMSE M criterion, the FP-GARCH(1,1) conditional variance forecasts yielded the least variance of predicted innovations.

41 Table 2.5 : Results of out of sample forecasts of daily conditional variance for 360 days sing parameters estimated from sample consisting of first 1,010 observations. Conditional variance forecasts were computed from past returns and prior conditional variance forecasts. S&P 500 Method σ ε 2 γ ε (1) σ M 2 ARCH(2) LAD-ARCH(2) NL-ARCH(2) FP-ARCH(2) GARCH(1, 1) LAD-GARCH(1, 1) NL-GARCH(1, 1) FP-GARCH(1, 1) Crude futures ARCH(2) LAD-ARCH(2) NL-ARCH(2) FP-ARCH(2) GARCH(1, 1) LAD-GARCH(1, 1) NL-GARCH(1, 1) FP-GARCH(1, 1) Euro spot ARCH(2) LAD-ARCH(2) NL-ARCH(2) FP-ARCH(2) GARCH(1, 1) LAD-GARCH(1, 1) NL-GARCH(1, 1) FP-GARCH(1, 1) Conclusion It has been shown that the nonlinear GARCH model that applies the power transform y (y 2 ) α,where 0 < α 1 is chosen based on magnitude of y 2, yields condi-