A New Independence Test for VaR violations

Transcription

1 A New Independence Test for VaR violations P. Araújo Santos and M.I. Fraga Alves Abstract Interval forecasts evaluation can be reduced to examining the unconditional coverage and independence properties of the hit sequence. In this work we propose a definition for tendency to clustering of violations and an exact independence test for the hit sequence. This test is suitable to detect models with a tendency to generate clusters of violations and is based on an exact distribution that does not depend on any unknown parameter. Moreover, we provide evidence through a simulation study that the suggested test performs better than other tests presented in the literature. 1 Introduction We consider a time series of daily log returns, R t+1 = log(v t+1 /V t ), where V t is the value of the portfolio at time t. The corresponding one-day-ahead VaR forecasts made at time t for time t +1, VaR t+1 t (p), is defined by P[R t+1 VaR t+1 t (p) Ω t ] = p, where Ω t is the information set up to time-t and p is the coverage rate. Considering a violation the event that a return on the portfolio is lower than the reported VaR, we define the hit function, also represented by I t, as I t+1 (p) = { 1 if Rt+1 < VaR t+1 t (p) 0 if R t+1 VaR t+1 t (p). (1) P. Araújo Santos Departamento de Informática e Métodos Quantitativos, Escola Superior de Gestão e Tecnologia, Instituto Politécnico de Santarém. CEAUL. paulo.santos@esg.ipsantarem.pt M.I. Fraga Alves Departamento de Estatística e Investigacão Operacional, Faculdade de Ciências, Universidade de Lisboa. CEAUL. isabel.alves@fc.ul.pt 1

2 2 P. Araújo Santos and M.I. Fraga Alves Christoffersen [7] showed that evaluating interval forecasts can be reduced to examining whether the hit sequence,{i t } t=1 T, satisfies the unconditional coverage (UC) and independence (IND) properties. UC hypothesis means P[I t+1 (p) = 1] = p, t. IND hypothesis means that past violations do not hold information about future violations. A problematic non verification of the IND hypothesis is the one that leads to clustering of violations, which corresponds to several large losses occurring in a short period of time. As noted by Campbell [5], comparatively with the UC property, the IND property represents a more subtle yet equally important property. When both properties are valid then we write P[I t+1 (p) = 1 Ω t ] = p, t, and we say that forecasts have a correct conditional coverage (CC). In Lemma 1 of Christoffersen [7] it is shown that condition CC is equivalent to I t+1 (p) iid Bernoulli(p), where iid denotes independent and identically distributed. In a recent paper, Berkowitz et al. [1] extend and unify the existing tests by noting that the de-meaned hits {I t+1 p} form a martingale difference sequence. The hit function and condition CC, imply that E[(I t+1 p) Ω t ] = 0 and then for any variable Z t in the time-t information set, E[(I t+1 p)z t ] = 0. This is the motivation for tests based on the martingale property. There are several backtesting procedures for evaluating intervals forecasts; for a detailed review see Campbell [5] and Berkowitz et al. [1]. The Christoffersen [7] Markov IND and CC likelihood ratio tests (Markov), are perhaps the most widely used in the literature. These tests, based on asymptotic distributions, are only sensible to one violation immediately followed by other, ignoring all other patterns of clustering. If we set Z t = I t k for any k 0, we have E[(I t+1 p)(i t k p)] = 0. Based on this condition Berkowitz et al. [1] suggested the Ljung-Box statistic (LB), for a joint test of whether the first m autocorrelations of (I t+1 p) and (I t+1 k p), k = 1,...,m, are zero. Considering other data in the information set such as past returns, under CC we have E[(I t+1 p)g(i t,i t 1,...,R t,r t 1,...)] = 0 for any nonanticipating function g(.). In the same line as Engle and Manganelli [11], Berkowitz et al. [1] consider the autoregression I t = α + n k=1 β 1k I t k + n k=1 β 2k g(i t k,i t k 1,...,R t k,r t k 1 ) + ε t (2) with n = 1 and g(i t k,i t k 1,...,R t k,r t k 1 ) = VaR t k+1 t k (p). These authors proposed the logit model and test the CC hypothesis with a likelihood ratio test considering for the null hypothesis P(I t = 1) = 1/(1+e α ) = p and the coefficients β 11 and β 21 equal to zero. For the IND hypothesis, the test is adapted considering β 11 and β 21 equal to zero. We refer these tests as the CAViaR tests of Engle and Manganelli (CAViaR). A duration-based approach emerged in the literature (e.g. Danielsson and Morimoto [9], Christofferson and Pelletier [6], Haas [13]). In this set-up, let us define the duration between two consecutive violations as D i := t i t i 1 (3) where t i denotes the day of violation number i. If the IND hypothesis is valid then I t+1 (p) iid Bernoulli(π), with 0 < π < 1, and the common distribution of durations

3 A New Independence Test for VaR violations 3 (3) is geometric with probability mass function (pmf) f D (d;π) = (1 π) (d 1) π,d N,0 < π < 1. (4) The exponential distribution with density function (df) f D (d;β) = β exp( βd), d > 0 and β > 0, (5) is the continuous analogue of the geometric distribution. Based on the exponential, Christoffersen and Pelletier [6] suggested tests using the duration based approach. Haas [13] showed that tests based on discrete distributions for durations, have higher power. The Generalised Method of Moments test framework suggested by Bontecamps [2] to test for distributional assumptions was extended by Candelon et al. [4] to the case of VaR forecasts accuracy. In the group of duration-based tests it is shown that the proposed GMM tests are the best performers. For the CC and IND hypothesis, the Markov tests are perhaps the most widely used in the literature and this is why we have chosen the Markov independence test for the comparative study. In the group of available duration-based tests we chose the best performers GMM tests. We also selected the CAViaR test, the best performer in the comparative simulation study done by Berkowitz et al. [1]. The rest of the paper is organized as follows. In Section 2 we present the new independence test. Finally, in Section 3, we compare its performance with other tests. 2 A new independence test Let D 1:N... D N:N be the order statistics (o.s. s) of durations D 1,...,D N defined in (3). The first motivation behind the class proposed is the following: when violations generated by the hit function (1) occur in clusters, the majority of durations are short (the short durations between violations in the clusters) and some durations are very long (the durations between the last violation of one cluster and the first violation of the following cluster). If the majority of durations are short then the median, D [N/2]:N, is short (notation: [x] denotes the integer part of x). If some durations are very long, the maximum, D N:N, is very long. Finally, with a short median and a very long maximum, the ratio D N:N /D [N/2]:N is large. We illustrate this motivation with an example: we have chosen the returns from the Deutscher Aktien index (DAX) from January 2, 1997 up until December 30, 2008, and we have calculated durations between violations using the popular Historical Simulation (HS) method for VaR(0.05) with a moving window of size 250. Figure 1 shows the geometric pmf, with π = 0.05, and the frequency of durations. For short durations, the frequencies in the frequency plot are much higher than the corresponding probability masses in the geometric pmf. The majority of durations are short, either equal or lower than 6 days and the empirical median is 6, contrasting with the expected value of D 85:170, under IND, which is close to 14. Moreover, for durations above 60 days we note higher frequencies in the frequency plot than the probability masses in the geomet-

4 4 P. Araújo Santos and M.I. Fraga Alves ric pmf. The maximum duration, d 170:170, is 208 days, almost double the expected value under IND, which is close to 112. The ratio is 34.66, much higher than the median of D 170:170 /D 85:170 under IND, which is 8.03 (see the cumulative distribution function (cdf) of Proposition 2.1). In this example, where violations occur in clusters, the majority of durations are short, some durations are very long and, as mentioned before, a high ratio D N:N /D [N/2]:N gives strong evidence against the IND hypothesis. Based on this motivation, we suggest the following definition. Fig. 1 Geometric(π = 0.05) pmf (left) and frequency of durations (right) between violations for DAX index from 2 January 1997 until 30 December Definition 1 (Tendency to clustering of violations). A hit function (1) has a tendency to clustering of violations if the median of D N:N /D [N/2]:N is higher than the median under the IND hypothesis. For explicitly testing the IND hypothesis versus tendency to clustering of violations, we propose the following test statistic R N,[N/2] := D N:N 1 D [N/2]:N. (6) The correction 1 made to D N:N, allows us to obtain a pivotal test. The Proposition 2.2 allows us to do that as well as to present in Proposition 2.3 a level α test. We will denote Y i instead of D i, the durations, when we use the exponential model (5). From now on, we denote ( ) ( ) N [N/2] 1 [N/2] 1 a w = b s =, c w,s = N [N/2] w + s w s γ N = N! ([N/2] 1)!(N [N/2] 1)! and RE N = Y N:N /Y [N/2]:N Proposition 2.1 Let Y 1,...,Y N, be iid exponential random variables (rvs) with common df (5). The cdf of R E N is

5 A New Independence Test for VaR violations 5 1 γ N N [N/2] 1 w=0 with 1 r. [N/2] 1 s=0 ( 1) w+s a w b s ( [cw,s (w + 1)] 1 [c w,s (w c w,s /r] 1), Proof: For the Weibull distribution with density function f X (x; p;θ) = θ p(xp) θ 1 e (px)θ, x > 0, p > 0, θ > 0, Malik and Trudel [14] proved that the density of the ratio of the i-th and j-th o.s. s with i < j N, is f ZN (z; p;θ) = θc j (i 1)!( j i 1)! j i 1 w=0 i 1 s=0 ( 1)w+s( j i 1 ( w i 1 s ) z θ 1 [N j + w ( j i w + s)z θ ] 2, with 0 w 1 and where C j = j v=1 (N v + 1). To obtain the ratio of the i-th and j-th o.s. s, with i < j N, from the (5) model, in (8) we substitute θ by 1. We also replace i and j respectively by [N/2] and N. Calculating the integral, the cdf for the ratio Z N = Y [N/2]:N /Y N:N is γ N N [N/2] 1 w=0 [N/2] 1 s=0 ( 1) w+s a w b s ( [cw,s (w + 1)] 1 [c w,s (w c w,s z)] 1), ) (7) (8) with 0 z 1. For R E N = 1/Z N the cdf is 1 F ZN (1/r), and the result follows. Proposition 2.2 Let D 1,...,D N, be iid rv s whose common distribution is geometric with pmf (4). If we consider R N,[N/2] and R E N, then we have FR N,[N/2] (1 α) < F R E (1 α), for all 0 < p < 1, and 0 < α < 1. N Proof: Let Y be an exponential rv with df (5) and denote [Y ] the integer part of Y and < Y > the fractional part of Y. If we define X = [Y ] + 1, then f X (x) = F Y (x) F Y (x 1) = ( exp( β) ) (x 1)( 1 exp( β) ) with x N. Note that X is distributed as geometric with π = (1 exp( β)). Now, for π = (1 exp( β)), D i:n d = Xi:N = [Y ] i:n + 1 d = [Y i:n ] + 1, and since Y β d = E, we have D N:N 1 D [N/2]:N d = [Y N:N ] [Y [N/2]:N ] + 1 < [Y N:N]+ < Y N:N > [Y [N/2]:N ]+ < Y [N/2]:N > = Y N:N Y [N/2]:N d = R E N. Proposition 2.3 Let us consider D := {D i } N i=1, the sample of the N durations defined in (3). Denote by Med(R N,[N/2] ) the median of R N,[N/2] and r 1/2,N,[N/2] the

6 6 P. Araújo Santos and M.I. Fraga Alves particular value under geometric distribution with pmf (4). At level α, for testing the IND hypothesis H 0,IND : D i iid D Geometric(π), with 0 < π < 1 and i = 1,...,N against alternatives expressing tendency to clustering patterns H 1 : Med(R N,[N/2] ) > r 1/2,N,[N/2], the rejection region is defined by R N,[N/2] > r α,n,k, where r α,n,[n/2] denotes a quantile 1 α of R E N. Proof: The proof follows straightforward using Propositions 2.1 and 2.2. Remark 1. The critical point r α,n,[n/2] implies a conservative approach with a test of level α and not of size α, i.e., we have P[type I error] α.the test is pivotal in the sense that is based on a distribution that does not depend on an unknown parameter. Remark 2. The test suggested in Proposition 2.3 is based on an exact distribution. The other independence tests, referred in Section 1, are based on asymptotic distributions and suffer from small sample bias. To aggravate the problem, the presence of the nuisance parameter p makes it impossible to control the size of the tests using the Monte Carlo testing approach of Dufour [8] as other authors do for the case of joint testing UC and IND (e.g. Christoffersen and Pelletier [6], Candelon et al. [4] and Berkowitz et al. [1]); see the paper of Dufour [8] for details. 3 Comparative Simulation Study In the context of a Monte Carlo study, we compare the power of the test we suggest in Proposition 2.3 with the Markov, the CAViaR and the GMM independence tests, denoted by M IND, CAViaR and J IND (k). We employ the R language (R Development Core Team [15]) and the fgarch package of Wuertz et al. [16] in order to develop the programs. Following other authors (e.g. Christofferson [7], Christofferson and Pelletier [6], Haas [13], Candelon et al. [4] and Berkowitz et al. [1]) we consider a GARCH specification for the returns process. Additionally, we use a APARCH model which nests some of the GARCH models with leverage effect. Gaussian GARCH(1,1) model (Bollerslev [3]), r t+1 = σ t+1 z t+1 with σ 2 t+1 = w + αr 2 t + βσ 2 t, (9) where the innovations z t+1 s are drawn independently from a standard normal distribution. As in Christofferson [7], we chose the parameterization w = 0.05, α = 0.1 and β = 0.85.

7 A New Independence Test for VaR violations 7 APARCH(1,1) model (Ding et al. [10]), r t+1 = σ t+1 z t+1 with σ δ t+1 = w + α( r t γr t ) δ + βσ δ t, (10) where the innovations z t+1 s are drawn independently from a skewed Student s t(ν) distribution with asymmetry coefficient ϕ, proposed by Fernandez and Steel [12]. We assume a portfolio that replicates the DAX index and we use daily data from beginning of 1997 until the end of 2008, for estimation. The parametrization achieved was w = 0.03, α = 0.086, γ = 0.64, β = 0.91, δ = 1.15, ϕ = 0.88 and ν = 10. As in other power studies with the same purpose, we have chosen the HS method which generates clusters of violations when applied to heteroscedastic processes. We conducted our study with p = 0.01,0.05, T = 250,500,750,1000 and set the size of the rolling window equal to 500. For each T and p, we have simulated returns using the models (9) and (10) over 10,000 replications. The empirical power of the tests is obtained by rejection frequencies with 0.1 significance level, excluding the samples with less than 2 violations. The frequency of excluded samples (FES) are presented in the Tables. To explicitly test the IND hypothesis, it is impossible to have a test of size α using a Monte Carlo approach. Therefore, and for all test statistics except (6), we apply the asymptotic distributions in order to find critical values, conscious of the limitations in the small sample cases. From Table 3.1, it is clear that the proposed test performs better than the other tests under study. In order to study the empirical type I error rates, we have simulated iid Bernoulli samples. In the CAViaR test we have generated the VaR regressors with a GARCH model that are independent of the Bernoulli samples. Table 3.2 shows that the Markov and CAViaR tests are undersized for small sample sizes and oversized for large sample sizes. The GMM tests are extremely undersized for small samples. These results confirm that the asymptotic critical values are misleading. Table Empirical power of tests (α = 0.1). Gaussian GARCH(1,1) p = 0.01 p = 0.05 T=250 T=500 T=750 T=1000 T=250 T=500 T=750 T=1000 R N,[N/2] M IND CAViaR J IND(3) J IND(5) FES Skewed t APARCH(1,1) p = 0.01 p = 0.05 T=250 T=500 T=750 T=1000 T=250 T=500 T=750 T=1000 R N,[N/2] M IND CAViaR J IND(3) J IND(5) FES

8 8 P. Araújo Santos and M.I. Fraga Alves Table Empirical type I error rates with α = 0.1. p = 0.01 p = 0.05 T=250 T=500 T=750 T=1000 T=250 T=500 T=750 T=1000 R N,[0.5N] M IND CAViaR J IND(3) J IND(5) FES Acknowledgements Research partially supported by Fundação para a Ciência e Tecnologia (FCT/ PROTEC and FCT/POCI 2010 project) and Center of Statistics and Applications of University of Lisbon (CEAUL). The authors would like to thank the two referees for their comments and suggestions which lead to improvements of an earlier version of this article. References [1] Berkowitz, J., Christoffersen P., Pelletier D.: Evaluating Value-at-Risk models with desk-level data. Management Science, Published online in Articles in Advance (2009) [2] Bontemps, C.: Testing distributional assumptions: A GMM approach. Working Paper (2006) [3] Bollerslev, T., Generalized Autoregressive Conditional Heteroskedasticity. Journal of Econometrics, 31, (1986) [4] Candelon, B., Colletaz, G., Hurlin, C., and Tokpavi, S.: Backtesting value-at-risk: A GMM Duration-Based Test. HAL, Working Paper (2008) [5] Campbell, S.D.: A review of backtesting and backtesting procedures. Journal of Risk, 9(2), 1-18 (2007) [6] Christoffersen, P. and Pelletier, D.: Backtesting Value-At-Risk: A Duration-Based Approach. Journal of Financial Econometrics, 2(1), (2004) [7] Christoffersen, P.: Evaluating Intervals Forecasts. International Economic Review, 39, (1998) [8] Dufour, J.M.: Monte Carlo tests with nuisance parameters: a general approach to finite sample inference and nonstandard asymptotics. Journal of Econometrics, 127(2), (2006) [9] Danielsson, J. and Morimoto, Y.: Forecasting Extreme Financial Risk: A Critical Analysis of Practical Methods for the Japanese Market. Monetary and Economic Studies, 18(2), (2000) [10] Ding, Z., Engle, R.F and Granger, C.W.J.: A long memory property of stock market return and a new model. Journal of Empirical Finance, 1, (1993) [11] Engel, R.F. and Manganelli, S.: CAViaR: Conditional Autoregressive Value-at-Risk by Regression Quantiles. Journal of Business and Economics Statistics, 22, (2004) [12] Fernández, C. and Steel, M.F.j.: On Bayesian modelling of fat tails and skewness. Journal of the American Statistical Association, 93, (1998) [13] Haas, M.: Improved duration-based backtesting of Value-at-Risk. Journal of Risk, 8(2), (2005) [14] Malik, R.J., Trudel, R.: Probability density function of quotient of order statistics from the pareto, power and weibull distributions. Communications in Statistics - Theory and Methods, 11(7), (1982) [15] R Development Core Team: R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN , URL (2008) [16] Wuertz, D., Chalabi, Y. and Miklovic, M.: fgarch: Rmetrics - Autoregressive Conditional Heteroskedastic Modelling. R package version (2008)