Backtesting Marginal Expected Shortfall and Related Systemic Risk Measures

Transcription

1 Backtesting Marginal Expected Shortfall and Related Systemic Risk Measures Denisa Banulescu, Christophe Hurlin, Jérémy Leymarie, Olivier Scaillet February 13, 2016 Preliminary version - Please do not cite Abstract This paper proposes two backtesting tests to assess the validity of the systemic risk measure forecasts. This new tool meets the need of financial regulators of evaluating the quality of systemic risk measures generally used to identify the financial institutions contributing the most to the total risk of the financial system SIFIs. The tests are based on the concept of cumulative joint violation process and it is built up in analogy with the recent backtesting procedure proposed for the ES Expected Shortfall. First, we introduce two backtests that apply for the case of the MES Marginal Expected Shortfall forecasts. The backtesting methodology is then generalised to MES-based systemic risk measures SES, SRISK and to the CoVaR. Second, we study the asymptotic properties of the tests in presence of estimation risk and we investigate their finite sample performances via Monte Carlo simulations. Finally, we use our backtests to asses the validity of the MES, SRISK and CoVaR forecasts on a panel of EU financial institutions. Keywords: Systemic risk, Backtesting, Marginal Expected Shortfall, SRISK, CoVaR JEL classification: C32 C58 G01 G32 1 Introduction Many systemic risk measures have been proposed in the academic literature over the past years see Benoit et al. 2016, for a survey, the most well-known being the Marginal Expected Shortfall MES and the Systemic Expected Shortfall SES of Acharya et al. 2010, the Systemic Risk Measure SRISK of Acharya et al and Brownlees and Engle 2015, and the Delta Conditional Value-at-Risk CoVaR of Adrian and Brunnermeier These measures are designed to summarize the systemic risk contribution of each financial institution into a single figure, in order to identify the so-called systemically important financial institutions SIFIs, i.e., the firms whose failure might trigger a crisis in the whole financial system. The identification of the SIFIs is crucial for the systemic risk regulation, whatever the regulation tools considered higher capital University of Orléans LEO, UMRS CNRS University of Orléans LEO, UMRS CNRS University of Orléans LEO, UMRS CNRS University of Geneva and Swiss Finance Institute. 1

2 requirements, specific regulation, systemic risk tax, etc.. As a consequence, regulators and other end-users of these measures need hence guidance on how to select the ones most adapted to their objectives. In this context, the validation is a key requirement for any systemic risk measure to become an industry standard. Some attempts of validation procedures have been proposed in the literature. Following the coherent risk approach of Artzner et al. 1999, Chen, Iyengar, and Moallemi 2013 define an axiomatic framework for systemic risk measures. Brownlees and Engle 2015 show that banks with higher SRISK before the financial crisis were more likely to be bailed out by the government and to receive capital injections from the Federal Reserve. Engle, Jondeau, and Rockinger 2015 compare the empirical ranking obtained with a given measure with the one computed by the FSB-BCBS, which is based on confidential bank supervisory data. However, to the best of our knowledge, no backtesting procedure have been proposed yet for the systemic risk measures. In this paper we propose a general framework for backtesting the MES and the related systemic risk measures. Introduced by Acharya et al. 2010, the MES of a financial firm is defined as its short-run expected equity loss conditional on the market taking a loss greater than its Valueat-Risk VaR. The MES is simple to compute and therefore easy for regulators to consider. Furthermore, the MES constitutes one of the key elements with the leverage and the market value of two popular systemic risk measures, i.e., the SRISK and the SES. The latter is equal to the expected amount a bank is undercapitalized in a future systemic event in which the overall financial system is undercapitalized Acharya et al Thus, the SES increases in the bank s expected losses during a crisis. The SRISK corresponds to the expected capital shortfall of a given financial institution, conditional on a crisis affecting the whole financial system Brownlees and Engle This index can be used to rank the financial institutions according to their systemic risk i.e., the firms with the highest SRISK are the largest contributors to the undercapitalization of the financial system in times of distress. What it is important to note is that these two systemic risk measures are theoretically related to the the current leverage, the current equity value of the financial institution and its MES. As a consequence, testing for the validity of the MES forecasts, implies to test for the validity of SRISK or SES forecasts. As defined by Jorion 2007, backtesting is a formal statistical framework that consists in verifying if actual losses are in line with projected losses. This involves a systemic comparison of the history of model-generated risk measure forecasts with actual returns. Since the true value of the risk measure is unobservable, this comparison generally relies on violations. When one comes to backtest the VaR, a violation is said to occur when the ex-post portfolio return is lower than the VaR forecast. The trick of our paper consists in defining an appropriate concept of violation for the MES forecast. Then, it is possible to adopt the same standard backtesting procedures Kupiec, 1995; Christoffersen, 1998, etc. as those that are currently used for the VaR forecasts see Christoffersen, 2009 for a survey on backtesting. Our approach is the following. First, we introduce a concept of Conditional-VaR CoVaR, inspired by the systemic risk measure proposed by Adrian and Brunnermeier The β, α-covar is defined as the β-quantile of the truncated 2

3 distribution of the firm s returns given that the market takes a loss greater than its α-var. We express the MES as an integral of the CoVaRs for all the coverage rate β between 0 and 1. To the best of our knowledge, this is the first time that a relationship is established between the CoVaR and the MES, and so on with the SES and SRISK. Second, we define a concept of joint violation of the β, α-covar of the firm s returns and α-var of the market returns. Finally, we extend the concept of cumulative violation recently proposed by Du and Escanciano 2015 for the Expected Shortfall ES backtests, to a bivariate case. We define a cumulative joint violation process defined as the integral of the joint violation processes for all the coverage rate β between 0 and 1. This process can be viewed as a violation counterpart of the definition that we propose for the MES. Furthermore, we show that this process has the same properties as the cumulative violation process introduced by Du and Escanciano for the ES. In particular, this process is a martingale difference sequence mds. Exploiting the mds properties, we propose two backtests for the MES : an Unconditional Coverage UC test and an Independence IND test Christoffersen, The UC test refers to the fact that the frequency of violations should be in line with the theoretical probability to observe a violation. Failure of UC means that the MES forecasts does not measure the risk accurately. Besides UC, MES forecasts should satisfy the independence property. The independence property refers to the clustering of violations. One advantage of our approach is that it allows to backtest either conditional with respect to the past information set MES Brownless and Engle, 2015 or unconditional MES Acharya et al., 2010 forecasts. We consider similar test statistics to those used by Du and Escanciano 2015 for the backtesting of the ES, since they are similar to those generally used by the regulator or the risk manager in the context of the VaR backtesting Kupiec, We derive the asymptotic distribution of the two test statistics, while taking into account the estimation risk Escanciano and Olmo 2010, 2011; Gouriéroux and Zakoian, Indeed, the MES forecasts are generally issued from a parametric framework in which the parameters have to be estimated. Therefore, the use of standard backtesting procedures to assess the MES model in an out-of-sample basis can be misleading, because they do not take into account the impact of the estimation risk. This is why we propose a robust version of our test statistics to the estimation risk. Within a Monte Carlo simulation study we show that our robust test statistics have good finite sample properties for realistic sample sizes. Finally, we extend the analysis and we derive backtesting tests for the CoVaR and the MESbased systemic risk measures SES, SRISK. The CoVaR is defined as the difference between the conditional VaR CoVaR of the financial system conditional on an institution being in distress and the CoVaR conditional on the median state of the institution. The backtesting of the CoVaR is based on a vector of two joint violations defined for the two conditioning events. Thus, the intuition of the test is similar to the backtests proposed for multi-level VaR Francq and Zakoian, 2015, i.e. the VaR defined for a finite set of coverage rates Hurlin and Tokpavi, 2006; Pérignon and Smith, 2008; Leccadito, Boffelli and Urga, Since the SRISK and the SES can be written as a deterministic function of the MES, we adapt the two MES backtesting tests UC and IND, 3

4 as well as their robust versions, for these specific systemic risk measures. The paper is organized as follows. In Section 1, we define the MES and we express it as a simple integral of conditional Value-at-Risk CoVaR. Section 2 introduces the concept of cumulative joint violation process and its properties, subsequently used for the backtesting of the MES. The backtesting tests for the MES are presented in Section 3 and their finite sample properties are illustrated by Monte Carlo simulations in Section 4. Section 5 extends our backtests to the main systemic risk measures that depends on the MES, i.e. the SES and the SRISK of Acharya et al and Brownlees and Engle 2015, and to the CoVaR of Adrian and Brunnermeier An empirical application is proposed in Section 6. Finally, the last section concludes. 2 Marginal Expected Shortfall Let Y t = Y 1t, Y 2t denote the vector of stock returns of two assets at time t. In the specific context of systemic risk, Y 1t generally corresponds to the stock return of a financial institution, whereas Y 2t corresponds to the market return. 1 Denote by Ω t 1 the information set available at time t 1, with Y t 1, Y t 2,... Ω t 1 and F.; Ω t 1 the joint cumulative distribution function cdf of Y t given Ω t 1 such that F y t ; Ω t 1 Pr Y 1t < y 1, Y 2t < y 2 Ω t 1 for any y = y 1, y 2 R 2. Assume that F.; Ω t 1 is continuous. Following Acharya et al. 2010, we define the MES of a financial firm as its short-run expected equity loss conditional on the market taking a loss greater than its Value-at-Risk VaR. This definition of the MES was extended to a Ω t 1 -conditional version by Brownlees and Engle Formally, the α-level MES of the financial institution at time t given Ω t 1 is defined as MES 1t α = EY 1t Y 2t V ar 2t α; Ω t 1, 1 where V ar 2t α denotes the α-level VaR of Y 2t, such that Pr Y 2t V ar 2t α Ω t 1 = α with α [0, 1]. Notice that if the market return Y 2t is defined as the value-weighted average of the firm s returns for all the firms that belong to the financial system, then the MES of one firm corresponds to the derivative of the market s Expected Shortfall ES with respect to the firm s market share Scaillet, 2004, hence the term Marginal. Hence, the MES can be interpreted as a measure of the participation of one financial institution to the overall systemic risk. As usual, V ar 2t α is defined as the α-th percentile of the marginal distribution of Y 2t, denoted F Y2., Ω t 1, with V ar 2t α = F 1 Y 2., Ω t 1. 2 Similarly, the MES can be expressed as a function of the quantiles of the conditional distribution of Y 1t given Y 2t V ar 2t α. For that, we introduce a concept of Conditional-VaR CoVaR inspired from the systemic risk measure proposed by Adrian and Brunnermeier For any coverage level β [0, 1], the CoVaR for the firm 1 at time t is the quantity CoV ar 1t β, α such that Pr Y 1t CoV ar 1t β, α Y 2t V ar 2t α; Ω t 1 = β. 2 1 Our results can be easily extended to the general case with m 2 assets. 2 For simplicity in the notations, we do not use the usual convention that defines the VaR as the opposite of the α-quantile of the returns distribution. Similarly, we define the MES as the conditional expectation. 4

5 There are two main differences between CoV ar 1t β, α and the CoVaR introduced by Adrian and Brunnermeier First, the conditioning event is based on an inequality, i.e., Y 2t V ar 2t α as in Girardi and Ergun 2013, rather than on the equality Y 2t = V ar 2t α. Second we introduce a distinction between the coverage level β of the CoVaR and the coverage level α of the VaR, which is used to define the conditioning event. Then, the β, α-level CoVaR can also be defined as CoV ar 1t β, α = F 1 Y 1 Y 2 V ar 2tα β; Ω t 1, 3 where F Y1 Y 2 V ar 2tα.; Ω t 1 is the cdf of the conditional distribution of Y 1t given Y 2t V ar 2t α and Ω t 1. The definition of conditional probability and a change in variables yield a useful representation of the MES in terms of CoVaR: MES 1t α = 1 0 CoV ar 1t β, αdβ. 4 Equation 4 gives a simple relationship between two risk measures, i.e., the MES and the CoVaR, that are broadly used in the systemic risk literature. Notice that this expression can be used to determine either the Ω t 1 -conditional MES 1t α, as in Brownlees and Engle 2015 or the unconditional MES 1 α, as in Acharya et al Furthermore, this definition of the MES is valid for any bivariate distribution and for any dynamic model DCC, CCC, etc.. For some particular distributions, the conditional cdf F Y1 Y 2 V ar 2tα.; Ω t 1, which defined previously the CoVaR, has a closed form expression. For instance, Arnold et al calculated the marginal of a bivariate normal distribution with double truncation over one variable. Horrace 2005 formalized analytical results on the truncated multivariate normal distribution, where the truncation is one-sided and at an arbitrary point. Ho et al proposed a study of the truncated multivariate t distribution. However, whatever the distribution considered, it is possible to express the cdf of the truncated distribution of Y 1t given Y 2t V ar 2t α as a simple function of the cdf of the joint distribution of Y t, such that where the vector ỹ is defined as ỹ = y 1, V ar 2t α. F Y1 Y 2 V ar 2tαy 1 ; Ω t 1 = 1 α F ỹ; Ω t 1, 5 In general, the MES forecasts are issued from a parametric model specified by the researcher, the risk manager or the regulation authority. For instance, Brownlees and Engle 2015 or Acharya, Engle, and Richardson 2012 consider a bivariate DCC model to compute the MES and the SRISK. In practice, the cdf F y; Ω t 1, θ 0 of the joint distribution of Y t, the cdf F Y2.; Ω t 1, θ 0 of the marginal distribution of Y 2t and the cdf F Y1 Y 2 V ar 2tα;θ 0y; Ω t 1, θ 0 of the truncated distribution of Y 1t given Y 2t V ar 2t α, θ 0 depend on θ 0, an unknown parameter in Θ R p. This parameter has to estimated before producing MES forecasts. 3 Cumulative Joint Violation Process As defined by Jorion 2007, backtesting is a formal statistical framework that consists in verifying if actual losses are in line with projected losses. This involves a systemic comparison of the history 5

6 of model-generated risk measure forecasts with actual returns. This comparison generally relies on tests over violations. When one comes to backtest the VaR, a violation is said to occur when the ex-post portfolio return is lower than the VaR forecast. In order to backtest the CoVaR and the MES, we define a joint violation of the β, α-covar of Y 1t and the α-var of Y 2t at time t. This violation process is represented by the following binary variable h t α, β, θ 0 = 1 Y 1t CoV ar 1t β, α, θ 0 Y 2t V ar 2t α, θ 0, 6 where 1. denotes the indicator function. The violation takes the value one if the losses of the firm exceeds its CoVaR and the losses of the market exceeds its VaR, and it is zero otherwise. The VaR backtesting procedures Kupiec, 1995; Christoffersen, 1998; Berkowitz et al., 2011, among others are generally based on the mds property of the violation process see Christoffersen, 2009; Hurlin and Pérignon, 2012, for a survey. Here, we adopt the same approach for backtesting the CoVaR, and thus the MES. Notice that the Bayes theorem implies that Pr h t α, β, θ 0 = 1 Ω t 1 = αβ. Therefore, equation 2 indicates that the violations are Bernouilli variables with mean αβ, and that the centered violation {h t α, β, θ 0 αβ} is a mds with risk levels α, β [0, 1] 2. E h t α, β, θ 0 αβ Ω t 1 = 0. 7 In order to test for the validity of the MES, we consider a cumulative joint violation process which can be viewed as a kind of violations counterpart of the MES definition in equation 4. This cumulative joint violation process is defined as the integral of the violations h t α, β, θ 0 for all the risk levels β between 0 and 1, with H t α, θ 0 = 1 0 h t α, β, θ 0 dβ. 8 This cumulative joint violation process is similar to the cumulative violation process introduced by Du and Escanciano 2015 to backtest the ES, even though the two definitions are slightly different. The mean and the variance of H t α, θ 0 are equal to α/2 and α 1/3 α/4, respectively see appendix A. Furthermore, the Fubini s theorem implies that the mds property of the sequence {h t α, β, θ 0 αβ}, α, β [0, 1]2 is preserved by integration. As a consequence, the sequence {H t α, θ 0 α/2} is also a mds for any α [0, 1], such that E H t α, θ 0 α/2 Ω t 1 = 0. 9 The backtesting tests that we propose for the MES are based on this mds property. Finally, it is possible to rewrite H t α, θ 0 in a more convenient way, through the Probability Integral Transformation PIT. Notice that the cumulative joint violation process H t α, θ 0 depends on the distribution of Y t as follows H t α, θ 0 = 1 Y 2t V ar 2t α, θ 0 1 = 1 F Y2 Y 2t ; Ω t 1, θ 0 α Y 1t CoV ar 1t β, α, θ 0 dβ F Y1 Y 2 V ar 2tα;θ 0 Y 1t ; Ω t 1, θ 0 β dβ.

7 Let us introduce two terms that can be interpreted as generalized errors, namely u 2t θ 0 u 2t = F Y2 Y 2t ; Ω t 1, θ 0 and u 12t θ 0 u 12t = F Y1 Y 2 V ar 2tα,θ 0Y 1t ; Ω t 1, θ 0. Then, the cumulative joint violation process becomes H t α, θ 0 = 1 u 2t α u 12t β dβ = 1 u 2t α 1dβ. u 12t 11 Thus, the process H t α, θ 0 can be expressed as a simple function of the transformed i.i.d. variables u 2t and u 12t, defined over [0, 1], such as H t α, θ 0 = 1 u 12t θ 0 1u 2t θ 0 α. 12 The PIT implies that the variable u 2t has a uniform U [0,1] distribution. The generalized error term u 12t has also a U [0,1] distribution as soon as the transformation F Y1 Y 2 V ar 2tα,θ 0.; Ω t 1, θ 0 is applied to observations Y 1t for which Y 2t V ar 2t α, θ 0. 4 Backtesting MES Exploiting the mds property of the cumulative joint violation process, we propose two backtests for the MES. These tests are similar to those generally used by the regulator or the risk manager for VaR backtesting Christoffersen, The unconditional coverage hereafter UC test corresponds to the null hypothesis H 0,UC : E H t α, θ 0 = α/2. 13 Since E H t α, θ 0 = 1 0 E h tα; β, θ 0 dβ, the null H 0,UC can also be written as H 0,UC : Pr h t α, β, θ 0 = 1 = αβ, β [0, 1]. The null hypothesis UC means that for any coverage level β, the joint probability to observe an ex-post return y 1t exceeding its β α-covar and an ex-post return y 2t exceeding its α-var must be equal to αβ. The second backtest is based on the independence hereafter IND property of the cumulative violation process {H t α, θ 0 α/2} : the cumulative violations observed at two different dates for the same coverage rate α must be distributed independently. As Christoffersen 1998, we propose a simple Box-Pierce test Box and Pierce, 1970 in order to test for the nullity of the first K autocorrelations of H t α, θ 0, denoted ρ k. The null of the IND test is then defined as with ρ k = corr H t α, θ 0 α/2, H t k α, θ 0 α/2. H 0,IND : ρ 1 =... = ρ K = However, these two tests require the estimation of the parameters θ 0 Θ in order to forecast the MES. For simplicity, we consider a fixed forecasting scheme. An in-sample period from t = T + 1 to t = 0 is used to estimate θ 0. Denote by Ω 1 the information set available at the end of the 7

8 in-sample period, with {Y T +1,..., Y 0 } Ω 1 and θ T a consistent estimator of θ 0. This estimator is used to forecast the MES for all the dates from t = 1 to t = n. The backtesting tests are then based on the out-of-sample forecasts of the cumulative violation process process given by H t α, θ T = 1 u 12t θ T 1 u 2t θ T α, t = 1,..., n Unconditional Coverage Test By analogy with the backtest proposed by Du and Escanciano 2015 for ES, and the Kupiec s backtest 1995 for VaR, we propose a standard t-test for the null hypothesis of unconditional coverage H 0,UC for the MES. This test statistic, denoted UC MES, is defined as n Hα, θt α/2 UC MES = α 1/3 α/4, 16 with Hα, θ T the out-of-sample mean of H t α, θ T Hα, θ T = 1 n n H t α, θ T. 17 In order to give the intuition of the asymptotic properties of the statistic UC MES, let us define a similar statistic UC MES α, θ 0 based on the true value of the parameters θ 0 rather than on its estimator ˆθ T. Under the null hypothesis, the sequence {H t α, θ 0 α/2} n is a mds with a variance equal to α 1/3 α/4. As a consequence, the Lindeberg-Levy central limit theorem implies that UC MES α, θ 0 has an asymptotic standard normal distribution. holds for the feasible statistic UC MES λ = n/t 0, i.e. when there is no estimation risk. A similar result UC MES α, θ T when T and n, whereas However, in the general case T, n and n/t λ <, there is an estimation risk as soon as λ 0. In this case, the asymptotic distribution of UC MES is not standard and it depends on the ratio of the in-sample size T to the out-of-sample size n. Theorem 1 gives the corresponding asymptotic distribution of UC MES when T, n and n/t λ with 0 < λ <. Theorem 1 Under assumptions A1-A4 UC MES d N 0, σ 2 λ, 18 where d denotes the convergence in distribution and where the asymptotic variance σ 2 λ is where R MES = E 0 H t α, θ 0 / and V as ˆθ T = Σ 0 /T. σ 2 λ = 1 + λ R MES Σ 0R MES α 1/3 α/4, 19 The proof of Theorem 1 is reported in appendix B. The vector R MES quantifies the parameter estimation effect on the test statistic UC MES. Indeed, this estimation risk that comes from the 8

9 difference between the estimate θ T and the true value of the parameter θ 0. This difference affects the UC MES test statistic as follows UC MES = n 1 H t α, θ 0 α/2 σ H n }{{} UC MES θ 0 λ 1 n H t α, θ + E Ω t 1 T θ T θ 0 + o p σ H n } {{ } Estimation risk Whatever the dynamic model considered for the returns, the vector R MES can be simply deduced from the cdf of the joint distribution of Y t given Ω 1. Indeed, since the derivative of a step function is a Dirac function and a continuous distribution has no point mass, we get Ht α, θ 0 R MES = E 0 = 1 F α E ỹt, θ 0 0 where ỹ t = y 1t, V ar 2t α, θ 0 and 21 F ỹ t, θ 0 = y1t f u, V ar 2t α, θ 0 ; θ 0 du V ar 2t α, θ 0 }{{} y1t V ar2t Impact on the truncation f u, v; θ 0 + dudv }{{} impact on the pdf of the joint distribution 22 with the pdf of the conditional distribution of Y 1t given Y 2t = V ar 2t α. An analytical expression for the derivative F ỹ t, θ 0 / can be determined for some particular bivariate distributions see appendix C for the case of the bivariate normal distribution. In the general case, it can be obtained by numerical differentiation. Corollary 2 When there is no estimation risk, i.e. when λ = 0, under assumptions A1-A4, UC MES d N 0, When the estimation period T is much larger than the evaluation period n, the unconditional coverage test is simplified since it does not require to evaluate R MES and Σ 0. Given these results, it is possible to define a robust test statistic, denoted UCMES C, that explicitly takes into account the estimation risk and that has standard limit distribution for any λ with 0 λ <, when T and n tends to infinity. The feasible robust UC backtest statistics is n Hα, θt α/2 UCMES C = α 1/3 α/4 + n R MES V as ˆθ T R 24 MES 9

10 with V as ˆθ T = Σ 0 /T a consistent estimator of the asymptotic variance covariance matrix of ˆθ T and R MES, a consistent estimator of R MES given by R MES = 1 αn with ŷ t = y 1t, V ar 2t α, ˆθ T. 4.2 Independence Test n F ŷ t, θ T 1y 2t V ar 2t α, ˆθ T, 25 To test the independence hypothesis H 0,IND : ρ 1 =... = ρ m = 0, we use a Portmanteau Box- Pierce test on the sequence of cumulative joint violation forecasts. The Box-Pierce test statistic is defined as follows IND MES = n m ˆρ 2 j, 26 with ˆρ j the sample autocorrelation of order j of the estimated cumulative joint violation H t α, θ T given by ˆρ j = γ j γ 0 and γ j = 1 n j n +j j=1 H t α, θ T α/2 H t j α, θ T α/2, 27 where γ j denotes a consistent estimator of the j-lag autocovariance of H t α, ˆθ T. Theorem 3 gives the asymptotic distribution of the statistic IND MES when T, n and n/t λ <. Theorem 3 Under assumptions A1-A4: IND MES d m π j Zj 2, 28 j=1 where {π j } m j=1 are the eigenvalues of the matrix with the ij-th element given by R j = ij = δ ij + λr iσ 0 R j, 29 1 α 1/3 α/4 E 0 H t j α, θ 0 α/2 H tα, θ 0, 30 δ ij is a dummy variable that takes a value 1 if i = j and 0 otherwise, {Z j } m j=1 are independent standard normal variables and V as ˆθ T = Σ 0 /T. The proof of Theorem 3 is reported in appendix D. The test statistic IND MES has an asymptotic distribution which is a weighted sum of chi-squared variables. The weights depends on the asymptotic variance-covariance matrix of the estimator θ T, on the cumulative joint violation process and on its derivative with respect to the model parameter θ, as for the UC test. However, this limit distribution becomes standard when λ = 0, i.e., when there is no estimation risk. Corollary 4 When there is no estimation risk, i.e. when λ = 0, under assumptions A1-A4, IND MES d χ 2 m

11 From the previous results, we can deduce a robust test statistics for the independence hypothesis which has standard distribution for any λ with 0 λ <, when T and n tends to infinity. Denote ρ m the vector ρ 1,..., ρ m and ρ m its empirical counterpart. The feasible robust IND backtest statistics is defined as where is a consistent estimator for, such that 1 1 R j = α 1/3 α/4 n j IND C MES = n ρ m 1 ρ m 32 ij = δ ij + n R i V as ˆθ T R j, 33 n H t j α, θ Ht α, T α/2 θ T, 34 t=j+1 and V as ˆθ T is a consistent estimator of the asymptotic variance covariance matrix of ˆθ T. When T and n tends to infinity, the robust statistic INDMES c converges to a chi-squared distribution with m degrees of freedom whatever the relative value of n and T. 5 Monte Carlo Simulation Study This section assesses the finite sample properties of the test statistics, computed with and without taking into account the estimation risk. The first part describes the data generating process DGP used to build up the realistic setup, along with the algorithm required to construct the cumulative violation series. The second part is devoted to the analysis of the empirical size and power of the test. 5.1 Monte Carlo Design In line with the current literature, we consider two definitions of MES to examine the finite sample proprieties of our test. First, we present MES as a conditional systemic risk measure à la Brownlees and Engle, Second, we deal with the particular case of a time-invariant MES defined à la Acharya et al., The DGP, as well as the size and power exercises, are drawn up in line with these aspects. Conditional MES For the dynamic case, the DGP consists in a dynamic conditional correlation DCC model Engle, 2002 computed on demeaned return processes Y t = Y 1t Y 2t : Y t = Σ 1/2 t z t, 35 where the 2x2 matrix Σ t is the time-varying conditional covariance matrix of Y t and z t is an i.i.d. Gaussian vector error process such that E[z t ] = 0 and E[z t z t] = I 2. The conditional covariance matrix is defined as follows: Σ t = D t R t D t, 36 11

12 where D t = diag{ σ1,t 2 ; σ2,t 2 } contains conditional standard deviations of the Y t system and [ ] 1 ρt R t = is the conditional correlation matrix of Y ρ t 1 t. The conditional variances are most often modeled using the standard univariate GARCH1,1 framework: σ 2 i,t = α i,0 + α i,1 Y 2 i,t 1 + α i,2 σ 2 i,t 1, i = 1, 2 Furthermore, the time varying correlation matrix;, R t, that can be obtained as follows: E t 1 ε t ε t = Dt 1 Σ t Dt 1 = R t, since ε t = Dt 1 Y t. We consider: Q t = 1 a br + aε t 1 ε t 1 + bq t 1, where a and b are non-negative scalar parameters such that a + b < 1, R is the unconditional correlation matrix of the standardized errors ε t, and Q 0 is positive definite. The correlation matrix is subsequently obtained by rescaling Q t, such as: R t = I Q t 1/2 Q t I Q t 1/2. For a more realistic scenario we use the parameter estimates obtained from the GARCH1,1- DCC1,1 model on real log-returns series i.e., JP Morgan Chase Co. and S&P500 index from January 1st, 2005 to October 9th, Based on this dynamic specification, we compute the conditional M ES as follows : MES t = E t 1 [Y 1t Y 2t V ar 2t α] = xf Y1t Y 2t V ar 2tαx; θ0 Ω c t 1 dx R where f Y1t Y 2t V ar 2t x; θ c 0 Ω t 1 is the pdf of Y 1t Y 2t V ar 2t, θ c 0 is the set of true parameters, and Ω t 1 is the known information set at time t 1, so that MES t depends on past information. The vector of parameters is estimated at each iteration using the following two-step procedure: i first, we estimate the univariate GARCH parameters α 1,0, α 1,1, α 1,2, α 2,0, α 2,1, α 2,2 by CMLE; ii second, given the estimated parameters from step one, we adjust the DCC parameters a, b based on a bivariate Normal distribution. Unconditional MES A natural extension of the conditional specification is represented by the time invariant representation, given by: 37 Y t = Σ 1/2 z t 38 where the 2x2 matrix Σ is the non time-varying covariance matrix of Y t and z t is the same error term than in dynamic case. The unconditional covariance matrix is defined as follows : Σ = DRD 39 where D t = diag{ σ1 2; [ ] σ2 2} contains unconditional standard deviations of the Y t system and 1 ρ R = is the unconditional correlation matrix of Y ρ 1 t. 3 GARCH1,1 parameters for the conditional variances of the first and second asset, α 1,0, α 1,1, α 1,2, α 2,0, α 2,1, α 2,2, are set to , , and , , DCC1,1 [ parameters for the ] conditional correlation between the two series, a, b, are set to , , and R =

13 In line with the previous specification, the parameters of this specific static case correspond with the unconditional counterpart of the GARCH1,1-DCC1,1 model parameters. 4 The unconditional M ES associated with this static specification is computed as follows: MES = E[Y 1t Y 2t V ar 2t α] = xf Y1t Y 2t V ar 2tαx; θ0 u dx R 40 where f Y1t Y 2t V ar 2t x, θ0 u is the pdf of Y 1t Y 2t V ar 2t, and θ0 u is the vector of parameters aggregating all the parameters of this unconditional specification. At each replication, we estimate θ0 u by MLE using the proprieties of the bivariate Normal distribution. The Monte Carlo simulation exercise is based on 10,000 replications, and we consider in-sample sizes T = 250, 2500 and out-of-sample sizes N = 250, 500. The coverage levels are set to α = 0.01, Size and Power The empirical size and power of the test are assessed within the two configurations previously presented. Note that when studying the power, two types of misspecification are considered under the alternative: i on the volatility dynamics; i on the correlation dynamics. For instance, in the H 1 A and H 1 B frameworks, the volatility of Y 1,t i.e., the firm and Y 2,t i.e., the market, respectively, is undervalued. Such underestimation of the riskiness on the firm/market losses might wrongly reduce the estimated MES. In the H 1 C structure, the misspecification occurs in the dependence between the two series. We focus hence on a scenario in which the correlation between the firm and the market is undervalued. This can imply a strong undervaluation of MES when the market is in times of distress. Consider first the conditional MES case. The DGP structure of Y t under the null and the alternative is given by: H 0 : GARCH1,1-DCC1,1 model: Y t = Σ 1/2 t z t Σ t = D t R t D t z t i.i.d. N0, I H 1 A : GARCH1,1-DCC1,1 model with undervalued variance of the firm Y 1,t : Y t = Σ 1/2 t z t σ 2 1,t = α H1A 1,0 + α 1,1 Y 2 1,t 1 + α 1,2 σ 2 1,t 1 with α H1A 1,0 = 25% α 1,0, 50% α 1,0, successively. 4 Thus, σ1 2, σ2 2, and ρ match the unconditional variances and the unconditional correlation of our dynamic specification. We set hence σ1 2, σ2 2, ρ to , ,

14 H 1 B : GARCH1,1-DCC1,1 model with undervalued variance fo the market Y 2,t : Y t = Σ 1/2 t z t σ 2 2,t = α H1B 2,0 + α 2,1 Y 2 2,t 1 + α 2,2 σ 2 2,t 1 with α H1B 2,0 = 25% α 2,0, 50% α 2,0, successively. H 1 C : GARCH1,1-DCC1,1 model with undervalued correlation : Y t = Σ 1/2 t z t Σ t = D t RD t [ 1 ρ R = H1C ρ H1C 1 ] with ρ H1C = 0.3, 0.6, successively. Second, in the unconditional MES case, the DGP structure of Y t under the null and the alternative is given by: H 0 : Static model: Y t = Σ 1/2 z t, Σ = DRD z t i.i.d. N0, I H 1 A : Static model with undervalued variance of Y 1,t : Y t = Σ 1/2 z t [ ] σ 2,H1A Σ = 1 ρσ H1A 1 σ 2 ρσ H1A 1 σ 2 σ2 2 with σ 2,H1A 1 = 25% σ 2 1, 50% σ 2 1, successively. H 1 B : Static model with undervalued variance of Y 2,t : Y t = Σ 1/2 z t [ Σ = σ 2 1 ρσ 1 σ H1B 2 ρσ 1 σ H1B 2 σ 2,H1B 2 ] with σ 2,H1B 2 = 25% σ 2 2, 50% σ 2 2, successively. H 1 C : Static model with undervalued correlation : Y t = Σ 1/2 z t [ σ1 Σ = 2 ρ H1C σ 1 σ 2 ρ H1C σ 1 σ 2 σ2 2 ] 14

15 with ρ H1C = 0.3, 0.6, successively. Table 1 and Table 2 present the empirical sizes and size-corrected power of the UC and IND tests, at 5% nominal level. Note that these results are performed for both the conditional and unconditional MES at 5% significance level and they rely on the scenarios previously presented for the null and the alternatives. Specific configurations on the length of the in-sample and out-ofsample series are considered. So far, this exercise provides us satisfactory sizes for the two tests, as well as a quite well identification of the alternatives H A 1, H B 1, H C 1. However, the estimation risk is not yet taken into account, since we are only simulating MES forecasts based on a simple normal bivariate distribution. 5 6 Backtesting other Systemic Risk Measures Other systemic risk measures can be written as a function of the MES or the CoVaR, and therefore, they can be backtested according to our methodology. It is especially the case for the SES Acharya et al., 2010 and the SRISK Acharya et al., 2012, and Brownlees and Engle, 2015 which extend the MES in order to take into account both the liabilities and the size of the financial institution. It is also the case for the CoVaR Adrian and Brunnermeier, 2016 which is fundamentally based on the difference of two conditional VaR. 6.1 Backtesting MES-based Systemic Risk Measures The SRISK corresponds to the expected capital shortfall of the financial institution, conditional on a crisis affecting the whole financial system. Brownlees and Engle 2015 define the capital shortfall CS it as the capital reserves the firm needs to hold for regulation or prudential management minus the firm s equity. Then, we have CS 1t 1 = k L 1t 1 + W 1t 1 W 1t 1, 41 with L 1t the book value of debt, W 1t the market value of the firm s equity and k a prudential ratio. As a consequence, if the financial system crisis is defined as a situation in which Y 2t < V ar 2t α, the SRISK becomes SRISK 1t = E t 1 CS 1t Y 2t < V ar 2t α 42 = ke t 1 L 1t Y 2t < V ar 2t α 1 k E t 1 W 1t Y 2t < V ar 2t α 43 Brownlees and Engle assume that the debt is constant, i.e., E t 1 L 1t Y 2t < V ar 2t α = L it 1. Since E t 1 W 1t Y 2t < V ar 2t α = W it E t 1 Y 1t Y 2t < V ar 2t α, we get SRISK 1t = k L 1t 1 1 k W 1t 1 MES 1t α 44 5 Further on, we will take into account the estimation risk by generating our forecasts as the result of a DCC model estimation. These results will be soon integrated in the paper. 15

16 Similarly, we can define a linear relationship between the SES Acharya et al., 2010 and the MES. Indeed, the SES corresponds to the amount a bank s equity drops below its target level defined as a fraction k of assets in case of a systemic crisis when aggregate capital is less than k times the aggregate assets. Acharya et al show that the SES of bank i can be expressed as linear function of its MES, with: SES it = k LV it 1 + θ MES it α + W it 1, 45 where θ and are constant terms, W it the market capitalization or market value of equity, and L it = L it 1 + W it 1 /W it 1 the leverage. Within this two examples, a MES-based systemic risk measure for the firm i at time t, denoted RM it, can defined as a deterministic function of the MES it α given Ω t 1, such as: RM it = g t MES it α, X i,t 1, 46 with g t. an decreasing function with MES and X t 1 a set of variables that belong to Ω t 1. Define a new joint violation process h t α, β, θ 0, X t 1 such that h t α, β, θ 0, X t 1 = 1 1 g t Y 1t, X t 1 g t CoV ar 1t β, α, θ 0, X t 1 1 Y 2t V ar 2t α, θ 0, 47 and a corresponding cumulative joint violation process H t α, θ 0, X t 1 = For these systemic risk measures, we consider the following test H 0,RM : E H t α, θ 0, X t 1 = α 2. The test statistic is then similar to that used for the MES test n Hα, θt α/2 UC RM =, α 1/3 α/4 1 0 h t α, β, θ 0, X it 1 dβ. 48 with Hα, θ T = 1 n n H t α, θ T, X t 1. When there is no estimation risk, i.e., λ = 0, under assumptions A1-A4, d UC RM N 0, When T, n and n/t λ <, the UC RM test statistic has a non standard asymptotic distribution. However, by using the same approach as in Section 4, it is possible to derive a robust test statistic that has a standard normal distribution whatever the relative value of n and T. 16

17 6.2 Backtesting CoVaR Adrian and Brunnermeier 2016 proposed a measure for systemic risk, called the CoVaR, which is defined as the difference between the conditional VaR CoVaR of the financial system conditional on an institution being in distress and the CoVaR conditional on the median state of the institution. Contrary to the MES case, denote by Y 1t the return of a portfolio of financial institutions and by Y 2t the stock return of a given institution. Adrian and Brunnermeier define the stress of the institution as a case in which the return Y 2t is equal to V ar 2t α. This choice is justified by their estimation methodology based on a quantile regression model. A more general approach consists in defining the financial stress as a situation in which Y 2t in using truncated distributions Girardi and Ergun, V ar 2t α and Similarly, the median state of the institution can be represented by an interquartile range, V ar 2t β inf Y 2t V ar 2t β sup, with for instance β inf = 25% and β inf = 75%. Then, the CoVaR of the financial system corresponds to the quantity with β = β inf, β sup and CoVaR 1t α = CoV ar 1t α, α, θ 0 CoV ar 1t α, β, θ 0, 50 Pr Y 1t CoV ar 1t α, α, θ 0 Y 2t V ar 2t α, θ 0 ; Ω t 1 = α, Pr Y 1t CoV ar 1t α, β, θ 0 V ar 2t β inf, θ 0 Y 2t V ar 2t β sup, θ 0 ; Ω t 1 = α. 51 In order to backtest each of the two CoVaRs, we define two violations. These violation processes are represented by the following binary variables h t α, α, θ 0 = 1 Y 1t CoV ar 1t α, α, θ 0 Y 2t V ar 2t α, θ 0, h t α, β, θ 0 = 1 Y 1t CoV ar 1t α, β, θ 0 V ar 2t β inf, θ 0 Y 2t V ar 2t β sup, θ 0. If the risk model is well specified, the two centered violation processes satisfy E h t α, α, θ 0 α 2 Ωt 1 = 0 52 E h t α, β, θ 0 α β sup β inf Ω t 1 = 0 53 The intuition of the test is then similar to the backtests proposed for the multi-level VaR see Francq and Zakoian, 2015, for the estimation method, i.e., the VaR defined for a finite set of coverage rates Hurlin and Tokpavi, 2006; Pérignon and Smith, 2008; Leccadito, Boffelli and Urga, It consists in considering a vector a violations, denoted h t α, β, θ 0, with h t α, β, θ 0 = h t α, α, θ 0, h t α, β, θ Thus, a test for the validity of the two components of the CoVaR can be expressed as H 0,UC : E h t α, β, θ 0 = µ h = α 2, α β sup β inf. 55 The corresponding test statistic is defined as UC CoV ar = n hα, β, θ T µ h Vhα, β, θt 1 hα, β, θ T µ h, 56 17

18 with hα, β, θ T the out-of-sample mean of h t α, β, θ T, and Vhα, β, θ T a consistent estimator of the variance-covariance matrix of hα, β, θ 0. Notice that as soon as α < β inf, this matrix is diagonal since the covariance between the two violations h t α, α, θ 0 and h t α, β, θ 0 is null. When there is no estimation risk, i.e., when λ = 0, under assumptions A1-A4, d UC CoV ar χ When T, n and n/t λ <, test statistic UC CoV ar has an asymptotic distribution which is a weighted sum of chi-squared variables. However, by using the same approach as in Section 4, it is possible to derive a robust test statistic that has a chi-squared distribution with 2 degrees of freedom whatever the relative value of n and T. 7 Empirical Application - To be done 8 Conclusion This paper develops a backtesting procedure for systemic risk measures, since, to the best of our knowledge, this type of tool has not been yet computed. Inspired by Escanciano and Du 2015 who proposed recently a backtest procedure for the aggregate risk of the financial system as measured by ES, we extend this literature by defining a backtesting setup specifically designed for systemic risk measures. These tests are based on the concept of cumulative joint violation process. To fix the main concepts and give the intuition, we first introduce a backtesting procedure, which applies for the case of the MES. In the same spirit, we generalize this approach to more complex specifications e.g., the SRISK of Brownlees and Engle 2015, the CoVaR of Adrian and Brunnermeier 2016, etc.. This original procedure has many advantages. First, its implementation is easy since it only implies to evaluate a cdf of a bivariate distribution. Second, it allows for a separate test for unconditional coverage and independence hypothesis Christoffersen, Third, Monte-Carlo simulations show that for realistic sample sizes, our tests have good finite sample properties. Finally, we pay a particular attention to the consequences of the estimation risk and propose a robust version of our test statistics. 18

19 Table 1. Empirical rejection rates for backtesting MES0.05 at 5% significance level UC MES θ T UC C MES θ T IND MES θ T IND C MES θ T T = 250, n = 250, Size and Power size corrected H H A 1 σ 2 1 = 25% σ 2 1 = 50% H B 1 σ 2 2 = 25% σ 2 2 = 50% H C 1 ρ H1 = ρ H1 = T = 2500, n = 2500, Size and Power size corrected H H A 1 σ 2 1 = 25% σ 2 1 = 50% H B 1 σ 2 2 = 25% σ 2 2 = 50% H C 1 ρ H1 = ρ H1 =

20 Table 2. Empirical rejection rates for backtesting MES0.05 at 5% significance level UC MES θ T UC C MES θ T IND MES θ T IND C MES θ T T = 2500, n = 250, Size and Power size corrected H H A 1 σ 2 1 = 25% σ 2 1 = 50% H B 1 σ 2 2 = 25% σ 2 2 = 50% H C 1 ρ H1 = ρ H1 = T = 250, n = 2500, Size and Power size corrected H H A 1 σ 2 1 = 25% σ 2 1 = 50% H B 1 σ 2 2 = 25% σ 2 2 = 50% H C 1 ρ H1 = ρ H1 =

21 A Moments of the Cumulative Joint Violation Process Let us define the binary variable H t α, θ 0 = 1 u 12t θ 0 1 u 2t θ 0 α with u 2t θ 0 u 2t = F Y2 Y 2t ; Ω t 1, θ 0 and u 12t θ 0 u 12t = F Y1 Y 2 V ar 2tα,θ 0Y 1t ; Ω t 1, θ 0. The two first conditional moments of the cumulative joint process H t α, θ 0 are then given by E H t α, θ 0 Ω t 1 = Pr u 2t θ 0 α Ω t 1 E H t α, θ 0 u 2t θ 0 α, Ω t 1 = α α E u 12t θ 0 u 2t θ 0 α, Ω t 1, E H 2 t α, θ 0 Ωt 1 = α E 1 2u12t θ 0 + u 2 12t θ 0 u2t θ 0 α, Ω t 1. Since the conditional distribution of u 2t θ 0 given Ω t 1 is U [0,1] with E u 2t θ 0 Ω t 1 = 1/2 and E u 2 2t θ 0 Ωt 1 = 1/3, then we get E H t α, θ 0 Ω t 1 = α 2, B Proof of Theorem 1 1 V H t α, θ 0 Ω t 1 = α 3 α To derive the asymptotic properties of the statistic CC MES, we introduce the following assumptions. A1: The vectorial process Y t = Y 1t, Y 2t is strictly stationary and ergodic. A2: The marginal distribution of Y 2t is given by F Y2 Y 2t ; Ω t 1, θ 0 and the truncated distribution of Y 1t given Y 2t V ar 2t α, θ 0 is given by F Y1 Y 2 V ar 2tα,θ 0Y 1t ; Ω t 1, θ 0. A3: θ 0 Θ, with Θ a compact subspace of R p. A4: The estimator θ T is consistent for θ 0 and is asymptotically normally distributed such that: T θt θ 0 d N 0, Σ0, with Σ 0 a positive definite p p matrix. Denote V as θ T = Σ 0 /T. Proof. Denote H t α, θ = 1 u 12t θ 1 u 2t θ α the cumulative violation process, with u 2t θ = F Y2 Y 2t ; Ω t 1, θ and u 12t θ = F Y1 Y 2 V ar 2tα,θY 1t ; Ω t 1, θ, t = 1,..., n and θ Θ. Under the null hypothesis H 0,UC, the sequence {H t α, θ 0 α/2} n is a mds with σ2 H = V H t α, θ 0 = α 1/3 α/4. For simplicity, we assume that Ω t 1 only includes a finite number of lagged values of Y t, i.e. there is no information truncation. The test statistic UC ES can be rewriten as UC MES = 1 σ H n Under assumptions A1-A4, the continuous mapping theorem implies that 1 n n H t α, θ T E H t α, θ T Ω t 1 n = 1 n H t α, θ T α/2. 59 n H t α, θ 0 E H t α, θ 0 Ω t 1 + o p 1. 21

22 Rearranging these terms gives 1 n n H t α, θ T E H t α, θ 0 Ω t 1 The mean value theorem implies that = 1 n + 1 n n n H t α, θ 0 E H t α, θ 0 Ω t 1 60 E H t α, θ T H t α, θ 0 Ωt 1 + o p 1. H t α, θ T = H t α, θ 0 + Ht α, θt θ θ 0, 61 where θ is an intermediate point between θ 0 and θ T. Equation 60 becomes 1 σ H n n H t α, θ T α/2 = n 1 H t α, θ 0 α/2 62 σ H n λ + T θt θ 0 1 n H t α, E θ σ H n Ω t 1 + o p 1. Assume that T, n and n/t λ with 0 λ <. Under the null hypothesis H 0,UC, the first term on the right hand converges in distribution to a standard normal distribution. The covariance between the first term and T θ T θ 0 is 0 as θ T depends on the in-sample observations and the summand in the first term is for out-of-sample observations. Under assumption A4, θ p θ 0 and since H t α, θ 0 / is also a mds, we have 1 n H t α, E θ n Ω p Ht α, θ 0 t 1 R MES = E 0, 63 where E 0. denotes the expectation with respect to the true distribution of H t α, θ 0. So, we get λ 1 n H t α, E θ T σ H n Ω t 1 θt θ 0 d λ N 0, R MESΣ 0 R MES. 64 and finally d UC MES N 0, 1 + λ R MES Σ 0R MES. 65 α 1/3 α/4 σ 2 H C Computation of R MES in the Bivariate Normal Case Let us assume that Y t = Y 1t, Y 2t such that Y t = Σ 1/2 t ε t 66 where ε t = ε 1t, ε 2t are i.i.d. N 0, I 2, where I 2 denotes the 2 2 identity matrix and Σ t = Σ t θ 0 is the conditional variance covariance matrix of Y t given Ω t 1. Denote by f y, Σ t f y 1, y 2, Σ t the pdf and by F y, Σ t F y 1, y 2, Σ t the cdf of the joint distribution of Y t such that f y, Σ t = 1 2π Σ t 1 2 exp y Σ t y

23 y1 y2 F y, Σ t = Pr Y 1 y 1 Y 2 y 2 = f a, b, Σ t dadb 68 Using the Dwyer s formula 1967, we know that f y, Σ t = ln f y, Σ t f y, Σ t = f y, Σ t Σ 1 t Σ t Σ t 2 Σ 1 t yy Σ 1 t 69 and we get F y, Σ t Σ t = y1 y2 = Σ 1 t 2 F y, Σ t f a, b, Σ t dadb 70 Σ t y1 y2 f a, b, Σ t Σ 1 t a, b Σ 1 dadb with a, b a 2 2 matrix equal to a, b a, b. If the conditional variance covariance matrix is defined as σ 2 Σ t = 1t σ 12t σ 12t the matrix expression given in equation 70 can be decomposed as follows F y, Σ t σ 2 1t F y, Σ t σ 2 2t σ 2 2t = σ2 2t F y, Σ t + 1 y1 y2 σ 2 2 t 2 2 2t a 2 2σ 12t σ2tab 2 + σ12tb 2 2 f a, b, Σ t dadb t = σ2 1 F y, Σ t + 1 y1 y2 σ 2 2 t t a 2 2σ 12t σ1ab 2 + σ1tb 2 2 f a, b, Σ t dadb t F y, Σ t = σ 12t F y, Σ t σ 12t 2 t t y1 y2 σ 2 2t σ 12t a 2 + σ12t 2 + σ1tσ 2 2t 2 ab σ 2 1t σ 12t b 2 f a, b, Σ t dadb with t = σ 2 1tσ 2 2t σ 2 12t. If θ denotes the parameters vector of the conditional variance-covariance model CCC, DCC, etc., then for any vector θ Θ, we have F y, Σ t = F y, Σt vec Σ t vec Σt where vec. denotes the vectorization operator of a matrix. A consistent estimate of R MES is given by R MES = 1 αn n vec D Proof of Theorem 3 Proof. F ỹ t, Σ t Σ t θt vec Σt t θt 73 Define the lag-j autocovariance and autocorrelation of the cumulative joint violation H t α, θ 0 for j 0 by ρ j = γ j γ 0 and γ j = 1 n j n +j H t α, θ 0 α/2 H t j α, θ 0 α/2, 74 23

24 { } n respectively. The sample counterparts of ρ j and γ j based on the sample u 12t θ T, u 2t θ T are ˆρ j = γ j γ 0 and γ j = 1 n j n +j H t α, θ T α/2 H t j α, θ T α/2. 75 The sketch of the proof is similar to that used for the Theorem 1. Under assumptions A1-A4, the continuous mapping theorem implies n γj E γ j Ω t 1 = n γ j E γ j Ω t 1 + o p Since the sequence {H t α, θ 0 α/2} n is a mds, we have E γ j Ω t 1 = 0 for j 0 and E γ 0 Ω t 1 = α 1/3 α/4. Rearranging these terms gives n γj γ j = n E γ j Ω t 1 E γ j Ω t 1 + o p The mean value theorem implies that γ j = γ j + γj α, θt θ θ 0, 78 where θ is an intermediate point between θ 0 and θ T. Equation 77 becomes n γj γ j = λ T θ T θ 0 γ j α, E θ Ω t 1 + o p Under assumption A4, θ p θ 0 and γ j α, E θ Ω p γj α, θ 0 t 1 E Ω t 1 Notice that γj α, θ 0 E Ω t 1 = 1 n j n E H t j α, θ 0 α/2 H tα, θ 0 Ω t 1 +j since E H t α, θ 0 α/2 H t j α, θ 0 / Ω t 1 = H t j α, θ 0 / E H t α, θ 0 α/2 Ω t 1 = 0 for j > 0. So, when T and n, we have 1 n γ j α, E θ n j Ω p t 1 R j γ 0, 82 +j with γ 0 = α 1/3 α/4 and γj α, θ 0 R j γ 0 = E Hence, we proved that = E H t j α, θ 0 α/2 H tα, θ 0, 83 n γj γ j = λ T θ T θ 0 R j γ 0 + o p We then have n ρj ρ j = n γ j γ j γ 0 + o p 1 = R j λ T θt θ 0 + o p