Comparing the Accuracy of Copula-Based Multivariate Density Forecasts in Selected Regions of Support

Transcription

1 Comparing the Accuracy of Copula-Based Multivariate Density Forecasts in Selected Regions of Support Cees Diks CeNDEF, Amsterdam School of Economics University of Amsterdam Valentyn Panchenko School of Economics University of New South Wales Oleg Sokolinskiy Rutgers Business School Rutgers University Dick van Dijk Econometric Institute Erasmus University Rotterdam January 25, 2012 Abstract This paper develops a testing framework for comparing the predictive accuracy of copula-based multivariate density forecasts, focusing on a specific part of the joint distribution. The test is framed in the context of the Kullback-Leibler Information Criterion, and using (out-of-sample conditional likelihood and censored likelihood in order to restrict the evaluation to the region of interest. Monte Carlo simulations show that the resulting test statistics have satisfactory size and power properties in small samples. In an empirical application to daily exchange rate returns we find evidence that the dependence structure varies with the sign and magnitude of returns, such that different parametric copula models achieve superior forecasting performance in different regions of the copula support. Our analysis highlights the importance of allowing for lower and upper tail dependence for accurate forecasting of common extreme appreciation and depreciation of different currencies. Keywords: Copula-based density forecast; Kullback-Leibler Information Criterion; out-of-sample forecast evaluation. JEL Classification: C12; C14; C32; C52; C53 Corresponding author: Center for Nonlinear Dynamics in Economics and Finance, Faculty of Economics and Business, University of Amsterdam, Roetersstraat 11, NL-1018 WB Amsterdam, The Netherlands. c.g.h.diks@uva.nl School of Economics, Faculty of Business, University of New South Wales, Sydney, NSW 2052, Australia. v.panchenko@unsw.edu.au Rutgers Business School. oleg.sokolinskiy@business.rutgers.edu Econometric Institute, Erasmus University Rotterdam, P.O. Box 1738, NL-3000 DR Rotterdam, The Netherlands. djvandijk@ese.eur.nl

2 1 Introduction The dependence between asset returns typically is nonlinear and time-varying. Traditionally, efforts to accommodate these features have focused on modeling the dynamics of conditional correlations (or covariances by means of multivariate GARCH and stochastic volatility (SV models; see the surveys by Silvennoinen and Teräsvirta (2009 and Chib et al. (2009, respectively. Recently, copulas have become an increasingly popular tool for modeling multivariate distributions in finance, see Patton (2009 and Genest et al. (2009. The copula approach provides more flexibility than multivariate GARCH and SV models in terms of the type of asymmetric dependence that can be captured. In addition, an attractive property of copulas is that they allow for modeling the marginal distributions and the dependence structure of the asset returns separately. Many parametric copula specifications (or families are available, with rather different dependence properties. Therefore, an important issue in empirical applications is the choice of an appropriate copula family. In practice, this is usually addressed by estimating a number of econometric models with alternative copula specifications, and comparing them indirectly by subjecting each of them to a number of goodness-of-fit tests, see Berg (2009 for a detailed review. A more direct way to compare alternative copulas from different parametric families has been considered by Chen and Fan (2006 and Patton (2006, adopting the approach based on pseudo likelihood ratio (PLR tests for model selection originally developed by Vuong (1989 and Rivers and Vuong (2002. These tests compare the candidate copula specifcations in terms of their Kullback-Leibler Information Criterion (KLIC, which measures the distance from the true (but unknown copula. Similar to the goodness-of-fit tests, these PLR tests are based on the in-sample fit of the competing copulas. Diks et al. (2010 approach the copula selection problem from an out-of-sample forecasting perspective. Specifically, the PLR testing approach is extended to compare the predictive accuracy of alternative copulas, by using out-of-sample log-likelihood values corresponding with copula density forecasts. An important motivation for considering the 1

3 (relative predictive accuracy of copulas is that multivariate density forecasting is one of the main purposes in empirical applications. Comparison of out-of-sample KLIC values for assessing relative predictive accuracy has recently become popular for the evaluation of univariate density forecasts, see Mitchell and Hall (2005, Amisano and Giacomini (2007 and Bao et al. (2007. Amisano and Giacomini (2007 provide an interpretation of the KLIC-based comparison in terms of scoring rules, which are loss functions depending on the density forecast and the actually observed data. The expected difference between the log-likelihood score for two competing density forecasts corresponds exactly to their relative KLIC values. The same interpretation holds for the copula-based multivariate density forecasts considered here. In many applications of density forecasts, we are mainly interested in a particular region of the density. Financial risk management is an example in case. Due to the regulations of the Basel accords, among others, the main concern for banks and other financial institutions is an accurate description of the left tail of the distribution of their portfolio s returns, in order to obtain accurate estimates of Value-at-Risk and related measures of downside risk. Correspondingly, Bao et al. (2004, Amisano and Giacomini (2007 and Diks et al. (2011 consider the problem of evaluating and comparing univariate density forecasts in a specific region of interest. Diks et al. (2011 demonstrate that the approach based on out-of-sample KLIC values can be adapted to this case, by replacing the full likelihood by the conditional likelihood, given that the actual observation lies in the region of interest, or by the censored likelihood, with censoring of the observations outside the region of interest. In this paper we develop tests of equal predictive accuracy of different copula-based multivariate density forecasts in a specific region of the support. For this purpose, we combine the testing framework for comparing univariate forecasts in specific regions developed by Diks et al. (2011, with the logarithmic score decomposition for copula models considered in Diks et al. (2010. The resulting test of equal predictive accuracy can be 2

4 applied to fully parametric, semi-parametric and nonparametric copula-based multivariate density models. The test is valid under general conditions on the competing copulas. This is achieved by adopting the framework of Giacomini and White (2006, under which the estimation of unknown model parameters is considered as part of the forecast method. To enable this interpretation, a fixed-length finite estimation window is used to construct the forecasts based on different copula families. Comparing scores for forecast methods rather than for models simplifies the resulting test procedures considerably, because parameter estimation uncertainty does not play a role (it essentially is part of the respective competing forecast methods. In addition, the asymptotic distribution of our test statistic in this case does not depend on whether the competing copulas belong to nested families or not. We examine the size and power properties of our copula predictive accuracy test via Monte Carlo simulations. We adopt the framework of semi-parametric copula-based multivariate dynamic (SCOMDY models developed in Chen and Fan (2005, 2006, which combines parametric specifications for the conditional mean and conditional variance with a semi-parametric specification for the distribution of the (standardized innovations, consisting of a parametric copula with nonparametric univariate marginal distributions. Our simulation results demonstrate that the predictive accuracy tests have satisfactory size and power properties in realistic sample sizes. We consider an empirical application to daily exchange rate returns of the Canadian dollar, Swiss franc, euro, British pound, and Japanese yen against the US dollar over the period from 1992 until Based on the relative predictive accuracy of one-step ahead density forecasts we find that different parametric copula specifications achieve superior forecasting performance in different regions of the support. Our analysis highlights the importance of accommodating positive upper (lower tail dependence for accurate forecasting of common extreme appreciation (depreciation of different currencies. The paper is organized as follows. In Section 2 we propose weighted scoring rules for 3

5 evaluating multivariate density forecasts, as well as a formal test for comparing competing density forecasts. Section 3 then describes how this general framework can be applied within a copula comparison setting, by choosing weight functions defined on the copula domain. In Section 4 we investigate the copula tests size and power properties by means of Monte Carlo simulations. In Section 5 we illustrate our test with an application to daily exchange rate returns for several major currencies. We conclude in Section 6. 2 Multivariate density forecast evaluation with weighted scoring rules This section extends the weighted likelihood-based scoring rules for univariate density forecast evaluation, proposed by Diks et al. (2011, to the multivariate density forecast setting. Density forecast evaluation Consider a stochastic process {Z t : Ω R d+k } T t=1, defined on a complete probability space (Ω, F, P, where Z t = (Y t, X t with Y t : Ω R d being the real valued d-dimensional random variable of interest and X t : Ω R k a vector of exogenous or pre-determined variables. The information set at time t is defined as F t = σ(z 1,..., Z t. We consider the case where two competing forecast methods are available, each producing one-step ahead density forecasts, i.e. predictive densities of Y t+1 based on F t. The predictive probability density functions (pdfs of these forecasts are denoted by ˆf A,t (y and ˆf B,t (y, respectively. As in Giacomini and White (2006, by forecast method we mean that the given density forecast results from various choices that the forecaster makes at the time of the prediction. These include the variables X t, the econometric model (if any, and the estimation method. The only requirement that we impose on the forecast methods is that the density forecasts depend on a finite number R of most recent observations 4

6 Z t R+1,..., Z t. Forecast methods of this type arise naturally, for instance, when density forecasts are obtained from time series models, for which parameters are estimated with a moving window of R observations. The advantage of comparing forecast methods rather than forecast models is that this allows for treating parameter estimation uncertainty as an integral part of the forecast methods. The use of a finite (rolling window of R past observations for parameter estimation considerably simplifies the asymptotic theory of tests of equal predictive accuracy, as argued by Giacomini and White (2006. It also turns out to be more convenient in that it enables comparison of density forecasts based on both nested and non-nested models, in contrast to alternative approaches such as West (1996. Scoring rules One of the approaches that has been put forward for density forecast evaluation in general is by means of scoring rules, which are commonly used in probability forecast evaluation, see Diebold and Lopez (1996. A scoring rule is a loss function S ( ˆf t ; y t+1 depending on the density forecast and the actually observed value y t+1, such that a density forecast that is better receives a higher score. Note that, as argued by Diebold et al. (1998 and Granger and Pesaran (2000, any rational user would prefer the true conditional density p t of Y t+1 over an incorrect density forecast. This suggests that it is natural to focus on scoring rules for which incorrect density forecasts ˆf t do not receive a higher average score than the true conditional density p t, that is, E t (S ( ˆf t ; Y t+1 E t (S (p t ; Y t+1, for all t. Following Gneiting and Raftery (2007, a scoring rule satisfying this condition will be called proper. It is useful to note that the correct density p t does not depend on estimated parameters, while density forecasts typically do. This implies that even if the density forecast ˆf t is based on a correctly specified model, but the model includes estimated parameters, the average score E t (S ( ˆf t ; Y t+1 may not achieve the upper bound E t (S (p t ; Y t+1 due 5

7 to nonvanishing estimation uncertainty. As a consequence, a density forecast based on a misspecified model with limited estimation uncertainty may be preferred over a density forecast based on the correct model specification but having larger estimation uncertainty. Null hypothesis and testing approach Given a scoring rule of one s choice, there are various ways to construct tests of equal predictive ability. Giacomini and White (2006 distinguish tests of unconditional predictive ability and conditional predictive ability. In the present paper, we focus on tests for unconditional predictive ability for clarity of exposition. The suggested approach can be extended to obtain tests of conditional predictive ability in a straightforward manner. Assume that two competing density forecasts ˆf A,t and ˆf B,t and corresponding realizations of the variable Y t+1 are available for t = R, R + 1,..., T 1 and let P = T R. We may then compare ˆf A,t and ˆf B,t based on their scores, by testing formally whether their difference is statistically significantly different from zero on average. Defining the score difference d t+1 = S ( ˆf A,t ; y t+1 S ( ˆf B,t ; y t+1, for a given scoring rule S, the null hypothesis of equal scores is given by H 0 : E(d t+1 = 0, for all t = R, R + 1,..., T 1. To test this null hypothesis, we may use a Diebold and Mariano (1995 type statistic t R,P = d R,P ˆσ 2R,P /P, (1 where d R,P denotes the sample average of the score differences, that is, d R,P = P 1 T 1 t=r d t+1, and ˆσ R,P 2 is a heteroskedasticity and autocorrelation-consistent (HAC variance estimator ( P of σr,p 2 = Var d R,P. The following theorem characterizes the asymptotic distribu- 6

8 tion of the test statistic under the null hypothesis. Theorem 1 The statistic t R,P in (1 is asymptotically (as P with R fixed standard normally distributed under the null hypothesis if: (i {Z t } is φ-mixing of size q/(2q 2 with q 2, or α-mixing of size q/(q 2 with q > 2; (ii E d t+1 2q < for all t; and ( P (iii σr,p 2 = Var d R,P > 0 for all P sufficiently large. Proof: This is Theorem 4 of Giacomini and White (2006, where a proof can also be found. The logarithmic scoring rule Mitchell and Hall (2005, Amisano and Giacomini (2007, Bao et al. (2004, 2007 and Diks et al. (2010, among others, focus on the logarithmic scoring rule S l ( ˆf t ; y t+1 = log ˆf t (y t+1, (2 such that the score assigned to a density forecast varies positively with the value of ˆf t evaluated at the observation y t+1. Based on the P observations available for evaluation, y R+1,..., y T, the density forecasts ˆf A,t and ˆf B,t can be ranked according to their average scores P 1 T 1 t=r log ˆf A,t (y t+1 and P 1 T 1 t=r log ˆf B,t (y t+1. Obviously, the density forecast yielding the highest average score would be the preferred one. The log score differences d l t+1 = log ˆf A,t (y t+1 log ˆf B,t (y t+1 may be used to test whether the predictive accuracy is significantly different, using the test statistic defined in (1. Note that this coincides with the log-likelihood ratio of the two competing density forecasts. The log-likelihood score S l ( ˆf t ; y t+1 is a proper scoring rule. This can be understood by noting that the expected value of the log score is proportional to the KLIC value of the density forecast, given by KLIC( ˆf t = ( p t (y log p t (y/ ˆf t (y dy = E t (logp t E t (log ˆf t. It is straightforward to show that KLIC( ˆf t 0 or E t (log ˆf t E t (logp t for any density 7

9 ˆf t, such that the log scoring rule is proper. Weighted likelihood-based scoring rules The most obvious way to adapt the logarithmic scoring rule for evaluating and comparing density forecasts in a specific region of interest, M t R d, say, would be to consider the weighted logarithmic score I(y t+1 M t log ˆf t (y t+1, as suggested by Amisano and Giacomini (2007. However, as argued by Diks et al. (2011 and?, by construction the resulting test statistic would be biased towards (possible incorrect density forecasts with more probability mass in the region of interest. This reflects the fact that the weighted log scoring rule is not proper. As shown by Diks et al. (2011 in the context of univariate density forecasts, replacing the full likelihood in (2 either by the conditional likelihood, given that the observation lies in the region of interest, or by the censored likelihood, with censoring of the observations outside M t, does lead to scoring rules which do not suffer from this problem and remain proper. The conditional likelihood (cl score function is given by ( S cl ( ˆf ˆft (y t ; y t+1 = I(y t+1 M t log t+1, (3 ˆft (ydy while the censored likelihood (csl score function is given by ( S csl ( ˆf t ; y t+1 = I(y t+1 M t log ˆf t (y t+1 +(1 I(y t+1 M t log 1 M t M t ˆft (ydy. Note that the cl scoring rule does not take into account the accuracy of the density forecast for the total probability of Y t+1 falling into the region of interest, while the csl scoring rule does. The conditional and censored likelihood scoring rules focus on a sharply defined region of interest M t. It is possible to extend this idea by using a more general weight function w t (y t+1, where the scoring rules in (3 and (4 can be recovered for the specific (4 8

10 choice w t (y t+1 = I(y t+1 M t : ( S cl ( ˆf ˆf t (y t ; y t+1 = w t (y t+1 log t+1 wt (y ˆf, (5 t (ydy and ( S csl ( ˆf t ; y t+1 = w t (y t+1 log ˆf t (y t+1 + (1 w t (y t+1 log 1 w t (y ˆf t (ydy. At this point, we make the following assumptions concerning the density forecasts that are to be compared, and the weight function. (6 Assumption 1 The density forecasts ˆf A,t and ˆf B,t satisfy KLIC( ˆf A,t < and KLIC( ˆf B,t <, where KLIC(h t = p t (y log (p t (y/h t (y dy is the Kullback-Leibler divergence between the density forecast h t and the true conditional density p t. Assumption 2 The weight function w t (y is such that (a it is determined by the information available at time t, and hence a function of F t, (b 0 w t (y 1, and (c wt (yp t (y dy > 0. Assumption 1 ensures that the expected score differences for the competing density forecasts are finite. Assumption 2 (c is needed to avoid cases where w t (y takes strictly positive values only outside the support of the data. The following lemma states that the generalized cl and csl scoring rules in (5 and (6 are proper, and hence cannot lead to spurious rejections against incorrect alternatives just because these have more probability mass in the region(s of interest (as in the case of the weighted log scoring rule. Lemma 1 Under Assumptions 1 and 2, the generalized conditional likelihood scoring rule given in (5 and the generalized censored likelihood scoring rule given in (6 are proper. 9

11 The proof of this Lemma is given in Appendix A. The proof clarifies that the scoring rules in (5 and (6 can be interpreted in terms of Kullback-Leibler divergences between weighted versions of the density forecast and the actual density. We may test the null hypothesis of equal performance of two density forecasts ˆf A,t (y t+1 and ˆf B,t (y t+1 based on the conditional likelihood score (5 or the censored likelihood score (6 in the same manner as before. That is, given a sample of density forecasts and corresponding realizations for P time periods t = R, R + 1,..., T 1, we may form the relative scores d cl t+1 = S cl ( ˆf A,t ; y t+1 S cl ( ˆf B,t ; y t+1 and d csl t+1 = S csl ( ˆf A,t ; y t+1 S csl ( ˆf B,t ; y t+1 and use these for computing Diebold-Mariano type test statistics as given in (1. 3 Copula comparison with weights on the copula domain We proceed by focusing on the comparison of two multivariate density forecasts that are obtained using a copula approach. Specifically, we consider predictive densities differing only in their copulas, and use weighted likelihood-based scores with weight functions defined on the copula domain. Patton s (2006 extension of Sklar s (1959 theorem to the time-series case describes how the time-dependent multivariate cumulative distribution function (CDF F t (y t+1 can be decomposed into conditional marginal CDFs F j,t (y j,t+1, j = 1,..., d, and a conditional copula C t (, that is F t (y t+1 = C t (F 1,t (y 1,t+1, F 2,t (y 2,t+1,..., F d,t (y d,t+1. (7 The attractiveness of the copula approach for modeling multivariate distributions follows directly from the decomposition in (7. It provides a clear separation between univariate information on the individual variables Y j,t+1, contained in the marginal distributions F j,t, j = 1..., d, and their dependence, which is governed completely by the copula 10

12 function C t. As the choice of marginal distributions does not restrict the choice of dependence function, or vice versa, a wide range of joint distributions can be obtained by combining different marginals with different copulas. The predictive log-likelihood associated with y t+1 is seen to be given by d log f j,t (y j,t+1 + log c t (F 1,t (y 1,t+1, F 2,t (y 2,t+1,..., F d,t (y d,t+1, (8 j=1 where f j,t (y j,t+1, j = 1,..., d, are the conditional marginal densities and c t is the conditional copula density, defined as c t (u 1, u 2,..., u d = d u 1 u 2... u d C t (u 1, u 2,..., u d, which we will assume to exist throughout. Using (8, the conditional likelihood and censored likelihood scoring rules in (5 and (6 can be decomposed as S cl t+1( ˆf t ; y t+1 = w t (y t+1 and ( d log j=1 ( ( ˆfj,t (y j,t+1 wt (y ˆf + log t (y dy ( d St+1( csl ˆf t ; y t+1 = w t (y t+1 log ˆf j,t (y j,t+1 + log ĉ t (û t+1 j=1 ( + (1 w t (y t+1 log 1 ĉ t (û t+1 wt (y ˆf t (y dy, (9 w t (y ˆf t (y dy, (10 where ĉ t is the conditional copula density associated with the density forecast ˆft, and û t+1 = ( ˆF 1,t (y 1,t+1,..., ˆF d,t (y d,t+1 the multivariate conditional probability integral transform (PIT according to the density forecast ˆf t. As in Diks et al. (2010 we assume that the two competing multivariate density forecasts differ only in their copula specifications and have identical predictive marginal den- 11

13 sities ˆf j,t, j = 1,..., d. The copulas of the two competing density forecasts are assumed to have well-defined densities ĉ A,t and ĉ B,t. The null hypothesis of equal predictive ability is H 0 : E(SA,t+1 = E(S B,t+1, as before. Since the conditional marginals are identically specified under both density forecasts, the logarithms of the marginal densities in (9 and (10 cancel out, so that an equivalent formulation of the null hypothesis is H 0 : E(S A,t+1 = E(S B,t+1, where, with a similar abuse of notation as above (leaving out the subscripts A and B, S cl t+1( ˆf t ; y t+1 = w t (y t+1 log ( ĉ(û t+1 wt (y ˆf, t (y dy and ( St+1( csl ˆf t ; y t+1 = w t (y t+1 log ĉ t (û t+1 + (1 w t (y t+1 log 1 w t (y ˆf t (y dy. These scores still allow for a general weight function w t (y t+1. Our aim here is to use the weight function to focus on specific regions of the copula. This can be achieved by taking weight functions of the form w t (y t+1 = w t (û 1,t+1 (y 1,t+1,..., û d,t+1 (y d,t+1, where w t (u 1,..., u d is a weight function defined on the copula support. Note that 12

14 w t (y ˆf t (y dy = w t (û 1,t+1 (y 1,..., û d,t+1 (y d ˆf t (y dy = w t (û 1,t+1 (y 1,..., û d,t+1 (y d d ˆF t (y = w t (û 1,t+1,..., û d,t+1 dĉt(û t+1 = w t (uĉ t (u du. This allows us to rewrite the scores S t+1 as S cl t+1( ˆf t ; y t+1 = w t (û t+1 log ( ĉ t (û t+1 wt (uĉ t (udu, (11 and ( St+1( csl ˆf t ; y t+1 = w t (û t+1 (log ĉ t (û t+1 + (1 w t (û t+1 log 1 w t (uĉ t (udu. (12 Note that these reduced scoring rules take the same form as the weighted likelihoodbased scoring rules (5 and (6 derived before, but now involving the copula density ĉ t, associated with the density forecast, instead of the full density forecast and the observed conditional PITs û t+1 instead of the variable y t+1. The weight function w t (u can be chosen directly in the copula support. In the cases considered in this paper, w t will be time independent, and will take the form of an indicator function of a given fixed subset of the copula support. In some cases this allows for a simplification of the scoring rules. For instance, for w t (u = I(u 1 a,... u d a, it follows that w t (uĉ t (u du = Ĉt(a,..., a, so that the reduced scoring rules take the form S cl ( ˆf t ; y t+1 = I(u 1,t+1 a,... u d,t+1 a log ( ĉ t (û t+1 Ĉ t (a,..., a 13

15 and S csl ( ˆf t ; y t+1 = I(u 1,t+1 a,... u d,t+1 a (log ĉ t (û t+1 + (1 I(u 1,t+1 a,... u d,t+1 a log ( 1 Ĉt(a,..., a. Again, the Diebold-Mariano type test statistics as given in (1 may be adopted to test the null hypothesis of equal predictive accuracy of two copula-based density forecasts ĉ A,t (û t+1 and ĉ B,t (û t+1 based on the conditional likelihood score (11 or the censored likelihood score (12 in the same manner as before. In the above we assume that the two competing multivariate density forecasts differ only in their copula specifications and have identical predictive marginal densities ˆf j,t, j = 1,..., d. Implicitly this assumes that the parameters in the marginals and the copula can be separated from each other, so that they can be estimated in a multi-stage procedure. No other restrictions are put on the marginals. In particular, they may be specified parametrically, nonparametrically, or semi-parametrically. An important class of models that satisfies these properties is that of SCOMDY models, discussed next. SCOMDY models The class of semi-parametric copula-based multivariate dynamic (SCOMDY models has been introduced by Chen and Fan (2006. We discuss this class in some detail here, as we use it in the Monte Carlo simulations and the empirical application in subsequent sections. The SCOMDY models combine parametric specifications for the conditional mean and conditional variance of Y t+1 with a semi-parametric specification for the distribution of the (standardized innovations, consisting of a parametric copula with nonparametric univariate marginal distributions. The general SCOMDY model is specified as Y t+1 = µ t (θ 1 + H t (θε t+1, (13 14

16 where µ t (θ 1 = (µ 1,t (θ 1,..., µ d,t (θ 1 = E [Y t+1 F t ] is a specification of the conditional mean, parametrized by a finite dimensional vector of parameters θ 1, and H t (θ = diag(h 1,t (θ,..., h d,t (θ, where h j,t (θ = h j,t (θ 1, θ 2 = E [ (Y j,t+1 µ j,t (θ 1 2 F t ], j = 1,..., d, is the conditional variance of Y j,t+1 given F t, parametrized by a finite dimensional vector of parameters θ 2, where θ 1 and θ 2 do not have common elements. The innovations ε t+1 = (ε 1,t+1,..., ε d,t+1 are independent of F t and independent and identically distributed (i.i.d. with E(ε j,t+1 = 0 and E(ε 2 j,t+1 = 1 for j = 1,..., d. Applying Sklar s theorem, the joint distribution function F (ε of ε t+1 can be written as F (ε = C(F 1 (ε 1,..., F d (ε d ; α C(u 1,..., u d ; α, (14 where C(u 1,..., u d ; α: [0, 1] d [0, 1] is a member of a parametric family of copula functions with finite dimensional parameter vector α. An important characteristic of SCOMDY models is that the univariate marginal densities F j (, j = 1,..., d are not specified parametrically (up to an unknown parameter vector but are estimated nonparametrically. Specifically, Chen and Fan (2006 suggest the following three-stage procedure to estimate the SCOMDY model parameters. First, estimates of the parameters θ 1 and θ 2 are obtained by means of univariate quasi maximum likelihood (assuming normality of the standardized innovations ε j,t+1. Second, the empirical CDF transformation of the standardized residuals ˆε j,t+1 (z j,t+1 µ j,t (ˆθ 1 / h j,t (ˆθ is used to estimate the marginal distributions F j (. Finally, the pa- 15

17 rameters of a given copula specification are estimated by maximizing the corresponding copula log-likelihood function. 4 Monte Carlo simulations In this section we use Monte Carlo simulation to examine the finite-sample behavior of our predictive accuracy test for comparing alternative copula specifications in specific regions of interest. We use the SCOMDY model (13 as data generating process (DGP. In all experiments, we use an AR(1 specification for the conditional means and a GARCH(1,1 specification for the conditional variances with coefficients that are typical for daily exchange rates, in particular, Y j,t = 0.1Y j,t 1 + h j,t ε j,t (15 h j,t = (Y j,t Y j,t h j,t 1, (16 for j = 1,..., d. The innovations ε j,t are i.i.d. and drawn from the univariate Student-t distributions with 5 degrees of freedom, which are standardized to have variance equal to 1. In the size and power experiments reported below, we use the Gaussian copula, the Student-t copula, the Clayton copula and the Clayton survival copula. The Gaussian and Student-t copulas can be obtained using the so-called inversion method, that is C Ga (u 1, u 2,..., u d = F (F 1 1 (u 1, F 1 2 (u 2,..., F 1 d (u d, (17 where F is the joint CDF and F 1 i (u = min{x u F i (x} is the (quasi-inverse of the corresponding marginal CDF F i. The Gaussian copula is obtained from (17 by taking F to be the multivariate normal 16

18 distribution with mean zero, unit variances, and correlations ρ ij, i, j = 1,..., d, and standard normal marginals F i. The corresponding copula density is given by ( c Ga (u; Σ = Σ 1/2 exp 1 2 (Φ 1 (u (Σ 1 I d Φ 1 (u, (18 where I d is the d-dimensional identity matrix, Σ is the correlation matrix, and Φ 1 (u = (Φ 1 (u 1,..., Φ 1 (u d, with Φ 1 ( denoting the inverse of the standard normal CDF. In the bivariate case d = 2, the correlation coefficient ρ 12 = ρ 21 is the only parameter of the Gaussian copula. The Student-t copula is obtained similarly, but using a multivariate Student-t distribution instead of the Gaussian. The corresponding copula density is given by c St t (u, Σ, ν = Σ 1/2 Γ([ν + d]/2γ d 1 (ν/2 Γ d ((ν + 1/2 where T 1 ν ( 1 + T ν 1 d i=1 ( 1 + (ν+d/2 (u Σ 1 Tν 1 (u ν 1 (Tν (u i 2 (ν+1/2, ν (19 (u = (T 1 (u 1,..., T 1 (u d, and T 1 ( is the inverse of the univariate ν ν Student-t CDF, Σ is the correlation matrix and ν is the number of degrees of freedom. In the bivariate case the Student-t copula has two parameters, the number of degrees of freedom ν and the correlation coefficient ρ 12. Note that the Student-t copula nests the Gaussian copula when ν =. A major difference between the Gaussian copula and the Student-t copula is their ability to capture tail dependence, which may be important for financial applications. For the Gaussian copula both tail dependence coefficients are equal to zero, while for the Studentt copula the tail dependence is symmetric and positive. Specifically, in the bivariate case d = 2, the tail dependence coefficients are given by λ L = λ U = 2T ν+1 ( (ν + 1(1 ρ 12 /(1 + ρ 12, ν 17

19 which is increasing in the correlation coefficient ρ 12 and decreasing in the degrees of freedom ν. The Clayton and Clayton survival copulas belong to the family of Archimedean copulas (see Nelsen (2006 for details. The d-dimensional Clayton copula is given by C Cl (u 1, u 2,..., u d ; α = ( d j=1 u α j d + 1 1/α, with α > 0. In contrast to the Gaussian and Student-t copulas, the Clayton copula is able to capture asymmetric tail dependence. In fact, it only exhibits lower tail dependence, while upper tail dependence is absent. In the bivariate case the lower tail dependence coefficient for the Clayton copula is λ L = 2 1/α, which is increasing in the parameter α. The density function of the Clayton copula is ( d ( d c Cl (u, α = (1 + (j 1α j=1 j=1 u (α+1 j ( d j=1 u α j d + 1 (α 1 +d The Clayton survival copula is obtained as a mirror image of the Clayton copula, with its density function given by c Cl-s (u, α = c Cl (1 u, α.. Consequently, in the bivariate case the upper tail dependence coefficient for the Clayton survival copula is λ U = 2 1/α, and is increasing in the parameter α, while the lower tail dependence coefficient is zero. In the simulation experiments we consider bivariate and trivariate cases, i.e., d = 2 and 3. We use the same number of observations for the moving in-sample window and for the out-of-sample forecasting period, and set R = P = 1, 000. The SCOMDY model is estimated using the three-step procedure outlined at the end of Section 3. Having estimated the parameters in the marginal models given by (15 and (16, the in-sample PITs, Û s for s = t R + 1,..., t, are obtained from the empirical CDF transformation 18

20 Û j,s = R j,s R+1 where R j,s is the rank of ˆε j,s among the residuals ˆε j,t R+1,..., ˆε j,t, for j = 1,..., d. These are then used to estimate the copula parameters and, finally, the out-ofsample PIT Û t+1, corresponding to the one-step ahead forecast errors ˆε j,t+1 t, are obtained from the empirical CDF transformation based on its rank among the in-sample residuals. The number of replications in each experiment is set equal to 1, Size In order to assess the size properties of the test, a case is required with two competing copulas that are both equally (incorrect. We achieve this with the following set-up. We consider two different DGPs. First, we take the innovations ε j,t, j = 1,..., d, in the SCOMDY model (15 and (16 to be independent. Second, we consider a DGP with a Student-t copula with correlation coefficients ρ i,j = 0.3 and degrees of freedom ν = 6. For both DGPs, we test the null hypothesis of equal predictive accuracy of Clayton and Clayton survival copulas with its parameter being either fixed (α = 1.5 or estimated using the moving in-sample window. In both cases, the two competing copula specifications are equally distant from the true copula. We conduct the predictive accuracy tests for the copula support region [0.25, 0.75] d (where d = 2, 3 using censored and conditional scores. Results for the test of equal predictive accuracy on the full copula support, as in Diks et al. (2010, are included for comparison. [Table 1 about here.] The observed rejection rates of the test for nominal sizes 0.01, 0.05 and 0.10 are reported in Table 1, where the null hypothesis is tested against the two-sided alternative that the average scores of the two copulas are not equal. In Panels I and II the PITs are obtained using the empirical cumulative distribution functions (ECDFs, while in Panel III the marginal distributions are assumed to be known exactly. In the absence of any parameter estimation uncertainty about the marginals, we observe that the empirical rejection rate 19

21 is close to the nominal size. However, when the ECDF procedure is used the observed size deviates from the nominal significance level. The tests based on the censored and conditional scores give similar results. The size properties of both tests compare favorably against the Diks et al. (2011 test of copula predictive accuracy for the full support. 4.2 Power We evaluate the power of the test of equal predictive accuracy by performing a simulation experiment where one of the competing copula specifications corresponds with the DGP, while the distance of the alternative, incorrect copula specification to the DGP varies depending on a certain parameter of the DGP. Specifically, the DGP is a Student-t copula of dimension d = 3 with all correlation coefficients set to ρ ij = 0.3, i, j = 1, 2, 3. The number of degrees of freedom ν is varied over the interval [5, 50]. We compare the predictive accuracy of the correct Student-t copula specification (albeit both parameters Σ and ν are being estimated against an incorrect Gaussian copula specification (with the correlation matrix Σ being estimated in the left tail region [0, 0.25] d. Hence, we focus on the question whether the suggested tests can distinguish between copulas with and without tail dependence. Note, however, that the Student-t copula approaches the Gaussian copula as ν increases, and the tail dependence disappears with the coefficients λ L and λ U converging to zero. Intuitively, the higher the value of ν in the DGP, the more difficult it is to distinguish between these two copula specifications. [Figure 1 about here.] The results are shown in Figure 1 in the form of power plots, showing the observed rejection rates (for nominal sizes of 0.01, 0.05, 0.1 as a function of the degrees of freedom parameter, ν, in the DGP, for tests based on the censored and conditional scoring rule. The results displayed are for the null hypothesis that the Gaussian and Student-t copulas perform equally well, against the one-sided alternative hypothesis that the correctly specified Student-t copula has a higher average score. Intuitively, since the true DGP uses the 20

22 Student-t copula, we might expect the Student-t copula to perform better. Note, however, that as the number of the degrees of freedom ν in the copula describing the DGP becomes large, the Gaussian copula might outperform the Student-t copula. This is a consequence of the fact that the Gaussian copula is almost equivalent to the Student-t copula for large values of ν, but requires one parameter (ν less to be estimated. Indeed, the rejection rates become smaller than the nominal size for very large values of ν. Consequently, Figure 1 shows that the test has higher power for smaller values of ν. Finally, the comparison with the results of Diks et al. (2010 shows that the tests based on the full copula support have higher power than the tests focusing on the left tail only. This is to be expected, as the tests for predictive accuracy in a given region of support attempt to solve a much more difficult statistical problem (the observations outside of the targeted region are of limited value to the testing of the hypothesis. In summary, although the suggested tests of predictive accuracy in the selected region of support are moderately conservative, they have satisfactory statistical power. 5 Empirical application We examine the empirical usefulness of the predictive accuracy test for comparing alternative copula specifications in given regions of the support with an application to exchange rate returns for several major currencies. Specifically, we consider daily returns on the US dollar exchange rates of the Canadian dollar (CAD, euro (EUR, British pound sterling (GBP and Japanese yen (JPY over the period from August 28, 1992 until July 21, 2008, giving a sample of exactly 4,000 observations. Up to December 31, 1998, the euro series actually concerns the exchange rate of the German Deutschmark, while the euro is used as of January 1, The data are noon buying rates in New York and are obtained from the Federal Reserve Bank of New York. We consider the dependence between the EUR and GPB exchange rates, as well as a group of three exchange rates, namely CAD, JPY and EUR. 21

23 We employ the SCOMDY framework, as discussed in Section 3, to model the daily exchange rate returns and their dependence. For the conditional mean and the conditional variance of the return on currency j we use an AR(5-GARCH(1,1 specification, given by 5 Y j,t = c j + φ j,l Y j,t l + h j,t ε j,t (20 l=1 h j,t = κ j + γ j (Y j,t 1 c j 2 5 φ j,l Y j,t 1 l + β j h j,t 1, (21 l=1 where κ j > 0, β j 0, γ j > 0 and β j + γ j < 1. The joint distribution of the standardized innovations ε j,t is specified semi-parametrically, combining nonparametric univariate marginal distributions F j with a parametric copula C. We consider a substantial number of alternative copula specifications, which we compare in terms of their relative performance in out-of-sample density forecasting for different regions of the support. In particular, in the bivariate analysis of the pound sterling and euro exchange rates, we consider the Gaussian (Ga and Student-t (St-t elliptic copulas and the classic Archimedean copulas and their mixtures, that is, the Clayton (Cl, Clayton survival (Cl-s, mixture of Clayton and Clayton survival (Cl/Cl-s, Gumbel (Gu, Gumbel survival (Gu-s, mixture of Gumbel and Gumbel survival (Gu/Gu-s, and Frank (F copulas. Finally, we also include the bivariate symmetrized Joe-Clayton (SJC copula, see Patton (2006. We compare the one-step ahead density forecasting performance of the different copula specifications using a rolling window scheme. We estimate the SCOMDY model parameters using the three-stage procedure described in Section 3. The length of the rolling estimation window is set to R = 2, 000 observations, such that P = 2, 000 observations during the period August 10, June 21, 2008 are left for out-of-sample forecast evaluation. For comparing the accuracy of the resulting copula-based density forecasts we use the Diebold-Mariano type test based on the conditional likelihood in (11 and the censored 22

24 likelihood in (12. As both scoring rules give qualitatively similar results, to save space we only report results of the tests based on the censored likelihood. 1 We focus on three specific regions of the copula support. The first region, labeled D, corresponds to all currencies suffering a simultaneous depreciation against the USD, and is defined as D = {(u 1,..., u d u j < r for all j = 1,..., d}, where the threshold r {0.25, 0.30, 0.35}. Note that we only consider regions with identical thresholds for all currencies. This obviously is an arbitrary choice, but it is made in order to limit the number of regions under consideration. Below, we present detailed results for r = 0.30 only, but include the two alternative values in the discussion to address the sensitivity of the results to the specific choice of the threshold value. The second region, denoted U, is the mirror image of D in the sense that it represents a simultaneous appreciation against the USD and is defined as U = {(u 1,..., u d u j > 1 r for all j = 1,..., d}, where again the threshold r {0.25, 0.30, 0.35}. The third region concerns the central part of the copula support. This region M is defined as M = {(u 1,..., u d r < u j < 1 r for all j = 1,..., d}, where we use the same values of r as for the regions D and U. Region M corresponds to regular trading conditions, but it can also be relevant for risk management purposes if a market participant acquires positive exposure to exchange rate volatility (for example, by acquiring FX options. Figure 2 illustrates the three regions for the conditional PITs from one-step ahead density forecasts for daily EUR/USD and GBP/USD return innovations. 1 Detailed results based on the conditional likelihood are available upon request. 23

25 [Figure 2 about here.] As an additional diagnostic tool, we use the model confidence set (MCS concept of Hansen et al. (2011 to identify the collection of models which includes the best copula specification with a certain level of confidence. Starting with the full set of models, at each iteration we test the null hypothesis that all the considered models have equal predictive ability according to the selected scoring rule. If the null hypothesis is rejected, the worst performing model is omitted, and equal predictive ability is tested again for the remaining models. This procedure is repeated until the null hypothesis cannot be rejected and the collection of models that remains at this point is defined to be the MCS. In the current application of the MCS procedure, we always exclude the worst performing model and repeat the algorithm until only one model remains in the confidence set. This modification of the MCS analysis allows us to obtain a complete ranking of the competing models. We report the p-values corresponding to the hypothesis tests at every iteration. The implementation of the MCS is based on bootstrapping. To accommodate the possibility of autocorrelation in the scoring rules, we use the stationary bootstrap methodology of Politis and Romano (1994, with the probability of sampling the consecutive observation set to EUR/USD - GBP/USD exchange rates Table 2 reports the values of the pairwise Q R,P test statistic based on the censored likelihood score for the regions D, U and M with the threshold r = 0.3. Obviously, the matrices in the different panels are antisymmetric, that is Q R,P (i, j = Q R,P (j, i for copula specifications i and j. We nevertheless report the full matrices as this allows for an easy assessment of the relative performance of the various copulas. Given that in each panel the (i, jth entry is based on the score difference d csl t+1 = S csl j,t+1(y t+1 S csl i,t+1(y t+1, positive values of the test statistic indicate that the copula in column j achieves a higher average score than the one in row i. Hence, the more positive values in a given column, 24

26 the higher the ranking of the corresponding copula specification. [Table 2 about here.] A first observation from Table 2 is that, indeed, the (relative performance of a given copula can vary widely across different regions of the support. The Clayton copula is an illustrative example. While it performs second-best in region D, it performs (second-to- worst in region U (M. Also, while the Gumbel survival copula significantly outperforms the Student-t copula in the D region, the ranking is reversed in the U region, where the Student-t copula is significantly better than the Gu-s copula at the 5% level. In the remainder of this section, we focus on the Student-t and Gumbel survival copulas as well as the mixture of the Gumbel and Gumbel survival copulas. These copula specifications turn out to provide the most competitive density forecasts for the current application. The Gu-s copula, which features positive lower tail dependence, is clearly favored for region D. Based on one-sided tests, it significantly outperforms all competing specifications at the 10% level or better. Panel (a in Figure 3 shows the time series of differences between the censored likelihood scores for the Gu-s and Student-t copulas. We observe that the Gu-s copula achieves the largest gains when the two PITs are close, that is, when both the British pound sterling and euro suffer a depreciation of comparable magnitude against the US dollar. For these observations, the censored likelihood score for the Gu-s copula is substantially higher than for the Student-t copula. At the same time the Gu-s copula struggles with unbalanced pairs of PITs, that is, when the (unexpected movements in the two currencies are quite different. For these observations, the Student-t copula provides more accurate density forecasts. [Figure 3 about here.] We explore this issue in more detail by means of the scatter plots of censored likelihood score differences for the Gu-s copula vis-a-vis the Student t and Gu/Gu-s copulas 25

27 in Figure 4. To provide a finer resolution, we further divide region D into nine nonoverlapping subregions. Specifically, for both currencies we separate observations with û t , < û t , and 0.10 < û t = r. Then, each panel of the graph contains the score differences corresponding to the PITs of the GBP/USD and EUR/USD returns satisfying a particular combination of these conditions. While the horizontal axis of each panel indicate the PIT of the EUR/USD return, note that the vertical axes measure the score differences. The scatter plots in Figure 4 supports the above conclusions. It clearly shows that the Gu-s copula is better at capturing the dependence structure of return innovations with balanced PITs, as indicated by the positive values of censored likelihood score differences in the blocks along the main diagonal (that is, blocks corresponding to both PITs lying in the squares defined by [0, 0.025], (0.025, 0.1], and (0.1, 0.3]. The largest gains for the Gu-s copula are obtained for extreme simultaneous depreciations of the two currencies against the US dollar (region [0, 0.025] [0, 0.025], although these occur infrequently by definition. Notably, the Gu-s copula also dominates both the Student-t and Gu/Gu-s mixture copulas in the (0.025, 0.1] (0.025, 0.1] and (0.1, 0.3] (0.1, 0.3] regions. We also observe that large negative score differences, indicating far worse density forecasts of the Gu-s copula, occur exclusively when the PITs are unbalanced, that is, in the upper-left and bottom-right blocks. 2 From the MCS results, we observe that the first copula specifications to be dropped from the model confidence set are the Clayton survival and Gumbel copulas. Both models can accommodate only positive upper tail dependence, which is of no use in the D region. The Gu-s, Cl, and Gu/Gu-s mixture copulas are in the top three model confidence set. [Figure 4 about here.] In region U the Student-t copula is the best choice for out-of-sample forecasting, with a (one-sided p-value of the test statistic against the second best Gu/Gu-s mixture copula 2 It is also interesting to note that the patterns in the relative performance of the Gu-s copula against the Student-t copula and the Gu/Gu-s mixture copula are very similar, although the magnitudes of score differences vary. This indicates the proximity of the shape of the dependence captured by the Student-t and Gu/Gu-s mixture copulas. 26

28 of 0.126, see panel B in Table 2. The strong performance of the Student-t copula relative to the Gu/Gu-s mixture is mainly due to accurate forecasting of extreme joint appreciations of the euro and British pound sterling against the US dollar, as illustrated by the time series of censored likelihood score differences in panel (b of Figure 3. This conclusion is corroborated by the scatter plot in Figure 5. Here we divide region U in four subregions using the value 0.9 as an additional threshold for both PITs. When only one currency dramatically increases its value relative to the US dollar, the Student-t copula tends to achieve a better score, although the gains in forecasting performance are relatively modest in these cases. However, unbalanced PITs also cause all of the large failures of the Student-t copula, but these observations are very scarce. The Student-t copula suffers the most in the (0.7 < U EUR < 0.9, 0.7 < U GBP < 0.9 region of the support, with the score difference being negative for most observations. The MCS results suggest that the elliptic Gaussian and Student-t copulas excel in capturing the dependence structure in region U. Copulas with positive lower tail dependence features (Clayton and Gumbel survival are among the first to be excluded from the model confidence set, which is the mirror image of the MCS results for region D. [Figure 5 about here.] In region M the Gu/Gu-s copula significantly outperforms the Student-t copula. Interestingly, the MCS results indicate that the differences in predictive accuracy of the density forecasts from the different copula specifications is much less pronounced for this middle region. Using a 20% significance level, for example, only the Clayton and Clayton survival copulas would not be included in the model confidence set. For the region D, for example, we find a much smaller model confidence set consisting of only three copulas at this significance level. The relative performance of the various copula specifications is robust with respect to moderate shifts of the cutoff value r defining the relevant regions of support, although the significance of the difference in predictive ability varies. For region D, for example, 27

29 the results for r = 0.25 and 0.35 indicate that the superiority of the Gu-s copula is especially evident for the larger tail region, with all test statistics significant at the (one-sided 1% level. While the Gu-s copula remains the best choice for smaller tail regions, the significance of the test statistic decreases. This suggests that the Gu-s copula model accurately matches the dependence structure for non-extreme levels of common depreciation. The Student-t copula demonstrates the highest relative accuracy for region U defined by r = 0.30, while its performance slightly worsens for both smaller and larger U regions (r = 0.25, Finally, the Gu/Gu-s copula gains substantially in relative performance against the Frank copula, if we consider a greater M region (r = Unreported results for the test of Diks et al. (2010 that compares the predictive accuracy of density forecasts on the whole copula support show that the Gu/Gu-s copula outperforms the (second best Student-t copula with the p-value of the corresponding test statistic equal to The superiority of the Gu/Gu-s copula comes from its performance in the M region of support, while the Student-t copula better captures asymmetric changes in the EUR/USD and GBP/USD returns. This illustrates how the suggested testing framework can be used to identify the sources of relative forecasting accuracy over the whole copula support. 5.2 CAD/USD - JPY/USD - EUR/USD exchange rates Table 3 show the values of the pairwise Q R,P test statistic based on the censored likelihood score for the regions D, U and M with the threshold r = 0.3, applied to copula-based density forecasts for daily CAD/USD, JPY/USD and EUR/USD returns. [Table 3 about here.] In the region D of the copula domain the Gu-s and Gu/Gu-s mixture copulas deliver the best predictive accuracy, with their average censored likelihood scores being almost identical. The Gu/Gu-s copula seems to be preferable, in light of its greater advantages 28

30 against all other copula specifications (reflected in higher values of the corresponding test statistics. Panel (a of Figure 6 shows that, relative to the Student-t copula, the performance of the Gu/Gu-s mixture copula is much stronger when only the Canadian dollar substantially depreciates against the US dollar. On the contrary, extreme common depreciations of the euro and Yen against the US dollar together with more moderate changes of the CAD/USD rate tend to cause lapses in the performance of the Gu/Gu-s mixture copula. [Figure 6 about here.] The situation is similar in the U region of support. In this case the Gumbel copula delivers the best predictive accuracy, but the difference with the mixture of the Gumbel and Gunmbel survival copulas is not significant. Both Gu and Gu/Gu-s copulas can accommodate positive upper tail dependence, but the simplicity of the Gu copula allows it to gain advantage over the Gu/Gu-s copula (most likely due to smaller estimation uncertainty. However, once again, the greater robustness of the Gu/Gu-s mixture copula transfers into better performance against a variety of alternative copula specifications. This becomes evident from comparing the test statistics in the columns corresponding to the Gu and Gu/Gu-s mixture copulas. The Gu/Gu-s mixture copula is the superior model for capturing common balanced and unbalanced extreme appreciations of the exchange rates against the US dollar, but its performance relative to the Student-t copula suffers during modest appreciations of the Canadian dollar against the US dollar, see panel (b of Figure 6. Looking at the model confidence sets, we see that the first copula specifications to be omitted from the model confidence set for region D (region U are the ones that feature only positive upper (lower tail dependence. Thus, like in the bivariate case of the EUR/USD and GBP/USD returns, we conclude that the ability of a copula to capture positive lower tail dependence in the D region and upper tail dependence in the U region significantly enhances its forecasting performance in the corresponding region. 29

31 In the M region of the copula support the Student-t copula achieves the highest average censored likelihood score, although the difference in performance with the Gu/Gu-s mixture copula is not significant. The relative accuracy of the Student-t copula is robust to different combinations of PITs in the region. The gains in performance of the Student-t copula are due to many positive but fairly small score differences, see panel (c of Figure 6. However, its advantage over the Gu/Gu-s mixture copula becomes significant for a smaller M region (r = The ranking of the copula models in the regions D, U and M is robust to variations in the volume of these regions (parameterized by the cutoff value, r. Interestingly, the results become significantly sharper for regions corresponding to r = Also, the Gu-s and Gu/Gu-s mixture copulas increase their gains against the Student-t copula in the smaller region D, defined by the cutoff r = Finally, based on the test of Diks et al. (2010 to the predictive accuracy of density forecasts on the whole copula support, we find overwhelming evidence for the superiority of the Student-t copula: it outperforms the second-best Gu/Gu-s mixture copula at the significance level. The above analysis suggests that this result is largely due to the Student-t copula s superior fit of the dependence structure in the middle region M and possibly in regions away from the main diagonal of the copula support (that is, the complement of the D, U, and M regions. In the tail regions D and U, which may very well be of particular interest in practice, the Gu/Gu-s mixture copula performs substantially better. 6 Conclusions Many practical applications involving joint density forecasts of multivariate asset returns focus on a particular part of the domain of support. Given that the dependence structure may vary, for example, with the sign and magnitude of returns, it becomes imperative to 30

32 identify the best forecast method for the targeted part of the distribution. Copula modeling allows for straightforward construction of flexible multivariate distribution via their decomposition into the dependence structure, represented by a copula function, and marginal distributions. In this paper, develop Kullback-Leibler Information Criterion (KLIC based test of equal (out-of-sample forecasting accuracy of different copula specifications in a selected region of the support. The test combines the approaches suggested by Diks et al. (2010 and Diks et al. (2011, making use of censored and conditional logarithmic scoring rules. Monte Carlo simulation shows that the tests, albeit moderately conservative, possess satisfactory size and power properties. The application of the tests to daily exchange rate returns clearly demonstrates that the best copula specification varies with the targeted region of the support. This finding highlights the practical usefulness of the suggested test. 31

33 A Appendix This Appendix provides a proof of Lemma 1. Generalized conditional likelihood score It is to be shown that E t (d cl t+1(p t, ˆf t 0, where d cl t+1(p t, ˆf t = S cl (p t ; Y t+1 S cl ( ˆf t, Y t+1. Define I p,t w t (sp t (s ds and I ˆf,t w t (s ˆf t (s ds. The time-t conditional expected score difference for the density forecasts p t and ˆf t is ( E t d cl t+1(p t, ˆf t ( ( pt (y = p t (y w t (y log dy I p,t ( ( ˆft (y p t (y w t (y log dy I ˆf,t ( wt (yp t (y w t (yp t (y/i p,t = I p,t log I p,t w t (y ˆf dy ( t (y/i ˆf,t w t (yp t (y = I p,t K, w t(y ˆf t (y 0, I p,t I ˆf,t where K(, represents the Kullback-Leibler divergence between the pdfs in its arguments, which is finite as a consequence of Assumption 1. Assumption 1 implies that support( ˆf t = support(ĝ t = support(p t. This, together with Assumption 2 (c guarantees that w t (yp t (y/i p,t and w t (y ˆf t (y/i ˆf,t can be interpreted as pdfs, while Assumption 2 (a ensures that w t (y can be treated as a given function of y in the calculation of the expectation, which is conditional on F t. 32

34 Generalized censored likelihood score then ( E t d csl t+1(p t, ˆf t where K If d csl t+1(p t, ˆf t = S csl (p t ; Y t+1 S csl ( ˆf t, Y t+1, = p t (y log ((p t (y wt(y (1 I p,t 1 wt(y dy ( ( wt(y p t (y log ˆft (y (1 I ˆf,t 1 wt(y dy ( wt(y ( 1 wt(y = p t (y log p t (y 1 I p,t dy ˆf t (y 1 I ˆf,t wt(y ( wt(y ( pt (yi = p t (y log ( 1 wt(y ˆf,t I p,t 1 I p,t dy I p,t ˆft (y I ˆf,t 1 I ˆf,t ( ( pt (yi ˆf,t = p t (y w t (y log + w t (y log I p,t I p,t ˆft (y I ˆf,t ( 1 I p,t + (1 w t (y log dy 1 I ˆf,t ( pt (yw t (y w t (yp t (y/i p,t = I p,t log I p,t w t (y ˆf dy t (y/i ˆf,t ( +I p,t log I p,t 1 I p,t + (1 I p,t log I ˆf,t 1 I ( ˆf,t w t (yp t (y = I p,t K, w t(y ˆf t (y ( + K Bin(1, I p,t, Bin(1, I I p,t I ˆf,t 0, ˆf,t ( Bin(1, I p,t, Bin(1, I ˆf,t is the Kullback-Leibler divergence between two Bernoulli distributions with succes probabilities I p,t and I ˆf,t, respectively. Assumption 2 (b, which requires w t (y to be scaled between 0 and 1 for the csl rule, is essential for this interpretation because it implies that I p,t and I ˆf,t can be interpreted as probabilities. Again, Assumptions1 and 2 (c guarantee that w t (yp t (y/i p,t and w t (y ˆf t (y/i ˆf,t can be interpreted as pdfs, while Assumption 2 (a ensures that w t (y can be treated as a given function of y in the calculation of the expectation, which is conditional on F t. 33

35 References Amisano, G. and Giacomini, R. (2007. Comparing density forecasts via weighted likelihood ratio tests. Journal of Business and Economic Statistics, 25, Bao, Y., Lee, T.-H. and Saltoğlu, B. (2004. A test for density forecast comparison with applications to risk management. Working paper 04-08, UC Riverside. Bao, Y., Lee, T.-H. and Saltoğlu, B. (2007. Comparing density forecast models. Journal of Forecasting, 26, Berg, D. (2009. Copula goodness-of-fit testing: An overview and power comparison. European Journal of Finance, 15, Chen, X. and Fan, Y. (2005. Pseudo-likelihood ratio tests for semiparametric multivariate copula model selection. Canadian Journal of Statistics, 33, Chen, X. and Fan, Y. (2006. Estimation and model selection of semiparametric copulabased multivariate dynamic models under copula misspecification. Journal of Econometrics, 135, Chib, S., Omori, Y. and Asai, M. (2009. Multivariate stochastic volatility. In Handbook of Financial Time Series (eds T.G. Andersen, R.A. Davis, J.-P. Kreiss and T. Mikosch, pp Springer-Verlag, Berlin. Diebold, F.X., Gunther, T.A. and Tay, A.S. (1998. Evaluating density forecasts with applications to financial risk management. International Economic Review, 39, Diebold, F.X. and Lopez, J.A. (1996. Forecast evaluation and combination. In Handbook of Statistics, Vol. 14 (eds G.S. Maddala and C.R. Rao, pp Amsterdam: North-Holland. 34

36 Diebold, F.X. and Mariano, R.S. (1995. Comparing predictive accuracy. Journal of Business and Economic Statistics, 13, Diks, C., Panchenko, V. and van Dijk, D. (2010. Out-of-sample comparison of copula specifications in multivariate density forecasts. Journal of Economic Dynamics and Control, 34, Diks, C., Panchenko, V. and van Dijk, D. (2011. Likelihood-based scoring rules for comparing density forecasts in tails. Journal of Econometrics, 163, Genest, C., Gendron, M. and Bourdeau-Brien, M. (2009. The advent of copulas in finance. European Journal of Finance, 15, Giacomini, R. and White, H. (2006. Tests of conditional predictive ability. Econometrica, 74, Gneiting, T. and Raftery, A.E. (2007. Strictly proper scoring rules, prediction and estimation. Journal of the American Statistical Association, 102, Granger, C.W.J. and Pesaran, M.H. (2000. Economic and statistical measures of forecast accuracy. Journal of Forecasting, 19, Hansen, P. R., Lunde, A. and Nason, J. M. (2011. The model confidence set. Econometrica, 79, Mitchell, J. and Hall, S.G. (2005. Evaluating, comparing and combining density forecasts using the KLIC with an application to the Bank of England and NIESR fan charts of inflation. Oxford Bulletin of Economics and Statistics, 67, Nelsen, R.B. (2006. An Introduction to Copulas, Second edn. Springer Verlag, New York. Patton, A.J. (2006. Modelling asymmetric exchange rate dependence. International Economic Review, 47,

37 Patton, A.J. (2009. Copula-based models for financial time series. In Handbook of Financial Time Series (eds T.G. Andersen, R.A. Davis, J.-P. Kreiss and T. Mikosch, pp Springer-Verlag, Berlin. Politis, D.N. and Romano, J.P. (1994. The stationary bootstrap. Journal of the American Statistical Association, 89, Rivers, D. and Vuong, Q. (2002. Model selection tests for non-linear dynamic models. Econometrics Journal, 5, Silvennoinen, A. and Teräsvirta, T. (2009. Multivariate GARCH models. In Handbook of Financial Time Series (eds T.G. Andersen, R.A. Davis, J.-P. Kreiss and T. Mikosch, pp Springer-Verlag, Berlin. Sklar, A. (1959. Fonction de répartitions à n dimensions et leurs marges. Publications de l Institut Statistique de l Université de Paris, 8, Vuong, Q. (1989. Likelihood ratio tests for model selection and non-nested hypotheses. Econometrica, 57, West, K.D. (1996. Asymptotic inference about predictive ability. Econometrica, 64,

38 Figure 1: Power of the test of equal predictive accuracy based on conditional and censored likelihood scores nominal size 0.01 Censored Conditional nominal size 0.05 Censored Conditional power 0.4 power degrees of freedom degrees of freedom (a Nominal size 0.01 (b Nominal size nominal size 0.10 Censored Conditional power degrees of freedom (c Nominal size 0.10 The figure displays observed rejection rates (on the vertical axis of a one-sided test of equal performance of the Gaussian and Student-t copulas, against the alternative hypothesis that the correctly specified Student-t copula has a higher average score. The tests are based on the left tail copula region [0, 0.25] d (d = 3 and use either censored or conditional scores. The horizontal axis displays the degrees of freedom parameter of the Student-t copula characterizing the DGP. The DGP is the SCOMDY model (13 with: (1 marginal distributions specified in ((15-(16; (2 a Student-t copula with dimension d = 3 with all correlation coefficients set to ρ = 0.3 and varying degrees of freedom ν. The test of equal predictive accuracy compares a Student-t copula (with both parameters Σ and ν estimated, rather than known against a Gaussian copula with the parameter Σ also being estimated. The three plots correspond to different nominal sizes 0.01, 0.05, 0.1, indicated by horizontal lines. The number of observations in the moving in-sample estimation window is R = 1, 000 and the number of out-of-sample evaluations is P = 1, 000. Reported results are based on 1,000 replications. 37

39 Figure 2: Conditional PITs from one-step ahead density forecasts for daily EUR/USD and GBP/USD returns for the period August 10, July 21,

40 Figure 3: Censored likelihood score differences for daily EUR/USD and GBP/USD returns. (a Gumbel survival minus Student-t for region D (b Student-t minus Gu/Gu-s mixture for region U (c Gu/Gu-s mixture minus Student-t for region M Note: The graphs also depict the PITs of most notable score differences (PIT EUR/USD, PIT GBP/USD. 39