Quality control of risk measures: backtesting VAR models

Transcription

1 De la Pena Q 9/2/06 :57 pm Page 39 Journal o Risk (39 54 Volume 9/Number 2, Winter 2006/07 Quality control o risk measures: backtesting VAR models Victor H. de la Pena* Department o Statistics, Columbia University, Room 027, Mail Code 4690, 255 Amsterdam Avenue, New York, NY 0027, USA Ricardo Rivera State o New York Banking Department and New York University, One State Street, New York, NY 0025, USA Jesus Ruiz-Mata Lehmann Brothers, 745 7th Avenue, Second Floor, New York, NY, 009, USA This paper introduces a new statistical approach to assessing the quality o risk measures: quality control o risk measures (QCRM. The approach is applied to the problem o backtesting value-at-risk (VAR models. VAR models are used to predict the maimum likely losses in a bank s portolio at a speciied conidence level and time horizon. The widely accepted VAR backtesting procedure outlined by the Basel Committee or Banking Supervision controls the probability o rejecting the model when the model is correct. A drawback o the Basel approach is its limited power to control the probability o accepting an incorrect VAR model. By eploiting the binomial structure o the testing problem, QCRM provides a more balanced testing procedure, which results in a uniorm reduction o the probability o accepting a wrong model. QCRM consists o three elements: the irst is a new hypothesis-testing problem in which the null and alternative hypotheses are echanged to control the probability o accepting an inaccurate model. The second element is a new approach or comparing the power o the QCRM and Basel tests in terms o the probability o rejecting correct and incorrect models. The third element involves the use o the technique o pivoting the cumulative distribution unction to obtain one-sided conidence intervals or the probability o an eception. The use o these conidence intervals results in new acceptance/rejection regions or tests o the VAR model. We compare these to ones commonly used in the inancial literature. Introduction Banking regulators and risk managers o inancial institutions oten wish to assess the adequacy o a model. They requently test the null hypothesis that the model is correct against an alternative hypothesis that the model is incorrect. In this paper we present an approach to assist regulators and risk managers in this *Corresponding author. The irst author acknowledges support rom NSF grant DMS

2 De la Pena Q 9/2/06 :57 pm Page Victor H. de la Pena, Ricardo Rivera and Jesus Ruiz-Mata task. We illustrate the approach by presenting an application to the problem o backtesting value-at-risk (VAR models. The ( δ00% VAR number is the ( δ 00 percentile o the distribution o the portolio losses or a speciied time horizon. VAR backtesting is the process by which inancial institutions periodically compare daily proits and losses with the model-generated risk measures to gauge the accuracy o their VAR models. In 996 the Basel Committee or Banking Supervision ( Basel developed a ramework or backtesting the internal models used to calculate regulatory capital or market risk (see Basel (996. Alternative statistical evaluation methodologies have been proposed. For eample, the evaluation based on the binomial distribution is discussed by Kupiec (995; the interval orecast method proposed by Christoersen (998; the distribution orecast method by Crnkovic and Drachman (996; and the magnitude loss unction method by Lopez (996. We start with the ollowing question: What are regulators and bank managers really doing when they apply the Basel VAR backtesting methodology? The Basel backtesting procedure tests the null hypothesis that the bank s VAR model predicts losses accurately against the alternative hypothesis that the model predicts losses incorrectly: H 0 B : VAR model is correct vs. H B : VAR model is incorrect ( The test statistic used is based on the number o eceptions generated by the VAR model. For a given trading day, an eception occurs when the loss eceeds the model-based VAR. The test postulates that the probability o an eception, p, is equal to 0.0 and tests it against the alternative hypothesis that the probability o an eception is greater than 0.0. The test is based on the number o eceptions in 250 trading days. The test rejects the VAR model i the number o eceptions is greater than or equal to 0 and accepts the model otherwise. When evaluating statistical tests, it is common practice to eamine their type I and II error rates. Under the Basel backtesting procedure, the type I error rate is the probability o rejecting the VAR model when the model is correct, while the type II error rate is the probability o accepting the model when the model is incorrect. The Basel backtesting procedure is designed or controlling the level, the probability o rejecting the VAR model when the model is correct. For this test, the level is the probability that number o eceptions, out o 250 daily observations, is greater than or equal to 0, when the probability o an eception is p = 0.0. The level o the test is (or 0.03%. Although the test establishes a very conservative threshold or controlling the type I error rate, it is not designed to control the type II error rate. Thereore, Journal o Risk

3 De la Pena Q 9/2/06 :57 pm Page 4 Quality control o risk measures: backtesting VAR models 4 it does not control the probability o accepting the VAR model when the model is incorrect. This drawback is pointed out on page 5 o Basel (996: The Committee o course recognizes that tests o this type are limited in their power to distinguish an accurate model rom an inaccurate model. To address this issue we introduce an alternative approach to the Basel backtesting procedure. We call this methodology quality control o risk measures (QCRM. Its goal is to enhance the ability o the test to reject an incorrect model. It consists o three elements: irst, the test introduces a new hypothesis testing problem in which the null and alternative hypotheses are echanged. The goal is to control the probability o accepting a wrong model. The second element consists o a new deinition o the power o the tests that allows the comparison o QCRM and Basel backtesting procedures. The third element involves the use o a technique to obtain accurate estimates o the acceptance/rejection regions. The new hypothesis testing problem is: H 0 Q : VAR model is incorrect vs. H Q : VAR model is correct (2 Under QCRM, the acceptance o the null hypothesis is equivalent to the rejection o the VAR model. The model is accepted when the null hypothesis in (2 is rejected and, hence, when there is overwhelming evidence supporting the model. A type I error or QCRM occurs when the VAR model is accepted but the model is incorrect, and a type II error happens when the model is rejected although the model is correct. Table shows correct and incorrect decisions when the null and the alternative hypotheses are true. The ollowing relationship is important or understanding the properties o the QCRM test: Probability o rejecting the VAR model when the model is incorrect = Probability o accepting the VAR model when it is incorrect. Under QCRM the probability o accepting the VAR model when the model is incorrect is set at level : this means that the procedure controls the type error rate. By setting the level to a small value ( %, QCRM guarantees a high probability ( 99% o rejecting a wrong model. Once this level is ied, the use TABLE Type I and II errors or QCRM. True Decision : Decision 2: hypothesis VAR model is rejected VAR model is accepted Model correct Type II error Correct assessment Model incorrect Correct assessment Type I error Volume 9/Number 2, Winter 2006/07

4 De la Pena Q 9/2/06 :57 pm Page Victor H. de la Pena, Ricardo Rivera and Jesus Ruiz-Mata o the Neyman Pearson lemma provides a uniormly most powerul (UMP test that minimizes the type II error (or the probability o rejecting a correct model. We introduce an approach or comparing the power o the QCRM and Basel tests in terms o the probability o rejecting correct and incorrect models. Using this approach, it is shown that, relative to the Basel procedure, QCRM provides a balanced trade-o between type I and type II error rates (see Table 3. Table 5 and Figure provide numerical comparisons o the power o the tests. Similar to the Basel backtesting procedure, QCRM establishes the acceptance/rejection zones based on probabilistic arguments (see (8. More precisely, QCRM uses the technique o pivoting the true cumulative distribution unction o the observations to ind one-sided conidence intervals, which provides the new three zones or accepting/rejecting the VAR model. The paper is organized as ollows: Section 2 reviews the Basel methodology or testing VAR models. Section 3 introduces the basic idea o QCRM and the new hypothesis testing problem. Section 3.2 presents the statistical techniques to compute appropriate coverage conidence intervals or the true (unknown probability o an eception, p. The proposed rules or accepting/rejecting the VAR model are obtained in Section 3.3. The approach or comparing the relative power o the two tests is then addressed in Section 3.4. Finally, Section 4 summarizes our work. Some technical proos are presented in Appendices A and B at the end o the paper. 2 Basel VAR backtesting methodology The VAR backtesting methodology compares actual losses against model-based VAR calculations. Let the losses observed in every trading day be denoted by L, L 2,, L n (3 Let p be the probability o an eception on a given trading day. The computed ( p 00% one-day VAR numbers or the corresponding trading days are V 0, V 2,, V n n, (4 where Vi i denotes the VAR calculated or day i using the inormation available on day i. Deine the indicator random variables Y i = {L Vi }, i (5 such that Y i = i L i Vi i and Y i = 0 otherwise. Hence, an eception occurs on trading day i i and only i Y i =, ie, there is a loss eceeding VAR. The Basel backtesting procedure assumes that the conditional probability o an eception at day i, given the inormation up to day i, is a ied value, ie, P(Y i I i = p. This implies that the indicator variables Y, Y 2,, have ied Journal o Risk

5 De la Pena Q 9/2/06 :57 pm Page 43 Quality control o risk measures: backtesting VAR models 43 conditional probabilities. In Appendi A we show that the assumption o constant conditional probability is a necessary and suicient condition or independence o the indicator variables Y i,,y n. The Basel backtesting procedure implicitly tests the ollowing hypothesis: H 0 B : p = p 0 vs. H A : p > p 0 (6 based on the sample o n = 250 observations Y,,Y 250 (where p 0 = 0.0. The Basel procedure uses a p 0 value o %. This hypothesis is tested against the alternative hypothesis that the probability o an eception is greater than p 0. The Neyman Pearson lemma provides a uniormly most powerul (UMP test with rejection region o the orm R = {# o eceptions k} where k is a threshold obtained rom the desired probability o the type I error, ie, P(# o eceptions k p = p 0 = P(R H 0 = (7 When k = 0, Basel backtesting gives = or 0.3%, which implies that with a % probability the VAR model will not be rejected when the model is correct (ie, p = 0.0. Based on the number o eceptions o the VAR model, Basel deines three zones. The green zone consists o our or ewer eceptions, and in this case the VAR model is assessed as correct. The yellow zone includes ive to nine eceptions, and the accuracy o the VAR model is questioned. The red zone corresponds to 0 or more eceptions, and the model is rejected. The limits or each o the zones can be linked to a probabilistic statement through the cumulative distribution o the number o the eceptions. The green zone etends rom 0 to 95% o the cumulative probability distribution (cd o the observed eceptions. The yellow zone starts where the cd o the observed eceptions eceeds 95% up to 99.97%. The red zone corresponds to values o probability greater than the 99.99% percentile o the cd o the observed eceptions. For eample, using the variables in (3, we compute: 250 P Yi p= P Yi p = 250 = , = , i= i= 250 P Yi p= P Yi p = =.., 0. 0 = i= i= (8 Note that these probabilities are calculated assuming that the true parameter p is equal to 0.0. Volume 9/Number 2, Winter 2006/07

6 De la Pena Q 9/2/06 :57 pm Page Victor H. de la Pena, Ricardo Rivera and Jesus Ruiz-Mata 3 Quality control o risk measures 3. New hypothesis testing problem QCRM starts with the hypothesis that the VAR model is incorrect and then tests this against the alternative hypothesis that the VAR model is correct. Accepting the null hypothesis then implies the rejection o the VAR model, while rejecting the null hypothesis leads to the acceptance o the model. Let us assume that the probability o an eception is p 0 when the VAR model is correct and is some unknown value p when the model is incorrect. More speciically, the rival hypotheses are H 0 Q : p > p vs. H Q : p p 0 (9 where p is the unknown probability o an eception. Note that, using monotonic properties o tests, the hypothesis test in (9 is equivalent to saying H 0 Q : p = p vs. H Q : p p 0. This results rom the act that the likelihood ratio used in developing the test is a monotone unction o the parameter p (see Equation (8 in Appendi B. The test in (9 above can be seen as a quality control problem in which regulators and risk managers assess the quality o the risk model and reject it i the proportion o eceptions (model ailures is statistically larger than an acceptable threshold; eg, p 0 = 0.0. To construct the test, we assume that the observations are independent so that the number o eceptions 250 S = Y + + Y = Y = n n i i= (0 ollows a binomial distribution with parameters n and p. The statistic S n is also a suicient statistic or p, and this means that it contains all the inormation available about the parameter p. In Appendi B we show that the statistic S n has a cumulative distribution unction that decreases in the parameter p. Thereore, using the theorem developed by Karlin-Rubin (see Casella and Berger (2002, the test that rejects H 0 Q : p > p vs. H Q : p p 0 i and only i {S n s( p } is uniormly most powerul level test, where = P p (S n s(p. Intuitively, the unknown probability p cannot be large i the number o eceptions is small. Under the new hypotheses testing problem, QCRM controls the probability o accepting the VAR model when the model is incorrect. By setting to a small level, QCRM saeguards against incorrectly accepting a alse model. 3.2 Optimal conidence intervals or hypothesis testing In this section we provide the theoretical underpinnings o the QCRM acceptance/rejection regions. More speciically, we use the correspondence between Journal o Risk

7 De la Pena Q 9/2/06 :57 pm Page 45 Quality control o risk measures: backtesting VAR models 45 tests o hypotheses and interval estimation to obtain optimal conidence intervals. For each sample value Y = (Y,,Y n, deine the conidence region as ollows: C(Y ={p : Y A( p } ( where A( p is the acceptance region or p = p against p = p 0. The traditional method or developing a one-sided conidence interval or p uses the normal approimation or the sample proportion, pˆ = n and computes the lower boundary as p p pl (, p ˆ ˆ( ˆ = z n where z is the ( 00% quantile o the standard normal distribution. The value is used to obtain a right one-sided ( 00% conidence interval (p L (,, ] or p. However, as indicated in Brown, Cai and DasGupta (200, the intervals have deiciencies in coverage probability originating rom the discreteness o the distribution, even when n is large, when np is small (as when n = 250 and p = 0.0. To overcome this problem, we use a technique in which we pivot the true cd o the observations to compute one-sided intervals with a conidence level at least as large as the desired conidence level (see Casella and Berger (2002. One advantage o this technique is that it gives conidence intervals with the desired conidence level. Consider the tail probability o the sample proportion o eceptions, pˆ, in a period o n days. The interval (p L (,, ] is a right one-sided interval o p with a coverage level greater than or equal to ( 00%, where p L (, is the smallest value o p which satisies the ollowing: F = P (2 P ( ˆ n p L(, As shown below, Equation (2 provides the basis or the development o QCRM s acceptance/rejection regions. 3.3 Probabilistic-based rules or accepting/rejecting VAR models The proposed rules or accepting/rejecting VAR models are computed by inverting the rejection region o the test in (8 with level o signiicance (see Equation (2. When inverting the rejection region, we obtain a right one-sided conidence interval, (p L (,, ], or p with a coverage level greater than or equal to ( 00%, ie, P(p (p L (,,]=. Thereore, the test certiies that the model is correct, with at least ( 00% conidence, every time p 0 Technical proos o pivoting the true cd are provided in Appendi B. Volume 9/Number 2, Winter 2006/07

8 De la Pena Q 9/2/06 :57 pm Page Victor H. de la Pena, Ricardo Rivera and Jesus Ruiz-Mata TABLE 2 95% and 99% right-sided conidence intervals or the probability o an eception ater observing k eceptions in n = 250 trading days. Regions 95% 99% Green k = [0, ] [0, ] k = 2 ( , ] ( , ] k = 3 (0.0033, ] (0.007, ] k = 4 (0.0055, ] (0.0033, ] k = 5 (0.0079, ] (0.005, ] Yellow k = 6 (0.005, ] (0.0072, ] k = 7 (0.032, ] (0.0094, ] Red k = 8 (0.060, ] (0.07, ] k = 9 (0.089, ] (0.042, ] k = 0 (0.29, ] (0.067, ] belongs to the interval (p L (,, ], or alternatively, when p 0 does not belong to (0, p L (, ]. Note the ollowing property o these one-sided conidence intervals: (p L (,, ] (p L (, *, ] i * (3 By analogy to the Basel supervisory ramework, QCRM deines the ollowing new zones: New green zone The VAR model is certiied as correct i p 0 is in the 95% oneside conidence interval or p, (p L (, 0.05, ]. New yellow zone When p 0 is not in the one-sided 95% conidence interval but is in the 99% one-sided conidence interval or p, (p L (, 0.0, ], then the validity o the model is questioned. New red zone I p 0 is not in the 99% conidence interval or p,(p L (,0.0, ], then the VAR model is rejected. We employ a Newton Raphson algorithm to obtain a solution to Equation (2. Table 2 presents the 95% and 99% right one-sided conidence intervals or the probability o an eception ater observing k eceptions (0 k 0. Using Table 2 and our criteria or deining the zones, the 99% VAR model is certiied when zero to ive eceptions are observed. The model is questioned when si or seven eceptions are observed. Finally, the model is rejected when eight or more eceptions are realized. Journal o Risk

9 De la Pena Q 9/2/06 :57 pm Page 47 Quality control o risk measures: backtesting VAR models 47 Thereore, the proposed three zones using QCRM are: New green zone = {0 to 5 eceptions} New yellow zone = {6 or 7 eceptions} New red zone = {8 or more eceptions} 3.4 A new approach or comparing powers: a tale o two powers As noted, QCRM controls the probability o accepting the model when the model is incorrect at a low level or the test. Thereore, it rejects the null hypothesis that the model is inaccurate when there is overwhelming evidence supporting the model (ie, when p 0.0. This results in the statistical validation o the model. To compare the results o the QCRM and Basel procedures, we look at the relative power o the tests. Since the tests are dierent in nature (the null and alternative hypotheses are dierent, it is not possible to directly compare their powers. However, by deining the power unction o the tests in terms o the probability o rejecting correct and incorrect models, we are able to make appropriate comparisons. The proposed approach or comparing test powers is based on the probability o rejecting correct and incorrect models. The irst step is to evaluate type I and type II error rates. Using the results o the tests or 250 observations and a % VAR, the type I and type II error rates are as shown in Table 3. The net step is to deine the power unction or QCRM, β Q (p, as: For p > 0.0, β Q (p = P(Rejecting the model Model is incorrect = P(Accepting the model Model is incorrect = P( 7 Given p = P( 8 Given p, and For p 0.0, β Q (p = P(Rejecting the model Model is correct = P( 8 Given p. Likewise, the power unction or Basel, β B (p, is: For p > 0.0, β B (p = P(Rejecting the model Model is incorrect = P(Accepting the model Model is incorrect = P( 9 Given p = P( 0 Given p, and For p 0.0, β B (p = P(Rejecting the model Model is correct = P( 0 Given p. Table 5 compares the power o the tests under the new approach. Relative to Basel, QCRM signiicantly increases the probability o rejecting a wrong model. The relative power gain is partially oset by an increase o the probability o rejecting a correct model. Volume 9/Number 2, Winter 2006/07

10 De la Pena Q 9/2/06 :57 pm Page Victor H. de la Pena, Ricardo Rivera and Jesus Ruiz-Mata TABLE 3 Type I and II error rates or the QCRM test. Decision with QCRM test True hypothesis Accept H 0 Q Reject H 0 Q H Q 0 : p > 0.0 OK Type I error = P( 7 p > 0.0 H A Q : p 0.0 Type II error = P( 8 p 0.0 = [0, 0.004] OK TABLE 4 Type I and II error rates or Basel test. Basel test decision True hypothesis Accept H 0 B Reject H 0 B H B 0 : p = 0.0 OK Type I error = P( 0 p = 0.0 = H B A : p > 0.0 Type II error = OK P( 9 p > 0.0 TABLE 5 Powers o Basel and QCRM tests. Probability o rejecting the model when it is: Test Correct Incorrect Basel Less than * P( 0 Given p > 0.0 QCRM Less than P( 8 Given p > 0.0 *Assumes a composite null hypothesis or the Basel test with p 0.0. QCRM rejects the VAR model when the number o eceptions is equal to or greater than eight or alternative values o p greater 0.0. See Section 3.3 or a proo o this. The Basel test is very conservative in controlling the probability o rejecting a correct model (ie, 0.03%; however, it does so at the cost o increasing, relative to QCRM, the probability o accepting a wrong model. On the other hand, the QCRM test provides a balanced trade-o between type I and type II errors since it uses a less etreme probability o rejecting a correct model. As a result, QCRM delivers signiicant gains in power. For eample, Figure shows the percentage gains o QCRM over Basel, calculated as percentages o the respective probabilities o rejecting a wrong model, or values o the (unknown probabilities o an eception greater than 0.0. Note that QCRM outperorms the Basel test, especially or parameter values closer to 0.0. For instance, there are rate gains in the range o 637% to 5% or p values between 0.05 and 0.03, respectively. Journal o Risk

11 De la Pena Q 9/2/06 :57 pm Page 49 Quality control o risk measures: backtesting VAR models 49 FIGURE Relative power gain (RPG curve. 400% 200% 000% 800% P( p. RPG( p = 8 > 00 % P( 0 p> % 400% 200% 0% Conclusions In this paper we have developed an approach or validating risk models and applied it to the problem o validating a value-at-risk model. The main contribution o our approach is to develop a probabilistic-based test to control the probability o accepting an incorrect VAR model. As a result, a model is validated when there is enough evidence to support it. A second contribution is the introduction o an approach or comparing the powers o the tests in terms o the probability o rejecting correct and incorrect models. Using this approach, we have shown that the QCRM procedure delivers signiicant gains in power, relative to the Basel procedure, in terms o the probability o rejecting a wrong model. The third element is the calculation o improved estimates o the probability o acceptance/rejection regions. The traditional method or obtaining a one-sided conidence interval or the parameter p uses the normal approimation or the sample proportion. However, the intervals have deiciencies in coverage probability originating rom the discreteness o the distribution. We have obtained conidence intervals with the eact desired conidence level by pivoting the cumulative distribution unction o the number o eceptions. Our results show that a VAR model should be certiied as correct when no more than ive eceptions are observed and should be rejected i eight or more eceptions are observed in 250 trading days. The proposed methodology can be etended to address other model validation problems within or outside the inancial world. Volume 9/Number 2, Winter 2006/07

12 De la Pena Q 9/2/06 :57 pm Page Victor H. de la Pena, Ricardo Rivera and Jesus Ruiz-Mata Appendi A: Independence In this section we prove the statement that Bernoulli random variables with ied conditional probabilities must be independent. Consider a sequence o Bernoulli random variables,, n, P( i ==p and P( i =0= p, and assume that or every n there eists a ied p such that P( n+ = n+ 0 = 0,, n = p n+ ( p n+ (A We now prove by induction that (A implies that, 2, are independent Bernoulli random variables with parameter p. Consider the case n = 0; then (A implies that P( 0 = 0, = = P( = 0 = 0 P( 0 = 0 = p ( p P( 0 = 0 = P( 0 = 0 P( = (A2 which implies that 0 and are independent Bernoulli random variables. Let us assume now that the irst n random variables 0,, n are independent Bernoulli random variables with parameter p; then, using the induction hypothesis and (A gives P( 0 = 0,, n = n, n+ = n+ = P( n+ = n+ 0 = 0,, n = n P( 0 = 0,, n = n = p n+ ( p n+ p n ( p n p 0 ( p 0 = n p i + n+ i n = 0 i= 0 i ( P n+ = P ( = i= 0 i i For the general case, consider a set o indees i i 2 i m ; then, using the law o total probabilities, P ( =,, = = P=,, = i i im im im im s i = 0, i ji, j=,, m ( = P( = P = s i = 0, i ji, j=,, m ( im im = P ( i = i P ( = im im (A3 Journal o Risk

13 De la Pena Q 9/2/06 :57 pm Page 5 Quality control o risk measures: backtesting VAR models 5 Equation (A3 proves that events i = i,, im = im are independent. Thereore, the sequence o Bernoulli random variables,, n is independent. Appendi B: Pivoting a CDF We provide here a detailed proo o Theorem o Casella and Berger (2002, Chapter 9 or computing the conidence intervals. THEOREM Let χ be a discrete statistic with cd F ( =P(. Suppose that or each χ, L ( and U ( are deined as ollows:. I F ( is a decreasing unction o or each, deine L ( and U ( by P ( U( =, P ( L( = 2 2 (B 2. I F ( is an increasing unction o or each, deine L ( and U ( by P ( U( =, P ( L( = 2 2 (B2 Then the interval [ L (, U (] is a 00 ( % conidence interval or. PROOF OF ( Assume irst that F ( is a decreasing unction o or each. Consider the new random variable T = F ( and assume that the range o is an ordered set { 0,, }; then the event {T > y} ={F ( > y} is equal to the event { n }, where n = in{ i : F ( i > y}. Thereore, PF ( ( > y = P ( = i= n = P ( n i (B3 By deinition o n, P( n = F ( n y. Thereore, P(F ( > y y (B4 I y is between n and n, then the inequality in (B4 is strict. This implies that P(F ( y y. Likewise, we can prove that P(F ( y y, where F ( = P(. Thereore, the set { } A( = : F( and F( 2 2 (B5 Volume 9/Number 2, Winter 2006/07

14 De la Pena Q 9/2/06 :57 pm Page Victor H. de la Pena, Ricardo Rivera and Jesus Ruiz-Mata is a rejection region with level o signiicance. This implies that (see Casella and Berger (2002 the set C( ={ : A(} (B6 is a 00 ( % conidence set or the parameter. We consider now L ( and U ( that satisy the equations F( U( =, F( L( = 2 2 (B7 Note that F ( is a decreasing unction o or each implies that F ( is a non-decreasing unction o. I > U (, then F ( 2; and i L (, then F ( 2 and { : F( and F( } 2 2 = { : ( ( } L Thereore, the 00 ( % conidence set or the parameter is C( = [ L (, U (]. PROOF OF (2 To prove (2 or the increasing case, we consider the unction h( = F (, where the parameter is moving in the domain o. The new unction is a decreasing unction o and we can use the proo given above. In order to apply the ormer result to the binomial case it is enough to observe that the cd o the binomial distribution is a decreasing unction o the parameter. This derives rom the act that has a monotone likelihood ratio (MLR, ie, or 2, the likelihood ratio U ( 2 ( (B8 is a decreasing unction o and the ollowing result. MLR implies a decreasing cd PROOF We prove this result or the continuous case without loss o generality. Let 2 and F ( = P( be the cd o, F ( F ( 2 = R {y } ( (y (y 2 dy = R {y } R(y (y dy (B9 Journal o Risk

15 De la Pena Q 9/2/06 :57 pm Page 53 Quality control o risk measures: backtesting VAR models 53 where R ( = ( 2 ( (B0 is a decreasing unction o rom the MLR property. We have now two cases:. R ( 0 ( 2,, y ( or every 2. ( 2 R (,, change sign in the interval (, ( The irst case implies that F ( F ( 2 0. For the second case, consider and A = {y (, :R(y > 0} and B = {y (, :R(y 0}; then, rom (B8, we obtain F ( F ( 2 = A {y } R(y (y dy + B {y } R(y (y dy (B Because R(y is decreasing, we have that all points in A have to be smaller than the points in B. Let A ( and B (, the smallest and largest values that {y } can achieve in A and B, respectively. Then, rom (B0, we obtain F ( F ( 2 A ( A R(y (y dy + B ( B R(y (y dy We have now that (B2 A Ry ( ( y dy = P y ( 2 2 y ( y 2 P ( ( y (B3 and B Ry ( ( y dy = P y ( 2 2 y ( y 2 P ( ( y (B4 Then (B becomes ( y 2 y 2 F ( F ( 2 [ A B ] P P 2 y ( ( ( y ( ( (B5 Volume 9/Number 2, Winter 2006/07

16 De la Pena Q 9/2/06 :57 pm Page Victor H. de la Pena, Ricardo Rivera and Jesus Ruiz-Mata Then, P 2 y ( 2 y ( y y y 2 2 = ( d ( = ( y ( y 2 ( y ( y 2 ( y y ( 2 y ( ( y dy y ( dy = P y ( 2 ( y (B6 I y A with y >, then A ( = B ( = 0. Otherwise, A ( = and B ( = 0 or A ( = B ( =. This implies that A ( B (. This act, together with (B5 and (B6, results in the inequality F ( F ( 2 This implies that the cd o is a decreasing unction o the parameter. (B7 REFERENCES Basel Committee on Banking Supervision (996. Supervisory ramework or the use o back testing in conjunction with the internal models approach to market risk capital requirements. Bank or International Settlements, Basle, January. Brown, L., Cai, T., and DasGupta, A. (200. Interval estimation or a binomial proportion. Statistical Science 6(2, Casella, G., and Berger, R. (2002. Statistical inerence, Second edition. Dubury Advance Series, Paciic Grove, CA 93950, USA. Christoersen, P. F. (998. Evaluating interval orecasts. International Economic Review 39, Crnkovic, C., and Drachman, J. (996. Quality control. Risk 9 (September, Jorion, P. (2000. Value at risk: the new benchmark or managing inancial risk, Second edition. McGraw-Hill, New York. Fallon, W. (996. Calculating value-at-risk. Working paper presented at the Wharton Financial Institution Center s Conerence on Risk Management in Banking. Haas, M. (200. New methods in backtesting. Working paper, Financial Engineering Research Center caesar, Bonn. Kupiec, P. (995. Techniques or veriying the accuracy o risk measurement models. Journal o Derivatives, Winter, Lopez, J. A. (996. Regulatory evaluation o value-at-risk models. Working paper presented at the Wharton Financial Institution Center s Conerence on Risk Management in Banking. Journal o Risk