Elicitability and backtesting

Transcription

1 Elicitability and backtesting Johanna F. Ziegel University of Bern joint work with Natalia Nolde, UBC 17 November 2017 Research Seminar at the Institute for Statistics and Mathematics, WU Vienna 1 / 32

2 Introduction Let X be the single-period return of some financial asset. A risk measure ρ assigns a real number ρ(x ) to X (interpreted as the risk of the asset). Risk measures are used for external regulatory capital calculation management, optimization and decision making performance analysis capital allocation 2 / 32

3 Introduction Let X be the single-period return of some financial asset. A risk measure ρ assigns a real number ρ(x ) to X (interpreted as the risk of the asset). Risk measures are used for external regulatory capital calculation management, optimization and decision making performance analysis capital allocation 2 / 32

4 Risk measures We assume that the risk ρ(x ) only depends on the distribution of X, that is, ρ : P R, where P is a suitable class of probability measures on R. (We write both: ρ(x ) = ρ(p) for X P.) Impose theoretical requirements on ρ motivated by economic principles: monetary risk measures, coherent risk measures,.... Consider statistical aspects of the resulting functionals. Sign convention Losses are positive, profits are negative. Risky positions yield positive values of ρ. 3 / 32

5 Risk measures We assume that the risk ρ(x ) only depends on the distribution of X, that is, ρ : P R, where P is a suitable class of probability measures on R. (We write both: ρ(x ) = ρ(p) for X P.) Impose theoretical requirements on ρ motivated by economic principles: monetary risk measures, coherent risk measures,.... Consider statistical aspects of the resulting functionals. Sign convention Losses are positive, profits are negative. Risky positions yield positive values of ρ. 3 / 32

6 Risk measures We assume that the risk ρ(x ) only depends on the distribution of X, that is, ρ : P R, where P is a suitable class of probability measures on R. (We write both: ρ(x ) = ρ(p) for X P.) Impose theoretical requirements on ρ motivated by economic principles: monetary risk measures, coherent risk measures,.... Consider statistical aspects of the resulting functionals. Sign convention Losses are positive, profits are negative. Risky positions yield positive values of ρ. 3 / 32

7 Examples Example (Value-at-Risk (VaR)) For α (0, 1) and a random variable X with distribution function F, we define VaR α (X ) = inf{x R : F (x) α} = F 1 (α). VaR is a co-monotonic additive, monetary risk measure. Example (Expected shortfall (ES)) If X has finite mean, we define ES α (X ) = 1 1 α 1 α VaR u (X ) du ( ) = E[X X VaR α (X )]. ES is a co-monotonic additive, coherent risk measure. 4 / 32

8 Examples Example (Value-at-Risk (VaR)) For α (0, 1) and a random variable X with distribution function F, we define VaR α (X ) = inf{x R : F (x) α} = F 1 (α). VaR is a co-monotonic additive, monetary risk measure. Example (Expected shortfall (ES)) If X has finite mean, we define ES α (X ) = 1 1 α 1 α VaR u (X ) du ( ) = E[X X VaR α (X )]. ES is a co-monotonic additive, coherent risk measure. 4 / 32

9 Examples Example (Value-at-Risk (VaR)) For α (0, 1) and a random variable X with distribution function F, we define VaR α (X ) = inf{x R : F (x) α} = F 1 (α). VaR is a co-monotonic additive, monetary risk measure. Example (Expected shortfall (ES)) If X has finite mean, we define ES α (X ) = 1 1 α 1 α VaR u (X ) du ( ) = E[X X VaR α (X )]. ES is a co-monotonic additive, coherent risk measure. 4 / 32

10 Statistical aspects Returns {X t } t N (covariates {Z t } t N ) adapted to filtration {F t } t N Historical data: returns x 1,..., x n, (covariates z 1,..., z n ) Estimation Approximate ρ(x n+1 F n ) by r n+1 = ˆρ(x 1,..., x n, z 1,..., z n ). ( elicitability/identifiability) Forecast evaluation Given r 1,..., r n, evaluate the quality of the predictions (traditional backtesting). ( identifiability) Given r 1,..., r n, r 1,..., r n, say which method has better predictive power (comparative backtesting). ( elicitability) 5 / 32

11 Statistical aspects Returns {X t } t N (covariates {Z t } t N ) adapted to filtration {F t } t N Historical data: returns x 1,..., x n, (covariates z 1,..., z n ) Estimation Approximate ρ(x n+1 F n ) by r n+1 = ˆρ(x 1,..., x n, z 1,..., z n ). ( elicitability/identifiability) Forecast evaluation Given r 1,..., r n, evaluate the quality of the predictions (traditional backtesting). ( identifiability) Given r 1,..., r n, r 1,..., r n, say which method has better predictive power (comparative backtesting). ( elicitability) 5 / 32

12 Statistical aspects Returns {X t } t N (covariates {Z t } t N ) adapted to filtration {F t } t N Historical data: returns x 1,..., x n, (covariates z 1,..., z n ) Estimation Approximate ρ(x n+1 F n ) by r n+1 = ˆρ(x 1,..., x n, z 1,..., z n ). ( elicitability/identifiability) Forecast evaluation Given r 1,..., r n, evaluate the quality of the predictions (traditional backtesting). ( identifiability) Given r 1,..., r n, r 1,..., r n, say which method has better predictive power (comparative backtesting). ( elicitability) 5 / 32

13 Statistical aspects Returns {X t } t N (covariates {Z t } t N ) adapted to filtration {F t } t N Historical data: returns x 1,..., x n, (covariates z 1,..., z n ) Estimation Approximate ρ(x n+1 F n ) by r n+1 = ˆρ(x 1,..., x n, z 1,..., z n ). ( elicitability/identifiability) Forecast evaluation Given r 1,..., r n, evaluate the quality of the predictions (traditional backtesting). ( identifiability) Given r 1,..., r n, r 1,..., r n, say which method has better predictive power (comparative backtesting). ( elicitability) 5 / 32

14 Statistical aspects Returns {X t } t N (covariates {Z t } t N ) adapted to filtration {F t } t N Historical data: returns x 1,..., x n, (covariates z 1,..., z n ) Estimation Approximate ρ(x n+1 F n ) by r n+1 = ˆρ(x 1,..., x n, z 1,..., z n ). ( elicitability/identifiability) Forecast evaluation Given r 1,..., r n, evaluate the quality of the predictions (traditional backtesting). ( identifiability) Given r 1,..., r n, r 1,..., r n, say which method has better predictive power (comparative backtesting). ( elicitability) 5 / 32

15 Elicitability and identifiability Let A R be such that ρ: P A, P ρ(p) Definition A loss function L : A R R is consistent for ρ (relative to P), if E[L(ρ(P), X )] E[L(r, X )], P P, X P, r A. It is strictly consistent (relative to P) if = implies r = ρ(p). The risk measure ρ is called elicitable (relative to P) if there exists a loss function L that is strictly consistent for it. In other words ρ(p) = arg min EL(r, X ). r A 6 / 32

16 Elicitability and identifiability Let A R be such that ρ: P A, P ρ(p) Definition A loss function L : A R R is consistent for ρ (relative to P), if E[L(ρ(P), X )] E[L(r, X )], P P, X P, r A. It is strictly consistent (relative to P) if = implies r = ρ(p). The risk measure ρ is called elicitable (relative to P) if there exists a loss function L that is strictly consistent for it. In other words ρ(p) = arg min EL(r, X ). r A 6 / 32

17 Elicitability and identifiability Let A R be such that ρ: P A, P ρ(p) Definition ρ is identifiable (relative to P), if there is a function V : A R R such that for all P P, X P, r A. E(V (r, X )) = 0 r = ρ(x ), 7 / 32

18 Some examples Mean L(r, x) = (x r) 2 V (r, x) = x r Least squares regression Comparison of models/forecast performance in terms of MSE α-quantiles/var α (Median) L(r, x) = (1{x r} α)(r x) Quantile/Median regression α-expectiles (Newey and Powell, 1987) L(r, x) = 1{x r} α (r x) 2 V (r, x) = 1{x r} α (r x) Expectile regression V (r, x) = 1{x r} α 8 / 32

19 Elicitable and non-elicitable functionals Elicitable Mean, moments Median, quantiles/value-at-risk Expectiles (Newey and Powell, 1987) Not elicitable Variance Expected shortfall (Weber, 2006, Gneiting, 2011) Spectral risk measures (Weber, 2006, Z, 2014) 9 / 32

20 k-elicitability and k-identifiability Let A R k be such that ρ: P A, P ρ(p) Definition A loss function L : A R R is consistent for ρ (relative to P), if E[L(ρ(P), X )] E[L(r, X )], P P, X P, r = (r 1,..., r k ) A. It is strictly consistent (relative to P) if = implies r = ρ(p). The vector of risk measures ρ is called k-elicitable (relative to P) if there exists a loss function L that is strictly consistent for it. In other words ρ(p) = arg min EL(r, X ). r A 10 / 32

21 k-elicitability and identifiability Let A R k be such that ρ: P A, P ρ(p) Definition ρ is k-identifiable (relative to P), if there is a function V : A R R k such that E(V (r, X )) = 0 r = ρ(x ), for all P P, X P, r = (r 1,..., r k ) A. 11 / 32

22 Elicitable and identifiable functionals 1-Elicitable and 1-identifiable Mean, moments Median, quantiles/var Expectiles (Newey and Powell, 1987) 2-Elicitable and 2-identifiable Mean and variance Second moment and variance VaR and ES (Acerbi and Szekely, 2014, Fissler and Z, 2016) k-elicitable and k-identifiable Some spectral risk measures together with several VaRs at certain levels (Fissler and Z, 2016) 12 / 32

23 ρ = (VaR α, ES α ) Theorem (Fissler and Z, 2016) Let α (0, 1), and A 0 := {r = (q, e) R 2 : q e}. Let P be a class of probability measures on R with finite first moments and unique α-quantiles. Any loss function L: A 0 R R of the form L(q, e, x) = ( 1 α 1{x > q} ) g(q) + 1{x > q}g(x) ( ) + φ q (e) (1 α 1{x > q}) 1 α + 1{x > q} x 1 α e + φ(e) is consistent for ρ = (VaR α, ES α ) if 1 [q, ) g is P-integrable and g is increasing and φ is increasing and concave. It is strictly consistent if, additionally, φ is strictly increasing and strictly concave. 13 / 32

24 Evaluating forecasts of expected shortfall Filtration F = {F t } t N Prediction-observation triples (Q t, E t, X t ) t N Q t : VaR α prediction for time point t, F t 1 -measurable E t : ES α prediction for time point t, F t 1 -measurable X t : Realization at time point t, F t -measurable 14 / 32

25 Absolute evaluation Let V be an identification function for (VaR α, ES α ), i.e. ( ) 1 α 1{x > q} V (q, e, x) = A(q, e) q e 1, 1 α1{x > q}(q x) where A(q, e) R 2 2 with det(a(q, e)) 0. (We take A = I 2.) Definition (Calibration) The sequence of predictions {(Q t, E t )} t N is conditionally calibrated for (VaR α, ES α ) if E (V (Q t, E t, X t ) F t 1 ) = 0 for all t N. Compare Davis (2016). 15 / 32

26 Calibration and optimal prediction Given information F t 1 at time point t 1, the best prediction for (VaR α, ES α ) of X t is ( VaRα ( L(Xt F t 1 ) ), ES α ( L(Xt F t 1 ) )). This is the only F t 1 -measurable prediction which is conditionally calibrated. If F t 1 F t 1, then ( VaRα ( L(Xt F t 1) ), ES α ( L(Xt F t 1) )). is also conditionally calibrated (with respect to F t 1 ). 16 / 32

27 Calibration and optimal prediction Given information F t 1 at time point t 1, the best prediction for (VaR α, ES α ) of X t is ( VaRα ( L(Xt F t 1 ) ), ES α ( L(Xt F t 1 ) )). This is the only F t 1 -measurable prediction which is conditionally calibrated. If F t 1 F t 1, then ( VaRα ( L(Xt F t 1) ), ES α ( L(Xt F t 1) )). is also conditionally calibrated (with respect to F t 1 ). 16 / 32

28 Calibration and optimal prediction Given information F t 1 at time point t 1, the best prediction for (VaR α, ES α ) of X t is ( VaRα ( L(Xt F t 1 ) ), ES α ( L(Xt F t 1 ) )). This is the only F t 1 -measurable prediction which is conditionally calibrated. If F t 1 F t 1, then ( VaRα ( L(Xt F t 1) ), ES α ( L(Xt F t 1) )). is also conditionally calibrated (with respect to F t 1 ). 16 / 32

29 Traditional backtesting H C 0 : The sequence of predictions {(Q t, E t )} t N is conditionally calibrated. Backtesting decision: If we do not reject H0 C, the risk measure estimates are adequate. Many existing backtests can be described as a test for conditional calibration (with different choices for the identification function, different model assumptions). (McNeil and Frey, 2000, Acerbi and Szekely 2014) Does not give guidance for decision between methods. Does not respect increasing information sets. 17 / 32

30 Traditional backtesting H C 0 : The sequence of predictions {(Q t, E t )} t N is conditionally calibrated. Backtesting decision: If we do not reject H0 C, the risk measure estimates are adequate. Many existing backtests can be described as a test for conditional calibration (with different choices for the identification function, different model assumptions). (McNeil and Frey, 2000, Acerbi and Szekely 2014) Does not give guidance for decision between methods. Does not respect increasing information sets. 17 / 32

31 Constructing conditional calibration tests E(V (Q t, E t, X t ) F t 1 ) = 0 is equivalent to E(h t V (Q t, E t, X t )) = 0 for all F t 1 -measurable R 2 -valued functions h t. Choose F-predictable sequence {h t } t N of q 2-matrices h t of test functions: Ideally, the rows of h t generate F t 1. Construct test statistic: For example, n n ( 1 T 1 = n n where t=1 Ω n = 1 n ) ( 1 h t V (Q t, E t, X t ) Ω 1 n n t=1 ) h t V (Q t, E t, X t ), n (h t V (Q t, E t, X t ))(h t V (Q t, E t, X t )). t=1 Giacomini and White (2006) give general conditions under which T 1 is asymptotically χ 2 q. For these traditional ES α backtests, VaR α is also needed but they are robust with respect to model misspecification. 18 / 32

32 Constructing conditional calibration tests E(V (Q t, E t, X t ) F t 1 ) = 0 is equivalent to E(h t V (Q t, E t, X t )) = 0 for all F t 1 -measurable R 2 -valued functions h t. Choose F-predictable sequence {h t } t N of q 2-matrices h t of test functions: Ideally, the rows of h t generate F t 1. Construct test statistic: For example, n n ( 1 T 1 = n n where t=1 Ω n = 1 n ) ( 1 h t V (Q t, E t, X t ) Ω 1 n n t=1 ) h t V (Q t, E t, X t ), n (h t V (Q t, E t, X t ))(h t V (Q t, E t, X t )). t=1 Giacomini and White (2006) give general conditions under which T 1 is asymptotically χ 2 q. For these traditional ES α backtests, VaR α is also needed but they are robust with respect to model misspecification. 18 / 32

33 Constructing conditional calibration tests E(V (Q t, E t, X t ) F t 1 ) = 0 is equivalent to E(h t V (Q t, E t, X t )) = 0 for all F t 1 -measurable R 2 -valued functions h t. Choose F-predictable sequence {h t } t N of q 2-matrices h t of test functions: Ideally, the rows of h t generate F t 1. Construct test statistic: For example, n n ( 1 T 1 = n n where t=1 Ω n = 1 n ) ( 1 h t V (Q t, E t, X t ) Ω 1 n n t=1 ) h t V (Q t, E t, X t ), n (h t V (Q t, E t, X t ))(h t V (Q t, E t, X t )). t=1 Giacomini and White (2006) give general conditions under which T 1 is asymptotically χ 2 q. For these traditional ES α backtests, VaR α is also needed but they are robust with respect to model misspecification. 18 / 32

34 Examples of a traditional ES α backtests Mean of exceedance residuals (McNeil and Frey, 2000): h t = 1ˆσ ( ) Et V t t 1 α, 1, where ˆσ t is an F t 1 -measurable estimator of the volatility of X t. They assume a model of the form X t = µ t + σ t Z t to calculate the distribution of (1/n) n t=1 h tv (Q t, E t, X t ) under H C 0. h t = (0, 1). Other options: Acerbi and Szekely (2014) 19 / 32

35 A simulation study AR(1)-GARCH(1,1)-model: X t = µ t + ε t, µ t = X t 1, ε t = σ t Z t, σ 2 t = ε 2 t σ 2 t 1, (Z t ) iid with skewed t distribution with shape = 5 and skewness = 1.5. Estimation procedures: Fully parametric (n-fp, t-fp, st-fp) Filtered historical simulation (n-fhs, t-fhs, st-fhs) EVT based semi-parametric estimation (n-evt, t-evt, st-evt) Moving window of size 500 for estimation 5000 out-of-sample verifying observations 20 / 32

36 P-values of traditional backtests for (VaR α, ES α ) α = simple general n-fp n-fhs n-evt t-fp t-fhs t-evt st-fp st-fhs st-evt opt α = simple general / 32

37 Comparative evaluation Filtrations F = {F t } t N and F = {F t } t N Q t, Q t : VaR α predictions for time point t E t, E t : ES α predictions for time point t Q t, E t : internal model, F t 1 -measurable Q t, E t : standard model, F t 1 -measurable X t : Realization at time point t, F t -measurable and F t -measurable 22 / 32

38 Comparative evaluation Filtrations F = {F t } t N and F = {F t } t N Q t, Q t : VaR α predictions for time point t E t, E t : ES α predictions for time point t Q t, E t : internal model, F t 1 -measurable Q t, E t : standard model, F t 1 -measurable X t : Realization at time point t, F t -measurable and F t -measurable 22 / 32

39 Comparative evaluation Filtrations F = {F t } t N and F = {F t } t N Q t, Q t : VaR α predictions for time point t E t, E t : ES α predictions for time point t Q t, E t : internal model, F t 1 -measurable Q t, E t : standard model, F t 1 -measurable X t : Realization at time point t, F t -measurable and F t -measurable 22 / 32

40 Forecast dominance Let L be a consistent loss function for (VaR α, ES α ). Definition (S-Dominance) The sequence of predictions {(Q t, E t )} t N (L-)dominates {(Q t, E t )} t N if E(L(Q t, E t, X t ) L(Q t, E t, X t )) 0, for all t N. Diebold-Mariano tests for difference in predictive performance (Diebold and Mariano, 1995). Test statistic: n ˆΣ n 1 n n (L(Q t, E t, X t ) L(Qt, Et, X t )). t=1 Asymptotically normal under suitable conditions (Giacomini and White, 2006). 23 / 32

41 Forecast dominance Let L be a consistent loss function for (VaR α, ES α ). Definition (S-Dominance) The sequence of predictions {(Q t, E t )} t N (L-)dominates {(Q t, E t )} t N if E(L(Q t, E t, X t ) L(Q t, E t, X t )) 0, for all t N. Diebold-Mariano tests for difference in predictive performance (Diebold and Mariano, 1995). Test statistic: n ˆΣ n 1 n n (L(Q t, E t, X t ) L(Qt, Et, X t )). t=1 Asymptotically normal under suitable conditions (Giacomini and White, 2006). 23 / 32

42 Comparative backtesting H 0 H + 0 : Internal model dominates the standard model. : Internal model is dominated by the standard model. Backtesting decision using H 0 : If we do not reject H 0, the risk measure estimates are acceptable (compared to the standard). Backtesting decision using H 0 + : If we reject H+ 0, the risk measure estimates are acceptable (compared to the standard). Elicitability is crucial for robust comparative backtests. Allows for comparison between methods. Necessitates a standard reference model. Respects increasing information sets (Holzmann and Eulert, 2014). 24 / 32

43 Comparative backtesting H 0 H + 0 : Internal model dominates the standard model. : Internal model is dominated by the standard model. Backtesting decision using H 0 : If we do not reject H 0, the risk measure estimates are acceptable (compared to the standard). Backtesting decision using H 0 + : If we reject H+ 0, the risk measure estimates are acceptable (compared to the standard). Elicitability is crucial for robust comparative backtests. Allows for comparison between methods. Necessitates a standard reference model. Respects increasing information sets (Holzmann and Eulert, 2014). 24 / 32

44 Comparative backtesting H 0 H + 0 : Internal model dominates the standard model. : Internal model is dominated by the standard model. Backtesting decision using H 0 : If we do not reject H 0, the risk measure estimates are acceptable (compared to the standard). Backtesting decision using H 0 + : If we reject H+ 0, the risk measure estimates are acceptable (compared to the standard). Elicitability is crucial for robust comparative backtests. Allows for comparison between methods. Necessitates a standard reference model. Respects increasing information sets (Holzmann and Eulert, 2014). 24 / 32

45 Choice of a loss function for (VaR α, ES α ) A loss function L is called positively homogeneous of degree b if L(cr, cx) = c b L(r, x), for all c > 0. Important in regression, forecast ranking; implies unit consistency (Efron, 1991, Patton, 2011, Acerbi and Szekely, 2014). Let A = (R (0, )) A 0. There are positively homogeneous strictly consistent loss functions of degree b if and only if b (, 1)\{0}. There are strictly consistent loss functions such that the loss differences are positively homogeneous of degree b = 0: L 0 (q, e, x) = 1{x > q} x q e ( q ) + (1 α) e 1 + log(e) 25 / 32

46 Choice of a loss function for (VaR α, ES α ) A loss function L is called positively homogeneous of degree b if L(cr, cx) = c b L(r, x), for all c > 0. Important in regression, forecast ranking; implies unit consistency (Efron, 1991, Patton, 2011, Acerbi and Szekely, 2014). Let A = (R (0, )) A 0. There are positively homogeneous strictly consistent loss functions of degree b if and only if b (, 1)\{0}. There are strictly consistent loss functions such that the loss differences are positively homogeneous of degree b = 0: L 0 (q, e, x) = 1{x > q} x q e ( q ) + (1 α) e 1 + log(e) 25 / 32

47 Three zone approach for comparative backtesting H + 0 pass fail H 0 pass fail 1.64 Σ N 1.64 Σ L N 0 26 / 32

48 P-values of traditional backtests and L 0 -ranking for (VaR α, ES α ) α = simple general L 0 n-fp n-fhs n-evt t-fp t-fhs t-evt st-fp st-fhs st-evt opt α = simple general L / 32

49 st EVT opt st EVT opt ν = ν = internal model internal model n FP n FHS n EVT t FP t FHS t EVT st FP st FHS st EVT opt n FP n FHS n EVT t FP t FHS standard model t EVT st FP st FHS st EVT opt n FP n FHS n EVT t FP t FHS t EVT st FP st FHS st EVT opt standard model n FP n FHS n EVT t FP t FHS t EVT st FP st FHS st EVT opt ν = ν = / 32

50 Backtesting with small sample size (VaR 0.975, ES ) n-fp n-fhs n-evt t-fp t-fhs t-evt st-fp st-fhs st-evt opt n-fp n-fhs n-evt t-fp t-fhs t-evt st-fp st-fhs st-evt opt / 32

51 Conclusion Traditional backtests that are robust with respect to model misspecification necessitate an identifiable risk measure. Robust comparative backtests necessitate an elicitable risk measure. Robust traditional and comparative backtests for the whole tail of the distribution are also available. (Holzmann & Klar, 2017, Gordi, Lok & McNeil, 2017) k-elicitability allows to find loss functions for functionals that are not elicitable individually. A relevant example in banking and insurance is the non-elicitable risk measure ES α which is 2-elicitable with VaR α. 30 / 32

52 Conclusion Traditional backtests that are robust with respect to model misspecification necessitate an identifiable risk measure. Robust comparative backtests necessitate an elicitable risk measure. Robust traditional and comparative backtests for the whole tail of the distribution are also available. (Holzmann & Klar, 2017, Gordi, Lok & McNeil, 2017) k-elicitability allows to find loss functions for functionals that are not elicitable individually. A relevant example in banking and insurance is the non-elicitable risk measure ES α which is 2-elicitable with VaR α. 30 / 32

53 Outlook Characterization result for loss functions for (VaR α, ES α ) allows for Murphy diagrams: Forecast comparison without the choice of a specific loss function (Z, Krüger, Jordan, Fasciati, 2017). The loss functions for (VaR α, ES α ) allow for M-estimation (Zwingmann & Holzmann, 2016), generalized regression (Bayer & Dimitriadis, 2017, Barendse, 2017), semi-parametric time series models (Patton, Z, Chen, 2017) 31 / 32

54 Some references T. Fissler and J. F. Ziegel (2016). Higher order elicitability and Osband s principle. Annals of Statistics, 44: T. Fissler, J. F. Ziegel and T. Gneiting (2016). Expected Shortfall is jointly elicitable with Value at Risk Implications for backtesting. Risk Magazine, January N. Nolde and J. F. Ziegel (2016). Elicitability and backtesting. Annals of Applied Statistics, to appear with discussion. Thank you for your attention! 32 / 32