CVaR (Superquantile) Norm: Stochastic Case

Transcription

1 CVaR (Superquantile) Norm: Stochastic Case Alexander Mafusalov, Stan Uryasev Risk Management and Financial Engineering Lab Department of Industrial and Systems Engineering 303 Weil Hall, University of Florida, Gainesville, FL 36. s: Abstract The concept of Conditional Value-at-Risk (CVaR) is used in various applications in uncertain environment. This paper introduces CVaR (superquantile) norm for a random variable, which is by denition CVaR of absolute value of this random variable. It is proved that CVaR norm is indeed a norm in the space of random variables. CVaR norm is dened in two variations: scaled and non-scaled. L- and L-innity norms are limiting cases of the CVaR norm. In continuous case, scaled CVaR norm is a conditional expectation of the random variable. A similar representation of CVaR norm is valid for discrete random variables. Several properties for scaled and non-scaled CVaR norm, as a function of condence level, were proved. Dual norm for CVaR norm is proved to be the maximum of L- and scaled L-innity norms. CVaR norm, as a Measure of Error, is related to a Regular Risk Quadrangle. Trimmed L-norm, which is a non-convex extension for CVaR norm, is introduced analogously to function L-p for p <. Linear regression problems were solved by minimizing CVaR norm of regression residuals. Keywords: CVaR norm, L-p norm, superquantile, risk quadrangle, linear regression

2 . Introduction The concept of Conditional Value-at-Risk (CVaR) is widely used in risk management and various applications in uncertain environment. This paper introduces a concept of CVaR norm in the space of random variables. CVaR norm in R n was introduced and developed in (Pavlikov and Uryasev, 04; Gotoh and Uryasev, 03), and is a particular case of general error measures introduced and developed by (Rockafellar et al., 008). The term ¾superquantile, free from dependence on nancial terminology, can be used as a neutral alternative name for ¾CVaR, like it was done in (Rockafellar and Royset, 00; Rockafellar and Uryasev, 03). For the similar reason the alternative name ¾superquantile norm is proposed to replace ¾CVaR norm when desired. For the sake of consistency within the paper and with the earlier study (Pavlikov and Uryasev, 04), this paper will use mostly the ¾CVaR norm term. This section provides a short introduction in the CVaR norm in R n and shows the relation with the CVaR norm in the space of random variables. This paper is motivated by applications of norms in optimization. We consider norms in R n and in the space of random variables. We use symbols x and x i for a vector and an i-th vector component in R n, i.e. x = (x,..., x n ). We use symbol X for a random variable. l p norms are broadly used in R n, and L p norms are considered in the space of random variables. For p [, ], the norms l p and L p are dened as follows : l p (x) = ( n ) /p n x i p, L p (X) = (E X p ) /p, where E is the expectation sign. The most popular cases are p =,,, i.e., i= l (x) = n n i= x i, L (X) = E X ; l (x) = max i=,...,n x i, L (X) = sup X ; l (x) = ( n n i= x i ) /, L (X) = (EX ) /. It is known that l (x) l (x) l (x) and L (X) L (X) L (X), which follow from l p (x) l q (x) and L p (X) L q (X) for p < q, see (e.g. Brezis, 00, page 8). The other family of norms, CVaR norm for R n was dened in (Pavlikov and Uryasev, 04) and studied in (Gotoh and Uryasev, 03). According to (Pavlikov and Uryasev, 04, Denition 3), the non-scaled CVaR norm with parameter α, or x α, is dened on x R n as a sum of absolute values of biggest n( α) components. If n( α) is not an integer, then x α is dened as a weighted average of two norms x α and x α for closest values α, α such that n( α ) and n( α ) are integers. A similar norm, called D-norm, was introduced in (Bertsimas et al., 004, Section 3) in a dierent way. D-norm is dened as a maximum of a sum of weighted absolute values of vector components. The maximization is performed over all sets of indexes for components in the sum, with a constraint on cardinality. For α [ ] 0, n n, the CVaR norm x α coincides with the D-norm x p with parameter p dened by p = n( α), see (Pavlikov and Uryasev, 04, Proposition 3.4). (Bertsimas et al., 004, Proposition ) nds a dual norm to D-norm; this result was generalized with Item of Proposition. of this paper for the stochastic case. Both CVaR norm and D-norm can be viewed as important special cases of Ordered Weighting Averaging (OWA) operators, see (Yager, 00; Merig o and Yager, 03; Torra and Narukawa, 007). A subfamily of OWA operators with monotonically non-increasing weights, when implied to the absolute values of the vector, were formalized as norms in (Yager, 00). The worst-case averages, corresponding to CVaR, were also studied, e.g., in (Ogryczak and Zawadzki, 00; Romeijn et al., 005). The paper (Pavlikov and Uryasev, 04, Denition ) has also dened scaled CVaR norm x S α. Scaled version calculates average value of components instead of sum: n( α) x S α = x α. This Note that the classic denition for l p norm is l p(x) = ( n i= x i p) /p does not satisfy inequality l p(x) l q(x) for p < q. This paper uses an equivalent scaled version of this norm l p(x) = ( n n i= x i p) /p, which satises that inequality. L p norm is commonly dened as L p(f) = f p ( S f p dµ ) /p, where S is a considered space. It is known (e.g. Brezis, 00, page 8) that for p and q norms inequality f p µ(s) p q f q holds for p q, where S is a considered space and µ(s) is the measure of the space S. When S is a probability space, µ(s) = and inequality L p(x) L q(x) holds for p q, where L p(x) = (E X p ) /p.

3 paper denes (scaled) CVaR norm of a random variable X as an expectation of X in its right ( α)- tail. It can be shown that proposed norm is a generalization of x S α in a following way. Consider mapping X(x) : R n L (Ω) from Eucledian space of dimension n to the space of L -nite random variables. Denote x = (x,..., x n ). Let X(x) be discretely distributed over atoms x,..., x n with equal probabilities n. Then it is easy to see that x S α = CVaR α ( X(x) ) X(x) S α, see (Pavlikov and Uryasev, 04, Denition ). This paper also denes non-scaled CVaR norm X α = ( α) X S α, which corresponds to x α from (Pavlikov and Uryasev, 04). Non-scaled version has attractive properties with respect to parameter α, see Items 6,7 from Section. Risk Quadrangle considers risk R(X), deviation D(X), regret V(X), error E(X) and statistic S(X), related with a set of equations, called The General Relationships, see (Rockafellar and Uryasev, 03, Diagram 3). If a functional satises a corresponding set of axioms, it is called regular, see (Rockafellar and Uryasev, 03, Section 3). It can be proved that if R(X) is a coherent and regular Measure of Risk, then R( X ) is both a norm and a regular Measure of Error, however, this proof is beyond the scope of this paper. This paper proves that X α is a regular Measure of Error and nds the corresponding functions R(X), D(X), V(X) and S(X) in risk quadrangle related to the Measure of Error E(X) = X α, see Section. Item 3 from Proposition. can be viewed as a stochastic generalization of (Hall and Tymoczko, 0, Lemma ). Paper (Hall and Tymoczko, 0) considers functions Σ j (x) on nonnegative orthant R n +, corresponding to R n + reduction of special cases of CVaR norm. Paper (Hall and Tymoczko, 0) relies on majorization theory, see (Marshall et al., 0), which is generalized for the stochastic case with a concept of stochastic dominance, see (Ogryczak and Ruszczynski, 00; Dentcheva and Ruszczynski, 003, e.g.). This paper considers also non-convex functions closely related to CVaR norm. In deterministic case, by denition, CVaR norm is the average of biggest ( α)n absolute values of components of a vector. The trimmed L-norm is also known as a trimmed sum of absolute deviations estimator for LTA regression, see (Hawkins and Olive, 999; Bassett Jr, 99; H ossjer, 994, e.g.). L-based LTA regression is introduced similarly to the more common L-based Least-Trimmed-Squares (LTS) regression, see (Zabarankin and Uryasev, 04b, e.g.). Trimmed L-norm is denoted here by t α, it is the average of smallest αn absolute values of components of a vector. Calculation formulas and mathematical properties for t α (x) in Eucledian space and T α (X) in the space of random variables are considered in Section 3. Trimmed L-norm is also related to the sparse optimization, similar to functions l p for 0 < p <, see (Ge et al., 0). Note that the constraint on trimmed L-norm directly species sparsity of the solution vector (see Item 6 of Section 3), compared to ¾indirect sparsity specication with l p function. This paper also provides an illustration for t α in Euclidean space. Paper (Krzemienowski, 009, Denition ) denes conditional average CAVG function. Both average quantile and CVaR are subfamilies of CAVG family, therefore, both X S α and T α (X) are subfamilies of CAVG β,γ ( X ) function family. Unfortunately, these functions are not convex or concave in general, and are out of the scope of this study, although robust regression applications based on these functions is a promising research direction. The paper is organized as follows. Section gives a formal denition of CVaR norm in stochastic case and enlists various mathematical of CVaR norm, including that it is indeed a norm and a regular measure of error. CVaR norm is a parametric family of norms with respect to the condence parameter α, properties of CVaR norm as a function of α are proved. Dual representation of the CVaR norm is derived, and a dual norm to the CVaR norm is dened. A short introduction to the concept of Risk Quadrangle is given. We derive the quadrangle related to the CVaR norm as a measure of error and we prove that this quadrangle is regular. Section 3 denes the trimmed L-norm, both in R n and in the space of random variables, and enlists several basic properties. The trimmed L-norm is an extension of CVaR norm, but it is not actually a norm. Section 4 illustrates properties of CVaR norm with a case study. Section 5 provides concluding remarks and acknowledgements.. CVaR (Superquantile) Norm Properties and Connection to Risk Quadrangle This section gives a formal denition of CVaR norm in stochastic case and proves various properties of the norm. Let us denote [x] + = max0, x}, [x] = max0, x}. Consider cumulative distribution function F X (x) = P (X x). If, for a probability level α (0, ), there is a unique x such that F X (x) = α, then this x is called the α-quantile q α (X). In general, however, the value x is not unique, or 3

4 may not even exist. There are two boundary values: q + α (X) = infx F X (x) > α}, q α (X) = supx F X (x) < α}. We will call by the quantile the entire interval between the two boundary values, q α (X) = [q α (X), q + α (X)]. () We will use notation q p (X)dp q p (X)dp, which is reasonable since q + p (X)dp = q p (X)dp. The CVaR norm is dened as follows. Denition. Let X be a random variable with E X <. Then CVaR (superquantile) norm of X with parameter α [0, ] is dened by X S α = CVaR α ( X ) = q α ( X ). Following the logic of (Pavlikov and Uryasev, 04), X S α is called scaled CVaR norm, while Denition introduces X α, corresponding to non-scaled CVaR norm for R n in (Pavlikov and Uryasev, 04, Denition 3). By default we call by CVaR norm the function X S α. The second version of the norm, non-scaled CVaR norm, is dened as follows. Denition. Let X be a random variable with E X <. Then non-csaled CVaR (superquantile) norm of X with parameter α [0, ) is dened as follows: X α = ( α) X S α. Note that by continuity X lim α X α = 0. That is, for α =, the function X α is not a norm. Recall some properties of CVaR (e.g. Rockafellar and Uryasev, 03, page 0), see also (Rockafellar and Uryasev, 000, 00): Positive homogeneity: Subadditivity: Monotonicity: CVaR α (λx) = λcvar α (X), for λ > 0. () CVaR α (X + Y ) CVaR α (X) + CVaR α (Y ). (3) CVaR α (X) CVaR α (Y ), for X Y. (4) Properties and Representations of CVaR (Superquantile) Norm. X S 0 = L (X), X S = L (X). (Follows from Denition ).. X S α is a norm on L (Ω) space of random variables. (Norm positive homogeneity and convexity follow from the positive homogeneity and convexity of the CVaR and the absolute value, whereas CVaR α ( X ) = 0 X 0 is obvious.) 3. The representaions X S α = min c X S α = α c + α } E[ X c]+. (5) α q p ( X )dp. (6) X S α = E ( X X > q α ( X )), if X is a continuous random variable, (7) follow directly from CVaR calculation formulas, see (e.g. Rockafellar and Uryasev, 000, formulas (3),(5)), (e.g. Rockafellar and Uryasev, 00, Denitions 3,4, Theorem 0). Also, X S X, with probability α = CVaR (+α)/ (Y ), where Y = ; X, with probability ; (8) see the proof in Appendix A. 4

5 4. Let X be a discrete random variable, i.e., it takes values x i } N i= with positive probabilities p i} N i=, where N i= p i = and N N (N also can be ). Let us denote by x (i) } N i= an ordered sequence x i } N i=, i.e., x (i) x (i+). We also denote by p (i) } N i= a corresponding to the x (i) } N i= sequence of probabilities from the p i} N i=. Then (a) for α =, X S x (N), for N < ; = lim i x (i), for N = ; (b) for α j = j i= p (i) and j < N, j Z + N 0}, (c) for α j < α < α j+, X S α j = α j N i=j+ x (i) p (i), for N < ; α j i=j+ x (i) p (i), for N = ; X S α = ( λ) α j α X S α j + λ α j+ α X S α j+, where λ = α α j α j+ α j. See the proof in Appendix A. 5. X S α is a continuous increasing function of α. (Follows from the fact that CVaR is a continuous increasing function of α, see (Rockafellar and Uryasev, 00, Proposition 3).) 6. X α is a concave and decreasing function of α. (The integral α q p( X )dp is a convex function of α as shown in (Ogryczak and Ruszczynski, 00, page 6), and X α = E X α q p( X )dp.) 7. X α and ( α)cvar α (X) are piecewise-linear functions of α for a discretely distributed random variable X. (Since quantile function for discretely distributed random variable is a step function, then its integral is piecewise-linear, and X α = E X α q p( X )dp.) 8. The norm X S α generates a Banach space for α [0, ]. (From the denition of Banach space 3 follows that since L (X) = E X generates 4 a Banach space and E X X S α α X S α = L (X) for α =.) 9. E(X) = X S α is a regular measure of error. See the proof in Appendix A. E X, then for α [0, ) CVaR norm also generates a Banach space, and The asterisk denotes the dual norm 5 to a norm. Therefore, Y S α denotes the norm dual to the CVaR norm X S α. Proposition.. For X L (Ω) and the norm S α, the following statements hold: }. X S α = sup Y Y EXY, where Y = Y Y α, E Y = Y } EXY X S α is a closed convex set. Note that ordered sequence x (i) } may not exist for some sets x i }. As a counterexample, consider x i = i, for i =,...,. 3 A Banach space is a vector space X over R, which is equipped with a norm and which is complete with respect to that norm. By denition, completeness means that for every Cauchy sequence x n} n= in X (i.e., for every ε > 0 exists N such that x m x n < ε for all m, n > N), there exists an element x in X such that lim xn = x, i.e., lim xn x = 0. n n 4 We say that norm L generates space (X L, L), where X L = X L(X) < }. 5 Let X be a normed space over R with norm (i.e., X R for X X). Then, the dual (or conjugate) normed space X is dened as the set of all continuous linear functionals from X into R. For f X, the dual norm of f is dened by } f(x) f = sup f(x) : x X, x } = sup x : x X, x 0. 5

6 . The norm Y S α = maxe Y, ( α) sup Y } is dual to the norm X S α for α (0, ). 3. Problem min d R X d S α has the following solution: arg min X d S α = ( q( α)/ (X) + q (+α)/ (X) ), d min X d S α = ( + α d α CVaR ( α)/ (X) + α CVaR (+α)/ (X) EX ). Proof.. Papers (Rockafellar et al., 006), (Rockafellar and Uryasev, 03, (6.9)) proved that CVaR α (X) = sup Q Q EXQ, where Q = Q 0 Q } α, EQ =. Denote Y = Y Y α, E Y } and Y = Y EXY X S α }. Then X S α = sup E X Q sup EXY sup E X Y sup E X Q. Q Q Y Y Y Y Q Q The rst inequality holds since for any Q Q exists Y = (I(X > 0) I(X < 0))Q Y such that X Q = XY. Hence, X S α = sup Y Y EXY. Note that X S α = sup Y Y EXY. Then, Y is a closed } convex hull of Y. Y is closed convex as an intersection of two closed convex sets Y Y α and Y E Y }. That is, Y = Y. }. Item implies that Y = Y Y α, E Y is a unit ball for the dual norm Y S α = EXY sup X 0. Then, the unit sphere Y S X α = for the dual norm is the set S α } } Y sup Y = α, E Y Y sup Y α, E Y =. Therefore, the dual norm equals Y S α = maxe Y, ( α) sup Y }. 3. Note that for c 0 and arbitrary x, d R, we have d c x, for x d c; [ x d c] + = 0, for x [d c, d + c]; [d c x] + + [x (d + c)] + = x (d + c), for x d + c. = [x (d c)] + + d c x + [x (d + c)] +. Let c = d c and c = d + c, then this relationship yields ( α)c + [ x d c] + = ( α) c c = + α + [x c ] + + c x + [x c ] + = c + [x c ] + + α c + [x c ] + x. (9) An optimal value c in X S α = min c R c + ( α) E[ X c] +} is the quantile q α ( X ) and hence nonnegative. Thus, c can be restricted in this optimization problem to be nonnegative, and, applying (9), min X d R d S α = min c + ( α) E[ X d c] +} = c 0,d R + α = α min c,c R = ( + α α c + E[X c ] + + α } c + E[X c ] + EX CVaR ( α)/ (X) + α CVaR (+α)/ (X) EX where optimal ( c and c are determined by q ( α)/ (X) and q (+α)/ (X), respectively, and yield optimal d = q( α)/ (X) + q (+α)/ (X) ). ), = 6

7 Risk Quadrangle (Rockafellar and Uryasev, 03) relates risk R(X), deviation D(X), regret V(X), error E(X) and statistic S(X) with the following equations (Rockafellar and Uryasev, 03, Diagram 3): V(X) = EX + E(X), R(X) = EX + D(X), (0) D(X) = mine(x C)}, R(X) = minc + V(X C)}, () C C S(X) = arg mine(x C)} = arg minc + V(X C)}. () C Following the paper (Rockafellar and Uryasev, 03, Section 3), we consider the L (Ω) space of random variables with nite second moment, EX <, which implies nite rst moment, E X <. The natural (¾strong ) convergence in L (Ω) of a sequence of random variables X k to a random variable X is characterized as follows: L - lim k Xk = X lim k L (X k X) = 0. The functional F is closed if for any C R the set X F(X) C} is closed with respect to L - convergence. The functional F is convex if F(λX + ( λ)y ) λf(x) + ( λ)f(y ) for all X, Y and λ (0, ). Measure of error E(X) is regular if: ) E(X) [0, ]; ) E(X) is closed convex; 3) E(0) = 0; 4) E(X) > 0 for any X 0; 5) E(X) ψ(ex) with a convex function ψ on (, ) having ψ(0) = 0 but ψ(t) > 0 for t 0. Denitions for regular measures of risk, deviation and regret are available in (Rockafellar and Uryasev, 03, Section 3). The quadrangle (R, D, E, V, S) is regular if equations (0) () hold and if also R(X) is a regular measure of risk, D(X) is a regular measure of deviation, V(X) is a regular measure of regret, and E(X) is a regular measure of error. (Rockafellar and Uryasev, 03, Quadrangle Theorem) implies that if equations (0)() hold for functions R, D, E, V, S, and if also E(X) is a regular measure of error, then (R, D, E, V, S) is a regular quadrangle. Since X S α is a regular measure of error, then X α is a regular measure of error, and the quadrangle, related to CVaR norm as a measure of error, is regular. If E(X) = X α and equations (0)() hold, then the corresponding measure of risk and statistic are calculated from Item 3 of Proposition., and the whole corresponding quadrangle is presented below. Proposition. (CVaR (Superquantile) Norm Quadrangle). For α [0, ) the error measure E(X) = X α is related to the following regular quadrangle: S(X) = ( q( α)/ (X) + q (+α)/ (X) ), R(X) = α CVaR (+α)/ (X) + + α CVaR ( α)/ (X), D(X) = α CVaR (+α)/ (X EX) + + α CVaR ( α)/ (X EX), V(X) = X α + EX, E(X) = X α. C CVaR norm quadrangle is similar to the Mixed-Quantile-Based quadrangle, see (Rockafellar and Uryasev, 03), for α = ( + α)/, α = ( α)/, λ = ( α)/, λ = ( + α)/. (3) For k =, dene [ ] αk E αk (X) = E X + + X, V αk (X) = EX +. α k α k With parameters from (3) we obtain the Mixed-Quantile-Based quadrangle with the same risk and deviation as in CVaR norm quadrangle, but with dierent statistic, error and regret: S(X) = α q (+α)/ (X) + + α q ( α)/ (X), V(X) = min λ V α (X B ) + λ V α (X B ) λ B + λ B = 0}, B,B E(X) = min B,B λ E α (X B ) + λ E α (X B ) λ B + λ B = 0}. 7

8 Suppose one is optimizing measure of error over some parametric family X(θ): min θ E i (X(θ)), (4) where i = for error from CVaR norm quadrangle, and i = for error from Mixed-Quantile-Based quadrangle. Assume that X(θ) = θ 0 + Y ( θ), where θ = (θ 0, θ), and θ 0 is a free parameter. Dene θ i = arg min θ E i (X(θ)). Then θ = θ = arg min θ D(Y ( θ)) = θ. Therefore, Y ( θ ) = Y ( θ ) and two optimal points X(θ ) and X(θ ) for problems (4) can be obtained from each other by adding a constant shift X(θ ) = (θ 0) + Y ( θ ), X(θ ) = (θ 0) + Y ( θ ), X(θ ) X(θ ) = (θ 0) (θ 0). 3. Trimmed L-Norm Paper (Ge et al., 0) considers a class of functions dened similar to L p norms, but for p [0, ). These functions are not norms and they are concave for some regions of the space they are dened 6. Such functions are used in optimization problems to achieve a sparse solution vector. We will dene a similar functions in terms of CVaR concept. Further we dene trimmed L-norm. Contrary to CVaR norm, this function takes average over smallest α-fraction of absolute values X of a random variable X. The word ¾norm here is potentially deceptive, since it corresponds to L and not the resulting function itself: trimmed L-norm is not actually a norm. The term ¾trimmed is widely used in robust regression, when the average of a few smallest regression residuals is minimized. Before averaging, residuals are usually transformed with some function φ : R [0, ). The case of φ(x) = x is the most commonly used and corresponds to the least trimmed squares regression (LTS), see (Rousseeuw, 984; Rousseeuw and Van Driessen, 999; Zabarankin and Uryasev, 04b). The case of φ(x) = x corresponds to the least trimmed sum of absolute deviations (LTA) regression, see (Hawkins and Olive, 999; Bassett Jr, 99; H ossjer, 994), it also corresponds to the trimmed L-norm function here, see below. The general case for arbitrary function φ was described in (Neykov et al., 0, formula (3)) and formalized for random variables via average quantile function in (Zabarankin and Uryasev, 04b, problem (6.5.9)). One possible way to dene trimmed L-norm, or T α, is as follows. Denition 3. Let X be a random variable with E X <. Trimmed L-norm T α (X) for α [0, ] is dened by T α (X) = CVaR α ( X ). Below we provide some mathematical properties for trimmed L-norm. These properties mostly follow from the ones presented in Section. Properties and Representations of Trimmed L-Norm. For α = 0, trimmed L-norm T α (X) = inf X. For α (0, ] trimmed L-norm can be calculated using one of the formulas below: T α (X) = α (E X ( α)cvar α( X )), (5) T α (X) = α α T α (X) = max c 0 q p ( X )dp, (6) c α } E[ X c]. (7). 0 T α (X) L (X) = E X, 3. T α (λx) = λ T α (X), 4. if XY 0, then T α (λx + ( λ)y ) λt α (X) + ( λ)t α (Y ), 5. T α (0) = 0, however, there exists X 0 such that T α (X) = 0, 6 For p [0, ) there is l p(x) in R n and L p(x) in the space of random variables. Concavity holds, for example, for region x 0 in R n and for region X 0 in the space of random variables. 8

9 6. T α (X) 0 T α (X) = 0 implies that P (X = 0) α. 7. T α (X) is a continuous non-decreasing function w.r.t. α. 8. αt α (X) is a convex non-decreasing function w.r.t. α. Formula (5) follows from (Pug, 000, Proposition (iii)), when Y = X is taken. T α (X) can be interpreted as an expectation of X in left α-tail. Note that T α (X) is then the average quantile of the random variable X, hence, formula (6) holds, see (Zabarankin and Uryasev, 04a, formula (.4.)). Formula (7) follows from (Rockafellar et al., 006, formula (5)). For p (0, ) the following inequality holds L p (X) L (X), where L p (X) = (E X p ) /p. Since x p is a concave function for 0 < p <, using Jensen's inequality we have E X p (E X ) p, therefore, (E X p ) /p E X, and Item shows that for trimmed L-norm. Items 4 follow from the fact that α α 0 q p(x)dp is a coherent, expectation bounded risk measure, see (Rockafellar et al., 006, Example 3). Item 5 follows from example of X = 0 with probability 0.5 and X = with probability 0.5. Then, for α [0, 0.5] function T α (X) = 0. Item 6 follows from formula (6). Note that Item 6 can be used in optimization problem settings for a constraint T α (X) 0 to achieve a given sparsity of a solution variable vector, or as max α subject to T α (X) 0 to maximize the sparsity of a solution variable vector. Item 4 implies that T α (X) is a concave function for X 0. Notice that this property cannot be strengthened to concavity in the whole space of random variables. Consider a function g(x) such that g(x) 0, g(0) = 0 and g(x) 0. Assume that g(x) is concave in the space of random variables. Since g(x) 0, then there exists X such that g(x) > 0. Then g(x) + g( X) > 0 = g(0) = g(x X), which implies that g(x) is not a concave function. Consider x = (x,..., x n ) R n. Let X(x) be a discretely distributed random variable taking values x,..., x n with equal probabilities n. Then the trimmed L-norm on Rn is dened as t α (X) = T α (X). Properties of t α (x) are similar to properties of the T α (X) and follow directly from Denition 3 and enlisted properties. For x R n, trimmed L-norm is calculated as follows: t α (x) = α (l (x) ( α) x S α), t αj (x) = ( x () x (j) )/j, for α = α j = j/n, t 0 (x) = min x i, i for α = 0, t (x) = l (x), for α =. Figure shows level-sets of x S α and t α (x) in R for dierent values of α. The function t α ( ) is a natural extension of S α. When α varies from 0 to, the function t α (x) changes from min i x i to l (x) = n n i= x i, and the function x S α changes from l (x) to max i x i. 4. Case Study 4.. Linear Regression: Financial Optimization Dataset We illustrate CVaR norm quadrangle, see Proposition., with the following case study. The case study results, data, and codes are posted at this link 7. We considered a linear regression problem with CVaR norm error. Let X be a n d design matrix, where n is a number of observations, d is a number of explanatory variables. Let y R n be a vector of observations on the dependent variable. Let e R n be a vector of ones. We denote by X = [e, X] an extended design matrix including additional unit column. We considered a linear regression: ŷ = Xa, where a R d+ is a vector of parameters. To solve this regression problem we minimized CVaR norm of vector of residuals y ŷ: min y Xa α. (8) a R d+ 7 cvar-norm-regression/ 9

10 3.5 x S 0.5 = max( x, x )= x S 0.5 = x S 0 = l (x) = t (x)= t 0.75 (x)= t 0.5 (x) = min( x, x )= x x Figure : Level-sets of CVaR norm x S α for α = 0, 0.5, 0.5 and level-sets of trimmed L-norm t α(x) for α = 0.5, 0.75, in R space. For α [0.5, ] norm x S α = max i x i. For α [0, 0.5] function t α(x) = min i x i. Equality x S 0 = l (x) = t (x) holds. 0

11 rlv rlg ruj ruo intercept objective Table : Optimal vector of parameters and objective for linear regression with CVaR norm. It is desirable to use CVaR norm in regression when we want to control directly large absolute values of residuals. We are indierent to the sign of the residual. We just do not want to have large absolute values, but are tolerant to small absolute values. Similar purpose can be achieved by minimizing L p norm, but in this case we do not control directly some specic percentage of largest outcomes. We can directly specify the percentage of larges absolute residuals with CVaR norm, e.g., 0% of largest outcomes. We also want to mention that the percentile regression (Koenker and Bassett Jr, 978) with Koenker and Bassett function is quite close to CVaR norm regression. However, percentile regression is concentrated on large outcomes in one tail, while CVaR norm regression pays attention to large outcomes without identifying the sign of the residual. Similar type of error has been considered earlier in OR literature. For instance, (Zabarankin and Uryasev, 04b, formula (6.5.9)) considered so called ¾average alfa-quantile error minimization applied to the transformed residual. In this problem, the average is taken over the left tail of the distribution. Such approach corresponds to trimmed error measures and to robust regression, it produces regression which is stable to outliers. By selecting the absolute value as a transformation function in ¾average alfaquantile error we are coming to trimmed L-norm minimization, or to LTA regression. Here we consider averaging over the right tail of distribution, which leads to CVaR, or superquantile, functions. Similarly, by selecting the absolute value as a transformation function in CVaR we are coming to CVaR norm minimization. CVaR norm regression is not stable to outliers, moreover, it is a ¾pessimistic estimator focused is on a fraction of the most ¾problematic observations. Aside from potential benets of the pessimistic approach, problem (8) is a convex problem and can be solved precisely and eciently. We consider the dataset from the case study ¾Estimation of CVaR through Explanatory Factors with Mixed Quantile Regression 8. The data contains returns of the Fidelity Magellan Fund as a dependent variable. Russell Value Index (RUJ), Russell 000 Value Index (RLV), Russell 000 Growth Index (RUO) and Russell 000 Growth Index (RLG) are taken as independent variables. Data include,64 observations. Solving Time on a PC with.83ghz is 0.0 sec. The CVaR norm is minimized with Portfolio Safeguard (AORDA, 009) software package. Condence level α in CVaR norm equals α = 0.9. We minimized CVaR instead of CVaR norm, according to calculation formula (8). Denote ȳ = [y; y] R n and X = [ X; X] R n d. Formula (8) implies Then, problem (8) is equivalently stated as follows y Xa S α = CVaR (+α)/ (ȳ Xa). min a R d+ CVaR (+α)/ (ȳ Xa). Optimization results for this problem are in Table. CVaR norm quadrangle is a regular quadrangle, see Proposition.. According to (Rockafellar and Uryasev, 03, Regression Theorem), rst stated as the Error-Shaping Decomposition of Regression Theorem (Rockafellar et al., 008, Theorem 3., page 7), the intercept, obtained in regression, equals to the Statistic of a modied residuals. In CVaR norm quadrangle, statistic equals S(X) = (q (+α)/ (X) + q ( α)/ (X))/. Denote the optimal vector of parameters obtained in regression by a = [c, b ], where c R is an optimal intercept. According to the Regression Theorem, c S(y Xb ) (we write because, in general, quantile q p (X) is an interval, see (), therefore S(X) is also an interval). At the optimal point, c = 0.00, q0.05 (y Xb ) = 0.03, q0.95 (y Xb ) = Therefore, S(y Xb ) (q 0.05 (y Xb ) + q 0.95 (y Xb ))/ = ( )/ = 0.00 = c. That is, numerical experiment conrms theoretical results for CVaR norm quadrangle, namely statistics S(y Xb ) = (q ( α)/(y Xb ) + q (+α)/ (y Xb )) including the optimal intercept, c. 8 estimation-of-cvar-through-explanatory-factors-with-mixed-quantile-regression/

12 y x y x y x y x Figure : Samples generated for regression. A true law is the red line. Observations are the blue dots. 4.. Linear Regression on Simulated Data In the following case study we consider L-norm as an out-of-sample criterion. For the in-sample criteria we consider elements of the CVaR norm parametric family. L-norm is an element of this family with the corresponding parameter value 0. We vary value of the parameter between 0 and for in-sample learning to optimize L-norm of residuals in out-of-sample. We show that using CVaR norms with parameter value bigger than 0 can lead, for small training samples, to better out-of-sample performance than direct in-sample minimization of L-norm. We illustrate CVaR norm regression with a controlled numerical experiment. In this case study we set true law as y(x) = x for x [0, ]. We pick sample points (x,..., x ) = (0, 0.,..., 0.9, ). Then we simulate dependent variable as y i = x i + ε i, where error terms ε i Laplace(0, 0.5) are distributed according to the ( Laplace ) distribution (double exponential distribution, probability density function f(x; µ, b) = b exp x µ b ). L-norm of regression residuals is chosen as the out-of-sample error criterion. Similar to exponential distribution, Laplace distribution has heavy tails, which makes L-based regression a reasonable choice. Moreover, minimization of L-norm of residuals is equivalent to likelihood maximization for the case of Laplace-distributed error. Since the true distribution is known, there is no need to divide sample into training and testing subsamples to measure error of the estimated model. Let ŷ i denote model estimation for the data point x i. First, ¾expected error is calculated as EE = n n E ε y(x i ) + ε ŷ i, i= where n = and each expectation is taken for ε Laplace(0, 0.5). Also, since the true law is know, then ¾true error is calculated as TE = n y(x i ) ŷ i. n i= As in Section 4., the problem (8) is solved. L-norm minimization is a special case of (8) when α = 0. The goal of the case study is to test whether ¾pessimistic estimation provided by CVaR norm minimization with α > 0 can achieve better out-of-sample quality of estimators, measured by EE and TE, than direct L-norm minimization with α = 0. To smooth the results obtained with the randomly generated sample, number of dierent samples generated N = 000, each sample size n =. For each sample, the problem (8) is solved for values α k = (k )/(K ) for k =,..., K, K = (that is, values of α are (0, 0.05, 0.0,..., )). For sample j, where j N, and parameter α k, regression estimator is found, and errors EE k j and TE k j are calculated. Figure 3 shows dependence on α for average error and standard deviation of error, scaled to the average error for α = 0 and the standard deviation of error for α = 0 correspondingly. These values are calculated for the two types of error we consider: ¾expected error EE, and ¾true error TE. Minimal average EE is achieved for α = 0.5 and is % lower than average EE for α = 0. Despite the modest drop in average error, CVaR norm model is more stable than L-norm model: standard deviation of EE is 0% lower for α = 0.5 than for α = 0. Average TE is minimized by the same value α = 0.5 and is 5% lower than for α = 0; standard deviation of TE is 8% lower for α = 0.5 than for α = 0. This case study showed that even if CVaR norm is not an out-of-sample criterion itself, as it was assumed in Section 4., the minimization of CVaR norm can be more advantageous than direct optimization of the out-of-sample criterion.

13 3 EE Average and Deviation. TE Average and Deviation.5 average deviation.8 average deviation error.5 error α α Figure 3: Average error and standard deviation of error, scaled to the average error for α = 0 and the standard deviation of error for α = 0 correspondingly, as functions of α. On the left: ¾Expected Error. On the right: ¾True Error. 5. Conslusion This paper extended the denition of CVaR norm from Eucledian space to a L (Ω) space of random variables and proved that CVaR norm is indeed a norm. CVaR norm is dened in two variations: scaled and non-scaled. Several properties for scaled and non-scaled CVaR norm, as a function of condence level, were proved. Dual norm for CVaR norm was proved to be the maximum of L and scaled L norms. CVaR norm was proved to be a regular Measure of Error and components of the corresponding CVaR (Superquantile) Norm Risk Quadrangle were found. Trimmed L-norm, which is a non-convex extension for CVaR norm, was introduced analogously to function L p for p <. Linear regression problems were solved by minimizing CVaR norm of regression residuals. CVaR norm has an intuitive interpretation to be chosen as an out-of-sample criterion, and also minimization of CVaR norm can be more advantageous than direct optimization of the out-of-sample criterion. Acknowledgements: Authors would like to thank Prof. R. Tyrrell Rockafellar, Prof. Michael Zabarankin, Prof. Jun-Ya Gotoh and Dr. Konstantin Pavlikov for their helpful comments and suggestions. Authors would also like to thank Prof. Georg Pug and two anonymous reviewers for their tremendous help shaping the material of the paper in a clearer way and for the help connecting this study with existing work in related elds. This work was partially supported by the USA AFOSR grants: ¾Design and Redesign of Engineering Systems, FA , and ¾New Developments in Uncertainty: Linking Risk Management, Reliability, Statistics and Stochastic Optimization, FA References AORDA (009). Portfolio Safeguard (PSG) version.. American Optimal Decisions, Inc., Gainesville, FL. http: // Bassett Jr, G. W. (99). Equivariant, monotonic, 50% breakdown estimators. The American Statistician, 45():3537. Bertsimas, D., Pachamanova, D., and Sim, M. (004). Robust linear optimization under general norms. Operations Research Letters, 3(6):5056. Brezis, H. (00). Functional Analysis, Sobolev Spaces and Partial Dierential Equations. Springer Science & Business Media. Dentcheva, D. and Ruszczynski, A. (003). Optimization with stochastic dominance constraints. SIAM Journal on Optimization, 4(): Ge, D., Jiang, X., and Ye, Y. (0). A note on the complexity of lp minimization. Math. Program., 9():8599. Gotoh, J.-y. and Uryasev, S. (03). Two pairs of families of polyhedral norms versus lp-norms: Proximity and applications in optimization. University of Florida, Research Report TwoPairsOfPolyhedralNormsVersusLpNorms_0305_UFISEPD.pdf. Hall, R. W. and Tymoczko, D. (0). Submajorization and the geometry of unordered collections. The American Mathematical Monthly, 9(4):

14 Hawkins, D. M. and Olive, D. (999). Applications and algorithms for least trimmed sum of absolute deviations regression. Computational Statistics & Data Analysis, 3():934. H ossjer, O. (994). Rank-based estimates in the linear model with high breakdown point. Journal of the American Statistical Association, 89(45):4958. Koenker, R. and Bassett Jr, G. (978). Regression quantiles. Econometrica: journal of the Econometric Society, pages Krzemienowski, A. (009). Risk preference modeling with conditional average: An application to portfolio optimization. Annals of Operations Research, 65():6795. Marshall, A. W., Olkin, I., and Arnold, B. C. (0). Inequalities: Theory of Majorization and Its Applications. Springer, New York. Merig o, J. M. and Yager, R. R. (03). Norm aggregations and OWA operators. In Aggregation Functions in Theory and in Practise, pages 45. Springer. Neykov, N., C zek, P., Filzmoser, P., and Neytchev, P. (0). The least trimmed quantile regression. Computational Statistics & Data Analysis, 56(6): Ogryczak, W. and Ruszczynski, A. (00). Dual stochastic dominance and related mean-risk models. SIAM Journal on Optimization, 3():6078. Ogryczak, W. and Zawadzki, M. (00). Conditional median: A parametric solution concept for location problems. Annals of Operations Research, 0(-4):678. Pavlikov, K. and Uryasev, S. (04). CVaR norm and applications in optimization. Optimization Letters, 8(7): Pug, G. C. (000). Some remarks on the value-at-risk and the conditional value-at-risk. In Probabilistic constrained optimization, pages 78. Springer. Rockafellar, R. T. and Royset, J. O. (00). On buered failure probability in design and optimization of structures. Reliability Engineering and System Safety, 95(5): Rockafellar, R. T. and Uryasev, S. (000). Optimization of conditional value-at-risk. Journal of Risk, :4. Rockafellar, R. T. and Uryasev, S. (00). Conditional value-at-risk for general loss distributions. Journal of banking & nance, 6(7): Rockafellar, R. T. and Uryasev, S. (03). The fundamental risk quadrangle in risk management, optimization and statistical estimation. Surveys in Operations Research and Management Science, 8():3353. Rockafellar, R. T., Uryasev, S., and Zabarankin, M. (006). Generalized deviations in risk analysis. Finance and Stochastics, 0():574. Rockafellar, R. T., Uryasev, S., and Zabarankin, M. (008). Risk tuning with generalized linear regression. Mathematics of Operations Research, 33(3):779. Romeijn, H. E., Ahuja, R. K., Dempsey, J. F., and Kumar, A. (005). A column generation approach to radiation therapy treatment planning using aperture modulation. SIAM Journal on Optimization, 5(3): Rousseeuw, P. J. (984). Least median of squares regression. Journal of the American statistical association, 79(388): Rousseeuw, P. J. and Van Driessen, K. (999). Computing LTS regression for large data sets. In Institute of Mathematical Statistics Bulletin. Citeseer. Torra, V. and Narukawa, Y. (007). Modeling decisions: Information Fusion and Aggregation Operators. Springer, Berlin. Yager, R. R. (00). Norms induced from OWA operators. Fuzzy Systems, IEEE Transactions on, 8():5766. Zabarankin, M. and Uryasev, S. (04a). Random variables. In Statistical Decision Problems, pages 37. Springer. Zabarankin, M. and Uryasev, S. (04b). Regression models. In Statistical Decision Problems, pages 787. Springer. Appendix A. Proofs for Properties of CVaR (Superquantile) Norm Proof for Item 3 from Section Proof. Formulas (5)(7) follow directly from CVaR calculation formulas, see (e.g. Rockafellar and Uryasev, 000, formulas (3),(5)), (e.g. Rockafellar and Uryasev, 00, Denitions 3,4, Theorem 0). Let us prove formula (8). For any x 0, hence, and nally, P (X x) + P ( X x) F Y (x) = P (Y x) = = P (X x) + P (X < x) + P ( x X x) = = q + (+α)/ x (Y ) = inf + F X (x) X S α = α = α α = + F X (x), > + α } = inf x F X (x) > α } = q α + ( X ), q + p ( X )dp = α (+α)/ α q + (+p)/ (Y )d ( + p q + (+p)/(y )dp = ) = CVaR (+α)/ (Y ). 4

15 Proof for Item 4 from Section Proof. Let us proceed with the proof item by item.. The proof follows directly from CVaR ( X ) = sup x i } N i=.. If X is a discrete random variable, then q p ( X ) is a step function of p. Step number i is having length α i α i = p (i) and height q αi ( X ) = x (i). Then, using additivity property of the integral (6), ( α j ) X S α j = α j q p ( X )dp = N i=j+ αi α i q p ( X )dp = N i=j+ p (i) x (i). 3. Since q p ( X ) is a step function of p and α j < α < α j+, then, using additivity property of the integral (6), α q p ( X )dp = ( λ) q p ( X )dp + λ q p ( X )dp, λ = α α j, α j α j+ α j+ α j X S α = ( λ) α j α X S α j + λ α j+ α X S α j+. Proof for Item 9 from Section Proof. We further prove that axioms of the regular measure of error hold for X S α. E(X) [0, ], E(0) = 0 and E(X) > 0 for any X 0 follows from the fact that X S α is a norm. E(X) is closed and convex, which follows from Item from Proposition.. E(X) = X S α E X EX = ψ(ex) for ψ(x) = x on (, ) having ψ(0) = 0 and ψ(t) > 0 for t 0. 5