arxiv: v4 [math.oc] 13 Aug 2013

Transcription

1 Piecewise linear approimations of the standard normal first order loss function arxiv: v4 [math.oc] 13 Aug 2013 Roberto Rossi,,1 S. Armagan Tarim, 2 Steven Prestwich, 3 Brahim Hnich 4 1 Business School, University of Edinburgh, Edinburgh, UK roberto.rossi@ed.ac.uk 2 Dept. of Management, Hacettepe University, Ankara, Turkey armagan.tarim@hacettepe.edu.tr 3 Dept. of Computer Science, University College Cork, Cork, Ireland s.prestwich@cs.ucc.ie 4 Dept. of Computer Engineering, Izmir University of Economics, Izmir, Turkey hnich.brahim@gmail.com Abstract The first order loss function and its complementary function are etensively used in practical settings. When the random variable of interest is normally distributed, the first order loss function can be easily epressed in terms of the standard normal cumulative distribution and probability density function. However, the standard normal cumulative distribution does not admit a closed form solution and cannot be easily linearised. Several works in the literature discuss approimations for either the standard normal cumulative distribution or the first order loss function and their inverse. However, a comprehensive study on piecewise linear upper and lower bounds for the first order loss function is still missing. In this work, we initially summarise a number of distribution independent results for the first order loss function and its complementary function. We then etend this discussion by focusing first on random variable featuring a symmetric distribution, and then on normally distributed random variables. For the latter, we develop effective piecewise linear upper and lower bounds that can be immediately embedded in MILP models. These linearisations rely on constant parameters that are independent of the mean and standard deviation of the normal distribution of interest. We finally discuss how to compute optimal linearisation parameters that minimise the maimum approimation error. keywords: first order loss function; complementary first order loss function; piecewise linear approimation; minima; Jensen s; Edmundson-Madansky. Corresponding author: Roberto Rossi, University of Edinburgh Business School, EH8 9JS, Edinburgh, United Kingdom. phone: +44(0) roberto.rossi@ed.ac.uk 1

2 1 Introduction Consider a random variable ω and a scalar variable. The first order loss function is defined as L(,ω) = E[ma(ω,0)], (1) where E denotes the epected value. The complementary first order loss function is defined as L(,ω) = E[ma( ω,0)]. (2) The first order loss function and its complementary function play a key role in several application domains. In inventory control [10] it is often used to epress epected inventory holding or shortage costs, as well as service level measures such as the widely adopted fill rate, also known as β service level [1], p. 94. In finance the first order loss function may be employed to capture risk measures such as the so-called conditional value at risk (see e.g. [8]). These eamples illustrate possible applications of this function. Of course, the applicability of this function goes beyond inventory theory and finance. Despite its importance, to the best of our knowledge a comprehensive analysis of results concerning the first order loss function seems to be missing in the literature. In Section 2, we first summarise a number of distribution independent results for the first order loss function and its complementary function. We then focus on symmetric distributions and on normal distributions; for these we discuss ad-hoc results in Section 3. According to one of these results, the first order loss function can be epressed in terms of the cumulative distribution function of the random variable under scrutiny. Depending on the probability distribution adopted, integrating this function may constitute a challenging task. For instance, if the random variable is normally distributed, no closed formulation eists for its cumulative distribution function. Several approimations have been proposed in the literature, see e.g. [12, 9, 11, 3, 7, 4], which can be employed to approimate the first order loss function. However, these approimations are generally nonlinear and cannot be easily embedded in mied integer linear programming (MILP) models. In Section 4 and 5, we introduce piecewise linear lower and upper bounds for the first order loss function and its complementary function for the case of normally distributed random variables. These bounds are based on standard bounding techniques from stochastic programming, i.e. Jensen s lower bound and Edmundson-Madansky upper bound [6], p The bounds can be readily used in MILP models and do not require instance dependent tabulations. Our linearisation strategy is based on standard optimal linearisation coefficients computed in such a way as to minimise the maimum approimation error, i.e. according to a minima approach. Optimal coefficients for approimations comprising from two to eleven segments will be presented in Table 1; these can be reused to approimate the loss function associated with any normally distributed random variable. 2

3 2 The first order loss function and its complementary function Consider a continuous random variable ω with support over R, probability density function g ω () : R (0,1) and cumulative distribution function G ω () : R (0,1). The first order loss function can be rewritten as L(,ω) = ma(t,0)g ω (t)dt = The complementary first order loss function can be rewritten as L(,ω) = ma( t,0)g ω (t)dt = (t )g ω (t)dt. (3) ( t)g ω (t)dt. (4) Lemma 1. The first order loss function L(,ω) can also be epressed as Proof. the integration of L(,ω) = L(,ω) = = (1 G ω (t)) dt (5) (t )g ω (t)dt (6) tg ω (t)dt tg ω (t)dt g ω (t)dt (7) is a well-known integration by parts that proceeds as follows. Let u(t) = t, u (t) = 1, v(t) = (1 G ω (t)), v (t) = g ω (t). Rewrite the integral as b b tg ω (t)dt = [ t(1 G ω (t))] b + (1 G ω (t))dt and take the limit since b. The product term in the integration by parts formula converges to (1 G ω ()) as b. We therefore take the limit to obtain the identity tg ω (t)dt = (1 G ω ())+ (1 G ω (t))dt and by substituting tg ω(t)dt with this epression we obtain L(,ω) = (1 G ω ())+ = (1 G ω (t))dt (1 G ω ()) (8) (1 G ω (t))dt (9) 3

4 The following well-known lemma is introduced, together with its proof, for completeness. Lemma 2. The complementary first order loss function L(,ω) can also be epressed as L(,ω) = G ω (t)dt. (10) Proof. L(,ω) = = ( t)g ω (t)dt (11) = G ω () g ω (t)dt = G ω () G ω ()+ = tg ω (t)dt (12) tg ω (t)dt (13) G ω (t)dt (14) G ω (t)dt (15) There is a close relationship between the first order loss function and the complementary first order loss function. Lemma 3. The first order loss function L(,ω) can also be epressed as where ω = E[ω]. Proof. L(,ω) = = = = = = L(,ω) = L(,ω) ( ω) (16) (t )g ω (t)dt (17) tg ω (t)dt tg ω (t)dt tg ω (t)dt tg ω (t)dt G ω ()+ g ω (t)dt (18) tg ω (t)dt g ω (t)dt (19) tg ω (t)dt (1 G ω (t)) (20) G ω (t)dt (1 G ω (t)) (21) G ω (t)dt ( ω) (22) = L(,ω) ( ω) (23) 4

5 Because of the relation discussed in Lemma 3, in what follows without loss of generality most of the results will be presented for the complementary first order loss function. Another known result for the first order loss function and its complementary function is their conveity, which we present net. Lemma 4. L(,ω) and L(,ω) are conve in. Proof. We shall prove the result for L(,ω). Recall that L(,ω) = G ω(t)dt. From the fundamental theorem of integral calculus d d L(,ω) = G ω () and d 2 d L(,ω) 2 = g ω (). Since g ω () is nonnegative the result follows immediately; furthermore, the proof for L(,ω) follows from Lemma 3 and from the fact that is conve. For a random variable ω with symmetric probability density function, we introduce the following results. Lemma 5. If the probability density function of ω is symmetric about a mean value ω, then L(,ω) = L(2 ω,ω). Proof. L(,ω) = = (1 G ω (t)) dt (24) ω ( ω) G ω (t)dt (25) = L(2 ω,ω) (26) Lemma 6. If the probability density function of ω is symmetric about a mean value ω, then L(,ω) = L(2 ω,ω)+( ω) and L(,ω) = L(2 ω,ω) ( ω). Proof. Follows immediately from Lemma 3 and Lemma 5. The results presented so far are easily etended to the case in which the random variable is discrete. In the following section we present results for the case in which the random variable is normally distributed. 5

6 3 The first order loss function for a normally distributed random variable Let ζ be a normally distributed random variable with mean µ and standard deviation σ. Recall that the Normal probability density function is defined as g ζ () = 1 σ ( µ) 2 2π e 2σ 2. (27) No closed form epression eists for the cumulative distribution function G ζ () = g ζ ()d. Let φ() be the standard Normal probability density function and Φ() the respective cumulative distribution function. We net present three known results for a normally distributed random variable: a standardisation result in Lemma 7, and two closed form epressions for the computation of the loss function and of its complementary function in Lemmas 8 and 9. Lemma 7. The complementary first order loss function of ζ can be epressed in terms of the standard Normal cumulative distribution function as µ ( ) σ µ L(,ζ) = σ Φ(t)dt = σ L σ,z, (28) where Z is a standard Normal random variable. Proof. Recall that the complementary first order loss function is defined as L(,ζ) = We change the upper integration limit to by noting that it follows that L(,ζ) = f() = µ σ f 1 (y) = σy +µ, = σ = σ f() µ σ µ σ ( t)g ζ (t)dt. df 1 (y) dy = σ ( f 1 (t))g ζ (f 1 (t)) df 1 (t) dt. (29) dt ( σt µ) 1 φ(t)dt. (30) σ ( ) µ 1 σ t φ(t)dt. (31) σ σ 6

7 Lemma 8. The complementary first order loss function L(,ζ) can be rewritten in closed form as ( ( ) ( ) ) µ µ µ L(,ζ) = σ φ +Φ σ σ σ Proof. Integrate by parts Eq. 28 and observe that µ σ tφ(t)dt = φ( µ σ ). From Lemma 3 the first order loss function of ζ can be epressed as µ L(,ζ) = ( µ)+σ σ Φ(t) dt (32) Recall that an alternative epression is obtained via Lemma 1, L(,ζ) = σ µ σ (1 Φ(t))dt. (33) From Lemma 3 and Lemma 8 we obtain the closed form epression ( ( ) ( ) ) µ µ µ L(,ζ) = ( µ)+σ φ +Φ σ σ σ (34) Lemma 9. The first order loss function L(,ζ) can be rewritten in closed form as ( ( ) ( ( )) ) µ µ µ L(,ζ) = σ φ 1 Φ σ σ σ Proof. Integrate by parts Eq. 32 and observe that µ σ tφ(t)dt = φ( µ σ ). Furthermore, since the Normal distribution is symmetric, both Lemma 5 and Lemma 6 hold. 4 Jensen s lower bound for the standard normal first order loss function We introduce a well-known inequality from stochastic programming [6], p Jensen s lower bound Theorem 1 (Jensen s inequality). Consider a random variable ω with support Ω and a function f(,s), which for a fied is conve for all s Ω, then Proof. [2], p E[f(,ω)] f(,e[ω]). 7

8 Common discrete lower bounding approimations in stochastic programming are etensions of Jensen s inequality. The usual strategy is to find a low cardinality discrete set of realisations representing a good approimation of the true underling distribution. [2], p. 288, discuss one of these discrete lower bounding approimations which consists in partitioning the support Ω into a number of disjoint regions, Jensen s bound is then applied in each of these regions. More formally, let g ω ( ) denote the probability density function of ω and consider a partition of the support Ω of ω into N disjoint compact subregions Ω 1,...,Ω N. We define, for all i = 1,...,N p i = Pr{ω Ω i } = g ω (t)dt Ω i and E[ω Ω i ] = 1 p i Ω i tg ω (t)dt Theorem 2. Proof. [2], p E[f(,ω)] N p i f(,e[ω Ω i ]) i=1 Theorem 3. Given a random variable ω Jensen s bound (Theorem 1) is applicable to the first order loss function L(,ω) and its complementary function L(,ω). Proof. Follows immediately from Lemma 4. Having established this result, we must then decide how to partition the support ω in order to obtain a good bound. In fact, to generate good lower bounds, it is necessary to carefully select the partition of the support ω. The optimal partitioning strategy will depend, of course, on the probability distribution of the random variable ω. 4.2 Minima discrete lower bounding approimation We discuss a minima strategy for generating discrete lower bounding approimations of the(complementary) first order loss function. In this strategy, we partition the support ω into a predefined number of regions N in order to minimise the maimum approimation error. Consider a random variable ω and the associated complementary first order loss function L(,ω) = E[ma( ω,0)]; assume that the support Ω of ω is partitioned into N disjoint subregions Ω 1,...,Ω N. Lemma 10. For the (complementary) first order loss function the lower bound presented in Theorem 2 is a piecewise linear function with N +1 segments. 8

9 Proof. Consider the bound presented in Theorem 2 and let f(,ω) = ma( ω,0), this function is equivalent to L lb (,ω) = L lb (,ω) = N p i ma( E[ω Ω i ],0) i=1 0 E[ω Ω 1 ] p 1 p 1 E[ω Ω 1 ] E[ω Ω 1 ] E[ω Ω 2 ] (p 1 +p 2 ) (p 1 E[ω Ω 1 ]+p 2 E[ω Ω 2 ]) E[ω Ω 2 ] E[ω Ω 3 ].. (p 1 +p p N ) (p 1 E[ω Ω 1 ]+p 2 E[ω Ω 2 ]+...+p N E[ω Ω N ]) E[ω Ω N 1 ] E[ω Ω N ] which is piecewise linear in with breakpoints at E[ω Ω 1 ],E[ω Ω 2 ],...,E[ω Ω N ]. The proof for the first order loss function follows a similar reasoning. Lemma 11. Consider the i-th linear segment of L lb (,ω) i L i lb (,ω) = p k k=1 i p k E[ω Ω k ] E[ω Ω i ] E[ω Ω i+1 ], k=1 where i = 1,...,N. Let Ω i = [a,b], then L i lb (,ω) is tangent to L(,ω) at = b. Furthermore, the 0-th segment = 0 is tangent to L(,ω) at =. Proof. Note that and that it follows and i L i lb (,ω) = k=1 Ω k g ω (t) dt i k=1 Ω 1 Ω 2... Ω i =),b] L i lb (,ω) = b Ω k tg ω (t) dt b g ω (t) dt tg ω (t) dt b L i lb (,ω) = G ω(b)( b)+ G ω (t) dt. which is the equation of the tangent line to L(,ω) at a given point b, that is y = L(b,ω) ( b)+ L(b,ω). = 0 is tangent to L(,ω) at = since L(,ω) is conve, positive and lim L(,ω) = 0. The very same reasoning can be easily applied to the first order loss function. 9

10 Lemma 12. The maimum approimation error between L lb (,ω) and L(,ω) will be attained at a breakpoint. Proof. By recalling that L(,ω) is conve (Lemma 4), since L lb (,ω) is piecewise linear (Lemma 10) and each segment of L lb (,ω) is tangent to L(,ω) (Lemma 11), it follows that the maimum error will be attained at a breakpoint. Theorem 4. Given the number of regions N, Ω 1,...,Ω N is an optimal partition of the support Ω of ω under a minima strategy, if and only if approimation errors at breakpoints are all equal. Proof. The approimation errors for and are both 0; since we have N+1 segments, we only have N breakpoints to check. We first show that ( ) if Ω 1,...,Ω N is an optimal partition of the support Ω of ω under a minima strategy, then approimation errors at breakpoints are all equal. A first observation that follows immediately from Lemma 12 is that, if the slope i+1 of segment L lb (,ω) remains unchanged, and the breakpoint between L i lb (,ω) and L i+1 i+1 lb (,ω) moves towards the point at which L lb (,ω) is tangent to L(,ω), the error at such breakpoint decreases. If one changes the size of region Ω i so that the upper limit becomes b i +, then E[ω Ω i ] will increase if > 0, or will decrease if < 0. This immediately follows from the definition of E[ω Ω i ]. Therefore, thebreakpoint between segment i and segment i+1, which occurs at E[ω Ω i ], will move accordingly. However, the slope of segment i + 1, which we recall is equal to G ω (b i+1 ), depends uniquely on the upper limit of the region Ω i+1, b i+1, and is not affected by a change in the upper limit of region Ω i. Therefore, the error at the breakpoint between segment i and segment i+1 will decrease if > 0, or will increase if < 0. Now, assume that Ω 1,...,Ω N is an optimal partition of the support Ω of ω and approimation errors at breakpoints are not all equal. Furthermore, assume that the maimum approimation error occurs at breakpoint i. By increasing the size of the region Ω i, i.e. by setting the upper limit to b i +, where > 0, it is possible to decrease the maimum error until it becomes equal to the error at breakpoint k, where k {1,...,i 1}. The procedure can be repeated until all approimation errors are equal. Second, we show that ( ) if approimation errors at breakpoints are all equal, then Ω 1,...,Ω N is an optimal partition of the support Ω of ω under a minima strategy. If approimation errors at breakpoints are all equal and we change the size of region Ω i by setting the upper limit to b i +, where > 0, then the approimation error at breakpoint i 1 will increase; conversely, if < 0, then the approimation error at breakpoint i + 1 will increase. By using this last result it is possible to derive a set of equations that can be solved for computing an optimal partitioning. Let us consider the error e i at breakpoint i, this can be epressed as e i = L(E[ω Ω i ],ω) L i lb (E[ω Ω i],ω), 10

11 where Ω i = [a i,b i ]. Since we have N breakpoints to check, we must solve a system comprising the following N 1 equations under the following restrictions e 1 = e i for i = 2,...,N. a 1 = b N = a i b i for i = 1,...,N b i = a i+1 for i = 1,...,N 1 The system therefore involves N 1 variables, each of which identifies the boundary between two disjoint regions Ω i and Ω i+1. Theorem 5. Assume that the probability density function of ω is symmetric about a mean value ω. Then, under a minima strategy, if Ω 1,...,Ω N is an optimal partition of the support Ω of ω, breakpoints will be symmetric about ω. Proof. This follows from Lemma 6 and Theorem 4. In this case, by eploiting the symmetry of the piecewise linear approimation, an optimal partitioning can be derived by solving a smaller system comprising N/2 equations, where N is the number of regions Ω i and rounds to the net integer value. Unfortunately, equations in the above system are nonlinear and do not admit a closed form solution in the general case Normal distribution We will net discuss the system of equations that leads to an optimal partitioning for the case of a standard Normal random variable Z. This partitioning leads to a piecewise linear approimation that is, in fact, easily etended to the general case of a normally distributed variable ζ with mean µ and standard deviation σ via Lemma 7. This equation suggests that the error of this approimation is independent of µ and proportional to σ. Consider a partitioning for the support Ω of Z into N adjacent regions Ω i = [a i,b i ], where i = 1,...,N. From Theorem 5, if N is odd, then b N/2 = 0 and b i = b N+1 i, if N is even, then b i = b N+1 i. We shall use Lemma 8 for epressing L(,Z). Then, by observing that bi a i tφ(t)dt = φ(a i ) φ(b i ) and that p 1 +p p i = Φ(b i ), we rewrite L i lb (E[Z Ω i],z) = Φ(b i )E[Z Ω i ] i (φ(a i ) φ(b i )) (35) k=1 = Φ(b i )E[Z Ω i ] (φ(a 1 ) φ(b i )) (36) = Φ(b i )E[Z Ω i ]+φ(b i ) (37) 11

12 Toepresstheconditional epectation E[Z Ω i ]weproceedasfollows: let p i = Φ(b i ) Φ(a i ), it follows E[Z Ω i ] = φ(a i) φ(b i ) Φ(b i ) Φ(a i ). To solve the above system of non-linear equations we will eploit the close connections between finding a local minimum and solving a set of nonlinear equations. In particular, we will use the Gauss-Newton method to find a partition Ω 1,...,Ω N of the support of Z that minimises the following sum of squares N (e 1 e k ) 2 k=2 This minimisation problem can be solved by software packages such as Mathematica (see NMinimize) Numerical eamples The classical Jensen s bound for the complementary first order loss function of a standard NormalrandomvariableZ isshowinfig. 1. Thisisobtainedbyconsideringadegenerate partition of the support of Z comprising only a single region Ω 1 = [, ]. In practice, we simply replace Z by its epected value, i.e. zero. Therefore we simply have L lb = ma(, 0). The maimum error of this piecewise linear approimation occurs for = 0 and it is equal to 1/ 2π. 2.0,Z lb,z 1.5,Z lb,z Figure 1: classical Jensen s bound for L(,Z) If we split the support of Z into four regions (Fig. 2), the solution to the system of nonlinear equations prescribes to split Ω at b 1 = , b 2 = 0, b 3 = The maimum error is and it is observed at {± , ± }. 12

13 2.0,Z lb,z 1.5,Z lb,z Figure 2: five-segment piecewise Jensen s bound for L(,Z) In Table 1 we report parameters of L lb (,Z) with up to eleven segments. In Fig. 3 we present the approimation error of L lb (,Z) with up to eleven segments. Absolute error Number of segments Figure 3: approimation error of L lb (,Z) with up to eleven segments 13

14 14 Piecewise linear approimation parameters Segments Error i b i p i 1 E[ω Ω i ] 0 b i p i E[ω Ω i ] b i p i E[ω Ω i ] b i p i E[ω Ω i ] b i p i E[ω Ω i ] b i p i E[ω Ω i ] b i p i E[ω Ω i ] b i p i E[ω Ω i ] b i p i E[ω Ω i ] b i p i E[ω Ω i ] Table 1: parameters of L lb (,Z) with up to eleven segments

15 In Fig. 4 we eploited Lemma 7 to obtain the five-segment piecewise Jensen s bound for L(,ζ), where ζ is a normally distributed random variable with mean µ = 20 and standard deviation σ = 5. The maimum error is σ and it is observed at {σ(± ) +µ,σ(± ) +µ}. 10 8,Ζ 6 lb,ζ 4,Ζ lb,ζ Figure 4: five-segment piecewise Jensen s bound for L(,ζ), where µ = 20 and σ = 5 5 An approimate piecewise linear upper bound for the standard normal first order loss function In this section we introduce a simple bounding technique that eploits conveity of the (complementary) first order loss function to derive a piecewise linear upper bound. 5.1 A piecewise linear upper bound Without loss of generality we shall introduce the bound for the complementary first order loss function. Consider a random variable ω with support Ω. From Lemma 4, L(,ω) is conve in regardless of the distribution of ω. Given an interval [a,b] R, it is possible to construct an upper bound by eploiting the very same definition of conveity, that is by constructing a straight line L ub (,ω) between the two points (a, L(a,ω)) and (b, L(b,ω)). The slope (α) and the intercept (β) of this line can be easily computed α = L(b,ω) L(a,ω) b a, β = b L(a,ω) a L(b,ω). b a 15

16 The upper bound is then L ub (,ω) = α+β a b (38) = b a L(a,ω) + L(b,ω) b a b a a b (39) We can improve the quality of this bound by partitioning the domain R of L(,ω) into N disjoint regions D i = [a i,b i ], i = 1,...,N. The selected regions must be all compact and adjacent. Because of the conveity of L(,ω) the bound can be then applied to each of these regions considered separately. However, since L(,ω) is defined over R, it is not possible to guarantee a complete covering of the domain by using compact regions. We must therefore add two etreme regions D 0 = [,a 1 ] and D N+1 = [a N+1 = b N, ] to ensure the one obtained is indeed an upper bound for each R. By noting that lim L(,ω) = 0 and lim L(,ω) = it is easy to derive equations for the lines associated with these two etra regions. In particular, we associate with D 0 a horizontal line with slope α = 0 and intercept β = L(a 1,ω), and with D N+1 a line with slope α = 1 and intercept β = L(b N,ω) b N. Also in this case, we must then decide how to partition the domain R into N + 2 intervals D 0,...,D N+1 to obtain a tight bound. Once more, the optimal partitioning strategy will depend on the probability distribution of the random variable ω. 5.2 Minima piecewise linear upper bound We discuss a minima strategy for generating a piecewise linear upper bound of the (complementary) first order loss function L(,ω). In this strategy, we partition of the domain R of into a predefined number of regions N + 2 in order to minimise the maimum approimation error. Note that, since this domain is not compact, one needs at least two regions to derive a piecewise linear upper bound. Consider a random variable ω and the associated complementary first order loss function L(,ω) = E[ma( ω,0)]; assume that the domain R of is partitioned into N + 2 disjoint adjacent subregions D 0,...,D N+1, where D 0 = [,a 1 ], D i = [a i,b i ], for i = 1,...,N, and D N+1 = [b N, ], and consider the following piecewise linear upper bound L(a 1,ω) D 0 L ub (,ω) =. L(a i,ω) b i b i a i + L(b i,ω) a i b i a i D i. + L(b N,ω) b N D N+1 Let L i ub (,ω) be the linear segment of L ub (,ω) over D i, for i = 0,...,N

17 Lemma 13. Consider L i ub (,ω), where i = 1,...,N; the maimum approimation error between L i ub (,ω) and L(,ω) will be attained for ) i = G 1 ω ( L(bi,ω) L(a i,ω) b i a i Proof. The idea here is to derive a line that is tangent to L(,ω) and that has a slope equal to that of the i-th linear segment of Lub (,ω). We have already discussed in Lemma 11 that the equation of the tangent to L(,ω) at a given point i is y = L( i,ω) ( p)+ L( i,ω) that is i L i lb (,ω) = G ω( i )( p)+ G ω (t) dt. The slope G ω ( i ) only depends on i. To find a tangent with a slope equal to that of the i-th linear segment of L ub (,ω), we simply let G ω ( i ) = L(b i,ω) L(a i,ω) b i a i and invert the cumulative distribution function. Note that the maimum approimation error for the linear segment over D 0 is L(a 1,ω) and the maimum approimation error for the linear segment over D N+1 is L(b N,ω) b N. This can be inferred from the fact that L(,ω) monotonically approaches 0 for and for. Theorem 6. D 0,...,D N+1 is an optimal partition under a minima strategy, if and only if the maimum approimation error between L(,ω) and each linear segment of L ub (,ω) is the same. Proof. The proof of this theorem can be obtained from Lemma 13 and from Theorem 4. In particular, the key insight needed to understand this result is the following. In Theorem 4 we showed that the approimation errors at breakpoints for the piecewise linear lower bound presented are all equal to each other and also equal to the maimum approimation error; furthermore, in Lemma 11 we showed that the i-th piecewise linear segment of this lower bound agrees with the original function at point b i, where Ω i = [a i,b i ] is the i-th partition of the support of Z, for i = 1,...,N. Since the first order loss function is conve and we know the maimum approimation error, by shifting up the piecewise linear lower bound by a value equal to the maimum approimation error we immediately obtain a piecewise linear upper bound comprising N + 1 segments. This upper bound agrees with the original function at E[Z Ω i ], for i = 1,...,N. The maimum approimation error will be attained at those points in which the lower bound was tangent to the original function, that is a 1,b 1,b 2,...,b N. By using a reasoning similar to the one developed for Theorem 4, it is possible to show that, if we increase or decrease at least one E[Z Ω i ], the maimum approimation error can only increase.. 17

18 By using this result it is possible to derive a set of equations that can be solved for computing an optimal partitioning. Let us consider the maimum approimation error e i associated with the i-th linear segment of L ub (,ω), this can be epressed as e i = L i ub ( i,ω) L( i,ω), where i = 1,...,N; furthermore e 0 = L(a 1,ω) and e N+1 = L(b N,ω) b N. Since we have N +2 segments to check, we must solve a system comprising the following N +1 equations e 0 = e i for i = 1,...,N +1 under the following restrictions a i b i for i = 1,...,N b i = a i+1 for i = 1,...,N 1 The system involves N + 1 variables, each of which identifies the boundary between two disjoint adjacent regions D i and D i+1. Theorem 7. Assume that the probability density function of ω is symmetric about a mean value ω. Then, under a minima strategy, if D 0,...,D N+1 is an optimal partition of the domain, breakpoints will be symmetric about the mean value ω. Proof. This follows from Lemma 6 and Theorem 6. In this case, by eploiting the symmetry of the piecewise linear approimation, an optimal partitioning can be derived by solving a smaller system comprising (N + 1)/2 equations, where N +2 is the number of regions D i and rounds to the net integer value. As in the case of Jensen s bound, equations in the above system are nonlinear and do not admit a closed form solution in the general case. For sake of completeness, we will briefly discuss net how to derive the system of nonlinear equations for the case of a standard normal random variable Z. However, one should note that in practice, by eploiting the properties illustrated in the proof of Theorem 6, one does not need to solve a new system of nonlinear equations to derive the piecewise linear upper bound. All information needed, i.e. maimum approimation error at breakpoints and locations of the breakpoint, are in fact immediately available as soon as the system of equation presented for the piecewise linear lower bound is solved (Table 1). The upper bound presented is closely related to a well-known inequality from stochastic programming, see e.g. [6], p. 168, [5], p. 316, and [2], pp As pointed out in [6], p. 168, Edmundson-Madanski s upper bound can be seen as a bound where the original distribution is replaced by a two point distribution and the problem itself is unchanged, or it can be viewed as a bound where the distribution is left unchanged and the original function is replaced by a linear affine function represented by a straight line. The above discussion clearly demonstrates the dual nature of this upper bound. 18

19 5.3 Normal distribution We will net discuss the system of equations that leads to an optimal partitioning for the case of a standard Normal random variable Z. This partitioning leads to a piecewise linear approimation that is, in fact, easily etended to the general case of a normally distributed variable ζ with mean µ and standard deviation σ via Lemma 7. Also for this second approimation this equation suggests that the error is independent of µ and proportional to σ. Consider a partitioning for the domain of in L(,ω) into N +2 adjacent regions D i = [a i,b i ], where i = 0,...,N +1. From Theorem 7, if N is odd, then b N/2 = 0 and b i = b N+1 i, if N is even, then b i = b N+1 i. Also in this case, we shall use Lemma 8 for epressing L(,Z), and we will eploit the close connections between finding a local minimum and solving a set of nonlinear equations. We will therefore use the Gauss- Newton method to minimize the following sum of squares N+1 (e 0 e k ) 2 k=1 This minimisation problem can be solved by software packages such as Mathematica (see NMinimize) Numerical eamples A two-segment piecewise linear upper bound for the complementary first order loss function of a standard Normal random variable Z is shown in Fig. 5. This bound has been obtained, under the minima criterion previously described, by considering a single breakpoint in the domain, i.e. = 0. Of course, the maimum error of this piecewise linear approimation occurs for = ± and it is equal to 1/ 2π. It is easy to observe that this upper bound can be obtained by adding to the classical Jensen s lower bound presented in Fig. 1 a constant value equal to its maimum approimation error, i.e. 1/ 2π. We net present a more interesting case, in which the domain has been split into five regions (Fig. 6). Breakpoints are positioned at {± , ± }. These were the locations at which the maimum error, i.e , was observed in Fig. 2. Also in this case, the five-segment piecewise linear upper bound can be obtained by adding to the five-segment piecewise Jensen s lower bound a value equal to its maimum approimation error. Finally, in Fig. 7, we show an eample in which we eploited Lemma 7 to obtain, from the approimation presented in Fig. 6, the five-segment piecewise linear upper bound for L(,ζ), where ζ is a normally distributed random variable with mean µ = 20 and standard deviation σ = 5. The maimum error is σ and it is observed at {±,σ(± ) +µ,µ}. 19

20 ,Z 2.0 ub,z 1.5,Z ub,z Figure 5: two-segment piecewise linear upper bound for L(,Z) 2.0,Z 1.5 ub,z 1.0,Z ub,z Conclusions Figure 6: five-segment piecewise linear upper bound for L(,Z) We summarised a number of distribution independent results for the first order loss function and its complementary function. We then focused on symmetric distributions and on normal distributions; for these we discussed ad-hoc results. To the best of our knowledge a comprehensive analysis of results concerning the first order loss function seems to be missing in the literature. The first contribution of this work was to fill this 20

21 10 8,Ζ 6 ub,ζ 4,Ζ ub,ζ Figure 7: five-segment piecewise linear upper bound for L(,ζ), where µ = 20 and σ = 5 gap in the literature. Based on the results discussed, we developed effective piecewise linear approimation strategies based on a minima framework. This is the second contribution of our work. More specifically, we developed piecewise linear upper and lower bounds for the first order loss function and its complementary function. These bounds rely on constant parameters that are independent of the means and standard deviation of the normal distribution considered. We discussed how to compute optimal parameters that minimise the maimum approimation error and we also provided a table with pre-computed optimal parameters for piecewise bound with up to eleven segments. These bounds can be easily embedded in eisting MILP models. Acknowledgements INSERT ACKNOWLEDGEMENTS HERE References [1] Sven Asater. Inventory control. Springer Verlag, [2] John R. Birge and François Louveau. Introduction to Stochastic Programming (Springer Series in Operations Research and Financial Engineering). Springer, corrected edition, July [3] Wlodzimierz Bryc. A uniform approimation to the right normal tail integral. Applied Mathematics and Computation, 127(2-3): , April

22 [4] Steven K. De Schrijver, El-Houssaine Aghezzaf, and Hendrik Vanmaele. Double precision rational approimation algorithm for the inverse standard normal first order loss function. Applied Mathematics and Computation, 219(3): , October [5] Karl Frauendorfer and Michael Schürle. Stochastic linear programs with recourse and arbitrary multivariate distributions. In Christodoulos A Floudas and Panos M Pardalos, editors, Encyclopedia of Optimization, pages Springer US, [6] Peter Kall and Stein W. Wallace. Stochastic Programming (Wiley Interscience Series in Systems and Optimization). John Wiley & Sons, August [7] Jean M. Linhart. Algorithm 885: Computing the logarithm of the normal distribution. ACM Trans. Math. Softw., 35(3), October [8] Renata Mansini, W lodzimierz Ogryczak, and Maria Grazia Speranza. Conditional value at risk and related linear programming models for portfolio optimization. Annals of Operations Research, 152(1): , July [9] Haim Shore. Simple approimations for the inverse cumulative function, the density function and the loss integral of the normal distribution. Journal of the Royal Statistical Society. Series C (Applied Statistics), 31(2):pp , [10] Edward A. Silver, David F. Pyke, and Rein Peterson. Inventory Management and Production Planning and Scheduling. John-Wiley and Sons, New York, [11] Gary R. Waissi and Donald F. Rossin. A sigmoid approimation of the standard normal integral. Applied Mathematics and Computation, pages 91 95, June [12] Marvin Zelen and Norman C. Severo. Probability functions. In Milton Abramowitz and Irene A Stegun, editors, Handbook of Mathematical Functions, volume 5 of Applied Mathematics Series, pages GPO,