Gradient Consistency for Integral-convolution Smoothing Functions

Transcription

1 Set-Value Var. Anal (2013) 21: DOI /s Graient Consistency for Integral-convolution Smoothing Functions James V. Burke Tim Hoheisel Christian Kanzow eceive: 4 September 2012 / Accepte: 18 March 2013 / Publishe online: 29 March 2013 Springer Science+Business Meia Dorrecht 2013 Abstract Chen an Mangasarian (Comput Optim Appl 5:97 138, 1996) evelope smoothing approximations to the plus function built on integral-convolution with ensity functions. X. Chen (Math Program 134:71 99, 2012) has recently picke up this iea constructing a large class of smoothing functions for nonsmooth minimization through composition with smooth mappings. In this paper, we generalize this iea by substituting the plus function for an arbitrary finite max-function. Calculus rules such as inner an outer composition with smooth mappings are provie, showing that the new class of smoothing functions satisfies, uner reasonable assumptions, graient consistency, a funamental concept coine by Chen (Math Program 134:71 99, 2012). In particular, this guarantees the esire limiting behavior of critical points of the smooth approximations. Keywors Smoothing metho Subifferential calculus Integral-convolution Piecewise-affine function Mathematics Subject Classifications (2010) 49J52 90C26 90C30 This research was partially supporte by the DFG (Deutsche Forschungsgemeinschaft) uner grant HO 4739/1-1. J. V. Burke Department of Mathematics, University of Washington, P.O. Box , Seattle, WA , USA burke@math.washington.eu T. Hoheisel (B) C. Kanzow Institute of Mathematics, University of Würzburg, Campus Hublan Nor, Emil-Fischer-Straße 30, Würzburg, Germany hoheisel@mathematik.uni-wuerzburg.e C. Kanzow kanzow@mathematik.uni-wuerzburg.e

2 360 J.V. Burke et al. 1 Introuction An unconstraine optimization problem takes the form min f (x), (1) x n for a function f : n. Incasethat f is continuously ifferentiable, see, e.g., [1] or [23], powerful numerical solution methos have been introuce an successfully employe. Things are more subtle when the traitional smoothness assumption is not satisfie. An important class of potentially nonsmooth functions are the convex ones, see [13] for an overview on solution methos for convex optimization. A generalization of convexity on the one han an continuous ifferentiability on the other, is local Lipschitz continuity, cf., e.g., [11] for solution proceures for locally Lipschitz problems. A well-recognize technique for the numerical solution of (1) in the nonsmooth case is to replace f by a smooth approximation, an solve a sequence of smooth problems, while riving the approximation closer an closer to the original function, with the intention of approximating minimizers (critical points) of f by those of the smooth approximations. This technique of replacing a nonsmooth problem by a sequence of smooth problems is, in general, known as smoothing an it has been employe extensively for several ifferent kins of problems, see, e.g., [2, 3, 5, 9, 12, 16, 17, 22], or the recent survey [6] which mentions several important applications of smoothing an inclues an extensive list of references. Certain smoothing methos are also closely relate to the class of interior-point methos, cf. [18]. In this paper, we are concerne with a class of smoothing functions for f inite max-functions, seesection4 for a formal efinition, that are special piecewise af f ine mappings. These approximations are shown to be well-behave uner both outer an inner composition with smooth functions giving rise to a satisfactory calculus, an hence a class of smoothing approximations for a broa class of nonsmooth, locally Lipschitz functions is obtaine. Following [6] an[9], respectively, the smoothing functions for the finite maxfunctions are constructe via integral-convolution with special ensity functions. A major aspect of the analysis consists in showing that the smoothing functions consiere satisfy graient consistency, see Section 3. Graient consistency, as efine in [6], is a funamental tool for establishing limiting stationarity properties of smoothing methos for optimization. In particular, it guarantees that (accumulation points of) sequences of first-orer critical points of the smooth approximations yiel critical points of the original function f. This paper can be viewe as an extension to parts of the recent paper [6] by Chen, in which the author constructs an analysis of smoothing approximations buil on the plus function, seesection2. The plus function is a special case of a finite max-function. Moreover, we fill a voi which arises from an insufficient proof of [6, Theorem 1 (i)], see Section 5 an [7]. The latter result is key for establishing graient consistency for composite smoothing functions, an hence of funamental importance. Not withstaning the insuffiency of the current proof of [6, Theorem 1 (i)], we conjecture that the assertion is vali, yet not achievable via a chain rule approach without the assumptions iscusse in the sequel.

3 Graient Consistency for Integral-convolution Smoothing Functions 361 The organization of the paper is as follows: In Section 2 we review some necessary concepts from nonsmooth analysis. In Section 3 we lay out a general framework for smoothing functions. Section 4 establishes the class of smoothing functions for finitemax functions an provies calculus rules for compositions with smooth mappings. We close with some final remarks in Section 5. In particular, we compare [6, Theorem 1 (i)] with our main theorem. Most of the notation use is stanar. An element x n is unerstoo as a column vector. The symbol n + enotes the set of all vectors whose components are nonnegative. The space of all real m n-matrices is enote by m n,anfor A m n, A T is its transpose, an rank A enotes its rank. An n n iagonal matrix D with the vector x on its iagonal is enote by D = iag (x) = iag (x i ). The Eucliean norm on n is enote by, i.e., x = x T x x n. The close Eucliean ball centere aroun x n with raius r 0 is enote by B r ( x), i.e., B r ( x) := {x n x x r}. For a set S n its convex hull is enote by conv S. Given a real-value function f : n ifferentiable at x,thegraient is given by f ( x) which is unerstoo as a column vector. For a function F : n m ifferentiable at x, thejacobian of F at x is enote by F ( x), i.e., F ( x) = F 1 ( x) T. F m ( x) T m n, whereas F( x) is the transpose Jacobian. In orer to istinguish between singlean set-value maps, we write S : n m to inicate that S maps vectors from n to subsets of m. Finally, the symbol x k X x inicates that {x k } is a sequence converging to the limit point x such that all iterates x k belong to a set X n. 2 Preliminaries In this section we review certain concepts from variational an nonsmooth analysis, which will be use in the subsequent analysis. The notation is mainly base on [25]. A major role is playe by ifferent kins of subifferentials as a tool for ealing with nonsmoothness of the functions consiere. To this en, we commence by introucing the so-calle regular an limiting subif ferential. In the efinition of the limiting subifferential, we employ the outer limit for a set-value mapping, which we now efine: For S : n m an X n,weefinetheouter limit by Lim sup S(x) := { v {x k } X x, {v k } v : v k S(x k ) k N }. x X x

4 362 J.V. Burke et al. Definition 2.1 (egular an limiting subifferential) Let f : n be continuous an x n. a) The regular subif ferential of f at x is the set given by { f (x) f ( x) v ˆ T (x x) f ( x) := v lim inf x x x x b) The limiting subif ferential of f at x is the set given by f ( x) := Lim sup ˆ f (x) x x } 0. { } = v {x k } x, {v k } v : v k ˆ f (x k ) k N. Note that there are ifferent ways of obtaining the limiting subifferential than the one escribe above. This erivation goes back to Morukhovich, cf. [19]. In this context, see [15] (or[4]) for a construction of the limiting subifferential via Dinierivatives. A very important class of potentially nonsmooth, nonconvex functions are the locally Lipschitz functions. We say F : n m is locally Lipschitz at x n if there exist ε>0 an L 0 such that F(x) F(y) L x y for all x, y B ε ( x). For such an F, weefinethegeneralize Jacobian as in Clarke ([10, Definition 2.6.1]) via aemacher s Theorem, see[4, Theorem 9.1.2] or [25, Theorem 9.60], which says that the complement of the set D F := {x F is ifferentiable at x} has Lebesgue measure 0. Hence the set F( x) := { V {x k } D F : x k x an F (x k ) V } is well-efine, even compact, cf. [25, Theorem 9.62] or [10, p. 63]. The set F( x) is usually calle the B-subif ferential of F at x, see, e.g., [24], though we o not use this terminology here. Definition 2.2 (Generalize Jacobian) For a locally Lipschitz function F : n m, the generalize Jacobian of F at x is given by F( x) := conv F( x). In the case m = 1, we recover the Clarke subif ferential, see[10] for an extensive treatment. In the general case, to be consistent with the generalize Jacobian, the elements from the Clarke subifferential are row vectors, but we prefer to think of them as column vectors, so everytime a generalize Jacobian is involve, we have to transpose accoringly. The Clarke subifferential of a locally Lipschitz function f : n can alternatively be obtaine via the limiting subifferential. In fact, see [25, Theorem 9.61], we have f ( x) = conv f ( x) x n. (2)

5 Graient Consistency for Integral-convolution Smoothing Functions 363 An important concept in the context of subifferentiation is (subif ferential) regularity, which we efine only for the locally Lipschitz case. For the general case, cf. [25, Definition 7.25]. Definition 2.3 (Subifferential regularity) Let f : n be locally Lipschitz. Then f is sai to be (subifferentially) regular at x n if ˆ f ( x) = f ( x). If the above equality hols for all x n, we simply say that f is (subifferentially) regular. Note that this notion of regularity coincies with the one given by [10, Definition 2.3.4], cf. [10, Theorem (ii)] in connection with [25, Definition 7.25]. A prominent class of (potentially nonsmooth) locally Lipschitz, regular functions are the convex functions. Here we refer to [4] or[14] for the fact that (finitevalue) convex functions are locally Lipschitz, an to [10, Proposition b)] or [25, Example 7.27] to see that they are inee regular. It is a well-known fact, see [25, Proposition 8.12], that if f : n is convex, then all subifferentials coincie with the subifferential of convex analysis, i.e., f ( x) = f ( x) = ˆ f ( x) = { v f (x) f ( x) + v T (x x) x n} for all x n. To illustrate these concepts consier the function ( ) + : given by (t) + := max{t, 0}, calle the plus function. This function is important to our stuy an reappears in Section 4. Example 2.4 (Subifferentiation of the plus function) a) Let f 1 : be given by f 1 (t) := (t) +. Then the convexity of f 1 implies that 0 if t < 0, ˆ f 1 (t) = f 1 (t) = f 1 (t) = [0, 1] if t = 0, (3) 1 if t > 0, whereas we have 0 if t < 0, f 1 (t) = {0, 1} if t = 0, 1 if t > 0. b) Let f 2 : be given by f 2 (t) := (t) +. Consiering the only interesting point t = 0, an elementary calculation shows that ˆ f 2 (0) =, f 2 (0) = f 2 (0) ={0, 1} an f (0) =[ 1, 0]. We close this section by introucing the coerivative, a erivative concept for setvalue maps that goes back to Morukhovich, see [20]. Here we are only intereste in a special case where F : n m is single-value an locally Lipschitz. In this case

6 364 J.V. Burke et al. the coerivative of F at x can be efine as the set-value map D F( x) : m n given by D F( x)(u) = (u T F)( x), where u T F : n is given by (u T F)(x) := m i=1 u i F i (x). Thisiscallethescalarization formula, see[21, Theorem 1.90] or [25, Proposition 9.24 (b)]. Furthermore, we have F( x) T u = conv D F( x)(u) u m, see [25, Theorem 9.62]. If F is continuously ifferentiable, we have D F( x)(u) = { F ( x) T u }, see [21, Theorem 1.38] or [25, Example 8.34]. 3 The General Smoothing Setup Let f : n be continuous. We say s f : n + is a smoothing function for f if the following assumptions are fulfille: s f (, ) converges continuously to f in the sense of [25, Definition 5.41], i.e., lim s f (x, ) = f ( x) x n, (4) 0,x x s f (, ) is continuously ifferentiable for all > 0. Suppose one has sequences {x k } x an { k } 0such that lim s ( f x k ), k 0. k The following is a crucial question from the perspective of algorithmic evelopment: Is x acritical point of f in the sense that 0 f ( x) (or 0 f ( x))? The answer is positive if Lim sup x s f (x, ) f ( x), x x, 0 where, for the sake of completeness, we recall that, accoring to the general efinition of the outer limit, we have Lim sup x s f (x, ) = { v {( x k )}, k ( x, 0) : x s f (x k, k ) v }. x x, 0 The next result shows that the converse inclusion is always vali if s f is a smoothing function for f. Lemma 3.1 Let f : n be continuous an let s f be a smoothing function for f. Then for x n we have f ( x) Lim sup x s f (x, ). x x, 0

7 Graient Consistency for Integral-convolution Smoothing Functions 365 If further f is locally Lipschitz at x, then { } f ( x) conv Lim sup x s f (x, ) x x, 0. Proof Let v f ( x) be given. Since, by assumption, s f (, ) converges continuously to f, it converges, in particular, epigraphically, cf.[25, Theorem 7.11], an hence we may invoke [25, Corollary 8.47] in orer to obtain sequences { k } 0, {x k } x an {v k } with v k x s f (x k, k ) such that v k v.now,sinces f (, k ) is continuously ifferentiable by assumption, we have v k = x s f ( x k, k ), which ientifies v as an element of Lim sup x x, 0 x s f (x, ) an thus, the first inclusion follows. The secon inclusion is an immeiate consequence of the first one an the fact that conv f ( x) = f ( x) in the presence of local Lipschitz continuity, see (2). A trivial consequence is the following corollary. Corollary 3.2 Let f : n be locally Lipschitz at x anlets f be a smoothing function for f. Then we have the inclusions Lim sup x s f (x, ) { } x x, 0 f ( x) conv Lim sup x s f (x, ). x x, 0 f ( x) In the locally Lipschitz setting, it is clear that the conition Lim sup x s f (x, ) = f ( x) (5) x x, 0 implies { } conv Lim sup x s f (x, ) x x, 0 = f ( x). (6) Conition (6) is calle graient consistency in [6]. In particular, both conitions coincie, when f ( x) = f ( x) (which is the case when f is locally Lipschitz an subifferentially regular). The following example shows that, for locally Lipschitz f, conition (6) is in fact weaker than (5). Example 3.3 Let f : 2, f (a, b) := min{a, b}. Thens f ((a, b), ) := 1 2 (a + b (a b)2 + 4) is a smoothing function for f, sometimes calle the CHKS-function ue to its origin from [8, 18, 27]. It hols that for all a we have {( ) ( )} 1 0 f (a, a) =, 0 1 {( ) 1 conv, 0 ( 0 1 )} = f (a, a),

8 366 J.V. Burke et al. but Lim sup s f ((x, y), ) = conv (x,y) (a,a), 0 { } Lim sup s f ((x, y), )(a, a) (x,y) (a,a), 0 = f (a, a). The following result is the main motivation for the analysis in Section 4. In this context, for f : n locally Lipschitz, we call x n Clarke-stationary, C-stationary for short, if 0 f ( x). Theorem 3.4 Let f : n be locally Lipschitz an let s f be a smoothing function for f. Furthermore let {x k } n an { k } 0 such that x s f (x k, k ) ck k N, (7) forsomec> 0. Then every accumulation point x of{x k } such that the graient consistency conition (6) hols at x is a C-stationary point of f. Proof Let x be a limit point of a subsequence {x k } K such that graient consistency hols at x. Since{ k } K 0 on the same subsequence, we can euce from (7)that 0 Lim sup x s f (x, ). x x, 0 This implies { } 0 conv Lim sup x s f (x, ) x x, 0 = f ( x) by the graient consistency assumption. We point out that, in particular, the smoothing graient metho propose in [6] fits into the framework of Theorem 3.4, cf. [6, Theorem 2]. We close this section with the remark that Theorem 3.4 can be refine in the following sense. Suppose that in Theorem 3.4 the stronger conition (5) hols at x. Then it follows from the previous proof that 0 f ( x), which is (without regularity) a tighter property than C-stationarity (typically calle M-stationarity in the corresponing literature). However, we are not aware of a class of (non-regular) functions, for which (5) hols. Example 3.3 isplays our impression that, in the nonregular case, the graient consistency (6) is substantially weaker, hence much more likely to hol than (5) in the smoothing setup escribe above. This is also confirme by the analysis in the upcoming section. 4 Smoothing via Integral-convolution Inthis sectionwe generalize (antoa certainextentcorrect,see [7]) the results from [6, Section 3 4], in the sense that we o not entirely focus on the plus function. Let p : be the f inite max-function given by p(t) = max i=1,...,r { f i(t)}

9 Graient Consistency for Integral-convolution Smoothing Functions 367 Fig. 1 Illustration for the choices in (8) (10) where f i : is affine linear, i.e., f i (t) = a i t + b i with scalars a i, b i for all i = 1,...,r (r N). Note that p is, in particular, piecewise affine, hence (globally) Lipschitz, see [26, Proposition 2.2.7] an convex, cf. [14, Proposition B 2.1.2]. Moreover, it can be seen (cf. Fig. 1) that, after skipping all inices which o not contribute in the maximization, an after reorering the remaining inices, we can assume without loss of generality that an there exists a partition of the real line such that a 1 < a 2 < < a r 1 < a r, (8) = t 1 < t 2 < < t r < t r+1 =+ a i t i+1 + b i = a i+1 t i+1 + b i+1 i = 1,...,r 1, (9) an a 1 t + b 1 if t t 2, p(t) = a i t + b i if t [t i, t i+1 ] (i {2,...,r 1}), a r t + b t if t t r. (10) Let ρ : be a piecewise continuous, symmetric ensity function, i.e., ρ(t) = ρ( t) (t ) an ρ(t) t = 1, (11)

10 368 J.V. Burke et al. such that ρ 0 an t ρ(t) t < +. (12) We enote the istribution function that goes with the ensity ρ by F, i.e., F : [0, 1] is given by F(x) = x ρ(t) t. In particular, since ρ is piecewise continuous, F is continuous with lim F(x) = 1 an lim F(x) = 0. x + x Lemma 4.1 Let p : be a f inite max-function. Furthermore, let ρ : + be a piecewise continuous function satisfying (11) an (12). Then the convolution s p (t, ) := p(t s)ρ(s) s is a (well-ef ine) smoothing function for p with Lim sup t t, 0 t s p(t, ) = p( t) t. Proof The fact that s p (t, ) exists for all > 0 is a consequence of the conitions impose on ρ in (11) an(12) an the representation of p from (8) (10), which we can assume without loss of generality. Next we show that lim t t, 0 s p (t, ) = p( t) for all t. This is ue to the fact that p( t) s p (t, ) = p( t)ρ(s) s p(t s)ρ(s) s p( t) p(t s) ρ(s) s L p t t + s ρ(s) s L p t t +L p s ρ(s) s, where L p is the (global) Lipschitz constant of p. Taking into account assumption (12), it follows that the last expression tens to 0 as t t an 0. This shows that s p (, ) converges continuously to p. We will now compute the erivative of s p (, ) for some fixe > 0. To this en, recall that, without loss of generality, we may assume that p amits a representation as in (10). Hence, we get s p (t, ) = r i=1 t t i t t i+1 [ ai (t s) + b i ] ρ(s) s,

11 Graient Consistency for Integral-convolution Smoothing Functions 369 where we have t t1 t tr+1 + an t s p(t, ) = r i=1 t. Thus, we obtain [ t t i ] [a i (t s) + b i ]ρ(s) s, (13) t t i+1 where the existence of the corresponing erivatives follows, e.g., from the Leibniz integral rule with variable limits. More precisely, this rule allows us to compute the erivatives explicitly. For the summans i = 2,...,r 1, we obtain t t t 2 [ t t i t t i+1 t ti = a i [ ai (t s) + b i ] ρ(s) s ] t t i+1 ρ(s) s + (a i t i + b i ) ρ(t ti ) t t 2 ) (a i t i+1 + b i ) ρ(t ti+1 For the summan i = 1 we compute [ ] + + [a 1 (t s) + b 1 ]ρ(s) s = a 1 ρ(s) s (a 1 t 2 + b 1 ) ρ(t t2 ), t an for i = r we get t [ t tr ] t tr [a r (t s) + b r ]ρ(s) s = a r ρ(s) s + (a r t r+1 + b r ) ρ(t tr ). Inserting these expressions in (13) an exploiting the fact that a i t i+1 + b i = a i+1 t i+1 + b i+1 (i = 1,...,r 1) (see (9)), we obtain t s p(t, ) = r i=1 t ti a i t t i+1 ρ(s) s ( t t2 )) r 1 ( t ti ) ( t ti+1 )) ( t tr ) = a 1 (1 F + a i (F F + a r F i=2 ue to the telescoping structure of the resulting sum. In particular, since F is continuous, so is t s p(, ) for all > 0. Altogether, we have shown that s p is a smoothing function for p. In orer to verify the remaining assertion, first note that, in view of (8) (10), we have {a 1 } if t < t 2, {a p( t) = i } if t (t i, t i+1 )(i = 2,...,r 1), (15) {a r } if t > t r, [a i, a i+1 ] if t = t i+1 (i = 1,...,r 1). Now, recall that Lemma 3.1 guarantees that Lim sup t t, 0 t s p(t, ) p( t), as s p is a smoothing function for p. In orer to see the converse inclusion, let v. (14)

12 370 J.V. Burke et al. Lim sup t t, 0 t s p(t, ) be given. Then there exist sequences { k } 0 an {t k } t such that t s p(t k, k ) v. Clearly, if t (t j, t j+1 ) for some j {2,...,r 1}, we obtain ( ) ( ) tk t i tk t i+1 F F 0 i = j, an k k k ( ) ( ) tk t j tk t j+1 F F 1 0 = 1, ( ) tk t 2 1 F 0, k k ( ) tk t r F 0. (16) The representation (14) of the graient of s p therefore shows that v = a j,hencewe have v = a j {a j }= p( t). Furthermore, if t < t 2, we infer that v = a 1 an, if t > t r we get v = a r, which yiels v p( t) also in these cases. It remains to consier the cases where t = t j+1 for some j {1,...,r 1}. First consier the case where j {2,...,r 2}.Then ( ) ( ) tk t i tk t i+1 F F 0 i / {j, j + 1}, k an since F : [0, 1], we get (at least on a subsequence) ( ) ( ) tk t j tk t j+1 F F 1 λ an k k k k k ( ) ( ) tk t j+1 tk t j+2 F F λ, for some λ [0, 1]. Using once more the limit conitions from (16) aswellasthe representation (14), we obtain v = a j (1 λ) + a j+1 λ [a j, a j+1 ]= p( t). Finally, the arguments are similar if t = t 1 or t = t r 1, hence we skip the etails. We point out that Lemma 4.1 is still vali for a function p, whichamitsa piecewise-affine representation as given by (9) an(10), without emaning (8). The latter conition correspons to convexity an hence, regularity of p, which is neee alreay in the following result. Corollary 4.2 Let p : be a f inite max-function, h : n continuously if ferentiable an let f : n be given by f (x) := p(h(x)). Then, if s p is ef ine as in Lemma 4.1, the function s f (, ) := s p (h( ), ) is a smoothing function for f with k Lim sup x s f (x, ) = f ( x) x. x x, 0

13 Graient Consistency for Integral-convolution Smoothing Functions 371 In particular, the graient consistency property (6) hols. Proof The fact that s f is a smoothing function for f is ue to the fact that s p has this property with respect to p an h is continuously ifferentiable. Hence, given x n, the inclusion Lim sup x x, 0 x s f (x, ) f ( x) is clear from Lemma 3.1. In orer to establish the converse inclusion, first note that f ( x) = h( x) g(h( x)), since p is convex, hence regular an h is smooth, cf. [25, Theorem 10.6]. Let v Lim sup x x, 0 x s f (x, ). Then, there exist sequences {x k } x an { k } 0 such that h ( x k) t s ( ( p h x k ) ) (, k = x s f x k ), k v. (17) Since we have (see (14)) t s ( ( p h x k ) ), k ( ( = a 1 (1 ) h x k t 2 )) F k ( ( ) h x k t r ) + a r F, k r 1 + i=2 ( ( h x k a i (F ) t i ) ( ( ) h x k t i+1 )) F k k an F : [0, 1], the sequence { t s p(h(x k ), k )} is boune. Hence Lemma 4.1 implies that { t s p(h(x k ), k )} converges (at least on a subsequence) to some element τ g(h( x)). It therefore follows from (17)that v = h( x)τ h( x) g(h( x)) = f ( x), which conclues the proof. In the following example, we compute a smoothing function for a simple finite max-function an compare it to the plus function smoothing approach escribe in [6]. Example 4.3 Consier the finite max-function p :, p(t) = max{ 2t, t} an the piecewise continuous, symmetric ensity ρ : given by { 1 if s < 1 ρ(s) := 2, 0 if s 1 2. Accoring to [6], a smoothing of the plus function can be compute (using a chain rule) as { (t) + if t 2 φ(t, ) := (t s) + ρ(s) s =, t t 2 + if t < 8 2.

14 372 J.V. Burke et al. Due to the representation p(t) = (t) + + ( 2t) +, a smoothing function for p in the sense of [6] is given by p(t) if t 2, φ(t, ) + φ( 2t, ) = ( 2t) + + t2 2 + t 2 + if t [ 8 4, 2 ), t 2 t 2 + if t < 4 4. On the other han, using the smoothing formula of Lemma 4.1, we have { p(t) if t 2 s p (t, ) = p(t s)ρ(s) s =, t 2 t if t < 2. To prepare the main theorem of this section, we nee the following preliminary result. Lemma 4.4 Let H : n m be continuously if ferentiable as well as G : m m given by G(y) := (ϕ i (y i )) m i=1, where ϕ i : (i = 1,...,m) is regular. Then for F := G H the following hols: a) For all x n F( x) T = H ( x) T G(H( x)) m +. b) If rank H ( x) = m, it hols that F( x) T = H ( x) T G(H( x)) m. Proof a) For m + an x n we have F( x) T = conv D F( x)() = conv ( T F)( x) = ( T F)( x) [ m ] = i F i ( x) = = i=1 m i F i ( x) i=1 m i ϕ i (H i ( x)) H i ( x) i=1 = H ( x) T G(H( x)). Here, the first equality follows from [25, Theorem 9.62], the secon is the scalarization formula, see Section 2. The thir uses the fact that the Clarke subifferential is the convex hull of the limiting subifferential, cf. (2). The fourth is the efinition of the function x ( T F)(x), an the fifth is ue to the

15 Graient Consistency for Integral-convolution Smoothing Functions 373 fact that the functions F i are regular by [10, Theorem (iii)], an hence the sum rule from [10, Proposition 2.3.3] hols with equality, see [10, Corollary 3] (note that 0 is require here!). The sixth equality is once again the chain rule from [10, Theorem (iii)], an the final line is a short-han form of the previous expression. b) If H ( x) has rank m it follows from [21, Theorem 1.66] that for m we have D F( x)() = D (G H)( x)() = H ( x) T D G(H( x))(). Taking the convex hull an using [25, Theorem 9.62] yiels F( x) T = conv D F( x)() = conv { H ( x) T D G(H( x))() } = H ( x) T conv { D G(H( x))() } = H ( x) T G(H( x)). This completes the proof The following example shows that if in the above theorem m has negative components an H ( x) is not onto, the esire assertion may fail. It is this chain rule which is erroneously applie in the proof of [6, Theorem 1 (i)] without further assumptions, an leas to the insufffiency of the proof given there. However, we point out, again, that we believe the assertion of [6, Theorem 1 (i)] to be true regarless. Example 4.5 (Failure in Lemma 4.4 a) when / m + ) Consier the function F := G H with H : 2, H(x) := ( x) x an G : 2 2, G(y) := ( (y 1) + ) ( (y 2) +, i.e., F(x) = (x)+ ) (x) +. It follows that {( ) } a F(0) = a [0, 1], a hence, for := ( 1 1) we have F(0) T ={0} = [ 1, 1] =H (0) T G(H(0)). We now give the main result of this section. Theorem 4.6 Let H : n m an g : m be continuously if ferentiable an ef ine f : n by f (x) := g(g(h(x))), where G(y) := [p i (y i )] i=1 m an p i : (i = 1,...,m) is a f inite piecewise linear max-function. Then s f : n + given by s f (x, ) := g([s pi (H i (x), )]) i=1 m ), where s p i is given as in Lemma 4.1, is a smoothing function for f. If furthermore x n is such that g(g(h( x))) m + or rank H ( x) = m

16 374 J.V. Burke et al. hols, then an hence Lim sup x s f (x, ) f ( x), x x, 0 conv { Lim sup x s f (x, ) } = f ( x), x x, 0 i.e., the graient consistency property (6) hols. Proof First, note that s f is a smoothing function for f since s pi has this property with respect to p i for all i = 1,...,m an g an H are continuously ifferentiable. Moreover, we compute the Clarke subifferential of f at x as f ( x) = (G H)( x) T g(g(h( x))) = H ( x) T G(H( x)) T g(g(h( x))) = H ( x) T iag ( p i (H i ( x))) g(g(h( x))), where the first equality is ue to [10, Theorem 2.6.6], the secon one follows (with = g(g(h( x)))) from Lemma 4.4, an the thir one exploits the componentwise structure of G. Now, we show that Lim sup x x, 0 x s f (x, ) f ( x) for all x n. To this en, we first note that ( ) x s f (x, ) = H (x) T iag t s p i (H i (x), ) g([s pi (H i (x), )] i=1 m ), by the orinary chain rule. Now, let v Lim sup x x, 0 x s f (x, ) be given. Then there exist sequences {x k } x an { k } 0such that ( ) H (x k ) T iag t s p i (H i (x k ), k ) g([s pi (H i (x k ), k )] i=1 m ) v. Since, ue to (14), we have t s p i (H i (x k ), k ) = ( Hi (x = a 1 (1 k ) t )) 2 F k ( Hi (x k ) t ) r + a r F, k r 1 + i=2 ( Hi (x a i (F k ) t ) ( i Hi (x k ) t )) i+1 F k k an F : [0, 1], the sequence {iag ( t s p i (H i (x k ), k ))} is boune, hence convergent on a subsequence, with a cluster point D iag ( p i (H i ( x))), ue to Lemma 4.1, an hence v = H ( x) T D g(g(h i ( x))) f ( x), which gives the asserte inclusion. The remaining statements now follow from Lemma 3.1.

17 Graient Consistency for Integral-convolution Smoothing Functions 375 We close this section by rawing the reaer s attention to the following result which is an immeiate consequence of Example 7.19 an Theorem 9.67 in [25]. Theorem 4.7 Let f : n be locally Lipschitz. Let ψ : n + be continuous with ψ(x) x = 1 an such that the sets B() := {x φ( x )>0} form a boune n sequence that converges to {0} as 0. Then the function s f given by s f (x, ) := f (x z) 1 ψ( z ) z is a smoothing function for f with conv Lim sup x s f (x, ) = f ( x) x n, x x, 0 an when f is regular the convex hull is superf luous. n Although this is a very powerful an very general result it oes not cover our analysis, since we o not restrict ourselves to mollif iers ψ that are continuous or have compact support, in the sense that is suggeste by the bouneness conition above. Mollifiers with non-compact support are essential since many interesting smoothing functions can be recovere using them. Two prominent examples are the Neural Network smoothing function obtaine via the mollifier ρ :,ρ(s) := e s (1+e s ) 2 an the Chen-Harker-Kanzow-Smale smoothing function obtaine via the mollifier ρ :,ρ(s) := 2, both by integral-convolution with the plus function, see (s 2 +4) 3 2 [6, 9] for etails. 5Finalemarks We investigate smoothing functions, base on integral-convolution, for a class of finite max-functions, which generalizes the analysis in [6] carrie out for the plus function. In the main result it was shown that, uner reasonable assumptions, (inner an outer) compositions with smooth functions fully agree with the framework laye out for the finite max-functions, an hence a satisfactory calculus is available. The analysis of Section 4 allows us to overcome a subtle flaw in [6, Theorem 1 (i)]. In [6, Theorem 1 (i)] it is state that the assertions of our Theorem 4.6, when applie to p i := ( ) + (i = 1,...,m), are vali without further assumptions on H or g. However, in the proof of [6, Theorem 1 (i)], the chain rule representation (using our notation) f ( x) = H ( x) T G(H( x)) g(g(h( x))) is invoke. This chain rule is shown to be false by Example 4.5. Nonetheless, we conjecture that the assertions in [6, Theorem1(i)] are vali. However, for the general case consiere in Theorem 4.6, we believe that the rank conition on H ( x) is essential. Acknowlegements We gratefully acknowlege the open iscussion with Prof. X. Chen [7] onthe gap in the proof of [6, Theorem 1 (i)].

18 376 J.V. Burke et al. eferences 1. Bazaraa, M.S., Sherali, H.D., Shetty, C.M.,: Nonlinear Programming. Theory an Algorithms, 2n en. John Wiley & Sons (1993) 2. Beck, A., Teboulle, M.,: Smoothing an first orer methos: a unifie framework. SIAM J. Optim. 22(2), (2012) 3. Bertsekas, D.P.: Nonifferentiable optimization via approximation. Math. Program. Stu. 3, 1 25 (1975) 4. Borwein, J.M., Lewis, A.S.: Convex Analysis an Nonlinear Optimization. Theory an Examples. CMS Books in Mathematics, Springer-Verlag, New York (2000) 5. Burke, J.V., Hoheisel, T.: Epi-convergent smoothing with applications to convex composite functions. Optimization Online (2012) 6. Chen, X.: Smoothing methos for nonsmooth, nonconvex minimization. Math. Program. 134, (2012) 7. Chen, X.: Private Communication. By (2012) 8. Chen, B., Harker, P.T.: A non-interior-point continuation metho for linear complementarity problems. SIAM J. Matrix Anal. Appl. 14, (1993) 9. Chen, C., Mangasarian, O.L.: A class of smoothing functions for nonlinear an mixe complementarity problems. Comput. Optim. Appl. 5, (1996) 10. Clarke F.H.: Optimization an Nonsmooth Analysis. John Wiley & Sons, New York (1983) 11. Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities an Complementarity Problems, vol. I an II. Springer, New York (2003) 12. Facchinei, F., Jiang, H., Qi, L.: A smoothing metho for mathematical programs with equilibrium constraints. Math. Program. 85, (1999) 13. Hiriart-Urrruty, J.-B., Lemaréchal, C.: Convex Analysis an Minimization Algorithms I+II. Secon Correcte Printing, Springer, Berlin, Heielberg (1996) 14. Hiriart-Urrruty, J.-B., Lemaréchal, C.: Funamentals of Convex Analysis. Grunlehren Text Eitions, Springer, Berlin, Heielberg (2001) 15. Ioffe, A.D.: Approximate subifferentials an applications I: The finite imensional case. Trans. Am. Math. Soc. 281, (1984) 16. Jongen, H.T., Stein, O.: Smoothing by mollifiers. Part I: semi-infinite optimization. J. Glob. Optim. 41, (2008) 17. Jongen, H.T., Stein, O.: Smoothing by mollifiers. Part II: nonlinear optimization. J. Glob. Optim. 41, (2008) 18. Kanzow, C.: Some noninterior continuation methos for linear complementarity problems. SIAM J. Matrix Anal. Appl. 17, (1996) 19. Morukhovich, B.S.: Maximum principle in the optimal time control problem with nonsmooth constraints. J. Appl. Math. Mech. 40, (1976) 20. Morukhovich, B.S.: Metric approximations an necessary optimality conitions for general classes of nonsmooth extremal problems. Sov. Math. Dokl. 22, (1980) 21. Morukhovich, B.S.: Variational Analysis an Generalize Differentiation I. Basic Theory. In: A Series of Comprehensive Stuies in Mathematics, vol Springer, Berlin, Heielberg (2006) 22. Nesterov, Y.: Smooth minimization of non-smooth functions. Math. Program. 103, (2005) 23. Noceal, J., Wright, S.J.: Numerical Optimization. Springer Series in Operations esearch, Springer-Verlag, New York (1999) 24. Qi, L.: Convergence analysis of some algorithms for solving nonsmooth equations. Math. Oper. es. 18, (1993) 25. ockafellar,.t., Wets,.J.-B.: Variational analysis. In: A Series of Comprehensive Stuies in Mathematics, vol Springer, Berlin, Heielberg (1998) 26. Scholtes, S.: Introuction to Piecewise Smooth Equations. Habilitation Thesis, University of Karlsruhe (1994) 27. Smale, S.: Algorithms for solving equations. In: Proceeings of the International Congress of Mathematicians, pp American Mathematical Society, Provience (1987)