Self-dual Smooth Approximations of Convex Functions via the Proximal Average

Transcription

1 Chapter Self-dual Smooth Approximations of Convex Functions via the Proximal Average Heinz H. Bauschke, Sarah M. Moffat, and Xianfu Wang Abstract The proximal average of two convex functions has proven to be a useful tool in convex analysis. In this note, we express the Goebel self-dual smoothing operator in terms of the proximal average, which allows us to give a different proof of self duality. We also provide a novel self-dual smoothing operator. Both operators are illustrated by smoothing the norm. Key words: approximation, convex function, Fenchel conjugate, Goebel smoothing operator, Moreau envelope, proximal average. AMS 00 Subject Classification: Primary 6B5; Secondary 6B05, 65D0, 90C5.. Introduction Let X be the standard Euclidean space R n, with inner product, and induced norm. It will be convenient to set q =.. Heinz H. Bauschke University of British Columbia, 3333 University Way Kelowna, BC Canada VV V7, heinz.bauschke@ubc.ca Sarah M. Moffat University of British Columbia, 3333 University Way Kelowna, BC Canada VV V7 sarah.moffat@ubc.ca Xianfu Wang University of British Columbia, 3333 University Way Kelowna, BC Canada VV V7 shawn.wang@ubc.ca

2 Heinz H. Bauschke, Sarah M. Moffat, and Xianfu Wang Now let f : X ], + ] be convex, lower semicontinuous, and proper. Since many convex functions are nonsmooth, it is natural to ask: How can one approximate f with a smooth function? The most famous and very useful answer to this question is provided by the Moreau envelope [5, 7], which, for λ > 0, is defined by e λ f = f λ q.. It is well known that e λ f is smooth i.e., continuously differentiable and that lim λ 0 + e λ f = f point-wise; see, e.g., [7, Theorem.5 and Theorem.6]. Parenthetically, other approaches to smoothing are Ghomi s integral convolution method [9], Seeger s ball rolling technique [8], and Teboulle s entropic proximal mappings [9]. Let us now consider the norm, which is nonsmooth at the origin. Example.. Moreau envelope of the norm Let λ ]0, [, set f =, and denote the closed unit ball by C. Then, for x and x in X, we have x, if x λ; λ e λ f x = x λ.3, if x > λ, eλ f = ιc + λq, and e λ f x = λ max{0, x }. Consequently, eλ f eλ f. Proof. Either a straight-forward computation or [7, Example.6a] yields Next, if y X, then f = ι C..4 e /λ ι C y = inf λqy c.5 c C = λ d Cy.6, = λ y if y > ; 0, if y,.7 The symbol denotes infimal convolution: f f x = inf y f y + f x y. Here, ι C is the indicator function defined by ι C x = 0, if x C; ι C x = +, if x / C, f x = sup x X x,x f x is the Fenchel conjugate of f, and d C x = inf c C x c = ι C x is the distance function.

3 Self-dual Smooth Approximations of Convex Functions via the Proximal Average 3 and thus, λ x/λ if x > λ; e /λ ι C x/λ = 0, if x λ. By [7, Example.6b on page 495], we obtain.8 e λ f x = λ qx e /λ f x/λ.9 = λ x λ x λ x +, if x > λ; λ.0 0, if x λ x λ, if x > λ; = x λ, if x λ. and e λ f = f + λq = ι C + λq. Alternatively, one may use [7, Example.6], which provides the proximal mapping of f, and then use the proximal mapping calculus to obtain these results. Finally, a referee pointed out that can also be derived by reducing the computation of the Moreau envelope to e λ f x = f λ qx. = inf f y + λ qx y.3 y = inf y = inf y + λ η 0 y =η = inf η 0 y =η x + y x,y.4 inf y + x + y x,y.5 λ inf η + x + η η x.6 λ η + λ x η,.7 = x λ + λ inf η 0 which can now be treated by one-dimensional calculus. While the Moreau envelope has many desirable properties, we see from Example. that the smooth approximation e λ f is not self-dual in the sense that eλ f eλ f..8 It is perhaps surprising that self-dual smoothing operators even exist. The first example appears in []. Specifically, Goebel defined

4 4 Heinz H. Bauschke, Sarah M. Moffat, and Xianfu Wang G λ f = λ e λ f + λq.9 and proved that Gλ f = Gλ f,.0 i.e., Fenchel conjugation and Goebel smoothing commute! For applications of the Goebel smoothing operator, see []. The purpose of this note is two-fold. First, we present a different representation of the Goebel smoothing operator which allows us to prove self-duality using the Fenchel conjugation formula for the proximal average. Second, the proximal average is also utilized to obtain a novel smoothing operator. Both smoothing operators are computed explicitly for the norm. The formulas derived show that the new smoothing operator is distinct from the one provided by Goebel. For f and f, two functions from X to ],+ ] that are convex, lower semicontinuous and proper, and for two strictly positive convex coefficients λ + λ =, the proximal average is defined by pav f, f ;λ,λ = λ f + q + λ f + q q.. The proximal average, which is actually a convex function, has been a useful tool for constructing primal-dual symmetric antiderivatives [5] and for extending monotone operators []; see also [3, 4, 6,, ] for further information and applications. One of the key properties is the Fenchel conjugation formula pav f, f ;λ,λ = pav f, f ;λ,λ ;. see [6, Theorem 6.], [4, Theorem 4.3], or [3, Theorem 5.]. We use standard convex analysis calculus and notation as, e.g., in [6, 7, ]. In Section, we consider the Goebel smoothing operator from the proximal-average view point. The new smoothing operator is presented in Section 3.. The Goebel smoothing operator Definition.. Goebel smoothing operator Let f : X ], + ] be convex, lower semicontinuous and proper, and let λ ]0,[. Then the Goebel smoothing operator [] is defined by G λ f = λ e λ f + λq..3 Note that 3 and standard properties of the Moreau envelope imply that point-wise and that each G λ f is smooth. lim G λ 0 + λ f = f.4

5 Self-dual Smooth Approximations of Convex Functions via the Proximal Average 5 Our first main result provides two alternative descriptions of the Goebel smoothing operator. The first description, item in Theorem., shows a pleasing reformulation in terms of the proximal average. The second description, item in Theorem., is less appealing but has the advantage of providing a different proof of the self-duality, item 3, observed by Goebel. Theorem.. Let f : X ], + ] be convex, lower semicontinuous and proper, and let λ ]0,[. Then the following hold 3.. G λ f = + λpav f,0; λ,λ + λq.. G λ f = + λ pav f,q; λ +λ, λ +λ + λ Id. 3. Goebel G λ f = Gλ f. Proof. Let x X. Then, using and standard convex calculus, we obtain + λ pav f,q; λ + λ, λ + λ Id x.5 + λ λ = + λ + λ f + q + λ x q + q q + λ + λ.6 λ = + λ λ x f + q + + λ + λ q qx.7 + λ = + λ λ f + q + λq x qx.8 = + λ λ f + q + λ 0 + q q x + λqx.9 = + λpav f,0; λ,λ + λq x..30 We have verified that 8 as well as the right sides of and coincide. Starting from 8 and again applying standard convex caluclus, we see that + λ λ f + q + λq x qx.3 = + λ λ f + q λq x qx.3 = + λ λ f + q λ λ q x qx.33 3 Here Id: X X : x x is the identity operator.

6 6 Heinz H. Bauschke, Sarah M. Moffat, and Xianfu Wang = + λinf λ f + q y + y λ λ qx y qx.34 y y = + λinf λ f + λq + qx y y λ λ λ + λ qx.35 y y = λ inf f + q + qx y y λ λ λ λ λ qx..36 Simple algebra shows that for every y X, y q + λ qx y λ λ λ qx = λ q x y + λ λ λ qx..37 Therefore, + λ λ f + q + λq x qx.38 y y = λ inf f + q + y λ λ qx y λ λ λ qx.39 y = λ inf f + y λ λ q x y + λ λ λ qx = λ inf f z + qx z + λ z λ λ qx.40.4 = λ e λ f + λq x.4 = G λ f x,.43 which completes the proof of and. 3: In view of the conjugate formula β h β Id = β h β Id,, and, we obtain Gλ f = + λ pav f,q; λ + λ, λ + λ Id.44 + λ = + λ pav f,q; λ + λ, λ + λ Id.45 + λ

7 Self-dual Smooth Approximations of Convex Functions via the Proximal Average 7 = + λ pav f,q ; λ + λ, λ + λ Id.46 + λ = + λ pav f,q; λ + λ, λ + λ Id.47 + λ = G λ f..48 The proof is complete. Remark..3 Theorem.& gives two representations of the Goebel smoothing operator in terms of the proximal average. Goebel [0] discovered a converse formula, which we state next without proof: pav f,q;λ, λ = λ G 4 λ/ λ f λ Id..49 Example..4 Let λ ]0, [ and set f =. Then, for every x X, x, if x λ; λ G λ f x = λ x + λ x λ λ, if x > λ. Proof. Combine 3 and A new smoothing operator We now provide a novel smoothing operator that has a very simple expression in terms of the proximal average. Definition.3. new smoothing operator Let f : X ], + ] be convex, lower semicontinuous and proper, and let λ ]0,[. Then the S λ f is defined by S λ f = pav f,q; λ,λ..5 Theorem.3. Let f : X ], + ] be convex, lower semicontinuous and proper, and let λ ]0,[. Set µ = λ/ λ. Then the following hold.. S λ f = λe µ f λ Id + µq.. Sλ f = Sλ f.

8 8 Heinz H. Bauschke, Sarah M. Moffat, and Xianfu Wang Proof. : Let x X. Then, using 5, and standard convex calculus, we obtain Sλ f x = λ f + q + λq + q x qx.5 = λ f + q + λ q x qx.53 = λ f + q λ λ q x qx.54 y y = inf λ f + λq + y λ λ λ qx y qx.55 y y = λinf f + q + y λ λ Simple algebra shows that for every y X, q y + qx y λ λ λ λ qx = λ x λ q λ y λ qx y λ λ + λ qx..56 λ qx..57 λ λ Therefore, Sλ f x.58 y = λinf f + λ x y λ λ q λ y + λ λ λ λ qx.59 = λinf f z + λ x z λ q λ z + λ qx.60 λ = λ f x µ q + µqx,.6 λ as claimed. : Using 5 and, we get

9 Self-dual Smooth Approximations of Convex Functions via the Proximal Average 9 Sλ f = pav f,q; λ,λ.6 = pav f,q ; λ,λ.63 = pav f,q; λ,λ.64 = S λ f..65 The proof is complete. Note that Theorem 3. and standard properties of the Moreau envelope imply that point-wise lim S λ 0 + λ f = f.66 and that each S λ f is smooth. Example.3.3 Let λ ]0, [ and set f =. Then, for every x X, λ x, if x λ λ ; S λ f x = λ x λ λ λ + x λ λ λ, if x > λ. Proof. Combine 3 and Theorem Remark.3.4 Let f =. The explicit formulas provided in Example.4 and Example 3.3 imply that G α f S β f, for all α and β in ]0,[. Thus, the smoothing operator defined by 5 is indeed new and different from the Goebel smoothing operator. Remark.3.5 It would be desirable to obtain further explicit formulas beyond the example of the norm. Given a more complicated function f, the explicit computation of the smoothing operators G λ f and S λ f may not be easy. However, computational convex analysis provides tools [8, 3, 4] to compute the Moreau envelope numerically which due to the Moreau envelope formulations 3 and Theorem 3. make it possible to compute the smoothing operators G λ f and S λ f numerically. It would also be interesting to extend the present results to infinite-dimensional settings. Promising starting points for this endeavour are [, ]. Finally, self-dual regularizations of maximal monotone operators are studied in [0]. Acknowledgements The authors would like to thank the two referees for their constructive comments. Heinz Bauschke was partially supported by the Natural Sciences and Engineering Research Council of Canada and by the Canada Research Chair Program. Sarah Moffat was partially supported by the Natural Sciences and Engineering Research Council of Canada. Xianfu Wang was partially supported by the Natural Sciences and Engineering Research Council of Canada.

10 0 Heinz H. Bauschke, Sarah M. Moffat, and Xianfu Wang References. Attouch, H.: Variational Convergence for Functions and Operators. Pitman, Boston 984. Bauschke, H.H. and Wang, X.: The kernel average for two convex functions and its applications to the extension and representation of monotone operators. Trans. Amer. Math. Soc. 36, Bauschke, H.H., Goebel, R., Lucet, Y., and Wang, X.: The proximal average: basic theory. SIAM J. Optim. 9, Bauschke, H.H., Lucet, Y., and Trienis, M.: How to transform one convex function continuously into another. SIAM Rev. 50, Bauschke, H.H., Lucet, Y., and Wang, X.: Primal-dual symmetric intrinsic methods for finding antiderivatives for cyclically monotone operators. SIAM J. Control Optim. 46, Bauschke, H.H., Matoušková, E., and Reich, S.: Projection and proximal point methods: convergence results and counterexamples. Nonlinear Anal. 56, Combettes, P.L., and Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4, Gardiner, B. and Lucet, Y.: Graph-matrix calculus for computational convex analysis. Preprint Ghomi, M.: The problem of optimal smoothing for convex functions. Proc. Amer. Math. Soc. 30, Goebel, R.: personal communication 006. Goebel, R.: Self-dual smoothing of convex and saddle functions. J. Convex Anal. 5, Goebel, R.: The proximal average for saddle functions and its symmetry properties with respect to partial and saddle conjugacy. J. Nonlinear Convex Anal., Lucet, Y.: Faster than the fast Legendre transform, the linear-time Legendre transform. Numer. Algorithms 6, Lucet, Y., Bauschke, H.H., and Trienis, M.: The piecewise linear-quadratic model for computational convex analysis. Comput. Optim. Appl. 43, Moreau, J.J.: Proximité et dualité dans un espace hilbertien. Bull. Soc. Math. France 93, Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton Rockafellar, R.T. and Wets, R.J-B.: Variational Analysis. c-*orrected 3rd printing, Springer- Verlag, Berlin Seeger, A.: Smoothing a nondifferentiable convex function: the technique of the rolling ball. Rev. Mat. Apl. 8, Teboulle, M.: Entropic proximal mappings with applications to nonlinear programming. Math. Oper. Res. 7, Wang, X.: Self-dual regularization of monotone operators via the resolvent average. Preprint 00. Zălinescu, C.: Convex Analysis in General Vector Spaces. World Scientific Publishing, River Edge, NJ 00