Proximal Point Method for a Class of Bregman Distances on Riemannian Manifolds

Transcription

1 Proximal Point Method for a Class of Bregman Distances on Riemannian Manifolds E. A. Papa Quiroz and P. Roberto Oliveira PESC-COPPE Federal University of Rio de Janeiro Rio de Janeiro, Brazil erik@cos.ufrj.br, poliveir@cos.ufrj.br September 2005 Abstract This paper generalizes the proximal point method using a class of Bregman distances to solve convex and quasiconvex optimization problems on complete Riemannian manifolds. We will prove, under standard assumptions, that the sequence generated by our method is well defined and converges to an optimal solution of the problem. Also, we give some examples of Bregman distances in non-euclidean spaces. Keywords: Proximal point algorithms; Riemannian manifolds; Bregman distances; diagonal Riemannian metrics.

2 1 Introduction Let consider the problem min f(x), x X where f : IR n IR is a convex function on a closed totally convex set X of IR n. The proximal point algorithm with Bregman distance, henceforth abreviated PBD algorithm, generates a sequence {x k } defined by Given x 0 S, x k = arg min {f(x) + λ k D h (x, x k 1 )}, (1.1) x X S where h is a Bregman function with zone S, such that X S φ, λ k is a positive parameter and D h is a Bregman distance defined as D h (x, y) = h(x) h(y) f(x), x y, where, denotes the usual inner product on IR n. Convergence and rate of convergence results, under appropriate assumptions on the problem, have been proved by several authors for certain choices of the regularization parameters λ k, (see, for example, [2, 4, 14, 15]). That algorithm has also been generalized for variational inequalities problems in Hilbert and Banach spaces, see [1, 13]. Variational Inequalities Problems arise naturally in several Engineering applications and recover optimization problems as a particular case. On the other hand, generalization of known methods in optimization from Euclidean space to Riemannian manifolds is in a certain sense natural, and advantageous in some cases. For example, we can consider the intrinsic geometry of the manifold, and constrained problems can be seen as unconstrained ones. Another application is that certain non convex optimization problems become convex through the introduction of an adequate Riemannian metric on the manifold, so we can use more efficient optimization techniques, see [8, 7, 11, 16, 20, 23, 24], and the references therein. Another advantage of that approach is that we can use Riemannian metric to introduce new algorithms in interior point methods, (see, for example, [21, 5, 9, 19]). In this paper we generalize the PBD algorithm to solve optimization problems on complete Riemannian manifolds. Our approach is new and it is related to the work of Ferreira and Oliveira [12]. In that paper, the authors have generalized the proximal point method using the intrinsic Riemannian distance for Hadamard manifolds. Here, we consider the following regularization parameters lim λ k = 0, with λ k > 0, (1.2) k + for the quasiconvex case, and 0 < λ k < λ, (1.3) for the convex one. For λ k satisfying (1.2), we obtain the convergence of the PBD algorithm, for quasiconvex functions, with no assumptions on the Bregman function.for λ k satisfying (1.3), we obtain the convergence of the PBD algorithm using a certain class of Bregman functions. That class satisfies either a Lipschitz condition on its gradient or a relation between the gradient and the geodesic triangle of the manifold. In particular, the second condition is satisfied by any Bregman function in IR n. The paper is divided as follows. In Section 2 we give the notation on Riemannian geometry that we will use along the paper. In Section 3, we recall some facts on convex analysis on Riemannian manifolds. In Section 4 the definition of Bregman function is introduced, besides 2

3 some properties. Section 5 presents the Moreau-Yosida regularization, by considering Bregman distances. In Section 6 we introduce the PDB algorithm to solve minimization problems on Riemannian manifolds and we prove the convergence of the sequence generated by the algorithm. In Section 7 are presented some explicit examples of Bregman distances in non Euclidean spaces, in the following section we give our conclusions and future works. An appendix with some new geodesic curves for diagonal metrics is also presented. 2 Some Tools of Riemannian Geometry In this section we recall some fundamental properties and notation on Riemannian manifolds. Those basic facts can be seen, for example, in [10]. Let M be a differential manifold. We denote by T x M the tangent space of M at x and T M = x M T xm. T x M is a linear space and has the same dimension of M. Because we restrict ourselves to real manifolds, T x M is isomorphic to IR n. If M is endowed with a Riemannian metric g, then M is a Riemannian manifold and we denoted it by (M, G) or only by M when no confusion can arise, where G denotes the matrix representation of the metric g. The inner product of two vectors u, v T x S is written u, v x := g x (u, v), where g x is the metric at the point x. The norm of a vector v T x S is defined by v x := v, v x 1/2. The metric can be used to define the length of a piecewise smooth curve α : [t 0, t 1 ] S joining α(t 0 ) = p to α(t 1 ) = p through L(α) = t 1 t 0 α (t) dt. Minimizing this length functional over the set of all curves we obtain a Riemannian distance d(p, p) which induces the original topology on M. Given two vector fields V and W in M (a vector field V is an application of M in T M), the covariant derivative of W in the direction V is denoted by V W. In this paper is the Levi-Civita connection associated to (M, G). This connection defines an unique covariant derivative D/dt, where for each vector field V, along a smooth curve α : [t 0, t 1 ] M, another vector field is obtained, denoted by DV/dt. The parallel transport along α from α(t 0 ) to α(t 1 ), denoted by P α,t0,t 1, is an application P α,t0,t 1 : T α(t0 )M T α(t1 )M defined by P α,t0,t 1 (v) = V (t 1 ) where V is the unique vector field along α such that DV/dt = 0 and V (t 0 ) = v. Since that is a Riemannian connection, P α,t0,t 1 P α,t0,t 1 = P α,t,t1 P α,t0,t, for all t [t 0, t 1 ]. A curve γ : [0, 1] M starting from x on the direction v T p M is called a geodesic if d 2 γ k dt 2 + n i,j=1 where γ i are the coordinates of γ and Γ k ij Γ m ij = 1 2 n k=1 is a linear isometry, furthermore P 1 α,t 0,t 1 = P α,t1,t 0 and Γ k dγ i dγ j ij dt dt γ(0) = x and γ (0) = v, { x i g jk + = 0, k = 1,..., n, (2.4) are the Christoffel s symbols expressed by x j g ki } g ij g km, (2.5) x k (g ij ) denotes the inverse matrix of the metric G = (g ij ), and x i are the coordinates of x. A Riemannian manifold is complete if its geodesics are defined for any value of t IR. Let x M, the exponential map exp x : T x M M, defined as exp x (v) = γ(1). If M is complete, then exp x is defined for all v T x M. Besides, there is an unique minimal geodesic (its length is equal to the distance between the extremes). 3

4 Given the vector fields X, Y, Z on M, we denote by R the curvature tensor defined by R(X, Y )Z = Y X Z X Y Z + [X,Y ] Z, where [X, Y ] := XY Y X is the Lie bracket. Now, the sectional curvature with respect to X and Y is defined by K(X, Y ) = R(X, Y )Y, X X 2 Y 2 X, Y 2. The gradient of a differentiable function f : M IR, gradf, is a vector field on M defined through df(x) = gradf, X = X(f), where X is also a vector field on M. 3 Convex Analysis on Riemannian Manifolds In this section we give some definitions and results of Convex Analysis on Riemannian manifolds. We refer the reader to [24] for more details and the proofs of the theorems. Definition 3.1 Let M be a Riemannian manifold. A subset A is said totally convex in M if, for any pair of points all geodesics joining these points are contained in A, that is, given x, y A and γ : [0, 1] M, the geodesic curve such that γ(0) = x, γ(1) = y verifies γ(t) A, for all t [0, 1]. Definition 3.2 Let A be a totally convex set in a complete Riemannian manifold M and f : A IR be a real-valued function. f is called a quasiconvex function on A if for all x, y A, t [0, 1], it holds that f(γ(t)) max{f(x), f(y)}, for any geodesic γ : [0, 1] IR, such that γ(0) = x and γ(1) = y. Theorem 3.1 Let A be a totally convex set in a complete Riemannian manifold M. The function f : A IR is quasiconvex if and only if the set {x A : f(x) c} is totally convex for each c IR. Definition 3.3 Let A be a totally convex set in a complete Riemannian manifold M and f : A IR be a real-valued function. f is called a convex function on A if for all x, y A, t [0, 1], it holds that f(γ(t)) tf(y) + (1 t)f(x), for any geodesic γ : [0, 1] IR, such that γ(0) = x and γ(1) = y. When the preceding inequality is strict, for x y and t (0, 1) the function f is said to be strictly convex. Theorem 3.2 Let M be a complete Riemannian manifold and A be a totally convex set in M. The function f : A IR is convex if and only if x, y A and γ : [0, 1] M (the geodesic joining x to y) the function f(γ(t)) is convex in [0, 1]. Proof. See [24]. Let M be a complete Riemannian manifold and let f : M IR be a convex function. Take y M, the vector s T y M is said to be a subgradient of f at y if, for any geodesic γ of M, with γ(0) = y, f(γ(t)) f(y) + t s, γ (0) y, (3.6) for all x M. The set of all subgradients of f at y is called the subdifferential of f at y and is denoted by f(y). 4

5 Theorem 3.3 Let M be a complete Riemannian manifold and let f : M IR be a convex function. Then, for any y M, there exists s T y M such that x M (3.6) is true. From previous theorem the subdifferential f(x) of a convex function f at x M is nonempty. Theorem 3.4 Let M be a complete Riemannian manifold and f : M IR be a convex function. Then 0 f(x) if and only if x is a minimum point of f in M. 4 Bregman Distances and Functions on Riemannian Manifolds To construct generalized proximal point algorithms with Bregman distances for solving optimization problems on complete Riemannian manifolds, it is necessary to extend the definitions of Bregman distances and Bregman functions to that framework. Starting from Censor and Lent [3] definition, we propose the following. Let M be a complete Riemannian manifold and S a nonempty open totally convex set of M with a topological closure S. Let h : M IR be a strictly convex function on S and differentiable in S. From now on, the exponential map is associated to the minimal geodesic. Then, the Bregman distance associated to h, denoted by D h, is defined as a function D h (.,.) : S S IR such that D h (x, y) := h(x) h(y) gradh(y), exp 1 y x y. (4.7) Notice that the expression of the Bregman distance depends on the definition of the metric. Some examples for different manifolds will be given in Section 7. Let us adopt the following notation for the partial level sets of D h. For α IR, take Γ 1 (α, y) := {x S : D h (x, y) α}, Γ 2 (x, α) := {y S : D h (x, y) α}. Definition 4.1 Let M be a complete Riemannian manifold. A real-valued function h : M IR is called a Bregman function if there exists a nonempty open totally convex set S such that a. h is continuous on S; b. h is strictly convex on S; c. h is continuously differentiable in S; d. For all α IR the partial level sets Γ 1 (α, y) and Γ 2 (x, α) are bounded for every y S and x S, respectively; e. If lim k y k = y S, then lim k D h (y, y k ) = 0; f. If lim k D h (z k, y k ) = 0, lim k y k = y S and {z k } is bounded then lim k z k = y. We denote the family of Bregman functions by B and refer to the set S as the zone of the function h. Lemma 4.1 Let h B with zone S. Then i. grad D h (., y)(x) = grad h(x) P γ,0,1 grad h(y), for all x, y S, where γ : [0, 1] M is the minimal geodesic curve such that γ(0) = y and γ(1) = x. 5

6 ii. D h (., y) is strictly convex on S for all y S. iii. For all x S and y S, D h (x, y) 0 and D h (x, y) = 0 if and only if x = y. Proof. i. From definition (4.7), we have p(t) := D h (γ(t), γ(0)) = h(γ(t)) h(γ(0)) t grad h(γ(0)), γ (0) γ(0). Derivation with respect to t gives p (t) = grad h(γ(t)), γ (t) γ(t) grad h(γ(0)), γ (0) γ(0). Using the fact that P γ,0,t γ (0) = γ (t) we obtain On the other hand, p (t) = grad h(γ(t)) P γ,0,t grad h(γ(0)), γ (t) γ(t). (4.8) p (t) = d dt (D h(γ(t), γ(0))) = d(d h (., γ(0))) γ(t) (γ (t)) = grad D h (., γ(0))(γ(t)), γ (t) γ(t). From (4.8) and (4.9) follows Taking t = 1 gives grad D h (., γ(0))(γ(t)) = grad h(γ(t)) P γ,0,t grad h(γ(0)). gradd h (., y)(x) = grad h(x) P γ,0,1 grad h(y). ii. As h is a strictly convex function on S and grad h(y), exp 1 y x y is a linear function in x M, then D h (., y) is strictly convex on S. iii. Use, again, the strict convexity of h. Observe that D h is not a distance in the usual sense of the term. In general, the triangular inequality is not valid, as the symmetry property. From now on, we use the notation grad D h (x, y) to mean grad D h (., y)(x). So, if γ is the minimal geodesic curve such that γ(0) = y and γ(1) = x, from Lemma 4.1, i, we obtain gradd h (x, y) = grad h(x) P γ,0,1 grad h(y). Definition 4.2 Let Ω M, S an open totally convex set, and let y S. A point P y Ω S for which D h (P y, y) = min x Ω S D h (x, y) (4.10) is called a D h projection of the point y on the set Ω. The next Lemma furnishes the existence and uniqueness of D h projection for a Bregman function, under an appropriate assumption on Ω, Lemma 4.2 Let Ω M a closed totally convex set, and h B with zone S. If Ω S φ then, for any y S, there exists a unique D h projection P y on Ω. (4.9) 6

7 Proof. For any x Ω S, the set B := {z S : D h (z, y) D h (x, y)} is bounded (from Definition 4.1, d) and closed (because D h (., y) is continuous in S, due to Definition 4.1, a). Therefore, the set T := (Ω S) B is nonempty, because x B Ω, and bounded. Now, as the intersection of closed sets is closed, then T is also closed, hence compact. Consequently, D h (z, y), a continuous function in z, takes its minimum on the compact set T at some point, let denote it by x. For every z Ω S which lies outside B D h (x, y) < D h (z, y); hence, x satisfies (4.10). The uniqueness follows from the strict convexity of D h (., y), therefore x = P y. Lemma 4.3 Let h B with zone S and y S. Suppose that P y S, where P y is the D h projection on some closed totally convex set Ω such that Ω S φ. Then, the function is convex on S. Proof. From (4.7) g(x) := D h (x, y) D h (x, P y) D h (x, y) D h (x, P y) = h(p y) h(y) + gradh(p y), exp 1 P y x P y gradh(y), exp 1 y x y. Due to the linearity of the functions gradh(p y), exp 1 P y x P y and gradh(y), exp 1 y x y in x the result follows. Proposition 4.1 Let h B with zone S; S and Ω M are closed totally convex sets such that Ω S φ. Let y S and assume that P y S, where P y denotes the D h projection of y on Ω. Then, for any x Ω S, the following inequality is true D h (P y, y) D h (x, y) D h (x, P y). Proof. Let γ : [0, 1] M be the minimal geodesic curve such that γ(0) = P y and γ(1) = x. Due to Lemma 4.3 the function G(x) = D h (x, y) D h (x, P y) is convex on S. Then in particular G(γ(t)) is convex for t (0, 1) (see Theorem 3.2). Thus, which gives, G(γ(t)) tg(x) + (1 t)g(p y), D h (γ(t), y) D h (γ(t), P y) t(d h (x, y) D h (x, P y)) + D h (P y, y) td h (P y, y), 7

8 where we used the fact that D h (P y, P y) = 0. The above inequality is equivalent to (1/t)(D h (γ(t), y) D h (P y, y)) (1/t)D h (γ(t), P y) D h (x, y) D h (x, P y) D h (P y, y). (4.11) As Ω S is totally convex, and x, P y Ω S, we have γ(t) Ω S for all t (0, 1). Then, use the fact that P y is the projection to get Using this inequality in (4.11) we obtain (1/t)(D h (γ(t), y) D h (P y, y)) 0. (1/t)D h (γ(t), P y) D h (x, y) D h (x, P y) D h (P y, y). Now, as D h (., z) is differentiable for all z S, we can take the limit in t, obtaining gradd h (P y, P y), exp 1 P y x P y D h (x, y) D h (x, P y) D h (P y, y). Clearly, the left side is null, leading to the aimed result. 5 Regularization Let M be a complete Riemannian manifold and f : X M IR a real-valued function. Let S an open totally convex set and h : S IR a differentiable function in S. For λ > 0, the Moreau-Yosida regularization f λ : S IR of f is defined by f λ (y) = inf {f(x) + λd h (x, y)} (5.12) x X S where D h (x, y) is given in (4.7). In order to prove that the function (5.12) is well defined, h and f should satisfy some conditions. Proposition 5.1 If f : X M IR is a bounded below quasiconvex and continuous function over a closed totally convex set X, and h B with zone S such that X S φ, then, for every y S and λ > 0 there exists a unique point, denoted by x f (y, λ), such that Proof. Let β a lower bound for f on X, then f λ (y) = f(x f (y, λ)) + λd h (x f (y, λ), y). (5.13) f(x) + λd h (x, y) β + λd h (x, y), for all x X S. It follows from Definition 4.1, d, that the level sets of the function f(.) + λd h (., y) are bounded. Also, this function is continuous on X S, due to Definition 4.1, a, and the hypothesis on f. So, the level sets of (f(.) + λd h (., y)) are closed, hence compact. Now, from continuity and compactness arguments, f(.) + λd h (., y) has a minimum on X S. Finally, the strict convexity of D h (., y), see Lemma 4.1, ii, together the quasiconvexity of f, lead to the uniqueness of x f (y, λ), the equality (5.13) following from (5.12). Now, we let a different condition on h, that also ensures that (5.12) is well defined. We consider the unconstrained case X = S = M, so (5.12) is reduced to Let us introduce the following definition. f λ (y) = inf x M {f(x) + λd h(x, y)}. 8

9 Definition 5.1 A function g : M IR is 1 coercive at y M if lim d(x,y) g(x) d(x, y) = +. Note that if g : M IR is a 1 coercive function at y M, then it is easy to show that the minimizer set of g on M is nonempty. Lemma 5.1 If f : M IR is bounded below, λ > 0, and h : M IR is 1 coercive at y M, then the function f(.) + λd h (., y) : M IR is 1 coercive at y M. Proof. As above, let β a lower bound for f. Then: f(x) + λd h (x, y) d(x, y) β d(x, y) + λd h(x, y) d(x, y) = β d(x, y) + λ h(x) d(x, y) λ h(y) d(x, y) λ gradh(y), exp 1 y x d(x, y) y β d(x,y) + λ h(x) d(x,y) λ h(y) d(x,y) λ gradh(y), where the equality comes from the definition of D h, and the last inequality results from the application of Cauchy inequality, and the fact that exp 1 y x = d(x, y). Taking d(x, y), we use the 1-coercivity assumption of h at y, to get (f(.) + λd h (., y))(x) lim = +. d(x,y) d(x, y) Proposition 5.2 Let h : M IR be a 1 coercive strictly convex function at y M, f : M IR a quasiconvex bounded below function. Then, there exists an unique point x f (y, λ) such that f λ (y) = f(x f (y, λ)) + λd h (x f (y, λ), y) Proof. The result follows from the Lemma above and, as in the proof of the preceding proposition, from the strict convexity of D h (., y) and the quasiconvexity of f. 6 Proximal Point Algorithm We are interested in solving the optimization problem: (p) min x M f(x) where M is a complete Riemannian manifold. The main convergence results will be given when f is a continuous quasiconvex or a convex function on M. The PBD algorithm is defined as x 0 M, (6.14) x k = arg min x M {f(x) + λ kd h (x, x k 1 )}, (6.15) 9

10 where h is a Bregman function with zone M, D h is as in (4.7) and λ k is a positive parameter. In the particular case where M is the Euclidean space IR n, and h(x) = (1/2)x T x, we have x k = arg min x IR n{f(x) + (λ k/2) x x k 1 2 }. Therefore, the PBD algorithm is a natural generalization of the proximal point algorithm on IR n. We will use the following parameter conditions to the PBD algorithm: 0 < λ k < λ, or (6.16) Note that (6.16) implies that considered in the following. lim λ k = 0, with λ k > 0. (6.17) k + k=1 (1/λ k ) = +. Next, we set some Hypothesis that wil be Assumption A1. The optimal set of the problem (p), denoted by X, is nonempty. Assumption A2. h B satisfies one of the following conditions: A2.1 gradh is a Lipschitz function on the bounded subsets of M, that is, for any x, y B, for any bounded subset B of M, it holds gradh(x) P γ,0,1 gradh(y) Ld(x, y), where L = L(B) > 0, γ(t) is the minimal geodesic curve such that γ(0) = y and γ(1) = x, and d is the Riemannian distance. A2.2 Given x, y, z M and letting γ : [0, 1] M the geodesic curve joining γ(0) = y and γ(1) = z, then h satisfies P γ,0,1 gradh(y), exp 1 z x exp 1 z y P γ,0,1 exp 1 y x z 0. Remark 1. The alternative Assumptions A2 will be used to prove that any limit point of the sequence {x k }, with λ k as in (6.16), is an optimal solution of the problem (p). When λ k verifies (6.17), we will show that Assumption A2 is not necessary to prove convergence of the PBD algorithm. On the other hand, it is worthwhile observe that when M is the Euclidean space IR n, any Bregman function satisfies the Assumption A2.2. Clearly, in the Euclidean space geodesic curves are straight lines, so Therefore, exp 1 z y + P γ,0,1 exp 1 y x = (y z) + (x y) = x z = exp 1 z x, P γ,0,1 gradh(y), exp 1 z Furthermore, that hypothesis implies x exp 1 z y P γ,0,1 exp 1 y x z = 0. D h (x, y) D h (z, y) D h (x, z) gradh(z) P γ,0,1 gradh(y), exp 1 z x z. (6.18) 10

11 Indeed, using the definition of D h, it gets D h (x, y) D h (z, y) D h (x, z) = gradh(y), exp 1 y z exp 1 y x y + grad h(z), exp 1 z x z Taking Parallel transport from γ(0) = y to γ(1) = z in the first term on the right side gives D h (x, y) D h (z, y) D h (x, z) = = P γ,0,1 grad h(y), exp 1 z y P γ,0,1 exp 1 y x z + grad h(z), exp 1 z x z, Now, add and subtract the term P γ,0,1 gradh(y), exp 1 z x z, to produce D h (x, y) D h (z, y) D h (x, z) = grad h(z) P γ,0,1 grad h(y), exp 1 z x z + P γ,0,1 gradh(y), exp 1 z x exp 1 z y P γ,0,1 exp 1 y x z. The aimed result is evident after using the assumption A2.2. Remark 2. In order to verify that h : M IR is a Bregman function, with zone M, it is sufficient to show that conditions a to d in Definition 4.1 are satisfied, as e and f are an immediate consequence of a, b, c and d. 6.1 Convergence Results In this subsection we prove the convergence of the proposed algorithm. Our results are motivated by the works [14, 4, 2] The quasiconvex case Theorem 6.1 Assume Assumption A1 and that f is a quasiconvex and continuous function. Then, the sequence {x k } generated by the PBD algorithm is well defined. Proof. The proof proceeds by induction. It holds for k = 0, due to (6.14). Assume that x k is well defined; from Proposition 5.1, for X = S = M, we have that x k+1 exists and it is unique. Theorem 6.2 Under the same hypothesis above, the sequence {x k }, generated by the PBD algorithm, is bounded. Proof. Since x k satisfies (6.15) we have f(x k ) + λ k D h (x k, x k 1 ) f(x) + λ k D h (x, x k 1 ), x M. (6.19) Hence, x M such that f(x) f(x k ) is true that D h (x k, x k 1 ) D h (x, x k 1 ). Therefore x k is the unique D h projection of x k 1 on the closed totally convex set (see Theorem 3.1) Ω := {x X : f(x) f(x k )}. 11

12 Using Proposition 4.1 and the fact that X Ω we have for every x X. Thus 0 D h (x k, x k 1 ) D h (x, x k 1 ) D h (x, x k ) (6.20) D h (x, x k ) D h (x, x k 1 ). (6.21) This means that {x k } is D h Fejér monotone with respect to set X. We can now apply Definition 4.1,d, to see that x k is bounded, because with α = D h (x, x 0 ). x k Γ 2 (x, α), Proposition 6.1 Under the assumptions of the precedent theorem, the following facts are true a. For all x X the sequence D h (x, x k ) is convergent; b. lim k D h (x k, x k 1 ) = 0; c. {f(x k )} is noincreasing; d. If lim j + x k j = x then, lim j + x k j+1 = x. Proof. a. From (6.21), {D h (x, x k )} is a bounded below nonincreasing sequence and hence convergent. b. Taking limit when k goes to infinity in (6.20) and using the previous result we obtain lim k D h (x k, x k 1 ) = 0, as desired. c. Considering x = x k 1 in (6.19) it follows that 0 D h (x k, x k 1 ) (1/λ k )(f(x k 1 ) f(x k )), (6.22) since D h (x k 1, x k 1 ) = 0. Thus {f(x k )} is noincreasing. d. Taking z k = x k j+1 and y k = x k j in Definition 4.1, f, we obtain the result. Theorem 6.3 Under Assumption A1 and that f is a quasiconvex and continuous function, any limit point of {x k } generated by the PBD algorithm with λ k satisfying (6.17) is an optimal solution of (p). Proof. Let x M be an optimal solution of ( p) and let x M be a cluster point of {x k } then, there exists a subsequence {x k j } such that As x k j This rewrites is a solution of (6.15) we have lim j xk j = x. f(x k j ) + λ kj D h (x k j, x k j 1 ) f(x ) + λ kj D h (x, x k j 1 ). λ kj (D h (x k j, x k j 1 ) D h (x, x k j 1 )) f(x ) f(x k j ). Using the differential characterization of convex functions for D h (., x k j 1 ) gives: λ kj gradd h (x, x k j 1 ), exp 1 x xk j x f(x ) f(x k j ). Taking, above, j and considering the hypothesis (6.17) we obtain f( x) f(x ). Therefore, any limit point is a solution of the problem (p). 12

13 Theorem 6.4 Under Assumption A1 and that f is a quasiconvex and continuous function, the sequence {x k } generated by the PBD algorithm, with λ k satisfying (6.17), converges to an optimal solution of (p). Proof. From Theorem 6.2 {x k } is bounded so there exists a convergent subsequence. Let {x k j } be a subsequence of {x k } such that lim j x k j = x. From Definition 4.1, e, it is true that lim D h(x, x k j ) = 0. j Now, from Theorem 6.3, x is an optimal solution of (p), so from Proposition 6.1, a, D h (x, x k ) is a convergent sequence, with the subsequence converging to 0, hence the overall sequence converges to 0, that is, lim k D h(x, x k ) = 0. To prove that {x k } has a unique limit point let x X be another limit point of {x k }. Then lim l D h (x, x k l) = 0 with lim l x k l = x. So, from Definition 4.1, f, x = x. It follows that {x k } cannot have more than one limit point and therefore, lim k + xk = x X The convex case Theorem 6.5 Suppose that assumptions A1 and A2.1 are satisfied, and that f is convex. If λ k satisfies (6.16) then, any limit point of {x k } is an optimal solution of the problem (p). Proof. Let x M be a limit point of {x k } then, there exists a subsequence {x k j } such that From (6.15) and Theorem 3.4 or, lim j xk j = x. 0 [f(.) + λ kj +1D h (., x k j )](x k j+1 ), λ kj +1gradD h (x k j+1, x k j ) f(x k j+1 ). Let γ kj be the geodesic curve such that γ kj (0) = x k j and γ kj (1) = x k j+1. By Lemma 4.1, i, we obtain λ kj +1[P γkj,0,1grad h(x k j ) grad h(x k j+1 )] f(x k j+1 ). Let x be an optimal solution of (p). Using (3.6) for x = x and y = x k j+1 we have where, On the other hand, from Cauchy inequality f(x ) f(x k j+1 ) y k j, exp 1 x k j +1 x x k j +1 (6.23) y k j := λ kj +1[P γkj,0,1grad h(x k j ) grad h(x k j+1 )]. y k j, exp 1 x k j +1 x x k j +1 y k j x k j +1 exp 1 x k j +1 x x k j

14 We have exp 1 x x k j +1 x k j +1 = d(x, x kj+1 ), also, from Theorem 6.2, there exists M > 0 such that y k j, exp 1 x x k j +1 x k j +1 M y k j x k j +1. Using this fact in the inequality (6.23) we obtain To conclude the proof we will show that f(x ) f(x k j+1 ) M y k j x k j +1. (6.24) Indeed, from the assumption A2.1 lim j yk j x k j +1 = 0. 0 y k j x k j +1 λ kj+1 Ld(x k j+1, x k j ). Now, Taking j + and using the boundedness of {λ k } and Proposition 6.1, d, we obtain lim j y k j x k j = 0, as wanted. Finally, taking j + in (6.24), use the continuity of f to get f(x ) f( x). Therefore, any limit point is an optimal solution of the problem (p). Theorem 6.6 Suppose that assumption A1 and A2.2 are satisfied. If λ k satisfies (6.16) then, any limit point of {x k } is an optimal solution of the problem (p). Proof. Analogously to the previous theorem, given x M and using (6.15), and (3.6) for x and y = x k we have where, 1 λ k y k, exp 1 x k, x x k 1 λ k (f(x k ) f(x)) (6.25) y k := λ k [P γk,0,1grad h(x k 1 ) grad h(x k )], and γ k is the geodesic curve joining γ k (0) = x k 1 to γ k (1) = x k. On the other hand, taking y = x k 1 and z = x k in (6.18) we have D h (x, x k 1 ) D h (x k, x k 1 ) D h (x, x k ) 1 λ k y k, exp 1 x k, x x k. In that inequality, use (6.25) and the boundedness of {λ k }, giving D h (x, x k 1 ) D h (x k, x k 1 ) D h (x, x k ) 1 λ(f(x k ) f(x)) (6.26) Now, Let x M be a limit point of {x k } then, there exists a subsequence {x k j } such that From (6.26) for x = x we have lim j xk j = x. D h (x, x k j 1 ) D h (x k j, x k j 1 ) D h (x, x k j ) 1 λ(f(x k j ) f(x )). Take j and apply Proposition 6.1, a and b, to obtain f(x ) f( x). Therefore, any limit point of {x k } is an optimal solution of (p). 14

15 Theorem 6.7 Under Assumptions A1 and A2 the sequence {x k } generated by the PBD algorithm, with λ k satisfying (6.16), converges to an optimal solution of (p). Proof. Analogous to the proof of Theorem Examples Examples 7.1 to 7.4 are Riemannian manifolds with zero sectional curvature. In example 7.5, it is negative. All those are complete simply connected Riemannian manifolds with nonpositive curvature, the known Hadamard manifolds. They have nice properties as uniqueness of geodesics between any two points, and the diffeomorphic property relating to the Euclidean space. Therefore, totally convex sets can be just denominated convex, and the convexity concept for functions is determined on the (unique) geodesic joining the points. We observe that we have no example of complete noncompact Riemannian manifold of positive sectional curvature (they have also the interesting property of diffeomorphism to IR n, see [22]). Aditionnaly, notice that if the sectional curvature of a Riemannian manifold K(x) δ, x M, for some positive δ, the manifold is compact, and (at least), homeomorphic to the sphere, see [22]. As quasi convex functions are constant on spheres, see [24], the typical sphere examples are out of order. Example 7.1 The Euclidean space is a Hadamard manifold with the metric G(x) = I. Its geodesic curves are the straight lines and the Bregman distances have the form Example 7.2 Let IR n with the metric n D h (x, y) = h(x) h(y) (x i y i ) h(y) y i=1 i G(x) = x 2 n 1 2x n x n 1 1 Thus (IR n, G(x)) is a Hadamard manifold isometric to (IR n, I) through the application φ : IR n IR n defined by Φ(x) = (x 1, x 2,..., x n 1, x 2 n 1 x n), see [6]. The unique geodesic curve, joining the points γ(0) = y and γ(1) = x is γ(t) = (γ 1,..., γ n ), where γ i (t) = y i +t(x i y i ), i = 1,...n 1 and γ n (t) = y n +t((x n y n ) 2(x n 1 y n 1 ) 2 )+2t 2 (x n 1 y n 1 ) 2. Then the Bregman distance is n D h (x, y) = h(x) h(y) (x i y i ) h(y) + 2 h(y) (x n y n ) y i y n i=1 Example 7.3 M = IR n ++ with the Dikin metric X 2 is a Hadamard manifold. Defining π : M IR n such that π(x) = ( ln x 1,..., ln x n ), it can be proved that π is an isometry. It is well known, see for example [20], that the unique geodesic curve joining the points γ(0) = y and γ(1) = x is ( ) γ(t) = x t 1y1 1 t,..., x t nyn 1 t, 15

16 with γ (t) = Then the Bregman distance is: ( ) x t 1y1 1 t (ln x 1 ln y 1 ),..., x t nyn 1 t (ln x n ln y n ). D h (x, y) = h(x) h(y) n i=1 y i ln(x i /y i ) h(y) y i. Example 7.4 Let M = (0, 1) n. We will consider three metrics. 1. (M, X 2 ((I ( X) 2 ) ) is a ( Hadamard manifold; it is isometric to IR n through the function π(x) = ln x1 1 x 1,..., ln xn 1 x n )). The unique geodesic curve, see Section 9, joining the points γ(0) = y and γ(1) = x is γ(t) = (γ 1,..., γ n ) such that γ i (t) = { ( ) ( )} ( )] [(1/2) 2 tanh xi yi yi ln ln t + (1/2) ln, 1 x i 1 y i 1 y i with Then, the Bregman distance is D h (x, y) = h(x) h(y) γ i(t) = ln (x i/(1 x i )) ln (y i /(1 y i )) 4 cosh ((1/2)t + (1/2) ln(y i /(1 y i ))). n i=1 (1 y i ) 2 { ( ) ( )} xi yi h(y) 4yi 2 ln ln. cosh2 (1/2) 1 x i 1 y i y i 2. (M, csc 4 (πx)) is a Hadamard manifold, isometric to IR n, through the function π(x) = 1 π (cot(πx 1),..., cot(πx n )). The unique geodesic curve, see Section 9, joining the points γ(0) = y and γ(1) = x is γ(t) = (γ 1,..., γ n ) such that with and the Bregman distance is γ i (t) = 1 π arg cot[(cot πx i cot πy i )t + cot(πy i )], γ i(t) = (1/π) (cot(πy i ) cot(πx i )) sin 2 (πγ i (t)), D h (x, y) = h(x) h(y) n i=1 1 π (cot(πy i) cot(πx i )) sin 2 (πy i ) h(y) y i. 3. Finally, we consider (M, csc 2 (πx)). It is a Hadamard manifold isometric to IR n, see [17]. The unique geodesic curve joining the points γ(0) = y and γ(1) = x is γ(t) = (γ 1,..., γ n ) such that γ i (t) = ψ 1 (ψ(y i ) + t(ψ(x i ) ψ(y i ))), where So, γ i(t) = (1/π) ln ψ(τ) := ln (csc(πτ) cot(πτ)). ( ) csc(πxi ) cot(πx i ) sin(πγ i (t)). csc(πy i ) cot(πy i ) Therefore, the Bregman distance is D h (x, y) = h(x) h(y) 1 n ( ) csc(πxi ) cot(πx i ) ln sin(πy i ) h(y). π csc(πy i=1 i ) cot(πy i ) y i 16

17 Example 7.5 M = S n ++, the set of the n n positive definite symmetric matrices, with the metric given by the Hessian of ln det(x), is a Hadamard manifold with nonpositive curvature. The geodesic curve joining the points γ(0) = Y and γ(1) = X, see [17], is given by with Then, the Bregman distance is γ(t) = X 1/2 (X 1/2 Y X 1/2 ) t X 1/2, γ (t) = X 1/2 ln(x 1/2 Y X 1/2 )(X 1/2 Y X 1/2 ) t X 1/2. D h (X, Y ) = h(x) h(y ) tr[ h(y )X 1/2 ln(x 1/2 Y X 1/2 )X 1/2 ]. 8 Conclusion and Future Works We generalize the PBD algorithm to solve optimization problems defined on Riemannian manifolds. When the PBD algorithm works with λ k satisfying lim k + λ k = 0, with λ k > 0, the sequence generated by this algorithm converges to an optimal solution of the problem without any assumption on the Bregman functions. On the other hand when λ k satisfies 0 < λ k < λ, we use a class of Bregman function that guarantees the convergence results. We are working to remove that assumption in a general framework of Variational Inequality Problems on Riemannian manifolds. 9 Appendix: Geodesic Equation for a Special Diagonal Metric In this Appendix we give a simplified equation to obtain geodesic curves in closed form for a special class of diagonal Riemannian metric defined in manifolds that are open subset of IR n. This approach recover known metrics that are associated to some interior point algorithms, for example, the Dikin metric and some metrics obtained by the Hessian of separated barrier functions. Those results are extracted from [18]. Let M = M 1 M 2... M n be a manifold in IR n where M i are open subsets of IR, i = 1,..., n. Given x M( and u, v T x M IR n ) we induce in M the metric u, v x = u T G(x)v, where G(x) = diag p 1 (x 1, ) 2 p 2 (x 2,..., ) 2 p n(x n) and p 2 i : M i IR ++ are differentiable functions. In this case we have g ij (x) = δ ij p i (x i )p j (x j ). Considering the Riemannian manifold (M, G(x)) we obtain the following results: 1. Christoffel s Symbols. We use the relation between metrics and Christoffel s symbols, given in (2.5). If k m then g mk = 0, and the expression is reduced to: We consider two cases. First case: i = j Γ m ij = 1 2 Γ m ii = 1 2 { x i g jm + { x i g im + 17 x j g mi x i g mi } g ij g mm. x m } g ii g mm. x m

18 If m = i then otherwise, Second case: i j : If m = i then m j and: If m = j then m i and: If m i and m j then: In both cases we have Γ m ij = 1 2 Γ m ii = 1 p i (x i ), p i (x i ) x i Γ m ii = 0. { g im + } g mi g mm. x i x j Γ i ij = 0. Γ j ij = 0. Γ m ij = 0. Γ m ij = 1 p i (x i ) δ im δ ij. (9.27) p i (x i ) x i 2. Geodesic Equation. Substituting the Christoffel s symbols (9.27) in the equation (2.4) we have that the unique geodesic curve γ starting from x = (x 1, x 2,..., x n ) M, with direction v = (v 1, v 2,..., v n ) T p (M) IR n is obtained by solving the following equation: d 2 γ i dt 2 1 p i (γ i ) p i (γ i ) γ i ( ) 2 dγi = 0, i = 1,..., n (9.28) dt with initial conditions: γ i (0) = x i, i = 1,..., n, γ i (0) = v i, i = 1,..., n. We can see that the above equation has the following equivalent form: d dt ( γ (t) p i (γ i (t)) ) = 0. Then, the geodesic equation (9.28) can be solved through: 1 p i (γ i ) dγ i = a i t + b i i = 1,..., n (9.29) where a i and b i are real constants such that γ i (0) = x i and γ i (0) = v i. With the equation (9.29) we can reproduce some known geodesic curves and obtain new geodesic curves. Next, we present some examples: (a) Consider the Riemannian manifold (IR n ++, X 2 ) with p i (x i ) = x i. Using (9.29) the geodesic curves γ(t) = (γ 1 (t), γ 2 (t),..., γ n (t)) such that γ(0) = x and γ (0) = v are: ( ) vi γ i (t) = x i exp t, i = 1, 2..., n. x i 18

19 (b) Consider the manifold ((0, 1) n, csc 4 (πx)), where p i (x i ) = sin 2 (πx i ). The geodesic curve γ(t) = (γ 1 (t), γ 2 (t),..., γ n (t)) starting in γ(0) = x on the direction γ (0) = v is: γ i (t) = 1 ) ( π π arctan csc 2 (πx i )v i t + cot(πx i ), i = 1, 2..., n. The geodesic curve is well defined for all t IR. Therefore this manifold is complete with null curvature. The Riemannian distance from x = γ(0) to y = γ(t 0 ), t 0 > 0, is given by: d(x, y) = t0 0 { n } 1 γ (t) dt = [cot(πy i ) cot(πx i )] 2 2. i=1 (c) Consider ((0, 1) n, X 2 (I X) 2 ) with p i (x i ) = x i (1 x i ). The geodesic curve γ(t) = (γ 1 (t), γ 2 (t),..., γ n (t)) defined in this manifold, such that γ(0) = x and γ (0) = v is: γ i (t) = 1 { ( 1 v i 1 + tanh 2 2 x i (1 x i ) t + 1 )} 2 ln x i i = 1, 2..., n. 1 x i where tanh(z) = (e z e z )/(e z + e z ) is the hyperbolic tangent function. The geodesic curve is well defined for all t IR. Therefore, this manifold is complete with null curvature. The Riemannian distance from x = γ(0) to y = γ(t 0 ), t 0 > 0, is given by: d(x, y) = t0 0 x (t) dt = { n i=1 [ ( ) ( )] } yi xi ln ln 1 y i 1 x i The geodesic curves and Riemannian distances in (b) and (c) are new to our knowledge. Therefore, new explicit geodesics, metric variables and proximal point algorithms can be defined to solve non convex unconstrained optimization problems on (0, 1) n, whose convergence results are known for general metrics (see [7, 11, 12, 20]). References [1] Burachik, R. S. and Iusem, A. (1998), Generalized Proximal Point algorithm for the Variational Inequality Problem in Hilbert Space, SIAM, Journal of Optimization, 8, [2] Censor, Y. and Zenios, A. (1992), Proximal minimization algorithms with D-functions, Journal of Optimization Theory and Applications, 73, 3, [3] Censor, Y. and Lent, A. (1981), An Iterative Row-Action Method for Interval Convex Programming, Journal of Optimization Theory and applications, 34, 3, [4] Chen, G. and Teboulle, M. (1993), Convergence Analysis of the Proximal-Like Minimization Algorithm Using Bregman Functions, SIAM Journal of Optimization, 3, [5] Cunha, G. F. M., Pinto, A. M., Oliveira, P. R. and da Cruz Neto, J. X. (2005), Generalization of the primal and dual affine scaling algorithms, Technical Report ES 675/05, PESC/COPPE, Federal University of Rio de Janeiro. 19

20 [6] da Cruz Neto, J. X., Ferreira, O. P., Lucambio Perez, L. and Nmeth, S. Z. Convex-and Monotone-Transformable Mathematical Programming and a proximal-like Point Method, to apper in Journal Of Global Optimization. [7] da Cruz Neto, J. X., Lima, L. L. and Oliveira, P. R. (1998), Geodesic Algorithms in Riemannian Geometry, Balkan Journal of Geometry and its Applications, 3, 2, [8] Gabay, D. Minimizing a Differentiable Function over a Differential Manifold (1982) Journal of Optimization Theory and Applications, 37, 2, [9] den Hertog, D., Roos, C. and Terlaky, T. (1991), Inverse Barrier Methods for Linear Programming, Report 91-27, Faculty of Technical Mathematics and Informatics, Delf University of Technology, The Netherlands. [10] do Carmo, M. P. Riemannian Geometry (1992), Bikhausen, Boston. [11] Ferreira, O. P. and Oliveira, P. R. Sub gradient Algorithm on Riemannian Manifolds (1998), Journal of Optimization Theory and Application, 97, 1, [12] Ferreira, O. P. and Oliveira, P. R. (2002), Proximal Point Algorithm on Riemannian Manifolds, Optimization, 51, 2, [13] Garciga, O. and Iusem, A. (2002), Proximal Methods with Penalization Effects in Banach Spaces, IMPA, Technical Report, Serie A-119. [14] Iusem, A. (1995), Métodos de Ponto Proximal em Otimização, Colóquio Brasileiro de Matemática, IMPA, Brazil. [15] Kiwiel, K. C. (1997), Proximal Minimization Methods with Generalized Bregman Functions, SIAM Journal of Control and Optimization, 35, [16] Luenberger, D. G. The Gradient Projection Method Along Geodesics (1972), Management Science, 18, 11, [17] Nesterov, Y. E. and Todd, M. J. (2002), On the Riemannian Geometry Defined by Self- Concordant Barrier and Interior-Point Methods, Foundations of Computational Mathematics, 2, [18] Papa Quiroz, E. A. and Oliveira, P. R. (2004), New Results on Linear Optimization Through Diagonal Metrics and Riemannian Geometry Tools, Technical Report ES-645/04, PESC/COPPE, Federal University of Rio de Janeiro. [19] Pereira, G. J. and Oliveira, P. R. (2005), A new Class of Proximal Algorithms for the Nonlinear Complementary Problems, In: Qi, Liqun; Teo Koklay; Yang, Xiaoqi, Optimization and Control with Applications 1 ed, 69, [20] Rapcsák, T. (1997), Smooth Nonlinear Optimization, Kluwer Academic Publishers. [21] Saigal, R. (1993), The primal power affine scaling method, Tech. Report 93-21, Dep. Ind. and Oper. Eng., University of Michigan. [22] Sakai, T. (1996), Riemannian Geometry, American Mathematical Society, Providence, RI. 20

21 [23] Smith, S. T. (1994), Optimization Techniques on Riemannian Manifolds, Fields Institute Communications, AMS, Providence, RI, 3, [24] Udriste, C. (1994), Convex Function and Optimization Methos on Riemannian Manifolds, Kluwer Academic Publishers. 21