Variational inequalities for set-valued vector fields on Riemannian manifolds Chong LI Department of Mathematics Zhejiang University Joint with Jen-Chih YAO Chong LI (Zhejiang University) VI on RM 1 / 28
Outline 1 Problems on manifolds 2 Existence and uniqueness results Existence of solutions Uniqueness of solutions 3 Convexity of solution sets 4 Proximal point methods 5 Application to convex optimization Chong LI (Zhejiang University) VI on RM 1 / 28
Problems on manifolds 1 Problems on manifolds 2 Existence and uniqueness results 3 Convexity of solution sets 4 Proximal point methods 5 Application to convex optimization Chong LI (Zhejiang University) VI on RM 2 / 28
Problems on manifolds Various problems posed on manifolds arise in many natural contexts: eigenvalue problems invariant subspace computations constrained minimization problems boundary value problems [A. Edelman, T. A. Arias and T. Smith, SIAM J. Matrix Anal. Appl., 1998], geometric model for human spine Adler et al, IMA J. Numer. Anal., 2002) Chong LI (Zhejiang University) VI on RM 3 / 28
Problems on manifolds Numerical methods extended to Riemannian manifolds: Newton s method on Riemannian manifolds [O.P. Ferreira, B.F. Svaiter, J. Complexity 2002] [J.P. Dedieu, P. Priouret, G. Malajovich, IMA J. Numer. Anal. 2003] [C. Li, J.H.Wang, Sci. China Ser. A 2005] [J.H. Wang, C. Li, J. Complexity, 2006] [C. Li and J. H. Wang, IMA J. Numer. Anal. 2006] [C. Li, J.H.Wang, J Complexity, 2008] [Alvarez et al, Found. Comput. Math. 2008] Chong LI (Zhejiang University) VI on RM 4 / 28
Problems on manifolds Newton s method on Lie groups [ R. E. Mahony, Linear Algebra Appl,1996] [B. Owren, B. Welfert, BIT Numer. Math, 2000] [C. Li, J. H.Wang, J. P. Dedieu, J. Complex.,2009] [J.H.Wang, C. Li, IMA J Numer. Anal., 2011] Conjugate gradient method [S. Smith, Fields Institute Communications, AMS, 1994] Trust-region method [P. A. Absil, C. G. Baker, K. A. Gallivan, Found. Comput. Math, 2007] Chong LI (Zhejiang University) VI on RM 5 / 28
Problems on manifolds Notions extended to the Riemannian manifolds: subdifferential calculus [Azagra et al. J. Funct. Anal., 2005] [Y. S. Ledyaev and Q. J. Zhu, Trans. Amer. Math. Soc., 2007] monotonicity [S. Nmeth, Nonlinear Anal. 2003] [C. Li, G. López and V. Martín-Máquez, J. London Math. Soc. 2009, Set-valued Var. Anal. 2011] [Wang, J.H., López, G., Martín-Máquez, V., Li, C., J. Optim. Theory Appl. 2010] [C. Li G. López V. Martín-Máquez J. H. Wang, Set-Valued Anal., 2011] weak sharp minima [C. Li, B. Mordukhovich, J. H. Wang and J. C. Yao, SIAM J. Optim. 2011] Chong LI (Zhejiang University) VI on RM 6 / 28
Problems on manifolds Variational inequality Variational inequalities on R n are powerful tools of constrained optimization problems equilibrium problems complementary problems [P. T. Harker and J. S. Pang, Math. Programming, 2003] Variational inequality problems on Hadamard manifolds: [S. Z. Németh, 2003] established some basic results on existence and uniqueness of the solution. An open problem: how to extend the existence and uniqueness results to Riemannian manifolds. Answer: [S. L. Li, C. Li, Y. C. Liou and J. C. Yao, Nonlinear Anal. 2009)] Chong LI (Zhejiang University) VI on RM 7 / 28
Problems on manifolds Proximal point algorithm The well-known proximal point algorithms are among the most powerful methods for solving variational inequalities and other classes of optimization-related problems. [R. T. Rockafellar, SIAM J. Control Optim. 1976] [O. Güler, SIAM J. Control Optim. 1991] [H. Bauschke and P. L. Combettes, Springer 2011] Proximal point algorithms were extended to the setting of Hadamard manifolds for optimization problems [O. P. Ferreira and P. R. Oliveira, Optim. 2002] variational inequality problems [C. Li, G. López and V. Martín-Máquez, J. London Math. Soc. 2009] Chong LI (Zhejiang University) VI on RM 8 / 28
Problems on manifolds Notations M the finite-dimensional Riemannian manifold T x M the tangent space at x to M c : [0, 1] M the piecewise smooth curve joining x to y l(c) := 1 0 ċ(t) dt the arc-length of c d(x, y) := inf c l(c) Riemannian distance between x and y exp x : T x M M the exponential map at x P γ,, the parallel transport on the tangent bundle TM B Z (p, r) the open metric ball at p with radius r B Z (p, r) the closed metric ball at p with radius r Γ Ā x,y the set of all geodesics γ joining x to y satisfying γ A. geodesic: a curve c : [0, 1] M joining x to y satisfying c(0) = x, c(1) = y and ċċ = 0 on [0, 1]. Chong LI (Zhejiang University) VI on RM 9 / 28
Problems on manifolds Definition Let A be a nonempty subset of M. The set A is said to be (a) weakly convex if, for any p, q A, there is a minimal geodesic of M joining p to q lying in A; (b) strongly convex if, for any p, q A, there is just one minimal geodesic of M joining p to q and it is in A; (c) locally convex if, for any p A, there is a positive ε > 0 such that A B(p, ε) is strongly convex. The strong convexity = the weak convexity = the local convexity. All convexities in a Hadamard manifold coincide. Definition A point o A is called a weak pole of A if for each x A, the minimal geodesic of M joining o to x is unique and lies in A. Chong LI (Zhejiang University) VI on RM 10 / 28
Existence and uniqueness results 1 Problems on manifolds 2 Existence and uniqueness results 3 Convexity of solution sets 4 Proximal point methods 5 Application to convex optimization Chong LI (Zhejiang University) VI on RM 11 / 28
Existence and uniqueness results Existence of solutions The variational inequality problem on Riemannian manifold M: Find x A such that (VI1) v V ( x) satisfying v, γ xy (0) 0, y A, γ xy Γ Ā x,y. The solution set of (VI1) is denoted by S(V, A). Chong LI (Zhejiang University) VI on RM 12 / 28
Existence and uniqueness results Existence of solutions Definition Let A M be a nonempty subset and V be a set-valued vector field on A. Let x 0 A. V is said to be (a) upper semicontinuous at x 0 if for any open set W satisfying V (x 0 ) W T x0 M, there exists an open neighborhood U(x 0 ) of x 0 such that P x0,xv (x) W for any x U(x 0 ). (b) upper Kuratowski semicontinuous at x 0 if for any sequences {x k } A and {u k } TM with each u k V (x k ), relations lim k x k = x 0 and lim k u k = u 0 imply u 0 V (x 0 ). The upper semicontinuity implies the upper Kuratowski semicontinuity. The converse is also true if A is compact and V is compact-valued. Chong LI (Zhejiang University) VI on RM 13 / 28
Existence and uniqueness results Existence of solutions V(A): the set of all upper Kuratowski semicontinuous set-valued vector fields V satisfying that V (x) is compact and convex for each x A. Theorem (S. L. Li, C. Li, Y. C. Liou and J. C. Yao, Nonlinear Anal.2009) Let A M be a compact, locally convex subset of M, and let V be a continuous vector field on A. Suppose that A has a weak pole o int R A. Then S(V, A), that is (VI1) admits at least one solution x. Theorem (Existence for compact A) Let A M be a compact, and locally convex subset of M. Suppose that A has a weak pole o int R A and that V V(A). Then the variational inequality problem (VI1) admits at least one solution. Corollary (Extend the result in S. Z. Németh, 2003 ) Suppose that M is a Hadamard manifold and let A M be a compact convex set. Let V V(A). Then the variational inequality problem (VI1) admits at least one solution. Chong LI (Zhejiang University) VI on RM 14 / 28
Existence and uniqueness results Uniqueness of solutions Definition Let A M be a nonempty weakly convex set and let V be a set-valued vector field on A. The vector field V is said to be (1) monotone on A if for any x, y A and γ xy Γ A x,y the following inequality holds: v x, γ xy (0) v y, γ xy (1) 0, v x V (x), v y V (y); (2) strictly monotone on A if for any x, y A and γ xy Γ A x,y the following inequality holds: v x, γ xy (0) v y, γ xy (1) < 0, v x V (x), v y V (y); (3) strongly monotone on A if there exists ρ > 0 such that for any x, y A and γ xy Γ A x,y the following inequality holds: v x, γ xy (0) v y, γ xy (1) ρ l 2 (γ xy ), v x V (x), v y V (y). Chong LI (Zhejiang University) VI on RM 15 / 28
Existence and uniqueness results Uniqueness of solutions Theorem (Uniqueness) Let A 0 A M. Suppose that A 0 is weakly convex and V is strictly monotone on A 0 satisfying S(V, A) A 0. Then the variational inequality problem (VI1) admits at most one solution. In particular, if A is weakly convex and V is strictly monotone on A, then the variational inequality problem (VI1) admits at most one solution. Chong LI (Zhejiang University) VI on RM 16 / 28
Convexity of solution sets 1 Problems on manifolds 2 Existence and uniqueness results 3 Convexity of solution sets 4 Proximal point methods 5 Application to convex optimization Chong LI (Zhejiang University) VI on RM 17 / 28
Convexity of solution sets Definition Let A be a nonempty subset of M and let r (0, + ). The set A is said to be (a) weakly r-convex if, for any p, q K with d(p, q) < r, there is a minimal geodesic of M joining p to q lying in A; (b) r-convex if, for any p, q A with d(p, q) < r, there is just one minimal geodesic of M joining p to q and it is in A. Chong LI (Zhejiang University) VI on RM 18 / 28
Convexity of solution sets Theorem (D κ -convexity) Suppose that M is of the sectional sectional curvature bounded above by κ and that A is closed and D κ -convex. Then the solution set S(V, A) is D κ -convex. A Hadamard manifold is of the sectional curvature bounded above by κ = 0 and D κ = +. Corollary Suppose that M is a Hadamard manifold and that A is convex. Then the solution set S(V, A) is convex. Theorem (Locally convexity) Suppose that M is a complete Riemannian manifold and A is locally convex. Then the solution set S(V, A) is locally convex. Chong LI (Zhejiang University) VI on RM 19 / 28
PPM 1 Problems on manifolds 2 Existence and uniqueness results 3 Convexity of solution sets 4 Proximal point methods 5 Application to convex optimization Chong LI (Zhejiang University) VI on RM 20 / 28
PPM Proximal point method Recall the variational inequality problem on Riemannian manifold M: Find x A such that (VI1) v V ( x) satisfying v, γ xy (0) 0, y A, γ xy Γ Ā x,y. Proximal point method: (PPM) x n+1 J λn (x n ), where J λ (z) := S(V λ,z, A), V λ,z (x) = λv (x) E z (x). Chong LI (Zhejiang University) VI on RM 21 / 28
PPM Theorem (Local convergence of PPM) Suppose that M is of the sectional curvature bounded above by κ and that A is weakly convex. Let V V(A) be a monotone vector field satisfying S(V, A). Let x 0 A be such that d(x 0, S(V, A)) < Dκ 8 and algorithm (PPM) is well-defined. Suppose that {λ n } (0, + ) satisfies n=0 λ 2 n = and λ n V (x n ) D κ 4 for each n = 0, 1,. Then the sequence {x n } converges to a solution of the variational inequality problem (VI1). Chong LI (Zhejiang University) VI on RM 22 / 28
PPM Corollary (Global convergence of PPM on Hadamard manifold) Suppose that M is a Hadamard manifold and that A is convex. Let V V(A) be a monotone vector field with S(V, A). Let x 0 A and suppose that {λ n } (0, + ) satisfies λ 2 n =. n=0 (1) Then algorithm (PPM) is well-defined and the generated sequence {x n } converges to a solution of the variational inequality problem (VI1). Chong LI (Zhejiang University) VI on RM 23 / 28
Application to convex optimization 1 Problems on manifolds 2 Existence and uniqueness results 3 Convexity of solution sets 4 Proximal point methods 5 Application to convex optimization Chong LI (Zhejiang University) VI on RM 24 / 28
Application to convex optimization Definition Let f : M R be a proper function with weakly convex domain D(f ). The function f is said to be (a) weakly convex if, for any x, y D(f ), there is a minimal geodesic γ xy Γ f x,y such that the composition f γ xy is convex on [0, 1]; (b) convex if, for any x, y D(f ) and any geodesic γ xy Γ f x,y, the composition f γ xy is convex on [0, 1]. Chong LI (Zhejiang University) VI on RM 25 / 28
Application to convex optimization Optimization problem Consider the optimization problem (P) min f (x). x A The solution set of problem (P) is denoted by S f (A). Assumption: (AP) A, f, f + δ A are weakly convex and A intd(f ). Chong LI (Zhejiang University) VI on RM 26 / 28
Then the algorithm (PPM) is well-defined and the sequence {x n } converges to a solution of the minimization problem (P). Chong LI (Zhejiang University) VI on RM 27 / 28 Application to convex optimization PPM for optimization problem (P): { (PPM) x n+1 argmin f (x) + 1 } d(x n, x) 2 : x A. 2λ n The well-definedness of (PPM) follows from assumption (AP). Theorem (Local convergence of PPM for OP) Let A be a weakly convex subset of M of the sectional curvature bounded above by κ. In addition to assumption (AP), we further suppose that f : M R is a convex function such that S(f, A). Let x 0 A be such that d(x 0, S(f, A)) < Dκ 8 and let {x n} be a sequence generated by algorithm (PPM). Suppose that {λ n } (0, + ) satisfy n=0 λ n = and λ n f (x n ) < D κ 4 for each n = 0, 1,.
Application to convex optimization Thank You for Your Attention. Chong LI (Zhejiang University) VI on RM 28 / 28