Variational inequalities for fixed point problems of quasi-nonexpansive operators 1. Rafał Zalas 2

Transcription

1 University of Zielona Góra Faculty of Mathematics, Computer Science and Econometrics Summary of the Ph.D. thesis Variational inequalities for fixed point problems of quasi-nonexpansive operators 1 by Rafał Zalas 2 Supervisor: Prof. Dr. Andrzej Cegielski Auxiliary supervisor: Dr. Robert Dylewski Zielona Góra 2014 In this summary we present the most important results selected from the Ph.D. thesis. Introduction Let H be a real Hilbert space equipped with an inner product, and the corresponding norm. We consider the following variational inequality problem: Given a monotone operator F : H H and a closed convex subset C H, find ū C such that F ū, z ū 0 for all z C. (1) Variational inequality, denoted by VI(F, C), plays an important role in many optimization problems. This is due to the fact that the variational inequality is a necessary and sufficient optimality condition for the minimization of a convex differentiable function f : H R over C with F (x) := Df(x) being the Gâteaux derivative of f at x. Despite this simple motivation there are many practical reasons which justify studying solution methods for VIs. Some of them can be found in the monograph of D. Kinderlehrer and G. Stampacchia [16] and in the monograph of F. Facchinei and J. S. Pang [13]. Variational inequality is usually difficult to solve directly. One possibility is to apply an iterative method which should approximate an unknown solution of VI. The standard method for VI(F, C) is the gradient projection (GP) method x k+1 := P C (x k µf x k ), (2) 1 Polish title: Nierówności wariacyjne dla problemów punktów stałych operatorów quasi-nieoddalających. 2 Author was financially supported by the Polish National Science Centre within the framework of Etiuda funding scheme under agreement No. DEC-2013/08/T/ST1/

2 where x 0 H is arbitrary and P C : H H denotes the metric projection onto C, i.e., P C x := argmin z C x z. One can prove that if F is η-strongly monotone and κ-lipschitz continuous, where κ, η > 0, then the solution of VI(F, C) is uniquely determined. Moreover, sequences {x k } k=0 defined by (2) converge in norm to ū C being the solution of VI(F, C), whenever µ (0, 2η κ ), 2 see, e.g. [20]. The GP method can be efficiently applied if the metric projection P C can be easily computed. Otherwise, the method can be essentially affected because P C should be computed in each iteration. Moreover, an estimation of µ has to be known a priori. To overcome these obstacles one can replace the metric projection P C by a sequence of nonexpansive, or at least quasi-nonexpansive operators T k with C Fix T k and the constant µ > 0 by a sequence λ k with λ k 0. This, by some additional reformulation of the GP method, leads to the iteration u k+1 = T k u k λ k F T k u k, (3) where u 0 H is arbitrary. Iterative method described by (3) was introduced by Deutsch and Yamada [12] and was called the hybrid steepest descent method (HSD method). Special cases of HSD method (with F := Id a) were studied by Halpern [14], Lions [17], Wittmann [18], Bauschke [2] and also (for more general F ) by Yamada [20], Xu and Kim [19], Yamada and Ogura [21], Hirstoaga [15], Cegielski and Zalas [5, 7], Aoyama and Kohsaka [1]. The aim of this Ph.D. thesis is to present a comprehensive convergence analysis of the hybrid steepest descent method described in (3) when F is Lipschitz continuous and strongly monotone, C H is nonempty, closed and convex, and T k are strongly quasi-nonexpansive, k 0. Chapter 1. Basic theorems and definitions For the convenience of the reader we recall basic definitions presented in Chapter 1 which are used in this summary. Let F : H H. We say that F is monotone if F x F y, x y 0 for all x, y H. We say that F is η-strongly monotone if F x F y, x y η x y 2 for all x, y H, where η > 0. We say that F is κ-lipschitz continuous if F x F y κ x y for all x, y H, where κ > 0. Let U : H H be an operator having a fixed point, i.e. Fix U. We say that U is quasinonexpansive (QNE) if Ux z x z for all x H and all z Fix U. We say that U is ρ-strongly quasi-nonexpansive (ρ-sqne), where ρ > 0, if Ux z 2 x z 2 ρ Ux x 2 for all x H and all z Fix U. We say that U is a cutter if z Ux, x Ux 0 for all x H and all z Fix U. One can show that U is a cutter if and only if U is 1-SQNE. If U is QNE, then Fix U is closed and convex. We say that an operator U : H H is demi-closed at 0 if for any sequence {x k } k=0 H converging weakly to x H and such that Ux k 0 we have Ux = 0. We say that U : H H is nonexpansive (NE) if U is 1-Lipschitz continuous. A NE operator having a fixed point is QNE. Moreover, if U is nonexpansive then U Id is demi-closed at 0. A comprehensive study of the presented operators can be found in [4]. Let C i H, i I := {1, 2,..., m}, be closed convex subsets with C := i I C i. Denote C := {C i i I}. We say that the family C is boundedly regular (BR) [3] if for any bounded 2

3 sequence {x k } k=0 H the following implication holds max k i I d(xk, C i ) = 0 = k d(xk, C) = 0. Chapter 2. Approximately shrinking operators In Chapter 2 we have introduced a subclass of quasi-nonexpansive operators called the class of approximately shrinking operators. This subclass has been used as a tool for the convergence analysis in Chapter 4, where we have formulated one of the main results of this Ph.D. thesis. The results presented in this chapter are based on [7]. AS operators were introduced by A. Cegielski and R. Zalas in [5]. Below we present a definition of an AS operator and the most important results of Chapter 2. Definition 1 (Definicja 2.1). We say that a quasi-nonexpansive operator U : H H is approximately shrinking (AS) if for any bounded sequence {x k } k=0 H the following implication holds k Uxk x k = 0 = k d(xk, Fix U) = 0. The underlying meaning of the approximately shrinking property is the following: the displacement Ux k x k can be small only in the neighborhood of Fix U. A simple example of an AS operator is the metric projection. It was shown (Twierdzenie 2.6) that if U is AS, then U Id is demi-closed at zero. The converse implication is also true for a QNE operator U : R n R n for which U Id is closed at zero. Consequently, in the finite dimensional case the subgradient projection is AS (Wniosek 2.7). Moreover, if U : R n R n is NE and Fix U, then U is AS (Wniosek 2.8). Additionally, it was shown (Twierdzenie 2.16) that AS operators are more general than quasi-shrinking ones [21]. Theorem 2 (Twierdzenie 2.12 and Twierdzenie 2.14). Let U i : H H be ρ i -SQNE, ρ i > 0, i I := {1,..., m}, with i I Fix U i. Let U := i I ω iu i, where ω i > 0, i I, i I ω i = 1, or U := U m... U 1. If U i, i I, are approximately shrinking, then for any bounded sequence {x k } k=0 H the following implication holds k Uxk x k = 0 = max U ix k x k = 0. (4) k i I If, additionally, C := {Fix U i i I} is boundedly regular, then U is approximately shrinking. Chapter 3. Hybrid steepest descent method Below we present the main result of Chapter 3. Theorem 3 (Twierdzenie 3.16). Let F : H H be Lipschitz continuous and strongly monotone and let C H be nonempty, closed and convex. Let T k : H H be ρ k -SQNE satisfying C Fix T k, k 0. Let {u k } k=0 be generated by the HSD method (3), i.e., u 0 H, u k+1 = T k u k λ k F T k u k, where {λ k } k=0 [0, ). Assume that inf k ρ k > 0 and that for some s 1 and for all {n k } k=0 the following implication holds {k} k=0 s 1 k l=0 T nk lu nk l u nk l = 0 = 3 k d(un k, C) = 0. (5)

4 Then k d(uk, C) = 0 whenever k λ k = 0. If, additionally, k=0 λ k =, then where ū C is a unique solution of VI(F, C). k uk ū = 0, Examples of methods satisfying implication (5) are presented in Chapter 4. Conditions similar to (5) were proposed by Yamada and Ogura [21], Hirstoaga [15] and by Aoyama and Kohsaka [1]. Chapter 4. Iterative methods for solving variational inequalities Let F : H H be monotone and let C := i I Fix U i, where U i : H H are QNE for all i I := {1,..., M}. We call U i input operators. In this chapter we want to apply the HSD method to VI(F,C). In order to do it, we have to define operators T k and relate them to input operators. In Chapter 4 we have shown many examples of such constructions. Most of them are special cases of the extended string averaging procedure, which we define below. For technical reasons, let us define an additional input operator U 0 := Id. Let N (N 1) be the number of steps. Let ε (0, 1). Procedure 4 (Procedura 4.5). Extended string averaging (ESA). Step 1. (Initialization) Set n := 1. Define auxiliary operators V j := U j for all j I {0}. Step 2. Assume that operators V M,..., V n 1 are defined. In order to define V n, choose one of the following cases: a) Fix a subset J n = {j n } { M,..., 0}, a relaxation parameter α n [ε, 2 ε] and set V n := Id +α n (V jn Id). (6) b) Fix a nonempty subset J n { M,..., n 1} and weights ω j,n [ε, 1 ε] satisfying j J n ω j,n = 1 and set V n := j J n ω j,n V j. (7) c) Fix a finite sequence (string) J n { M,..., n 1} with length less than M + n and set V n := j J n V j. (8) Step 3. If n < N, then set n := n + 1 and go to step 2. Otherwise the procedure is finished. The underlying idea presented in the extended string averaging procedure is to combine repeatedly convex combinations and compositions of input operators. The structure of the final operator V N can be represented graphically, see Figure 1. By the procedure described above we can define many important operators used in the iterative methods for solving VIs. Especially, if we restrict the construction to a convex combination 4

5 U 1 U 2 U M U 1 U 2 U M U 1 U 2 U 3 U 4 U 5 U 6 U 7 U 8 Figure 1: Graphical representation of structures related to convex combination: M ωiui, composition: U M... U 1 and combination of both (string averaging): ω 1U 3U 2U 1 + ω 2U 6U 5U 4 + ω 3U 8U i=1 7. (average) of compositions of the input operators along sequences (strings) J n, then we obtain a standard string averaging procedure. By using ESA procedure combined with HSD method and selected properties of AS operators we obtain the main result of this Ph.D. thesis. For all k 0 let V k N k : H H be an operator generated by ESA procedure with N k steps. Formally, by the definition of ESA procedure, we can present V k N k only by using relaxations, convex combinations and compositions of several input operators, not necessarily all. By I k, let us denote the subset of indices of those input operators which were really applied in the construction of V k N k. We have the following result. Theorem 5 (Twierdzenie 4.12). For all k 0 define T k : H H by T k := V k N k. Let {u k } k=0 be a sequence generated by the HSD method, i.e. u 0 H, u k+1 := T k u k λ k F T k u k, where {λ k } k=0 [0, ). Assume that the following conditions are satisfied: (i) F is strongly monotone and Lipschitz continuous, (ii) U i are approximatelly shrinking cutters, i I, (iii) {N k } k=0 is bounded, (iv) for some s 1 and for all k s 1 we have I s 1 l=0 I k l, (v) the family C := {Fix U i i I} is boundedly regular. If k λ k = 0, then k d(u k, C) = 0. If, additionally, k=0 λ k =, then {u k } k=0 converges to ū C, being the unique solution of VI(F, C). Theorem 5 extends the results presented in [5] and [7]. It follows from Theorem 5 that the convergence of the HSD method does not depend on the underlying structure of the operators T k. This structure may vary within the iterations. The same holds for the relaxation parameters and for the convex combination parameters. We only have 5

6 to guarantee that the size N k of the structure is uniformly bounded from above (condition (iii)). We do not have to use all input operators at once. We can restrict ourselves only to the chosen subset I k. The additional input operator U 0 = Id may not be used at all. But in each consecutive s iterations we have to use any of the input operators U 1,... U M at least once (condition iv). Special cases of Theorem 5 are HSD methods with a cyclic control (T k := U ik, i k := (k mod M) + 1), parallel control (T k := i I k ω k i U i) and sequential control (T k := i I k U i ). Such schemes with I k = I, constant weights ω k i and with no relaxations were studied by Bauschke [2], Yamada [20], Xu and Kim [19] and Hirstoaga [15] with NE input operators U i. Theorem 5 allows us also to combine the HSD method with an almost cyclic control (T k := U ik, I = {i k s+1,..., i k } for all k 0), which is more general than the cyclic one. The standard string averaging procedure was studied by Y. Censor et al. for solving the convex feasibility problems [8], the common fixed point problems [9] and recently for the constrained minimization problems [11]. The idea of extending the standard string averaging was mentioned in [10]. It is also possible to combine the HSD method with controls other than covered by the ESA procedure such as remotest set control or maximal displacement control, i.e. T k = U ik with i k Argmax i I d(u k, Fix U i ) or i k Argmax i I U i u k u k, respectively. Concluding remarks The HSD method is a general iterative scheme which depends on how we refer to the constraint subset C defining the variational inequality. By combining the HSD method with selected properties of AS operators and the ESA procedure, due to Theorem 5, we can adapt techniques commonly used for solving the convex feasibility problems and the common fixed point problems to VIs and particularly to the constrained minimization problems. References [1] K. Aoyama, F. Kohsaka, Viscosity approximation process for a sequence of quasinonexpansive mappings, Fixed Point Theory and Applications, 2014:17 (2014). [2] H. H. Bauschke, The approximation of fixed points of composition of nonexpansive mappings in Hilbert space, J. Math. Anal. Appl., 202(1996), [3] H. H. Bauschke, J. Borwein, On projection algorithms for solving convex feasibility problems, SIAM Review, 38 (1996), [4] A. Cegielski, Iterative Methods for Fixed Point Problems in Hilbert Spaces, Springer, Heidelberg, (2012). [5] A. Cegielski, R. Zalas, Methods for variational inequality problem over the intersection of fixed point sets of quasi-nonexpansive operators, Numer. Funct. Anal. Optim., 34 (2013), [6] A. Cegielski, A. Gibali, S. Reich, R. Zalas, An algorithm for solving the variational inequality problem over the fixed point set of a quasi-nonexpansive operator in Euclidean space, Numer. Funct. Anal. Optim., 34 (2013), [7] A. Cegielski and R. Zalas, Properties of a class of approximately shrinking operators and their applications, Fixed Point Theory, 15 (2014), in print. 6

7 [8] Y. Censor, T. Elfving, G. T. Herman, Averaging strings of sequential iterations for convex feasibility problems, w: Inherently Parallel Algorithms in Feasibility and Optimization and their Applications, Editors: D. Butnariu, Y. Censor, S. Reich, Elsevier, Amsterdam, (2001), [9] Y. Censor and A. Segal, On the string averaging method for sparse common fixed point problems, Int. Trans. Oper. Res., 6 (2009), [10] Y. Censor, E. Tom, Convergence of string-averaging projection schemes for inconsistent convex feasibility problems, Optimization Methods and Software, 18 (2003), [11] Y. Censor, A. J. Zaslavski, String-averaging projected subgradient methods for constrained minimization, Optimization Methods and Software, 29 (2014), [12] F. Deutsch, I. Yamada, Minimizing certain convex functions over the intersection of the fixed point sets of nonexpansive mappings, Numer. Funct. Anal. Optim., 19 (1998), [13] F. Facchinei, J. S. Pang, Finite-Dimensional Variational Inequalities and Complementarity Problems, Volume I, Volume II, Springer, New York, (2003). [14] B. Halpern, Fixed points of nonexpanding maps, Bull. Amer. Math. Soc., 73 (1967), [15] S. A. Hirstoaga, Iterative selection methods for common fixed point problems, J. Math. Anal. Appl., 324 (2006), [16] D. Kinderlehrer, G. Stampacchia, An Introduction to Variational Inequalities and Their Applications, Academic Press, New York, (1980). [17] P. L. Lions, Approximation de points fixes de contractions, C. R. Acad. Sci. Paris Sér. A, 284 (1977), [18] R. Wittmann, Approximation of fixed points of nonexpansive mappings, Arch. Math., 58 (1992), [19] H. K. Xu, T. H. Kim, Convergence of hybrid steepest-descent methods for variational inequalities, Journal of Optimization Ttheory and Aplications, 119 (2003), [20] I. Yamada, The hybrid steepest descent method for the variational inequality problem over the intersection of fixed point sets of nonexpansive mappings, w: Inherently Parallel Algorithms in Feasibility and Optimization and their Applications, Editors: D. Butnariu, Y. Censor, S. Reich, Elsevier, Amsterdam, (2001), [21] I. Yamada, N. Ogura, Hybrid steepest descent method for variational inequality problem over the fixed point set of certain quasi-nonexpansive mapping, Numer. Funct. Anal. Optim., 25 (2004),