Hybrid Steepest Descent Method for Variational Inequality Problem over Fixed Point Sets of Certain Quasi-Nonexpansive Mappings

Transcription

1 Hybrid Steepest Descent Method for Variational Inequality Problem over Fixed Point Sets of Certain Quasi-Nonexpansive Mappings Isao Yamada Tokyo Institute of Technology Wellington, Feb. 13, 2004

2 This talk is based on a joint work with N. Ogura (Tokyo Institute of Technology). 1

3 We are trying to solve: in Real Hilbert Sp H Variational Inequality Problem over F ix(t ) For given T : H H and Θ : H R (Convex func.), Find u F ix(t ) := {x H T (x) = x} closed convex s.t. u u, Θ (u ) 0, u F ix(t ). For T : Convex Projection Gradient Projection Method (Goldstein 64/Levitin&Polyak 66) We propose Hybrid Steepest Descent Method T : H H Nonexpansive Mapping (Yamada et al 96 / Deutsch & Yamada 98 / Yamada 01) Appl: Convexly Constrained Inverse Problems T : H H Quasi-Nonexpansive(Yamada&Ogura 03) Appl: Optimization of Fixed Point of Subgradient Projector 2

4 Part 1 Background / Preliminaries Original Idea of Gradient Projection Method 3

5 P K (X) X K P K (Y) Y Convex Projection: Basic Properties P K (x) P K (y) x y, x, y H F ix(p K ) := {x H P K (x)= x}= K K must be simple to compute P K. 4

6 Gradient Projection Method (1964 ) u n+1 := P K (un λ n+1 Θ (un) ), n = 0, 1, 2,... under certain conditions converges (strongly / weakly) to a solution to Smooth Convex Optimization Problem (P1) Minimize Subject to Θ : H R G-differentiable convex func. x K ( H) closed convex set NOTE: where H : Real Hilbert Space u K is a solution of (P1) u K satisfies u u, Θ (u ) 0, u K. 5

7 Part 2 Hybrid Steepest Descent Method From Projection to Nonexpansive Mapping / Quasi-Nonexpansive Mapping 6

8 T : H H is called κ-lipschitzian if κ > 0 s.t. T (x) T (y) κ x y for all x, y H. If κ = 1 T : H H is Nonexpansive mapping. F ix (T ) := {x H T (x) = x} is closed convex. Generalization κ < 1 κ < 1 or κ = 1 broadens Fixed Point Theory significantly. Many choices of T s.t. F ix(t ) = K, e.g., F ix m i=1 w i T i = m i=1 F ix(t i ) if m i=1 F ix(t i ). 7

9 Is It Possible to Extend from Gradient Projection Method v n+1 := P K ( vn λ n+1 Θ (vn) ) to v n+1 := T ( vn λ n+1 Θ (vn) ) where T : H H: Nonexpansive Mapping for Minimizing Θ over F ix(t )? 8

10 To Answer to the Question, we introduce Hybrid Steepest Descent Method (Yamada et al, 1996 ) u n+1 := T (un) λ n+1 Θ (T (un)) where T :H H: Nonexpansive Mapping This is because vn := T (un) is generated by v n+1 := T ( vn λ n+1 Θ (vn) ) and If s- n lim un = u F ix(t ) s- n lim vn = u F ix(t ) 9

11 In short, Hybrid Steepest Descent Method (Yamada2001): u n+1 := T (u n ) λ n+1 Θ (T (u n )) can minimize Θ over F ix(t ), where T : H H: nonexpansive, and (λ n ) n=1 R+ : slowly decreasing. 10

12 Sequence Generation by Hybrid Steepest Descent Method Θ Π H T(H) Fix(T) un+1 T(un) -λn+1θ (T(un)) 11

13 Hybrid Steepest Descent Method(Yamada 2001) Suppose that (a) T : H H: Nonexp. mapping, (b) Θ : H R:Convex function, (c) Θ : Lipschitzian & Strongly monotone over T (H), (d) (λ n ) n 1 [0, ) satisfies (i) lim λ n n = 0, (ii) λ n =, (iii) λ n λ n+1 <. n 1 n 1 u n+1 := T (u n ) λ n+1 Θ (T (u n )) satisfies s- lim n u n = u arg inf x F ix(t ) Θ(x). (Unique) 12

14 If we specially choose Θ(x) := 1 2 x a 2 in the Hybrid Steepest Descent Method, Halpern ( 67), P.L.Lions ( 77), Wittmann ( 92) u n+1 := λ n+1 a + (1 λ n+1 )T (u n ), converges strongly to P F ix(t ) (a), where T : H H: nonexpansive, and (λ n ) n=1 R+ : slowly decreasing. More general cyclic versions were given by P.L. Lions (1977) and H.H. Bauschke (1996) 13

15 Generalization of Θ Θ : Lipschitzian & Paramonotone (Ogura,Yamada 2002) Robust Hybrid Steepest Descent Method (Yamada, Ogura, Shirakawa 2002) u n+1 := T (n) (u n ) λ n+1 Θ ( T (n) (u n ) ) where T (n) := (1 t n+1 )I + t n+1 T is gifted with notable numerical robustness. For detail, see Contemporary Mathematics 313 (2002) 14

16 Let Convexly Constrained Generalized Inverse Problem K H: a closed convex set, Ψ : H R: the 1st convex function, satisfying Then the problem is K Ψ := arg inf x K Find a point x arg Ψ(x). inf Θ(x) =: Γ( ), x K Ψ where Θ : H R is the 2nd convex function. 15

17 Suppose that Ψ : H H (G-derivative) is γ-lipschitzian. Apply Hybrid Steepest Descent Method u n+1 := T (u n ) λ n+1 Θ (T (u n )), [ T := PK (I νψ ), ν (0, 2/γ] ] Solves the Problem, i.e., n lim d(u n, Γ) = 0. NOTE: Projected Landweber Iteration (Eicke 1992): v n+1 := P K ( λn+1 A b + β n (I λ n+1 A A)v n ) is the simplest realization for Θ(x) := 1 2 x 2 and Ψ(x) := 1 2 A(x) b 2 o (A : H H o : bdd linear). 16

18 Part 3 Hybrid Steepest Descent Method From Nonexpansive Mapping to Quasi-Nonexpansive Mapping 17

19 Quasi-Nonexpansive Mapping T : H H is called Quasi-Nonexpansive if T (x) T (f) x f, (x, f) H F ix(t ). In this case, F ix (T ) := {x H T (x) = x} is closed convex set. Quasi-nonexpansive mapping T is not necessarily continuous. 18

20 Convex Projection Firmly Nonexpansive Attracting Nonexpansive Nonexpansive Quasi-Nonexpansive Next Example shows The level set of continuous convex function can be expressed as Fixed Point Set of Simple Quasi-Nonexpansive Mapping. 19

21 Example (Subgradient Projection T sp(φ) ) Subgradient Projection for Cont. convex function Φ T sp(φ) : x x Φ(x) g(x) 2 g(x) if Φ(x) > 0 x if Φ(x) 0, where g(x) Φ(x) : subgradient of Φ at x H. See for example (Bauschke & Combettes 01) T sp(φ) : ( 1 2-averaged) quasi-nonexpansive, F ix(t sp(φ) ) = {x H Φ(x) 0} =: lev 0 Φ 20

22 Subgradient Projection : Approximation of Convex Projection x lev < = 0 Φ Tsp( ) Φ (x) F ix ( T sp(φ) ) = lev 0 (Φ) 21

23 Is It Possible to Extend from u n+1 := T (un) λ n+1 Θ (T (un)) where T : H H: Nonexpansive to u n+1 := T (un) λ n+1 Θ (T (un)) where T : H H: Quasi-Nonexpansive for Minimizing Θ over F ix(t )? 22

24 Quasi-shrinking (Yamada & Ogura 03) Let T : H H : quasi-nonexpansive with F ix(t ) C for C( H): closed convex set. T : H H is called quasi-shrinking on C if D : r [0, ) inf u (F ix(t ),r) C d(u, F ix(t )) d(t (u), F ix(t )) if (F ix(t ), r) C otherwise satisfies D(r) = 0 r = 0. where (F ix(t ), r) := {x H d(x, F ix(t )) r}. 23

25 Hybrid Steepest Descent Method (Quasi-Nonexpansive) Suppose that (a)t : H H: Quasi-Nonexpansive, (b)θ : κ-lipschitzian& η-strongly monotone over T (H), (c) f F ix(t ), s.t. T is quasi-shrinking on C f (u 0 ) := where µ (0, 2η κ 2), x H x f max With (λ n ) n 1 [0, 1] s.t. (i) u 0 f, 1 µf(f) 1 µ(2η µκ 2 ) lim λ n n = 0, (ii) λ n =, n 1 u n+1 := T (u n ) λ n+1 µθ (T (u n )) satisfies s- lim n u n = u arg inf x F ix(t ) Θ(x) (Unique) 24

26 Proposition Suppose Φ : H R (cont. convex function) satisfies lev 0 Φ and Φ bounded. Define T α := (1 α)i + αt sp(φ) (α (0, 2)). Then (a) If dim(h) <, T α : quasi-shrinking on any bdd closed convex C satisfying C lev 0 Φ. (b) If Φ Φ: Uniformly monotone over H, T α : quasi-shrinking on any bdd closed convex C satisfying C lev 0 Φ. 25

27 Hybrid Steepest Descent Method (for T sp(φ) ) Suppose that (a)φ : H R:Cont. Convex,lev 0 Φ & Φ: bdd, Let T α := (1 α)i + αt sp(φ) (α (0, 2)). (b)θ : κ-lipschitzian& η-strongly monotone over T α (H), With (λ n ) n 1 [0, ) s.t. (i) When dim(h) < lim λ n n = 0, (ii) λ n =, n 1 u n+1 := T α (u n ) λ n+1 Θ (T α (u n )) satisfies lim n u n = u arg inf x lev 0 Φ Θ(x) (Unique) 26

28 Hybrid Steepest Descent Method (for T sp(φ) over K) Suppose that (a)φ : H R: Cont. Convex with Φ: bdd, (b) K: bdd closed convex set s.t. lev 0 Φ K, (c)θ : Lipschitzian& Paramonotone over K, When dim(h) < With (λ n ) n 1 [0, ) s.t. (i) lim λ n n = 0, (ii) λ n =, n 1 u n+1 := P K T α (u n ) λ n+1 Θ (P K T α (u n )) satisfies lim n d(u n, Γ) = 0, where Γ := arg inf K lev 0 Φ Θ(x). 27

29 For related results to this talk, See for example : 28

30 Hybrid Steepest Descent Method and Its Applications 1. I. Yamada: The hybrid steepest descent method for the variational inequality problem over the intersection of fixed point sets of nonexpansive mappings, pp , in Inherently Parallel Algorithm for Feasibility and Optimization and Their Applications, Elsevier I. Yamada, N. Ogura and N. Shirakawa: A numerically robust hybrid steepest descent method for the convexly constrained generalized inverse problems, pp , in Inverse Problems, Image Analysis, and Medical Imaging, Contemporary Mathematics, 313, Amer. Math. Soc.,

31 3. K. Slavakis, I. Yamada and K. Sakaniwa: Computation of symmetric positive definite Toeplitz matrices by the Hybrid Steepest Descent Method, Signal Processing, vol.83, pp , H.K. Xu and T.H. Kim: Convergence of hybrid steepest descent methods for variational inequalities, Journal of Optimization Theory and Applications, vol. 119, no. 1, pp , I. Yamada and N. Ogura: Two Generalizations of the Projected Gradient Method for Convexly Constrained Inverse Problems Hybrid steepest descent method, Adaptive projected subgradient method, Proceedings of NANIT 03, RIMS, Kyoto, Dec.,

32 Thank you very much!! 31

33 What is Subgradient? Subgradient of Φ at x Let Φ : H R : Cont. Convex Function. Φ(x) := {s H : y x, s + Φ(x) Φ(y), y H }. s Θ(x) is called Subgradient of Φ at x. 0 Φ(x) Φ(x) = min y H Φ(y). Φ(x) = { Φ(x)} if Φ:G-differentiable at x. generalization of Gradient. 32

34 Subgradient: a generalization of Gradient Gradient Subgradient 33