An inertial forward-backward method for solving vector optimization problems

Transcription

1 An inertial forward-backward method for solving vector optimization problems Sorin-Mihai Grad Chemnitz University of Technology gsor research supported by the DFG project GR 3367/4-1 joint work with Radu Ioan Boţ Splitting Algorithms, Modern Operator Theory, and Applications Workshop Oaxaca, September 19, 2017

2 Outline Preliminaries An inertial forward-backward proximal method Alternative constructions and hypotheses Numerical experiments 2 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

3 Preliminaries X - Hilbert space, Y - separable Banach space C Y pointed (i.e. C ( C) = {0}) closed convex cone (λc C λ 0) partial ordering C on Y C - greatest element with respect to C, C / Y, denote Y = Y { C } we write x C y if x C y and x y C = {y Y : y, y 0 y C} - dual cone to C A vector function F : X Y is proper: dom F = {x X : F (x) Y } C-convex: F (tx + (1 t)y) C tf (x) + (1 t)f (y) for all x, y Y and all t [0, 1] positively C-lsc: z, F ( ) is lower semicontinuous for all z C \ {0} 3 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

4 Solution concepts Consider the vector optimization problem (P) WMin F (x), x X where F : X Y proper and int C. An x X is a weakly efficient solution to (P): ( x int C) F (X) = notation: x WE(P) an efficient solution to (P): x X s.t. F (x) C F ( x) notation: x E(P) Proposition. F is C-convex x WE(P) z C \ {0} s.t. z, F ( x) z, F (x) x X. 4 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

5 Example Consider the vector optimization( problem ) x 2 (P) WMin 1 x 2, x 1,x 2 R x 2 where the vector-minimization is considered with respect to R 2 +. For all λ = (λ 1, λ 2 ) R 2 + \ {0} with λ 1 λ 2, one has inf{λ 1 (x 2 1 x 2 ) + λ 2 x 2 : x 1, x 2 R} = one cannot identify the weakly efficient solutions of (P) Only for λ = ( λ 1, λ 2 ) R 2 + \ {0} with λ 1 = λ 2 > 0 one gets λ 1 ( x 2 1 x 2 )+ λ 2 x 2 λ 1 (x 2 1 x 2 )+ λ 2 x 2 x 1, x 2 R for x 1 = 0, x 2 R. an unfortunate choice of the scalarization may lead nowhere (even for simple vector optimization problems) 5 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

6 Iterative methods for solving multiobjective/vector optimization problems algorithms for delivering one (weakly) efficient solution Newton: Fliege, Graña Drummond & Svaiter; Graña Drummond, Raupp & Svaiter steepest descent: Fliege & Svaiter; Graña Drummond & Svaiter projected gradient: Fukuda & Graña Drummond; Graña Drummond & Iusem proximal point: Bonel, Iusem & Svaiter; Villacorta & Oliveira algorithms for finding (approximating) the whole (weakly) efficient set Benson type: Löhne & Weißing adaptive scalarization: Berger & Büskens 6 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

7 Problem formulation Consider the vector optimization problem [ ] (VP) WMin F (x) + G(x), x X where F : X Y - Fréchet differentiable with an L-Lipschitz-continuous gradient G : X Y - proper int C 7 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

8 Inertial forward-backward proximal algorithm choose x 0, x 1 X 1 9 > β β n 0 n 0: (β n ) n is nondecreasing (z n) n C \ {0}: z n = 1 e n int C n 0: z n, e n = 1 n 0 1 let n = 1 2 if x n WE(VP): STOP { 3 find x n+1 WE G(x) + L 2 x ( x n + β n (x n x n 1 ) 1 L (z nf)(x n ) ) 2 e n : x Ω n }, where Ω n = {x X : (F + G)(x) C (F + G)(x n )} 4 let n := n + 1 and go to 2 8 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

9 Remarks F 0 & β n = 0 n 0 proximal point method [Bonel, Iusem & Svaiter] Y = R & C = R + inertial forward-backward proximal point method [Moudafi & Oliny] Y = R & C = R + & F 0 inertial proximal point method [Alvarez & Attouch] Y = R & C = R + & F 0 & β n = 0 n 0 ISTA [Beck & Teboulle] it is not necessary to impose the existence of a weakly efficient solution to (P) any z C \ {0} provides a good scalarization function for the vector optimization problems in Step 3 under additional hypotheses (e.g. δ > 0 : {z Y : z, x δ x z x C} ) the method delivers efficient solutions to (VP) 9 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

10 Convergence statement If F and G are C-convex G is positively C-lsc (F + G)(X) (F (x 0 ) + G(x 0 ) C) is C-complete (i.e. (a n ) n X with a 0 = x 0 s.t. (F + G)(a n+1 ) C (F + G)(a n ) n 0 a X: (F + G)(a) C (F + G)(a n ) n 0) then x n x WE(VP). 10 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

11 Proof steps the strong convexity of the scalarized intermediate problems guarantees the existence of new iterates descent lemma for (z F ) (convex and Fréchet differentiable with an L z -Lipschitz continuous gradient z C ) existence of a weak cluster point of (x n ) n that is weakly efficient to (VP) Opial s Lemma guarantees the weak convergence of (x n ) n to a point of {x X : F (x) C F (x n ) n 0} uniqueness of the weak cluster point of (x n ) n main challenges the necessity of using constrained intermediate vector optimization problems the considered function changes at each step we deal with two vector functions with different properties 11 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

12 Alternative hypotheses/stopping rule/inexact version imposing the condition (cf. [Alvares & Attouch]) + k=1 β k x k x k 1 2 < + (β n ) n needs not be nondecreasing and β [0, 1[ considering instead of 2 if x n WE(VP): STOP the following stopping rule 2 if x n+1 = x n = x n 1 : STOP or replacing 3 with 3 find x n+1 X such that 0 εn ( zn, G( ) + L 2 x n β n (x n x n 1 ) + 1 L (z n F )(x n ) 2 e n + δ Ωn ( ))(x n+1 ) with (ε n ) n fulfilling e.g. n 1 ε n < +, the converge statement remains valid 12 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

13 Forward-backward proximal algorithm choose x 0 X, (z n) n C \ {0}: z n = 1 e n int C n 0: z n, e n = 1 n 0 If 1 let n = 1 2 if x n WE(VP): STOP 3 find { x n+1 WE G(x)+ L 2 x ( x n 1 L (z nf)(x n ) ) 2 } e n : x Ω n 4 let n := n + 1 and go to 2 F and G are C-convex G is positively C-lsc (F + G)(X) (F (x 0 ) + G(x 0 ) C) is C-complete z n = z C \ {0} n 1 then for any n 0 and x Ω = n 0 Ω n one has z, F (x n ) + G(x n ) F ( x) G( x) L x x n 13 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

14 Second inertial forward-backward proximal algorithm choose x 0, x 1 X 0 β n β R n 0: (β n ) n is nondecreasing (z n) n C \ {0}: z n = 1 e n int C n 0: z n, e n = 1 n 0 If 1 let n = 1 2 if x n WE(VP): { STOP 3 find x n+1 WE G(x) + L 2 x ( x n + β n (x n x n 1 ) 1 L (z nf)(x n + β n (x n x n 1 )) ) 2 } e n : x Ω n 4 let n := n + 1 and go to 2 β < 1 9 F and G are C-convex G is positively C-lsc (F + G)(X) (F (x 0 ) + G(x 0 ) C) is C-complete then x n x WE(VP). 14 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

15 Convergence rate If F and G are C-convex G is positively C-lsc (F + G)(X) (F (x 0 ) + G(x 0 ) C) is C-complete z n = z C \ {0} n 1 β n = tn 1 t n, where t 1 = 1, t n = tn 2 2 n 0 then for any n 0 and x Ω one has z, F (x n ) + G(x n ) F ( x) G( x) 2 L x x 0 2 (n + 1) 2. Open problems derive convergence rates without taking (zn) n constant alternative hypotheses to the C-completeness avoid using Ω n without losing the convergence 15 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

16 Numerical experiments The portfolio vector optimization problem ( x (EP) WMin u x=(x 1,...,x d ) R d + d, x Vx x i =1 i=1 where u R d and V R d d is symmetric positive semidefinite, can be recast as a special case of (VP) by taking X = R d, Y = R 2, C = R 2 +, F (x) = ( x u, x Vx) and G(x) = (δ R d + T (x), δ R d + T (x)), where T = {x = (x 1,..., x d ) R d + : d i=1 x i = 1}. x - portfolio vector for d given assets short sales are excluded: x R d + expected return: x u variance of the portfolio: x Vx ), 16 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems

17 Matlab implementation real data collected in [Duan, 2007] stocks: IBM, Microsoft, Apple, Quest Diagnostics, and Bank of America, between 02/01/ /01/2007 d = 5, u = (0.4, 0.513, 4.085, 1.006, 1.236) and V = z n = (1/ 2, 1/ 2), e n = (1, 1) n 0 x 0 = (0.25, 0.25, 0, 0.25, 0.25), x 1 = (0.15, 0.25, 0.25, 0.2, 0.15) stopping rule: x n+1 x n ε = x n x n 1 the intermediate scalar problems solved with fmincon (interior point methods) approximate weakly efficient solution x = ( , , , , ) 17 Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems.

18 β n iterations time (s) / / / / / / / /10 1/10n /10 1/(n + 10) /15 1/n /20 1/(n + 20) /30 1/(n + 30) /50 1/(n + 50) Sorin-Mihai Grad An inertial forward-backward proximal point method for solving vector optimization problems