under general perturbations (B., Kassay, Pini, Conditioning for optimization problems under general perturbations, NA, 2012) Dipartimento di Matematica, Università degli Studi di Genova February 28, 2012
Content 1 Introduction to conditioning The case of linear equation in finite dimensional space The case of generalized equation Condition number for scalar optimization problem 2 3 Sensitivity result Distance theorem Strongly convex programming
The case of linear equation in finite dimensional space The case of generalized equation Condition number for scalar optimization problem The goal of the analysis in a parametric (optimization) problem is to study how a change in the data affects the solution of the given problem. Qualitative approach: continuity properties of the solution map (stability analysis); Quantitative approach: Lipschitzian properties of the solution map (sensitivity analysis and condition number theory). Given a parametric (optimization) problem, a condition number essentially gives the following information: upper bound of the ratio of the size of the solution error to the size of the data error (Lipschitz modulus); how the initial problem can be perturbed in order to preserve its good properties.
The case of linear equation in finite dimensional space The case of generalized equation Condition number for scalar optimization problem Let A L(R n, R n ) and consider the equation Ax = p. If A in nonsingular, then the unique solution is x = A 1 p and A 1 (p) A 1 (p ) p p A 1 Theorem (Eckart-Young) Let A L(R n, R n ), nonsingular. Then min { B : A + B singular } = 1 B L(R n,r n ) A 1
The case of linear equation in finite dimensional space The case of generalized equation Condition number for scalar optimization problem Let X and P be Banach spaces and F : X P. The set of solution of the generalized equation p F (x) is given by S F (p) = {x X : x = F 1 (p)} and it is nonempty for p F (X ). i) How does S F (p) behave with respect to perturbations in p? (Lipschitzian property) ii) How does S F (p) behave with respect to perturbations of F? (Distance-type theorem)
The case of linear equation in finite dimensional space The case of generalized equation Condition number for scalar optimization problem (A.L.Dontchev, A.S. Lewis and R.T. Rockafellar, The radius of metric regularity, Trans. AMS, 2002) Fix a point (x, p) gph(f ). Definition (Metric regularity) The map F is said to be metrically regular around (x, p) when there exists k > 0 such that d(x, F 1 (p)) kd(p, F (x)) (1.1) for all (x, p) close to (x, p). The infimum of k such that (1.1) holds, is the regularity modulus of F around (x, p) and it is denoted by reg(f ; x p). For a single-valued linear map A in a finite dimensional space, the regularity modulus is the same for all (x, p) gph(a) and it is given by reg(a) = A 1
The case of linear equation in finite dimensional space The case of generalized equation Condition number for scalar optimization problem B X (x, r) denotes the open ball in X centered at x with radius r and B X (x, r) its closure. Definition (Aubin property) We say that F has the Aubin property around (x, p) when there exists k > 0 and a neighbourhood V of p such that F (x ) V F (x) + k x x B P (0, 1) (1.2) for all x, x close to x. The infimum of k such that (1.2) holds, is the Lipschitz moduls of F around (x, p) and it is denoted by lip(f ; x p).
The case of linear equation in finite dimensional space The case of generalized equation Condition number for scalar optimization problem Remark If F is single-valued around x, the Aubin property is the ordinary Lipschitz continuity of map around x: lip(f ; x) = lip(f ; x F (x)) = F (x) F (x ) lim sup x,x x,x x x x Remark Notice that reg(f ; x p) = lip(f 1 ; p x)
The case of linear equation in finite dimensional space The case of generalized equation Condition number for scalar optimization problem Given a map F such that reg(f ; x p) < +, how far can we perturbe it, with a map G in a suitable class, before loosing metrical regularity of F + G around a suitable point of its graph? Theorem (Estimates for Lipschitz perturbations) Consider a map F : X P and (x, p) gph(f ) at which gph(f ) is locally closed. Let k and µ be non negative constants such that reg(f ; x p) k, kµ < 1. Then for any function g : X P such that lip(g; x) < µ, we have reg(g + F ; x p + g(x) < k 1 kµ
The case of linear equation in finite dimensional space The case of generalized equation Condition number for scalar optimization problem Corollary (Distance-type result) Consider a map F : X P and (x, p) gph(f ) at which gph(f ) is locally closed. Let g : X P a Lipschitz map around x, then inf g {lip(g; x) : F + g is not metrically regular around (x, p + g(x)} 1 reg(f ; x p).
The case of linear equation in finite dimensional space The case of generalized equation Condition number for scalar optimization problem (T. Zolezzi, Condition number theorem in optimization, SIAM J. Optim., 2003) E is a real Banach space and E is its dual, denotes the duality pairing B(x, r) (B (p, r)) is the closed ball in E (E ) with center x (p) and radius r C 1,1 denotes the class of Fréchet differentiable functions such that the Fréchet derivative Df is a Lipschitz function. Given f C 1,1 (B(0, L)), and p E, consider the problem and the solution map min f p = min (f p, ) (O p ) B(0,L) B(0,L) m : B (0, r) E E, m(p) = argmin(b(0, L), f p )
The case of linear equation in finite dimensional space The case of generalized equation Condition number for scalar optimization problem The condition number cond(f ) is defined, under the assumption that the map m is single valued close to 0, as follows: cond(f ) = m(p) m(q) lim sup p,q 0, p q p q If m(0) = 0, i.e. argmin(b(0, L), f ) = {0}, then cond(f ) = lip(m; 0) = reg(m 1 ; 0 0). Furthermore, if m(p) B(0, L) for sufficiently small p, then Df (m(p)) = p and cond(f ) = lip(df 1 ; 0 0)
The case of linear equation in finite dimensional space The case of generalized equation Condition number for scalar optimization problem Class T 1 : functions f C 1,1 (B(0, L)) such that argmin(b(0, L), f ) = {0} argmin(b(0, L), f p ) for small p p argmin(b(0, L), f p ) upper hemicontinuous at p = 0. Class W 1 : functions f : B(0, L) R such that argmin(b(0, L), f p ) singleton for small p cond(f ) < +. Define I 1 = {g T 1 : g / W 1 } (ill conditioned problems).
The case of linear equation in finite dimensional space The case of generalized equation Condition number for scalar optimization problem The class C 1,1 (B(0, L)) is endowed with the pseudodistance { } Df1 (u) Df 2 (u) Df 1 (v) + Df 2 (v) d(f 1, f 2 ) = u v sup u,v Bm(0,L) Theorem (Distance-type theorem) Let f T 1 W 1 be such that Df one to one near 0. Let g T 1 1 be such that d(f, g) < cond(f ). Then g W 1. Corollary Let f T 1 W 1 be such that Df one to one near 0. Then dist(f, I 1 ) 1 cond(f ).
Definition (Condition number) Given f C 1,1 (B(0, L)) such that Df (0) = 0 Df is one to one near 0 we define the condition number as the positive extended real number: u v ĉ(f ) = lim sup u,v 0, u v Df (u) Df (v). Notice that ĉ(f ) 1 L Df > 0 where L Df is the Lipschitz constant of Df on B(0, L).
Assuming that Df is open at 0, i.e. 0 intdf (B(0, s)) s small enough ĉ(f ) is just the lipschitz modulus lip((df ) 1 ; 0) of (Df ) 1 at 0: ĉ(f ) = lim sup p,q 0, p q (Df ) 1 (p) (Df ) 1 (q). p q Notice that ĉ(f ) < + if and only if (Df ) 1 is Lipschitz near 0.
Sensitivity result Distance theorem Strongly convex programming Perturbed optimization problems Given a function f C 1,1 (B(0, L)) and a perturbation function g : E B(0, L) R, we consider for each p E the class of perturbed optimization problems: min F g (p, u) = min f (u) g(p, u) u B(0,L) u B(0,L) (OP(F g, p)) satisfying the assumptions i) g(0, ) = 0; ii) g(p, ) C 1,1 (B(0, L)), for each p E ; iii) sup u B(0,L) Dg(p, u) 0 if p 0, where Dg is the Fréchet derivative of g w.r.t. the second argument.
Sensitivity result Distance theorem Strongly convex programming The class T g Fix a perturbation function g. The class T g contains all the functions f C 1,1 (B(0, L)) such that: Argmin (B(0, L), f ) = {0}; Argmin (B(0, L), F g (p, )) is nonempty for small p; the map p Argmin(B(0, L), F g (p, )) is upper hemicontinuous at p = 0.
Sensitivity result Distance theorem Strongly convex programming The class W g The class W g contains all functions f T g such that: Df is one to one in a neighbourhood of 0; Df satisfies the openness property; ĉ(f ) < + ; there exists 0 < s < L such that Argmin(B(0, L), F g (p, )) B(0, s) is a singleton, for every sufficiently small p. The functions in W g give rise to what will be called a well conditioned optimization problem.
Sensitivity result Distance theorem Strongly convex programming Denote by m(p) the Argmin(B(0, L), F g (p, )) B(0, s) for p small. For a function f W g, the map p m(p) is single valued if we restrict our attention to a neighbourhood of 0 in E, and continuous at p = 0, with m(0) = {0}. The case where the map is locally constant, i.e. m(p) = {0} for all p in a neighborhood of 0, is not interesting; we can get rid of it by assuming that Dg(p, 0) 0 if p 0.
Sensitivity result Distance theorem Strongly convex programming Theorem 1 (Sensitivity) Suppose that the Fréchet derivative Dg : E B(0, L) E of the perturbation function g is Lipschitz, i.e. there exists L > 0 such that Dg(p, u) Dg(q, v) L( p q + u v ). Take f W g such that ĉ(f ) < 1/L. If Dg(p, 0) 0 when p 0, then, for every 0 < ɛ < (1 Lĉ(f ))/L, there exists a neighborhood of 0 E such that the map p m(p) is Lipschitz with constant L m (ĉ(f ) + ɛ)l 1 ĉ(f )L Lɛ.
Sensitivity result Distance theorem Strongly convex programming Theorem 2 (Distance-type theorem) Let f W g, and suppose that there exists γ > 1 such that, for p small enough, L Dg(p, ) ĉ(f ) < 1 γ. (3.1) Denote by h any function in the class T g such that Then h W g. d(f, h) < γ 1 γ 1 ĉ(f ). Notice that condition (3.1) holds for some γ > 1 if the Fréchet derivative Dg : E B(0, L) E of the perturbation function is Lipschitz and ĉ(f ) < 1/L.
Sensitivity result Distance theorem Strongly convex programming Corollary Let f W g, and suppose that there exists γ > 1 such that for p small enough, If h T g \ W g then L Dg(p, ) ĉ(f ) < 1 γ. d(f, h) γ 1 γ 1 ĉ(f ).
Sensitivity result Distance theorem Strongly convex programming Strongly convex functions E is a reflexive Banach space. Definition Let f : C E R be a function on the convex set C. We say that f is strongly convex with convexity modulus α > 0 if f ((1 t)u + tv) (1 t)f (u) + tf (v) α(1 t)t u v 2, u, v C, t [0, 1]. If f is Fréchet differentiable, strong convexity is equivalent to the strong monotonicity of Df on C, i.e. < Df (u) Df (v), u v > 2α u v 2, u, v C.
Sensitivity result Distance theorem Strongly convex programming Notice that strong monotonicity of Df trivially implies that f is one to one near 0; moreover ĉ(f ) u v 2 lim sup u,v 0, u v Df (u) Df (v), u v 1 2α. Proposition 1 Let f be a function in C 1,1 (B(0, L)) such that Df (0) = 0 holds. Suppose that f is strongly convex with convexity modulus equal to α > 0. Then Df in open at 0.
Sensitivity result Distance theorem Strongly convex programming Problem stable under perturbation Let us now consider a perturbation function g : E B(0, L) R satisfying the following conditions: i) g(0, ) = 0; ii) g(p, ) C 1,1 (B(0, L)), and it is a concave function for all small p E ; iii) the mapping (p, u) Dg(p, u) is continuous at (0, 0). These assumptions entail that the parametric optimization problem is stable under perturbation (see Kassay, G. and J. Kolumban, 2000).
Sensitivity result Distance theorem Strongly convex programming Proposition 2 Let f be a function in C 1,1 (B(0, L)) such that Df (0) = 0 holds, and assume that f is strongly convex. Consider a perturbation function g : E B(0, L) R satisfying i) iii). Then the function f belongs to W g.
Sensitivity result Distance theorem Strongly convex programming Main references 1. M. Bianchi, G. Kassay and R. Pini, Conditioning for optimization problems under general perturbations, NA, Vol. 75, pp. 37-45, 2012. 2. A.L.Dontchev, A.S. Lewis and R.T. Rockafellar, The radius of metric regularity, Trans. AMS, 2002. 3. A.L.Dontchev and R.T. Rockafellar, Implicit Functions and Solution Mappings, Springer, 2009. 4. G. Kassay and J. Kolumban, Multivalued parametric variational inequalities with α-pseudomonotone maps, J. Optim. Theory and Appl., Vol. 107, pp.35-50, 2000. 5. T. Zolezzi, Condition number theorems in optimization, SIAM J. OPTIM, Vol. 14, pp. 507-516, 2003.