Tame variational analysis

Transcription

1 Tame variational analysis Dmitriy Drusvyatskiy Mathematics, University of Washington Joint work with Daniilidis (Chile), Ioffe (Technion), and Lewis (Cornell) May 19, 2015

2 Theme: Semi-algebraic geometry is a powerful addition to the Variational Analysis toolkit.

3 Illustration For closed, convex f : R n R, the following are equivalent Quadratic growth: f(x) f( x) + α x x 2 for 2 x near x. Error bound: x x κ dist ( 0, f(x) ) for x near x. 3/16

4 Illustration For closed, convex f : R n R, the following are equivalent Quadratic growth: f(x) f( x) + α x x 2 for 2 x near x. Error bound: x x κ dist ( 0, f(x) ) for x near x. (Proved by Kummer 91, Aragon-Geoffroy 08) 3/16

5 Illustration For closed, convex f : R n R, the following are equivalent Quadratic growth: f(x) f( x) + α x x 2 for 2 x near x. Error bound: x x κ dist ( 0, f(x) ) for x near x. (Proved by Kummer 91, Aragon-Geoffroy 08) Is this true generally? 3/16

6 Illustration For closed, convex f : R n R, the following are equivalent Quadratic growth: f(x) f( x) + α x x 2 for 2 x near x. Error bound: x x κ dist ( 0, f(x) ) for x near x. (Proved by Kummer 91, Aragon-Geoffroy 08) Is this true generally? The implication always holds (D-Mordukhovich-Nghia 14). 3/16

7 Illustration For closed, convex f : R n R, the following are equivalent Quadratic growth: f(x) f( x) + α x x 2 for 2 x near x. Error bound: x x κ dist ( 0, f(x) ) for x near x. (Proved by Kummer 91, Aragon-Geoffroy 08) Is this true generally? The implication always holds (D-Mordukhovich-Nghia 14). But the converse can easily fail. 3/16

8 Illustration For closed, convex f : R n R, the following are equivalent Quadratic growth: f(x) f( x) + α x x 2 for 2 x near x. Error bound: x x κ dist ( 0, f(x) ) for x near x. (Proved by Kummer 91, Aragon-Geoffroy 08) Is this true generally? The implication always holds (D-Mordukhovich-Nghia 14). But the converse can easily fail. Theorem (D-Ioffe) The equivalence holds at local minimizers of semi-algebraic f. 3/16

9 Illustration For closed, convex f : R n R, the following are equivalent Quadratic growth: f(x) f( x) + α x x 2 for 2 x near x. Error bound: x x κ dist ( 0, f(x) ) for x near x. (Proved by Kummer 91, Aragon-Geoffroy 08) Is this true generally? The implication always holds (D-Mordukhovich-Nghia 14). But the converse can easily fail. Theorem (D-Ioffe) The equivalence holds at local minimizers of semi-algebraic f. (More in Ioffe s talk tomorrow.) 3/16

10 Outline Basics of semi-algebraic geometry Consequences for Variational Analysis: Subgradient descent Sweeping process Sard theorem Size of subdifferential graphs Approximation on singular domains 4/16

11 Semi-algebraic geometry Semi-algebraic set: finite union of sets { } p x : i (x) < 0 for i I p j (x) = 0 for j J where p i, p j are polynomials. 5/16

12 Semi-algebraic geometry Semi-algebraic set: finite union of sets { } p x : i (x) < 0 for i I p j (x) = 0 for j J where p i, p j are polynomials. A mapping F : R n R m is semi-algebraic if gph F = {(x, y) : y F (x)} is semi-algebraic. 5/16

13 Semi-algebraic geometry Semi-algebraic set: finite union of sets { } p x : i (x) < 0 for i I p j (x) = 0 for j J where p i, p j are polynomials. A mapping F : R n R m is semi-algebraic if gph F = {(x, y) : y F (x)} is semi-algebraic. Robustness: Boolean operations preserve semi-algebraic sets. Linear mappings preserve semi-algebraic sets (Tarski-Seidenberg). 5/16

14 Semi-algebraic geometry Semi-algebraic set: finite union of sets { } p x : i (x) < 0 for i I p j (x) = 0 for j J where p i, p j are polynomials. A mapping F : R n R m is semi-algebraic if gph F = {(x, y) : y F (x)} is semi-algebraic. Robustness: Boolean operations preserve semi-algebraic sets. Linear mappings preserve semi-algebraic sets (Tarski-Seidenberg). Eg: Q semi-algebraic = {x : y (x, y) Q} semi-algebraic. 5/16

15 Semi-algebraic geometry Semi-algebraic set: finite union of sets { } p x : i (x) < 0 for i I p j (x) = 0 for j J where p i, p j are polynomials. A mapping F : R n R m is semi-algebraic if gph F = {(x, y) : y F (x)} is semi-algebraic. Robustness: Boolean operations preserve semi-algebraic sets. Linear mappings preserve semi-algebraic sets (Tarski-Seidenberg). Eg: Q semi-algebraic = {x : y (x, y) Q} semi-algebraic. Conclusion: f, f, sur F, Lip F,... remain semi-algebraic. 5/16

16 Basic properties Curve selection: Given x cl Q, there is an analytic curve γ with γ(0) = x and γ(0, η) Q. (Bruhat-Cartan 50, Milnor 68) γ Q x 6/16

17 Basic properties Curve selection: Given x cl Q, there is an analytic curve γ with γ(0) = x and γ(0, η) Q. (Bruhat-Cartan 50, Milnor 68) γ Semi-algebraic selection: Semialgebraic F : R n R m admit semi-algebraic selections f F. Q x 6/16

18 Basic properties Curve selection: Given x cl Q, there is an analytic curve γ with γ(0) = x and γ(0, η) Q. (Bruhat-Cartan 50, Milnor 68) γ Semi-algebraic selection: Semialgebraic F : R n R m admit semi-algebraic selections f F. Stratification: Semialgebraic sets stratify into finitely many manifolds {M i }, and so have dimension. (Whitney 65, Łojasiewicz 71) Q x 6/16

19 Basic properties Curve selection: Given x cl Q, there is an analytic curve γ with γ(0) = x and γ(0, η) Q. (Bruhat-Cartan 50, Milnor 68) γ Semi-algebraic selection: Semialgebraic F : R n R m admit semi-algebraic selections f F. Stratification: Semialgebraic sets stratify into finitely many manifolds {M i }, and so have dimension. (Whitney 65, Łojasiewicz 71) Q x Łojasiewicz inequality: If f is semi-algebraic, then on compacta dist(x; f 1 (0)) C f(x) α. (Łojasiewicz 91, Kurdyka 98, Bolte-Daniilidis-Lewis 06) 6/16

20 What are consequences for Variational Analysis? 7/16

21 Subgradient systems Theorem (D-Ioffe-Lewis) f semi-algebraic, x not a local minimizer = there exists a nontrivial solution to ẋ f(x) and x(0) = x. 8/16

22 Subgradient systems Theorem (D-Ioffe-Lewis) f semi-algebraic, x not a local minimizer = there exists a nontrivial solution to Moreover, then ẋ f(x) and x(0) = x. ẋ is the shortest element of f(x) a.e. 8/16

23 Subgradient systems Theorem (D-Ioffe-Lewis) f semi-algebraic, x not a local minimizer = there exists a nontrivial solution to Moreover, then ẋ f(x) and x(0) = x. ẋ is the shortest element of f(x) a.e. Theorem (Daniilidis-Bolte-Lewis) If f is semi-algebraic, then any bounded subgradient curve has finite length. 8/16

24 Subgradient systems Theorem (D-Ioffe-Lewis) f semi-algebraic, x not a local minimizer = there exists a nontrivial solution to Moreover, then ẋ f(x) and x(0) = x. ẋ is the shortest element of f(x) a.e. Theorem (Daniilidis-Bolte-Lewis) If f is semi-algebraic, then any bounded subgradient curve has finite length. Many analogues for descent methods; e.g. proximal point, splitting, Gauss-Seidel, etc (Attouch, Bolte, Bot, Noll, Peypouquet, Soubeyran, Svaiter,... ) 8/16

25 Sweeping process Sweeping process (Moreau 77): ẋ(t) N S(t) (x(t)) with S(t) a moving set. S(t) N S(t) (x(t)) x(t) 9/16

26 Sweeping process Sweeping process (Moreau 77): ẋ(t) N S(t) (x(t)) with S(t) a moving set. The monotone case S(t) = [f t] is subgradient descent. S(t) N S(t) (x(t)) x(t) 9/16

27 Sweeping process Sweeping process (Moreau 77): ẋ(t) N S(t) (x(t)) with S(t) a moving set. The monotone case S(t) = [f t] is subgradient descent. S(t) N S(t) (x(t)) x(t) Theorem (D-Daniilidis) If S : R R n is semi-algebraic, then every bounded solution of the sweeping process has finite length. 9/16

28 Sweeping process Sweeping process (Moreau 77): ẋ(t) N S(t) (x(t)) with S(t) a moving set. The monotone case S(t) = [f t] is subgradient descent. S(t) N S(t) (x(t)) x(t) Theorem (D-Daniilidis) If S : R R n is semi-algebraic, then every bounded solution of the sweeping process has finite length. Key estimate: ẋ(t) Lip S ( t x(t) ) sup Lip S ( t x ) x S(t) X and the upper-bound is integrable by the Łojasiewicz inequality. 9/16

29 Sard Theorem Mapping F : R n R m is metrically regular at ( x, ȳ) gph F if dist(x, F 1 (y)) dist(y, F (x)) is bounded near ( x, ȳ). If F is not metrically regular at ( x, ȳ), then ȳ is a critical value. 10/16

30 Sard Theorem Mapping F : R n R m is metrically regular at ( x, ȳ) gph F if dist(x, F 1 (y)) dist(y, F (x)) is bounded near ( x, ȳ). If F is not metrically regular at ( x, ȳ), then ȳ is a critical value. Theorem (Ioffe 08) Semi-algebraic F : R n R m have almost no critical values. Justifies typical linear convergence of basic schemes, e.g. alternating projections (D-Ioffe-Lewis 13). A x 0 B 10/16

31 Sard Theorem Mapping F : R n R m is metrically regular at ( x, ȳ) gph F if dist(x, F 1 (y)) dist(y, F (x)) is bounded near ( x, ȳ). If F is not metrically regular at ( x, ȳ), then ȳ is a critical value. Theorem (Ioffe 08) Semi-algebraic F : R n R m have almost no critical values. Justifies typical linear convergence of basic schemes, e.g. alternating projections (D-Ioffe-Lewis 13). A x 0 B Sard Theorem & gph f is thin = generic properties of semi-algebraic functions. (cf. Lewis talk) 10/16

32 Generic properties Consider min x f(x) + h(g(x) + y) v, x where f, h, G are semi-algebraic and G is C 2 -smooth. 11/16

33 Generic properties Consider min x f(x) + h(g(x) + y) v, x where f, h, G are semi-algebraic and G is C 2 -smooth. Optimality conditions: [ ] v y [ ] G(x) λ ( + f ( h) 1) (x, y). G(x) 11/16

34 Generic properties Consider min x f(x) + h(g(x) + y) v, x where f, h, G are semi-algebraic and G is C 2 -smooth. Optimality conditions: [ ] v y [ ] G(x) λ ( + f ( h) 1) (x, y). G(x) Sard theorem & thinness = generic properties: qualification conditions, strict complementarity, smooth dependance of (x, λ), existence of identifiable manifolds, second order sufficient conditions are necessary at local minimizers. 11/16

35 Generic properties Consider min x f(x) + h(g(x) + y) v, x where f, h, G are semi-algebraic and G is C 2 -smooth. Optimality conditions: [ ] v y [ ] G(x) λ ( + f ( h) 1) (x, y). G(x) Sard theorem & thinness = generic properties: qualification conditions, strict complementarity, smooth dependance of (x, λ), existence of identifiable manifolds, second order sufficient conditions are necessary at local minimizers. Remark: Without semi-algebraicity, one needs geometric measure theory and not all properties above are generic. 11/16

36 Approximation of functions Set-up: Q R n f R. Q Assume Q is a disjoint union of manifolds Q = M 1 M 2... M k 1 M k 12/16

37 Approximation of functions Motivation: Integration by parts g f dx Q 13/16

38 Approximation of functions Motivation: Integration by parts g f dx = g f, ˆn ds Q Q Q g, f dx. 13/16

39 Approximation of functions Motivation: Integration by parts g f dx = g f, ˆn ds Q Q Q g, f dx. Goal: approximate f by a C 2 smooth f so that the f ˆn. 13/16

40 Approximation of functions Motivation: Integration by parts g f dx = g f, ˆn ds Q Q Q g, f dx. Goal: approximate f by a C 2 smooth f so that the f ˆn. Theorem (D-Larsson) Given a continuous f : R n R and any ɛ > 0, there exists a C 1 -smooth f satisfying 1. Closeness: f(x) f(x) < ɛ for all x R n, 2. Neumann Boundary condition: x M i = f(x) T x M i. 13/16

41 Approximation of functions Motivation: Integration by parts g f dx = g f, ˆn ds Q Q Q g, f dx. Goal: approximate f by a C 2 smooth f so that the f ˆn. Theorem (D-Larsson) Given a continuous f : R n R and any ɛ > 0, there exists a C 1 -smooth f satisfying 1. Closeness: f(x) f(x) < ɛ for all x R n, 2. Neumann Boundary condition: x M i = f(x) T x M i. provided {M i } is a Whitney stratification of Q. 13/16

42 Approximation of functions Motivation: Integration by parts g f dx = g f, ˆn ds Q Q Q g, f dx. Goal: approximate f by a C 2 smooth f so that the f ˆn. Theorem (D-Larsson) Given a continuous f : R n R and any ɛ > 0, there exists a C 1 -smooth f satisfying 1. Closeness: f(x) f(x) < ɛ for all x R n, 2. Neumann Boundary condition: x M i = f(x) T x M i. provided {M i } is a Whitney stratification of Q. For semi-algebraic Q, Whitney stratifications always exist! 13/16

43 Conclusion Semi-algebraic geometry is a powerful addition to the Variational Analysis toolkit. Applications: quadratic growth and error bounds, subgradient descent and the sweeping process, Sard theorem, and approximation on singular domains. 14/16

44 Thank you. 15/16

45 References Generic minimizing behavior in semi-algebraic optimization, D-Ioffe-Lewis, preprint arxiv: , Trajectory length of the tame sweeping process, D-Daniilidis, preprint arxiv: , Quadratic growth and critical point stability of semi-algebraic functions, D-Ioffe. To appear in Math. Program., Clarke subgradients for directionally Lipschitzian stratifiable functions, Math. Oper. Res., 40(2), , Approximating functions on stratified sets, D-Larsson, Trans. Amer. Math. Soc., 367, , Curves of descent, D-Ioffe-Lewis, SIAM J. Control and Optim., 53(1), , Semi-algebraic functions have small subdifferentials, D-Lewis, Math. Program., 140(1): 5-29, Available at ddrusv 16/16