Analysis of the convergence rate for the cyclic projection algorithm applied to semi-algebraic convex sets

Transcription

1 Analysis of the convergence rate for the cyclic projection algorithm applied to semi-algebraic convex sets Liangjin Yao University of Newcastle June 3rd, 2013 The University of Newcastle Joint work with Jon Borwein and Guoyin Li Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

2 Outline 1 Motivation 2 Notation and auxiliary results Notation 3 References Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

3 Motivation Given finitely many closed convex sets C 1, C 2,, C m in R n with m i=1 C i. Let x 0 R n. The sequence of cyclic projections, (x k ) k N, is defined by x 1 := P 1 x 0, x 2 := P 2 x 1,, x m := P m x m 1, x m+1 := P 1 x m..., where P i denotes the Euclidean projection to the set C i. Bregman showed that In this talk, we study m x k x C i. i=1 m how fast (x k ) k N converges a point in C i. i=1 Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

4 Notation and definitions Throughout this talk, R n is a Euclidean space with the norm and the inner product,. Let C R n. The interior of C is int C and C is the norm closure of C. B(x 0, δ) := { x R n x x 0 δ }. [α] + := max{α, 0}. The distance function to the set C is dist(x, C) := inf c C x c. The projector operator to the set C is P C (x) := {c C y c = dist(x, C)}. Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

5 Notation and definitions distc(x0) = x0 PC(x0) x0 C PC(x0) Figure : The projection of the point x 0 to the set C. β(n) := ( n [n/2]) is the central binomial coefficient with respect to n. Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

6 Notation and definitions f : R n R is a polynomial if f (x) := λ α x α, 0 α r where λ α R, x = (x 1,, x n ), x α := x α 1 1 x n αn, α i N {0}, and α := n j=1 α j. The corresponding constant r is called the degree of f. A polynomial f is constant on [x, y] f is constant on aff [x, y]. We say that C is a semi-algebraic convex set if C = {x R n g i (x) 0, i = 1,, N}, where g i is convex polynomial for every i = 1,..., N. Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

7 Notation and definitions From now on we assume that m N, γ i N, i = 1,..., m g i,1, g i,2,, g i,γi are convex polynomials on R n { } C i := x R n g i,1 (x) 0, g i,2 (x) 0,, g i,γi (x) 0 P i := P Ci m C := C i. i=1 Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

8 Auxiliary results Fact 1. (Kollár, 1999) Let g i be polynomials on R n with degree d for every i = 1,, m. Let g(x) := max 1 i m g i (x). Suppose that there exists ε 0 > 0 such that g(x) > 0 for all x B(0, ε 0 )\{0}. Then there exist constants c, ε > 0 such that 1 x c g(x) β(n 1)d n, x ε. Fact 2. (Li, 2010) Let g be a convex polynomial on R n with degree at most d. Let S := {x g(x) 0} and x S. Then there exist constants c, ε > 0 such that (d 1) dist(x, S) c [g(x)] n +1 +, x x ε. 1 Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

9 Theorem 1. (Local error bounds for convex polynomial systems) Let g i be convex polynomials on R n with degree at most d for every i = 1,, m. Let S := {x R n g i (x) 0, i = 1,, m} and x S. Then there exist c, ε > 0 such that ( ) τ dist(x, S) c max 1 i m [g i (x)] + x x ε, where [α] + := max{α, 0}, τ := max { 2 (2d 1) n +1, 1 β(n 1)d n }, β(n 1) is the central binomial coefficient with respect to n 1 which is given by ( [(n 1)/2]). Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

10 Theorem 2. (Hölderian regularity) Let θ > 0 and K R n be a compact set. Then there exists c > 0 such that ( m τ dist θ (x, C) c dist θ (x, C i )), x K, i=1 1 where τ := { (2d 1) n +1 } and β(n 1) is the central binomial min, β(n 1)d 2 n coefficient with respect to n 1 which is given by ( n 1 [(n 1)/2]). Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

11 Our Main result Theorem 3. (Cyclic convergence rate) Suppose that d > 1. Let x 0 R n and the sequence of cyclic projections, (x k ) k N, be defined by x 1 := P 1 x 0, x 2 := P 2 x 1,, x m := P m x m 1, x m+1 := P 1 x m.... Then x k converges to x C, and there exists M > 0 such that x k x M 1, k N, k ρ 1 where ρ := { } and β(n 1) is the central min (2d 1) n 1, 2β(n 1)d n 2 binomial coefficient with respect to n 1 which is given by ( n 1 [(n 1)/2]). Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

12 von Neumann alternating projection When m = 2, we can consider the general case where the intersection of these two sets is (possibly) empty. We assume that g i, h j are convex polynomials with degree at most d A := {x R n g i (x) 0, i = 1,, m} B := {x R n h j (x) 0, j = 1,, l} b 0 R n, a k+1 := P A b k, b k+1 := P B a k+1. Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

13 von Neumann alternating projection Theorem 4. (von Neumann alternating projection) Assume d > 1. Then a k ã A and b k b B with b ã = v where v := P B A 0. Moreover, there exists M > 0 such that a k ã M 1 k ρ and b k, b M 1, k N, k ρ 1 where ρ := { } and β(n 1) is the central min (2d 1) n 1, 2β(n 1)d n 2 binomial coefficient with respect to n 1 which is given by ( n 1 [(n 1)/2]). Remark 1. The intersection of these two sets A and B is (possibly) empty. Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

14 Examples and Applications Example 1. Let C 1 := {(x, y) R 2 (x + 1) 2 + y 2 1 0} C 2 := {(x, y) R 2 x + y 1 0} C 3 := {(x, y) R 2 (x 1) 2 + y 2 1 0} C 4 := {(x, y) R 2 x + (y + 2) 2 4 0}. Take x 0 R 2. Let (x k ) k N be defined by x 1 := P 1 x 0, x 2 := P 2 x 1, x 3 := P 3 x 2, x 4 := P 4 x 3, x 5 := P 1 x 4... Then x k = O( 1 ). k 1 6 Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

15 Examples and Applications Example 2. Let α 0 and A := ( 1, 0) + B(0, 1) and B := { (x, y) R 2 x + α 0 }. Let (a k ) k N and (b k ) k N be defined by Then for every k 2 ( b k = α, a k+1 = b 0 R 2, a k+1 := P A b k, b k+1 := P B a k+1. ( 1 + t 1 (1 + α) 2(k 1) + t 2 1 (α + 1) 2 + t 1 (1 + α) 2k + t 2 1 k 2 i=0 ((1 + α)2i α + 1 t 2 1 (1+α) 2(k 1) +t 2 1 k 1 i=0 ((1 + α)2i ). ) k 2 i=0 ((1+α)2i, Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

16 Example 2 continued Consequently, If α = 0, a k 0 and b k (α, 0) at the rate of k 1 2 if α 0(then A B = ), a k 0 and b k (α, 0) at the rate of (1 + α) k. Remark 2. According to Theorem 4, we can only deduce that (a k ) k N in Example 2 converges to (0, 0) and (b k ) k N converge to (α, 0) at the rate of at least of k 1 6. Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

17 Examples and Applications Example 3. Let A := { (x, y) R 2 (x + 1) 2 + y } B := { (x, y) R 2 (x 1) 2 + y }. Let (x 0, y 0 ) R 2 and (x k, y k ) k N be defined by (x 1, y 1 ) := P A (x 0, y 0 ), (x 2, y 2 ) := P B (x 1, y 1 ), (x 3, y 3 ) := P A (x 2, y 2 ),. Figure 2 depicts the algorithm s trajectory with starting point (0, 2). Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

18 Example 3 continued Figure : The iteration commencing at (0, 2). Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

19 Example 3 continued Since (x k, y k ) bdry A bdry B, r k := where α k := x k. Hence 1 α k+1 = 1 + α k 1 + 4αk. x 2 k + y 2 k satisfies r 2 k = 2α k, Linearizing the above equation, set w k := 4α k to obtain w k+1 w k (1 w k ) and then 1 w k+1 1 w k = 1 1 w k. When summing and dividing by N, leads to lim N ( 1 1 = lim 1 ) N 1 1 = lim Nw N N Nw N Nw 0 N N k=0 = lim N 1 1 w k 1 1 w N = 1 (since (x k, y k ) 0). Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

20 Example 3 ended Then we have x 2 k + y 2 k 1 2k. Hence (x k ) k N and (y k ) k N converge to 0 and at the rate of k 1 2. Remark 3. According to Theorem 4, we can only deduce that (x k ) k N and (y k ) k N in Example 3 converge to (0, 0) at the rate of at least of k 1 6. Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

21 Examples and Applications Example 4. Let A, B be defined by A := { (x, y) R 2 x 0 } and B := { (x, y) R 2 y 2 x 0 }. Let b 0 R R + with b 0 1, and (a k ) k N, (b k ) k N be defined by Then a k+1 := P A b k, b k+1 := P B a k+1. (a k ) k N and (b k ) k N converge to 0 at the rate of at exactly of k 1 2. Remark 4. According to Theorem 4, we can only deduce that (a k ) k N and (b k ) k N in Example 4 converge to (0, 0) at the rate of at least of k 1 6. Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

22 Conclusion and questions Our explicit examples show that, in general, our estimate of the convergence rate of the cyclic projection algorithm will not be tight. It would be interesting to see how one can sharpen the estimate obtained in this talk and get a tight estimate for the cyclic projection algorithm. Can we extend the approach here to analyze the convergence rate of the Douglas-Rachford algorithm? Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

23 References H.H. Bauschke and P.L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Springer, H.H. Bauschke and J.M. Borwein, On the convergence of von Neumann s alternating projection algorithm for two sets", Set-Valued Analysis, vol. 1, pp , H.H. Bauschke and J.M. Borwein: On projection algorithms for solving convex feasibility problems", SIAM Review, vol. 38, pp , J.M. Borwein, G. Li, and L. Yao, Analysis of the convergence rate for the cyclic projection algorithm applied to semi-algebraic convex sets, submitted; April Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

24 References L.M. Brègman, Finding the common point of convex sets by the method of successive projection. Doklady Akademii Nauk SSSR, vol. 162, pp , L.G. Gubin; B.T. Polyak, and E.V. Raik, The method of projections for finding the common point of convex sets, USSR Computational Mathematics and Mathematical Physics, vol. 7, pp. 1 24, J. Kollár, An effective Łojasiewicz inequality for real polynomials, Periodica Mathematica Hungarica, vol. 38, pp , G. Li, On the asymptotic well behaved functions and global error bound for convex polynomials", SIAM Journal on Optimization, vol. 20, pp , Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25

25 Thanks for your attention Thanks for your attention. Liangjin Yao (University of Newcastle) cyclic projection algorithm on semialgebraic convex sets June 3rd, / 25