Analysis of the convergence rate for the cyclic projection algorithm applied to basic semi-algebraic convex sets

Transcription

1 Analysis of the convergence rate for the cyclic projection algorithm applied to basic semi-algebraic convex sets Jonathan M. Borwein, Guoyin Li, and Liangjin Yao Second revised version: January 04 Abstract In this paper, we study the rate of convergence of the cyclic projection algorithm applied to finitely many basic semi-algebraic convex sets. We establish an explicit convergence rate estimate which relies on the imum degree of the polynomials that generate the basic semi-algebraic convex sets and the dimension of the underlying space. We achieve our results by exploiting the algebraic structure of the basic semialgebraic convex sets. 00 Mathematics Subject Classification: Primary 4A5, 90C5; Secondary 4A50, 90C3 Keywords: Cyclic projection algorithm, convex polynomial, distance function, Fejér monotone sequence, Hölderian regularity, Lojasiewicz s inequality, projector operator, basic semi-algebraic convex set, von Neumann alternating projection method. Introduction A very common problem in diverse areas of mathematics and engineering consists of trying to find a point in the intersection of closed convex sets C i, i =,..., m. This problem is often referred to as the convex feasibility problem. One popular method for solving the convex feasibility problem is the so-called cyclic projection algorithm. Mathematically, the CARMA, University of Newcastle, Newcastle, New South Wales 308, Australia. jonathan.borwein@newcastle.edu.au. Laureate Professor at the University of Newcastle and Distinguished Professor at King Abdul-Aziz University, Jeddah. Department of Applied Mathematics, University of New South Wales, Sydney 05, Australia. g.li@unsw.edu.au. CARMA, University of Newcastle, Newcastle, New South Wales 308, Australia. liangjin.yao@newcastle.edu.au.

2 cyclic projection algorithm is formulated as follows. Given finitely many closed convex sets C, C,, C m in R n with m i= C i, let x 0 R n and P i := P Ci, i =,,, m, where P Ci denotes the Euclidean projection to the set C i. The sequence of cyclic projections, x k k N, is defined by. x := P x 0, x := P x,, x m := P m x m, x m+ := P x m... When m =, the cyclic projection method reduces to the well known von Neumann alternating projection method APM see [4] and also [8,, 3, 7] for some recent developments. The cyclic projection method has attracted much recent interest due to its simplicity and to numerous applications to diverse areas such as engineering and the physical sciences, see [7, 6, 0, 0, ] and the references therein. The convergence properties of cyclic projection methods have been examined by many researchers. In particular, Bregman [9] showed that the sequence x k k N generated by the cyclic projection algorithm, always converges to a point in C. Moreover, linear convergence of cyclic projection algorithm onto convex sets with regular intersections was shown in [3]. On the other hand, for convex sets with irregular intersections for example, when the intersection is a singleton, the cyclic projection algorithm may not exhibit linear convergence even for simple two dimensional cases as observed by [9, Example 5.3] see Section 5 for more examples. This then raises the following basic question: Can we estimate the convergence rate of the cyclic projection algorithm for convex sets with possibly irregular intersections? In this paper, we provide an answer for the above question by focusing on the case where each set C i is a basic semi-algebraic convex set in R n in the sense that there exist γ i N and convex polynomial functions, g i j, j =,..., γ i such that C i = x R n g i j x 0, j =,, γ i }. The main motivation for examining basic semi-algebraic convex sets lies with the following two facts. First, as recently established in [, 3], optimization problems involving semi-algebraic structure have a number of remarkable properties such as the celebrated Kurdyka- Lojasiewicz inequality which enables us to obtain useful qualitative information of the problem. Second, the class of basic semi-algebraic convex sets is a broad class of convex sets which covers polyhedra and convex sets described by convex quadratic functions. Additionally, the structure can often be relatively easily identified [4]. By exploiting the precise algebraic structure, we are able to provide an explicit rate for the cyclic projection algorithm applied to finitely many basic semi-algebraic convex sets without any regularity conditions. More precisely, let C i be basic semi-algebraic convex sets generated by polynomials in R n with degree at most d N. We show that the sequence of cyclic projections x k k N. converges at least at the rate of k when d >, where ρ

3 ρ := min d n, βn d n } and βs denotes the central binomial coefficient with respect to s which is given by s [s/]. When d =, the sequence of cyclic projections converges linearly. The remainder of this paper is organized as follows. In Section, we collect notation and auxiliary results for future use and for the reader s convenience. In Section 3, we give a Hölderian regularity result for finitely many basic semi-algebraic convex sets. The proof of our main result Theorem 4. forms the bulk of Section 4. In Section 5, we explore various concrete examples. Finally, we end the paper with some conclusions and open questions. Preliminaries and auxiliary results We assume throughout that R n is a Euclidean space with the norm and inner product,, where n N :=,, 3, }. We reserve d N. We denote by Bx, ε := y R n y x < ε}. We adopt standard notation used in these books [, 8, 37, 39, 4]. Given a subset C of R n, int C is the interior of C, bd C is the boundary of C, aff C is the affine hull of C and C is the norm closure of C. The orthogonal set is C := x R n c C x, c = 0}. The distance function to the set C, written dist, C, is defined by x inf c C x c. The projector operator to the set C, denoted by P C, is defined by P C x := c C y c = distx, C}, x R n. Let D R n. The distance of two sets: C and D, is distc, D := inf c C,d D c d. Given f : X ], + ], we set dom f := f R. We say f is proper if dom f. Let f be a proper function on R n. Its associated recession function f is defined, for any v R n, by f v := lim inf t, v v ftv. t If f is further assumed to be lower semicontinuous and convex, one has see [4, Proposition.5.]. f fx + tv fx fx + tv fx v = lim = sup t t t>0 t for all x domf.. Notation and facts on polynomials Recall that f : R n R is a polynomial if there exists a number r N such that fx := λ α x α, 0 α r Here, [a] denotes the integer part of a 3

4 where λ α R, x = x,, x n, x α := x α xαn n, α i N 0}, and α := n j= α j. The corresponding constant r is called the degree of f. Next let us recall a useful property of polynomial functions. Fact. See [5, Remark 4] Let f : R n R be polynomial, and x, x } R n. If f is constant on D := [x, x ], then f is constant on affd. We now summarize some basic properties of convex polynomials that will be used later. The first property is a Frank-Wolfe type result for convex polynomial optimization problems while the second one is a directional-constancy property for a convex polynomial. Fact. Belousov See [5, Theorem 3, Section 4, Chapter II] or [6, Theorem 3] and [35]. Let f be a convex polynomial on R n. Consider a set D := x g i x 0, i =,, m}, where each g i, i =,, m, is a convex polynomial on R n. Suppose that inf x D fx >. Then f attains its minimum on D. Fact.3 See [4, Proposition 3..]. Let f be a convex polynomial on R n and v R n. Assume that f v = 0. Then fx + tv = fx for all t R and for all x R n.. Notation and facts on semialgebraic sets/functions Following [7], a set D R n is said to be semi-algebraic if D := l i= j= s x R n f ij x = 0, h ij x 0} for some integers l, s and some polynomial functions f ij, h ij on R n i l, j s. Moreover, a function f : R n R is said to be semi-algebraic if its graph gphf := x, fx x R n } is semi-algebraic. We now summarize below some basic properties of semi-algebraic sets and semi-algebraic functions. These properties will be useful for our later work. Fact.4 The following statements hold the properties P and P4 are direct from the definitions. P Any polynomial is a semi-algebraic function. P See [7, Proposition..8]. Let D be a semi-algebraic set. Then dist, D is a semi-algebraic function. P3 See [7, Proposition..6]. If f, g are semi-algebraic functions on R n and λ R then f + g, λf, f, g}, fg are semi-algebraic. 4

5 P4 If f i are polynomials, i =,..., m, and λ R, then the sets x f i x = λ, i =,..., m}, x f i x λ, i =,..., m} are semi-algebraic sets. P5 Lojasiewicz s inequalitysee [7, Corollary.6.7]. If φ, ψ are two continuous semialgebraic functions on compact semi-algebraic set K R n such that = φ 0 ψ 0 then there exist constants c > 0 and τ 0, ] such that ψx c φx τ for all x K. Remark.5 As pointed out by [34], the corresponding exponent τ in Lojasiewicz s inequality P5 is hard to determine and is typically not known. Remark.6 Let g i, i =,..., m be polynomials on R n and set S := x R n g i x 0}. Let x S. Then, P and P4 imply that φ = i m [g i ] + with [g i ] + := g i, 0} and ψ = dist, S are semi-algebraic functions. Applying P5 it follows that there exist c, ε > 0 and τ 0, ] such that. distx, S c i m [g ix] τ + for all x B x, ε. As explained in Remark.5, the exponent τ in. is hard to determine and is typically unknown. However, there are some special cases where we can provide some effective estimates on the exponent τ: To formulate these results, we introduce the following notation. Define.3 κn, d := d n +. We now present various results which show that the exponent τ in. can be effectively estimated when g i have some appropriate extra structure. Fact.7 Gwoździewicz See [, Theorem 3]. Let g be a polynomial on R n with degree no larger than d. Suppose that g0 = 0 and there exists ε 0 > 0 such that gx > 0 for all x B0, ε 0 \0}. Then there exist constants c, ε > 0 such that.4 x c gx κn,d, x B0, ε. We denote by βs the central binomial coefficient with respect to an integer s: s [s/] with 0 0 = [6]. Fact.8 Kollár See [6, Theorem 3i]. Let g i be polynomials on R n with degree d for every i =,, m. Let gx := i m g i x. Suppose that there exists ε 0 > 0 such that gx > 0 for all x B0, ε 0 \0}. Then there exist constants c, ε > 0 such that x c gx βn d n, x B0, ε. 5

6 Fact.9 See [9, Theorem 4.]. Let g be a convex polynomial on R n with degree at most d. Let S := x gx 0} and x S. Then, g has a Hölder type local error bound with exponent κn, d, i.e., there exist constants c, ε > 0 such that κn,d distx, S c [gx] + for all x B x, ε. Fact.0 See [9, Theorem 4.]. Let g be a convex polynomial on R n. Let S := x gx 0}. Suppose that there exists x 0 R n such that gx 0 < 0. Then, g has a Lipschitz type global error bound, i.e., there exists a constant c > 0 such that distx, S c [gx] + for all x R n. Fact. See [38, Theorem ] Let g be a continuous convex function R n. Let S := x R n gx 0}. Suppose that there exists x 0 R n such that gx 0 < 0. Then, for every compact subset K of R n, there exists c > 0 such that distx, S c [gx] + for all x K. The following example shows that the conclusion of Fact. can fail if we allow K to be noncompact. Example. Shironin See [9, Example 4.] or [40]. defined by Let g, g : R 4 R be g x, x, x 3, x 4 := x, g x, x, x 3, x 4 := x 6 + x 8 + x x x 3 x x x 4 x 3 + x x x 4 x 4 3 Then g,g are convex polynomials and + x 4 x 6 + x x 6 + x + x + x 3 x 4. g k, 0, 0, k 6 + k + k = g k, 0, 0, k 6 + k + k = k < 0, k N. Let gx := g x, g x} for all x R 4. Then, g is a continuous convex function and g k, 0, 0, k 6 + k + k < 0 for all k N. On the other hand, as shown in [40], there exists a sequence x k k N in R 4 such that [gx k ] +, k but distx k, S +, where S := x R 4 g x 0, g x 0}. Let K be a noncompact set such that x k } K. Then, the conclusion of Fact. distx, S c [gx] + for all x K must fail in this case. Recall that a set C R n is a basic semi-algebraic convex set if there exist γ N and convex polynomial functions, g j, j =,..., γ such that C = x R n g j x 0, j =,, γ}. Clearly, any basic semi-algebraic convex set is convex and semi-algebraic. However, the following example shows that a convex and semi-algebraic set does not need to be a basic semi-algebraic convex set. 6

7 Example.3 Consider the set A := x, x R x x 0, x 0, x 0}. Clearly, A is convex and semi-algebraic while the polynomial x, x x x is not convex. We now show that A is not a basic semi-algebraic convex set, i.e., it cannot be written as x : g i x 0, i =,, l} for some convex polynomials g i, i =,, l, l N. To see this, we proceed by the method of contradiction. Let f : R R be defined by fx, x := x. Clearly inf x=x,x A fx = 0. Then, then by Fact., f should attain its minimum on A. This leads to a contradiction, and so, justifies the claim..3 Notation and facts for projection methods From now on, we assume that m N, γ i N, i =,..., m g i,, g i,,, g i,γi are convex polynomials on R n, i =,..., m } C i := x R n g i, x 0, g i, x 0,, g i,γi x 0, i =,..., m P i := P Ci, i =,,, m m C := C i. i= Fact.4 Bregman See [9]. Let x 0 R n. The sequence of cyclic projections, x k k N, defined by.5 x := P x 0, x := P x,, x m := P m x m, x m+ := P x m... converges to a point in C. Fact.5 Bauschke and Borwein See [9, Lemma. and Theorem 4.8], or [8, Fact.iii and Fact.ii]. Let A, B be nonempty convex subsets of R n such that A B is closed. Let b 0 X and a k k N, b k k N be defined as below: Let v := P A B 0. Then, we have ak+ := P A b k b k+ := P B a k+. i v = dista, B and a k a, b k a + v. ii P B x = P B A+v x = x+v, x A B v and P A y = P A B v y = y v, y B A + v 7

8 Definition.6 Let A be a nonempty convex subset of R n. We say the sequence x k k N in R n is Fejér monotone with respect to A if x k+ a x k a, k N, a A. Fact.7 Bauschke and Borwein See [8, Theorem 3.3iv]. Let A be a nonempty closed convex subset of R n and let x k k N be Fejér monotone with respect to A, and x k x A. Then x n x distx n, A. 3 Hölderian regularity for basic semi-algebraic convex sets In this section, we will establish Hölderian regularity for basic semi-algebraic convex sets and shall provide an effective estimate of the exponent in the regularity results. This result plays an important role in our following estimation of the convergence speed of the cyclic projection methods. To do this, we first establish an error bound result which estimates the distance of a point to a basic semi-algebraic convex set S in terms of the polynomials which define S. More explicitly, we obtain an explicit exponent τ > 0 such that there exist c, ε > 0, τ distx, S c [g ix] + whenever x x ɛ, i m where S := x R n g i x 0, i =,, m}. We note that this error bound property plays an important role in convergence analysis of many algorithms for optimization problems [36] as well as for the variational inequality problem [30], and that the exponent τ in the error bound property has a close relationship with the convergence rate of the algorithm. However, existing results such as the powerful Lojasiewicz s inequality do not provide any insight on how to explicitly estimate the exponent τ. Before we proceed, let us use a simple example to illustrate that the exponent τ can be related to the imum degree of the polynomials defined the basic semi-algebraic convex set and the dimension of the underlying space. This example is partially inspired by [6, Example ]. Example 3. Let d be an even number. Consider convex polynomials g i, i =,..., n on R n given by g x := x d and g i, i =,, n given by g i x := x d i x i, i =,, n. Then, direct verification gives us that S := x R n g i x 0, i =,, n} = 0}, and so, distx, S = x. In this case, consider xt = t dn, t dn,, t R n, t 0,. Then distxt, S = Ot and i m [g i xt] + = t dn. Therefore, we see that if there exist c, ɛ > 0 and τ > 0 such that τ distx, S c [g ix] + whenever x ɛ, i m 8

9 then, τ d. Thus, we see that the exponent τ is related to the imum degree of n the polynomials defined the basic semi-algebraic convex set and the dimension of the underlying space. We now introduce a decomposition of the index set. Definition 3. For convex polynomials g,..., g m on R n with S = x R n g i x 0, i =,..., m}, the index set,..., m} can be decomposed as J 0 J with J 0 J = where 3. J 0 := i,..., m} g i x = 0 x S } and J :=,..., m}\j 0. Now we come to our key technical result which provides a local error bound for convex polynomial systems. The main idea of the proof is to use the extreme rigidity of convex polynomial functions see Fact. and.3 to reduce the problem to the one of comparing a power of a nonnegative convex polynomial vanishing only at 0 with the norm so that the estimate of the Lojasiewicz inequality that we introduced in Fact.7 and Fact.8 can be applied. Theorem 3.3 Local error bounds for convex polynomial systems Let g i be convex polynomials on R n with degree at most d for every i =,, m. Let S := x R n g i x 0, i =,, m} and x S. Then there exist c, ε > 0 such that distx, S c i J [g i x] + + i J 0 [g i x] + τ whenever x x ε, where [a] + := a, 0}, τ := κn,d, } βn d, κn, d := d n +, βn n is the central binomial coefficient with respect to n which is given by [n /], and J 0, J are defined as in 3.. Proof. We prove the desired conclusion by induction on the number of the polynomials m. [Trivial Case] Suppose that m =. Then J 0 = } or J 0 =. If J 0 = }, then the conclusion follows by Fact.9 since κn,d, } βn d n κn,d. If J 0 =, then there exists x 0 such that g x 0 < 0. In this case, the conclusion follows by Fact.0. [Reduction to the active cases] Let us suppose that the conclusion is true for m p, p N, and look at the case for m = p. We first consider the case where J 0,..., m}. Then,..., m}\j 0. Let i 0 / J 0. Then there exists x 0 S such that g i0 x 0 < 0. Set J := i,,..., m} g i x 0 < 0 }. Then i 0 J J and J J 0 =. Let A, B be defined by A := x R n g i x 0, i J } B := x R n g i x 0, i J }, 9

10 where J :=,,..., m}\j. Thus, J 0 J. Then we have x 0 int A B and S = A B. Since S B, we have 3. J 0 := i,,..., m}\j g i B 0 } J 0. Since int A B, [8, Corollary 4.5] implies that for every compact set K there exist γ, δ > 0 such that 3.3 distx, S = distx, A B γ distx, A, distx, B} for all x K. Now applying Fact. with fx = i J [g i x] +, there exists c > 0 such that 3.4 distx, A c i J [g ix] +, x K. From the induction hypothesis and 3., we see that there exist ε > 0 and c > 0 such that for every x x ε, i m [g i x] + such that distx, B c [g i x] + + τ [g i x] + i J\ J 0 i J 0 = c } τ [g i x] +, [g i x] + + [g i x] + i J\J 0 i J 0 \ J 0 i J 0 c [g i x] + + [g i x] + + τ [g i x] + i J\J 0 i J 0 \ J 0 i J 0 c [g i x] + + τ τ [g i x] + + [g i x] + i J\J 0 i J 0 \ J 0 i J 0 c [g i x] + + τ [g i x] + i J\J 0 i J 0 c [g i x] + + τ [g i x] +. i J\J 0 i J 0 since J 0 J 0 J Thus, combining 3.3 and 3.4, and noting that J\J 0 J J, we see that the conclusion follows when J 0,..., m}. From now on, we may assume that J 0 =,..., m}. That is to say, x gi x 0, i =,, m } = x g i x = 0, i =,, m }. This implies that inf x R n i m g i x} = 0. Then, 0 R m / g x,, g m x x R n} + int R m +. Hence the convex separation theorem ensures that there exist α i 0 with m i= α i = such that m i= α ig i x 0 for all x R n. Denote I := i α i > 0}. Then, we have α i = and 3.5 α i g i x 0 for all x R n. 0

11 [Decompose the underlying space into sum of two subspaces M and M ] Consider D := x R n g i x 0, i I}. Clearly, D is a convex set and x D. Moreover, for any v D, 3.5 implies that 3.6 g i v = 0, i I. In other words, g i takes constant value 0 on D. Then, Fact. implies that D is either a singleton or an affine set with dimension larger than one. Let M := D x. Then M is a subspace. We may decompose R n = M + M. Denote dimm = k k n. 3.7 We now see that gi x > 0 for all x x M \0}. Otherwise, there exists x 0 R n such that x 0 x M \0} and g i x 0 = 0 for all i I. This shows that x 0 D. Thus x 0 x M and hence x 0 x M M. This contradicts the fact that x 0 x 0. Similarly, we have 3.8 g ix > 0 for all x x M \0}. [Distance estimation on M ] We first show that there exist ε 0, γ 0 > 0 such that 3.9 x x γ 0 gi x κn k,d, x x M B0, ε 0. Since dim M = n k, there exists an n n k matrix Q 0 with the rank n k such that Q 0 R n k = M. Then Q 0 is a bijective operator from R n k to M. Then 3.7 shows that 3.0 gi x + Q 0 b > 0, b R n k \0}. Define h : R n k R by hb := g i x + Q 0b. Then h0 = g i x = 0 by 3.6. Thus by 3.0 and Fact.7, there exist ε, γ > 0 such that for all b γ hb κn k,d Setting x := x + Q 0 b, it follows that = γ gi κn k,d x + Q 0 b, b ε. x x = Q 0 Q 0 x x Q 0 Q 0 x x Q 0 γ gi κn k,d x, Q 0 x x ε.

12 Hence, there exist ε, γ > 0 such that x x γ 0 gi x κn k,d, x x M B0, ε 0. Thence 3.9 holds. By Fact.8 and 3.8, there exist ε 0, γ 0 > 0 such that ε 0 ε 0 and, for all x with x x M B0, ε 0, g ix and 3. x x γ 0 g βn k d ix n k = γ 0 [g ix] + βn k d n k. j I\i} α j [Distance estimation on M] Set r := α i i I} > 0, if I ;, otherwise.. Thus r. Note that α ig i x 0, we have for each i I [g j I\i} ix] + g i x α jg j x r α [g ix] +. i Hence we have g i x r [g i x] +. This together with 3.9 implies that x x γ 0 r I [g κn k,d ix] +, x x M B0, ε 0. Combining this with 3., we see that, for every x x M B0, ε 0, 3. x x γ 0 r I + γ 0 [g κn k,d, βn k d ix] n k } +. We now consider two cases. Case : dim M = 0}. We have D = S = x}. Thus M = 0 and M = R n. We can assume that [g i x] + for all x x B0, ε 0. Then by 3., we have distx, S = x x γ 0 r } I + γ 0 [g κn,d, βn d n ix] +, x x ε 0. Case : k = dim M.

13 Since dim M = k, there exists a full rank matrix Q R n k such that QR k = M. For each u M and i I, 3.6 implies that Then, Fact.3 implies that g i g i x + tu g i x u = lim = 0. t t 3.3 g i x + u = g i x for all x R n, u M, i I. Since S D = x + M, it follows that S = x x + M R n g i x 0, i / I} = x + QŜ, where Ŝ := a Rk g i x + Qa 0, i / I}. Note that 0 Ŝ. The induction hypothesis implies that there exist ε, γ > 0 such that i/ I [g i x + Qa] + and } dista, Ŝ γ [g κk,d i x + Qa], βk d k + for all a ε. i/ I This implies that there exist ɛ, γ > 0 such that } 3.4 distx, S γ [g κk,d ix], βk d k + i/ I for all x x M B0, ε. [Combining the estimation and simplification] Now let ε min ε 0,ε } be such that i m [g i x] + for all x B x, ε. Let K be a compact set containing B x, ε B0, ε. Denote the Lipschitz constant of g i over K by L i, i.e., g i x g i x L i x x for all x, x K. Set L := i m L i and γ := γ 0 r I + γ 0, γ }. To arrive at the conclusion, we only need to show that for any x B x, ε, } κn,d, βn d n distx, S c i m [g ix] + where c := γ + Lγ. To see this, let us fix an arbitrary x B x, ε. Note that R n = M + M. Then, one can decompose x x = u + v for some u M B0, ε and v M B0, ε. This together with 3.4 and 3. implies that } distu + x, S γ [g κk,d iu + x], βk d 3.5 k + i/ I } v γ [g κn k,d, βn k d 3.6 iv + x] n k +. Therefore, distx, S distu + x, S + x u + x, 3

14 = distu + x, S + v γ [g iu + x] + i/ I = γ [g iu + x] + i/ I γ i m [g iu + x] + κk,d, βk d k } κk,d, βk d k } κk,d, βk d k } + γ [g iv + x] + + γ [g ix] + + γ i m [g ix] + } κn k,d, βn k d n k κn,d, βn d n } where the second equality follows by 3.3 and v + x + u = x. Note that 3.7 i m [g iu + x] + i m [g ix] + i m g iu + x g i x L u + x x = L v Lγ [g iv + x] + = Lγ [g ix] + κn k,d, βn k d n k } κn k,d, βn k d n k } As i m [g i x] + for all x B x, ε, it follows that distx, S γ 3.8 γ i m [g iu + x] + κk,d [g ix] + + Lγ i m + γ γ + γ i m [g ix] + κn k,d [g ix] + γ [g ix] + i m κn k,d + } κn,d, βn d n [g ix] + + Lγ [g ix] i m i m [g ix] + } κn,d, βn d n κk,d κn k,d + κn,d, βn d n } by 3.6 by 3.3 and v + x + u = x. } κn,d, βn d n by 3.7 κk,d γ + Lγ [g 4 κn k,d κk,d ix] + + γ [g ix] + i m i m 4 = γ + Lγ [g d ix] n k + d k γ [g ix] + i m i m γ + Lγ [g d ix] n γ [g ix] + i m i m Similarly, we also have distx, S γ i m [g iu + x] + βk d k + γ i m [g ix] + } κn,d, βn d n } κn,d, βn d n κn,d, βn d n } } κn,d, βn d n., 4

15 γ [g ix] + + Lγ i m + γ γ i m [g ix] + [g ix] βn k d n k βn k d n k + } κn,d, βn d n βk d k by 3.7 βn k d n k + [g ix] + + Lγ [g ix] i m } + γ [g κn,d, βn d n ix] + i m βn k d = γ + Lγ [g ix] n k βk d k + + γ i m γ + Lγ [g βn d ix] n + + γ [g ix] + i m i m βk d k i m [g ix] + } κn,d, βn d n where the last inequality was obtained by the Chu-Vandermonde identity. In combination with 3.8 we obtain } distx, S γ + Lγ [g ix] d n +, βn d n + + γ [g ix] + i m i m = γ + Lγ [g ix] d n +, βn d n } +. i m } κn,d, βn d n, } κn,d, βn d n This completes the proof. As a corollary, we obtain a local error bound result which is independent of the partition of the index set. Corollary 3.4 Let g i be convex polynomials on R n with degree at most d for every i =,, m. Let S := x R n g i x 0, i =,, m} and x S. Then there exist c, ε > 0 such that τ distx, S c ix] + i m whenever x x ε, where [a] + := a, 0}, τ := κn,d, βn d n } = min d n +, βn d n } and βn is the central binomial coefficient with respect to n. Proof. Choose ε small enough so that i m [g i x] +. Then, the conclusion follows immediately from the preceding Theorem 3.3 by noting that [g i x] + i m κn,d [g i x] + for each i =,..., m. Remark 3.5 Discussion of the exponent Let g i be convex polynomials on R n with degree at most d for every i =,, m. Let S := x R n g i x 0, i =,, m} and x S. We now make some discussion on the exponent in our local error bound results. 5

16 Theorem 3.3 shows that in the case when d = or if there exists x 0 R n such that g i x 0 < 0, i =,..., m, we indeed obtain a Lipschitz type local error bound. That is to say, in these cases, there exist c, ε > 0 such that distx, S c i m [g ix] + whenever x x ε, where [a] + := a, 0}. To see this, if d =, then κn,d = and βn d. n So, τ = κn,d, } βn d = and the conclusion follows immediately from n Theorem 3.3. On the other hand, if there exists x 0 R n such that g i x 0 < 0, i =,..., m, then, J 0 =. So, the conclusion follows immediately from the same theorem. In the case when n =, we see that κn,d = βn d = n d. So, when each g i is a univariate convex polynomial, then there exist c, ε > 0 such that distx, S c [g d ix] + whenever x x ε. i m Note that, for the naive simple example g x := x d, local error bound holds at 0 with exponent d. This suggests that our result matches what one might expect in the univariate case. 3 On the other hand, in general, our estimation on the exponent will not be optimal. For example, if the inequality system consists of one single convex polynomial, Fact.9 shows that the exponent can be set as d n + while our results produce a weaker exponent d n +, } βn d. An interesting feature of the exponent n d n + in Fact.9 is that, in the convex quadratic case, it collapses to which is independent of the dimension of the underlying space and which agrees with the known result presented in [3]. By contrast, our estimate 3 n +, } βn n depends heavily on the dimension n. Moreover, as indicated in Example 3., the best possible exponent might be d see n [6] for some relevant discussion regarding the best possible exponent for general nonconvex polynomial system. It would be interesting to find how to could improve our estimate here. Making better sense of these estimates will be one of our future research topics. Given D R n, we set dist r, D := dist, D r for every r R. Theorem 3.6 Hölderian regularity Let γ i N, i =,..., m, and g i,j be are convex polynomials on R n with degree d N, j =,..., γ i, i =,..., m. Recall that } m C i = x R n g i,j x 0, j =,..., γ i and C = C i. 6 i=

17 Let θ > 0 and K R n be a compact set. Then there exists c > 0 such that m τ dist θ x, C c dist θ x, C i, x K, i= where τ := d n + } and βn is the central binomial coefficient with min, βn d n respect to n which is given by n [n /]. Proof. To see the conclusion, we only need to show that for each x R n, there exist c, ε > 0 such that m τ 3.9 dist θ x, C c dist θ x, C i, for all x x ε. i= Indeed, granting this and fixing a compact set K, then for any x K there exist c x, ε x > 0 such that m τ dist θ x, C c x dist θ x, C i, for all x x ε x. i= As K is compact and x K B x; ε x K, we can find finitely many points x,, x s K, s N, such that s i= B x i; ε xi K. Let c := c x,, c xs }. Then, for any x K, there exists i 0,, s} such that x B x i0 ; ε xi0, and hence dist θ x, C c xi0 m i= τ m τ dist θ x, C i c dist θ x, C i. We now show 3.9 holds. Fix x R n. We consider two cases. Case : x / C. Then there exist ε, η, M > 0 such that m dist θ x, C i η and dist θ x, C M for all x x ε. i= Therefore, dist θ x, C M = M η τ η τ M η τ m i= distθ x, C i τ for all x x ε and hence, 3.9 holds. Case : x C. i= We have C = x R n g i, x 0, g i, x 0,, g i,γi x 0, } i =,, m. By Corollary 3.4, there exist positive constants c 0 and δ such that distx, C c } τ θ 0 [gi, x] +,, [g i,γi x] +, x x δ. i m 7

18 Hence 3.0 dist θ x, C c 0 i m [gi, x] +,, [g i,γi x] + } θτ, x x δ. Now we claim that there exists β > 0 such that 3. i m } θ gi, x] +,, [g i,γi x] + β m dist θ x, C i, x x δ. Suppose to the contrary that there exists a sequence x k k N in B x, δ such that 3. i m } θ m [gi, x k ] +,, [g i,γi x k ] + > k dist θ x k, C i, k N. Without loss of generality, we can assume that g i,, g i,, g i,γi have the Lipschitz constant L > 0 on B x, δ for every i =,,, m. Then, there exists a subsequence x kl l N of x k k N, i 0 m and j 0 γ i0 such that It follows from 3. that } [gi, x kl ] +,, [g i,γi x kl ] + = [gi0,j 0 x kl ] +, l N. i m i= i= 3.3 θ m [gi0,j 0 x kl ] + > kl dist θ x kl, C i, l N. i= Then [g i0,j 0 x kl ] + = g i0,j 0 x kl and hence for every l N, 3.4 gi0,j 0 x kl θ m > kl dist θ x kl, C i = k l i= m i= xkl P i x kl θ k l x kl P i0 x kl θ. Since P i0 x kl C i0, we have g i0,j 0 Pi0 x kl 0 and x kl P i0 x kl x kl x < δ by x C i0. Combining this with 3.4, we have L θ x kl P i0 x kl θ g i0,j 0 x kl g i0,j 0 Pi0 x kl θ > kl x kl P i0 x kl θ, l N. Hence we have L θ > k l for every l N, this contradicts the fact that k l +. Thus, 3. holds. Combining 3. and 3.0, we see that m τ dist θ x, C c 0 β τ dist θ x, C i, x x δ, i= and so the conclusion follows. 8

19 4 Convergence rate for the cyclic projection algorithm In this section, we derive explicit convergence rate of the cyclic projection algorithm applied to finite intersections of basic semi-algebraic convex sets. Before we come to our main result, we need the following useful lemma, Lemma 4., which is a special case of Alber and Reich s result in []. For the reader s convenience, we provide a direct and self-contained proof. Lemma 4. Recurrence relationships Let p > 0, and let δ k } k=0 and β k} k=0 be two sequences of nonnegative numbers satisfying the conditions β k+ β k δ k β p k as k = 0,,.... Then, we have 4. β k β p k 0 + p i=0 δ i p for all k N. We use the convention that 0 δ k =. k=0 = +. In particular, we have lim k β k = 0 whenever Proof. It follows from our assumption that 0 β i+ β i β 0 and δ i β p+ i β i β i+ as i N. Fix k N. We consider two cases. Case : β k = 0. Clearly, 4. holds. Case : β k 0. Thus β k > 0 and hence β i > 0 for every i k. Define the nonincreasing function h : R ++ ], + ] by hx := x p+. As δ i hβ i = δ i β p+ i β i β i+, then we get βi δ i β i β i+ hβ i hxdx = β p i+ β p i. β i+ p This implies that 4. β p i+ β p i pδ i for all i N 0}. 9

20 Now fix any k N and, summing 4. from i = 0 to i = k, we get β p k k β p 0 p δ i. i=0 which implies the conclusion in 4.. We also need the following technical result. The proof of it follows in part that of [3, Lemmas 3&4], one may also consult [0]. Proposition 4. Cyclic convergence rate Let D i R n be a closed convex set, i =,..., m, and D := m i= D i. Let x 0 R n and the sequence of cyclic projections, x k k N, be defined by x := P x 0, x := P x,, x m := P m x m, x m+ := P x m..., where we set P i := P Di for the convenience. Suppose that Hölderian regularity with exponent τ 0 < τ holds: for any compact set K R n and θ > 0, there exists c 0 > 0 such that m τ dist θ x, D c 0 dist θ x, D i, x K. i= Then x k converges to x D. Moreover, there exist M > 0 and r 0 ]0, [ such that x k x M k τ if τ ]0, [, M r0 k if τ =, k N, Proof. We denoted by α i := i mod m +, i N. Thus x k+ = P Dαk x k. By Fact.4, there exists x D such that x k x. We first follow closely the proofs of [3, Lemmas 3&4] to get that 4.3 dist x k, D dist x k+, D dist x k, D αk, k N. Indeed, using the definition of projection operator, we have dist x k, D dist x k+, D x k P D x k x k+ P D x k = x k P D x k P αk x k P D x k = x k P D x k Pαk x k x k + x k P D x k = x k P D x k P αk x k x k x k P D x k + x k P αk x k, x k P D x k x k P αk x k + x k P αk x k, x k P αk x k + P αk x k P D x k = x k P αk x k + x k P αk x k + x k P αk x k, P αk x k P D x k x k P αk x k = dist x k, D αk, k N. Hence 4.3 holds. 0

21 Next we claim that for every i N 4.4 distx k, D αi distx k, D αk + distx k+, D αk distx k+m, D αk+m. To see this, note that there exists i 0 m such that α i0 +k = α i. Then, we have distx k, D αi = distx k, D x αi0 +k k x i0 +k + x i0 +k P x αi0 +k i 0 +k x k x k+ + + x i0 +k x i0 +k + x i0 +k P x αi0 +k i 0 +k = x k P αk x k + + x i0 +k P x αi0 +k i 0 +k + xi0 +k P x αi0 +k i 0 +k = distx k, D αk + distx k+, D αk+ + + distx i0 +k, D αi0 +k distx k, D αk + distx k+, D αk+ + + distx k+m, D αk+m by i 0 m. Hence 4.4 holds. 4.5 Thus by 4.4, dist x k, D αi m distx i, D αi k i k+m m dist x k, D αk + dist x k+, D αk dist x k+m, D αk+m. By the assumption, there exists c 0 > 0 such that m τ dist x k, D c 0 dist x k, D i, k N. i= By enlarging c 0 if necessary, we may assume that c 0 >. Let r := τ. Then by 4.5, for every k N, c r 0 dist r x k, D m dist x k, D i m i m dist x k, D i i= m 3 dist x k, D αk + dist x k+, D αk dist x k+m, D αk+m 4.6 k+m m 3 i=k dist x i, D dist x i+, D by 4.3 = m 3 dist x k, D dist x k+m, D. Thus we have 4.7 dist x k+m, D dist x k, D m 3 c r dist r x k, D. 0 Now we consider two cases.

22 Case : τ ]0, [. Thus r >. Fix k 0 N. Let β i := dist x k0 +im, D, i N 0}. Then 4.7 shows that 4.8 β i+ β i m 3 c r βi r = β i 0 m 3 c r βi r. 0 By Lemma 4., dist x k0 +im, D = β i Thus, there exists M 0 > 0 such that Hence there exists M > 0 such that 4.9 So, we have β0 r + r i m 3 c r 0 distx k0 +im, D M 0 r i, i N. r, i N. distx k, D M r k = M k τ, k N. 4.0 x k x x k P D x k + P D x k x = distx k, D + P D x k x. By [3, Lemma 3], x k+l P D x k x k P D x k = distx k, D, l N. Letting l in the above inequality, we obtain that 4. x P D x k distx k, D. Combining 4.0, 4. and 4.9, 4. x k x distx k, D M k τ, k N. Thus, the conclusion follows by letting M := M. Case : τ =. Then we have r = τ =, and so, 4.7 implies that for all k N distx k+m, D m 3 c r distx k, D. 0 Hence there exist M > 0 and r 0 ]0, [ such that distx k, D M rk 0, using a similar method of proof as in Case, we obtain that k N. Then, 4.3 x k x distx k, D M r k 0, k N. Thus, the conclusion follows by letting M := M.

23 Remark 4.3 Connection to the existing result on linear convergence In the case where there exists i 0,..., m} such that D i0 int i i 0 D i in this case, we say the intersection is regular, then the Hölderian regularity result holds with exponent τ =. So, the preceding proposition implies that the cyclic projection algorithm converges linearly in the regular intersection case. Thus, this recovers the linear convergence result for cyclic projection algorithm established in [3]. We are now ready for one of our main results. Theorem 4.4 Estimate of the cyclic convergence rate Let x 0 R n and the sequence of cyclic projections, x k k N, be defined by x := P x 0, x := P x,, x m := P m x m, x m+ := P x m.... Then x k converges to x C, and there exist M > 0 and r 0 ]0, [ such that where ρ := min M k, if d > ; x k x ρ, k N,, if d = Mr k 0 d n, βn d n [n /]. respect to n which is given by } and βn is the central binomial coefficient with Proof. Combining Theorem 3.6 and Proposition 4., we directly obtain τ := d n + }. Note that τ = min d n, βn d n }. min, βn d n Thus the conclusion follows from the preceding proposition. Remark 4.5 Discussion on our estimation of the convergence rate Although our estimate of the convergence rate works for cyclic projection algorithm with finitely many basic semialgebraic convex sets without any regularity condition, the estimated convergence rate is quite poor when the dimension n of the underlying space and the imal degree d are large. This is mainly due to the fact that the estimated convergence rate is derived by using the local error bound result for general convex polynomial systems. It would be interesting to see how one could improve the estimation of the convergence rate by either adopting other approaches or by further exploiting the structure of the underlying convex sets. For example, one possibility would be to examine problems involving some suitable additional curvature or uniform convexity assumptions. This will be another of our future research topics. 3

24 4. Alternating projection algorithm In this subsection, we discuss the convergence rate of the alternating projection algorithm. We assume throughout this subsection that g i, h j are convex polynomials with degree at most d, i =,,, m, j =,,, l A := x R n g i x 0, i =,, m} B := x R n h j x 0, j =,, l} b 0 R n, a k+ := P A b k, b k+ := P B a k+. As an immediate corollary of Theorem 4.4, we first obtain the following estimate on the convergence rate of the alternating projection algorithm in the case where the two sets have nonempty intersection. The case of two sets with nonempty intersection Corollary 4.6 Alternating convergence rate Suppose that A B. Let the sequence a k, b k } be generated by the alternating projection algorithm. Then, a k, b k c A B. Moreover, there exist M > 0 and r 0 ]0, [ such that for every k N a k c where ρ := min M k, if d > ; M ρ Mr0 k, if d =, and b k, if d > ; k c ρ Mr0 k, if d =, d n, βn d n [n /]. respect to n which is given by } and βn is the central binomial coefficient with Recently, [] established a local convergence rate analysis for proximal alternating projection methods for very general nonconvex problems, and where the corresponding convergence rate involves the exponent of the Kurdyka- Lojasiewicz inequality. The proximal alternating projection method is a variant of the alternating projection algorithm we discussed here. On the other hand, as we discussed before, in general, the actual exponent of the Kurdyka- Lojasiewicz inequality is typically unknown and hard to estimate. Corollary 4.6 above complements the result of [] in the case of basic convex semialgebraic cases by providing an explicit estimate of the convergence rate. The case of two sets with empty intersection In this part, we consider the general case where the intersection of these two sets is possibly empty. We first need the following lemma. 4

25 Lemma 4.7 The difference B A of two basic semi-algebraic sets A, B is closed. Proof. Let b k B and a k A be such that b k a k c. We now show that c B A. Consider the following convex polynomial optimization problem P min x R n,y Rn y x c s.t. g i x 0, i =,, m, h j y 0, j =,, l. Note that a k, b k are feasible for P. Hence we see that infp = 0. By Fact., the optimal solution of P exists. Thus there exists x A and y B such that c = y x B A. Hence the conclusion follows. Remark 4.8 With A and B defined as above, Fact. implies that B A is closed convex. Hence P B A 0. Let v := P B A 0. Then, there exist a A and b B such that v = b a and hence dista, B = v. Remark 4.9 In general, the distance between two convex and semi-algebraic sets need not be attained. For instance, consider D := x, x R x x, x 0, x 0} and E := x, x R x = 0}. It is clear that D, E are both convex and semialgebraic; while D is not a basic semi-algebraic convex set as explained in Example.3. Clearly, distd, E = 0 but D E =. Thus, the distance is not attained in this case. The proof of Theorem 4.0 partially follows that of [8, Theorem 3.]. Theorem 4.0 Convergence rate in the infeasible case Let the sequence a k, b k } be generated by the alternating projection algorithm. Then a k ã A and b k b B with b ã = v where v := P B A 0. Moreover, there exist M > 0 and r 0 ]0, [ such that for every k N 4.4 a k ã M k, if d > ; M ρ k, if d > ; Mr0 k, if d = and b k, b ρ Mr0 k, if d =, where ρ := min d n, βn d n [n /]. respect to n which is given by } and βn is the central binomial coefficient with Proof. Lemma 4.7 implies that B A is closed. Then by Fact.5i, there exist ã A, b B such that a k ã A and b k b B with b ã = v. By Theorem 3.6, there exists c 0 > such that 4.5 dista k, A B v c 0 distak, A + dista k, B v r = c 0 dist r ak, B v, 5

26 where r := min d n +, βn d n}. Fix x A B v. Note that v = P B A 0 we have P B x = x + v by Fact.5ii. This implies that dist a k, B v a k b k v = a k x b k v + x = a k x P B a k P B x a k x P B a k P B x by [, Proposition 4.8] = a k x b k x + v a k x P A b k P A x + v = a k x a k+ x by Fact.5ii. In particular, choose x = P A B v a k. Then, we have dist a k, B v dist a k, A B v a k+ P A B v a k Combining with 4.5, we have dist a k, A B v dist a k+, A B v. c r dist r a k, A B v dist a k, B v dist a k, A B v dist a k+, A B v. Thus dista k+, A B v dista k, A B v c r dista k, A B v r. 0 Now, let β k := dista k, A B v, k N. Then, we have 4.7 Now we consider two cases. Case : d >. β k+ β k c r 0 β r k. In this case, we have r >. Applying the preceding Lemma 4. with δ k := c r 0 p := r, by 4.7, we see that dist a k, A B v = β k Thence there exists M 0 > 0 such that where ρ := min β r 0 + dista k, A B v M 0 k ρ, d n, βn d n r r c r k 0 monotone with respect to A B v. Thus, by Fact.7, k N for all k N. and }. Then, [8, Example 3.] shows that a k k N is Fejér a k ã dista k, A B v M 0, k N. kρ 6

27 Case : d =. Thus r =. Then by 4.7, dista k+, A B v θ dista k, A B v, where θ :=. Then θ < since c c 0 >. Hence there exists M > 0 such that 0 dista k, A B v M θ k. From [8, Example 3.], we see that a k k N is Fejér monotone with respect to A B v. Thus, by Fact.7, a k ã dista k, A B v M θ k. Set M := M 0, M }. Combining the above two cases, we have a k ã M k, if d > ; ρ, k N. M θ k, if d = Similarly, we can show that there exist L > 0 and η ]0, [ such that b k b L k, if d > ; ρ, k N. Lη k, if d = Therefore, the conclusion follows by taking M := M, L} and r 0 := θ, η}. 5 Examples and remarks In this section, we will provide several examples of the rates of convergence of the cyclic projection algorithm and the von Neumann alternating projection algorithm. We first start with some examples where the basic semialgebraic convex sets are described by convex quadratic functions. Subsequently, we will examine examples where the basic semialgebraic convex sets are described by higher degree convex polynomials. Basic semialgebraic convex sets described by convex quadratic functions Example 5. Let C := x, y R x + + y 0} C := x, y R x + y 0} C 3 := x, y R x + y 0} C 4 := x, y R x + y + 4 0}. 7

28 Take x 0 R. Let x k k N be defined by x := P x 0, x := P x, x 3 := P 3 x, x 4 := P 4 x 3, x 5 := P x 4... Then x k = O. k 6 Proof. Clearly, 4 i= C i = 0}. Then apply n = and d = to Theorem 4.. Example 5. Let α 0 and A := x, y R x + + y 0 } =, 0 + B0, B := x, y R x + α 0 }. Let a k k N := u k, v k k N and b k k N := s k, t k k N be defined by Then for every k b k = α, a k+ = + b 0 R, a k+ := P A b k, b k+ := P B a k+. t + α k + t α + + k i=0 + αi α + t +α k +t k i=0 +αi, t + α k + t k i=0 + αi Consequently, a k 0 and b k α, 0 at the rate of k when α = 0. When α 0then A B =, a k 0 and b k α, 0 at the rate of + α k.. Proof. We first claim that 5. b k+ = α, t k+ = α, t k, k. + α + t k By [, Examples 3.7&3. and Proposition 3.7], we have 5. P A x, y =, 0 + x +, y, x +, y }, P B x, y = α, y, x, y / int B. x, y R Let k. Since A B = 0} or A B =, a k / int B. Then by 5., b k = α, v k and then a k+ = P A b k =, 0 + b k+ = P B a k+ = α, + α, v k, + α, v k } =, α, v k + α + vk v k = α, + α + vk t k + α + t k. 8

29 Hence 5. holds. Next we show that 5.3 b k = α, t + α k + t We prove 5.3 by the induction on k. k i=0 + αi, k. By 5., 5.3 holds when k =. Now assume that 5.3 holds when k = p, where p. Now we consider the case of k = p +. By the assumption, we have 5.4 Then by 5., we have Hence 5.3 holds. b p+ = = = b p = α, α, α, α, t + α p + t t p + α + t p t + α p + t + α + + α p + t t + α p + t p i=0 + αi p i=0 + αi Combining5. and 5.3, we have for every k. p i=0 + αi t p i=0. + αi a k+ = P A b k = + α + + α + t +α k +t k i=0 +αi, t + α k + t k i=0 + αi Hence a k 0 and b k α, 0 at the rate of k when α = 0. When α 0, a k 0 and b k α, 0 at the rate of + α k.. Remark 5.3 According to Theorem 4.0,we can only deduce that a k k N in Example 5. converge to 0, 0 and b k k N converge to α, 0 at the rate of at least of k 6. Example 5.4 Let A := x, y R x + + y 0 } B := x, y R x + y 0 }. 9

30 Figure : The iteration commencing at 0,. Let x k, y k k N be defined by x 0, y 0 R, x, y := P A x 0, y 0, x, y := P B x, y, x 3, y 3 := P A x, y,. Note that P A x, y = P B x, y = + + x x + + y, y, x, y R + R ++ x + + y x + x + y, y, x, y R R ++. x + y Figure depicts the algorithm s trajectory with the starting point 0,. Suppose, without loss of generality, that one starts on a point on one the half-circles nearest the other circle. Then the distance from zero for every k N, r k := x k + y k satisfies rk = α k where α k := x k since x k, y k bd A bd B. Hence α k+ = + α k + 4αk. Linearizing, we obtain that w k := 4α k approximately satisfies the logistics equation This can be explicitly solved by writing w k+ w k w k w k+ w k = 30 w k

31 When summing and dividing by N, leads to 5.5 lim N Nw N = lim N = lim N = lim N Nw N Nw 0 N N k=0 w k w N =, since Césaro summability is conservative and w N 0. Hence α k /4k and so x k + y k = r k k. For instance, with α 0 =, N = 0 6, we obtain α N analysis can be performed in the previous example A similar Remark 5.5 According to Theorem 4.0, we can only deduce that a k k N and b k k N in Example 5.4 converge to 0, 0 at the rate of at least of k 6. Basic semialgebraic convex sets described by convex polynomials In general, identifying the exact convergence rate of the cyclic projection method in a direct way can be quite arduous when applied to finitely many basic semialgebraic convex sets. Below, we provide two simple examples to illustrate how the convergence rate depends on the imum degree of the polynomials that described the basic semialgebraic convex sets and on the dimension of the underlying space. Example 5.6 Let A, B be defined by A := x, y R x 0 } B := x, y R y d x 0 }, where d is an even number. Let a k k N, b k k N := x k, y k k N be defined by b 0 := x 0, y 0 R R ++ with b 0, a k+ := P A b k, b k+ := P B a k+. Then for every k N a k+ = 0, y k, b k = y d k, y k, and d y d k+ + y k+ = y k. Consequently, a k k N and b k k N converge to 0 at the rate of at least of k d. 3

32 Proof. Let k N. Since A B = 0}, b k / int A. Since b k bd B and then x k = y d k. Thus a k+ = P A b k = P A x k, y k = 0, y k. Then we have y k 0, b k+ = x k+, y k+ = y d k+, y k+ = P B a k+ = P B 0, y k. Thus y k+ is a minimizer of the function y y d, y 0, y k = y d, y y k = y d + y yy k + y k. Thus d y d k+ + y k+ y k = d y d k+ + y k+ y k = 0. Then we have d y d k+ + d y d k+ = y d k Now, for the function hx := x+ d, we have Note that y k 0. So, for all large k, d y d k+ h x = d for all x R. x + d = hd yd k+ + h0 + h 0d y d k+ d This gives us that for all large k, y d k dd y d k+ y d k+ In other words, there exists l 0 N such that for all l l 0 = dd yd k+. = y d dd. k+ 5.6 y d l+ y d l dd. Let k > l 0. Summing the above relation from l = l 0 to k, we have y d k y d l 0 dd k l 0 + and so, y k d y d + dd k l 0 + = Ok d. l 0 Thence a k k N and b k k N converge to 0 at that rate. Remark 5.7 Similarly, according to Theorem 4.0, we can only deduce that a k k N and b k k N in Example 5.6 converge to 0, 0 at the rate of at least of k d. 3

33 Example 5.8 Let Let a k k N, b k k N be defined by A := x,..., x n R n x B := x,..., x n R n x 4 + n x 4 i 0 } i= n x 4 i 0 }. i= b 0 R, a k+ := P A b k, b k+ := P B a k+. Then a k = O k, b ρn k, 0,..., 0 = O k with ρ ρn n = min 7 n, βn 4 n }. Proof. By the assumption, there exist unique points a 0 bd A, b 0 bd B such that = dista, B = a 0 b 0. Clearly, a 0 = 0, 0,..., 0 and b 0 =, 0,..., 0. Then, the conclusion follows by applying Theorem 4.0 with d = 4. 6 Conclusion and Open Questions In this paper, we studied the rate of convergence of the cyclic projection algorithm applied to finitely many basic semi-algebraic convex sets. We established an explicit convergence rate estimate which relies on the imum degree of the polynomials that generate the basic semi-algebraic convex sets and the dimension of the underlying space. We also examined some concrete examples and compared the actual convergence rate with our estimate. Although our estimate of the convergence rate works for cyclic projection algorithm with finitely many basic semialgebraic convex sets without any regularity condition, the limitation of our approach is that the estimated convergence rate behaves quite badly when the dimension n of the underlying space and the imal degree d are large. Our results have suggested the following future research topics and open questions: The explicit examples Examples 5., 5.4 and 5.6 show that, in general, our estimate of the convergence rate of the cyclic projection algorithm will not be tight. It would be interesting to see how one can sharpen the estimate obtained in this paper and get a tight estimate for the cyclic projection algorithm. In particular, finding the right exponent when each set is defined by convex quadratic functions would be a good starting point. Can we extend the approach here to analyze the convergence rate of the Douglas- Rachford algorithm? Almost nothing is known except for affine sets. These will be our future research topics and will be examined later on. 33

34 Acknowledgments The authors are grateful to Dr. Simeon Reich and the two anonymous referees and the editor for their pertinent and constructive comments. Jonathan Borwein, Guoyin Li and Liangjin Yao were partially supported by various Australian Research Council grants. References [] Y. Alber and S. Reich, An iterative method for solving a class of nonlinear operator equations in Banach spaces, Panamerican Mathematical Journal, vol. 4, pp , 994. [] H. Attouch, J. Bolte, P. Redont and A. Soubeyran, Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka- Lojasiewicz inequality, Mathematics of Operation Research, Vol. 35, no.,pp , 00. [3] H. Attouch, J. Bolte and B. Svaiter, Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Mathematical Programming Vol. 37, no. -, Ser. A, pp. 9-9, 03. [4] A. Auslender and M. Teboulle, Asymptotic cones and functions in optimization and variational inequalities. Springer Monographs in Mathematics. Springer-Verlag, New York, 003. [5] A. Auslender, Existence of optimal solutions and duality results under weak conditions, Mathematical Programming Vol. 88, pp 45-59, 000. [6] J.-B. Baillon, P. L. Combettes, and R. Cominetti, There is no variational characterization of the cycles in the method of periodic projections, Journal of Functional Analysis, vol. 6, pp , 0. [7] H.H. Bauschke, Projection algorithms: results and open problems, Inherently parallel algorithms in feasibility and optimization and their applications, pp. -, 00. [8] H.H. Bauschke and J.M. Borwein, On the convergence of von Neumanns alternating projection algorithm for two sets, Set-Valued Analysis, vol., pp. 85, 993. [9] H.H. Bauschke and J.M. Borwein, Dykstra s alternating projection algorithm for two sets, Journal of Approximation Theory, vol. 79, pp , 994. [0] H.H. Bauschke and J.M. Borwein: On projection algorithms for solving convex feasibility problems, SIAM Review, vol. 38, pp , 996. [] H.H. Bauschke, J.M. Borwein, and A.S. Lewis, The method of cyclic projections for closed convex sets in Hilbert space, Recent developments in optimization theory and nonlinear analysis, Contemporary Mathematic, pp. 38, 997 [] H.H. Bauschke and P.L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Springer, 0. [3] H.H. Bauschke, D.R. Luke, H. M. Phan, and X. Wang, Restricted normal cones and the method of alternating projections, Set-Valued and Variational Analysis, in press, 038v. [4] E.G. Belousov, Introduction to convex analysis and integer programming, Izdat. Moskov. Univ., Moscow, 977 [5] E.G. Belousov, On types of Hausdorff discontinuity from above for convex closed mappings, Optimization, vol. 49, pp , 00. [6] E. G. Belousov and D. Klatte, A Frank-Wolfe type theorem for convex polynomial programs, Computational Optimization and Applications, vol., pp , 00. [7] J. Bochnak, M. Coste and M.F. Roy, Real algebraic geometry, Springer-Verlag, Berlin,