Journal of Complexity. Smale s point estimate theory for Newton s method on

Transcription

1 Journal of Complexity Contents lists available at ScienceDirect Journal of Complexity journal homepage: Smale s point estimate theory for Newton s method on Lie groups Chong Li a,, Jin-Hua Wang b, Jean-Pierre Dedieu c a Department of Mathematics, Zhejiang University, Hangzhou 37, PR China b Department of Mathematics, Zhejiang University of Technology, Hangzhou 33, PR China c Institut de Mathématiques, Université Paul Sabatier, 36 Toulouse Cedex 9, France a r t i c l e i n f o a b s t r a c t Article history: Received February 8 Accepted 6 November 8 Available online 4 November 8 Keywords: Newton s method Lie group The γ -condition Smale s point estimate theory We introduce the notion of the one-parameter subgroup γ -condition for a map f from a Lie group to its Lie algebra and establish α-theory and γ -theory for Newton s method for a map f satisfying this condition. Applications to analytic maps are provided, and Smale s α-theory and γ -theory are extended and developed. Examples arising from initial value problems on Lie group are presented to illustrate applications of our results. 8 Elsevier Inc. All rights reserved.. Introduction Numerical problems arising on manifolds have a lot of applications in many areas of applied mathematics, see for example [ 6]. Such problems can usually be formulated as computing zeroes of mappings or vector fields on a Riemannian manifold. One of the most famous and efficient methods to solve approximately these problems is Newton s method. An analogue of the well-known Kantorovich theorem for Newton s method on Banach spaces cf. [7,8] was obtained in [9] for Newton s method on Riemannian manifolds; while extensions of Smale s α-theory and γ -theory in [,] to analytic vector fields on Riemannian manifolds were done in []. In the recent paper [3], by using the Riemannian connection, we extended the notion of the γ -condition for operators on Banach spaces cf. [4] to vector fields on Riemannian manifolds and established α-theory and γ -theory of Newton s method for the vector fields on Riemannian The first author is supported in part by the National Natural Science Foundation of China grants 6775 and 736 and Program for New Century Excellent Talents in University. The third author is supported by the ANR Gecko, France. Corresponding author. addresses: cli@zju.edu.cn C. Li, wjh@zjut.edu.cn J.-H. Wang, jean-pierre.dedieu@math.ups-tlse.fr J.-P. Dedieu X/$ see front matter 8 Elsevier Inc. All rights reserved. doi:.6/j.jco.8..

2 C. Li et al. / Journal of Complexity manifolds satisfying the γ -condition, which consequently improve the corresponding results in []. Recently, Alvarez, Bolte and Munier introduced in [5] a Lipschitz-type radial function for the covariant derivative of vector fields and mappings on Riemannian manifolds, and established a unified convergence criterion of Newton s method on Riemannian manifolds, improving some results in [] and [3]. On the other hand, some numerical problems such as symmetric eigenvalue problems, optimization problems with equality constraints and ordinary differential equations on manifolds, etc., can be actually considered as problems on Lie groups, see for example [,6 9,4,5]. In particular, motivated by ordinary differential equations on manifolds, Owren and Welfert introduced in [9] two kinds of Newton s methods for finding zeros of f, where f is a map from a Lie group to its Lie algebra, and showed that, under classical assumptions on f, these two methods converge locally to a zero of f at a quadratic rate. Recall that a Lie group is a Hausdorff topological group equipped with a structure of analytic manifold such that the group product and the inversion are analytic operations cf. [,]. As is well-known, the most important construction associated with a Lie group and its Lie algebra is the exponential map or equivalently, the one-parameter semigroup on the Lie group. Naturally, a Lie group with the left invariant metric becomes a Riemannian manifold. However, generally, the oneparameter semigroup is not a geodesic even for the left invariant metric. As showed in [9], Newton s methods presented in [9] are only dependent on the one-parameter semigroup but not on the affine connections. This means that Newton s methods on Lie groups are completely different from Newton s method on the corresponding Riemannian manifolds, which is defined via the Riemannian connection cf. [5,,9,3]. In the present paper, we study the convergence issue for one of two Newton s methods presented in [9]. Newton s method considered here is defined as follows with initial point : x n+ = x n exp df x n f x n for each n =,,...,. where df is the derivative of f and is defined in terms of the exponential map and independent of the Riemannian metric. We introduce the notion of the one-parameter subgroup γ -condition for maps f from a Lie group to its Lie algebra. Unlike the notion of the γ -condition on Riemannian manifolds, this condition is defined via the one-parameter subgroup, instead of the Riemannian connection, and so is independent of the Riemannian connection. We establish the generalized α- theory and the generalized γ -theory for Newton s method for maps from a Lie group to its Lie algebra satisfying the γ -condition. We also show that any analytic map on a Lie group satisfies this kind of the one-parameter subgroup γ -condition and so the classical Smale s α-theory and γ -theory for analytic maps on Banach spaces are extended and developed to the setting of Lie groups. Moreover, as an application to the initial value problem on Lie groups studied in [8,9], two examples are presented in the last section.. Notions and preliminaries Most of the notions and notations which will be used in the present paper are standard, see for example [,,]. The dimension of a Lie group G is that of the underlying manifold, and we shall always assume that it is finite. The symbol e designates the identity element of G. Let G be the Lie algebra of the Lie group G which is the tangent space T e G of G at e, equipped with the Lie bracket [, ] : G G G. For y G fixed, let L y : G G be the left multiplication defined by L y z = y z for each z G.. Then the left translation is defined to be the differential L y e of L y at e, which clearly determines a linear isomorphism from G to the tangent space T y G. The exponential map exp : G G is certainly the most important construction associated with G and G, and is defined as follows. Given u G, let σ u : R G be the one-parameter subgroup of G determined by the left invariant vector field X u : y L y e u; i.e., σ u satisfies that σ u = e and σ t = u X uσ u t = L σu t e u for each t R..

3 3 C. Li et al. / Journal of Complexity The value of the exponential map exp at u is then defined by expu = σ u. Moreover, we have and exptu = σ tu = σ u t for each t R and u G.3 expt + su = exptu expsu for any t, s R and u G..4 In general, the exponential map is not surjective. Hence, in general, for any two points x, y G, there may be no one-parameter subgroup of G to connect them. However, the exponential map is a diffeomorphism on an open neighborhood of G. Let N := B, ρ be the largest open ball containing such that exp is a diffeomorphism on N and set Ne = expn. Then for each y Ne, there exists a unique v N such that y = expv. When G is Abelian, exp is also a homomorphism from G to G, i.e., expu + v = expu expv for all u, v G..5 In the non-abelian case, exp is not a homomorphism and, by the Baker Campbell Hausdorff BCH formula cf. [, p.4],.5 must be replaced by expw = expu expv for all u, v in a neighborhood of G where w is defined by w := u + v + [u, v] + [u, [u, v]] + [v, [v, u]] +..7 In what follows, we will make use of the left invariant Riemannian metric on Lie group G produced by an inner product on G, see for example []. Let, e be an inner product on G. Then the left invariant Riemannian metric is defined by u, v x = L x x u, L x x v e, for each x G and u, v T x G..8 Let x be the norm associated with the Riemannian metric, where the subscript x is sometimes omitted if there is no confusion. Then L y e is a linear isometry from G to T yg for each y G. Let x, y G be distinct and let c : [, ] G be a piecewise smooth curve connecting x and y. The arc-length of c is defined by lc := c t ct dt. The distance from x to y is defined by dx, y := inf c lc, where the infimum is taken over all piecewise smooth curves c : [, ] G connecting x and y when such curves exist, and dx, y := + otherwise. We now introduce the notion of a piecewise one-parameter subgroup, which will play a basic role. Definition.. A curve c : [, ] G connecting x and y is called a piecewise one-parameter subgroup connecting x and y if there exist m + real numbers {t i } m+ i=, with = t < t < < t m < t m+ =, and m + elements {w i } m i= G such that ct = ct i expt t i w i for each t [t i, t i+ ] and i =,,..., m..9.6 Remark.. Clearly, a curve connecting x and y is a piecewise one-parameter subgroup if and only if there exist m+ elements u,..., u m G such that the curve can be expressed as c : [, m+] G satisfying c = x, cm + = y and ct = ci expt iu i for each t [i, i + ] and i =,,..., m.. The following proposition regarding the arc-length of a piecewise one-parameter subgroup will be useful.

4 C. Li et al. / Journal of Complexity Proposition.. Let x, y G, m N and let c : [, m+] G be a piecewise one-parameter subgroup connecting x and y as expressed in.. Then and lc = u i i= dx, y u i. i=.. Proof. Obviously, by the definition of the distance,. is direct from.. Below we will prove.. To this end, let i =,,..., m. Since c t = L ci expt i e u i for each t [i, i + ].3 and L ci expt i e is an isometry, one has that Hence c t = u i for each t [i, i + ]. lc = i= i+ and. is proved. i c t dt = u i.4 i= Throughout the whole paper we will assume that G is complete. Thus each connected component of G, d is a connected and complete metric space. For a Banach space or a Lie group Z, we use B Z x, r and B Z x, r to denote respectively the open metric ball and the closed metric ball at x with radius r, that is, B Z x, r = {y Z : dx, y < r}, B Z x, r = {y Z : dx, y r}. We usually omit the subscript if no confusion occurs. Assume that f : G G is a C -map and let x G. We use f x to denote the differential of f at x. Then, by [, P. 9] the proof there for smooth maps is still valid for C -maps, for each x T x G and any non-trivial smooth curve c : ε, ε G with c = x and c = x, one has d f x x = f ct..5 dt In particular, f x x = t= d dt f x exptl x x x Define the map df x : G G by d df x u = f x exptu dt Then, by.6, df x = f x L x e and so df x is linear. Moreover, by definition, we have for all t d t= t= for each x T x G..6 for each u G..7.8 dt f x exptu = df x exptuu for each u G.9

5 3 C. Li et al. / Journal of Complexity and f x exptu f x = t df x expsu uds for each u G.. Let k be a positive integer and assume further that f : G G is a C k -map. Define the map d k f x : G k G by k d k f x u u k = f x exp t k u k exp t u. t k t t k = =t = for each u,..., u k G k. In particular, d k d k f x u k = f x exp tu dtk for each u G.. t= Let i k. Then, in view of the definition, one has d k f x u u k = d k i d i f u u i x u i+ u k for each u,..., u k G k..3 In particular, for fixed u,..., u i, u i+,..., u k G, d i f x u u i = d d i f u u i..4 x This implies that d i f x u u i u is linear with respect to u G and so is d k f x u u i uu i+ u k by.3. Consequently, d k f x is a multilinear map from G k to G because i k is arbitrary. Thus we can define the norm of d k f x by d k f x = sup{ d k f x u u u k : u,..., u k G k with each u j = }..5 For the remainder of the paper, we always assume that f is a C -map from G to G. Then taking i = in.4 we have d f z vu = d df v z u for any u, v G and each z G..6 Thus,. is applied with df v in place of f for each v G to conclude the following formula. df x exptu df x = t d f x expsu uds for each u G and t R..7 The γ -conditions for nonlinear operators in Banach spaces were first introduced and explored by Wang [4,3] to study Smale s point estimate theory. Below we define the one-parameter subgroup γ -condition for a map f from a Lie group to its Lie algebra in view of the map d f. Let r > and γ > be such that γ r. Definition.. Let G be such that df exists. f is said to satisfy: i the one-parameter subgroup γ -condition at on B, r if, for any x B, r and u B G, r satisfying x = expu, d f x γ u 3 ;.8 ii the pieces one-parameter subgroup γ -condition at on B, r if, for any x B, r and any piecewise one-parameter subgroup c connecting and x with its arc-length less than r, d f x γ lc..9 3

6 C. Li et al. / Journal of Complexity Clearly, the pieces γ -condition implies the γ -condition. The following lemma will be used frequently in what follows. For notational simplicity, we make use of the real-valued function ψ defined by [ ψs = 4s + s for each s,..3 Note that ψ is strictly monotonic decreasing on [,. Lemma.. Let r and let x G be such that df exists. Let x B, r be such that there exists a piecewise one-parameter subgroup c connecting and x with its arc-length less than r. Suppose that f satisfies the pieces γ -condition at on B, r. Then df x exists and x df x γ lc..3 ψγ lc Proof. By assumption, there exist m + elements u,..., u m G and a curve c : [, m + ] G such that c =, cm + = x and. holds. Then, by.7, one has for each i m, df ci expui df ci d f ci exptui u i dt γ u i 3 dt u j + t u i i j= = i γ u j γ j=..3 u j i j= Consequently, df x I G = df cm expum df x df ci expui df ci i= γ lc <. Thus the conclusion follows from the Banach lemma and the proof is complete. We end this section with the notion of convergence of the sequence on Lie groups, see for example [9]. Let G. Definition.3. Let {x n } n be a sequence of G and x G. Then {x n } n is said to be: i convergent to x if for any ε > there exists a natural number K such that x x n Ne and exp x x n ε for all n K ; ii quadratically convergent to x if { exp x x n } is quadratically convergent to ; that is, {x n } n is convergent to x and there exist a constant q and a natural number K such that exp x x n+ q exp x x n for all n K. Note that convergence of a sequence {x n } n in G to x in the sense of Definition.3 above is equivalent to lim n + dx n, x =.

7 34 C. Li et al. / Journal of Complexity Generalized α-theory The majorizing function h, which is due to Wang [4], will play a key role in this section. Let β > and γ >. Define γ t ht = β t + γ t for each t < γ. 3. Let {t n } denote the sequence generated by Newton s method for h with initial value t =, that is, t n+ = t n h t n ht n for each n =,, Then we have the following proposition which can be found in [4,3]. Proposition 3.. Suppose that α := γ β 3. Then the zeros of h are r = + α + α 8α, r = + α + + α 8α 4γ 4γ and they satisfy β r Moreover, if α < 3, where β r. 3.4 < t n+ t n ν n β for each n =,,..., 3.5 ν = α + α 8α α + + α 8α. 3.6 In the remainder of this section, we assume that G is such that df β := f. exists and let Theorem 3.. Suppose that α := βγ 3 and that f satisfies the pieces γ -condition at on B, r. Then Newton s method. with initial point is well-defined and the generated sequence {x n } converges to a zero x of f in B, r. Moreover, if α < 3, dx n+, x n ν n β for each n =,,,..., 3.7 where ν is given by 3.6. Proof. Recall from. that x n+ = x n exp df x n f x n n =,, By assumptions, v := df f is well-defined and v = β = t t. Hence, x is well-defined and dx, t t. We now proceed by mathematical induction on n. For this purpose, assume that v n := df x n f x n is well-defined for each n k and dx n+, x n v n t n+ t n for each n =,,..., k. 3.9

8 Then x k = expv expv expv k and C. Li et al. / Journal of Complexity k k v n t n+ t n = t k < r n= n= 3. thanks to 3.9. Therefore, df x k exists by Lemma. and so v k is well-defined. Furthermore, by.3, we have that γ k v n x k df x ψ n=. 3. v n γ k n= Noting that t ψt = h t/γ for each t,, it follows x k df x k h h t k 3. v n n= because h t is monotonically increasing on [,. Applying. and.7, we deduce that f x k = f x k f x k df xk v k = = τ Observing that h t = df xk expτv k v k dτ df xk v k f x k d f xk expsv k v k dsdτ. γ t 3, one has from.9 that τ d f xk expsv k v k dsdτ τ h t k + s v k v k dsdτ τ h t k + st k t k t k t k dsdτ = ht k ht k h t k t k t k = ht k, 3.3 where the last equality holds because ht k h t k t k t k = by 3. with n = k. Consequently, combining 3. and 3.3 yields that v k = df x k f x k x k df x f x k h t k ht k = t k+ t k. Hence, x k+ is well-defined and, by Proposition., dx k+, x k v n t k+ t k. 3.4 This completes the proof.

9 36 C. Li et al. / Journal of Complexity Generalized γ-theory For the whole section, we always assume that x G is such that f x = and df x exists. The main purpose of this section is to give estimates of the convergence ball of Newton s method on G around the zero x of f. Recall that the function ψ is defined by [ ψs = 4s + s for each s,. The following lemma gives an estimates of the quantity f. Lemma 4.. Let r = x exp v and let Bx, r be such that for some v G with v < r. Suppose that f satisfies the γ -condition at x on Bx, r. Then df and 4. exists f γ v v. 4. ψγ v Proof. By Lemma., df df x exists and γ v. 4.3 ψγ v On the other hand, it follows from 4.,. and.7 that f = f f x df x v + df x v = = τ df x expτvvdτ df x v + df x v d f x expsvv dsdτ + df x v. Hence the assumed γ -condition is applied to get that τ x f x d f x expsvv dsdτ + v = τ v γ v. Combining 4.3 and 4.4 yields that sγ v 3 v dsdτ + v f df x x f γ v ψγ v v. This completes the proof. 4.4 Let a = be the smallest positive root of the equation a ψa =

10 C. Li et al. / Journal of Complexity Theorem 4.. Let < r a γ and set r = + γ r + r. Suppose that f satisfies the pieces ψγ r γ -condition at x on Bx, r. Let G be such that there exists v G satisfying = x exp v and v < r. 4.6 Then Newton s method. with initial point is well-defined and converges to a zero, say y, of f in Bx, r. Proof. By Lemma 4., df Set γ = β := exists and f γ v v. 4.7 ψγ v γ ψγ v γ v. Then ᾱ := β γ γ v ψγ v a ψa = because the function t t ψt is strictly monotonic increasing on [,. Let r = + ᾱ + ᾱ 8ᾱ γ Then, by 4.8, it follows from 3.4 that β r + β. 4. To apply Theorem 3. we have to show the following assertion: there exists ˆr r such that f satisfies the pieces γ -condition at on B, ˆr. For this purpose, let ˆr = + γ r r. ψγ r 4. Since v < r and the function t t ψt is strictly monotonic increasing on [, a ], we have ˆr = + γ r ψγ r r + γ v v + β r 4. ψγ v thanks to 4.7 and 4.. Below we shall show that f satisfies the pieces γ -condition at on B, ˆr. To do this, let x B, ˆr and let c : [, ] G be a piecewise one-parameter subgroup connecting x and so c = x, c = with lc < ˆr. Define the curve ĉ : [, ] G by ĉt = { ct t [, ], expt v t [, ]. 4.3 Then, in view of 4.6, ĉ is a piecewise one-parameter subgroup connecting x and x. Moreover, by 4., lĉ = lc + v < ˆr + r = r. Since f satisfies the pieces γ -condition at x on Bx, r by the assumption, it follows that x d f x γ lĉ = 3 γ v + lc

11 38 C. Li et al. / Journal of Complexity Since < ψt < for t,, we have γ = γ ψγ v γ v Consequently, it follows from Lemma. and 4.4 that d f x = df x d f x x γ γ v. 4.5 γ v ψγ v γ v + lc 3 γ v 3 = ψγ v γ v γ v γ lc 3 γ = γ γ v lc 3 γ γ lc 3, where the last inequality holds by 4.5. Therefore, the assertion stands. Thus, Theorem 3. is applicable and Newton s method. with initial point converges to a zero y of f in B, r. Hence, it follows from 4. that dx, y dx, + d, y < r + r r + ˆr = r, which completes the proof. In particular, taking r = a γ in Theorem 4., one has the following corollary. Note the following elementary inequality: + a + a <. 4.6 ψa Corollary 4.. Suppose that f satisfies the pieces γ -condition at x on Bx,. Let G be such that there exists v G satisfying = x exp v and γ v < a. 4.7 Then Newton s method. with initial point is well-defined and converges to a zero, say y, of f in Bx,. In general, we do not know whether the solution y is equal to x in Theorem 4.. The following corollary provides an estimate of the convergence domain depending only on the diffeomorphism ball around the origin, which guarantees the convergence to x of Newton s method with initial point from the domain. Recall that a =.885 is given by 4.5 and set ψa s = + a + ψa. 4.8 Corollary 4.. Suppose that f satisfies the pieces γ -condition at x on Bx,. Let ρ > be the { } largest number such that Be, ρ expb, a and let r = min γ, s ρ. Write Nx, r := x expb, r. Then, for each Nx, r, Newton s method. with initial point is well-defined and converges quadratically to x.

12 C. Li et al. / Journal of Complexity Proof. Let Nx, r. Then there exists v G such that = x exp v and { } a v < r = min γ, s ρ. 4.9 Note that the function t t ψt is strictly monotonic increasing on [, a ]. It follows that + γ r ψγ r + + a ψa This together with 4.6 implies that + γ r + r <. ψγ r Thus Theorem 4. is applicable and Newton s method. with initial point is well-defined and converges to a zero, say y, of f in B x, + γ r + r. Hence, by 4.9 and 4., one has ψγ r that dy, x + γ r < + r + a + ψa r ρ. ψγ r ψa This means that there exists u G such that u < and y = x exp u. By.7 and.9, one has df x df x expτ udτ I = = df x τ τ = γ u γ u <, df x expτ u df x dτ df x d f x expsuudsdτ γ s u 3 u dsdτ where the last inequality holds because γ u < <. Thus, it follows from the Banach lemma that df x df x expτ udτ is invertible and so is df x expτ udτ. Note that df x expτ udτ u = f y f x =. We get that u = ; hence y = x and the proof is complete. Recall that in the special case when G is a compact connected Lie group, G has a bi-invariant Riemannian metric cf. [, p. 46]. Below, we assume that G is a compact connected Lie group and endowed with a bi-invariant Riemannian metric. Then an estimate of the convergence domain with the same property as in Corollary 4. is described in the following corollary. Corollary 4.3. Let G be a compact connected Lie group and endowed with a bi-invariant Riemannian metric. Suppose that f satisfies the pieces γ -condition at x on Bx,. Let x G be such that there exists v G satisfying = x exp v and γ v < a. 4.

13 4 C. Li et al. / Journal of Complexity Then Newton s method. with initial point is well-defined and converges quadratically to x. Proof. By Corollary 4., Newton s method. with initial point is well-defined and converges to a zero, say y, of f in Bx,. Thus, there is a minimizing geodesic c connecting x y and e. Since G is a compact connected Riemannian manifold and endowed with a bi-invariant Riemannian metric, it follows from [5, p. 4] that c is a one-parameter subgroup of G. Consequently, there exists u G such that y = x exp u and u = dx, y <. Since df x df x expτ udτ I = = df x τ τ = γ u γ u <, df x expτ u df x dτ df x d f x expsuudsdτ γ s u 3 u dsdτ where the last inequality holds because γ u < <. Thus, from the Banach lemma, it follows that df x df x expτ udτ is invertible and so is df x expτ udτ. As df x expτ udτ u = f y f x =, u = and so y = x. This completes the proof. 5. Applications to analytic maps Throughout this section, we always assume that f is analytic on G. For x G such that df x exists, we define γ x := γ f, x = sup i df x d i f x i! i. 5. Also we adopt the convention that γ f, x = if df x is not invertible. Note that this definition is justified and, in the case when df x is invertible, γ f, x is finite by analyticity. The Taylor formula for a real-valued function on G can be found in [, p. 95]; and its extension to the map from G to G is trivial. Proposition 5.. Let < r γ f,x. Let y Bx, r be such that y = x expv with v B G, r. Then, d k f y = j= j! dj+k f x v j for each k =,, Recall that G is such that df exists. Below, we will show that an analytic map satisfies the pieces γ -condition at in B, r with γ := γ f, and r =. The following lemma is clear, see for example [].

14 C. Li et al. / Journal of Complexity Lemma 5.. Let k be a positive integer. Then k + j! t j = for each t,. k! j! t k+ j= Proposition 5.. Let γ = γ f, and let r =. Then f satisfies the pieces γ -condition at on B, r. Proof. Let x B, r and let c : [, m + ] G be a piecewise one-parameter subgroup with lc < r and {u, u,..., u m } G as stated in Remark.. Write y =, y m+ = x and y i+ = y i expu i for i =,,..., m. 5.3 We shall verify that, for each i =,,,..., m +, and d j f yi y i df x j!γ j j+ for each j =,,..., 5.4 i γ x u k k= i γ x ψ k= γ x i u k 5.5 u k k= γ x γ f, y i i γ x u k ψ k= γ x i k=, 5.6 u k where we adopt the convention that k= u k =. To do this, let i =,,,..., m and consider the following inequality: j!γ j d j f yi+ df d j x f yi j!γ j x j+ j i i γ x u k γ x u k We claim that 5.4 and k= k= 5.8 To show the claim, note that tψt > t for each t [,, 5.9 := γ i x k= u k. Then t + and t,. It follows from 5.6 and 5.9 that which can be proved by elementary differential techniques. Set t γ x u i γ x lc < γ x u i γ f, y i u i i γ x u k ψ k= γ x i k= t < <. 5. tψt u k

15 4 C. Li et al. / Journal of Complexity Thus, Proposition 5. is applicable and df d j f yi+ df d j f yi = By assumption 5.4, d p+j f yi p= dfx p! d p+j f yi u p i for each j =,, p + j!γ p+j p+j+ for each p =,,... and j =,, i γ x u k k= This, together with 5., implies that d j f yi+ df d j p + j!γ p+j x f yi p! p+j+ u i p. 5.3 i p= γ x u k γ x u i Using Lemma 5. with t = γ i to calculate the quantity on the right-hand side of 5.3, x k= u k we see that 5.7 holds and claim 5.8 is proved. We now use mathematical induction to verify that assertions hold for each i =,,..., m. Clearly, hold for i = by the definition of γ x in 5.. Assume that hold for all i n for some integer n. Then 5.7 holds for all i n. It follows that k= d j f yn+ d j f yn+ df d j f yn + d j f yn j!γ j n j+ 5.4 γ x u k k= holds for each j =,,... and 5.4 is true for i = n+. To show that 5.5 and 5.6 hold for i = n+, note by 5.7 with j = that n df yn+ I G k= df yk+ df df yk n γ x k= <, u k where the last inequality is valid because γ n x k= u k γ x lc <. Thus, by the Banach lemma, df y n+ exists and y n+ df x n γ x u k ψ γ x k=, 5.5 n u k k= that is 5.5 holds for i = n +. This and 5.4 imply that d j f yn+ df df x y n+ j! dfy n+ df x d j f yn+ j! ψ γ x n k= u k k= γ x n γ x u k j 5.6

16 holds for each j =,,.... Therefore, df y γ f, y n+ = n+ d j f yn+ sup j j! γ f, n γ x u k C. Li et al. / Journal of Complexity k= k= sup j j ψ γ x n k= γ f, = n n γ x u k ψ γ x k= j u k u k because the supremum is attained at j = as < ψγ x n k= u k < by 5.9. Hence, 5.6 is true for i = n +. Thus, hold for each i =,,..., m. In particular, taking i = m + and j = in 5.4, it follows that d f x γ x x k= x 3 = γx lc 3 u k because lc = m k= u k by.. This completes the proof. 5.7 By Proposition 5., the results in previous sections are applicable. Hence the following corollaries are direct. Recall that G is such that df exists, β = f, γ = γ f, and α = βγ. Corollary 5.. If α 3, then Newton s method. with initial point is well-defined and the generated sequence {x n } converges to a zero x of f in B, r. Moreover, if α < 3, dx n+, x n ν n β for all n =,,..., where ν and r are given by 3.6 and 3.3 respectively. Corollary 5.. If α , then Newton s method. with initial point is well-defined and the generated sequence {x n } converges to a zero x of f in B, r. Moreover, n dx n+, x n β for all n =,,.... Proof. It follows from 3.4 that r. Thus, by Proposition 5., f satisfies the pieces f, γ -condition at on B, r with γ = γ f,. Note also that α < 3. Theorem 3. is applicable to concluding that dx n+, x n ν n β,

17 44 C. Li et al. / Journal of Complexity where ν = α + α 8α α + + α 8α. Therefore, we have that ν because ν increases as α does on [, ] and the value of ν at 4 α = is. 4 Recall that a and s are defined respectively by 4.5 and 4.8. Assume as in the previous section that x G is such that f x = and df x exists. Corollary 5.3. Let G be such that = x exp v for some v G with γ f, x v < a. Then Newton s method. with initial point is well-defined and converges to a zero of f in B x, f,x. Corollary 5.4. Let ρ > be the largest number such that Be, ρ expb, f,x and let { } a r = min γ f,x, s ρ. Write Nx, r := x expb, r. Then, for each Nx, r, Newton s method. with initial point is well-defined and converges quadratically to x. Corollary 5.5. Suppose that G is a compact connected Lie group and endowed with a bi-invariant Riemannian metric. Let G be such that = x exp v for some v G with γ f, x v < a. Then Newton s method. with initial point is well-defined and converges quadratically to x. 6. Examples This section is devoted to an application to initial value problems on the special orthogonal group SON, R. The following two examples have been considered in [9] by Owren and Welfert. Let N be a positive integer and let I N denote the N N identity matrix. Following [,6], let G be the special orthogonal group under standard matrix multiplication, that is, G = SON, R := {x R N N x T x = I N and det x = }. 6. Then its Lie algebra is the set of all N N skew-symmetric matrices, that is, G = son, R := {v R N N v T + v = }. 6. We endow G with the standard inner product u, v = tru T v for any u, v G; 6.3 hence the corresponding norm is the Frobenius norm. Note that [u, v] = uv vu and [u, v], w = u, [w, v] for any u, v, w G. One can easily verify cf. [, p. 4] that the left invariant Riemannian metric induced by the inner product in 6.3 is a bi-invariant metric on G. Moreover, the exponential exp : G G is given by expv = k v k k! for each v G, and its inverse is the logarithm cf [6, p. 34]: exp z = k k z I N k k for each z G with z I N <. 6.4 Let g : G R G be a differential map and x a random starting point. Consider the following initial value problem on G studied in [8,9]: { x = x gx, t x = x 6.5.

18 C. Li et al. / Journal of Complexity The application of one step of the backward Euler method on 6.5 leads to the fixed-point problem x = x exphgx, h, where h represents the size of the discretization step. Let f : G G be defined by f x = exp x x hgx, h for each x G. Thus, solving 6.6 is equivalent to finding a zero of f. To apply our results obtained in the previous sections, we have to estimate the norms of df x. To do this, write w = exp x and assume x = exp v for some v G. Let x := exp v exp v exp v m for some v, v,..., v m G m with v i= i <. Since 4 e t 5 4 t for each t [, 4 one can use mathematical induction to prove that x I N e Consequently, x x I N 5 4 v i ], 6.7 v i. 6.8 v i < 5 < i= Thus, by definition and using 6.4, one has that, for each u G, Since dw x u = k k k x x I N i x xux x I N k i i= x xu = x xu u + u x x I N + u, k. 6. it follows from 6. that k x x I N + x x I N k dw x u u + x x I N u k Hence, thanks to 6.9, dw x I G k = x x I N x x I N u. x x I N x x I N where I G is the identity on G. Similarly, one has d w x k v i i=, v i i= k x x I N + x x I N k + kk x x I N + x x I N k = 3 + x x I N x x I N. k

19 46 C. Li et al. / Journal of Complexity Combining this and 6.9 gives m v 4 i d i= w x m v 4 i i= In the following examples, we consider two special functions g which were used in [8,9]. Example 6.. Let x = exp v with v G such that v 3 and let g be the function defined by where gx, t = sintxx 5x sintxx 5x T for each x, t G R, sin tx = i i txi i! for each x, t G R. Consider the special case when h =. Let = I N and γ =. Below, we will show that f satisfies the pieces γ -condition at on B,. To do this, let x := exp v 5 exp v exp v m for some v, v,..., v m G m with v i <. Then, by 6., we have that 5 dw x I G v 4 5 v 7 because v 3 <. Since m v i= i + m v i < 3, it follows from 6. that 5 d w x 5 4 On the other hand, note that m v i v i gx, = i xi x i T 5 i xi+ x i+ T. 6.5 i! i! i Then, by definition, dg x = 6 cos 6 sin I G. Hence dg = It follows from 6.3 that i 6 cos 6 sin I G and I G dg x = I G dg x df x I G dg x = I G dg x dw x I G + 6 cos + 6 sin 7 <. Thus the Banach lemma is applied to conclude that +6 cos +6 sin +6 cos +6 sin 7 Let l, k be integers and let θ : G G be defined by θx = x l x l T for each x G. + 6 cos + 6 sin I G

20 C. Li et al. / Journal of Complexity Then, for each k-tuple vector u k,..., u G k, one can use mathematical induction to verify that d k θ x u k u is the sum of l k items, each of which has the form u i u ik u i u ik T, where {i,..., i k } is a rearrangement of {,..., k}. Consequently, d k θ x l k. Note that, for each i, i k i! i + k i + i i! Then, by elementary calculations, we have that i i k i! l l + k l + l! Similarly, i + k i! k +! k e. i + i + k ii. i! = k! k e. Combining 6.8 and 6.5, together with above two inequalities, gives the following estimate i i 6.8 i k d k g x 4 i! + i + k i! k! + 5 k +! k e. 6.9 This together with 6.6 implies that dg x sup d k g x k 7 + 5k k e k 68e sup k k! k 6 cos + 6 sin 6 cos + 6 sin. Thus applying Proposition 5., one has and dg d g x 36e 6 cos +6 sin 68e 6 cos +6 sin d g x dg x dg d g x 3 6. v i < 36e 68e 6 cos +6 sin v i thanks to 6.6 and 6.7. Combining this with 6.4 and 6.7 yields that d f x d w x + d g x 3 + v i v i 3 v i v i

21 48 C. Li et al. / Journal of Complexity This shows that f satisfies the pieces γ -condition at on B, 5. Moreover, r < 5, so that f satisfies the pieces γ -condition at on B, r. Since f = v and v 3, it follows from 6.7 that α = γ f γ v 3. Thus, Theorem 3. is applicable to concluding that the sequence generated by. with initial point = I N converges to a zero x of f. To illustrate the application of Corollary 4.3, we take x = I N, that is, f : G G is defined by f x = exp x sinxx x + sinxx x T for each x G. Then x := I N is a zero of f. Furthermore by 6. and 6.6, dw x = I G and df x = + 6 cos + 6 sin I G. 6. Let x := exp v exp v exp v m for some v, v,..., v m G m with v i <. Then, by 6. 5 noting that v =, one has that 3 + d 4 w x v 4 i Note that x = = I N. Then, using and 6.6, one can verify with almost the same argument as we did for 6. that x d f x 4 3. v i Hence, f satisfies the pieces γ -condition at x on Bx, as < with γ =. Take 5 = x exp v with v G and v < a, where a =.885 is given by 4.5. Then γ v < a. Corollary 4.3 is applicable to concluding that the sequence generated by. with initial point is well-defined and converges quadratically to x. Example 6.. Let x = exp v with v G such that v 3 and let g be the function defined 7 by gx, t := gx = diagdiagx,, diagdiagx,, for each x, t G R, where diag is in Matlab notation. Consider the special case when h =. Let x 5 = I N and γ = 7. We claim that f satisfies the pieces γ -condition at on B, 3. To show the claim, let x := exp v exp v exp v m for some v, v,..., v m G m with v i < 3. Note by [9] that Since dg x u = gxu and d g x u u = gxu u for any u, u G. 6.4 gxu u xu u x I N + u u, 6.5 it follows from 6.8 and the fact that m v i < 3 that d g x x I N v i Define E ij := b lk N N R N N for each pair i, j with i < j N, where b lk is equal to if l, k = i, j, if l, k = j, i and otherwise. Write M = N N. Then {E, E,..., E M } = {E, E 3,..., E N,N, E 3, E 4,..., E N,N,..., E N } is a basis of son, R. Let son, R be endowed

22 C. Li et al. / Journal of Complexity with the -norm, namely, u = a,..., a M for each u = im a ie i son, R. Then it is a routine to verify that u = u for each u son, R. Moreover, by 6.4, IN dg x E,..., E M = E,..., E M. 6.7 Hence where C = I G 5 dg E,..., E M = E,..., E M C, 5 4 I N I M N+ I G 5 dg u = I G 5 dg u. Let u = im a ie i son, R. Then = Ca,..., a M T C u = C u, where C = λ max C T C. Consequently, I G 5 dg C = Since v 3 < and m v 7 4 i < 3, it follows from 6. and 6. that and dw x I G v 4 5 v d w x m < v i 7 Noting df x I G dg 5 = dw x I G, one has from 6.8 and 6.9 that I G 5 dg df x I G 5 dg <. Hence, by the Banach lemma, we have I G 5 dg 49 I G dg dfx 5 I G dg 5 x Then, combining 6.6, 6.3 and 6.3 yields that d f x d w x + 5 df d g x < v i v i =

23 5 C. Li et al. / Journal of Complexity Thus, the claim stands. Moreover, r < 3, so that f satisfies the pieces γ -condition at x on B, r. Noting that f = v, by 6.3, we have α = γ f γ v < 3. 7 Hence, Theorem 3. is applicable to concluding that the sequence generated by. with initial point = I N converges to a zero x of f. To illustrate the application of Corollary 4.3, we take x = I N, that is, f : G G is defined by f x = exp x gx for each x G Clearly, x := I N satisfies that f x =. Furthermore, dw x = I G and df x = dw x dg 5 x = I G dg 5 x. This together with 6.8 implies that dfx 5. Let x = exp v 4 exp v exp v m with some v, v,..., v m G m such that v i <. Then, by 6., d 4 w x v 4 i because x = I N and so v =. On the other hand, by 6.8 and 6.5,we get that d g x x I N + 5 v i Combining this with 6.33, together with the fact that x 5, one can prove that 4 x d f x x d w x + 5 df x d g x < v i Hence, f satisfies the pieces γ -condition at x on Bx, with γ = 5 as <. Take 5 = x exp v with v G and v < a, where a 5 =.885 is given by 4.5. Then γ v < a. Corollary 4.3 is applicable to concluding that the sequence generated by. with initial point is well-defined and converges quadratically to x. 7. Concluding remarks In the present paper, we have introduced the notion of the γ -condition for maps from a Lie group to its Lie algebra and have established the generalized α-theory and the generalized γ -theory for Newton s method on Lie group for the maps satisfying the γ -condition. Applications to analytic maps on Lie groups extend and develop the classical Smale s point estimate theory. The main feature of our results for Newton s method on Lie groups is on two folds: one is that the generalized α-theory, without a prior assumption of existence of the zeros, provides a convergence criterion depending on the information around the initial point, which has not been studied before on Lie groups; the other is that the generalized γ -theory provides some clear estimates for the convergence domains, while the corresponding results in [9] just ensure the existence of the convergence domains. In [3], we defined the notion of γ -condition for maps on Riemannian manifolds and established the α-theory and the γ -theory for Newton s method on Riemannian manifolds for the maps satisfying the γ -condition. Although a Lie group with the left invariant Riemannian metric is also a Riemannian manifold, the differences between our results in the present paper and the corresponding ones reported in [3] are clear because, as explained in the introduction section, Newton s method and

24 C. Li et al. / Journal of Complexity the γ -condition on Riemannian manifolds in [3] are completely different from Newton s method. and the γ -condition on Lie groups introduced in the present paper. Moreover, the convergence criterion and the estimates in [3] depend on the curvature of the underlying Riemannian manifold, but in the present paper, they do not. References [] R. Adler, J.P. Dedieu, J. Margulies, M. Martens, M. Shub, Newton method on Riemannian manifolds and a geometric model for human spine, IMA J. Numer. Anal. 3. [] A. Edelman, T.A. Arias, T. Smith, The geometry of algorithms with orthogonality constraints, SIAM J. Matrix Anal. Appl [3] D. Gabay, Minimizing a differentiable function over a differential manifold, J. Optim. Theory Appl [4] S.T. Smith, Optimization techniques on Riemannian manifolds, in: Fields Institute Communications, vol. 3, American Mathematical Society, Providence, RI, 994, pp [5] S.T. Smith, Geometric optimization method for adaptive filtering, Ph. D. Thesis, Harvard University, Cambridge, MA, 993. [6] C. Udriste, Convex Functions and Optimization Methods on Riemannian Manifolds, in: Mathematics and Its Applications, vol. 97, Kluwer Academic, Dordrecht, 994. [7] L.V. Kantorovich, On Newton method for functional equations, Dokl. Acad. Nauk [8] L.V. Kantorovich, G.P. Akilov, Functional Analysis, Pergamon, Oxford, 98. [9] O.P. Ferreira, B.F. Svaiter, Kantorovich s theorem on Newton s method in Riemannian manifolds, J. Complexity [] L. Blum, F. Cucker, M. Shub, S. Smale, Complexity and Real Computation, Springer-Verlag, New York, 997. [] S. Smale, Newton s method estimates from data at one point, in: R. Ewing, K. Gross, C. Martin Eds., The Merging of Disciplines: New Directions in Pure, Applied and Computational Mathematics, Springer, New York, 986, pp [] J.P. Dedieu, P. Priouret, G. Malajovich, Newton s method on Riemannian manifolds: Covariant alpha theory, IMA J. Numer. Anal [3] C. Li, J.H. Wang, Newton s method on Riemannian manifolds: Smale s point estimate theory under the γ -condition, IMA J. Numer. Anal [4] X.H. Wang, D.F. Han, Criterion α and Newton s method in weak condition, Chinese J. Numer. Appl. Math [5] F. Alvarez, J. Bolte, J. Munier, A unifying local convergence result for Newton s method in Riemannian manifolds, Found. Comput. Math [6] R.E. Mahony, The constrained Newton method on a Lie group and the symmetric eigenvalue problem, Linear Algebra Appl [7] R.E. Mahony, J. Manton, The geometry of the Newton method on non-compact Lie groups, J. Global Optim [8] H. Munthe-Kaas, High order Runge Kutta methods on manifold, Appl. Numer. Math [9] B. Owren, B. Welfert, The Newton iteration on Lie groups, BIT, Numer [] V.S. Varadarajan, Lie Groups Lie Algebras and their Representations, in: GTM,, Springer-Verlag, New York, 984. [] F.W. Warner, Foundations of Differentiable Manifolds and Lie Groups, in: GTM, vol. 94, Springer-Verlag, New York, 983. [] M.P. DoCarmo, Riemannian Geometry, Birkhauser, Boston, 99. [3] X.H. Wang, Convergence on the iteration of Halley family in weak conditions, Chinese Sci. Bull [4] X.H. Wang, D.F. Han, On the dominating sequence method in the point estimates and Smale s theorem, Scientia Sinica Ser. A [5] S. Helgason, Differential Geometry Lie Groups Symmetric Spaces, Academic Press Inc., New York, 978. [6] B.C. Hall, Lie Groups Lie Algebras Representations, in: GTM, vol., Springer-Verlag, New York, 4.