Lecture Notes on Numerical Analysis of Nonlinear Equations

Transcription

1 Lecture Notes on Numerical Analysis of Nonlinear Equations Eusebius J Doedel Department of Computer Science, Concordia University, Montreal, Canada Numerical integrators can provide valuable insight into the transient behavior of a dynamical system. However, when the interest is in stationary and periodic solutions, their stability, and their transition to more complex behavior, then numerical continuation and bifurcation techniques are very powerful and efficient. The objective of these notes is to make the reader familiar with the ideas behind some basic numerical continuation and bifurcation techniques. This will be useful, and is at times necessary, for the effective use of the software Auto and other packages, such as XppAut [7], Content [24], Matcont [2], and DDE-Biftool [6], which incorporate the same or closely related algorithms. These lecture notes are an edited subset of material from graduate courses given by the author at the universities of Utah and Minnesota [9] and at Concordia University, and from short courses given at various institutions, including the Université Pierre et Marie Curie (Paris VI), the Centre de Recherches Mathématiques of the Université de Montréal, the Technische Universität Hamburg-Harburg, and the Benemérita Universidad Autónoma de Puebla.. The Implicit Function Theorem Before starting our discussion of numerical continuation of solutions to nonlinear equations, it is important first to discuss under what conditions a solution will actually persist when problem parameters are changed. Therefore, we begin with an overview of the basic theory. The Implicit Function Theorem (IFT) is central to our analysis and we discuss some examples. The discussion in this section follows the viewpoint of Keller in graduate lectures at the California Institute of Technology, a subset of which was published in [23].

2 2 Eusebius J Doedel.. Basic Theory Let B denote a Banach space, that is, a complete, normed vector space. In the presentation below it will be implicitly assumed that B is R n, although the results apply more generally. For x B, we denote by S ρ (x ) the closed ball of radius ρ centered at x, that is, S ρ (x ) = {x B x x ρ}. Existence and uniqueness of solutions is obtained by using two theorems. Theorem (Contraction Theorem). Consider a continuous function F : B B on a Banach space B and suppose that for some x B, ρ >, and some K with K <, we have F(u) F(v) K u v, for all u,v S ρ (x ), F(x ) x ( K ) ρ. Then the equation x = F(x), x B, has one and only one solution x S ρ (x ), and x is the limit of the sequence Proof. Let x = F(x ). Then x k+ = F(x k ), k =,,2,.... x x = F(x ) x ( K ) ρ ρ. Thus, x S ρ (x ). Now assume inductively that x,x,,x n S ρ (x ). Then for k n we have Thus, x k+ x k = F(x k ) F(x k ) K x k x k = K k x x K k ( K ) ρ. x n+ x x n+ x n + x n x n + + x x (K n + K n + + ) ( K ) ρ = ( K n+ ) ρ ρ. Hence x n+ S ρ (x ), and by induction x k S ρ (x ) for all k. We now show that {x k } is a Cauchy sequence:

3 Numerical Analysis of Nonlinear Equations 3 x k+n x k x k+n x k+n + + x k+ x k (K n + K n ) K k ( K ) ρ = ( K n ) K k ρ K k ρ. For given ε >, choose k such that K k ρ < 2 ε. Then x k+l x k+m x k+l x k + x k+m x k 2K k ρ < ε, independently of l and m. Hence, {x k } is a Cauchy sequence and, therefore, converges to a unique limit limx k = x, where x S ρ (x ). Since we assumed that F is continuous, we have x = limx k = limf(x k ) = F(limx k ) = F(limx k ) = F(x ). This proves the existence of x. We get uniqueness as follows. Suppose there are two solutions, say, x,y S ρ (x ) with x = F(x) and y = F(y). Then x y = F(x) F(y) K x y. Since K <, this is a contradiction. The second theorem ensures the parameter-dependent existence of a solution. Theorem 2 (Implicit Function Theorem). Let G : B R m B satisfy: G(u,λ ) = for u B and λ R m ; G u (u,λ ) is nonsingular with bounded inverse, G u (u,λ ) M for some M > ; G and G u are Lipschitz continuous, that is, for all u,v S ρ (u ), and for all λ,µ S ρ (λ ) the following inequalities hold for some K L > : G(u,λ) G(v,µ) K L ( u v + λ µ ), G u (u,λ) G u (v,µ) K L ( u v + λ µ ). Then there exists δ, with < δ ρ, and a unique function u(λ) that is continuous on S δ (λ ), with u(λ ) = u, such that G(u(λ),λ) =, for all λ S δ (λ ). If G(u, λ ) = and if G u (u,λ ) is invertible with bounded inverse, then u is called an isolated solution of G(u,λ ) =. Hence, the IFT states that isolation (plus Lipschitz continuity assumptions) implies the existence of a locally unique solution family (or solution branch) u = u(λ), with u(λ ) = u.

4 4 Eusebius J Doedel Proof. We use the notation G u = G u (u,λ ). Then we rewrite the problem as G(u,λ) = G u u = G u u G(u,λ) u = ( G u) [G u u G(u,λ)]. }{{} F(u,λ) Hence, G(u,λ) = if and only if u is a fixed point of F(,λ). (Note that the corresponding fixed point iteration is, in fact, the Chord Method for solving G(u,λ) =.) We must verify the conditions of the Contraction Theorem. Pick u,v S ρ (u ), and any fixed λ S ρ (λ ), where ρ is to be chosen later. Then F(u,λ) F(v,λ) = ( G u) { G u [u v] [G(u,λ) G(v,λ)] }. (.) By the Fundamental Theorem of Calculus, we have G(u,λ) G(v,λ) = = d G(tu + ( t)v,λ) dt dt G u (tu + ( t)v,λ) dt [u v] = Ĝu(u,v,λ) [u v], where in the last step we used the Mean Value Theorem to get Ĝ. Then (.) becomes F(u,λ) F(v,λ) M G u Ĝu(u,v,λ) u v = M M M G u (u,λ ) G u (tu + ( t)v,λ) dt u v G u (u,λ ) G u (tu + ( t)v,λ) dt u v K L ( u (tu + ( t)v) + λ λ ) dt u v }{{} S ρ (u ) M K L 2ρ }{{} K u v. Therefore, if we take ρ < 2M K L, then K <. The second condition of the Contraction Theorem is also satisfied, namely,

5 F(u,λ) u = F(u,λ) F(u,λ ) Numerical Analysis of Nonlinear Equations 5 = ( G u) [G u u G(u,λ)] ( G u) [G u u G(u,λ )] = ( G u) [G(u,λ ) G(u,λ)] M K L λ λ M K L ρ, where ρ (with < ρ ρ ) is to be chosen. We want the above to be less than or equal to ( K )ρ, so we choose ρ ( K ) ρ M K L. Hence, for each λ S ρ (λ ) we have a unique solution u(λ). We now show that u(λ) is continuous in λ. Let λ,λ 2 S ρ (λ ), with corresponding solutions u(λ ) and u(λ 2 ). Then u(λ ) u(λ 2 ) = F(u(λ ),λ ) F(u(λ 2 ),λ 2 ) F(u(λ ),λ ) F(u(λ 2 ),λ ) + F(u(λ 2 ),λ ) F(u(λ ),λ 2 ) K u(λ ) u(λ 2 ) + ( G u) [G u u(λ 2 ) G(u(λ 2 ),λ )] ( G u) [G u u(λ 2 ) G(u(λ 2 ),λ 2 )] K }{{} < u(λ ) u(λ 2 ) +M K L λ λ 2. Hence, which concludes the proof of the IFT. u(λ ) u(λ 2 ) M K L K λ λ 2, So far, under mild assumptions, we have shown that there exists a locally unique solution family u(λ). If we impose the condition that F(u,λ) is continuously differentiable in λ, then we can show that u(λ) is also continuously differentiable. To this end, the Banach Lemma is very useful. Lemma (Banach Lemma). Let L : B B be a linear operator with L <. Then (I + L) exists and (I + L) L. Proof. Suppose I + L is not invertible. Then there exists y B, y, such that (I + L)y =.

6 6 Eusebius J Doedel Thus, y = Ly and y = Ly L y < y, which is a contradiction. Therefore, (I+L) exists. We can bound the inverse as follows: (I + L)(I + L) = I This proves the Banach Lemma. (I + L) = I L(I + L) (I + L) + L (I + L) (I + L) L. The Banach Lemma can be used to show the following. Lemma 2. Under the conditions of the IFT, there exists M > and δ > such that G u (u,λ) exists and G u (u,λ) M in S δ (u ) S δ (λ ). Proof. Using again the notation G u = G u (u,λ ), we have G u (u,λ) = G u + G u (u,λ) G u = G u [I + (G u) (G u (u,λ) G }{{ u) ]. } L Similar to how we verified the second condition of the Contraction Theorem in the proof of the IFT, we can show that L M K L ( u u + λ λ ) M K L 2δ. As for the IFT, we choose δ < 2M K L, and conclude that, therefore, (I + L) exists and Hence, G u (u,λ) exists and as required. (I + L) M K L 2δ. G u (u,λ) = (G u) (I + L) M M K L 2δ M, We are now ready to prove differentiability of the solution branch. Theorem 3. In addition to the assumptions of the IFT, assume that the derivative G λ (u,λ) is continuous in S ρ (u ) S ρ (λ ). Then the solution branch u(λ) has a continuous derivative u λ (λ) on S δ (u ) S δ (λ ).

7 Numerical Analysis of Nonlinear Equations 7 Proof. Using the definition of (Fréchet) derivative, we are given that there exists G u (u,λ) such that G(u,λ) G(v,λ) = G u (u,λ) (u v)+r (u,v,λ), where R (u,v,λ) is such that R (u,v,λ) u v as u v. (.2) Similarly, there exists G λ (u,λ) such that G(u,λ) G(u,µ) = G λ (u,λ) (λ µ) + R 2 (u,λ,µ), where R 2 (u,λ,µ) satisfies R 2 (u,λ,µ) λ µ We must show that there exists u λ (λ) such that as λ µ. (.3) u(λ) u(µ) = u λ (λ) (λ µ) + r(λ,µ), with Now r(λ,µ) λ µ as λ µ. = G(u(λ),λ) G(u(µ),µ) = G(u(λ),λ) G(u(µ),λ) + G(u(µ),λ) G(u(µ),µ) = G u (u(λ),λ) (u(λ) u(µ)) + R (u(λ),u(µ),λ) +G λ (u(µ),λ) (λ µ) + R 2 (u(µ),λ,µ). Lemma 2 guarantees the existence of G u (u(λ),λ), and we find where u(λ) u(µ) = G u (u(λ),λ) [G λ (u(µ),λ) (λ µ) (R + R 2 )] = G u (u(λ),λ) [G λ (u(λ),λ) (λ µ) r], r = [G λ (u(λ),λ) G λ (u(µ),λ)] (λ µ) + R + R 2. Let us, for the moment, ignore the harmless factor G u (u(λ),λ) and consider each term of r. Since u and G λ are continuous, we have [G λ (u(λ),λ) G λ (u(µ),λ)] (λ µ) λ µ Also, the existence of G λ implies (.3) Using (.2), we have R 2 (u(λ),λ,µ) λ µ as λ µ. as λ µ.

8 8 Eusebius J Doedel R (u(λ),u(µ),λ) λ µ = R (u(λ),u(µ),λ) u(λ) u(µ) u(λ) u(µ) λ µ, as λ µ because the second factor is bounded due to continuity of u(λ) (see the end of the proof of the IFT). Thus, u λ (λ) = G u (u(λ),λ) G λ (u(λ),λ). To prove that u λ is continuous it suffices to show that G u (u(λ),λ) is continuous. Indeed, G u (u(λ),λ) G u (u(µ),µ) = G u (u(λ),λ) [G u (u(µ),µ) G u (u(λ),λ)]g u (u(µ),µ) M 2 K L ( u(µ) u(λ) + µ λ ), which concludes the proof of Theorem 3 Remark. In fact, if G λ is Lipschitz continuous then u λ is Lipschitz continuous (we already assume that G u is Lipschitz continuous). More generally, it can be shown that u λ is C k if G is C k, that is, u inherits the degree of continuity of G. We now give some examples where the IFT is used to show that a given solution persists, at least locally, when a problem parameter is changed. We also identify some cases where the conditions of the IFT are not satisfied...2 A Predator-Prey Model Our first example is that of a predator-prey model defined as { u = 3u ( u ) u u 2 λ( e 5u ), u 2 = u 2 + 3u u 2. (.4) We can think of u as fish and u 2 as sharks, while the term λ( e 5u ) represents fishing, with fishing-quota λ. When λ = the stationary solutions are } 3u ( u ) u u 2 = (u u 2 + 3u u 2 =, u 2 ) = (,), (,), ( 3,2). The Jacobian matrix is ( ) 3 6u u J = 2 5λe 5u u = J(u u 2 + 3u, u 2 ;λ). Hence, we have

9 Numerical Analysis of Nonlinear Equations 9 u lambda u2 Fig... Stationary solution branches of the predator-prey model (.4). Solution 2 and solution 4 are branch points, while solution 8 is a Hopf bifurcation point. ( ) 3 J(,; ) =, eigenvalues 3, (unstable); ( ) 3 J(,; ) =, eigenvalues 3, 2 (unstable); 2 ( µ)( µ) + 2 = ( ) J( 3,2; ) = µ 2 + µ + 2 = 3, eigenvalues 6 µ ± = ± 7 2 ; Re(µ ± ) < (stable). All three Jacobians at λ = are nonsingular. Thus, by the IFT, all three stationary points persist for (small) λ. In this problem we can explicitly find all solutions (see Fig..): I: (u, u 2 ) = (,). II: u 2 = and λ = 3u ( u ). (Note that lim 5u e λ = lim 3( 2u ) = 3 u u 5e 5u 5.) III: u = 3 and u 2 λ( e 5/3 ) = u 2 = 2 3λ( e 5/3 ). These solution families intersect at two branch points, one of which is (u, u 2, λ) = (,,3/5). The stability of Branch I follows from: J(,;λ) = ( 3 5λ ), eigenvalues 3 5λ,.

10 Eusebius J Doedel MAX U lambda Fig..2. Bifurcation diagram of the predator-prey model (.4). The periodic solution branch is also shown. For stationary solutions the vertical axis is simply u, while for periodic solutions max(u ) is plotted. Solid/dashed lines denote stable/unstable solutions. Open squares are branch points; the solid square is a Hopf bifurcation. Hence, the trivial solution is unstable if λ < 3/5, and stable if λ > 3/5, as indicated in Fig..2. Branch II has no stable positive solutions. At λ H.67 on Branch III (Solution 8 in Fig..2) the complex eigenvalues cross the imaginary axis. This crossing is a Hopf bifurcation. Beyond λ H there are periodic solutions whose period T increases as λ increases; see Fig..3 for some representative periodic orbits. The period becomes infinite at λ = λ.7. This final orbit is called a heteroclinic cycle. From Fig..2 we can deduce the solution behavior for increasing λ: Branch III is followed until λ H ; then the behavior becomes oscillatory due to the periodic solutions of increasing period until λ = λ ; finally, the dynamics collapses to the trivial solution (Branch I)...3 The Gelfand-Bratu Problem The IFT is not only useful in the context of solution branches of equilibria. The periodic orbits in Sect...2 are also computed using the IFT principle. This section gives an example of a solution branch of a two-point boundary value problem. The Gelfand-Bratu problem [2] is defined as { u (x) + λe u(x) =, x [,], (.5) u() = u() =.

11 Numerical Analysis of Nonlinear Equations U U Fig..3. Some periodic solutions of the predator-prey model (.4). The final orbits are very close to a heteroclinic cycle. MAX U LAMBDA Fig..4. Bifurcation diagram of the Gelfand-Bratu equation (.5). Note that there are two solutions for < λ < λ C, where λ C 3.5. There is one solution for λ = λ C and for λ, and are no solutions for λ > λ C.

12 2 Eusebius J Doedel U X Fig..5. Some solutions to the Gelfand-Bratu equation (.5). If λ = then u(x) is a solution. We show here that this solution is isolated, so that there is a continuation u = ũ(λ), for λ small. Consider } u (x) λe u(x) =, u() =, u u = u(x;q, λ). () = q, We want to solve u(;q, λ) =, for λ small. Here F(,) =. }{{} F(q,λ) We must show (IFT) that F q (,) u q (;,) : Now u q (x;,) satisfies } u q(x) λ e u(x) u q =, u q () =, u where u q() =,. { u q =, u q () =, u q() =. Hence, u q (x;,) = x, so that u q (;,) =...4 A Nonlinear Eigenvalue Problem The equations for column buckling (from nonlinear elasticity theory) [3] are given by

13 Numerical Analysis of Nonlinear Equations 3 for x [,], with { u (x) + µu (x) =, u 3(x) + 2 u (x) 2 + µβ =, { u () = u () =, u 3 () = u 3 () = λ (λ > ), where µ is a stress (to be determined) and β is another physical constant; take β =. Note that the boundary conditions are overspecified (to determine µ). We rewrite the equations as the first order system u = u 2, u () = u () =, u 2 = µu, (.6) u 3 = 2 u2 2 µ, u 3 () = u 3 () = λ. Note that u u 2 implies u 3 = µ, so that u 3 (x) = λ µx, with u 3 () = λ and u 3 () = λ µ = λ. Thus, we must have µ = 2λ, so that u 3 (x) = λ 2λx = λ( 2x). Hence, u u 2, u 3 (x) = λ( 2x), µ = 2λ, is a solution for all λ. Are these solutions isolated? In the formal set-up, consider u = u 2, u () =, u 2 = µu, u 2 () = p, u = u(x, p, µ;λ). u 3 = 2 u2 2 µ, u 3 () = λ. We must have } u (, p, µ;λ) =, F(p, µ;λ) =, u 3 (, p, µ;λ) + λ =, with F : R 2 R R 2. So the question is: Is (F p F µ )(λ) nonsingular along the basic solution branch? To answer the above question quickly, we omit explicit construction of F. We linearize (.6) about u,u 2, u 3, µ, and λ, with respect to u, u 2, u 3, and µ, acting on v, v 2, v 3, and µ, to obtain the linearized homogeneous equations v = v 2, v () = v () =, v 2 = µv µu, v 3 = u 2 v 2 µ, v 3 () = v 3 () =. In particular, the linearized homogeneous equations about u u 2, u 3 (x) = λ( 2x), and µ = 2λ are v = v 2, v () = v () =, v 2 = 2λv, v 3 µ =, v 3. = µ, v 3 () = v 3 () =.

14 4 Eusebius J Doedel Now, if 2λ k 2 π 2, k =,2,3,..., then { v + 2λv =, v () = v () =, has the unique solution v and, hence, also v 2. Thus, if λ 2 k2 π 2 then the basic solution branch is locally unique. However, if λ = 2 k2 π 2 then the linearization is singular, and there may be bifurcations. (In fact, there are buckled states.)..5 The Pendulum Equation The equation of a damped pendulum subject to a constant torque is given by m R φ (t) + ε m φ (t) +m g sinφ(t) = }{{}}{{} I, damping torque that is, φ (t) + ε R φ (t) + g R sinφ(t) = Scaling time as s = c t we have φ = dφ we obtain dt = dφ ds c 2 φ(s) + ε c R φ(s) + g R sinφ(s) = g φ + ε R c φ + g R c 2 sinφ = I m R. ds dt = c φ and, similarly, φ = c 2 φ, I m R, I m R c 2. Choose c such that R c =, that is, c = g/r, and set ε = ε/(r c) and 2 Ĩ = I/(m R c 2 ). Then the equation becomes φ + ε φ + sin φ = Ĩ, or, dropping the, and using, φ + ε φ + sinφ = I. (.7) We shall consider special solutions, called rotations, that satisfy φ(t + T) = φ(t) + 2π, for all t or, equivalently, φ(t) = {}}{ φ() +2π φ (T) = φ (), (= 2π), where T is the period.

15 The Undamped Pendulum First consider the undamped unforced pendulum Numerical Analysis of Nonlinear Equations 5 φ + sinφ =, that is, (.7) with ε = I =. Suppose the initial data for a rotation are φ() =, and φ () = p >. We have φ(t) = 2π, and φ (T) = φ () = p. Integration gives t φ φ dt + t φ sinφ dt = 2 φ 2 t cos φ t = Thus, 2 φ (t) 2 cos φ(t) = 2 p2 2 φ (t) 2 + cos φ(t) = }{{}}{{} 2 p2. kinetic energy potential energy φ (t) = dφ dt = p cos φ(t) dt dφ = p cos φ We see that and 2π 2π dt dφ dφ = T = T 2π In fact, rotations exists for all p > 2. The Forced Damped Pendulum 2π T as p, p cos φ dφ p cos φ dφ cos φ dφ = as p 2. We now consider the forced damped pendulum (.7), φ + ε φ + sinφ = I, with φ() = (which sets the phase) and φ () = p. We write the solution as φ = φ(t;p, I, ε). Do there exist rotations, i.e., does there exist T such that φ(t;p, I, ε) = 2π and φ (T;p, I, ε) = p?

16 6 Eusebius J Doedel Theorem 4. Let φ be a rotation of the undamped unforced pendulum: φ + sinφ =, φ () =, φ () = p, φ (T ) = 2π, φ (T ) = p. Then there exist (smooth) functions T = T(p, ε) and I = I(p, ε), with T(p,) = T and I(p,) =, such that φ(t;p, I(p, ε), ε) is a rotation of period T(p, ε) of the damped forced pendulum φ + ε φ + sinφ = I, φ(;p, I(p, ε), ε) =, φ (;p, I(p, ε), ε) = p, φ(t(p, ε);p, I(p, ε), ε) = 2π, φ (T(p, ε);p, I(p, ε), ε) = p. for all (p, ε) sufficiently close to (p,). Proof. The Jacobian matrix with respect to T and I, of the algebraic system { φ(t;p, I, ε) 2π =, φ (T;p, I, ε) p =, evaluated at p = p, T = T, and I = ε =, is ( φ J = φ ) I (T ). φ φ I We must show that detj. We have φ + sinφ = φ (T ) = sin (φ (T )) = sin (2π) =, φ (T ) = p. Thus, detj if φ I (T ). Here, φ I satisfies In particular, From and we have Using integration, we find φ I + ε φ I + φ I cos φ =, φ I () = φ I() =. φ I + φ I cos φ =, φ I() = φ I () =. φ I φ + φ I cos φ φ = φ, φ + sinφ = φ + cos φ φ =, φ I φ φ I φ = φ.

17 φ I φ T T Numerical Analysis of Nonlinear Equations 7 T φ I φ φ I T φ φ I φ }{{} φ sin φ φ I φ T T = φ = 2π, T + φ I φ = 2π, I (T ) φ (T ) φ I () φ }{{}}{{} () = 2π. p Hence, φ I (T ) = 2π p. A more general analysis of this type for coupled pendula can be found in [] (see also Chap. 5)..2 Continuation of Solutions As mentioned, the IFT plays an important role in the design of algorithms for computing families of solutions to nonlinear equations. Such continuation methods are applied in a parameter-dependent setting. Hence, we consider the equation G(u, λ) =, u, G(, ) R n, λ R. Let x (u, λ). Then the equation can be written as G(x) =, G : R n+ R n..2. Regular Solutions A solution x of G(x) = is regular [22] if the n (rows) by n + (columns) matrix G x G x (x ) has maximal rank, i.e., if Rank(G x) = n. In the parameter formulation G(u, λ) =, we have (i) G u is nonsingular, or Rank(G x) = Rank(G u G λ) = n dimn(g u) =, (ii) and G λ R(G u). Here, N(G u) denotes the null space of G u, and R(G u) denotes the range of G u, i.e., the linear space spanned by the n columns of G u.

18 8 Eusebius J Doedel u u x λ λ Fig..6. A solution branch of G(u, λ) = ; note the two folds. Theorem 5. Let x (u, λ ) be a regular solution of G(x) =. Then, near x, there exists a unique one-dimensional continuum of solutions x(s), called a solution family or a solution branch, with x() = x. Proof. Since Rank(G x) = Rank(G u G λ ) = n, either G u is nonsingular and by the IFT we have u = u(λ) near x, or else we can interchange columns in the Jacobian G x to see that the solution can locally be parametrized by one of the components of u. Thus, a unique solution family passes through a regular solution. Remark 2. We remark here that the second case in the above proof is that of a simple fold; see also Fig Parameter Continuation In the parameter-dependent setting we assume that the continuation parameter is λ. Suppose we have a solution (u, λ ) of G(u, λ) =, as well as the direction vector u = du/dλ, and we want to compute the solution u at λ = λ + λ; this is illustrated in Fig..7. To compute the solution u we use Newton s method { G u (u (ν), λ ) u (ν) = G(u (ν), λ ), As initial approximation, we use u (ν+) = u (ν) + u (ν), ν =,,2,.... (.8) u () = u + λ u. If G u (u, λ ) is nonsingular and λ is sufficiently small, then the convergence theory for Newton s method guarantees that this iteration will converge.

19 Numerical Analysis of Nonlinear Equations 9 "u" u u u () u (= du d λ at λ ) λ λ λ λ Fig..7. Graphical interpretation of parameter continuation. After convergence, the new direction vector u can be computed by solving G u (u, λ ) u = G λ (u, λ ). This equation follows from differentiating G(u(λ), λ) = with respect to λ at λ = λ. Note that, in practice, the calculation of u can be done without another LU-factorization of G u (u, λ ). Thus, the extra work to find u is negligible. As an example, consider again the Gelfand-Bratu problem of Sect...3 given by { u (x) + λe u(x) =, x [,], u() = u() =. If λ = then u(x) is an isolated solution; see Sect...3. We discretize this problem by introducing a mesh, The discrete equations are: = x < x < < x N =, x j x j = h, j N, h = /N. u j+ 2u j + u j h 2 + λe uj =, j =,..., N, with u = u N =. (More accurate discretization is discussed in Sect..3..) Let u u u 2. u N Then we can write the above as G(u, λ) =, where G : R n R R n, with n = N.

20 2 Eusebius J Doedel "u" u u u s λ ( u, ) λ λ λ λ Fig..8. Graphical interpretation of pseudo-arclength continuation. For the parameter continuation we suppose that we know λ, u, and u. Then we set λ = λ + λ and apply Newton s method (.8) with u () = u + λ u. After convergence find u from G u (u, λ ) u = G λ (u, λ ), and repeat the above procedure to find u 2, u 3, and so on. Here, 2 h + λe u 2 h 2 h 2 G u (u, λ) = 2 h + λe u2 2 h h 2 2 h + λe u 2 N Hence, we must solve a tridiagonal system for each Newton iteration. The solution branch has a fold where the parameter-continuation method fails; see Figs..4 and Keller s Pseudo-Arclength Continuation In order to allow for continuation of a solution branch past a fold, Auto [8,, 2] uses Keller s Pseudo-Arclength Continuation [22]. Suppose we have a solution (u, λ ) of G(u, λ) =, as well as the direction vector ( u, λ ) of the solution branch. Pseudo-arclength continuation solves the following equations for (u, λ ): { G(u, λ ) =, (u u ) u + (λ λ ) λ (.9) s =. Figure.8 shows a graphical interpretation of this continuation method. Newton s method for pseudo-arclength continuation becomes

21 Numerical Analysis of Nonlinear Equations 2 "X-space" x s x x Fig..9. Parameter-independent pseudo-arclength continuation. ( (G u ) (ν) (G λ )(ν) u λ ) ( ) ( (ν) u = λ (ν) with the new direction vector defined as ( ) ( ) ( ) G u G λ u =, λ λ Note that u G(u (ν), λ(ν) ) (u (ν) u ) u + (λ (ν) λ ) λ s In practice ( u, λ ) can be computed with one extra back-substitution; The orientation of the branch is preserved if s is sufficiently small; The direction vector must be rescaled, so that indeed u 2 + λ 2 =. Theorem 6. The Jacobian of the pseudo-arclength system is nonsingular at a regular solution point. Proof. Let x = (u, λ) R n+. Then pseudo-arclength continuation can be written as G(x ) =, (x x ) ẋ s =, ( ẋ = ). Figure.9 shows ( a) graphical interpretation. The matrix in Newton s method G at s = is x ẋ. At a regular solution we have N(G x) = Span{ẋ }. We ( ) G must show that x ẋ is nonsingular at a regular solution. Suppose, on the ( ) G contrary, that x ẋ is singular. Then there exists some vector z with G x z = and ẋ z =. Thus, z = cẋ, for some constant c. But then = ẋ z = cẋ ẋ = c ẋ 2 = c, ), so that z =, which is a contradiction.

22 22 Eusebius J Doedel Consider pseudo-arclength continuation for the discretized Gelfand-Bratu problem of Sect...3. Then the matrix ( ) ( ) Gx Gu G λ ẋ = u λ in Newton s method is a bordered tridiagonal matrix of the form. We now show how to solve such linear systems efficiently..2.4 The Bordering Algorithm The linear systems in Newton s method for pseudo-arclength continuation are of the form ( ) ( ) ( ) A c x f b =. d y h The special structure of this extended system can be exploited; a general presentation of the numerical linear algebra aspects of extended systems can be found in [2, 24]. If A is a sparse matrix whose LU-decomposition can be found relatively cheaply (e.g., if A is tridiagonal), then the following bordered LU-decomposition [22] will be efficient: ( ) A c b d = ( ) ( ) L U γ β. δ After decomposing A = LU (which may require pivoting) we compute γ, β, and δ from Lγ = c, U β = b, δ = d β γ. The linear system can then be written as

23 Numerical Analysis of Nonlinear Equations 23 ( ) ( ) ( ) ( ) L U γ x f β =, δ y h }{{} ˆf ĥ and we can compute the solution (x, y) through the following steps: Lˆf = f, ĥ = h β ˆf, y = ĥ/δ, Ux = ˆf yγ. Theorem 7. ( The bordering ) algorithm outlined above works if A and the full A c matrix A b are nonsingular. d In the proof of Theorem 7 we make use of the Bordering Lemma [22] ( ) A c Lemma 3 (Bordering Lemma). Let A b. Then d (a) A nonsingular A nonsingular if and only if d b A c; { c R(A), (b) dimn(a) = dimn(a ) = A nonsingular if b R(A ); (c) If dimn(a) 2 then A is singular. Proof. (a) (A nonsingular) ( ) ( ) A I A In this case A = c b, where e = d b e A c. Clearly, A is nonsingular if and only if e. (b) (dimn(a) = ) Suppose A is singular in this case. Then there exist z R n and ξ R, not both zero, such that ( ) ( ) ( ) ( ) A c z Az + ξc A = b = d ξ b =. z + ξd We see that c R(A) if ξ, which contradicts the assumptions. On the other hand, if ξ = and z then N(A) = Span{z} and b N(A).

24 24 Eusebius J Doedel Since, in general, N(A) = R(A ), it follows that b R(A ), which also contradicts the assumptions. (c) (dim N(A) 2) This case follows from a rank argument. Proof (Theorem 7). The crucial step in the bordering algorithm is the computation of z = ĥ/δ. Namely, we must have δ. Since δ is determined in the bordered LU-decomposition, we have δ = d β γ = d (U b) (L c) = d b U L c = d b (LU) c = d b A c, which is nonzero by Conclusion (a) of the Bordering Lemma. Remark 3. In pseudo-arclength continuation we have ( ) ( ) A c Gu G λ A = b = d λ, that is, A = G u, which is singular at a fold. Therefore, the bordering algorithm will fail when it is used exactly at a fold. In practice, the method may still work. We consider another approach, used in Auto, when discussing collocation methods in Sect..3.. u.3 Boundary Value Problems Consider the first-order system of ordinary differential equations where subject to boundary conditions and integral constraints u (t) f(u(t),µ, λ) =, t [,], u( ), f( ) R n, λ R, µ R nµ, b(u(),u(),µ, λ) =, b( ) R n b, q(u(s),µ, λ) ds =, q( ) R nq. We want to solve this boundary value problem (BVP) for u( ) and µ. In order for this problem to be formally well posed we require that n µ = n b + n q n. We can think of λ as the continuation parameter in which the solution (u,µ) may be continued. A simple case is n q =, n b = n, for which n µ =.

25 .3. Orthogonal Collocation Numerical Analysis of Nonlinear Equations 25 Auto solves boundary value problems using the method of orthogonal collocation with piecewise polynomials [2, 7]. This method is very accurate, and allows adaptive mesh-selection. The set-up is as follows. First, we introduce a mesh { = t < t < < t N = }, with h j = t j t j, ( j N). Define the space of (vector-valued) piecewise polynomials Ph m { Ph m = p h C[,] p h Pm} [tj,t j], as where P m is the space of (vector-valued) polynomials of degree m. The orthogonal collocation method with piecewise polynomials [3] consists of finding p h Ph m and µ, such that the following collocation equations are Rnµ satisfied: p h(z j,i ) = f(p h (z j,i ),µ, λ), j =,..., N, i =,..., m, and such that p h satisfies the boundary and integral conditions. The collocation points z j,i in each subinterval [t j, t j ] are the (scaled) roots of the mth-degree orthogonal polynomial (Gauss points); see Fig.. for a graphical interpretation. Since each local polynomial is determined by (m + )n, coefficients, the total number of degrees of freedom (considering λ as fixed) is (m + )nn + n µ. This is matched by the total number of equations: collocation: mnn, continuity: (N )n, constraints: n b + n q (= n + n µ ). If the solution u(t) of the BVP is sufficiently smooth then the order of accuracy of the orthogonal collocation method is m, i.e., p h u = O(h m ). At the main meshpoints t j we have superconvergence: max j p h (t j ) u(t j ) = O(h 2m ). The scalar variables µ are also superconvergent [7].

26 26 Eusebius J Doedel t t t t 2 N t j- t j z j, z j,2 z j,3 t j- t j-2/3 t j-/3 lj,3(t) lj,(t) t j Fig... The mesh { = t < t < < t N = }. Collocation points and extended-mesh points are shown for the case m = 3, in the jth mesh interval. Also shown are two of the four local Lagrange basis polynomials..3.2 Implementation in Auto The implementation in Auto [2] is done via the introduction of Lagrange basis polynomials for each subinterval [t j, t j ]. Define by where {l j,i (t)}, j =,..., N, i =,,..., m, l j,i (t) = m k=,k i t t j k m t j i t, m j k m t j i m = t j i m h j. The local polynomials can then be written as p j (t) = With the above choice of basis m i= l j,i (t)u j i m. u j approximates u(t j ) and u j i m approximates u(t j i m ), where u(t) is the solution of the continuous problem. Then the collocation equations are p j(z j,i ) = f(p j (z j,i ),µ, λ), i =,..., m, j =,..., N,

27 Numerical Analysis of Nonlinear Equations 27 u u 3 u 2 3 u u 2 u N T λ Fig... Structure of the Jacobian for the case of n = 2 differential equations with the number of mesh intervals N = 3, the number of collocation points per mesh interval m = 3, the number of boundary conditions n b = 2, and the number of integral constraints n q =. The last row corresponds to the pseudo-arclength equation, which is not included in the n q = count. From E.J. Doedel, H.B. Keller, J.P. Kernévez, Numerical analysis and control of bifurcation problems (II): Bifurcation in infinite dimensions, Internat. J. Bifur. Chaos Appl. Sci. Engrg. (4) (99) c 99 World Scientific Publishing; reproduced with permission. the discrete boundary conditions are b i (u,u N,µ, λ) =, i =,..., n b, and the integrals constraints can be discretized as N m j= i= ω j,i q k (u j i m,µ, λ) =, k =,..., n q, where the ω j,i are the Lagrange quadrature coefficients. The pseudo-arclength equation is (u(t) u (t)) u (t) dt + (µ µ ) µ + (λ λ ) λ s =, where (u,µ, λ ), is the previously computed point on the solution branch, and ( u, µ, λ ), is the normalized direction of the branch at that point. The

28 28 Eusebius J Doedel u u 3 u 2 3 u u 2 u N T λ Fig..2. The system after condensation of parameters. The entries have been eliminated by Gauss elimination. discretized pseudo-arclength equation is N m j= i= ω j,i [u j i m (u ) j i m ] ( u ) j i m +(µ µ ) µ + (λ λ ) λ s =. The implementation in Auto includes an efficient method to solve these linear systems [2]; this is illustrated in Figs Note that the figures only illustrate the matrix structure; the indicated operations are also carried out on the right-hand side, which is not shown in the figures. Figure.2 shows the system after condensation of parameters. The entries marked with have been eliminated by Gauss elimination. These operations can be done in parallel [34]. The condensation of parameters leads to a system with a fully decoupled sub-system that can be solved separately. The decoupled subsystem is marked by in Fig Numerical Linear Algebra The complete discretization consists of mnn + n b + n q +,

29 Numerical Analysis of Nonlinear Equations 29 u u u 2 u u 2 u N T λ 3 3 Fig..3. This is the same matrix as in Fig..2, except that some entries are now marked by a. The sub-system is fully decoupled from the remaining equations and can, therefore, be solved separately. From E.J. Doedel, H.B. Keller, J.P. Kernévez, Numerical analysis and control of bifurcation problems (II): Bifurcation in infinite dimensions, Internat. J. Bifur. Chaos Appl. Sci. Engrg. (4) (99) c 99 World Scientific Publishing; reproduced with permission. nonlinear equations, in the unknowns {u j i m } RmnN+n, µ R nµ, λ R. These equations can be solved by a Newton-Chord iteration. The structure of the associated Jacobian is illustrated in Fig.. for a system of n = 2 differential equations, with N = 3 mesh intervals, m = 3 collocation points per mesh interval, n b = 2 boundary conditions, and n q = integral constraint. In a typical problem N will be larger, say, N = 5 for very easy problems, and N = 2 for very difficult problems. The standard choice of the number of collocation points per mesh interval is m = 4. The decoupled sub-system can be solved by nested dissection. This procedure eliminates some of the -entries, but also introduces some new nonzero entries due to fill-in; see Fig..4. However, the structure reveals a new decoupled sub-system that can be solved completely; this subsystem is highlighted in Fig..5 with +. The + sub-system consists of two sub-matrices A and A, as in Fig..5. For periodic solutions, the Floquet multipliers are the eigenvalues of the matrix A A [8].

30 3 Eusebius J Doedel u u u 2 u u 2 u N T λ 3 3 Fig..4. The decoupled sub-system solved by nested dissection. This procedure eliminates some of the -entries, but also introduces some new nonzero entries due to fill-in..4 Computing Periodic Solutions Periodic solutions can be computed very effectively by using a boundary value approach. This method also determines the period very accurately. Moreover, the technique allows asymptotically unstable periodic orbits to be computed as easily as asymptotically stable ones..4. The BVP Approach. Consider the first-order system u (t) = f(u(t), λ), u( ), f( ) R n, λ R. Fix the interval of periodicity by the transformation t t T. Then the equation becomes u (t) = T f(u(t), λ), u( ), f( ) R n, T, λ R, (.) and we seek solutions of period, i.e., u() = u(). (.)

31 Numerical Analysis of Nonlinear Equations 3 u u u 2 u u 2 u N T λ 3 3 A A Fig..5. The same matrix as in Fig..4, except with some entries now marked by +. Note that the + sub-system is decoupled from the other equations, and can, therefore, be solved separately. Note that the period T is one of the unknowns. Equations (.) (.) do not uniquely specify u and T. Assume that we have computed (u k ( ), T k, λ k ) and we want to compute the next solution (u k ( ), T k, λ k ). Then u k (t) can be translated freely in time: if u k (t) is a periodic solution then so is u k (t + σ) for any σ. Thus, a phase condition is needed. An example is the Poincaré orthogonality condition (u k () u k ()) u k () =, where the phase of the next condition is fixed such that the difference at time t = is perpendicular to the tangent vector of the current solution; this is illustrated in Fig..6. In the next section we derive a numerically more suitable phase condition..4.2 Integral Phase Condition If ũ k (t) is a solution then so is ũ k (t + σ), for any σ. We want the solution that minimizes D(σ) = ũ k (t + σ) u k (t) 2 2 dt.

32 32 Eusebius J Doedel u k- () u k- () u () k Fig..6. Graphical interpretation of the Poincaré phase condition. The optimal solution ũ k (t+ˆσ), must satisfy the necessary condition D (ˆσ) =. Differentiation gives the necessary condition Writing u k (t) ũ k (t + ˆσ), gives (ũ k (t + ˆσ) u k (t)) ũ k(t + ˆσ) dt =. (u k (t) u k (t)) u k(t) dt =. Integration by parts, using periodicity, gives This is the integral phase condition [8]..4.3 Pseudo-Arclength Continuation u k (t) u k (t) dt =. (.2) In practice, we use pseudo-arclength continuation to follow a family of periodic solutions; see Sect In particular, this allows calculation past folds along a family of periodic solutions. It also allows calculation of a vertical family of periodic solutions, which has important applications to the computation of periodic solutions to conservative systems [4, 3] (see also Chap. 9). For periodic solutions the pseudo-arclength equation is (u k (t) u k (t)) u k (t) dt + (T k T k ) T k + (λ k λ k ) λ k = s. (.3) Equations (.) (.3) are the equations used in Auto for the continuation of periodic solutions. In summary, given u k, T k, and λ k, we solve the system

33 Numerical Analysis of Nonlinear Equations 33 MAX U LAMBDA Fig..7. Bifurcation diagram of the stationary solution u(t) of (.4). where u k(t) = T f(u k (t), λ k ), u k () = u k (), u k (t) u k (t) dt =, (u k (t) u k (t)) u k (t) dt + (T k T k ) T k + (λ k λ k ) λ k = s, u( ), f( ) R n, λ, T R..4.4 A Vertical Family of Periodic Orbits Consider the system of equations { u = λu u 2, u 2 = u ( u ). (.4) Note that u(t) ( is) a stationary solution for all λ. Another stationary solution is u(t). λ The bifurcation diagram for u(t) is shown in Fig..7, but we can also analyze the behavior analytically. The Jacobian along the solution family u(t) is ( ) λ,

34 34 Eusebius J Doedel U U. Fig..8. A phase plot of some periodic solutions of (.4). U SCALED TIME Fig..9. Solution component u of (.4) as a function of the scaled time variable t. with eigenvalues λ ± λ

35 Numerical Analysis of Nonlinear Equations 35 U SCALED TIME Fig..2. Solution component u 2 of (.4) as a function of the scaled time variable t. Hence, the eigenvalues are complex for λ ( 2,2). The eigenvalues cross the imaginary axis when λ passes through zero. Thus, there is a Hopf bifurcation along u(t) at λ =, and a family of periodic solutions bifurcates from u(t) at λ =. As shown in Fig..7, the emanating family of periodic solutions is vertical. Some periodic solutions are shown in Fig..8 in the (u, u 2 )-plane. These solutions are plotted versus time in Figs..9 and.2. Along this family the period tends to infinity. The final infinite-period orbit is homoclinic to (u, u 2 ) = (,). The time diagrams in Figs..9 and.2 illustrate how the peak in the solution remains in the same location. This is a result of the integral phase condition (.2) and very advantageous for discretization methods..4.5 FitzHugh-Nagumo Equations The Fitzhugh-Nagumo equations of nerve-conduction are { v = c ( v 3 v3 + w ), w = (v a + bw)/c. (.5) Let b =.8 and c = 3. Note that there is a stationary solution (v(t), w(t)) = (,) for a =. We compute the solution family, starting at (v(t), w(t)) = (,) for a =, with Auto. The bifurcation diagram is shown in Fig..2. Note that the solution is unstable for a small and becomes stable after a Hopf bifurcation at a Figure.2 also shows the emanating family of periodic solutions,

36 36 Eusebius J Doedel MAX V Parameter a Fig..2. Bifurcation diagram of the Fitzhugh-Nagumo equations (.5). V Scaled Time Fig..22. The periodic solution of (.5) at a =. which turns back toward a = ; the periodic solution at a = is shown in Fig..22.

37 Numerical Analysis of Nonlinear Equations 37 MAX V Forcing Amplitude r Fig..23. Continuation of (.6) from r = to r = Periodically Forced and Non-Autonomous Systems In this section we illustrate computing periodic solutions to non-autonomous systems. The classical example of a non-autonomous system is a periodically forced system. In Auto periodic orbits of a periodically forced system can be computed by adding a nonlinear oscillator with the desired periodic forcing as one of its solution components. An example of such an oscillator is { x = x + βy x(x 2 + y 2 ), y = βx + y y(x 2 + y 2 ), which has the asymptotically stable solution x(t) = sin (βt), y(t) = cos (βt). As an example, consider again the FitzHugh-Nagumo equations of Sect..4.5, where we assume that the first component of the equations is periodically forced by r cos βt. Coupling the oscillator to the Fitzhugh-Nagumo equations gives: x = x + βy x(x 2 + y 2 ), y = βx + y y(x 2 + y 2 ), v = c(v (.6) 3 v3 + w ry), w = (v a + bw)/c, where we take b =.8, c = 3, and β =. For a = and r = there exists the solution

38 38 Eusebius J Doedel MAX V Scaled Time Fig..24. Solutions along the continuation path of (.6) from r = to r = 2. x(t) = sin (βt), y(t) = cos (βt), v(t), w(t). We continue this solution in the forcing amplitude r, from r = to, say, r = 2. The result is shown in Fig..23, with some of the solutions along this family plotted versus time in Fig..24. If the forcing is not periodic, or difficult to model by an autonomous oscillator, then the equations can be rewritten in autonomous form as follows. The non-autonomous system { u (t) = f(t,u(t)), u( ), f( ) R n, t [,], b(u(),u()) =, b( ) R n, can be transformed into u (t) = f(v(t),u(t)), u( ), f( ) R n, t [,], v (t) =, v( ) R, b(u(),u()) =, b( ) R n, v() =, which is autonomous, with n + ODEs and n + boundary conditions..5 Computing Connecting Orbits Orbits that connect fixed points of a vector field are important in many applications. A basic algorithm, which can be represented in various forms [25, 6, 9] consists of continuation of solutions to the equations

39 Numerical Analysis of Nonlinear Equations 39 u (t) = T f(u(t),λ), u( ), f(, ) R n, λ R n λ, (.7) { f(w,λ) =, (.8) f(w,λ) =, { fu (w,λ)v i = µ i v i, i =,..., n, (.9) f u (w,λ)v i = µ i v i, i =,..., n, { v i v i =, i =,..., n, vi v (.2) i =, i =,..., n, (f(u,λ) f(û, ˆλ)) f u (û, ˆλ)f(û, ˆλ) dt =, (.2) { n u() = w + ε i= c n iv i, i= c2 i =, n u() = w + ε i= c n iv i, i= c2 i =. (.22) Equation (.7) is the ODE with independent variable t scaled to [,]. Equation (.8) defines two fixed points w and w. We assume in (.9) that f u (w,λ) has n distinct real positive eigenvalues µ i with eigenvectors v i, and f u (w,λ) has n distinct real negative eigenvalues µ i with eigenvectors v i. Equation (.2) normalizes the eigenvectors. Equation (.2) gives the phase condition, with reference orbit û(t), which is a necessary condition for D(σ) = u (t + σ) û (t) 2 dt to be minimized over σ; here we use u (t) = f u (u,λ)u (t) = f u (u,λ)f(u,λ). Finally, (.22) requires u() to lie in the tangent manifold U at distance ε from w ; similarly, u() must lie in S at distance ε from w. Using (.22) we can eliminate w and w, to be left with n coupled differential equations subject to n c = 2n + (n + )(n + n ) + 3 constraints. In addition to u(t) R n we have scalar variables λ R n λ, ε, ε R, µ i, c i R, v i R n, i =,..., n, µ i, c i R, v i R n, i =,..., n. The total number of scalar variables equals n v = n λ + (n + 2)(n + n ) + 2. Formally, we need n v = n c n for a single heteroclinic connection; this gives n λ = n (n + n ) +. For a family of connecting orbits, we must use n (n + n ) + 2 free parameters. Note that T is large and fixed in this continuation.

40 4 Eusebius J Doedel v w (a) w u() v u() v w (b) w v v w (c) w v Fig..25. Geometric interpretation of the equations for computing heteroclinic orbits in the special case n = 2, n = n =. First consider the special case of a heteroclinic connection between two saddle points in R 2, that is, n = 2, n = n = ; a graphical illustration of this case is shown in Fig..25. Then n λ + = 2, i.e., a branch of heteroclinic orbits requires two free problem parameters λ = (λ, λ 2 ). Consider λ 2 as fixed here. For λ = λ we assume the existence of the heteroclinic connection in Fig..25(b). Generically, perturbation of λ will produce either Fig..25(a) or Fig..25(c), depending on the sign of the perturbation. If ε and ε are sufficiently small, then there exists a λ close to λ for which (.7) (.2) (and (.22)) can be satisfied; here, this is satisfied for λ as in Fig..25(a). Furthermore, the radii ε and ε can be chosen such that the period of the orbit equals a given large value T, and such that the phase condition (.2) is satisfied. Some more particular cases are:. The connection of a saddle to a node in R 2. Here n = 2, n =, n = 2, so n λ =. A branch of connections requires one problem parameter; 2. If n = 3, n = 3, n = 2, then n λ =, which means that a two-dimensional manifold of connecting orbits is already possible for fixed problem parameters;

41 Numerical Analysis of Nonlinear Equations 4 NORM U GAMMA Fig..26. Bifurcation diagram of the singularly-perturbed BVP (.23). 3. The homoclinic orbit. In this case w = w and n + n = n, so that n λ =. Such orbits can also be computed as the limit of periodic orbits as the period T..6 Other Applications of BVP Continuation We end this chapter with two examples where the boundary value continuation of Auto is applied in special contexts. Singularly Perturbed BVP Auto is well suited for computing solutions in systems with multiple timescales. The numerical sensitivity caused by the difference in timescales is dealt with by the orthogonal collocation solution technique with adaptive meshes. The pseudo-arclength continuation ensures detection of changes along the solution family. Consider the singularly perturbed system [26] with boundary conditions εu (x) = u(x) u (x) (u(x) 2 ) + u(x), u() = 3, u() = γ. 2 The computational formulation is in the form

42 42 Eusebius J Doedel U X Fig..27. Some solutions along the solution branch of (.23). with boundary conditions { u = u 2 u 2 = λ ( u u 2 (u 2 ) (.23) ) + u, ε u () = 3 2, u () = γ. The parameter λ is a homotopy parameter to locate a starting solution. In the first run λ varies from to and ε = is fixed. In the second run ε is decreased by continuation to the desired value. We use ε = 3. Once a starting solution is obtained, we continue the solution for ε = 3 in the parameter γ. This third run takes many continuation steps. Figure.26 shows the bifurcation diagram with the solution family obtained by continuation in γ. A selection of the solutions along the branch is shown in Fig Orbit Continuation in IVP One can also use continuation to compute solution families of initial value problems (IVP). Using continuation instead of integration of a large number of initial conditions has the advantage that the manifold described by the orbits is well covered, even in problems with very sensitive dependence on initial conditions. As an example, we consider the Lorenz equations given by x = σ(y x), y = ρx y xz, (.24) z = xy βz,

43 Numerical Analysis of Nonlinear Equations 43 MAX X RHO Fig..28. Bifurcation diagram of the Lorenz equations (.24). with σ = and β = 8/3. Let us first analyze the stationary solutions of (.24) as a function of ρ. A bifurcation diagram is shown in Fig..28. The zero solution is unstable for ρ >. Two nonzero (symmetric) stationary solutions bifurcate at ρ =. These nonzero stationary solutions become unstable for ρ > ρ H At ρ H there are Hopf bifurcations, and a family of unstable periodic solutions emanates from each of the Hopf bifurcation points; only the maximal x-coordinate is shown in Fig..28, and Fig..29 shows some of these periodic orbits in the (x, y)-plane. The families of periodic solutions end in homoclinic orbits (infinite period) at ρ 3.9. Now let ρ = 28. For this parameter value the Lorenz equations have a strange attractor. Let and write the Lorenz equations as u = x y, z u (t) = f(u(t)). The origin is a saddle point, with eigenvalues µ 2.66, µ , µ 3.82, and corresponding normalized eigenvectors v, v 2, and v 3, respectively. We want to compute the stable manifold of the origin. We compute an initial orbit u(t), for t from to T (where T < ), with u() close to in the eigenspace spanned by v and v 2, that is,

44 44 Eusebius J Doedel Y Fig..29. Periodic orbits of the Lorenz equations (.24). ( cos(θ) u() = + ε v sin(θ) ) v 2, µ µ 2 for, say, θ =. The IVP can be solved with Auto as follows. Scale time t t T. Then the initial orbit satisfies u (t) = T f(u(t)), t, X and The initial orbit has length u() = ε µ v. L = T f(u(s)) ds. Thus the initial orbit is a solution of the equation F(X) =, where X = (u( ), θ, T) (for given L and ε) and u (t) T( f(u(t)), cos(θ) F(X) = u() ε v sin(θ) v 2 ), µ µ 2 T f(u(s)) ds L. Once the initial orbit has been integrated up to a sufficiently long arclength L, we can use pseudo-arclength continuation to find a family of solution segments

45 Numerical Analysis of Nonlinear Equations 45 (a) (b) Fig..3. The stable manifold of the origin in the Lorenz equations (.24). Panel (a) shows the family of orbits that represent part of the manifold. Panel (b) shows another section of the Lorenz manifold. that forms an approximation of the Lorenz manifold, the stable manifold of the origin. The set-up for pseudo-arclength continuation is now:

46 46 Eusebius J Doedel F(X ) =, (X X ) Ẋ s =, ( Ẋ = ), with X = (u( ), θ, T) and L and ε fixed. It is important to note here that we do not just change the initial point (i.e., the value of θ). The continuation stepsize s measures the change in X. An impression of part of the computed Lorenz manifold is shown in Fig..3. For more detailed results see [3]..7 Outlook We discussed the set-up in Auto for the numerical continuation of families of solutions to first-order systems of ordinary differential equations. Auto uses Keller s pseudo-arclength continuation [22], which can equally well be applied to solution families of algebraic problems, e.g., families of stationary solutions. When applied to families of orbits, each continuation step involves solving a boundary value problem. Auto uses piecewise polynomial collocation with Gauss-Legendre collocation points (orthogonal collocation) [7, 3], similar to Colsys [2] and Coldae [4], with adaptive mesh selection [32]. The basic objective behind the continuation methods of Auto is the ability to perform a numerical bifurcation analysis. Such computational results give a deeper understanding of the solution behavior, stability, multiplicity, and bifurcations, and they often provide direct links to the underlying mathematical theories. We highlighted only the basic set-up in Auto. For multi-parameter bifurcation analysis the system that implicitly defines the solution branch is extended to contain bifurcation conditions; see, for example, [, 2]. By monitoring the appropriate bifurcation condition Auto detects, say, a Hopf bifurcation when continuing a family of stationary solutions in one parameter. This bifurcation point can subsequently be continued by extending the set-up for pseudo-arclength continuation with extra equations (the bifurcation condition), and freeing a second parameter. For so-called minimally extended systems see [2, 24]. There is a need for further refinement of existing continuation algorithms and software for bifurcation analysis, and there is a need for their extension to new classes of problems. Probably the greatest challenges lie in the development of numerical continuation and bifurcation software for partial differential equations. There is such a package for scalar nonlinear elliptic PDEs on general domains in R 2 [5], which is based on multigrid solution techniques; see also [27, 28, 29]. Good results have also been obtained with stabilized simple iteration schemes for computing stationary PDE solutions with mostly stable modes ; see, for example, [33]. There remains a need for general bifurcation software for systems of elliptic PDEs, subject to general boundary conditions and integral constraints. For the case of such systems on simple domains in R 2, the generalization of the collocation method of Sect..2.3 carries some promise. To become comparable in performance to current ODE bifurcation