Extrapolation and the Cayley Transform

Transcription

1 Extrapolation and the Cayley Transform Apostolos Hadjidimos Department of Computer and Communication Engineering University of Thessaly GR Volos, Greece and Michael Tzoumas c/o Department of Mathematics, University of Ioannina, GR Ioannina, Greece HERCMA 2007: Athens, Greece (20-22/09/2007) This work is dedicated to all those innocent victims, all over Greece, who have lost their lives or their fortunes and properties in the recent big fires

2 Outline of talk: 1. The Cayley Transform 2. Applications 3. The Optimization Problem 4. Generalizing Optimization Problem / Möbius Transformation 5. Analyzing Möbius Transformation / Geometric Interpretation 6. Propositions 7. The Algorithm / Comments on Algorithm 8. Examples 9. Notes / References 2

3 1. The Cayley Transform Definition 1. Given the Cayley Transform is defined by A C n,n, 1 /σ(a), (1) F := C(A) = (I + A) 1 (I A). (2) (See, e.g., Fallatt and Tsatsomeros, ELA (2002), for properties and other references). Definition 2. Under the assumptions of Definition 1, we call Extrapolated Cayley Tranform, with extrapolation parameter ω, the function F ω := (I + ωa) 1 (I ωa), ω C\{0}, 1 /σ(ωa). (3) Basic Assumption: From now on it is assumed that A R n,n is positive stable, that is its eigenvalues a σ(a) have Re a > 0. 3

4 2. Applications The (Extrapolated) Cayley Transform or their scalar analogues (e.g., w = 1+ωa 1 ωa, (Möbius transformation)), appear, among others, in the solution of: 1) The Linear Complementarity Problem (LCP) when the basic matrix A is, in addition, real symmetric positive definite, by the Modulus Algorithm (W.M.G. van Bokhoven (1981)). 2) The problem of the determination of optimal acceleration parameter in the a) Classical Stationary Alternating Direction Implicit (ADI) Iterative Method (D.W Peaceman and H.H. Rachford Jr. SINUM (1955)). b) A complex linear system, with matrix coefficient positive stable by i) The HS Splitting (Z.-Z. Bai, G.H. Golub and M.K. Ng SIMAX (2003)). ii) The NS Splitting (Z.Z. Bai, H.G. Golub and M.K. Ng NLAA (2006)). 4

5 3. Optimization Problem Problem I: For A R n,n positive stable, determine the Extrapolation Parameter ω (> 0) that minimizes the spectral radius of the Extrapolated Cayley Tranform, i.e. min ρ(f ω) = min max ω>0 ω>0 a σ(a) 1 ωa (< 1). (4) 1 + ωa 5

6 4. Generalizing Optimization Problem / Möbius Transformation Definition 3. Convex Hull H of σ(a) is the smallest convex polygon that contains σ(a) in the closure of its interior. Since A is real positive stable, σ(a) and H are symmetric (wrt) the positive real semiaxis. Problem II: Determine the extrapolation parameter ω that solves the minimax problem min max ω>0 a H 1 ωa (< 1). (5) 1 + ωa 6

7 Consider the function w := w(a) = 1 ωa, a H, ω > 0. (6) 1 + ωa i) w is a Möbius transformation ii) w has no poles iii) w(a) is not the constant function iv) w(a) maps the point a onto w of the same complex plane. Its inverse transformation w 1 satisfies w 1 (w(a)) = a = i) w 1 is a Möbius transformation ii) w 1 has no poles iii) w 1 is not the constant function iv) w 1 maps back w onto its pre-image a. 1 w, w = w(a), a H, ω > 0. (7) ω(1 + w) 7

8 5. Analyzing Möbius Transformation / Geometric Interpretation Consider the images w(σ(a)) and w(h) through (6). Analyze into a sequence of elementary transformations 1 w = 2 1. (8) 1 + ωa a w 1=ωa w w 2 =1+w w 3 = 1 w 1 1 w 2 w 2 w 4 =2w 3 3 w 4 w=w 4 1 w (9) and w 3 = w 4 w 2 = 1 w 2 4 w 3 w w 4=w+1 w 3 w 2 w 1 =w 2 1 a= w 1 w ω 1 a. (10) 8

9 Interpret each simple transformation (9) in geometric terms: i) w 1 = ωa : Similitude (Homothesy), center at O(0, 0), ratio ω ii) w 2 = w : Translation by +1 iii) w 3 = 1 w 2 : Inversion, pole at z = 0 iv) w 4 = 2w 3 : Similitude, center at z = 0, ratio 2 v) w = w 4 1 : Translation by 1. 9

10 Observe that: Similitudes: centers at z = 0, ratios positive real numbers, Translations: parallel to real axis, Inversion: pole at point z = 0 lying strictly outside pre-images it inverts. 10

11 Interpret each simple transformation (10) in geometric terms: i) w 4 = w + 1 : Translation by +1. ii) w 3 = w 4 2 : Similitude (Homothesy), center at O(0, 0), ratio 1 2. iii) w 2 = 1 w 3 : Inversion, pole at z = 0. iv) w 1 = w 2 1 : Translation by 1. v) a = w 1 ω : Similitude, center at z = 0, ratio 1 ω. 11

12 Observe that: Translations: parallel to real axis, Similitudes: centers at z = 0, ratios positive real numbers, Inversion: pole at point z = 0 lying strictly outside pre-images it inverts. 12

13 Images w(σ(a)) and w(h) will be symmetric wrt the real axis (due to the nature of the Möbius transformations (6) and (7) (real coefficients and no poles)). For an ω > 0, let C ω the circle, center at O(0, 0) and radius ρ := ρ(c ω ) = max a H w(a) (< 1). (11) C ω passes through a boundary point of w(h). Therefore, in view of nature of the inverse Möbius transformation (7), C ω must be image of a circle C. 13

14 C ω : w = ρ w 2 = ρ 2 ww = ρ 2 (12) ωa 1 ωa 1 + ωa = ρ2 ω 2 (1 ρ 2 )aa ω(1+ρ 2 )(a+a)+(1 ρ 2 ) = 0 aa (1 + ρ2 ) ω(1 ρ 2 ) (a + a) + 1 ω 2 = 0 aa (1 + ρ2 ) ω(1 ρ 2 (a + a) + ) ( (1 + ρ 2 ) ω(1 ρ 2 ) ) 2 = ( (1 + ρ 2 ) ω(1 ρ 2 ) ) 2 1 ω 2 14

15 where a (1 + ρ2 ) ω(1 ρ 2 ) c := a (1 + ρ2 ) ω(1 ρ 2 ) = 2 2ρ = ω(1 ρ 2 ) 1 + ρ2 ω(1 ρ 2 ), R := 2ρ ω(1 ρ 2 ) ( 2ρ ω(1 ρ 2 ) ) 2 a c = R : C, (13) (c > R 0). (14) From (14) solving for ω and ρ we obtain ω = 1 (15) c 2 R2, ρ = c c 2 R 2 R = c + R c R c + R + c R. (16) 15

16 a C Cω Ο ρ w θ ρ 2 w 3 ωr ρ w 4 w 1 R ωr w 2 Geometric representation of the five transformations of a cc C to C ω and vice versa 16

17 By virtue of (11), (12) and (13), circle C 1) will have center on positive real semiaxis, 2) will lie in open right half complex plane, 3) will pass through a boundary point of H, namely a vertex (and its symmetric wrt positive real semiaxis), and 4) will contain all vertices of H in the closure of its interior. Definition 4. A circle C satisfying the above four properties will be called capturing circle (cc) of H.. Note: There are infinitely many cc s of a certain H. From now on the word captures will mean contains in the closure of its interior 17

18 It has been proved that: ρ(f ω ) max w(a) ρ(w(c)) max w(a) ρ(c ω) ρ. a σ(a) a H Theorem 1. The solutions to Problems I and II are identical min ρ(c ω) = min max ω>0 ω>0 a H 1 ωa 1 + ωa, Re a > 0. (17) 18

19 6. Propositions Theorem 2. Let C be a cc of H, K(c, 0) (c > 0) and R (< c) be its center and radius, respectively, and C ω be its image via (6). Then, from (14) solving for ω and ρ, it is obtained that ω = 1 c 2 R 2, ρ := ρ (C ω) = c + R c R c + R + c R. (18) 19

20 Lemma 1. The function f(x) := 1 + x 1 x 1 + x + 1 x (19) is continuously increasing in [0, 1). Also, for x [d, e) [0, 1), f(x) attains its minimum at the minimum value of x = d. Theorem 3. The solution to either Problem I or Problem II is equivalent to the determination of the optimal capturing circle (cc) C of H so that the ratio R c is a minimum. 20

21 7. The Algorithm / Comments on Algorithm The extrapolation problem in a simpler case, 1 ωa, was solved In the Real Case in A.J. Hughes Hallett Proceedings (1981), CAM (1982) A.H. IJCM (1983) In the Complex Case in A.H. LAA (1984), LAA (2004) G. Opfer and G. Schober LAA (1984) Under the notation and the assumptions made so far, C of H, of Theorem 3, is determined as follows: The Algorithm Step 1. P i (β i, γ i ), i = 1(1)k, 0 < β i < β i+1, i = 1(1)k 1, γ i 0, i = 1(1)k, vertices of H, in the first quadrant of the complex plane. 21

22 Step 2. Find P i corresponding to largest polar angle θ i, max tan θ i = i=1(1)k max i=1(1)k γ i β i. (20) If two such vertices exist go to Step 3; otherwise Let i {1, 2,..., k} be the index and C i the circle that is tangent to the line OP i at P i and has center on the real axis. If C i captures H, it is C ω of H (one-point optimal cc). If no such a cc C ω exists go to next Step. Step 3. Determine all circles through pairs of vertices P i, P j, i = 1(1)k 1, j = i + 1(1)k, with centers on the real axis. Let K i,j (c i,j, 0) and R i,j = (K i,j P i ) = (K i,j P j ) centers and radii. Discard those that either capture O or do not capture H. From remaining the one corresponding to the smallest ratio R i,j (OK i,j ) is C ω of H (two-point optimal cc). 22

23 P * R P O θ A * K ( c*, 0 ) * * A Optimal elements in a One-point optimal cc 23

24 θ c Optimal elements in a Two-point optimal cc 24

25 a) Let i {1, 2,..., k} be the index corresponding to the optimal one-point cc. Then, its center and radius are given by K i (c i, 0), c i = β2 i + γ2 i β i, R i = γ i β 2 i + γ2 i β i. (21) b) To determine optimal two-point cc from centers K i,j and radii ( ) k R i,j of the possible cc s, find 2 c i,j = (β2 j + γ2 j ) (β2 i + γ2 i ), R i,j = 2(β j β i ) [(β 2 j + γ2 j ) + (β2 i + γ2 i ) 2β iβ j ] 2 4γ 2 i γ2 j 2(β j β i ) (22) Discard circles for which c i,j 0 or 0 < c i,j R i,j. From rest find optimal cc as the unique cc that captures the other k 2 vertices of H and corresponds to the smallest ratio. R i,j c i,j = [(β 2 j + γ 2 j ) + (β2 i + γ2 i ) 2β iβ j ] 2 4γ 2 i γ2 j (β 2 j + γ2 j ) (β2 i + γ2 i ). (23) 25

26 8. Examples LCP A R n,n, det(a) 0, b R n, R n x 0, r = Ax b 0, x T r = 0 Example 1: A = tridiag( 1, 2, 1) R 9,9, b = [ ] T. A is symmetric positive definite (P matrix and positive stable). Its extreme eigenvalues are a m = 4 sin 2 ( π 20 ) and am = 4 cos 2 ( π 20 ). Using the Modulus Algorithm, with C = (I + A) 1 (I A), C 2 = ρ(c) = , 125 iterations for LCP solution. Using the Optimal (Extrapolated) Modulus Algorithm, ω = , C ω 2 = ρ(c ω ) = , 93 iterations for same LCP solution. x = [ ] T, r = Ax b = [ ] T. (24) 26

27 Example 2: A = [ ], b = [ 2 1 A positive stable, σ(a) = {1, 2}. A P matrix, all its principal minors are positive. 32 iterations for LCP solution. x = [2 0] T, r = [ ] T. ω = 1 2 = , ρ(c ω ) = iterations for same LCP solution. ]. 27

28 NOTES 1) The Optimal Capturing Circle C ω has also been found in case the Convex Hull H under consideration is : a) A Circle or a Circular Region (Sector, Section or Zone) and b) An Ellipse or an Elliptical Region (Sector, Section, or Zone). 2) The Modulus Algorithm by van Bokhoven and also its extension by Kappel and Watson (1996) can be improved by stationary and/or nonstationary extrapolation. 3) The Optimal Extrapolation Parameter ω of the present work generalizes a known Mathematical Problem. Consider the Poisson Equation in the Unit Square under Dirichlet Boundary Conditions. Use a 5-Point Discretization with Equal Mesh Size in each Co-ordinate Direction. Then the Optimal Acceleration Parameter r of the corresponding Stationary Alternating Direction Implicit (ADI) Iterative Method (Peaceman-Rachford Method) is found as the solution to the following Minmax Problem 28

29 Rewrite it as min max r>0 0<δ a ɛ min 1 r >0 max 0<δ a ɛ r a r + a 1 1 r a r a. In this Problem H := [δ, ɛ] and ω := 1 r! 4) For the solution of a Complex Linear System by the Method of a) Hermitian/Skew Hermitian Splitting (H is as above). b) Normal/Skew Hermitian Splitting (H is a rectangle). If some more information on the spectrum is known it can be improved further using the Algorithm described. 29

30 References 1. H. Samelson, R.M. Thrall and O. Wesler, Proc. Amer. Math. Soc. (1958) 2. G.B. Dantzig and R.W. Cottle, In Nonlinear Programming, North-Holland (1967) 3. C.E. Lemke, In Mathematics of the Decision Sciences, AMS (1968) 4. O.L. Mangasarian, Nonlinear Programming. McGraw Hill, (1969) (Reprint: SIAM (1994)) 5. A. Berman and R.J. Plemmons, Nonnegative Matrices in the Mathematical Sciences, SIAM (1994) 30

31 6. C.W. Cryer, SIAM J. Control (1971) 7. O.L. Mangasarian, J. Opt. Theory Appl. (1977) 8. B.H. Ahn, J. Opt. Theory Appl. (1981) 9. J.S. Pang, J. Opt. Theory, Appl. (1984) 10. R.W. Cottle, J.-S. Pang and R.E. Stone, The Linear Complementarity Problem, Academic Press (1992) 11. K.G. Murty, Linear Complementarity, Linear and Nolinear Programming, Internet Edition (1997)

32 12. R.S. Varga, Matrix Iterative Analysis. Prentice-Hall (1962) (Also: 2nd Edition, Revised and Expanded, Springer (2000)) 13. D.M. Young, Iterative Solution of Large Linear Systems, Academic Press (1971) 14. E. Hille, Analytic Function Theory, 4th Printing, Blaisdel (1965) 15. H.S.M. Coxeter, Introduction to Geometry, John Wiley & Sons (1989) 16. A.J. Hughes Hallett, In Proceedings of the Econometric Society European Meeting 1979, North-Holland (1981) 17. A.J. Hughes Hallett, J. Comput. Appl. Maths (1982)

33 18. A. Hadjidimos, Intern. J. Comput. Math. (1983) 19. A. Hadjidimos, Linear Algebra Appl. (1984) 20. G. Opfer and G. Schober, Linear Algebra Appl. (1984) 21. A. Hadjidimos, Linear Algebra Appl. (2004) 22. K. Pantazopoulos, PhD Thesis, Department of Computer Sciences, Purdue Univ. (1999) 23. M.D. Koulisianis and T.S. Papatheodorou, Appl. Numer. Math. (2003)