Fixed Points and Contractive Transformations. Ron Goldman Department of Computer Science Rice University

Transcription

1 Fixed Points and Contractive Transformations Ron Goldman Department of Computer Science Rice University

2 Applications Computer Graphics Fractals Bezier and B-Spline Curves and Surfaces Root Finding Newton s Method Trivial Fixed Point Method Solving Large Systems of Linear Equations -- Relaxation Methods Jacobi Gauss-Seidel Existence and Uniqueness Theorem for Ordinary Differential Equations Reconstruction of Curves and Surfaces from Point Clouds

3 Part I: Theory

4 Fixed Points of Functions Fixed Point F(P) = P Example F(x) = x 2 F(0) = 0 and F(1) =1

5 Fixed Points by Iteration Iteration T 1 = F(T 0 ) T 2 = F 2 (T 0 ) = F(T 1 ) F(T ) = F +1 (T 0 ) = T ( fixed point) T = F (T 0 ) Another Iteration S 1 = F(S 0 ) S 2 = F 2 (S 0 ) = F(S 1 ) F(S) = F +1 (S 0 ) = S ( fixed point) S = F (S 0 ) Observations Iteration Might Not Converge If Iteration Converges Fixed Point Different Starting Values May Generate Different Fixed Points

6 Contractive Transformation Contractive Map (T) Dist( T(P), T(Q) ) sdist(p,q) 0 < s <1 Examples Uniform Scaling -- 0 < s <1 Image of Three Points -- Triangle Shrinks on All Sides Necessary Condition Necessary and Sufficient Conditions Det(T) < 1 Eigenvalues(T ) <1 Lemma T Contractive T Continuous

7 Iterations and Transformations Iteration -- Arbitrary Transformations Either Converges to a Fixed Point or Diverges Possibly Many Distinct Fixed Points Starting Point Matters Iteration -- Contractive Transformations Always Converges to a Fixed Point Fixed Point is Unique Starting Point does not Matter

8 Fixed Points of Contractive Maps Theorem 1: Suppose that T is a Contractive and P n +1 = T(P n ) for all n 0. If P = Lim n P n exists, then P is a fixed point of T. Proof: T(P) = T( Lim n P n ) = Lim n T(P n ) = Lim n P n+1 = P. Theorem 2: T Contractive Fixed Point is Unique. Proof: If P and Q are both fixed points of T, then T(P) = P and T(Q) = Q Dist( T(P), T(Q) ) = Dist(P,Q). Hence T is not contractive. Contradiction.

9 Cauchy Sequences Definition A sequence P n { } is Cauchy if Dist(P n +m, P n ) < ε for all n > N. Intuition A sequence of points P n n gets larger and larger. { } is Cauchy if the points get closer and closer as Cauchy s Theorem (Completeness) Every Cauchy sequence converges.

10 Triangular Inequality P 3 P 1 P 2 Dist(P 1,P 3 ) Dist(P 1,P 2 )+ Dist(P 2,P 3 ) Dist(P 1,P n ) Dist(P 1,P 2 )+ Dist(P 2,P 3 )+L+ Dist(P n 1,P n )

11 Contractive Maps and Cauchy Sequences Theorem 3 Suppose that T is a Contractive Map, and P n +1 = T(P n ) for all n 0. Then P n { } is a Cauchy Sequence for Any Choice of P 0. Pr oof: Dist P n+1, P n ( ) = Dist( T(P n ),T(P n 1 )) ( ) = sdist( T(P n 1 ),T(P n 2 )) sdist P n, P n 1 M ( ) s n Dist P 1, P 0 Dist( P n+m+1,p n ) Dist( P n+m+1,p n+m )+L+ Dist( P n+1,p n ) ( ) (s n+m +L+ s n )Dist P 1,P 0 s n Dist P 1,P 0 ( ) 1 s < ε

12 Trivial Fixed Point Theorem Trivial Fixed Point Theorem Suppose that T is a Contractive Map P n +1 = T(P n ) for all n 0. Then Lim n P n is the Unique Fixed Point of T for Any Choice of P 0! Proof: The Sequence { P n } is Cauchy. (Theorem 3) Therefore, Lim n P n Exists. (Cauchy s Theorem) Hence, Lim n P n is a Fixed Point of T. (Theorem 1) Moreover, this Fixed Point is Unique. (Theorem 2)

13 Part II: Applications

14 Equation Solving Finding Roots of Real Valued Functions Guess and Iterate -- Newton s Method -- Trivial Fixed Point Method Solving Large Systems of Linear Equations Relaxation Methods -- Jacobi -- Gauss-Seidel

15 Generating Curves and Surfaces Ordinary Differential Equations Existence and Uniqueness Theorem Solutions by Iteration Fractals Iterated Function Systems Fractal Algorithm

16 Root Finding by Newton s Method Problem Solve F(x) = 0 Newton s Method 1. Replace F(x) by G(x) = x F(x) / F (x) F(x) has a root G(x) has a Fixed Point F(x*) = 0 G(x*) = x * 2. Select an initial guess x 0 for x *. 3. Compute x k+1 = G(x k ) = x k F(x k ) / F (x k ) 4. Stop when x k +1 x k < ε for k = 1,K,n. Observations May Diverge -- F (x k ) 0 May Converge to Different Roots for Different Initial Guesses

17 Finding a Fixed Point by Iteration y = G(x 0 ) y = x x 1 = G(x 0 ) Fixed Point G( x ˆ ) = x ˆ y = G(x) x 0 x 2 ˆ x x 3 x1

18 Root Finding by Trivial Fixed Point Method Problem Solve F(x) = 0 Fixed Point Method 1. Replace F(x) by G(x) = x + F(x) F(x) has a root G(x) has a Fixed Point F(x*) = 0 G(x*) = x * 2. Select any(!) initial guess x 0 for x *. 3. Compute x k +1 = G(x k ) for k = 1,K,n. 4. Stop when x k +1 x k < ε Observation Works when G(x) is a Contractive Map.

19 Root Finding for Differentiable Functions Mean Value Theorem G(b) G(a) G (c)(b a) a c b Observations G (c) < 1 G(x) is Contractive G (c) < 1/ 2 Fixed Point Method is Faster than Bisection Method Example Find a root of F(x) = cos(x) x Find a fixed point of G(x) = cos(x)

20 Solving Large Systems of Linear Equations System of Linear Equations M 11 x 1 + M 12 x 2 +L+ M 1n x n = b 1 M 21 x 1 + M 22 x 2 +L+ M 2n x n = b 2 M M x i = b i M ii M n1 x 1 + M n2 x 2 +L+ M nn x n = b n M ij x j 1 i n M j i ii Matrix Form M 11 M 12 L M 1n x 1 b 1 M 21 M 22 L M 2n x 2 = b 2 M M O M M M M n1 M n2 L M nn 3 { x n { b n M X B M X = B

21 Relaxation Methods Jacobi Relaxation p b x i = i M ii M ij p 1 x M j j i ii 1 i n Gauss-Seidel Relaxation i 1 M ij p b x i = i p M M ii x j ij M j=1 ii n p 1 x M j j=i+1 ii 1 i n

22 System of Linear Equations Jacobi Relaxation M 11 x 1 + M 12 x 2 +L+ M 1n x n = b 1 M 21 x 1 + M 22 x 2 +L+ M 2n x n = b 2 M M M X = B B M X = 0 M n1 x 1 + M n2 x 2 +L+ M nn x n = b n. Equivalent System of Linear Equations M 11 x 1 = b 1 M 12 x 2 L M 1n x n M 22 x 2 = b 2 M 21 x 1 L M 2n x n M M M D X = B (M D) X M nn x n = b n M n1 x 1 L M 11 0 L 0 0 M D = 22 L 0 M O O M 0 L 0 M NN

23 Jacobi Relaxation (continued) Equivalent System of Linear Equations x 1 = b 1 M 11 M 12 M 11 x 2 L M 1n M 11 x n x 2 = b 2 M 22 M 21 M 22 x 1 L M 2n M 22 x n M M M X = D 1 B D 1 (M D) X x n = b n M nn M n1 M nn x 1 L Fixed Point T(X) = D 1 B +(I D 1 M)X T(X) = X (fixed point)

24 Gauss-Seidel Relaxation System of Linear Equations M 11 x 1 + M 12 x 2 +L+ M 1n x n = b 1 M 21 x 1 + M 22 x 2 +L+ M 2n x n = b 2 M M M X = B B M X = 0 M n1 x 1 + M n2 x 2 +L+ M nn x n = b n. M n1 x 1 + M n2 x 2 +L+ M nn x n = b n. Equivalent System of Linear Equations. M 11 x 1 = b 1 M 12 x 2 L M 1n x n M 21 x 1 + M 22 x 2 = b 2 M 23 x 1 L M 2n x n M M L X = B (M L) X M n1 x 1 +L+ M nn x n = b n M 1,1 0 L 0 M 2,1 M 2,2 L 0 L = M O O M M n,1 L M n 1,n 1 M n,n

25 Gauss-Seidel Relaxation (continued) Equivalent System of Linear Equations L X = B (M L) X X = L 1 B L 1 (M L) X Fixed Point T(X) = L 1 B+ L 1 (L M)X T(X) = X (fixed point)

26 Diagonally Dominant Convergence Both methods converge for any initial guess when M is diagonally dominant M ii M ij (diagonally dominant) j i Fixed Point M X = B Q X = (Q M )X + B X = (I Q 1 M )X +Q 1 B T(X) = (I Q 1 M)X +Q 1 B T(X) = X (fixed point) Contractive Map T is a contractive map when -- M is diagonally dominant -- Q is the diagonal part of M (Jacobi) -- Q is the lower triangular part of M (Gauss-Seidel)

27 Ordinary Differential Equations General Case y(n) = a n 1 (x)y (n 1) +L+ a 1 (x)y ' + a 0 (x)y +b(x) (Differential Equation) y(0) = c 0,K,y (n 1) (0) = c n 1 (Initial Conditions) Example 1 y'= y y(0) =1 y = e x Example 2 y''= y y(0) =1 y'(0) = 0 y = cos(x)

28 Contractive Map for Ordinary Differential Equations Ordinary Differential Equation y(n) = a n 1 (x)y (n 1) +L+ a 1 (x)y ' + a 0 (x)y +b(x) (Differential Equation) y(0) = c 0,K,y (n 1) (0) = c n 1 (Initial Conditions) Ordinary Differential Equation y(n) = F(x,y,y',K,y (n 1) ) (Differential Equation) y(0) = c 0,K,y (n 1) (0) = c n 1 (Initial Conditions) Contractive Map T( f )(x) = c 0 +c 1 x +L+ c n 1 xn 1 (n 1)! x t t + L F t n, f (t n ), f '(t n ),K, f (n 1) 1 n 1 (t n ) dt n Ldt 1 Fixed Point of T = Solution of ODE

29 Examples Example 1 y'= y y(0) =1 x 0 T( f )(x) =1+ f (x) dx T(e x x ) =1+ e t 0 dt = e x Example 2 y''= y y(0) =1 y'(0) = 0 x 0 T( f )(x) =1+ f (u) dudt T cos(x) t 0 x t ( ) =1+ cos(u) 0 0 du dt = cos(x)

30 Solving Ordinary Differential Equations Ordinary Differential Equation y(n) = a n 1 (x)y (n 1) +L+ a 1 (x)y ' + a 0 (x)y +b(x) (Differential Equation) y(0) = c 0,K,y (n 1) (0) = c n 1 (Initial Conditions) Fixed Point Method 1. Select any continuous function f 0 (x) as the initial guess. 2. Compute f k +1 = T( f k ) k =1,K,n. 3. Stop when f k +1 f k < ε.

31 Spaces and Functions Euclidian Space Points Function Space Continuous Functions on Closed Intervals Euclidian Distance f g = Max f g Cauchy Sequences of Points Converge Contractive Maps Fixed Points Trivial Fixed Point Theorem Cauchy Sequences of Functions Converge Integrals Solutions of ODE s Trivial Fixed Point Theorem Root Finding Algorithm Algorithm for Solving ODE s Start with Any Point and Iterate Start with Any Function and Iterate