Introduction to Simulation - Lecture 9. Multidimensional Newton Methods. Jacob White

Transcription

1 Introduction to Simulation - Lecture 9 Multidimensional Newton Methods Jacob White Thanks to Deepak Ramaswamy, Jaime Peraire, Michal Rewienski, and Karen Veroy

2 Outline Quick Review of -D Newton Convergence Testing Multidimensonal Newton Method Basic Algorithm Description of the Jacobian. Equation formulation. Multidimensional Convergence Properties Prove local convergence Improving convergence

3 -D Reminder Newton Idea ) * * Problem: ind such that f = Use a Taylor Series Epansion * f ) ) * f * f = f If is close to the eact solution f ) ) 2 ) ) ) * ) f ) 2

4 -D Reminder Newton Algorithm k = = Initial Guess, Repeat { f k k ) k+ k ) k = f ) = k + } Until? < threshold + ) k+ k k? f < threshold?

5 -D Reminder Newton Algorithm Algorithm Picture

6 -D Reminder Newton Algorithm Convergence Checks Need a "delta-" check to avoid false convergence f) > ε + ε k + k k + a r k + k * k + ) f < ε f a X

7 -D Reminder Newton Algorithm Convergence Checks Also need an " f ) " check to avoid false convergence f) k + ) f > ε f a * k + k X < ε + ε k + k k + a r

8 -D Reminder Newton Algorithm Local Convergence Convergence Depends on a Good Initial Guess f) 2 X

9 Multidimensional Newton Method Eample Problem Strut and Joint L ) = f f y + L = + = L y l = + y 2 2 lo l) = EAc = ε lo l) lo f = = ε lo l) l l y y f y = = ε lo l) l l l y l ε ε l l) + = o L l l) + = o OR L y

10 Multidimensional Newton Method Eample Problem Nonlinear Resistors i + b v - v i b 2 + v2 - v 2 i 3 Nonlinear Resistors i = g v ) + b v3 - Nodal Analysis At Node : i + i = 2 ) ) g v + g v v = At Node 2: i i = ) ) g v g v v = 3 2 Two coupled nonlinear equations in two unknowns

11 Multidimensional Newton Method General Setting ) * * Problem: ind such that = and : * N N N Use a Taylor Series Epansion *) ) ) * J ) = + + H. O. T. Jacobian Matr i If is close to the eact solution J ) * ) )

12 Multidimensional Newton Method Nodal Analysis Strut and Joint and : * L l y l ε ε l l) + = o L l l) + = o L y J )?? =??

13 + v b - Multidimensional Newton Method v i 2 + v b 2 - v i 2 i 3 Nodal Analysis and : * v b 3 - Nonlinear Resistor At Node : i+ i2 = v = g v + g v v = ) ) ) 2 At Node 2: i3 i2 = v = g v g v v = ) ) ) 2 3 2?? J ) =??

14 Multidimensional Newton Method Jacobian Matri J + ) ) ) ) ) N J ) ) ) N N N N

15 Multidimensional Newton Method Jacobian Matri Singular Case Suppose J is singular? ) ) ) N J ) = = ) ) N N N N What does it mean?

16 Multidimensional Newton Method Newton Algorithm k = = Initial Guess, Repeat { k) k) Compute, J Solve k = k + ) ) = ) J for k k+ k k k+ k k } Until k + f + ), small enough

17 Multidimensional Newton Method Computing the Jacobian and the unction Consider the contribution of one nonlinear resistor Connected between nodes n and n 2 b b i + v - b b i = g v ) n n2 Summing currents at Node n : n v g vn vn = + ) ) ) ) 2 Summing currents at Node n 2: n v g vn vn Differenting at Node n : n v = v) g vn vn ) n v) g vn vn ) n 2 = + v n g v v n 2 = + v g v n 2 2

18 Multidimensional Newton Method g v ) ) n v n g v 2 n v n2 v v g v ) ) n v n g v 2 n v n2 v v J v) i Computing the Jacobian and the unction b v + - n n2 n n2 n n n2 n 2 Stamping a Resistor g v v ) g v ) n v n2 v ) n n 2

19 Multidimensional Newton Method Initial Guess, k = = Repeat { k) k) Compute, J More Complete Newton Algorithm Zero J and for each element Compute element currents and derivatives Sum currents to, sum derivatives to J Solve k = k + ) ) = ) J for k k+ k k k+ k k } Until k + f + ), small enough

20 Multidimensional Newton s Method Eample: Heat low in leaky bar T ) T ) T TN i = kt+ kt h 2 3 vs = T) vs = T) What is the Jacobian?

21 Multidimensional Newton Method If Main Theorem k ) ) a J β ) Inverse is bounded Multidimensional Convergence Theorem Theorem Statement ) ) ) b) J J y y Derivative is Lipschitz Cont Then Newton s method converges given a sufficiently close initial guess

22 Multidimensional Newton Method Multidimensional Convergence Theorem Key Lemma If ) ) Derivative is Lipschitz Cont) J J y y Then ) y) J y) y) y 2 2 There is no multidimensional mean value theorem.

23 Multidimensional Newton Method Multidimensional Convergence Theorem Theorem Proof By definition of the Newton Iteration and the assumed bound on the inverse of the Jacobian ) ) β ) = J k+ k k k k Again applying the Newton iteration definition ) ) ) ) k+ k k k k k k β J inally using the Lemma β 2 k+ k k k 2

24 Multidimensional Newton Method Multidimensional Convergence Theorem Theorem Proof Continued Reorganizing the equation β 2 k+ k k k k k β 2 γ < If k+ k k k+ k k = ) γ + converges

25 Non-converging Case f) -D Picture X Must Somehow Limit the changes in X

26 Newton Method with Limiting Newton Algorithm Newton Algorithm for Solving ) = k = = Initial Guess, Repeat { k) k) Compute, J Solve ) ) J = for k k+ k k+ ) = + limited k+ k k+ k = k + k } Until k + + ), small enough

27 Newton Method with Limiting Limiting Methods Direction Corrupting limited k + ) = NonCorrupting limited k ) i = α + k+ γ = min, α k + if < γ k+ k+ i i k + ) γ sign i otherwise limited limited γ k + ) γ k + ) k + k + Heuristics, No Guarantee of Global Convergence

28 Newton Method with Limiting Damped Newton Scheme General Damping Scheme Solve ) ) J = for k k+ k k+ = + α k+ k k k+ Key Idea: Line Search Pick α to minimize + + α ) 2 k k k k 2 k k k+ ) k k k+ ) k k k+ + α + α + α ) 2 Method Performs a one-dimensional search in Newton Direction 2 T

29 Newton Method with Limiting If Damped Newton Convergence Theorem k ) ) a J β ) Inverse is bounded ) ) ) b) J J y y Derivative is Lipschitz Cont Then ] k There eists a set of α ' s, such that k+ ) k k k+ ) k = + < ) α γ with γ< Every Step reduces -- Global Convergence!

30 Newton Method with Limiting Initial Guess, k = = Repeat { Solve k) k) ) ) Compute, J J = for Damped Newton Nested Iteration k k+ k k+ ] k k k+ + α ) k ind α, such that is minimizd e = + α k k+ k k k+ = k + k } Until k + + ), small enough How can one find the damping coefficients?

31 Newton Method with Limiting f) Damped Newton Singular Jacobian Problem 2 D X Damped Newton Methods push iterates to local minimums inds the points where Jacobian is Singular

32 Summary Quick Review of -D Newton Convergence Testing Multidimensonal Newton Method Basic Algorithm Description of the Jacobian. Jacobian Construction. Local Convergence Theorem Damped Newton Method Nested Algorithm with line search Global convergence I Jacobian nonsingular