Pysically Based Modeling: Principles and Practice Implicit Metods for Differential Equations David Baraff Robotics Institute Carnegie Mellon University Please note: Tis document is 997 by David Baraff Tis capter may be freely duplicated and distributed so long as no consideration is received in return, and tis copyrigt notice remains intact
Implicit Metods for Differential Equations David Baraff Robotics Institute Carnegie Mellon University Implicit Metods Te metods we ave looked at for solving differential equations in te first section of tese notes (eg Euler s metod, te midpoint metod are all called explicit metods for solving ODE s However, sometimes an ODE can become stiff, in wic case explicit metods don t do a very good job of solving tem Wenever possible, it is desirable to cange your problem formulation so tat you don t ave to solve a stiff ODE Sometimes owever tat s not possible, and you ave just ave to be able to solve stiff ODE s If tat s te case, you ll usually ave to use an ODE solution metod wic is implicit Example Stiff ODE First, wat is te meaning and cause of stiff equations? Lets consider an example tat arises frequently in dynamics Suppose tat we ave a particle, wit position (x(t, y(t, and suppose tat we want te y-coordinate to always be zero One way of doing tis is to add a component ky(t to ẏ(t were k is a large positive constant If k is large enoug, ten te particle will never move too far away from y(t, since te ky(t term always brings y(t back towards zero However, lets assume tat tere is no restriction on te x-coordinate, and tat we want a user to be able to pull te particle arbitrarily along te x-axis So lets assume tat over some time interval our differential equation is simply Ẏ(t d dt ( x(t y(t ( x(t ky(t ( (We ll also assume tat te particle doesn t start exactly wit y Wat s appening ere is tat te particle is strongly attracted to te line y, and less strongly towards x If we solve te ODE far enoug forward in time, we expect te particle s location to converge towards (, and ten stay tere once it arrives Now suppose we use Euler s metod to solve te equation If we take a step of size,weget ( ( x x Y new Y + Ẏ(t + y ky Tis yields ( x x Y new y ky ( ( x ( ky E
If we look at te y component of tis equation, we see tat if k > ten te y new we compute will ave an absolute value wic is larger tan y In oter words, if k >, Euler s metod will not converge to an answer: eac step will result in a value of y new wic is larger tan te last Tecnically, Euler s metod is unstable for k > Tus, we better ave k > ork < 2 if we ope to converge Te largest step we can ope to take is less tan 2/k Now, if k is a large number, we ll ave to take very small steps Tis means tat te particle slides towards (, excruciatingly slowly Even toug te particle may nearly satisfy y, we ave to take suc small steps tat te particles progress along te x-axis is pretty muc nonexistent Tat s te embodiment of a stiff ODE In tis case, te stiffness arises from making k very large in order to keep te particle close to te line y Later on, wen we connect particles wit secondorder dynamics togeter wit springs, we ll experience exactly te same effect: stiff ODE s Even if we use a more sopisticated explicit metod suc as fourt-order Runge-Kutta, we may do a little better in te size of our steps, but we ll still ave major problems Now as we said above, te name of te game is to pose your dynamics problems so tat you don t experience stiff ODE s However, wen you can t, you ll ave to turn towards an implicit solution metod Te metod we ll sow below is te simplest of te implicit metods, and its based on taking an Euler step backwards 2 Solving Stiff ODE s Instead of writing Y new Y + f(y(t,weuseteformula Y new Y + f(y new ( 2 Tat is, we re going to evaluate f at te point we re aiming at, rater tan were we came from (If you tink about reversing te world and running everyting backwards, te above equation makes perfect sense Ten te equation says if you were at Y new, and took a step f(y new, you d end up at Y So if your differential equation represents a system tat is reversible in time, tis step makes sense It s just finding a point Y new suc tat if you ran time backwards, you d end up at Y So, we re looking for a Y new suc tat f, evaluated tere, times, points directly back at were we came from Unfortunately, we can t in general solve for Y new, unless f appens to be a linear function To cope wit tis, we ll replace f (Y new wit a linear approximation, again based on f s Taylor series Lets define Y by Y Y new Y Using tis, we rewrite equation ( 2 as or just Next, lets approximate f (Y + Y by Y + Y Y + f(y + Y Y f(y + Y f (Y + f (Y Y (Note tat since f (Y is a vector, te derivative f (Y is a matrix Using tis approximation, we can approximate Y wit Y ( f (Y + f (Y Y SIGGRAPH 97 COURSE NOTES E2 PHYSICALLY BASED MODELING
or Y f (Y Y f(y Rewriting tis as ( I f (Y Y f (Y, were I is te identity matrix, we can solve for Y as ( Y I f (Y f (Y ( 3 Computing Y new Y + Y is clearly more work tan using an explicit metod, since we ave to solve a linear system at eac step Wile tis would seem a serious weakness, computationally speaking, don t despair (yet For many types of problems, te matrix f will be sparse for example, if we are simulating a spring-lattice, f will ave a structure wic matces te connectivity of te particles As a result, it is usually possible to solve equation ( 3 in linear time (ie time proportional to te dimension of Y In suc cases, te payoff is dramatic: we can usually take considerably large timesteps witout losing stability (ie witout divergence, as appens wit te explicit case if te stepsize is too large Te time taken in solving eac linear system is tus more tan offset by te fact tat our timesteps are often orders of magnitude bigger tan we could manage using an explicit metod (Of course, te code needed to do all tis is muc more complicated tan in te explicit case; like we said, make your problems un-stiff if you can, and if not, pay te price Lets apply te implicit metod to equation ( We ave tat f (Y(t is ( x(t f (Y(t ky(t Differentiating wit respect to Y yields f (Y(t Y f (Y(t Ten te matrix I f (Y is ( + +k ( k ( + +k Inverting tis matrix, and multiplying by f (Y yields ( + ( Y x +k ky ( ( + x +k ky + x +k ky SIGGRAPH 97 COURSE NOTES E3 PHYSICALLY BASED MODELING
Wat is te limit on te stepsize in tis case? Te answer is: tere is no limit! In tis case, if we let grow to infinity, we get lim Y lim + x + k ky x k ky x ( y Tis means tat we acieve Y new Y + ( Y in a single step! For a general stiff ODE, we won t be able to take steps of arbitrary size, but we will be able to take muc larger steps using an implicit metod tan using an explicit metod Te extra cost of solving a linear equation is more tan made up by te time saved by taking large timesteps SIGGRAPH 97 COURSE NOTES E4 PHYSICALLY BASED MODELING