CS522 - Partial Di erential Equations

Transcription

1 CS5 - Partial Di erential Equations Tibor Jánosi April 5, 5 Numerical Di erentiation In principle, di erentiation is a simple operation. Indeed, given a function speci ed as a closed-form formula, its di erentiation involves te mecanical application of te cain rule, and of te rules tat refer to te di erentiation of elementary functions. Numerical di erentiation is more callenging. We will now understand some of te di culties. Consider an example wen te function to di erentiate is inferred based on a set of sampled values (measurements), and tese measurements are a ected by noise. If one interpolates te function so tat te interpolated function "goes" troug all measured points, te implied function derivative will often ave no relationsip to te underlying "trut." Figure below illustrates tis point..5 Effect of Noise on Function Values and Numerical Differentials Function values.5 Te noise is normal, wit mu= and sigma=.5..5 log(+x) log(+x)+noise x Differential (exact and approximate) - /(+x) x Figure : E ect of noise on te inferred value of numerical derivatives. Functions interpolated from noisy measurements sould not be used directly to generate approximate values for teir derivatives. In our example, a simple, well-beaved function is perturbed by normal noise wit

2 mean and standard deviation.5. On te grap, te values of te noisy function are practically indistinguisable from te unperturbed function. Indeed, te noisy values could be used directly in certain applications; for example, to integrate te function. If we use tese perturbed values to compute numerical derivatives, owever, te results bear no relation to te underlying reality. So wat can one do? One solution is to infer a reasonably smoot function from te noisy measurements, ten use tis function to compute approximate derivatives. If one knows te general form of te underlying function, ten a least-squares metod can be used to infer values for te unknown parameters. Once tese parameters are determined, one can di erentiate te function eiter numerically, or analytically. It is, of course, possible for te underlying "true" function not to be known. In suc cases one can try to coose functions wose general sape is in agreement wit te measured data, or use spline tecniques to approximate te unknown underlying function. Suc metods are often reasonably successful, but tey involve some amount of experimentation before obtaining a satisfactory outcome.. Tecniques for Di erentiation Te simplest - and peraps most obvious - tecnique is tat of functions provided as closed-form formulas. Suc functions can be di erentiated "by and," or by using symbolic tools like Matematica. Te result can ten be implemented directly... Automated Di erentiation A generalization of tis idea resulted in te idea of automated di erentiation. Even if a function is not available as a closed-form formula, te program tat computes it consists of a (peraps very long) sequence of elementary operations, wic can be seen as function applications. Te derivatives of suc elementary functions are known, so all one as to do is to suitably combine tese derivatives (by using te cain rule) in order to obtain te derivative of te initial function. It is tus possible to rewrite te source code of a function in order to generate te derivative of te respective function wit respect to one, or even all of its input arguments. In practice, tere are many problems tat automated di erentiation must overcome. Also, tere are some inerent limitations. Wat sould appen, for example, if certain libraries are only available in compiled form - sould one try to reverse engineer tem and develop automated di erentiation for binary code as well? But wat if legal impediments prevent libraries from being reverse engineered? Despite some of tese problems, automated di erentiation enjoys increasing popularity, due to te ig precision of tese metods, and due to te low computational cost tat tey entail.

3 .. Finite Di erences Consider a di erentiable function f : R! R, and te usual de nition of te derivative f f(x+) f(x) (x) = lim!. Traditionally, di erentiation as been approaced as a purely numerical problem, and te limit above as been approximated by using nite di erences. Consider te Taylor-series expansions of function f around x: f(x + ) = f(x) +! f (x) +! f (x) + 3! f (x) 3 + ::: f(x ) = f(x)! f (x) +! f (x) 3! f (x) 3 + ::: Let us now examine at te following nite-di erence approximations of te derivative: f (x) = f (x) = f (x) = f(x + ) f(x)! f (x) f(x) f(x ) +! f (x) f(x + ) f(x ) 3! f (x) ::: 3! f (x) + ::: 3! f (x) + ::: Assuming tat is small, and ignoring all terms containing we get te following approximations: f (x) f (x) f (x) f(x + ) f(x) f(x) f(x ) f(x + ) f(x ) Te tree metods to compute numerical derivatives are called "forward approximation," "backward approximation," and "symmetric approximation," respectively, and all of tem are exact in te limit. For nite values of, te residual term of te forward and backward approximation is O(), wile te residual term of te symmetric approximation is O( )...3 Ricardson s Extrapolation Te nite precision of numerical computations can severely limit our practical ability to use any of te tree nite-di erences approximation given above. In oter words, we migt not reasonably approac te limit!. Ricardson s metod can elp us overcome tis di culty by extrapolating te value obtained for te derivative for nite s to te point =. In te following we will implicitly assume tat functions ave all te properties tat are needed in te context tat we are using tem. 3

4 Let us denote by f () te approximate value of te derivative f (x), computed for a step size of. Furter, let us assume tat a relationsip of te form given below olds: f () = a + a p + O( r ); r > p > Te value tat we want to compute is f () = a. Let us now compute function f two values of,, and. We get te following: for f ( ) = a + a p + O( r ) f ( ) = a + a p + O( r ) Ignoring te residuals, we can solve for a : f () = a = f ( ) + f ( ) f ( ) p Note tat f () is still an approximation. Tis idea can be extrapolated so tat more terms are considered in te expansion of f () given above; if n terms are considered, ten n approximate values of te derivative must be computed for n di erent values of. Te resulting system of equations can ten be solved to recover te value of a. For su ciently well-beaved functions f, tis metod can produce very accurate results...4 Higer-Order Derivatives Analogously to te matematical de nition of iger-order derivatives, we can de ne nite-di erence approximations to suc derivatives using derivatives of lower order. Consider te matematical de nition of te second derivative: f (x) = lim! f (x + ) f (x) Since we ave several ways to approximate rst-order derivatives, we will also ave several approximations of te second derivative. We will, owever, only consider te symmetric central-di erence approximation given below: f (x) f (x) f (x ) = f(x+) f(x) f(x) f(x ) f(x + ) f(x) + f(x ) As it can be seen, te formula above is te backward di erence of te forward di erence approximations of te rst derivative. Tis approximation is accurate to te order of O( ). 4

5 . Partial Di erential Equations For our immediate purposes, te most important di erential equation will be te (t; x) f (t; Because te igest-order partial derivative tat appears in equation above is, tis is a second-order partial derivative equation. Let us consider te general form of a second-order partial di erential equation of a function f wit arguments t and f + + f c 5f + c 6 = Suc partial di erential equations are classi ed wit respect to te sign of te expression E = c 4c c 3 as yperbolic (if E > ), parabolic (if E = ), and elliptic (if E < ). Informally speaking, yperbolic pde s describe time-dependent processes tat are not evolving toward a steady state, parabolic pde s describe processes tat are evolving toward a steady state, wile elliptic processes ave already reaced a steady state (teir solution is not time-dependent). Wen te coe cients c i are not constant, te equation can cange type, so tis classi cation is of limited value. Te eat equation is a parabolic equation... Initial and Boundary Conditions As in te case of regular di erential equations, solving a pde yields a family of solutions. To select a unique function from suc a family one needs to impose furter conditions; typically tese consist of te initial condition and boundary conditions. For te speci c case of te eat equation as applied to te problem of option values, our set of conditions will consist of te following: f(; x) = f (x) lim x! = f lim x! = f In te formula above f is a function independent of t, wile f and f are constants. As we will see below, practical considerations prevent us from directly using limits at in nity. Rater, we will coose a pair of values x min and x max so tat tey represent suitable approximations of and, respectively. We ten obtain te following set of conditions: f(; x) = f (x) f(t; x max ) = f f(t; x min ) = f 5

6 .. Semidiscrete Solutions Te eat equation can be solved by discretizing te space variable x wile leaving te time variable t continuous. To acieve tis, we rst divide te interval [x min ; x max ] into an integer number of subintervals of lengt x; let N = xmax x min. In te second stage we x associate a function f i (t), wit eac point x i = x min + ix, were < i < N. Using te symmetric central-di erence approximation for te second partial derivative wit respect to x, we get te following: f i (t) = f i+(t) f i (t) + f i (t) (x) ; < i < N Te initial and boundary conditions yield te following relations: f i () = f (x i ); < i < N f N (t) = f f (t) = f In e ect, tis metod corresponds to computing te curves tat result wen we slice te surface f(t; x) wit planes of te form x = x i ; ence, tis approac is called te metod of lines. By introducing te notation F (t) = [f (t) f (t) f 3 (t) ::: f n (t)] T, were T represents te transpose operator, we obtain te following system of ordinary di erential equations: 3 3 : : : f F (t) = : : : (x) 6 : : : F (t) + 4: : : : : : : : : : : : : : : 5 (x) 6 4 : : : 5 {z : : : } f C Te matrix of coe cients C is of size (N ), but only 3(N ) = 3N 5 coe cients are non-zero. Te iger N grows (i.e. te smaller x becomes), te smaller te proportion of non-zero coe cients becomes. In te limit, te proportion of non-zero coe cients tends to, but even for N = we only get a ratio of approximately 3% for suc coe cients. For te bene t of readers wit more background in numerical analysis, we note ere tat te system of ordinary di erential equations above tends to be very sti, tus one must be careful to coose an appropriate metod to solve it. Sparse Matrices Matrices wose proportion of non-zero coe cients is very small wit respect to teir total number of coe cients are called sparse matrices. Sparse matrices are commonly encountered in several important areas of numerical computations. Special tecniques ave been developed to represent and manipulate suc matrices so as to 6

7 minimize te amount of memory tat would be wasted if te full matrix were explicitly represented. In tis area, a lot of e ort is expended to de ne matrix algoritms tat minimize te number of new non-zero elements (te " ll") tat are produced as a result of various matrix operations. Te speci c sparse matrix representation and algoritms tat are cosen depend in general on te degree of sparsity, as well as on te sparsity pattern (i.e. te distribution of non-zero coe cients) exibited by te matrices at and. Matrices encoded using sparse matrix representations incur an overead wit respect to regular matrices wen various operations are performed. Depending on tis overead, any advantage related to memory savings can be swamped by te increased processing time tat te sparse matrix representation entails. As a very approximate rule of tumb, one sould probably not contemplate using sparse matrix representations unless te total number of coe cients coe cients is at least of a few tousand (or even iger). For te example at and, we ave a particularly simple sparsity pattern wic we could address by linearizing te matrix. To acieve tis, we can transfer all coe cients into an array C linear of size 3N 5, going line by line in te original matrix C. We obtain te following representation of C linear : C linear = : : : Given te pair of indices (i; j) in matrix C, te corresponding index k in array C linear will be given by [3(i ) min(i ; )] + [j max(i ; )]. 3 Of course, te previous formula assumes tat (i; j) denotes a non-zero coe cient. Can you establis wat relation must old between i and j so tat tis assumption is true? Matlab provides support for sparse-matrix computations; type lookfor sparse at your Matlab prompt to learn more about tis topic...3 Discrete Solutions In te preceding section we ave discretized te space dimension x wile leaving te time dimension t continuous. We can, of course, also discretize te time dimension. As before, we rst divide te interval [x min ; x max ] into an integer number of subintervals of lengt x; let N x = xmax x min. Next, we coose a su ciently large maximum value of t, x t max, and we divide te interval [; t max ] into an integer number of subintervals of lengt t; let N t = tmax. Tese two steps fully discretize te domain of te pde at and. Te t corresponding situation is illustrated in gure. We introduce te notation fn m = f(mt; x min + nx), m N t, n N x, to denote te values of te function at te points of te resulting mes. Using te various nite-di erence approximations for te rst and second-order derivative, we can now write down te equations needed for solving te eat equation. Here, as elsewere in te course, we stick wit Matlab s convention and use -based indices. 3 Try to understand ow tis formula was derived. Our use of parenteses sould elp you in tis process.

8 t max mdt "upper boundary" time "lower boundary" passing of time "initial condition" x min x min +ndx x max x Figure : Illustration of te eat equation s discretized domain. Te Explicit Finite-Di erence Metod We coose te forward di erence equation for te time derivative, and te symmetric central di erence for te space derivative. Ignoring residual terms we get te approximate equality below: f m+ n t f m n = f n+ m fn m + fn m ; < m < N (x) t ; < n < N x Introducing te notation = t, we get: (x) f m+ n = f m n+ + ( )f m n + f m n ; < m < N t ; < n < N x Te relationsip between tese quantities is illustrated in gure 3. Since te values f n = f(x min + nx; ) = f (x min + nx), n N x, and f m = f and f m N x = f, m N t are known (given), we can compute te values of te function at all points in te domain of interest. In principle, we know ow to compute an approximate solution for te eat equation. But ow good will tis solution be? Can we understand ow te quality of te result depends on te coice of x and t? Te explicit nite di erence metod as an error term of te form O(t) + O((x) ), i.e. it is rst-order accurate in time and second-order accurate in space. 8

9 f(n,m+) f(n-,m) f(n,m) f(n+,m) Figure 3: Relationsip between known values (in black), and te newly computed value (in red) in te explicit nite-di erence discretization. Ideally, te solution to a pde problem sould be bot stable and consistent. Loosely speaking, stability means tat small perturbations do not cause te corresponding approximate solutions to diverge witout bound. Consistency means tat as te step sizes x and t decrease toward te truncation error (te error due to te discrete approximation of te derivative) decreases toward. For te convergence of an approximate solution to te true solution of te underlying problem it is necessary for te respective solution to be bot consistent and stable. As we can see from te discussion of te error terms, te explicit nite-di erence metod is consistent. Witout providing more details, we will state tat te nitedi erence metod is stable if, and not stable if >. Te Fully-Implicit Metod We again use te forward di erence for te time derivative and te symmetric central-di erence approximation for te second-order space derivative. For te latter second derivative, owever, we will not coose te time coordinate to be mt, as we did before; we will x it at (m + )t. We ten get: f m+ n t f m n = f n+ m+ fn m+ + fn m+ ; < m < N (x) t ; < n < N x Keeping te same interpretation for = t, we get: (x) f m+ n + ( + )f m+ n f m+ n+ = f m n ; < m < N t ; < n < N x Te relationsip between tese quantities is illustrated in gure 4. As te reader can immediately see, te fully implicit metod does not allow for te immediate computation of te new function values. Rater, a system of equations must 9

10 f(n-,m+) f(n,m+) f(n+,m+) f(n,m) Figure 4: Relationsip between known value (in black), and te newly computed values (in red) in te fully implicit nite-di erence discretization. be set up and solved. Let F m = f m f m f3 m fn m T x, be column of unknown values at time mt (and note tat f m and fn m x are known). We obtain te following system of equations: : : : + : : : + : : : 6 : : : : : : : : : : : : : : : : : : F m+ = F m : : : {z : : : + } M We can write te relation above in te muc more compact form: MF m+ = b m f m+ : : : f m+ N x 5 {z } b m If M were invertible, we could immediate compute F m+ = M b m. But is M invertible? Not necessarily, if is arbitrary. However, if >, as it must be in tis case, M is invertible. We prove tis statement below. Teorem (Gersgorin s Circle Teorem) Let A be a complex square matrix A = (a ij ) i;j=;n and let be equal to one of its eigenvalues. De ne R i = P n j= ja ij j, for eac i, i n. i6=j Ten will be in at least one of te disks D i : fz j jz a ii j R i g. Proof. Let be an eigenvalue of A, and v = v v : : : v n its corresponding eigenvector. We immediately get tat Av = v. Furter, coose i so tat jv i j = maxfjv j j j j ng. We immediately get tat jv i j > (wy?).

11 Now, consider te i -t component of te equality Av = v; we get: nx a i jv j = v i j= We can rewrite tese equations as follows: nx a ijv j = ( j= j6=i Finally, we ave: j a i i j = nx j= v j a ij v i j6=i a ii)v i nx = ja ijj j= j6=i v j v i {z} nx ja ijj j= j6=i Lemma Matrix M de ned below as no negative real eigenvalues. 3 : : : : : : M = : : : 6: : : : : : : : : : : : : : : : : : 4 : : : 5 : : : Proof. By applying Gersgorin s teorem we immediately get tat all eigenvalues of matrix M must be in at least one of te disks given below: D : fz j jz j g D : fz j jz j g But D is included in D, so we conclude tat all eigenvalues of M must be in disk D. Hence M as no negative eigenvalues. Teorem 3 For any real, positive, matrix M (as de ned above) is invertible. Proof. Let us assume tat M is not invertible. Ten det(m) =. We can immediately establis te obvious equality M = I + M, were I is te identity matrix of size n, and M as been de ned in te previous lemma. We now ave: det(m) = det(i + M) = det(m + I) =

12 Te last relation implies tat equation My = y admits a non-trivial solution (i.e. y 6= ), wic is te same ting as saying tat one of M s eigenvalues must be. Since we assumed tat >, tis means tat M admits a negative eigenvalue. We ave reaced a contradiction, ence M is not invertible. Returning now to te problem of solving te system of equations MF m+ = F m, we know tat matrix M is in fact non-singular for any coice of x > and t >. Hence F m+ = M F m. However, tis approac is not usually pursued. Witout providing more details, we state tat in general matrix inversions are avoided in numerical computations; tey are expensive and can be sensitive to small errors in te initial matrix (if te matrix is ill-conditioned). In tis case tere is a furter disadvantage: wile te original matrix M is tridiagonal, ence sparse, its inverse M would be dense. If we could avoid computing M we could peraps avoid te need to store (N x ) coe cients, tus greatly reducing te memory consumption of our algoritm. One approac to solving te system of equations is to use te so-called LU ("lowerupper") decomposition of matrix M. In a general LU decomposition of a matrix A, te respective matrix will be written as te product L A U A, were A is a lower-triangular matrix wit unitary main diagonal (i.e. te main diagonal contains only s), and U A is an upper-triangular matrix. Given te special form of our matrix M, its LU decomposition is especially simple to obtain by solving te following matrix equation: 3 + : : : + : : : + : : : 6 : : : : : : : : : : : : : : : : : : = 4 : : : + 5 : : : : : : d u : : : l : : : d u : : : = l : : : d 3 : : : 6: : : : : : : : : : : : : : : : : : 6: : : : : : : : : : : : : : : : : : 4 : : : 54 : : : d Nx u Nx 5 : : : l Nx : : : d Nx {z } {z } L We can easily solve tis system of equations "by and." First, by equating te terms tat give te coe cient wit indices (; ) of te result, we immediately obtain d = +. Te equations for te coe cients wit indices (i; i + ) immediately yield te values u i =, < i < N x. Similarly, from te coe cients wit indices (i; i ) we get tat l i d i =, < i < N x. Tis implies tat l i = d i, < i < N x. Finally, we determine te values d i by examining te equations tat correspond to te coe cients (i; i) in te resulting matrix. We get tat l i u i + d i = +, < i < N x. We immediately get tat d i = + d i, < i < N x. Tese relations allow us to determine matrices L and U in a simple loop. U

13 Note tat we if we do not explicitly represent te s on te main diagonal of L M, ten we can represent bot L M and U M in te same space tat we used originally to store M. Tis is important if we used a linearization metod like te one described above. We can now solve te system of equations in two steps: L M v m = b m U M F m+ = v m We can solve te rst system of equations by "forward substitution" in one pass; te second system can ten be solved by "backward substitution." Can you estimate te total number of operations (i.e. te number of additions, subtractions, multiplications and divisions) needed to solve tese systems? At eac step m we compute all te values fn m by solving te system of equations MF m+ = b m. Te advantage of tis metod, owever, is tat its stability is muc better tan tat of te explicit nite-di erence metod; i.e. te time step t can be signi cantly bigger relative to x in te case of te fully implicit metod. Te Crank-Nicolson Metod Recall tat te error term for bot te explicit nitedi erence and fully implicit metod was of te form O(t) + O((x) ). Te Crank- Nicolson metod improves tis error term to O((t) ) + O((x) ). Let us recall te formulas we wrote for te explicit nite-di erence metod and te fully implicit metod: f m+ n f m+ n t t f m n f m n = f n+ m fn m + fn m ; < m < N (x) t ; < n < N x = f n+ m+ fn m+ + fn m+ ; < m < N (x) t ; < n < N x We immediately obtain te following relation: fn m+ fn m = f m n+ fn m + fn m + f m+ t (x) Again, using te notation = t, we get: (x) n+ f m+ n + fn m+ (x) f m+ n f m n = (f m n+ f m n + f m n + f m+ n+ f m+ n + f m+ n ) Separating te terms tat refer to time mt and time (m + )t we get: m+ fn + ( + )fn m+ m+ fn+ = f n m + ( )fn m + f n+ m {z } Zn m Te relationsip between tese quantities is illustrated in gure 5. 3

14 f(n-,m+) f(n,m+) f(n+,m+) f(n-,m) f(n,m) f(n+,m) Figure 5: Relationsip between known values (in black), and te newly computed values (in red) in te Crank-Nicolson metod. By analogy to te fully implicit metod above we can write te following system of equations: + : : : Z + : : : m f + : : : Z m m+ 6 : : : : : : : : : : : : : : : : : : F m+ = Z m 3 4 : : : + 6 : : : + 6 : : : 5 4Z m 5 4 : : : + N x 5 Z m N {z } x f m+ N {z x } M CN b m CN We use te subscript CN to denote matrices related to te Crank-Nicolson metod, so tat tey are distinguisable from te analogous matrices we de ned for te fully implicit metod. Also, we reused te de nition F m = f m f m f3 m fn m T x. Te system of equations M CN F m+ = b m CN can be solved by te metod of LU decomposition as described above. 4