Krylov Integration Factor Method on Sparse Grids for High Spatial Dimension Convection Diffusion Equations

Transcription

1 DOI.7/s Krlov Integration Factor Method on Sparse Grids for High Spatial Dimension Convection Diffusion Equations Dong Lu Yong-Tao Zhang Received: September 5 / Revised: 9 March 6 / Accepted: 8 April 6 Springer Science+Business Media New York 6 Abstract Krlov implicit integration factor (IIF) methods were developed in Chen and Zhang (J Comput Phs 3: , ) for solving stiff reaction diffusion equations on high dimensional unstructured meshes. The methods were further etended to solve stiff advection diffusion reaction equations in Jiang and Zhang (J Comput Phs 53: , 3). Recentl we studied the computational power of Krlov subspace approimations on dealing with high dimensional problems. It was shown that the Krlov integration factor methods have linear computational compleit and are especiall efficient for high dimensional convection diffusion problems with anisotropic diffusions. In this paper, we combine the Krlov integration factor methods with sparse grid combination techniques and solve high spatial dimension convection diffusion equations such as Fokker Planck equations on sparse grids. Numerical eamples are presented to show that significant computational times are saved b appling the Krlov integration factor methods on sparse grids. Kewords Implicit integration factor methods Sparse grids Krlov subspace approimation High spatial dimensions Convection diffusion equations Introduction Integration factor (IF) methods are a class of eactl linear part time discretization methods for the solution of nonlinear partial differential equations (PDEs) with the linear highest spatial derivatives. This class of methods performs the time evolution of the stiff linear operator via evaluation of an eponential function of the corresponding matri. Hence the integration factor tpe time discretization can remove both the stabilit constrain and time direction numerical errors from the high order derivatives [,5,4 6,]. In [], a class of efficient implicit integration factor (IIF) methods were developed for solving sstems with B Yong-Tao Zhang zhang@nd.edu Department of Applied and Computational Mathematics and Statistics, Universit of Notre Dame, Notre Dame, IN 46556, USA 3

2 both stiff linear and nonlinear terms. A novel propert of the methods is that the implicit terms are free of the eponential operation of the linear terms. Hence the eact evaluation of the linear part is decoupled from the implicit treatment of the nonlinear terms. As a result, if the nonlinear terms do not involve spatial derivatives, the size of the nonlinear sstem arising from the implicit treatment is independent of the number of spatial grid points; it onl depends on the number of the original PDEs. This distinguishes IIF methods [] from implicit eponential time differencing (ETD) methods in []. For problems with high spatial dimensions, the major computational challenge in appling the IIF methods is how to deal with matri eponentials for ver large matrices. Currentl there are two approaches to deal with the large matri eponential problem in IIF methods. One is the class of compact implicit integration factor (ciif) methods in [3]. ciif methods reduce the cost prohibitive large matri eponentials for linear diffusion operators with constant diffusion coefficients in high spatial dimensions to a series of much smaller one dimensional computations. This approach is further etended in [8] as an arra-representation technique to deal with more complicated high dimensional reaction diffusion equations with cross-derivatives in diffusion operators. The method is termed as arra-representation compact implicit integration factor (AcIIF) method. Another approach is to use Krlov subspace approimations to efficientl calculate large matri eponentials. In [4], Krlov subspace approimation is directl applied to the IIF methods for solving high dimensional reaction diffusion problems. The Krlov IIF methods were further etended to solve high dimensional stiff convection diffusion reaction (CDR) equations in [,3]. Recentl in [9] we studied the computational power of Krlov subspace approimations on dealing with high spatial dimension convection diffusion problems. Sstematical numerical comparison and compleit analsis are carried out for the computational efficienc of the two different approaches. It was shown that the Krlov integration factor methods have linear computational compleit and are especiall efficient for solving high dimensional convection diffusion problems with anisotropic diffusions, such as high dimensional Fokker Planck equations [6,4]. In this paper, we aim at achieving more efficient computations of Krlov IIF schemes than the eisting work in the literature b developing the Krlov IIF schemes on sparse grids for high spatial dimension problems. In recent ears, sparse-grid has become a major approimation tool for high-dimensional problems. It has been successfull used in man scientific and engineering applications. Discretizations on sparse grids involve O(N (log N) d ) degrees of freedom onl, where d denotes the underling problem s dimensionalit and N is the number of grid points in one coordinate direction. A detailed review on sparse-grid technique can be found in [3]. Sparse-grid techniques were introduced b Zenger [9] in 99 to reduce the number of degrees of freedom in finite-element calculations. The sparse-grid combination technique, which was introduced in 99 b Griebel et al. [8], can be seen as a practical implementation of the sparse-grid technique. In the sparse-grid combination technique, the final solution is a linear combination of solutions on semi-coarsened grids, where the coefficients of the combination are chosen such that there is a canceling in leading-order error terms and the accurac order can be kept to be the same as that on single full grids [8,7,8]. The rest of the paper is organized as following. In Sect., we review the Krlov IIF schemes. Krlov IIF methods on sparse grids are developed in Sect. 3. In Sect. 4, we perform etensive numerical eperiments to test the sparse-grid Krlov IIF methods and show significant savings in computational costs b comparisons with single-grid computations. Conclusions are given in Sect. 5. 3

3 Krlov IIF Methods for CDR Equations In this section, we briefl review the Krlov IIF methods for high spatial dimension convection diffusion-reaction equations which were developed in []. We consider convection diffusion-reaction equations d u t + f i (u) i = (D u) + r(u), () i= where u is the unknown, f i, i =,...,d are flu functions in d spatial dimensions respectivel, D is the diffusion matri and it could be non-constant and have variable entries, and r is the reaction term. Often the CDR models in applications have nonlinear convection and reaction terms, but a linear diffusion term (D u),whered is independent of u but could be functions of spatial variables. We use central differences schemes to discretize the diffusion terms, and nonlinear stable weighted essentiall non-oscillator (WENO) [] or upwind schemes for convection terms. After spatial discretizations, a semi-discretized ODE sstem d U = F d ( U) + F a ( U) + R( U) () dt is obtained. Here U = (u i ) i N, F d ( U) = ( Fˆ d i ( U)) i N, F a ( U) = ( Fˆ ai ( U)) i N, R = (r(u i )) i N. N is the total number of grid points, F d ( U) is the approimation for the diffusion terms b the second or fourth order finite difference schemes, and its each component Fˆ d i is a linear or nonlinear function of numerical values on the approimation stencil. If the diffusion term is linear, F d ( U) = C U where C is the approimation matri for the linear diffusion operator b the central finite difference schemes. F a ( U) is the approimation for the nonlinear advection terms b WENO scheme or upwind scheme, and its each component Fˆ ai is a nonlinear function of several numerical values on the WENO or upwind approimation stencil. R( U) is the nonlinear reaction term, and its each component r(u i ) is a nonlinear function which onl depends on numerical value at one grid point. In [], we developed a method to deal with the nonlinear diffusion terms b factoring out the linear part which mainl contributes to the stiffness of the nonlinear diffusion terms, then appling the integration factor approach to remove this stiffness. In this paper, we focus on the problems with linear diffusions of either constant or variable diffusion coefficients, i.e., F d ( U) = C U. IIF methods for () are constructed b eactl integrating the linear part of the sstem. Directl multipl () b the integration factor e Ct and integrate over one time step from t n to t n+ t n + t n to obtain U(t n+ ) = e C t n U(t n ) + e C t n + e C t n tn tn e Cτ F a ( U(t n + τ))dτ e Cτ R( U(t n + τ))dτ. (3) Two of the nonlinear terms in (3) have different properties. The nonlinear reaction term R( U) is usuall stiff but local, while the nonlinear term F a ( U) derived from approimations to the convection term is nonstiff but couples numerical values at grid points of the stencil. Hence we use different methods to treat them and avoid solving a large coupled nonlinear sstem. For the stiff reaction term e Cτ R( U(t n + τ)), we approimate it implicitl b an (r )-th order Lagrange polnomial with interpolation points at t n+, t n,...,t n+ r.the nonlinear convection term e Cτ F a ( U(t n + τ)) is approimated eplicitl b an (r )-th 3

4 order Lagrange polnomial with interpolation points at t n, t n,...,t n+ r.ther-th order IIF scheme for CDR equations is obtained as U n+ = e C t n U n + t n {α n+ R( U n+ ) + α n+i e C( t n τ i ) R( U n+i ) + i= r where the coefficients α n+i = tn t n β n+i = tn t n i= r } β n+i e C( t n τ i ) F a ( U n+i ), (4) j= r, j =i j= r, j =i τ τ j τ i τ j dτ, i =,,,..., r; (5) τ τ j τ i τ j dτ, i =,,,..., r. (6) τ = t n, τ =, τ i = k=i t n+k for i =,, 3,..., r. U n+i is the numerical solution for U(t n+i ). Specificall, the second order scheme (IIF) is of the following form U n+ = e C t n U n + t n {α n+ R( U n+ ) + α n e C t n R( U n ) } + β n e C( t n+ t n ) F a ( U n ) + β n e C t n F a ( U n ), (7) where α n =,α n+ =,β n = t n,β n = t n t n ( ) tn + t n. And the third order scheme (IIF3) is U n+ = e C t n U n + t n {α n+ R( U n+ ) + α n e C t n R( U n ) + α n e C( t n+ t n ) R( U n ) } + β n e C( t n+ t n + t n ) F a ( U n )+β n e C( t n+ t n ) F a ( U n )+β n e C t n F a ( U n ), where ( tn α n+ = ( t n + t n ) 3 + t ) n, α n = ( tn t n 6 + t ) n, t n α n = 6 t n ( t n + t n ), β n = + t n ( t n + t n ) [ t β n = n t n t n 3 + t n β n = 3 t n ( t n + t n ) [ t n 3 + t ] n ( t n + t n ), ] ( t n + t n ), ( t n 3 + t ) n t n. (8)

5 The efficienc of IIF schemes for high dimensional problems largel depends on the methods to evaluate the product of the matri eponential and a vector, for eample e C t v. For PDEs defined on high spatial dimensions (D and above), a large and sparse matri C is generated in the schemes (4). But the eponential matri e C t is dense. For high dimensional problems, direct computation and storage of such eponential matri are prohibitive in terms of both CPU cost and computer memor. Here we review Krlov subspace approimation method which was applied to IIF schemes in [4] and for solving CDR equations in []. The computational power of Krlov subspace approimation method for IIF schemes on high spatial dimension problems has been studied in [9] which shows its high efficienc. Notice that we do not need the full eponential matrices such as e C t itself, but onl the products of the eponential matrices and some vectors in the schemes (4). Follow the literature (e.g. [7,]), we describe the Krlov subspace methods to approimate e C t v as following. The large sparse matri C is projected to the Krlov subspace K M = span{v,cv,c v,...,c M v}. (9) The dimension M of the Krlov subspace is much smaller than the dimension N of the largesparsematric. In all numerical computations of this paper, we take M = 5 for different N, and accurate results are obtained in the numerical eperiments. An orthonormal basis V M =[v,v,v 3,...,v M ] of the Krlov subspace K M is generated b the well-known Arnoldi algorithm [7] as the following.. Compute the initial vector: v = v/ v.. Perform iterations: Do j =,,..., M: () Compute the vector w = Cv j. () Do i =,,..., j: (a) Compute the inner product h i, j = (w, v i ). (b) Compute the vector w = w h i, j v i. (3) Compute h j+, j = w. (4) If h j+, j, then stop the iteration; else compute the net basis vector v j+ = w/h j+, j. In the Arnoldi algorithm, if h j+, j forsome j < M, it means that the convergence has occurred and the Krlov subspace K M = span{v,v,...,v j }, so the iteration can be stopped at this step j, and we assign the value of this j to M. This algorithm will produce an orthonormal basis V M of the Krlov subspace K M. Denote the M M upper Hessenberg matri consisting of the coefficients h i, j b H M. Since the columns of V M are orthogonal, we have H M = V T M CV M. () This means that the ver small Hessenberg matri H M represents the projection of the large sparse matri C to the Krlov subspace K M, with respect to the basis V M. Also since V M is orthonormal, the vector V M V T M ec t v is the orthogonal projection of e C t v on the Krlov subspace K M, namel, it is the closest approimation to e C t v from K M. Therefore e C t v V M V T M ec t v = βv M V T M ec t v = βv M V T M ec t V M e, 3

6 where β = v, and e denotes the first column of the M M identit matri I M.Usethe fact of (), we have the approimation e C t v βv M e H M t e. () Thus the large e C t matri eponential problem is replaced with a much smaller e H M t problem. The small matri eponential e H M t will be computed using a scaling and squaring algorithm with a Padé approimation with onl a small computational cost, see [7,,]. Then the Krlov approimations are directl applied in schemes (4), (7) or(8) to obtain Krlov IIF schemes for CDR equations []. The r-th order Krlov IIF scheme for CDR equations has the following form U n+ = t n α n+ R( U n+ ) + γ,n V M,,n e H M,,n t n e + t n (β n+ r γ r,n V M, r,n e H M, r,n( t n τ r ) e + i= r ) γ i,n V M,i,n e H M,i,n( t n τ i ) e, () where γ,n = U n + t n (α n R( U n ) + β n F a ( U n )), V M,,n and H M,,n are orthonormal basis and upper Hessenberg matri generated b the Arnoldi algorithm with the initial vector U n + t n (α n R( U n ) + β n F a ( U n )). γ r,n = F a ( U n+ r ), V M, r,n and H M, r,n are orthonormal basis and upper Hessenberg matri generated b the Arnoldi algorithm with the initial vector F a ( U n+ r ). γ i,n = α n+i R( U n+i ) + β n+i F a ( U n+i ), V M,i,n and H M,i,n are orthonormal basis and upper Hessenberg matri generated b the Arnoldi algorithm with the initial vectors α n+i R( U n+i ) + β n+i F a ( U n+i ),fori = r, 3 r,...,. Notice that V M,,n, V M, r,n and V M,i,n, i = r, 3 r,..., are orthonormal bases of different Krlov subspaces for the same matri C, which are generated with different initial vectors in the Arnoldi algorithm. Specificall, the second order Krlov IIF (KrlovIIF) scheme has the following form U n+ = t n R( U n+ ) + γ,n V M,,n e H M,,n t n e ( t n) ( ) γ,n V M,,n e H M,,n( t n + t n ) e, (3) t n ( where γ,n = U n + t n R( U n ) + t n ( t n + t n ) F a ( U n )), V M,,n and H M,,n are orthonormal basis and( upper Hessenberg matri generated b the ) Arnoldi algorithm with the initial vectoru n + t n R( U n ) + t n ( t n + t n ) F a ( U n ). γ,n = F a ( U n ), V M,,n and H M,,n are orthonormal basis and upper Hessenberg matri generated b the Arnoldi algorithm with the initial vector F a ( U n ). And the third order Krlov IIF (Krlov IIF3) scheme has the form U n+ = t n + 3 t n 6( t n + t n ) t n R( U n+ ) + γ,n V M,,n e H M,,n t n e 3 + t n ( ( tn ) + 3 t n t n 6 t n ( t n + t n ) γ,nv M,,n e H M,,n( t n + t n + t n ) e + γ,n V M,,n e H M,,n( t n + t n ) e ), (4)

7 where γ,n = U n + t n (α n R( U n ) + β n F a ( U n )), V M,,n and H M,,n are orthonormal basis and upper Hessenberg matri generated b the Arnoldi algorithm with the initial vector U n + t n (α n R( U n ) + β n F a ( U n )). γ,n = F a ( U n ), V M,,n and H M,,n are orthonormal basis and upper Hessenberg matri generated b the Arnoldi algorithm with the initial vector F a ( U n ). γ,n = α n R( U n ) + β n F a ( U n ), V M,,n and H M,,n are orthonormal basis and upper Hessenberg matri generated b the Arnoldi algorithm with the initial vectors α n R( U n ) + β n F a ( U n ).SeetheEq.(8) for values of α n,β n,α n,β n. As that pointed out in [], in the implementation of the Krlov approimation methods we do not store matrices C, because onl multiplications of matrices C with a vector are needed in the methods, and the correspond to certain finite difference operations. Remark Theoretical analsis including linear stabilit and error analsis of the IIF schemes for convection diffusion-reaction equations is given in []. As an eample, we present the linear stabilit analsis of the second order IIF scheme for convection diffusion-reaction equations in the Appendi section of this paper. 3 Krlov IIF Schemes on Sparse Grids To achieve further efficienc in solving the CDR Eq. () on high spatial dimensions b Krlov IIF schemes, we present the Krlov IIF schemes on sparse grids b sparse-grid combination technique. The basic idea of sparse-grid combination technique is that b combining several solutions on different semi-coarsened grids (sparse grids), a final solution on the most refined mesh is obtained. The most refined mesh is corresponding to the usual single full grid. Since the PDEs are solved on semi-coarsened grids which have much fewer grid points than the single full grid, computation costs are saved a lot. The final solution obtained b sparse-grid combination technique is required to have the similar accurac as that b solving the PDEs directl on a single full grid. For eample see [8,7,8]. Here we use two dimensional (D) case as the eample to illustrate the idea. Higher dimensional cases are similar. Consider a D domain [a, b]. The construction of semicoarsened grids is as follows. We first partition the domain into the coarsest mesh, which is called a root grid, with N r cells in each direction. The root grid mesh size is H = b a N r. The multi-level refinement on the root grid is performed to obtain a famil of semi-coarsened grids { l,l }. The semi-coarsened grid { l,l } has mesh sizes h l = l H in the direction and h l = l H in the direction, where l =,,...,N L, l =,,...,N L,seeFig.. Superscripts l, l indicate the level of refinement relative to the root grid,,andn L indicates the finest level. Therefore, our finest grid is N L,N L with mesh size h = N L H for both and directions. To solve Eq. (), we will use the second order Krlov IIF (Krlov IIF) method (3) or the third order Krlov IIF (Krlov IIF3) scheme (4) for time discretization. Spatial discretizations are the classical second or fourth order central schemes for diffusion terms, and the third order WENO scheme or the upwind scheme for convection terms. Following the spare-grid combination techniques, rather than on a single full grid, the PDE () issolved on the following (N L + ) sparse grids { l,l } I : {,N L,,N L,..., N L,, N L, } and {,N L,,N L,..., N L,, N L, }. 3

8 Fig. Semi-coarsened sparse grids { l,l } with the finest level NL = 3 And I denotes the inde set I = { (l, l ) l + l = N L or l + l = N L }. B carring out time marching of the PDE using Krlov IIF schemes on these (N L + ) sparse grids, we obtain (N L + ) sets of numerical solutions {U l,l } I (one set of numerical solution is obtained on each sparse grid). The net step is to combine solutions on sparse grids to obtain the final solution on the finest grid N L,N L. The ke point here is that the PDE is never solved directl on N L,N L in order to save computational costs. To etend numerical solutions on sparse grids to that on the finest grid, we appl a prolongation operator P N L,N L (defined in the spare-grid combination techniques [8,7,8]) on each sparse grid solution U l,l to obtain (N L + ) solutions on the finest grid N L,N L. And finall, these solutions are combined to form the final solution Û N L,N L on N L,N L. Net we provide details on the prolongation operator P N L,N L. Prolongation operator P N L,N L maps numerical solutions {U l,l } I on sparse grids onto the finest grid N L,N L.And a prolongation operator is basicall an interpolation operator. For eample, U l,l is numerical solution on l,l,thenp N L,N L U l,l gives numerical values on the most refined mesh N L,N L. For the D case, first in one direction(e.g. the direction), we construct (N r l ) quadratic interpolation polnomials Pi (), i =,...,N r l, b the third order Lagrange interpolation. Each interpolation uses three adjacent grid points to construct a quadratic polnomial. Note that a higher order interpolation is needed for comparable numerical accurac as that of the numerical schemes, if higher order accurac numerical schemes are used to solve PDEs on sparse grids (see [8,7,8]). Then we evaluate Pi () on the grid points of N L,l, which is the most refined meshes in the direction. Net, in the other direction (e.g. the direction), we construct (N r l ) quadratic interpolation polnomials Pj (), j =,...,N r l, and evaluate them on the grid points of N L,N L.Atlastweget P N L,N L U l,l, defined on the finest grid N L,N L. We summarize the algorithm of Krlov IIF scheme on sparse grids as following. 3

9 Algorithm: Krlov IIF Scheme with Sparse-Grid Combination Technique Step : Restrict the initial condition u(,, ) to (N L +) sparse grids { l,l } I defined above; Step : On each sparse grid l,l,solvetheeq.() b Krlov IIF scheme to reach the final time T.Thenweget(N L + ) sets of solutions {U l,l } I ; Step 3: At the final time T, on each grid l,l, appl prolongation operator P N L,N L on U l,l.thenweget P N L,N L U l,l, defined on the most refined mesh N L,N L. do the combination to get the final solution Û N L,N L = P NL,NL U l,l P N L,N L U l,l. (5) l +l =N L l +l =N L For three dimensional (3D) or higher dimensional problems, the algorithm is similar although prolongation operations are performed in additional spatial directions. The sparsegrid combination formula for higher dimensional cases can be found in the literature, e.g. in [8]. Specificall the 3D formula is Û N L,N L,N L = P NL,NL,NL U l,l,l3 P N L,N L,N L U l,l,l 3 l +l +l 3 =N L l +l +l 3 =N L (6) + P N L,N L,N L U l,l,l 3. l +l +l 3 =N L 4 Numerical Eperiments In this section, we use various numerical eamples to show the computational efficienc of Krlov IIF schemes with sparse-grid combination technique on sparse grids, b comparing to the same schemes on regular grids. Eamples include reaction diffusion equations without convection, and convection diffusion problems. Equations with different tpes of diffusions are tested, namel, equations with constant diffusion coefficients, with variable diffusion coefficients, and with/without cross derivatives. We test eamples with an eact solution and a three dimensional Fokker Planck equation which has broad applications. For each eample, we compute numerical accurac errors and convergence orders of the schemes, and record CPU times. We also list the ratios of corresponding CPU times on an N h N h mesh to that on a N h N h mesh, to stud the computational compleit of the schemes on sparse grids and on regular single grids. Here in the data Tables and tets of this section, N h N h (or a coarser one N h N h in the tet description) denotes the most refined mesh in sparse grids or a regular mesh in single grid computations. Since Krlov IIF schemes remove time step size constraint of stiff diffusion and reaction terms, the time step sizes can be taken as that for a pure hperbolic problem, i.e., proportional to the spatial grid sizes. For computations on sparse grids, PDEs are evolved on different semi-coarsened sparse grids. How to choose time step sizes for each individual time evolution is an interesting question. Via numerical eperiments, we found that for the Eample, which is a relativel simple constant diffusion problem without cross derivatives and convection terms, if the grids are uniform, the time step sizes are taken to be proportional to the minimum spatial grid size of each spatial direction on each individual semi-coarsened sparse grid l,l, i.e. t = c min(h l, h l ). c is a constant. Hence time step sizes ma take different values for solving the PDE on different semi-coarsened sparse grid, although each individual time evolution reaches the same final 3

10 time. The resulting numerical accurac orders keep the desired values. However for more complicated problems such as Eamples, 3, 4 and 5, time step sizes on all semi-coarsened sparse grids need to take the same value. It is determined b the spatial grid size h of the most refined grid N L,N L, namel, it is proportional to h with t = c h. Numerical eperiments show that the desired numerical accurac orders are reached with time step sizes taken this wa. Hence for a general problem, the numerical eperiments in this paper suggest that time step sizes on all semi-coarsened sparse grids should be determined b the spatial grid size h of the most refined grid N L,N L. All of the numerical simulations in this paper are performed on a.3 GHz, 6GB RAM Linu workstation. Eample (Isotropic diffusion problems). We consider a reaction diffusion problem with isotropic diffusion u =. ( u) +.u. t First we test the two dimensional case defined on the domain ={ < < π, < < π}, subject to periodic boundar conditions, i.e., u(,, t) = u(π,, t); u(,, t) = u(, π, t). The initial condition is u(,, ) = cos() + sin(). The eact solution of the problem is u(,, t) = e.t (cos() + sin()). We compute the problem till final time T = bthe second order Krlov IIF scheme (Krlov IIF) (3) on both single grids and sparse grids, and compare their computational efficienc. We present the L errors, L errors, the corresponding numerical accurac orders, and CPU times on successivel refined meshes to show the efficienc of computations on sparse grids. There are two different was to refine meshes for computations on sparse grids. One wa is to refine the root grid,, and keep the number of semi-coarsened sparse-grid levels (total N L + levels) unchanged. For eample, sparse-grid with a root grid and N L = 3 has the finest mesh 8 8. If the root grid is refined once to be, with N L = 3 unchanged we can obtain the finest mesh 6 6. The other wa is to increase the number of levels (refine level), and keep the root grid, unchanged. For eample, if we increase N L = 3toN L = 4 with a root grid, the finest mesh which is 8 8 for the N L = 3 case is refined to be 6 6 for the N L = 4 case. The numerical errors, accurac orders, and CPU times are listed in Table for computations b the Krlov IIF scheme on single-grid and sparse-grid. The computations on single-grid, and sparse grids with two different mesh refinement methods achieve the similar numerical errors and the second order accurac. However, computations on sparse-grid are much more efficient than those on single-grid. Comparing the CPU times in Table, we can see that for computations on sparse grids with the first mesh refinement method (i.e., refine root grids), more than 5 % computation time can be saved, especiall on more refined meshes. Moreover, the CPU time savings are even more significant for computations on sparse grids with the second mesh refinement method (i.e., refine level). As thatshownintable, 9 % CPU time can be saved for the computation on a mesh. We also list the ratios of corresponding CPU times on an N h N h mesh to that on a N h N h, to stud the computational compleit of the methods. For this two dimensional time dependent parabolic problem, we achieve large time step size computation t = O(h) b using the Krlov IIF method. A linear computational compleit method should have the CPU time ratio be 8 for a complete time evolution. The CPU time ratios shown in Table for computations on single-grid verif its linear computational compleit. For computations on sparse grids, the CPU time ratio is around 8 for the refining root grid case, and around 4 for 3

11 Table Eample, D case, Krlov IIF scheme, comparison of numerical errors and CPU times for computations on single-grid and sparse-grid Single-grid N h N h L error Order L error Order CPU(s) Ratio N r N L N h N h L error Order L error Order CPU(s) Ratio Sparse-grid, refine root grids Sparse-grid, refine level Final time T =.. For single-grid computations, t =.5 h. For sparse-grid computations, t =.5min(h l, h l ) on each semi-coarsened sparse grid l,l. Nr : number of cells in each spatial direction of a root grid. N L : the finest level in a sparse-grid computation. CPU: CPU time for a complete simulation. Ratio is the ratio of corresponding CPU times on an N h N h mesh to that on a N h N h mesh. CPU time unit: seconds the refining level case. Hence the computational compleit on sparse-grid is also linear for the first mesh refinement method, and much better than linear for the second mesh refinement method. We perform the same test for the third order scheme. The third order Krlov IIF scheme (Krlov IIF3) (4) on single-grid and the same scheme with sparse-grid combination technique are used to compute this two-dimensional problem till final time T =. Again we use two different was to refine meshes on sparse grids. The numerical results are reported in Table. Comparable numerical errors and fourth order accurac order are obtained for all three different approaches. The fourth order accurac order here is due to the fourth order central difference scheme to discretize the diffusion terms. It is obvious here that the spatial errors dominate and are larger than the temporal errors. Again, computations on sparse-grid are more efficient than those on single-grid as that shown in Table. Especiall for the the second mesh refinement method (i.e., refine level), 8 % CPU time can be saved for the computation on a mesh. In terms of computational compleit, the Krlov IIF3 scheme shows a linear computational compleit on single-grid as that for the second order scheme. The computational compleit of the Krlov IIF3 scheme on sparse-grid is also linear for the first mesh refinement method, and much better than linear for the second mesh refinement method. Then we test the three dimensional case defined on the domain ={ π, π, z π}, subject to no-flu boundar conditions. The initial condition is u(,, z, ) = 3

12 Table Eample, D case, Krlov IIF3 scheme, comparison of numerical errors and CPU times for computations on single-grid and sparse-grid Single-grid N h N h L error Order L error Order CPU(s) Ratio N r N L N h N h L error Order L error Order CPU(s) Ratio Sparse-grid, refine root grids Sparse-grid, refine level Final time T =.. For single-grid computations, t =.5 h. For sparse-grid computations, t =.5min(h l, h l ) on each semi-coarsened sparse grid l,l. Nr : number of cells in each spatial direction of a root grid. N L : the finest level in a sparse-grid computation. CPU: CPU time for a complete simulation. Ratio is the ratio of corresponding CPU times on an N h N h mesh to that on a N h N h mesh. CPU time unit: seconds cos() + cos() + cos(z). The eact solution is u(,, z, t) = e.t (cos() + cos() + cos(z)). We compute the problem till final time T =. The numerical errors, accurac orders, CPU times for a complete simulation and the ratios of CPU times on an N h N h mesh to that on a N h N h mesh are listed in Table 3 for the Krlov IIF scheme on single-grid and on sparsegrid with two different mesh refinement approaches. The computation on the single-grid can not be performed due to the computer memor restriction. Computer memor is saved significantl b using sparse-grid and computations can be successfull done for the mesh case. We observe that all computations give comparable numerical errors and the second order accurac. For a three dimensional time dependent problem with t = h/3, a linear computational compleit method should have the CPU time ratio be 6. For single-grid computation, the Krlov IIF scheme s CPU time ratios shown in Table 3 verif its linear computational compleit. We also observe that the Krlov IIF scheme on sparse-grid with the first mesh refinement method ( refining root grid) has CPU time ratio be around 6, so it also has linear computational compleit. And computations on sparse-grid with the second mesh refinement method (i.e., refining level) has CPU time ratio be around 5 as thatshownintable3, hence its computational compleit is much better than linear. In terms of computational efficienc, the savings of CPU times and improvement of the efficienc for solving this three dimensional problem on sparse-grid are more significant than that for two dimensional problems, as that shown in Table 3. For eample, we compare the CPU times for 3

13 Table 3 Eample, 3D case, Krlov IIF scheme, comparison of numerical errors and CPU times for computations on single-grid and sparse-grid Single-grid N h N h N h L error Order L error Order CPU(s) Ratio , , N r N L N h N h N h L error Order L error Order CPU(s) Ratio Sparse-grid, refine root grids , , Sparse-grid, refine level , Final time T =.. For single-grid computations, t = h/3. For sparse-grid computations, t = /3min(h l, h l, h l3 ) on each semi-coarsened sparse grid l,l,l 3. Nr : number of cells in each spatial direction of a root grid. N L : the finest level in a sparse-grid computation. CPU: CPU time for a complete simulation. Ratio is the ratio of corresponding CPU times on an N h N h N h mesh to that on a N h N h N h mesh. CPU time unit: seconds computations on a mesh. With the number of cells in each spatial direction of a root grid N r = 4 and the finest level N L = 3, the CPU time for the computation on sparsegrid (5,44.3 s) is about / of that on a single-grid (5,543.8 s). And with a coarser root grid N r = and the finest level N L = 5, the CPU time for the computation on sparsegrid (3.95 s) is about / of that on a single-grid (5,543.8 s), so 99 % CPU time is saved. Since higher dimensional problems generall demand much more computational time than low dimensional ones, the efficienc achieved here verifies advantages of Krlov IIF schemes designed on sparse-grid for solving higher dimensional problems. We use the Krlov IIF scheme here as an eample to further analze the computational compleit on singe-grid and sparse-grid. We estimate the number of multiplication and division operations in one time step for computations on single-grid and sparse-grid for the D case. The number of operations is (M + 4M + 9)[( +.5N L ) N L N r + (6 N L 4)N r + N L +] for the computation on sparse-grid with an N r N r root grid and N L fine levels. For the computation on an N h N h single-grid, the number of operations is (M +4M+9)N h. M is the dimension of Krlov subspace, and M = 5 here. In Table 4, we list the number of operations for these grids used in this eample. It shows that computations on sparse-grid need fewer operations than that on single-grid, especiall the savings of operations are ver significant for computations on sparse-grid with the second mesh refinement method (i.e., refining level). It is interesting to compare the computational efficienc of Krlov IIF method on singe-grid and sparse-grid studied in this paper with a full implicit scheme with an advanced linear ss- 3

14 Table 4 Eample, D case, Krlov IIF scheme, comparison of the number of multiplication and division operations in one time step for computations on single-grid and sparse-grid N h N h Single-grid Sparse-grid (refine root grid) Sparse-grid (refine level) 8 8 6,97,6 4,769,448 4,769, ,9,4 8,9,8,934, ,76,6 7,,38 8,65, ,46,4 8,564,968 66,77,99 tem solver such as a multigrid method. As an eample, we appl the Crank-Nicolson scheme [9] in discretizing the D case here. A multigrid solver (the Two-Grid correction scheme) [] is implemented to solve the linear sstem at ever time step. We take the number of relaation times to be 3 in the Two-Grid correction scheme []. The results including numerical errors, accurac orders and CPU times are reported in Table 5. The Crank Nicolson scheme with the Two-Grid correction multigrid solver for solving this problem has similar numerical errors and the second order accurac order as the Krlov IIF scheme on singe-grid and sparse-grid (Table ). In terms of computational efficienc, the Crank Nicolson scheme with the Two-Grid correction multigrid solver is more efficient on relativel coarse mesh (e.g. the 8 8 mesh) than the Krlov IIF scheme. However, on more refined meshes the Krlov IIF scheme is more efficient. Especiall, the improvement of efficienc is ver obvious for computations on sparse-grid. More sstematic comparisons of Krlov IIF schemes and full implicit schemes with efficient multigrid solvers will be carried out in our future research. Remark The numerical methods with sparse grid combination technique are presented using uniform rectangular meshes in this paper. The approach can be straightforwardl implemented on non-uniform rectangular meshes. Here we test the Krlov IIF scheme with sparse grid combination technique on non-uniform rectangular meshes b appling it in solving the D case of this eample. The non-uniform meshes are obtained b randoml perturbing - coordinates and -coordinates of a uniform mesh in the range of (.3h,.3h).We use five points in one spatial direction to approimate the diffusion terms on non-uniform meshes. Hence the approimations to the diffusion terms are on a centered stencil and the accurac order for the diffusion terms is 3. The numerical errors, accurac orders, and CPU times are listed in Table 6 for computations b the Krlov IIF scheme on single-grid and sparse-grid. We draw consistent conclusion with computations on uniform meshes. Namel, the computations on single-grid, and sparse grids with two different mesh refinement methods achieve the similar numerical errors, while computations on sparse-grid are much more efficient than those on single-grid. Eample (A 3D problem with anisotropic diffusion and constant diffusion coefficients). We consider a three-dimensional reaction diffusion problem with cross-derivative diffusion terms and constant diffusion coefficients u t = (.u.5u +.u ) + (.u +.u z +.u zz ) + (.u +.5u z +.u zz ) +.8u, 3

15 Table 5 Eample, D case, Crank Nicolson scheme with a multigrid solver (the Two-Grid correction scheme) for the linear sstems N h N h L Error Order L error Order CPU(s) Numerical errors, accurac orders and CPU times are presented. Final time T =.. t =.5 h. CPU: CPU time for a complete simulation. CPU time unit: seconds Table 6 Eample, D case, Krlov IIF scheme, Non-uniform grids Single-grid N h N h L error Order L error Order CPU(s) Ratio N r N L N h N h L error Order L error Order CPU(s) Ratio Sparse-grid, refine root grids Sparse-grid, refine level Comparison of numerical errors and CPU times for computations on single-grid and sparse-grid. Final time T =.. t =.5 h. For single-grid computations, h is the smallest grid size in all spatial directions. For sparse-grid computations, h is the smallest grid size of the most refined grid N L,N L. Nr : number of cells in each spatial direction of a root grid. N L : the finest level in a sparse-grid computation. CPU: CPU time for a complete simulation. Ratio is the ratio of corresponding CPU times on an N h N h mesh to that on a N h N h mesh. CPU time unit: seconds where (,, z) ={ < < π, < < π, < z < π} with periodic boundar conditions. The initial condition is u(,, z, ) = sin( + + z). The eact solution of the problem is u(,, z, t) = e.t sin( + + z). In [9], we show that for high dimensional problems with anisotropic diffusion terms, Krlov IIF schemes are more efficient than compact IIF methods [3]. It is interesting to test Krlov 3

16 Table 7 Eample, Krlov IIF scheme, comparison of numerical errors and CPU times for computations on single-grid and sparse-grid Single-grid N h N h N h L error Order L error Order CPU(s) Ratio , N r N L N h N h N h L error Order L error Order CPU(s) Ratio Sparse-grid, refine root grids , , Sparse-grid, refine level , Final time T =.. t = h/3. N r : number of cells in each spatial direction of a root grid. N L : the finest level in a sparse-grid computation. CPU: CPU time for a complete simulation. Ratio is the ratio of corresponding CPU times on an N h N h N h mesh to that on a N h N h N h mesh. CPU time unit: seconds IIF scheme on sparse-grid for such problems with anisotropic diffusion terms. We compute the problem till final time T = b the Krlov IIF scheme (3) on both single-grid and sparse-grid. The L errors, L errors, the corresponding numerical accurac orders, and CPU times on successivel refined meshes are reported in Table 7. As that in the last eample, the computation on the single-grid can not be performed due to the computer memor restriction. Computer memor is saved significantl b using sparse-grid and computations can be successfull done for the mesh case. We observe that computations on both single-grid and sparse-grid give similar numerical errors and the second order accurac. Again, it is shown in Table 7 that b preforming computations on sparse grids, a significant amount of CPU time can be saved, especiall if we use a relativel large finest level N L and a small number of cells N r in each spatial direction of the root grid. For eample, we compare the CPU times for computations on a mesh. With the number of cells in each spatial direction of a root grid N r = 4 and the finest level N L = 3, the CPU time for the computation on sparse-grid (9,573.6 s) is about % of that on a single-grid (3, s), and 78 % CPU time is saved. Furthermore, with a coarser root grid N r = and the finest level N L = 5, the CPU time for the computation on sparse-grid (437.8 s) is onl 3.3 % of that on a singlegrid (3, s), and 96.7 % CPU time is saved. We can also observe that if the mesh refinement is done b refining root grids, the Krlov IIF scheme on sparse grids has the linear computational compleit as that for the Krlov IIF scheme on single-grid, with CPU time ratios around 6. If the mesh refinement is done b refining level, the CPU time ratios are 3

17 around 6, and the computations on sparse grids have much better than linear computational compleit. Eample 3 (A 3D problem with anisotropic diffusion and variable diffusion coefficients). In this eample, we consider a three-dimensional reaction diffusion problem with crossderivative diffusion terms and variable diffusion coefficients u t =.5u.5sin( + )u +.5u +.5u 3 cos u z + 3 u zz (7) +.5( + cos )u.5( + cos )u z + 3 ( + cos )u zz + f (,, z, u), where (,, z) ={ < < π, < < π, < z < π} with periodic boundar conditions. The initial condition is u(,, z, ) = sin( + + z). The source term f (,, z, u) = ( sin( + ) + 3 (cos cos )) u. The eact solution of this problem is u(,, z, t) = e.t sin( + + z). As the last eample, in [9] we show that for this problem with anisotropic diffusion terms, Krlov IIF schemes are more efficient than compact IIF methods [3]. Here we use this eample to show the significant improvement of computational efficienc of Krlov IIF scheme on sparse grids. We compute the problem till final time T = b the Krlov IIF scheme (3) on both single-grid and sparse-grid. Again we use two different was to refine meshes for computations on sparse grids. The numerical results are reported in Table 8. We obtain the similar observations and draw the same conclusion as the last eample which has constant diffusion coefficients. Computer memor is saved significantl b using sparse-grid and computations can be successfull done for the mesh case, for which the computation on single-grid can not be performed due to computer memor restriction. Again, appling sparse-grid combination technique in the Krlov IIF scheme brings in a huge benefit in terms of CPU time savings while the similar numerical errors and accurac orders are kept as that for the single-grid computations. For eample, we compare the CPU times for computations on a mesh. With the number of cells in each spatial direction of a root grid N r = 4 and the finest level N L = 3, the CPU time for the computation on sparse-grid (55,6.3 s) is about 36 % of that on a single-grid (53, 95.4 s), and 64 % CPU time is saved. Furthermore, with a coarser root grid N r = and the finest level N L = 5, the CPU time for the computation on sparse-grid ( s) is onl 5.3 % of that on a single-grid (53, 95.4 s), and 94.7 % CPU time is saved. We can also observe that if the mesh refinement is done b refining root grids, the Krlov IIF scheme on sparse grids has the linear computational compleit as that for the Krlov IIF scheme on single-grid, with CPU time ratios around 6. If the mesh refinement is done b refining level, the CPU time ratios are around 6, and the computations on sparse grids have much better than linear computational compleit. Eample 4 (A convection diffusion problem). In this eample, we test the method for solving problems with convection terms. Consider a two-dimensional convection-diffusion problem ( ) ( ) ( u ) t + u u + u =. + u + f (,, t), 3

18 Table 8 Eample 3, Krlov IIF scheme, comparison of numerical errors and CPU times for computations on single-grid and sparse-grid Single-grid N h N h N h L error Order L error Order CPU(s) Ratio , N r N L N h N h N h L error Order L error Order CPU(s) Ratio Sparse-grid, refine root grids , , Sparse-grid, refine level , Final time T =.. t = h/3. N r : number of cells in each spatial direction of a root grid. N L : the finest level in a sparse-grid computation. CPU: CPU time for a complete simulation. Ratio is the ratio of corresponding CPU times on an N h N h N h mesh to that on a N h N h N h mesh. CPU time unit: seconds where (, ) ={ < < π, < < π} with periodic boundar conditions. The initial condition is u(,, ) = cos() + sin(). The eact solution is u(,, t) = e.t (cos() + sin()). The source term f (,, t) is ( ) f (,, t) =. + e.t ( sin() + cos()) e.t (cos() + sin()). The Krlov IIF scheme (3) with the third order WENO approimation for the convection terms is used here. We compute the problem till final time T = on both single-grid and sparse-grid. Here the time step sizes are determined onl b the convection (hperbolic) part of the equation since the IIF schemes remove stabilit constraint of diffusion and reaction terms []. The CFL number for the convection terms is taken to be.5 in the computations. Numerical errors, numerical accurac orders, CPU times for a complete simulation, and the ratios of CPU times on an N h N h mesh to that on a N h N h mesh are reported. Again, two approaches to perform mesh refinement in sparse-grid computations are used, i.e., the refining root grid approach and the refining level approach. In this eample, for mesh refinement in sparse-grid computations b the refining root grid approach, we test performance of the method with two different finest levels N L = 3andN L = 4. Numerical results are reported in Table 9. We observe that the desired second order accurac due to the second order Krlov IIF scheme is achieved for all methods. About computational efficienc, we observe that in general a big amount of CPU time is saved if computations are performed on sparse grids. 3

19 Table 9 Eample 4, Krlov IIF scheme, comparison of numerical errors and CPU times for computations on single-grid and sparse-grid Single-grid N h N h L error Order L error Order CPU(s) Ratio N r N L N h N h L error Order L error Order CPU(s) Ratio Sparse-grid, refine root grids, N L = Sparse-grid, refine root grids, N L = , Sparse-grid, refine level CFL number for the hperbolic terms is.5. Final time T =.. N r : number of cells in each spatial direction of a root grid. N L : the finest level in a sparse-grid computation. CPU: CPU time for a complete simulation. Ratio is the ratio of corresponding CPU times on an N h N h mesh to that on a N h N h mesh. CPU time unit: seconds Specificall, for eample for the mesh case, computations on sparse grids can save 57 % CPU time (the N r = 4, N L = 4 case), and even 83 % CPU time (the N r =, N L = 6 case) comparing with the single-grid computation, and keep comparable numerical errors. See Table 9. Again we can also observe that if the mesh refinement is done b refining root grids, Krlov IIF scheme on sparse grids has the linear computation compleit as that for the Krlov IIF scheme on single-grid, with CPU time ratios around 8. If the mesh refinement is done b refining level, the CPU time ratios are around 5, and the computations on sparse grids have better than linear computational compleit. Eample 5 (Three dimensional Fokker Planck equations). The Fokker Planck equation (FPE) [6,4] describes in a statistical sense how a collection of initial data evolves in time, e.g., in describing Brownian motion. It is a N-dimensional convection diffusion equation and has been applied in computing statistical properties in man sstems. In [8], Arrarepresentation integration factor scheme was applied in solving FPEs which describe the time 3