AN EFFICIENT PARALLEL ALGORITHM FOR THE NUMERICAL SOLUTION OF FRACTIONAL DIFFERENTIAL EQUATIONS. Kai Diethelm 1,2

Transcription

1 RESEARCH PAPER AN EFFICIENT PARALLEL ALGORITHM FOR THE NUMERICAL SOLUTION OF FRACTIONAL DIFFERENTIAL EQUATIONS Kai Diethelm 1,2 Abstract The numerical solution of differential equations of fractional order is known to be a computationally very expensive problem due to the nonlocal nature of the fractional differential operators. We demonstrate that parallelization may be used to overcome these difficulties. To this end we propose to implement the fractional version of the second-order Adams- Bashforth-Moulton method on a parallel computer. According to many recent publications, this algorithm has been successfully applied to a large number of fractional differential equations arising from a variety of application areas. The precise nature of the parallelization concept is discussed in detail and some examples are given to show the viability of our approach. MSC 2010 : Primary 65Y05; Secondary 65L05, 65R20 Key Words and Phrases: fractional differential equation, numerical solution, Adams-Bashforth-Moulton method, parallel algorithm 1. Introduction In recent years, fractional differential equations, i.e. differential equations involving differential operators of non-integer order, have proven to be very useful tools for modeling many phenomena in physics, engineering, finance, life sciences, and many other areas; see, e.g., [5, 17, 19, 20, 23, 27, 29] c 2011 Diogenes Co., Sofia pp , DOI: /s

2 476 K. Diethelm and the references cited therein for a number of examples. From a mathematical point of view this amounts to models of the form D α y(x) =f(x, y(x)), y(k) (0) = y (k) 0 (k =0, 1,..., α 1) (1.1) with some α>0. Here, denotes the ceiling function that rounds up to the nearest integer not less than its argument. In most applications one has 0 <α<1, so we have exactly one initial condition in the initial value problem (1.1), but for our purposes it will not be necessary to impose such a restriction; we shall rather consider the fully general case α>0. The standard literature in fractional calculus [5, 27, 29, 30] is aware of a great number of mutually non-equivalent definitions of fractional derivatives. The differential operator D α that we use in eq. (1.1) is the so-called Caputo derivative, given by D α y := J α α D α y, (1.2) where D α is the classical differential operator of (integer) order α and J μ denotes the Riemann-Liouville integral operator of order μ 0, defined to be the identity operator for μ = 0 and by the equation J μ y(x) = 1 x (x t) μ 1 y(t)dt (1.3) Γ(μ) 0 for μ>0(see,e.g.,[5, 3]). Here, Γ denotes Euler s Gamma function. As discussed in [5, 3 and 6], the Caputo operator is known to have properties that make it particularly suitable for real-world applications using models of the form (1.1). An important feature of fractional differential operators in general and the Caputo operator in particular is their non-locality [5, Remark 3.2] (except for the trivial case α N where the operators reduce to the well known classical differential operators of integer order). This means that, as is evident from eqs. (1.2) and, in particular, (1.3), it is not sufficient for the computation of D α y(x) to take into account only information on the function y in a small neighbourhood of the point x. Instead, it is necessary to use the complete history of y from the starting point 0 up to the point of interest x. This observation is often expressed by saying that fractional differential operators have a memory. When it comes to differential equations of fractional order, this implies that the behaviour of the solution at the point x + h, say, also depends on its values on the entire interval [0,x]and not only on a small neighbourhood of x. For the development of numerical methods for such equations, we thus have to be aware that the computation of a numerical solution at some point x j+1 of a discretization grid cannot be based solely on the approximate solution at the points x j,x j 1,...,x j k with some fixed k N 0 as one would do in a traditional (k+1)-step method

3 AN EFFICIENT PARALLEL ALGORITHM for a first-order equation [15, 16]. Rather, it is necessary to take into account the history up to the initial point x 0 in a proper way. Specifically it is possible to leave out a significant part of the grid points in this process but there is a minimum number of points that must remain in the computations in order to achieve a certain accuracy [10], and this implies that the arithmetic complexity of a solver for a fractional differential equation must be higher than the complexity of a corresponding classical algorithm for first-order equations. Consequently, the user of a simple sequential approach must be prepared to accept much longer run times in the fractional case than would be necessary in the first-order situation. It is therefore a natural question to ask whether parallelization can be a way to reduce the run times, and we shall demonstrate in this paper that this is indeed the case. We note that while we only consider the solution of the fractional ordinary differential equation (1.1) here, it is clear that our findings can be transferred to obtain corresponding results also for time-fractional partial differential equations. Moreover they can be applied to situations like finite element codes where, for example, in a deformation analysis the integer-order differential equation model for the material law is replaced by a fractional-order model to allow the simulation of viscoelastic materials which is a potential future extension of, e.g., the sheet material forming code INDEED [13]. 2. The basic numerical method Many numerical algorithms for initial value problems of the form (1.1) have been proposed in recent years; see, e.g., [1, 4, 6, 7, 8, 21, 22] and the references cited therein. We have chosen one of these methods as a prototype for our parallelization project, namely the fractional Adams- Bashforth-Moulton scheme of [6]. This is a particularly well understood algorithm for which a detailed theoretical analysis is available [7]. Moreover, it has been shown to be stable and reliable also in cases where the differential equation to be solved was considered to be difficult [31, 32], and comparative studies [9] have demonstrated that its performance compares favourably with other methods. Moreover it can be seen that the basic principle of our approach can be carried over to other numerical methods too, and the fundamental properties will remain intact. In view of the availability of precise information about this method in the publications mentioned above, we shall be very brief here and only give a most fundamental description and refer the reader to those papers for further details. Specifically, the fractional Adams-Bashforth-Moulton method (or, for short, Adams method) has been constructed to solve the

4 478 K. Diethelm fractional initial value problem (1.1) on the interval [0,T] withsomet>0. TothisendwechooseanintegerN and define the step size h := T/N and the grid points x j := jh, j =0, 1,...,N,sothatx 0 =0andx N = T are the two end points of the interval on which the solution is sought. The approximate solution at the point x j will be denoted by y j. For j =0,thevaluey j = y 0 is given by the initial condition in eq. (1.1). The values y j for j =1, 2,...,N will then be computed in an iterative manner. In particular, assuming that y 0,y 1,...,y j have already been calculated, the Adams method determines a preliminary approximation yj+1 P, the so-called predictor, by α 1 yj+1 P = k=0 where the weights b μ are given by x k j+1 y (k) 0 + h α k! j b j k f(x k,y k ), (2.1) k=0 b μ = (μ +1)α μ α. (2.2) Γ(α +1) This intermediate step is then followed by the computation of the final approximation y j+1 via the formula y j+1 = α 1 k=0 x k j+1 y (k) 0 (2.3) k! j a j k f(x k,y k )+ f(x j+1,yj+1 P ) ) Γ(α +2) + h α (c j f(x 0,y 0 )+ k=1 with weights a μ and c μ being defined as and a μ = (μ +2)α+1 2(μ +1) α+1 + μ α+1 Γ(α +2) (2.4) c μ = μα+1 (μ α)(μ +1) α, (2.5) Γ(α +2) respectively. Under reasonable assumptions we can then expect the convergence of the algorithm in the sense that max y(x j) y j = j=0,1,...,n { O(h 1+α ) for 0 <α<1, O(h 2 ) for α 1, see [7], and it is evident that the arithmetic complexity of the algorithm is O(N 2 )=O(h 2 ).

5 AN EFFICIENT PARALLEL ALGORITHM Parallelization of the fractional Adams method We now come to the principal task of this paper, namely the implementation of the Adams method described in eqs. (2.1), (2.2), (2.3), (2.4) and (2.5) above in a parallel computing environment. First of all we note that the computational effort for the calculation of the weights of our scheme according to eqs. (2.2), (2.4) and (2.5) is negligible in comparison to the cost for the solution algorithm itself, i.e. the evaluation of the formulas (2.1) and (2.3). For example, on a single core of a 1.6GHz Itanium2 Montecito dual core processor we needed less than half a second for the calculation of all these weights for N =10 6. Thus, while the structure of the weights implies that a parallelization of the computation of the weights is a simple and straightforward matter, such a parallelization is not likely to have any significant influence on the overall performance of the algorithm. For the parallelization of the remaining part of the problem, i.e. the evaluation of eqs. (2.1) and (2.3) for j =0, 1,...,N 1, we assume a system with p cores to be available. Our basic strategy then is to divide the set y 1,y 2,...,y N of values to be computed into blocks of size p so that the lth block contains the variables y (l 1)p+1,...,y lp (l =1, 2,..., N/p 1), and the N/p th block contains the remaining variables y ( N/p 1)p+1,...,y N. (This last block contains p elements if and only if p is a divisor of N; otherwise it will contain less than p elements. No problems will result from this fact.) We will then process one block after the other in such a way that during the handling of each block, each processor will be assigned to exactly one of the variables contained in the block. At the end of each block, all processors then send their results to the other processors so that, at the beginning of the lth block, all y j computed in the previous blocks are known to all processors. Thus we shall now describe the exact procedure used in the computations for the lth block. The key to an efficient parallelization of the Adams method is the exploitation of the structure of the sums in eqs. (2.1) and (2.3). In this context we rewrite these two sums in the forms and respectively, where y P j+1 = I j+1 + h α H P j,l + hα L P j,l (3.1) y j+1 = I j+1 + h α H j,l + h α L j,l, (3.2) I j+1 := α 1 k=0 x k j+1 y (k) 0 k!

6 480 K. Diethelm is a sum with a fixed and typically very small number of summands that appears in both sums. Moreover, the other quantities appearing in eqs. (3.1) and (3.2) are defined by H P j,l := L P j,l := (l 1)p k=0 j k=(l 1)p+1 b j k f(x k,y k ), b j k f(x k,y k ), (l 1)p H j,l := c j f(x 0,y 0 )+ a j k f(x k,y k ), k=1 and L j,l := j k=(l 1)p+1 a j k f(x k,y k )+ f(x j+1,yj+1 P ). Γ(α +2) Recall that, in the block l that we need to consider, we have to compute these expressions for j =(l 1)p, (l 1)p +1,...,lp 1. Each of the p processors that we had assumed to be available will then be assigned with the task of computing y P j+1 and y j+1 for exactly one value of j. To this end, each processor first computes the sums I j+1, H P j,l and H j,l for its value of j. As far as these computations are concerned, we observe a few facts: (1) For these computations, one needs to know only the initial values (which are given as part of the initial value problem), the weights of the algorithm (that have been computed in advance) and the y k for k (l 1)p, i.e. approximate solution values that have already been computed in previous blocks. Thus, at the beginning of the lth block, all required data are available. (2) The computations to be performed by any of the processors are completely independent of the computations of the others processors. In particular, no communication is required between the processors in this part of the block. (3) Each processor has the task of computing I j, i.e. a sum of α terms, and two sums of (l 1)p + 1 terms each, namely H P j,l and H l,j. Thus, the work load is distributed across the processors in a uniform way; we hence have a very good load balancing, and it may be expected that all processors require similar amounts of time for this part of the task.

7 AN EFFICIENT PARALLEL ALGORITHM (4) The amount of time required for this part of the block depends on l in a linear way. When l is small, it can be completed very quickly, but as l increases it will take longer and longer. In the second part of the block we have to deal with the sums L P j,l and L j,l. Specifically we note that the sum L P j,l is empty for j =(l 1)p (i.e. the smallest of the indices under consideration in the current block). Thus, for this value of j the value yj+1 P can be computed by the appropriate processor. Moreover, the expression L j,l = L (l 1)p,l in this case reduces to f(x (l 1)p+1,y(l 1)p+1 P )/Γ(α + 2) which can now also be computed, allowing us to finalize the calculation of y (l 1)p+1. The processor associated with this task then additionally computes f(x (l 1)p+1,y (l 1)p+1 ) and passes all its results to all other processors. Its job is finished for this block, and so it becomes idle. (Strictly speaking, it would not have been necessary to compute and store the value f(x j,y j ), but since this value will be used again and again in future steps, it is more efficient to do so than to call the function f with the same set of arguments for a large number of times.) The other processors have had to wait for this information about y (l 1)p+1, but once they have received it, they are in a position to compute the first summands of their respective values L P j,l and L j,l. This sum only consists of this one summand for the next value of j, viz.j =(l 1)p +1, sothe associated processor can complete the calculation of yj+1 P = yp (l 1)p+2 and subsequently also the computation of y (l 1)p+2 and f(x (l 1)p+2,y (l 1)p+2 ) itself since all the data required for the evaluation of L (l 1)p+1,l are now present. This processor then also passes the result of the computation to all other processors (including the first one that has already completed its task and that needs this value in the next block). In this fashion, we work our way through all y j so that, at the end of this process, all processors have computed their y j and f(x j,y j ) and passed the results to all other processors. This concludes the lth block, and we now have the data of y j, j =1, 2,...,lp available at all processors. With respect to this part of the block, none of our four observations above applies. Instead we find: (1) Only the first value y (l 1)p+1 could be immediately computed. All other y j needed to have input from earlier values computed in this block and had to wait for the other processors to provide these values. (2) As a consequence of the first comment, a certain amount of communication between the processors is required. Specifically, each processor computes its assigned value y j in the indicated way and then passes the result to all other processors.

8 482 K. Diethelm (3) The work load is small for the first processor and monotonically increases up to the last processor, so in this part of the block we do not have a good load balance. On average, each processor remains idle for half of the time spent in the second part of the block. (4) The absolute amount of time required for this part of the block is independent of l. Thus, in view of our previous observation on the first part of the block, the relative amount of time that this part of the block takes becomes smaller as l increases, and hence, as the algorithm proceeds, the inefficiency caused by the load imbalance becomes less and less severe. The procedure described above for the lth block then needs to be repeated in an iterative manner for l =1, 2,..., N/p 1. This will give us the numerical solution at the points x = x j, j =1, 2,...p( N/p 1). For the remaining values x j with j = p( N/p 1) + 1,...,N we proceed in an almost identical way, except that now the number of values that need to be computed may be smaller than the number p of processors. (As mentioned above, the numbers will be equal if and only if p is a divisor of N.) Thus, for this last block we only employ as many processors as we have approximate function values that need to be computed, and these processors perform their computations in the same way as they did in the previous blocks; the other processors may remain idle. Using the tracing system VAMPIR [2, 3, 14, 25, 26] we can visualize the structure of the algorithm described above. For an example with 8 processors, the results are depicted in Figures 1 and 2. Specifically, Figure 1 shows a time slice of approximately 30μs that corresponds to one block of the algorithm. The periods marked in light gray are those required for computational purposes, the dark periods indicate communication. The latter includes time spent by individual processors waiting for data from other processors. We can clearly see the large part on the left where all processors are busy with computations. This is just the first part of the block. Once this part is completed, we enter the second part that is mainly devoted to communication and waiting for data from other processors, with only a small amount of time spent for actual computations in between. A more detailed view of the second part of the same block is provided in Figure 2. This part covers a time slice of about 12μs. Here, in addition to the elements shown in Figure 1 we have also included a visualization of the messages that are being passed. Each message is indicated by a black line connecting the sender (at the left end of the line) and the receiver (at the right end). For example, the black lines associated with the first process (displayed at the top) indicate the values of the first grid point

9 AN EFFICIENT PARALLEL ALGORITHM Figure 1. Visualization of one block of the algorithm. Light gray indicates time spent by the individual processes for actual computation; dark gray indicates time for communication. being passed from this processor to the seven others (the seven lines on the left) and then the values of the other grid points that it receives from the other processors (the seven lines on the right). Similarly, the last processor must receive the data from all others before it can finish its computations and send the result to the remaining processors. The processors in the middle receive some data from their respective predecessors, finish their computation, pass the results to all other processors and finally receive the results from their successors. An analysis of the algorithm using the performance evaluation toolset Scalasca [11, 12, 18] has also been carried out to find out possible performance problems and to introduce appropriate countermeasures. The results confirmed our expectation that the load balancing was very good

10 484 K. Diethelm Figure 2. Visualization of second part only of one block of the algorithm. Light gray indicates time spent by the individual processes for actual computation; dark gray indicates time for communication. Black lines indicate messages passed between processes. in the sense that the effort for both computation and communications was evenly distributed among the processors. Evidently, the algorithm described above may be implemented both on a distributed memory system, e.g. using MPI [24] as a means to establish the communication between the processors, or on a shared memory system using, e.g., the OpenMP system [28] for the parallelization. It is clear that, in the latter case, the passing of the information about the newly computed y j need not be done explicitly because the memory is shared and so all processors have access to the full set of data anyway. Thus, in such an environment it is only necessary for every processor to send a signal to the other processors when it has completed its calculations so that the other

11 AN EFFICIENT PARALLEL ALGORITHM processors know that the required data has been stored in the memory and that they may proceed with their computations. We have implemented, tested and analyzed both these variants and we shall report the results in the next section. 4. Numerical examples We conclude the paper with a look at some numerical examples. The computations were performed on a standard workstation with two Intel Xeon Gainestown quad core processors with a clock rate of 3.2GHz. Our primary test case is the simple linear differential equation D α y(x) = y(x), x [0, 5], (4.1) subject to the initial conditions y(0) = 1 and y (k) (0) = 0 (k =1, 2,..., α 1). An inspection of the algorithm reveals that its behaviour is independent of the chosen value of α except for the computation time required for the sums I j which, however, is completely negligible compared to the overall run time. We thus concentrate on only one value for α, inourcaseα =1.3, so the initial condition for eq. (4.1) reduces to y(0) = 1, y (0) = 0. (4.2) It is well known [5, Theorem 6.11] that the exact solution of this problem can be given with the help of the Mittag-Leffler function E α and takes the form y(x) =E α ( x α ). This example is particularly important in view of the fundamental significance of the Mittag-Leffler functions in fractional calculus (see, e.g., [5, 4] and the references cited therein) and the lack of efficient direct numerical methods for their evaluation, in particular for α>1. Tests with other differential equations have been done too, but since their results showed no major differences to the results of this problem, we do not give any details here for the sake of brevity. In Table 1 we report the run times required by the shared memory implementation of our code on the two hardware platforms for two different choices of the number of grid points N. We notice that the shared memory implementation scales relatively well. The fact that the speedup for N = is better than for N =10 6 is due to the synchronization procedure that is contained in the second part of each block. Specifically, since all processes need to wait for each other, an interruption of one of the processes (typically due to the fact that the corresponding core has to work on some unaviodable operating system task) implies that all other processes have to wait as well, and in a

12 486 K. Diethelm number of processor run time cores (seconds) speedup number of processor run time cores (seconds) speedup Table 1. Run times for the solution of eqs. (4.1), (4.2) for α =1.3 withn = (left) and N =10 6 (right) on an 8-core shared memory system. longer running process this will simply happen more often than in a shorter process. The corresponding results for the distributed-memory version are listed in Table 2. number of processor run time cores (seconds) speedup number of processor run time cores (seconds) speedup Table 2. Run times for the solution of eqs. (4.1), (4.2) for α =1.3 withn = (left) and N =10 6 (right) on an 8-core system with distributed memory. The timing behaviour of the distributed memory version is not as good as for the shared memory version. This is due to the fact that much more communication is involved here for passing the individual results of each process to the other processes which necessarily takes up additional time.

13 AN EFFICIENT PARALLEL ALGORITHM We have attempted to modify the algorithm in an attempt to reduce the number of messages that need to be passed between the processes. To this end we have introduced another parameter b, a local block length. The idea is that each block is now b times larger than before and is now subdivided into p subblocks of length b each. Then, the first processor will compute the first b approximate values, pass all of them to the other processors in one message instead of passing each of them separately, and the next processor handles the next b values, and so on. In this way the number of messages is reduced by a factor of b, and each message now contains b values instead of only one. However, the reduction in communication comes at a price of longer waiting times in each block because the processes now need to do more work before the messages can be passed, i.e. before the next processes can continue their work. This observation explains the fact that the performance of this modified version did not differ significantly from the original version. It is evident from the derivation of the parallelization approach that the number of processors must be in a reasonable relation to the number of grid points. If there are too many processors then the time required for the communication between the processes and the time spent by some processes waiting for other processes to provide the required data in the second part of each block will exceed the time required for the actual computation of the numerical solution and thus will dominate the total running time of the program. But if the number of processors is chosen in an appropriate way then the parallelization approach introduced in this paper can be used successfully to obtain a substantial reduction of the run times. 5. Conclusions We have developed a parallel version of the fractional Adams-Bashforth-Moulton algorithm and demonstrated that it scales well on a shared memory parallel computer. Hence it follows that parallelization can be an efficient means of reducing the normally very long run times required by traditional solvers for fractional differential equations. The method can also be used on a distributed memory system but in such an environment the performance is somewhat weaker. An investigation of the concept reveals that it can be successfully combined with other acceleration methods like the nested mesh principle of Ford and Simpson [10], thus (at least on a shared memory system) providing a very fast overall algorithm. Moreover we remark that the approach can also be applied to algorithms for the numerical simulation of systems with memory that use concepts other than fractional derivatives for modeling the influence of past states.

14 488 K. Diethelm Acknowledgement The work described in this paper was supported by the German Federal Ministry of Education and Research (BMBF) under grant no. 01IH08006C ( Scalable infrastructure for the automatic performance analysis of parallel codes ). References [1] D. Baleanu, K. Diethelm, E. Scalas, and J. J. Trujillo, Fractional Calculus: Models and Numerical Methods. World Scientific, Singapore (In preparation). [2] H. Brunst, D. Hackenberg, G. Juckeland and H. Rohling, Comprehensive performance tracking with Vampir 7. Tools for High Performance Computing 2009 (M. S. Müller, M. M. Resch, A. Schulz and W. E. Nagel, Eds.) Springer, Berlin (2010), [3] H. Brunst, M. Winkler, W. E. Nagel and H.-C. Hoppe, Performance optimization for large scale computing: The scalable VAMPIR approach. Computational Science ICCS 2001, Part II (V. N. Alexandrov, J. J. Dongarra, B. A. Juliano, R. S. Renner and C. J. K. Tan, Eds.), Springer, Berlin (2001), [4] K. Diethelm, Efficient solution of multi-term fractional differential equations using P(EC) m E methods. Computing 71 (2003), [5] K. Diethelm, The Analysis of Fractional Differential Equations. Springer, Berlin (2010); dynamical+systems/book/ [6] K. Diethelm, N.J. Ford and A.D. Freed, A predictor-corrector approach for the numerical solution of fractional differential equations. Nonlinear Dynamics 29 (2002), [7] K. Diethelm, N.J. Ford and A.D. Freed, Detailed error analysis for a fractional Adams method. Numer. Algorithms 36 (2004), [8] K. Diethelm, N.J. Ford, A.D. Freed and Y. Luchko, Algorithms for the fractional calculus: A selection of numerical methods. Comput. Methods Appl. Mech. Eng. 194 (2005), [9] N.J. Ford and J.A. Connolly, Comparison of numerical methods for fractional differential equations. Commun. Pure Appl. Anal. 5 (2006), [10] N.J. Ford and A.C. Simpson, The numerical solution of fractional differential equations: Speed versus accuracy. Numer. Algorithms 26 (2001), [11] M. Geimer, F. Wolf, B.J.N. Wylie, E. Ábrahám, D. Becker and B. Mohr, The Scalasca performance toolset architecture. Concurrency and Computation: Practice and Experience 22 (2010),

15 AN EFFICIENT PARALLEL ALGORITHM [12] M. Geimer, F. Wolf, B.J.N. Wylie, D. Becker, D. Böhme, W. Frings, M.-A. Hermanns, B. Mohr and Z. Szebenyi, Recent developments in the Scalasca toolset. Tools for High Performance Computing 2009 (M.S. Müller, M.M. Resch, A. Schulz and W.E. Nagel, Eds.) Springer, Berlin (2010), [13] GNS mbh, The Numerical Forming Simulation Software Package IN- DEED; [14] GWT-TUD GmbH, Vampir Performance Optimization; [15] E. Hairer, S.P. Nørsett and G. Wanner, Solving Ordinary Differential Equations I: Nonstiff Problems. 2nd ed., Springer, Berlin (1993). [16] E. Hairer and G. Wanner, Solving Ordinary Differential Equations II: Stiff and Differential-Algebraic Problems. Springer, Berlin (1991). [17] R. Hilfer (Ed.), Applications of Fractional Calculus in Physics. World Scientific, Singapore (2000). [18] Jülich Supercomputing Centre, SCALASCA Scalable Performance Analysis of Large-Scale Applications; [19] R. Klages, G. Radons and I. M. Sokolov (Eds.), Anomalous Transport: Foundations and Applications. Wiley-VCH, Weinheim (2008). [20] A. Le Méhauté, J.A. Tenreiro Machado, J.C. Trigeassou and J. Sabatier (Eds.), Fractional Differentiation and its Applications. Ubooks, Neusäß (2005). [21] C. Lubich, Runge-Kutta theory for Volterra and Abel integral equations of the second kind. Math. Comp. 41 (1983), [22] C. Lubich, Fractional linear multistep methods for Abel-Volterra integral equations of the second kind. Math. Comp. 45 (1985), [23] R. Magin, Fractional Calculus in Bioengineering. Begell House, Redding (2006). [24] Message Passing Interface Forum, MPI: A Message-Passing Interface Standard; [25] M. S. Müller, A, Knüpfer, M. Jurenz, M. Lieber, H. Brunst, H. Mix and W.E. Nagel, Developing scalable applications with Vampir, VampirServer and VampirTrace. Parallel Computing: Architectures, Algorithms and Applications (C. Bischof, M. Bücker, P. Gibbon, G. R. Joubert, T. Lippert, B. Mohr and F. Peters, Eds.), IOS Press, Amsterdam (2008), [26] W.E. Nagel, A. Arnold, M. Weber, H.-C. Knoppe and K. Solchenbach, VAMPIR: Visualization and analysis of MPI resources. Supercomputer 12 (1996), [27] K.B. Oldham and J. Spanier, The Fractional Calculus. Academic Press, New York (1974).

16 490 K. Diethelm [28] OpenMP Architecture Review Board, The OpenMP API Specification for Parallel Programming; [29] I. Podlubny, Fractional Differential Equations. Academic Press, San Diego (1999). [30] S.G. Samko, A.A. Kilbas and O.I. Marichev, Fractional Integrals and Derivatives: Theory and Applications. Gordon and Breach, Yverdon (1993). [31] M.S. Tavazoei, Comments on stability analysis of a class of nonlinear fractional-order systems. IEEE Trans. Circuits Systems II 56 (2009), [32] M.S. Tavazoei, M. Haeri, S. Bolouki and M. Siami, Stability preservation analysis for frequency-based methods in numerical simulation of fractional-order systems. SIAM J. Numer. Anal. 47 (2008), GNS Gesellschaft für numerische Simulation mbh Am Gaußberg Braunschweig, GERMANY diethelm@gns-mbh.com 2 Institut Computational Mathematics Technische Universität Braunschweig Pockelsstr Braunschweig, GERMANY k.diethelm@tu-bs.de Received: April 15, 2011 Please cite to this paper as published in: Fract. Calc. Appl. Anal., Vol. 14, No 3 (2011), pp ; DOI: /s