Gregory Capra. April 7, 2016

Transcription

1 ? Occidental College April 7, / 39

2 Outline? diffusion? Method Gaussian Elimination Convergence, Memory, & Time Complexity Conjugate Gradient Squared () Algorithm Concluding remarks 2 / 39

3 Standard You should be familiar with these: Food coloring in water cigarette smoke into the air water into cooking noodles (pasta)? 3 / 39

4 Standard? You should be familiar with these: Food coloring in water cigarette smoke into the air water into cooking noodles (pasta) and maybe these: oxygen from plant cells into the air oxygen from blood cells in the human body s blood stream into muscles 3 / 39

5 Everything isn t Standard Standard random motion & mean-free path Gaussian Distribution? 4 / 39

6 Everything isn t Standard Standard random motion & mean-free path Gaussian Distribution? σr 2 Dt 1 u t = D 2 u x 2 4 / 39

7 Everything isn t Standard Standard random motion & mean-free path Gaussian Distribution? σr 2 Dt 1 u t = D 2 u x 2 Anomalous inter-dependencies & random fluctuations Pareto Distribution 4 / 39

8 Everything isn t Standard Standard random motion & mean-free path Gaussian Distribution? σr 2 Dt 1 u t = D 2 u x 2 Anomalous inter-dependencies & random fluctuations Pareto Distribution σr 2 Dt α u t = D α u x α 4 / 39

9 Gaussian vs. Pareto Distribution? 5 / 39

10 Anomalous? Definition: Anomalous is a diffusion process that has a non-linear relationship to time, i.e., it is affected by random fluctuations and dependencies.[1] Examples include: Transport of electrons in a photocopier Foraging behavior of animals Trapping of contaminants in groundwater Proteins across cell membranes Super-: which exceeds linear relationship with time, faster than classical diffusion. Dt α, α > 1 Sub-: less than linear, slower than classical diffusion. Dt α, α < 1 [1] 6 / 39

11 Anybody?? basic definition of the whole derivative of a function f(x): d f(x) f(x h) f(x) = lim dx h 0 h Repeated composition of this operation leads to d n 1 n ( ) n f(x) = lim dxn h 0 h n ( 1) j f(x jh), n Z + j j=0 Grünwald-Letnikov fractional p-th order derivative of f(x) [2]: d p f(x) = lim dxp h 0 x x0 h j=0 Γ(j p) f(x jh) Γ( p)γ(j + 1) p R + 7 / 39

12 Half-Derivative of x? Blue:f(x) = x Red:f (x) = 1 Purple:f ( 1 x 2 ) (x) = 2 π 8 / 39

13 x n dx = xn+1 n C n 1? 9 / 39

14 ? x n dx = xn+1 n C n 1 result of integrating a function is non-local; it depends on how the function behaves over the range for which the integration is performed, not just at a single point. 9 / 39

15 ? x n dx = xn+1 n C n 1 result of integrating a function is non-local; it depends on how the function behaves over the range for which the integration is performed, not just at a single point. d dx xn = nx n 1 9 / 39

16 ? x n dx = xn+1 n C n 1 result of integrating a function is non-local; it depends on how the function behaves over the range for which the integration is performed, not just at a single point. d dx xn = nx n 1 However, differentiation is thought of as local, because whole derivatives happen to possess this attribute. 9 / 39

17 ? x n dx = xn+1 n C n 1 result of integrating a function is non-local; it depends on how the function behaves over the range for which the integration is performed, not just at a single point. d dx xn = nx n 1 However, differentiation is thought of as local, because whole derivatives happen to possess this attribute. apparent paradoxes of fractional derivatives arise from the fact that, in general, differentiation is non-local, just as in integration! 9 / 39

18 Where Do We Go From Here Anomalous diffusion is modeled with Partial Differential (PDE s) that incorporate these fractional derivatives.? 10 / 39

19 Where Do We Go From Here Anomalous diffusion is modeled with Partial Differential (PDE s) that incorporate these fractional derivatives. Non-local? 10 / 39

20 Where Do We Go From Here Anomalous diffusion is modeled with Partial Differential (PDE s) that incorporate these fractional derivatives. Non-local knowledge of previous solutions? 10 / 39

21 Where Do We Go From Here? Anomalous diffusion is modeled with Partial Differential (PDE s) that incorporate these fractional derivatives. Non-local knowledge of previous solutions need to store these solutions 10 / 39

22 Where Do We Go From Here? Anomalous diffusion is modeled with Partial Differential (PDE s) that incorporate these fractional derivatives. Non-local knowledge of previous solutions need to store these solutions full matrices 10 / 39

23 Where Do We Go From Here? Anomalous diffusion is modeled with Partial Differential (PDE s) that incorporate these fractional derivatives. Non-local knowledge of previous solutions need to store these solutions full matrices Most problems cannot be solved directly, instead you have to approximate the solution 10 / 39

24 Often the framework for setting up an approximate solution is with a discretization.? 11 / 39

25 ? Often the framework for setting up an approximate solution is with a discretization. b a f(x)dx = lim n f(x i ) x i=1 11 / 39

26 Typical Initial-Boundary Value Problem (IBVP)? two-sided diffusion equation, 1 < α < 2: u(x, t) t Domain: d + (x, t) α u(x, t) + x α d (x, t) α u(x, t) x α = f(x, t). Boundary Conditions: x L x x R, 0 < t T, Initial Condition: u(x L, t) = 0, u(x R, t) = 0. u(x, 0) = u 0 (x) 12 / 39

27 IBVP cont.? [10]: step size h: h = (x R x L ) N time step t: t = T M spatial partition: x i = x L + ih for i = 0, 1,..., N temporal partition: t m = m t for m = 0, 1,..., M notation: u m i = u(x i, t m ), d m +,i = d +(x i, t m ), d m,i = d (x i, t m ), and fi m = f(x i, t m ). 13 / 39

28 Simplifying the Equation? To replace the fractional derivatives we use the Grünwald approximations, shown earlier. resulting equation is such: u(x, t) t d (x, t) h α d +(x, t) h α ( N i+1 k=0 ( i+1 k=0 g (α) k um i k+1 + O(h) g (α) k um i+k 1 + O(h) ) ) = f(x, t). Grünwald weights gk α are defined with gα k = ( 1)k( ) α k where ( α k) are binomial coefficients of order α. se weights satisfy the following recursive relation [14]: ( g (α) 0 = 1, g (α) k = 1 α + 1 ) g (α) k k 1 for k / 39

29 (CN) Method general diffusion equation:? 15 / 39

30 (CN) Method? general diffusion equation: finite difference scheme [11]: ( u t = F u, x, t, u ) x, α u x α 15 / 39

31 (CN) Method? general diffusion equation: finite difference scheme [11]: u m+1 i u m i t = 1 2 ( u t = F u, x, t, u ) x, α u x α [ ( F m+1 i u, x, t, u ) x, α u x α + Fi m ( u, x, t, u )] x, α u x α 15 / 39

32 (CN) Method? general diffusion equation: finite difference scheme [11]: u m+1 i u m i t = 1 2 ( u t = F u, x, t, u ) x, α u x α [ ( F m+1 i u, x, t, u ) x, α u x α + Fi m ( u, x, t, u )] x, α u x α purpose of the finite difference method is to approximate the solution to a differential equation by approximating the derivatives with finite differences. 15 / 39

33 (CN) Method? general diffusion equation: finite difference scheme [11]: u m+1 i u m i t = 1 2 ( u t = F u, x, t, u ) x, α u x α [ ( F m+1 i u, x, t, u ) x, α u x α + Fi m ( u, x, t, u )] x, α u x α purpose of the finite difference method is to approximate the solution to a differential equation by approximating the derivatives with finite differences. This method is second-order in time, i.e., O(( t) 2 ). It is first-order in space, i.e., O(( x)). se are convergence rates. [11] 15 / 39

34 It was stated earlier that the converges at a rate of O(( t) 2 ) + O( x).? 16 / 39

35 ? It was stated earlier that the converges at a rate of O(( t) 2 ) + O( x). Convergence is dragged down by O( x). It will follow linear rate. 16 / 39

36 ? It was stated earlier that the converges at a rate of O(( t) 2 ) + O( x). Convergence is dragged down by O( x). It will follow linear rate. In order to improve to second-order in space, we will use the well-known [11] So O( x) O(( x) 2 ) 16 / 39

37 How it Works : 2u m+1 i ( ) h u m+1 2 i (h) u m+1 i (h)? 17 / 39

38 How it Works? : 2u m+1 i ( ) h u m+1 2 i (h) u m+1 i (h) for example, the regular solution (N = 5) and twice-refined (N = 10): u 1 u 2 u 3 u 4 v 1 v 2 v 3 v 4 v 5 v 6 v 7 v 8 v 9 17 / 39

39 Test Problem? Order: α = 1.8 Spatial Domain: x L = 0, x R = 1 Time Interval: t L = 0, t R = 1 d + coefficient: γ(1.2)(x α ) d coefficient: γ(1.2)((1 x) α ) Source term: f(x, t) =.0032e (5000x t 2( 1 x ) 2 ( x 2 + ( 1 x ) 2) 13.2 ( x 3 + ( 1 x ) 3) ( + 12 x 4 + ( 1 x ) 4) ) Initial Condition: f(x) = 16x 2 (1 x) 2 True : f(x, t) = 16e t x 2 (1 x) 2 18 / 39

40 Measuring Error? Since we know the true solution, we can generate an error vector by taking the difference between the true solution vector and our generated approximation. two most common ways to analyze that error vector is with the L 2 or L norm, defined below: x 2 = n i=1 x 2 i x = max i=1,...,n x i 19 / 39

41 Non-Extrapolated Results? Without the : N=M x error ratio / 39

42 Extrapolated Results? With the : N=M x error ratio / 39

43 Non-Extrapolated Graph? 22 / 39

44 Extrapolated Graph? 23 / 39

45 cont.? When applying to our PDE from earlier, the resulting equation can be expressed in matrix form as such: ( I + t ) ( 2h α Am+1 u m+1 = I t ) 2h α Am u m + t ( f m + f m+1) / 39

46 cont.? When applying to our PDE from earlier, the resulting equation can be expressed in matrix form as such: ( I + t ) ( 2h α Am+1 u m+1 = I t ) 2h α Am u m + t ( f m + f m+1). 2 A x = b 24 / 39

47 cont.? When applying to our PDE from earlier, the resulting equation can be expressed in matrix form as such: ( I + t ) ( 2h α Am+1 u m+1 = I t ) 2h α Am u m + t ( f m + f m+1). 2 A x = b What can we use to solve this? 24 / 39

48 cont.? When applying to our PDE from earlier, the resulting equation can be expressed in matrix form as such: ( I + t ) ( 2h α Am+1 u m+1 = I t ) 2h α Am u m + t ( f m + f m+1). 2 A x = b What can we use to solve this? Gaussian Elimination. 24 / 39

49 cont.? When applying to our PDE from earlier, the resulting equation can be expressed in matrix form as such: ( I + t ) ( 2h α Am+1 u m+1 = I t ) 2h α Am u m + t ( f m + f m+1). 2 A x = b What can we use to solve this? Gaussian Elimination. x = A 1 b 24 / 39

50 Gaussian Elimination In order to get A 1 from A, we must use Gaussian Elimination. An overview of this process:? 25 / 39

51 Gaussian Elimination? In order to get A 1 from A, we must use Gaussian Elimination. An overview of this process: A series of row operations I (identity matrix) simultaneously: I same series of row operations A 1 [9] 25 / 39

52 Gaussian Elimination? In order to get A 1 from A, we must use Gaussian Elimination. An overview of this process: A series of row operations I (identity matrix) simultaneously: I same series of row operations A 1 [9] Row operations Gaussian Elimination 25 / 39

53 Gaussian Elimination? In order to get A 1 from A, we must use Gaussian Elimination. An overview of this process: A series of row operations I (identity matrix) simultaneously: I same series of row operations A 1 [9] Row operations Gaussian Elimination How efficient is this process? 25 / 39

54 Gaussian Elimination requires O(N 3 ) time, demonstrated by counting the number of arithmetic operations to get the necessary numbers in each spot in the matrix. [9]? 26 / 39

55 ? Gaussian Elimination requires O(N 3 ) time, demonstrated by counting the number of arithmetic operations to get the necessary numbers in each spot in the matrix. [9] What does this mean? 26 / 39

56 ? Gaussian Elimination requires O(N 3 ) time, demonstrated by counting the number of arithmetic operations to get the necessary numbers in each spot in the matrix. [9] What does this mean? n we must do A 1 b. This process is O(N 2 ). Total computational cost: O(N 3 ). Why? Conclusion: This is VERY inefficient. Can we do better? 26 / 39

57 Room for Improvement We saw earlier how matrix inversion is expensive.? 27 / 39

58 Room for Improvement We saw earlier how matrix inversion is expensive. O(N 3 ) computation and O(N 2 ) memory? 27 / 39

59 Room for Improvement? We saw earlier how matrix inversion is expensive. O(N 3 ) computation and O(N 2 ) memory To replace this costly inversion process done via Gaussian Elimination, we will use an iterative method known as Conjugate Gradient Squared Method. the goal is to get the resultant vector at each time step with significantly less computation 27 / 39

60 What Is An Iterative Method? Initial guess, x 0, evaluate residual make some modification & generate approximation residual: b Ax i? 28 / 39

61 What Is An Iterative Method?? Initial guess, x 0, evaluate residual make some modification & generate approximation residual: b Ax i After every approximation, a residual is calculated and compared to the convergence criteria- also known as the tolerance. tolerance used in the test cases was / 39

62 What Is An Iterative Method?? Initial guess, x 0, evaluate residual make some modification & generate approximation residual: b Ax i After every approximation, a residual is calculated and compared to the convergence criteria- also known as the tolerance. tolerance used in the test cases was Error analysis is crucial in carrying out an iterative method. 28 / 39

63 What Is An Iterative Method?? Initial guess, x 0, evaluate residual make some modification & generate approximation residual: b Ax i After every approximation, a residual is calculated and compared to the convergence criteria- also known as the tolerance. tolerance used in the test cases was Error analysis is crucial in carrying out an iterative method. L 2 and L norms. 28 / 39

64 Trust the Process? iterative process [7]: x i : current solution x i = x i 1 + αr i 1 x i 1 : previous solution generated by r i 1 : the residual vector r (line in the direction of steepest descent) α: the factor that identifies how far down the line to go residual in this case is b Ax i 29 / 39

65 CG vs.? Conjugate Gradient (CG) Method: CG requires a symmetric matrix simpler algorithm, requires less calculation Conjugate Gradient Squared () Method: Does not require symmetry Requires more scalar computations, including 6 matrix-vector multiplications An extended version of CG 30 / 39

66 Benefit: method has a computational cost of O(N 2 ) per time step, as opposed to O(N 3 ).? Note: did you catch that the matrix A is still being used in the? Still O(N 2 ) memory. 31 / 39

67 Less is More, Or Just Better? Claim: A m+1 can be stored using only O(N) memory. Consider the following decomposition: ( ) ( A m+1 = d m+1 +,i A m+1 L + d m+1,i ) A m+1 R 32 / 39

68 Less is More, Or Just Better? Claim: A m+1 can be stored using only O(N) memory. Consider the following decomposition: ( ) ( A m+1 = d m+1 +,i A m+1 L + d m+1,i ) A m+1 R diffusion coefficients line the main diagonal of their own matrices 32 / 39

69 Less is More, Or Just Better? Claim: A m+1 can be stored using only O(N) memory. Consider the following decomposition: ( ) ( A m+1 = d m+1 +,i A m+1 L + d m+1,i ) A m+1 R diffusion coefficients line the main diagonal of their own matrices A m+1 L and A m+1 R are Toeplitz matrices made up of the Grünwald Weights 32 / 39

70 Less is More, Or Just Better? Claim: A m+1 can be stored using only O(N) memory. Consider the following decomposition: ( ) ( A m+1 = d m+1 +,i A m+1 L + d m+1,i ) A m+1 R diffusion coefficients line the main diagonal of their own matrices A m+1 L and A m+1 R are Toeplitz matrices made up of the Grünwald Weights Definition: Toeplitz matrices are matrices that are constant, left-to-right, along all diagonals. 32 / 39

71 ? Definition: Circulant matrices are matrices with the property that each row or column are rotations of another. Let T 3 = B 3 = then the Toeplitz matrix T n can be embedded into a Circulant matrix C n with the help of B n as follows: C 6 = / 39

72 ? A circulant matrix C 2N can be decomposed as C 2N = F 1 2N diag(f 2Nc)F 2N This decomposition also helps reduce the complexity of matrix-vector multiplications: 34 / 39

73 ? A circulant matrix C 2N can be decomposed as C 2N = F 1 2N diag(f 2Nc)F 2N This decomposition also helps reduce the complexity of matrix-vector multiplications: We want to do A m+1 u m+1, which means we will need C m+1 2N um+1 2N. 34 / 39

74 ? A circulant matrix C 2N can be decomposed as C 2N = F 1 2N diag(f 2Nc)F 2N This decomposition also helps reduce the complexity of matrix-vector multiplications: We want to do A m+1 u m+1, which means we will need C m+1 2N um+1 2N. F 2N u m+1 2N can be carried out in O(N log N) operations via FFT 34 / 39

75 ? A circulant matrix C 2N can be decomposed as C 2N = F 1 2N diag(f 2Nc)F 2N This decomposition also helps reduce the complexity of matrix-vector multiplications: We want to do A m+1 u m+1, which means we will need C m+1 2N um+1 2N. F 2N u m+1 2N can be carried out in O(N log N) operations via FFT C m+1 2N um+1 2N can be evaluated in O(N log N) operations 34 / 39

76 ? A circulant matrix C 2N can be decomposed as C 2N = F 1 2N diag(f 2Nc)F 2N This decomposition also helps reduce the complexity of matrix-vector multiplications: We want to do A m+1 u m+1, which means we will need C m+1 2N um+1 2N. F 2N u m+1 2N can be carried out in O(N log N) operations via FFT C m+1 2N A m+1 L,N um+1 2N um+1 N and A m+1 O(N log N) operations can be evaluated in O(N log N) operations R,N um+1 N can be evaluated in 34 / 39

77 ? A circulant matrix C 2N can be decomposed as C 2N = F 1 2N diag(f 2Nc)F 2N This decomposition also helps reduce the complexity of matrix-vector multiplications: We want to do A m+1 u m+1, which means we will need C m+1 2N um+1 2N. F 2N u m+1 2N can be carried out in O(N log N) operations via FFT C m+1 2N A m+1 um+1 2N um+1 can be evaluated in O(N log N) operations L,N N and A m+1 R,N um+1 N can be evaluated in O(N log N) operations A m+1 u m+1 can be evaluated in O(N log N) operations! 34 / 39

78 ? This process brings the computational cost of the from O(N 2 ) to O(NlogN) per time step. C 2N = F 1 2N diag(f 2Nc)F 2N Notice how all we need is a diagonal from each circulant matrix? O(N) memory achieved! Overview: Decompose A embed Toeplitz parts into Circulant matrices decompose Circulant matrices using FFT evaluate C L u and C R u 35 / 39

79 ? This process brings the computational cost of the from O(N 2 ) to O(NlogN) per time step. C 2N = F 1 2N diag(f 2Nc)F 2N Notice how all we need is a diagonal from each circulant matrix? O(N) memory achieved! Overview: Decompose A embed Toeplitz parts into Circulant matrices decompose Circulant matrices using FFT evaluate C L u and C R u Reduced memory and computational complexity! 35 / 39

80 Complexity Analysis-? Naive vs. : N=M Naive Time (s) Fast Time (s) Naive ratio Fast ratio DNF NaN seconds 8 hours seconds 14 minutes 36 / 39

81 What Did We Accomplish?? : Overall convergence increased from first to second-order in space Properties of Toeplitz & Circulant Matrices: Memory required reduced from O(N 2 ) to O(N) Iterative Method & FFT Decomposition: Time complexity improved from O(N 3 ) to O(NlogN) per time step. 37 / 39

82 Future Advances? Although this algorithm handles the one-dimensional case well (variables x and t), most problems in nature are two or three dimensional in space. so u = u(x, y, t) or u = (x, y, z, t) refore the Method,, and fast vector-multiplication technique all need to be applied to these extended differential equations. Additional Concern: Moving boundary problem 38 / 39

83 Acknowledgments? Special Thanks to: Occidental College Library Computers Professor Treena Basu 39 / 39

84 ? [1] Joseph Klafter and Igor M Sokolov. Anomalous diffusion spreads its wings. Physics World. physicsweb.org, August, [2] Donato Cafagna. : A Mathematical Tool from the Past for Present Engineers. IEEE Industrial Electronics Magazine, pp , [3] Treena S. Basu and Hong Wang. Second-Order Finite Difference -. International Journal of Numerical Analysis and Modeling, Volume 9, Number 3, pp , [4] Hong Wang and Treena S. Basu. Finite Difference Two-Dimensional -. SIAM J. Sci. Comput. Vol. 34, No. 5, pp. A2444-A / 39

85 ? [5] H. O. A. Wold and P. Whittle. A Model Explaining the Pareto Distribution of Wealth. Econometrica, Volume 25, Number 4, pp , Oct [6] G. E. Uhlenbeck and L. S. Ornstein. On the ory of the Brownian Motion. Physics Review, Volume 36, Number 5, pp , Sep [7] Jonathan R. Shewchul. An Introduction to the Conjugate Gradient Method Without the Agonizing Pain. Carnegie Mellon University, Pittsburgh, PA, [8] Gene H.Golub and Charles F.Van Loan. Matrix Computations. Johns Hopkins University Press, Third Edition, Oct / 39

86 ? [9] John H.Mathews and Kurtis D.Fink. Numerical Methods Using Matlab. Prentice Hall, Upper Saddle River, Third Edition, pp , NJ, [10] Mark M. Meerschaert, Hans-Peter Scheffler, and Charles Tadjeran. Finite difference methods for two-dimensional fractional dispersion equation. Journal of Computational Physics, 211, pp , [11] Charles Tadjeran and Mark M. Meerschaert. A second-order accurate numerical method for the two-dimensional fractional diffusion equation. Journal of Computational Physics, 220, pp , [12] Phillip J. Davis. Circulant Matrices. John Wiley & Sons, New York, / 39

87 ? [13] Robert M. Gray. Toeplitz and circulant matrices: A review. Foundations and Trends in Communications and Information ory, 2, pp , [14] Mark M. Meerschaert and Charles Tadjeran. Finite Difference Approximations for Two-Sided - Partial Differential Applied Numerical Mathematics, Volume 56, Issue 1, pp , Jan / 39