arxiv: v1 [math.na] 9 Mar 2015

Transcription

1 APPROXIMATION OF FUNCTIONS OF LARGE MATRICES WITH KRONECKER STRUCTURE MICHELE BENZI AND VALERIA SIMONCINI arxiv:53.265v [ath.na] 9 Mar 25 Abstract. We consider the nuerical approxiation of f(a)b where b R N and A is the su of Kronecker products, that is A = M 2 I + I M R N N. Here f is a regular function such that f(a) is well defined. We derive a coputational strategy that significantly lowers the eory requireents and coputational efforts of the standard approxiations, with special ephasis on the exponential function, for which the new procedure becoes particularly advantageous. Our findings are illustrated by nuerical experients with typical functions used in applications. Key words. atrix functions, sparse atrices, Krylov ethods, Kronecker structure. Introduction. We consider the proble of approxiating f(a)b (.) where f is a sufficiently regular function defined on the spectru of A (see [2]), and A = M 2 I +I M R N N (.2) is the su of Kroneckerproducts with M R n n, M 2 R n2 n2 so that N = n n 2, b = vec(b) with B a atrix of low rank with diensions copatible with that of b. The Kronecker (or tensor) product of two atrices X and Y of size n x x and n y y, respectively, is defined as x Y x 2 Y x x Y x 2 Y x 22 Y x 2x Y X Y = ; Rnxny xy x nxy a nx2y x nx x Y the vec operator stacks the coluns of a atrix X = [x,...,x ] R n one after the other as vec(x) = x. x R n. The proble of approxiating (.) for general A is very iportant in several applications and has long attracted considerable attention; we refer the reader to [2] and to [3] for coprehensive treatents of the proble and for any ways of nuerically approxiating its solution. For the case when A has large diensions, new effective approaches have been devised, aking the use of atrix function evaluations an iportant tool for solving large scale (three-diensional) scientific and engineering probles involving discretized partial differential equations; see, e.g., [4, 22, 23]. In Departent of Matheatics and Coputer Science, Eory University, Atlanta, Georgia 3322, USA (benzi@athcs.eory.edu). The work of this author was supported by National Science Foundation grants DMS-5692 and DMS Dipartiento di Mateatica, Università di Bologna, Piazza di Porta S. Donato 5, I-427 Bologna, Italy (valeria.sioncini@unibo.it). The work of this author was partially supported by the FARB2SIMO grant of the Università di Bologna.

2 2 M. Benzi and V. Sioncini particular, the Kronecker structure above arises whenever the doain is a rectangle or a parallelepiped and finite difference or certain low-order finite eleent ethods are eployed to discretize differential equations with separable coefficients; see, e.g., [7, 9] and references therein. Other applications leading to atrices with Kronecker su structure include iage processing [9], queueing theory [3, Chapter 9], graph analysis [, Chapter 3.4], and network design [36]. A significant body of literature is now available on efficient nuerical ethods for approxiately evaluating the product of f(a) ties a vector b, using particular spectral properties of A and under certain regularity conditions on f. To the best of our knowledge, the coputational advantages of exploiting, for a general function f, the possible Kronecker structure of A have not been addressed in the context of Krylov subspace ethods for large-scale probles of the for (.). By taking into account this structure, and also the possible low rank of B, the coputational setting changes significantly. We will show that the eory requireents can be drastically reduced: in fact, we show that by preserving the structure of the proble, faster convergence and significantly lower eory requireents can be achieved. More precisely, we acknowledge that the approxiation to functions of A is the coposition of distinct approxiations in ters of M and M 2, which are uch saller atrices. Siilar considerations can be ade for other properties of functions of atrices that are Kronecker sus, as is the case for their sparsity and decay patterns; see [6] for a recent analysis. Our results strongly rely on the low rank of the atrix B. In fact, but without loss of generality, we shall assue that B has rank equal to one, so that we can write B = b b T 2, b R n, b 2 R n2. For larger rank l in{n,n 2 }, we could still write B = B B2 T and proceed in a siilar anner. Our results also apply when B is nuerically low rank, that is, only a few singular values of B are above achine precision, or soe other sall tolerance. In this case, we could write b = b+b ǫ where b = vec( B) with B of low rank, and bǫ. If f(a) is not too large, f(a)b = f(a) b+f(a)b ǫ f(a) b. An outline of the paper is as follows. In section 2 we review soe standard techniquesforapproxiating(.)whenaislarge,andsetupthenotationfortherest of the paper. In section 3 we derive the structure-exploiting approxiation for general functions f such that f(a) is well defined. In section 4 we focus on the exponential function, for which the new procedure becoes particularly advantageous. Another iportant special case, the atrix inverse, is briefly discussed in section 5, and ore general atrix functions in section 6. Conclusions are given in section 7. Our findings are illustrated by nuerical experients with typical functions used in applications. 2. General approxiation by projection. A coon procedure for large A constructs an approxiation space, and a atrix V whose orthonoral coluns span that space, and obtain f(a)b Vf(H)e, H = V T AV, e = V T b. (2.) Depending on the spectral properties of the atrix A and on the vector b, the approxiation space diension ay need to be very large to obtain a good approxiation. Unfortunately, the whole atrix V ay need to be stored, liiting the applicability of the approach. This is the otivation behind the recently introduced restarted ethods, which try to cope with the growing space diensions by restarting the

3 Functions of atrices with Kronecker structure 3 approxiation process as soon as a fixed axiu subspace diension is reached [, 3]. A classical choice as approxiation space is given by the (standard) Krylov subspace K (A,b) = span{b,ab,...,a b}. An orthonoral basis {v,...,v } can be constructed sequentially via the Arnoldi recurrence, which can be written in short as AV = V H +hv + e T, V = [v,...,v ]; here e is the th vector of the canonical basis of R, H = V TAV (as stated earlier), and h = Av i= [H ] i v i. The past few years have seen a rapid increase in the use of richer approxiation spaces than standard Krylov subspaces. More precisely, rational Krylov subspaces, naely K (A,b,σ ) = span{b,(a σ I) b,..., (A σ i I) b}, (2.2) have been shown to be particularly well suited for atrix function approxiations; we refer the reader to [6] for a recent survey on various issues related to rational Krylov subspace approxiations of atrix functions. A special case is given by the extended Krylov subspace, which alternates powers of A with powers of A [7, 26]. 3. Exploiting the Kronecker structure. Assue that A has the for in(.2) and that, for siplicity, B has rank one, that is B = b b T 2. We generate distinct approxiations for the atrices M and M 2 ; in the case of the classical Krylov subspace these are given as and i= K (M,b ), M Q = Q T +q + t () e T K (M 2,b 2 ), M 2 P = P T 2 +p + t (2) e T. Note that the two spaces could have different diensions; we will use the sae diension for siplicity of presentation. The atrices Q and P have orthonoral coluns, and have a uch saller nuber of rows than V (the square root of it, if n = n 2 ). We thus consider the following quantity to define the approxiation: so that V = P Q AV = A(P Q ) = M 2 P Q +P M Q = (P T 2 Q +P Q T )+p + t (2) e T Q +P q + t () e T = (P Q )(T 2 I +I T )+low rank. Following the general setting in (2.), and defining T = T 2 I +I T we thus consider the approxiation f(a)b x := (P Q )z, z = f(t )(P Q ) T b. (3.) In case b is a atrix, the definition of a block Krylov subspace is copletely analogous, that is K (A,b) = range([b,ab,...,a b]).

4 4 M. Benzi and V. Sioncini WestressthattheatrixP Q doesnotneedtobeexplicitlycoputedandstored. Indeed, letting Z R be such that z = vec(z), it holds that x = vec(q ZP T ); oreover, (P Q ) T b = vec((q T b )(b T 2 P )). The following proposition provides a cheaper coputation in case both T and T 2 are diagonalizable, as is the case for instance when they are both syetric. Proposition 3.. Assue that the atrices T,T 2 are diagonalizable, and let T = XΛX, T 2 = YΘY be their eigendecopositions. Let g = f(θ I +I Λ)vec(X Q T b b T 2 P Y T ) R 2. With the notation and assuptions above, for G such that g = vec(g) it holds that x = vec(q XGY T P T ). Proof. Using the properties of the Kronecker product (see, e.g., [24, Corollary 4.2. and Theore 4.4.5]), the eigendecoposition of T = T 2 I +I T is given by T 2 I +I T = (Y X)(Θ I +I Λ)(Y X ), so that f(t ) = (Y X)f(Θ I +I Λ)(Y X ), where f(θ I +I Λ) is a diagonal atrix. The result follows fro explicitly writing down the eigenvector atrices associated with each Kronecker product; note that f(θ I +I Λ) can be coputed cheaply as both Θ and Λ are diagonal. In addition to providing a coputational procedure for deterining x, Proposition 3. reveals that, in exact arithetic, the true vector x = f(a)b can be obtained using inforation fro spaces of diension at ost = ax{n,n 2 }, whereas the standard approxiation x ay require a uch larger diension space. This fact is due to both the Kroneckerfor of A and the coposition of b, as b corresponds to the vectorization of the rank-one atrix b b T 2. The following exaples illustrate this property, while ore explicit forulas can be obtained for the exponential function, as we will describe in section 4. Exaple 3.2. We consider f(x) = x and M = M 2 = tridiag(, 2,) R n n, n = 5, each corresponding to the (scaled) centered three-point discretization oftheone-diensionalnegativelaplaceoperatorin(,). Wefirstconsiderb equalto the vector of all ones, and b 2 a vector of rando values uniforly distributed in (,); the results are shown in Table 3.. We observe that convergence is faster, in ters of space diension, for x. Moreover, once subspaces of diension = n = 5 are reached, a rather accurate approxiation is obtained with the structure-preserving approach, as the full eigenspace of M is generated. We next consider the case of b 2 = b (the vector of all ones) and report the nuerical experients in Table 3.2. Convergence is even faster in the structure preserving ethod, as apparently convergence is faster with b than with the original b 2. No ajor difference is observed in the standard procedure. Exaple 3.3. Data for this exaple are taken fro [3]. We consider the function f(z) = (e s z )/z with s = 3, and A = M I + I M, where M is the n n tridiagonal atrix of the finite difference discretization of the onediensional Laplace operator; b := b = b 2 is the vector of all ones. Table 3.

5 Functions of atrices with Kronecker structure 5 f(a)b x f(a)b x 5.446e e e- 2.75e e e e e e e e e e e e e e e e e-3 Fig. 3.. Exaple 3.2 for f(x) = x. b 2 b. f(a)b x f(a)b x 5.937e+.593e e e e-.3538e e e e-2.79e e-2.34e e e e-3.86e e e e e-2 Fig Exaple 3.2 for f(x) = x. b 2 = b. x x,old f(a)b x f(a)b x x x,old x x e e-.e+.e e- 2.25e- 2.27e e e-.365e-.366e-.297e e e e e e-2.76e e e e e e-2.935e e e e e e e e-3.366e e e-2.754e e e e e e e e e e e e e e-3 Table 3. Exaple 3.3. For f(x) = (e s z )/z with s = 3. Here n = 5. shows the approxiation history of the standard ethod and of the new approach for n = 5. Because of the sall size, we could copute and onitor the true error. In the last two coluns, however, we also report the relative difference between the last two approxiation iterates, which ay be considered as a siple-inded stopping criterion for larger n; see, e.g., [26] or [2] for ore sophisticated criteria. The results are as those of the previous exaples. In Table 3.2 we report the runs for n =, for which we could not copute the exact solution, so that only the error estiates are reported. The results are very siilar to the saller case. In this case, eory requireents of the structured approxiation becoe significantly lower than for the standard approach. 4. The case of the atrix exponential. The evaluation of (.) with f(a) = exp(a) presents special interest owing to its iportance in the nuerical solution of tie-dependent ODEs and PDEs [2, 22, 23]. The proble also arises in network science, when evaluating the total counicability of a network [5, ]. The exponential function provides a particularly favorable setting in the case of a atrix having Kronecker for. Indeed, due to the property (see, e.g., [2, Theore

6 6 M. Benzi and V. Sioncini x x,old x x x,old x 4.e+.e e e- 2.5e-.6289e- 6.76e-.966e e e e e e e e e e e e e e e e e e e e-3.354e e e-3 Table 3.2 Exaple 3.3. For f(x) = (e s z )/z with s = 3 and b 2 = b. Here n =..9]) exp(t 2 I +I T ) = exp(t 2 ) exp(t ), (4.) forula (3.) siplifies even further. Indeed, we obtain x = (P Q )exp(t 2 I +I T )(P Q ) T b = (P Q )(exp(t 2 ) exp(t ))(P Q ) T b = vec(q exp(t )Q T b b T 2P exp(t 2 ) T P T ) = vec(x () (x(2) )T ), (4.2) with x () = Q exp(t )Q T b and x (2) = P exp(t 2 )(P T b 2). We observe that the final approxiation x is the siple cobination of the two separate approxiations of exp(m )b and exp(m 2 )b 2. Indeed, the sae approxiation could be obtained by first writing exp(a)b = exp(m 2 ) exp(m )b = vec((exp(m )b )(b T 2 exp(m 2 ) T ), (4.3) and then using the standard approxiations exp(m )b Q exp(t )Q T b and exp(m 2 )b 2 P exp(t 2 )P T b 2. In the following we illustrate the behavior of the approxiation to the atrix exponential with a few nuerical exaples. Here the standard Krylov subspace is used in all instances for approxiating the corresponding vector. We stress that becauseofthe decreasedeoryallocationstogeneratek (M i,b i ), the coputation of x can afford significantly larger values of, than when building K (A,b). Exaple 4.. We consider the approxiation of exp(a)b with A as in (.2) and M = tridiag(, 2,) and M 2 = tridiag(2, 3,2), both of size n = n 2 = 7. Therefore, A has diension 49. Moreover, we take b = [,...,] T and b 2 a vector with rando values uniforly distributed in (, ). Thanks to the proble size, the

7 Functions of atrices with Kronecker structure 7 2 X X X X 2 X X error nor x () x () x (2) x (2) error nor X X x () () x x (2) x (2) space diension space diension Fig. 4.. Convergence history to exp(a)b. Left: Exaple 4.. Right: Exaple 4.2. vector exp(a)b could be coputed explicitly. Figure 4.(left) reports the convergence history as the space diension increases when using two different approaches: the first one uses K (A,b) as approxiation space, so that x = V exp(h )(Vb) T with H = VAV T ; the second one uses x in (4.2). We observe that the convergence of x is faster than that of x ; this fact will be explored in section 4.. The plot also reports the error nor in the approxiation of x () and x (2) : the error nor for x is ainly driven by that of the ost slowly converging approxiation between x () and x (2). Exaple 4.2. We odify Exaple 4. by setting M 2 = tridiag(, 2,), while M to be equal to the discretization by finite differences of the one-diensional nonselfadjoint operator L(u) = u xx u x on the interval [,]. Hence, M (and therefore A) is nonsyetric. The atrix diensions and the vectors b,b 2 are as in the previous exaple. The convergence history is reported in the right plot of Figure 4.. Siilar coents as for Exaple 4. can be deduced. Exaple 4.3. The next exaple arises in graph and network analysis. Given two graphs G = (V,E ) and G 2 = (V 2,E 2 ), we consider the Cartesian product G = G G 2 of the two given graphs, defined as follows. The vertex set of G is just the Cartesian product V V 2, and there is an edge between two vertices (u,u 2 ) and (v,v 2 ) of G if either u = v and (u 2,v 2 ) E 2, or u 2 = v 2 and (u,v ) E. The adjacency atrix of G is then the Kronecker su of the adjacency atrices of G and G 2 [, page 37]; see also [36] for definitions (and applications) in the case of directed graphs. A useful notion in the analysis of coplex networks is the total counicability, which is defined as the rowsu ofthe exponentialof the adjacencyatrix, see [5]. The entries of this vector provide a easure of the iportance of the nodes in the network, and can be coputed as exp(a)b where now b is the vector of all ones (note that the corresponding atrix B has rank one). We consider five Cartesian product graphs of the for G i = G i G i, with each G i being a Barabasi Albert graph constructed using the preferential attachent odel. The coand pref in the Matlab toolbox contest [34] was used (with the default choice of paraeters) to generate five graphs on n nodes, where n =,2,...,5. Thus, the adjacency atrices of the corresponding Cartesian product graphs G i have diension ranging between

8 8 M. Benzi and V. Sioncini estiate of error nor X X X X estiate of error nor 5 A A2 A3 A4 A space diension space diension Fig Exaple 4.3. Convergence history to exp(a)b. Left: Exaple 4.3, case n =. Right: Exaple 4.3, convergence history for all five cases. one and twenty-five illions. All the resulting atrices are syetric indefinite. Table 4. reports the CPU tie required to copute a basis for the Krylov subspace of diension = 3 as the graph atrix size increases (all runs were perfored with Matlab R2b [29] on a laptop with Intel Core i7-3687u CPU running at 2.Ghz with 7.7GiB eory). The last colun reports the tie when A is used, so that V exp(h )e b is coputed; the iddle colun refers to the case when M is used, sothat x in (4.2) is coputed. As expected, the CPUtie for K (M,b ) is several orders of agnitude saller than for K (A,b). In the latter case, tiings becae prohibitive for n = 3,, since the generation of the basis for the space entails the orthogonalization of vectors in R n2. The coputational costs reain extreely low when coputing a basis for K (M,b ). The left plot of Figure 4.2 shows the convergence history of the two approaches, in ters of space diensions, when the sallest atrix in the set is used. Convergence is onitored by easuring the difference between the last two iterates, as done in the previous exaples. Once again, convergence is faster when the Kronecker for is exploited. The right plot of Figure 4.2 reports the convergence history of x for all atrices in the set. All spectra are roughly contained in the interval [ 5, 5], therefore the expected convergence rate is approxiately the sae for all atrices. n CPU Tie CPU Tie K (M,b ) K (A,b) Table 4. Exaple 4.3. CPU Tie for the construction of the Krylov approxiation space of diension = 3 when using either M R n n (syetric) or A = M I + I M R n2 n 2. Only results for the sallest graph atrices A are reported when building K (A,b).

9 Functions of atrices with Kronecker structure 9 Reark 4.4. Besides the exponential, the atrix sine and cosine are also wellbehaved with respect to the Kronecker su structure. Indeed, the following identities hold [2, Theore 2.2]: and sin(m M 2 ) = sin(m ) cos(m 2 )+cos(m ) sin(m 2 ) (4.4) cos(m M 2 ) = cos(m ) cos(m 2 ) sin(m ) sin(m 2 ). (4.5) These identities can be exploited to greatly reduce the coputational cost and storage requireents for the evaluation of f(a)b when f is either the sine or cosine or a cobination of these functions. 4.. Convergence considerations. An expression for the error can be deduced by using the for in (4.3). Indeed, letting x () = exp(m )b and x (2) = exp(m 2 )b 2, and also X = x () (x (2) ) T and X = x() (x (2) ) T, it holds exp(a)b x = X X F = x () (x (2) x (2) ) T +(x () x () )(x (2) ) T F x () x (2) x (2) + x() x () x(2). (4.6) Therefore, the error nor in the approxiation to exp(a)b is bounded by the errors of the two separate approxiations with M and M 2. The relation (4.6) can also be used for deriving a priori convergence bounds for exp(a)b in ters of the bounds for M and M 2. We will give such bounds in the case M and M 2 are Heritian and positive definite. The following result was proved in [2]. Theore 4.5. Let M be a Heritian positive seidefinite atrix with eigenvalues in the interval [,4ρ]. Then the error in the Arnoldi approxiation of exp(τm)v with v =, naely, ε := exp( τm)v V exp( τt )e, is bounded in the following ways: i) ε exp( 2 /(5ρτ)), for ρτ and 4ρτ τ; ii) ε (ρτ) exp( ρτ) ( ) eρτ for τ. We next show how Theore 4.5 can be used to copare the difference in convergence rates between x and x. To siplify the presentation, we assue that M = M = M 2, and that b = b 2, with b =. We refer the reader to [4, 25] for estiates siilar to those of Theore 4.5. It can be shown that if λ i, i =,...,n are the eigenvalues of M (in decreasing order), then the n 2 eigenvalues of A are given by λ i +λ j, i,j {,...,n}; see, e.g., [24, Theore 4.4.5]. Therefore in particular, the largest and sallest eigenvalues of A equal 2λ and 2λ n, respectively. If we apply Theore 4.5 to M λ n I for τ =, then we obtain that for large enough the error is bounded as ε (M) exp( ρ) ρ ( eρ ), λ λ n ρ =. 4 On the other hand, Theore 4.5 applied to A 2λ n I yields ε (A) exp( ρ) ρ ( ) e ρ, ρ = 2λ 2λ n =. 4

10 M. Benzi and V. Sioncini The ratio between the two bounds is given by exp ( ) ρ 2 2, which is in favor of the coputation with M for sall ρ. In case ρ is very large, both ethods becoe very slow. 5. The case of the atrix inverse. Also very iportant in applications is the case of the inverse, f(a) = A where A has Kronecker su structure. The solution of linear systes of the for Ax = b arises in any applications (PDEs, iaging, Markov chains, networks, etc.) and Krylov subspace ethods are widely used, typically together with preconditioning, to solve such systes. The Kronecker structure can be readily exploited if b is the result of the vectorization of a low rank atrix. For siplicity, let us assue that b = vec(b b T 2). Then the syste Ax = b is equivalent to the following Sylvester equation (see, e.g., [24, Sec. 4.4]): M X +XM T 2 = b b T 2, x = vec(x). (5.) Nuerical ethods that exploit the sall size of M,M 2 can be used to solve the linear atrix equation in (5.). If n and n 2 are of order up to a few thousands, then the Bartels Stewart algorith can be used [2]. Otherwise, X can be approxiated by using different approaches, depending on the relative size of n and n 2 ; we refer the reader to [32] for detailed discussion of the available ethods and the related references. Here we briefly describe the idea of approxiate solution by projection onto an appropriate subspace, which will be used in section 6.2. For siplicity of exposition we assue M 2 = M and b 2 = b ; if this is not the case, straightforward odifications can be included; see [32]. If the orthonoral coluns of P (= Q ) are a basis for the considered subspace of R n of diension, then an approxiation to X is sought as X X = P Y P, T where Y R is deterined by iposing additional conditions. A coon strategy consists of iposing that the residual R := M X + X M T b b T be orthogonal to the generated subspace, that is, (P P ) T vec(r ) =, or, in atrix ters, PR T P =, where a zero atrix appears on the right-hand side. Substituting in this last equation the definition of R and X, gives P T M P Y P T P +P T P Y P T M T P P T b b T P =. Recalling that P T P = I and that P T M P = T, we obtain the sall scale linear equation whose solution yields Y. In particular, T Y +Y T T b b =, b = P T b, x x = X P Y P T F, (5.2) where F denotesthe Frobeniusnor. While wereferto [32]foradetailedanalysis, here we notice that the approxiate solution x = vec(x ) to f(a)b = A b can be written in the ore failiar for x = vec(p Y P T ) = (P P )vec(y ) = (P P )(T I +I T ) (P P ) T b. (5.3) This for will be used in section 6.2 to express the approxiation error of Cauchy- Stieltjes functions.

11 Functions of atrices with Kronecker structure 6. Copletely onotonic functions. Both the atrix exponential (in the for f(a) = exp( A)) and the inverse are special cases of an iportant class of analytic functions, naely, the copletely onotonic functions [35]. We recall the following definitions. Definition 6.. Let f be defined in the interval (a,b) where a < b +. Then, f is said to be copletely onotonic in (a,b) if ( ) k f (k) (x) for all a < x < b and all k =,,2,... Moreover, f is said to be strictly copletely onotonic in (a,b) if ( ) k f (k) (x) > for all a < x < b and all k =,,2,... Here f (k) denotes the kth derivative of f, with f () f. An iportant theore of Bernstein states that a function f is copletely onotonic in (, ) if and only if f is the Laplace Stieltjes transfor of α(τ); f(x) = e τx dα(τ), (6.) where α(τ) is nondecreasing and the integral in (6.) converges for all x >. See [35, Chapter 4]. For this reason, copletely onotonic functions on (, ) are also referred to as Laplace Stieltjes functions. Iportant exaples of Laplace Stieltjes functions include:. f (x) = /x = e xτ dα (τ) for x >, where α (τ) = τ for τ. 2. f 2 (x) = e x = e xτ dα 2 (τ) for x >, where α 2 (τ) = for τ < and α 2 (τ) = for τ. 3. f 3 (x) = ( e x )/x = e xτ dα 3 (τ) for x >, where α 3 (τ) = τ for τ, and α 3 (τ) = for τ. Also, the functions x σ (for any σ > ), log(+/x) and exp(/x), are all strictly copletely onotonic on(, ). Moreover, products and positive linear cobinations of strictly copletely onotonic functions are strictly copletely onotonic. Forula (6.) suggests the use of quadrature rules to approxiate f(a)b when A is a Kronecker su and f is strictly copletely onotonic on (, ): f(a)b = exp( τa)bdα(τ) q w k exp( τ k A)b, (6.2) where τ,...,τ k (, ) are suitably chosen quadrature nodes and w,...,w k R are the quadrature weights; see, for exaple, [7, Sec. 5]. As shown in the previous section, the Kronecker su structure can be exploited in the coputation of the individual ters exp( τ k A)b. Also note that each contribution to the quadrature can be coputed independently of the others, which could be useful in a parallel setting. This approach could be especially useful in cases where convergence of the Krylov subspace approxiation is slow. In the case of interpolatory quadrature and Heritian atrices, the absolute error f(a)b q k= w kexp( τ k A)b (in the 2-nor) is easily seen to be bounded by ε b, where ε is defined as ε = ax i,j exp( τ(λ i +µ j ))dα(τ) k= q w k exp( τ k (λ i +µ j )) k=

12 2 M. Benzi and V. Sioncini and λ i,µ j range over the spectra of M, M 2, where A = M 2 I + I M. In the case M = M 2 = M = M, the error can be bounded by q ε ax exp( τ(λ+µ))dα(τ) w k exp( τ k (λ+µ)) λ,µ [λ in,λ ax], where λ in,λ ax are the extree eigenvalues of M. For additional discussion of error bounds associated with the use of quadrature rules of the for (6.2), see [7, Sec. 5.7]. Analogous considerations apply to ore general types of functions. For a function f analytic inside a contour Γ C containing the eigenvalues of A in its interior and continuous on Γ we can write f(a) = 2πi Γ k= f(z)(a zi) dz. Quadrature rules can be used to obtain approxiations of the for f(a)b = f(z)(a zi) bdz 2πi Γ q w k (A z k I) b, requiring the solution of the q linear systes (A z k I)x = b, possibly in parallel for k =,...,q. We refer to [8] for details on how to apply this technique efficiently. Again, the Kronecker su structure of A, if present, can be exploited to greatly reduce the coputational cost and storagerequireents. In particular, if b = vec(b b T 2 ), then according to section 5, each syste (A z k I)x = b is equivalent to solving the linear atrix equation (M z k I)X +XM2 T = b b T 2, with x = vec(x). Another iportant class of functions is given by the Cauchy Stieltjes (or Markovtype) functions, which can be written as f(z) = Γ k= dγ(ω) z ω, z C\Γ, where γ is a (coplex) easure supported on a closed set Γ C and the integral is absolutely convergent. This class is closely related to, but distinct fro, the class of Laplace Stieltjes functions; see [35, Chapter VIII] for a general treatent. In this paper we are especially interested in the particular case Γ = (,] so that f(x) = dγ(ω), x C\(,], (6.3) x ω where γ is now a (possibly signed) real easure. Iportant exaples of Cauchy Stieltjes function that are frequently encountered in applications (see [6]) include z 2 = e t z z log(+z) z z ω π ω dω, = = z ω z ω sin(t ω) dω, πω ( ω) dω.

13 Functions of atrices with Kronecker structure Convergence analysis for Laplace Stieltjes functions. For Laplace Stieltjes functions and syetric positive definite atrices, in this section we analyze the convergence rate of the approxiation obtained by exploiting the Kronecker for. Moreover, we copare this rate with that of the standard approxiation with A, and that of the approxiation of M. We will ainly deal with standard Krylov approxiations, as error estiates for the exponential are available. A few coents are also included for rational Krylov subspaces Analysis of Krylov subspace approxiation. In this section we show that the convergence rate of the approxiation when using x is saller than that with x = V f(h )Vb. T Moreover, it is also saller than the convergence rate of the approxiation x () to f(m )b. For siplicity of exposition we assue that M = M 2 and b = b 2, with b =. Proposition 6.2. Let f be a Laplace Stieltjes function, and x = (P P )f(t )(P P ) T b be the Kronecker approxiation to f(a)b. Moreover, let x () = exp( τm )b and x () = P exp( τt )P Tb. We also define the scaled quantity x () := e λinτ x () = e (M+λinI)τ b ; analogously for x (). Then f(a)b x 2 x () x () dα(τ). Proof. Recalling the notation of (4.2) and (4.3) leading to (4.6), we have f(a)b x = = 2 = 2 (e τa b (P P )e τt (P P ) T b)dα(τ) ( x () + x () ) x() x () dα(τ) 2e λinτ x () x () dα(τ) e τ(m+λini) b P e τ(t+λini) P T b dα(τ) x () x () dα(τ), where in the last inequality we have used λ in (M ) λ in (T ), so that x () e λin(t)τ e λin(m)τ. We reark that the extra shift in the atrix M is what akes the solution x converge faster than x (). In light of Proposition 6.2, bounds for the error nor can be found by estiating the error nor in the approxiation of the exponential function under the easure dα. Depending on the function α(τ), different approxiation strategies need to be devised. Here we analyze the case dα(τ) = τ dτ for soe γ ; for instance, the γ function f(x) = / x falls in this setting with γ = 3/2. Then f(a)v x 2 τ γ x() x () dτ.

14 4 M. Benzi and V. Sioncini The case γ = is special. Since τ γ =, the integral on the right-hand side can be bounded as in [33, proof of (3.)], so that it holds ) κ+ ( κ f(a)v x 2, λ in κ κ+ where κ = (λ ax +λ in )/(λ in +λ in ). We focus next on the case γ >. We split the integral as 2 τ γ x() x () 4ρ dτ = τ γ x() x () dτ + 2 τ γ x() x () dτ 4ρ =: I +I 2. (6.4) Lea 6.3. With the previous notation, for γ > it holds ( ) ) γ 4ρ κ+ ( κ I 2 2, λ in κ κ+ where κ = (λ ax +λ in )/(λ in +λ in ). Proof. We note that in the given interval τ γ I 2 ( ) γ 4ρ 2 2 4ρ ( 2 4ρ ) γ, so that ( ) γ 4ρ x () x () dτ 2 x () x () dτ. Following [33, Prop.3.], an explicit bound for the last integral can be obtained, fro which the bound follows. The derivation of an upper bound for I in (6.4) is a little ore involved. Lea 6.4. With the previous notation, for γ > it holds ( ( ( ) eρ γ I γ( γ, ρ ) 2λ ) ( ) in +ρ γ ( 2 π + 2λ in +ρ 2λ in ) 2λ 2 2 in e 5ρ where γ is the lower incoplete Gaa function. Proof. We observe that the quantity x () x () in I can be bounded by using Theore 4.5 (see [2]), hence we further split I as I = τ γ x() x () dτ + 2 4ρ τ γ x() x () dτ. (6.5) Theore 4.5 can be applied to positive seidefinite atrices. Therefore we write e (M+λinI)τ = e 2λinτ e (M λini)τ, with M λ in I positive seidefinite. For the first integral in (6.5) we thus have (see Theore 4.5(ii)) τ γ x() x () dτ = e 2λinτ τ γ e (2λin+ρ)τ e (M λini)τ b (P P )e (T λini)τ b dτ ρτ γ+ = ( eρ ρ ) = ( ( eρ ρ ) 2λ in +ρ ( eρτ ) dτ τ γ e (2λin+ρ)τ dτ ) γ γ( γ, 2λ ) in +ρ.,

15 Functions of atrices with Kronecker structure 5 For the second integral in (6.5), after the sae spectral transforation and also using τ γ (/()) for τ [ γ, 2 4ρ ], we obtain 2 4ρ τ γ x() x () 2 dτ 4ρ e 2λinτ τ γ ( ) γ /2 2 4ρ ( ) γ /2 e 2 5ρτ dτ τ 2 τ 2 2 2λinτ e 5ρτ dτ 2 2λinτ e 5ρτ dτ. We then use [5, forula ], naely x 2e βx β 2 x dx = π β e 2 β β 2, to finally write ( ) γ /2 τ 2 ( ) γ /2 2 2λinτ e 5ρτ π dτ = e 2 2λ in 2 5ρ. 2λ in By collecting all bounds we can prove a final upper bound for the error, and give its asyptotic convergence rate. To this end, we first need the following technical lea, whose proof is given in the appendix. Lea 6.5. For < x αn with < α < it holds that γ(n,x) x n α n e x. Theore 6.6. For γ > and with the notation above, it holds that f(a)v x 2(I +I 2 ) ( ( eρ ) ( 2 ρ ( + ( ) γ 4ρ κ+ +2 = O 2λ in +ρ ) γ /2 π 2λ in e 2 2 λ in κ ( ( exp κ 2 )) ) γ γ( ) 2λin 5ρ ) ( κ κ+ γ, 2λ in +ρ for and κ large enough. (6.6) ) Proof. We ( only ( need )) to show that the ter involving γ is asyptotically bounded abovebyo exp κ 2 forand κlarge. Siple calculationsshowthatthe second arguent of γ satisfies (2λ in + ρ)/() α with α < for λ ax /λ in > 9; the larger this eigenvalue ratio, the saller α, so that for a large ratio, the bound (2λ in + ρ)/() α( γ) also holds. Hence, we can use Lea 6.5 to write

16 6 M. Benzi and V. Sioncini γ(n,x) e xxn n. Therefore2, ρ ( ( ) eρ γ γ( γ, ) 2λ ) in +ρ 2λ in +ρ ( ( ) eρ γ ( ) γ 2λin +ρ ρ ) 2λ in +ρ γ e 2 γ ρ γ ( e ) e = 2λ in +ρ γ 2 γ ρ γ = ( γ) 2 γ ( γ) e 2λ in +ρ 2λ in +(ln4 )ρ. Writing down ρ = 4 (λ ax λ in ), and after a few algebraic calculations we obtain 2λ in +(ln4 )ρ = (ln4 )λ ax +(8 (ln4 ))λ in = ( ) 4 2 λ ax λ in 2 κ +(ln4 ) κ 2, where the last inequality holds for κ large enough (naely for κ ). The theore above states that the convergence rate of the approxiation depends on the condition nuber of the shifted atrix. Reark 6.7. If f(a)b is approxiated in the Krylov subspace K (A,b), then the error nor can be written as f(a)b x e τa b V e τh V T b dα. Therefore, all the previous steps can be replicated, leading to an estiate of the type ( f(a)b x exp 2 ) κ where now κ = λ ax /λ in. The iproveent in the convergence rate when exploiting the Kronecker for thus becoes readily apparent, with the shift acting as an accelerator. It is also iportant to realize that the error nor f(m )b x () is also driven by the sae quantity κ, since the condition nuber of M and A is the sae. Therefore, it is really by using the Kronecker for that convergence becoes faster. We next illustrate our findings with a siple exaple. A diagonal atrix is considered so as to be able to copute exact quantities, while capturing the linear convergence of the approxiation. Exaple. We consider f(x) = / x (so that γ = 3/2) and a diagonal atrix M of size n = 5 with logarithically distributed eigenvalues in [, 3 ], giving ρ 247; b is the vector of all ones, noralized to have unit nor. We wish to approxiate f(a)b, with b = vec(b b T ). We copare the convergence curves of the usual approxiation x = V f(h )e (solid thin line), with that of x = (Q Q )f(t ) b (dashed thick line). As expected, the convergence rate of the latter is better (saller) than for the standard ethod. The estiate in (6.6) is reported as thick crosses, and it well approxiates the correct convergence slope. For copleteness we also report the error nor f(m )b Q f(t )Q T b, which behaves like that of the standard ethod, as noticed in the previous reark. 2 We assue here that γ is a positive integer, otherwise we can take the Gaa function associated with the closest integer larger than γ.

17 Functions of atrices with Kronecker structure 7 error nor and estiate f(a)b x f(a)b x () f(m )b x Estiate space diension Fig. 6.. Convergence curves for various approxiation ethods, and estiate in (6.6) Rational Krylov subspace approxiation. Convergence bounds of Laplace-Stieltjes functions are harder to obtain when rational Krylov subspaces are used, for few results on error bounds for the exponential functions are available. In thissectionwediscusssoeofthe resultsthatcanbeobtained. However,wewillshow that additional results can be derived for the subclass of Cauchy-Stieltjes functions, since they involve inverses in place of exponential functions. If the approxiation space is the extended Krylov subspace, which can be defined as K (M,b )+K (M,M b ), error bounds are difficult to obtain for Laplace Stieltjes functions, unless dα =dτ (that is, γ = ). Indeed, for γ =, we have ( ( ) f(a)b x = vec exp( τm )b b T exp( τm ) P exp( τt ) b bt exp( τt )P T dτ The integral coincides with the error atrix in the approxiation of the nuerical solution to the Lyapunov equation in the space Range(P ). This connection was already used in Lea 6.3 to establish an upper bound for the Krylov subspace approxiation. Here, we just ention that an asyptotic bound can be obtained by using results in [27, 28, 3], giving (( 4 ) ) κ f(a)b x O 4, κ+ where κ = λ ax (A)/λ in (A) = λ ax (M )/λ in (M ). Note that while there is no beneficial shifted atrix in this bound, the fourth root of κ appears, ensuring significantly faster convergence rate than for standard Krylov subspaces. For γ >, lack of explicit error bounds for the exponential function leaves the derivation of bounds for our setting an open proble. Nonetheless, experiental evidence sees to suggest a convergence rate siilar to the one for γ =. Upper bounds for the exponential function when rational Krylov subspaces (2.2) are eployed are usually asyptotic and specialized to optial values of the paraeters σ,...,σ. We refer the reader to [6, sec.4.2] for ore details Convergence analysis for Cauchy Stieltjes functions. Functions belonging to the Cauchy Stieltjes class provide a ore favorable setting, as they are ).

18 8 M. Benzi and V. Sioncini based on the resolvent function. Assue then that f is a Cauchy-Stieltjes function. Using the definition in (6.3) and assuing that b = vec(b b T ), we can write f(a)b x = f(a)b (P P )f(t )(P P ) T b = = (A ωi) bdγ(ω) (P P ) (T ωi) (P P ) T bdγ(ω) ( (A ωi) b (P P )(T ωi) (P P ) T b ) dγ(ω). Let M(ω) = M ω 2I, so that for any v and V such that v = vec(v) we can write (T ωi)v = vec(m(ω)v + VM(ω) T ). Then, recalling the derivation in (5.3), we obtain f(a)b x = vec(x(ω) X (ω))dγ(ω), where X(ω) and X (ω) are the exact and approxiate solutions to the linear atrix equation M(ω)X +XM(ω) T = b b T when the space K (M ω 2 I,b ) is used. Note that this space is invariant under shift, that is K (M,b ) = K (M ωi,b ), so that the coluns of P are still a basis for this space. An upper bound for the error can then be obtained as f(a)b x X(ω) X (ω) F dγ(ω). Available convergence bounds for the approxiation of X(ω) onto standard and rational Krylov subspaces can be eployed as a first step towards an upper bound for the error nor above. Final bounds will then be obtained for specific choices of f, which yield specific functions γ(ω). For standard Krylov subspaces, we can once again use [33, Proposition 3.] to get κω f(a)b x + ( κω 2 (λ in 2 ω) κ dγ(ω), ω κω +) where κ ω = (λ ax +λ in 2 ω)/(λ in+λ in 2ω). For consistency with the previous notation, we shall use κ = κ. Therefore, where f(a)b x 2 ( κ κ+ ) κω + (λ in 2 ω) κ ω dγ(ω), (6.7) ( κ κ+ ) exp( 2/ κ) for κ large. The final upper bound can be obtained once the easure dγ(ω) is ade ore explicit, and this ay influence the integration interval as well. For instance, for f(x) = x 2, κω + κω + (λ in 2 ω) κ dγ(ω) = ω (λ in 2 ω) κ dω ω ω 2 = 2 (λ in 2 ω) ω dω (λ in 2 ω) ω dω +2 dω +2 ( ω) ω λ in (λ in 2 ω) dω ω ω dω = 2+ 4 λ in.

19 Functions of atrices with Kronecker structure 9 Since the inverse square root is both a Laplace-Stieltjes function and a Cauchy- Stieltjes function, it is not surprising that we get a siilar convergence rate. The setting of this section allows one to deterine a sipler expression for the bound. Following siilar steps as for the inverse square root, for f(x) = ln(+x)/x we have (note the change in the integration interval) κω + (λ in 2 ω) κ dγ(ω) = ω 2 κω + (λ in 2 ω) κ ω ω dω (λ in 2 ω) dω 2 2 = 4. ( ω) In both cases, the integral appearing in (6.7) is bounded by a constant of odest size, unless λ in is tiny in the inverse square root. Suarizing, when using standard Krylov subspace ( approxiation, ) f(a)b x is bounded by a quantity whose asyptotic ter is κ κ+ as grows. When using the extended Krylov ( subspace for an Heritian positive definite 4 ) M, this quantity is replaced by κ, 4 κ+ where is the subspace diension and κ = λ ax /λ in (see, e.g., [26],[28]), as conjectured in the case of Laplace-Stieltjes functions. Here, an explicit upper bound can actually be obtained. Rational Krylov subspaces can also be used, and the ter X X F can be estiated using, e.g., [8, Theore 4.9]. 7. Conclusions. In this paper we have shown how to take advantage of the Kronecker su structure in A when using Krylov subspace ethods to evaluate expressions of the for f(a)b. Special attention has been devoted to the iportant case of the atrix exponential. Nuerical experients deonstrate that considerable savings can be obtained when the Kronecker su structure is exploited. A detailed analysis of the convergence rate of the new approxiation for syetric (or Heritian) and positive definite atrices was also proposed. Finally, while we have liited our presentation to the case where A is the Kronecker su of two atrices, the sae observations and techniques apply to the ore general case where A is the Kroneckersu of three or ore suands, since this can be reduced to the Kronecker su of two atrices. For instance, if we can write A = M M 2 M 3 := M I I +I M 2 I +I I M 3, A = M (I I)+I (M 2 I +I M 3 ) =: M I +I M, and apply the techniques in this paper in a recursive fashion. Acknowledgeent. The authors would like to thank Paola Boito for her careful reading of the anuscript and helpful coents. Appendix. In this appendix we prove Lea 6.5. Lea 6.5. For < x αn with < α < it holds that γ(n,x) x n α n e x.

20 2 M. Benzi and V. Sioncini Proof. We have n γ(n,x) = (n )!e (e x x k= ) ( x k = (n )!e x k! k=n = (n )!e xxn x + j n! (n+) (n+j) j= e xxn + α j n j n (n+) (n+j) j= e xxn α j = e xxn n n α. j= ) x k k! REFERENCES [] R. B. Bapat, Graphs and Matrices, Universitext, Springer, London; Hindustan Book Agency, New Dehli, 2. [2] R. H. Bartels and G. W. Stewart, Algorith 432: Solution of the Matrix Equation AX + XB = C, Co. ACM, 5(9) (972), pp [3] B. Beckerann, An error analysis for rational Galerkin projection applied to the Sylvester equation, SIAM J. Nuer. Anal., 49(6) (2), pp [4] B. Beckerann and L. Reichel, Error estiation and evaluation of atrix functions via the Faber transfor, SIAM J. Nuer. Anal., 47(5) (29), pp [5] M. Benzi and C. Klyko, Total counicability as a centrality easure, J. Coplex Networks, (2) (23), pp [6] M. Benzi and V. Sioncini, Decay bounds for functions of atrices with banded or Kronecker structure, arxiv: (25), pp. 2. [7] V. Druskin and L. Knizhneran, Extended Krylov subspaces: approxiation of the atrix square root and related functions, SIAM J. Matrix Anal. Appl., 9 (998), pp [8] V. Druskin, L. Knizhneran, and V. Sioncini, Analysis of the rational Krylov subspace and ADI ethods for solving the Lyapunov equation, SIAM J. Nuer. Anal., 49 (2), pp [9] V. Druskin, L. Knizhneran, and M. Zaslavsky, Solution of large scale evolutionary probles using rational Krylov subspaces with optiized shifts, SIAM J. Sci. Coput., 3 (29), pp [] M. Eierann and O. Ernst, A restarted Krylov subspace ethod for the evaluation of atrix functions, SIAM J. Nuer. Anal., 44 (26), pp [] E. Estrada, N. Hatano,and M. Benzi, The physics of counicability in coplex networks, Phys. Rep., 54 (22), pp [2] A. Froer, S. Güttel, and M. Schweitzer, Convergence of restarted Krylov subspace ethods for Stieltjes functions of atrices, SIAM J. Matrix Anal. Appl, 35 (24). pp [3], Efficient and stable Arnoldi restarts for atrix functions based on quadrature, SIAM J. Matrix Anal. Appl., 35 (24), pp [4] A. Froer and V. Sioncini, Matrix functions, in Model Order Reduction: Theory, Research Aspects and Applications, W. H. A. Schilders, H. A. van der Vorst, and J. Roes, eds., Matheatics in Industry, Springer, Heidelberg, 28. [5] I. S. Gradshteyn and I. M. Ryzhik, Table of Integrals, Series, and Products, Seventh Edition, Acadeic Press, New York, 27. [6] S. Güttel, Rational Krylov approxiation of atrix functions: Nuerical ethods and optial pole selection, GAMM Mitt., 36 (23), pp [7] W. Hackbusch, Nuerical tensor calculus, Acta Nuerica 24, pp [8] N. Hale, N. J. Higha, and L. N. Trefethen, Coputing A α, log(a), and related atrix functions by contour integrals, SIAM J. Nuer. Anal., 46 (28), pp [9] P. C. Hansen, J. G. Nagy, and D. P. O Leary, Deblurring Iages. Matrices, Spectra, and Filtering, Society for Industrial and Applied Matheatics, Philadelphia, 26.

21 Functions of atrices with Kronecker structure 2 [2] N. J. Higha, Matrix Functions Theory and Applications, SIAM, Philadelphia, USA, 28. [2] M. Hochbruck and C. Lubich, On Krylov subspace approxiations to the atrix exponential operator, SIAM J. Nuer. Anal., 34 (997), pp [22] M. Hochbruck and C. Lubich, Exponential integrators for quantu-classical olecular dynaics, BIT, Nuerical Matheatics, 39 (999), pp [23] M. Hochbruck and A. Osterann, Exponential integrators, Acta Nuerica, 9 (2), pp [24] R. A. Horn and C. R. Johnson, Topics in Matrix Analysis, Cabridge University Press, Cabridge, 99. [25] L. Knizhneran, Calculus of functions of unsyetric atrices using Arnoldi s ethod, Coput. Math. Math. Phys., 3 (99), pp. 9. [26] L. Knizhneran and V. Sioncini, A new investigation of the extended Krylov subspace ethod for atrix function evaluations, Nuer. Linear Algebra Appl., 7 (2), pp [27] L. Knizhneran and V. Sioncini, Convergence analysis of the Extended Krylov Subspace Method for the Lyapunov equation, Nuer. Math., 8(3) (2), pp [28] D. Kressner and C. Tobler, Krylov subspace ethods for linear systes with tensor product structure, SIAM J. Matrix Anal. Appl., 3(4) (2), pp [29] The MathWorks, Inc. MATLAB 7, Septeber 24. [3] C. Moler and C. Van Loan, Nineteen dubious ways to copute the exponential of a atrix, twenty-five years later, SIAM Rev., 45 (23), pp [3] M. K. Ng, Iterative Methods for Toeplitz Systes, Oxford University Press, Oxford, 24. [32] V. Sioncini, Coputational Methods for Linear Matrix Equations, Technical report, Ala Mater Studioru - Università di Bologna, 23. [33] V. Sioncini and V. Druskin, Convergence analysis of projection ethods for the nuerical solution of large Lyapunov equations, SIAM J. Nuer. Anal., 47(2) (29), pp [34] A. Taylor and D. J. Higha, CONTEST: A Controllable Test Matrix Toolbox for MATLAB, ACM Trans. Math. Software, 35 (29), pp. 26: 26:7. [35] D. V. Widder, The Laplace Transfor, Princeton University Press, 946. [36] C. Yang and J.-M. Xu, Reliability of interconnection networks odeled by Cartesian product digraphs, Networks, 52 (28), pp