Risoluzione di equazioni matriciali con matrici infinite quasi-toeplitz

Transcription

1 Risoluzione di equazioni matriciali con matrici infinite quasi-toeplitz Leonardo Robol, ISTI-CNR, Pisa, Italy joint work with: D. A. Bini, B. Meini (UniPi) S. Massei (EPFL) Montecatini, 16 Feb / 26

2 GNCS Project / Participants Project: Metodi numerici avanzati per equazioni e funzioni di matrici con struttura, coordinated by B. Meini. Dario Bini Gianna Del Corso Dario Fasino Luca Gemignani Bruno Iannazzo Nicola Mastronardi Federico Poloni Valeria Simoncini Massimiliano Fasi Stefano Massei Davide Palitta Leonardo Robol 2 / 26

3 GNCS Project / Participants Project: Metodi numerici avanzati per equazioni e funzioni di matrici con struttura, coordinated by B. Meini. Dario Bini Gianna Del Corso Dario Fasino Luca Gemignani Bruno Iannazzo Nicola Mastronardi Federico Poloni Valeria Simoncini Massimiliano Fasi Stefano Massei Davide Palitta Leonardo Robol 2 / 26

4 Random walks on a quadrant We consider random walks on N N: Allowed moves only to adjacent states. Probabilities of going up/down/left/right are eventually independent of position (i, j). The problem can be stated also on the infinite strip {1,..., m} N. 3 / 26

5 The link with Toeplitz matrices Several queuing problems can be recasted in this framework. The simpler 1D case considers movements on a line: The probability transition matrix is as follows: ρ 0 + ρ 1 ρ 1 P = ρ 1 ρ 0 ρ ,.. where ρ 1 is the probability of moving left, ρ 1 right, and ρ 0 = 1 ρ 1 ρ 1 : a quasi-toeplitz semi-infinite matrix. 4 / 26

6 Toeplitz matrices Toeplitz matrices have constant diagonals. In our case, the entry (1, 1) is different: we have a top-left correction (quasi-toeplitz structure). In the 2D case, the matrix P is (quasi-)tbbt (Toeplitz-block block-toeplitz): Â 0 A 1 P = A 1 A 0 A ,.. with A i, Â 0 quasi-toeplitz (and semi-infinite). 5 / 26

7 Computing the invariant vector The steady state probability vector solving π T P = π T can be recovered by finding the minimal non-negative solutions to A 1 + A 0 G + A 1 G 2 = G, R 2 A 1 + RA 0 + A 1 = R Cyclic reduction (CR) is a matrix iteration that converges to the correct solutions. Needs to perform multiplications, sum, and inversions. Complexity O(m 3 ) if the matrices A i are m m for instance a random walk on {1,..., m} N. 6 / 26

8 A quick look at the matrix iteration A (h+1) 0 = A (h) 0 + A (h) 1 S (h) A (h) 1 + A (h) 1 S (h) A (h) 1 A (h+1) 1 = A (h) 1 S (h) A (h) 1 A (h+1) 1 = A (h) 1 S (h) A (h) 1 S (h) = (I A (h) 0 ) 1 The initial matrices are A (0) i structure preserved? := A i, so they are Toeplitz. Is the 7 / 26

9 A quick look at the matrix iteration A (h+1) 0 = A (h) 0 + A (h) 1 S (h) A (h) 1 + A (h) 1 S (h) A (h) 1 A (h+1) 1 = A (h) 1 S (h) A (h) 1 A (h+1) 1 = A (h) 1 S (h) A (h) 1 S (h) = (I A (h) 0 ) 1 The initial matrices are A (0) i structure preserved? := A i, so they are Toeplitz. Is the Experimentally, approximately, up to a top-left correction. Non-trivial to exploit in practice. In fact, typical implementation ignore the structure and require O(m 3 ) flops = m = is unsolved. 7 / 26

10 Basic facts about Toeplitz matrices a 0 a a 1 a 0 a 1 T (a(z)) =.. a , a(z) = a j z j j Z a(z) is in the Wiener algebra W if a j < j Z a(z) W 1 a (z) W j Z i a i <. 8 / 26

11 Hankel matrices Given a power series f (z) = j=0 f jz j, we have f 1 f 2 f 3... f 2 f 3 H(f (z)) =. f.. 3. the Hankel matrix defined by f (z). Often, we use the notation a + (z) = j 1 a j z j, a (z) = j 1 a j z j. 9 / 26

12 Toeplitz matrices are not an algebra The power series in W and W 1 form an algebra. However, T (a(z)) T (b(z)) T (c(z)), c(z) = a(z)b(z). The link between Toeplitz matrices and Laurent series is indeed an isomorphism for bi-infinite matrices. 10 / 26

13 Toeplitz matrices are not an algebra The power series in W and W 1 form an algebra. However, T (a(z)) T (b(z)) T (c(z)), c(z) = a(z)b(z). The link between Toeplitz matrices and Laurent series is indeed an isomorphism for bi-infinite matrices. Theorem (Gohberg Feldman) Let a(z), b(z) W. Then, T (a)t (b) = T (c) H(a )H(b + ), H(f ) Hankel matrix. The correction H(a )H(b + ) is a compact operator on l / 26

14 Toeplitz matrices are not an algebra The power series in W and W 1 form an algebra. However, T (a(z)) T (b(z)) T (c(z)), c(z) = a(z)b(z). The link between Toeplitz matrices and Laurent series is indeed an isomorphism for bi-infinite matrices. Theorem (Gohberg Feldman) Let a(z), b(z) W. Then, T (a)t (b) = T (c) H(a )H(b + ), H(f ) Hankel matrix. The correction H(a )H(b + ) is a compact operator on l 2. Toeplitz matrices are an algebra up to compact corrections. 10 / 26

15 The Hankel correction Since a(z), b(z) are in W, then a j, b j 0 for j, so: T (a), T (b) are numerically banded. H(a ), H(b + ) have the support in the top-left corner. Therefore, T (a) T (b) = T (c) H(a )H(b + ) is numerically: = + 11 / 26

16 A new class of matrices Consider the norm E F := i,j E ij, and F := {E R N N E F < }. 12 / 26

17 A new class of matrices Consider the norm E F := i,j E ij, and F := {E R N N E F < }. Definition A = T (a(z)) + E a is quasi-toeplitz (QT), if: a(z) W 1 W. E a satisfies E a F <. We define A QT := a(z) W + a (z) W + E a F. 12 / 26

18 A new class of matrices Consider the norm E F := i,j E ij, and F := {E R N N E F < }. Definition A = T (a(z)) + E a is quasi-toeplitz (QT), if: a(z) W 1 W. E a satisfies E a F <. We define A QT := a(z) W + a (z) W + E a F. QT-matrices form a (computationally-friendly) algebra. E a F < = E a compact operator on l / 26

19 Numerical representation of a QT matrix We can represent QT matrices with a finite number of parameters and arbitrary accuracy. If A = T (a) + E a : T (a) can be stored by a (truncated) approximation of its symbol, since i >j a i < ; E a UV T by means of SVD; U, V have compact support and σ j (E a ) 0, since E a is compact and so numerically low-rank. In our case we use E a F < to pick a top-left block as approximation to E a. 13 / 26

20 QT is an algebra We can check that: A, B QT = A + B QT : T (a) + E a + T (b) + E b = T (a + b) + (E a + E b ). 14 / 26

21 QT is an algebra We can check that: A, B QT = A + B QT : T (a) + E a + T (b) + E b = T (a + b) + (E a + E b ). A, B QT = AB QT : (T (a) + E a ) (T (b) + E b ) = T (ab)+ (T (a)e b + E a T (b) + E a E b H(a )H(b + )) 14 / 26

22 QT is an algebra We can check that: A, B QT = A + B QT : T (a) + E a + T (b) + E b = T (a + b) + (E a + E b ). A, B QT = AB QT : (T (a) + E a ) (T (b) + E b ) = T (ab)+ (T (a)e b + E a T (b) + E a E b H(a )H(b + )) A = T (a) + E a implies A 1 = T (a 1 ) +? 14 / 26

23 QT is an algebra (cont d) If a(z), b(z) W 1 and E a, E b F, then the corrections obtained by sum and multiplication are in F. 15 / 26

24 QT is an algebra (cont d) If a(z), b(z) W 1 and E a, E b F, then the corrections obtained by sum and multiplication are in F. Proof: some (not so interesting) computations. a(z) W 1 plays an important role. 15 / 26

25 QT is an algebra (cont d) If a(z), b(z) W 1 and E a, E b F, then the corrections obtained by sum and multiplication are in F. Proof: some (not so interesting) computations. a(z) W 1 plays an important role. And the inverse? 15 / 26

26 Computing the inverse For the inverse of T (a), we need a few steps: Factor a(z) = u(z)l(z 1 ), where u, l are power series (Wiener-Hopf factorization). This corresponds to the UL factorization T (a) = T (u(z)) T (l(z 1 )) Triangular Toeplitz matrices have Toeplitz inverses: T (a(z)) 1 = T (l 1 (z 1 ))T (u 1 (z)), and we can use what we know about multiplications. 16 / 26

27 Computational remarks We can operate on quasi-toeplitz matrices combining: Operations/Factorizations on power series (FFT-based); Toeplitz-vector multiplications (again FFT-based); Compression of low-rank matrices of the form H(a )H(b + ). For the last item, fast matvec is available, so we can run either Lanczos or random sampling to compress the products of Hankel matrices. 17 / 26

28 Computational remarks We can operate on quasi-toeplitz matrices combining: Operations/Factorizations on power series (FFT-based); Toeplitz-vector multiplications (again FFT-based); Compression of low-rank matrices of the form H(a )H(b + ). For the last item, fast matvec is available, so we can run either Lanczos or random sampling to compress the products of Hankel matrices. Ranks of the corrections are small in practice: is a typical value in applications. 17 / 26

29 Keeping the rank bounded When we perform arithmetic operations, the rank of the correction increases in the representation. This can be kept low by recompression. 18 / 26

30 Keeping the rank bounded When we perform arithmetic operations, the rank of the correction increases in the representation. This can be kept low by recompression. Assume E a = U a V T a, U a, V a with k columns. [Q U, R U ] = qr(u a ), and [Q V, R V ] = qr(v a ). 18 / 26

31 Keeping the rank bounded When we perform arithmetic operations, the rank of the correction increases in the representation. This can be kept low by recompression. Assume E a = U a V T a, U a, V a with k columns. [Q U, R U ] = qr(u a ), and [Q V, R V ] = qr(v a ). [U 1, Σ 1, V 1 ] svd(r U RV T ) (truncated SVD). 18 / 26

32 Keeping the rank bounded When we perform arithmetic operations, the rank of the correction increases in the representation. This can be kept low by recompression. Assume E a = U a V T a, U a, V a with k columns. [Q U, R U ] = qr(u a ), and [Q V, R V ] = qr(v a ). [U 1, Σ 1, V 1 ] svd(r U RV T ) (truncated SVD). E a Q U U 1 Σ1 (Q V V 1 Σ1 ). The new representation has k < k columns. 18 / 26

33 Keeping the rank bounded When we perform arithmetic operations, the rank of the correction increases in the representation. This can be kept low by recompression. Assume E a = U a V T a, U a, V a with k columns. [Q U, R U ] = qr(u a ), and [Q V, R V ] = qr(v a ). [U 1, Σ 1, V 1 ] svd(r U RV T ) (truncated SVD). E a Q U U 1 Σ1 (Q V V 1 Σ1 ). The new representation has k < k columns. The cost of the compression is O(nk 2 + k 3 ) flops, where n is the support of E a. 18 / 26

34 Keeping the rank low (pictorial version) = +

35 Keeping the rank low (pictorial version) = = + 19 / 26

36 A Banach algebra We can say even more about QT. Theorem The set of QT-matrices, endowed with the norm is a Banach algebra. T (a) + E a := a W + a W + E a F, This means that it is an algebra, it is a Banach space, and that AB A B. 20 / 26

37 Back to our matrix iteration Recall we wanted to use cyclic reduction to solve A 1 + A 0 G + A 1 G 2 = G, R 2 A 1 + RA 0 + A 1 = R Theorem Let A i B, a Banach algebra. If ϕ(z) = z 1 A 1 + (A 0 I ) + za 1 is invertible on {t 1 < z < t} for t > 1, then CR converges quadratically and G, R can be obtained from the limits of A (h) i. 21 / 26

38 Good news The hypotheses of the previous results are always satisfied in the Markov chain setting. QT-matrices are a Banach algebra so cyclic reduction works in this class, and we know how to compute the iteration. The semi-infinite solutions G, R, will also be represented in the QT format. We have a MATLAB toolbox that makes it easy to play with these matrices: You can google cqt-toolbox. 22 / 26

39 A simple example Consider a tandem Jackson queue. It has been shown that finite truncation may not approximate the infinite-dimensional case behavior 1. 1 Motyer, A.J. and Taylor, P.G., Decay rates for quasi-birth-and-death processes with countably many phases and tridiagonal block generators. Advances in applied probability, 38(2), pp / 26

40 Explicit solution We can compute the steady state vector π using CR: Problem Time (s) Residue Band Support Rank The residue is π T P π T, and each problem corresponds to different rates and parameters. Ranks and support refer to the corrections in G, R, and the band to their Toeplitz part / 26

41 Conclusions and future outlook Computing with infinite matrices might be fun, if you have the right structure. Some hypotheses on the norms can be relaxed this is work in progress. We can do much more fancy things: compute matrix functions, solve other kinds of matrix equations,.... This has applications to other fields, such as finance. Finite case handled as well. A MATLAB toolbox is available for you to try: feedback is very welcome! 25 / 26

42 Thank you for your attention! 26 / 26