Let ^A and ^Z be a s.p.d. matrix and a shift matrix of order

FAS IMPLEMENAION OF HE QR FACORIZAION IN SUBSPACE IDENIFICAION N. Mastronardi, P. Van Dooren, S. Van Huffel Università della Basilicata, Potenza, Italy, Katholieke Universiteit Leuven, Leuven, Belgium Catholic University of Louvain, Louvain-la-Neuve, Belgium Katholieke Universiteit Leuven, Leuven, Belgium wo recent approaches [, ] in subspace identification problems require the computation of the R factor of the QR factorization of a block Hankel matrix H, which, in general has a huge number of rows. Since the data are perturbed by noise, the involved matrix H is, in general, full rank. It is well known that, from a theoretical point of view, the R factor of the QR factorization of H is equivalent to the Cholesky factor of the correlation matrix H H, apart from a multiplication by a sign matrix. In [] a fast Cholesky factorization of the correlation matrix, exploiting the block Hankel structure of H, is described. In this paper we consider a fast algorithm to compute the R factor based on the generalized Schur algorithm. he proposed algorithm allows to handle the rank deficient case. Keywords: identification methods least squares solutions multivariable systems singular values. INRODUCION Subspace based system identification techniques have been studied intensively in the last decades []. he major drawbacks of these direct state space identification techniques are the high computation and storage costs. Let u k and y k be the m dimensional input vector and the l dimensional output vector, respectively, of the basic linear time invariant state space model x k+ Ax k + Bu k + w k y k Cx k + Du k + v k where x k is the n dimensional state vector at time k fw k g and fv k g are state and output disturbances or noise sequences, and A B C and D are unknown real matrices of appropriate dimensions. For non sequential data processing, the N (m + l)s N fl (m + l)s matrix H [UsN Y sn ] is constructed, where U sn and Y sn are block Hankel matrices defined in terms of the input and output data, respectively, U sn Y sn u u u ::: u N u u u ::: u N+ u u u :: u N+..... u s u s+ u s+ ::: u N+s y y y ::: y N y y y ::: y N+ y y y :: y N+..... y s y s+ y s+ ::: y N+s 5 hen the R factor of a QR factorization H QR is used for data compression. In [] a fast Cholesky factorization of the correlation matrix, exploiting the block Hankel structure of H, is described. In this paper we consider a fast algorithm to compute the R factor of the QR factorization of H based on the generalized Schur algorithm, exploiting its displacement structure. he implementation of the algorithm depends, in particular, on the involved matrix. he paper is organized as follows. First the generalized Schur algorithm to compute the Cholesky factor of a symmetric positive definite matrix is described. hen the generalized Schur algorithm applied to the matrix H is considered. he rank deficient case is also discussed. Finally, some numerical experiments are reported. HE GENERALIZED SCHUR ALGORIHM FOR SYMMERIC POSIIVE DEFINIE MARICES In this section the key properties of the generalized Schur algorithm to compute the Cholesky factor of a symmetric positive definite (s.p.d.) matrix, which will be used in the next section, are summarized. More details can be found in [, 8]. Let ^A and ^Z be a s.p.d. matrix and a shift matrix of order

^N respectively. Let ^Z 0 0.............. 0 0 r ^A ^A ^Z ^A ^Z : 5 hen rank(r ^A) is called the displacement rank of ^A: If the displacement rank of ^A is ^M then r ^A ^M ^p +^q: ^px i ^g (+) i ^g (+) i ^qx i ^g ( ) i ^g ( ) i () he vectors ^g (+) i ^g ( ) i R ^N are called the positive and the negative generators of ^A respectively. Let h i ^G ^g (+) ^g (+) ^p ^g ( ) ^g ( ) ^q : he generator matrix ^G is said to be in proper form if the entries in the first non zero column are equal to zero, except for a non-zero in the first row. Define I r the identity matrix of order r: Let J^p^q I^p Φ ( I^q ) where the symbol» Φ denotes the concatenated direct sum, i.e., A 0 A Φ B : hen () can be written in the 0 B following way r ^A ^G J^p^q ^G: Amatrix ^Q is said J^p^q orthogonal if ^Q J^p^q ^Q J^p^q : A generator matrix is not unique. In fact, if ^Q is a J^p^q orthogonal matrix, then ^Q ^G is a generator matrix, too. he following theorem holds []. heorem Let ^A be a s.p.d. matrix. Let ^G () [g (+) :::g (+) ^p g ( ) :::g ( ) ^q ] be a generator matrix in proper form of ^A with respect to the shift matrix ^Z: hen g (+) is the first row of ^R the Cholesky factor of ^A: Furthermore the generator matrix for the Schur complement is given by ^G () h ^A ^A g (+) g (+) ^Zg (+) g (+) ^p g ( ) ^g ( ) ^q i : Let F be a full rank rectangular matrix. hen F F is a s.p.d. matrix, with Cholesky decomposition ~ R ~ R ~ R upper triangular. If» R is the R factor of the QR decomposition of F it is well known that»r ~ R where diag(±:::±): Hence both the problem of computing the R factor of the QR factorization of F and that of computing the Cholesky factor of F F are equivalent. We observe that the entries of the first row (and of the first column) of ^A are equal to 0. he entries of the first column of ^G () are equal to 0, too. o compute the second row of the Cholesky factor, we need to find a J^p^q orthogonal matrix ^Q such that ~ G () ^Q ^G () is in proper form. hen the first row of ~ G () becomes the second row of ^R: If we pose ~G () [g (+) :::g (+) ^p g ( ) :::g ( ) ^q ] then a generator matrix for the Schur complement is given by ^A ^A g (+) g (+) ^G () [^Zg (+) :::g (+) ^p g ( ) :::g ( ) ^q ] : By successively repeating the described procedure, we obtain the triangular factor ^R: Generalized Schur algorithm Input: ^G () [g (+) :::g (+) ^p g ( ) :::g ( ) ^q ] Output: ^R for i : ^N Find a J^p^q orthogonal matrix ^Q i such that G ~ (i) ^Q ^G i (i) is in proper form. i th row of ^R : first row of G ~ (i) ^G (i+) [^Zg (i+) :::g (i+) ^p g (i ) :::g (i ) ^q ] end he computational complexity is O( ^M ^N ): Hence the generalized Schur algorithm can be efficiently used when ^M fi ^N i.e., when ^A is a oeplitz like matrix. he next section describes the particular choice of (i) ^Q i ^G and ^Z for solving the System Identification problem. REDUCION OF HE GENERAORS MARIX IN PROPER FORM At each iteration of the generalized Schur algorithm a J^p^q orthogonal matrix Q i is chosen in order to reduce the generator matrix in proper form. his can be accomplished in a few different ways (see for instance [0, 8, ]). However, for the sake of the stability, we consider an implementation that allow us to compute the R factor also when the matrix H is rank deficient. First of all, we observe that the product of two J^p^q orthogonal matrices is a J^p^q orthogonal matrix. Furthermore, if Q and Q are orthogonal matrices of order ^p and ^q respectively, then the orthogonal matrix Q Q Φ Q is a J^p^q orthogonal matrix, too.

If jρj < the matrix product, called modified hyperbolic rotation [],» #» ρ» p 0 " 0 p ρ ρ ρ 0 is J orthogonal w.r.t. the matrix J» 0 0 : 0 Let ^G (i) be the generator matrix at the beginning of the ith iteration of the generalized Schur algorithm. he J^p^q orthogonal matrix Q i such that G ~ (i) Q ^G i (i) is in proper form, can be constructed in the following way. Let» ^G ^G (i) (i+) ^G (i ) where ^G (i+) g (i+).. g (i+) ^p 5 and ^G (i ) g (i ).. g (i ) ^q Let Q i and Q i be two orthogonal matrices of order ^p and ^q respectively, such that G ~ (i+) Q ^G (i+) i and ~G (i ) Q ^G (i ) i are both in proper form and let ^Q i Q i Φ Q i :Q i and Q i can be either Householder matrices or products of Givens rotations chosen in order to annihilate all the entries of the ith column except for the first one of the matrices ^G (i+) and ^G (i ) respectively. Let v R whose components are the nonzero entries in the ith column of ~ G (i+) and ~ G (i ) respectively. Consider the J orthogonal matrix ^ such that the second component of the vector ^v ^v is 0: Let ~ i ^ 0 ^ 0 0 I^p 0 0 ^ 0 ^ 0 0 0 0 I^q 5 where I^p and I^q are identity matrices of order ^p and ^q respectively, and the 0 s in the matrix ~ i are null matrices of appropriate dimension. It is easy to see that ~ i is a J^p^q orthogonal matrix. he J^p^q orthogonal matrix ^Q i ~ i is such that ^Q i ~ i ^G (i) is in proper form. Hence Q i ^Q i ~ i is the sought matrix. FAS COMPUAION OF HE R FACOR OF HE QR FACORIZAION OF H In this section the computation of the R factor of the QR factorization of H defined in x, exploiting the displacement structure of H H is described. We first show that the displacement rank of H H is (m + l +)at most, with the same number of positive and negative generators. Hence the generalized Schur algorithm to compute the R factor requires O s (m + l) flops. Furthermore the computation of the generators requires O ((m + l)n ) flops. Let» W H UsN UsN H U sn YsN Y sn UsN Y sn YsN Let Z m and Z l be shift matrices of order m and l respectively. Denote Ms Ms Z sm Z m Z sl Z l and i Z Z sm M Z sl : i Partition U sn and Y sn in the following way,»» ^u ^u ^y ^y U sn Y ^u ^u sn ^y ^y with ^u R m (N ) ^u R m ^u R (s )m (N ) ^u R (s )m and ^y R l (N ) ^y R l ^y R (s )l (N ) ^y R (s )l : hen r W W ZWZ ^u ^u ^u ^u ^u ^y ^u ^y ^u ^u 0 ^u ^y 0 ^y ^u ^y ^u ^y ^y ^y ^y ^y ^u 0 ^y ^y 0 5 +g (+) m+l+ g(+) ( ) m+l+ g m+l+ g( ) m+l+ ( ) g ^W + g (+) m+l+ g(+) m+l+ m+l+ g( ) m+l+ where g (+) m+l+ [^u ^u ^y ^y ] g ( ) m+l+ ZH( : ) : he symmetric matrix ^W has rank (m + l) at most. Hence the displacement rank of W is (m + l +) i:e: r W W ZWZ m+l+ X i X m+l+ i g (+) i g (+) i g ( ) i g ( ) i : he computation of the remaining generators requires the computation of ^W that is, of the first m rows and the rows sm + i i : l: his can be accomplished in O ((l + m)n ) flops. We observe that the computation of the matrix ^W requires O((s(m + l) N ) flops. Furthermore we point out that the matrix H involved in the computation of ^W is not explicitly stored. he computation indeed is done considering only the vectors y and u taking into account the block Hankel structure of H:

COMPUAIONAL BURDEN o compute the generators of H H the product H H(: [ : m sm +:sm + l]) () is needed. his can be accomplished in N (m + l) flops. All the techniques described in the appendix require O(s(m + l)) flops to compute the generators. If Householder matrices are chosen as the orthogonal matrices involved at each iteration, the total number of flops of the generalized Schur algorithm is 8s (m + l) + s (m + l) : As N is significantly larger than s m l the bulk of the algorithm is the computation of (). HE GENERALIZED SCHUR ALGORIHM FOR RANK DEFICIEN MARICES In this section we describe how the generalized Schur algorithm can be modified to compute the R factor in case H is a rank deficient matrix. More details can be found in [5]. We show that the computational complexity is reduced to O(rs(m + l)) in this case, where r is the rank of H: Without loss of generality, we suppose that the first k k :::s(m + l) columns of H are full rank and the (k +) th column linearly depends on the first k columns, i.e., H H H w H Λ where H R N k w R k H R N (m+l)s k : hen H H H H H H w H H w H H w H H w w H H H H H H w H H he Schur complement of H H with respect to H H is (H H) (k) 0 0 0 H (I H (H H ) H )H We observe that the first row (and column) of (H H) (k) is a null vector. Let ~ G (k+) and ~ G (k ) be the matrix of positive and negative generators of (H H) (k) respectively, in proper form. hen the first row of ~ G (k+) is equal to the first row of ~ G (k ) : Hence these rows can be dropped from the corresponding matrices, reducing the number of generators by and the (k +)th row of the R factor is a null vector. From a numerical point of view we can say that the described procedure works in an accurate way when it is applied to matrices H such that the gap between the large singular values and the negligible ones is sufficiently able : Numerical results for Example R M R S b. e. R M b. e. R S n. r. # flops # flops 0 8 :5 0 : 0 5 5 large. A loss of accuracy in the computed factor R happens when the distribution of the small singular values of H shows a uniform and slow decrease. his behaviour of the algorithm is described in the following examples. Example Consider the matrix H [U jy ] with Y U, where the first row and the last column of U are [ 0 9 8 ::: ] [ 5 ] respectively. he rank of the matrix H is 5: 0 5 0 0 0 5 0 0 0 5 0 0 0 5 0 0 0 5 0 8 0 8 0 Figure : Distribution of the singular values, in logarithmic scale, of the matrix considered in Example In able the results of the computation of the R factor of the matrix H by means of the standard QR and the generalized Schur algorithm are shown. We denote by R M b.e.r S b.e.r Λ and n.r., the R factor of the QR factorization of H computed by the matlab function triu(qr(h)) and by the generalized Schur algorithm, the backward error of H H defined as kh H R Λ R Λk kh Hk and the rank of H detected by the generalized Schur algorithm, respectively. In this case, the R factor is accurately computed by the generalized Schur algorithm, because of the big difference between the significant singular values and the negligible ones of H (see Figure ). Example his is the fourth application considered in the next section. In Figure we can see that the distribution of the small singular values of the involved matrix H slightly decreases. We point out that the correlation matrix H H computed by matlab is not numerically

able : Numerical results for Example R M R S b. e. R M b. e. R S n. r. # flops # flops 9 0 5: 0 s.p.d. because of the nearly rank deficiency of H: So, in this case the fast Cholesky factorization, exploiting the block Hanckel structure of H and described in [], can not be used. In able we can see that, although the generalized Schur algorithm is very fast w.r.t. the standard QR algorithm, the achieved accuracy is not satisfactory. able : Summary description of applications. Appl. # Application m l s t N Glass tubes 0 0 Labo dryer 5 000 90 Glass oven 0 Mechanical flutter 0 0 90 5 Flexible robot arm 0 0 98 Evaporator 0 05 85 CD player arm 5 08 08 8 Ball and beam 0 000 90 9 Wall temperature 0 800 0 0 0 0 0 0 0 0 0 0 0 5 0 0 0 0 0 0 0 50 0 0 80 able : Comparative results for the computation of the R factor. Figure : Distribution of the singular values, in logarithmic scale, of the matrix considered in Example NUMERICAL RESULS In this section results computing the R matrix by means of the generalized Schur algorithm are summarized. he data sets considered are publicly available on the DAISY web site Appl. # Application R M R S # flops # flops Glass tubes : 0 :9 0 Labo dryer :0 0 :00 0 5 Glass oven : 0 9:8 0 Mechanical flutter :5 0 : 0 5 Flexible robot arm :5 0 : 0 5 Evaporator :8 0 8 9: 0 CD player arm 5: 0 : 0 8 Ball and beam : 0 :8 0 5 9 Wall temperature : 0 :8 0 www.esat.kuleuven.ac.be/sista/daisy For all the considered applications, the generators have been computed using the second technique described in the Appendix. At each iteration of the generalized Schur algorithm, two products of Givens rotations and a modified hyperbolic rotation are performed in order to reduce the generator matrix in proper form. All the numerical results have been obtained on a Sun workstation Ultra 5 using Matlab 5.. able gives a summary description of the applications considered in our comparison, indicating the number of inputs m the number of outputs l the number of block rows s the total number of data samples used t and the number of rows of H: In able and 5 some results for the computation of the R factor of the QR factorization of H are presented. Rel. residual denotes able 5: Comparative results for the computation of the R factor. Appl. # Application backward error R S Rel.residual Glass tubes :0 0 0 Labo dryer :9 0 00 0 Glass oven :8 0 : 0 9 Mechanical flutter ΛΛΛΛΛΛ ΛΛΛΛΛΛΛ 5 Flexible robot arm : 0 9 0 Evaporator :89 0 9:5 0 CD player arm : 0 : 0 9 8 Ball and beam 8: 0 5 5:8 0 9 Wall temperature 5:5 0 8 0 kjr M j jr S jk kjr M jk :

he results in able and 5 are comparable with those described in [], where the R factor is obtained considering the Cholesky factorization of the correlation matrix H H and exploiting the block-hankel structure of H: he analysis of the fourth application is described in Example of the previous section. We can not compare, in this case, the elements of the matrices computed by both considered algorithms, since the generalized Schur detects a rank deficiency in the matrix H. CONCLUSIONS In this paper the generalized Schur algorithm to compute the R factor of the QR factorization of block Hankel matrices, arising in some subspace identification problems, is described. It is shown that the generalized Schur algorithm is significantly faster than the classsical QR factorization. A rank revealing implementation of the generalized Schur algorithm in case of rank deficient matrices is also discussed. Algorithmic details and numerical results have been presented. ACKNOWLEDGEMENS his paper presents research results of the Belgian Programme on Interuniversity Poles of Attraction (IUAP P-0 and P-), initiated by the Belgian State, Prime Minister s Office - Federal Office for Scientific, echnical and Cultural Affairs, of the European Community MR Programme, Networks, project CHRX-C9-00, of the Brite Euram Programme, hematic Network BRR-C9-500 Niconet, of the Concerted Research Action (GOA) projects of the Flemish Government MEFISO- (Mathematical Engineering for Information and Communication Systems echnology), of the IDO/99/0 project (K.U.Leuven) Predictive computer models for medical classification problems using patient data and expert knowledge, of the FWO Krediet aan navorsers G.0.98 and the FWO project G.000.00. References [] A.W. Bojanczyk, R.P. Brent, P. Van Dooren and F.R. De Hoog, A note on downdating the Cholesky factorization, SIAM J. Sci. Stat. Comput., (980),pp. 0 0. [] Y.M. Cho, G.Xu and. Kailath, Fast recursive identification of state space models via exploitation of displacement structure, Automatica, Vol.0. No., (99) pp. 5 59. [] J. Chun,. Kailath and H. Lev ari, Fast parallel algorithms for QR and triangular factorization, SIAM J. Sci. and Stat. Comp., Vol. 8, No. (98), pp. 899 9. [] B. De Moor, P. Van Overschee and W. Favoreel, Numerical Algorithms for Subspace State-Space System Identification An Overview, in Applied and Computational Control, Signals and Circuits, Vol., Chapter, Birkhäuser, Boston, 999. [5] K.A. Gallivan, S. hirumalai, P. Van Dooren and V. Vermaut, High performance algorithms for oeplitz and block oeplitz matrices, Linear Algebra Appl., -,(99) -88. []. Kailath, S. Kung and M. Morf, Displacement ranks of matrices and linear equations, J. Math. Anal. Appl., 8, (99), pp. 95 0. []. Kailath and A.H. Sayed, Displacement structure: theory and applications, SIAM Review,, (995), pp. 9 8. [8]. Kailath, Displacement structure and array algorithms, in Fast Reliable Algorithms for Matrices with Structure,. Kailath and A. H. Sayed, Ed., SIAM, Philadelphia, 999. [9] M. Moonen, B. De Moor, L. Vandenberghe and J. Vandewalle, On- and off-line identification of linear state space models, Int. J. Control, 9(), (989) pp. 9-. [0] C.M. Rader and A.O. Steihardt, Hyperbolic Householder transforms, SIAM J. Matrix Anal. Appl., 9, (988), pp. 9 90. [] V. Sima, Subspace based algorithms for multivariable system identification, Studies in Informatics and Control, Vol.5, No., (99) pp.5. [] V. SIMA, Cholesky or QR factorization for data compression in subspace based identification?, Proceedings Second NICONE Workshop, Paris Versailles, Dec., (999), pp.5 80. [] M. Stewart and P. Van Dooren, Stability issues in the factorization of structured matrices, SIAM J. Matrix Anal. Appl., Vol. 8, (99), pp. 0 8. [] P. Van Overschee and B. De Moor, Subspace Identification for Linear Systems: heory, Implementation, Applications, Kluwer Academic Publishers, Dordrecht, 99. [5] P. Van Overschee and B. De Moor, NSID: wo subspace algorithms for the identification of combined deterministic-stochastic systems, Automatica, 0(), (99) pp. 5 9. [] M.Verhaegen, Subspace model identification. Part :Analysis of the ordinary output error state space model identification algorithm, Int. J. Control, 58() (99), pp. 555 58.