An Improved Algorithm for Computing the Singular Value Decomposition

Transcription

1 An Imprved Algrithm fr Cmputing the Singular Value Decmpsitin TONY F. CHAN Yale University The mst well-knwn and widely used algrithm fr cmputing the Singular Value Decmpsitin (SVD) A --- U ~V T f an m x n rectangular matrix A is the Glub-Reinsch algrithm (GR-SVD). In this paper, an imprved versin f the riginal GR-SVD algrithm is presented. The new algrithm wrks best fr matrices with m >> n, but is mre efficient even when m is nly slightly greater than n (usually when m ~ 2n) and in sme cases can achieve as much as 50 percent savings. If the matrix U ~s exphcltly desired, then n 2 extra strage lcatins are required, but therwise n extra strage is needed. The tw main mdificatins are: (1) first triangularizing A by Husehlder transfrmatins befre bldmgnahzing it (thin idea seems t be widely knwn amng sme researchers in the field, but as far as can be determined, neither a detailed analysis nr an lmplementatmn has been published befre), and (2) accumulating the left Givens transfrmatins in GR-SVD n an n x n array instead f n an m x n array. A PFORT-verified FORTRAN Implementatin m included. Cmparisns with the EISPACK SVD rutine are given. Categries and Subject Descriptrs" G.1.3 [Numerical Analysis]: Numerical Linear Algebra-- e~genvalues General Terms: Algrithms, Perfrmance Additinal Key Wrds and Phrases Husehlder transfrmatins, singular values The Algrithm. An Imprved Algrithm fr Cmputing the Singular Value Decmpsitin. ACM Trans. Math. Sftw. 8, 1 (Mar. 1982), INTRODUCTION Let A be a real m x n matrix, with m _> n. It is well knwn [5, 6] that the fllwing decmpsitin f A always exists: A = UXV T, (1.1) where U is an m n matrix and cnsists f n rthnrmalized eigenvectrs assciated with the n largest eigenvalue f AA T, V is an n n matrix and cnsists f the rthnrmalized eigenvectrs f ATA, and 5] is a diagnal matrix cnsisting This wrk was supprted by NSF Grant DCR and NASA Ames Cntract NCA 2-OR The cmputing time was prvided by the Stanfrd Linear Acceleratr Center (SLAC). Authr's address: Department f Cmputer Scmnce, Yale University, 10 Hfllhuse Avenue, P.O. Bx 2158 Yale Statin, New Haven, CT Permissin t cpy withut fee all r part f thin matetaal is granted prvided that the cpies are nt made r distributed fr direct cmmercial advantage, the ACM cpyright ntice and the title f the publicatin and its date appear, and ntice is given that cpying is by permissin f the Assciatin fr Cmputing Machinery T cpy therwme, r t republish, requires a fee and/r specific permissin ACM /82/ $00.75 ACM Transactins n Mathematmal Sftware, Vl. 8, N. 1, March 1982, Pages 72-83

2 Imprved Algrithm fr Cmputing the Singular Value Decmpsitin. 73 f the "singular values" f A, which are the nnnegative square rts f the eigenvalues f ATA. Thus, utu-~- vtv = VV T = In and It is usually assumed fr cnvenience that Z = diag(~... On). (1.2) O] >--~ O2 > =... >-~ On> ~0. The decmpsitin (1.1) is called the Singular Value Decmpsitin (SVD) f A. Remarks (1) If rank(a) = r, then r+l = r+2... On -~" O. (2) There is n lss f generality in assuming that m _ n, fr if m < n, then we can instead cmpute the SVD f A T. If the SVD f A T is equal t UZV T, then the SVD f A is equal t V~U T. The SVD plays a very imprtant rle in linear algebra. It has applicatins in such areas as least squares prblems [5, 6, 11], in cmputing the pseudinverse [6], in cmputing the Jrdan cannical frm [7], in slving integral equatins [9], in digital image prcessing [1], and in ptimizatin [2]. Many f the applicatins ften invlve large matrices. It is therefre imprtant that the cmputatinal prcedures fr btaining the SVD be as efficient as pssible. 2. THE GOLUB-REINSCH ALGORITHM (GR-SVD) We use the same ntatins as in [5] and [3]. This algrithm cnsists f tw phases. In the first phase ne cnstructs tw finite sequences f Husehlder transfrmatins and such that p(k), Q(k), k -- 1, 2... n k=l, 2,...,n-2, n p(n)... p(1)aq(l).." Q(n-2) x 0 x x 0 x n = j(0), (2.1) 0 (m - n) an upper bidiagnal matrix. Specifically, P(') zers ut the subdiagnal elements in clumn i and Q(J) zers ut the apprpriate elements in rw j. ACM Transactins n Mathematmal Sftware, Vl. 8, N. 1, March 1982

3 74 Tny F. Chan Because all the transfrmatins intrduced are rthgnal, the singular values f j() are the same as thse f A. Thus, if is the SVD f j(~, then s that j() = G]EH T A = PG~.HTQ T U = PG, V ffi QH (2.2) with p = p(1)... p(n), Q = Q(1)... Q(n-2). The secnd phase is t iteratively diagnalize j(0) by the QR methd s that j(). j(i)... ) ~, (2.3) where j(,+l) = (S(,))Tj.~T(,). where S (') and T (') are prducts f Givens transfrmatins and are therefre rthgnal. The matrices T (') are chsen s that the sequence M (') = (j(,))tj(o cnverges t a diagnal matrix, while the matrices S (') are chsen s that all J(') are f bidiagnal frm. The prducts f the T (') and the S ( are exactly the matrices H T and G T, respectively, in eq. (2.2). Fr mre details, see [5]. It has been reprted in [5] that the average number f iteratins n J(') in (2.3) is usually less than 2n. In ther wrds, j(2,) in eq. (2.3) is usually a gd apprximatin t a diagnal matrix. We briefly describe hw the cmputatin is usually implemented. Assume fr simplicity that we can destry A and return U in the strage fr A. In the first phase the p(0 are stred in the lwer part f A, and the Q(') are stred in the upper triangular part f A. After the bidiagnalizatin the Q (') are accumulated in the strage prvided fr V, the tw diagnals f j() are cpied t tw ther linear arrays, and the P(') are accumulated in A. In the secnd phase, fr each i, S (') (T( ) w is applied t P frm the right, and is applied t QT frm the left in rder t accumulate the transfrmatins. 3. THE MODIFIED ALGORITHM (MOD-SVD) Our riginal mtivatin fr this algrithm is t find an imprvement f GR-SVD when rn >> n. In that case tw imprvements are pssible: (1) In eq. (2.1), each f the transfrmatins P(') and Q") has t be applied t a submatrix f size (m - ~ + 1) (n - i + 1) (see Figure 1). Nw, since mst entries f this submatrix are ultimately ging t be zers, it is intuitive that if it can ACM Transactins n Mathematmal Sftware, Vl 8, N. 1, March 1982

4 Imprved Algrithm fr Cmputing the Singular Value Decmpsitin * 75 n A n-~+l A m -m-i+l Fig. 1. P(') and Q(') affect the shaded prtin f the matrix. smehw be arranged that the Q(') des nt have t be applied t the subdiagnal part f this submatrix, then we will be saving a great amunt f wrk when m>>n. This can indeed be dne by first transfrming A int upper triangular frm by Husehlder transfrmatins n the left: where R is n x n upper triangular and L is rthgnal, and then prceed t bidiagnalize R. The imprtant difference is that this time we will be wrking with a much smaller matrix, R, than A (if n 2 << ran), and s it is cnceivable that the wrk required t bidiagnalize R is much less than that riginally dne by the right transfrmatins when m >> n. The questin still remains as t hw t bidiagnalize R. An bvius way is t treat R as an input matrix t GR-SVD, using alternating left and right Husehlder transfrmatins. In fact, it can be easily verified that if the SVD f R is equal t X~ yt, then the SVD f A is given by We can identify U with L[X/O] and V with Y. Ntice that in rder t btain U, we have t frm the extra prduct L[X/O]. If U is nt needed explicitly (e.g., in least squares), then we d nt have t accumulate any left transfrmatins, and in that case, fr m _ n, it seems likely that we will make a substantial saving. ACM Transactlns n Mathematwal Sftware, Vl. 8, N. I, March 1982.

5 76 Tny F. Chan It is als pssible t take advantage f the structure f R t bidiagnalize it. This is discussed in Sectin 4. (2) The secnd imprvement f GR-SVD that can be made is the fllwing: In GR-SVD, if U is wanted explicitly, each f the S (') is applied t the m n matrix P frm the right t accumulate U. If m >> n, then this accumulatin may invlve a large amunt f wrk, because a single Givens transfrmatin affects tw clumns f P (f length m) and each S (') is the prduct f n the average n/2 Givens transfrmatins. Therefre, in such cases it wuld seem mre efficient t first accumulate all S (') n an n n array, say Z, and later frm the matrix prduct PZ after J('~ has cnverged t ~. In essence, imprvement (1) wrks best when U is nt needed, imprvement (2) wrks best when U is needed, and bth wrk best when m >> n. These imprvements are bth incrprated in the fllwing mdified GR-SVD algrithm: MOD-SVD (1) LT[A] --,, [R/0] where R is n n upper triangular. (2) Find the SVD f R by GR-SVD, R = X~Y w. (3) Frm A -- L[X/O]ZY T, the SVD f A. The idea f transfrming A t upper triangular frm when m >> n and then calculating the SVD f R is mentined in Lawsn and Hansn [11, pp. 119 and 122] in the cntext f least squares prblems where U is nt explicitly required. 4. SOME COMPUTATIONAL DETAILS (1) As in mst GR-SVD implementatins, the input matrix A is first cpied int the array U. Thereafter, the array A is never referenced and all peratins are perfrmed n the array U. It shuld be bvius that when U is nt needed, MOD-SVD des nt require any extra strage. When U is needed, we can stre L w in the lwer part f array U, cpy R int anther n x n array Z, and ask GR- SVD t return X in Z. Therefre we need at mst n 2 extra strage lcatins (array Z), which is relatively small (when rn >> n) cmpared t mn lcatins already needed fr array U. (2) The next questin is hw t frm L[X/O] withut using extra strage. This can be dne by nting that s we can first accumulate L[//0] in the space prvided fr U in place, and then d a matrix multiplicatin by X. Anther pssibility is t actually accumulate the Husehlder transfrmatins L n [X/0]. With the usual implementatin, this requires mn instead f n 2 extra strage lcatins, but slightly less wrk (2ran 2 - n 3 versus 2ran 2 - n3/3 multiplicatins). Our current implementatin uses the first methd. A ptential imprvement has been suggested by Kauffman [10]. (3) The questin arises whether it is pssible t bidiagnalize R in a way that takes advantage f the zers that are already in R. The usual Husehlder transfrmatins cannt take advantage f this structure. One way is t use Givens ACM Transactins n Mathematmal Sftware, Vl 8, N. 1, March 1982

6 Imprved Algrithm fr Cemputing the Singular Value Decmpsitin 77 First rtatin intrduces nnzer element here "~ ~ ~ M --,",~ ~.... First rtatin t zer )~ ut the (1, 5) element _,; _ Secnd rtatin t zer ut the (2, 1) element intrduced by the first rtatin Figure 2 transfrmatins t zer ut the elements at the upper right-hand crner f R, ne clumn r ne rw at a time. Pictrially (fr n ffi 5), t zer ut the {1, 5) element, we d tw Givens transfrmatins, as shwn in Figure 2. It turns ut, hwever, by simple cunting and verificatin by experiments, that this methd takes abut the same peratins (4n3/3 multiplicatins) as the previus methd t bidiagnalize R, prvided that we d nt have t accumulate transfrmatins. If we d need t accumulate either the left r right transfrmatins, then this methd will require mre wrk (4n 3 versus 4n~/3 multiplicatins), mainly because it requires tw rtatins t zer ut each element and these rtatins have t be accumulated. S it seems that taking advantage f the zer structure f R in this fashin actually makes the methd less efficient. We have t nte, hwever, that Givens transfrmatins invlve fewer additins and array accesses than Husehlder transfrmatins per multiplicatin (see Sectin 5). Therefre this methd may turn ut t be mre cmpetitive n mdern cmputers where the time taken fr flating-pint additins and multidimensinal array indexings are nt negligible cmpared t that fr multiplicatins. Als, the use f fast Givens [4] may result in substantial imprvement in efficiency. 5. OPERATION COUNTS In Sectin 3 we indicated that MOD-SVD shuld be mre efficient than GR- SVD when m >> n. In this sectin we study the relative efficiency between GR- SVD and MOD-SVD as a functin f m and n. We d this by cmputing the asympttic peratin cunts fr each algrithm. In the fllwing peratin cunts, we nly keep the highest rder terms in m and n, and s the results are crrect fr relatively large m and n. GR-SVD (1) Bidiagnalizatin {using Husehlder transfrmatins) j = p(,)... p(1)aq(~)... Q(,-2) 2(ran 2- n3/3) mult. ACM Transactins n Mathematical Sftware, Vl. 8, N. 1, March 1982

7 78 Tny F. Chan accumulate P ffi ptl}... pt,} mn 2 _ n3/3 accumulate Q ffi Qtl}... Qt,-2} 2n3/3 (2) Diagnalizatin (using Givens transfrmatins) accumulate S ~'} n P Cmn 2 (C -- 4) mult. accumulate T t'} n Q Cn 3 (C ffi 4) mult. mult. mult. MOD-SVD (1) Triangularizatin (using Husehlder transfrmatins) LT[A] --* [R/O] mn 2 - n3/3 mult. (2) GR-SVD f R, R ~- X~,Y w depends n whether accumulatins are needed (3) Frm L[UO]X (using Husehlder transfrmatins) 2mn 2 - n3/3 mult. Sme cmments are in rder: (1) The entries Cmn 2 and Cn a in the diagnalizatin phase f GR-SVD are btained by assuming that the iterative phase f the SVD takes n the average tw cmplete QR iteratins per singular value [5], [11, p. 122]. We have checked this experimentally and fund it t be quite accurate. It is assumed that slw Givens is used thrughut the calculatin. If fast Givens [4] had been used, then the entries wuld becme apprximately 2mn 2 and 2n 3, instead. (2) As with mst peratin cunts, we have used the number f multiplicatins as a measure f wrk. Fr the Husehlder transfrmatins, each multiplicatin als invkes ne additin and apprximately tw array addressings. Fr the Givens transfrmatins, each multiplicatin invkes ne-half an additin and ne array addressing. On many large cmputers tday, a flating-pint multiplicatin is nt much slwer than a flating-pint additin. Als, array indexing (invlving integer arithmetic) is usually quite expensive. In such cases, a Husehlder multiplicatin actually invlves mre wrk than a Givens multiplicatin because f the extra additins and array indexings. Therefre, the peratin cunts given fr the diagnalizatin phase f GR-SVD may be misleading because it may actually invlve relatively less wrk. The ttal effect, hwever, can be accunted fr by using a smaller value fr C. Fr example, if ne Givens "multiplicatin" takes half the wrk needed by a Husehlder "multiplicatin," then the effect n the relative efficiency can be accunted fr by setting C ffi 2 instead f C On lder r nnscientific machines where multiplicatins take much mre time than additins and array addressings, the peratin cunt based n multiplicatins alne is usually a gd measure f relative efficiency. (3) The applicatin f S (')w and T (') n J(') is actually f rder O(n 2) and is therefre nt included in the previus cunts. (4) We have t distinguish between fur cases in the cmparisn: Case a: Bth U and V are required explicitly. Case b: Only U is required explicitly. Case c: Only V is required explicitly. Case d: Only ~ is required explicitly. ACM Transactins n Mathematical Sftware, Vl 8, N l, March 1982

8 Imprved Algrithm fr Cmputing the Singular Value Decmpsitin 79 Table I. Ttal Operatin Cunts f GR-SVD and MON-SVD fr Each f the Cases a, b, c, and d Case GR-SVD MOD-SVD a (3 + C)mn 2 + (C - 1/3)n '~ 3mn 2 + (2C + 2)n ~ b (3 + C)mn 2 - n 'j 3ran e + (C + 4/3)n 3 c 2mn 2 + Cn ~ mn 2 + (C + 5/3)n 3 d 2mn 2-2n'~/3 mn 2 + n 3 Table II. Rati f Operatin Cunt f MOD-SVD t that f GR-SVD, r = m/n Crssver pint hmit as Case Rati ~* r --) 0 a [3r + (2C + 2)]/[(3 + C)r + (C - 1/3)] (C + 7/3)/C 3/(3 + C) b [3r + (C + 4/3)]/[(3 + C)r -1] (C + 7/3)/C 3/(3 + C) c [r + (C + 5/3)]/[2r + CI 5/3 1/2 d [r + 1]/[2r - 2/3] 5/3 1/2 These fur cases d arise in applicatins. We will mentin a few here: Case a arises in the cmputatin f pseudinverse [5]. Case b is Case c fr A w. Case c arises in least squares applicatins [5, 11] and in the slutin f hmgeneus linear equatins [5]. Case d arises in the estimatin f the cnditin number f a matrix and in the determinatin f the rank f a matrix [13]. The ttal peratin cunt fr each case is given in Table I. Using Table I, we can cmpute the rati f the peratin cunts f MOD-SVD t that f GR-SVD fr each f the fur cases. The results are given in Table II, where the rati is expressed as a functin f r = m/n. These ratis are pltted in Figure 3a-d fr C = 2, 3, 4. The crssver pint r* is the value f r that makes the rati equal t 1. If r > r*, then MOD-SVD is mre efficient than GR-SVD. We see that, in all fur cases, MOD-SVD becmes mre efficient than GR-SVD when r starts t get bigger than 2, apprximately, and the savings can be as much as 50 percent when r is abut 10. On the ther hand, when r is abut 1, GR-SVD is mre efficient. This agrees with ur earlier cnjectures. Hwever, the imprtant thing is that all the ratis decrease quite fast as r becmes large. If we assume that it is equally likely t encunter matrices with any value f r >-- 1 (this is nt an unreasnable assumptin fr designers f general mathematical sftware, fr example), then MOD-SVD is bviusly preferable. In any case, these ratis give indicatins as t when ne f the methds is mre efficient, at least when m and n are large enugh s that ur peratin cunts apply. In the cntext f least squares applicatins, we can als cmpare the peratin cunts f GR-SVD and MOD-SVD t thse f the rthgnal triangularizatin methds (OTLS) [8], which are ften used fr such prblems. Analgus t Table I, the ratis f peratin cunts are nw OTLS: GR-SVD = [r- 1/3]/[2r + C] OTLS: MOD-SVD = [r - 1/3]/[r + C + 5/3]. ACM TransacUns n Mathematical Sftware, VL 8, N. 1, March 1982

9 == b C3 a "K "u; 3 "u; fl v; 0 0~/ -H C'! 0"! 9"0 9"0 OIl~ _f 0-'~1 9 '0 9:0 0 I.T.Ua ~"0 c) II d~ II // :) OI i ; d ;> 03 r..b ;> E] -K c) "K 0cg -ug O c~ -,d r' :) ",,i 0 '7 8 "0 9' 0 Ollg~ ~,'0 8"! 0"7! 9:0 9 "J0 0IJ.U~J ~'0

10 Imprved Algrithm fr Cmputing the Singular Value Decmpsitin 81 One sees frm these ratis that fr m nearly equal t n(r ~ 1), the tw SVD algrithms require much mre wrk than OTLS. Hwever, when r is bigger than abut 3, MOD-SVD requires nly abut 3 times mre wrk than OTLS. It may therefre becme ecnmically feasible t slve the least squares prblems at hand by MOD-SVD instead f OTLS. The reward is that the SVD returns much mre useful infrmatin abut the prblem than OTLS [11]. It is easy t see that as r becmes arbitrarily large, MOD-SVD is as efficient as OTLS, since the bulk f the wrk is in the triangularizatin f the data matrix A. Hwever, GR-SVD can be at mst half as efficient as OTLS. 6. A HYBRID ALGORITHM On the basis f the results f earlier sectins, we can implement a hybrid methd fr cmputing the SVD f a rectangular matrix A, which autmatically chses t use the mre efficient algrithm between GR-SVD and MOD-SVD. Fr each f the fur Cases a, b, c, and d, if the input matrix A has a value f r(ffi m/n), which is less than the crssver pint r* fr that case, we use GR-SVD; therwise we use MOD-SVD. The crssver pints depend n the value f C used. As nted befre, the value f C t be used depends n the relative efficiencies f flatingpint multiplicatins, flating-pint additins, and array indexings n the particular machine cncerned. Hwever, C can be determined nce fr any particular machine and cmpiler cmbinatin. Fr example, if flating-pint multiplicatins take much mre time than flating-pint additins and array indexings n the machine in questin, then we shuld use C apprximately equal t 4. In mst situatins, ne can prbably d just as well by setting the crssver pint equal t a fixed value.~2, since this pint is nt sensitive t C at all. In the algrithm (see p. 84) we give the cdes f a PFORT [3a] verified FORTRAN subrutine called HYBSVD which implements the previusly mentined hybrid algrithm. HYBSVD will need t call a standard Glub-Reinsch SVD subrutine during part f its cmputatin, and s we have included such a rutine, called GRSVD, in the package t be used with HYBSVD. The rutine GRSVD is actually a slightly mdified versin f the subrutine SVD in the EISPACK [12] package. The main mdificatin that we have made is t eliminate the requirement in subrutine SVD that the rw dimensin f V declared in the calling prgram be equal t that f A. This minimized the strage requirements f GRSVD at the cst f ne mre argument in the argument list. There is ne additinal feature implemented in HYBSVD (and als in GRSVD). In least squares applicatins, where we have verdetermined linear system Ax = b, the left transfrmatins U w have t be accumulated n the righthand-side vectrs b (there may be mre than ne b). This can be dne by putting the vectrs b in the matrix argument B when calling HYBSVD and setting IRHS t the number f b's. This is analgus t rutine MINFIT in EISPACK. The calling sequences and usages f HYBSVD are explained in the cmments in the beginning f the subrutine. 7. COMPUTATIONAL RESULTS The cnclusins in Sectin 5 hld nly if m and n are bth large. In this sectin sme cmputatinal experiments are carried ut t see if the cnclusins are still valid fr matrices with realistic sizes. ACM Transactins n Mathematical Sftware, Vl. 8, N. 1, March 1982.

11 82 Tny F. Chan We cmputed the SVD f sme randmly generated matrices using bth HYBSVD and the SVD rutine in EISPACK [12]. All tests were run n the IBM 370/168's at the Stanfrd Linear Acceleratr Center (SLAC). Lng precisin was used thrughut the calculatin. The mantissa f a flating-pint number is represented by 56 bits (apprximately 16 decimal digits}. The FORTRAN H ptimizing cmpiler (OPT = 2) was used thrughut. Fr each f the fur cases, we fixed sme values fr n and cmputed the SVD f a sequence f randmly generated matrices with different values f r. The executin times taken by HYBSVD and EISPACK SVD were then cmpared, tgether with the accuracies f the cmputed answers. Since we are wrking in a multiprgramming envirnment, the executin times we measured cannt be taken as the actual cmputing time taken, althugh the timing was dne with a lt f care; fr example, by averaging ver large samples. Thus, keeping these pints in mind, we can still expect a qualitative agreement with the analysis based n peratin cunts. On the IBM 370/168's at SLAC, a flating-pint multiplicatin takes nly abut 1.5 times the wrk taken fr a flating-pint additin. Als, the cst f array indexing is nt negligible. Therefre, as nted in Sectin 5, we shuld use a value fr C that is cnsiderably less than 4 fr the purpse f cmparing the relative efficiency f the tw algrithms n the basis f cmputatinal results. The results f the cmputatins are pltted in Figure 3a-d. In general, they agree very well qualitatively with the asympttic results we btained by peratin cunts, the best fit being with c - 3. We bserve that the larger n is, the better the agreement, as it shuld be. Hwever, even when n is small, the theretical results based n asympttic peratin cunts still describe very well the qualitative behavir f the cmputatinal results in many cases. The cmputatinal results als shw that large savings in wrk are indeed realizable fr reasnably sized matrices, and that indeed HYBSVD becmes mre efficient than GRSVD when r ~ 2. We als checked the accuracies f sme f the cmputed results. The singular values returned by bth prcedures HYBSVD and EISPACK SVD agree t within a few units f the machine precisin in almst all the cases that we have tested. The matrices U and V als agree t the same precisin, but the signs f the crrespnding clumns may be reversed. Hwever, the SVD is nly unique t within such a sign change, s this is acceptable [13]. Nte Added in Prf." Subsequent cmparisn tests were perfrmed with rutine SSVDC f LINPACK [3a]. The results were similar. 8. CONCLUSIONS We have presented an imprved algrithm and its FORTRAN implementatin HYBSVD, fr cmputing the SVD. We have demnstrated that the HYBSVD rutine wrks substantially better than the EISPACK SVD rutine fr matrices that have many mre rws than clumns (m ~ 2n), and since it uses the EISPACK SVD rutine when m ~ 2n, it is as efficient as the EISPACK SVD rutine in thse cases. It is als as accurate as the EISPACK SVD rutine. We ACM Transacttns n Mathematlca Sftware, Vl 8, N 1, March 1982.

12 Imprved Algrithm fr Cmputing the Singular Value Decmpsitin 83 have als seen that the cst f slving a least squares prblem by HYBSVD can ften be less than twice that f the usual rthgnal triangularizatin algrithms. It may therefre becme ecnmically feasible t slve many least squares prblems by SVD. The authr hpes that the imprved algrithm can be included in future versins f mathematical sftware packages, such as EISPACK. ACKNOWLEDGMENTS The authr wuld like t thank Jhn Gregg Lewis, Gene Glub, Charles Van Lan, and Bill Cughran fr their helpful discussins. The fllwing persns als helped at ne time r anther: C. Lawsn, R. Hansn, M. Gentleman, J. Oliger, P. Gill, J. Dennis, J. Blstad, and J. S. Pang. Finally, the authr thanks SLAC fr prviding the cmputing time that was used. REFERENCES 1. ANDREWS, H.C, AND PATTERSON, C.L. Singular value decmpsitins and digital image prcessmg. IEEE Trans Acust., Speech, S~gnal Prcesstng ASSP-24, 1 (Feb 1976). 2. BARTELS, R.H., GOLUB, G.H, AND SAUNDERS, M.A. Numermal techniques in mathematical prgramming. In Nnhnear Prgramming, Academm Press, New Yrk, pp CHAN, T.F. On cmputing the singular value decmpsitin. Rep. STAN-SC , Cmputer Science Dep. Stanfrd Umv., Stanfrd, Calif., a. DONGARRA, J. J., BUNCH, J. R., MOLER, C. B., AND STEWART, G.W. LINPACK Users's Gutde SIAM, Philadelphia, GENTLEMAN, W M Least sqaures cmputatins by Givens transfrmatins withut square rts. Rep. CSRR-2062, Univ. f Waterl, Waterl, Ont, Canada. 5. GOLUB, G.H., AND REINSCH, C Singular value decmpsitin and least squares slutins. In Handbk fr Autmattc Cmputatmn, II, Ltnear Algebra J.H. Wilkinsn and C. Reinsch (Eds.), Springer-Verlag, New Yrk, GOLUB, G.H., AND KAHAN, W. Calculating the singular values and pseudinverse f a matrix SIAMJ Numer. Anal 2, 3 (1965), GOLUB, G.H, AND WILKINSON, J.H. Ill-cnditined Eigensystems and the cmputatin f the Jrdan cannical frm. SIAM Rev. 18, 4 (Oct. 1976). 8. GOLUB, G H, AND BUSINGER, P.A. Linear least squares slutin by Husehlder transfrmatins Numer. Math. 7 (Handbk Series Linear Algebra), 1965, pp HANSON, R.J. A numermal methd fr slving Fredhlm integral equatins f the first kind using singular values. SIAM J. Numer. Anal 8, 3 (1971), KAUFFMAN, L. Applicatin f Husehlder transfrmatins t a sparse matrix. Cmputer Scmnce Tech. Rep. N. 63, Bell Labratries, Murray Hill, N.J., Nv LAWSON, C.L, AND HANSON, R.J Slwng Least Squares Prblems. Prentice-Hall, Englewd Cliffs, N.J., SMITH, B.T., ET AL. Matrtx Etgensystem Rutmes--EISPACK Guide, 2nd ed. (Lecture Ntes m Cmputer Sctence Sertes), Springer-Verlag, New Yrk, STEWART, G.W. Intrductmn t Matrix Cmputatins Academic Press, New Yrk, Received Octber 1976; revised July 1978; accepted December 1980 ACM Transactmns n Mathematmal Sftware, Vl. 8, N. 1, March 1982