COMPUTATION OF A TEST STATISTIC IN DATA QUALITY CONTROL

Transcription

1 SIAM J SI OMPUT Vo 26, No 6, pp c 25 Society for Industria and Appied Mathematics OMPUTATION OF A TEST STATISTI IN DATA QUALITY ONTROL XIAO-WEN HANG, HRISTOPHER PAIGE, AND HRISTIAN J M TIBERIUS Abstract When processing observationa data, statistica testing is an essentia instrument for rendering harmess incidenta anomaies and disturbances in the measurements A commony used test statistic based on the genera inear mode is the generaized ikeihood ratio test statistic The standard formua given in the iterature for this test statistic is not defined if the noise covariance matrix is singuar, and is not suitabe for computation if any of the matrices invoved are i-conditioned Based on Paige s generaized inear east suares method [omm Statist B Simuation omput, 7 (1978), pp , a numericay stabe approach is proposed for the computation of the test statistic, as we as for the estimates of the parameter vectors, and reiabe representations of the error covariance matrices for these estimates are presented This approach aows the noise covariance matrix to be singuar and can be appied directy to the inear mode with inear euaity constraints Key words data uaity contro, statistica testing, generaized ikeihood ratio, generaized east suares, generaized QR factorization, numerica stabiity AMS subject cassifications 62J12, 62J2, 65F2, 65F25 DOI 11137/ Introduction For a wide range of appications, data processing invoves east suares (LS) estimation The LS estimate in itsef, however, does not offer any indication of the vaidity of the resut Once some basic conditions are met, LS estimates for the unknown parameters, for exampe, the position coordinates in a Goba Positioning System (GPS) appication, are computed Usuay these give no indication as to whether, or to what extent, the input measurements are corrupted; ie, they do not satisfy the assumed mathematica mode In critica appications such as rea-time navigation, the user has to be protected against miseading (position) information caused by one or more anomaies in the measurements underying the resuting position estimate The incoming data need to be carefuy vaidated by means of statistica testing, using the redundancy in the measurement set-up and the conseuent strength of the mathematica mode Ony then can the integrity of the eventua position soution be assured A discussion of statistica testing for data uaity contro can be found, for exampe, in [11, hap 7 (mainy for GPS data processing) and [12 A commony used genera test statistic for mode vaidation based on inear modes is caed the generaized ikeihood ratio test statistic, which foows the hi-suare distribution A direct evauation of the standard formua for the test statistic (see (28)) may ead to arge rounding errors in the resut when any of the matrices invoved are i-conditioned (ie, have a arge ratio of maximum to minimum singuar vaue) In particuar, those methods which use the inverse of the covariance matrix Received by the editors October 3, 23; accepted for pubication (in revised form) June 23, 24; pubished eectronicay May 2, 25 Schoo of omputer Science, McGi University, Montrea, Q, anada H3A 2A7 (chang@cs mcgica, paige@csmcgica) The research of the first author was supported by NSER of anada grant RGPIN and FAR of Quebec, anada grant 21-N The research of the second author was supported by NSER of anada grant RGPIN Section of Mathematica Geodesy and Positioning, Deft University of Technoogy, Thijsseweg 11, 2629 JA Deft, The Netherands (ccjmtiberius@geotudeftn) 1916

2 OMPUTATION OF A TEST STATISTI 1917 are numericay unstabe and so can ead to arge errors with finite precision computation This was iustrated in [5, section 4 Furthermore, the formua is not defined when the noise covariance matrix is singuar To our knowedge, there has not yet been a numericay sound approach for computing this test statistic The goa of this paper is to present a numericay stabe agorithm to compute the test statistic, and the estimates of the parameter vectors as we, based on Paige s generaized inear LS method (see [5, 6, 7, 8) Our approach can hande the case when the noise covariance matrix is singuar This wi aso aow us to dea with inear modes with inear euaity constraints without atering the agorithm It appears unikey that a very i conditioned noise covariance matrix wi arise directy in standard GPS appications But constraints do appear, and these have been handed by introducing an additiona noise vector whose covariance matrix has very sma diagona eements, eading to an inexact mode whose noise covariance matrix is very i conditioned I-conditioned or singuar noise covariance matrices can, of course, appear in other appications This paper is organized as foows In section 2 we introduce the genera inear mode and derive the test statistic; then in section 3 we show the dangers of some obvious ways of computing this test statistic In section 4 we first reformuate the optimization probems for estimating the parameter vectors and give an euivaent formua for the test statistic, then present an agorithm for computing the test statistic and the estimates of the parameter vectors as we, and give reiabe representations of the error covariance matrices for the estimates The backward numerica stabiity of the agorithm is proven in section 5 In section 6 we show how to compute the test statistic and the estimates of parameter vectors when the noise covariance matrix may be singuar We aso iustrate how inear euaity constraints resut in a mode which effectivey has a singuar noise covariance matrix so that the method here can be appied directy Finay, we give a brief summary in section 7 Throughout this paper we work with reas ony R m denotes the cass of rea m-dimensiona vectors, whie R m n denotes the cass of rea m n matrices R(A) denotes the range of the matrix A The identity matrix is denoted by I and its ith coumn by e i For a matrix A, A denotes its Moore Penrose generaized inverse We write x 2 = x T x for vectors and A F = ij a2 ij for matrices We use E{ } to denote the expected vaue and cov{ } to denote the covariance, that is, cov{x} = E{(x E{x})(x E{x}) T } for a random vector x If v is a normay distributed random vector with mean v and covariance σ 2 V, we write v N( v, σ 2 V ) V is sometimes caed the cofactor matrix in geodesy; see, for exampe, [3, p 25 If a random variabe δ foows a hi-suare distribution with noncentraity parameter λ and degrees of freedom, we write δ χ 2 (, λ) 2 Linear mode and test statistic Suppose we have a inear mode (21) y = Ax + v, v N(,σ 2 V ), where y R m is a random measurement vector, A R m n with m n is a design matrix with fu coumn rank, x R n is an unknown parameter vector, and v R m is a random noise vector foowing a norma distribution with zero mean and a known symmetric nonnegative definite covariance matrix σ 2 V The above mode, however, may not propery refect reaity in a cases The mode is misspecified when, for instance, one or more measurements are corrupted by outiers They wi invaidate the resuts of estimation To safeguard against

3 1918 X-W HANG, PAIGE, AND J M TIBERIUS anomaous measurements, it is important to vaidate the data and mode used Here we restrict the misspecification to the mean of the measurement vector y, ie, an error of additive nature Suppose (21) above represents the working mode, the nu hypothesis H In a genera form the aternative hypothesis H a then reads (22) y = Ax + + v, v N(,σ 2 V ), where the known matrix R m with 1 m n specifies the type of mode error that occurred, [A, has fu coumn rank, and R is an unknown constant vector (see, for exampe, [12, section 41) One specia case eading to the so-caed w-test statistic is that in which the cofactor matrix V is the identity matrix and there is ony a possibe outier in the ith measurement, ie, = e i If we take i =1,,m, we wi have m aternative hypotheses For other specia cases that may arise from GPS appications, see [11, p 28 Initiay we assume that V is nonsinguar Foowing [12, section 41, we briefy derive the generaized ikeihood ratio test statistic for testing the nu hypothesis H against the aternative hypothesis H a Let p(y x) and p(y x, ) denote the density functions of y under H and H a, respectivey Then (23) 1 p(y x) = (2π) m 2 σ det(v ) 1 2 (24) 1 p(y x, )= (2π) m 2 σ det(v ) 1 2 [ exp 1 2σ 2 (y Ax)T V 1 (y Ax) [ exp 1 2σ 2 (y Ax )T V 1 (y Ax ) Let x be the maximum ikeihood estimator of x under H, and et {x a, a } be the maximum ikeihood estimator of {x, } under H a, respectivey, ie, p(y x ) = max p(y x) p(y x a, a ) = max p(y x, ) x We expect these vaues to be comparabe under H Obviousy x and {x a, a } are, respectivey, the soutions of the foowing two generaized inear east suares (GLLS) probems (aso referred to as weighted inear east suares probems in some iterature): x,, (25) (26) GLLS : GLLS a : min x min x, (y Ax) T V 1 (y Ax); (y Ax )T V 1 (y Ax ) The two estimators are aso the best inear unbiased estimators (BLUEs) of x under H and {x, } under H a, respectivey (see, for exampe, [4, section 32) For ater use, we define the corresponding residuas (27) r y Ax, r a y Ax a a The generaized ikeihood ratio is defined to be (see, for exampe, [12, section 41) max x p(y x) L(H, H a ) max {x, } p(y x, ) = p(y x [ ) p(y x a, a ) = exp 1 ( r T 2σ 2 V 1 r ra T V 1 ) r a, and the test statistic can be defined by (28) δ TS 2 og e L(H, H a )=σ 2 (r T V 1 r r T a V 1 r a )

4 OMPUTATION OF A TEST STATISTI 1919 For some euivaent formuas for δ TS, see [12, section 41 It can be shown that (we iustrate this in section 4; see (423)) δ TS χ 2 (, ) under H, (29) δ TS χ 2 (, λ), λ = σ 4 T T V 1 cov{r }V 1 under H a, where cov{r } = σ 2 [V A(A T V 1 A) 1 A T When δ TS is arger than a given threshod, one decides to reject the nu hypothesis H in favor of the aternative hypothesis H a Otherwise the nu hypothesis H wi be accepted The threshod is usuay determined by the reuirements of the specific appication For a discussion on this issue for genera hypothesis testing, see, for exampe, [4, section 421 and [12, section 14 3 omputationa dangers It is easy to assume that any reasonabe method can be used to compute a simpe resut ike δ TS in (28) However, this is not true, and we give a sma artificia exampe to iustrate some computationa dangers On the surface the foowing exampe ooks fairy harmess, since the 2-norm condition numbers of A and V are about 444 and 33,, respectivey: y = [ V =,A= [ ,= [ [ , , σ 2 =1 This exampe was chosen to have x =[1, 2 T and (see beow) a arge δ TS = δ δ a = 2 1 = 1, which we woud expect to be abe to compute uite accuratey We carried out computations using Matab 6 on an Appe Macintosh 45MHz PowerP G4 running OSX 126, so the foating point reative precision was ɛ Now from (25), x =(A T V 1 A) 1 A T V 1 y and r = y Ax,so δ r T V 1 r = y T V 1 y y T V 1 A(A T V 1 A) 1 A T V 1 y We computed this ast expression using the foowing Matab code: Vinv=inv(V); Atide=A *Vinv; W=Atide*A; deta_=y *(Vinv*y)-y *(Atide *(inv(w)*(atide*y))); and computed δ a ra T V 1 r a by repacing A with [A, in the above From (28) we see δ TS = δ δ a But our Matab resut was δ δ a = , an obviousy nonsensica resut This was not caused by a faut in our programming or in Matab (our computations gave reasonabe answers for a reasonabe probems) It is a simpe reminder that combining a seuence of seemingy reiabe computations does not necessariy ead to an overa numericay stabe computation The approach of computing x above, then r = y Ax, then δ r T V 1 r, and simiary for δ a, gave δ δ a = , which is meaningfu but sti in error by more than 5% The numericay stabe method we recommend in section 4 gives δ TS = and x T = [11 21 Since a numericay stabe computation gives the exact resut to a nearby probem (see section 5) we suspect that the above probem is reasonaby we conditioned and see that it is important to use a proven numericay stabe agorithm for computing δ TS This is a the more important in many rea-time appications where we might not have the uxury of IEEE standard doube precision foating point arithmetic The numerica instabiities of the above two obvious methods for δ TS were reveaed here by [A, having a 2-norm condition number of about

5 192 X-W HANG, PAIGE, AND J M TIBERIUS 4 The agorithm An obvious approach to soving probems GLLS (25) and GLLS a (26) is to transform the probems into their corresponding ordinary LS probems Suppose the symmetric positive definite V has the factorization (41) V = BB T, where B R m m is nonsinguar In some appications B itsef is given rather than V ; then computing V can resut in oss of important information and coud cause the computed V to be singuar even when B is nonsinguar; see, for exampe, [1, p 163 Even if B is not given, we can compute the hoesky factorization of V to find B Then by forming B 1 y, B 1 A, and B 1, (25) and (26) can be transformed into (42) (43) GLLS : GLLS a : min(b 1 y B 1 Ax) T (B 1 y B 1 Ax); min(b 1 y B 1 Ax B 1 ) T (B 1 y B 1 Ax B 1 ) For these ordinary LS probems, the QR factorization method or the norma euation method can then be used to compute the optima soutions and the test statistic δ TS This approach is sti often used in some practica areas But the soutions may ose accuracy unnecessariy when B is i-conditioned Another probem is that for some appications the cofactor matrix V may be singuar so that the GLLS formuations (25) and (26) (or (42) and (43)) and the test statistic δ TS in (28) are not defined Fortunatey we can use Paige s approach to soving GLLS probems (see [5, 6, 7, 8) to design a numericay stabe method to compute the test statistic and the estimates of x under H and under H a as we Our agorithm can hande a singuar cofactor matrix V, but in order to expose the ideas uicky, we sti assume that V is nonsinguar in this section and the next We wi show how to hande singuar V in section 6 With (41), for the random noise vector v in (21) and (22) we can write (44) v Bu, u N(,σ 2 I) Then the inear modes (21) and (22) can be repaced with (45) (46) y = Ax + Bu, u N(,σ 2 I) under H ; y = Ax + + Bu, u N(,σ 2 I) under H a Therefore probems GLLS in (42) and GLLS a in (43) can be reformuated as foows: (47) (48) GLLS : GLLS a : min u,x u 2 2 min u,x, u 2 2 subject to y = Ax + Bu; subject to y = Ax + + Bu Even if B is not suare or is singuar, we can sti find the optima soutions for GLLS and GLLS a as ong as the constraints in (47) and (48) are consistent (see section 6) Let u and u a denote the optima u for probems GLLS and GLLS a, respectivey Then with the assumption that B is nonsinguar (see (27)) (49) u = B 1 (y Ax )=B 1 r, u a = B 1 (y Ax a a )=B 1 r a When B = I, u and u a are just the ordinary LS residuas r and r a, respectivey So they can be caed the generaized east suares (GLS) residuas From (28), we have (41) δ TS = σ 2 ( u 2 2 u a 2 2)

6 OMPUTATION OF A TEST STATISTI 1921 Introducing decreases the suare of the GLS residua norm from u 2 2 to u a 2 2, so δ TS is a scaed measure of this decrease Note that in the formuations (47), (48), and (41), no inverse of B or V appears This is the key to the design of the numericay stabe agorithm that we now deveop First consider soving probem GLLS a in (48) Let [A, and B have the foowing generaized QR (GQR) factorization (see [9): [ UA U P T A n [A, = U, P T BQ = R A R A R A3 n (411) R R 3, m n R 3 m n where n n m n (412) P =[P A,P, P 3 R m m and Q =[Q A,Q, Q 3 R m m n m n n m n are orthogona (P 1 = P T, Q 1 = Q T ), and U A, U, R A, R, and R 3 are nonsinguar upper trianguar Mutipying the constraint euaity in (48) by P T from the eft and using QQ T = I, wehave P T y = P T Ax + P T + P T BQQ T u under H a Then by substituting (411) into the above euation and defining z A n w A n (413) z P T y z, w Q T u w z 3 m n w 3 m n, we can transform probem GLLS a in (48) into (414) min( w A w w 3 2 2) subject to z A U A U A R A R A R A3 w A z = x + U + R R 3 w (ie, under H a ) z 3 R 3 w 3 Obviousy the optima soution x a, a, w a [(wa) a T, (w) a T, (w3) a T T satisfies U A U A R A3 x a z A (415) w a A =, w a =, U R 3 a = z R 3 w3 a z 3 The upper trianguar system in (415) can easiy be soved by back substitution From (414) and (415) we have w a 3 = w 3 under H a Since w = Q T u N(,σ 2 I m ), (416) w a 3 N(,σ 2 I m n ) under H a Thus w a 3 is a vector of uncorreated residuas with respect to the mode (414), which can be regarded as the extension of the inear unbiased with scaar covariance matrix using Househoder transformations (LUSH) residuas to a inear mode with a genera noise covariance matrix For the LUSH residuas, see [1 Thus the GLS residua u a (the optima u under H a ) satisfies (see (413)) (417) u a = Qw a = Q 3 w3, a u a 2 2 = w a 2 2 = wa a w a w3 a 2 2 = w3 a 2 2

7 1922 X-W HANG, PAIGE, AND J M TIBERIUS Note that, unike the eements of w a 3, the eements of u a are correated, since from (416) and (417) cov{u a } = σ 2 Q 3 Q T 3 Now consider soving probem GLLS in (47) Note that the difference between GLLS and GLLS a is that the constraint euaity in GLLS does not have the term Thus from (414) it foows that GLLS in (47) can be transformed into (418) subject to min( w A w w 3 2 2) z A z = U A x + R A R A R A3 R R 3 w A w z 3 R 3 w 3 It foows that the optima soution x, w [(w A) T, (w ) T, (w 3) T T satisfies (419) w A =, U A R A R A3 R R 3 x w = z A z R 3 w3 z 3 This upper trianguar system can aso be soved by back substitution Note that the matrix coefficients and the right-hand sides [ of both (415) and (419) were found from the one GQR in (411) Simiar to w3, a w is a vector of uncorreated residuas, w 3 since from (418), (419), (413), and (44), [ [ w w (42) w3 = N(,σ 2 I w m n ) under H 3 Thus the GLS residua u (the optima u under H ) satisfies (421) u = Qw = Q w + Q 3 w 3, u 2 2 = w 2 2 = w w From (415) and (419) we observe that w a 3 = w 3 By using the euivaent formua of δ TS (see (41)), from (417) and (421) we obtain a resut not invoving differences: (422) δ TS = σ 2 ( u 2 2 u a 2 2)=σ 2 w 2 2 From (42) we see w N(,σ 2 I ) under H, and we know w N(,σ 2 I ); see (413), (44) ombining (419) with (414), we have under H a, [ [ [ R R 3 z w R 3 [ w w3 = z 3 = [ U + [ R R 3 R 3 so w = R 1 U + w N(R 1 U,σ 2 I ) under H a Therefore from (422) and the definition of the hi-suare distribution, we have w 3, (423) δ TS χ 2 (, ) under H ; δ TS χ 2 (, σ 2 R 1 U 2 2) under H a It can be verified from (411) that the noncentraity parameter σ 2 R 1 U 2 2 is just the λ given in (29) The above derivation eads to the foowing agorithm to compute the test statistic, the BLUE x under H, and the BLUE x a under H a

8 OMPUTATION OF A TEST STATISTI 1923 Agorithm ompute [ the GQR [ factorization [ of [A, and B (411) R R Sove 3 w z R 3 w3 = (see (419)) z 3 ompute δ TS = σ 2 w 2 2 (see (422)) If δ TS is smaer than a given threshod, accept the nu hypothesis, and sove U A x = z A R A w R A3w3 (see (419)) ese accept the [ aternative [ hypothesis, [ UA U and sove A xa za R = A3w3 U a z R 3w3 (see (415)) In the foowing we give a few comments about the above agorithm The GQR factorization (411) can be computed using Househoder transformations P and Q do not need to be formed in the computation Note that R A is not invoved in the computation Thus it is not reuired to be upper trianguar, or even suare (R A may not be suare when B is not suare; see section 6) After we compute the QR factorization of A, we appy P T to B and obtain P T P T A B (424) B = P T B P3 T B Then we determine Q by the RQ factorization: [ [ P T B R R P3 T Q = 3 B R 3 Athough both R A and R A3 are invoved in computing x, and R A3 is invoved in computing x a, we do not need to expicity compute them In fact, what we need to compute are R A w + R A3w3 and R A3w3 Note that by (411), R A w + R A3w3 = P T A BQ w, R A3w w3 3 = P T A BQ, w3 where P T A B has been computed in (424) The above computation can be performed by using ony matrix-vector mutipication Sometimes one is interested in the covariance matrices cov{x } under H and cov{x a } under H a First, we consider computing cov{x } under H From (414), without the term on the right-hand side, and (419), we obtain U A R A R A3 R R 3 R 3 x x w w w3 w 3 This shows that w 3 w 3 = and w w =,so U A (x x) =R A w A R A w A = Since w = Q T u N(,σ 2 I) (see (413) and (44)) we have (425) U A cov{x } U T A = σ 2 R A R T A

9 1924 X-W HANG, PAIGE, AND J M TIBERIUS We see that here R A is needed, athough it is not needed in computing the test statistic and the estimates x and x a If cov{x } wi be invoved in other computations, it might be usefu to have an upper trianguar R A Notice that, if R A is upper trianguar, σu 1 A R A is the hoesky factor of cov{x } omputing this, however, coud ead to oss of accuracy if U A is i-conditioned In many appications we may not need to compute cov{x } expicity In genera (425) gives the most reiabe representation of cov{x } Now we consider computing cov{x a } under H a From (414) and (415) we obtain U A U A R A3 U R 3 x a x a = R A R 3 w3 a w 3 R A R This shows that w3 a w 3 =,so [ [ [ UA U A xa x RA R = A U a R [ wa [ wa w Again since w N(,σ 2 I), we have the representation of cov {[ x a } a : [ {[ } [ T [ [ T UA U A xa UA U cov A = σ 2 RA R A RA R (426) A, U a U R R and cov{x a } can be found by soving this If we want to get the reiabe representation of cov{x a }, which is simiar to (425), we find two orthogona matrices G 1 and G 2 such that [ [ [ [ G T UA U A ŨA 1 =, G T RA R A RA (427) U 1 G Ũ A Ũ R 2 =, R A R where G 1 is a product of Givens rotations, G 2 is a product of Househoder transformations, and ŨA and R A are sti upper trianguar This computation can be impemented to make fu use of the structures of the reevant matrices for efficiency ombining (426) and (427) shows [ {[ } [ T [ [ T ŨA xa ŨA cov = σ 2 RA RA Ũ A Ũ a Ũ A Ũ R A R R A R The (1,1) bock of the two sides of this euaity gives the representation of cov{x a }: Ũ A cov{x a } Ũ T A = σ 2 RA RT A 5 Numerica stabiity of the agorithm In this section we wi prove the numerica stabiity of the computation of the test statistic δ TS, the BLUE x of x under H, and the BLUE x a under H a The proof extends the approach in [7 Let the computed trianguar matrices in the GQR factorization (411) be denoted by Û and ˆR We can show (see, for exampe, [2, Lem 193, Thm 194) that there exist orthogona P R m m and Q R m m such that (51) Û A Û A w P T [A +ΔA 1, +Δ 1 = Û, ΔA 1 F = O(ɛ) A F, Δ 1 F = O(ɛ) A F,

10 (52) OMPUTATION OF A TEST STATISTI 1925 ˆR A ˆRA ˆRA3 P T (B +ΔB 1 ) Q = ˆR ˆR3, ΔB 1 F = O(ɛ) A F, ˆR3 where ɛ is the unit roundoff, and for simpicity we use O(ɛ) instead of the product of ɛ and a constant dependent ony on the dimensions of the probem Here and throughout we ignore any O(ɛ 2 ) terms Note that z = P T y (see (413)) We can show that the computed z satisfies (see, for exampe, [2, Lem 193) ẑ = P T (y +Δy), Δy 2 = O(ɛ) y 2, and the computed soution ˆx, ŵ, ŵ3 of (419) satisfies (see [2, Thm 85) Û A +ΔU A ˆRA +ΔR A ˆRA3 +ΔR A3 ˆR +ΔR ˆR3 +ΔR 3 ˆx ŵ = ẑa ẑ, ˆR3 +ΔR 3 ŵ 3 ẑ 3 (53) ΔR A ΔR A3 ˆR A ˆRA3 ΔU A F = O(ɛ) ÛA F, ΔR ΔR 3 = O(ɛ) ˆR ˆR3 ΔR 3 F ˆR3 F For the computation of δ TS δ TS satisfies (see [2, sect 31) = σ 2 w 2 2, it is easy to show that the computed (54) ˆδ TS =(σ +Δσ) 2 ŵ 2 2, Δσ = O(ɛ)σ ombining (51), (52), (53), and (54), we can easiy verify that ˆδ TS and ˆx are the exact test statistic and the BLUE of x under H, respectivey, for the initia data ỹ y +Δy, ỹ y 2 = O(ɛ) y 2, ΔU A Ã A +ΔA 1 + P, Ã A F = O(ɛ) A F, ΔR A ΔR A3 B B +ΔB 1 + P ΔR ΔR 3, B B F = O(ɛ) B F, ΔR 3 σ σ +Δσ, σ σ = O(ɛ) σ Thus the computations of δ TS and x are numericay stabe It is peasing that ˆδ TS and ˆx are exact for initia data with the same sma perturbations Simiary, we can easiy show that the computation of the BLUE x a of x under H a is numericay stabe 6 Extension to the singuar case In this section we assume that the rank of the cofactor matrix V is k m Suppose V sti has the factorization (41), where B R m k has fu coumn rank If B is not given, we can use diagona pivoting to compute the hoesky factorization of V (see, for exampe, [1, section 222) to give (41), where the stricty upper trianguar part of B is zero, ie, b ij = for i<j,or to compute the reverse hoesky factorization of V to give (41), where b ij = for i>m k + j So we sti have modes (45) and (46)

11 1926 X-W HANG, PAIGE, AND J M TIBERIUS If y/ R([A, B), then y coud not have come from the mode (21), and this has to be checked For simpicity we assume that the two modes are consistent so that uniue optima soutions {x,u } and {x a, a,u a } exist for probems GLLS in (47) and GLLS a in (48), respectivey Now (see [5, section 3), the soution of GLLS resuts in the BLUE of any estimabe function of x under H Aso since B has fu coumn rank we have B B = I, and writing v = Bu in (21) we see that in (45) and (46), (61) u = B v N(,σ 2 I k ) From this we wi show that we can sti use the formua for the test statistic δ TS (41) Unike u and u a in (49), y = Ax + Bu, etc ead to in u = B (y Ax ), u a = B (y Ax a a ) In the foowing we wi make a seuence of orthogona transformations to compute GQR factorizations invoving A,, and B Based on these, we then discuss the computation of δ TS, x, and x a Just as in (411), we find the QR factorization of [A, : (62) P T [A, = [ UA U A U n n m n Then we appy P T to B to get B A (63) P T B = B B 3 k n m n We woud ike to transform B 3 into a nonsinguar upper trianguar matrix First, find the RQ factorization of B 3 with row pivoting: Π T 3 B 3 Q = [ (64) R3 m n, k where Π 3 is a permutation matrix, Q is orthogona, and R 3 is a fu-coumn-rank trapezoida matrix with zeros in its stricty ower eft triange Then find the QR factorizations of R 3 : [ (65) ˇP 3 T R R3 3 =, m n where ˇP 3 is orthogona and R 3 is nonsinguar upper trianguar ombining (64) and (65) eads to [ ˇP 3 T Π T (66) R3 3 B 3 Q = m n k

12 OMPUTATION OF A TEST STATISTI 1927 Defining P P I n I, Π 3 ˇP3 we then have from (62), (63), and (66) that U A U A n P T [A, = U (67), P T BQ n m n B A R A3 B R 3 R 3 k n m n Here B does not have any specia structure As in section 4, defining z A n (68) z P T y z z 3 z 4, w Q T u m n we transform probem GLLS a in (48) into (cf (414)) [ wa w 3 k, (69) subject to z A z z 3 z 4 min( w A w 3 2 2) U A U A B A = x + U + B R A3 R 3 R 3 [ wa w 3 Here the term wi disappear when we transform probem GLLS in (47) by the same transformations Note that z 4 in (69) must be a zero vector, since the origina mode (46) is consistent Obviousy the optima soution x a, a, w a [(wa) a T, (w3) a T T satisfies (cf (415)) U A U A R A3 x a z A (61) w a A =, U R 3 a = z R 3 w3 a z 3 Then it foows that the GLS residua u a (the optima u under H a ) satisfies (611) u a = Qw a, u a 2 2 = w a 2 2 = w a Now we consider soving probem GLLS in (47) From (69) we observe that GLLS can be transformed into (612) subject to min( w A w 3 2 2) z A U A B A R A3 z = x + B R 3 z 3 R 3 [ wa w 3

13 1928 X-W HANG, PAIGE, AND J M TIBERIUS Note that (612) is just (69) except that the term and z 4 rows have been removed In order to estimate x, we have to transform B into a nonsinguar upper trianguar matrix As we did to B 3 (see (66)), we can find orthogona ˇP R and ˇQ R (k ) (k ), and a permutation Π R, such that [ ˇP T Π T B (613) R p ˇQ =, p k p p where R (614) (615) is nonsinguar upper trianguar Let [ž ˇP T Π T p z z, T [Ř3 ˇP Π T R 3 p R 3 [ wa B A ˇQ [ R A,R A n, ˇQT w A k p p w p p p, k p Therefore from (612) with (613) (615) we observe that probem GLLS further transformed into can be (616) subject to min( w A w w 3 2 2) z A U A R A R A R A3 ž z = x + R Ř 3 w A R3 w w z 3 3 R 3 Notice that the bottom bock euation in (616) can uniuey determine the estimate of w 3, since R 3 is nonsinguar Since the euations in (616) are consistent, the third bock euation can be removed Then obviousy the optima soution x, w [(wa) T, (w) T, (w3) T T satisfies (617) w A =, U A R A R A3 R Ř 3 R 3 x w w 3 = Then from the second euaity in (68) and the second euaity in (615), we observe that the GLS residua u (the optima u under H ) satisfies [ w ˇQ A (618) u Q w, u I 2 w3 2 = w w3 2 2 As in section 4, w a 3 = w 3 (see (61) and (617)), so using the formua of the test statistic in (41) we have from (611) and (618) that z A ž z 3 (619) δ TS = σ 2 ( u 2 2 u a 2 2)=σ 2 w 2 2, so that once again introducing decreases the suare of the GLS residua from u 2 2 to u a 2 2, and δ TS is a scaed measure of this decrease Now we ook at the distribution of δ TS Using the fact that orthogona transformations of a unit covariance random vector do not change its covariance matrix, we see from (616) and (617) that w = w and from (615), (68), and (61) that w N(,σ 2 I p ) under H

14 OMPUTATION OF A TEST STATISTI 1929 Under H a the transformed mode has the form (69) Let (cf (614)) [Ǔ ˇP T Π T U Ũ p p ; then (69) with the z 4 rows removed can be transformed into (cf (616)) z A U A U A R A R A R A3 ž z = x + Ǔ Ũ + R Ř 3 w A (62) R3 w w z 3 3 R 3 Then from (617) and (62) we see R w =ž = Ǔ + R w,so w N(R 1 Ǔ,σ 2 I p ) under H a Thus δ TS in (619) is the desired test statistic, since (cf (423)) δ TS χ 2 (p, ) under H ; δ TS χ 2 (p, σ 2 R 1 Ǔ 2 2) under H a From (613) we see that p is the rank of B, and this can be determined in computing (613) Here we give a theoretica formua for it Suppose for P in (67) P [P 1,P 2,P 3 Thus it foows that such that [ P T 2 P T 3 BQ = B R 3 R 3 m n (621) But rank([p 2,P 3 T BQ) = rank( B ) + rank(r 3 ), rank(p T 3 BQ) = rank(r 3 )= (622) rank([p 2,P 3 T BQ) = rank((p 2 P2 T + P 3 P3 T )B), rank(p3 T BQ) = rank(p 3 P3 T B), and from the QR factorization of [A, in (67) (623) P 2 P T 2 + P 3 P T 3 = I AA, P 3 P T 3 = I [A, [A, Substituting (622) and (623) into (621), we obtain p = rank( B ) = rank((i AA )B) rank((i [A, [A, )B) Notice that when B is nonsinguar, p = rank(i AA ) rank(i [A, [A, )=(m n) (m n ) = This is the case we discussed in section 4 We coud discuss the computation of the covariance matrices for the estimates in the same way as we did in section 4 But for brevity we omit this discussion

15 193 X-W HANG, PAIGE, AND J M TIBERIUS Since the theory and agorithm here can hande a singuar noise covariance matrix, we can now easiy hande inear euaity constraints without needing to deveop a new theory or agorithm Suppose the two hypotheses are H : y = Ax + v, v N(,σ 2 V ), subject to Ex = d; H a : y = Ax + + v, v N(,σ 2 V ), subject to Ex = d, where [A, has fu coumn rank and the symmetric nonnegative definitive V has the factorization (41) with B being of fu coumn rank Then we can appy our agorithm directy to the foowing GLLS probems to find the test statistic and the estimates as we: [ [ [ GLLS : min u 2 y A B 2 subject to = x + u; d E [ [ [ [ GLLS a : min u 2 y A B 2 subject to = x + + u d E Note that [ E A, [ A E, and [ B have fu coumn rank 7 Summary The standard formua for the generaized ikeihood ratio test statistic is not suitabe for numerica computation when any of A, [A,, and the noise covariance matrix σ 2 V in (21) and (22) are i-conditioned Aso the formua is not defined when the noise covariance matrix is singuar To overcome these probems, we gave a new formua for the test statistic after reformuating the two GLLS probems for estimating the parameter vectors The new formua is mathematicay euivaent to the standard one, in which the noise covariance matrix is nonsinguar, and is we defined when the noise covariance matrix is singuar With this formua, a numericay stabe agorithm based on Paige s GLLS method was proposed to compute it The computations of the test statistic and the BLUEs of the parameter vectors are cosey connected they are obtained from the soutions of two upper trianguar inear euations, (415) and (419) We aso showed how to compute reiabe representations of the error covariance matrices for the BLUEs; see (425) and (426) In our agorithm, we did not assume that B has any specia structure If B comes from the hoesky factorization of the noise covariance matrix, then we can use the structure of B to design a faster agorithm by foowing the approach in [8 REFERENES [1 Å Björck, Numerica Methods for Least Suares Probems, SIAM, Phiadephia, 1996 [2 N J Higham, Accuracy and Stabiity of Numerica Agorithms, 2nd ed, SIAM, Phiadephia, 22 [3 B Hofmann-Weenhof, H Lichtenegger, and J oins, GPS Theory and Practice, 4th ed, Springer, New York, 1997 [4 K R Koch, Parameter Estimation and Hypothesis Testing in Linear Modes, 2nd ed, Springer, Berin, 1999 [5 S Kouroukis and Paige, A constrained east suares approach to the genera Gauss- Markov inear mode, J Amer Statist Assoc, 76 (1981), pp [6 Paige, Numericay stabe computations for genera univariate inear modes, omm Statist B Simuation omput, 7 (1978), pp [7 Paige, omputer soution and perturbation anaysis of generaized inear east suares probems, Math omp, 33 (1979), pp [8 Paige, Fast numericay stabe computations for generaized inear east suares probems, SIAM J Numer Ana, 16 (1979), pp

16 OMPUTATION OF A TEST STATISTI 1931 [9 Paige, Some aspects of generaized QR factorizations, in Reiabe Numerica omputation, M G ox and S Hammaring, eds, Oxford University Press, New York, 199, pp [1 G P H Styan, LUSH and other uncorreated residuas from inear modes, anad J Statist, 1 (1973), pp [11 P J Teunissen, Quaity contro and GPS, in GPS for Geodesy, 2nd ed, P J Teunissen and A Keusberg, eds, Springer, Berin, 1998, pp [12 P J G Teunissen, Testing Theory: An Introduction, Deft University Press, Deft, 2