Some Properties of a Recursive Procedure for High Dimensional Parameter Estimation in Linear Model with Regularization

Transcription

1 Open Journa of Statstcs, 4, 4, 9-93 Pubshed Onne December 4 n ScRes. Some Propertes of a Recurse Procedure for Hgh Dmensona Parameter Estmaton n Lnear Mode wth Reguarzaton Hong Son Hoang, Remy Barae SHOM/HOM/REC, 4 a Gaspard Coros 357 ououse, France Ema: hhoang@shom.fr Receed 8 September 4; resed 6 October 4; accepted Noember 4 Copyrght 4 by authors and Scentfc Research Pubshng Inc. hs work s censed under the Create Commons Attrbuton Internatona Lcense (CC BY). Abstract heoretca resuts reated to propertes of a reguarzed recurse agorthm for estmaton of a hgh dmensona ector of parameters are presented and proed. he recurse character of the procedure s proposed to oercome the dffcutes wth hgh dmenson of the obseraton ector n computaton of a statstca reguarzed estmator. As to dea wth hgh dmenson of the ector of unknown parameters, the reguarzaton s ntroduced by specfyng a pror non-negate coarance structure for the ector of estmated parameters. Numerca exampe wth Monte-Caro smuaton for a ow-dmensona system as we as the state/parameter estmaton n a ery hgh dmensona oceanc mode s presented to demonstrate the effcency of the proposed approach. Keywords Lnear Mode, Reguarzaton, Recurse Agorthm, Non-Negate Coarance Structure, Egenaue Decomposton. Introducton In [] a statstca reguarzed estmator s proposed for an optma near estmator of unknown ector n a near mode wth arbtrary non-negate coarance structure z = Hx () where z s the p-ector obseraton, H s the ( p n) parameters to be estmated, s the p-ector representng the obseraton error. It s assumed obseraton matrx, x s the n-ector of unknown How to cte ths paper: Hoang, H.S. and Barae, R. (4) Some Propertes of a Recurse Procedure for Hgh Dmensona Parameter Estmaton n Lnear Mode wth Reguarzaton. Open Journa of Statstcs, 4,

2 H. S. Hoang, R. Barae, E = E = V () E x = x, E ee = M, E e = N, e : = x x (3) where E ( ) s the mathematca expectaton operator. hroughout ths paper et p, n be any poste nte- gers, the coarance matrx of the jont ector = ( e, ) may be snguar (and hence the mode ()-(3) s caed a near mode wth arbtrary non-negate coarance structure). No partcuar assumpton s made regardng the probabty densty functon of and p, n are any poste ntegers. In [] the optma near estmator for the unknown ector x n the mode ()-(3) s defned as, xˆ = Gz, G = H I V V, V = V I H H (4) where V s the transpose of V, H denotes the pseudo-nerson of H. As n practce a the matrces H, M, N, R and the obseraton ector z are gen ony approxmatey, D= HM,, NRz,, we are gen ther δ -approxmatons nstead of data set δ hence the resutng estmate xˆ xˆ ( D ) (,,,, ) D δ = H δ M δ N δ R δ z δ (5) δ =. δ As shown n [], there are stuatons when the error e: = xˆ xˆ between the true estmate ˆx and ts perturbed ˆx δ may be ery arge een for sma data error δ. he reguarzaton procedure has been proposed n [] to oercome ths dffcuty. In ths paper we are nterested n obtanng a smpe recurse agorthm for computaton of ˆx subject to the stuaton when n or p or/and n, p may be ery hgh. hs probem s ery mportant for many practca appcatons. As exampe, consder data assmaton probems n meteoroogy and oceanography []. For typca data set n oceanography, at each assmaton nstant we 4 5 p, hae the obseraton ector wth compute the estmate (4) usng standard numerca methods. We w show that there exsts a smpe agorthm to oercome these dffcutes by expotng a recurse character of the agorthm wth an approprate reguarzaton n the form of the pror coarance matrx M for the ector of unknown parameters x. 6 7 n. Under such condtons t s unmagnabe to. Smpe Recurse Method for Estmatng the Vector of Parameters.. Probem Statement: Free-Nose Obseratons Frst consder the mode () and assume that =. We hae then the system of near equatons z = Hx, z R p, n x R (6) p and z R for the nose-free obseratons z. Suppose that the system (6) s compatbe,.e. there exsts x such that Hx z. In what foows et (,, ) = p, H ( h,, ) hp z z z =,.e. z s the th component of z, h s the th row-ector of H. he probem s to obtan a smpe recurse procedure to compute a souton of the system (6) when the dmenson of z s ery hgh... Iterate Procedure o fnd a souton to Equaton (6), et us ntroduce the foowng system of recurse equatons [ ] x = x K z h x, x = x, =,, p (7a) K = M h h M h (7b) M = M K h M, =,,, M s gen (7c) Menton that the system s compatbe f z R[ H] where [ ] R H s a near space spanned by the coumns 9

3 H. S. Hoang, R. Barae of H. hroughout of the paper, for defnteness, et h, =,, p. heorem. Suppose the system (6) s compatbe. hen for any fnte x and symmetrc poste defnte (SPD) matrx M we hae Hxp = z. In order to proe heorem we need Lemma. he foowng equates hod Proof. By nducton. We hae for j =, hm =, =,, j (8) j hm = hm hkhm = snce hk = hm h hm h =. Let the statement be true for some < < p. We show now that t s true aso for. As the statement s true for, t mpes hm =, =,,. We hae to proe hm =, =,, Substtutng M = M K h M nto hm, takng nto account the form of K one sees that as hm =, =,, t mpes hm =, =,, (End of Proof). Lemma. he foowng equates hod hx Proof. By nducton. We hae for = z. hx = z, =,, j (9) j j =, hx = h x K ( z hx ). As hk =, t s edent that Let the statement be true for some < < p. We show now that t s true aso for. As the statement s true for, t mpes hx = z, =,,. We hae to proe hx = z, =,,. From the defnton of x, Howeer from Lemma, as hx = h x K z h x = z hk z h x hm =, [ ] hk = hm h M h M = for a. (End of Proof). Proof of heorem. heorem foows from Lemma snce under the condtons of heorem, from Equaton (9) for j = p t foows hx p = z, =,, p or Hxp = z. (End of Proof). Coroary. Suppose the rows of H : = h,, h are neary ndependent. hen under condtons of heorem, rank [ ] M = n. Proof. By nducton. he fact that for =, hm = mpes at east the nu subspace of M has one nonzero eement hence dm R( M ) n. We show now that t s mpossbe that R( M ) Suppose that dm R( M ) n n, n =,3, For smpcty, et a such that any eement from the subspace neary ndependent ectors,, an on the bass of these n eements. As to the matrx M : = M h hm hm h ts subspace R( M ) has the dmenson hence any eement from R( M ) b. hus any eement from the subspace : dm < n. n =. It means that there exst n R M can be represented can be represented on the bass of some ector R M wth M = M M can be represented as a near combnaton of n eements a,, an, b. On the other hand, as M M = M s non-snguar, any eement of R( M ) must be represented as a near combnaton of n neary ndependent ectors. It contradcts the fact that any eement from R( M ) coud be wrtten as a near combnaton of n neary ndependent eements. We concude that t s mpossbe dm R( M ) < n hence dm R( M ) = n. he same argument s true for n = 3, 4,, n. Suppose now Coroary s true for and we hae to proe that t hods for : =. For 93

4 H. S. Hoang, R. Barae :, M = Mh h M h Mh M = M M hm =, =,, hence (as a the rows of H are neary ndependent). It fo- we hae dm R ( M ) = n, dm[ M ] =. From Lemma t foows the nu subspace N[ M ] has the dmenson ows that the dmenson of the subspace RM [ ] s at east ess or equa to n. By the same way as proed for = one can show that t s mpossbe dm R M = n (End of Proof). dm R M < n hence Comment. By erfyng the rank of M, Coroary aows us to check f the computer code s correct. In partcuar f H s non-snguar, at the end of the terate procedure the matrx M n shoud be zero. he recurse Equatons (7a)-(7c) then yed the unque souton of the equaton z = Hx after n teratons. Usng the resut (8) n Lemma, t s easy to see that: Coroary. Suppose h s neary dependent on h,, h. hen n (7), M = M. Coroary 3. Suppose h,, h are neary ndependent, h s neary dependent on h,, h. hen under the condtons of heorem, rank R( M ) = n. Coroary 3 foows from the fact that when h s neary dependent on h,, h from Coroary, M = M and hence by Coroary, rank R( M ) = n. Comment. Equaton (7c) for M s smar to that gen n (3.4.) n [3] snce from Coroary t foows automatcay M = M f h s neary dependent on h,, h. her dfference s that n (7c) the nta M may be any SPD matrx whereas n (3.4.) of [3] t s assumed M = I. In the next secton we sha show that the fact M may be any SPD matrx s mportant to obtan the optma n mean squared estmator for x. 3. Optma Propertes of the Souton of (7) 3.. Reguarzed Estmate heorem says ony that Equatons (7a)-(7c) ge a souton to (). We are gong now to study the queston on whether a souton of Equatons (7a)-(7c) s optma, and f yes, n what sense? Return to Equatons ()-(3) and assume that V =, N=. he optma estmator (4) s then gen n the form ˆx x MH HMH z Hx = where H s the pseudo-nerson of H. Consder frst the equaton z = hx. In ths case, () yeds the optma estmator (obtaned on the bass of the frst obseraton) = ˆx x Mh h Mh z h x and one can proe aso that the mean square error for ˆx s equa to = M M Mh h Mh h M If we appy Equaton (3.4.) n [3], M = I then nstead of ˆx we hae () (a) (b) xˆ = h hh z = h z, M = I h hh h () For smpcty, et x =. Comparng Equaton () wth Equaton () shows that f ˆx s the orthogona projecton of x onto the subspace spanned by h, the estmate ˆx beongs to the subspace spanned by M. hus the agorthm () takes nto account the fact that we known a pror x beongs to the space RM [ ]. hs fact s ery mportant when the number of obseratons p s much ess than the number of the estmated parameters n as t happens n oceanc data assmaton: today usuay p =, n = In [4] a smar queston has been studed whch concerns the choce of adequate structure for the Error Coarance Matrx (ECM) M. We proe now a more strong resut sayng that a the estmates x, =,, are projected onto the space RM [ ]. heorem. Consder the agorthm (7). Suppose x R M. hen a the estmates x, =,, beong to 94

5 H. S. Hoang, R. Barae the space spanned by the coumns of M,.e. x R M. Proof. For = the statement s edent as shown aboe. Suppose the statement s true for some. We w show that t s true aso for : =. Reay as x R M, t s suffcent to show that K ζ R M, ζ : = z h x. From x R M, x = x Kζ, as x R M t foows that K ζ R M. But coumns of ζ = ζ K Mh hmh hence the M must beong to R M. Agan from the equaton for M n Equaton (7) we hae M R M. It proes K ζ R M snce M R M (End of proof). heorem says that by specfyng M R M the agorthm (7) w produce the estmates beongng to the subspace R M. Specfcaton of the pror matrx M pays the most mportant task f we want the agorthm (7) to produce the estmate wth hgh quaty. Comment 3 ) If x does not beong to R M, by consderng e : = x x n pace of x heorem 3 remans ad for the agorthm (7) wrtten for e. In ths case at the frst teraton, as e: = x x represents the pror error for x, t s natura for the agorthm to seek the correcton (.e., the estmate for the error e: = x x ) beongng to the space R M. In what foows, uness otherwse stated, for smpcty we assume x =. ) heorem says that there s a possbty to reguarze the estmate when the number of obseratons s ess than the number of estmated parameters by choosng M = M. hus the agorthm can be consdered as that whch fnds the souton Hx = z under the constrant x R M. In the procedure n Abert (97) puttng M = I means that there s no constrant on x hence the best way to do s to project x orthogonay onto subspace of R H. 3.. Mnma Varance Estmate Suppose x s a random arabe hang the mean x and coarance matrx M. We hae then the foowng resut. heorem 3. Suppose x s a random arabe hang the mean x and the coarance matrx M. hen x generated by the recurse Equaton (7) s an unbased and mnmum arance estmate for x n the cass of a unbased estmates neary dependent on x and z,, z. Proof. Introduce for the system (6), and the cass of a estmates or he condton for unbasedness of x( AB, ) from whch foows A I BH z : = z,, z, H : = h,, h (3) x neary dependent on x and z,, z s (, ) x A B = Ax Bz (, ) ( ). E x A B = E Ax Bz = x A BH x = x x R, =. Substtutng ths reaton nto x ( A, B) Ax Bz x ( B) = x B z Hx It means that a the estmate n Equaton (4) s unbased. Consder the mnmzaton probem We hae = trace argmn, : = J B E ee e x x B B n = eads to (4) 95

6 H. S. Hoang, R. Barae E ee = E x x B z H x x x B z H x = M MH B BH M BH MH B,, akng the derate of J( B ) wth respect to B mpes the foowng equaton for fndng B, from whch foows one of the soutons BH MH = MH,,,, B = MH H MH If now nstead of Equaton (3) we consder the system, z= Hx, E = α I, E x x = and repeat the same proof, one can show that the unbased mnmum arance estmate for x s gen by = xˆ α x Kα z Hx,, K α = MH HMH αi Usng the propertes of the pseudo-nerse (heorem 3.8 [3]), one can proe that hus for a ery sma x B. α > the estmate ( α ) m K B ˆ (5) (6a) (6b) α = (7) x α s unbased mnmum arance whch can be made as cose as possbe to ( ) On the other hand, appyng Lemma n [5] for the case of uncorreated sequence { } xˆ ( α) = xˆ ( α) k ( α) z h xˆ ( α) Lettng α one comes to x 3.3. Nosy Obseratons, one can show that, (8a) k α = M α h h M α h α, (8b) ˆ ( α) ( α) ( α) ( α) M = M k h M (8c) M α x n (7) (End of proof). α = M (8d) he agorthm (8a)-(8d) thus yeds an unbased mnma arance (UMV) estmates for x n the stuaton when α represents the obseraton nose arance. We want to stress that these agorthms produce the UMV estmates ony f M = M where M s the true coarance of the error e: = x x before arrng z, =,, Very Hgh Dmenson of x : Smpfed Agorthms In the fed of data assmaton n meteoroogy and oceanography usuay the state ector x s of ery hgh 6 7 dmenson, the orders of []. hs happens because x s a coecton of seera arabes defned n the three dmensona grd. If the agorthm (7) aows to oercome the dffcutes wth the hgh dmenson of the obseraton ector z (each teraton noes one component of z ), due to hgh dmenson of x, t s 4 mpossbe to hande Equaton (7c) to eauate the matrces M (wth the number of eements ). hs secton s deoted to the queston on how one can oercome such dffcutes. Let us consder the egen-decomposton for M [6] M In (9) the coumns of U are the egenectors of M and D s dagona wth the eements λ λ λn at the dagona the egen-aues of M. Let U = [ U, U], D = dag [ D, D]. If we put n the agorthms (7) or (8) M = U DU, then the agorthm (7), for exampe, w yed the best estmate for x = UDU (9) 96

7 projected n the subspace RU ( ). Let U U( m) = has the dmenson n m, m n. Let M m = U md mu m Man heoretca Resuts heorem 4 Consder two agorthms of the type (7) subject to two matrces where the coumns of nequates hod M = M m = U m D m U m, =, U m consst of H. S. Hoang, R. Barae m eadng egenectors of M, m m. hen the foowng E e m ˆ E e m, m m, e m = x m x, =, xˆ ( m ) s the estmate produced by the agorthm (7) subject to M M ( m ) : () =, where the strct nequaty takes pace f λ m >. Proof Wrte the representaton of x n the terms of decomposton of M on the bass of ts egenectors (for smpcty, et x = ), x UD y Ly L UD M UDU = =, : =, = () where y s of zero mean and has the coarance matrx I. Let y s a sampe of y. heorem 3 states that for a x : = Ly, the agorthm (7) w yed the estmate wth the mnma arance. In what foows we ntroduce the notaton: M = UDU the true ECM of x ; M m = U mdmu m a truncated coarance comng from M ; M the ntazed ECM n the agorthm (7a)-(7c). he sampes x of x are comng from a arabe hang zero mean and coarance M. x ( m ) sampe comng from a arabe hang zero mean and coarance M ( m ). here are the foowng dfferent cases ) M = M : By heorem 3 the agorthm (7) w produce the estmates of mnma arance for a x m. ) M = M ( m), m< n, λ m > : a) For sampes beongng to R U ( m) : he estmates w be of mnma arance. s true aso f Equaton (7) s apped to n b) For sampes beongng to R R U( m) w not be of mnma arance. hus n the mean sense (.e. beongng to n,, ˆ R but not to R U( m) E e n E e m m n e m = x m x M M m M = M m, m m n, we hae a) M = M ( m ) : ) x R M ( m ) : the estmates are of mnma arance; 3) Consder two ntazatons = and n ) x R R M ( m ) b) M M ( m ) ) of mnma arance for x R M ( m ) : the estmates are not of mnma arance. = : he agorthm (7) w produce the estmates ; ) of mnma arance for x R M ( m ) R M ( m ) ; x ; hs ): he estmates () λ m >. In the same way 97

8 H. S. Hoang, R. Barae ) not of mnma arance for x R M ( n) R M ( m ) hus n the mean sense. E e m ˆ E e m, m m, e m = x m x, =, (3) Smpfed Agorthm heorem 5 =, m n. Consder the agorthm (7) subject to M : = M ( m), M( m) U( mdmu ) ( m) hen ths agorthm can be rewrtten n the form m x = U m xe, xe R, (4a) xe = xe Ke z he xe, x = x, =,,, p (4b) = ( ) ( ) ( ) ( ) Ke Me he he Me he, (4c) ( ), h = h U m M = D m (4d) e e m It s seen that n the agorthm (4a)-(4d), the estmate xe R beongs to the near space of dmenson m. In data assmaton, t happens that the dmenson of x may be ery hgh but there s ony some eadng drectons (eadng egenectors of M representng the drectons of most rapd growth of estmaton error) to be captured. hus the agorthm (4a)-(4d) s qute adapted for song such probems: frst the nta coarance M s constructed from physca consderatons or numerca mode, and next to decompose t (numercay) to obtan an approxmated decomposton M m = U mdmu m, m n Menton that the erson (4a)-(4d) s ery cosed to that studed n [7] for ensurng a stabty of the fter. 4. Numerca Exampe Consder the system () subject to the coarance M, M = Here we assume that the st and 3rd components of x s obsered,.e. Numerca computaton of egendecomposton of z = Hx, H = M = UDU yeds E 6.66 U = [ u, u, u3] = E E E 5 (5) (6) (7) D = dag (.5,.64,.6) (8) he agorthm (4a)-(4d) s apped subject to three coarance matrces M M( m) U( mdmu ) ( m) = =, m =,, 3. hey are denoted as ALG(m). In Fgure we show the numerca resuts obtaned from the Monte-Caro smuaton. here are sampes smuatng the true x whch are generated by a random generator dstrbuted accordng to the norma dstrbuton N(./ xm, ). he cures n Fgure represent rms of the estmaton error e= x xˆ obtaned by dfferent agorthms. Here the cures m =, m =, m = 3 correspond to the three agorthms ALG(), ALG(), ALG(3). 98

9 H. S. Hoang, R. Barae Fgure. Performance (rms) of the agorthm (4) subject to dfferent projecton subspaces. he cure Id denotes the 4th agorthm ALG (4) whch s run subject to M = I dentty matrx. hs s equaent to the orthogona projecton (usng the pseudo-nerson of H ) of z nto the subspace R H. As seen from Fgure, the estmaton error s hghest n ALG (). here s practcay no dfference between ALG () and ALG (3) whch are capabe of decreasng consderaby the estmaton error (5%) compared to ALG (). As to the ALG (4), ts performance s stuated between ALG () and ALG (). hs experment confrms the theoretca resuts and demonstrates that f we are gen a good pror nformaton on the estmated parameters, there exsts a smpe way to mproe the quaty of the estmate by appropratey ntroducng the pror nformaton n the form of the reguarzaton matrx M. he resuts produced by ALG () and ALG (4) show aso that when the pror nformaton s nsuffcenty rch, the agorthm naturay produces the estmates of poor quaty. In such stuaton, smpe appyng orthogona projecton can yed a better resut. For the present exampe, the reason s that usng the nd mode u aows to capture the mportant nformaton contaned n the second obseraton. Ignorng t (as does ALG ()) s equaent to gnorng the second obseraton z. As to the thrd mode u 3, t has a weak mpact on the estmaton snce the correspondng egenaue λ3 s sma. hat expans why ALG () and ALG (3) hae produced amost the same resuts. 5. Experment wth Oceanc MICOM Mode 5.. MICOM Mode In ths secton we w show mportance of the reguarzaton factor n the form of the pror coarance M n the desgn of a fter for systems of ery hgh dmenson. he numerca mode used n ths experment s the MICOM (Mam Isopycnc Coordnate Ocean Mode) whch s exacty as that presented n [8]. We reca ony that the mode confguraton s a doman stuated n the North Atantc from 3 N to 6 N and 8 W to 44 W. he grd spacng s about. n ongtude and n attude, requrng the horzonta mesh =,,4 ; j =,,8. he dstance between two ponts x = x x km, y = y y km. he number of ayers n the mode n = 4. We note that the state of the mode x: = ( hu,, ) where h= h (, j, ) s the thckness of the th ayer, u= u(, j, ), = (, j, ) are two eocty components. he true ocean s smuated by runnng the mode from cmatoogy durng two years. Each ten days the sea-surface heght (SSH) are stored at the grd ponts o =,,,4 ; j o =,,,8 whch are consdered as obseratons n the assmaton experment. he sequence of true states w be aaabe and aows us to compute the estmaton errors. hus the obseraton operator H s constant at a assmaton nstants. he assmaton experment conssts of usng the SSH to correct the mode souton, whch s ntazed by some arbtrary chosen state resutng from the contro run. 5.. Dfferent Fters he dfferent fters are mpemented to estmate the oceanc crcuaton. It s we known that determnng the 99

10 H. S. Hoang, R. Barae fter gan s one of the most mportant tasks n the desgn of a fter. As for the consdered probem t s mpossbe to appy the standard Kaman fter [9] snce n the present experment, n = O( ) and the number of 6 eements n the ECM s of order O( n ). At each assmaton nstant, the estmate for the system state n a fters s computed n the form () wth the correspondng ECM M. As the number of obseratons p = 5 s argey nferor to the dmenson of the system state, the choce of M as a reguarzaton factor has a great mpact on the quaty of the produced estmates. In ths assmaton experment the foowng fters w be empoyed. Frst the Predcton Error Fter (PEF) whose ECM s obtaned on the bass of eadng rea Schur ectors [8]. Paraey two other fters, one s the Cooper-Hanes fter (CHF) [] and another s an EnOI (Ensembe based Optma Interpoaton) fter [] w be used. Menton that the ECM n the CHF s obtaned on the bass of the prncpe of a ertca rearrangement of water parces (see aso [8]). he method conseres the water masses and mantans geostrophy. he man dfference between PEF and EnOI s yng n the way to generate the ensembes of Predcton Error (PE) sampes. In the PEF, the ensembe of PE sampes s generated usng the sampng procedure descrbed n [8] (and t w be denoted as En(PEF)). As for the EnOI, the ensembe of background errors sampes (the term used n [8]) and w be denoted by En(EnOI)) w be used. he eements of En(EnOI) are constructed accordng to the method n []. It conssts of usng -year mean of true states as the background fed and the error sampes are cacuated as dfferences between nddua -day true states durng ths perod and the background. Accordng to Coroary 4. n [4], usng the hypothess on separabe ertca-horzonta structure for the ECM, we represent M = M Mh where M, M h are the ECM of ertca and horzonta arabes respectey. In the case of sea-surface heght obseratons, from the representaton M = M Mh, the gan fter can be represented n the form [ ] K = K K, K = k, k, k, k h 3 4 where denotes the Kronecker product. he gan K aows the correcton aaabe at the surface to propagate nto a ertca subsurface ayers. As to K h, t represents an operator of (horzonta) optma nterpoaton whch nterpoates the obseratons oer a horzonta grd ponts at the surface. Menton that the eements of M and the correaton ength parameter n M h are estmated by mnmzng the mean dstance between the data matrx M d and M usng a smutaneous perturbaton stochastc approxmaton agorthm []. he data matrx M d s obtaned from sampes of the eadng rea Schur ectors as descrbed n [8]. Fgure shows the estmated ertca coeffcents k, =,,3,4 obtaned on the bass of the ECM spanned by the eements of En(PEF). It s seen that the estmates conerge qute qucky. he estmated ertca gan coeffcents K [ ] = k, k, k3, k4 computed on the bass of the ECM from two ensembes En(PEF), En(EnOI) at the teraton t = 7 are pef [ 44.59, 9.53, 34.44, 8.] K = (9a) eno [ 34.4, 7.53, 3.3,.] K = (9b) eno We remark that a the gan coeffcents n two fters are of dentca sgn but the eements of K are of pef much ess magntudes than that of K. It means that the EnOI w make ess correcton (compared to the PEF) to the forecast estmate. wo gans n (9a), (9b) w be used n the two fters PEF and EnOI to assmate the obseratons. he ertca gan coeffcents for the CHF are taken from [8] and are equa to 5.3. Numerca Resuts chf [ 85.97,,, 84.97] K = (3) In Fgure 3 we show the nstantaneous arances of the SSH nnoaton produced by three fters EnOI, CHF and PEF. It s seen that ntazed by the same nta state, f the nnoaton arances n EnOI, CHF hae a tendency to ncrease, ths error remans stabe for the PEF durng a assmaton perod. At the end of assmaton, the PE n the CHF s more than two tmes greater than that produced by the PEF. he EnOI has produced poor estmates, wth error about two tmes greater than the CHF has done. For the eocty estmates, the same tendency s obsered as seen from Fgure 4 for the surface eocty PE 93

11 H. S. Hoang, R. Barae Fgure. Vertca gan coeffcents obtaned durng appcaton of the Sampng Procedure for ayer thckness correcton durng teratons. Fgure 3. Performance comparson of EnOI, CHF and PEF: Varance of SSH nnoaton resutng from the fters EnOI, CHF and PEF. Fgure 4. he predcton error arance of the u eocty component at the surface (cm/s) resutng from the EnOI, CHF and PEF. errors. hese resuts proe that the choce of ECM as a reguarzaton factor on the bass of members of the En(PEF) aows to much better approach the true system state compared to that based on the sampes taken from En(EnOI) or to that constructed on the bass of the physca consderaton as n the CHF. he reason s that the members of En(PEF) by constructon [8] are sampes of the drectons aong whch the predcton error ncreases 93

12 H. S. Hoang, R. Barae most rapdy. In other words, the correcton n the PEF s desgned to capture the prncpa mportant components n the decomposon of the coarance of the predcton error. 6. Concuson We hae presented some propertes of an effcent recurse procedure for computaton of a statstca reguarzed estmator for the optma near estmator n a near mode wth arbtrary non-negate coarance structure. he man objecte of ths paper s to obtan an agorthm whch aows oercomng the dffcutes concerned wth hgh dmensons of the obseraton ector as we as that of the estmated ector of parameters. As t was seen, the recurse nature of the proposed agorthm aows deang wth hgh dmenson of the obseraton ector. By ntazaton of the assocated matrx equaton by a ow rank approxmaton coarance whch accounts for ony frst eadng components of the egenaue decomposton of the pror coarance matrx, the proposed agorthm permts to reduce greaty the number of estmated parameters n the agorthm. he effcency of the proposed recurse procedure has been demonstrated by numerca experments, wth the systems of sma and ery hgh dmenson. References [] Hoang, H.S. and Barae, R. (3) A Reguarzed Estmator for Lnear Regresson Mode wth Possby Snguar Coarance. IEEE ransactons on Automatc Contro, 58, [] Daey, R. (99) Atmospherc Data Anayss. Cambrdge Unersty Press, New York. [3] Abert, A. (97) Regresson and the Moore-Penrose Pseudo-Inerse. Academy Press, New York. [4] Hoang, H.S. and Barae, R. (4) A Low Cost Fter Desgn for State and Parameter Estmaton n Very Hgh Dmensona Systems. Proceedngs of the 9th IFAC Congress, Cape own, 4-9 August 4, [5] Hoang, H.S. and Barae, R. () Approxmate Approach to Lnear Fterng Probem wth Correated Nose. Engneerng and echnoogy, 5, -3. [6] Goub, G.H. and an Loan, C.F. (996) Matrx Computatons. 3rd Edton, Johns Hopkns Unersty Press, Batmore. [7] Hoang, H.S., Barae, R. and aagrand, O. () On the Desgn of a Stabe Adapte Fter for Hgh Dmensona Systems. Automatca, 37, [8] Hoang, H.S. and Barae, R. () Predcton Error Sampng Procedure Based on Domnant Schur Decomposton. Appcaton to State Estmaton n Hgh Dmensona Oceanc Mode. Apped Mathematcs and Computaton, 8, [9] Kaman, R.E. (96) A New Approach to Lnear Fterng and Predcton Probems. Journa of Basc Engneerng, 8, [] Cooper, M. and Hanes, K. (996) Atmetrc Assmaton wth Water Property Conseraton. Journa of Geophysca Research,, [] Greensade, D.J.M. and Young, I.R. (5) he Impact of Atmeter Sampng Patterns on Estmates of Background Errors n a Goba Wae Mode. Journa of Atmospherc and Oceanc echnoogy, [] Spa. J.C. () Adapte Stochastc Approxmaton by the Smutaneous Perturbaton Method. IEEE ransactons on Automatc Contro, 45,

13