Incorporating Ensemble Covariance in the Gridpoint Statistical Interpolation Variational Minimization: A Mathematical Framework

Transcription

1 2990 M O N T H L Y W E A T H E R R E V I E W VOLUME 38 Incorporting Ensemble Covrince in the Gridpoint Sttisticl Interpoltion Vritionl Minimiztion: A Mthemticl Frmework XUGUANG WANG School of Meteorology, University of Oklhom, nd Center for Anlysis nd Prediction of Storms, Normn, Oklhom (Mnuscript received 7 October 2009, in finl form 2 Februry 200) ABSTRACT Gridpoint sttisticl interpoltion (GSI), three-dimensionl vritionl dt ssimiltion method (3DVAR) hs been widely used in opertions nd reserch in numericl wether prediction. The opertionl GSI uses sttic bckground error covrince, which does not reflect the flow-dependent error sttistics. Incorporting ensemble covrince in GSI provides nturl wy to estimte the bckground error covrince in flowdependent mnner. Different from other 3DVAR-bsed hybrid dt ssimiltion systems tht re preconditioned on the squre root of the bckground error covrince, commonly used GSI minimiztion is preconditioned upon the full bckground error covrince mtrix. A mthemticl derivtion is therefore provided to demonstrte how to incorporte the flow-dependent ensemble covrince in the GSI vritionl minimiztion.. Introduction Gridpoint sttisticl interpoltion (GSI; Wu et l. 2002; Kleist et l. 2009), three-dimensionl vritionl dt ssimiltion method (3DVAR) hs been widely used in opertions nd reserch in numericl wether prediction (NWP). GSI is cpble of ingesting lrge vriety of tmospheric observtions nd hs developed cpbilities for dt thinning, qulity control, nd stellite rdince bis correction. The opertionl GSI uses sttic bckground error covrince, which does not reflect the flow-dependent error sttistics. The nisotropic recursive filter (e.g., Purser et l. 2003,b) provides the possibility to introduce nisotropic nd inhomogeneous covrinces. How to determine the prmeters used in the nisotropic recursive filter to relisticlly represent vrying bckground error covrince, however, still remins n issue. A four-dimensionl vritionl dt ssimiltion system (4DVAR) implicitly includes time-evolving covrince model through the evolution of initil errors Corresponding uthor ddress: Dr. Xugung Wng, School of Meteorology, University of Oklhom, 20 Dvid L. Boren Blvd., Normn, OK E-mil: xugung.wng@ou.edu under tngent liner dynmics (Lorenc 2003). However, the evolved, flow-dependent covrince model my still be limited by the usge of sttic covrince model t the beginning of ech 4DVAR cycle (Buehner et l. 200,b). On the other hnd, the ensemble Klmn filter (EnKF) hs been extensively explored s n lterntive dt ssimiltion method (e.g., Houtekmer et l. 2005; Whitker et l. 2008; Szunyogh et l. 2005; Liu et l. 2008; Torn nd Hkim 2008; Zhng et l. 2009; Aksoy et l. 2009; Jung et l. 2008). The min dvntge of the EnKF is tht it cn conveniently provide, through ensemble covrince, flow-dependent estimte of the bckground error covrince, nd therefore the bckground forecst nd the observtions re more ppropritely weighted during the ssimiltion. A hybrid method ws proposed (e.g., Hmill nd Snyder 2000; Lorenc 2003; Zupnski 2005; Wng et l. 2007) nd hs been developed nd implemented recently for rel NWP models (e.g., Buehner 2005; Wng et l. 2008,b; Buehner et l. 200,b). The hybrid method combines the dvntges of both the vritionl method nd the EnKFs nd therefore demonstrtes gret potentil for future reserch nd opertions. In the hybrid method, the vritionl frmework is used to conduct dt ssimiltion within which the flow-dependent ensemble covrinces re effectively incorported to DOI: 0.75/200MWR3245. Ó 200 Americn Meteorologicl Society

2 JULY 200 N O T E S A N D C O R R E S P O N D E N C E 299 estimte the bckground error covrince. Studies using simple models nd full NWP models hve demonstrted the potentil of the hybrid method s compred with stndlone vritionl method (VAR; e.g., Hmill nd Snyder 2000; Wng et l. 2007b, 2008,b, 2009; Buehner et l. 200,b). Recent studies lso suggest potentil dvntges of the hybrid method in comprison with stnd-lone EnKF. For exmple, Wng et l. (2007b, 2009) suggested tht for lrge-scle ppliction the hybrid EnKF 3DVAR ws more robust thn the EnKF for smll ensemble size nd lrge model errors since the sttic covrince used in 3DVAR helped to reduce the smpling errors, which indictes smller ensemble size my be needed by the hybrid thn by the EnKFs to chieve similr performnce. This is prticulrly ttrctive for opertionl forecsts for which the size of the ensemble my be constrined by the vilbility of the computtionl resources nd the timing of the forecsts. Studies by Buehner et l. (200,b) suggested the 4DVAR-bsed hybrid is better thn the stnd-lone EnKF, likely becuse the temporl evolution of the bckground error covrince ws in spce with much higher dimension by the 4DVAR-bsed hybrid method thn the EnKF. Another dvntge of the hybrid is tht it is built bsed on the existing opertionl vritionl frmework so tht the estblished cpbility in VAR cn be esily dopted. In ddition, since the hybrid dopts the vritionl frmework, the dynmic constrint cn be conveniently dded during the dt ssimiltion (Kleist et l. 2009). In the current implementtion of the hybrid method, the ensemble covrince is incorported in the vritionl frmework through the extended control vrible method (Lorenc 2003; Buehner 2005; Wng et l. 2007, 2008), nd the ensemble covrince locliztion is conducted in the model stte vrible spce (i.e., model spce locliztion); therefore, no ssumption bout the explicit position of the observtion is required during this procedure of the covrince locliztion. For widely used EnKFs, explicit positions of the observtions re needed to pply covrince locliztion (so-clled observtion spce locliztion; Hmill et l. 200). For stellite rdinces, for which there is no explicit verticl position, such observtion spce locliztion is thus inpproprite. Results in Cmpbell et l. (200) suggested tht model spce locliztion such s used in the hybrid frmework is more sensible nd effective in ssimilting stellite rdinces. In n opertionl vritionl dt ssimiltion system, the bckground error covrince nd its inversion re never explicitly formed during the minimiztion becuse of the lrge dimension of the problem. Incorporting the ensemble covrince therefore cnnot be conducted by simple weighted sum of the sttic nd ensemble covrinces. For VAR tht is preconditioned upon the squre root of the bckground error covrince, Buehner (2005) nd Wng et l. (2008,b) described method to modify the vritionl minimiztion to incorporte the ensemble covrince. Different from Buehner (2005) nd Wng et l. (2008,b), in the widely used GSI the minimiztion procedure is preconditioned upon the full bckground error covrince (Derber nd Rosti 989). A frmework to define how to incorporte the ensemble covrince in GSI is therefore needed. The gol of this pper is to provide mthemticl derivtion demonstrting this frmework. Such frmework cn be pplied in the vritionl system with similr preconditioning. Given the populrity of the GSI nd encourging results from previous work on hybrid dt ssimiltion, this pper represents the first step towrd developing nd investigting the hybrid dt ssimiltion method bsed on the GSI. Section 2 briefly summrizes the key components in GSI minimiztion, nd section 3 provides mthemticl detils on how ensemble covrince is incorported in the GSI minimiztion. Section 4 suggests using the existing GSI cpbilities to conduct ensemble covrince locliztion nd discusses wys to reduce computtionl cost. Section 5 summrizes the pper nd provides further discussion. 2. The GSI minimiztion lgorithm GSI dopts 3DVAR cost function s J(x9 ) 5 0.5(x9 ) T B (x9 ) 0.5(y o 9 Hx9 ) T R (y o 9 Hx9 ), () where x9 is the nlysis increment, B is the sttic bckground error covrince, y o 9 is the innovtion vector, H is the linerized observtion opertor, nd R is the observtion error covrince. A preconditioned conjugte grdient lgorithm is pplied for minimizing Eq. () (Derber nd Rosti 989): tht is, the minimiztion is preconditioned through defining new vrible: z9 5 B x9. (2) Then the grdients of the cost function with respect to x9 nd z9 become $ x9 J 5 z9 H T R (Hx9 y o 9) nd (3) $ z9 J 5 B z9 B H T R (Hx9 y o 9) 5 B $ x9 J. (4)

3 2992 M O N T H L Y W E A T H E R R E V I E W VOLUME 38 After estblishing Eqs. (3) (4), the itertive minimiztion steps re then followed (Derber nd Rosti 989; Wu et l. 2002) to find the finl nlysis. In other words, the GSI minimiztion is preconditioned on the full bckground error covrince, nd there is no need to invert the bckground error covrince explicitly. 3. Incorporting ensemble covrince in GSI minimiztion The gol of this section is to provide mthemticl frmework to show how the GSI minimiztion described in section 2 will be modified to incorporte the ensemble covrince s prt of the bckground error covrince. The ide is to follow the sme preconditioning of the originl GSI. We first briefly summrize the chnges needed in the GSI cost function, following the method of Wng et l Then the derivtion on how chnges need to be mde to fit the GSI minimiztion outlined in Eqs. (2) (4) is given. As in Wng et l. 2008, in the hybrid system, the nlysis increment, denoted s x9, is sum of two terms, defined s x95x9 å K k5 ( k 8 xe k ). (5) The first term, x9, in Eq. (5) is the increment ssocited with the GSI sttic bckground covrince. The second term is the increment ssocited with the flow-dependent ensemble covrince. In the second term of Eq. (5), x k e is the kth ensemble perturbtion normlized by (K 2 ) /2, where K is the ensemble size. The vectors k, k 5,..., K, denote the extended control vribles for ech ensemble member. The open-circle symbol denotes the Schur product (element by element product) of the vectors k nd x k e. In other words, the second term of Eq. (5) represents locl liner combintion of ensemble perturbtions. The nlysis increment x9 is obtined by minimizing the following hybrid cost function: J(x9, ) 5 b J b 2 J e J o 5 b 0.5(x9 ) T B (x9 ) b 2 0.5() T A () 0.5(y o 9 Hx9) T R (y o 9 Hx9). (6) As compred with norml 3DVAR cost function, weighted sum of J nd J e terms in Eq. (6) replces the usul bckground term. Here J is the trditionl GSI bckground term ssocited with the sttic covrince B. In the term J e, is vector formed by conctenting K vectors k, k 5,..., K. The extended control vribles re constrined by block-digonl mtrix A. Ech of the K blocks contins the sme prescribed correltion mtrix, which constrins the sptil vrition of k. In other words, A defines the sptil covrince, here sptil correltion (since vrince is equl to ) of. The term J o in Eq. (6) is the observtion term s in the trditionl 3DVAR except tht x9 is replced by Eq. (5). In Eq. (6), there re two fctors b nd b 2 tht define the weights plced on the sttic bckground-error covrince nd the ensemble covrince. Note tht lthough ensemble covrince is not explicitly shown in Eq. (6), the ensemble covrince ws incorported through the second term in Eq. (5) during the minimiztion. To understnd further how ensemble covrince is incorported in Eqs. (5) (6), Wng et l. (2007) nd Wng et l. (2008) proved the equivlence of using Eqs. (5) (6) to find the nlysis to tht by replcing the bckground error covrince in Eq. () with the weighted sum of the sttic bckground error covrince nd the ensemble covrince modulted by the correltion mtrix in A. The sme ppers lso show tht A determines the covrince locliztion on the ensemble covrince. For detils plese see Wng et l. (2007) nd Eqs. (6) (8) in Wng et l. (2008). As described by Eqs. (2) (4), the GSI minimiztion is conducted through the conjugte grdient method preconditioned on the full bckground error covrince. Next we derive tht to minimize the new cost function in Eq. (6) we cn follow the sme minimiztion procedure used in the originl GSI. The key of the new derivtion is tht the minimiztion of the new cost function cn be preconditioned in the sme wy s shown in Eqs. (2) (4). In other words, the sme conjugte grdient minimiztion procedure from the originl GSI will be followed, except tht we will just need to extend the control vrible nd extend the bckground error covrince. Note lso tht the second term in Eq. (6) is preconditioned on A nd not on the ensemble covrince. In other words, similr to the fct tht the preconditioning for the first term in Eq. (6) is with respect to the sttic covrince, the dded second term in Eq. (6) is preconditioned with respect to A, which plys role of constrining the covrince of extended control vribles tht is similr to tht of the sttic covrince to the originl control vribles. Denote the new control vrible s x 5 x9. (7) The hybrid increment in Eq. (5) cn be expressed s

4 JULY 200 N O T E S A N D C O R R E S P O N D E N C E 2993 x95x9 å K k5 k 8 x e k 5 x9 [dig(xe )... dig(xe K )], (8) where dig is n opertor tht turns vector into digonl mtrix where the nth digonl element is given by the nth element of the vector (Wng et l. 2007). We further denote D 5 [dig(x e )...dig(x e K )] nd C 5 (I, D), where I is n identity mtrix. Then the hybrid increment becomes x95x9 D 5 (I, D) x9 5 Cx. (9) Denote the new bckground error covrince s 0 B b 0 B 5 B A. (0) 0 A b 2 As in the originl GSI, the hybrid is preconditioned by defining new vrible: z 5 B x 5 b! B 0 0 b 2 A x9 5 b B x9 b 2 A!. () In the rest of the derivtion, we will show tht $ z J 5 B$ x J nd therefore the minimiztion for the hybrid cost function cn follow the sme conjugte grdient method used by the originl GSI described in section 2. First, we derive the grdient of the hybrid cost function with respect to x5 x9. The grdients of the new cost function with respect to the originl control vribles $ x9 J nd the extended control vribles $ J re given s $ x9 J 5 b B x9 HT R (Hx9 y o 9) nd (2) $ J 5 b 2 A D T H T R (Hx9 y o 9). (3) Therefore, using Eqs. (9) (3), we obtin $ x J 5 $! x9j 5 B x C T H T R (HCx y o 9) $ J 5 z C T H T R (HCx y o 9). (4) Next we derive the grdient of the hybrid cost function with respect to z. The grdients of the new cost function with respect to b B x9 nd b 2A 2 re given by $ b B x9 J 5 x9 B b H T R (Hx9 y o 9) nd (5) $ b2 A J 5 b 2 AD T H T R (Hx9 y o 9). (6) Using Eqs. (9), (0), (), (5), nd (6), we obtin $ z J 5 $! b B x9 J $ b2 A J 5 x BC T H T R (HCx y o 9). (7) Compring $ x J in Eq. (4) nd $ z J in Eq. (7), we thus obtin $ z J 5 B$ x J. (8) As in the originl GSI minimiztion described in section 2, using Eqs. (4) nd (8), the sme procedure of conjugte grdient minimiztion procedure cn be followed. As discussed in section 2, there is no need to invert both B nd A explicitly. The min differences of the hybrid system reltive to the originl GSI system re tht ) in the clcultion of $ x J in Eq. (4) extr clcultion of grdient with respect to the extended control vrible is needed nd 2) in the clcultion of $ z J [Eq. (8)] the extended bckground error covrince ssocited with the mtrix A is needed. 4. Modeling of the error covrince of the extended control vrible A nd thoughts to reduce computtionl cost In Eq. (8), the bckground error covrince consists of two components, the originl B nd the error covrince A tht constrins the extended control vrible. As discussed in section 3, effectively A conducts the covrince locliztion on the ensemble covrince. In the originl GSI, B is pproximted by the recursive filter trnsform (Hyden nd Purser 995; Wu et l. 2002; Purser et l. 2003,b). The recursive filter cpbility in GSI cn lso be used to pproximte A, which is different from Buehner (2005) in which the correltion ws modeled with truncted spectrl expnsion. Note tht A is block digonl mtrix with K identicl submtrices. Ech submtrix will be pproximted by the sme recursive filter trnsform. As discussed in Wng et l. 2008, the prmeters in the recursive filter will determine the correltion length scle in A nd therefore prescribe the covrince locliztion length scle for the ensemble covrince. The number of extended control vribles is equl to the dimension of the model spce where the covrince

5 2994 M O N T H L Y W E A T H E R R E V I E W VOLUME 38 locliztion is pplied times the number of ensemble members. Becuse with reltively long locliztion scle the extended control vribles re sptilly smooth fields, the extended control vribles nd the recursive filter trnsform ssocited with A cn both be on corser grid. A lrge svings in computtionl cost is expected without loss of ccurcy. A similr ide ws suggested by Wng et l. (2008) nd ws dopted by Buehner (2005) in which the extended control vrible ws in severely truncted spectrl spce nd by Yng et l. (2009) in the context of the locl ensemble trnsform Klmn filter. Wng et l. (2007c) showed tht the number of itertions during the minimiztion in the hybrid system for the Wether nd Forecsting Model (WRF) in which the ensemble covrince is incorported is less thn tht of the originl WRF vritionl system in which the sttic bckground error covrince is used, suggesting fster convergence in the minimiztion of the hybrid system in which ensemble covrince nd sttic covrince were ssigned zero weights thn in the originl VAR system with only the sttic covrince. They lso found tht the number of itertions is not sensitive to the length of the covrince locliztion scles. Such tests will be conducted for the GSI-bsed hybrid dt ssimiltion system. 5. Summry nd discussion The populrity of GSI nd the potentil dvntges of the hybrid ensemble GSI reltive to GSI nd pure ensemble Klmn filter suggest need to develop the hybrid ensemble GSI system. Different from the hybrid system in Buehner (2005) nd Wng et l. (2008,b), in the widely used GSI the minimiztion procedure is preconditioned upon the full bckground error covrince (Derber nd Rosti 989). A frmework to define how to incorporte the ensemble covrince in GSI is therefore provided. Possible wys to reduce computtionl cost re lso discussed. The method cn be pplied in the vritionl system with similr preconditioning. The improvement in the ccurcy of the nlysis obtined from hybrid method over stndrd vritionl pproch my depend on how ccurtely the shortrnge ensemble forecsts used in the hybrid estimte the flow-dependent forecst error covrince. The current opertionl ensemble my not optimize for this ppliction, nd lterntive ensemble-genertion methods such s the ensemble Klmn filter bsed method my need to be explored. Acknowledgments. The uthor ws supported by newfculty strtup wrd from the University of Oklhom. The uthor lso gretly cknowledges discussions with Dve Prrish nd the vluble comments by Mrk Buehner nd n nonymous reviewer tht helped to improve the mnuscript. REFERENCES Aksoy, A., D. C. Dowell, nd C. Snyder, 2009: A multicse comprtive ssessment of the ensemble Klmn filter for ssimiltion of rdr observtions. Prt I: Storm-scle nlyses. Mon. We. Rev., 37, Buehner, M., 2005: Ensemble-derived sttionry nd flow-dependent bckground-error covrinces: Evlution in qusi-opertionl NWP setting. Qurt. J. Roy. Meteor. Soc., 3, , P. L. Houtekmer, C. Chrette, H. L. Mitchell, nd B. He, 200: Intercomprison of vritionl dt ssimiltion nd the ensemble Klmn filter for globl deterministic NWP. Prt I: Description nd single-observtion experiments. Mon. We. Rev., 38, ,,,, nd, 200b: Intercomprison of vritionl dt ssimiltion nd the ensemble Klmn filter for globl deterministic NWP. Prt II: One-month experiments with rel observtions. Mon. We. Rev., 38, Cmpbell, W. F., C. H. Bishop, nd D. Hodyss, 200: Verticl covrince locliztion for stellite rdinces in ensemble Klmn filters. Mon. We. Rev., 38, Dryl, T. K., D. F. Prrish, J. C. Derber, R. Treton, W. Wu, nd S. Lord, 2009: Introduction of the GSI into the NCEP globl dt ssimiltion system. We. Forecsting, 24, Derber, J., nd A. Rosti, 989: A globl ocenic dt ssimiltion system. J. Phys. Ocenogr., 9, Hmill, T. M., nd C. Snyder, 2000: A hybrid ensemble Klmn filter- 3D vritionl nlysis scheme. Mon. We. Rev., 28, , J. S. Whitker, nd C. Snyder, 200: Distnce-dependent filtering of bckground error covrince estimtes in n ensemble Klmn filter. Mon. We. Rev., 29, Hyden, C. M., nd R. J. Purser, 995: Recursive filter objective nlysis of meteorologicl fields: Applictions to NESDIS opertionl processing. J. Appl. Meteor., 34, 3 5. Houtekmer, P., G. Pellerin, M. Buehner, nd M. Chrron, 2005: Atmospheric dt ssimiltion with n ensemble Klmn filter: Results with rel observtions. Mon. We. Rev., 33, Jung, Y., G. Zhng, nd M. Xue, 2008: Assimiltion of simulted polrimetric rdr dt for convective storm using the ensemble Klmn filter. Prt I: Observtion opertors for reflectivity nd polrimetric vribles. Mon. We. Rev., 36, Kleist, D. T., D. F. Prrish, J. C. Derber, R. Tredon, W.-S. Wu, nd S. Lord, 2009: Introduction of the GSI into the NCEP Globl Dt Assimiltion System. We. Forecsting, 24, Liu, H., J. Anderson, Y. Kuo, C. Snyder, nd A. Cy, 2008: Evlution of nonlocl qusi-phse observtion opertor in ssimiltion of CHAMP rdio occulttion refrctivity with WRF. Mon. We. Rev., 36, Lorenc, A. C., 2003: The potentil of the ensemble Klmn filter for NWP A comprison with 4D-VAR. Qurt. J. Roy. Meteor. Soc., 29, Purser, R. J., W. S. Wu, D. F. Prrish, nd N. M. Roberts, 2003: Numericl spects of the ppliction of recursive filters to vritionl sttisticl nlysis. Prt I: Sptilly homogeneous nd isotropic Gussin covrinces. Mon. We. Rev., 3, ,,, nd, 2003b: Numericl spects of the ppliction of recursive filters to vritionl sttisticl nlysis. Prt II: Sptilly inhomogeneous nd nisotropic generl covrinces. Mon. We. Rev., 3,

6 JULY 200 N O T E S A N D C O R R E S P O N D E N C E 2995 Szunyogh, I., E. J. Kostelich, G. Gyrmti, D. J. Ptil, B. R. Hunt, E. Klny, E. Ott, nd J. A. York, 2005: Assessing locl ensemble Klmn filter: Perfect model experiments with the NCEP globl model. Tellus, 57A, Torn, R., nd G. J. Hkim, 2008: Performnce chrcteristics of pseudo-opertionl ensemble Klmn filter. Mon. We. Rev., 36, Wng, X., C. Snyder, nd T. M. Hmill, 2007: On the theoreticl equivlence of differently proposed ensemble/3d-vr hybrid nlysis schemes. Mon. We. Rev., 35, , T. M. Hmill, J. S. Whitker, nd C. H. Bishop, 2007b: A comprison of hybrid ensemble trnsform Klmn filter-oi nd ensemble squre-root filter nlysis schemes. Mon. We. Rev., 35, , D. Brker, C. Snyder, nd T. M. Hmill, 2007c: A hybrid ensemble trnsform Klmn filter (ETKF)-3DVAR dt ssimiltion scheme for WRF. Preprints, th Symp. on Integrted Observing nd Assimiltion Systems for the Atmosphere, Ocens, nd Lnd Surfce, Sn Antonio, TX, Amer. Meteor. Soc., P4.0.,,, nd, 2008: A hybrid ETKF 3DVAR dt ssimiltion scheme for the WRF model. Prt I: Observing system simultion experiment. Mon. We. Rev., 36, ,,, nd, 2008b: A hybrid ETKF 3DVAR dt ssimiltion scheme for the WRF model. Prt II: Rel observtion experiments. Mon. We. Rev., 36, , T. M. Hmill, J. S. Whitker, nd C. H. Bishop, 2009: A comprison of the hybrid nd EnSRF nlysis schemes in the presence of model errors due to unresolved scles. Mon. We. Rev., 37, Whitker, J. S., T. M. Hmill, X. Wei, Y. Song, nd Z. Toth, 2008: Ensemble dt ssimiltion with the NCEP Globl Forecst System. Mon. We. Rev., 36, Wu, W. S., R. J. Purser, nd D. F. Prrish, 2002: Three-dimensionl vritionl nlysis with sptilly inhomogeneous covrinces. Mon. We. Rev., 30, Yng, S.-C., E. Klny, B. Hunt, nd N. E. Bowler, 2009: Weight interpoltion for efficient dt ssimiltion with the locl ensemble trnsform Klmn filter. Qurt. J. Roy. Meteor. Soc., 35, Zhng, F., Y. Weng, J. A. Sippel, Z. Meng, nd C. H. Bishop, 2009: Cloud-resolving hurricne initiliztion nd prediction through ssimiltion of Doppler rdr observtions with n ensemble Klmn filter. Mon. We. Rev., 37, Zupnski, M., 2005: Mximum likelihood ensemble filter: Theoreticl spects. Mon. We. Rev., 33,