Algorithm for Estimation of the Specific Differential Phase

Transcription

1 DECEMBER 2009 W A N G A N D C H A N D R A S E K A R 2565 Algorithm for Estimation of the Specific Differential Phase YANTING WANG AND V. CHANDRASEKAR Colorado State University, Fort Collins, Colorado (Manuscript received and in final form 13 July 2009) ABSTRACT The specific differential phase K dp is one of the important parameters measured by dual-polarization radar that is being considered for the upgrade of the current Next Generation Weather Radar (NEXRAD) system. Estimation of the specific differential phase requires computing the derivative of range profiles of the differential propagation phase. The existence of possible phase wrapping, noise, and associated fluctuation in the differential propagation phase makes the evaluation of derivatives an unstable numerical process. In this paper, a robust algorithm is presented to estimate the specific differential phase, which is able to work on wrapped phases and keep up with the spatial gradients of rainfall, to provide a high-resolution specific differential phase. 1. Introduction Corresponding author address: Yanting Wang, 1373 Campus Delivery, Colorado State University, Fort Collins, CO ytwang@engr.colostate.edu In dual-linear-polarization bases, the specific differential phase K dp is defined as the slope of range profiles of the differential propagation phase shift F dp between horizontal (H) and vertical (V) polarization states (Seliga and Bringi 1978; Jameson 1985; Bringi and Chandrasekar 2001). The specific differential phase is an important parameter for meteorological applications, because it is not affected by propagation attenuation and it is closely related to rain intensity, even in the presence of hail (Aydin et al. 1995; Chandrasekar et al. 2008). However, the estimation of K dp involves estimating the slope of F dp profiles, which is known to be a noisy unstable computation. Evaluation of the derivative is essentially a high-pass filtering, and it expects a smooth and continuous function as the input. The range profile of total differential phase C dp contains both F dp and differential backscatter phase shift d hv,aswellasmeasurement fluctuation. The fluctuation in the estimates of C dp will be magnified during the differentiation, resulting in large variance in the estimates of K dp. Furthermore, phase wrapping may exist in F dp, and special care is generally required to unfold the wrapped phase profiles. The unambiguous range of F dp usually is 1808 in the alternate H/V transmission mode and 3608 in the simultaneous H/V transmission mode. The system phase offset needs to be carefully adjusted to near the lower unambiguous limit for making use of the full unambiguous range. Nevertheless, for a long propagation path in rain or at higher-frequency bands such as X-band, F dp values can easily exceed the unambiguous range. All these challenges have to be addressed to achieve a satisfactory estimation of K dp, which can be applied for quantitative applications such as rainfall estimation. The artificial discontinuities introduced by phase wrapping have to be detected first from the noisy phase profiles and to be adjusted. Usually, a piecewise fitting is used to predict the local trend, and any phase sample deviated far from this trend can be attributed to phase wrapping. This solution is neither elegant nor stable because of the statistical fluctuations present in the F dp field. Indeed, it is a common unfolding technique to estimate the slope first and then recover the intrinsic phase profiles based on the detected discontinuities. In this paper, a completely different approach to estimate K dp is introduced to avoid the need for unfolding phases. The current techniques to reduce the variance of K dp estimates include range filtering, linear fitting, or both (Golestani et al. 1989; Hubbert and Bringi 1995). All these techniques reduce the peak K dp values in the estimation that could introduce biases. In addition, only limited adaptivity can be achieved with filtering or fitting to follow the steep slopes within intense rain cells and reduce the estimation variance in the rest of the segments simultaneously. This paper presents an adaptive scheme to estimate K dp, DOI: /2009JTECHA Ó 2009 American Meteorological Society

2 2566 J O U R N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L O G Y VOLUME 26 which has better range resolution in intense rain cells to capture the small-scale variability. A regularization technique is introduced to control the balance between estimation bias and variance, and it incorporates the adaptivity to keep up with steep change of F dp. The procedure is developed essentially to improve the robustness in real-time operations. This paper is organized as follows: the conventional estimation of K dp is summarized first in section 2 with the state-of-the-art Colorado State University University of Chicago Illinois State Water Survey (CSU CHILL) algorithm as an example. Then, the new estimator is described in section 3 to deal with wrapped phases, and an adaptive algorithm for its implementation is developed in section 4. Its application to radar observations from CSU CHILL S-band radar and the Center for Collaborative Adaptive Sensing of the Atmosphere (CASA) Integrated Project 1 (IP1) X-band radars is presented to demonstrate the performance of the new algorithm. 2. Conventional estimation of specific differential phase The specific differential phase K dp is estimated as the range derivative of the differential propagation phase F dp, following the fundamental definition K dp (r) 5 1 df dp (r). (1) 2 dr Radar does not measure the forward propagation parameter F dp directly. Instead, the total differential phase C dp is estimated from copolar covariance that consists of both the phase shifts resulting from forward propagation and backscattering; that is, C dp (r) 5F dp (r) 1 d hv (r). (2) If the differential backscattering phase d hv is relatively constant or negligible, the profile of C dp can be used to estimate K dp. Because of different pulse sequencing, separate estimators are used to estimate C dp in the alternative transmission mode (VH) and in the simultaneous transmission mode (VHS). In the VH mode, C dp can be estimated from the received alternate samples at H and V polarization states as (Mueller 1984; Sachidananda and Zrnić 1989; Bringi and Chandrasekar 2001) ^C dp argf ^R a ^Rb *g, (3) where ^R a 5 hh* 2i11 V 2i i and (4a) ^R b 5 hv* 2i12 H 2i11 i, (4b) where H 2i11 and V 2i are the alternately sampled horizontal and vertical polarization returns, respectively. Note that the V polarization is assumed leading the pulse-topulse polarization switching on transmission. In the VHS mode, C dp is estimated directly from the simultaneous samples V i and H i as ^C dp 5 arg hh i *V i i. (5) The argument operation in (3) and (5) leads to finite unambiguous ranges of 1808 and 3608 in the VH and VHS modes, respectively. As a cumulative value, F dp is a continuous range function. However, discontinuities usually appear in the estimated C dp because of phase wrapping, statistical fluctuations in estimation, and gateto-gate variation of d hv. The artifact introduced by phase wrapping needs to be identified and corrected for, whereas the statistical errors and gate-to-gate variation should be then suppressed using various smoothing techniques. The error model is presented to describe the estimated C dp in term of K dp as ^C dp (r) 5F dp (0) 1 2 ð r 0 K dp (r) dr 1 «, (6) where F dp (0) represents the system differential phase and «represents fluctuations resulting from statistical errors and the variation of d hv. In the presence of significant d hv over a short range such as a bump, Hubbert and Bringi (1995) introduced an iterative technique to detect and remove the extra change resulting from the d hv bumps. a. Phase unfolding Phase wrapping can occur, depending on the system differential phase F dp (0), even if the total F dp accumulation is within the unambiguous range. For this reason, the system differential phase usually is intentionally set near the lower bound of the unambiguous range. Nevertheless, this system phase can drift, and F dp can exceed the unambiguous range. Therefore, phase unfolding needs to be done first at the very beginning during the evaluation. In CSU CHILL, a set of straightforward but complex logic is adopted to detect the wrapped phase and accordingly unfold the range profile of F dp. The procedure is described in the following.

3 DECEMBER 2009 W A N G A N D C H A N D R A S E K A R 2567 FIG. 1. Flowchart of the phase unfolding algorithm used in the CSU CHILL radar. The judgments and adjustments are valid in VH mode. The first step is to determine the location and phase offset where the meaningful F dp profile begins. Within the path free of weather echoes, low signal-to-noise ratio (SNR) and low copolar correlation result in large phase fluctuation, which should not be misjudged as phase wrapping. This decision is made based on the standard deviation of F dp (s f ) and the copolar correlation coefficient r hv. A consensus of small s f and high r hv over consecutive gates is used to ensure robustness in the detection of weather echoes. Subsequently, s f and the slope of F dp are continuously computed. A segment of profile with large s f is labeled as nonweather echoes. Otherwise, the estimated slope is accumulated into a smoothed F dp profile for reference. Based on this knowledge, the current F dp value is compared against the reference value of preceding F dp to determine whether the current one needs to be unfolded. The flowchart describing the current CSU CHILL algorithm is shown in Fig. 1. The computation for s f and the slope is done to prevent false phase unfolding resulting

4 2568 J O U R N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L O G Y VOLUME 26 from the contamination by noise. It should be noted that hard thresholds are used throughout the procedure that are obtained from the system metrics of the CSU CHILL radar. It is also assumed that F dp would not be wrapped at the beginning range. b. Range filtering The differentiation process is equivalent to applying high-pass filtering on the range profiles of F dp, as seen in the frequency domain K dp (v) 5 jv 2 F (v), (7) dp where K dp (v)andf dp (v) are Fourier transforms of range profiles of K dp (r) and F dp (r), respectively. Therefore, statistical errors in the estimated C dp will be magnified in the estimated K dp profile. If K dp is estimated using finite difference as ^K dp (r) ^C(r 2 ) ^C(r 1 ), (8) 2(r 2 r 1 ) then the standard deviation of the estimate of K dp is related to the phase variation through (Bringi and Chandrasekar 2001) s[ ^K dp ] 5 s[ ^C dp ] p ffiffi 2 (r2 r 1 ), (9) assuming uniform differential phase variance across range. However, the differential phase variance depends on Doppler spectrum, r hv, and d hv. Low-pass range filtering can be further employed to reduce the estimation variance. To gain insight into this filtering process, the error term in (6) is assumed as a white, ergodic, widesense stationary process. Using a filter as specified by F(v), the variance of the estimates of C dp after filtering can be written as s 2 ( ^C filt ) 5 1 2p s2 ( ^C dp ) ð 2p 0 jf(v) j 2 dv. (10) The variance reduction is approximately proportional to the normalized cutoff frequency of the filter. The cutoff frequency can be translated to a range scale. At gate spacing Dr, the Nyquist frequency can be written as 1/(2Dr). Assuming the filter is expected to allow the trend of F dp down to a range scale of r c, the normalized cutoff frequency for the passband of the filter must be (Oppenheim and Schafer 1989) v c 5 4pDr r c. (11) Then, the fluctuations at scales smaller than r c will be suppressed. A good design of the range filter should suppress gate-to-gate fluctuations but preserve the underlying trend of F dp (Hubbert et al. 1993). In the CSU CHILL algorithm, the cutoff scale is chosen at 3 km, and iterative filtering is used to adaptively compensate for the persistence of large d hv. Note that at different gate spacing the filter needs to be redesigned as shown in appendix A. c. Least square fitting Polynomial regression can be applied as an alternate estimator to finite difference (Bringi and Chandrasekar 2001), where the range profile of C dp is approximated by a polynomial function as C dp (r) 5 b 0 1 b 1 r 1 1 b n r n. (12) As the order of this polynomial function increases, the regression can better capture the intrinsic change of F dp. However, high-order regression will also increase the computational complexity and the estimation sensitivity. In practice, low-order piecewise polynomial fitting is rather performed. For short paths, linear approximation can be used, and the estimate of K dp is given by b 1. The standard deviation of estimated K dp in this case can be derived as (Gorgucci et al. 1999) 1 s[k dp ] 5 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi s[ ^C Dr N(N 1)(N 1 1)/3 dp ], (13) where N is the number of consecutive gates for the piecewise linear regression. The estimation variance can by reduced by increasing pathlength (i.e., NDr) or decreasing the gate spacing. In the current CSU CHILL algorithm, piecewise linear regression is done to estimate K dp using adaptive length N, which is adjusted according to three reflectivity levels, such as 8 < 30 Z h, 35 dbz N dbz, Z h, 45 dbz. : 10 Z h. 45 dbz (14) This adaptive adjustment is meant to follow the steep phase changes in intensive precipitation regions while keeping the variation of estimated K dp low in light precipitation regions. The regression length is tailored for the CSU CHILL radar, which has a range resolution of 150 m. Therefore, the estimation can keep up with a small-scale variability up to 1.5 km in heavy rain and a large-scale variability up to 4.5 km in light rain.

5 DECEMBER 2009 W A N G A N D C H A N D R A S E K A R New estimator for specific differential phase A practical phase unfolding approach has been described in the previous section. One feature of the procedure is somewhat complicated in the sense that a slope is first approximated for unfolding the phase profiles and then a more accurate slope is estimated for K dp. Its performance also relies on the choice of several hard thresholds. Nevertheless, wrapped phase profiles have to be unfolded first before the subsequent filtering and fitting, because the discontinuities are not physical features. In this section, a new estimator will be proposed for estimating K dp based on the complex-valued range profiles of the differential phase shift. To clarify the difference from the conventional description, hereafter the differential phase shift in the complex domain will be referred as the angular F dp, whereas its counterpart in the real domain (e.g., from 0 to 2p) will be referred as the principal F dp. For example, F dp repeats in periods of 2p in the VHS mode. When the profile crosses a full period, even though the principal phase wraps back with an abrupt jump, locally the profile is still continuous in the complex domain. The angular F dp profile can be obtained by defining a profile of unit vector as q(r) 5 e j ^C dp (r). (15) In the VH mode, the period is only p and the angular F dp in this case is known as an axial variable, which can be transformed to circular variable by magnifying the angle by a factor of 2. Again, the phase profile can be vectorized as q(r) 5 e j2 ^C dp (r). (16) The only difference between (15) and (16) is the factor 2, which needs to be applied back to the final estimates of K dp. Without loss of generality, the following discussion will be based on the profile as defined in (15). The same algorithm works for the VH mode as well. Taking the derivative of (15) with respect to r, we have q9(r) 5 je j ^C d ^C dp (r) dp (r). (17) dr Therefore, the estimate of K dp can be written as q9(r) ^K dp (r) 5 j 1 e(r). (18) q(r) This shows that the quotient q9(r)/q(r) is an imaginary number, provided the continuity in the complex-valued phase profile and its imaginary part corresponds to K dp with an extra term associated with estimation error. On the other hand, the existence of phase fluctuations can disturb the continuity. In addition, the numerical errors in derivatives make the quotient likely not a pure imaginary number. If the phase fluctuation is sufficiently small, as implied earlier in (8), the imaginary part of the quotient can be taken as the estimates of K dp. The statistical fluctuations of the differential phase shift can be also studied through the angular F dp. Its mean and its dispersion can be defined as M(F dp ) 5 arghe jf dp i and (19) D(F dp ) 5he jf dp i, (20) respectively, which satisfy the following properties under a constant shift by f 0 : M(F dp f 0 ) 5 M(F dp ) f 0 and (21) D(F dp f 0 ) 5 D(F dp ). (22) It can be shown that the mean defined in (19) is the same as the mean of unfolded principal F dp if it follows a symmetric distribution. The dispersion defined in (20) decreases as the standard deviation of F dp increases, and it approaches unity if F dp is highly concentrated, whereas it approaches 0 if F dp is uniformly distributed between 0 and 2p. Specially, if F dp isanarrowlydistributed Gaussian variable, it can be shown that D(F dp ) e s2 f /2. (23) Under a Gaussian distribution assumption, the dispersion can be used to examine the degree of fluctuation in the estimated C dp and predict the data quality for the estimates of K dp. Figure 2 illustrates the simple logic for data masking with use of the dispersion parameter. Numerical derivative errors can be reduced by using high-order estimation techniques. Linear finite difference is a genuine slope estimator for a linear function. However, the definition in (15) and (16) does require higherorder numerical differentiation because of the exponential term, unless the sampling interval is sufficiently small. As a simple example, Table 1 shows the numerical derivative errors in the angular F dp profiles using different solutions. The principal F dp profile is assumed linear from which all these estimators can accurately retrieve the true slope. However, as the slope increases, finite difference gives larger errors in the estimates of K dp from the angular F dp profiles. Numerical differentiation

6 2570 J O U R N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L O G Y VOLUME 26 FIG. 2. Flowchart of decision for rain cells and good data mask, where M G and M B are constants set for consecutive good and bad samples, respectively. Usually, M B is smaller than M G. based on interpolation polynomials of a higher degree can substantially reduce the errors, simply because the high-degree polynomials can better approximate the underlying function. Note that cubic spline is also a piecewise interpolation polynomial with only extra constraint of continuities at the knots. It can be seen that cubic spline performs uniformly better than a four-degree Lagrange interpolation polynomial. Therefore, a feasible solution for K dp estimates will be filtering angular F dp profiles based on angular C dp statistics and estimating K dp using cubic spline. Thus, phase unfolding is avoided, and the accuracy is also ensured. An adaptive estimation of K dp is developed based on the principles described and is presented in the next section. 4. Adaptive estimation of specific differential phase The instability in numerical differentiation of noisy data can be well controlled in a regularization framework. To reach stable solutions, the differentiation of an arbitrary function f() (which is the range profile of angular F dp in this paper) is to find a function s() tominimize both the smoothness of the underlying regression function and the regression errors (Reinsch 1967):

7 DECEMBER 2009 W A N G A N D C H A N D R A S E K A R 2571 TABLE 1. Numerical derivative errors in K dp estimates from angular F dp profile. The range spacing is assumed at 0.5 km, and the intrinsic K dp is assumed constant. The derivative using higher-order polynomials is computed from piecewise Lagrangian interpolation polynomials. K dp (8 km 21 ) Three-point finite difference ; Fourth-degree polynomial ;10 27 ;10 25 ; Sixth-degree polynomial ; ;10 28 ;10 26 ; Cubic spline ;10 28 ;10 26 ; < ð min js9(r) j 2 dr 1 l å M s(r) : k51 ð rk f (r k ) 9 2= s(r) dr ;, (24) where f(r k ) represents the observations and l is the Lagrangian parameter. The first term in (24) evaluates the degree of smoothness. An extreme smoothness occurs when l 5 0, in which case a linear continuous function is the solution to the minimization. The second term in (24) assesses the degree of data fidelity. As l /, the regression function is dictated to pass all the measurement. It can be shown that the cubic spline function has the greatest smoothness over all secondorder continuous functions that pass the nodes (Pollock 1999). Other intermediate values of l control the tradeoff between the expected model smoothness and the allowed regression errors. The minimization will impose more emphasis on the degree of smoothness if small l is used. Through the choice of l, the solution balances between the bias and variance of the estimates. Assuming cubic splines as the regression function, the polynomial function for the kth interval between nodes r k and r k11 can be written as g k (r) 5 a k (r r k ) 3 1 b k (r r k ) 2 1 c k (r r k ) 1 d k, r 2 [r k, r k11 ]. (25) Normally, K dp can be assumed as constant at both ends of a radial profile over a rain cell, which means that the second-order derivatives at end nodes should be zero. This end condition imposes the complete determination of all the coefficients and leads to natural cubic splines. On the cubic splines solved, the coefficient d k corresponds to the smoothed angular F dp and the coefficient c k corresponds to its range derivatives. Note that only the derivatives at the nodes are of our interest to estimate K dp. Therefore, the estimates can be simply obtained as ^K dp (r k ) Im c k. (26) d k Thus, the conventional filtering process is avoided in this regularization framework, and stable numerical derivative can be obtained with an appropriate l. However, cubic smoothing spline still is largely a global smoothing procedure. Smoothing the segments of high variation at a satisfactory level leads to an oversmoothed profile in the segments of low variation. In addition, oversmoothness needs to be avoided in high K dp region to track the peak K dp. The corresponding adaptivity is enabled by introducing varying weights in the minimization (24) as 8 < ð min s(r) : q 2 (r) js9(r) j 2 dr 1 lå M k51 w 2 (r k ) f (r k ) ð rk 9 2= s(x) dx ;. (27) Substituting (25) into (27), the objective function for minimization can be rewritten as M 1 4q L 5 å 2 k (r k11 r k ) [jb k51 3 k j 2 1 jb k11 j 2 1 Re(b k b* k11 )] 1lå M w 2 k f (r k ) d 2 k. (28) k51 Its solution is provided in appendix B. The new introduced parameters q and w are both real-valued scaling factors. With them, the balance between bias and variance can be selectively controlled for different segments. The former one will be used to compensate for the variation in slope (i.e., K dp ); the latter one will be used to compensate for the statistical fluctuation of the input profiles. a. Adaptivity for statistical fluctuation To minimize the second term in (28), the smoothed profile d k should be equal to the mean of f(r k ). Therefore,

8 2572 J O U R N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L O G Y VOLUME 26 w can be chosen as the inverse of the standard deviation of the angular F dp such that the variation is normalized. The variance of angular expfjf dp g can be evaluated as w 2 5 E[je jf dp j 2 ] je[e jf dp ]j je[e jf dp ]j 2. (29) The maximum variation occurs when noise dominates, in which case F dp distributes uniformly between 2p and p. It then follows that w (30) When the variation in the angular F dp is small, Taylor expansion can be used to approximate (29) as w s2 f s 2 f. (31) This shows that the variance of the angular F dp is approximately equal to the variance of the principal F dp. The variance s 2 f can be evaluated from the Doppler spectrum parameters (Bringi and Chandrasekar 2001). It depends both on the Doppler spread and the copolar correlation coefficient. For simplicity, it can be approximated for large r hv as! w r hv. (32) r hv Note that the approximations in (31) and (32) are only valid for small variation or large r hv. However, the weighting function (32) captures well the overall relation between the variation of angular F dp and the copolar correlation coefficient, noting that it also satisfies the maximum variation condition (30) as the correlation becomes poor. Nonnegligible d hv can result from melting hydrometeors (Hubbert and Bringi 1995). In this case, the radar signatures will have lower r hv values and the adaptive method will smooth the differential phase profiles more over the bumped segments. It should also be noted that the iterative filtering technique developed in Hubbert and Bringi (1995) can be additionally applied, if the user desires. b. Adaptivity for K dp To follow the profiles in large K dp, larger variation shall be allowed such that bias is minimized to avoid excessive smoothness. Therefore, let q K dp, where the factor 2 is in place to compensate the two-way F dp profile. With this adjustment, the smoothness requirement at large K dp will be less emphasized. On the contrary, the smoothness at small K dp will be heavily weighted. The TABLE 2. Approximate quantitative evaluation of the smoothness of the cubic regression splines in single interval. ð rk11 K dp (8 km 21 ) jk9 dp /K dp j 2 (K dp ) 2 q 2 (r)js9(r)j 2 dr Dr Dr Dr Dr scaling factor q 21 cuts off at a K dp value of 0.18 km 21 for numerical stability. Based on the angular F dp profile, the smoothness term in (27) can be evaluated as js9(x) j 2 5 je jf dp F0dp e jf dp (F9dp ) 2 4K 2 dp 4K 2 dp 5 K9! 2 dp 1 4K 2 dp K. (33) dp One should note that this term (33) is not unitless after the scaling factor q is applied. It is still related to the actual value of slope K dp owing to the translation from the principal F dp to the angular F dp. c. Choice of the Lagrangian parameter The adaptive approach is only left with the freedom in choosing the smoothing factor l. Cross-validation can be used to find the optimal l from a dataset (Craven and Wahba 1979). Such determination does not suit this application because of excessive ray-by-ray variation in radar observations. However, with the scaling scheme, a simple procedure is used to make the evaluation of smoothness and that of data fidelity equally important. The variation of angular F dp has been normalized using w; therefore, a reasonable choice of l should be close to the expected smoothness by evaluating (33). As an approximation, constant K dp can be assumed in each range interval. At different K dp levels, the smoothness is quantitatively assessed and shown in Table 2 by allowing a 10% variation in K dp. The last column can be used as a baseline for choosing l. The existence of extra K dp in (33) requires a different smoothing factor for different peak K dp values. For example, targeted at K dp of 308 km 21, the smoothing factor should be around 1.1Dr. Figure 3 shows a simulated K dp profile, composed of a Gaussian function at 15 km, to demonstrate an infinitely differentiable profile, and a triangular function at 35 km, to demonstrate K dp discontinuities. The profile exhibits sharp gradient on K dp, which changes from 08 to 308 km 21 within a 3-km range. Statistical fluctuation r k 2

9 DECEMBER 2009 W A N G A N D C H A N D R A S E K A R 2573 are band limited. Most importantly, the calculation for phase unfolding is completely avoided. 5. Evaluation In this section, a simulated storm and actual radar observations are used to evaluate and demonstrate the new K dp estimation. The scaling as shown in (33) cannot be applied directly in practice. However, the K dp in (33) serves only as a relative weighting factor and does not need to be exact. An approximation of K dp can be first translated from the measured reflectivity using the following power-law relation: K dp 5 az b h, (34) FIG. 3.K dp estimation from principal F dp (black dashed line) using simple finite difference and from angular F dp using adaptive cubic smoothing spline: (a) the mean and (b) standard deviation of estimated K dp based on 100 realizations. The intrinsic K dp profile (thick gray line) is simulated as (left) a narrow Gaussian function and (right) a triangular function, at gate spacing of 0.5 km. Fluctuation resulting from noise, Doppler spread, and copolar correlation is simulated. is introduced to F dp by setting the intrinsic r hv at 0.99, the normalized spectral width at 0.12, and the integration length at 64 samples. The Lagrangian factor is chosen between 0.1Dr and 1.1Dr for comparison. The conventional K dp estimation is also plotted from the filtered F dp profiles with a six-order finite impulse response (FIR) filter (designed to keep the variation at 3-km scale). Near the K dp peaks, it can be seen that at l of 1.1Dr the estimation biases are almost completely removed and the standard deviation of the estimation increases but less than 18 km 21. In the nonstorm regions, both the biases and the standard deviation of the estimation are reduced. With smaller l, oversmoothed K dp estimates are obtained that exhibit large biases. It should be noted that the l value is not guaranteed to be the optimal, but Table 2 can be used as a baseline to empirically pick an appropriate l. The value at 1.1Dr is used for all the case studies that are presented in the next section. The algorithm can be easily implemented in a realtime environment. Even though matrix inversion and matrix multiplication are involved in the solution, the computation complexity is well within the capability of modern-day computation power because all the matrices which can be obtained assuming a given raindrop size and shape distribution. This conversion generally works very well at S band. At shorter wavelengths, rain attenuation on the measured reflectivity would limit its applicability. Hail contamination will be another factor violating the conversion. To solve these problems, a two-step iteration is adopted, in which a rough estimation of K dp is made first using the nonadaptive cubic splines derived from (24) and then it is used in the adaptive cubic splines to perform the final K dp estimation. This introduces a slight increase of computation time without any perceivable impact on real-time implementation. During the first step, a smaller Lagrangian parameter fixed at 0.1 is used to obtain the overall trend of the K dp profile and the resulting scaling functions very well. This two-step iteration technique is adopted in this paper to estimate K dp from radar observations. a. Simulated storm Chandrasekar et al. (2006) have developed a procedure to simulated X-band radar observation from S-band radar data with the natural variation of microphysical parameters preserved. X-band radar observations are simulated using the CSU CHILL radar observations of a thunderstorm during the Severe Thunderstorm Electrification and Precipitation Study (STEPS). The simulated X-band observations are shown in Fig. 4, with K dp of up to 78 km 21. Figures 4a c present the radar reflectivity, the estimated K dp using the conventional filtering and fitting approach, and the estimated K dp using the new adaptive approach, respectively. Both the conventional approach and the adaptive approach result in a similar K dp field, with the adaptive approach showing less range streaks. The adaptive approach presents better resolution, which captures the K dp peaks as well as the storm texture that is exhibited in the radar

10 2574 J O U R N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L O G Y VOLUME 26 FIG. 4. Simulated X-band radar observations at elevation angle of 0.58 and gate spacing of 0.1 km. (a) PPI plot of the radar reflectivity; (b) PPI plot of the estimated K dp using conventional approach on the principal F dp ; (c) PPI plot of the estimated K dp using the adaptive estimation on the angular F dp ; and (d) the scattering plot between the estimated K dp and the intrinsic K dp, where the black points illustrate the conventional estimates and the gray points illustrate the adaptive estimates. reflectivity. In addition, the K dp field is much cleaner with smaller variation in the regions of low radar reflectivities. Because the true radar parameters are available in the simulation, the scatterplot between the estimated K dp and the intrinsic K dp is shown in Fig. 4d. If the intrinsic K dp is small, the new estimates are more stable; if the intrinsic K dp is large, the new estimates have a larger standard deviation, but the mean bias is smaller. The results of Fig. 4 clearly illustrate the advantage of the new adaptive approach. b. CSU CHILL S-band radar observation Figure 5 shows the S-band observations of a heavy rain event on 31 May 2003 UTC from the CSU CHILL radar. A large area of thunderstorms developed southeast of the CSU CHILL radar and some of the beams were continuously filled with heavy rain over range segments of nearly 50 km. The value of F dp is accumulated over 2008 around an azimuth of 1288, where phase is wrapped. Again, it can be found that both approaches give similar K dp estimates in the overall pattern by comparing Figs. 5b,c. The K dp peaks are well estimated by the adaptive estimator in the storm core, whereas the K dp estimates outside of storm core are fairly smooth compared to these by the conventional estimator. Note that the storm is located at range of 50 km and beyond. The spatial resolution there is much lower than the previous simulated cases. The scatterplot between the estimated K dp and radar reflectivity is presented in Fig. 5d. It can be seen once again that the large number of extraneous points from the conventional algorithm are eliminated. c. CASA X-band radar observation On 14 June 2007, an intense rainfall event developed over the CASA IP1 test bed in Oklahoma (Chandrasekar et al. 2007) for which a flash flood warning was issued by the National Weather Service (NWS). Reflectivity observations by the X-band IP1 radar in Rush Springs are shown in Fig. 6a. The estimated K dp from the principal F dp is shown in Fig. 6b, and the new K dp estimates

11 DECEMBER 2009 W A N G A N D C H A N D R A S E K A R 2575 FIG. 5. CSU CHILL S-band radar observations at elevation angle of 0.58 and gate spacing of 0.15 km, at 0059 UTC 31 May 2003, Greeley, Colorado. (a) PPI plot of the attenuated reflectivity; (b) PPI plot of the estimated K dp using conventional approach on the principal F dp ; (c) PPI plot of the estimated K dp using the adaptive estimation on the angular F dp ; and (d) the scattering plot between the estimated K dp and the attenuation-corrected Z h, where the black points illustrate the conventional estimates and the gray points illustrate the adaptive estimates. are shown in Fig. 6c. Visually, these two estimators give a comparable K dp field in terms of the overall pattern and intensity. For this case, the radar has a range resolution of 48 m, and the conventional approach has better estimates for large K dp observations than the case of lower range resolutions. However, the K dp field in Fig. 6c matches the storm structure much better, and the negative K dp in Fig. 6b is largely eliminated. The improved quality of the new adaptive K dp is also shown in the K dp Z h scatterplot in Fig. 6d. 6. Discussion and summary Challenges in the estimation of the specific differential propagation phase lie in the existence of phase wrapping and statistical fluctuations in the range profiles of the differential propagation phase, because differentiation is involved. Even though practical unfolding approach exists, the current implementation involves several ad hoc adjustments. Nevertheless, this procedure has been very useful so far and has been extensively used at CHILL. Phase wrapping is in fact an artifact coming from mapping an angular variable along unit circle into the real axis. In this paper, the estimation is shifted to the complex-valued range profile of the differential propagation phase exponentials. This yields a simpler estimation that is inherently immune to phase wrapping problem. Its numerical evaluation requires better accuracy as a result of the exponential form function. To this end, a regularization framework was developed, and a cubic smoothing spline was adopted to fit a better regression function. The specific differential phase is an important parameter derived from dual-polarization radars. Using the specific differential phase can improve the accuracy of quantitative rainfall estimation. However, its estimates are highly susceptible to large variance in light and moderate rain, where K dp is intrinsically small; on the other hand, its estimates are likely underestimated in heavy rain, where it is intrinsically large. These two factors could deter its wider use in quantitative rainfall mapping. A new algorithm is introduced to incorporate adaptivity through scaling the regression errors for estimation of K dp.

12 2576 J O U R N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L O G Y VOLUME 26 FIG. 6. CASA X-band radar observations at elevation angle of 2.08 and gate spacing of km, at 0708 UTC 14 Jun 2007, in Rush Springs, Oklahoma. (a) PPI plot of the attenuated reflectivity; (b) PPI plot of the estimated K dp using conventional approach on the principal F dp ; (c) PPI plot of the estimated K dp using the adaptive estimation on the angular F dp ; and (d) the scattering plot between the estimated K dp and the attenuation-corrected Z h, where the black points illustrate the conventional estimates and the gray points illustrate the adaptive estimates. Bias and variance are both typically a dilemma in inferring estimation from noisy data. During filtering, the bias and variance can be controlled through the cutoff frequency of the filter. As the sampling interval changes, new filter design needs to be in place. In the new adaptive algorithm, the bias and variance are controlled through a single Lagrangian parameter, or smoothing factor, which is also the only control parameter bounded. A simple method is proposed based on equilibrium principletochoosethesmoothingfactor.theproposedalgorithm is well suitable for real-time implementation, primarily because of the banded matrix structure in the governing equations. It takes the same order of computation time as the conventional filtering or fitting procedure. The adaptive technique has been implemented in the CSU CHILL radar, and it is expected to find importance in operational applications (Chandrasekar et al. 2008). Acknowledgments. This research is supported by NASA via the Precipitation Measurement Mission (PMM) program and by Colorado State University. The authors acknowledge the opportunity to test the algorithm for CASA radar data. The CSU-CHILL radar facility is supported by the National Science Foundation through ATM

13 DECEMBER 2009 W A N G A N D C H A N D R A S E K A R 2577 TABLE A1. Specification of FIR range filters to pass F dp trend at 3-km scale. Gate spacing (km) Normalized frequency Order APPENDIX A Range Filter Design for Differential Propagation Phase The Nyquist sampling rate in range is directly related to the along-range gate spacing. As shown in (11), the cutoff frequency depends on the gate spacing and the required cutoff scale of range. Because of these changes, the range filter needs to be redesigned for different gate spacings. As Dr decreases, the normalized passband width becomes narrower and more samples are available for the same pathlength. Using an FIR filter, its order can be correspondingly increased. The specifications of the FIR filters used in this paper are listed in Table A1, and their frequency responses are plotted in Fig. A1. APPENDIX B Solution of the Cubic Smoothing Spline The cubic splines are required to be continuous in the intermediate nodes; therefore, where a k h 3 k 1 b k h2 k 1 c k h k 1 d k 5 d k11, h k 5 r k11 r k. (B1) (B2) Additionally, their first-order and second-order derivatives are also required to be continuous, which leads to 3a k h 2 k 1 2b k h k 1 c k 5 c k11 and (B3) 6a k h k 1 2b k 5 2b k11, respectively. The following define vectors and matrices as (B4) y 5 [ f (r 1 ) f (r 2 )... f (r M ) ] T ; (B5) b 5 [ b 2 b 2... b M 1 ] T ; (B6) d 5 [ d 1 d 2... d M ] T ; (B7) FIG. A1. Frequency responses of FIR range filters at various gate spacings. 2 3 p 1 h 1 h 1 p 2 h 2 M 5 h 2 p., p M 1 p k 5 2(h k 1 1 h k ), h ; (B8) 2 p 1 q 1 h 1 q 1 h 1 p 2 q 2 h 2 M q 5 q 2 h 2 p p M 1 3, 7 5 p k 5 2(q k 1 h k 1 1 q k h k ), q 0 5 h 0 5 0; and (B9) 2 3 l 1 u 2 l 2 l 2 u 3 l 3 Q 5 6., l M 1 u M u k 5 (l k 1 1 l k ), l k 5 3 h k. (B10) The continuity constraints in (B1), (B3), and (B4) can be combined in matrix form as (Pollock 1999) M b 5 Q d, l M (B11) where M is an invertible square matrix. Similarly, the scaled objective function (28) used for adaptive smoothing can be written as L 5 l(y d) H W(y d) bh M q b. (B12)

14 2578 J O U R N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L O G Y VOLUME 26 Substitute (B11) into (B12) to express the objection function in term of d, and the minimization can be solved by setting the derivative with respect to d to zero; that is, lw(y d) QH M H M q M 1 Q d. Using (B11) in (B13), we can further derive (B13) Q y M b 5 2 3l QW 1 Q T M 1 M q b and (B14) y d 5 2 3l W 1 Q T M 1 M q b. (B15) Therefore, the coefficients b and d can be solved as b 5 M l QW 1 Q T M 1 M q Q y and (B16) d 5 y 2 3l W 1 Q T M 1 M q b. (B17) The solution to the cubic smoothing spline as defined in (24) can be easily found by setting W as an identity matrix and M q 5 M; that is, b 5 M l QQT Q y and (B18) d 5 y 2 3l QT b. (B19) With b and d available, a can be solved from (B4). Replacing a in (B1) using (B4) arrives the solution to c as c k 5 d k11 d k x k11 x k b k11 1 2b k 3(x k11 x k ) REFERENCES k M 1. (B20) Aydin, K., V. N. Bringi, and L. Liu, 1995: Rain-rate estimation in the presence of hail using S-band specific differential phase and other radar parameters. J. Appl. Meteor., 34, Bringi, V. N., and V. Chandrasekar, 2001: Polarimetric Doppler Weather Radar: Principles and Applications. Cambridge University Press, 662 pp. Chandrasekar, V., S. Lim, and E. Gorgucci, 2006: Simulation of X-band rainfall observations from S-band radar data. J. Atmos. Oceanic Technol., 23, , D. J. McLaughlin, J. Brotzge, M. Zink, B. Philips, and Y. Wang, 2007: Distributed collaborative adaptive radar network: The CASA IP-1 network and tornado observations. Preprints, 33rd Conf. Radar Meteor., Cairns, Australia, Amer. Meteor. Soc., 13A.3A., A. Hou, E. Smith, V. N. Bringi, S. A. Rutledge, E. Gorgucci, W. A. Petersen, and G. S. Jackson, 2008: Potential role of dualpolarization radar in the validation of satellite precipitation measurements: Rationale and opportunities. Bull. Amer. Meteor. Soc., 89, Craven, P., and G. Wahba, 1979: Smoothing noisy data with spline functions. Numer. Math., 31, Golestani, Y., V. Chandrasekar, and V. N. Bringi, 1989: Intercomparison of multiparameter radar measurements. Preprints, 24th Conf. Radar Meteorology, Tallahassee, FL, Amer. Meteor. Soc., Gorgucci, E., G. Scarchilli, and V. Chandrasekar, 1999: Specific differential phase estimation in the presence of nonuniform rainfall medium along the path. J. Atmos. Oceanic Technol., 16, Hubbert, J., and V. N. Bringi, 1995: An iterative filtering technique for the analysis of copolar differential phase and dualfrequency radar measurements. J. Atmos. Oceanic Technol., 12, ,, and V. Chandrasekar, 1993: Processing and interpretation of coherent dual-polarized radar measurements. J. Atmos. Oceanic Technol., 10, Jameson, A., 1985: Microphysical interpretation of multiparameter radar measurements in rain. Part III: Interpretation and measurement of propagation differential phase shift between orthogonal linear polarizations. J. Atmos. Sci., 42, Mueller, E. A., 1984: Calculation procedure for differential propagation phase shift. Preprints, 22nd Conference on Radar Meteorology, Zurich, Switzerland, Amer. Meteor. Soc., Oppenheim, A. V., and R. W. Schafer, 1989: Discrete-Time Signal Processing. Prentice Hall, 870 pp. Pollock, D. S. G., 1999: A Handbook of Time-Series Analysis, Signal Processing and Dynamics. Academic Press, 782 pp. Reinsch, C. H., 1967: Smoothing by spline functions. Numer. Math., 10, Sachidananda, M., and D. S. Zrnić, 1989: Efficient processing of alternately polarized radar signals. J. Atmos. Oceanic Technol., 6, Seliga, T. A., and V. N. Bringi, 1978: Differential reflectivity and differential phase shift: Applications in radar meteorology. Radio Sci., 13,