Validation of the CloudSat precipitation occurrence algorithm using the Canadian C band radar network

Transcription

1 JOURNAL OF GEOPHYSICAL RESEARCH, VOL. 113,, doi: /2008jd009992, 2008 Validation of the CloudSat precipitation occurrence algorithm using the Canadian C band radar network David Hudak, 1 Peter Rodriguez, 1 and Norman Donaldson 1 Received 19 February 2008; revised 13 May 2008; accepted 19 June 2008; published 19 September [1] The ability of CloudSat to detect precipitation in cold season cloud systems is examined using data from the Environment Canada C band weather radar at King City, Ontario. The factors complicating the comparison are the time mismatch, the differences in sensitivity, and the changes to the geometry of cross section with range from the ground radar, W band radar attenuation, and the effect of ground clutter. A total of 40 overpasses with precipitation were observed over the King City radar from September 2006 to April In about 14% of the precipitation profiles, time mismatches were diagnosed. When these cases were removed, the skill scores of the CloudSat precipitation occurrence product were excellent. The most frequent cause of a false detection was an incorrect precipitation threshold in the algorithm. The most frequent cause of a miss in detection was ground clutter removal of valid echoes by the algorithm. Overall, the CloudSat algorithm handled the effect of attenuation very well. Improvement to the algorithm would arise from a better tuning of the precipitation threshold, a threshold of 10 dbz instead of 18 dbz being more appropriate for winter storms in the Great Lakes area, and more effective ground clutter filtering in the lowest four range bins of the CloudSat data. The methodology employed here and the 1456 verified precipitation profiles from CloudSat can serve as a framework for a test bed to evaluate precipitation products from CloudSat. Citation: Hudak, D., P. Rodriguez, and N. Donaldson (2008), Validation of the CloudSat precipitation occurrence algorithm using the Canadian C band radar network, J. Geophys. Res., 113,, doi: /2008jd Cloud Physics and Severe Weather Research Section, Environment Canada, King City, Ontario, Canada. Published in 2008 by the American Geophysical Union. 1. Introduction [2] CloudSat has been collecting science data since the beginning of operations on 2 June An important yet novel feature of the W band radar on board CloudSat is its ability to observe jointly the properties of clouds and precipitation [Stephens and the CloudSat Science Team, 2002]. This enables cloud properties and the frequency of occurrence of detectable precipitation in these same clouds to be deduced. Recent work by Mace et al. [2007] and Marchand et al. [2008] have addressed the ability of CloudSat to detect clouds. In this work, we will address the ability of CloudSat to detect precipitation. [3] With growing demand for water and with the impact of floods/drought on society, the need to better understand the water cycle and the availability of water are of critical concern to all nations. A recent review entitled Earth Science and Applications from Space: National Imperatives for the Next Decade [NRC, 2005] advocated in the strongest terms the need for an international effort to provide more accurate and frequent precipitation measurements from space. The Global Precipitation Mission (GPM), a flagship mission of NASA s Earth Science program that is addressing this concern, is scheduled for launch in GPM ( would build on the success of the Tropical Rainfall Measuring Mission that was launched in 1997 [Kummerow et al., 2000]. [4] CloudSat bridges a critical gap between the precipitation measurements of TRMM at low latitudes and the more global precipitation measuring goals of GPM. The precipitation information contained in the CloudSat observations also serves as an important precursor, global database for developing and testing new global precipitation algorithms required for GPM during the prelaunch phase of the mission. Precipitation products derived from CloudSat observations are attempting to estimate the precipitation efficiency of cloud systems [Haynes and Stephens, 2007], study the timeline of the autoconversion process from nonprecipitating to precipitating clouds [Stephens and Haynes, 2007], and more generally to study critical hydrological processes in the context of weather and climate variability. An enhanced science effort to develop more elaborate and sophisticated precipitation products for snow and rainfall is underway. [5] The CloudSat cloud profiling 94 GHz W band radar (CPR) has a broad dynamic range that is well suited to detect light precipitation including drizzle and snowfall, which are important modes of precipitation at middle-to-higher latitudes. The disadvantage of the relatively high frequency of the CPR is the significant attenuation of the signal in even 1of12

2 Figure 1. Typical daily CloudSat satellite overpasses over the Environment Canada (EC) C band radar network. King City radar coverage is highlighted in red. modest rain rates and the issue of multiple scattering. There has been an emphasis on enhanced precipitation characterization in the CloudSat community that can alleviate some of these shortcomings. L Ecuyer and Stephens [2002] developed an optimal estimation-based algorithm for retrieving profiles of rainfall that accounts for attenuation. In this technique, the path-integrated attenuation derived from surface echo measurements is used as a constraining feature. Matrosov [2007] takes advantage of the high attenuated reflectivity and the fact that the variability of nonattenuated reflectivity of rainfall diminishes as Mie scattering becomes dominant at W band frequencies. The technique estimates height derivatives of measured reflectivities to retrieve profiles of rainfall. Battaglia et al. [2007] has evaluated the CloudSat CPR configuration to provide an estimate of positive biases in retrieved rainfall estimates due to multiple scattering enhancement of the signal. This effect becomes significant at rain rates R > 5 mm h 1 and overwhelming at R > 25 mm h 1. [6] The current work addresses CloudSat results from cold season precipitation and has two main objectives. First, it uses data collected by an Environment Canada (EC) C band weather network radar to assess Release 4 output of the precipitation detection algorithm in the CloudSat 2B CLDCLASS product. Second, the analysis will be used to provide a framework for validation of future CloudSat precipitation related products. [7] The paper describes the validation data from the EC radar network and the CLDCLASS precipitation algorithm in section 2 and the methodology used to match the two data sets in section 3. Section 4 gives specific examples and the global statistics for one winter of the comparison. The final section describes the significance of the results with respect to the specific operational constraints of the CPR. 2. Data [8] The EC radar network consists of 30 C band Doppler radars concentrated in the southern part of Canada (Figure 1). Joe and Lapczak [2002] gives the radar specifications. The scan strategy employed throughout the network operates on a 10-min cycle. It consists of a 24 angle reflectivity-only volume scan (termed CONVOL) out to a range of 256 km with 1.0 km bins. This scan takes approximately 5 min and can be used to build vertical cross sections through echoes from the radar. This is followed by four Doppler plan position indicator (PPI) scans (DOPVOL) at elevation angles from 0.2 to 3.5 out to 113 km with 0.5 km bins. The Doppler scans are performed with a higher spatial resolution and a slower scanning rate and includes a FFT Doppler ground clutter filter. The lowest DOPVOL scan is scheduled 1 min after the lowest angle of CONVOL. The Doppler processing from DOPVOL can be used to determine whether low-level echoes are clutter or precipitation. [9] On a given day CloudSat will pass over only a few radars, given the narrow CPR swath and CloudSat s 16 day return cycle. Figure 1 displays the CloudSat passes over Canada on a typical day. Usually there are four ascending 2of12

3 Figure 2. An example depiction of the output from the CloudSat CLDCLASS product showing (top) cloud type and (bottom) precipitation occurrence indicator. Terrain is shown as a yellow line. See text for further explanation. and four descending orbits that pass over at least one of the radars in the EC network per day. In a given winter, there would be about 100 satellite overpasses over an individual radar. [10] The CloudSat data processing operates in a hierarchical structure. The 1B-CPR product handles the conversion of the CPR measurements to calibrated values. Elements in the processing include an estimate of the noise level and standard deviation, the identification and removal of the surface bin, and a means for identifying the impact of surface clutter on the lowest four bins above the surface. The conversion to calibrated reflectivities and the application of a cloud mask, along with a quality flag is performed in the 2B-GEOPROF product [Marchand et al., 2008]. The quality flag indicates the likelihood of detection of hydrometeors. It is based on signal to noise considerations as well as the application of a spatial box filter to assess spatial correlations. This filter, seven along track bins wide by five vertical bins in height, is used to reduce false detections due to signals near the noise level or spikes in the data by nonmeteorological targets such as airplanes. The product 2B-CLDCLASS uses the GEOPROF output at a quality level that rejects most noise and spikes as input to determine cloud type. In addition, the precipitation algorithm in CLDCLASS which is an integral part of the cloud typing makes a determination of ground precipitation occurrence and its phase. Detailed information about the CloudSat data products is contained in the CloudSat Data Products Handbook ( [11] The algorithms contained in the CLDCLASS product are described in detail by Sassen and Wang [2008]. Figure 2 shows an example of the output. The top time-height plot shows cloud type as a colored cloud mask (legend at right side) and the precipitation information is contained in the time bar at the bottom. This indicates whether drizzle, solid (snow), or liquid (rain) was identified at the base of the corresponding column. Cloud clustering is first carried out in both the vertical and horizontal directions to identify extended cloud layers. The reflectivity (Z) at the fifth range gate above the surface (1.2 km agl) is used as a proxy for the surface echo due to ground clutter contamination of the CPR signal in the lowest four range bins. Precipitation occurrence is then identified if where Z 5 Z thresh ðdbzþ ð1þ Z thresh ¼ 10 if T < 10 CTin ð C at the height of Z 5 Þ ¼ Tif 10 < T < 0 ¼ 0ifT> 0 C [12] Temperature information is obtained from the ECMWF model analysis that is an auxiliary part of the CloudSat data package. Precipitation occurrence is also declared if it is determined that the regional clear sky reference signal at the surface minus the measured surface 3of12

4 Figure 3. An example of a CloudSat overpass over an EC C band radar (Marble Mountain, Newfoundland on 21 September 2006): (a) plan position indictor view with the satellite swath crossing the EC ground radar coverage. Beam heights are indicated for the corresponding range rings. (b) (top) Matched reflectivity fields from the CloudSat cloud profiling 94 GHz W band radar (CPR) and (bottom) the vertical cross section derived from the EC radar volume scan data. The dark red curve on both plots indicates the lower spatial bound of the EC radar data. return is more than 6 db. This would indicate that strong W band attenuation has taken place. [13] Precipitation phase is identified as follows. If a bright band is found in the vertical profile of reflectivity or if the surface temperature is >2 C, then the precipitation type is liquid. Otherwise it is solid. In addition, if there is a single low stratus or stratocumulus layer identified and the max- 4of12

5 Figure 4. Distribution of reflectivities from the CPR and vertical cross sections of reflectivity (VCS) above 5 km corresponding to Figure 3b. imum reflectivity is > 18 dbz, then precipitation is identified as drizzle. 3. Methodology [14] The analysis strategy is to determine the projection of the satellite track across the radar coverage and compare the CPR echoes to a cross section through nearly synchronous data from the C band surface radars. Precipitation occurrence estimates from both systems will then be compared. [15] The first step in the matching of data from the CPR and from an EC radar is to create vertical cross sections of reflectivity (VCS) from the volume scan. This is carried out through the use of a lookup table for each radar site derived from the nadir track of the satellite. This transforms slant range, and antenna azimuth and elevation of the data from a radar in the EC National Radar Program (NRP) to ground range and height at the CloudSat data resolution of 1.0 km in the horizontal and 240 m in the vertical. In this way, a nadir satellite ray from the CPR can be matched to a corresponding EC radar vertical profile. In this approach, no interpolation is performed; the closest actual measurement is used in the creation of the VCS. Since the NRP radar beam broadens with range, the quality of the matching will be better closer to the radar. [16] Figure 3 is an example of the comparison of reflectivity products between an EC radar and CloudSat. Figure 3a gives the location of the satellite track on the closest lowlevel PPI from CONVOL. Figure 3b shows the CPR reflectivity swath within the EC radar coverage (top) and the corresponding VCS along the satellite track (bottom). Figure 4 shows a comparison of the distributions of reflectivity for the colocated coincident echoes above 5 km in Figure 3b. At those altitudes during winter, the CPR attenuation would not be significant. The comparison is quite good for Z < 15 dbz. For Z > 15 dbz, the effect of resonant scattering of W band would limit the maximum reflectivity and account for the discrepancy [Matrosov, 2007]. The good correspondence of reflectivity in Figure 4 is reassuring for the purpose of using the EC radar data to validate the precipitation algorithm from CLDCLASS. Nevertheless, a number of factors need to be considered in the interpretation of the cross sections. [17] The first factor is whether times are matched sufficiently well. With the EC radar scanning cycle repeat time of 10 min, the time offset between the CPR data and the VCS can be up to 5 min. This source of error was minimized through an inspection of the horizontal variability in the precipitation field on the EC radar. When this was suspected to be a significant problem, the corresponding sections of the swath were removed from the data set. [18] The second factor is variation of radar sensitivity with location along the VCS. The minimum detectable signal (MDS) of the EC radar will decrease with range from the radar along the VCS. The CPR, on the other hand, has effectively a constant MDS. In Figure 3b this effect is demonstrated in the cloud layer from 6.0 to 8.0 km in height between about 49 N and 51 N. Near the radar (49 N), the cloud layer is detected by the EC radar. Further north beyond 49.6 N (locations A), this upper layer is not detected due to its increasing distance from the ground radar. [19] The third factor is the variation of the cross-section geometry of the VCS along the swath. The dark red curve indicates the bottom of the VCS that rises with range from the radar due to Earth curvature effects. For example, at locations B in Figure 3b a low-level echo seen by CPR (top) is not seen by NRP (bottom) due to overshooting of the VCS. Consequently, the comparison is limited to those parts of the radar coverage where the bottom of the VCS is below 2.0 km. This corresponds approximately to a range of 180 km from the radar. [20] The fourth factor is radar attenuation. For the cold season conditions considered in this study, attenuation should not significantly affect a C band radar at the close ranges dictated by the VCS geometry restriction. However, the CPR with its higher frequency would have significant attenuation effects. At locations C in Figure 3b, the CPR reflectivity is comparable to the VCS in the higher layers above 5 km. Below 3.0 km, the CPR reflectivity is significantly less that the VCS reflectivity. At locations D, there is the complete attenuation of the CPR in the strong reflectivity core as seen on the VCS. That there is some residual echo on the CPR swath at its location D just below the main attenuating layer near 2.5 km suggests that multiple scattering may be occurring as well. [21] The final factor is the effect of ground clutter and the attempt of the CPR and EC radar algorithms to remove it. Figure 5 compares the echo mask occurrence corresponding to Figure 3b. Area A is an example where there is a discrepancy. The CloudSat cloud mask has removed the echo below 1.0 km. This is due to the ground clutter filtering algorithm whereby low level CPR data that is below that magnitude expected from ground clutter in the first four bins is flagged as suspect in the 1B-CPR product and then removed in later processing algorithms in GEO- PROF and CLDCLASS. The EC radar shows a valid echo to the bottom of the VCS. Although not applicable in this case, other examples where low-level echoes extend just a bin or two above the ground clutter area can also removed by the spatial box filter applied as part of GEOPROF. Most of the areas where there is a CPR echo and no EC radar 5of12

6 Figure 5. A comparison of the echo occurrence between CPR and VCS (21 September 2006) corresponding to Figure 3b. See text for further explanation. echo are due to sensitivity as discussed previously. However, area B demonstrates a column of data that has been removed from the VCS due to ground clutter filtering, but it is still evident in the CPR data. [22] Precipitation occurrence along the VCS is determined from the EC data with the following criteria. Echoes must extend upward from the VCS base and be at least 480 m thick (two CPR bins). Then, the low-level DOPLVOL PPI is compared with the lowest angle of CONVOL on which the VCS is based. If the Doppler-based ground clutter filter has removed an echo or if the reflectivity in CONVOL is more than 10 dbz more than the DOPVOL value, then the profile is declared as ground clutter and removed from the analysis. For this reason, the comparison is further limited to those portions of the VCS that lie within the DOPVOL PPI range (113 km). [23] Tables 1a and 1b summarize the comparison statistics. The precipitation occurrence from the CloudSat algorithm in the CLDCLASS product is considered the estimate. The precipitation occurrence from the EC radar algorithm is taken as truth. The following four measures are used to assess the skill of the CLDCLASS precipitation occurrence results: critical success index (CSI), bias, probability of detection (POD), and false alarm ratio (FAR). They are defined in Tables 1a and 1b [Wilks, 1995]. [24] Some aspects of the precipitation algorithm in CLDCLASS could lead to systematic errors. Errors that Table 1a. Contingency Table a Observation = EC Radar Yes No Estimate = CloudSat Yes HIT FALSE positive No MISS Correct negative ( NEG ) a Contingency table and definitions are from Wilks [1995]. will result in a FALSE categorization in Tables 1a and 1b are as follows: [25] 1. The first error is the precipitation threshold. The CPR with a minimum MDS 30 dbz is sensitive to both cloud and precipitation. CLDCLASS could mistakenly classify a nonprecipitating cloud as precipitation occurrence depending on the algorithm threshold. [26] 2. The second error is ground clutter filtering. The VCS at times can detect echoes closer to the surface than the CPR due to the CLDCLASS ground clutter filter. This filter would remove the bins in the lowest kilometer where the ground clutter signal is masking the fact that the echo that the CPR collected above 1 km is not reaching the surface. CLDCLASS could declare precipitation occurrence based on the measurement in the fifth range bin, whereas in fact virga is occurring as the precipitation is dissipating before it reaches the surface. [27] Errors that could result in a MISS categorization in Tables 1a and 1b are as follows. [28] 1. The first error is the reflectivity gradients. The CLDCLASS algorithm performs clustering of the echoes in both the horizontal and vertical. Also, the GEOPROF product performs spatial box filtering. Both effects could distort or alter bins with valid CPR measurements initially above the precipitation threshold to below the threshold. Table 1b. Scoring Definitions a Score Meaning Formula CSI critical success index 100 [HIT/(HIT + MISS + FALSE)] Bias forecast : observation 100 [(HIT + FALSE)/ (HIT + MISS) 1] POD probability of detection 100 [HIT/(HIT + MISS)] FAR false alarm ratio 100 [FALSE/(HIT + FALSE)] a Contingency table and definitions are from Wilks [1995]. 6of12

7 Figure 6. (a) Reflectivity fields for CPR (top) and VCS (bottom) (18 September 2006 overpass of King City radar). The color bars indicate the precipitation occurrence flag for the profile according to the legend directly below. For the CPR, liquid, sold, and drizzle are identified. For the VCS, categories are clear, no precipitation, ground clutter, precipitation occurrence based on 24 angle reflectivity-only volume scan (CONVOL) data, and precipitation occurrence based on Doppler plan position indicator scans (DOPVOL) data. (b) Vertical profile of reflectivity for the CPR and VCS at the location corresponding to the arrow in Figure 6a. 7of12

8 Figure 7. Same as Figure 6a for 23 December 2006 overpass of King City radar. The multiple arrows mark locations where the CLDCLASS and EC radar algorithms disagree. [29] 2. The second is attenuation. The attenuation of the CPR signal can be attenuated in significant precipitation cores or multilayer cloud systems. The CPR measurement may be reduced below the precipitation occurrence threshold. [30] 3. The third is ground clutter filtering. Shallow precipitation echoes that have sharp increases in reflectivity with decreasing altitude and that only go above the precipitation threshold in the lowest km could be missed by the CLDCLASS algorithm due to ground clutter removal in the lowest 1.0 km. [31] These five factors will all be considered in the evaluation of the performance of CLDCLASS. The first stage of the study, reported here, is to consider the data for the cold season of for CloudSat swaths over the EC radar site at King City, Ontario. This radar operates as both an operational radar and a research radar and also has dual polarization technology [Hudak et al., 2006a]. The data is considered of the highest quality and there is a good familiarity with the local ground clutter patterns. [32] For the 7 months of this study (September 2006 to March 2007) there were a total of 104 overpasses of CloudSat. In 40 cases there was detectable precipitation by both the CPR and the King City radar. Specific examples of MISS and FALSE will be presented in section 4; then the overall statistics of the comparison will be given. 4. Results [33] The first case is an example of a swath on 28 September 2006 where some of the profiles were incorrectly classified as no precipitation by CLDCLASS. Figure 6a shows the two reflectivity swaths (CPR and VCS). The arrow indicates a region where an extensive area of precipitation was indicated by the King City radar, but there was a distinct break in the precipitation occurrence classification by CLDCLASS. This resulted in nine CPR profiles classified as MISS. It appears that there are two factors at work here. The region has a multilayered cloud system with precipitation. The clustering that is done may be in part responsible for the misidentification. Marchand et al. [2008] found the same problem in identifying cloud features with the CPR when compared to lidar measurements from CALIPSO. Second, the upper layer is attenuating the CPR signal somewhat. Figure 6b gives the mean profiles of CPR and VCS in this area. The reduction in the CPR reflectivity with decreasing altitude is primarily due to attenuation of 8of12

9 Figure 8. (a) Same as Figure 6a for 5 December 2006 overpass of King City radar. (b) Vertical profile of reflectivity for the CPR and VCS at the location corresponding to the arrow in Figure 8a. the CPR, most of it due to the higher reflectivity in the lower cloud layer. The reflectivity in the lowest bin of CPR falls below the 0 dbz threshold for precipitation occurrence. Hence an incorrect no precipitation occurrence determination was made by CLDCLASS. [34] A second example of a MISS is shown in Figure 7. South of approximately 44.1 N, the VCS shows an area of precipitation interspersed with sections declared as ground clutter (Figure 7, bottom). The area is in a heavy ground clutter zone and the EC algorithm at times indicated 9of12

10 Table 2a. Contingency Table Results Result Total Error Characterization Subtotal HIT 1456 FALSE 107 Precipitation threshold 80 Ground clutter filtering 27 MISS 81 Reflectivity gradients 16 Attenuation 6 Ground clutter filtering 59 NEG 4198 ground clutter rather than precipitation. In this same area, CLDCLASS classified most of the profiles as no precipitation (Figure 7, top). In this case, 31 profiles were incorrectly classified as no precipitation by CLDCLASS. The ground clutter filter removed the echoes in the lowest 1.0 km, where according to the VCS the reflectivity was sufficiently strong to have triggered precipitation occurrence in the algorithm. Consequently, CLDCLASS incorrectly identifies this as no precipitation when in fact there was significant precipitation from low topped echoes. The magnitude of the CPR reflectivity was generally lower than the King City radar VCS reflectivity. This might have been due to a slight mistiming of the data. However, an examination of the low-level PPI at the time confirmed that widespread snow was in the area and the correct designation should be precipitation occurrence. These conditions are typical of shallow snow streamers produced to the lee of the Great Lakes during cold air outbreaks, as described by Niziol et al. [1995]. [35] Figure 8 is an example of a FALSE detection from CLDCLASS. The VCS (Figure 8a, bottom) shows an elevated echo but that precipitation is not reaching the ground. This is a case of virga. The CPR also detects this layer (Figure 8a, top). Owing to the ground clutter masking of the lowest 1.0 km, CLDCLASS extrapolates this echo to the surface and falsely declared it a precipitation profile. Figure 8b compares the profiles near the arrow in Figure 8a. The VCS reflectivity profiles clearly show the decrease in reflectivity in the lowest levels to below the precipitation threshold, thus indicating no precipitation occurrence. This decrease was not detected by the CPR and CLDCLASS incorrectly identified this profile as precipitation occurrence. [36] An examination was done on all of the profiles within the 40 swaths in which precipitation occurrence was identified in one of CPR or VCS, but not both, i.e., all the profiles that were either FALSE or MISS. A determination was first made by a subjective inspection of the low-level PPIs whether the precipitation field was sufficiently variable that the timing error between the two data sets might be responsible for the discrepancy. Those profiles in the derived cross sections which could be significantly affected by timing errors were then removed Table 2b. Corresponding Skill Scores a Score Value CSI 88.6% Bias 2% POD 94.7% FAR 6.9% a See text for details. from the data set. There were approximately 200 profiles that were removed due to possible mistiming errors. This is 13.7% of the total number of hits. Without the removal of these profiles, this source of uncertainty would dominate the analysis. The remaining profiles were then categorized by the nature of the error. Tables 2a and 2b summarize the results of the comparison. [37] The CSI is 88.6%, the POD is 94.7%, and the FAR 6.9%. There is a positive bias of 2% in the occurrence of precipitation. The few errors that are either FALSE or MISS are almost equal so that the precipitation occurrence estimate is essentially unbiased. The unbiased result combined with a high POD and low FAR indicates that the CLDCLASS precipitation occurrence algorithm is performing very well. [38] Of the 107 profiles that were misidentified as precipitation and were FALSE, 80 were due to sensitivity thresholds in the algorithm. Twenty-seven times the FALSE classification was due to the presence of virga. Of the 81 profiles that were misclassified as no precipitation and were MISS, 59 were due to the ground clutter filter, six were due to the effects of attenuation on CPR, and 16 were due to errors in the clustering of the CPR data. [39] There was a sense during the course of the analysis that the threshold for the CLDCLASS drizzle category at 18 dbz was not appropriate for Canadian winter clouds. This was reflected in the fact that 75% of the false category was deemed due to sensitivity thresholding. Nonprecipitating low clouds with a high preponderance of ice, a frequent occurrence in the study area in the winter, could regularly achieve Z > 18 dbz. As a result, the analysis was repeated with the drizzle category set to no precipitation in CLDCLASS. The revised contingency table is given in Table 3. The effect of this change was that 124 profiles had their designation changed from precipitation (drizzle) to no precipitation. Fifty-nine profiles deemed as HIT were changed to MISS; 65 FALSE were turned into correct negative (NEG). That the change in MISS and FALSE were comparable meant that the CSI stayed about the same. Both the POD and FAR dropped about 4%. The bias indicates that the CLDCLASS now underestimated precipitation occurrence by 6%. The net result was that there was no improvement in the precipitation occurrence if the drizzle class were eliminated. It captures a segment of the precipitation, although in this climate regime it is indicative of low-level snow as well as drizzle cases. [40] The distribution of maximum reflectivity for the 124 profiles classified by CLDCLASS as drizzle, subdivided by those cases in which precipitation was observed Table 3. Contingency Table Results and the Corresponding Skill Scores With Drizzle Cases Set to No Precipitation in CLDCLASS Result Total HIT 1397 FALSE 42 MISS 140 NEG 4263 Score Value CSI 88.5% Bias 6% POD 90.9% FAR 2.9% 10 of 12

11 (blue) and not observed (red), is given in Figure 9. A bimodal distribution is very evident. If a threshold of 10 dbz was applied to the drizzle instead of 18 dbz, only 12 of the original HIT would change to MISS, while 54 of the FALSE would change to NEG. The skill scores for this intermediate threshold is better than either the original algorithm or by eliminating the drizzle category altogether. The BIAS and POD are within 1% of those in the original algorithm, while CSI improves by 2% and FAR decreases by 3%. The improvement in the statistics with this new threshold is representative of the Great Lakes area but would need to be reexamined using other EC radars in various regions across Canada. [41] Further insight into the challenges involved in characterizing winter precipitation is seen by comparing reflectivity for the 1456 profiles that were deemed hits (Figure 10). The distribution was constructed from bins in which both radars reported nonzero echoes after filtering. The CPR values are systematically lower. The mean of CPR reflectivity was 2.1 dbz, compared to 10.4 dbz from King City radar VCS. Also, the overall distribution of CPR reflectivity is narrower than for the VCS. Ninety five percent of the CPR data lie between 15.7 dbz and 13.0 dbz, while for King City radar the range is from 5.5 dbz to 29.0 dbz. The results of Figure 10 are consistent with both Mie scattering and attenuation limiting the magnitude of the CPR reflectivity. At the lower end of the distribution, the lower values of CPR reflectivity is an indication of the effect of attenuation and perhaps at the extreme lowest end (< 20 dbz), multiple scattering is introducing spurious weak signals. Matrosov [2007] estimated the values of nonattenuated W band data in rain would not exceed 26 dbz due to resonant scattering effects at this frequency. The maximum reflectivity from the CPR is 17 dbz. It should also be noted that stronger reflectivities in the study area in the winter frequently occur either with wet snow or a very low altitude so-called bright band. This suggests a complex, nonlinear correlation between W band radar attenuation and precipitation rate in winter conditions. Also, the cold season reflectivity values of the King City radar as given in Figure 10 are significantly lower than its values in the summertime. The higher reflectivity and associated precipitation rates in the summer will more consistently fall in the Mie scattering regime than in the winter. This implies that the nonattenuated reflectivity will be more variable in the winter than the summer. These factors may restrict the effectiveness of height derivative techniques [e.g., Matrosov, 2007] to derive quantitative precipitation information in cold season cloud systems. Figure 9. Distribution of maximum reflectivity with CloudSat reported drizzle locations. Blue curve are indicative of profiles when EC radar also reported precipitation. Red curve are for profiles when the EC radar did not report precipitation. 5. Summary and Conclusions [42] The study has described the use of the Canadian C band weather radar network as a validation source for precipitation occurrence from CloudSat. In the development of the analysis methodology, limitations on the comparison of space-based nadir pointing radar and ground-based scanning radar have been illustrated. [43] The results demonstrated the care that must be taken in doing the data matching as a result of the limited horizontal extent of the CloudSat radar swath. In about 14% of the precipitation profiles, time mismatches of the CPR and EC radar VCS would result in discrepancies. When these errors were removed from the comparison, the performance of the CloudSat precipitation occurrence product was excellent. The precipitation occurrence product from CLDCLASS was essentially unbiased. The POD was 94.7% and the FAR was 6.9%. The method was also able to deduce the types of errors that contributed to the cases that were misclassified. As a result, it was possible to quantify their relative importance of the errors. The most frequent cause of a false detection was an incorrect precipitation threshold in the algorithm. The most frequent cause of a miss in detection was ground clutter removal of valid echoes by the CLDCLASS algorithm. In the area studied these cases were exclusively from low topped lake effect snow streamers off the Great Lakes. This points out that the precipitation statistics from CloudSat in a given climate regime need to be considered in the context of the specific operational constraints of the W band radar. In the case of Figure 10. Comparative reflectivity distribution of CPR and King City radar for the 1456 profiles designated as HIT. 11 of 12

12 southern Ontario, the snow streamers are not well represented in the CPR data. [44] Overall, the effect of attenuation of the CPR, an issue of considerable concern beforehand, was handled very well by the CLDCLASS algorithm for winter cases. Improvement to the algorithm would arise from a better tuning of the precipitation threshold to regional circumstances and more effective ground clutter filtering in the lowest four ranges bins of the CPR. [45] The methodology employed here provides a framework for a test bed for the evaluation of precipitation products from CloudSat. The next step is to automate the comparison procedure based on the subjective experience from this study. Some automation issues do remain, such as a new technique to recognize time mismatches, as a replacement for human pattern recognition. It will be possible then to assess the precipitation occurrence product in other climate regimes across Canada using data from the entire EC surface radar network of 30 radars. When suitably matched with validation data, the very good detection of precipitation occurrence by the CPR was expected. This confirms the soundness of the matching strategy employed in this study. And this in turn suggests that the validation database here, consisting of 1456 verified precipitation profiles is well suited to the validation of more sophisticated precipitation products such as precipitation rates and types. Additional sources of validation from EC surface precipitation observations [Hudak et al., 2006b] can also contribute to these comparisons. [46] Acknowledgments. The CloudSat validation program was made possible through funding provided by the Canadian Space Agency (CSA). We are especially indebted to Stella Mello and Thomas Piekutowski of the CSA. References Battaglia, A., M. O. Ajewole, and C. Simmer (2007), Evaluation of radar multiple scattering effects in CloudSat configuration, Atmos. Chem. Phys., 7, Haynes, J., and G. L. Stephens (2007), Tropical oceanic cloudiness and the incidence of precipitation: Early results from CloudSat, Geophys. Res. Lett., 34, L09811, doi: /2007gl Hudak, D., P. Rodriguez, G. W. Lee, A. Ryzhkov, F. Fabry, and N. Donaldson, (2006a), Winter precipitation studies with a dual polarized C-band radar, paper presented at 4th European Conference on Radar in Hydrology and Meteorology, Serv. Meteorol. de Catalunya, Barcelona, Spain. Hudak, D., H. Barker, P. Rodriguez, and D. Donovan, (2006b), The Canadian CloudSat Validation Project, paper presented at 4th European Conference on Radar in Hydrology and Meteorology, Serv. Meteorol. de Catalunya, Barcelona, Spain. Joe, P., and S. Lapczak (2002), Evolution of Canadian operational radar network, in Proceedings of the Second European Conference on Radar Meteorology (ERAD), pp , Copernicus, Katlenburg-Lindau, Germany. Kummerow, C., et al. (2000), The status of the Tropical Rainfall Measuring Mission (TRMM) after two years in orbit, J. Appl. Meteorol., 39, , doi: / (2001)040<1965:tsottr> 2.0.CO;2. L Ecuyer, T. S., and G. L. Stephens (2002), An estimation-based precipitation retrieval algorithm for attenuating radars, J. Appl. Meteorol., 41, , doi: / (2002)041<0272:aebpra>2.0.co;2. Mace, G. G., R. Marchand, Q. Zhang, and G. L. Stephens (2007), Global hydrometeor occurrence as observed by CloudSat: Initial observations from summer 2006, Geophys. Res. Lett., 34, L09808, doi: / 2006GL Marchand, R., G. G. Mace, T. Ackerman, and G. L. Stephens (2008), Hydrometeor detection using CloudSat - An Earth orbiting 94 GHz cloud radar, J. Atmos. Oceanic Technol., 25(4), , doi: /207jte- CHA Matrosov, S. Y. (2007), Potential for attenuation-based estimations of rainfall rate from CloudSat, Geophys. Res. Lett., 34, L05817, doi: / 2006GL National Research Council (NRC) (2005), Earth Science and Applications from Space: Urgent Needs and Opportunities to Serve the Nation, 400 pp., Natl. Acad., Washington, D. C. Niziol, T. A., W. R. Snyder, and J. S. Waldstreicher (1995), Winter weather forecasting throughout the eastern United States: Part IV: Lake effect snow, Weather Forecast., 10, 61 77, doi: / (1995) 010<0061:WWFTTE>2.0.CO;2. Sassen, K., and Z. Wang (2008), Classifying clouds around the globe with the CloudSat radar: 1-year of results, Geophys. Res. Lett., 35, L04805, doi: /2007gl Stephens, G. L., and J. Haynes (2007), Near global observations of the warm rain coalescence process, Geophys. Res. Lett., 34, L20805, doi: /2007gl Stephens, G. L., and the CloudSat Science Team (2002), The CloudSat mission and the A-train. A new dimension of space observations of clouds and precipitation, Bull. Am. Meteorol. Soc., 83, , doi: /bams Wilks, D. S. (1995), Statistical Methods in the Atmospheric Sciences, 467 pp., Academic, San Diego, Calif. N. Donaldson, D. Hudak, and P. Rodriguez, Cloud Physics and Severe Weather Section, Environment Canada, King City, ON L7B 1A3, Canada. (david.hudak@ec.gc.ca) 12 of 12