Comparison of cloud products within IASI footprints for the assimilation of cloudy radiances

Transcription

1 Quarterly Journal of the Royal Meteorological Society Q. J. R. Meteorol. Soc. 137: , October 2011 B Comparison of cloud products within IASI footprints for the assimilation of cloudy radiances L. Lavanant, a * N. Fourrié, b A. Gambacorta c,g.grieco, d S. Heilliette e,f.i.hilton f, M.-J. Kim, g A. P. McNally, h H. Nishihata, i E. G. Pavelin f and F. Rabier b a Météo-France, Centre de Météorologie Spatiale, Lannion, France b Météo-France and CNRS/CNRM-GAME, Toulouse, France c NOAA/NESDIS/PSGS, Camp Springs, Maryland, USA d DIFA, Universita della Basilicata, Potenza, Italy e Meteorological Research Division, Environment Canada, Dorval, Quebec, Canada f Met Office, Exeter, UK g Cooperative Institute for Research in the Atmosphere (CIRA), Colorado State University, Fort Collins, USA and Joint Center for Satellite Data Assimilation (JCSDA), USA h ECMWF, Reading, UK i Japan Meteorological Agency, Tokyo, Japan *Correspondence to: L. Lavanant, Météo-France, DP/CMS, BP 50747, LANNION, 22300, France, , lydie.lavanant@meteo.fr The contributions of these authors to this article were prepared as part of their official duties as United States Federal Government employees. The contributions of these authors to this article were prepared as part of their official duties as Canadian government employees. The contribution of these authors was written in the course of their employment at the Met Office, UK, and is published with the permission of the Controller of HMSO and the Queen s Printer for Scotland. This article compares different methods of deriving cloud properties in the footprint of the Infrared Atmospheric Sounding Interferometer (IASI), onboard the European MetOp satellite. Cloud properties produced by ten operational schemes are assessed and an intercomparison of the products for a 12 h global acquisition is presented. Clouds cover a large part of the Earth, contaminating most of the radiance data. The estimation of cloud top height and effective amount within the sounder footprint is an important step towards the direct assimilation of cloud-affected radiances. This study first examines the capability of all the schemes to detect and characterize the clouds for all complex situations and provides some indications of confidence in the data. Then the dataset is restricted to thick overcast single layers and the comparison shows a significant agreement between all the schemes. The impact of the retrieved cloud properties on the residuals between calculated cloudy radiances and observations is estimated in the long-wave part of the spectrum. Copyright c 2011 Royal Meteorological Society, Crown in the right of Canada, and British Crown copyright, the Met Office Key Words: cloud detection; cloud properties; MetOp; overcast scenes; NWP assimilation; high spectral resolution sounder Received 10 January 2011; Revised 31 May 2011; Accepted 1 August 2011; Published online in Wiley Online Library 16 September 2011 Citation: Lavanant L, Fourrié N, Gambacorta A, Grieco G, Heilliette S, Hilton FI, Kim M-J, McNally AP, Nishihata H, Pavelin EG, Rabier F Comparison of cloud products within IASI footprints for the assimilation of cloudy radiances. Q. J. R. Meteorol. Soc. 137: DOI: /qj.917

2 Comparison of IASI Cloud Products Introduction Infrared Atmospheric Sounding Interferometer (IASI) data for temperature and humidity sounding are now assimilated in clear conditions at many operational meteorological centres, providing good impact on forecast skill (e.g. Hilton et al., 2009). However, more than 80% of the whole globe is covered by clouds. All the centres have begun to handle these data in recent years, starting with the assimilation of cloud-affected radiances for restricted conditions such as overcast scenes or middle and low-level cloud layers (e.g. Pavelin et al., 2008; McNally, 2009; Pangaud et al., 2009). Full simulation of cloudy radiances from numerical weather prediction (NWP) model fields is still some way off. The first step for operational applications is to detect and characterize the clouds within the footprint of the sounder before the assimilation. The main useful cloud parameters are the cloud top height, the effective amount, and of course correct detection of cloudy scenes. When the cloud properties within the satellite footprint are known, the information can then be used by the radiative transfer model within the data assimilation system, allowing the direct assimilation of the cloud-affected radiances. However, it is not obvious how accurate the cloud information is, as direct validation with in situ verification data is difficult. One way of investigating the limitations of a particular method is to perform a careful intercomparison of the results of different processing schemes for the same set of observations covering a wide range of cloud and atmospheric situations. The benefit of performing a cloud property intercomparison was discussed and recommended during a meeting of the IASI Sounding Science Working Group (ISSWG) with an initial group of participating centres which then expanded when the study was endorsed by the World Meteorological Organization (WMO) Working Group on Numerical Experimentation. For this study, ten operational or pre-operational cloud schemes were applied to a 12-hour global IASI dataset from 18 November The various methods of cloud detection and characterization in the IASI footprint used by the different participants are briefly described in section Description of the cloud analysis methods Table I is a summary of the different schemes together with a reference paper. For the readability of the following sections, abbreviations for schemes and models as named in this paper are given: CMC: Canadian Meteorological Centre, Canadian operational assimilation system CMS: Centre de Météorologie Spatiale, in charge of the local satellite data acquisition and treatment for Météo-France CNRM: Centre National de Recherches Météorologiques, in charge of developments for the Météo-France operational assimilation system CRTM: Community Radiative Transfer Model under development at the Joint Center for Satellite Data Assimilation (JCSDA) EC: in charge of developments for the Environment Canada operational assimilation system JMA: Japan Meteorological Agency METO: current operational assimilation system at the Met Office METO o: previous operational assimilation system for IASI at the Met Office NCEP: National Centers for Environmental Prediction, National Oceanic and Atmospheric Administration (NOAA) operational assimilation system NESDIS: NOAA National Environmental Satellite, Data and Information System ϕ-iasi: physical forward/inverse scheme under development at UNIBAS RTTOV: Radiative Transfer for TIROS Operational Vertical Sounder (TOVS), under development at the European Meteorological Satellite system (EUMET- SAT) NWP Satellite Application Facility SARTA: Stand-alone Atmospheric InfraRed Sounder (AIRS) Radiative Transfer Algorithm under development at the University of Maryland, Baltimore County (UMBC) UNIBAS: Universita della Basilicata 2.1. Cloud detection schemes Until recently, cloud detection for IASI at the Met Office was performed using a Bayesian method applied to a small selection of IASI channels (English et al., 1999). This method has now been superseded by a 1D-Var analysis scheme (see section 2.2), but the Bayesian cloud detection results (METO o in next sections) are included in this study for validation purposes. The UNIBAS cloud detection scheme (Masiello et al., 2009) is in development in the course of a EUMETSAT contract for the preparation of the Meteosat Third Generation/InfraRed Sounding (MTG/IRS) programme. It is based on IASI window channel tests and long-wave/shortwave regression tests. Cloud detection data only were provided by METO o and UNIBAS for this study. EC and CMS implement the cloud detection using the Advanced Very High Resolution radiometer (AVHRR) radiance cluster analysis. The cluster information is available in the IASI level 1c files (Cayla, 2001; EPS programme, 2004) and corresponds to a detailed multi-spectral characterization of AVHRR pixel properties and their separation into a limited number of classes (up to 7) for each IASI footprint. For each class, the fraction of the field of view (FOV) covered and the mean value of AVHRR channel radiances are given, together with the standard deviation of the data which should provide information about compactness of the cluster. JMA directly uses the full resolution AVHRR cloud mask mapped to the IASI FOV. Except these teams, most of the schemes also apply an IASI window channel test with the forecast temperature before starting the characterization. Otherwise their cloud detection is embedded in the cloud characterization (NESDIS) Cloud characterization schemes AVHRR cluster information The CMS cloud classification is done for each AVHRR cluster (Lavanant and Lee, 2005) and is an adaptation to the CMS cloud mask derived from the AVHRR data at full resolution (Lavanant, 2002). The mask is based on a series of tests which allow the use of visible channels during the day, and take advantage of the possible emissivity differences between long-wave and short-wave CO 2 bands. The cloud

3 1990 L. Lavanant et al. Affiliation Researcher [scheme reference] EC ECMWF JMA Météo France /CMS Météo France /CNRM Met Office (METO scheme) Met Office (METO o scheme) NOAA /NCEP NOAA /NESDIS UNIBAS Heilliette [Garand, 2011] McNally [McNally, 2009] Nishihata [Eyre, 1989] Lavanant [Dahoui, 2005] Fourrié [Guidard, 2011] Pavelin [Pavelin, 2008] Hilton [English, 1999] Kim [Eyre, 1989] Gambacorta [Susskind, 2003] Grieco [Masellio, 2009] Scheme status Table I. Summary of the different schemes. Scheme description IASI operational Detection: AVHRR, assimilation IASI comparison with T s Characterization:CO 2 Slicing IASI operational it Detection and assimilation Characterization: First-guess: Minimum residual method and 4D-Var assimilation Development Detection: AVHRR, IASI comparison with T s Characterization: Minimum residual method IASI Level2 Detection: AVHRR Characterization: AVHRR for opaque, CO 2 Slicing for homogeneous semitransparent. Up to 3 cloud layers AIRS: operational IASI: pre-operational assimilation AIRS and IASI: operational assimilation IASI detection NOAA operational assimilation NOAA operational Level2 Development for MTG/IRS Detection: IASI tests Characterization: CO 2 Slicing IASI channels used by the scheme Radiative transfer A priori 13 CO 2 pairs RTTOV8.7 CMC shortrange forecast Guess: 5 CO 2 channels RTTOV9.3 ECMWF short-range forecast 74 channels RTTOV9.3 JMA shortrange forecast 40 CO 2 channels in 366 Detection and Guess: 10 Characterization : CO 2 channels First-guess: Minimum 1D-Var: residual 92 channels method and 1D-Var retrieval of cloud parameters together with atmospheric profile cloud Detection: Bayesian 4 window and other cloud tests. channels Clear channels assimilation Characterisation:No IASI Detection and Characterization: Minimum residual method IASI Cloud clearing 69 channels method using the from the 2 2 IASI footprints in conjunction with and AMSU and MHS. Up cm 1 to 2 cloud layers Detection: IASI window channels and LW-SW regression tests Characterization:No RTTOV9.3 ECMWF 12h-18h forecast 36 channels RTTOV8.7 Meteo- France short-range forecast RTTOV7 Met Office short-range forecast RTTOV7 Met Office short-range forecast 165 channels CRTM NCEP global short-range forecast 148 channels from 668 and 2078 cm 1 SARTA10 ϕ-iasi Surface emissivity Sea: Masuda [1988] Land: Wilber[1999] Sea: [Sherlock, 1999] ISEM6 Sea: ISEM6 Land: 0.98 Sea ice: 0.99 Sea: ISEM6 Land: 0.98 Sea ice: 0.99 Sea: ISEM6 Land: 0.98 Sea ice: 0.99 Sea: ISEM6 Land: 0.98 Sea ice: no data used Sea: ISEM6 Land: 0.98 Sea ice: no data used Sea: Nally et al. [2008] Land: Han et al. [2005] Climatology Sea: Masuda and atmospheric Land: regression state on cloudcleared radiances ECMWF climatology Sea: Masuda, [1988] Land: 0.98

4 Comparison of IASI Cloud Products 1991 height is computed from the cluster information when the cloud layer is classified thick and overcast. The calculation takes into account the radiance absorption above the cloud. This method is particularly useful for low-level layers, as the accuracy of the result is less sensitive to the errors in the surface parameters. The geometrical cloud amount is the cluster cover in the IASI footprint. The scheme also characterizes the heterogeneity of the scene and provides up to three layers (clear or cloudy) in the FOV. For clusters classified as semi-transparent, no cloud height is given from AVHRR and an additional step is added based on the CO 2 Slicing method when a single cloud layer is found in the FOV CO 2 Slicing method The CO 2 Slicing method (Menzel et al., 1983; Smith and Frey, 1990) is used by several schemes (CNRM, EC, CMS for single non-opaque layers) to retrieve the cloud top pressure and the effective cloud amount. Assuming that the cloud emissivity is constant in the long-wave CO 2 band, the method is based on the ratio of a pair of channels (a channel k paired with a reference channel ref ), and searches for the pressure level p minimizing the difference: (Rclr Rm) k /(Rclr Rm) ref (Rclr Rp) k /(Rclr Rp) ref (1) where Rm is the measured radiance, Rclr the calculated clear radiance from the background and Rp is the computed overcast radiance from any possible pressure level p. The high spectral resolution of the sounder allows the selection of several pairs of channels peaking at different levels in the atmosphere, and the final cloud pressure cp is computed as the weighted average of the different solutions, taking into account the sensitivity of each pair to the cloud height. EC computes the median of the thirteen outputs instead of a weighted average. Once this is done, the effective amount Ne, which represents the product of the geometrical cloud fraction and the grey-body emissivity of the cloud, can be found from either channel: Ne = (Rclr Rm)/(Rclr Rcp), (2) with Rcp the radiance of the final cloud pressure cp. Several teams involved in this study use the CO 2 Slicing method but with different settings, such as different a priori atmospheric profiles, different channel pairings, or by using one reference channel only for all pairs (CNRM) or several couples of channels (EC), and so results can be different with the same method. In addition, when the CO 2 Slicing reference channel is flagged clear, the CNRM considers the IASI pixel as clear and the CNRM cloud detection relies on the McNally and Watts (2003) cloud detection scheme NESDIS cloud characterization The NESDIS cloud characterization approach is described in Susskind et al. (2003) and is an extension of that used by Smith (1968) and Chahine (1974, 1977). The method is a regularized least-square minimization between observed and computed cloudy radiances considering the four instantaneous fields of view (IFOVs) of the effective FOV (EFOV) and solves for each IFOV, cloud fraction and cloud top pressure for up to two cloud formations. In this approach the only assumption made is that the IFOVs are homogeneous except for the amount of cloud cover in each field of view. The regularization is based on a global mean climatology of cloud properties. The forward computation is made using the SARTA forward model and a clear atmospheric state obtained through a radiance regression technique Minimum residual method JMA and NCEP run a minimum residual method. As described by Eyre et al. (1989), the method consists of a simultaneous minimization of the residuals between the measured Rm and calculated Rc radiances for a set of k channels: k {(Rm Rc) k Ne(Rp Rc) k } 2. (3) The effective amount is first minimized for each possible cloud top pressure in the radiative transfer model and then, substituting the value into the equation above, the retrieved cloud top pressure corresponds to the minimum residual. The method assumes a constant cloud emissivity over the range of channels, and also a constant estimate of the surface and atmospheric variables coming from the forecast. The European Centre for Medium-Range Weather Forecasts (ECMWF) uses the minimum residual method applied to five IASI channels to identify overcast scenes (Ne = 1) and to get an initial guess at the cloud top pressure. The cloud guess is then used as a linearization point for the 4D-Var assimilation. Inside the assimilation, the cloud estimates are adjusted (in practice slightly within 10 hpa to at most 50 hpa) using all the other remaining IASI channels and through the constraint by information from all other satellites and conventional observations. The dataset supplied for this study corresponds to the final analysed estimates of the assimilation system, which explains the small number of 255 samples (only cloudy scenes) D-Var method The Met Office (METO in next sections) runs a 1D-Var retrieval system (Pavelin et al., 2008) where the cloudtop pressure and effective cloud fraction are retrieved simultaneously with the atmospheric profile variables by minimizing the cost function J(x): J(x)=(x x 0 ) T B 1 (x x 0 )+{Rm Rc(x)} T R 1 {Rm Rc(x)}, (4) where R is the measurement and forward model error covariance matrix, B the background error covariance matrix. Rc(x) represents the calculated radiances corresponding to the atmospheric state x (atmospheric profile and cloud variables) and x 0 is the a priori value of x. The scheme has been operational for IASI since early 2010 and for AIRS since summer The 1D-Var retrieved cloud parameters are passed to the 4D-Var assimilation system, where they are used to constrain the radiative transfer calculation. For each observation, the channel selection is chosen to reduce the sensitivity to errors in the forward modelling of multi-level and non-grey cloud. For IASI, up to 183 channels from different part of the IASI spectrum are used in the 1D-Var

5 1992 L. Lavanant et al. analysis. In this scheme also, the cloud emissivity is assumed to be independent of the wavelength. The cloud first-guess is derived using the minimum residual method Errors due to the a priori knowledge of the atmospheric and surface state vector Different authors (i.e. Wielicki and Coakley, 1981; Eyre and Menzel, 1989; Susskind et al., 2003; Szyndel et al., 2004) discussed the accuracy of their retrieved cloud products from the method used. Potential sources of errors in both the measured and calculated radiances were considered, since in all methods (except for AVHRR) it is the radiance difference which determines the retrieved cloud height. Areas of difficulty arise when the difference in radiance between the cloud and clear sky is small and comparable to the instrument noise, mainly for optically thin cirrus or low-level clouds. In these conditions, small errors in any of both radiances have a large impact on the retrieved product accuracy. Errors in the measured radiances are small with IASI due to the good instrument noise in the CO 2 band compared to previous instruments. Errors in the simulated radiances come from inaccuracies in the forward model and from errors in the background profile. Surface skin temperature errors dominate in the selected channels as these errors can be very large mainly over land (they could be more than 3 4 K) when the surface emissivity is not correctly defined. Table I indicates the use of surface emissivity spectra by the different groups, which should help understanding the differences over continental surfaces. Besides, the cloud accuracy increases with the cloud height and amount, due to the higher contrast of the cloud radiation and the surface radiation. McNally (2009) showed that for high cloud contrasts the cloud top pressure estimate is not significantly affected by the background profile errors, giving an accuracy similar to a system for which the profile is known perfectly. For the cloud detection, it is important also that thresholds are different over sea and land. For example for some tests with thresholds defined by experience, CMS adds railings on the thresholds with larger values over desert than vegetation which account for the accuracy of the skin surface temperature. Considering the previous remarks, it is obvious that in some regions on the Earth the cloud retrievals are more reliable than in others. Due to the background accuracy, we should expect higher confidence over sea than over land, in the Northern Hemisphere than in the Southern Hemisphere, and in midlatitude regions than in polar regions. Also, some environmental conditions can degrade the product accuracy, for example the different methods require a temperature gradient to infer the cloud height. Polar regions with relatively isothermal atmospheres or strong thermal inversions during winter could be problematic areas. Nevertheless, we have considered it important to define the intercomparison in all possible regions (and two seasons) as NWP forecasting models concerned in this article are global models, with the most sensitive areas in polar regions. Maps in Figures 1 and 2 provide some way to see the behaviour of the different methods in all areas. It should be noted that data provided by the different teams are considered operational and confident enough to be assimilated in their respective systems. So far, cloudy radiances are assimilated with caution in NWP systems. The same type of rules as for clear radiances are applied to these data but are generally more severe as the sensitivity of the cloudy radiances to the state is highly nonlinear. For example, METO, CNRM and ECMWF reject all cloudy radiances over land and polar regions. Moreover, CNRM assimilates only the data with residuals smaller than the thresholds also used in clear conditions, and channels with a significant response at or below the cloud height are rejected by METO (Renshaw et al., 2010) and EC (in near future: Heilliette, 2011). ECMWF (McNally, 2009) discards all non-opaque or non-overcast situations Some remarks In all cloud retrieval methods, the cloud is considered to be a single thin layer of opaque or semi-transparent cloud with a constant emissivity over the range of selected wavelengths, and for that reason the schemes are mostly applied in the long-wave CO 2 band. Furthermore, the radiative transfer models used in the different schemes (RTTOVv7 to v9.3, SARTA, CRTM, ϕ-iasi) all consider a simple cloud model and do not take into account the optical properties of the clouds. However, for ice crystals for example, the microphysical properties of the cloud cannot be neglected in an accurate calculation: the cloud emissivity can change dramatically in this band with a possible impact of several degrees on the IASI spectrum. For these situations, cloud pressures are biased depending on the set of channels used and the observed-calculated residuals can be quite large, larger than the threshold applied to reject the IASI channels in the assimilation systems. Current methods, based on the IASI information in individual footprints only, cannot correctly retrieve cloud pressures for multi-layer systems, which represent at least half of the cloudy situations. The output pressure is then an intermediate value between the layers depending on the cover and thickness of the upper layer. We then consider in section 5 the cases with single opaque layers, which are the simplest cases to deal with in data assimilation. Schemes need anapriori atmospheric profile which could be a climatology (UNIBAS), a cloud distribution climatology together with an atmospheric state first-guess (NESDIS), or an NWP forecast. Results can be very sensitive to the background accuracy as discussed above and, for the 1D-Var scheme, to the assumed background error covariance matrix. For example, in METO implementation, cloud-top pressure (CTP) and effective cloud amount (ECA) background error variances are assumed to be very large, so that the cloud parameters in the background vector are effectively ignored, but with a cloud first-guess as good as possible to account for the highly nonlinear nature of the cloudy problem. In ECMWF 4D-Var assimilation, the cloud background errors are put to very small values (5 hpa on CTP and 0 on ECA) so that the a priori value is not modified by the variational assimilation. So the same method applied in two different centres could give different results depending on the settings. Therefore this study should not be considered as a strict assessment of the validity of any of the methods used. The focus of this analysis is to compare and characterize the differences in cloud properties retrieved from a global sample of IASI data by the different methods under consideration. As requested and accepted by all the authors, the supplied datasets correspond to the outputs of their schemes as used

6 Comparison of IASI Cloud Products 1993 Figure 1. Geographical position of the clear situations over the globe for the three schemes which provided their results at each IASI footprint. (Middle) UNIBAS cloud flag. (Upper) NESDIS results based on their effective amount <5%. (Lower) CMS based on an AVHRR cloud cover in the footprint <5%. operationally. Each of the schemes adopts its own technique for the footprint selection, for example there are different rules for the thinning of the FOVs prior to the processing (e.g. the selection of a different footprint number in the Field Of Regard) and in particular, each scheme uses different rejection criteria. Consequently, for this study, it has proved difficult to intercompare the results over an identical subset of observations for all centres. Nevertheless, we are able to provide some indications of confidence in the retrieved cloud information. 3. Cloud detection results The first useful indicator for a scheme is its capability to detect the clouds or conversely to select the clear situations, if it is important for an application to use only IASI footprints unaffected by clouds. For this study, the participants were asked to provide their retrieved geometric cloud amount (CMS) or effective cloud amount in the IASI footprint, or if that was not possible, a flag indicating clear/cloudy (UNIBAS, METO o). A threshold of 5% on the cloud amount or effective cloud amount was used to separate clear and cloudy situations. The statistical agreement PC (Proportion of Correct) between two schemes is derived from the contingency table shown in Table II. PC = (n a + n d )/(n a + n b + n c + n d )is the percentage of agreement, which should be as large as possible. Table III summarizes the PC agreement between the different schemes together with their number of collocated

7 1994 L. Lavanant et al. (a) (b) (c) Figure 2. Cloud top pressure (hpa) for schemes based on different methods. CNRM (a) and EC (b) use a CO 2 Slicing method. METO (c) uses 1D-Var. NCEP (d) uses a minimum residual method. NESDIS (e) uses a cloud-clearing method and CMS (f) is based on AVHRR radiance analysis together with aco 2 Slicing method. [Correction added on 31 October 2011 after original online publication: part (f) in Figure 2 was missing and has been added.]. This figure is available in colour online at wileyonlinelibrary.com/journal/qj situations. The numbers in the diagonal of the table correspond to the number of situations in each dataset after removing the missing data. Note that ECMWF data are not shown in Table III as all data of this dataset are cloudy, those effectively assimilated by its 4D-Var system (only 255 situations for this 12-hour period). The supplied datasets correspond to the outputs of the different schemes as used operationally and consequently the processed number of data for each team was very different. This induces some difficulties when considering more local conditions (e.g. sea by day, etc.) in the comparison. In Table III, the agreements are probably overestimated and have to be taken with caution because the large proportion of cloudy situations (more than 80% of data) tends to bias the statistics towards better agreements. The disagreement between the clear scenes detected with the METO o scheme and the others is partly a consequence of the fact that the scheme is very conservative and in

8 Comparison of IASI Cloud Products 1995 (d) (e) (f) Figure 2. (Continued). particular rejects almost all scenes over land. Furthermore, the threshold of <5% cloud applied to the other schemes could cause a discrepancy in the results. Only CMS, NESDIS and UNIBAS provided their results for each situation, and Figure 1 shows, for these three schemes, the position of the clear situations which successfully passed their own respective retrieval acceptance criteria. Compared to the two others, CMS indicates more clear situations over land (Asia, North America) but is slightly cloudier for marine fractional clouds. Most cloud detection systems are based on a succession of thresholds tests applied to various combinations of channels and the quality of the process is largely a function of the capability of the scheme to compute their thresholds in-line. Thresholds are dependent on the environmental conditions (surface type and estimated temperature, daytime period, viewing geometry, etc.) and a validation effort involving intensive comparisons with in situ data or a visual inspection is

9 1996 L. Lavanant et al. Table II. Contingency table. Cloud detected scheme A Cloud detected scheme B n a n b Clear detected scheme B n c n d Clear detected scheme A necessary to achieve high confidence in the mask. Notice that UNIBAS reports no polar clear-sky, which could be problematic if not solved, as it tends to systematically exclude these regions in final applications. The UNIBAS scheme is still under development and their thresholds have been tuned as a result of the current study through a collaboration with CMS. 4. Cloud characterization results Important products for careful and accurate assimilation of cloudy radiances are the retrieved cloud pressure and effective cloud amount. These two parameters are often linked together in the cloud scheme retrievals. For the intercomparison, we have considered the existence of a cloud layer when the effective cloud amount or geometrical cloud cover is larger than 10%. Indeed, for very small cloud amounts, the accuracy of the retrieved cloud height, for all the applied methods, is very sharply degraded. As a consequence, large differences between products are observed without an easy interpretation of the results. Of course, this slightly reduces the number of co-registrations which is nevertheless very large except with ECMWF. Figure 2 shows cloud pressure maps for six different schemes based on different retrieval methods and their respective rejection criteria. It is necessary to re-emphasize the fact that these results are not based on a common subset as explained in section 2.3. The main meteorological structures have been well retrieved by all the schemes but their cloud heights can be very different. Similar methods lead to similar results (i.e. CNRM and EC). For CMS and NESDIS multi-layer situations, the cloud pressure of the upper layer is selected in the intercomparison because of its importance in cloudy data assimilation. The NESDIS scheme is able to detect and characterize very thin clouds above lower clouds, which explains the colder map, compared to the other schemes. These thin clouds are often, but not always, detected by CMS but are not characterized because AVHRR channels are not suitable for estimating the height of semi-transparent clouds and because the CMS scheme is cautious with regard to the determination of cloud heights for multi-layer systems from single IASI footprints. For these situations, the pressure of the lower layer is thus visualized on the map but the situation is rejected in the following statistics, which probably biases the output cloud pressures towards larger values. The slightly inferior agreement in cloud detection of METO with the other schemes is seen here on the map: a large amount of clear areas are flagged cloudy by the 1D-Var system, for example over desert but not only there, with a cloud pressure value near the surface pressure. This is probably due to ambiguity between low cloud and errors in background surface temperature. This effect is worse over land due to additional uncertainty in the surface emissivity. This is a known issue at the Met Office and it is of little importance to the assimilation scheme because the retrieved cloud in these situations is very low; it therefore does not have a significant effect on the choice of channels to be assimilated in 4D-Var. Tests on residuals are often used to prevent assimilation of those channels peaking near the surface. Large residuals may result from differences between the effective cloud cover and the surface emissivity, over land for example. Because the discrepancy between clear and lowcloud scenes can probably be corrected by some tuning, in order to get statistics representative of the cloud parameters we have eliminated in the next sections the METO cloudy situations with a cloud pressure below 950 hpa and classified clear by CMS. Table III. Statistical agreement between the different schemes. EC JMA CMS CNRM METO METO o NCEP NESDIS UNIBAS EC JMA CMS CNRM METO METO o NCEP NESDIS UNIBAS The first number corresponds to the number of collocated situations and the second to the statistical agreement PC in per cent.

10 Comparison of IASI Cloud Products 1997 Figure 3. Examples of bi-dimensional histograms (reduced to scatter plots for figures involving ECMWF). Symbols correspond to the retrieved effective cloud amount of the scheme shown on the abscissa. For clarity, every comparison is split into 2 figures, the six upper figures corresponding to cloud amounts larger than 0.7 ( : star in blue; : square in red), the six lower figures for cloud amount less than 0.7 ( : circle in blue; : cross in green; : triangle in red). The size of the symbol indicates the percentage of all observations, except for the graphs with ECMWF where each observation has been plotted individually. In the title, n: number of co-registrations, m: mean of the differences, s: standard deviation of differences, c: correlation. The different schemes sampling explains the various numbers of co-registrations. This figure is available in colour online at wileyonlinelibrary.com/journal/qj We also have eliminated from the statistics NCEP situations with cloud pressures between 155 and hpa corresponding to more than 10% of the results and which appeared to result from the initialization. Figure 3 shows typical examples of bi-dimensional histograms computed in steps of 50 hpa. For clarity, the results are split into two figures depending on the cloud amount. The different symbols indicate the retrieved effective cloud amount of the scheme shown on the abscissa. The size of the characters indicates the percentage of observations, all categories merged, of the bi-dimensional histograms. For readability, distributions with less than ten elements are discarded. The agreement between all the schemes is always better for high clouds even for

11 1998 L. Lavanant et al. Table IV. Cloud pressure statistics. ECMWF JMA CMS CNRM METO NCEP NESDIS EC / / / / / / / ECMWF / / / / / JMA / / / / / CMS / / / / CNRM / / / METO / / NCEP / Top: Number of co-registrations. Middle: Mean/standard deviation of departures. Bottom: Correlation. small effective cloud amount, and for opaque and overcast situations for the lower-level layers. For middle and lowlevel clouds, the agreement decreases very quickly with the effective cloud amount. Situations with less than 10% cloud amount have been discarded from the following statistics because the agreement was generally too poor and pollutes the understanding of the results. As mentioned before, ECMWF provided this experiment with only the marine cloudy data effectively assimilated by its 4D-Var system which discards all non-opaque and non-overcast situations, or those outside the quality tests. The size of the ECMWF dataset being then very small (255 situations), we have plotted all the situations individually on the graphs. Table IV summarizes the statistics of all comparisons. Standard deviations of differences are generally about hpa with biases of around 50 hpa or less. As expected, CMS data exhibit a positive bias (larger pressures). In spite of different retrieval methods, ECMWF, CMS, CNRM, METO and NCEP outputs are close, with correlations larger than Large correlations can, however, be explained by some similarities between the schemes. For example, the high correlation between CNRM and CMS is partly explained by the teams collaboration and by some tuning of CNRM outputs during this study. The good agreement observed between METO and NCEP could be due to the fact that the METO 1D-Var cloud first-guess is derived using the minimum residual method, which is also used by NCEP. It would have been interesting to know how much the cloud initialization is modified by the 1D-Var system. The ECMWF dataset, which is limited to a small set of overcast and opaque clouds (cloud fraction = 1), exhibits very good correlations in cloud pressure with most of the other schemes and particularly with NCEP and CMS. Figure 4 shows the global distribution of the cloud pressures. For NESDIS and CMS multi-layer systems, all Figure 4. Global cloud pressure (hpa) distribution. This figure is available in colour online at wileyonlinelibrary.com/journal/qj layers are shown. Except for NESDIS and EC, the other schemes have two main peaks around hpa and at 300 hpa, with approximately the same strength in the distribution. The higher layer is probably underestimated by CMS due to the difficulty of retrieving the upper cloud formation of high-level semi-transparent clouds using this method. Figure 5 shows the departure of the effective cloud amount,withmetoresultsasareference.ncepresults seem to be biased compared to the others. Again, as explained

12 Comparison of IASI Cloud Products 1999 Figure 5. Cloud effective emissivity departures versus METO results. This figure is available in colour online at wileyonlinelibrary.com/journal/qj in section 2.3, both comparisons in Figures 4 and 5 are performed on data which passed the rejection criteria of the schemes. 5. Overcast and opaque single layers Several methods use information from single IASI FOVs, which makes it difficult to perform an accurate retrieval of the cloud parameters for multi-layer cases. Until more comprehensive schemes are developed, which in many cases will depend upon advances in radiative transfer models and in data assimilation schemes, the priority is to avoid assimilating cloudy radiances from these complex systems. For example ECMWF is testing the assimilation of opaque and overcast footprints only. For AIRS, CNRM is assimilating middle and low-level clouds only. A simple way to detect most of the multi-layer situations is to test the heterogeneity within the footprint by the use of the AVHRR radiance analysis available in the IASI level 1c files in conjunction with the spectrum itself. Figure 6 shows the number of cloud layers determined by the CMS process after applying the cloud mask to the AVHRR radiance analysis. To define a layer, AVHRR clusters are aggregated if the difference between the retrieved cloud-top temperatures is less than 1 K. This threshold could seem severe as it often corresponds to pressure departures smaller than 100 hpa but it is consistent with the threshold applied to the residuals in section 6. All clusters with missing cloud-top pressure are put together in the same layer. Comparison of Figures 6 and 2 indicates that single layers are not necessarily connected to meteorological structures. They could correspond to multilayer systems but with a thick, overcast upper layer. There is no obvious coherence in position between single cloud layers and meteorological structures. We then consider only those IASI footprints declared by the AVHRR tests to be covered by a single thick overcast (>98%) layer. Table V corresponds to the ratio of these situations then declared cloudy by each scheme relative to the total number of cloudy situations in the datasets. It represents approximately 25% of the cloudy situations for all the schemes except for ECMWF which seems able to do a pre-selection consistent with this criterion. Similarly, the CNRM method rejects cloudy pixels when the CO 2 Slicing does not converge and flagged them as clear observations. The ratio is slightly smaller for NESDIS because the analysed sample collects only cases that passed the full set of the retrieval rejection criteria, and overcast cases are normally likely to be rejected. Table VI summarizes the statistics of all comparisons for the single-layer opaque overcast scenes. Values in brackets correspond to the cloud detection agreement similar to Table III. There is almost no false categorisation of these situations as clear scenes by any of the schemes. The cloud characterization agreement between the schemes is much better, with correlations often larger than 0.95, standard deviations of differences smaller than hpa and biases less than 20 hpa. The percentage of outliers with absolute differences larger than 200 hpa and the number with the absolute difference larger than 500 hpa are also given in Table VI. Except with JMA, outliers are very few. An additional detailed study (results not seen on a figure) indicates that the histograms of departures approximately follow a Gaussian shape and that more than 90% of important departures are in polar regions, where specific problems are encountered (low-level thermal inversions, isothermal temperatures). Notice that statistics from ECMWF have to be taken with care because of the small number of scenes in the comparison. For example, the agreement is very good between EC and ECMWF but two outliers artificially degrade the statistics in the table. The global distribution of the cloud pressures seen in Figure 7 is also much more coherent, with the same behaviour for EC, CNRM, METO, CMS and NCEP. NESDIS exhibits a smaller peak around 700 hpa, which could be due to the difficulty of retrieving overcast cloud layers with this scheme especially near the surface, or to the fact that the method often finds an upper layer above middle or low-level layers which is not seen by the CMS. This could be due to an overestimation of thin cirrus by the cloud-clearing method or to an underdetection of thin clouds by CMS due to the poor sensitivity to high levels of the AVHRR channels, which peak in the window bands. Table V. Percentage of detected cloudy FOVs corresponding to an overcast and opaque single layer. EC ECMWF JMA CMS CNRM % of total cloudy METO METO o NCEP NESDIS UNIBAS % of total cloudy

13 2000 L. Lavanant et al. Table VI. Cloud pressure statistics for overcast and opaque single layers. ECMWF JMA CMS CNRM METO NCEP NESDIS UNIBAS EC [99%] 50.9/ / / / / / / /7.4/0 0.9/2.7/ /5.3/24 2.5/2.7/0 1.5/8.9/7 1.6/5.0/2 2.2/8.6/ [100%] 0.94 [100%] 0.93 [100%] 0.91 [99%] 0.92 [100%] 0.82 [88%] ECMWF [99%] 112/ / / / / /12.5/2 1.4/3.4/0 0./6.7/0 1.9/5.7/0 0./0./ JMA [99%] 144.8/ / / / / /28.5/ /24.0/ /12.1/ /18.9/ /23.5/ [100%] 0.48 [100%] 0.75 [100%] 0.80 [100%] 0.31 [88%] CMS [99%] 39.5/ / / / /2.7/5 2.1/4.0/8 2.3/4.4/17 2.8/8.8/ [100%] 0.95 [98%] 0.95 [100%] 0.86 [92%] CNRM [100%] 8.4/ / / /3.1/0 1.7/2.3/1 1.9/3.8/ [99%] 0.96 [100%] 0.91 [95%] METO [98%] 12.2/ / /2.9/0 2./7.5/ [100%] 0.91 [93%] NCEP 3198 [100%] 34.9/ / [95%] (1 st row) Number of co-registrations. (2 nd ) Mean and standard deviation of departures. (3 rd ) % of data to more than 3 standard deviations/% of outliers with absolute differences larger than 200 hpa/number of outliers with absolute differences larger than 500 hpa. (4 th ) Correlation. Values in brackets are cloud detection agreements in per cent. Figure 6. Geographical positions of the multi-cloud layers from CMS. This figure is available in colour online at wileyonlinelibrary.com/journal/qj 6. Impact on residuals for single layers Assimilation of cloud-affected channels is often limited to those having observed-calculated radiance residuals smaller than some threshold. Figure 8 shows the histogram of the number of channels with residuals in brightness temperature smaller than 1 K in the CO 2 long-wave band (wave numbers less than 950 cm 1 ) among the channels belonging to preselected subsets of channels done for assimilation purposes. For METO and CNRM, the maximum number is of 145 channels among the 314 subset selected by Collard (2007), and for CMS and NESDIS, the maximum number is

14 Comparison of IASI Cloud Products 2001 Figure 7. Cloud pressure (hpa) distribution for single overcast and thick clouds on the globe. This figure is available in colour online at wileyonlinelibrary.com/journal/qj of 189 among the 366 channels subset used in Collard and McNally (2009). The 1 K value is a value typically used to allow a channel to be assimilated, but not all schemes actually apply such a threshold. The CO 2 longwave region has been chosen because it is mainly sensitive to temperature. The histograms are done for five classes of cloud height ( , , , and below 900 hpa). Because the cloud parameters are retrieved using a relatively small set of channels but are applied across the whole spectrum, the greater the number of channels with a small residual, the greater the confidence in the retrieved cloud parameters. It is, of course, not true to say that small residuals always correspond to accurate results. Nevertheless, it is an indirect way to estimate the quality of the cloud parameters, bearing in mind that we cannot separate errors in the cloud parameters from errors in the radiative transfer model or temperature profile. In this section, we have restricted the comparison to single-layer scenes only, but which can be semi-transparent or non-overcast clouds. As seen in the previous section, the cloud characterization agreement between the schemes is much better for single layers and the assimilation of these scenes is recommended as a priority. Moreover, these situations being simpler to treat, fewer rejection flags are applied by the different schemes and consequently the interpretation of the results is easier. A last reason is that most radiative transfer models (e.g. RTTOV up to version 9) cannot deal with more than one cloud layer. Nevertheless, some differences still exist in the computation of the residuals. CMS and CNRM did not apply a bias correction before computing the residuals; METO does one. These three schemes computed the differences between the observed and the calculated cloudy brightness temperatures while NESDIS residuals are the final computation of differences between cloud-cleared radiances and computed radiances from the retrieved state. The residuals for the four schemes are very good. For low and middle-level clouds, CMS residuals are small which is probably due to the use of the AVHRR information which is more efficient for situations of low contrast with the surface. The five histograms of CNRM are identical whatever the height of cloud with numbers always around 140 among the 145 channels. Probably the good residuals could be explained by the method which discards difficult cases (low-level clouds, small effective cloud amounts) which is also brought to light in Table V. The CNRM scheme treats these difficult situations as clear pixels to rely on the cloud detection of McNally and Watts (2003) and to discard cloud-affected channels during the assimilation step. Despite very good correlations of CMS and METO cloud pressures which suggests good cloud information, the residual histograms indicate less agreement for high-level clouds. The bias correction applied by METO could be one of the explanations. Another explanation is that CMS does not rely on residuals to reject some doubtful results. For high-level clouds, the optical properties are not taken into account in the RTTOV radiative transfer model used to compute the residuals by both schemes and the differences can be quite large (several degrees) for these situations and maybe have been discarded by METO, which applies a 1.5 K threshold to discard difficult cases. The NESDIS residuals should be considered separately since they are the difference of the cloud-cleared spectrum minus the computed spectrum from the final retrieval output, and for this reason represent a collection of samples that have passed further rejection selections. This justifies the better results shown by this scheme with respect to the others mainly for high-level clouds as shown in Figure Conclusion Retrieved cloud properties within the field of view of the Infrared Atmospheric Sounding Interferometer (IASI), onboard the European MetOp satellite, from ten operational or developing schemes using different methods have been assessed and an intercomparison of the cloud products over a 12 h global acquisition has been presented. JMA and CNRM exhibit a high agreement with NCEP for the detection of the cloudy situations. The main meteorological structures were retrieved by all the schemes, but the cloud heights can be very different. However, ECMWF, CMS, CNRM, METO and NCEP cloud pressures are close, with correlations larger than This is an encouraging result as the retrieval methods are different. The agreement between the schemes is always better for high-level clouds (even for small effective cloud amounts), and for opaque and overcast situations for the lower-level layers. The schemes based on the departure between calculated clear radiance and overcast radiance at the cloud pressure level are very sensitive to errors in the surface parameters, which explains the greatly decreased level of agreement of the effective cloud amount for middle and low-level clouds. This problem is to some extent avoided with methods which use AVHRR due to its higher spatial resolution. The occurrence of complex situations with multi-layer cloud is important, and the success of each method varies with the complexity of the situation. When restricting the comparisons to situations covered by single, overcast, thick clouds, the agreement between all the schemes is very good. The characterization of the complexity of the cloud formation through use of the AVHRR radiance analysis

15 2002 L. Lavanant et al. Figure 8. Number of channels in the CO 2 band among ECMWF subsets of IASI channels (at most 189 channels among 366 for CMS and NESDIS and at most 145 among 314 for METO and CNRM) for which the difference between the observed and the calculated brightness temperature is smaller than 1 K. The 1 K value is chosen as a typical threshold which would allow a channel to be assimilated with confidence. This figure is available in colour online at wileyonlinelibrary.com/journal/qj is very easy and is recommended for methods based on the treatment of single IASI FOVs. The NESDIS cloudclearing method is the only one able to assess the thin upper layers above underlying cloud cover. The integration of the AVHRR radiances in the NESDIS scheme is under way and is expected to bring significant improvements in the retrieval of cloud parameters, especially in terms of acceptance yield. Indeed, the NESDIS current method misses situations which would have an important effect on the assimilation of IASI data in NWP assimilation. The impact of the retrieved cloud properties on the residuals between calculated cloudy radiances and observations is estimated in the long-wave part of the spectrum for single-layer scenes. CMS residuals are often larger for ice clouds than the other schemes. Taking into account the cloud optical properties should improve the simulation of the observation for high-level cloud layers and should have a large impact on the ability to assimilate these data. The intercomparison has never been thought of as a competition between teams to get the best result. All methods have advantages for some specific conditions but they also all show some deficiencies in other conditions. It was a good tool for everybody to understand problems and to improve their own scheme during these past months. A typical example is the interaction between CMS and CNRM resulting in the detection of bugs in the software and the improvement in methods. The consequence is a better agreement between the two methods with correlations of 0.85 at the beginning of the study and 0.94 after tuning the schemes. Some authors have suggested considering the possibility of redoing the intercomparison on a common sample of accepted cases. In this study, it was difficult to get the results exactly on the same data, as everybody runs their scheme as it is. Moreover, the study is limited by the lack of a ground-truth dataset which could be used in a further intercomparison exercise as a validation reference. We have already compiled more than IASI spectra over more than four months in the north and south polar regions, with co-registered data from the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP), the Cloud Profiling Radar (CPR) on Cloudsat and from NWP fields along with data from in situ campaigns from the Concordiasi campaign (Rabier et al., 2010). This dataset will be used to gain more understanding of the cloud situations and the a priori errors in an additional intercomparison.