An Algorithm to Screen Cloud-A ected Data for Sky Radiometer Data Analysis

Transcription

1 Journal of the Meteorological Society of Japan, Vol. 87, No. 1, pp , DOI: /jmsj An Algorithm to Screen Cloud-A ected Data for Sky Radiometer Data Analysis Pradeep KHATRI and Tamio TAKAMURA Center for Environmental Remote Sensing Chiba University, Chiba, Japan (Manuscript received 14 July 2008, in final form 18 November 2008) Abstract Aerosol optical parameters obtained from sky radiometer instrument are important not only for studying aerosol e ects on climate change, but also for validating several results obtained from satellite retrievals and numerical simulations. However, the greatest challenge is to separate cloud-a ected and cloud free data from data measured by sky radiometer. In this study, we present an algorithm to separate such cloud-a ected and cloud free data. The proposed algorithm is comprehensively tested with observational data. The algorithm consists of three tests: (i) test with global irradiance data, (ii) spectral variability test, and (iii) statistical analyses test. Though the test with the global irradiance data is the most powerful test, our study shows that it has some limitations, which can sometimes cause some clear sky data to be detected as cloud-a ected data. In order to cope with this problem, a modified version of spectral variability algorithm is proposed. As the second test, the modified spectral variability algorithm is applied to filter clear sky data from data detected as cloud-a ected by the first test. Finally, statistical analyses tests are performed to remove any outlier, if exists, from clear sky data detected by the first and second tests. It is shown that our proposed algorithm can screen cloud-a ected data more e ectively in comparison to other cloud screening algorithms. An application of this algorithm to screen observation data of one year collected in Chiba, Japan produces the seasonal means of optical thickness at 500 nm (Angstrom @0.53(@1.21), for winter, spring, summer and autumn seasons, respectively. Depending on the season, the initial seasonal mean optical thicknesses at 500 nm decrease and mean Angstrom exponents increase due to cloud screening. An application of this algorithm to dust-loaded atmospheres is also discussed. The proposed algorithm can be applied to any sky radiometer observation site as long as global irradiance data are available. 1. Introduction Due to the importance of aerosols on climate change study, they have received considerable attentions of atmospheric and climate change research communities in recent years. As a result, several space- and ground-based remote sensing approaches were deployed in the past with the aim of better understanding aerosol e ects of climate change. Such remote sensing systems provide a large volume of data with su cient spatial and temporal coverage. For the purpose of scientific use, such data should be qualitative. Data obtained Corresponding author: Pradeep Khatri, Center for Environmental Remote Sensing, Chiba University, 1-33 Yayoi-cho, Inage-ku; Chiba, , Japan. pradeep.nep@gmail.com ( 2009, Meteorological Society of Japan from ground-based remote sensing approaches have been widely used not only for understanding aerosol e ects on atmospheric heat budget (e.g., Kim et al. 2005), but also for validating satellite products (e.g., Higurashi et al. 2000) and model simulations (e.g., Takemura et al. 2000). Therefore, quality check of data obtained from ground-based remote sensing system is very important. For the purpose of understanding aerosol, cloud, and radiation interaction in the atmosphere of East Asia, a ground-based measurement network named as SKYNET was established in the last decade. This network has several monitoring stations at various locations of East Asia. Those stations are equipped with several automatic instruments including sky radiometer, pyranometer, pyrgeometer, etc. One of the key instruments of the SKYNET network is sky radiometer manufactured by

2 190 Journal of the Meteorological Society of Japan Vol. 87, No. 1 PREDE Co. Ltd., Japan, which measures both diffuse and direct radiations. Inversions of such measured radiations provide important atmospheric parameters such as aerosol optical thickness, single scattering albedo, volume size distribution, and refractive index of aerosol particles (Nakajima et al. 1996). Such data are very useful for aerosol science community, however, the biggest challenge for using such data is the separation of cloud-a ected data from cloud free data. Detection of clouds is possible based on human observation for the analysis of small amount of data of short observation period. But, such work is laborious and consumes considerable time. Furthermore, human analyzers may not be consistent in their cloud detection criteria. On the other hand, in order to analyze multiyear data of various stations, such traditional approach is impossible. In this regard, it is necessary to develop certain criteria that can e ectively separate cloud-a ected data from cloud free data. But, such criteria should be well tested and applicable to all observation area regardless of observation time. In this paper, we describe an algorithm to screen cloud-a ected data from data observed by sky radiometer. A detailed description of the instrument can be found in past studies (e.g., Aoki and Fujiyoshi 2003; Uchiyama et al. 2005; Takamura et al. 2008). In brief, the instrument consists of a sunand sky-scanning spectral radiometer, a sun sensor, a sun tracker, a control unit, a rain sensor, and personal computer. The instrument can measure both direct and di use radiations using a single detector at predefined scattering angles at regular intervals. It is possible for observation data to be a ected by cloud under a single or multiple conditions of: (i) cloud covers the sun while measuring direct radiation, (ii) cloud exists over and/or around an observation point while measuring di use radiation, and (iii) no cloud exists over an observation point, but measured radiations get a ected by nearby clouds. Therefore, the term cloud-a ected data refers to observation data, which are a ected by cloud as mentioned above. The paper is organized in the following way: (i) Section 2 gives a description of an algorithm with examples, (ii) Section 3 describes comparison of results obtained from our algorithm with those of past algorithms as well as application of this algorithm to dust-loaded atmospheres, and (iii) the main results of this study are summarized in Section Description of an algorithm Figure 1 shows the schematic diagram of an algorithm. In brief, the present algorithm uses three important tests to decide each observation datum as cloud-a ected or cloud free. The first test separates cloud-a ected and cloud free periods of observation days by examining global irradiance data. Secondly, spectral dependency behaviors of aerosols and clouds are used taking an advantage of the fact that aerosols have stronger wavelength dependency in comparison to clouds (Kaufman et al. 2006). Finally, statistical analyses tests are performed to remove outliers that pass the first and second tests. The tests used in this study may have some similarities to algorithms proposed in the past studies, but there are some major di erences too. The first test of our algorithm may look similar to an algorithm proposed by Long and Ackerman (2000) because both studies used irradiance data to detect clear sky periods. Long and Ackerman (2000) used an approach, which is computationally expensive. On the other hand, based on directly observed and radiative transfer model simulated data, we will show that consideration of a diurnal cycle measured in a very clear sky day as a standard case is realistic for detecting clear sky and cloud-a ected periods from measured global irradiance regardless of any variation in aerosol optical properties, precipitable water content, observation area, observation time, etc. More recently, Kaufman et al. (2006) proposed a spectral variability cloud screening algorithm (SVA). Noting that some obvious cloud-a ected data could not be screened by an original SVA, we have made modification for the purpose of using in this study. Therefore, the second test is a modified form of SVA. The third test, which can be considered as an auxiliary test, uses some statistical analyses similar to Smirnov et al. (2000). In Section 3.1, we will show that combination of various tests according to a scheme proposed in this study can remove cloud-a ected data more e ectively in comparison to some past cloud screening algorithms. 2.1 Test with global irradiance data On the time scale of a single day, the solar zenith angle is the most important factor that determines the diurnal cycle of downwelling global irradiance (F down ) for the clear sky. Other parameters such as aerosol optical thickness, Angstrom exponent, single scattering albedo, asymmetry parameter, sur-

3 February 2009 P. KHATRI and T. TAKAMURA 191 Fig. 1. A schematic diagram of cloud screening algorithm. For detail, see text.

4 192 Journal of the Meteorological Society of Japan Vol. 87, No. 1 face reflectance, and water vapor content exhibit far less influence in comparison with solar zenith angle. On the contrary, cloud can e ectively influence the diurnal cycle of F down. Therefore, cloudy skies exhibit certain irradiance characteristics that clear skies do not. The first test of our algorithm uses such characteristics of cloud to distinguish observed F down data into two groups, namely clouda ected and cloud free data. We have chosen F down data for our test because such data may be easily available in most of the observation sites. The F down ( mm) data used in this study are measured by a Kipp and Zonnes s CM21 radiometer. One of the most e ective approaches of determining F down fluctuation due to cloud is to observe whether diurnal cycle of measured F down follows a specific trend applicable to clear sky day. For a small amount of data, such detection may be possible based on human observation, however, such human observation is impossible for a large amount of data. For the time scale of a single day, a specific relationship can be expected between F down and cosine of the solar zenith angle (m) if the sky is cloud free. Our investigation for a large volume of clear sky diurnal F down generated by SBDART radiative transfer model (Ricchiazzi et al. 1998) covering the variations of aerosol optical thickness at 500 nm from and precipitable water content from cm as well as observed very clear sky diurnal F down at various SKYNET sites indicates that the following relationship is suitable between F down and m if the sky is cloud free: F down ¼ X5 i¼0 a i :m i ; ð1þ where the values of the coe cients ða i Þ can vary depending on observation day and location. Though the values of a i can vary depending on observation day and location, one important behavior for all clear sky diurnal cycles is that they show quite similar pattern with the highest (lowest) value in the smallest (largest) solar zenith angle. Therefore, the ratios of F down of two clear sky diurnal cycles can be expected to be nearly smooth if data of the same consecutive zenith angles of the very short time period are considered. This may not be true for some cases of large zenith angles as discussed latter in this section. This fact suggests that any clear sky diurnal cycle of F down can be considered as a standard for the purpose of detecting observed F down as the cloud-a ected or clear sky data. For our purpose, a very clear sky day (April 11, 2007 of Fukuejima ( N, E) shown in Fig. 2) is chosen and a five order polynomial relationship is developed between F down and m. The relationship between F down and m is as follows: F down ¼ 1609:6m :1m 4 þ 4082:9m :7m 2 þ 1202:3m 79:64: ð2þ The above equation produces correlation coe - cient (R 2 ) value of By considering that this relationship represents a standard clear sky day, theoretical diurnal cycle of F down for each observation day and location is estimated by inputting diurnal values of m in the above equation. m can be calculated from latitude, longitude, and local time. After that observed values of F down are normalized (divided) by theoretical values, and standard deviations of every 5 minutes data are taken. If the standard deviation is greater than or equal to 0.02, that time period (time period of 5 minutes) is kept in the cloud-a ected data group, otherwise it is kept in the clear sky data group. Note that the standard relationship above is derived by taking the very clear sky day of specific observation area. It is worthless to mention that depending on observation day and location, theoretical values of F down obtained from above equation may not exactly match with observed values in magnitude. We suggest that such exact agreement in magnitude is not necessary for our purpose as long as theoretical diurnal cycle shows similar pattern with observed diurnal cycle for clear sky day because our test detects cloud-a ected or cloud free observation periods based on the ratio of observed and theoretical values at various zenith angles. Instead of Eq. (2), if equations for other observed very clear sky diurnal cycles or equations obtained from radiative transfer model are used, the di erence occurs only in the magnitudes of ratio, however, the standard deviation of 5 minutes normalized data remain more or less unchanged. Therefore, though Eq. (2) is used as a default equation in our algorithm, the user may use his/her own equation representing a very clear sky diurnal cycle if he/she wishes to do so. In order to provide evidences that the abovementioned relationship can be considered as standard for normalizing measured F down of various observation sites for the purpose of detecting cloud-a ected and clear sky periods, we have considered three pairs of measured F down in three dif-

5 February 2009 P. KHATRI and T. TAKAMURA 193 Fig. 2. Diurnal cycles of global irradiance for some observation days in some SKYNET sites. Chiba and Fukuejima are Japanese sites and Dunhuang is a Chinese site. JST and CST represent Japanese and Chinese standard times, respectively. JST corresponds to Chiba and Fukuejima sites, and CST corresponds to Dunhuang site. Fig. 3. Standard deviations of 5 minutes data obtained by normalizing (a) observed global irradiance data with the standard clear sky global irradiance data and (b) radiative transfer model simulated clear sky global irradiances with the standard clear sky global irradiance data. Model simulated clear sky diurnal cycles are for precipitable water contents of 0 and 10 under assumptions of no aerosols and for aerosol optical thicknesses at 500 nm of 0.1 and 2.0 under assumptions of precipitable water content, single scattering albedo and asymmetry parameter at 500 nm equal to 1.0 cm, 0.95 and 0.70, respectively. The horizontal half line shows the value of threshold standard deviation (0.02). The vertical half line shows the extreme zenith angle of clear sky days for not crossing the threshold standard deviation value. For detail, see text.

6 194 Journal of the Meteorological Society of Japan Vol. 87, No. 1 ferent observation sites of di erent latitude and longitude (Fig. 2). Along with clear sky days a partially cloudy day (March 28, 2007 of Fukuejima) is purposely chosen to test our threshold standard deviation of Though Fig. 2 shows dissimilar curves of measured F down depending on observation day and location, however, standard deviations of data for each 5 minutes, which are obtained by normalizing measured F down with calculated F down using Eq. (2), are less than 0.02 (denoted by horizontal half line in Fig. 3a) for all clear sky days regardless of observation day and location for zenith angles less (denoted by vertical half line in Fig. 3a). As shown in Fig. 3(a), the standard deviation easily crosses 0.02 even at the small solar zenith angle (@13:10 JST) for March 28, 2007 of the Fukuejima observation when measured values of F down do not follow specific trend of F down applicable to clear sky day. For the same day, standard deviation values are quite high for some data falling within zenith JST) due to the same reason mentioned above. Figure 3(a) further shows that standard deviations of normalized data cross the threshold value for very clear sky days in some cases when the solar zenith angles are greater This is probably due to the fact that F down measuring radiometers are known to have uncertainties at large solar zenith angles (Michalsky et al. 1995; Sateesh et al. 1999). Because of this reason and some additional reasons that will be described at the end of this section, not all data with standard deviation greater than or equal to 0.02 are considered as cloud-a ected data. In order to further verify that Eq. (2) is valid for the purpose of normalizing F down measured under various atmospheric conditions, additional clear sky diurnal cycles of F down are generated using SBDART radiative transfer model. Clear sky diurnal cycles of F down are generated for (i) precipitable water content ranging from cm and assuming that aerosols are not present in the atmosphere and (ii) aerosol optical thickness at 500 nm ranging from and assuming that precipitable water content, single scattering albedo at 500 nm, asymmetry parameter at 500 nm are constant with the values of 1.0 cm, 0.95 and 0.70, respectively. Those clear sky diurnal cycles are normalized by data obtained from Eq. (2) and 5-minutes standard deviations of such normalized values are taken. Figure 3(b) shows standard deviations at various solar zenith angles for some selected atmospheric scenarios. Fig. 4. Measured global irradiances and sky radiometer observation points for two typical observation days (Feb. 12 and Dec. 11, 2007). Data were collected in Chiba. As evidenced by Fig. 3(b), an application of Eq. (2) for the purpose of normalizing and standard deviation criteria of 0.02 are valid for a wide range of clear sky atmospheric scenarios with varying atmospheric components. Figure 4 shows global irradiances along with sky radiometer observation points for two typical observation days (Feb. 12, 2007 and Dec. 11, 2007) of Chiba observatory site ( N, E). Similarly, Fig. 5(a) shows optical thickness at 500 nm ðt 500 nm Þ and Angstrom exponent ðaþ of those observation days, which are obtained by analyzing sky radiometer data using skyrad.pack (version 4.2) software (Nakajima et al. 1996). a is a parameter describing wavelength dependency behavior of aerosol optical thickness. In general, a is high for small particles and low for large particles. Similarly, Fig. 5(b) shows values of t 500 nm and a, which fall in clear sky data group according to test with global irradiance data. Important observations to be noted in Figs. 5(a) and 5(b) are that (i) aerosol optical parameters measured in the early morning are detected to fall into cloud-a ected data group including data for clear sky day of Feb. 12 and (ii) some data with high values of a ða > 1:0Þ observed on Dec. 12 are detected to fall in cloud-a ected data group. As an alternation, sky images captured by skyview camera are analyzed. Though based on image analyses it is di cult to accurately distin-

7 February 2009 P. KHATRI and T. TAKAMURA 195 Fig. 5. Values of aerosol optical thickness at 500 nm and Angstrom exponent (a) before cloud screening and (b) after cloud screening with the global irradiance data for two observation days (Feb. 12 and Dec. 11, 2007). guish cloudy or clear skies (Long and Ackerman 2000), the images indicate that the sky was cloud free through out the whole observation day on Feb. 12. The inability of this test (test with global irradiance data) to correctly detect cloudy or clear sky in the morning of Feb. 12 is likely due to uncertainty of radiometer at large solar zenith angle as mentioned above. On the other hand, it is observed that thin or thick cloud prevailed over the observation area before 10:00 JST on Dec. 11. JST JST on the same day, the sky was observed to be clear. Hemispheric images further suggest that cloud gradually started to move towards the observation area JST, and JST the observation area was covered by cloud through out the whole observation day. Those observations with the aid of sky view camera indicate that the test with global irradiance data may sometimes detect cloud free data as cloud-a ected data. For a more detailed investigation, one-year data of 2006 collected in Chiba observatory are analyzed. Analyzed data are only for the days when both sky radiometer and global irradiance data are available. Figures 6(a) and 6(b) show data detected as cloud-a ected by our method and an empirical clear sky detection method of Long and Ackerman (2000) that uses 1-min global irradiance data. Our method of Fig. 6. Scatter plots of aerosol optical thickness at 500 nm and Angstrom exponent of data detected as cloud-a ected by (a) the test with global irradiance data of this study and (b) an algorithm of Long and Ackerman (2000).

8 196 Journal of the Meteorological Society of Japan Vol. 87, No. 1 total data as cloud-a ected where as a method of Long and Ackerman (2000) detected much more (@46%) data as cloud-a ected. Figures 6(a) and 6(b) further show that, though our method is computationally less expensive, the method of Long and Ackerman (2000) detects more data with small t 500 nm and large a as cloud-a ected. Nonetheless, both methods that use global irradiance data to detect clear-sky suggest that a use of only global irradiance data can sometimes detect small t 500 nm with large a as cloud-a ected data. Such data showing strong wavelength dependency, i.e., having large a, are less likely to be cloud-a ected (Kaufman et al. 2005; 2006). Rejection of such considerable amount of data (@35%) as cloud-a ected without any further investigation may cause to loose some important information related to aerosol climatology. Therefore, further tests are necessary. 2.2 Spectral variability test Kaufman et al. (2006) proposed spectral variability cloud screening algorithm (SVA) that screens data as cloud-a ected if the below criteria is met for the variation of measurements over 15 minutes: Dt 870 nm Dt 440 nm ðt 870 nm =t 440 nm Þ > 0:0075 þ 0:03t 675 nm ; ð3þ where Dt l is the maximum di erence between the observed aerosol optical thickness and the next or previous one, and t l is the measured aerosol optical thickness. In the above equation, the value of means that the equation allows for noise in the maximum optical thickness di erence less than not to be called a cloud, and the value of 0.03 means that the equation allows the variability of 100% RH in the wavelength independent component of aerosol optical thickness to be associated with aerosol growth rather than cloud contamination (Kaufman et al. 2005). It is important to note that sky radiometer does not have a 440 nm channel. Therefore, instead of t 440 nm, t 400 nm is used when Eq. (3) is used for data observed by sky radiometer in this study. The above equation is based on an assumption that cloud optical thickness is wavelength independent and aerosol optical thickness has wavelength dependence (Kaufman et al. 2005). Kaufman et al. (2006) suggested that this algorithm does not eliminate the highly variable aerosol plumes as cloud. Application of this algorithm to our observation data of various observation time and location indicates that some obvious cloud-a ected data could not be screened by this algorithm. An example of one-month observation data is shown in Fig. 10(c), which will be discussed in Section 3.1. Since this algorithm screens each observation datum by referring the adjacent observation datum (next or previous measured optical thickness), we suspect that cloud-a ected datum can easily pass the test if referred adjacent datum itself is cloud-a ected. Therefore, for the purpose of our study, we modified this algorithm. We screened each observation datum by referring the nearest observation datum that has significant wavelength dependence, instead of always referring an adjacent datum. Therefore, our modified algorithm automatically takes an adjacent datum as a reference datum if the latter has significant wavelength dependence, otherwise it searches the nearest observation datum that has significant wavelength dependence in the forward or backward direction. Additionally, if the observation datum itself has a greater than or equal to 1, such datum is considered to be cloud free without test. Though the physical principal behind our modified algorithm is same to the original algorithm of Kaufman et al. (2006), an additional inherent important characteristic of modified algorithm is that observed datum is not rejected as cloud, as long as it has similar spectral properties to the cloud free aerosol. As discussed in Section 2.1, considerable amounts of data are detected as cloud-a ected by the first test. Before rejecting those data, the spectral variability test is performed to filter some data that may not be cloud-a ected. Figure 7 shows time series of t 500 nm and a for two typical observation days (Feb. 12, 2007 and Dec. 11, 2007) after the application of second test to data detected as clouda ected by the first test. As shown in Fig. 7, data detected as cloud-a ected in the early morning of Feb. 12 by the first test are recovered by the second test. Similarly, some data are recovered for Dec. 12 by the second test. An interesting thing to be noted here is that after the recovery of those data by the second test, cloud-a ected and cloud free data of those days show satisfactory agreement with observations made by sky view camera as discussed in Section 2.1. For observation data of one year in Chiba, Figs. 8(a) and 8(b) show data detected as cloud free and cloud-a ected by the second test for the data detected as cloud-a ected by the first test. It is revealed from Figs. 8(a) and 8(b) that significant amounts of data, which are not likely to be cloud-a ected, are recovered by the second test. Though some data with high t 500 nm are also re-

9 February 2009 P. KHATRI and T. TAKAMURA 197 Fig. 7. Values of aerosol optical thickness at 500 nm and Angstrom exponent after recovery of clear sky data by modified spectral variability algorithm for two observation days (Feb. 12 and Dec. 11, 2007). Fig. 8. Scatter plots of aerosol optical thickness at 500 nm and Angstrom exponent of (a) clear sky data and (b) cloud-a ected data as detected by the modified spectral variability algorithm for data detected as clouda ected by the test with global irradiance data. covered as clear sky data by the second test, such data show large a, suggesting that they are due to heavy pollution concentration rather than cloud. Similarly, some data detected as cloud-a ected show relatively small t 500 nm, which are very di cult to be judged as cloud-a ected or clear sky data. Application of the second test reduced the amount of cloud-a ected data (see Section 2.1) for the total data of one year observation in Chiba. 2.3 Statistical analyses tests The primary purpose of the statistical analyses tests is to remove any outliers from the variation of data that passed the above-mentioned two tests. The statistical analyses tests in this study are similar to Smirnov et al. (2000), excluding triplet stability criteria test because such test is not applicable to our observation data. The triplet stability criteria test needs three measurements within one minute, and such data are not available for sky radiometer

10 198 Journal of the Meteorological Society of Japan Vol. 87, No. 1 Fig. 9. Scatter plots of aerosol optical thickness at 500 nm and Angstrom exponent (a) before cloud screening and (b) after cloud screening for one-year data collected in Chiba in Data enclosed within dotted oval fall in the spring season. observation. In brief, data screened by abovementioned two tests are passed through diurnal stability test, data smoothness test, and three standard deviation criteria tests (Smirnov et al. 2000). Statistical analyses tests for the above-mentioned two days (Feb. 12, 2007 and Dec. 11, 2007) do not eliminate any data. This is probably due to the fact that the above-mentioned two tests are directly related to cloud characteristics, which have removed all cloud-a ected data. However, for a data of longterm observation, we observed elimination of some data. For example, for observation data of one year in Chiba, we noticed that statistical analyses tests alone of total data. Figures 9(a) and 9(b) show t 500 nm and a of one year observation before cloud screening and after cloud screening using above mentioned three tests. Approximately 77% of initial data remained after cloud screening. As shown in Fig. 9(b), some data with t 500 nm of around 1.5 still exist. Those data are less likely to be cloud-a fected because of their high values of a. Similarly, some data with high t 500 nm and small a (indicated by dotted closed oval) also exist. Those data are noted to fall in the spring season. Our unpublished data of Chiba observation indicate the influence of dust aerosols in the spring season, suggesting that such values are likely due to the influence of dust aerosols. The influence of dust aerosols in the spring season over Chiba was also reported by Fukagawa et al. (2006) by means of optical monitoring, ground sampling, and wind data. 3. Discussion 3.1 Comparison with past algorithms This section discusses comparison of results obtained from our algorithm with those of some other cloud screening algorithms. For comparison, two AERONET cloud screening algorithms proposed by Smirnov et al. (2000) and Kaufman et al. (2006) and clear skies identification algorithm proposed by Long and Ackerman (2000) are considered as they are the most frequently used algorithms to screen cloud-a ected data for observation data similar to sky radiometer. A detailed description of each cloud screening algorithm can be found in respective publications. It is important to mention that triplet stability criteria test, which needs three measurements within one minute, of Smirnov et al. (2000) is not included here. Figures 10(a), 10(b), 10(c), 10(d), and 10(e) show unscreened data, data screened with algorithms of Smirnov et al. (2000) excluding triplet stability criteria test, Kaufman et al. (2006), Long and Ackerman (2000) and our algorithm, respectively. Those data were collected in Chiba in Dec Similarly, Table 1 shows seasonal statistics (data count, average, and standard deviation) of unscreened and cloud screened data for one year observation in Chiba. As shown in Fig. 10(b), after application of an

11 February 2009 P. KHATRI and T. TAKAMURA 199 Fig. 10. Time series of (a) unscreened aerosol optical thickness at 500 nm and Angstrom exponent and screened data using algorithms of (b) Smirnov et al. (2000) (excluding triplet stability criteria test), (c) Kaufman et al. (2006), (d) Long and Ackerman (2000), and (e) present algorithm. Data were collected in Chiba in Dec algorithm of Smirnov et al. (2000) (excluding triplet stability criteria test), some data with large t 500 nm and small a (data of Dec. 5 and Dec. 25) are not eliminated. Note that such data are eliminated by an algorithm of Long and Ackerman (2000) (excluding one data point) and our algorithm, which use global irradiance data to detect clear sky days. This suggests that measured global irradiances on those observation days do not follow typical pattern applicable to clear sky days, indicating the influence of cloud. Furthermore, Table 1 suggests that this algorithm screens very few data for all seasons if the triplet stability criteria test is excluded. As a result, seasonal means and standard deviations of t 500 nm and a remain more or less similar to those of unscreened data. It is important to note that triplet stability criteria test is the main part of an algorithm of Smirnov et al. (2000). Since triplet stability criteria test is not performed here, this might have caused to remain some data, which are likely to be cloud-a ected. As shown in Fig. 10(c), application of an algorithm of Kaufman et al. (2006) shows results more of less similar to Fig. 10(b). Though results shown

12 200 Journal of the Meteorological Society of Japan Vol. 87, No. 1 Table 1. Statistics of data count, aerosol optical thickness at 500 nm ðt 500 nm Þ, and Angstrom exponent ðaþ before and after cloud screenings using various algorithms. The given statistics are only for observation days of 2006 in Chiba observatory when both sky radiometer and global irradiance data were available. Data status or algorithm name Season Parameter Winter (Dec., Jan., and Feb.) Spring (Mar., Apr., and May) Summer (June, July, and Aug.) Autumn (Sep., Oct., and Nov.) Before screening Data count t 500 nm G std:dev: G G G G a G std:dev: G G G G Smirnov et al. (2000)* Data count t 500 nm G std:dev: G G G G a G std:dev: G G G G Kaufman et al. (2006) Data count t 500 nm G std:dev: G G G G a G std:dev: G G G G Long and Ackerman (2000) Data count t 500 nm G std:dev: G G G G a G std:dev: G G G G Present algorithm Data count * without triplet stability criteria test t 500 nm G std:dev: G G G G a G std:dev: G G G G in Figs. 10(b) and 10(c) are qualitatively similar, Table 1 suggests that the latter algorithm screens more data in comparison to the previous one. Table 1 further shows lower (higher) seasonal means of t 500 nm ðaþ for latter algorithm in comparison to the former one, indicating that more data with large t 500 nm and small a are screened by the latter algorithm. Our experience of analyzing a large volume of data of various sites indicate that generally obvious cloud-a ected data show large t 500 nm and small a, and they fall on the upper range (above 75 percentile) of t 500 nm for total data of one month period or more. Our examination of few months data of some sites indicate that some of such data still remain after applications of both of above mentioned algorithms. We suspect that the possible reason for escaping some data, which are likely to be clouda ected, in an algorithm of Kaufman et al. (2006) is associated with referred reference datum (adjacent measurement datum) while scanning each observation datum as cloud-a ected or clear sky (see Section 2.2). On the other hand, Fig. 10(d) shows that an algorithm of Long and Ackerman (2000) produces much better result in comparison to abovementioned algorithms, that is, some data that are cloud-a ected are screened by this algorithm. But, one important observation to be noted in Fig. 10(d) is that some data with small t 500 nm and large a are eliminated too. Table 1 shows more clearly that this algorithm eliminates initial data as cloud-a ected in various season. The possible reasons are discussed in Section 2.1. Table 1 further shows that this algorithm considerably decreases (increases) the initial seasonal means of t 500 nm ðaþ in comparison to first two algorithms. On the contrary, Fig. 10(e) shows that in addition of keeping mostly all data detected as clear sky by an algorithm of Long and Ackerman (2000), our algorithm still preserves additional data, which are

13 February 2009 P. KHATRI and T. TAKAMURA 201 not likely to be cloud-a ected. Table 1 also shows that the seasonal means of both of t 500 nm and a produced by our algorithm are comparable to those of Long and Ackerman (2000), but our algorithm eliminates initial data as cloud-a ected by in various seasons. As demonstrated in Table 1, the seasonal means of t 500 nm ðaþ are observed @0.53(@1.21), for winter, spring, summer and autumn seasons, respectively after an application of our algorithm. It is also revealed from Table 1 that depending on the season, the initial (unscreened) seasonal mean values of t 500 nm decrease whereas a increase after cloud screening. The largest amount of cloud-a ected data is noted in the summer season. The mean values of t 500 nm are low in the winter and autumn seasons and high in the spring and summer seasons. Similar results were also reported by Fukagawa et al. (2006) by analyzing optical monitoring, ground sampling, and wind data collected for the periods of in Chiba, Japan. Though cloud screening method and seasonal means of t 500 nm are not explicitly expressed by Fukagawa et al. (2006), they reported small values less than 0.2 in autumn and winter and large values in spring and summer. Similar to our result, they found low a in the spring season due to the influence of Asian dust. This feature is also preserved by our algorithm. Considerable di erences in both t 500 nm and a between unscreened data and cloud screened data by our algorithm (or an algorithm of Long and Ackerman (2000)) are interesting to be noted. For example, mean t 500 nm decreased in the summer season after cloud screening. Such result is important for certain studies that use ground-based remote sensing data for validation purposes. Since reduction of data volume may impact the timeaveraged values (e.g., hourly mean, daily mean, weekly mean etc.) necessary for such validation purposes, an algorithm that e ectively screens cloud-a ected data by eliminating fewer amounts of observation data is important. This algorithm may fulfill this gap. 3.2 Application of this algorithm to the dust loaded atmospheres Cloud screening for the heavy dust loaded atmospheres is a challenging work. To our knowledge, a detailed study for such atmospheric scenarios has not been performed in past studies. We suggest that our proposed method may be applicable for such atmospheres too. As shown in Fig. 3(b), results produced by the first test of our algorithm (test with global irradiance data) are not affected by any increase in aerosol optical thickness if the sky is cloud free. This may give a hint that our proposed method is applicable even for the atmospheric scenarios with the heavy dust loading. With an increase of dust loading in the atmosphere, direct irradiance decreases, which is compensated by an increase of di use component. Overall, in comparison to direct and di use irradiances, global irradiance is less sensitive to an increase of dust loading in the atmosphere. Since diurnal cycle of global irradiance is mainly driven by solar zenith angle in the clear sky day, any perturbation of aerosol optical properties due to dust aerosols in certain period of observation day and/or the whole observation day may become a minor factor to influence the diurnal cycle of global irradiance driven by the solar zenith angle. In this regard, the first test may be applicable even for the dusty atmosphere as long as global irradiance data are available. Even if extremely heavy dust loading in an atmosphere bears a capacity to influence the diurnal cycle of global irradiance driven by the solar zenith angle, we suggest that our proposed algorithm may still classify clear sky and cloud-a ected data more e ectively. This is due to an application of the second test. The second test is used to filter clear sky data, which are likely to be not cloud-a ected but detected as cloud-a ected by the first test. The second test can recover clear sky data, which are mistakenly determined as cloud-a ected by the first test, by taking into account the spectral dependency behavior of the nearest clear sky datum. In order to show results for the heavy dust loaded atmospheres, we have applied this algorithm to screen cloud-a ected data for observation data collected in Dunhuang ( N, E) in the spring season (march, april, and may) of 2006 (Fig. 11). Dunhuang is located in the east of the Taklimakan desert. This region is expected to experience high frequencies of the dust event under strong wind speed and low relative humidity conditions during the spring season (Kim et al. 2004). Figures 11(a) and 11(b) show scatter plots of t 500 nm and a before and after cloud screening, respectively. Comparing Figs. 11(a) and 11(b), it is interesting to note that data with low t 500 nm and high a, which are less likely to be cloud-a ected, are well preserved after cloud screening. On the

14 202 Journal of the Meteorological Society of Japan Vol. 87, No. 1 Fig. 11. Scatter plots of aerosol optical thickness at 500 nm and Angstrom exponent (a) before cloud screening and (b) after cloud screening for spring season data collected in Dunhuang in other hand, data eliminated as cloud-a ected have relatively high t 500 nm and low a. It is quite di cult to confirm that all data appearing (disappearing) in Fig. 11(b) are cloud free (cloud-a ected) because an alternative highly accurate technique to detect cloud-a ected and clear sky for sky radiometer and similar instrument has not been developed so far, however, well-preserved data with low t 500 nm and high a and eliminated data having relatively high t 500 nm and low a may indicate that the present algorithm screens cloud-a ected data e ectively even for the dust loaded atmospheres. Since data were collected in an arid region in the spring season, dust concentrations of various ranges prevail in an atmosphere over the observation area. As a result, existence of dust aerosols having relatively high t 500 nm and low a are obvious in this region. Therefore, such data with relatively high t 500 nm and low a, which passed all tests of our algorithm and appeared in Fig. 11(b), are likely to be dust aerosols rather than clouds. Overall, after cloud screening, the mean value of t 500 nm decreased whereas a increased by of initial data. 4. Summary The main results of this study can be summarized as follows: 1. An algorithm is developed to screen clouda ected data from data measured by sky radiometer. The important tests of this algorithm are: (i) test with global irradiance data, (ii) spectral variability test, and (iii) statistical analyses test. 2. The study suggests that application of only global irradiance data to screen cloud-a ected data can sometimes eliminate some clear sky data, reducing aerosol data volume. Reduction of data volume may impact time-averaged values necessary for certain studies for validation purposes. Therefore, a modified version of spectral variability algorithm is implemented to recover some cloud free data from data detected as cloud-a ected by the test with global irradiance data. Finally, statistical analyses tests are performed to remove any outlier, if exist. 3. The proposed algorithm is comprehensively tested with experimental data and compared with some other cloud screening algorithms. It is observed that the proposed algorithm can screen cloud-a ected data more e ectively in comparison to some other algorithms. 4. Application of present algorithm to observation data of one year collected in Chiba, Japan produces seasonal means of t 500 nm @0.53(@1.21), for winter, spring, summer and autumn seasons, respectively by of total data. 5. Furthermore, the present algorithm is applied to dust-loaded atmospheres by taking data col-

15 February 2009 P. KHATRI and T. TAKAMURA 203 lected in Dunhuang in the spring season of It is discussed that the present algorithm may screen cloud a ected data e ectively even in the dust loaded atmosphere. The mean value of t 500 nm ðaþ is observed to decrease (increase) (@0.073) by of initial data after an application of cloud screening algorithm. We would like to make a few comments regarding our cloud-screening algorithm. In order to generalize the cloud screening method, we attempt to use the least type of easily available data. For example, along with data measured by sky radiometer, only global irradiance data are necessary to operate the proposed algorithm. Instead of the global irradiance data, direct irradiance data may be applicable, however, for using the latter type of data and/or both types of data certain modifications are necessary in the criteria put in the first test of our algorithm. The present version of the algorithm may not be applicable to screen very thin cirrus cloud-a ected data since this algorithm has not been tested to screen such data. As a part of further improving the proposed cloud screening method, we will deal such problems in our future study. In order to make it easy for the analyses of a large amount of sky radiometer data, a computer code is developed. The code is written in the FOR- TRAN language. The code as well as operation manual will be uploaded in the SKYNET homepage ( very soon. An online cloud screening system will be also available in the SKYNET homepage in the near future. Acknowledgement The authors are grateful to Japan Society for the Promotion of Science (JSPS) for providing a JSPS fellowship to the first author. This research is supported by the JSPS Fellowship Program, and also performed as a part of the SKYNET activities by the Observational Research Project for Atmospheric Change in the Troposphere (GEOSS program) of the Ministry of Education, Culture, Sports, Science and Technology, Japan. The authors are also grateful to two anonymous reviewers for their valuable comments. References Aoki, K., and Y. Fujiyoshi, 2003: Sky radiomter measurements of aerosol optical properties over Sapporo, Japan. J. Meteorol Soc. Japan, 81, Fukagawa, S., H. Kuze, G. Bagtasa, S. Naito, M. Yabuki, T. Takamura, and N. Takeuchi, 2006: Characterization of seasonal variation of tropospheric aerosols in Chiba, Japan. Atmos. Environ., 40, Higurashi, A., T. Nakajima, B. N. Holben, A. Smirnov, R. Frouin, and B. Chatenet, 2000: A study of global aerosol optical climatology with two channel AVHRR remote sensing. J. Climate, 13, Kafuman, Y. J., L. A. Remer, D. Tanre, R. R. Li, R. Kleidman, S. Matto, R. C. Levy, T. F. Eck, B. N. Holben, C. Ichoku, J. V. Martins, and I. Koren, 2005: A critical examination of the residual cloud contamination and diurnal sampling e ects on MODIS estimates of aerosols over ocean. IEEE Trans. Geosci. Remote Sens., 43, Kaufman, Y. J., G. P. Gobbi, and I. Koren, 2006: Aerosol climatology using a tunable spectral variability cloud screening of AERONET data. Geophys. Res. Lett., 33, L07817, doi: / 2005GL Kim, D. -H., B. -J. Sohn, T. Nakajima, T. Takamura, T. Takemura, B. -C. Choi, and S. -C. Yoon, 2004: Aerosol optical properties over East Asia determined from ground-based sky radiation measurements. J. Geophys. Res., 109, D02209, / 2003JD Kim, D. -H., B. -J. Sohn, T. Nakajima, and T. Takamura, 2005: Aerosol radiative forcing over East Asia determined from ground-based solar radiation measurements. J. Geophys. Res., 110, D10S22, doi: /2004jd Long, C. N., and T. P. Ackerman, 2000: Identification of clear skies from broadband pyranometer measurements and calculation of downwelling shortwave cloud e ects. J. Geophys. Res., 105, 15,609 15,626. Michalsky, J. J., L. C. Harrison, and W. E. Berkheiser III, 1995: Cosine response characteristics of some radiometric and photometric sensors. Sol. Energy, 54, Nakajima, T., G. Tonna, R. Rao, Y. Kaufman, and B. Holben, 1996: Use of sky brightness measurements from ground for remote sensing of particulate polydispersions. Appl. Opt., 35, Ricchiazzi, P., S. Yang, C. Gautier, and D. Sowle, 1998: SBDART: A research and teaching software tool for plane-parallel radiative transfer in the Earth s atmosphere. Bull. Amer. Meteoro. Soc., 79, Satheesh, S. K., V. Ramanathan, X. -L. Jones, J. M. Lobert, I. A. Podgorny, J. M. Prospero, B. N. Holben, and N. G. Loeb, 1999: A model for the natural and anthropogenic aerosols over the tropical Indian Ocean derived from Indian Ocean Experiment data. J. Geophys. Res., 104, 27,421 27,440. Smirnov, A., B. N. Holben, T. F. Eck, O. Dubovik, and

16 204 Journal of the Meteorological Society of Japan Vol. 87, No. 1 I. Slutsker, 2000: Cloud screening and quality control algorithm for the AERONET database. Remote Sens. Environ., 73, Takamura, T., N. Sugimoto, A. Shimizu, A. Uchiyama, A. Yamazaki, K. Aoki, T. Nakajima, B. J. Sohn, and H. Takenaka, 2008: Aerosol radiative characteristics at Gosan, Korea, during the Atmospheric Brown Cloud East Asian Regional Experiment J. Geophys. Res., 112, D22S36, doi: / 2007JD Takemura, T., H. Okamoto, Y. Maruyama, A. Numaguti, A. Higurashi, and T. Nakajima, 2000: Global three-dimensional simulation of aerosol optical thickness distribution of various origins. J. Geophys. Res., 105, 17,853 17,873. Uchiyama, A., A. Yamazaki, H. Togawa, and J. Asano, 2005: Characteristics of Aeolian Dust Observed by Sky-Radiometer in the Intensive Observation Period 1 (IOP1). J. Meteor. Soc. Japan, 83,