CAPABILITIES OF THE HAVEMANN-TAYLOR FAST RADIATIVE TRANSFER CODE (HT-FRTC): HYPERSPECTRAL RADIANCE SIMULATIONS AND ATMOSPHERIC AND SURFACE RETRIEVALS

Transcription

1 CAPABILITIES OF THE HAVEMANN-TAYLOR FAST RADIATIVE TRANSFER CODE (HT-FRTC): HYPERSPECTRAL RADIANCE SIMULATIONS AND ATMOSPHERIC AND SURFACE RETRIEVALS Stephan Havemann, Jean-Claude Thelen and Jonathan P. Taylor Met Office, FitzRoy Road, Exeter, United Kingdom of Great Britain Abstract The Havemann-Taylor Fast Radiative Transfer Code (HT-FRTC) is based on Principal Components (PCs) and allows fast and exact radiance and/or transmittance calculations. It is ideally suited for the simulation of hyperspectral sensors with hundreds or thousands of channels. The HT-FRTC can simulate a full instrument spectrum for any atmosphere and surface within a few milliseconds. It works for satellite-based, airborne and ground-based sensors. The code has been applied in any part of the spectrum from the short-wave to the long-wave (i.e. infrared plus microwave). It includes the solar and the lunar source and can account for the spherical Earth. The HT-FRTC is trained on a wide variety of different atmospheric and surface conditions. The gaseous spectroscopy has been updated to the most recent version of LBLRTM (12.1). For the simulation of scattering by clouds and aerosols and Rayleigh scattering in the short-wave, an exact spherical harmonics line-by-line code has been integrated into the HT-FRTC, which is very similar to the Edwards-Slingo (ES) band radiation code. For clear-sky and scattering atmospheres, the Principal Components (PCs) can be derived from lineby-line monochromatic radiances/transmittances rather than those related to instrument channels This is a major advantage of the HT-FRTC as it makes the PCs completely sensor independent. For the prediction of the PC scores (weights) for a given atmosphere/surface, monochromatic radiances are used which have been selected by k-means clustering during the training. The HT-FRTC has been incorporated into a one-dimensional variational (1D-Var) retrieval system that also works solely in PC space. This keeps the dimensions of the matrices involved small. The solution of the full non-linear problem is achieved by an iterative Levenberg-Marquardt minimization procedure. The retrieval state vector includes the vertical profiles of atmospheric temperature, water vapour and ozone, and possibly other trace gases as well as the surface temperature and surface emissivity / reflectivity (the latter being represented by a set of PCs). For a scattering atmosphere, cloud parameters and aerosol parameters have been added to the state vector. The cloud part of the state vector for cirrus cloud includes cloud-top pressure, ice water content, cloud fraction and cloud geometrical thickness. For water cloud there is also an effective droplet size. The HT-FRTC has also been used to retrieve the surface properties from airborne/spaceborne hyperspectral radiance measurements, both in the visible and the infrared. As an example, full-scene surface reflectance retrievals based on radiance measurements made by the hyperspectral shortwave imager AVIRIS (Airborne Visible / Infrared Imaging Spectrometer) are presented INTRODUCTION With the advent of the new hyperspectral infrared satellite-based sensors like the Atmospheric Infrared Sounder (AIRS) (Aumann et al., 2003) and the Infrared Atmospheric Sounding Interferometer (IASI) (Chalon et al. 2001) with thousands of channels, the need has arisen for fast radiative transfer models

2 to be able to quickly simulate the radiance spectra of such instruments for given atmosphere and surface conditions for use in retrieval and assimilation schemes. One of these new fast models is based on the Optical Spectral Sampling (OSS) method (Moncet et al., 2008).The OSS model allows the rapid calculation of the channel transmittances/radiances by a weighted sum of monochromatic transmittances/radiances at selected frequencies within the interval spanned by the instrument response function. Since there are considerable correlations across different parts of hyperspectral spectra, the compression of these spectra into their Principal Components (PCs) is a promising approach to dimensionality reduction both in the radiative transfer model and in retrievals. Such PC radiative transfer schemes have been developed in the past and successfully applied for some years now. One example is PCRTM (Liu et al, 2006) and another is HT-FRTC (Havemann, 2006; Havemann et al. 2008). Both PCRTM and HT-FRTC use selected monochromatic radiances to predict the PC scores for a given atmosphere/surface. With a forward model operating in PC space it is logical to do retrievals solely in PC space with the related dimensionality reduction and consequently the reduction in computational costs. PCRTM and HT-FRTC have both been used in physical retrieval schemes (Liu et al. 2009; Thelen et al., 2009). Another fast radiative transfer model which is used operationally for hyperspectral sensors like AIRS and IASI is RTTOV (which stands for Radiative Transfer for TOVS) (Saunders et al. 1999). RTTOV is maintained by the EUMETSAT Satellite Application Facility for Numerical Weather Prediction (NWP SAF). A Principal Component radiative transfer model has been developed which builds on RTTOV (PC_RTTOV) (Matricardi, 2010). PC_RTTOV uses selected polychromatic channel radiances calculated by RTTOV to predict the PC scores. PC_RTTOV has so far been developed for night-time clear-sky cases over ocean. THE HAVEMANN-TAYLOR FAST RADIATIVE TRANSFER CODE (HT-FRTC) Like PCRTM and RC_RTTOV, the HT-FRTC needs to be trained. Typically, a thousand different atmospheric profiles and surface emissivity/reflectivity spectra are randomly selected to go into the training set for which line-by-line (LBL) transmittance/radiance calculations are then carried out. Vertical profiles of temperature and gas concentrations as well as cloud are taken from a diverse profile dataset (Chevallier et al. 2006) as well as the surface temperatures. The surface emissivity/reflectance spectra are taken from the ASTER database (Baldridge et al. 2009) and according to Snyder (Snyder et al. 1998). The gas absorption properties are parametrizations obtained from the recent spectroscopy in LBLRTM 12.1 (Clough et al. 1992; Clough et al. 2005). The cirrus optical properties are due to a parametrization by Baran, which uses ice cloud temperature and ice water content as the independent quantities (Baran, 2009; Baran et al, 2009) and is not explicitly related to an effective particle size. In addition to water and ice clouds, precipitation in the form of rain and snow can be considered in the radiative transfer. The transmittance/radiance spectra obtained at line-by-line resolution during the training phase are then normalized by the variability across the training set, individually for each line-by-line frequency. From these normalized spectra the Principal Components, also at line-by-line resolution, are then derived. This approach differs from that taken by models like PCRTM and PC_RTTOV, where the Principal Components are generated for the channels of a given instrument. While HT-FRTC can calculate versions of the Principal Components at line-by-line resolution, which can be convolved at a later stage with any instrument line shape, in the case of the other models the convolution is done before the Principal Components are calculated. The use of line-by-line PC in HT-FRTC has the advantage that the PC are completely independent of any instrument characteristics and are valid for all instruments within the spectral range for which the line-by-line training calculations have been carried out. This means that changes in the instrument characteristics or things like apodisation do not require a new training of the fast radiative transfer model, which is a clear practical advantage.

3 The PC scores are predicted from the results of monochromatic calculations at selected frequencies. This is the same approach as taken by PCRTM, while PC_RTTOV uses polychromatic (channel) predictors. In HT-FRTC, the selection of the monochromatic frequencies is obtained by k-means clustering of the normalized spectra from the training phase, where only the centroids of the clusters above a certain size are chosen (Forgy, 1965). For the simulation of scattering by clouds and aerosols and Rayleigh scattering in the short-wave, a spherical harmonics line-by-line code has been integrated into the HT-FRTC, which is similar to the Edwards-Slingo band radiation code (Edwards and Thelen, 2008).. The HT-FRTC is used in a 1D-Var retrieval system (Rogers, 2000), which operates solely in Principal Component space. The variational retrieval system creates a lot of diagnostics, in addition to the solution, like the degrees of signal as a measure of information content gained. The state vector includes the vertical profiles of atmospheric temperature, water vapour and ozone, and possibly other trace gases as well as the surface temperature and surface emissivity / reflectivity (the latter begin again represented in Principal Components). For a scattering atmosphere, cloud parameters and aerosol parameters have been added to the state vector. All state vector elements can be retrieved simultaneously or alternatively some elements of the state vector can be kept fixed and not retrieved. The cloud part of the state vector for cirrus cloud includes cloud-top pressure, ice water content, cloud fraction and cloud optical thickness. For water cloud there is also an effective droplet size. Results of cloudy retrievals were presented at a previous conference (Havemann, 2010). Full scene hyperspectral retrievals from short-wave radiance spectra will be presented in the following section as an example of an application of the HT-FRTC. FULL-SCENE SURFACE REFLECTANCE RETRIEVALS WITH THE HT-FRTC The HTFRTC-code, combined with a 1D-VAR physical retrieval scheme, has been used in the past to retrieve the surface emissivity, as well as the temperature and the humidity profiles, from infrared, hyperspectral air/space-borne radiance measurements (Thelen et al., 2009). Here this technique is applied to the short-wave in order to retrieve the surface reflectances from the data taken by the short-wave hyperspectral imager AVIRIS using a 1D-Var approach, at speeds comparable to those of traditional atmospheric correction schemes (Thelen and Havemann, 2012). The hyperspectral image studied here in more detail was acquired by AVIRIS over San Jose in California (37.45N,-122.0E) on the 20 th June 1997 at UTC (Figure 1). Figure 1. False color image for San Jose, obtained by combining channels 8 (at 438 nm), 18 (at 537 nm) and 29 (at 646 nm).

4 Hyperspectral imagers are characterized by a large number of channels, a high spatial resolution and high data rates, which makes the processing of and retrievals from such images computationally very expensive. To make the problem more tractable, a two stage approach has been developed. Before attempting to retrieve the surface emissivity spectra, the hyperspectral radiance spectra of all the individual pixels of the San Jose scene were clustered using the same k-means cluster algorithm that has been mentioned earlier in the different context of deriving suitable PC predictors. Pixels that end up in the same cluster should have similar surface properties. Figure 2. shows that this is indeed the case. Three of the clusters obtained, clusters 1, 3, and 15 have been overplotted in red on a greyscale image of the underlying AVIRIS scene. A comparison of these clusters to the false-color image (Figure 1) reveals that they correspond to some type of soil (cluster 1), some type of vegetation (cluster 3) and some type of manmade surface (concrete/asphalt) (cluster 15). Figure 2: Clusters 1 (left), 3 (middle) and 15 (right) obtained by clustering the San Jose image After determining the clusters and their centroids for the image, the surface reflectance spectra were then retrieved for the pixels corresponding to the cluster centroids. In the first step, the surface reflectance spectrum was retrieved together with the atmospheric profiles for the centroid pixel belonging to the largest cluster. For this retrieval, the ECMWF ERA Interim atmospheric profiles for the correct time and location were chosen as the a priori, together with the surface temperature and surface pressure. In the absence of any information about the aerosols, their optical properties were based on the rural and urban aerosols described in SBDART (Ricchiazzi et al. 1998) and it was assumed that the aerosols were concentrated below 750 hpa with an extinction of 0.1 km -1, corresponding to a horizontal visibility of 40 km as predicted by the UK MetOffice global forecast model for the time and location. Figure 3 shows the results of the profile retrievals for the centroid of cluster 1, the black and red lines denoting the retrieved and background profiles respectively. The 1D- Var retrieval decreases the specific humidity between approximately 900 hpa and 50 hpa while simultaneously decreasing the aerosol extinction at 0.55 microns by about 25%. In a next step, the retrieved atmospheric profiles for the largest cluster are then taken as the background for the retrievals of all the remaining clusters. As these profiles are assumed to represent the true state of the atmosphere, these are no longer allowed to vary, i.e. during this second stage only the surface reflectance spectra are retrieved. The idea behind this that the atmosphere will only be influenced significantly by the surface covering the largest area while surfaces covering only small areas will have no effect on the atmosphere. For the surface retrievals, the a priori is assumed to be water in all cases. The results for the surface reflectance retrievals are presented in Figure 4. The black and blue lines in the surface reflectance plots refer to spectra retrieved in the presence of aerosols and under the clearsky (no aerosol) assumption, resprectively. The dashed lines represent the retrieval error. In the case considered here, where the atmospheric aerosol loading is very low, the assumption of clear-sky conditions has little effect on the surface reflectance retrievals results. Apart from slightly reducing the overall amplitude of the surface reflectance, the spectral features of the spectrum remain practically unchanged. A comparison of the retrieved spectrum to the red line, which is a reference soil spectrum

5 from the ASTER database, confirms that cluster 1 corresponds to some type of soil as indicated earlier. For the clusters 3 and 15, in addition to the surface reflectance retrievals with and without aerosol (black and blue lines), spectra for vegetation and concrete/asphalt have been added from the U.S. Geological Survey (USGS) spectral database, respectively, which show that the retrieval results are in good agreement with the surface properties expected for the clusters under consideration. Figure 3: Retrieved atmospheric profiles (temperature, specific humidity, aerosol extinction at 0.55 microns), all in black. The red lines denote the corresponding background profiles Figure 4: Reflectance spectra (black) retrieved from cluster 1 (left), 3 (middle) and 15 (right). The dashed lines show the retrieval error while the red lines denote reference spectra taken from the USGS database. REFERENCES Aumann H.H., M.T. Chahine, C. Gautier, M. Goldberg, E. Kalnay, L. McMillin, H. Revercomb, P.W. Rosenkranz, W.L. Smith, D.H. Staelin, L. Strow, and J. Susskind, (2003) AIRS/AMSU/HSB on the Aqua Mission: Design, Science Objectives, Data Products and Processing Systems, IEEE Transactions on Geoscience and Remote Sensing, 41, 2, pp Baldridge A.M, S.J. Hook, C.I. Grove, and G. Rivera The ASTER Spectral Library version 2.0, Remote Sensing Environment, 113, Baran, A.J. (2009) A review of the light scattering properties of cirrus, Journal of Quantitative Spectroscopy and Radiative Transfer, 110, pp Baran A.J., P.J. Conolly, and C. Lee (2009) Testing an ensemble model of cirrus ice crystals using midlatitude in situ estimates of ice water content, volume extinction coefficient, and the total solar optical depth, Journal of Quantitative Spectroscopy and Radiative Transfer, 110, pp

6 Chalon G, F. Cayla, D. Diebel, (2001) IASI An advanced sounder for operational meteorology, Proceedings of the 52th Congress of IAF, Toulouse, France, 1-5 October. Chevallier F., S. D Michele, and A. P. McNally (2006) Diverse profile datasets from the ECMWF 91 level short-range forecasts, NWPSAF EC TR 010 Version 1.0. Clough, S.A., M.J. Iacono, and J.-L. Moncet, (1992) Line-by-line calculation of atmospheric fluxes and cooling rates:application to water vapour, J. Geophys. Res., 97, pp Clough, S. A., M. W. Shephard, E. J. Mlawer, J. S. Delamere, M. J. Iacono, K. Cady-Pereira, S. Boukabara, and P. D. Brown, (2005) Atmospheric radiative transfer modeling: a summary of the AER codes, Short Communication, J. Quant. Spectrosc. Radiat. Transfer, 91, Edwards J.M., J.-C. Thelen (2008) Solving for Radiances: A Technical Report, Met Office Internal Report, OBR. Forgy, E.W. (1965) Cluster analysis of multivariate data: efficiency versus interpretability of classification, Biometrics, 21, 768. Havemann, S, (2006) The development of a fast radiative transfer model based on an empirical orthogonal functions (EOF) technique, Proceedings Paper in Multispectral, Hyperspectral, and Ultraspectral Remote Sensing Technology, Techniques, and Applications, William L. Smith, Sr.; Allen M. Larar; Tadao Aoki; Ram Rattan, Editors. SPIE Proceedings 6405, 22 December, DOI: / Havemann, S., J.-C. Thelen, J.P. Taylor and A Keil, (2008) The Havemann-Taylor Fast Radiative Transfer Code: Exact fast radiative transfer for scattering atmospheres using Principal components (PCs), Current Problems in Atmospheric Radiation (IRS 2008), Proceedings of the International Radiation Symposium (IRC/IAMAS) edited by T. Nakajima and M.A. Yamasoe, AIP Conference Proceedings 1100, pp , 3-8 August, Foz do Iguacu, Brazil Havemann S., J.-C. Thelen, A.J. Baran and J.P. Taylor, (2010) Variational retrievals of atmospheric profiles including clouds in principal component space using the Havemann-Taylor Fast Radiative Transfer Code (HT-FRTC), American Meteorological Society (AMS) 13th Conference on Cloud Physics / 13th Conference on Atmospheric Radiation, Portland, Oregon, USA, 28th June 2nd July Liu, X, W.L. Smith, D.K. Zhou, and A. Larar, (2006) Principal component-based radiative transfer model for hyperspectral sensors: theoretical concept, Applied Optics, 45, 1, pp Liu, X., D.K. Zhou, A. M. Larar, W.L. Smith, P. Schluessel, S.M. Newman, J.P. Taylor, and W. Wu (2009) Retrieval of atmospheric profiles and cloud properties from IASI spectra using super channels, Atmos. Chem. Phys., 9, pp Matricardi, M. (2010) A principal component based version of the RTTOV fast radiative transfer model, Quarterly Journal of the Royal Meteorological Society, 136, pp Moncet, J.-L., G. Uymin, and A. Lipton, (2008) OSS radiative transfer method performance in real time atmosphere characterization from satellite sounding and imaging data. In Proc. Internat. Geoscience and Remote Sensing Symp., IEEE, 6-11 July, Boston. Ricchiazii P., S.R. Yang, C. Gautier, and D. Sowle, (1998) SBDART: A Research and Teaching Software Tool for Plane-Parallel Radiative Transfer in the Earth s Atmosphere, Bulletin of the American Meteorological Society, 79, pp Rogers, C.D. (2000) Inverse Methods for Atmospheric Sounding, Theory and Practice, Series on Atmospheric, Oceanic and Planetary Physics, 2, World Scientific, 240p.

7 Saunders, R., M. Matricardi, and P. Brunel (1999) An improved fast radiative transfer model for assimilation of satellite radiance observations, Quarterly Journal of the Royal Meteorological Society, 125, pp Snyder W.C., Z. Wan, Y. Zhang, and Y.Z. Feng (1998) Classification-based emissivity for land surface temperature measurement from space, International Journal of Remote Sensing, 19, pp Thelen J.-C., S. Havemann, S. Newman, and J. Taylor, (2009) Hyperspectral retrieval of land surface emissivities using ARIES, Quarterly Journal of the Royal Meteorological Society, 135, pp , DOI: /qj.520. Thelen J.-C. and S. Havemann, (2012) Hyperspectral retrieval of surface reflectances: A new scheme, Proceedings Paper in Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XVIII, S.S. Shen and P.E.Lewis, Editors. SPIE Proceedings 8390, April