A flexible inversion algorithm for retrieval of aerosol optical properties from sun and sky radiance measurements

Transcription

1 A flexible inversion algorithm for retrieval of aerosol otical roerties from sun and sky radiance measurements Oleg Dubovik (1,2), and Michael D. King (3) 1. Laboratory for Terrestrial Physics, NASA Goddard Sace Flight Center, Greenbelt, Maryland 2. Also at Science Systems and Alications, Inc., Lanham, Maryland 3. Earth Sciences Directorate, NASA Goddard Sace Flight Center, Greenbelt, Maryland Abstract The roblem of deriving comlete aerosol otical roerties from sun and sky radiance measurements is discussed. The algorithm develoment is focused on imroving aerosol retrieval by including into the inversion rocedure the detailed statistical otimization of the influence of noise. The otimized inversion algorithm is built on the rinciles of statistical estimation: the sectral radiances and various a riori constraints on aerosol characteristics are considered as multi-source data that are known with redetermined accuracy. The inversion is designed as a search for the best fit of all considered data by a theoretical model that takes into account the accuracy differences of the fitted data. The multivariable fitting is imlemented by a stable numerical rocedure combining matrix inversion and univariant relaxation. The algorithm design allows the use of different statistics of exerimental noise in the solution otimization, as well as using various a riori constraints on retrieved aerosol arameters. This flexibility in algorithm organization hels to achieve simultaneous and reliable inversions of comlex data sets, which include various radiative, and microstructure characteristics. The inversion algorithm is adated for the retrieval of aerosol characteristics 1

2 2 from radiances measured from ground based sun - sky scanning radiometers used in the AErosol RObotic NETwork (AERONET). The aerosol size distribution and comlex refractive index together with aerosol hase function and single scattering albedo are retrieved from the sectral measurements of direct and diffuse radiation. The aerosol articles are modeled as homogeneous sheres. The atmosheric radiative transfer modeling is imlemented with well-established ublicly available radiative transfer codes. The retrieved refractive indices can be wavelength deendent, however the extended smoothness constraints are alied to both retrieved size distributions and the sectral deendence of refractive index. The ositive effects of noise statistic otimization on the retrieval results as well as the imortance of alying a riori constraints are discussed in detail for the retrieval of both aerosol size distribution and comlex refractive index. The results of numerical tests together with examles of exerimental data inversions are resented. 1. Introduction Currently there are numerous studies focused on measuring and interreting aerosol otical roerties. Esecially high exectations are associated with satellite and ground based remote sensing (e.g., see King et al. [1999], Kaufman et al. [1997]); however, not every required radiative characteristic can be measured remotely. Corresondingly, a core asect of remote sensing is the inversion rocedure, whereby aerosol otical and radiative characteristics are derived from the remote sensing measurements. In the ast three decades, a number of inversion algorithms have been roosed for interreting the measured radiative characteristics of the cloud free atmoshere. For examle, the codes of King et al. [1978], Nakajima et al. [1983, 1996] and Wang and Gordon [1993] for deriving aerosol otical roerties from atmosheric radiances have been established. These codes differ in the set of retrieved aerosol arameters and/or set of required inut radiative characteristics. The resent aer describes a

3 3 new algorithm that retrieves an extended set of aerosol arameters from multi-angular and multi-sectral measurements of atmosheric radiances. The urose is to maximize the retrieved aerosol information by inverting simultaneously all available measurements of atmosheric radiances. Namely, in the resent aer we consider the simultaneous retrieval of aerosol article size distribution and comlex refractive index from sectral otical thickness measurements combined with the angular distribution of sky radiance measured at different wavelengths. This retrieval aroach is consistent with the methods develoed by King et al. [1978] and Nakajima et al. [1983,1996] for retrieving the article size distribution of aerosol in the total atmosheric column. The method of King et al. inverts sectral measurements of otical thickness only, whereas the method of Nakajima et al. inverts the angular distribution of sky radiance (with or without sectral otical thickness). Both methods model aerosol articles as homogeneous sheres with refractive indices assumed a riori. The concets for determining aerosol article refractive index from multi-angular radiance measurements were develoed by Wendish and von Hoyningen-Huene [1994] and Yamasoe et al. [1998]. These methods are based on the rincile of artial searation of the effects of refractive index and size distribution on the angular variability of sky radiance. Our aroach is significantly different from earlier studies in that we imlement retrieval via simultaneous fitting of radiances measured in the entire available angular and sectral range. Such an aroach should rovide higher retrieval accuracy through adotion of sohisticated mathematical rocedures. The resent aer addresses the simultaneous retrieval of a large number of significantly different arameters from multi-source data. For examle, direct sun and diffuse sky radiance are measured by sensors with different sensitivities and the accuracy requirements on measurements of direct sun radiation and diffuse sky radiance are rather different. Such accuracy differences should be taken into account when making multi-source data comatible. Similarly, the aerosol article size distribution and com-

4 4 lex refractive index are characteristics that are very different in nature. Corresondingly, the design of an algorithm for retrieving these characteristics should congruously rationalize the differences in units, ranges of variability, etc. Develoing any inversion algorithm demands two kinds of effort from the develoer. First of all, accurate forward modeling of measured atmosheric characteristics is required. The second necessary comonent of an inversion algorithm is a formal numerical rocedure that imlements a mathematical inverse transformation and that does not relate to a limited articular alication. In the following sections we will discuss both of these asects. For modeling atmosheric radiances we adot standardized ublicly available software, therefore leaving oen the ossibility of easily relacing one code with another as radiative transfer theory advances. Following this strategy in forward modeling, we ursue a similar goal of making the entire algorithm flexible and adjustable. In designing the algorithm, we tried to anticiate the ossibilities of ugrading forward modeling codes with new advanced versions and exanding the code alicability for new alications (e.g., accounting for light olarization, detailed characteristics of surface reflectance, incororating article nonshericity, etc.). We ursued a similar objective in imlementing the numerical inversion transformations in our retrieval algorithm. However, in contrast to forward modeling, designing a flexible numerical inversion algorithm requires clarification of inversion rinciles. Indeed, forward models differ mainly in the accuracy of describing a hysical henomenon and the seed of calculation. Corresondingly, for ractical alications, one always chooses the most accurate model rovided it satisfies the time standards. Choosing the best inversion method, on the other hand, is a more comlicated task, in that the evaluation of inversion accuracy is an ambiguous question, esecially for a case of the simultaneous retrieval of several variables. For examle, relacing a scalar model of light scattering by a model accounting for olarization results in doubtless

5 5 imrovement in the accuracy of describing any characteristic of scatted light. In contrast, retrieval errors are not so well correlated for different retrieved arameters. Due to a change of inversion methods the retrieval accuracy may imrove for one arameter but degrade for another arameter. Corresondingly, the reference between inversion methods is always rather uncertain. Detailed reviews of currently used methods can be found in various books, e.g. Twomey [1977], Tikhonov and Arsenin [1977], Houghton et al. [1983], Tarantolla [1987]. However, the existence of a variety of different well-established inversion rocedures creates an uncertainty for researchers in understanding how to choose the otimal technique for inversion imlementation. For examle, the widely used book by Press et al. [1992] rooses a diversity of inversion methods, however it does not direct the reader with exlanations as to which method and why it should be chosen for a articular alication. Such a situation is artly a result of the fact that most innovations were roosed under ressure of different secific ractical needs and derived in rather different ways. In the resent aer, we follow the inversion strategy roosed and refined in the revious studies by Dubovik et al. [1995, 1998a]. This strategy is focused on clarifying the connection between different inversion methods established in atmosheric otics and unifying the key ideas of these methods in a single inversion rocedure. Corresondingly, this strategy is rather helful for building otimized and flexible inversion techniques. For examle, in Sections 3 and 4.2 we outline the imortant connections of designed retrieval algorithms with the inversion methods widely adoted in the alication of atmosheric otics and remote sensing, such as the methods given by Phillis [1962], Twomey [1963, 1977], Tikhonov [1963, 1977], Chahine [1968], Rodgers [1976], etc. The effort of algorithm develoment was initiated under the AERONET (AErosol RObotic NETwork) roject [Holben et al., 1998] with the urose of meeting the high requirements of aerosol arameter retrieval accuracy needed for satellite data valida-

6 6 tion and imroved understanding of the radiative effects of aerosols. Therefore, the discussion of the algorithm design and retrieval accuracy will be focused on the interretation of radiances measured from AERONET ground based sun - sky scanning radiometers. 2. Forward modeling The AERONET network rovides globally distributed near real time observations of aerosol sectral otical thickness, aerosol size distributions, etc. in a manner suitable for integration with satellite data. This network has been develoed to rovide aerosol information from two kinds of ground-based measurements: sectral data of direct sun radiation extinction (i.e., aerosol otical thickness) and angular distribution of sky radiance. An inversion algorithm is required for the retrieval of aerosol size distribution, comlex refractive index, single scattering albedo, and hase function. Below, in this Section, we discuss the concet of atmosheric radiance modeling, which we emloy in our retrieval algorithm. 2.1 Radiative transfer modeling The atmosheric sky radiance can be modeled by solving the radiative transfer equation for a lane-arallel atmoshere. The angular distribution of diffuse radiation can be described by: [ ] ex( -m0 t)-ex( -m1 t) I( Q; l)= F0 m0 DW ( w0 tp( Q; l) + G(...)), if q ¹ q 0 (1a) m -m 0 1 ( ) ( ) ( ) + ( ) I( Q; l)= F0 m0 DW ex -m0 t w0 tp Q; l G..., if q = q 0, (1b) where I( Q;l) is the sectral sky-radiance measured at different wavelengths and at different scattering angles Q ; F 0 the exoatmosheric flux; DW the view solid angle; q 0 the solar zenith angle; q the observation zenith angle; m the air mass (m 0 = 1/cosq 0, m 1 = 1/cosq); t = t ext (l) the sectral extinction otical thickness; w 0 = w 0 (l) the single scatter-

7 7 ing albedo; and P(Q;l) the hase function at different wavelengths. The term G( ) = G(w 0 (l)t ext (l);p(q;l);a(l)) describes the multile scattering effects, where A( l) is the sectral surface reflectance. The above equation is written for a homogeneous atmoshere, without accounting for olarization effects and for angular indeendent ground reflectance (Lambertian aroximation). At resent, there are a number of wellestablished and ublicly available codes to account for multile scattering in diffuse radiance I( Q;l). For examle, in our studies we have used two indeendent discrete ordinates codes develoed by Nakajima and Tanaka [1988] and Stamnes et al. [1988]. These codes allow for vertical variability of atmosheric characteristics by dividing the atmoshere into a number of homogeneous layers. In these models, different otical thickness, hase function, and single scattering albedo characterize each layer. The modeling of t( l), w0( l) and P( Q;l) requires consideration of three main comonents under cloud-free conditions: gaseous absortion, molecular scattering, and aerosol scattering and absortion. These three atmosheric comonents comrise the total otical characteristics of an atmosheric layer as follows: total ext ( )= ( )+ ( )+ ( )+ ( ), (2) aer scat aer abs mol scat gas abs t l t l t l t l t l w aer total 0 l scat ( ) = mol ( )+ scat ( ) t ( l) t l t l total ext = t t total scat total ext ( ) l ( l), (3) where text l ( ) aer mol total t P scat l aer t total P scat l mol ( Q; l)= ( Q; l)+ total P ( Q; l), (4) tscat ( l) tscat ( l) aer ( ) is the aerosol otical thickness; w 0 aer ( l) the aerosol single scattering albedo; and P aer ( Q;l) the aerosol hase function. In the considered case of groundbased measurements of solar radiation, strong gaseous absortion can be avoided by instrumental design or aroriately accounted for from climatological data, and molecular scattering can easily be calculated from the surface ressure at the time of the measurements. For instance, the secified wavelengths of the four AERONET sky ra- ( )

8 8 diometer sectral channels (440, 670, 870, and 1020 nm) were carefully selected to avoid strong gaseous absortion [Holben et al., 1998]. Slight ozone absortion is accounted for from climatological data. The values of surface reflectance A(l) are also accounted for a riori, in site of the fact that A(l) can vary significantly deending on climatological and meteorological conditions. Indeed, uncertainty in a riori knowledge of surface reflectance A(l) is usually not critical for modeling of downward solar radiation for two rimary reasons. First, it is exected that values of A(l) can in some cases be obtained from accomanying measurements of uward radiation. Second, in most situations, light roagated from the sun dominates over reflected light in the downward radiation field and accuracy requirements on a riori estimates of A(l) are modest. Thus, local variability of atmosheric radiance I( Q;l) deends rimarily on the otical roerties of the aerosol articles, and for convenience of further discussion we can write: ( ) ( )= ( ) ( ) ( ) I Q; l I t aer l ; w aer l ; P aer ext 0 Q; l. (5) All of these characteristics (text aer ( l ), w 0 aer ( l), P aer ( Q;l)) are highly variable and will be considered below as unknown characteristics that can be retrieved from multi-angular and multi-sectral radiance data. In rincile, aerosol roerties vary in the vertical direction and a multi-layer model of atmoshere is required, in order to account for the vertical variations in t(l), w 0 (l), and P(Q;l). However, radiances measured at the ground are influenced by the whole atmosheric column and are not exected to be strongly deendent on the vertical distribution of aerosol. Consequently, most ground-based retrievals characterize the otical roerties of the aerosol in the total atmosheric column (columnar aerosol). Therefore, in our resent study we focus on designing an algorithm for the vertically homogeneous atmoshere. The strategy of accounting for vertical variability in the atmoshere will be outlined later in Section 4. Thus, from the viewoint of radiative transfer calculations, the radiance I(Q;l) measured from the ground is a function of the otical characteristics of columnar aero-

9 9 sol (t(l), w 0 (l) and P(Q;l)). This is why the inversion of atmosheric radiance can naturally be designed for the retrieval of these aerosol characteristics. For instance, Wang and Gordon [1993] and Box and Sendra [1999] emloy such an inversion strategy in their retrievals. Alternatively, the inversion can be focused on retrieving arameters of aerosol microstructure, such as article size, number, etc. We will utilize this aroach by extending the ideas reviously develoed in the aers of King et al. [1978] and Nakajima et al. [1983, 1996]. 2.2 Microhysics modeling of aerosol otical roerties The modeling of otical characteristics via arameters of microstructure is a rather common way of light scattering characterization in both laboratory and remote sensing methods (cf., McCartney [1977]). For examle, the aerosol otical characteristics (hase function (P( Q )), otical thickness of aerosol extinction, scattering and absortion (t ext (l); t scat (l); t abs (l))) can be modeled from microstructure arameters using the following aroximations: tscat( l) ( l)= æ 2 r ö max P Q; scat ( l ) n () è l ø ò K Q; ; m ; r r dr, (6) rmin ò r max t... ( l)= æ 2 ö Kt ( l; m ; r) n () r dr, (7) è l ø... rmin where r is article radius, n(r) = dn(r)/dr denotes article number size distribution, K scat ( ).is a scattering cross section and K t ( ) is an extinction cross section ( l 2 rq ext (...) in the case of Mie theory, where Q ext ( ) is the extinction efficiency factor). 2 In our studies we will assume aerosol articles are sherical. Corresondingly, the functions K scat ( ) and K t ( ) will be aroximated by Mie functions derived for sherical and homogeneous articles with the comlex refractive index: m( l)= n( l)- ik( l). Eqs. (6)-(7) allow one to consider size distribution and refractive index of aerosol arti-

10 10 cles instead of directly considering t(l), w 0 (l) and P(Q;l) of the aerosol. Finally, atmosheric radiance I(Q;l) given by Eq. (5) can be defined via Eqs. (6)- (7) as a function of the arameters of aerosol microstructure: I( Q; l)= I( dn() r dr; m ( l) ). (8) Thus, Eqs. (5) and (8) reresent two different strategies of atmosheric radiance modeling. Eq. (1) gives the formal radiative transfer modeling based on radiative characteristics (t(l), w 0 (l) and P(Q;l)) of the atmosheric layer with no assumtions on these characteristics. Therefore, the inversion of an atmosheric radiance can be designed for the retrieval of these aerosol characteristics. We emloy an alternative aroach and focus the inversion on retrieving arameters of the aerosol microstructure. In this case, some relationshi between otical thickness, single scattering albedo, and hase function is alied by assuming the aerosol articles are homogeneous sheres, as in Eq. (8). Additional discussion on details of atmosheric radiance modeling will be given in Section Inversion Strategy To formulate the criteria of inversion otimization we emloy rinciles of statistical estimation theory (cf., Edie et al. [1971]). Corresondingly, in designing the retrieval algorithm we account for the character and level of uncertainties in the initial data. This is esecially imortant when we invert the data measured under different exerimental conditions (i. e., data from different sources). Therefore, inversion of multi-source data is a subject of articular consideration here. Using a riori constraints is an another key asect, which requires a detailed deliberation. Phillis [1962], Twomey [1963], and Tikhonov [1963], have shown that alying a riori constraints (e.g., the smoothness of retrieved functions) is a critical comonent of designing a successful inversion with many arameters. Choosing the strength of a riori constraints is, however, an esecially challenging roblem (e.g., Rodgers

11 11 [1976], Twomey [1977], King [1982]), which becomes even more challenging when such different arameters as article size distribution and comlex refractive index are retrieved simultaneously. Our strategy is to consider measurements and a riori knowledge together as a single set of multi-source data. These data are combined in a single set using the rinciles of statistical estimation and strength of the influence of each data source on the retrieval result and assigned according to the relative accuracy of the data. Thus, the current Section discusses the rinciles of inversion otimization, which are the same for both measured and a riori data. The secific questions of alying a riori constraints are discussed in detail in Section Statistically otimized inversion of multi-source data The inversion is designed as a search for the best fit of all data considered by a theoretical model taking into account the accuracy differences of the fitted data. The errors in all inverted data are determined statistically. Both measured and a riori data are searated into grous assuming that data obtained from the same source (i.e., by the same way) have a similar error structure, indeendent of errors in the data obtained from another source. For examle, direct sun and diffuse sky radiances have different magnitudes and are measured by sensors with different sensitivity, i.e., errors should be indeendent (due to different sensors) and may have different values (due to different magnitudes). Thus, both measured and a riori data can formally be written as follows: fk * ( a)= fk( a)+d k (k=1, 2,,K), (9) where the vectors f 1 and f 2 relate to sky (at the selected wavelengths and angles) and sun (at the selected wavelengths) radiance measurements. The vector a denotes the aerosol arameters which should be retrieved. The vectors f k>2 include the values of a riori constraints on aerosol arameters or ossible accessory data. The asterisk *

12 12 denotes the data known with some uncertainties D k. Numerous studies have shown that the normal (or Gaussian) distribution is the most exected and aroriate function for describing random noise (detailed discussions can be found in the books by Edie et al. [1971] and Tarantolla [1987]). The normal Probability Density Function (PDF) for each vector f * k of initial data can be written in the form: ( )= ( ) ( ) ( ) - ( ( )- ) ( ) ( ( )- ) * m - 1/ 2 æ 1 * T - 1 * ö P fk( a) fk 2 det Ck ex fk a fk Ck fk a f è k, (10) 2 ø where T denotes matrix transosition, C k is the covariance matrix of the vector f k ; det(c k ) denotes determinate of the matrix, and m is the dimension of vectors f k and f * k. The vectors f * k are obtained from different sources and, corresondingly, they are statistically indeendent. This is why the joint PDF of all inverted data can be obtained by * simle multilication of the PDF of all vectors f k as follows: K K * * * æ 1 T 1 P( f1( a),.., fk( a) f1,..., fk)= ÕP( fk( a) fk) ~ exç- å( fk( a)-fk) ( C ) f ( a)-f è 2 k= 1 k= 1 ( ) * - * k k k ö. (11) ø According to MML (Method of Maximum Likelihood), the best estimates â of unknowns corresond to the maximum of likelihood function (PDF), i.e. P f1 aˆ,.., fk aˆ * * ( ( ) ( ) f1,..., fk)= max. (12) The MML is one of the strategic rinciles of statistical estimation and the obtained solution â is statistically the best in many senses (see Edie et al. [1971]). The solution is asymtotically (since PDF is defined asymtotically) normal and otimum (most accurate the retrieval errors have the smallest standard deviations). In addition, the MML solution kees many otimum characteristics even in the case of a limited number of observations. The otimum roerties of MML are closely connected with the Fisher information determination (see Edie et al. [1971]). The maximum of the PDF exonential term given by Eq.(11) corresonds to the

13 13 minimum of the quadratic form in the exonent. Therefore, the best solution â, which can be derived from all given data f * k, is a vector â corresonding to the minimum of the following form: K K é * T -1 * Y( a)= åg k Yk( a)= å g k fk - fk( a) Wk fk fk a ëê k= 1 k= 1 ( ) ( ) ( - ( )) ù. (13) ûú This equation is written via Lagrange multiliers g k and weight matrices W k defined as: Wk = 1 2 Ck, (14) ek where e 2 k denotes the variance of errors D k in the data vector f * k. Corresondingly, Lagrange multiliers get clear statistical interretation as the ratios of variances: g k e = 12 2 e. (15) It should be noted that there is no need to know the absolute value of the variance e 2 1, because the retrieval rocess is aimed at finding the global minimum of Y(a) and does not deend on the value of this minimum. At the same time, it is known that the value of Y(a) has a c 2 2 distribution and that the minimum of Y(a) statistically relates to e 1 as follows: k ( ) 2 Y min ( a)= Nf - N a e 1, (16) where N f is the number of values in all fitted vectors f * k and N a is the number of retrieved arameters. The above relation is often used for estimation of measurement error e 2 1. It is imortant to emhasize that MML only formulates the condition of otimality and it does not tell how to achieve the minimum of Y(a). Finding the minimum of quadratic form Y(a) is a technical question and choosing one or another rocedure does not imrove the solution rovided the roblem is not ill-osed and the solution is unique. According to our strategy of designing the inversion algorithm, the correct

14 14 osing of the roblem should be done at the stage of forming the initial data set given by Eq.(9). For examle, in our case of inverting sky (f * 1 ) and sun (f * 2 ) radiances, these two basic data sets will be sulemented by some a riori data of corresonding f * k with k>2. Therefore, the formulation of initial data sets denoted by Eq.(9) is a critical question in inversion algorithm develoment. In contrast, minimization of Y(a) is a technical question, which ractically does not affect the accuracy of the solution. Nevertheless, a good design of a minimizing technique is imortant for liberating comuter ower requirements and consequently reducing the time consumtion of the retrieval. 3.2 Minimization rocedure Modern scientific literature (e.g., Press et al. [1992]) rooses a variety of standardized mathematical methods and software for minimizing quadratic forms. As noted in the revious Section, the choice of method for finding the minimum of Y(a) (Eq. (13)) is not a critical issue and mainly deends on the comlexity of the deendencies f k (a) and the reference of the inversion algorithm develoer. Nevertheless, below we roose a generalized flexible scheme of minimization that can be easily reduced to different standard methods. The scheme shows the clear relationshi between different standard methods. Therefore, our exectations are that this scheme should be rather helful for designing inversion algorithms for different alications. For the general case of nonlinear functions f k (a), the minimization is usually imlemented by iteration: aˆ + 1 = aˆ - D a, (17a) where the correction Da can be aroximated by the linear estimator Dâ as follows: Da» t Daˆ. (17b) The multilier t 1 (arbitrary chosen) is tyically used in for roviding monotonic convergence of non-linear numerical algorithms (cf. Ortega and Reinboldt [1970]). Assuming that Dâ is in the close neighborhood of the solution â, a Taylor exansion can

15 15 be used: where U k, a a, i.e. U { } = k, a ji 2 fk( aˆ)= fk( a )+ U ( aˆ -a )+ aˆ -a... k, a o( ) + (18) is the Jacobi matrix of the first derivatives in the near vicinity of the vector ({ fk( a) } j) a i a ( ), and o â-a 2 denotes the function that aroaches zero as ( â-a ) 2 when ( â-a ) 0. Now, neglecting all terms of second or higher order in Eq. (18), we can consider f k (a) as linear functions in Eq. (13). Accordingly, the correction Dâ corresonds to the minimum of Y(a) with f k (a) linearly aroximated. Corresondingly, Dâ can be found (with account for noise otimization) as a solution of the so-called normal equation system, which for our case is the following (details are given in Aendices A-B): æ ç è K å k = 1 g ( ) ( ) ( )+ ( ) T -1-1ö U Wk U g W aˆ aˆ k, aˆ Da Da D = ø k k, K å = g ( U ) ( W ) f ( aˆ )-f g W Daˆ. k= 1 1 ( )+ ( ) ( ) T - 1 * - * k k, aˆ k k k Da Da (19a) This normal equation system is the solution of linear LSM (Least Square Method, e.g., Tarantolla [1987]) which gives the minimum of the quadratic form of Eq. (13) for linear functions f k (a). The normal equation system gives the solution of linear LSM and thus it gives an otimum linear estimate. The terms with multilier g Da are added in both the left and right arts of Eq. (19a) for imroving the convergence of the whole minimization rocedure given by Eqs. (17)-(19a) (details are given in Aendix B). These terms are incororated statistically in similar manner as all data in Eq. (9), i.e., the a riori exected correction ( Da ˆ) * is assumed statistically as estimates ( Daˆ) * =( Daˆ )+ D( Daˆ) with covariance matrix C Da. It should be noted that both the a riori estimate ( Da ˆ) * in Eq. (19) and the multilier t 1 in Eq. (17b) are mainly aimed to decrease the length of Dâ, because linear aroximation may strongly overestimate the Dâ correction. Un-

16 16 derestimation of Dâ does not affect the convergence, since underestimation may only slow down the arrival to the final solution and not to mislead the minimization. The key question of imlementing minimization by Eqs. (17)-(19) is the solving of the linear system Eq. (19a), which in the comact form can be rewritten as follows: F Daˆ = Ñ Y( aˆ ), (19b) where matrix F denotes the matrix on the left side of Eq. (19a). This matrix (at g Da = 0) closely relates to the matrix of Fisher information, widely considered in statistical estimation theory [Edie et al., 1971]. The vector Ñ Y( a ) (i.e., vector on the right side of Eq. (19a)) reresents the gradient of the quadratic form Y(a) (Eq. (13)). This vector has the rincial imortance for building otimum minimization [Ortega, 1988]. Thus, Eqs. (17)-(19) give rather a general and flexible form to the minimization of the quadratic form Y(a) (Eq. (13)). This rocedure can be easily transformed, by choosing a method for solving Eq. (19a), to many other well-established numerical rocedures based on matrix inversion, relaxation, combined iterations methods, etc. In our oinion, such freedom in incororating different linear inversion techniques to the generalized non-linear scheme (Eqs. (17)-(19)) is a great hel for both understanding the relationshis between existing inversion algorithms and in develoing our new algorithm. In our algorithm for inverting atmosheric radiance, we imlement two alternative techniques: matrix inversion (using singular value decomosition) and relaxation quasi-gradient techniques. A brief introduction to these methods is given below Matrix inversion The linear system given by Eq. (19) can be solved using matrix inversion oerations. First of all, the fundamental formula for linear LSM solution imlies matrix inversion (e.g., Press et al. [1992]). Corresondingly, a great number of the LSM related inversion methods use matrix inversion. For examle, Phillis [1962], Twomey [1963],

17 17 Tikhonov [1963], Turchin et al. [1970], Rodgers [1976], and others emloy matrix inversion in their methods. All of these methods are well known in otical alications and differ from basic LSM formulae by using differing a riori constraints (additional discussion can be found in Section 4 and in the aers of Dubovik et al. [1995, 1998a]). The basic scheme of solving a non-linear system is the traditional Newton-Gauss rocedure (e.g., Ortega and Reinboldt [1975]), which imlements the LSM rincile in the nonlinear case. Eqs. (17)-(19) can easily be reduced to the Newton-Gauss rocedure. Namely, if we define t = 1, g Da = 0 and g k = 0 (for k ³ 2) in these formulas, we obtain the Newton-Gauss method with statistical otimization at each -ste: + 1 T -1-1 T -1 a a U W U U W f f * ( ( )) = -( ) -, (20) where for simlicity we denote the vectors and matrices as follows: U denotes Jacobi matrix U 1, a ; W denotes weight matrix W 1 ; vector f denotes vector f(a ). In this Section, we always assume g k = 0 (for k ³ 2) only because the discussed standard numerical formulas are written for inverting a single data set. Obviously, Eq. (20) incororates the basic linear LSM formula. Indeed, Eq. (20) is reduced to linear LSM by assuming linear deendence f(a) = Ua: = -( ) ( ( - ))=( ) + 1 T -1-1 T a a U W U U W - 1 * T -1-1 T * Ua f U W U U W - 1 f. (20a) In ractice, Newton-Gauss iterations may not converge and need to be modified. The most established modification of Eq.(20) is widely known as the Levenberg- Marquardt method (e.g., Ortega and Reinboldt [1970], Press et al. [1992]). This method is also included in the scheme of Eqs. (17)-(19). Namely, if we assume t 1, g Da > 0 and ( Da ˆ) * = 0, then Eqs. (17)-(19) can be reduced to the Levenberg-Marquardt method: ( ) ( ) ( - ) + 1 a a U T -1-1 t W U ad U T - 1 W f * = - + g D f, (21) where D = (W D ) -1 and g Da = e0 2 ed 2. It should be noted that using the generalized inversion rocedure of Eqs. (17) and (19) hels to rovide an additional simle interretation

18 18 of the Levenberg-Marquardt method. Indeed, an a riori assumtion of ( Da ˆ) * = 0 means that we constrain the solutions Dâ to the smallest value (the closest to ( Da ˆ) * = 0). In addition, by assuming ( Da ˆ) * ¹ 0 and varying W D in Eq. (19a) the convergence character can be adjusted in the scheme of Eqs. (17)-(19) more flexibly than is ossible with standard Levenberg-Marquardt formula Eq. (21). The main difficulty of using the matrix method aears in the situation when the matrix F is of quasi-degenerate nature and the inverse oerator F ( ) -1 is very unstable. The ractical way of alying matrix inversion is to use matrix singular value decomosition. Singular value decomosition is an oeration of linear algebra, that allows one to decomose matrix F as F=VIw i A, where matrices V and A are orthogonal in the sense that V T V = I and A T A = I. Matrix I wi is diagonal with the elements on the diagonal equal to w i. Inversion of matrix F trivially follows from this decomosition as F - 1 T T = A I1 /w V. In the case of a singular matrix F, the inverse matrix of F is un- i certain, because some values w i are equal or close to zero. Corresondingly, by means of relacing w i = 0 by a moderately small non-zero w i, singular matrix F can be relaced by reasonably close non-singular matrix F which can be trivially inverted. The details of this method can be found in Press et al. [1992]. In many ractical situations singular value decomosition is very helful. Therefore, we emloy this rocedure in our algorithm to imlement matrix inversion. The main concern of alying this method comes from the fact that relacement of matrix F with matrix F is formal and has no relation to the hysics of an alication Alternatives to Matrix Inversion Methods Many methods are known in the mathematical literature that solve linear systems of equations without using matrix inversion. For examle, Jacobi and Gauss-

19 19 Seidel univariant iterations, steeest descent method, method of conjugated gradients, singular values decomosition, etc. Some of these methods can yield suerior results over matrix inversion oerations. For examle, in our algorithm we emloy linear iterations, which always give a result even if the linear system is singular and a solution is not unique. In contrast with inversions via singular value decomosition, iterations do not require any change of matrix F. In the aers by Dubovik et al. [1995, 1998a], solution of the -ste system (Eq. (19)) is imlemented by means of linear q-iterations and the whole minimization rocess is reresented via combined iterations (two kinds of iteration). Namely, Dâ is obtained from Eq. (19b) by means of q-linear iterations: [ ] Da q +1 Da q q H F Da q ( ) =( ) -( ) ( ) -Ñ Y( a ). (22a) Eqs. (17)-(18) and (22) formulate a search for the minimum a of the quadratic form Y(a) (Eq. (13)) via combined - and q-iterations. For each -iteration, a larger number of q-iterations can be made. The matrix H and vector ( Da ) q=0 can be chosen by various ways to assure that the iterations converge. Such a combined iteration technique is very helful for realizing statistical otimization (which usually is associated with matrix methods) by means of relaxation iterations ( H is a diagonal matrix) in situations where matrix inversion is not efficient. In addition, the consideration of combined iterations hels to understand relationshis between two categories of inversion methods: matrix inversion methods (Phillis [1962], Twomey [1963], Tikhonov [1963], Turchin et al. [1970], Rodgers [1976]) and relaxation techniques (Chahine [1968], Twomey [1975]). These two kinds of methods are very oular in atmosheric otics and remote sensing and they usually are considered as alternative. The steeest descent method deserves articular attention among all other relaxation techniques. This method has been deely elaborated in the mathematical lit-

20 20 erature (e.g., Forsythe and Wasow [1960], Ortega [1988]). The basic idea of the steeest descent method (or gradient search method) is to minimize the quadratic form Y(a) using it s gradient as a direction of the strongest local change of Y(a). The minimization rocedure given by Eqs. (17)-(18), (22), can be easily reduced to the steeest descent method by assuming H = t q 1, ( Da ) q=0 = 0 in Eq. (22): ( ( )) 1 T -1 a = a -t Ñ Y( a )= a -t U W f -f + *. (22b) Also, only one q-iteration is to be imlemented for each iteration in Eq. (22b), i.e. t,q = t and the combined iterations are reduced to only one kind of -iteration. As ointed out in Press et al. [1992], the steeest descent method is generalized by the Levenberg-Marquardt formula. Namely, Eq (19a) can be reduced to (21) by defining matrix D in Eq. (19a) as the unit matrix 1 and rescribing a large value to the arameter g Da. In Aendix D, we show that the oular Twomey-Chahine relaxation technique roosed by Twomey [1975] can be considered to be the steeest descent method. Equation (22b) solves both linear and non-linear equations. Corresondingly, the non-linear steeest descent iterations can be used directly for minimization of quadratic form in Eq. (13). However, such minimization can be very time consuming because, for the non-liner case, each iteration requires a recalculation of the Jacobi matrix U and the steeest descent method converges to the solution only after a very large number of iterations. Therefore, to reduce comutation time, we use the steeest descent method only to solve linear -ste systems Eq. (19b). In other words, we assume H = t q 1, ( Da ) q=0 = 0 in Eq. (22a). Then we imlement a large number N q of q- iterations. We choose the value of t,q roviding the fastest convergence of the rocess at each q-iteration. Forsythe and Wasow [1960] and Ortega [1988] describe the rinciles of defining such a value.

21 21 4. Sun-sky radiance inversion algorithm Sections 2 and 3 described two comlementary and necessary tools for realizing an inversion algorithm: a model of radiative transfer and a method of otimum inversion. Our intention was to structure and, in a certain sense, to standardize the rocess of designing an inversion algorithm. Namely, Section 3 outlined the otimization strategy common for any numerical inversion and roosed the scheme (Eqs. (17)-(19)) uniting a diversity of minimization methods. Our exectation is that the roosed inversion strategy enables one to create a flexible inversion algorithm, that can be easily ugraded with new develoments in forward modeling and/or numerical recies. At the same time, the ability to model radiance with available codes and to imlement numerical inversions does not reduce the design of sun-sky radiance inversion codes to a urely technical rocedure. There are many small and secific questions that need to be resolved in order to create an inversion rocedure that is efficient in ractice. Definitively, the key question in inversion algorithm develoment is quantifying the a riori constraints (defining Lagrange multiliers, formulating smoothing matrices, etc.) In addition, the forward model may also require some adjustments. For instance, numerical inversion of Eqs. (17)-(19) uses vectors of aerosol arameters, whereas the forward models (Eqs. (1) and (6)-(8)) oerate on continuous functions. Corresondingly, the vectors with a reasonable number of comonents should relace functions traditionally used in modeling. Thus, below in this Section, we roceed with the detailed design of a sun-sky radiance inversion algorithm, using the rinciles described in Sections 2 and Adatation of forward model to the inversion The scheme of numerical inversion given by Eqs. (17)-(19) requires extensive forward calculations. Namely, each -ste requires recalculation of fitted characteristics f(a) and Jacobi matrices U in the case of non-linear deendence f = f(a). Corresond-

22 22 ingly, adoting a fast technique of forward calculation is very imortant for making the inversion algorithm ractical and efficient. Possible ways of accelerating and adjusting the forward model for inversion uroses will be discussed below Otical thickness and hase function simulations Eq. (8) summarizes the modeling concet that relates otical roerties of the atmoshere with the size distribution (dn(r)/dr) and comlex refractive index ( m (l)) of the aerosol articles, assuming homogeneous sheres. Both size distribution (dn(r)/dr) and refractive index ( m(l)) will be the focus of the retrieval in the designed algorithm. The retrieval of article size distribution from the measurements of light scattered by olydisersions of sheres is a well-develoed otical alication. The concet of size distribution retrieval from single scattering measurements is articularly clear for a case of known refractive index; the integral equation (Eqs. (6) or (7)) can be reduced to a linear system, then solved by standard algebraic methods. In our case, the situation is more comlicated because the refractive index is unknown and the contribution of multile scattering to sky radiance is significant in some instances. Nevertheless, in our algorithm, relacing integral Eqs. (6) and (7) with linear systems is essential for making radiance simulations more raid. Also, Eqs. (6) and (7) are written for the size distribution of columnar aerosol article number concentration; however, ractical algorithms are often designed to retrieve the size distribution of surface area or volume of aerosol articles since light scattering of small single articles is a function of article surface area or volume (cf. Bohren and Huffman [1983]), rather than number concentration. Thus, for flexibility of our algorithm, we transform Eqs. (6) and (7) using different kinds of size distributions: number, radius, area and volume article size distributions. Then, to meet calculation seed requirements, we reduce the integral equations to a linear systems as follows:

23 ( ) max t l; ;... t... ( l)= æ r 2 ö K m r ( ) ln» t ( l; ; ) è... l øò xn ln r d r K n k x n (23) g () r r n min scat l tscat( l) ( l)= æ rmax 2 ö K ( Q; ; m ; r) P Q; n( )» scat( l ) n è l øò x ln r dln r K Q; ; n; k x. g () r rmin n Here, the index k (k = 0, 1, 2, 3) denotes the tye of distribution as follows: 0 dr () r 0 dn dn for n = 0 (number): x0( ln r)= = r = (i.e., g 0 = 1); dln r dln r dln r 1 dr r for n = 1 (radius): x r r dn dr 1 ln () ( )= = = d ln r d ln r d ln (i.e., g 1 = r); (24) r 2 dr () r 2 dn ds for n = 2 (area): x2( ln r)= = 2 r = (i.e., g 2 = 2r 2 ); dln r dln r dln r 3 dr () r 4 3 dn dv for n = 3 (volume): x3( ln r)= = r = (i.e., g 3 = 4/3r 3 ). dln r 3 dln r dln r The kernel functions of otical thickness K t... ( ) and differential scattering coefficient K scat ( ) aroximated in Eqs. (23)-(24) by matrices K t... ( ) and K scat ( ). The vector x n aroximates size distribution dr n (r)/dlnr by N r elements corresonding to the oints {x k } I = dr n (r i )/dlnr chosen with equal ste D ln r = ln ri+ 1- ln ri = const. The calculations of the matrices K t... (...) and K scat ( ) in our algorithm are imlemented in two different ways of interolating size distribution values between grid oints r i. First, the size distribution dr n (r)/dlnr between oints ln(r i )-(Dlnr)/2 and ln(r i )+(Dlnr)/2 can simly be assumed to be equal to dr n (r i )/dlnr, i.e., elements of the matrices are comuted as: 23 ln( ri )+ Dln r K... (...) ö K ji l ø ò ln( r )- ln n i D r 2 { } = æ è (...; r) g r () dln r. (25a) The traezoidal aroximation is another way of interolating between oints. In this case, the size distribution is aroximated between a grid oints ln(r i+1 ) and ln(r i ) linearly by dr n (r)/dlnr = a lnr + b, where a and b must be chosen to coincide with values dr n (r i+1 )/dlnr and dr n (r i )/dlnr. The matrix elements for this case are comuted according to Twomey [1977] as:

24 24 ln ri+1 2 ln ln... K... (...) ö ri+1 r K ji l ø ò ln ln ln( ) ( ri+1)- ( r r i) n i { } = æ è ( ) ( )- (...; r) g r () æ2 + ö èl ø dln r+ ln ln ( r ) ò i ( r ) i-1 ln r- ln( ri-1) ln r ln r ( )- ( ) i i-1 n K... (...; r) g r (). (25b) dln r The index j in Eqs. (25a)-(25b) relates to matrix elements with sun radiance at different wavelengths and sky radiance at different wavelengths and angles. The deendence of matrices K t... (...) and K scat (...) on real n and imaginary k art of the refractive index are aroximated from look-u tables over all ossible n and k values. Namely, we comute matrices in N n and N k grid oints, which cover the whole range of exected values. The matrices for the values of n and k between these grid oints are comuted using linear interolation on a logarithmic scale. It should be noted that in Eqs. (23)-(25), the size distributions are written in the logarithmic scale (dr n (r)/dlnr) instead of the linear scale (dn(r)/dr) used in Eqs. (6)-(7). This is because the kernel functions K... ( ) show much smoother variability for equal relative stes Dr/r (i.e., for equal logarithmic stes, since dr/r = dlnr) than for equal absolute stes Dr. Corresondingly, the logarithmic scale is commonly referred for viewing otically imortant details of the article size distributions and for making faster integrations over article size. According to Eqs. (25a)-(25b), the elements of the kernel matrices K ( ) are a roduct of the integration of kernel functions over article size. Such integration can be time consuming. Corresondingly, matrix aroximations (Eqs. (23)-(24)) are very efficient in ractice, because they allow romt calculation of otical thickness t (extinction and absortion otical thickness) and differential scattering coefficient t scat (l)p(q;l), give a vector of size distribution x k and refractive index m ( l). All of the above mentioned aroximations roduce some error even in socalled error free conditions. According to our estimations (for N r = 22 for the range: 0.05 r 15 mm; N n = N k = 15 for the ranges: 1.33 n 1.6 and k 0.5) these

25 25 errors can be considered as relative random errors with variance less than 0.01 for the tyical aerosol models given by Tanré et al. [1999]. For significantly narrower size distributions (which are rather unlikely for atmosheric aerosols) this error may increase to 2-3% Simulations of radiative transfer in the atmoshere As it was mentioned in the Section 2.1, we have emloyed a scalar discrete ordinates radiative transfer code to simulate diffuse radiance I(Q,l) in the lane-arallel atmoshere aroximation. To make ossible internal checks of the algorithm, we adoted two indeendent radiative transfer codes, one by Nakajima and Tanaka [1988] and the other by Stamnes et al. [1988]. However, for ractical reasons we mainly used the rogram of Nakajima and Tanaka [1988], since it emloys a truncation aroximation that allows fast and accurate calculation of downwelling radiance in the aureole angular range with a relatively small number of Gaussian quadratures oints. At the same time, it should be noted that we use radiative transfer codes only for modeling fitted characteristics f(a). Jacobi matrices U k,a of sun/sky radiance derivatives are calculated in the single scattering aroximation, i.e., for k = 1,2.: U k,a» U k,a (single scattering) (26) The elements of these matrices can be easily calculated from Eqs. (1a) and (1b) assuming G( ) equals zero. Our retrieval exerience shows that neglecting multile scattering in simulating first derivatives does not articularly affect the retrieval results. Thus, using Eqs. (23) (25), the aerosol otical thickness t (l), single scattering albedo w 0 (l) = t scat (l)/t ext (l), and hase function (P(Q,l)) are generated from the refractive index m (l) = n(l) ik(l) and the size distribution of aerosol articles dr n (r)/dlnr in the total atmosheric column. These aerosol characteristics weighted (as given by Eqs. (2)-(4)) with molecular scattering and gas absortion comose a set of atmosheric layer otical characteristics, that are necessary for radiative transfer com-

26 26 utations. Regarding vertical variability of the atmoshere, we consider two aroximations in our algorithm: (i) an atmoshere with vertically homogeneous otical roerties, and (ii) an atmoshere with a known vertical rofile of aerosol extinction coefficient. For the case of a vertically homogeneous atmoshere, the otical thickness of molecular scattering and gaseous absortion are calculated as described by Holben et al. [1998]. If the vertical rofile of the aerosol extinction coefficient is available, the radiative transfer calculations can be erformed for a multi-layered atmoshere. Therewith the rofiles of water vaor and ozone absortion together with climatological rofiles of temerature and ressure (for molecular scattering calculations) are required. However, we hardly can count uon having information on the vertical distribution of aerosol comlex refractive index, single scattering albedo and shae of the article size distribution. Therefore, these otical characteristics are assumed to be constant for the aerosol in the whole atmosheric column. We focus our rimary consideration on the simlest model of a homogeneous atmoshere. This is because information on aerosol vertical rofiles is not currently available for AERONET sun/sky radiometer locations. In addition, the effect of aerosol vertical variability on sky radiance ground measurements is often neglected, because it is rather modest in comarison with effects caused by aerosol size distribution variability. In addition, to minimize ossible retrieval uncertainty due to the assumtion of a homogeneous atmoshere, we concentrate our analysis on inverting sky radiances measured in the solar almucantar (Eq. (1b)). In observations with such a scheme (zenith angle of observations is equal to the solar zenith angle), all atmosheric layers are always viewed with similar geometry. Corresondingly, sky radiances in the solar almucantar are not sensitive to vertical variations of aerosol.