ISTINA - : Investigation of Sensitivity Tendencies and Inverse Numerical Algorithm advances in aerosol remote sensing

Transcription

1 STNA - : nvestigation of Sensitivity Tendencies and nverse Numerical Algorithm advances in aerosol remote sensing B. Torres, O. Dubovik, D. Fuertes, and P. Litvinov GRASP- SAS, LOA, Universite Lille-1, Villeneuve d'ascq, France; DEAS+ Task 3 Cal/val meeting, 5 and 6 July 216, ESRN/ESA, Frascati, taly

2 DEAS WP : Analysis of Numerical nverse Algorithm Advances in Atmospheric Remote Sensing; WP : Assessments of Sensitivity Tendencies in DEAS + Aerosol Remote Sensing Experimental data; WP : ntroduction to assimilation and inverse modelling; WP : ntroduction to assimilation and inverse modelling; DEAS+ Task 3 Cal/val meeting, 5 and 6 July 216, ESRN/ESA, Frascati, taly

3 DEAS WP : Analysis of Numerical nverse Algorithm Advances in Atmospheric Remote Sensing; WP : Assessments of Sensitivity Tendencies in DEAS + Aerosol Remote Sensing Experimental data; WP : ntroduction to assimilation and inverse modelling; WP : ntroduction to assimilation and inverse modelling; DEAS+ Task 3 Cal/val meeting, 5 and 6 July 216, ESRN/ESA, Frascati, taly

4 GRASP: Generalized Retrieval of Aerosol and Surface Properties MERS AERONET POLDER lidar AERONET Sentine - 4 laboratory single scattering GRASP open source MSR shape Surface reflectance BRDF BPDF

5 General structure of the algorithm GRASP NDEPENDENT MODULES!!! FORWARD MODEL Simulates observations f(a p ) for a given set of parameters a p Observation definition: Viewing geometry, spectral characteristics; coordinates, etc. a p f(a p ) nput : Observations f* f* NUMERCAL NVERSON Stat. optimized fitting of f* by f(a p ) under a priori constraints nversion settings: - description of error Δf*; - a priori constraints a p - final Retrieved parameters: a p describes optical properties of aerosol and surface

6 nverse Problem: Retrieval of particle size distribution from light scattering? Fa = f * measurements? P ij ( ) ( ) trans ( )

7 Which approach to use? - MML a = ( F T 1 C -1 1 F 1 ) -1 F T 1 C -1 * 1 f 1 a = F T C -1 1 ( f F + C ā ) -1 ( F T C -1 f f * + C 1 ā a * ) a = ( F T C -1 f F+gS T S) -1 ( F T C -1 f f * ) - LSM - «Optimal estimations», C. Rodgers - Kalman filter - Phillips-Tikhonov-Twomey - Steepest Desent Method - Twomey-Chahine a i p+1 = a i p f i * p f i a = ( F T F+g) -1 F T f * - Chahine - Tikhonov Regularization Assimilation, 4DVR SVD, gradient methods, etc.

8 - MML a = ( F T 1 C -1 1 F 1 ) -1 F T 1 C -1 * 1 f 1 - LSM Measurements f * -Fa 2 +g h(a * - a) 2 = min - Byaesian Approach A Priori Constrains a = F T C f -1 F + C ā 1 ( ) -1 F T C f -1 f * + C ā 1 a * ( ) Remote sensing assimilation, etc. - «Optimal estimations», C. Rodgers

9 Base idea of inversion f 1 * f 2 * = F 11 F 21 F 12 F 22 a 1 a 2 a = ( F) -1 f * f * F a square - parameters of size distribution f 1 * f 2 * f 3 * = F 11 F 12 F 21 F 22 F 31 F 32 a 1 a 2 a = ( F T F ) -1 F T f * f * F a rectangular

10 Base idea of constrained inversion det ( F T F ) ( F T F ) -1 -??? What to do?

11 Base idea of constrained inversion det ( F T F ) ( F T F ) -1 -??? 1 1 = - Diagonal matrix great for inversion!!!

12 Base idea of constrained inversion det ( F T F ) ( F T F ) -1 -??? ( F T F ) ( F T F +) ( ) > but F T F + det F T F + ( ) ¹ ( F T F)???

13 Base idea of constrained inversion det ( F T F ) ( F T F ) -1 -??? ( F T F ) ( F T F +g ) ( ) > and F T F +g det F T F +g ( )» ( F T F)

14 Base idea of constrained inversion det ( F T F ) ( F T F ) -1 -??? a = ( F T F ) -1 F T f * a = ( F T F +g ) -1 F T f * Solution is unique and almost correct!!!

15 Which approach to use? a = ( F T F+g) -1 F T f * - Tikhonov Regularization a = ( F T C -1 f F+gS T S) -1 ( F T C -1 f f * ) a = F T C -1 1 ( f F + C ā ) -1 ( F T C -1 f f * + C 1 ā a * ) - Phillips-Tikhonov-Twomey - «Optimal estimations», C. Rodgers a p+1 = a p - t p ( F T C -1 F +g p ) -1 F T C -1 Dlnf p - Levenberg - Marquardt - Kalman filter

16 CONTENT 1. ntroduction 2. Atmospheric remote sensing as an inverse problem 2.1 nteraction of radiation with the atmosphere Single scattering of electromagnetic radiation Multiple scattering effects 2.2 Main atmospheric components and their optical properties Atmospheric gases and molecular scattering Aerosols and clouds Underlying surfaces: land and water 2.3 Typical inverse problems of remote sensing Primary linear problems Essentially non-linear problems 3. Linear system of equations 3.2 Matrix inversion solutions 3.3 terative linear solutions 3.4 Solutions of non-linear systems 3.5 Methods of constrained inversions, basic concept of overcoming solution instability 4. Statistical estimation concept 4.1 Solving system of equation in the presence of noise in the data 4.2 Method of Maximum Likelihood 4.3 Optimality of the Method of Maximum Likelihood Cramer Rao nequality Fisher matrix Fisher definitions of information METHODS OF NUMERCAL NVERSON N ATMOSPHERC REMOTE SENSNG AND NVERSE MODELNG: AN NTRODUCTON

17 5. Least Squares Method 5.1 Gaussian Distribution of Noise (Normal Central Theorem) 5.2 Formulation of the LSM as a minimization procedure 5.3 Estimation of the solution covariance matrix 5.4 nformation content and its analysis 5.5 Estimations of linear functions of the retrieved parameters 6. Methods of constrained inversions 6.1 ll-posed problem definition 6.2 Strategy of constrained inversions General idea of using constraints for solving ill-posed problems Smoothness a priori constraints, equations by Phillips Tikhonov Twomey Solution constraints by means of using direct a priori estimates on unknown Kalman filter, Optimum estimations by Rogers, Bayesian statistics approach Methods for ensuring solution non-negativity and other diverse approaches 7. ncluding additional a priori information and Multi-Term Least Squares Method 7.1 Definition of Multi-Term LSM 7.2 Utilizing a priori estimates of unknowns 7.3 Utilizing a priori information about smoothness of the retrieved functions 7.4 Utilizing multiple a priori constraints simultaneously 7.5 Concept of statistically optimized Multi-Pixel nversion 8. Optimized solution of non-linear system of equations 8.1 Optimization of solution of non-linear system in presence of noise 8.2 Gauss Newton and Quasi-Newton iterations 8.3 Solution convergence, Levenberg Marquardt iterations 8.4 Steepest-decent and other gradient methods

18 9. Limitation of statistical estimation optimization of inverse solution 9.1 Utilization of a priori constraints on solution non-negativity: linear regularization methods, non-linear Chahine and Twomey Chahine inversion procedures 9.2 Application of a priori constraints on solution non-negativity in the framework of statistical optimization formalism 9.3 Accounting for effect of redundant observations 1. General recommendations, remote sensing applications, the GRASP algorithm 1.1 General recommendations for the inverse algorithm development 1.2 Satellite and ground-based atmospheric remote sensing 1.3 GRASP algorithm 1.4 Tropospheric aerosol remote sensing applications: Retrieval of aerosol properties from measurements of aerosol extinction and angular singe scattering Aerosol columnar properties retrieval from ground-based observations with sun-photometers Aerosol columnar properties and surface reflectance retrieval from satellite multi-angular and polarimetric observations Retrieval of aerosol vertical profiles from active lidar observations Enhanced retrieval of aerosol columnar and vertical properties from combined sun-photometer and lidar ground-based observations 11. ntroduction to assimilation and inverse modeling 11.1 Atmospheric chemistry transport modeling, equation of diffusion11.2 Assimilation and inverse modeling: gradient solutions using adjoint operators 11.3 Utilization of diverse a priori constraints in assimilation and inverse modeling Model forecast as an a priori estimate Smoothness constraints on temporal and spatial variability of retrieved geofields 11.4 Retrieval of aerosol emission sources from remote sensing by inverse modeling Retrieval of aerosol emission sources from MODS observations by GOCART inverse modeling Retrieval of aerosol emission sources from POLDER/PARASOL observations by GEOS-CHEM inverse modeling

19 Multi-term LSM Multi-Pixel Solution: W x = S x T S x ; W y = S y T S y ; W t = S t T S t ; a v a n a n a h a sph a Vc a brdf,1 g D,1 W 1 g D,2 W 2 g D,3 W 3 g D W = g D,4 W 4 g D,5 W 5 g D,6 W 6 g D,7 W 7 S T y S y = d11 d12 d13 d21 d22 d23 d31 d32 d33 d11 d12 d13 d21 d22 d23 d31 d32 d33 d11 d12 d13 d21 d22 d23 d31 d32 d33 ; a brdf,2 a brdf,3 a bpdf 43 parameters S T y S y = d11 d12 d13 d21 d22 d23 d31 d32 d33 d11 d12 d13 d21 d22 d23 d31 d32 d33 d11 d12 d13 d21 d22 d23 d31 d32 d33 ; S T t S t = d11 d11 d11 d21 d21 d21 d31 d31 d31 d12 d12 d12 d22 d22 d22 d32 d32 d32 d13 d13 d13 d23 d23 d23 d33 d33 d33.

20 Which approach to use? - MML a = ( F T 1 C -1 1 F 1 ) -1 F T 1 C -1 * 1 f 1 a = F T C -1 1 ( f F + C ā ) -1 ( F T C -1 f f * + C 1 ā a * ) a = ( F T C -1 f F+gS T S) -1 ( F T C -1 f f * ) - LSM - «Optimal estimations», C. Rodgers - Kalman filter - Phillips-Tikhonov-Twomey - Steepest Desent Method - Twomey-Chahine a i p+1 = a i p f i * p f i a = ( F T F+g) -1 F T f * - Chanine f * -Fa 2 +g h(a * - a) 2 = min - Tikhonov Regularization - Byaesian Approach

21 noise system is redundant noise can be accounted Maximum Likelihood Principle: Gaussian noise assumption: P (f * - f(a)) = max PDF(Likelihood function) P ~ exp å i (f * i -f i (a)) 2 s 2 = max Least Squares Principle å i (f i * -f i ( a )) 2 = min LSM gives optimum solution: LSM - Optimality s 2 ai = ( a i - a real ) 2 -Smallest!!!

22 1. if P (...) Gaussian, MML = MLS Y( a) = 1 2 (f ( a ) - f * ) T C -1 (f ( a) - f * ) = min ÑY a ( ) = Y ( a ) a i =, (i = 1,..,N a ) f Normal => a - Normal 2. Optimality of LSM: g - a characteristic linearly dependent on a (i.e. g = g T a, g is a vector of coefficients) <(Δg) 2 > = < g T Δã(g T Δã) T > = = g T Δã(Δã) T g = g T Δ ã g g T Δ LSM g - Cramer-Rao inequality

23 1. Fisher nformation: nformation Quantity: Gauss Probability Function ( ( )) = - 2 ln P( C ˆ ) h P C ˆ ò C ˆ 2 P( x ˆ )dˆ x = 1 DC ˆ 2 min P(C) <DC 2 > 1/2 <C> 2. Shannon nformation: ( ( )) = -log 2 P( C ˆ ) h P C ˆ ( ) ò P( C ˆ )dˆ x N bits N bits - number of bits (binary digits) needed to represent the number of distinct estimates that could have be obtained N symbols ~ 2 N bits