Inversion basics y = Kx + ε x ˆ = (K T K) 1 K T y Erkki Kyrölä Finnish Meteorological Institute Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 1
Contents 1. Introduction: Measurements, models, inversion 2. Classical inversion: LSQ 3. Bayes theory 4. Monte Carlo Markov Chain 5. References Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 2
Introduction Forward model target instrument data Inverse model inverse model = generalized inverse of the approximate forward model Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 3
Measurements Measurements either count or scale Remote and in situ measurements Direct and indirect measurements Classical or quantum measurements Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 4
Atmospheric remote measurements Atmosphere is continuously changing in time and space. No repeated measurements of the same quantity. Radiation field measurements are direct, all other measurements are indirect Measurements probe large atmospheric volume. Large averaging. Validation by in situ measurements is difficult. Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 5
Forward models The true nature G(x,z) + ε z=all other pertinent variables G known (x,z known = z fix ) + ε The best forward model available. Uninteresting variables fixed. G app (x,z known = z fix ) + ε Model used in simulation G inv (x,z known = z fix ) + ε Model used for inversion Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 6
Inverse problem y = K(x) + ε x = Parameters to be determined from measurements K = Forward model y = Measurements (data) ε = Noise Find best x when y is measured. Define best first. Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 7
Least squares solution (LSQ) Minimize S = (y p K p (x)) 2 Distance between data and the model prediction If we have a linear problem y = Kx we get simply ˆ x = (K T K) 1 K T y Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 8
A systematic basis for inversion theory is given by the Bayesian approach Model parameters are random variables Probability distribution of model parameters is retrieved A priori information is needed. This has led to many controversies about the Bayesian approach. Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 9
Bayesian method P(x y)p(y) = P(y x)p(x) P(x y) = P(y x)p(x) P(y) = P(y x)p(x) P(y x)p(x)dx P(x y) = Conditional probability distribution for model parameters x given data y P(x) = A priori probability for model parameters P(y x) = Conditional pdf for data y when x given. Also called as likelihood. P(y) = The normalization. It can usually be ignored. Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 10
Various point estimators for x Mean Minimum variance Maximum probability Mean = Minimum variance estimator Mean = Maximum probability if pdf symmetric Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 11
P(x y) = P(y x)p(x) Whole distribution Point estimation MCMC method max of max of Maximum likelihood MAP Gaussian errors LSQ LM method Linear model Closed solution Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 12
Example: Linear problem y = K true x true + ε Kx + ε Assume Gaussian noise and prior distribution: P(y x) = const e 1 2 (y Kx)T C 1 D (y Kx) const = ((2π) N det(c D )) 1 2 P(x) = const e 1 2 (x x a ) T C 1 a (x x a ) Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 13
Maximum a posteriori ˆ x = (K T C D 1 K + C a 1 ) 1 (K T C D 1 y + C a 1 x a ) Interpretation: weighted mean between data and a priori Posterior distribution Model covariance P(x y) = const e 1 2 (x ˆ x )T C x 1 (x ˆ x ) C x = (K T C D 1 K + C a 1 ) 1 The solution can be written also as ˆ x = x a + C a K T (KC a K T + C D 1 ) 1 (y Kx a ) This can be viewed as an update to a priori Assimilation Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 14
Properties of linear solution ˆ x = (K T C D 1 K + C a 1 ) 1 (K T C D 1 y + C a 1 x a ) = C x K T C D 1 K true x true + C x K T C D 1 ε + C x C a 1 x a Averaging kernel A = C x K T C D 1 K true ˆ x x = (A I)(x x a ) + C x K T C D 1 ε smoothing error random error Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 15
Quality of retrieval Bias ˆ x = C x K T C D 1 K true x true + C x K T C D 1 ε + C x C a 1 x a linear If C a and K = K true no bias The bias in retrieval can usually be checked only by computer simulations. Intercomparisons of real measurements can also be used to detect bias. Residual Investigate the difference d = y obs y mod = y obs Kˆ x Chi2 χ 2 = 1 N m ( (y Kˆ T obs x ) 1 CD (y obs Kˆ x ) + ( x ˆ x a ) T C 1 a ( x ˆ x a )) Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 16
Special cases 1. No apriori C a and K = K true A=1 and we obtain WLSQ 2. Connection to elementary data analysis. Take case 1 and only one parameter. Then the MAP estimator is mean and C x = σ 2 N i.e. the standard error of the mean. Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 17
Possible solutions for linear equations Number of data = N, number of unknowns = M 1. N = M Exact inversion possible 2. N > M Overdetermined problem. Additional information may be used to constrain the solution. 3. N < M Underdetermined problem. Needs additional information or constraints Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 18
Non-linear problems Gaussian errors and linear model give a quadratic optimisation problem. This leads to linear (in data) estimates. All other cases lead to non-linear problems. Gaussian errors and non-linear models can be approached by the Levenberg-Marquardt algorithm Sometimes a model can also be linearised Sometimes we can transform the problem to a new linear problem. Note: Error statistics will also change With very noisy data and/or complicated models several maxima of pdf can exist. Global methods, like simulated annealing, may help but it is better to try MCMC. Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 19
Ultimate estimators: Markov chain Monte Carlo Twin peaks drama Mr. Markov: Hold your horses Blind Mr. Levenberg: That s it! Top guy: Yes! Mean guy: <Sorry but...> Flatness dullness Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 20
Markov chain Monte Carlo Estimators from MCMC 1 N <x i >= Σ z t i N t Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 21
MCMC examples (GOMOS) Bright star Weak star Marginal posterior distributions at 30 km for different gases Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 22
A priori information Discrete grid: Assume that profile has only a finite number of free parameters Smoothness: Tikhonov constraint A priori profile Positivity constraint or similar Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 23
Literature and a reference Tarantola: Inverse problem theory, Methods for data fitting and model parameter estimation, Elsevier, 1987 Rodgers: Inverse Methods for Atmospheric Sounding: Theory and Practice, World Scientific, 2000 Menke: Geophysical data analysis: discrete inverse theory, Academic Press, 1984 Tamminen and Kyrölä, JGR, 106, 14377, 2001 Tamminen: Ph.D. thesis, FMI contributions 47, 2004. Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 24