Non-linear least-squares inversion with data-driven
|
|
- Jessie Wheeler
- 5 years ago
- Views:
Transcription
1 Geophys. J. Int. (2000) 142, Non-linear least-squares inversion with data-driven Bayesian regularization Tor Erik Rabben and Bjørn Ursin Department of Petroleum Engineering and Applied Geophysics, Norwegian University of Science and Technology, 7491 Trondheim, Norway SUMMARY The non-linear inverse problem is formulated in a Bayesian framework. The multivariate normal distribution is assumed in both the noise and prior distributions. However, only the structures of the covariance matrices have to be specified, estimation of the variance levels is included in the inversion procedure. The maximum a posteriori approximation is derived, and the final result is a weighted least-squares inversion algorithm with the ratio between the variance levels as an adaptive, data-driven regularization factor, hence the name Bayesian regularization. The algorithm is tested on inversion of seismic reflection amplitudes and compared with the L-curve approach for choosing the regularization parameter. The Bayesian regularization results in a better regularization value in only a fraction of the time. Key words: Inverse theory Probability distributions Acoustic properties 1 INTRODUCTION Inverse problems, in geophysics and other disciplines, are often ill-posed and non-unique. Ill-posed means that small changes in the measurements can cause large changes in the solution. In a situation where the measured data is contaminated with noise this would be a
2 2 T.E. Rabben and B. Ursin serious problem. The non-uniqueness problem can arise if the forward model is too simple or if the coverage of the measurements is insufficient. In any of these situations regularization is necessary. Regularizing the inverse problem means finding a physically meaningful stable solution (Tenorio, 2001). Historically, the most famous inversion method is least squares (also known as regression analysis), see e.g. Tarantola (1987) or Lines & Treitel (1984), and the goal is to find the set of parameters that minimizes the square of the misfit. The straight-forward solution to solve it is to find the gradient and set it equal to zero. This results in the normal equations. In statistical nomenclature this method is equal to assuming a Gaussian distribution of the error and maximizing the likelihood (ML estimate). In case of a linear forward model the normal equations will have a simple, analytic form. However, if the forward model is nonlinear the least-squares problem is solved by Taylor expanding the gradient and creating an iterative solution algorithm which (hopefully) converges to a global minimum. A very common regularization technique for least-squares problems is Tikhonov regularization (Tikhonov & Arsenin, 1977; Tenorio, 2001) (also known as ridge regression) which seeks to minimize both the residual and a property of the solution alone. Typical properties of the solution can be its derivative to ensure smoothness or the distance from an a priori expected solution. Statistically, this can be seen as allowing a bias in order to reduce the variance in the solution (Golub et al., 1979). The equivalence to Tikhonov regularization in the statistical nomenclature is to maximize the posterior distribution (MAP estimate) when assuming Gaussian error and prior distributions. The challenge in the Tikhonov regularization is the trade-off between minimizing the residuals or the parameter norms a trade-off between trusting measurements or a priori information. Several methods exist and the trade-off parameters is commonly denoted λ 2, a notation which will be adapted here. An intuitive solution is to parametrically display the residual and parameter norms as a function of different λ 2 -values. Typically this will give a figure with the shape of an L, and therefore the name L-curve (Lawson & Hanson, 1974; Hansen, 1992). The perfect λ 2 is in the corner point where the curvature is largest since this corresponds to a good
3 Non-linear least-squares inversion with data-driven Bayesian regularization 3 trade-off between minimization of the two norms. The singular value decomposition (SVD) (Golub & Van Loan, 1996) is a way to form a generalized inverse operator to solve the linear inverse problem. This procedure can easily be extended to include regularization by either truncating some singular values (Hansen, 1987) or applying filter factors (Lines & Treitel, 1984; Aster et al., 2004). The latter can be shown to be mathematically equivalent to solving the normal equations (Lines & Treitel, 1984). In both situations the basic concept is to remove amplification of eigenvectors corresponding to singular values below the noise level. Other approaches to discriminate between measurements and prior information are the generalized cross-validation (Golub et al., 1979), the chi-square test (Snedecor & Cochran, 1989; Aster et al., 2004) or the maximum entropy condition (Aster et al., 2004), coming from different statistical consideration. A common feature for all the methods mentioned here, to find the trade-off parameter λ 2, is that they are computer intensive in case of a large problem sizes. They all require the inversion performed for a range of possible values before finding the best value. A better solution would be to have the measurements itself determine λ 2 in an automatic, robust and effective procedure. In this paper we present a method which is adaptive, data-driven, based on a sound theoretical background, and more efficient than its competitors. We start by defining the Bayesian model where we follow closely the work of Rabben et al. (2008). The noise and the prior are multivariate Gaussian distributions with given structure of the covariance matrices. The scaling factors of these covariance matrices are included as stochastic parameters in the inversion procedure (Buland & Omre, 2003). Based on this model we iteratively compute the MAP estimate of the model parameters. The trade-off parameter λ 2 is the ratio between the two scale parameters of the covariance matrices. It is updated in each iteration, and the new value depends on the data misfit and the model misfit from the prior. The final result is a least-squares algorithm with adaptive and data-driven regularization. Hansen and O Leary (1993) did a comparison of several of the methods mentioned above, and concluded that L-curve was more robust than the main competitor, generalized cross-
4 4 T.E. Rabben and B. Ursin validation. We will therefore use this method as a comparison for the new inversion algorithm. Our numerical example will be inversion of seismic reflection amplitudes, both only PP inversion and joint PP and PS inversion, following the work of Rabben et al. (2008). 2 BAYESIAN MODEL In order to formulate our inversion procedure we start by introducing the non-linear forward problem d = f(m) + e. (1) Here, d is a vector containing a set of measurements, m is a vector of model parameters, f is a non-linear function, and e is a general noise term. The inversion problem is, given d and f, to find m. The corresponding Bayesian formulation of the inverse problem is, via Bayes rule, given by π(m d) π(d m) π(m), (2) where π( ) represents any probability distribution. In Bayesian inversion we are not only searching for one optimal m, but the full statistical distribution of m given the measured data d. This is the left hand side of (2) and is known as the posterior distribution. It is proportional to the product of π(d m) (the likelihood model, analog to the forward model) and π(m) (the prior model representation). To find the posterior model we need to assign probability distributions to the noise e and to the model parameters m. Common and convenient choices are the multivariate normal distribution (defined in Appendix A) π(e) = N (e; 0, Σ e ) (3) π(m) = N (m; µ m, Σ m ). (4) Under the assumption of a known, deterministic forward model f we easily find the likelihood to be π(d m) = N (d; f(m), Σ e ) using (1) and (3). In this formulation the choice of covariance matrices is very important since it greatly will influence the optimal solution. Our approach
5 Non-linear least-squares inversion with data-driven Bayesian regularization 5 is to define the covariance matrices as (Buland & Omre, 2003) Σ e = σ 2 es e Σ m = σ 2 ms m, (5) (6) where S e and S m are known correlation matrices, and then include the estimation of the scalar random variables σe 2 and σm 2 as a part of the inversion procedure. As a consequence, we need prior distributions on σe 2 and σm, 2 and we choose the inverse gamma distributions π(σe) 2 = IG(σe; 2 α e, β e ) (7) π(σ 2 m) = IG(σ 2 m; α m, β m ), (8) where α e, β e, α m and β m (known as hyperparameters (Robert, 2007)) are scalar constants, see Appendix A for further definitions. The inverse gamma distribution is flexible and defined for positive values and can thereby be adapted to varying prior knowledge. Moreover, it makes the mathematical treatment of the resulting posterior easier. It is a convenient (although maybe not optimal) choice as the normal distribution is for the medium parameters and the measurement noise. With these two new model parameters, our Bayesian inverse problem now becomes to estimate the joint posterior distribution π(m, σe, 2 σm d) 2 π(d m, σe, 2 σm) 2 π(m, σe, 2 σm). 2 (9) This equation will constitute the modelling basis for our inversion algorithm. 3 MAXIMUM A POSTERIORI SOLUTION There are several ways to assess posterior information. Rabben et al. (2008) showed how to sample the posterior (9) and hence assessing the full posterior distribution through a MCMC Metropolis-Hastings sampling approach. However, this is complicated and computationally expensive since the forward model is non-linear. We will therefore search for only the most likely solution, arg max π(m, σe, 2 σm d), 2 m,σe,σ 2 m 2 (10)
6 6 T.E. Rabben and B. Ursin also known as the maximum a posteriori (MAP) solution or posterior mode. Further, instead of trying to assess the joint posterior π(m, σe, 2 σm d) 2 we will update each parameter sequentially (known as Gibbs steps) in an iterative algorithm. We therefore need the three posterior expressions (one parameter conditioned on the two others) π(m d, σe, 2 σm) 2 π(d m, σe) 2 π(m σm) 2 (11) π(σe d, 2 m) π(d m, σe) 2 π(σe) 2 (12) π(σm m) 2 π(m σm) 2 π(σm) 2 (13) or π(m d, σe, 2 σm) 2 N (d; f(m), σes 2 e ) N (m; µ m, σms 2 m ) (14) π(σe d, 2 m) N (d; f(m), σes 2 e ) IG(σe; 2 α e, β e ) (15) π(σm m) 2 N (m; µ m, σms 2 m ) IG(σm; 2 α m, β m ). (16) For the posterior (14) we write { π(m d, σe, 2 σm) 2 exp 1 exp 2σ 2 e [d f(m)] T S 1 e } [d f(m)] } { 1 [m µ 2σm 2 m ] T S 1 m [m µ m ] Maximizing the posterior is equal to minimizing the expression. (17) ψ = d f(m) 2 S 1 e + σ2 e m µ σm 2 m 2, (18) S 1 m which is a non-linear weighted least-squares problem. The minimum is reached when the gradient of ψ is zero, and by expanding it in a Taylor series we find the iterative solution algorithm m k+1 = m k H 1 k g k = m k ( J T S 1 e ) J + λ 2 S 1 1 ( m λ 2 S 1 m m µ J T S 1 e d ), (19) where J = f/ m T mk, d = d f(m k ), m µ = m k µ m and λ 2 = σe/σ 2 m. 2 By omitting the first term in the gradient and assuming S e = S m = I it reduces to the famous Levenberg- Marquardt algorithm (Levenberg, 1944; Marquardt, 1963). The posterior (15) and (16) can, by using the definition of the normal and inverse gamma
7 Non-linear least-squares inversion with data-driven Bayesian regularization 7 distribution (Appendix A), be written π(σ 2 e d, m) IG π(σ 2 m m) IG ( σ 2 e; α e n e, β e d 2 S 1 e ( σ 2 m; α m n m, β m m µ 2 S 1 m ) (20) ), (21) where n e and n m are the number of elements in d and m. In other words, the posterior distributions of σ 2 e and σ 2 m are still inverse gamma distributed, only with modified parameters. The MAP solution of an inverse gamma distribution IG(σ 2 ; α, β ) is σ 2 = β /(1 + α ) (see (A5)), and with this result we find the MAP of (15) and (16) to be σe 2 = β e d 2 S 1 e 1 + α e + 1n 2 e σm 2 = β m + 1 m 2 µ 2 S 1 m 1 + α m + 1n. 2 m (22) (23) To speed up the algorithm we will perform only one update of m using (19) before updating the MAP of σ 2 e and σ 2 m. The final expression for λ 2 reads λ 2 k+1 = σ2 e,k+1 σ 2 m,k+1 = β e d 2 S 1 e β m + 1 m 2 µ 2 S 1 m 1 + α m + 1n 2 m 1 + α e + 1n. (24) 2 e Eqs (19) and (24) constitute our inversion algorithm. In addition to the parameters needed in the weighted least-squares algorithm, we also have to assign the hyperparameters α s and β s in the two inverse gamma distributions. Since σ 2 e and σ 2 m appears relative to each other, it will be convenient to scale the correlation matrices S e and S m in (5) and (6) such that the determinant of both equals one. When assigning values to the hyperparameters we can start by looking at eqs (7) and (8) which are distributions of the a priori knowledge about σ 2 e and σ 2 m. The safe choice would be a reasonable mean and a large variance such that the data itself can decide. However, it is not obvious how to choose this just by looking at eqs (A3) and (A4). We will therefor use (24) to guide us. Firstly, we will assume the dimensionality to be large such that n m 1 + α m and n d 1 + α e and set α m = α e = 0, resulting in λ 2 k+1 = n m n e β e d 2 1 Se β m + 1 m 2 µ 2 S 1 m (25)
8 8 T.E. Rabben and B. Ursin From here, if we assume both β s to be zero it becomes λ 2 k+1 = n m n e d 2 1 Se m µ 2 S 1 m, which is a purely data-driven damping. However, one should be careful here since both of the terms d 2 S 1 e and m µ 2 S 1 m (26) can be zero. If either one of the two norms approaches zero, it will lead to λ 2 approaching the undesirable value of zero or infinite. In this setting, the β s act as stabilizing terms by defining a priori lower bounds on the two norms. Lastly, the total opposite choice for the β s is β e d 2 S 1 e which leads to λ 2 k+1 n m n e β e β m, and β m m µ 2 S 1 m (27) a solution driven totally by the hyperparameters. Needless to say, this choice has to be avoided in order for our method to work. 4 NUMERICAL EXAMPLE: INVERSION OF SEISMIC REFLECTION AMPLITUDES To demonstrate our algorithm we apply it to the numerical example used in Rabben et al. (2008). It is a synthetic problem where the true model contains large contrasts, and are chosen to enhance non-linear effects. Our method for comparison is the L-curve which is a logarithmic plot ( d f(m) 2 S 1 e, m µ m 2 ) for a range of regularization values λ 2. S 1 m The point on the graph which has the larges curvature, the corner point, is considered the correct regularization value. The forward model is isotropic and the medium parameters m are defined over a lattice with the 3 elastic moduli in each point, resulting in n m = For the measurements we have two cases: only PP reflections and both PP and PS reflections. In both cases the exact Zoeppritz equations are used to generate the reflection amplitudes before correlated noise is added, this noise is generated using the correct correlation matrix in (3). For PP inversion we have 4 angles, and for joint PP and PS a total 7 angles, giving n e = and n e = respectively.
9 Non-linear least-squares inversion with data-driven Bayesian regularization 9 When inverting we use only the quadratic approximations to the Zoeppritz equation (Stovas & Ursin, 2003; Rabben et al., 2008) as our forward model f. The a priori expected mean, µ m, is half of the true model. For the correlation matrices S e and S m we have included spatial correlations, but no correlations within medium parameters or reflection angles. In the first case, inversion of only PP reflection amplitudes, the obvious first choice for β e and β m is zero for both, eq. (26). However, this solution converges to the prior because of our initial λ 2 0. This can easily be fixed by assigning a relative small value to β m. We have chosen β m = 5, and in Fig. A1 our Bayesian regularization is plotted together with the L-curve. The black, dotted line in the figure is the value of β m. We see that the classical L-shape is not present, this is because of the spatial correlation which prevents the solution from growing very large. However, our solution converges to a very plausible value, with a final value λ 2 = In Fig. A2 we see a zoom of the left end of the L-curve. Although (β e = 0, β m = 5) is our final choice for the inversion, we will also see how two other choices will influence the solution. In a situation where we have less confidence in the measurements we will assign a non-zero value to β e also. In Fig. A3 we have used the values (β e = 1, β m = 5), and we see that the algorithm converges to a solution within the rectangle defined by β e and β m. Based on this we can interpret β e and β m as a priori noise-level estimates. However, in this case the algorithm is still strongly data-driven. If we compare with the situation in Fig. A4 (β e = 10, β m = 100), we see that there the algorithm converges in a very few iterations, and the final λ 2 -value will be very close to the estimate from eq. (27). From purely geometrical considerations we can conclude that if the point (β e, β m ) is above or close to the L-curve, our algorithm will not be data-driven but controlled mainly by prior information. We will now continue with the solution from Fig. A1. Since we know the truth we can access the true bias, this is displayed in Fig. A5, plotted together with true bias in the L- curve calculations. Here we see that the optimal regularization value is λ , somewhat higher than our algorithm but far from the L-curve solution. The convergence of our algorithm is also interesting. Fig. A6 shows the update m k
10 10 T.E. Rabben and B. Ursin m k 1 and the damping factor λ 2 in each iteration. The medium parameters m does not converge until the damping factor has. We also see that our algorithm converges after 17 iterations. As a comparison, the L-curve in Fig. A1 which consists of 49 different values of λ 2, adds up to a total of 215 iterations. In Table A1 we have summarized this together with the total number of Conjugate Gradients (Saad, 2003) iterations in order to solve the matrix inversion in (19). We clearly see that our method is superior also when it comes to computational time, not only to find the regularization parameter. To conclude the PP example we compare the bias in our MAP estimate with the bias in the mean (Rabben et al., 2008) in Fig. A7. For the contrast in P-wave impedance the results are the same, for S-wave impedance the mean is slightly better, while for contrasts in density our result is best. In other words, the two methods yield very similar results. Our second example, joint inversion of PP and PS reflections, yields very similar results to the previous example. This time we plot both our method and the L-curve together in Figs A8 and A9, and again we see that the L-curve breaks down while our Bayesian regularization converges to a very plausible value. In Fig. A10 we use the known truth to calculate the true bias in the two approaches. Also this time the damping factor is slightly too low, but far better than the L-curve. However, if we compare with Fig. A5 we see that it performs better than for PP inversion. The bias in the inversion is also lower in the case of a too low damping factor. This is due to the forward model, joint inversion is less ill-posed than PP inversion. When it comes to convergence we see that the only major difference in Fig. A11 compared to Fig. A6 is the number of iteration. In our two examples joint inversion converges in 13 iterations while PP inversion requires 17, again due to a not so ill-posed problem. Table A2 confirms this by looking at both number of least-squares and conjugate gradients iterations, and comparing with Table A1. More important, it also shows how computationally superior our algorithm is to the L-curve approach. To conclude our numerical examples we compare the bias in our MAP estimate with the bias in the mean (Rabben et al., 2008) in Fig.
11 Non-linear least-squares inversion with data-driven Bayesian regularization 11 A12. Again we see that the two methods yield very similar results with contrasts in density impedance slightly better and the opposite for the S-wave impedance. 5 DISCUSSION We have used the Bayesian model from Rabben et al. (2008), but have to distinctly different approaches on how to assess posterior information. While they used a sampling methodology, we have used an optimization strategy. The advantage with sampling of the posterior is the ability to quantify uncertainties together with expected value and most likely solution, compared to our method which only finds the most likely solution. In this setting, their method is definitely preferred. However, it does not come for free. The sampling method is very computer intensive. In the example used, our method converges in minutes while the sampling algorithm may require more than one day to generate the large number of samples needed. Our method has its strength for non-linear problems. Since we have to alternate between updating the medium parameters m and the dampening factor λ 2 until convergence, the method requires more than one iteration - even for linear problems. Therefore, for linear problems which converges in one iteration, the workload of computing the L-curve would be about the same as our Bayesian regularization approach when the number of λ 2 values in the L-curve equals the number of iterations needed in our method. However, for non-linear problems each point on the L-curve will be as costly as our method. An other important point to make is the use of covariance matrices S e and S m. With a wrong assumption here, the misfit function (18) will be sensitive to changes in only a subset of the measurements and model parameters. This will clearly be a violation of our assumptions, and our method (specially in the case of eq. (26)) may not perform as expected. As a minimum, one should include correlation matrices with relative variances on the diagonal.
12 12 T.E. Rabben and B. Ursin 6 CONCLUSION We have formulated the non-linear inverse problem in a Bayesian setting where we, in addition to the medium parameters, have included estimation of the prior and noise level variance. Searching for the most likely solution, the maximum a posteriori solution, results in a weighted non-linear least-squares algorithm with an adaptive, data-driven regularization factor. The origin from a Bayesian formulation is reflected in the term Bayesian regularization. To test the algorithm we have compared it with the L-curve approach by applying both approaches to the problem non-linear inversion of reflection amplitudes. From the L-curve we see that our method converges to a very plausible regularization factor. We have also demonstrated how different strategies for choosing the hyperparameters will influence the inversion result. The parameters β e and β m are a priori lower bounds of the data and prior misfit. In a situation where the point (β e = 0, β m = 5) lies far below the L-curve, the algorithm will be purely data-driven, while when the point is close to or above the curve, the solution will be more and more constrained by the hyperparameters. For non-linear problems the new methods is computationally more efficient than the L-curve approach. 7 ACKNOWLEDGMENTS We wish to thank BP, Schlumberger, StatoilHydro and The Research Council of Norway for their support through the Uncertainty in Reservoir Evaluation (URE) project. Bjørn Ursin thanks StatoilHydro via the VISTA project for financial support. We thank Colin G. Farquharson for constructive comments in the review process. REFERENCES Aster, R., Borchers, B., & Thurber, C., Parameter Estimation and Inverse Problems, Elsevier Academic Press, Burlington. Buland, A. & Omre, H., Joint AVO inversion, wavelet estimation and noise-level estimation using a spatially coupled hierarchical Bayesian model, Geophysical Prospecting, 51,
13 Non-linear least-squares inversion with data-driven Bayesian regularization 13 Golub, G. H. & Van Loan, C. F., Matrix Computations, The Johns Hopkins University Press, Baltimore. Golub, G. H., Heath, M., & Wahba, G., Generalized cross-validation as a method for choosing a good ridge parameter, Technometrics, 21, Hansen, P. C., The truncated SVD as a method for regularization, BIT, 27, Hansen, P. C., Analysis of decrete ill-posed problems by means of the L-curve, SIAM Review, 34, Hansen, P. C. & O Leary, D. P., The use of the L-curve in the regularization of discrete ill-posed problems, SIAM Journal on Scientific Computing, 14, Lawson, C. L. & Hanson, R. J., Solving Least Squares Problems, Prentice-Hall, Englewood Cliffs. Levenberg, K., A method for the solution of certain non-linear problems in least squares, Quarterly of Applied Mathematics, 2, Lines, L. R. & Treitel, S., A review of least-squares inversion and its application to geophysical problems, Geophysical Prospecting, 32, Marquardt, D. W., An algorithm for least-squares estimation of nonlinear parameters, SIAM Journal on Applied Mathematics, 11, Rabben, T. E., Tjelmeland, H., & Ursin, B., Non-linear Bayesian joint inversion of seismic reflection coefficients, Geophysical Journal International, Accepted, available at OnlineEarly. Robert, C., The Bayesian Choice, Springer, New York. Saad, Y., Iterative Methods for Sparse Linear Systems, SIAM, Philadelphia. Snedecor, G. W. & Cochran, W. G., Statistical Methods, Iowa State University Press, Ames. Stovas, A. & Ursin, B., Reflection and transmission responses of layered transversely isotropic viscoelastic media, Geophysical Prospecting, 51, Tarantola, A., Inverse Problem Theory, Elsevier, Amsterdam. Tenorio, L., Statistical regularization of inverse problems, SIAM Review, 43, Tikhonov, A. N. & Arsenin, V. Y., Solutions of Ill-Posed Problems, John Wiley & Sons, New York. APPENDIX A: STATISTICAL DISTRIBUTIONS A multivariate Gaussian variable x with expectation vector µ and covariance matrix Σ has the probability function { 1 N (x; µ, Σ) = exp 1 } (2π) n/2 Σ 1/2 2 (x µ)t Σ 1 (x µ), (A1)
14 14 T.E. Rabben and B. Ursin where n is the dimension of x. The inverse gamma probability function is ( ) α+1 IG(x; α, β) = βα 1 e β/x (A2) Γ(α) x where x 0, α > 0 and β > 0. Its mean, variance and mode are E IG(x; α, β) = β α 1 β 2 Var IG(x; α, β) = (α 1) 2 (α 2) max IG(x; α, β) = β x α + 1. Given the prior distribution σ 2 (A3) (A4) (A5) IG(α, β) and measurements x N (µ, σ 2 S), the posterior distribution of σ 2 is π(σ 2 x) π(x σ 2 ) π(σ 2 ) where = N (µ, σ 2 S) IG(α, β) { 1 exp 1 } (σ 2 ) n/2 S 1/2 2 σ 2 (x µ) T S 1 (x µ) ( ) α+1 1 exp { βσ } 2 σ 2 ( ) α+1+n/2 1 exp { σ 2 (β + s 2 n2 } ) IG σ 2 ( σ 2 α n + 2, β + n ) s2 2 (A6) s 2 = 1 n (x µ)t S 1 (x µ) (A7) and n is the dimension of x. Clearly, the posterior is also inverse gamma but with modified parameters. This paper has been produced using the Blackwell Publishing GJI L A TEX2e class file.
15 Non-linear least-squares inversion with data-driven Bayesian regularization 15 LIST OF FIGURES A1 Bayesian regularization for PP inversion and comparison with the L-curve. The red x-marks show how our algorithm converges for (β e = 0, β m = 5), and the thick x-marks is the final iteration, with λ 2 = The L-curve is made using λ 2 -values from to with constant logarithmic increments. The smallest values of λ 2 is to the left on the curve. The black, dotted line in the figure is the value of β m, and the black square indicates the zoomed area in Fig. A2. A2 Zoom of the black square in Fig. A1. A3 Bayesian regularization for PP inversion and comparison with the L-curve. The red x-marks show how our algorithm converges for (β e = 1, β m = 5), and the thick x-marks are the final iteration. The black circle and dotted lines are the values of β e and β m. A4 Bayesian regularization for PP inversion and comparison with the L-curve. The red x-marks show how our algorithm converges for (β e = 10, β m = 100), and the thick x-marks are the final iteration. The black circle and dotted lines are the values of β e and β m. A5 True bias in the Bayesian regularization and in the L-curve calculations, both for PP inversion. A6 Convergence of the Bayesian regularization algorithm for inversion of PP reflection amplitudes. The black, dotted line is the convergence criterion for the update. A7 Absolute value of bias in the medium parameters m for PP inversion. The left column is our MAP estimate while the right column (Mean) is reproduced from Rabben et al. (2008). Some high values in the contrast in S-wave impedance are clipped in order to visually enhance the differences. A8 Bayesian regularization for joint PP and PS inversion and comparison with the L-curve. For the L-curve we have again used λ 2 values from to The red x-marks show how our algorithm converges for (β e = 0, β m = 5), and the thick x-mark to the left is the final iteration, with λ 2 = The black, dotted line in the figure is the value of β m, and the black square indicates the zoomed area in Fig. A9. A9 Zoom of the black square in Fig. A8. A10 True bias in the Bayesian regularization and in the L-curve calculations, both for joint PP and PS inversion. A11 Convergence of the Bayesian regularization algorithm for inversion of joint PP and PS reflection amplitudes. The black, dotted line is the convergence criterion for the update. A12 Absolute value of bias in the medium parameters m for joint PP and PS inversion. The left column is our MAP estimate while the right column (Mean) is reproduced from Rabben et al. (2008). A few high values in the contrast in S-wave impedance are clipped in order to visually enhance the differences.
16 16 T.E. Rabben and B. Ursin 10 3 Bayes. reg. L-curve 10 2 m µ m 2 S 1 m d f(m) 2 S 1 e Figure A1. Bayesian regularization for PP inversion and comparison with the L-curve. The red x-marks show how our algorithm converges for (β e = 0, β m = 5), and the thick x-marks is the final iteration, with λ 2 = The L-curve is made using λ 2 -values from to with constant logarithmic increments. The smallest values of λ 2 is to the left on the curve. The black, dotted line in the figure is the value of β m, and the black square indicates the zoomed area in Fig. A2.
17 Non-linear least-squares inversion with data-driven Bayesian regularization Bayes. reg. L-curve m µ m 2 S 1 m d f(m) 2 S 1 e Figure A2. Zoom of the black square in Fig. A1.
18 18 T.E. Rabben and B. Ursin Bayes. reg. L-curve (β e, β m ) m µ m 2 S 1 m d f(m) 2 S 1 e Figure A3. Bayesian regularization for PP inversion and comparison with the L-curve. The red x-marks show how our algorithm converges for (β e = 1, β m = 5), and the thick x-marks are the final iteration. The black circle and dotted lines are the values of β e and β m.
19 Non-linear least-squares inversion with data-driven Bayesian regularization m µ m 2 S 1 m Bayes. reg. L-curve (β e, β m ) d f(m) 2 S 1 e Figure A4. Bayesian regularization for PP inversion and comparison with the L-curve. The red x-marks show how our algorithm converges for (β e = 10, β m = 100), and the thick x-marks are the final iteration. The black circle and dotted lines are the values of β e and β m.
20 20 T.E. Rabben and B. Ursin 10 3 Bayes. reg. λ 2 -range True bias Figure A5. True bias in the Bayesian regularization and in the L-curve calculations, both for PP inversion. λ 2
21 Non-linear least-squares inversion with data-driven Bayesian regularization m k m k 1 λ 2 k λ 2 k λ2 k Iteration k Figure A6. Convergence of the Bayesian regularization algorithm for inversion of PP reflection amplitudes. The black, dotted line is the convergence criterion for the update.
22 22 T.E. Rabben and B. Ursin 2500 MAP Mean 0.15 Iα Iα Iβ Iβ ρ ρ Figure A7. Absolute value of bias in the medium parameters m for PP inversion. The left column is our MAP estimate while the right column (Mean) is reproduced from Rabben et al. (2008). Some high values in the contrast in S-wave impedance are clipped in order to visually enhance the differences
23 Non-linear least-squares inversion with data-driven Bayesian regularization Bayes. reg. L-curve m µ m 2 S 1 m d f(m) 2 S 1 e Figure A8. Bayesian regularization for joint PP and PS inversion and comparison with the L- curve. For the L-curve we have again used λ 2 values from to The red x-marks show how our algorithm converges for (β e = 0, β m = 5), and the thick x-mark to the left is the final iteration, with λ 2 = The black, dotted line in the figure is the value of β m, and the black square indicates the zoomed area in Fig. A9.
24 24 T.E. Rabben and B. Ursin Bayes. reg. L-curve m µ m 2 S 1 m d f(m) 2 S 1 e Figure A9. Zoom of the black square in Fig. A8.
25 Non-linear least-squares inversion with data-driven Bayesian regularization Bayes. reg. λ 2 -range True bias Figure A10. True bias in the Bayesian regularization and in the L-curve calculations, both for joint PP and PS inversion. λ 2
26 26 T.E. Rabben and B. Ursin m k m k 1 λ 2 k λ 2 k λ2 k Iteration k Figure A11. Convergence of the Bayesian regularization algorithm for inversion of joint PP and PS reflection amplitudes. The black, dotted line is the convergence criterion for the update.
27 Non-linear least-squares inversion with data-driven Bayesian regularization MAP Mean 0.15 Iα Iα Iβ Iβ ρ ρ Figure A12. Absolute value of bias in the medium parameters m for joint PP and PS inversion. The left column is our MAP estimate while the right column (Mean) is reproduced from Rabben et al. (2008). A few high values in the contrast in S-wave impedance are clipped in order to visually enhance the differences
28 28 T.E. Rabben and B. Ursin LIST OF TABLES A1 Comparison of the workload for the Bayesian regularization and the L-curve approaches in the PP example. LS is the number of least-squares iterations, while CG is the total number of conjugate gradient iterations. A2 Comparison of the workload for the Bayesian regularization and the L-curve approaches in the joint PP and PS example. LS is the number of least-squares iterations, while CG is the total number of conjugate gradient iterations.
29 Non-linear least-squares inversion with data-driven Bayesian regularization 29 Table A1. Comparison of the workload for the Bayesian regularization and the L-curve approaches in the PP example. LS is the number of least-squares iterations, while CG is the total number of conjugate gradient iterations. LS CG Bayes. reg L-curve
30 30 T.E. Rabben and B. Ursin Table A2. Comparison of the workload for the Bayesian regularization and the L-curve approaches in the joint PP and PS example. LS is the number of least-squares iterations, while CG is the total number of conjugate gradient iterations. LS CG Bayes. reg L-curve
Nonlinear Bayesian joint inversion of seismic reflection
Geophys. J. Int. (2) 142, Nonlinear Bayesian joint inversion of seismic reflection coefficients Tor Erik Rabben 1,, Håkon Tjelmeland 2, and Bjørn Ursin 1 1 Department of Petroleum Engineering and Applied
GEOPHYSICS. AVA inversion of the top Utsira Sand reflection at the Sleipner field
AVA inversion of the top Utsira Sand reflection at the Sleipner field Journal: Geophysics Manuscript ID: GEO-00-00 Manuscript Type: Technical Paper Date Submitted by the Author: -Sep-00 Complete List of
= (G T G) 1 G T d. m L2
The importance of the Vp/Vs ratio in determining the error propagation and the resolution in linear AVA inversion M. Aleardi, A. Mazzotti Earth Sciences Department, University of Pisa, Italy Introduction.
Bayesian Lithology-Fluid Prediction and Simulation based. on a Markov Chain Prior Model
Bayesian Lithology-Fluid Prediction and Simulation based on a Markov Chain Prior Model Anne Louise Larsen Formerly Norwegian University of Science and Technology, N-7491 Trondheim, Norway; presently Schlumberger
ECE521 week 3: 23/26 January 2017
ECE521 week 3: 23/26 January 2017 Outline Probabilistic interpretation of linear regression - Maximum likelihood estimation (MLE) - Maximum a posteriori (MAP) estimation Bias-variance trade-off Linear
NORGES TEKNISK-NATURVITENSKAPELIGE UNIVERSITET. Directional Metropolis Hastings updates for posteriors with nonlinear likelihoods
NORGES TEKNISK-NATURVITENSKAPELIGE UNIVERSITET Directional Metropolis Hastings updates for posteriors with nonlinear likelihoods by Håkon Tjelmeland and Jo Eidsvik PREPRINT STATISTICS NO. 5/2004 NORWEGIAN
Multiple Scenario Inversion of Reflection Seismic Prestack Data
Downloaded from orbit.dtu.dk on: Nov 28, 2018 Multiple Scenario Inversion of Reflection Seismic Prestack Data Hansen, Thomas Mejer; Cordua, Knud Skou; Mosegaard, Klaus Publication date: 2013 Document Version
STA 4273H: Sta-s-cal Machine Learning
STA 4273H: Sta-s-cal Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 2 In our
Bayesian lithology/fluid inversion comparison of two algorithms
Comput Geosci () 14:357 367 DOI.07/s596-009-9155-9 REVIEW PAPER Bayesian lithology/fluid inversion comparison of two algorithms Marit Ulvmoen Hugo Hammer Received: 2 April 09 / Accepted: 17 August 09 /
Parameter estimation: A new approach to weighting a priori information
c de Gruyter 27 J. Inv. Ill-Posed Problems 5 (27), 2 DOI.55 / JIP.27.xxx Parameter estimation: A new approach to weighting a priori information J.L. Mead Communicated by Abstract. We propose a new approach
GEOPHYSICAL INVERSE THEORY AND REGULARIZATION PROBLEMS
Methods in Geochemistry and Geophysics, 36 GEOPHYSICAL INVERSE THEORY AND REGULARIZATION PROBLEMS Michael S. ZHDANOV University of Utah Salt Lake City UTAH, U.S.A. 2OO2 ELSEVIER Amsterdam - Boston - London
Regularization Parameter Estimation for Least Squares: A Newton method using the χ 2 -distribution
Regularization Parameter Estimation for Least Squares: A Newton method using the χ 2 -distribution Rosemary Renaut, Jodi Mead Arizona State and Boise State September 2007 Renaut and Mead (ASU/Boise) Scalar
Introduction to Bayesian methods in inverse problems
Introduction to Bayesian methods in inverse problems Ville Kolehmainen 1 1 Department of Applied Physics, University of Eastern Finland, Kuopio, Finland March 4 2013 Manchester, UK. Contents Introduction
REGULARIZATION PARAMETER SELECTION IN DISCRETE ILL POSED PROBLEMS THE USE OF THE U CURVE
Int. J. Appl. Math. Comput. Sci., 007, Vol. 17, No., 157 164 DOI: 10.478/v10006-007-0014-3 REGULARIZATION PARAMETER SELECTION IN DISCRETE ILL POSED PROBLEMS THE USE OF THE U CURVE DOROTA KRAWCZYK-STAŃDO,
P191 Bayesian Linearized AVAZ Inversion in HTI Fractured Media
P9 Bayesian Linearized AAZ Inversion in HI Fractured Media L. Zhao* (University of Houston), J. Geng (ongji University), D. Han (University of Houston) & M. Nasser (Maersk Oil Houston Inc.) SUMMARY A new
STA414/2104 Statistical Methods for Machine Learning II
STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements
Discrete ill posed problems
Discrete ill posed problems Gérard MEURANT October, 2008 1 Introduction to ill posed problems 2 Tikhonov regularization 3 The L curve criterion 4 Generalized cross validation 5 Comparisons of methods Introduction
Frequentist-Bayesian Model Comparisons: A Simple Example
Frequentist-Bayesian Model Comparisons: A Simple Example Consider data that consist of a signal y with additive noise: Data vector (N elements): D = y + n The additive noise n has zero mean and diagonal
y(x) = x w + ε(x), (1)
Linear regression We are ready to consider our first machine-learning problem: linear regression. Suppose that e are interested in the values of a function y(x): R d R, here x is a d-dimensional vector-valued
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear
Probing the covariance matrix
Probing the covariance matrix Kenneth M. Hanson Los Alamos National Laboratory (ret.) BIE Users Group Meeting, September 24, 2013 This presentation available at http://kmh-lanl.hansonhub.com/ LA-UR-06-5241
Stat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
COS513 LECTURE 8 STATISTICAL CONCEPTS
COS513 LECTURE 8 STATISTICAL CONCEPTS NIKOLAI SLAVOV AND ANKUR PARIKH 1. MAKING MEANINGFUL STATEMENTS FROM JOINT PROBABILITY DISTRIBUTIONS. A graphical model (GM) represents a family of probability distributions
Stochastic Spectral Approaches to Bayesian Inference
Stochastic Spectral Approaches to Bayesian Inference Prof. Nathan L. Gibson Department of Mathematics Applied Mathematics and Computation Seminar March 4, 2011 Prof. Gibson (OSU) Spectral Approaches to
Fundamentals of Data Assimila1on
014 GSI Community Tutorial NCAR Foothills Campus, Boulder, CO July 14-16, 014 Fundamentals of Data Assimila1on Milija Zupanski Cooperative Institute for Research in the Atmosphere Colorado State University
Gaussian processes. Chuong B. Do (updated by Honglak Lee) November 22, 2008
Gaussian processes Chuong B Do (updated by Honglak Lee) November 22, 2008 Many of the classical machine learning algorithms that we talked about during the first half of this course fit the following pattern:
Estimation of Cole-Cole parameters from time-domain electromagnetic data Laurens Beran and Douglas Oldenburg, University of British Columbia.
Estimation of Cole-Cole parameters from time-domain electromagnetic data Laurens Beran and Douglas Oldenburg, University of British Columbia. SUMMARY We present algorithms for the inversion of time-domain
ECE295, Data Assimila0on and Inverse Problems, Spring 2015
ECE295, Data Assimila0on and Inverse Problems, Spring 2015 1 April, Intro; Linear discrete Inverse problems (Aster Ch 1 and 2) Slides 8 April, SVD (Aster ch 2 and 3) Slides 15 April, RegularizaFon (ch
DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING. Inversion basics. Erkki Kyrölä Finnish Meteorological Institute
Inversion basics y = Kx + ε x ˆ = (K T K) 1 K T y Erkki Kyrölä Finnish Meteorological Institute Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 1 Contents 1. Introduction: Measurements, models, inversion
Bayesian System Identification based on Hierarchical Sparse Bayesian Learning and Gibbs Sampling with Application to Structural Damage Assessment
Bayesian System Identification based on Hierarchical Sparse Bayesian Learning and Gibbs Sampling with Application to Structural Damage Assessment Yong Huang a,b, James L. Beck b,* and Hui Li a a Key Lab
The Bayesian approach to inverse problems
The Bayesian approach to inverse problems Youssef Marzouk Department of Aeronautics and Astronautics Center for Computational Engineering Massachusetts Institute of Technology ymarz@mit.edu, http://uqgroup.mit.edu
Linear Models 1. Isfahan University of Technology Fall Semester, 2014
Linear Models 1 Isfahan University of Technology Fall Semester, 2014 References: [1] G. A. F., Seber and A. J. Lee (2003). Linear Regression Analysis (2nd ed.). Hoboken, NJ: Wiley. [2] A. C. Rencher and
Uncertainty quantification for Wavefield Reconstruction Inversion
Uncertainty quantification for Wavefield Reconstruction Inversion Zhilong Fang *, Chia Ying Lee, Curt Da Silva *, Felix J. Herrmann *, and Rachel Kuske * Seismic Laboratory for Imaging and Modeling (SLIM),
Stochastic vs Deterministic Pre-stack Inversion Methods. Brian Russell
Stochastic vs Deterministic Pre-stack Inversion Methods Brian Russell Introduction Seismic reservoir analysis techniques utilize the fact that seismic amplitudes contain information about the geological
Uncertainty quantification for inverse problems with a weak wave-equation constraint
Uncertainty quantification for inverse problems with a weak wave-equation constraint Zhilong Fang*, Curt Da Silva*, Rachel Kuske** and Felix J. Herrmann* *Seismic Laboratory for Imaging and Modeling (SLIM),
Bayesian closed skew Gaussian inversion of seismic AVO data into elastic material properties
Bayesian closed skew Gaussian inversion of seismic AVO data into elastic material properties Omid Karimi 1,2, Henning Omre 2 and Mohsen Mohammadzadeh 1 1 Tarbiat Modares University, Iran 2 Norwegian University
Lecture 3. Linear Regression II Bastian Leibe RWTH Aachen
Advanced Machine Learning Lecture 3 Linear Regression II 02.11.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de This Lecture: Advanced Machine Learning Regression
Outline lecture 2 2(30)
Outline lecture 2 2(3), Lecture 2 Linear Regression it is our firm belief that an understanding of linear models is essential for understanding nonlinear ones Thomas Schön Division of Automatic Control
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen. Bayesian Learning. Tobias Scheffer, Niels Landwehr
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Bayesian Learning Tobias Scheffer, Niels Landwehr Remember: Normal Distribution Distribution over x. Density function with parameters
Inverse Theory. COST WaVaCS Winterschool Venice, February Stefan Buehler Luleå University of Technology Kiruna
Inverse Theory COST WaVaCS Winterschool Venice, February 2011 Stefan Buehler Luleå University of Technology Kiruna Overview Inversion 1 The Inverse Problem 2 Simple Minded Approach (Matrix Inversion) 3
Downloaded 08/29/13 to Redistribution subject to SEG license or copyright; see Terms of Use at
New approach to 3D inversion of MCSEM and MMT data using multinary model transform Alexander V. Gribenko and Michael S. Zhdanov, University of Utah and TechnoImaging SUMMARY Marine controlled-source electromagnetic
Analysis of a prior model for a blocky inversion of seismic amplitude versus offset data
Analysis of a prior model for a blocky inversion of seismic amplitude versus offset data Short title: Blocky inversion of seismic AVO data Ulrich Theune* StatoilHydro Research Center, 7005 Trondheim, NORWAY,
Hierarchical sparse Bayesian learning for structural health monitoring. with incomplete modal data
Hierarchical sparse Bayesian learning for structural health monitoring with incomplete modal data Yong Huang and James L. Beck* Division of Engineering and Applied Science, California Institute of Technology,
23855 Rock Physics Constraints on Seismic Inversion
23855 Rock Physics Constraints on Seismic Inversion M. Sams* (Ikon Science Ltd) & D. Saussus (Ikon Science) SUMMARY Seismic data are bandlimited, offset limited and noisy. Consequently interpretation of
Simultaneous Multi-frame MAP Super-Resolution Video Enhancement using Spatio-temporal Priors
Simultaneous Multi-frame MAP Super-Resolution Video Enhancement using Spatio-temporal Priors Sean Borman and Robert L. Stevenson Department of Electrical Engineering, University of Notre Dame Notre Dame,
Deconvolution. Parameter Estimation in Linear Inverse Problems
Image Parameter Estimation in Linear Inverse Problems Chair for Computer Aided Medical Procedures & Augmented Reality Department of Computer Science, TUM November 10, 2006 Contents A naive approach......with
5. Discriminant analysis
5. Discriminant analysis We continue from Bayes s rule presented in Section 3 on p. 85 (5.1) where c i is a class, x isap-dimensional vector (data case) and we use class conditional probability (density
ADAPTIVE FILTER THEORY
ADAPTIVE FILTER THEORY Fourth Edition Simon Haykin Communications Research Laboratory McMaster University Hamilton, Ontario, Canada Front ice Hall PRENTICE HALL Upper Saddle River, New Jersey 07458 Preface
Computer Vision Group Prof. Daniel Cremers. 3. Regression
Prof. Daniel Cremers 3. Regression Categories of Learning (Rep.) Learnin g Unsupervise d Learning Clustering, density estimation Supervised Learning learning from a training data set, inference on the
Lecture 5: GPs and Streaming regression
Lecture 5: GPs and Streaming regression Gaussian Processes Information gain Confidence intervals COMP-652 and ECSE-608, Lecture 5 - September 19, 2017 1 Recall: Non-parametric regression Input space X
Physics 403. Segev BenZvi. Numerical Methods, Maximum Likelihood, and Least Squares. Department of Physics and Astronomy University of Rochester
Physics 403 Numerical Methods, Maximum Likelihood, and Least Squares Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Review of Last Class Quadratic Approximation
Regularized Regression A Bayesian point of view
Regularized Regression A Bayesian point of view Vincent MICHEL Director : Gilles Celeux Supervisor : Bertrand Thirion Parietal Team, INRIA Saclay Ile-de-France LRI, Université Paris Sud CEA, DSV, I2BM,
Porosity Bayesian inference from multiple well-log data
Porosity inference from multiple well log data Porosity Bayesian inference from multiple well-log data Luiz Lucchesi Loures ABSTRACT This paper reports on an inversion procedure for porosity estimation
Probability and Estimation. Alan Moses
Probability and Estimation Alan Moses Random variables and probability A random variable is like a variable in algebra (e.g., y=e x ), but where at least part of the variability is taken to be stochastic.
Pattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
CSC 2541: Bayesian Methods for Machine Learning
CSC 2541: Bayesian Methods for Machine Learning Radford M. Neal, University of Toronto, 2011 Lecture 3 More Markov Chain Monte Carlo Methods The Metropolis algorithm isn t the only way to do MCMC. We ll
Linear Models for Regression CS534
Linear Models for Regression CS534 Prediction Problems Predict housing price based on House size, lot size, Location, # of rooms Predict stock price based on Price history of the past month Predict the
Chapter 2 Wiener Filtering
Chapter 2 Wiener Filtering Abstract Before moving to the actual adaptive filtering problem, we need to solve the optimum linear filtering problem (particularly, in the mean-square-error sense). We start
Generalized Elastic Net Regression
Abstract Generalized Elastic Net Regression Geoffroy MOURET Jean-Jules BRAULT Vahid PARTOVINIA This work presents a variation of the elastic net penalization method. We propose applying a combined l 1
Learning Gaussian Process Models from Uncertain Data
Learning Gaussian Process Models from Uncertain Data Patrick Dallaire, Camille Besse, and Brahim Chaib-draa DAMAS Laboratory, Computer Science & Software Engineering Department, Laval University, Canada
IJMGE Int. J. Min. & Geo-Eng. Vol.49, No.1, June 2015, pp
IJMGE Int. J. Min. & Geo-Eng. Vol.49, No.1, June 2015, pp.131-142 Joint Bayesian Stochastic Inversion of Well Logs and Seismic Data for Volumetric Uncertainty Analysis Moslem Moradi 1, Omid Asghari 1,
Linear Regression (9/11/13)
STA561: Probabilistic machine learning Linear Regression (9/11/13) Lecturer: Barbara Engelhardt Scribes: Zachary Abzug, Mike Gloudemans, Zhuosheng Gu, Zhao Song 1 Why use linear regression? Figure 1: Scatter
MODULE -4 BAYEIAN LEARNING
MODULE -4 BAYEIAN LEARNING CONTENT Introduction Bayes theorem Bayes theorem and concept learning Maximum likelihood and Least Squared Error Hypothesis Maximum likelihood Hypotheses for predicting probabilities
covariance function, 174 probability structure of; Yule-Walker equations, 174 Moving average process, fluctuations, 5-6, 175 probability structure of
Index* The Statistical Analysis of Time Series by T. W. Anderson Copyright 1971 John Wiley & Sons, Inc. Aliasing, 387-388 Autoregressive {continued) Amplitude, 4, 94 case of first-order, 174 Associated
The Inversion Problem: solving parameters inversion and assimilation problems
The Inversion Problem: solving parameters inversion and assimilation problems UE Numerical Methods Workshop Romain Brossier romain.brossier@univ-grenoble-alpes.fr ISTerre, Univ. Grenoble Alpes Master 08/09/2016
Linear Models. DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis.
Linear Models DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Linear regression Least-squares estimation
DS-GA 1002 Lecture notes 10 November 23, Linear models
DS-GA 2 Lecture notes November 23, 2 Linear functions Linear models A linear model encodes the assumption that two quantities are linearly related. Mathematically, this is characterized using linear functions.
1 Introduction When the model structure does not match the system, is poorly identiable, or the available set of empirical data is not suciently infor
On Tikhonov Regularization, Bias and Variance in Nonlinear System Identication Tor A. Johansen SINTEF Electronics and Cybernetics, Automatic Control Department, N-7034 Trondheim, Norway. Email: Tor.Arne.Johansen@ecy.sintef.no.
Likelihood Bounds for Constrained Estimation with Uncertainty
Proceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference 5 Seville, Spain, December -5, 5 WeC4. Likelihood Bounds for Constrained Estimation with Uncertainty
Least Absolute Shrinkage is Equivalent to Quadratic Penalization
Least Absolute Shrinkage is Equivalent to Quadratic Penalization Yves Grandvalet Heudiasyc, UMR CNRS 6599, Université de Technologie de Compiègne, BP 20.529, 60205 Compiègne Cedex, France Yves.Grandvalet@hds.utc.fr
Cheng Soon Ong & Christian Walder. Canberra February June 2018
Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2018 (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 254 Part V
DETECTING PROCESS STATE CHANGES BY NONLINEAR BLIND SOURCE SEPARATION. Alexandre Iline, Harri Valpola and Erkki Oja
DETECTING PROCESS STATE CHANGES BY NONLINEAR BLIND SOURCE SEPARATION Alexandre Iline, Harri Valpola and Erkki Oja Laboratory of Computer and Information Science Helsinki University of Technology P.O.Box
Bayesian Linear Models
Bayesian Linear Models Sudipto Banerjee 1 and Andrew O. Finley 2 1 Department of Forestry & Department of Geography, Michigan State University, Lansing Michigan, U.S.A. 2 Biostatistics, School of Public
Parametric Techniques
Parametric Techniques Jason J. Corso SUNY at Buffalo J. Corso (SUNY at Buffalo) Parametric Techniques 1 / 39 Introduction When covering Bayesian Decision Theory, we assumed the full probabilistic structure
Uncertainty Quantification for Inverse Problems. November 7, 2011
Uncertainty Quantification for Inverse Problems November 7, 2011 Outline UQ and inverse problems Review: least-squares Review: Gaussian Bayesian linear model Parametric reductions for IP Bias, variance
Linear Models for Regression. Sargur Srihari
Linear Models for Regression Sargur srihari@cedar.buffalo.edu 1 Topics in Linear Regression What is regression? Polynomial Curve Fitting with Scalar input Linear Basis Function Models Maximum Likelihood
Bayesian Decision Theory
Bayesian Decision Theory Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2017 CS 551, Fall 2017 c 2017, Selim Aksoy (Bilkent University) 1 / 46 Bayesian
SIO223A A Brief Introduction to Geophysical Modeling and Inverse Theory.
SIO223A A Brief Introduction to Geophysical Modeling and Inverse Theory. Forward modeling: data model space space ˆd = f(x, m) m =(m 1,m 2,..., m N ) x =(x 1,x 2,x 3,..., x km ) ˆd =(ˆd 1, ˆd 2, ˆd 3,...,
Pattern Recognition and Machine Learning. Bishop Chapter 2: Probability Distributions
Pattern Recognition and Machine Learning Chapter 2: Probability Distributions Cécile Amblard Alex Kläser Jakob Verbeek October 11, 27 Probability Distributions: General Density Estimation: given a finite
Lessons in Estimation Theory for Signal Processing, Communications, and Control
Lessons in Estimation Theory for Signal Processing, Communications, and Control Jerry M. Mendel Department of Electrical Engineering University of Southern California Los Angeles, California PRENTICE HALL
Discrete Mathematics and Probability Theory Fall 2015 Lecture 21
CS 70 Discrete Mathematics and Probability Theory Fall 205 Lecture 2 Inference In this note we revisit the problem of inference: Given some data or observations from the world, what can we infer about
Geophysical Data Analysis: Discrete Inverse Theory
Geophysical Data Analysis: Discrete Inverse Theory MATLAB Edition William Menke Lamont-Doherty Earth Observatory and Department of Earth and Environmental Sciences Columbia University. ' - Palisades, New
component risk analysis
273: Urban Systems Modeling Lec. 3 component risk analysis instructor: Matteo Pozzi 273: Urban Systems Modeling Lec. 3 component reliability outline risk analysis for components uncertain demand and uncertain
Bayesian Inference (A Rough Guide)
Sigmedia, Electronic Engineering Dept., Trinity College, Dublin. 1 Bayesian Inference (A Rough Guide) Anil C. Kokaram anil.kokaram@tcd.ie Electrical and Electronic Engineering Dept., University of Dublin,
STA414/2104. Lecture 11: Gaussian Processes. Department of Statistics
STA414/2104 Lecture 11: Gaussian Processes Department of Statistics www.utstat.utoronto.ca Delivered by Mark Ebden with thanks to Russ Salakhutdinov Outline Gaussian Processes Exam review Course evaluations
SIGNAL AND IMAGE RESTORATION: SOLVING
1 / 55 SIGNAL AND IMAGE RESTORATION: SOLVING ILL-POSED INVERSE PROBLEMS - ESTIMATING PARAMETERS Rosemary Renaut http://math.asu.edu/ rosie CORNELL MAY 10, 2013 2 / 55 Outline Background Parameter Estimation
Linear Models for Regression
Linear Models for Regression Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr
Dimension-Independent likelihood-informed (DILI) MCMC
Dimension-Independent likelihood-informed (DILI) MCMC Tiangang Cui, Kody Law 2, Youssef Marzouk Massachusetts Institute of Technology 2 Oak Ridge National Laboratory 2 August 25 TC, KL, YM DILI MCMC USC
Bayesian linear regression
Bayesian linear regression Linear regression is the basis of most statistical modeling. The model is Y i = X T i β + ε i, where Y i is the continuous response X i = (X i1,..., X ip ) T is the corresponding
Bayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
Neutron inverse kinetics via Gaussian Processes
Neutron inverse kinetics via Gaussian Processes P. Picca Politecnico di Torino, Torino, Italy R. Furfaro University of Arizona, Tucson, Arizona Outline Introduction Review of inverse kinetics techniques
Bayesian Linear Models
Bayesian Linear Models Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department
Bayesian Linear Regression. Sargur Srihari
Bayesian Linear Regression Sargur srihari@cedar.buffalo.edu Topics in Bayesian Regression Recall Max Likelihood Linear Regression Parameter Distribution Predictive Distribution Equivalent Kernel 2 Linear
Bayesian Regression Linear and Logistic Regression
When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we
Probabilistic Graphical Models
Probabilistic Graphical Models Brown University CSCI 2950-P, Spring 2013 Prof. Erik Sudderth Lecture 12: Gaussian Belief Propagation, State Space Models and Kalman Filters Guest Kalman Filter Lecture by
SCUOLA DI SPECIALIZZAZIONE IN FISICA MEDICA. Sistemi di Elaborazione dell Informazione. Regressione. Ruggero Donida Labati
SCUOLA DI SPECIALIZZAZIONE IN FISICA MEDICA Sistemi di Elaborazione dell Informazione Regressione Ruggero Donida Labati Dipartimento di Informatica via Bramante 65, 26013 Crema (CR), Italy http://homes.di.unimi.it/donida
A Process over all Stationary Covariance Kernels
A Process over all Stationary Covariance Kernels Andrew Gordon Wilson June 9, 0 Abstract I define a process over all stationary covariance kernels. I show how one might be able to perform inference that
Calibrating Environmental Engineering Models and Uncertainty Analysis
Models and Cornell University Oct 14, 2008 Project Team Christine Shoemaker, co-pi, Professor of Civil and works in applied optimization, co-pi Nikolai Blizniouk, PhD student in Operations Research now
Linear Methods for Prediction
Chapter 5 Linear Methods for Prediction 5.1 Introduction We now revisit the classification problem and focus on linear methods. Since our prediction Ĝ(x) will always take values in the discrete set G we
LINEAR MODELS FOR CLASSIFICATION. J. Elder CSE 6390/PSYC 6225 Computational Modeling of Visual Perception
LINEAR MODELS FOR CLASSIFICATION Classification: Problem Statement 2 In regression, we are modeling the relationship between a continuous input variable x and a continuous target variable t. In classification,