Scientific Data Computing: Lecture 3

Transcription

1 Scientific Data Computing: Lecture 3 Benson Muite benson.muite@ut.ee 23 April 2018

2 Outline Monday 10-12, Liivi Monday 12-14, Liivi Topics Introduction, statistical methods and their applications Linear algebra and higher order singular value decomposition Optimization and adjoints Clustering and applications Compressed sensing Text processing Time series analysis and wavelets

3 Outline Optimization Adjoints Image Denoising

4 Optimization Maximizing or minimizing a function Used in many processes (largest happiness, most profit, least cost) Typically distinguish between discrete and continuous. Continuous case typically uses calculus Discrete case typically uses guided search

5 L 1 and L matrix norms A 1 = sup j A ij A = sup i A ij Can use these to define L 1 and L steepest descent algorithms i j

6 Optimization find x such that f (x ) f (x) for all feasible x or find x such that f (x ) f (x) for all feasible x Constrained vs. Unconstrained Continuous vs. Discrete Differentiable vs. Non differentiable

7 Optimization: Hooke and Jeeves method a) choose a step size b) Successively sequentially check if f (x ± he i ) < f (x), and update x if so c) After checking all coordinate directions, check if x was updated. If the step size h < ɛ, stop, otherwise decrease h by a factor of 2. d) If x was updated If the step size h < ɛ, stop. Otherwise start searching from 2x new x old if f (2x new x old ) < f (x new x old ), if not search from f (x new x old ).

8 Optimization: Nelder and Mead method a) Generate a simplex of n + 1 points in R. b) Remove the vertex with the worst function value and replace it with a new point. Choose the point by reflecting, expanding or contracting the simplex along the line joining the worst vertex with the centroid of the remaining vertices. If this does not give a better value, keep the best vertex and shrink all other vertices towards the best one.

9 Optimization: Descent methods For a differentiable function x k+1 = x k + α k d k a) Newton s method: d k = H 1 (x k ) f (x k ) where H is the hessian of f b) Approximate Newton: d k = B 1 (x k ) f (x k ) where B approximates the hessian of f c) Steepest descent: d k = f (x k ) d) Conjugate gradient: d k = f (x k ) + β k d k 1

10 Constrained Optimization Given f : R n R maximize f (x) subject to g i (x) b i g i (x) b i g i (x) = b i x i m 0 i = 1,..., l i = l + 1,..., k i = k + 1,..., m i = m + 1,..., n Typically define a Lagrangian L = f (x) + m λ i [b i g i (x)] i=1 n i=m+1 λ i x i m and use Kuhn-Tucker conditions to check for constrained extrema.

11 Adjoints - Following Tekitek Consider the mass spring equation d 2 x dt 2 + ω2 x = 0 Suppose we measure the position as a function of time. Might we be able to estimate the initial position(x 0 ), initial velocity(v 0 ) and natural frequency(ω)? The exact solution would be Measured data is given by x x(t) = x 0 cos(ωt) + v 0 ω sin(ωt)

12 Adjoints The idea of Pontryagin is to minimize 0.5 T 0 x(t) x 2 dt as a function of x 0,v 0 and ω subject to the constraint of satisfying the differential equation

13 Adjoints T ( L = 0.5 (x(t) x) 2 d 2 ) x + p(t) 0 dt 2 + ω2 x dt Taking variations and integrating by parts, one finds the adjoint equation satisfies with p(t ) = 0 and dp dt = 0. The gradient is given by d 2 p dt 2 + ω2 p + x x = 0 J = dp T dt (0)δx 0 p(0)δv 0 + 2ωp(t)x(t)δωdt 0

14 Adjoints The gradient is given by J = dp T dt (0)δx 0 p(0)δv 0 + 2ωp(t)x(t)δωdt 0

15 Image Processing Image Denoising Image Segmentation

16 The heat equation u t = u Decreases L 2 norm Can be thought of as successive averaging (see first lecture)

17 Denoising with nonlinear diffusion Gray scale image where u is the intensity u t (Pu x ) x (Pu y ) y = 0 P = 1 η = 1+η 1 (ux 2 +uy 2 ) u 2 x +uy 2 dxdy 1dxdy

18 Denoising color images with nonlinear diffusion Color image in (r, g, b) space r t (Pr x ) x (Pr y ) y = 0 g t (Pg x ) x (Pg y ) y = 0 b t (Pb x ) x (Pb y ) y = 0 P = 1 det E E = 1 + r x 2 +gx 2 +bx 2 η r x r y +g x g y +b x b y η r x r y +g x g y +b x b y η 1 + r 2 y +g 2 y +b 2 y η η := r 2 x +r 2 y +g 2 x +g 2 y +b 2 x +b 2 y dxdy 1dxdy

19 Large scale optimization Some packages available Dakota

20 References Muite, Scientific Computing lecture slides (2015) Solomon and Breckon, Fundamentals of Digital Image Processing, Wiley-Blackwell (2011) Quarteroni, Saleri and Sacco, Numerical Mathematics Springer (2011) chapter 7 Boyd and Vandenberghe Convex Optimization Cambridge (2004) Nocedal and Wright Numerical Optimization Springer (2006)

21 References Perona, Malik Scale space and edge detection using anisotropic diffusion IEEE Trans. Pattern Anal (1990) Shapira Solving PDEs in C++ SIAM 2012 Tekitek, Bouzidi, Dubois, Lallemand Adjoint Lattice Boltzmann Equation for Parameter Identification Computers and Fluids, (2006) Dubois, Lallemand, Tekitek, Parameters identification and the adjoint method Application to Lattice Boltzmann Equation Lecture notes, Spring School: Lattice Boltzmann Methods with OpenLB Sofware Lab