Fraunhofer Institute for Computer Graphics Research Interactive Graphics Systems Group, TU Darmstadt Fraunhoferstrasse 5, Darmstadt, Germany

Transcription

1 Scale Space and PDE methods in image analysis and processing Arjan Kuijper Fraunhofer Institute for Computer Graphics Research Interactive Graphics Systems Group, TU Darmstadt Fraunhoferstrasse 5, Darmstadt, Germany Tel.: +49 (0) Fax.: +49 (0) Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 1/29

2 MCM Mean curvature motion Curve evolution Denoising Edge preserving Implementation Isophote vs. image implementation Taken from: Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 2/29

3 Alvarez, Guichard, Lions and Morel (1993) realized that the P&M variable conductance diffusion was complicated by the choice of the parameter k. They reasoned that the principal influence on the local conductivity should be to direct the flow in the direction of the gradient only: L s = L. L L They named the affine version (right-hand side to the power 1/3) the 'fundamental equation of image processing This is the unique model of multi-scale analysis of an image, affine invariant and morphological invariant. L. Alvarez, F. Guichard, P-L. Lions, and J-M. Morel. Axioms and fundamental equations of image processing. Arch. Rational Mechanics and Anal., 16(9): , Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 3/29

4 Grayscale invariance The non-affine version can be written as There are a number of differences between this equation and the Perona & Malik equation: the flow (of flux) is independent of the magnitude of the gradient; There is no extra free parameter, like the edge-strength turnover parameter k; in the P&M equation the diffusion decreases when the gradient is large, resulting in contrast dependent smoothing; this equation is gray-scale invariant (the function does not change value when the grayscale function L is modified by a monotonically increasing or decreasing function f(l), f 0.). This PDE is known as Euclidean shortening flow, curve shortening, L s = L vv Mean curvature Motion Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 4/29

5 Overview of Evolution Equations Alvarez et al. (1993) present a generalized framework for a number of nonlinear evolution equations -- an extensive treatment on evolution equations of the general form: Imposing various axioms on a multi-scale analysis the authors derive a number of evolution equations. We distinguish two approaches: i) evolution of the luminance function and ii) evolution of the level sets of the image. These approaches are dual in the sense that one determines the other Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 5/29

6 Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 6/29

7 Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 7/29

8 Choices for F(.) F = L In this case we find the linear diffusion equation. The luminance L is conserved under the flow L. F = c( L ) L The heat conduction coefficient c is not a constant anymore but depends on local image properties (Perona and Malik, 1990) resulting in nonlinear or geometry-driven diffusion. F = L / L This equation has been used in Rudin et al. (1992) to remove noise based on nonlinear total variation. Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 8/29

9 Curve evolution A general equation which evolves planar curves as a function of their geometry can be written as: In other words, a curve evolves as a function of curvature (and derivatives of curvature with respect to arc-length) only. The definition above follows from the following considerations. A general evolution of a curve can be written as: where T and N denote the tangential and normal unit vector to the curve respectively. However, the T component only affects the parameterization. Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 9/29

10 Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 10/29

11 Choices for g(.) g = c This choice results in normal motion: Isophotes move in the normal direction with velocity c. This equation is equivalent to the morphological operation of erosion (or depending on the sign of c dilation) with a disc as structuring element. g = k This equation evolves the curve as a function of curvature and is known as the Euclidean shortening flow. It implies the following evolution of the luminance function: Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 11/29

12 Some differential geometry This equation has been proposed since it is invariant under Euclidean transformations. An elegant generalization to find the flow which is invariant with respect to any Lie group action can be found is as follows: The evolution given by where r denotes the arc-length which is invariant under the group, defines a flow which is invariant under the action of the group. These equations locally behave as the geometric heat equation: where g = x r is the G-invariant metric (g=1 in Euclidean space). If r is Euclidean arc-length (re) we find the Euclidean shortening flow: Special case: Laplacian blurring on a surface: Laplace-Beltrami Flow Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 12/29

13 Why the name 'shortening flow'? For the metric of the curve, defined as g p, t = x p where p is an arbitrary parameterization of the curve, the evolution of the metric is equal to g t =-k 2 g The total length of the curve evolves as L = 0 2 p g t, t t so the length is always decreasing with time: L t = t 02 p g t, t t = p k 2 g t, t t = - 0 L k 2 v Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 13/29

14 Warning! Euclidean shortening flow is not the second order derivative of the spatial coordinates (left), but of the parameterization (right)! Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 14/29

15 Choices for g(.) This evolution is known as the affine shortening flow. (recall Alvarez et al.) If we insert the affine arc-length (r a ) we find: g = a + b k A combination of normal motion and Euclidean shortening flow. Note that a and b have different dimensions (the value b/ a is not invariant under a spatial rescaling x -> l x). We have to work in natural coordinates or multiply nth order derivatives with the n th power of scale. The luminance function evolves according to This is a Hamilton-Jacobi equation with parabolic right-hand side. Since there are two independent variables it generates a 2-dimensional Entropy Scale Space with a reaction axis (owing to the hyperbolic term) and a diffusion axis (owing to the parabolic right hand side). Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 15/29

16 Overview Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 16/29

17 Numerical stability When we approximate a partial differential equation with finite differences in the forward Euler scheme, we want to make large steps in evolution time (or scale) to reach the final evolution in the fastest way, with as little iterations as possible. How large steps are we allowed to make? In other words, can we find a criterion for which the equation remains stable? A famous answer to the question of stability was derived by Von Neumann, and is called the Von Neumann stability criterion. Alternative names: Courant stability criterion CFL condition (Courant-Friedrichs-Lewy condition) Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 17/29

18 Consider the 1D linear diffusion equation: L t = 2 L x 2 This equation can be approximated with finite differences as L j n+1 - L j n D t = L n j+1-2 L n j + Ln j-1 D x 2 We define R = D t, so rewrite to D x 2 L n+1 j - L n n j - R L j+1-2 L n n j + L j-1 = 0 i.e. f(j,n)=0 Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 18/29

19 Let the solution L j n of our PDE be a generalized exponential function, with k a general (spatial) wave number: j k x n When we insert this solution in our discretized PDE, we get jk x n jk x 1 n R 1 j k x n 2 j k x n 1 j k x n We want the increment function f(j,n) to be maximal on the domain j, so we get the condition f j, n j = 0 1 2R 2 R Cos k x Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 19/29

20 1 2R 2 R Cos k x The amplitude x n of the solution x n e  j k Dx should not explode for large n, so in order to get a stable solution we need the criterion x 1. This means, because Cos(k Dx)-1 is always non-positive, that R = D t D x This is an essential result. When we take a too large step size for Dt in order to reach the final time faster, we may find that the result gets unstable. The Von Neumann criterion gives us the fastest way we can get to the iterative result. It is safe to stay well under the maximum value, to not compromise this stability close to the criterion. Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 20/29

21 The pixel step Dx is mostly unity, so the maximum evolution step size should be D t < 1 4 pixel2 This is indeed a strong limitation, making many iteration steps necessary. Gaussian derivative kernels improve this situation considerably. We start again with a general possible solution for the luminance function L(x,j,n), where x is the spatial coordinate, j is the discrete spatial grid position, and n is the discrete moment in evolution time of the PDE. j x n Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 21/29

22 The Laplacian in 1D is just the second order spatial derivative: x2 4s 2s x 2 8 s 5 2 We recall that the convolution of a function f with a kernel g is defined as f g = - f y g y - x x For discrete location j at time step n we get for the blurred intensity: jy x y 2 4s 2s x y 2 n 8 s 5 2 y j j s x j 2 n Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 22/29

23 If we divide this by the original intensity function, we find a multiplication factor - - j2 s j 2 factor We are looking for the largest absolute value of the factor, because then we take the largest evolution steps. Because the factor is negative everywhere we need to find the minimum of the factor with respect to j, i.e. factor j = 0 -> j = 1 s We then find for the maximum size of the time step factor -1 s j time Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 23/29

24 So we find for the Gaussian derivative implementation x=1 - Dt s Dt s 2 so x 1 implies, thus Dt 2 s Introducing this in the time-space ratio R we get the limiting step size for a stable solution under Gaussian blurring: R = D t D x 2 2 s = s2 Note that this enables substantially larger step sizes then in the nearest neighbor case. Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 24/29

25 For the Gaussian blurring of an image with s=0.8 pixels for the Laplacian operator, we get Ds<.8 2 =1.74. We blur a test image to 128 pixels (which is equal to s=64, s = 1 ) in two ways: 2 s2 s= a) with normal Gaussian convolution and b) with the numerical implementation of the diffusion equation and Gaussian derivative calculation of the Laplacian. timestep= timestep= timestep= timestep= timestep= Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 25/29

26 Numerical examples shortening flow Same test image. By blurring the image the noise is gone, but the edge is gone too. MCM deblurs and keeps the edge. timestep= 2. timestep= timestep= timestep= timestep= timestep= timestep= timestep= timestep= 1.6 timestep= Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 26/29

27 The noise gradually disappears in this nonlinear scale-space evolution, while the edge strength is well preserved. Because the flux term, expressed in Gaussian derivatives, is rotation invariant, the edges are well preserved irrespective of their direction: this is edge-preserving smoothing. Original scale = 9 3D early embryogenesis image filtering by nonlinear partial differential equations Z. Krivá, et al. MedIA 14, , 2010 Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 27/29

28 Summary of Numerical Stability Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 28/29

29 Summary There is a strong analogy between curve evolution and PDE based schemes. They can be related directly to one another. Euclidean shortening flow (MCM) involves the diffusion to be limited to the direction perpendicular to the gradient only. The divergence of the flow in the equation is equal to the second order gauge derivative L vv with respect to v, the direction tangential to the isophote. Implementation with Gaussian derivatives may allow larger time steps. Many alternatives are possible with geometric reasoning. Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 29/29