Two Simple Nonlinear Edge Detectors

Transcription

1 Two Simple Nonlinear Edge Detectors Mladen Victor Wickerhauser Washington University in St. Louis, Missouri International Conference on Wavelet Analysis and Applications 28 May 2004, Chongqing, China Joint work with Wojciech Czaja Research supported in part by NSF grant DMS

2 Introduction Goal: Detect and characterize singularities in images by local dimensionality. Idea: Model images and functions as random vectors. Use an autocovariance matrix as a substitute for the Jacobian, or derivative. Application: Detect points (local dimension 0) and edges (local dimension 1) in images using the eigenvalues of the autocovariance matrix. 1

3 Prior Similar Ideas Old Observation: if a function is rough along some direction, then its local Fourier transform stays big along that direction. Duda and Hart [1972]: Hough transform for detection of lines and curves in images. Mallat and Zhong [1992]: edge detection from wavelet maxima. Aron and Kurz [1997]: detect lines and edges. linear hypothesis testing of variances in small windows to New View: recognize nonsmooth directions from eigenvalues of an autocovariance matrix accumulated from the localized Fourier transform. Use the eigenvector of the larger eigenvalue as the normal to the edge. 2

4 Probabilistic Motivation (2-D) Function φ : T C on the unit circle T R 2. Random direction vector Φ : T C 2 is Φ(θ) def = φ(θ)(cos θ, sin θ), θ T. Direction probability density is φ 2, normalized. Symmetric 2 2 Autocovariance Matrix 2π E(Φ Φ) ij = T Φ i Φ j = φ(θ) 2 τ i (θ)τ j (θ) dθ, 0 for i, j {1, 2}, τ 1 (θ) = sin θ, and τ 2 (θ) = cos θ. Then E(Φ Φ)v, v = T Φ, v 2 attain sup v =1 E(Φ Φ)v, v at a unit eigenvector v R 2 of the largest eigenvalue of E(Φ Φ). Such an eigenvector v always exists. Use v to approximate the max of φ = Φ. 3

5 Example (2-D) For f : R 2 C and large fixed R > 0, define φ(θ) = ˆf(R cos θ, R sin θ) 2 This samples ˆf 2 far out in the direction θ. Suppose φ is highly concentrated near the point θ 0 T. Then E(Φ Φ) is approximately proportional to ( cos 2 ) θ 0 cos θ 0 sin θ 0 cos θ 0 sin θ 0 sin 2. θ 0 Any nonzero vector in the θ 0 direction is an eigenvector of the largest eigenvalue. 4

6 Alternatives (A-1) Replace circle T with disc B: for i, j {1, 2}, and φ L 2 (B). E(Φ Φ) ij = B φ(ξ) 2 ξ i ξ j dξ, (A-2) Replace disk B with R 2, under additional integrability assumptions. 5

7 Localization Start with f : R 2 R, point of interest x, scale ɛ = 1/R. Localize by f gf, for some nonzero radial Schwartz function g : R 2 R: ( ) y x g ɛ (y) = g, ɛ > 0. ɛ which is: smooth, to avoid introducing new singularities; radial, to avoid introducing directional bias; nonzero and concentrated, to emphasize the point of interest x. 6

8 Dual Local Autocovariance Definition 1 The dual local autocovariance matrix of f at x is the 2 2 matrix whose ij-coefficient is: E ɛ,g (f; x) ij = B(0,1/ɛ) ξ iξ j ĝɛf(ξ) 2 dξ. This is the real, symmetric second moment matrix of the unnormalized probability density function ĝɛf 2. Localization gf is integrable are well defined. ĝf is bounded and continuous matrix coefficients f 0 near x E ɛ,g (f; x) is positive definite. 7

9 Straight Edges Change of variables: E ɛ,g (f; x) ij = B ξ iξ j ( g R 2 y + ɛ 1 ɛ ) x f(ɛy)e 2πiy ξ dy 2 dξ. If f is homogeneous of degree 0, and x = 0, this formula is independent of ɛ > 0 Example: f = 1 L, indicator function of left half-plane L = {(x 1, x 2 ) : x 1 0}. E ɛ,g (1 L ; 0) is a matrix with no ɛ-dependence. Use g(x) = exp( π x 2 ), for ease of computing Fourier transforms. Get dual local autocovariance matrix explicitly for the edge x 1 = 0. 8

10 Straight edges (continued... ) Lemma 1 E ɛ,g (1 L ; 0) has the eigensystem ( ) 1 λ 1, λ 2 0 satisfying λ 1 > λ 2 > 0, independently of ɛ. ( ) 0, 1 Numerical estimate: the matrix is diagonal, so λ i = B ξ2 i 0 2 e πξ2 2 e πx2 2πixξ 1 dx for i = 1, 2. In terms of the Gaussian integral G(ξ) = ξ 0 exp(πx2 ) dx, this becomes: [ ] 1 λ i = B ξ2 i e 2π ξ G2 (ξ 1 ) dξ, for i = 1, 2. dξ, By Mathematica, λ 1 λ

11 Off the Straight Edge Conversely, take g(ξ) to be a smooth radial function supported in B(0, 1), so: for any x / L, and 0 < ɛ < dist (L, x) small enough, since g ɛ 1 L = 0. E ɛ,g (1 L ; x) 0, for each point x IntL, E ɛ,g (1 L ; x) = const I, for all sufficiently small 0 < ɛ < dist (x, L). In either case, E has two equal eigenvalues. 10

12 Rotation and Translation Translation moves x, preserves eigenvectors: Lemma 2 Let T be a translation by x in R 2 : f T (y) = f(y + x). Then E ɛ,g (f; x) = E ɛ,g (f T ; 0). Rotation about x rotates eigenvectors, changes edge direction: Lemma 3 Let U be a rotation about x in R 2. Then E ɛ,g (f U; x) = U E ɛ,g (f; x) U 1. 11

13 Straight Edge Detection Let 1 H be the characteristic function of a half-plane H = {x R 2 : ν x β}, with nonzero normal vector ν R 2, for some constant β R. Its edge is the line H. Theorem 4 x H iff E ɛ,g (1 H ; x) has distinct eigenvalues λ 1 > λ 2 > 0 for every smooth, radial function g 0, and ɛ > 0. In that case, ν will be an eigenvector of the larger eigenvalue. Note; degree-0 homogeneity of 1 H eigenvalues of E ɛ,g (1 H ; x) do not depend on ɛ. For x / H off the edge, and each smooth, compactly-supported radial function g, E ɛ,g (1 H ; x) = const I has eventually constant and equal eigenvalues as ɛ 0. 12

14 Smooth-Curve Edges Idea: Boundaries of smooth domains look like boundaries of half-planes. Fix: Domain D R 2, smooth boundary D. 1 D, the characteristic function of D. g : R 2 R, a smooth radial function, supported in B = B(0, 1). H, half-plane with H D at some x. As ɛ 0, expect: x D distinct eigenvalues for E ɛ,g (1 D, x). x / D equal eigenvalues for E ɛ,g (1 D, x). 13

15 Smooth-Curve Edges (continued... ) Lemma 5 In any matrix norm, as ɛ 0. E ɛ,g (1 H ; x) E ɛ,g (1 D ; x) = O(ɛ), (Note: for Schwartz g, any δ > 0 gives as ɛ 0.) E ɛ,g (1 H ; x) E ɛ,g (1 D ; x) = O(ɛ 1 δ ), Theorem 6 For x D, and eigenvalues λ 1 (ɛ) and λ 2 (ɛ) of E ɛ,g (1 D ; x), we have For x / D, λ 1 (ɛ) λ 2 (ɛ) 0 as ɛ 0. lim inf ɛ 0+ λ 1(ɛ) λ 2 (ɛ) > 0. 14

16 Affine and Differentiable Functions At points x where f is differentiable, the eigenvalues λ 1 (ɛ), λ 2 (ɛ) of E ɛ,g (f; x) are equal in the limit: Lemma 7 Let A : R 2 R be an affine function: A(x) = a x + b for some constants a R 2, b R. Let g : R 2 R be a smooth radial function supported in B = B(0, 1). Then: lim [λ 1(ɛ) λ 2 (ɛ)] = 0. ɛ 0+ Lemma 8 If f : R 2 R is differentiable at x with affine A tangent to f at x, then as ɛ 0. E ɛ,g (f; x) E ɛ,g (A; x) = o(ɛ), (Continuously differentiable f near x satisfies as ɛ 0.) E ɛ,g (f; x) E ɛ,g (A; x) = O(ɛ 2 ), 15

17 Detecting Non-Edge Points Theorem 9 Suppose that f : R 2 R is differentiable at x. radial compactly-supported function g : R 2 R, Then for any smooth E ɛ,g (f; x) const I, as ɛ 0. Converse is false: lim ɛ 0+ E ɛ,g (f; x) = const I does not imply that f is differentiable at x, or even continuous. Symmetry can masquerade as smoothness, as in f = 1 x1,x 2 >0. Fix g(x) = exp( π x 2 ) as before. Then E ɛ,g (f; 0) = const I 0 for every ɛ > 0, so λ 1 (ɛ) = λ 2 (ɛ) = λ > 0 for every ɛ, even though f is discontinuous at 0. 16

18 Higher-Dimensional Theory Study the geometry of f : R p R d in high dimensions p, d. Suppose f = (f 1,..., f d ) is polynomially bounded. Fix x R p, Schwartz g : R p R d with radial components, and g ɛ (y) def = g ( ) y x, as before, for ɛ > 0. Definition 2 The dual local autocovariance matrix of f at x is the p p matrix: E ɛ,g (f; x) ij = for i, j {1,..., p}. ɛ ξ iξ j f g ɛ (ξ) 2 dξ, B(0,1/ɛ) Smooth p p case is like smooth 2 2: Theorem 10 Suppose that f : R p R d is differentiable at x. Then for any smooth radial function g : R p R d of compact support, the matrix E ɛ,g (f; x) const I, as ɛ 0. 17

19 Discretization Discrete Fourier transform on N real samples {f(n) : 0 n < N}: for integer k B N = [ N 2, N 2 ]. ˆf(k) = N 1 n=0 exp ( 2πi kn N ) f(n), If only the first q N samples of f are nonzero, then the sum is over {0, 1,..., q 1}. When f is real-valued, for k B N. ˆf(k) 2 = q 1 n,n =0 exp ( 2πi k(n n ) N ) f(n)f(n ), 18

20 Discretization (continued... ) The r-th moment of ˆf(k) 2 is q 1 n,n =0 k B N k r ˆf(k) 2 = f(n)f(n ) k r exp k B N ( 2πi k(n n ) N ). The innermost sum in k, if normalized with N r+1, is a Riemann sum for the integral evaluated at n (n n ): µ r (n) = 1 2 µ 0 (n) = 1 2 x r exp( 2πinx) dx, { 1, if n = 0, 0, otherwise; µ 1 (n) = µ 2 (n) = { 0, if n = 0, i ( 1)n 2πn, otherwise; { 1/12, if n = 0, ( 1) n 2π 2 n 2, otherwise. 19

21 Discrete Dual Local Autocovariance Image: f on {(m, n) : 0 m < M; 0 n < N}. Bump: g on {(m, n) : 0 m < p; 0 n < q}, for p M; q N. Autocovariance matrix: localize f gf, then compute E 11 = p 1 q 1 m,m =0 n=0 f(m, n)f(m, n)µ 2 (m m ); E 22 = p 1 q 1 m=0 n,n =0 f(m, n)f(m, n )µ 2 (n n ); E 12 = E 21 = p 1 q 1 m,m =0 n,n =0 f(m, n)f(m, n )µ 1 (m m )µ 1 (n n ). 20

22 Implementation in General Around each point z of the image, do: Localization. Extract pixel values on the square subgrid contained in z + [ ɛ, ɛ] 2. Multiply f(z + y) by the weighting bump function g(y). [O(ɛ 2 ) operations per pixel.] Dual autocovariance. Compute E = (E ij ), a real-valued, symmetric, positive semidefinite 2 2 matrix. [O(ɛ 4 ) operations per pixel.] Eigenvalues. For symmetric 2 2 matrices E: λ = 1 ( E 11 + E 22 ± (E 11 E 22 ) 2 + 4E where + gives λ 1, gives λ 2. [O(1) operations per pixel.] ), Edginess. Compute ratio λ 1 /λ 2 or difference λ 1 λ 2, or use reciprocal λ 2 /λ 1 [0, 1] in write-black mode so darker edgier. [O(1) operations per pixel.] 21

23 Special 7x7 Gridpoint Case Let z be of the form (m, n), a gridpoint with integer coordinates. Use ɛ = 3, to localize to 7 7 subgrids. Zero-pad at boundaries. Let g be the Gaussian bump g(y) = exp( π y 2 /ɛ 2 ). Use reciprocal edginess e = λ 2 /λ 1 [0, 1] in write-black mode so darker edgier. 22

24 Special 2x2 Midpoint Case Let z be of the form (m + 1 2, n ), a midpoint. Let ɛ = 1, so E is computed from the pixels immediately NE, SE, SW, and NW of z: (m, n + 1) (m + 1, n + 1) z (m, n) (m + 1, n) def = NW f(m, n + 1) SW def = f(m, n) [z] NE def = f(m + 1, n + 1) SE def = f(m + 1, n) Use eigenvalue difference for edginess: e def = λ 1 (ɛ, z) λ 2 (ɛ, z), All pixels are equidistant from z, so normalize g = 1 at that distance. 23

25 Special 2x2 Midpoint Case (continued... ) Evaluate µ 1 (0) = 0, µ 1 (±1) = 2πi ±1, µ 2(0) = 12 1,and µ 2(±1) = 1 2, so E 11 = (NW 2 + NE 2 )µ 2 (0 0) + (SW 2 + +SE 2 )µ 2 (1 1) + +(NW SW + NE SE) (µ 2 (0 1) + µ 2 (1 0)) = 1 ( NW 2 + NE 2 + SW 2 + SE 2 ) 1 (NW SW + NE SE) ; 12 π2 E 22 = 1 ( NW 2 + NE 2 + SW 2 + SE 2 ) 1 (NW NE + SW SE) ; 12 π2 E 11 E 22 = 1 (NW NE + SW SE NW SW NE SE) π2 = 1 (NW SE) (NE SW ) ; π2 E 12 = E 21 = 1 (NW SE NE SW ). 2π2 Edginess formula is a nonlinear four-tap filter with a square 2 2 pixel mask: e = λ 1 λ 2 = 1 π 2 (NW SE) 2 (NE SW ) 2 + (NW SE NE SW ) 2. 2π 24

26 Special 4+1 Centerpoint Case Let z be of the form (m, n). Let ɛ = 1, so E is computed from the central pixel at z = (m, n) and the 4 pixels N, E, S, and W of z: (m, n+1) N def = f(m, n+1) (m 1, n) z = (m, n) (m+1, n) def = W f(m 1, n) C def = f(m, n) E def = f(m+1, n) (m, n 1) S def = f(m, n 1) Use eigenvalue difference for edginess: e def = λ 1 (ɛ, z) λ 2 (ɛ, z), Use g with g(0) = 1 and g(1) = t > 0, so z gets 1 while N,E,S,W get t. 25

27 Special 4+1 Centerpoint Case (continued... ) Evaluate µ 1 (0) = 0, µ 1 (±1) = 2πi ±1, µ 1(±2) = 4π ±i, µ 2(0) = 12 1, µ 2(±1) = 1 2π2, and µ 2 (±2) = 1 8π 2: D def = 1 12 (C2 + t 2 [N 2 + E 2 + S 2 + W 2 ]); E 11 = D t t2 π2(n C + C S) + (N S) ; 4π2 E 22 = D t t2 π2(e C + C W ) + 4π 2 (E W ) ; E 12 = E 21 = t 2 2π 2 (N W N E + S E S W ) = t2 (N S) (E W ). 2π2 Edginess formule is a nonlinear five-tap filter with a mask shaped like a plus sign: λ 1 λ 2 = t [ π 2 C (E+W N S) + t 4 ] 2 (N S E W ) +t 2 (N S) 2 (E W ) 2. Each weight t (0, 1) tunes a different filter. Use t = 0.75 for no good reason. The choice t = 0 gives e = 0 everywhere. 26

28 Example Images 1. Geometrical figures, piecewise constant functions with jump discontinuities along rectifiable, mostly smooth curves. 2. Toy house demonstration image from the XHoughTool website. 3. Fingerprint, part of the FBI/NIST compliance test suite for WSQ compression. 4. Lena Sjøblum, the famous woman in a hat. 5. Cone, a ray-traced image by Craig Kolb. 6. Truck, one frame of video from Peng Li. 27

29 Example: Geometrical Figures 28

30 Geometrical Figures 7x7 Gridpoint Edginess 29

31 Geometrical Figures 2x2 Midpoint Edginess 30

32 Geometrical Figures 4+1 Centerpoint Edginess 31

33 Example: Toy House Image 32

34 Toy House 7x7 Gridpoint Edginess 33

35 Toy House 2x2 Midpoint Edginess 34

36 Toy House 4+1 Centerpoint Edginess 35

37 Example: Fingerprint Image 36

38 Fingerprint 7x7 Gridpoint Edginess 37

39 Fingerprint 2x2 Midpoint Edginess 38

40 Fingerprint 4+1 Centerpoint Edginess 39

41 Example: Lena Sjøblum Image 40

42 Lena 7x7 Gridpoint Edginess 41

43 Lena 2x2 Midpoint Edginess 42

44 Lena 4+1 Centerpoint Edginess 43

45 Example: Cone Image 44

46 Cone 7x7 Gridpoint Edginess 45

47 Cone 2x2 Midpoint Edginess 46

48 Cone 4+1 Centerpoint Edginess 47

49 Example: Truck Video Frame 48

50 Truck 7x7 Gridpoint Edginess 49

51 Truck 2x2 Midpoint Edginess 50

52 Truck 4+1 Centerpoint Edginess 51

53 Comparison: Toy House Image 7x7, 2x2, 4+1 Edginess 52

54 Comparison: Fingerprint Image 7x7, 2x2, 4+1 Edginess 53

55 Comparison: Lena Sjøblum Image 7x7, 2x2, 4+1 Edginess 54

56 Comparison: Cone Image 7x7, 2x2, 4+1 Edginess 55

57 Comparison: Truck Video Frame 7x7, 2x2, 4+1 Edginess 56