Image Analysis and Approximation via Generalized Polyharmonic Local Trigonometric Transforms

Transcription

1 Image Analysis and Approximation via Generalized Polyharmonic Local Trigonometric Transforms Naoki Saito with Jean-François Remy, Noel Smith, Ernest Woei, Katsu Yamatani, & Jim Zhao saito/ Department of Mathematics University of California, Davis Partially supported by ONR N , PECASE, and NSF DMS MGAWS1: Sep. 21, 2004 p.1

2 Outline Motivations: Issues of Localized Fourier Analysis Polyharmonic Local Sine Transform (PHLST) for Rectangular Domains Extension to 3D Extension to General Shape Domain Summary and Future Plan MGAWS1: Sep. 21, 2004 p.2

3 Motivations Want sparse data representation (compression) = Fast decay of the expansion coefficients Want a data representation as less statistically dependent among blocks as possible = No overlaps among blocks Want to develop a local signal analysis tool that can distinguish intrinsic singularities from the artificial discontinuities created by local windowing = many potential applications Want to efficiently represent regions of more general shapes other than rectangular blocks MGAWS1: Sep. 21, 2004 p.3

4 Life on an Interval (or a Domain) If data are periodic and smooth, the Fourier basis is very efficient: Fast decay of the Fourier coefficients Smoothness estimate (e.g., the Lipschitz/Hölder exponents) However, most signals and images of interest have compact supports, and are neither periodic nor smooth. What to do? MGAWS1: Sep. 21, 2004 p.4

5 Life on an Interval (or a Domain)... Design special wavelets ( wavelets on intervals ); Use multiwavelets of Alpert and Rokhlin (segmented orthogonal polynomials); Use Prolate Spheroidal Wave Functions ; Use the Continuous Boundary Local Fourier Transform (CBLFT) that periodizes the signal nicely and expand it into a periodic basis; or Use the Polyharmonic Local Trigonometric Transform (PHLTT) and its generalizations MGAWS1: Sep. 21, 2004 p.5

6 Basic Ideas of PHLTT Split the domain Ω into a set of subregions (often rectangular blocks) {Ω j } j and cut the data/function f into pieces using the characteristic functions, f j = χ Ωj f MGAWS1: Sep. 21, 2004 p.6

7 Basic Ideas of PHLTT Split the domain Ω into a set of subregions (often rectangular blocks) {Ω j } j and cut the data/function f into pieces using the characteristic functions, f j = χ Ωj f f j is decomposed into two parts u j + v j MGAWS1: Sep. 21, 2004 p.6

8 Basic Ideas of PHLTT Split the domain Ω into a set of subregions (often rectangular blocks) {Ω j } j and cut the data/function f into pieces using the characteristic functions, f j = χ Ωj f f j is decomposed into two parts u j + v j u j is a solution of the polyharmonic equations m u = 0 with Dirichlet/Neumann boundary conditions MGAWS1: Sep. 21, 2004 p.6

9 Basic Ideas of PHLTT Split the domain Ω into a set of subregions (often rectangular blocks) {Ω j } j and cut the data/function f into pieces using the characteristic functions, f j = χ Ωj f f j is decomposed into two parts u j + v j u j is a solution of the polyharmonic equations m u = 0 with Dirichlet/Neumann boundary conditions v j is a residual f j u j, which are expanded into the multiple Fourier series (either complex exponentials, sines, or cosines) MGAWS1: Sep. 21, 2004 p.6

10 Basic Ideas of PHLTT Split the domain Ω into a set of subregions (often rectangular blocks) {Ω j } j and cut the data/function f into pieces using the characteristic functions, f j = χ Ωj f f j is decomposed into two parts u j + v j u j is a solution of the polyharmonic equations m u = 0 with Dirichlet/Neumann boundary conditions v j is a residual f j u j, which are expanded into the multiple Fourier series (either complex exponentials, sines, or cosines) Sines and cosines are eigenfunctions of the Laplacian on a rectangular box MGAWS1: Sep. 21, 2004 p.6

11 1D Case In 1D, each u j is a low order algebraic polynomial (e.g., line, cubic poly.) MGAWS1: Sep. 21, 2004 p.7

12 1D Case In 1D, each u j is a low order algebraic polynomial (e.g., line, cubic poly.) v j is a trigonometric polynomial MGAWS1: Sep. 21, 2004 p.7

13 1D Case In 1D, each u j is a low order algebraic polynomial (e.g., line, cubic poly.) v j is a trigonometric polynomial These two compensate the shortcomings of each other: MGAWS1: Sep. 21, 2004 p.7

14 1D Case In 1D, each u j is a low order algebraic polynomial (e.g., line, cubic poly.) v j is a trigonometric polynomial These two compensate the shortcomings of each other: High order algebraic polynomial = Runge phenomenon MGAWS1: Sep. 21, 2004 p.7

15 1D Case In 1D, each u j is a low order algebraic polynomial (e.g., line, cubic poly.) v j is a trigonometric polynomial These two compensate the shortcomings of each other: High order algebraic polynomial = Runge phenomenon Trigonometric polynomial on an interval = Gibbs phenomenon MGAWS1: Sep. 21, 2004 p.7

16 1D Example: Compression Ratio MGAWS1: Sep. 21, 2004 p.8

17 Polyharmonic Global Sine Transform on a Rectangle Consider a function f C 2m (Ω), where m = 1, 2,..., and Ω R n (e.g., Ω = [0, 1] n ), but not periodic. MGAWS1: Sep. 21, 2004 p.9

18 Polyharmonic Global Sine Transform on a Rectangle Consider a function f C 2m (Ω), where m = 1, 2,..., and Ω R n (e.g., Ω = [0, 1] n ), but not periodic. Decompose this function into the following two components: f(x) = u(x) + v(x), MGAWS1: Sep. 21, 2004 p.9

19 Polyharmonic Global Sine Transform on a Rectangle Consider a function f C 2m (Ω), where m = 1, 2,..., and Ω R n (e.g., Ω = [0, 1] n ), but not periodic. Decompose this function into the following two components: f(x) = u(x) + v(x), u(x) satisfies the following polyharmonic equation: m u = 0 in Ω. MGAWS1: Sep. 21, 2004 p.9

20 PHGST on a Rectangle... The boundary condition for u is: p l u ν p l = p l f ν p l on Γ = Ω, l = 0,..., m 1, where p l is the order of the normal derivatives to be specified (p 0 0 u = f on Γ). MGAWS1: Sep. 21, 2004 p.10

21 PHGST on a Rectangle... The boundary condition for u is: p l u ν p l = p l f ν p l on Γ = Ω, l = 0,..., m 1, where p l is the order of the normal derivatives to be specified (p 0 0 u = f on Γ). Now set v(x) = f(x) u(x), which we will call the residual component with p l v ν p l = 0 on Γ, l = 0,..., m 1. MGAWS1: Sep. 21, 2004 p.10

22 A Specific Example: m = 1 (Laplace) Case u = 0 in Ω, u = f on Γ. Variational formulation = minimum gradient interpolation: u 2 dx subject to the above boundary condition. min u H 1 (Ω) Ω Note that in 1D, this is simply a line. MGAWS1: Sep. 21, 2004 p.11

23 A Specific Example: m = 2 (Biharmonic) Case 2 u = 0 in Ω, 2 u u = f, ν = 2 f on Γ. 2 ν 2 Variational formulation = minimum curvature interpolation: min u H 2 (Ω) Ω ( u + 2 j k subject to the above boundary condition. j k u) 2 dx, Note that in 1D, this is simply a cubic polynomial. MGAWS1: Sep. 21, 2004 p.12

24 PHGST on a Rectangle... The polyharmonic component u is smooth inside the domain Ω, and its values (and possibly its normal derivatives) matches those of data. MGAWS1: Sep. 21, 2004 p.13

25 PHGST on a Rectangle... The polyharmonic component u is smooth inside the domain Ω, and its values (and possibly its normal derivatives) matches those of data. The u component can be represented only by the boundary values f Γ. = No need to store the whole u. MGAWS1: Sep. 21, 2004 p.13

26 PHGST on a Rectangle... The polyharmonic component u is smooth inside the domain Ω, and its values (and possibly its normal derivatives) matches those of data. The u component can be represented only by the boundary values f Γ. = No need to store the whole u. The residual component v becomes 0 at the boundary Γ. MGAWS1: Sep. 21, 2004 p.13

27 PHGST on a Rectangle... Therefore, the v component is suitable for Fourier analysis. In fact, if Ω = [0, 1] n and p l = 2l, l = 0,..., m 1, then the Fourier sine analysis should be used to get the matching normal derivatives up to order 2m 1 by odd reflection at the boundaries. MGAWS1: Sep. 21, 2004 p.14

28 PHGST on a Rectangle... Therefore, the v component is suitable for Fourier analysis. In fact, if Ω = [0, 1] n and p l = 2l, l = 0,..., m 1, then the Fourier sine analysis should be used to get the matching normal derivatives up to order 2m 1 by odd reflection at the boundaries. The frequency content (in particular, mid to high frequency range) of the original is retained in the residual = textures remain in v; shading is captured by u. MGAWS1: Sep. 21, 2004 p.14

29 PHGST on a Rectangle... Therefore, the v component is suitable for Fourier analysis. In fact, if Ω = [0, 1] n and p l = 2l, l = 0,..., m 1, then the Fourier sine analysis should be used to get the matching normal derivatives up to order 2m 1 by odd reflection at the boundaries. The frequency content (in particular, mid to high frequency range) of the original is retained in the residual = textures remain in v; shading is captured by u. We can get the decay rate as follows: MGAWS1: Sep. 21, 2004 p.14

30 Theorem Let Ω = [0, 1] n, and f C 2m (Ω), but non-periodic. Assume further that i 2m+1 f, i = 1,..., n, exist and are of bounded variation. Furthermore, let f = u + v be the PHLST representation where the polyharmonic component u is the solution of the polyharmonic equation of order m with the boundary condition 2l u ν 2l = 2l f ν 2l on Γ, l = 0,..., m 1. Then, the Fourier sine coefficient b k of the residual v is of O ( k 2m 1) for all k 0, where k = (k 1,..., k n ), and k is the usual Euclidean (i.e., l 2 ) norm of k. MGAWS1: Sep. 21, 2004 p.15

31 Polyharmonic Local Sine Transform Now, consider a decomposition of Ω into a disjoint set of subdomains {Ω j }, i.e., Ω = J j=1ω j. A typical example is Ω = (0, 1) n, and Ω j is a dyadic subcube. Then, restrict f on Ω j, i.e., for each j, we decompose f locally as follows: fχ Ωj = f j = u j + v j, where we follow the same recipe locally as in the global case. We call this decomposition of f into {u j, v j } Polyharmonic Local Sine Transform (PHLST). For m = 1: Laplace Local Sine Transform (LLST); For m = 2: Biharmonic Local Sine Transform (BLST). MGAWS1: Sep. 21, 2004 p.16

32 Polyharmonic Local Sine Transform (a) Original MGAWS1: Sep. 21, 2004 p.17

33 Polyharmonic Local Sine Transform (a) Original (b) j u j MGAWS1: Sep. 21, 2004 p.17

34 Polyharmonic Local Sine Transform (a) Original (b) j u j (c) j v j MGAWS1: Sep. 21, 2004 p.17

35 Polyharmonic Local Sine Transform... No spatial overlaps MGAWS1: Sep. 21, 2004 p.18

36 Polyharmonic Local Sine Transform... No spatial overlaps Decay of the Fourier sine coefficients are fast if Ω j does not contain any singularity MGAWS1: Sep. 21, 2004 p.18

37 Polyharmonic Local Sine Transform... No spatial overlaps Decay of the Fourier sine coefficients are fast if Ω j does not contain any singularity Can distinguish intrinsic singularities from the artificial discontinuities at Γ j imposed by local windowing MGAWS1: Sep. 21, 2004 p.18

38 Remarks on the Laplace Solver A fast and accurate (no finite difference approximation of the Laplace operator) algorithm (for both 2D and 3D) exists via FFT if Ω and Ω j are dyadic cubes (Averbuch, Braverman, Israeli, and Vozovoi, 1998) MGAWS1: Sep. 21, 2004 p.19

39 Remarks on the Laplace Solver A fast and accurate (no finite difference approximation of the Laplace operator) algorithm (for both 2D and 3D) exists via FFT if Ω and Ω j are dyadic cubes (Averbuch, Braverman, Israeli, and Vozovoi, 1998) The u component can be computed as: u(x, y) = p(x, y) + k 1 where... {b (1) k h k(x, 1 y) + b (2) k h k(y, 1 x) + b (3) k h k(x, y) + b (4) k h k(y, x)}, MGAWS1: Sep. 21, 2004 p.19

40 Remarks on the Laplace Solver p(x, y) is a harmonic polynomial that agrees with f(x, y) at the four corner points of the domain, e.g., p(x, y) = a 3 xy + a 2 x + a 1 y + a 0, MGAWS1: Sep. 21, 2004 p.20

41 Remarks on the Laplace Solver p(x, y) is a harmonic polynomial that agrees with f(x, y) at the four corner points of the domain, e.g., h k (x, y) is defined as: p(x, y) = a 3 xy + a 2 x + a 1 y + a 0, h k (x, y) = sin(πkx) sinh(πky) sinh(πk), where b (j) k are the kth 1D Fourier sine coefficients of the boundary data of f p. MGAWS1: Sep. 21, 2004 p.20

42 Remarks on the Laplace Solver p(x, y) is a harmonic polynomial that agrees with f(x, y) at the four corner points of the domain, e.g., h k (x, y) is defined as: p(x, y) = a 3 xy + a 2 x + a 1 y + a 0, h k (x, y) = sin(πkx) sinh(πky) sinh(πk), where b (j) k are the kth 1D Fourier sine coefficients of the boundary data of f p. The computational cost of u is O(4N 2 log N). MGAWS1: Sep. 21, 2004 p.20

43 Remarks on PHLST Need to store the boundary values = can compress them using the lower dimensional version of PHLST MGAWS1: Sep. 21, 2004 p.21

44 Remarks on PHLST Need to store the boundary values = can compress them using the lower dimensional version of PHLST Can use complex exponentials, wavelets, etc., instead of sines with potentially slower decay MGAWS1: Sep. 21, 2004 p.21

45 Remarks on PHLST Need to store the boundary values = can compress them using the lower dimensional version of PHLST Can use complex exponentials, wavelets, etc., instead of sines with potentially slower decay Can do in the frequency domain = better wavelet packets MGAWS1: Sep. 21, 2004 p.21

46 Remarks on PHLST Need to store the boundary values = can compress them using the lower dimensional version of PHLST Can use complex exponentials, wavelets, etc., instead of sines with potentially slower decay Can do in the frequency domain = better wavelet packets Can be generalized to other geometries (e.g., discs, spheres, star shapes, general smooth boundaries) MGAWS1: Sep. 21, 2004 p.21

47 Remarks on PHLST Need to store the boundary values = can compress them using the lower dimensional version of PHLST Can use complex exponentials, wavelets, etc., instead of sines with potentially slower decay Can do in the frequency domain = better wavelet packets Can be generalized to other geometries (e.g., discs, spheres, star shapes, general smooth boundaries) Useful for interpolation and local feature computation (e.g., gradients, directional derivatives, etc.) MGAWS1: Sep. 21, 2004 p.21

48 f = u + v Models: Interpretation of PHLTT f = u + v models of Yves Meyer (2001): MGAWS1: Sep. 21, 2004 p.22

49 f = u + v Models: Interpretation of PHLTT f = u + v models of Yves Meyer (2001): u objects (e.g., BV (Ω)) MGAWS1: Sep. 21, 2004 p.22

50 f = u + v Models: Interpretation of PHLTT f = u + v models of Yves Meyer (2001): u objects (e.g., BV (Ω)) v textures and noise MGAWS1: Sep. 21, 2004 p.22

51 f = u + v Models: Interpretation of PHLTT f = u + v models of Yves Meyer (2001): u objects (e.g., BV (Ω)) v textures and noise In PHLTT, the polyharmonic component u is what is predictable from only using the boundary behavior of a function. MGAWS1: Sep. 21, 2004 p.22

52 f = u + v Models: Interpretation of PHLTT f = u + v models of Yves Meyer (2001): u objects (e.g., BV (Ω)) v textures and noise In PHLTT, the polyharmonic component u is what is predictable from only using the boundary behavior of a function. The residual component v is what is unpredictable using the boundary information. MGAWS1: Sep. 21, 2004 p.22

53 f = u + v Models: Interpretation of PHLTT f = u + v models of Yves Meyer (2001): u objects (e.g., BV (Ω)) v textures and noise In PHLTT, the polyharmonic component u is what is predictable from only using the boundary behavior of a function. The residual component v is what is unpredictable using the boundary information. Yet another interpretation is: u: trend, v: fluctuation. MGAWS1: Sep. 21, 2004 p.22

54 f = u + v Models: Interpretation of PHLTT... Many possible ideas here once we view this as predicition from the available data: MGAWS1: Sep. 21, 2004 p.23

55 f = u + v Models: Interpretation of PHLTT... Many possible ideas here once we view this as predicition from the available data: A. Cohen et al. uses inpainting algorithm for predicting u components MGAWS1: Sep. 21, 2004 p.23

56 f = u + v Models: Interpretation of PHLTT... Many possible ideas here once we view this as predicition from the available data: A. Cohen et al. uses inpainting algorithm for predicting u components Relationship with geometric harmonic analysis of Coifman & Lafon MGAWS1: Sep. 21, 2004 p.23

57 f = u + v Models: Interpretation of PHLTT... Many possible ideas here once we view this as predicition from the available data: A. Cohen et al. uses inpainting algorithm for predicting u components Relationship with geometric harmonic analysis of Coifman & Lafon Other inverse boundary problems... MGAWS1: Sep. 21, 2004 p.23

58 LLST Approximation Experiment Comparison with DCT, the fixed folding local cosine (LCT), and Wavelet (Coiflet 12). MGAWS1: Sep. 21, 2004 p.24

59 LLST Approximation Experiment Comparison with DCT, the fixed folding local cosine (LCT), and Wavelet (Coiflet 12). Split into a set of squares of pixels or pixels MGAWS1: Sep. 21, 2004 p.24

60 LLST Approximation Experiment Comparison with DCT, the fixed folding local cosine (LCT), and Wavelet (Coiflet 12). Split into a set of squares of pixels or pixels Select the largest 1% and 5% of coefficients. MGAWS1: Sep. 21, 2004 p.24

61 LLST Approximation Experiment Comparison with DCT, the fixed folding local cosine (LCT), and Wavelet (Coiflet 12). Split into a set of squares of pixels or pixels Select the largest 1% and 5% of coefficients. Compression using quantization is in progress. MGAWS1: Sep. 21, 2004 p.24

62 LLST Approximation Experiment: Top 1% (a) Original (b) DCT (c) LCT (d) C12 (e) LLST MGAWS1: Sep. 21, 2004 p.25

63 Zoomed up images: Top 1% (a) Original (b) DCT (c) LCT (d) C12 (e) LLST MGAWS1: Sep. 21, 2004 p.26

64 LLST Approximation Experiment: Top 5% (a) Original (b) DCT (c) LCT (d) C12 (e) LLST MGAWS1: Sep. 21, 2004 p.27

65 Zoomed up images: Top 5% (a) Original (b) DCT (c) LCT (d) C12 (e) LLST MGAWS1: Sep. 21, 2004 p.28

66 Approximation Test: Smooth Function PSNR (db) LLST: J=1 DCT: J=1 LCT: J=1 Daub6: J=3 Symm8: J= (a) Original Ratio of retained coefficients (%) (b) PSNR MGAWS1: Sep. 21, 2004 p.29

67 Approximation Test: Piecewise Smooth Function PSNR (db) LLST: J=1 DCT: J=1 LCT: J=1 Daub6: J=3 Symm8: J= (a) Original Ratio of retained coefficients (%) (b) PSNR MGAWS1: Sep. 21, 2004 p.30

68 Approximation Test: Oscillatory Function LLST: J=1 DCT: J=1 LCT: J=1 Daub6: J=3 Symm8: J= PSNR (db) (a) Original Ratio of retained coefficients (%) (b) PSNR MGAWS1: Sep. 21, 2004 p.31

69 Approximation Test: Oscillatory Function with Discontinuity LLST: J=1 DCT: J=1 LCT: J=1 Daub6: J=3 Symm8: J= PSNR (db) (a) Original Ratio of retained coefficients (%) (b) PSNR MGAWS1: Sep. 21, 2004 p.32

70 Extension to 3D Boundary data contains more information compared to 1D and 2D MGAWS1: Sep. 21, 2004 p.33

71 Extension to 3D Boundary data contains more information compared to 1D and 2D Fast 3D Laplace solver exists (the ABIV algorithm with our new organization of data): O(12N 3 log N) MGAWS1: Sep. 21, 2004 p.33

72 Extension to 3D Boundary data contains more information compared to 1D and 2D Fast 3D Laplace solver exists (the ABIV algorithm with our new organization of data): O(12N 3 log N) Nicely recursive: corners; edges; faces; body contents MGAWS1: Sep. 21, 2004 p.33

73 Extension to 3D Boundary data contains more information compared to 1D and 2D Fast 3D Laplace solver exists (the ABIV algorithm with our new organization of data): O(12N 3 log N) Nicely recursive: corners; edges; faces; body contents Many possibilities: hierarchical decomposition, parallelization, 3D feature extraction,... MGAWS1: Sep. 21, 2004 p.33

74 Extension to 3D Boundary data contains more information compared to 1D and 2D Fast 3D Laplace solver exists (the ABIV algorithm with our new organization of data): O(12N 3 log N) Nicely recursive: corners; edges; faces; body contents Many possibilities: hierarchical decomposition, parallelization, 3D feature extraction,... Promising applications include analysis and compression of 3D seismic, medical, and video data,... MGAWS1: Sep. 21, 2004 p.33

75 Extension to General Shape Domains Consider a general-shaped domain Ω R n. MGAWS1: Sep. 21, 2004 p.34

76 Extension to General Shape Domains Consider a general-shaped domain Ω R n. Want to analyze the frequency information inside of the object. MGAWS1: Sep. 21, 2004 p.34

77 Extension to General Shape Domains Consider a general-shaped domain Ω R n. Want to analyze the frequency information inside of the object. Want to avoid the Gibbs phenomenon due to Γ. MGAWS1: Sep. 21, 2004 p.34

78 Extension to General Shape Domains Consider a general-shaped domain Ω R n. Want to analyze the frequency information inside of the object. Want to avoid the Gibbs phenomenon due to Γ. = Smoothly extend the function defined on Ω to the outside! MGAWS1: Sep. 21, 2004 p.34

79 (c) Object (d) Anomalies MGAWS1: Sep. 21, 2004 p.35 Object-Oriented Image Analysis (a) Original (b) Background

80 Object-Oriented Image Analysis... Suppose its boundary Γ is already detected explicitly. MGAWS1: Sep. 21, 2004 p.36

81 Object-Oriented Image Analysis... Suppose its boundary Γ is already detected explicitly. Let f C 2 (Ω) C(Ω). MGAWS1: Sep. 21, 2004 p.36

82 Object-Oriented Image Analysis... Suppose its boundary Γ is already detected explicitly. Let f C 2 (Ω) C(Ω). Let S be a rectangular domain containing Ω. MGAWS1: Sep. 21, 2004 p.36

83 Object-Oriented Image Analysis... Suppose its boundary Γ is already detected explicitly. Let f C 2 (Ω) C(Ω). Let S be a rectangular domain containing Ω. Then, we will find a smooth extension ṽ to the entire S from Ω and the harmonic function u in Ω: f in Ω, ṽ = 0 in S\Ω, ṽ = 0 on S. MGAWS1: Sep. 21, 2004 p.36

84 Motivations for ṽ ṽ is essentially the same as f in Ω modulo some harmonic function (in fact, this harmonic function becomes the u component); MGAWS1: Sep. 21, 2004 p.37

85 Motivations for ṽ ṽ is essentially the same as f in Ω modulo some harmonic function (in fact, this harmonic function becomes the u component); ṽ is harmonic (i.e., very smooth) in S\Ω (outside of Ω); MGAWS1: Sep. 21, 2004 p.37

86 Motivations for ṽ ṽ is essentially the same as f in Ω modulo some harmonic function (in fact, this harmonic function becomes the u component); ṽ is harmonic (i.e., very smooth) in S\Ω (outside of Ω); can show ṽ C 1 (S) MGAWS1: Sep. 21, 2004 p.37

87 Motivations for ṽ ṽ is essentially the same as f in Ω modulo some harmonic function (in fact, this harmonic function becomes the u component); ṽ is harmonic (i.e., very smooth) in S\Ω (outside of Ω); can show ṽ C 1 (S) can also show ṽ S\Γ C 2 (S\Γ) MGAWS1: Sep. 21, 2004 p.37

88 Motivations for ṽ ṽ is essentially the same as f in Ω modulo some harmonic function (in fact, this harmonic function becomes the u component); ṽ is harmonic (i.e., very smooth) in S\Ω (outside of Ω); can show ṽ C 1 (S) can also show ṽ S\Γ C 2 (S\Γ) ṽ S = 0 = suitable for Fourier sine series expansion, with decay rate O( k 3 ). MGAWS1: Sep. 21, 2004 p.37

89 How to compute ṽ Introduce the potential function: P (x) = f(y)φ(x, y) dy for x R n. Ω where Φ(x, y) = 1 2π log x y if n = 2, x y 2 n (2 n)ω n if n > 2, is the so-called fundamental solution of the Laplace equation. MGAWS1: Sep. 21, 2004 p.38

90 How to compute ṽ... Can show that P satisfies the above Poisson equation: f in Ω, P = 0 in S\Ω, MGAWS1: Sep. 21, 2004 p.39

91 How to compute ṽ... Can show that P satisfies the above Poisson equation: f in Ω, P = 0 in S\Ω, But the boundary condition at the outside box is not satisfied, i.e., P S 0. MGAWS1: Sep. 21, 2004 p.39

92 How to compute ṽ... Introduce now a boundary correction function Q: Q = 0 in S, Q = P on S; MGAWS1: Sep. 21, 2004 p.40

93 How to compute ṽ... Introduce now a boundary correction function Q: Q = 0 in S, Q = P on S; Finally set ṽ(x) = P (x) + Q(x), which satisfies the desired conditions. MGAWS1: Sep. 21, 2004 p.40

94 How to compute ṽ... Can write the solution via single layer and double layer potentials as: P (x) = χ Ω (x)f(x)+ Γ ( f ν ) y) (y)φ(x, y) f(y) Φ(x, ν y ds(y), where χ Ω (x) = 1 if x Ω; 1/2 if x Γ; 0 if x R n \Ω. MGAWS1: Sep. 21, 2004 p.41

95 Object-Oriented Image Analysis... The u component satisfies: u = 0 in Ω, u = f ṽ on Γ. MGAWS1: Sep. 21, 2004 p.42

96 Object-Oriented Image Analysis... The u component satisfies: u = 0 in Ω, u = f ṽ on Γ. u can be recovered from f Γ, f, and ṽ via Fast ν Γ Multipole Method (just potential evaluation, no need to solve the Laplace equation). MGAWS1: Sep. 21, 2004 p.42

97 1D Example 4 3 f=u+χ I v f u v x MGAWS1: Sep. 21, 2004 p.43

98 2D Example (a) Orig.1 MGAWS1: Sep. 21, 2004 p.44

99 2D Example (a) Orig.1 (b) Orig.2 MGAWS1: Sep. 21, 2004 p.44

100 2D Example (a) Orig.1 (b) Orig (c) Ext.1 MGAWS1: Sep. 21, 2004 p.44

101 2D Example (a) Orig.1 (b) Orig (c) Ext.1 (d) Ext.2 MGAWS1: Sep. 21, 2004 p.44

102 2D Example (a) Original MGAWS1: Sep. 21, 2004 p.45

103 2D Example (a) Original (b) u MGAWS1: Sep. 21, 2004 p.45

104 2D Example (a) Original (b) u (c) ṽ MGAWS1: Sep. 21, 2004 p.45

105 The Fourier Transform Magnitudes (log-scale) (a) Original MGAWS1: Sep. 21, 2004 p.46

106 The Fourier Transform Magnitudes (log-scale) (a) Original (b) ṽ MGAWS1: Sep. 21, 2004 p.46

107 Another example MGAWS1: Sep. 21, 2004 p.47

108 Another example (a) Extracted shell (b) Extended (c) u component (d) ṽ component MGAWS1: Sep. 21, 2004 p.48

109 Related Previous Work Whitney (1934): Smooth extensions of C m (Ω) functions in R n, and the Whitney decomposition Zygmund (1935): Removable discontinuities Lanczos (1938,1966): Trigonometric interpolation with linear/polynomial component removal Kantrovich & Krylov (1958): Rapidly convergent trigonometric series by polynomial removal Briggs (1974): Minimum curvature surface interpolation Grimson (1981): Visual surface reconstruction Terzopoulos (1983): Multilevel visual surface reconstruction MGAWS1: Sep. 21, 2004 p.49

110 Related Previous Work... Leclerc (1989): Image segmentation using MDL and local polynomials Wahba (1990): Variational formulation/multivariate splines Dimitrov (1996): Polyharmonic functions for quadratures Kounchev ( ): Theory of polysplines Casas & Torres (1996): Two-stage coding Caselles, Morel, & Sbert (1998): Axiomatic interpolation All of them except Lanczos mainly focus on the u-component. Lanczos explored neither higher dimensions nor multiscale setting. MGAWS1: Sep. 21, 2004 p.50

111 Conclusions PHLTT and its generalizations: Provide a compact image/data representation scheme MGAWS1: Sep. 21, 2004 p.51

112 Conclusions PHLTT and its generalizations: Provide a compact image/data representation scheme Proactive use of boundary information MGAWS1: Sep. 21, 2004 p.51

113 Conclusions PHLTT and its generalizations: Provide a compact image/data representation scheme Proactive use of boundary information Can get faster decay expansion coefficients MGAWS1: Sep. 21, 2004 p.51

114 Conclusions PHLTT and its generalizations: Provide a compact image/data representation scheme Proactive use of boundary information Can get faster decay expansion coefficients Allow object-oriented image analysis & synthesis MGAWS1: Sep. 21, 2004 p.51

115 Conclusions PHLTT and its generalizations: Provide a compact image/data representation scheme Proactive use of boundary information Can get faster decay expansion coefficients Allow object-oriented image analysis & synthesis Variety of applications: segmentation, compression, interpolation, local feature computation, 3D,... MGAWS1: Sep. 21, 2004 p.51

116 Comments This general area is a meeting ground of image processing, harmonic analysis, PDEs, and shape optimization MGAWS1: Sep. 21, 2004 p.52

117 Comments This general area is a meeting ground of image processing, harmonic analysis, PDEs, and shape optimization A large possibility for prediction operators from boundary MGAWS1: Sep. 21, 2004 p.52

118 Comments This general area is a meeting ground of image processing, harmonic analysis, PDEs, and shape optimization A large possibility for prediction operators from boundary Anisotropy and many interesting issues MGAWS1: Sep. 21, 2004 p.52

119 Comments This general area is a meeting ground of image processing, harmonic analysis, PDEs, and shape optimization A large possibility for prediction operators from boundary Anisotropy and many interesting issues Reliability of boundary information and noise is an issue MGAWS1: Sep. 21, 2004 p.52

120 References Check saito/publications/ periodically for more preprints on this work. For preliminary work, see: N. Saito & J.-F. Remy: A new local sine transform without overlaps: A combination of computational harmonic analysis and PDE, in Wavelets X (M. A. Unser, A. Aldroubi, & A. F. Laine, eds.), Proc. SPIE 5207, pp , See also references therein. Thank you very much for your attention! MGAWS1: Sep. 21, 2004 p.53