Use of Laplacian Eigenfunctions and Eigenvalues for Analyzing Data on a Domain of Complicated Shape

Transcription

1 Use of Laplacian Eigenfunctions and Eigenvalues for Analyzing Data on a Domain of Complicated Shape Naoki Saito 1 Department of Mathematics University of California, Davis May 23, Partially supported by grants from National Science Foundation and Office of Naval Research. I thank Leo Chalupa (UCD, Neurobiology), Raphy Coifman (Yale), Julie Coombs (UCD, Neurobiology), Dave Donoho (Stanford), John Hunter (UCD), Peter Jones (Yale), Linh Lieu (UCD), Stan Osher (UCLA), Allen Xue (UCD), Katsu Yamatani (Meijo Univ.) for the fruitful discussions. saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 1 / 68

2 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 2 / 68

3 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 3 / 68

4 Motivations Consider a bounded domain of general (may be quite complicated) shape Ω R d. Want to analyze the spatial frequency information inside of the object defined in Ω = need to avoid the Gibbs phenomenon due to Γ = Ω. Want to represent the object information efficiently for analysis, interpretation, discrimination, etc. = fast decaying expansion coefficients relative to a meaningful basis. Want to extract geometric information about the domain Ω = shape clustering/classification. saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 4 / 68

5 Motivations...Data Analysis on a Complicated Domain saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 5 / 68

6 Motivations...Clustering Complicated Objects µ m µ m saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 6 / 68

7 Motivations... Clustering Complicated Objects... (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 7 / 68

8 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 8 / 68

9 Eigenfunctions of Laplacian Our previous attempt was to extend the object to the outside smoothly and then bound it nicely with a rectangular box followed by the ordinary Fourier analysis. Why not analyze (and synthesize) the object using genuine basis functions tailored to the domain? After all, sines (and cosines) are the eigenfunctions of the Laplacian on the rectangular domain with Dirichlet (and Neumann) boundary condition. Spherical harmonics, Bessel functions, and Prolate Spheroidal Wave Functions, are part of the eigenfunctions of the Laplacian (via separation of variables) for the spherical, cylindrical, and spheroidal domains, respectively. saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 9 / 68

10 Eigenfunctions of Laplacian... Consider an operator L = in L 2 (Ω) with appropriate boundary condition. Analysis of L is difficult due to unboundedness, etc. Much better to analyze its inverse, i.e., the Green s operator because it is compact and self-adjoint. Thus L 1 has discrete spectra (i.e., a countable number of eigenvalues with finite multiplicity) except 0 spectrum. L has a complete orthonormal basis of L 2 (Ω), and this allows us to do eigenfunction expansion in L 2 (Ω). saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 10 / 68

11 Eigenfunctions of Laplacian...Difficulties The key difficulty is to compute such eigenfunctions; directly solving the Helmholtz equation (or eigenvalue problem) on a general domain is tough. Unfortunately, computing the Green s function for a general Ω satisfying the usual boundary condition (i.e., Dirichlet, Neumann) is also very difficult. saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 11 / 68

12 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 12 / 68

13 Integral Operators Commuting with Laplacian The key idea is to find an integral operator commuting with the Laplacian without imposing the strict boundary condition a priori. Then, we know that the eigenfunctions of the Laplacian is the same as those of the integral operator, which is easier to deal with, due to the following Theorem (G. Frobenius 1878?; B. Friedman 1956) Suppose K and L commute and one of them has an eigenvalue with finite multiplicity. Then, K and L share the same eigenfunction corresponding to that eigenvalue. That is, Lϕ = λϕ and Kϕ = µϕ. saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 13 / 68

14 Integral Operators Commuting with Laplacian... Let s replace the Green s function G(x,y) by the fundamental solution of the Laplacian: 1 2 x y if d = 1, K(x,y) = 1 2π log x y if d = 2, x y 2 d (d 2)ω d if d > 2. The price we pay is to have rather implicit, non-local boundary condition although we do not have to deal with this condition directly. saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 14 / 68

15 Integral Operators Commuting with Laplacian... Let K be the integral operator with its kernel K(x,y): Kf (x) = K(x,y)f (y)dy, f L 2 (Ω). Theorem (NS 2005) Γ Ω The integral operator K commutes with the Laplacian L = with the following non-local boundary condition: K(x,y) ϕ (y)ds(y) = 1 K(x,y) ϕ(x) + pv ϕ(y)ds(y), ν y 2 ν y for all x Γ, where ϕ is an eigenfunction common for both operators. Γ saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 15 / 68

16 Integral Operators Commuting with Laplacian... Corollary (NS 2005) The integral operator K is compact and self-adjoint on L 2 (Ω). Thus, the kernel K(x,y) has the following eigenfunction expansion (in the sense of mean convergence): K(x,y) µ j ϕ j (x)ϕ j (y), j=1 and {ϕ j } j forms an orthonormal basis of L 2 (Ω). saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 16 / 68

17 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 17 / 68

18 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 18 / 68

19 1D Example Consider the unit interval Ω = (0, 1). Then, our integral operator K with the kernel K(x, y) = x y /2 gives rise to the following eigenvalue problem: ϕ = λϕ, x (0, 1); ϕ(0) + ϕ(1) = ϕ (0) = ϕ (1). The kernel K(x,y) is of Toeplitz form = Eigenvectors must have even and odd symmetry (Cantoni-Butler 76). In this case, we have the following explicit solution. saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 19 / 68

20 1D Example... λ , which is a solution of tanh λ0 2 = 2 λ0, ϕ 0 (x) = A 0 cosh ( λ 0 x 1 ) ; 2 λ 2m 1 = (2m 1) 2 π 2, m = 1, 2,..., ϕ 2m 1 (x) = 2cos(2m 1)πx; λ 2m, m = 1, 2,..., which are solutions of tan λ2m 2 = 2 λ2m, ϕ 2m (x) = A 2m cos ( λ 2m x 1 ), 2 where A k, k = 0, 1,... are normalization constants. saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 20 / 68

21 First 5 Basis Functions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 21 / 68

22 1D Example: Bases on Disconnected Intervals φ k (x) x (a) Union saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 22 / 68

23 1D Example: Bases on Disconnected Intervals φ k (x) 0 φ k (x) x x (a) Union (b) Separated saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 22 / 68

24 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 23 / 68

25 2D Example Consider the unit disk Ω. Then, our integral operator K with the kernel K(x,y) = 1 2π log x y gives rise to: ϕ = λϕ, in Ω; ϕ = ϕ = Hϕ, ν Γ r Γ θ Γ where H is the Hilbert transform for the circle, i.e., Hf (θ) = 1 π ( ) θ η 2π pv f (η) cot dη θ [ π, π]. 2 π Let β k,l is the lth zero of the Bessel function of order k, J k (β k,l ) = 0. Then, { J m (β m 1,n r) ( cos ϕ m,n (r, θ) = sin) (mθ) if m = 1, 2,..., n = 1, 2,..., J 0 (β 0,n r) if m = 0, n = 1, 2,..., { βm 1,n 2, if m = 1,..., n = 1, 2,..., λ m,n = β0,n 2 if m = 0, n = 1, 2,.... saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 24 / 68

26 First 25 Basis Functions (a) Our Basis (b) Dirichlet-Laplace (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 25 / 68

27 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 26 / 68

28 3D Example Consider the unit ball Ω in R3. Then, our integral operator K with 1. the kernel K (x, y) = 4π x y Top 9 eigenfunctions cut at the equator viewed from the south: saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 27 / 68

29 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 28 / 68

30 Discretization of the Problem Assume that the whole dataset consists of a collection of data sampled on a regular grid, and that each sampling cell is a box of size d i=1 x i. Assume that an object of our interest Ω consists of a subset of these boxes whose centers are{x i } N i=1. Under these assumptions, we can approximate the integral eigenvalue problem Kϕ = µϕ with a simple quadrature rule with node-weight pairs (x j, w j ) as follows. N w j K(x i,x j )ϕ(x j ) = µϕ(x i ), i = 1,...,N, w j = j=1 d x i. Let K i,j = wj K(x i,x j ), ϕ i = ϕ(xi ), and ϕ = (ϕ 1,...,ϕ N ) T R N. Then, the above equation can be written in a matrix-vector format as: Kϕ = µϕ, where K = (K ij ) R N N. Under our assumptions, the weight w j does not depend on j, which makes K symmetric. saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 29 / 68 i=1

31 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 30 / 68

32 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 31 / 68

33 Image Approximation; Comparison with Wavelets (a) What data? (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 32 / 68

34 Image Approximation; Comparison with Wavelets (a) What data? (b) χ J Barbara saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 32 / 68

35 First 25 Basis Functions (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 33 / 68

36 Next 25 Basis Functions (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 34 / 68

37 Reconstruction with Top 100 Coefficients (a) Reconstruction (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 35 / 68

38 Reconstruction with Top 100 Coefficients (a) Reconstruction (b) Error (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 35 / 68

39 Reconstruction with Top 100 2D Wavelets (Symmlet 8) (a) Reconstruction (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 36 / 68

40 Reconstruction with Top 100 2D Wavelets (Symmlet 8) (a) Reconstruction (b) Error (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 36 / 68

41 Reconstruction with Top 100 1D Wavelets (Symmlet 8) (a) Reconstruction (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 37 / 68

42 Reconstruction with Top 100 1D Wavelets (Symmlet 8) (a) Reconstruction (b) Error (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 37 / 68

43 Comparison of Coefficient Decay 10 4 Laplacian Eigenbasis 1D Symmlet 8 2D Symmlet D Symmlet 8 Tensor saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 38 / 68

44 A Real Challenge: Kernel matrix is of (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 39 / 68

45 Conjecture on the Coefficient Decay Rate Conjecture (NS 2005) For f C(Ω) with f BV(Ω) defined on C 2 -domain Ω, the expansion coefficients f, ϕ k w.r.t. the Laplacian eigenbasis decay as O(k 2 ). Thus, the N-term approximation error measured in the L 2 -norm should have a decay rate of O(N 1.5 ). saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 40 / 68

46 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 41 / 68

47 Comparison with PCA Consider a stochastic process living on a domain Ω. PCA/Karhunen-Loève Transform is often used. PCA/KLT incorporate geometric information of the measurement (or pixel) location through the data correlation, i.e., implicitly. Our Laplacian eigenfunctions use explicit geometric information through the harmonic kernel ϕ(x,y). saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 42 / 68

48 Comparison with PCA: Example Rogue s Gallery dataset from Larry Sirovich 72 training dataset; 71 test dataset Left & right eye regions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 43 / 68

49 Comparison with PCA: Basis Vectors (a) KLB/PCA 1:9 (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 44 / 68

50 Comparison with PCA: Basis Vectors (a) KLB/PCA 1:9 (b) Laplacian Eigenfunctions 1:9 (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 44 / 68

51 Comparison with PCA: Basis Vectors... (a) KLB/PCA 10:18 (b) Laplacian Eigenfunctions 10:18 (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 45 / 68

52 Comparison with PCA: Kernel Matrix (a) Covariance (b) Harmonic kernel (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 46 / 68

53 Comparison with PCA: Energy Distribution over Coordinates Magnitude of Top 50 Principal Components (Log scale) Magnitude of 50 Lowest Frequency Coefficients (Log scale) PC index frequency index face index face index (a) KLB/PCA (b) Laplacian Eigenfunctions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 47 / 68

54 Comparison with PCA: Basis Vector #7... c 7 :large c 7 :large ϕ 7 c 7 :small c 7 :small saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 48 / 68

55 Comparison with PCA: Basis Vector #13... c 13 :large c 13 :large ϕ 13 c 13 :small c 13 :small saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 49 / 68

56 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 50 / 68

57 Solving the Heat Equation on a Complicated Domain It is well known that the semigroup e t can be diagonalized using the Laplacian eigenbasis, i.e., for any initial heat distribution u 0 (x) L 2 (Ω), we have the heat distribution at time t formally as u(x, t) = e t u 0 = e tλ j u 0, ϕ j ϕ j (x), j=1 which is based on the expansion of the Green s function for the heat equation p t (x,y) via the Laplacian eigenfunctions as follows: p t (x,y) = e λjt ϕ j (x)ϕ j (y) (t,x,y) (0, ) Ω Ω. j=1 saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 51 / 68

58 Solving the Heat Equation on a Complicated Domain Due to the discretization of the problem, we can write e t in the matrix-vector notation as ( ) Φe tλ Φ T = Φ diag e tλ 1,...,e tλ N Φ T = N e λjt ϕ j ϕ T j, where Φ = (ϕ 1,...,ϕ N ) is the Laplacian eigenbasis matrix of size N N, and Λ is the diagonal matrix consisting of the eigenvalues of the Laplacian, which are the inverse of the eigenvalues of the kernel matrix, i.e., Λ k,k = λ k = 1/µ k. Given an initial heat distribution over the domain, u 0 R N, we can compute the heat distribution at time t as u(t) = Φe tλ Φ T u 0. j=1 saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 52 / 68

59 Solving the Heat Equation on a Complicated Domain... t=0 t=1 t=10 t=100 t=250 t=500 saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 53 / 68

60 Solving the Heat Equation on a Complicated Domain... It is well known that the eigenvalues of the Laplacian with the Dirichlet (or the Neumann/Robin) boundary condition are positive (or non-negative). At this point, precisely specifying the boundary values is difficult because our formulation satisfies neither the Dirichlet nor the Neumann nor the Robin conditions. Our empirical observation so far has led to the following conjecture: Conjecture (NS 2007) The eigenvalues of the Laplacian satisfying our boundary condition and defined over a bounded domain Ω R d are all positive possibly with a finite number of negative ones. saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 54 / 68

61 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 55 / 68

62 Clustering Mouse Retinal Ganglion Cells Objective: To understand how the structural/geometric properties of mouse retinal ganglion cells (RGCs) relate to the cell types and their functionality Why mouse? = great possibilities for genetic manipulation Data: 3D images of dendrites/axons of RGCs State of the Art: Process each image via specialized software to extract geometric/morphological parameters (totally 14 such parameters) followed by a conventional clustering algorithm These parameters include: somal size; dendric field size; total dendrite length; branch order; mean internal branch length; branch angle; mean terminal branch length, etc. = takes half a day per cell with a lot of human interactions! saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 56 / 68

63 Clustering Mouse Retinal Ganglion Cells...3D Data (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 57 / 68

64 Very Preliminary Study on Mouse Retinal Ganglion Cells Use 2D plane projection data instead of full 3D Compute the smallest k Laplacian eigenvalues using our method (i.e., the largest k eigenvalues of K) for each image Construct a feature vector per image Possible feature vectors reflecting geometric information: F 1 = (λ 1,...,λ k ) T ; F 2 = (µ 1,...,µ k ) T ; F 3 = (λ 1 /λ 2,..., λ 1 /λ k ) T ; F 4 = (µ 1 /µ 2,..., µ 1 /µ k ) T. Do visualization and clustering saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 58 / 68

65 Very Preliminary Study on Mouse RGCs... The first three Laplacian eigenvalues The ratios of the Laplacian eigenvalues 2.5 Diffusion map of manual features 2 x λ λ 1 /λ x λ 2 x 10 4 λ λ /λ λ 1 /λ φ 2 4 φ 1 (a) F 1 (b) F 3 φ 3 (c) Manual saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 59 / 68

66 Laplacian Eigenfunctions on a Mouse RGC φ φ φ φ φ φ φ (UC 500 saito@math.ucdavis.edu Davis) 0.05 φ φ Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 60 / 68

67 Challenges of Mouse Retinal Ganglion Cells Their shapes are very complicated. Interpretation of our eigenvalues are not yet fully understood compared to the usual Dirichlet-Laplacian case that have been well studied: the Payne-Pólya-Weinberger inequalities; the Faber-Krahn inequalities; the Ashbaugh-Benguria results, etc. For Ω R d, λ (D) 1 (Ω) ( B d 1 Ω ) 2 λ (D) 1 (B d 1), λ (D) k+1 (Ω) (Ω) λ λ (D) k (D) 2 (B d 1 ) λ (D) 1 (B d 1 ), k = 1, 2, 3. Note the related work on Shape DNA by Reuter et al. (2005), and classification of tree leaves by Khabou et al. (2007). Perhaps original 3D data should be used instead of projected 2D data. Reduce computational burden = need to develop fast algorithms. Heat propagation on the dendrites may give us interesting and useful information; after all the dendrites are network to disseminate information via chemical reaction-diffusion mechanism. Construct actual graphs based on the connectivity and analyze them directly via spectral graph theory and diffusion maps. saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 61 / 68

68 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 62 / 68

69 A Possible Fast Algorithm for Computing ϕ j s Observation: our kernel function K(x,y) is of special form, i.e., the fundamental solution of Laplacian used in potential theory. Idea: Accelerate the matrix-vector product Kϕ using the Fast Multipole Method (FMM). Convert the kernel matrix to the tree-structured matrix via the FMM whose submatrices are nicely organized in terms of their ranks. (Computational cost: our current implementation costs O(N 2 ), but can achieve O(N log N) via the randomized SVD algorithm of Martinsson-Rokhlin-Tygert.) Construct O(N) matrix-vector product module fully utilizing rank information (See also the work of Bremer (2007) and the HSS algorithm of Chandrasekaran et al. (2006)). Embed that matrix-vector product module in the Krylov subspace method, e.g., Lanczos iteration. (Computational cost: O(N) for each eigenvalue/eigenvector). saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 63 / 68

70 Splitting into Subproblems for Faster Computation (a) Whole islands (b) Separated islands (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 64 / 68

71 Eigenfunctions for Separated Islands (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 65 / 68

72 Outline 1 Motivations 2 Laplacian Eigenfunctions 3 Integral Operators Commuting with Laplacian 4 Examples 1D Example 2D Example 3D Example 5 Discretization of the Problem 6 Applications Image Approximation Statistical Image Analysis; Comparison with PCA Solving the Heat Equation on a Complicated Domain Clustering Mouse Retinal Ganglion Cells 7 Fast Algorithms for Computing Eigenfunctions 8 Conclusions saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 66 / 68

73 Conclusions Allow object-oriented image analysis & synthesis Can get fast-decaying expansion coefficients Can decouple geometry/domain information and statistics of data Can extract geometric information of a domain through the eigenvalues A variety of applications: interpolation, extrapolation, local feature computation,... Fast algorithms are the key for higher dimensions/large domains Connection to lots of interesting mathematics: spectral geometry, spectral graph theory, isoperimetric inequalities, Toeplitz operators, PDEs, potential theory, almost-periodic functions,... Many things to be done: Synthesize the Dirichlet-Laplacian eigenvalues/eigenfunctions from our eigenvalues/eigenfunctions How about higher order, i.e., polyharmonic? Features derived from heat kernels? Improve our fast algorithm saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 67 / 68

74 References The following articles are available at saito/publications/: N. Saito: Geometric harmonics as a statistical image processing tool for images defined on irregularly-shaped domains, in Proc. IEEE Workshop on Statistical Signal Processing, Bordeaux, France, Jul N. Saito: Data analysis and representation using eigenfunctions of Laplacian on a general domain, Submitted to Applied & Computational Harmonic Analysis, Mar Thank you very much for your attention! saito@math.ucdavis.edu (UC Davis) Image Analysis via Laplacian Eigenfunctions IPAM RSWS4 68 / 68