Harmonic Analysis and Geometries of Digital Data Bases

Transcription

1 Harmonic Analysis and Geometries of Digital Data Bases AMS Session Special Sesson on the Mathematics of Information and Knowledge, Ronald Coifman (Yale) and Matan Gavish (Stanford, Yale) January 14, 2010 () Harmonic Analysis of Data Bases January 14, / 41

2 Local geometry described by a graph Given a dataset X = {x 1,..., x N } with affinity matrix W i,j (aka covariance, kernel, etc) (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

3 Local geometry described by a graph Given a dataset X = {x 1,..., x N } with affinity matrix W i,j (aka covariance, kernel, etc) (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

4 Local geometry described by a graph Given a dataset X = {x 1,..., x N } with affinity matrix W i,j (aka covariance, kernel, etc) (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

5 Local geometry described by a graph Given a dataset X = {x 1,..., x N } with affinity matrix W i,j (aka covariance, kernel, etc) (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

6 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

7 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

8 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

9 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

10 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

11 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

12 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

13 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

14 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

15 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

16 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

17 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

18 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

19 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

20 Mission: Harmonic analysis on n-way arrays (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

21 Why organize {rows} and {columns} of a matrix? In the correct coupled intrinstic geometry of {rows} and {cols} we can analyze matrix effectively (below) (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

22 Why organize {rows} and {columns} of a matrix? In the correct coupled intrinstic geometry of {rows} and {cols} we can analyze matrix effectively (below) (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

23 e.g.: organize {rows},{columns} of potential operator Consider the matrix M i,j = x i y j 1 ({x i } - red, {y j }- blue ) (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

24 e.g.: organize {rows},{columns} of potential operator Consider the matrix M i,j = x i y j 1 ({x i } - red, {y j }- blue ) (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

25 e.g.: organize {rows},{columns} of potential operator Scramble rows and columns: consider M i,j = x σ(i) y τ(j) (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

26 e.g.: organize {rows},{columns} of potential operator Diffusion embedding of graphs on {rows}, {cols} recovers spatial point layout : (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

27 e.g.: organize {rows},{columns} of potential operator Diffusion embedding of graphs on {rows}, {cols} recovers spatial point layout : (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

28 e.g.: organize torus eigenfunctions Now consider ( the matrix ) sin 2πk j N M k,j = ( ) cos 2π (k m) j N k = 1... m k = m m Partition based correlation { } (Coifman, Gavish) l Harmonic l Analysis d of Data Bases January 14, / 41

29 e.g.: organize torus eigenfunctions Now consider ( the matrix ) sin 2πk j N M k,j = ( ) cos 2π (k m) j N k = 1... m k = m m Partition based correlation { } (Coifman, Gavish) l Harmonic l Analysis d of Data Bases January 14, / 41

30 e.g.: organize torus eigenfunctions Now consider ( the matrix ) sin 2πk j N M k,j = ( ) cos 2π (k m) j N k = 1... m k = m m Partition based correlation For mesh K l = 2 l {0,..., 2 l 1 } d and bump ψ define ρ (f, g) = 2 l l 1 k K l T d ( ) f (x) g (x)ψ 2 l x k dx Can show: ( ) ρ e im x, e im x 1 const m m 2 (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

31 e.g.: organize torus eigenfunctions Diffusion embedding of graphs on {rows}, {cols} recovers points and frequencies organization: (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

32 e.g.: organize torus eigenfunctions Diffusion embedding of graphs on {rows}, {cols} recovers points and frequencies organization: (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

33 Graph into Partition Tree (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

34 Graph into Partition Tree (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

35 Graph into Partition Tree (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

36 Graph into Partition Tree (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

37 Graph into Partition Tree (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

38 Graph into Partition Tree (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

39 Graph into Partition Tree (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

40 Graph into Partition Tree (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

41 Graph into Partition Tree (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

42 Graph into Partition Tree (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

43 Graph into Partition Tree (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

44 Graph into Partition Tree (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

45 organize torus eigenfunctions, again Now consider the matrix M i,j = { sin (2πk) cos (2πk) k = 1... m k = m m (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

46 organize torus eigenfunctions, again Now consider the matrix M i,j = { sin (2πk) cos (2πk) k = 1... m k = m m (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

47 organize torus eigenfunctions, again Build affinity on points Create partition tree on points Compute partition-based affinity between eigenvectors (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

48 organize torus eigenfunctions, again Build affinity on points Create partition tree on points Compute partition-based affinity between eigenvectors (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

49 organize torus eigenfunctions, again Build affinity on points Create partition tree on points Compute partition-based affinity between eigenvectors (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

50 e.g.: organize {rows},{columns} of data matrix Term-document matrix A, where A i,j is relative appearence frequency of word i in document j (Data prepared by Priebe et al, COMPSTAT 2004) We recover coupled geometry: hierarchical structure of words (concepts) and hierarchical structure of documents (contexts) (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

51 e.g.: organize {rows},{columns} of data matrix (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

52 e.g.: organize {rows},{columns} of data matrix (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

53 e.g.: Image texture separation By Ali Haddad, Yale To each pixel we associate the 8x8 patch around it and compute log absolute value of the Fourier transform as a list of 64 questions. Colors code the partition in each tree level. (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

54 e.g.: Image texture separation By Ali Haddad, Yale To each pixel we associate the 8x8 patch around it and compute log absolute value of the Fourier transform as a list of 64 questions. Colors code the partition in each tree level. (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

55 e.g.: Image texture separation By Ali Haddad, Yale To each pixel we associate the 8x8 patch around it and compute log absolute value of the Fourier transform as a list of 64 questions. Colors code the partition in each tree level. (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

56 e.g.: Image texture separation By Ali Haddad, Yale To each pixel we associate the 8x8 patch around it and compute log absolute value of the Fourier transform as a list of 64 questions. Colors code the partition in each tree level. (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

57 Questions Basic Questions 1 How to quantify the compatibility of database to proposed coupled geometry? 2 How to exploit a compatible coupled geometry for data analysis? (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

58 Questions Basic Questions 1 How to quantify the compatibility of database to proposed coupled geometry? 2 How to exploit a compatible coupled geometry for data analysis? (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

59 Questions Basic Questions 1 How to quantify the compatibility of database to proposed coupled geometry? 2 How to exploit a compatible coupled geometry for data analysis? (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

60 Enter J. Strömberg Strömberg (1998): tensor product of Haar bases extremely efficient in representing regular f : [0, 1] d R Regular = bounded mixed derivatives Using tensor product of Haar bases in all d dimensions sup x [0,1] d f (x) f, ψ R ψ R (x) < const ε logd 1 R R s.t R >ε Only O ( 1 ε logd 1 ( 1 ε)) coefficients needed ( ) 1 ε (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

61 Enter J. Strömberg Strömberg (1998): tensor product of Haar bases extremely efficient in representing regular f : [0, 1] d R Regular = bounded mixed derivatives Using tensor product of Haar bases in all d dimensions sup x [0,1] d f (x) f, ψ R ψ R (x) < const ε logd 1 R R s.t R >ε Only O ( 1 ε logd 1 ( 1 ε)) coefficients needed ( ) 1 ε (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

62 Enter J. Strömberg Strömberg (1998): tensor product of Haar bases extremely efficient in representing regular f : [0, 1] d R Regular = bounded mixed derivatives Using tensor product of Haar bases in all d dimensions sup x [0,1] d f (x) f, ψ R ψ R (x) < const ε logd 1 R R s.t R >ε Only O ( 1 ε logd 1 ( 1 ε)) coefficients needed ( ) 1 ε (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

63 Enter J. Strömberg Strömberg (1998): tensor product of Haar bases extremely efficient in representing regular f : [0, 1] d R Regular = bounded mixed derivatives Using tensor product of Haar bases in all d dimensions sup x [0,1] d f (x) f, ψ R ψ R (x) < const ε logd 1 R R s.t R >ε Only O ( 1 ε logd 1 ( 1 ε)) coefficients needed ( ) 1 ε (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

64 Enter J. Strömberg Strömberg (1998): tensor product of Haar bases extremely efficient in representing regular f : [0, 1] d R Regular = bounded mixed derivatives Using tensor product of Haar bases in all d dimensions sup x [0,1] d f (x) f, ψ R ψ R (x) < const ε logd 1 R R s.t R >ε Only O ( 1 ε logd 1 ( 1 ε)) coefficients needed ( ) 1 ε (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

65 Answers Basic Questions 1 How to quantify the compatibility of database to proposed coupled geometry? 2 How to exploit a compatible coupled geometry for data analysis? Answers Build Haar-like basis {ψ i } for {f : {rows} R} and {ϕ j }for {f : {cols} R} Expand the matrix as M = i,j M, ψ i ϕ j ψ i ϕ j 1 The coefficients l 1 norm i,j M, ψ i ϕ j quantifies compatibility 2 Process M in coefficient domain (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

66 Answers Basic Questions 1 How to quantify the compatibility of database to proposed coupled geometry? 2 How to exploit a compatible coupled geometry for data analysis? Answers Build Haar-like basis {ψ i } for {f : {rows} R} and {ϕ j }for {f : {cols} R} Expand the matrix as M = i,j M, ψ i ϕ j ψ i ϕ j 1 The coefficients l 1 norm i,j M, ψ i ϕ j quantifies compatibility 2 Process M in coefficient domain (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

67 Answers Basic Questions 1 How to quantify the compatibility of database to proposed coupled geometry? 2 How to exploit a compatible coupled geometry for data analysis? Answers Build Haar-like basis {ψ i } for {f : {rows} R} and {ϕ j }for {f : {cols} R} Expand the matrix as M = i,j M, ψ i ϕ j ψ i ϕ j 1 The coefficients l 1 norm i,j M, ψ i ϕ j quantifies compatibility 2 Process M in coefficient domain (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

68 Answers Basic Questions 1 How to quantify the compatibility of database to proposed coupled geometry? 2 How to exploit a compatible coupled geometry for data analysis? Answers Build Haar-like basis {ψ i } for {f : {rows} R} and {ϕ j }for {f : {cols} R} Expand the matrix as M = i,j M, ψ i ϕ j ψ i ϕ j 1 The coefficients l 1 norm i,j M, ψ i ϕ j quantifies compatibility 2 Process M in coefficient domain (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

69 Answers Basic Questions 1 How to quantify the compatibility of database to proposed coupled geometry? 2 How to exploit a compatible coupled geometry for data analysis? Answers Build Haar-like basis {ψ i } for {f : {rows} R} and {ϕ j }for {f : {cols} R} Expand the matrix as M = i,j M, ψ i ϕ j ψ i ϕ j 1 The coefficients l 1 norm i,j M, ψ i ϕ j quantifies compatibility 2 Process M in coefficient domain (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

70 The Haar Basis (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

71 The Haar Basis (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

72 The Haar Basis (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

73 The Haar Basis (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

74 The Haar Basis (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

75 The Haar Basis (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

76 The Haar Basis (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

77 The Haar Basis (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

78 The Haar Basis (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

79 The Haar Basis (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

80 The Haar Basis (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

81 Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

82 Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

83 Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

84 Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

85 Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

86 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

87 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

88 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

89 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

90 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

91 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

92 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

93 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

94 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

95 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

96 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

97 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

98 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

99 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

100 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

101 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

102 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

103 Haar-like bases With B. Nadler, Weizmann (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

104 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

105 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

106 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

107 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

108 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

109 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

110 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

111 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

112 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

113 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

114 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

115 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

116 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

117 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

118 Tensor product of Haar-like bases (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

119 Tensor Haar-like basis function (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

120 An approximation theorem Theorem Fix 0 < p < 2, f : X R and ε > 0. Denote A ε f = f, ψ i ψ i (x) 1 i N with f,ψ i >ε 1 p and R(ψ i ) >ε (i.e. retaining only coefficients which are (i) large, and (ii) correspond to basis functions supported on large folders) (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

121 An approximation theorem Theorem 1 # coefficients retained for A ε f ε 1 N i=1 f, ψ i p. In particular it does not depend on d. 2 Approximation in the mean (roughly): X A ε f f p 1 p ε( 1 p 1 2 ) ( N i=1 f, ψ i p ) 1 p. 3 Uniform pointwise approximation holds outside an exceptional set with E p < N i=1 f, ψ i p (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

122 Data Analysis Scheme 1 Construct affinity of rows according to columns 2 Construct affinity of columns according to rows 3 Construct correponding Haar-like bases and compute i,j M, ψ i ϕ j 4 Repreat until converge 5 Expand the matrix M in tensor Haar-like basis and process in coefficient domain (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

123 Data Analysis Scheme 1 Construct affinity of rows according to columns 2 Construct affinity of columns according to rows 3 Construct correponding Haar-like bases and compute i,j M, ψ i ϕ j 4 Repreat until converge 5 Expand the matrix M in tensor Haar-like basis and process in coefficient domain (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

124 Data Analysis Scheme 1 Construct affinity of rows according to columns 2 Construct affinity of columns according to rows 3 Construct correponding Haar-like bases and compute i,j M, ψ i ϕ j 4 Repreat until converge 5 Expand the matrix M in tensor Haar-like basis and process in coefficient domain (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

125 Data Analysis Scheme 1 Construct affinity of rows according to columns 2 Construct affinity of columns according to rows 3 Construct correponding Haar-like bases and compute i,j M, ψ i ϕ j 4 Repreat until converge 5 Expand the matrix M in tensor Haar-like basis and process in coefficient domain (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

126 Data Analysis Scheme 1 Construct affinity of rows according to columns 2 Construct affinity of columns according to rows 3 Construct correponding Haar-like bases and compute i,j M, ψ i ϕ j 4 Repreat until converge 5 Expand the matrix M in tensor Haar-like basis and process in coefficient domain (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

127 Data Analysis Scheme i,j M, ψ i ϕ j plotted over iteration number for the term-document matrix data: Σ <M,ψ i > Iteration # (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

128 Data Analysis Scheme X A εf f fraction of coefficients used and theoretical bound over folder volue cutoff ε e(a)*ε 1/2 l1(a A(ε)) l1(a A(ε)) fraction of coeffs used ε (keep coeffs >ε on normalized support>ε) x 10 4 (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

129 Data Analysis Scheme (left) original term-document matrix. (right) after retaining 15% of the tensor Haar-like coefficients (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

130 Application: Numerical operator compression Here is an extension of ideas by Belkin, Coifman, Rokhlin. Back to the charge distribution (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

131 Application: Numerical operator compression the corresponding potential operator (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

132 Application: Numerical operator compression and its scrambled version, which is our data matrix (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

133 Application: Numerical operator compression Coefficient matrix of scrambled operator in the tensor Haar-like basis is exteremely sparse (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

134 Application: Numerical operator compression Coefficient matrix of scrambled operator in the tensor Haar-like basis is exteremely sparse. Working in coefficient domain allows us to consider only coefficients corresponding to functions of support > ε and obtain a Belkin, Coifman, Rokhlin-type compression scheme that does not assume a-priori knowledge of the operator shape or underlying geometry! (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

135 Application: Numerical operator compression operator norm of the residual and fraction of coefficients used over the threshold ε A A(ε) fraction of coeffs used ε (keep coeffs >ε on normalized support>ε) x 10 7 (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

136 Take Home... Sometimes must consider graphs on both rows and columns Strömberg s theory Partition trees and induced tensor Haar-like bases allow quantitative analysis: - Smoothness of matrix w.r.t graphs - Signal processing on matrix New results for Haar bases on R d conjuctured by experimenting with data (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

137 Take Home... Sometimes must consider graphs on both rows and columns Strömberg s theory Partition trees and induced tensor Haar-like bases allow quantitative analysis: - Smoothness of matrix w.r.t graphs - Signal processing on matrix New results for Haar bases on R d conjuctured by experimenting with data (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

138 Take Home... Sometimes must consider graphs on both rows and columns Strömberg s theory Partition trees and induced tensor Haar-like bases allow quantitative analysis: - Smoothness of matrix w.r.t graphs - Signal processing on matrix New results for Haar bases on R d conjuctured by experimenting with data (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

139 Take Home... Sometimes must consider graphs on both rows and columns Strömberg s theory Partition trees and induced tensor Haar-like bases allow quantitative analysis: - Smoothness of matrix w.r.t graphs - Signal processing on matrix New results for Haar bases on R d conjuctured by experimenting with data (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

140 Take Home... Sometimes must consider graphs on both rows and columns Strömberg s theory Partition trees and induced tensor Haar-like bases allow quantitative analysis: - Smoothness of matrix w.r.t graphs - Signal processing on matrix New results for Haar bases on R d conjuctured by experimenting with data (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41

141 References R. Coifman and MG, Harmonic Analysis of Digital Data Bases, to appear in Wavelets: Old and New Perspectives, Springer 2010 B. Nadler, MG and R. Coifman, Adaptive Haar-like wavelet analysis on data: theory and inference algorithms, submitted J. O. Strömberg, (1998) Wavelets in higher dimensions, Documenta Mathematica extra volume ICM-1998 (3), R. Coifman and G. Weiss, Extensions of Hardy spaces and their use in analysis, Bul. Of the AMS, 83, #4, 1977, (Coifman, Gavish) Harmonic Analysis of Data Bases January 14, / 41