Sparse representations from moments

Size: px

Start display at page:

Download "Sparse representations from moments"

Bruno Claud Wilkins
5 years ago
Views:

1 Sparse representations from moments Bernard Mourrain Inria Méditerranée, Sophia Antipolis

2 Sparse representation of signals Given a function or signal f (t): decompose it as f (t) = (a i cos(µ i t) + b i sin(µ i t))e ν i t = r i=1 r ω i e ζ it i=1 B Mourrain Sparse representations from moments 2 / 29

3 Prony s method (1795) For the signal f (t) = r i=1 ω ie ζit, (ω i, ζ i C), Evaluate f at 2 r regularly spaced points: σ 0 := f (0), σ 1 := f (1), Compute a non-zero element p = [p 0,, p r ] in the kernel: σ 0 σ 1 σ r σ 1 σ r+1 σ r 1 σ 2r 1 σ 2r 1 Compute the roots ξ 1 = e ζ 1,, ξ r = e ζr Solve the system 1 1 ξ 1 ξ r ξ r 1 1 ξr r 1 ω 1 ω 2 ω r p 0 p 1 p r = 0 of p(x) := r i=0 p ix i = σ 0 σ 1 σ r 1 B Mourrain Sparse representations from moments 3 / 29

Symmetric tensor decomposition and Waring problem (1770) Tensor decomposition problem: given a homogeneous polynomial Ψ of degree d in the variables x = (x 0, x 1,, x n ): Ψ(x) = ( ) d σ α x α, α α

4 Symmetric tensor decomposition and Waring problem (1770) Tensor decomposition problem: given a homogeneous polynomial Ψ of degree d in the variables x = (x 0, x 1,, x n ): Ψ(x) = ( ) d σ α x α, α α =d find a minimal decomposition of Ψ of the form Ψ(x) = r ω i (ξ i,0 x 0 + ξ i,1 x ξ i,n x n ) d i=1 for ξ i = (ξ i,0, ξ i,1,, ξ i,n ) C n+1, ω i C The minimal r in such a decomposition is called the rank of τ B Mourrain Sparse representations from moments 4 / 29

Sylvester approach (1851) Theorem The binary form Ψ(x 0, x 1 ) = d i=0 σ d ) i( i x d i sum of r distinct powers of linear forms 0 x i 1 can be decomposed as a Ψ = r ω k (α k x 0 + β k x 1 ) d k=1

5 Sylvester approach (1851) Theorem The binary form Ψ(x 0, x 1 ) = d i=0 σ d ) i( i x d i sum of r distinct powers of linear forms 0 x i 1 can be decomposed as a Ψ = r ω k (α k x 0 + β k x 1 ) d k=1 iff there exists a polynomial p(x 0, x 1 ) := p 0 x0 r + p 1x0 r 1 x p r x1 r st σ 0 σ 1 σ r p 0 σ 1 σ r+1 p 1 σ d r σ d 1 σ d and of the form p = c r k=1 (β kx 0 α k x 1 ) p r = 0 B Mourrain Sparse representations from moments 5 / 29

6 Sparse interpolation Given a black-box polynomial function f (x) find what are the terms inside from output values Find r N, ω i C, α i N such that f (x) = r i=1 ω i x α i B Mourrain Sparse representations from moments 6 / 29

7 Choose ϕ C Compute the sequence of terms σ 0 = f (1),, σ 2r 1 = f (ϕ 2r 1 ); Construct the matrix H = [σ i+j ] and its kernel p = [p 0,, p r ] st σ 0 σ 1 σ r σ 1 σ r+1 σ r 1 σ 2r 1 σ 2r 1 Compute the roots ξ 1 = ϕ α 1,, ξ r = ϕ αr deduce the exponents α i = log ϕ (ξ i ) p 0 p 1 p r = 0 of p(x) := r i=0 p ix i and Deduce the weights W = [ω i ] by solving V Ξ W = [σ 0,, σ r 1 ] where V Ξ is the Vandermonde system of the roots ξ 1,, ξ r B Mourrain Sparse representations from moments 7 / 29

Decoding An algebraic code: E = {c(f ) = [f (ξ 1 ),, f (ξ m

an error with ω j 0 for j = i 1,, i r and ω j = 0 otherwise

8 Decoding An algebraic code: E = {c(f ) = [f (ξ 1 ),, f (ξ m )] f K[x]; deg(f ) d} Encoding messages using the dual code: C = E = {c c [f (ξ 1 ),, f (ξ l )] = 0 f V = x a F[x]} Message received: r = m + e for m C where e = [ω 1,, ω m ] is an error with ω j 0 for j = i 1,, i r and ω j = 0 otherwise Find the error e B Mourrain Sparse representations from moments 8 / 29

9 Berlekamp-Massey method (1969) Compute the syndrome σ k = c(x k ) r = c(x k ) e = r j=1 ω i j ξi k j Compute the matrix σ 0 σ 1 σ r σ 1 σ r+1 σ r 1 σ 2r 1 σ 2r 1 and its kernel p = [p 0,, p r ] p 0 p 1 Compute the roots of the error locator polynomial p(x) = r i=0 p i x i = p r r j=1 (x ξ i j ) Deduce the errors ω ij p r = 0 B Mourrain Sparse representations from moments 9 / 29

10 Sequences, series, duality (1D) Sequences: σ = (σ k ) k N K N indexed by k N Formal power series: σ = y k σ k k! K[[y]] (or σ k z k K[[z]]) k=0 k=0 Linear functionals: K[x] = {Λ : K[x] K linear} Example: p coefficient of x i in p = 1 i! i (p)(0) e ζ : p p(ζ) B Mourrain Sparse representations from moments 10 / 29

11 Series as linear functionals: For σ = k=0 σ k y k k! K[[y]], σ : p = k p k x k σ p = k σ k p k ( y k k! ) is the dual basis of the monomial basis (x k ) k N Example: e ζ = k=0 ζk y k k! = eζy K[[y]] Structure of K[x]-module: p(x) Λ : q Λ(p q) x σ = k=1 σ k y k 1 p(x) σ = p( )(σ(y)) (k 1)! = (σ(y)) B Mourrain Sparse representations from moments 11 / 29

12 Sequences, series, duality (nd) Multi-index sequences: σ = (σ α ) α N n K Nn α = (α 1,, α n ) N n Taylor series: indexed by σ(y) = y α σ α α! K[[y 1,, y n ]] (or σ α z α K[[z 1,, z n ]]) α N n α N n where α! = α i! for α = (α 1,, α n ) N Linear functionals: σ R = {σ : R K, linear} σ : p = α p α x α σ p = α σ α p α The coefficients σ x α = σ α K, α N n are called the moments of σ Structure of R-module: p R, σ R, p σ : q σ p q p σ = p( 1,, n )(σ)(y) B Mourrain Sparse representations from moments 12 / 29

13 Hankel operator: For σ K[[y]], H σ : K[x] K[[y]] p p σ = p( 1,, n )(σ)(y) σ is the symbol of H σ Truncated Hankel operator: V, V K[x], H V,V σ : p V p σ V V Example: V = x A, V = x B K[x] Ideal: H A,B σ = [σ α+β ] α A,β B I σ = ker H σ = {p K[x] p σ = 0}, = {p = α p α x α β N n α p α σ α+β = 0} Linear recurrence relations on the sequence σ = (σ α ) α N n Quotient algebra: A σ = K[x 1,, x n ]/I σ Studied case: dim A σ < B Mourrain Sparse representations from moments 13 / 29

14 Structure of an Artinian algebra A Hypothesis: A = K[x]/I is Artinian, ie dim K A < Hilbert nullstellensatz: A = K[x]/I Artinian V K (I ) = {ξ 1,, ξ r } is finite Assuming K = K is algebraically closed, we have I = Q 1 Q r where Q i is m ξi -primary where V K (I ) = {ξ 1,, ξ r } A = K[x]/I = A 1 A r, with Ai = u i A K[x 1,, x n ]/Q i, u 2 i = u i, u i u j = 0 if i j, u u r = 1 dim R/Q i = µ i is the multiplicity of ξ i B Mourrain Sparse representations from moments 14 / 29

15 Structure of the dual A Sparse series: PolExp = { σ(y) = } r ω i (y) e ξi (y) ω i (y) K[y], i=1 where e ξi (y) = e y ξ i = e y 1ξ 1,i + +y nξ n,i with ξ i,j K Inverse system generated by ω 1,, ω r K[y] Theorem For K = K algebraically closed, V K (I ) = {ξ 1,, ξ r } D i = D(ω i,1,, ω i,li ) = Q i D(ω 1,, ω r ) = α y (ω i ), α N n A = r i=1 D i e ξi (y) PolExp with ω i,j K[y] and I = Q 1 Q r µ(ω i,1,, ω i,li ) := dim K (D i ) = µ i multiplicity of ξ i B Mourrain Sparse representations from moments 15 / 29

16 The roots by eigencomputation Hypothesis: V K (I ) = {ξ 1,, ξ r } A = K[x]/I Artinian M a : A A u a u M t a : A A Λ a Λ = Λ M a Theorem The eigenvalues of M a are {a(ξ 1 ),, a(ξ r )} The eigenvectors of all (M t a) a A are (up to a scalar) e ξi : p p(ξ i ) Proposition If the roots are simple, the operators M a are diagonalizable Their common eigenvectors are, up to a scalar, interpolation polynomials u i at the roots and idempotent in A B Mourrain Sparse representations from moments 16 / 29

17 Example Roots of polynomial systems { f1 = x 2 1 x 2 x 2 1 I = (f 1, f 2 ) C[x] f 2 = x 1 x 2 x 2 A = C[x]/I 1, x 1, x 2 I = (x 2 1 x 2, x 1 x 2 x 2, x 2 2 x 2) M 1 = , M 2 = common eigvecs of M t 1, Mt 2 = 1 0 0, I = Q 1 Q 2 where Q 1 = (x 2 1, x 2), Q 2 = m (1,1) = (x 1 1, x 2 1) I = Q 1 Q 2 Q 1 = 1, y 1 = 1, y 1 e (0,0) (y) Q 2 = 1 e (1,1) (y) = e y1+y2 Solution of partial differential equations (with constant coeff) { 2 y1 y2 σ 2 y 1 σ = 0 f 1 σ = 0 σ I = Q 1 Q 2 y1 y2 σ y2 σ = 0 f 2 σ = 0 σ = a + b y 1 + c e y1+y2 a, b, c C B Mourrain Sparse representations from moments 17 / 29

18 Hankel operators and quotient algebra Hankel operator: For σ K[[y]], H σ : K[x] K[[y]] p p σ = p( 1,, n )(σ)(y) Quotient algebra: A σ = K[x 1,, x n ]/I σ A = K[x 1,, x n ]/I Artinian iff dim K A < A Gorenstein iff σ A such that A = A σ is a free A Isomorphism with the dual space A σ: 0 I σ K[x] H σ A σ 0 p p σ Correspondence between sequences σ K Nn with rankh σ < and Artinian Gorenstein algebras A σ := K[x]/I σ B Mourrain Sparse representations from moments 18 / 29

Univariate series: Kronecker (1881) Multivariate series: The Hankel operator H σ : (p m ) C N,finite ( m σ m+np m ) n N C N is of finite rank r iff ω 1,, ω r C[y] and ξ 1,, ξ r C distincts st σ(y) =

19 Univariate series: Kronecker (1881) Multivariate series: The Hankel operator H σ : (p m ) C N,finite ( m σ m+np m ) n N C N is of finite rank r iff ω 1,, ω r C[y] and ξ 1,, ξ r C distincts st σ(y) = y n r σ n n! = ω i (y)e ξi (y) with (deg(ω i )+1) = r n N i=1 i=1 Theorem (Generalized Kronecker Theorem) H σ is of rank r iff σ = r i=1 ω i(y) e ξi (y) PolExp with r = r i=1 µ(ω i) In this case, we have V C (I σ ) = {ξ 1,, ξ r } I σ = Q 1 Q r with Q i = D(ω i ) e ξi (y) A σ = A σ σ (free A σ -module of rank 1) (a, b) σ ab is non-degenerate in A σ B Mourrain Sparse representations from moments 19 / 29 r

20 The structure of A σ For σ = r i=1 ω i e ξi, with ω i C \ {0} and ξ i C n distinct rank H σ = r and the multiplicity of the points ξ 1,, ξ r in V(I σ ) is 1 For B, B be of size r, H B,B σ A σ = K[x]/I σ invertible iff B and B are bases of The matrix M i of multiplication by x i in the basis B of A σ is such that H B,x i B σ = H B,B x i σ = H B,B σ The common eigenvectors of M i are (up to a scalar) the Lagrange interpolation polynomials { u ξi at the points ξ i, i = 1,, r 1 ifi = j, u ξi (ξ j )= u 0 otherwise 2 ξi u ξi, r i=1 u ξ i 1 The common eigenvectors of Mi t [B(ξ i )], i = 1,, r M i are (up to a scalar) the vectors B Mourrain Sparse representations from moments 20 / 29

21 Decomposition algorithm Input: The first coefficients (σ α ) α A of the series σ = y α r σ α α! = ω i e ξi (y) α N n i=1 1 Compute bases B, B x A st that H B,B invertible and B = B = r = dim A σ ; 2 Deduce the tables of multiplications M i := (H B,B σ ) 1 H B,x i B σ 3 Compute the eigenvectors v 1,, v r of i l im i for a generic l = l 1 x l n x n ; 4 Deduce the points ξ i = (ξ i,1,, ξ i,n ) st M j v i ξ i,j v i = 0 and the weights ω i = 1 v i (ξ i ) σ v i Output: The decomposition σ = r 1 i=1 v i (ξ i ) σ v i e ξi (y) B Mourrain Sparse representations from moments 21 / 29

22 Multivariate Prony method (1) Let h(t 1, t 2 ) = t 1 2 t 2 3 t 1, σ = yα α N2 h(α) α! = 2e (1,1) (y) + 3e (2,2) (y) e (3,1) (y) Take B = {1, x 1, x 2 } and compute h(0, 0) h(1, 0) h(0, 1) H 0 := Hσ B,B = h(1, 0) h(2, 0) h(1, 1) H 1 := H B,x1 B σ = h(0, 1) h(1, 1) h(0, 2) =, H 2 := H B,x2 B σ = , Compute the generalized eigenvectors of (ah 1 + bh 2, H 0 ): U = 1/2 0 1/2 and H 0 U = /2 1 1/ This yields the weights 2, 3, 1 and the roots (1, 1), (2, 2), (3, 1) B Mourrain Sparse representations from moments 22 / 29

Multivariate Prony method (2) h(t 1, t 2 ) := r i=1 ω ie f 1t 1 +f 2 t 2 with f := 05 10 + 3141592654 i 1375328890 + 09992349291 i 01 + 2136283005 i 15 + 3267256360 i 1046162168 + 03399186938 i ω :=

23 Multivariate Prony method (2) h(t 1, t 2 ) := r i=1 ω ie f 1t 1 +f 2 t 2 with f := i i i i i ω := i i i i For the sampling [ 1 50, ], B = {1, x 1, x 2, x 1 x 2 }, the SVD of Hσ B,B 92 [ , , , ] and the computed decomposition is f = i i i i i i i i i ω i = i i B Mourrain Sparse representations from moments 23 / 29 is

24 Sparse interpolation f (x) = r i=1 ω i x α i σ = γ f (ϕγ ) yγ γ! = r i=1 ω i e ϕ α i (y) Example: f (x 1, x 2 ) = x 33 1 x x 1x Compute σ α = f (ϕ α 1 1, ϕα 2 2 ) for α 1 + α 2 3 and ϕ 1 = ϕ 2 = e 2iπ Compute the Hankel matrix H 1,2 σ : i i i i i i i i i i i i i i 9 Deduce the decomposition of σ = 3 i=1 ω ie ξi : Ξ = i i e 11 i e 10 i i i ω = and the exponents 50 Ξ 2πi mod 50 of the terms of f : e 7 i e 6 i, e e 8 i e e 8 i e 6 i e 6 i e 7 i e 7 i e 8 i B Mourrain Sparse representations from moments 24 / 29

25 Symmetric tensor and apolarity Apolar product: For f = α =d f α ( d α) x α, g = α =d g α ( d α) x α K[x] d, f, g d = α d ( ) d f α g α α Property: f, (ξ 0 x ξ n x n ) d = f (ξ 0,, ξ n ) Duality: τ = α N n+1, α =d ( ) d τ α x α τ = α where (α 1,, α n ) = (d i α i, α 1,, α n ) N n+1 α N n, α d y α τ α α! τ(x) = r i=1 ω i (1 + ξ i,1 x ξ i,n x n ) d τ (y) = r i=1 ω i e ξi (y) + (y) d+1 B Mourrain Sparse representations from moments 25 / 29

Symmetric tensor decomposition τ = (x 0 x 1 + 3 x 2 ) 4 + (x 0 + x 1 + x 2 ) 4 3 (x 0 + 2 x 1 + 2 x 2 ) 4 = x 4 0 24 x 3 0 x 1 8 x 3 0 x 2 60 x 2 0 x 2 1 168 x 2 0 x 1 x 2 12 x 2 2 0 x 2 96 x 0 x 3 1

26 Symmetric tensor decomposition τ = (x 0 x x 2 ) 4 + (x 0 + x 1 + x 2 ) 4 3 (x x x 2 ) 4 = x x 3 0 x 1 8 x 3 0 x 2 60 x 2 0 x x 2 0 x 1 x 2 12 x x 2 96 x 0 x x 0 x 2 1 x x 0 x 1 x x 0 x x x 3 1 x x 2 1 x x 1 x x 2 τ = e ( 1,3) (y) + e (1,1) (y) 3e (2,2) (y) (by apolarity) For B = {1, x 2, x 1 }, H 2,2 τ := H B,B τ = , HB,x 1 B τ = , HB,x 2 B τ = B Mourrain Sparse representations from moments 26 / 29

27 The matrix of multiplication by x 1 in B = {1, x 2, x 1 } is M 1 = (H B,B τ ) 1 H B,x 1 B τ = Its eigenvalues are [ 1, 1, 2] and the eigenvectors: U := that is the polynomials U(x) = [ 1 2 x x x x x x 1 We deduce the weights and the frequencies: H [1,x 1,x 2 ],U τ = Weights: 1, 1, 3; frequencies: ( 1, 3), (1, 1), (2, 2) Decomposition: τ (y) = e ( 1,3) (y) + e (1,1) (y) 3 e (2,2) (y) + (y) 4 ] B Mourrain Sparse representations from moments 27 / 29

28 A general framework F the functional space, in which the signal lives S 1,, S n : F F commuting linear operators: S i S j = S j S i : h F [h] C a linear functional on F Generating series associated to h F: Eigenfunctions: Generalized eigenfunctions: σ h (y) = α N n [S α (h)] yα α! = α N n σ α y α α! S j (E) = ξ j E, j = 1,, n σ E = e ξ (y) S j (E k ) = ξ j E k + k <k m j,k E k ) σ Ek = ω i (y)e ξ (y) If h σ h is injective unique decomposition of f as a linear combination of generalized eigenfunctions B Mourrain Sparse representations from moments 28 / 29

29 Other applications Decomposition of measures as sums of spikes from moments (images, spectroscopy, radar, astronomy, ) Decomposition of convolution operators of finite rank Sparse interpolation of PolyLog functions Polynomial optimisation and convex relaxations Vanishing ideal of points: σ = r i=1 e ξ i (y) Change of ordering for Grobner basis or basis representation for zero-dimensional (Gorenstein) ideals: σ α = u, N(x α ), Challenges, open questions Numerical stability, correction of errors, Efficient construction of basis, complexity, Super-resolution, collision of points, Super-extrapolation, Best low rank approximation, Thanks for your attention B Mourrain Sparse representations from moments 29 / 29

arxiv: v3 [math.ag] 20 Oct 2017

arxiv: v3 [math.ag] 20 Oct 2017 Polynomial-exponential decomposition from moments arxiv:160905720v3 [mathag] 20 Oct 2017 Bernard Mourrain Université Côte d Azur, Inria, aromath, France bernardmourrain@inriafr October 23, 2017 Abstract