A Potpourri of Nonlinear Algebra

Transcription

1 a a 2 a 3 + c c 2 c 3 =2, a a 3 b 2 + c c 3 d 2 =0, a 2 a 3 b + c 2 c 3 d =0, a 3 b b 2 + c 3 d d 2 = 4, a a 2 b 3 + c c 2 d 3 =0, a b 2 b 3 + c d 2 d 3 = 4, a 2 b b 3 + c 2 d 3 d =4, b b 2 b 3 + d d 2 d 3 =0 d i = X j6=i i + j A Potpourri of Nonlinear Algebra Chris Hillar Det 2,2,2 (A) = 4 apple apple apple a000 a det 00 a00 a + 0 a 00 a 0 a 0 a apple a000 a det 00 a 00 a 0 apple a00 a 0 a 0 a 2 a c b d u 2, b c + a d, c u a 2 + b 2, d u 2a b, a u c 2 + d 2, b u 2d c, a 2 c 2 b 2 d 2 u 2, b 2 c 2 + a 2 d 2, c 2 u a b 2 2, d 2 u 2a 2 b 2, a 2 u c d 2 2, b 2 u 2d 2 c 2, a 3 c 3 b 3 d 3 u 2, b 3 c 3 + a 3 d 3, c 3 u a b 2 3, d 3 u 2a 3 b 3, a 3 u c d 2 3, b 3 u 2d 3 c 3, a 4 c 4 b 4 d 4 u 2, b 4 c 4 + a 4 d 4, c 4 u a b 2 4, d 4 u 2a 4 b 4, a 4 u c d 2 4, b 4 u 2d 4 c 4, a 2 b 2 + a a 3 b b 3 + a 2 3 b 2 3, a 2 b 2 + a a 4 b b 4 + a 2 4 b 2 4, a 2 b 2 + a a 2 b b 2 + a 2 2 b 2 2, a 2 2 b a 2 a 3 b 2 b 3 + a 2 3 b 2 3, a 2 3 b a 3 a 4 b 3 b 4 + a 2 4 b 2 4, 2a b + a b 2 + a 2 b +2a 2 b 2, 2a 2 b 2 + a 2 b 3 + a 3 b 2 +2a 3 b 3, 2a b + a b 3 + a 2 b +2a 3 b 3, 2a b + a b 4 + a 4 b +2a 4 b 4, 2a 3 b 3 + a 3 b 4 + a 4 b 3 +2a 4 b 4, w 2 + w w7 2 + w det apple a000 a 00 a 00 a 0 det apple a00 a 0 a 0 a. a 000 x 0 y 0 + a 00 x 0 y + a 00 x y 0 + a 0 x y =0, a 00 x 0 y 0 + a 0 x 0 y + a 0 x y 0 + a x y =0, a 000 x 0 z 0 + a 00 x 0 z + a 00 x z 0 + a 0 x z =0, a 00 x 0 z 0 + a 0 x 0 z + a 0 x z 0 + a x z =0, a 000 y 0 z 0 + a 00 y 0 z + a 00 y z 0 + a 0 y z =0, a 00 y 0 z 0 + a 0 y 0 z + a 0 y z 0 + a y z =0, ICERM Computational Nonlinear Algebra June 204

2 Outline Computational complexity of nonlinear algebra Real-life examples: Tensor problems - graph theory, optimization, Groebner bases Neuroscience: The Retina Equations - bipartite graphs, probability, matrix analysis

3 Computational Nonlinear Algebra Problem: Solve on a finite computer in finite time a finite set of polynomial (quadratic) equations computability Undecidable ( Uncomputable ) ring Z reference [Hilbert s 0th Problem] [Davis, Putnam, Robinson, Matijasevič 6/ 70]????? [Poonen 03] Q Decidable ( Computable ) R C [Tarski Seidenberg] [Hironaka 64, Buchberger 70]

4 Some Random Polynomial Systems: a) a a 2 a 3 + c c 2 c 3 =2, a a 3 b 2 + c c 3 d 2 =0, a 2 a 3 b + c 2 c 3 d =0, a 3 b b 2 + c 3 d d 2 = 4, a a 2 b 3 + c c 2 d 3 =0, a b 2 b 3 + c d 2 d 3 = 4, a 2 b b 3 + c 2 d 3 d =4, b b 2 b 3 + d d 2 d 3 =0 b) a c b d u 2, b c + a d, c u a 2 + b 2, d u 2a b, a u c 2 + d 2, b u 2d c, a 2 c 2 b 2 d 2 u 2, b 2 c 2 + a 2 d 2, c 2 u a b 2 2, d 2 u 2a 2 b 2, a 2 u c d 2 2, b 2 u 2d 2 c 2, a 3 c 3 b 3 d 3 u 2, b 3 c 3 + a 3 d 3, c 3 u a b 2 3, d 3 u 2a 3 b 3, a 3 u c d 2 3, b 3 u 2d 3 c 3, a 4 c 4 b 4 d 4 u 2, b 4 c 4 + a 4 d 4, c 4 u a b 2 4, d 4 u 2a 4 b 4, a 4 u c d 2 4, b 4 u 2d 4 c 4, a 2 b 2 + a a 3 b b 3 + a 2 3 b 2 3, a 2 b 2 + a a 4 b b 4 + a 2 4 b 2 4, a 2 b 2 + a a 2 b b 2 + a 2 2 b 2 2, a 2 2 b a 2 a 3 b 2 b 3 + a 2 3 b 2 3, a 2 3 b a 3 a 4 b 3 b 4 + a 2 4 b 2 4, 2a b + a b 2 + a 2 b +2a 2 b 2, 2a 2 b 2 + a 2 b 3 + a 3 b 2 +2a 3 b 3, 2a b + a b 3 + a 2 b +2a 3 b 3, 2a b + a b 4 + a 4 b +2a 4 b 4, 2a 3 b 3 + a 3 b 4 + a 4 b 3 +2a 4 b 4, w 2 + w w7 2 + w8. 2 c) a 000 x 0 y 0 + a 00 x 0 y + a 00 x y 0 + a 0 x y =0, a 00 x 0 y 0 + a 0 x 0 y + a 0 x y 0 + a x y =0, a 000 x 0 z 0 + a 00 x 0 z + a 00 x z 0 + a 0 x z =0, a 00 x 0 z 0 + a 0 x 0 z + a 0 x z 0 + a x z =0, a 000 y 0 z 0 + a 00 y 0 z + a 00 y z 0 + a 0 y z =0, a 00 y 0 z 0 + a 0 y 0 z + a 0 y z 0 + a y z =0,

5 A Briefer on Computational Complexity I. Model of Computation - What are inputs / outputs? - What is a computation? Alan Turing II. Model of Complexity - Cost of computation? Stephen Cook III. Model of Reducibility - What are equivalent problems? Dick Karp Leonid Levin

6 I. Model of Computation: Turing Machine [Turing 37][Turing 936] Inputs: finite list of rational numbers Outputs: YES/NO or rational vectors II. Model of Complexity: Time complexity Number of Tape-Level moves III. Model of Reducibility: Classes: P (polynomial-time), NP,, NP-complete, NP-hard,... Turing Machine (Mike Davey) NP-Hard input I polynomial-sized transformation input I P Tensor Problems NP-Complete P2 P P2 NP P Matrix Problems YES/NO = YES/NO the world of all computational problems

7 NP-complete decision problems [Cook-Karp-Levin 97/2] Graph coloring: Given graph G, is there a proper 3-coloring? YES NO is an NP-complete (can verify quickly) problem Million $$$ prize (Clay Math)

8 Connection to nonlinear algebra Theorem [Bayer 82]: Whether or not a graph is 3- colorable can be encoded as whether a system of cubic equations over C has a nonzero solution Reformulation [H., Lim 3]: Whether or not a graph G on v vertices with edges E is 3-colorable can be encoded as whether the following homogeneous quadratics has a nonzero solution in C C G = ( x i y i u 2, y i u x 2 i, x iu yi 2 P, i =,...,v, j:{i,j}2e (x2 i + x ix j + x 2 j ), i =,...,v. x i = a i + ib i y i = c i + id i Quadratic System over the reals R

9 Example: The following graph is 3-colorable:! 2 2 primitive cube root of! x i = 4 3! x i =! x i =! 2 The system has a (nonzero) solution over the reals: b) 35 homogeneous quadratics in 35 indeterminates: A 2 Q a c b d u 2, b c + a d, c u a 2 + b 2, d u 2a b, a u c 2 + d 2, b u 2d c, a 2 c 2 b 2 d 2 u 2, b 2 c 2 + a 2 d 2, c 2 u a b 2 2, d 2 u 2a 2 b 2, a 2 u c d 2 2, b 2 u 2d 2 c 2, a 3 c 3 b 3 d 3 u 2, b 3 c 3 + a 3 d 3, c 3 u a b 2 3, d 3 u 2a 3 b 3, a 3 u c d 2 3, b 3 u 2d 3 c 3, a 4 c 4 b 4 d 4 u 2, b 4 c 4 + a 4 d 4, c 4 u a b 2 4, d 4 u 2a 4 b 4, a 4 u c d 2 4, b 4 u 2d 4 c 4, a 2 b 2 + a a 3 b b 3 + a 2 3 b 2 3, a 2 b 2 + a a 4 b b 4 + a 2 4 b 2 4, a 2 b 2 + a a 2 b b 2 + a 2 2 b 2 2, a 2 2 b a 2 a 3 b 2 b 3 + a 2 3 b 2 3, a 2 3 b a 3 a 4 b 3 b 4 + a 2 4 b 2 4, 2a b + a b 2 + a 2 b +2a 2 b 2, 2a 2 b 2 + a 2 b 3 + a 3 b 2 +2a 3 b 3, 2a b + a b 3 + a 2 b +2a 3 b 3, 2a b + a b 4 + a 4 b +2a 4 b 4, 2a 3 b 3 + a 3 b 4 + a 4 b 3 +2a 4 b 4, w 2 + w w7 2 + w8. 2

10 Example: The following 2 graph is not 3-colorable: 4 3 The system does not have (nonzero) solution over : b) a 2 2 b a 2 a 4 b 2 b 4 + a 2 4 b 2 4, 2a 2 b 2 + a 2 b 4 + a 4 b 2 +2a 4 b 4 R a c b d u 2, b c + a d, c u a 2 + b 2, d u 2a b, a u c 2 + d 2, b u 2d c, a 2 c 2 b 2 d 2 u 2, b 2 c 2 + a 2 d 2, c 2 u a b 2 2, d 2 u 2a 2 b 2, a 2 u c d 2 2, b 2 u 2d 2 c 2, a 3 c 3 b 3 d 3 u 2, b 3 c 3 + a 3 d 3, c 3 u a b 2 3, d 3 u 2a 3 b 3, a 3 u c d 2 3, b 3 u 2d 3 c 3, a 4 c 4 b 4 d 4 u 2, b 4 c 4 + a 4 d 4, c 4 u a b 2 4, d 4 u 2a 4 b 4, a 4 u c d 2 4, b 4 u 2d 4 c 4, a 2 b 2 + a a 3 b b 3 + a 2 3 b 2 3, a 2 b 2 + a a 4 b b 4 + a 2 4 b 2 4, a 2 b 2 + a a 2 b b 2 + a 2 2 b 2 2, a 2 2 b a 2 a 3 b 2 b 3 + a 2 3 b 2 3, a 2 3 b a 3 a 4 b 3 b 4 + a 2 4 b 2 4, 2a b + a b 2 + a 2 b +2a 2 b 2, 2a 2 b 2 + a 2 b 3 + a 3 b 2 +2a 3 b 3, 2a b + a b 3 + a 2 b +2a 3 b 3, 2a b + a b 4 + a 4 b +2a 4 b 4, 2a 3 b 3 + a 3 b 4 + a 4 b 3 +2a 4 b 4, w 2 + w w7 2 + w8. 2

11 Example: The graph G below is uniquely 3-colorable [Example of Akbari, Mirrokni, Sadjad 0 disproving a conjecture of Xu 90] The coloring ideal is trivial (< 2 sec computation) I G [H., Windfeldt 08]

12 Tensor eigenvalues Problem: Given A =[[a ijk ]] 2 Q n n n find (x, ) with x 6= 0 such that: nx a ijk x i x j = x k, k =,...,n i,j= [Lim 2005], [Qi 2005], [Ni, et al 2007], [Qi 2007], [Cartwright and Sturmfels 202] Some Facts: Generic or random tensors over complex numbers have a finite number of eigenvalues and eigenvectors (up to scaling equivalence), although their count is exponential. Still, it is possible for a tensor to have an infinite number of non-equivalent eigenvalues, but in that case they comprise a cofinite set of complex numbers Another important fact is that over the reals, every 3-tensor has a real eigenpair.

13 Decision problem Problem: Given A =[[a ijk ]] 2 Q n n n and does there exist 2 Q 0 6= x 2 C n nx a ijk x i x j = x k, k =,...,n i,j= Decidable (Computable on a Turing machine): - Quantifier elimination - Buchberger s algorithm and Groebner bases - Multivariate resultants All quickly become inefficient as n grows Is there an efficient algorithm?

14 No, because Quadratic equations are hard to solve [Bayer 982], [Lovasz 994], [Grenet et al 200],... NP-Hard Tensor Problems Corollary: NP-Complete Deciding if =0 is a tensor eigenvalue is NP-hard NP P Matrix Problems x i =! ( 2, 3 2 ) 3 4 x i = Corollary: Unless P = NP, (, 0) there is no polynomial time approximation scheme for 3 4 finding tensor eigenvectors to within =3/4 x i =! 2

15 Computational complexity of tensor problems [H., Lim 3]

16 Tensor Rank rank tensors: A =[[x= x i y j z k z, ]] = x y z x, y, z 2 F n - Segre variety Definition: Tensor rank over the field rank F (A) =min {A = r F rx x i y i z i }. } i= Theorem [Hastad 90]: Tensor rank is NP-hard over is Q Note: rank can change over changing fields (not true linear algebra) Question: Is tensor rank different over reals / complex?

17 The following system has no solution over the rationals [Singular, Macaulay 2, Maple,...] a) a a 2 a 3 + c c 2 c 3 =2, a a 3 b 2 + c c 3 d 2 =0, a 2 a 3 b + c 2 c 3 d =0, a 3 b b 2 + c 3 d d 2 = 4, a a 2 b 3 + c c 2 d 3 =0, a b 2 b 3 + c d 2 d 3 = 4, a 2 b b 3 + c 2 d 3 d =4, b b 2 b 3 + d d 2 d 3 =0 A = z p z z + z z z p z = x + 2y, z = x 2y x =[, 0] >, y =[0, ] > Theorem [H. Lim 3] There are rational tensors with different rank over the rationals versus the reals rank R (A) < rank Q (A)

18 The 2 x 2 x 2 hyperdeterminant of a tensor is zero: (Defining equation for the dual variety to Segre variety) Det 2,2,2 (A) = 4 apple apple a000 a det 00 + a 00 a 0 apple a00 a 0 a 0 a apple a000 a det 00 a 00 a 0 apple a00 a 0 a 0 a 2 4 det apple a000 a 00 a 00 a 0 det apple a00 a 0 a 0 a. [Cayley 845] There exist (x, y, z) 6= 0 such that: c) a 000 x 0 y 0 + a 00 x 0 y + a 00 x y 0 + a 0 x y =0, a 00 x 0 y 0 + a 0 x 0 y + a 0 x y 0 + a x y =0, a 000 x 0 z 0 + a 00 x 0 z + a 00 x z 0 + a 0 x z =0, a 00 x 0 z 0 + a 0 x 0 z + a 0 x z 0 + a x z =0, a 000 y 0 z 0 + a 00 y 0 z + a 00 y z 0 + a 0 y z =0, a 00 y 0 z 0 + a 0 y 0 z + a 0 y z 0 + a y z =0, Conjecture: NP-hard to compute hyperdeterminant

19 and now for something completely different...

20 Neuroscience Motivation: Spike Coding of Continuous Signals Santiago Ramón y Cajal????? continuous signal in the world neural sensor circuit (retina) binary spike train (ganglion neurons)

21 Shannon Entropy (948) Theseus first robot mouse Claude Shannon Father of Entropy Theory

22 Given a distribution on a finite number of states: p i = probability of being in state i p =(p,...,p N ) Definition: The entropy of a distribution is H(p) = NX i= p i log p i - entropy is a measure of the uncertainty in a random variable - entropy provides an absolute limit on the best possible lossless encoding or compression of a communication

23 Given a distribution on a finite number of states: p i = probability of being in state i p =(p,...,p N ) Definition: The entropy of a distribution is NX H(p) = i= p i log p i apple log N - entropy is a measure of the uncertainty in a random variable - entropy provides an absolute limit on the best possible lossless encoding or compression of a communication (hence bounded above by log N)

24 Example: Flipping coins Heads Tails H(p) = NX i= p i log p i p H = 2 p T = 2 H(p H,p T )= 2 log log 2 = A fair coin flip has bit of entropy or information (n fair coin flips have n bits of entropy) Example: The uniform distribution has highest entropy p = N,..., N H(p) = log N

25 Example: Compressing letters over the interwebs: probability of letter in Oxford English Dictionary H = 4.2 bits (per letter) e t a o i n s h r d l c u m w f g y p b v k j x q z - So it takes 4.2 < 4.7 = log(26) bits per character to code English letters - E.g., e = 0, t = 0, a = 0,...

26 Maximum Entropy Statistical Physics (Jaynes, 957) Biology, Neuroscience (Bialek) Statistical =? Process New measurements Measurements MaxEnt Model most random / generic distribution given constraints / measurements

27 What about computation? Learning (development) Statistical Process Measurements MaxEnt Model new measurement??? Input Computation Output Neural Circuit Example: Neural Processing

28 Distributions on graphs p = /2 8 /2 2 /2 3 i.i.d. Bernoulli distribution on each edge Example: Erdős-Renyi (ER) distribution on graphs

29 Maximum Entropy for graphs Example: The maximum entropy distribution on graphs is the ER distribution p = /2 /2 2 /2 3

30 Expected degree sequence Given a distribution on graphs, can compute an expected degree sequence: E[d i ]=E[ X j6=i w ij ] Example: ER with p = /2 E[d i ]= n 2

31 Maximum Entropy for graphs Problem: Given an expected degree sequence d, what is the maximum entropy distribution on graphs with this expected degree sequence? Answer (classical): +e + 2 d = d 2 = d 3 = +e e e e + 2 +e 2+ +e e Erdős-Renyi =0 3 +e 2+ 3

32 Chatterjee-Diaconis-Sly (20) n =300,r =2 0.5 Persi Diaconis MLE estimate True θ Theorem: One sample of a graph from such a maximum entropy distribution determines the distribution for large n

33 Application: Binary Coding of Continuous Signals Original d expected degrees Single graph sample ˆd emperical degree sequence Reconstruction ˆ

34 What about graphs which have [H-Wibisono]: r different edge values {0,...,r-} nonnegative integer edges values {0,,...} nonnegative real values Edges are exponential random variables with means i + j d i = X j6=i i + j

35 Sanyal-Sturmfels-Vinzant (203) The Retina Equations : () d i = X j6=i i + j, i =,...,N Bernd Sturmfels Studied (more generally) using matroid theory and algebraic geometry

36 Theorem [H., Wibisono 3]: There is almost surely a unique nonnegative solution to any retina equation. Moreover, given one sample from a graph distribution, solving the equations gives original parameters for large n: r log n b apple C n with high probability Proof ingredients: () Large deviation theory (2) Inverses of special class of matrices (positive symmetric diagonally dominant matrices)

37 Diagonally Dominant Matrices Definition: A positive matrix is diagonally dominant if each off-diagonal row sum is at most the diagonal entry J = S 4 = Theorem [H., Lin, Wibisono]: For a positive, symmetric diagonally dominant (n x n) matrix J with smallest entry > : J apple S = 3n 4 2(n 2)(n )

38 bipartite bipartite non-bipartite G N N = lim t! (S + tp ) (P is the signless Laplacian of G)

39 Theorem [H., Wibisono 3]: There is almost surely a unique nonnegative solution to any retina equation. Moreover, given one sample from a graph distribution, solving the equations gives original parameters for large n: Proof sketch: F ( ) =(d,...,d n ) d i = X j6=i i + j F ( ) F (ˆ )+J ( ˆ ) p n log n apple C r log n ˆ apple J d ˆd apple C n n Matrix theorem ( J is the Jacobian of the map F ) Large deviation theory (strengthening of central limit theorem)

40 r log n ˆ apple J d ˆd apple C n F d ˆ F ˆd ḓ d ˆ - Original parameters of graph distribution - Expected degree sequence of graph distribution - Degrees computed from single graph sample - Parameters inferred from sampled degrees

41 Problems: Solve these other Retina Equations Maximum entropy graph distributions with binary edges: d i = X j6=i i j +, i =,...,N with nonnegative integer edges: d i = X j6=i i j, i =,...,N also others...

42 Redwood Center for Theoretical Neuroscience (U.C. Berkeley) Mathematical Sciences Research Institute National Science Foundation