On (Effective) Analytic Combinatorics

Transcription

1 On (Effective) Analytic Combinatorics Bruno Salvy Inria & ENS de Lyon Séminaire Algorithmes Dec. 2017

2 Combinatorics, Randomness and Analysis From simple local rules, a global structure arises. A quest for universality in random discrete structures: probabilistic complexity of structures and algorithms. Quantitative results using complex analysis. 1/20

3 Combinatorics, Randomness and Analysis Typical questions From simple local rules, a global structure arises. A quest for universality in random discrete structures: probabilistic complexity of structures and algorithms. Quantitative results using complex analysis. 1/20

4 Combinatorics, Randomness and Analysis From simple local rules, a global structure arises. A quest for universality in random discrete structures: probabilistic complexity of structures and algorithms. Typical questions For a tree of size n,. probability that a leaf is a child of the root? Quantitative results using complex analysis. 1/20

5 Combinatorics, Randomness and Analysis From simple local rules, a global structure arises. A quest for universality in random discrete structures: probabilistic complexity of structures and algorithms. Quantitative results using complex analysis. Typical questions For a tree of size n,. probability that a leaf is a child of the root?. expected pathlength (sum of distances from nodes to the root)? 1/20

6 Combinatorics, Randomness and Analysis From simple local rules, a global structure arises. A quest for universality in random discrete structures: probabilistic complexity of structures and algorithms. Quantitative results using complex analysis. Typical questions For a tree of size n,. probability that a leaf is a child of the root?. expected pathlength (sum of distances from nodes to the root)?. limiting distribution of it? 1/20

7 Philippe Flajolet, the Father of Analytic Combinatorics Planned complete works span 7 volumes of approx 600 pp. each Combinatorial specification Generating functions counting; random generation; asymptotic analysis. If you can specify, you can analyze. 2/20

8 Constructible Structures Language: 1,Z, +,, SEQ, SET, CYC and recursion. 3/20

9 Constructible Structures Language: 1,Z, +,, SEQ, SET, CYC and recursion. Binary trees: B=1+ZxBxB 3/20

10 Constructible Structures Language: 1,Z, +,, SEQ, SET, CYC and recursion. Binary trees: B=1+ZxBxB Permutations: Perm=SET(CYC(Z)); 3/20

11 Constructible Structures Language: 1,Z, +,, SEQ, SET, CYC and recursion. Binary trees: B=1+ZxBxB Permutations: Perm=SET(CYC(Z)); Trees: T=ZxSET(T); 3/20

12 Constructible Structures Language: 1,Z, +,, SEQ, SET, CYC and recursion. Binary trees: B=1+ZxBxB Permutations: Perm=SET(CYC(Z)); Trees: T=ZxSET(T); Functional graphs: F=SET(CYC(T)); 3/20

13 Constructible Structures Language: 1,Z, +,, SEQ, SET, CYC and recursion. Binary trees: B=1+ZxBxB Permutations: Perm=SET(CYC(Z)); Trees: T=ZxSET(T); Functional graphs: F=SET(CYC(T)); Series-parallel graphs: G=Z+S+P, S=SEQ >0 (Z+P),P=SET >0 (Z+S); 3/20

14 Constructible Structures Language: 1,Z, +,, SEQ, SET, CYC and recursion. Binary trees: B=1+ZxBxB Permutations: Perm=SET(CYC(Z)); Trees: T=ZxSET(T); Functional graphs: F=SET(CYC(T)); Series-parallel graphs: G=Z+S+P, S=SEQ >0 (Z+P),P=SET >0 (Z+S);...hundreds of examples in the purple book. 3/20

15 I. Enumeration and Generating Functions

16 I. Enumeration and Generating Functions 1X F (z) = f n z n n=0

17 Example: Binary Trees B=1+ZxBxB B(z) = 1X n=0 B n z n B 0 = B 1 =1 B 2 =2 B 3 =5 number of binary trees with n nodes (Catalan) 4/20

18 Example: Binary Trees B=1+ZxBxB B(z) = 1X n=0 B n z n B 0 = B 1 =1 B 2 =2 B 3 =5 number of binary trees with n nodes (Catalan) For n 1, B n = nx 1 k=0 B k B n k 1 4/20

19 Example: Binary Trees B=1+ZxBxB B(z) = 1X n=0 B n z n B 0 = B 1 =1 B 2 =2 B 3 =5 number of binary trees with n nodes (Catalan) For n 1, B n = nx 1 B k B n k 1 ) B n z n = nx 1 z(b k z k )(B n k 1 z n k 1 ) k=0 k=0 4/20

20 Example: Binary Trees B=1+ZxBxB B(z) = 1X n=0 B n z n B 0 = B 1 =1 B 2 =2 B 3 =5 number of binary trees with n nodes (Catalan) For n 1, B n = nx 1 B k B n k 1 ) B n z n = nx 1 z(b k z k )(B n k 1 z n k 1 ) k=0 k=0 B(z) =1+zB(z) 2 4/20

21 Example: Binary Trees B=1+ZxBxB B(z) = 1X n=0 B n z n B 0 = B 1 =1 B 2 =2 B 3 =5 number of binary trees with n nodes (Catalan) For n 1, B n = nx 1 B k B n k 1 ) B n z n = nx 1 z(b k z k )(B n k 1 z n k 1 ) k=0 k=0 B(z) =1+zB(z) 2 =1+z +2z 2 +5z 3 + 4/20

22 Example: Binary Trees B=1+ZxBxB B(z) = 1X n=0 B n z n B 0 = B 1 =1 B 2 =2 B 3 =5 number of binary trees with n nodes (Catalan) For n 1, B n = nx 1 B k B n k 1 ) B n z n = nx 1 z(b k z k )(B n k 1 z n k 1 ) k=0 k=0 B(z) =1+zB(z) 2 =1+z +2z 2 +5z 3 + 4/20

23 What do we count? Inv[{1, 2, 3}] = { } ,,, involutions; 3 of them permuted by S 3 2 unlabelled structures. F (z) = F (z) = Exponential generating series (EGF): 1X n=0 f n z n n!, f n = nb. labelled structs of size n. Ordinary generating series (OGF): 1X n=0 f n z n, fn = nb. unlabelled structs of size n. Inv 3 (z) = 2 3 z3 Inv f 3 (z) =2z 3 5/20

24 A Dictionary for Generating Series Language: 1,Z, +,, SEQ, SET, CYC and recursion. describes constructible structures 6/20

25 A Dictionary for Generating Series Language: 1,Z, +,, SEQ, SET, CYC and recursion. describes constructible structures Structure EGF OGF A+B A(z)+B(z) Ã(z)+ B(z) AxB A(z) B(z) Ã(z) B(z) 6/20

26 A Dictionary for Generating Series Language: 1,Z, +,, SEQ, SET, CYC and recursion. describes constructible structures Structure EGF OGF A+B A(z)+B(z) Ã(z)+ B(z) AxB A(z) B(z) Ã(z) B(z) SEQ(A) 1 1 A(z) 1 1 Ã(z) 6/20

27 A Dictionary for Generating Series Language: 1,Z, +,, SEQ, SET, CYC and recursion. describes constructible structures Structure EGF OGF A+B A(z)+B(z) Ã(z)+ B(z) AxB A(z) B(z) Ã(z) B(z) SEQ(A) 1 1 A(z) 1 1 Ã(z) SET(A) exp(a(z)) exp( X Ã(z i )/i) 6/20

28 A Dictionary for Generating Series Language: 1,Z, +,, SEQ, SET, CYC and recursion. describes constructible structures Structure EGF OGF A+B A(z)+B(z) Ã(z)+ B(z) AxB A(z) B(z) Ã(z) B(z) SEQ(A) 1 1 A(z) 1 1 Ã(z) SET(A) exp(a(z)) exp( X Ã(z i )/i) CYC(A) log 1 1 A(z) X (i) i log 1 1 Ã(z i ) 6/20

29 Examples 7/20

30 Examples Binary trees: B=1+ZxBxB! B(z) =1+zB(z) 2 = B(z) 7/20

31 Examples Binary trees: B=1+ZxBxB! B(z) =1+zB(z) 2 = B(z) Cayley trees: T=ZxSET(T)! T (z) =z exp(t (z));! T (z) =z exp T (z)+ 1 T (z 2 )+ 1 T (z 3 ) /20

32 Examples Binary trees: B=1+ZxBxB! B(z) =1+zB(z) 2 = B(z) Cayley trees: T=ZxSET(T)! T (z) =z exp(t (z));! T (z) =z exp T (z)+ 1 T (z 2 )+ 1 T (z 3 )+ 2 3 Series-parallel graphs:! G = Z + S + P, S = Seq >0 (Z + P), P = Set >0 (Z + S) G(z) =z + S(z)+P (z),s(z) = 1 1 z P (z) 1,P(z) =e z+s(z) 1 7/20

33 II. Newton Iteration and Fast Enumeration 1671

34 Numerical Newton Iteration To solve (y) =0, iterate (y) =1+zy 2 y with y [n+1] = y [n] + u [n], (y [n] )+ 0 (y [n] )u [n] =0. z =0.23 8/20

35 Numerical Newton Iteration To solve (y) =0, iterate (y) =1+zy 2 y with y [n+1] = y [n] + u [n], (y [n] )+ 0 (y [n] )u [n] =0. y [n+1] = N (y [n] )=y [n] + 1+zy[n]2 1 2zy [n] y [n] z =0.23 8/20

36 Numerical Newton Iteration To solve (y) =0, iterate (y) =1+zy 2 y with y [n+1] = y [n] + u [n], (y [n] )+ 0 (y [n] )u [n] =0. y [n+1] = N (y [n] )=y [n] + 1+zy[n]2 1 2zy [n] y [n] z =0.23 y [0] =0, Quadratic convergence y [1] = , y [2] ' , y [3] ' , y [4] ' , y [5] ' , y [6] ' /20

37 Newton Iteration for Power Series Same Newton Iteration y [n+1] = N (y [n] )=y [n] + 1+zy[n]2 1 2zy [n] y [n] 9/20

38 Newton Iteration for Power Series Same Newton Iteration y [n+1] = N (y [n] )=y [n] + 1+zy[n]2 1 2zy [n] y [n] y [0] =0 y [1] =1 y [2] =1+z +2z 2 +4z 3 +8z z z z 7 + y [3] =1+z +2z 2 +5z z z z z 7 + 9/20

39 Newton Iteration for Power Series Same Newton Iteration y [n+1] = N (y [n] )=y [n] + 1+zy[n]2 1 2zy [n] y [n] y [0] =0 y [1] =1 y [2] =1+z +2z 2 +4z 3 +8z z z z 7 + y [3] =1+z +2z 2 +5z z z z z 7 + 9/20

40 Newton Iteration for Power Series Same Newton Iteration y [n+1] = N (y [n] )=y [n] + 1+zy[n]2 1 2zy [n] y [n] y [0] =0 y [1] =1 Proving numerical convergence requires control over the tails y [2] =1+z +2z 2 +4z 3 +8z z z z 7 + y [3] =1+z +2z 2 +5z z z z z 7 + 9/20

41 Newton Iteration for Power Series Same Newton Iteration y [n+1] = N (y [n] )=y [n] + 1+zy[n]2 1 2zy [n] y [n] y [0] =0 y [1] =1 Proving numerical convergence requires control over the tails y [2] =1+z +2z 2 +4z 3 +8z z z z 7 + y [3] =1+z +2z 2 +5z z z z z 7 + On power series: y y [1] = O(z m )=) N (y) y [1] = O(z 2m(+1) ) 9/20

42 Newton Iteration for Power Series Same Newton Iteration y [n+1] = N (y [n] )=y [n] + 1+zy[n]2 1 2zy [n] y [n] y [0] =0 y [1] =1 Proving numerical convergence requires control over the tails y [2] =1+z +2z 2 +4z 3 +8z z z z 7 + y [3] =1+z +2z 2 +5z z z z z 7 + On power series: y y [1] = O(z m )=) N (y) y [1] = O(z 2m(+1) ) 9/20

43 Newton Iteration for Power Series Same Newton Iteration y [n+1] = N (y [n] )=y [n] + 1+zy[n]2 1 2zy [n] y [n] y [0] =0 y [1] =1 Proving numerical convergence requires control over the tails y [2] =1+z +2z 2 +4z 3 +8z z z z 7 + y [3] =1+z +2z 2 +5z z z z z 7 + On power series: y y [1] = O(z m )=) N (y) y [1] = O(z 2m(+1) ) Expand(N) = { res=expand(n/2); a=ɸ(res); b=ɸ (res); u=solve(a+bx,x); return res+u; } 9/20

44 Newton Iteration for Power Series Same Newton Iteration y [n+1] = N (y [n] )=y [n] + 1+zy[n]2 1 2zy [n] y [n] y [0] =0 y [1] =1 Proving numerical convergence requires control over the tails y [2] =1+z +2z 2 +4z 3 +8z z z z 7 + y [3] =1+z +2z 2 +5z z z z z 7 + On power series: y y [1] = O(z m )=) N (y) y [1] = O(z 2m(+1) ) Expand(N) = { res=expand(n/2); a=ɸ(res); b=ɸ (res); u=solve(a+bx,x); return res+u; } Cost(N) ct Cost(last step) 9/20

45 Fast Enumeration Thm. First N coefficients of GFs of all constructible structures: 1. arithmetic complexity O(N log N) (both ogf & egf); 2. bit complexity O(N 2 log 2 NloglogN) (ogf); O(N 2 log 3 NloglogN) (egf). Ingredients: Newton iteration & FFT. [Pivoteau-S-Soria 2012] 10/20

46 Fast Enumeration Thm. First N coefficients of GFs of all constructible structures: 1. arithmetic complexity O(N log N) (both ogf & egf); 2. bit complexity O(N 2 log 2 NloglogN) (ogf); O(N 2 log 3 NloglogN) (egf). Ingredients: Newton iteration & FFT. [Pivoteau-S-Soria 2012] Demo NewtonGF 10/20

47 Derivative of Combinatorial Structures (origin: species theory) F F 11/20

48 Derivative of Combinatorial Structures (origin: species theory) F F (F+G) =F +G ;(FXG) =F XG+FXG ; 11/20

49 Derivative of Combinatorial Structures (origin: species theory) F F (F+G) =F +G ;(FXG) =F XG+FXG ; F(G) =F (G)XG ; 11/20

50 Derivative of Combinatorial Structures (origin: species theory) F F (F+G) =F +G ;(FXG) =F XG+FXG ; F(G) =F (G)XG ; 0 =1 =0; Z =1; SET =SET; CYC =SEQ; SEQ =SEQXSEQ. 11/20

51 Combinatorial Newton Iteration Y =1+Z Y Y =: H(Z, Y) [Décoste, Labelle, Leroux 1982] 12/20

52 Combinatorial Newton Iteration Y =1+Z Y Y =: H(Z, Y) Y [n+1] = Y [n] + Seq(Z Y [n]? + Z? Y [n] ) ((1 + Z Y [n] Y [n] ) \Y [n] ). [Décoste, Labelle, Leroux 1982] 12/20

53 Combinatorial Newton Iteration Y =1+Z Y Y =: H(Z, Y) Y [n+1] = Y [n] + Seq(Z Y [n]? + Z? Y [n] ) ((1 + Z Y [n] Y [n] ) \Y [n] ). (Z, Y[n] ) [Décoste, Labelle, Leroux 1982] 12/20

54 Combinatorial Newton Iteration Y =1+Z Y Y =: H(Z, Y) Y [n+1] = Y [n] + Seq(Z Y [n]? + Z? Y [n] ) ((1 + Z Y [n] Y [n] ) \Y [n] ). (Z, Y[n] ) [Décoste, Labelle, Leroux 1982] Ccl: numerical convergence of the Newton iteration starting from 0. 12/20

55 III. Random Generation

56 Why Random Generation? Simulation in the discrete world; helps evaluate parameters; compare/refine models; test software. 13/20

57 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X n T n x n x is a parameter of the algorithm 14/20

58 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) 14/20

59 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

60 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

61 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

62 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

63 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

64 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

65 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

66 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

67 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

68 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

69 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

70 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

71 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

72 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

73 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

74 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

75 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

76 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

77 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

78 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

79 Boltzmann Sampling Proba(t)= x t T (x) with T (x) = X t2t x t = X T n x n n x is a parameter of the algorithm Expected size: xt 0 (x) T (x) Generate(h H) = { if H=1 return 1; if H=Z return Z; if H=FxG { Generate(f F); Generate(g G); return (f,g); } if H=F+G { b:=bernoulli(f(x)/h(x)); if b=1 return Generate(f F); else return Generate(g G); if H=Set(F) { Example: binary trees B=1+ZxBxB with 1/B(.23) 1/ Coin: 0,0,1,0,1,1,1,1,0,1, Exp. size 1 2 p 1 4x F(x),H(x) by Newton iteration [Duchon, Flajolet, Louchard, Schaeffer ] 14/20

80 Example: XML-trees Grammar File size #eq. Newton rss 9.5k s. Relax NG 124k s. XSLT 168k s. XHTML Basic 284k s. SVG 6.3M s. OpenDocument 2.8M s. DocBook 11M s. Time for x s.t. E(size)=10,000 [Darrasse2010] 15/20

81 IV. Asymptotic Analysis 1X F (z) = f n z n f n, n!1 n=0

82 IV. Asymptotic Analysis 1X F (z) = f n z n f n, n!1 n=0

83 Singularity Analysis counts the number of objects of size n (a n ) 7! A(z) := X n 0 a n z n captures some structure A 3-Step Method: a n = 1 2 i I A(z) dz zn+1 16/20

84 Singularity Analysis counts the number of objects of size n (a n ) 7! A(z) := X n 0 a n z n captures some structure A 3-Step Method: 1. Locate dominant singularities 2. Compute local behaviour 3. Translate into asymptotics a n = 1 2 i I A(z) dz zn+1 16/20

85 Singularity Analysis counts the number of objects of size n (a n ) 7! A(z) := X n 0 a n z n captures some structure A 3-Step Method: 1. Locate dominant singularities 2. Compute local behaviour 3. Translate into asymptotics a n = 1 2 i I A(z) dz zn+1 A(z) z! c 1 z log m 1 1 z a n c n n 1 n!1 ( ) logm n ( 62 N) 16/20

86 Singularity Analysis counts the number of objects of size n (a n ) 7! A(z) := X n 0 a n z n captures some structure A 3-Step Method: 1. Locate dominant singularities 2. Compute local behaviour 3. Translate into asymptotics a n = 1 2 i I A(z) dz zn+1 A(z) z! c 1 z log m 1 1 z a n full asymptotic expansion available c n n 1 n!1 ( ) logm n ( 62 N) 16/20

87 Example: Binary Trees (Again!) B(z) =1+zB(z) 2 =1+z +2z 2 +5z 3 + = 1 p 1 4z 2z A 3-Step Method: 1. Locate dominant singularities; 2. Compute local behaviour; 3. Translate into asymptotics. 1. sing. en z =1/4 2. local behaviour: B(z) = z!1/4 2 2(1 4z)1/ translation: B n 4n n 3/2 p 1 9 8n + 17/20

88 Example: Binary Trees (Again!) B(z) =1+zB(z) 2 =1+z +2z 2 +5z 3 + = 1 p 1 4z 2z 1. sing. en z =1/4 2. local behaviour: B(z) = z!1/4 2 2(1 4z)1/ translation: B n 4n n 3/2 p 1 9 8n + A 3-Step Method: 1. Locate dominant singularities; 2. Compute local behaviour; 3. Translate into asymptotics. Probability that a leaf is a child of the root? [z n ]2zB(z) B n = n + O n 2 17/20

89 Example: Binary Trees (Again!) B(z) =1+zB(z) 2 =1+z +2z 2 +5z 3 + = 1 p 1 4z 2z 1. sing. en z =1/4 2. local behaviour: B(z) = z!1/4 2 2(1 4z)1/ translation: B n 4n n 3/2 p 1 9 8n + A 3-Step Method: 1. Locate dominant singularities; 2. Compute local behaviour; 3. Translate into asymptotics. Probability that a leaf is a child of the root? [z n ]2zB(z) B n = n + O n 2 Demo. 17/20

90 Parameters 18/20

91 Parameters Equations over combinatorial structures + parameters Ex.: path length in binary trees 18/20

92 Parameters Equations over combinatorial structures + parameters Ex.: path length in binary trees Multivariate generating series F (z,u) = X n,k f n,k u k z n 18/20

93 Parameters Equations over combinatorial structures + parameters Ex.: path length in binary trees Multivariate generating series F (z,u) = X n,k f n,k u k z n B(z,u) = X b2b u pl(b) z b 18/20

94 Parameters Equations over combinatorial structures + parameters Multivariate generating series X F (z,u) = f n,k u k z n n,k Ex.: path length in binary trees B(z,u) = X b2b u pl(b) z b P u=1 18/20

95 Parameters Equations over combinatorial structures + parameters Multivariate generating series X F (z,u) = f n,k u k z n n,k Complex analysis f n,k,n!1 Ex.: path length in binary trees B(z,u) = X b2b u pl(b) z b P u=1 18/20

96 Parameters Equations over combinatorial structures + parameters Multivariate generating series X F (z,u) = f n,k u k z n n,k Complex analysis f n,k,n!1 Ex.: path length in binary trees B(z,u) = X b2b u pl(b) z b P u=1 P n nb n p n 18/20

97 Parameters Equations over combinatorial structures + parameters Multivariate generating series X F (z,u) = f n,k u k z n n,k Complex analysis f n,k,n!1 Ex.: path length in binary trees B(z,u) = X b2b u pl(b) z b P u=1 P n nb n p n Limiting distribution 18/20

98 Parameters Equations over combinatorial structures + parameters Multivariate generating series X F (z,u) = f n,k u k z n n,k Complex analysis f n,k,n!1 Ex.: path length in binary trees B(z,u) = X b2b u pl(b) z b P u=1 P n nb n p n Limiting distribution Demo. 18/20

99 If you can specify it, you can analyze it! Permutations Mappings Words Strings Urns Trees Languages Integers Compositions Partitions + parameters Universality phenomena: Ex.: # trees of various types K n n 3/2 pathlength in n 1/2 19/20

100 Summary & Conclusion 20/20

101 Summary & Conclusion What we have: Combinatorial specification counting; random generation; asymptotic analysis. Automated? (not fully) 20/20

102 Summary & Conclusion What we have: Combinatorial specification Ultimate (dream) goal: counting; random generation; asymptotic analysis. Automated? (not fully) Combinatorial specification + Algorithm Automatic complexity analysis 20/20

103 Summary & Conclusion What we have: Combinatorial specification Ultimate (dream) goal: counting; random generation; asymptotic analysis. Automated? (not fully) Combinatorial specification + Algorithm Automatic complexity analysis Where to learn more: On-line course by R. Sedgewick at 20/20

104 Summary & Conclusion What we have: Combinatorial specification counting; random generation; asymptotic analysis. Automated? (not fully) Ultimate (dream) goal: Combinatorial specification + Algorithm Where to learn more: The End On-line course by R. Sedgewick at Automatic complexity analysis 20/20