Asymptotic and Exact Poissonized Variance in the Analysis of Random Digital Trees (joint with Hsien-Kuei Hwang and Vytas Zacharovas)

Transcription

1 Asymptotic and Exact Poissonized Variance in the Analysis of Random Digital Trees (joint with Hsien-Kuei Hwang and Vytas Zacharovas) Michael Fuchs Institute of Applied Mathematics National Chiao Tung University June 22nd, 2017 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

2 Digital Trees Three standard types: Trie: René de la Briandais, 1959; PATRICIA Trie: Donald R. Morrison, 1968; Digital Search Tree: Coffman and Eve, Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

3 Digital Trees Three standard types: Trie: René de la Briandais, 1959; PATRICIA Trie: Donald R. Morrison, 1968; Digital Search Tree: Coffman and Eve, Example: R 1 =0010, R 2 =0011, R 3 =1100, R 4 =1010 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

4 Digital Trees Three standard types: Trie: René de la Briandais, 1959; PATRICIA Trie: Donald R. Morrison, 1968; Digital Search Tree: Coffman and Eve, Example: R 1 =0010, R 2 =0011, R 3 =1100, R 4 = R 4 R R 1 R 2 Trie Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

5 Digital Trees Three standard types: Trie: René de la Briandais, 1959; PATRICIA Trie: Donald R. Morrison, 1968; Digital Search Tree: Coffman and Eve, Example: R 1 =0010, R 2 =0011, R 3 =1100, R 4 = R 4 R R 1 R 2 R 4 R R 1 R 2 PATRICIA trie Trie Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

6 Digital Trees Three standard types: Trie: René de la Briandais, 1959; PATRICIA Trie: Donald R. Morrison, 1968; Digital Search Tree: Coffman and Eve, Example: R 1 =0010, R 2 =0011, R 3 =1100, R 4 = R R 4 R R 2 R 3 0 R 1 R 2 R 4 R 3 R R 1 R 2 PATRICIA trie DST Trie Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

7 Random Digital Trees Bernoulli model: Bits are iid with P(bit = 1) = p and P(bit = 0) = q := 1 p. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

8 Random Digital Trees Bernoulli model: Bits are iid with P(bit = 1) = p and P(bit = 0) = q := 1 p. Gives two types: p = 1/2: symmetric digital tree; p 1/2: asymmetric digital tree. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

9 Random Digital Trees Bernoulli model: Bits are iid with Gives two types: P(bit = 1) = p and P(bit = 0) = q := 1 p. p = 1/2: symmetric digital tree; p 1/2: asymmetric digital tree. Bernoulli model is simple and prototypical but not very realistic! J. Clément, P. Flajolet, B. Vallée (2001). Dynamical sources in information theory: A general analysis of trie structures, Algorithmica, 29:1-2, Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

10 Shape Parameters Size S n Number of internal nodes in tries; for the other two standard types the size is deterministic. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

11 Shape Parameters Size S n Number of internal nodes in tries; for the other two standard types the size is deterministic. Path Lengths L n Internal path length, external path length, total path length, weighted path length, peripheral path lengths, etc. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

12 Shape Parameters Size S n Number of internal nodes in tries; for the other two standard types the size is deterministic. Path Lengths L n Internal path length, external path length, total path length, weighted path length, peripheral path lengths, etc. Depth Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

13 Shape Parameters Size S n Number of internal nodes in tries; for the other two standard types the size is deterministic. Path Lengths L n Internal path length, external path length, total path length, weighted path length, peripheral path lengths, etc. Depth Number of leaves and patterns Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

14 Shape Parameters Size S n Number of internal nodes in tries; for the other two standard types the size is deterministic. Path Lengths L n Internal path length, external path length, total path length, weighted path length, peripheral path lengths, etc. Depth Number of leaves and patterns External and Internal Profiles Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

15 Shape Parameters Size S n Number of internal nodes in tries; for the other two standard types the size is deterministic. Path Lengths L n Internal path length, external path length, total path length, weighted path length, peripheral path lengths, etc. Depth Number of leaves and patterns External and Internal Profiles Height Etc. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

16 Additive Shape Parameters X n+b d = XIn + X n I n + T n I n d = Binomial(n, p); Root 0 1 X n d = X n ; X n, X n, (I n, T n ) independent. T n toll-function. Size: I n Size: n I n Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

17 Additive Shape Parameters X n+b d = XIn + X n I n + T n I n d = Binomial(n, p); Root 0 1 X n d = X n ; X n, X n, (I n, T n ) independent. T n toll-function. Size: I n Size: n I n Thus, moments satisfy the recurrence: n ( ) n a n+b = p j q n j (a j + a n j ) + b n. j j=0 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

18 Two (Main) Approaches Rice-Noerlund Method: Exercise 54 in Section of Knuth s book. Developed into a systematic tool by Flajolet & Sedgewick (1986, 1995). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

19 Two (Main) Approaches Rice-Noerlund Method: Exercise 54 in Section of Knuth s book. Developed into a systematic tool by Flajolet & Sedgewick (1986, 1995). Two-stage Approach Introduced by Jacquet & Régnier (1988). Important contributions by Jacquet & Szpankowski (1998); Flajolet, Gourdon & Dumas (1995). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

20 Two (Main) Approaches Rice-Noerlund Method: Exercise 54 in Section of Knuth s book. Developed into a systematic tool by Flajolet & Sedgewick (1986, 1995). Two-stage Approach Introduced by Jacquet & Régnier (1988). Important contributions by Jacquet & Szpankowski (1998); Flajolet, Gourdon & Dumas (1995). Poisson-Mellin- Newton cycle: Poisson GF a n Noerlund integral f(z) Mellin transform F (s) Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

21 Rice-Noerlund Method Lemma (Noerlund) Let C be a positive oriented, closed curve encircling 0,..., n and f(z) be analytic within C. Then, n ( ) n ( 1) k f(k) = 1 n!γ( s) f(s) k 2πi C Γ(n + 1 s) ds. k=0 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

22 Rice-Noerlund Method Lemma (Noerlund) Let C be a positive oriented, closed curve encircling 0,..., n and f(z) be analytic within C. Then, n ( ) n ( 1) k f(k) = 1 n!γ( s) f(s) k 2πi C Γ(n + 1 s) ds. k=0 For additive trie parameters: where f(s) = F ( s) Γ( s) = G( s) (1 p s q s )Γ( s). F (s) = M e z a n z n /n!; s, n 0 G(s) = M e z b n z n /n!; s. n 0 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

23 Mean of Size in Symmetric Tries (i) We have, f(s) = (s 1)/(1 2 1 s ). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

24 Mean of Size in Symmetric Tries (i) We have, f(s) = (s 1)/(1 2 1 s ). Theorem For n 2, E(S n ) = n log 2 1 log 2 k 0 c k n! Γ(n + χ k ) 1, where with χ k = 2kπi/(log 2). c k = χ k Γ( 1 + χ k ). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

25 Mean of Size in Symmetric Tries (i) We have, f(s) = (s 1)/(1 2 1 s ). Theorem For n 2, E(S n ) = n log 2 1 log 2 k 0 c k n! Γ(n + χ k ) 1, where with χ k = 2kπi/(log 2). c k = χ k Γ( 1 + χ k ). Note that n! Γ(n + χ k ) n1 χ k + (1 χ k)χ k n χ k +. 2 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

26 Mean of Size in Symmetric Tries (ii) E(S n) n (n = 1,..., 1024) Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

27 Mean of Size in Symmetric Tries (ii) E(S n) n (n = 1,..., 1024) E(S n)+1 n (n = 1,..., 1024) Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

28 Variance via Rice-Noerlund Method Symmetric digital trees: done by Kirschenhofer & Prodinger & Szpankowski in a series of papers in the beginning of 90s. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

29 Variance via Rice-Noerlund Method Symmetric digital trees: done by Kirschenhofer & Prodinger & Szpankowski in a series of papers in the beginning of 90s. Second Moment Method: Use Rice-Noerlund twice for mean and second moment. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

30 Variance via Rice-Noerlund Method Symmetric digital trees: done by Kirschenhofer & Prodinger & Szpankowski in a series of papers in the beginning of 90s. Second Moment Method: Use Rice-Noerlund twice for mean and second moment. Compute the variance via Var(X) = E(X 2 ) (E(X)) 2. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

31 Variance via Rice-Noerlund Method Symmetric digital trees: done by Kirschenhofer & Prodinger & Szpankowski in a series of papers in the beginning of 90s. Second Moment Method: Use Rice-Noerlund twice for mean and second moment. Compute the variance via Var(X) = E(X 2 ) (E(X)) 2. Here, one needs identities of Ramanjuan: e.g. for α and β with αβ = π 2, we have 1 k(e 2αk 1) 1 4 log α + α 12 k 1 = k 1 1 k(e 2βk 1) 1 4 log β + β 12 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

32 Variance of Size in Symmetric Tries (i) Theorem (Kirschenhofer & Prodinger) We have, Var(S n ) n log 2 G( 1 + χ k )n χ k, k Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

33 Variance of Size in Symmetric Tries (i) Theorem (Kirschenhofer & Prodinger) We have, where Var(S n ) n log 2 G( 1 + χ k )n χ k, k G( 1 + χ k ) = 3χ k Γ( 1 + χ k ) χ kγ(1 + χ k ) log 2 (1 χ k )(2 χ k )Γ(χ k ) 1 2 (χ k + j) ( χ ) k j 1 (j + 1)(2 j 1) j 1 2Γ(1 + χ k ) 5 χ k 4(1 χ k ) (χ k + j + 1) ( χ k 1) j 1 (j + 1)(2 j 1) j log 2 j+m=k j,m 0 χ j Γ( 1 + χ j )χ mγ(1 + χ m). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

34 Variance of TPL in Symmetric DSTs (i) Theorem (Kirschenhofer, Prodinger, Szpankowski) We have, Var(L n ) n(c var + P (log 2 n)), where P (z) is a 1-periodic function with zero average value. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

35 Variance of TPL in Symmetric DSTs (i) Theorem (Kirschenhofer, Prodinger, Szpankowski) We have, Var(L n ) n(c var + P (log 2 n)), where P (z) is a 1-periodic function with zero average value. To describe C var we need the following: Let L = log 2, Q j = j i=1 (1 2 i ), Q(1) = lim j Q j. Put ω(z) = 1 Γ( 1 χ l )e 2lπiz ; L ψ(z) = 1 L l 0 ( l 0 1 χ l 2 ) Γ( χ l )e 2lπiz. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

36 C var = 28 3L l 1 l2 l (2 l 1) L l 1 2 ( 1) l+1 (l 5) L (l + 1)l(l 1)(2 l 1) l 3 2Q(1) L + l l Q l 1 2 l 1 + π2 2L L ( ) ( 1) l 2 l+1 2 L(1 2 l+1 )/2 1 L 1 2 l ( 1) r+1 r(r 1)(2 r+l 1) l 1 r 2 ( ) ( 1) r 2 r+1 2 ( j j+r+l+2 1 Q r+l 2 Q r 0 r ( l 1 ( l + 1 ) 2 l+1 2l i i=2 2 (1 2 l r ) 2 + 2l + 2 (1 2 1 l r ) 2 2 L l+1 ( l + 1 ) 2 j j=2 1 2 r+j L j=0 i r+i l r + 2 l+1 ( l + 1 ) L j j=1 l+1 ( l + 1 ) ( 1) i ) j (i + 1)(2 r+j+i 1) + l 1 ( l + 1 ) Qr 2 Q l r 1 1 r 2 l 3 r=2 l Q l 2 j 1 2[ωψ] 0 [ω 2 ] 0. j l+1 ) 1 2 r+j 1 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

37 Bernoulli and Poisson Model Bernoulli Model: Digital tree is built from n records. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

38 Bernoulli and Poisson Model Bernoulli Model: Digital tree is built from n records. Poisson Model: Digital tree is built from Poi(n) records. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

39 Bernoulli and Poisson Model Bernoulli Model: Digital tree is built from n records. Poisson Model: Digital tree is built from Poi(n) records. We have, E(X Poi(n) ) = e n j 0 E(X j ) nj j! = f 1 (n). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

40 Bernoulli and Poisson Model Bernoulli Model: Digital tree is built from n records. Poisson Model: Digital tree is built from Poi(n) records. We have, E(X Poi(n) ) = e n j 0 E(X j ) nj j! = f 1 (n). Poisson Heuristic: f 1 (n) sufficiently smooth = E(X Poi(n) ) E(X n ). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

41 Poisson-Charlier Expansion Let f(z) = e z n 0 a n z n n!. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

42 Poisson-Charlier Expansion Let f(z) = e z n 0 a n z n n!. Lemma (F., Hwang, Zacharovas) Let f(z) be entire, then a n = j 0 τ j (n) j! f (j) (n), where τ j (n) = n j e z. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

43 Poisson-Charlier Expansion Let f(z) = e z n 0 a n z n n!. Lemma (F., Hwang, Zacharovas) Let f(z) be entire, then a n = j 0 τ j (n) j! f (j) (n), where τ j (n) = n j e z. τ 0 (z) τ 1 (z) τ 2 (z) τ 3 (z) τ 4 (z) τ 5 (z) 1 0 n 2n 3n(n 2) 4n(5n 6) Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

44 J S -admissibility We say f(z) J S α,β if (I) Uniformly for arg(z) ɛ, f(z) = O ( z α log β z ). (O) Uniformly for ɛ < arg(z) π, f(z) := e z ( f(z) = O e (1 ɛ) z ). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

45 J S -admissibility We say f(z) J S α,β if (I) Uniformly for arg(z) ɛ, (O) Uniformly for ɛ < arg(z) π, Theorem (Jacquet & Szpankowski) If f(z) J S α,β, then f(z) = O ( z α log β z ). f(z) := e z f(z) = O ( e (1 ɛ) z ). a n j 0 τ j (n) j! f (j) (n). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

46 Closure Properties of J S -admissibility Proposition Let α (0, 1). (i) z m, e αz J S. (ii) If f J S, then f(αz), z m f J S. (iii) If f, g J S, then f + g J S. (iv) If f, g, then f(αz) g((1 α)z) J S. (v) If f J S, then f J S. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

47 Closure Properties of J S -admissibility Proposition Let α (0, 1). (i) z m, e αz J S. (ii) If f J S, then f(αz), z m f J S. (iii) If f, g J S, then f + g J S. (iv) If f, g, then f(αz) g((1 α)z) J S. (v) If f J S, then f J S. Proposition Let b j=0 ( b j) f(z) (j) = 2 f(z/2) + g(z). Then, g(z) J S f(z) J S. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

48 Two Stage Approach for Tries (and PATRICIA Tries) Uses first poissonization: f(z) = e z n 0 a n z n n!. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

49 Two Stage Approach for Tries (and PATRICIA Tries) Uses first poissonization: f(z) = e z n 0 a n z n n!. f(z) satisfies: f(z) = 2 f(z/2) + g(z) which is solved with the Mellin transform: F (s) = 0 f(z)z s 1 dz = G(s) s. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

50 Two Stage Approach for Tries (and PATRICIA Tries) Uses first poissonization: f(z) = e z n 0 a n z n n!. f(z) satisfies: f(z) = 2 f(z/2) + g(z) which is solved with the Mellin transform: F (s) = 0 f(z)z s 1 dz = G(s) s. Asymptotics of f(z) is obtained via inverse Mellin transform (Flajolet & Gourdon & Dumas, 1995). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

51 Two Stage Approach for Tries (and PATRICIA Tries) Uses first poissonization: f(z) = e z n 0 a n z n n!. f(z) satisfies: f(z) = 2 f(z/2) + g(z) which is solved with the Mellin transform: F (s) = 0 f(z)z s 1 dz = G(s) s. Asymptotics of f(z) is obtained via inverse Mellin transform (Flajolet & Gourdon & Dumas, 1995). Asymptotics of a n is obtained via (analytic) depoissonization (Jacquet & Szpankowski, 1998). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

52 Asymptotic Poissonized Variances Let f 1 (z) = e z n 0 E(X n ) zn n!, f2 (z) = e z E(Xn) 2 zn n!. n 0 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

53 Asymptotic Poissonized Variances Let f 1 (z) = e z n 0 E(X n ) zn n!, f2 (z) = e z E(Xn) 2 zn n!. n 0 Variance of Poisson model: Ṽ 1 (n) := Var(X Poi(n) ) = f 2 (n) f 1 (n) 2. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

54 Asymptotic Poissonized Variances Let f 1 (z) = e z n 0 E(X n ) zn n!, f2 (z) = e z E(Xn) 2 zn n!. n 0 Variance of Poisson model: Ṽ 1 (n) := Var(X Poi(n) ) = f 2 (n) f 1 (n) 2. However, for many shape parameters: Var(X n ) Ṽ1(n), i.e., Poisson model unsuitable for obtaining asymptotics of the variance! Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

55 Asymptotic Poissonized Variances Let f 1 (z) = e z n 0 E(X n ) zn n!, f2 (z) = e z E(Xn) 2 zn n!. n 0 Variance of Poisson model: Ṽ 1 (n) := Var(X Poi(n) ) = f 2 (n) f 1 (n) 2. However, for many shape parameters: Var(X n ) Ṽ1(n), i.e., Poisson model unsuitable for obtaining asymptotics of the variance! F., Hwang, Zacharovas (2010): Ṽ 2 (z) := f 2 (z) f 1 (z) 2 z f 1(z) 2. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

56 Ṽ 1 (z) vs. Ṽ 2 (z) Ṽ 1 (z) Ṽ 2 (z) depth; leader/loser selection; Paper-Stone-Scissors and variants; approximate counting; corner parking; partial match queries in k-d digital trees; node profile of asymmetric digital trees. size; path length; occurrence of pattern; node profile of symmetric digital trees; Wiener and Steiner index. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

57 Variance of TPL in Symmetric DSTs (ii) Theorem (F., Hwang, Zacharovas) We, have Var(L n ) n(c var + P (log 2 n)), where P (z) is a 1-periodic function with zero average value and C var = Q(1) L j,h,l 0 ( 1) j 2 (j+1 2 ) Q j Q h Q l 2 h+l ϕ(2 j h + 2 j l ), where ϕ(x) := { (x log x 1)/(x 1) 2, if x 1; 1/2, if x = 1. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

58 C var = 28 3L l 1 l2 l (2 l 1) L l 1 2 ( 1) l+1 (l 5) L (l + 1)l(l 1)(2 l 1) l 3 2Q(1) L + l l Q l 1 2 l 1 + π2 2L L ( ) ( 1) l 2 l+1 2 L(1 2 l+1 )/2 1 L 1 2 l ( 1) r+1 r(r 1)(2 r+l 1) l 1 r 2 ( ) ( 1) r 2 r+1 2 ( j j+r+l+2 1 Q r+l 2 Q r 0 r ( l 1 ( l + 1 ) 2 l+1 2l i i=2 2 (1 2 l r ) 2 + 2l + 2 (1 2 1 l r ) 2 2 L l+1 ( l + 1 ) 2 j j=2 1 2 r+j L j=0 i r+i l r + 2 l+1 ( l + 1 ) L j j=1 l+1 ( l + 1 ) ( 1) i ) j (i + 1)(2 r+j+i 1) + l 1 ( l + 1 ) Qr 2 Q l r 1 1 r 2 l 3 r=2 l Q l 2 j 1 2[ωψ] 0 [ω 2 ] 0. j l+1 ) 1 2 r+j 1 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

59 General Framework for Symmmetric Tries Consider X n d = XIn + X n I n + T n and let g 1 (z) = e z n 0 and W 2 (z) := g 2 (z) g 1 (z) 2 z g 1 (z). E(T n ) zn n!, g 2(z) = e z E(Tn) 2 zn n! n 0 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

60 General Framework for Symmmetric Tries Consider X n d = XIn + X n I n + T n and let g 1 (z) = e z n 0 and W 2 (z) := g 2 (z) g 1 (z) 2 z g 1 (z). E(T n ) zn n!, g 2(z) = e z E(Tn) 2 zn n! n 0 Theorem (F., Hwang, Zacharovas) Let g 1, W 2 J S α,β with α < 1 and g 2 J S. Then, E(X n ) np 1 (log 2 n), Var(X n ) np 2 (log 2 n), where P 1, P 2 are 1-periodic, computable functions. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

61 Variance of Size in Symmetric Tries (ii) Theorem (F., Hwang, Zacharovas) We have, where Var(S n ) n log 2 G( 1 + χ k )n χ k, k G( 1 + χ k ) = χ kγ( 1 + χ k )(1 + χ k ) ( 1) j j(j(j + χ k ) 1)Γ(j + χ k ) (j + 1)!(2 j 1) j 1 with χ k = 2kπi/(log 2). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

62 Exact Poissonized Variance Question: why not using the PGF of the variance, i.e., Ṽ (z) := e z n 0 Var(X n ) zn n!? Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

63 Exact Poissonized Variance Question: why not using the PGF of the variance, i.e., Ṽ (z) := e z n 0 Var(X n ) zn n!? Recall that Then, σ 2 n = 2 1 n where σ 2 n = Var(X n ). X n d = XIn + X n I n + T n. n ( ) n σj 2 + Var(T n ) j j=0 n ( ) n + 2 n (E(X j ) + E(X n j ) E(X n ) + E(T n )) 2, j j=0 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

64 Poissonization of the Variance Recurrence Set W (z) := e z n 0 Var(T n ) zn n!. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

65 Poissonization of the Variance Recurrence Set Proposition We have, W (z) := e z n 0 Var(T n ) zn n!. Ṽ (z) = 2Ṽ (z/2) + W (z) + h(z), where h(z) =2( f 1 f 1 )(z/2) + 2 f 1 (z/2) 2 ( f 1 f 1 )(z) + 2( f 1 g 1 )(z) ( g 1 g 1 )(z). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

66 Poissonization of the Variance Recurrence Set Proposition We have, W (z) := e z n 0 Var(T n ) zn n!. Ṽ (z) = 2Ṽ (z/2) + W (z) + h(z), where h(z) =2( f 1 f 1 )(z/2) + 2 f 1 (z/2) 2 ( f 1 f 1 )(z) + 2( f 1 g 1 )(z) ( g 1 g 1 )(z). Here, f g denotes the Hadamard product of f and g. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

67 Closure Property of Hadamard Product Proposition (F., Hwang, Zacharovas) If f(z) = e z n 0 are both J S -admissible, then is also J S -admissible. z n a n n!, g(z) = z n e z b n n! n 0 ( f g)(z) = e z n 0 a n b n z n n! Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

68 Closure Property of Hadamard Product Proposition (F., Hwang, Zacharovas) If f(z) = e z n 0 are both J S -admissible, then is also J S -admissible. z n a n n!, g(z) = z n e z b n n! n 0 ( f g)(z) = e z n 0 Proof uses the integral representation ( f g)(z 2 ) = e z2 ue u2 4πi 0 ξ =1 a n b n z n n! e zuξ+zu ξ f(zuξ) g(zu ξ) dξ ξ du. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

69 An IFF Condition for J S -admissibility Theorem (Jacquet) f J S iff there exists an analytic extension α(z) of a n with α(z) = O(z p ), for some p > 0, as z, in a cone {z : arg(z) ɛ}. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

70 An IFF Condition for J S -admissibility Theorem (Jacquet) f J S iff there exists an analytic extension α(z) of a n with α(z) = O(z p ), for some p > 0, as z, in a cone {z : arg(z) ɛ}. Was proved in two stages: Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

71 An IFF Condition for J S -admissibility Theorem (Jacquet) f J S iff there exists an analytic extension α(z) of a n with α(z) = O(z p ), for some p > 0, as z, in a cone {z : arg(z) ɛ}. Was proved in two stages: If : Jacquet & Szpankowski (1999); Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

72 An IFF Condition for J S -admissibility Theorem (Jacquet) f J S iff there exists an analytic extension α(z) of a n with α(z) = O(z p ), for some p > 0, as z, in a cone {z : arg(z) ɛ}. Was proved in two stages: If : Jacquet & Szpankowski (1999); Only If : Jacquet (2014). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

73 An IFF Condition for J S -admissibility Theorem (Jacquet) f J S iff there exists an analytic extension α(z) of a n with α(z) = O(z p ), for some p > 0, as z, in a cone {z : arg(z) ɛ}. Was proved in two stages: If : Jacquet & Szpankowski (1999); Only If : Jacquet (2014). Corollary If f, g are J S -admissible, then f g is J S -admissible. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

74 Asymptotic Expansion for Hadamard Product Theorem If f, g J S, then ( f g)(z) n 0 z k k! f (k) (z) g (k) (z). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

75 Asymptotic Expansion for Hadamard Product Theorem If f, g J S, then ( f g)(z) n 0 z k k! f (k) (z) g (k) (z). We have, where with Ṽ (z) = 2Ṽ (z/2) + W (z) + h(z), h(z) k 2 c ( k z ) k (k) f 1 (z/2) 2 k! 2 c k = 2(1 2 1 k ). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

76 General Framework for Symmetric Tries - Revisited d Recall: X n = XIn + Xn I n + T n. Theorem (F., Hwang, Zacharovas) Let g 1, W 2 J S α,β with α < 1 and g 2 J S. Then, E(X n ) np 1 (log 2 n), Var(X n ) np 2 (log 2 n), where P 1, P 2 are 1-periodic, computable functions. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

77 General Framework for Symmetric Tries - Revisited d Recall: X n = XIn + Xn I n + T n. Theorem (F., Hwang, Zacharovas) Let g 1, W 2 J S α,β with α < 1 and g 2 J S. Then, E(X n ) np 1 (log 2 n), Var(X n ) np 2 (log 2 n), where P 1, P 2 are 1-periodic, computable functions. Theorem Let g 1, W J S α,β with α < 1. Then, E(X n ) np 1 (log 2 n), Var(X n ) np 2 (log 2 n), where P 1, P 2 are 1-periodic, computable functions. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

78 Variance of Size in Symmetric Tries (iii) Theorem We have, where Var(S n ) 1 log 2 P 1(n) 1 1 log 2 2 k! P k(n), k 2 P k (n) = l d k,l n! Γ(n + k 1 + χ l ) with d 1,l as before and for k 2 d k,l = j χ j Γ(k 1 + χ j )χ l j Γ(k 1 + χ l j ). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

79 Variance of Size in Symmetric Tries (iii) Theorem We have, where Var(S n ) 1 log 2 P 1(n) 1 1 log 2 2 k! P k(n), k 2 P k (n) = l d k,l n! Γ(n + k 1 + χ l ) with d 1,l as before and for k 2 d k,l = j χ j Γ(k 1 + χ j )χ l j Γ(k 1 + χ l j ). Remark: Similar results for other (additive) shape parameters of symmetric tries and PATRICIA tries. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

80 Identity for Hadamard Product Theorem Under assumptions which include J S -admissibility, we have ( f g)(z) = k 0 z k k! f (k) (z) g (k) (z). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

81 Identity for Hadamard Product Theorem Under assumptions which include J S -admissibility, we have ( f g)(z) = k 0 z k k! f (k) (z) g (k) (z). Equivalently, we have: ( )( τ j (n) f (j) (n) j! j 0 = j 0 j 0 ) τ j (n) g (j) (n) j! ( ) (j) τ j (n) n k j! k! f (k) (n) g(k)(n). k 0 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

82 A Quick Proof (i) Let f(z) = A m m! zm, g(z) = B m m! zm. m 0 m 0 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

83 A Quick Proof (i) Let Then, f(z) = A m m! zm, g(z) = B m m! zm. m 0 m 0 ( j 0 τ j (n) j! = j 0 f (j) (n) )( j 0 ) τ j (n) g (j) (n) j! ( τ j (n) n k j! k! f (k) (n) g(k)(n) k 0 ) (j) can be rewritten to ( ( ) )( n A m m 0 m n 0 m n ( ) n )B m = n![z n ]e z z k m k! f (k) (z) g (k) (z). k 0 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

84 A Quick Proof (ii) Note that n![z n ]e z k! f (k) (z) g (k) (z) = n! 1 k! [zn ]z k e z f (k) (z) g (k) (z) k 0 k 0 z k Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

85 A Quick Proof (ii) Note that z k n![z n ]e z k! f (k) (z) g (k) (z) = n! k 0 k 0 1 = n! k! 0 k n i 1 +i 2 +i 3 =n k 1 k! [zn ]z k e z f (k) (z) g (k) (z) A i2 +kb i3 +k i 1!i 2!i 3! Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

86 A Quick Proof (ii) Note that z k n![z n ]e z k! f (k) (z) g (k) (z) = n! k 0 k 0 1 = n! k! = 0 k n ( 0 m n i 1 +i 2 +i 3 =n k ( ) n m A m )( 1 k! [zn ]z k e z f (k) (z) g (k) (z) A i2 +kb i3 +k i 1!i 2!i 3! ( ) n )B m m 0 m n Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

87 A Quick Proof (ii) Note that z k n![z n ]e z k! f (k) (z) g (k) (z) = n! k 0 k 0 1 = n! k! = 0 k n ( 0 m n and this is true since ( )( ) n n = s t 0 k n i 1 +i 2 +i 3 =n k ( ) n m A m )( 1 k! [zn ]z k e z f (k) (z) g (k) (z) A i2 +kb i3 +k i 1!i 2!i 3! ( ) n )B m m 0 m n ( )( n n k ) k s k, t k, n + k s t which counts twice the number of ways of choosing a set of size s and t from a set of size n. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

88 Exact vs. Asymptotic Poissonized Variance Recall Ṽ (z) = e z n 0 Var(X n ) zn n!. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

89 Exact vs. Asymptotic Poissonized Variance Recall Ṽ (z) = e z n 0 Var(X n ) zn n!. Corollary We have, Ṽ (z) = f 2 (z) k 0 z k (k) f 1 (z) 2. k! Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

90 Exact vs. Asymptotic Poissonized Variance Recall Corollary We have, Ṽ (z) = e z n 0 Ṽ (z) = f 2 (z) k 0 Var(X n ) zn n!. z k (k) f 1 (z) 2. k! This gives: Ṽ (z) = f 2 (z) f 1 (z) 2 z }{{} f 1(z) 2 =Ṽ1(z) } {{ } =Ṽ2(z) z2 2 f 1 (z) 2 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

91 Summary Poisson-Mellin- Newton cycle: Poisson GF a n Noerlund integral f(z) Mellin transform F (s) Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

92 Summary Poisson-Mellin- Newton cycle: Poisson GF a n Noerlund integral f(z) Mellin transform F (s) We used exact poissonized variance in contrast to asymptotic poissonized variances. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

93 Summary Poisson-Mellin- Newton cycle: Poisson GF a n Noerlund integral f(z) Mellin transform F (s) We used exact poissonized variance in contrast to asymptotic poissonized variances. This yields general frameworks for asymptotics of mean and variance of additive shape parameter in tries and PATRICIA tries under natural conditions. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

94 Summary Poisson-Mellin- Newton cycle: Poisson GF a n Noerlund integral f(z) Mellin transform F (s) We used exact poissonized variance in contrast to asymptotic poissonized variances. This yields general frameworks for asymptotics of mean and variance of additive shape parameter in tries and PATRICIA tries under natural conditions. This also yields full asymptotic expansions of the variance for symmetric tries and PATRICIA tries. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

95 Inverse Mellin and De-poissonization Combined Formally, we have a n = n! e z f(z) 2πi z =r z n+1 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

96 Inverse Mellin and De-poissonization Combined Formally, we have a n = n! 2πi = n! 2πi z =r z =r e z f(z) z n+1 e z z n+1 ( 1 2πi (c) F (s)z s ds ) dz Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

97 Inverse Mellin and De-poissonization Combined Formally, we have a n = n! 2πi = n! 2πi = n! 2πi z =r e z f(z) z n+1 e z z =r z n+1 (c) ( 1 2πi ( 1 F (s) 2πi (c) z =r F (s)z s ds ) e z z n s 1 dz dz ) ds Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

98 Inverse Mellin and De-poissonization Combined Formally, we have a n = n! 2πi = n! 2πi = n! 2πi = 1 2πi z =r e z f(z) z n+1 e z z =r z n+1 (c) (c) ( 1 2πi ( 1 F (s) 2πi F (s) Γ(s) (c) z =r n!γ(s) Γ(n s) ds F (s)z s ds ) e z z n s 1 dz dz ) ds Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

99 Inverse Mellin and De-poissonization Combined Formally, we have a n = n! 2πi = n! 2πi = n! 2πi = 1 2πi z =r e z f(z) z n+1 e z z =r z n+1 (c) (c) ( 1 2πi ( 1 F (s) 2πi F (s) Γ(s) (c) z =r n!γ(s) Γ(n s) ds which after s s is the Noerlund integral. F (s)z s ds ) e z z n s 1 dz dz ) ds Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

100 Inverse Mellin and De-poissonization Combined Formally, we have a n = n! 2πi = n! 2πi = n! 2πi = 1 2πi z =r e z f(z) z n+1 e z z =r z n+1 (c) (c) ( 1 2πi ( 1 F (s) 2πi F (s) Γ(s) (c) z =r n!γ(s) Γ(n s) ds which after s s is the Noerlund integral. F (s)z s ds ) e z z n s 1 dz Remark: justification needs e.g. polynomial growth of F ( s)/γ( s)! dz ) ds Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

101 Variance of Size in Symmetric Tries (iv) Recall Ṽ (z) = 2Ṽ (z/2) + h(z) where h(z) = k 2 c ( k z ) k (k) f 1 (z/2) 2. k! 2 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

102 Variance of Size in Symmetric Tries (iv) Recall Ṽ (z) = 2Ṽ (z/2) + h(z) where For the size: h(z) = j,l 1 h(z) = k 2 2 j+l 2 e z 2 j z 2 l k 2 c ( k z ) k (k) f 1 (z/2) 2. k! 2 2 (j+l 1)k c ( k z k z ) ( z ) k! 2 j k l k + 1 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

103 Variance of Size in Symmetric Tries (iv) Recall Ṽ (z) = 2Ṽ (z/2) + h(z) where For the size: h(z) = j,l 1 h(z) = k 2 2 j+l 2 e z 2 j z 2 l k 2 c ( k z ) k (k) f 1 (z/2) 2. k! 2 2 (j+l 1)k c ( k z k z ) ( z ) k! 2 j k l k + 1 From this one obtains an explicit expression for the Mellin transform! Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

104 Variance of Size in Symmetric Tries (iv) Recall Ṽ (z) = 2Ṽ (z/2) + h(z) where For the size: h(z) = j,l 1 h(z) = k 2 2 j+l 2 e z 2 j z 2 l k 2 c ( k z ) k (k) f 1 (z/2) 2. k! 2 2 (j+l 1)k c ( k z k z ) ( z ) k! 2 j k l k + 1 From this one obtains an explicit expression for the Mellin transform! Lemma M [ h(z); s]/γ( s) is of polynomial growth on R(s) > 0. Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

105 Variance of Size in Symmetric Tries (v) Start with: 1 2πi (3/2) M [ h(z); s] n!γ( s) (1 2 1 s )Γ( s) Γ(n + 1 s) Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

106 Variance of Size in Symmetric Tries (v) Start with: 1 2πi (3/2) M [ h(z); s] n!γ( s) (1 2 1 s )Γ( s) Γ(n + 1 s) Moving the line of integration to: Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

107 Variance of Size in Symmetric Tries (v) Start with: 1 2πi (3/2) M [ h(z); s] n!γ( s) (1 2 1 s )Γ( s) Γ(n + 1 s) Moving the line of integration to: + : gives Var(S n ); Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

108 Variance of Size in Symmetric Tries (v) Start with: 1 2πi (3/2) M [ h(z); s] n!γ( s) (1 2 1 s )Γ( s) Γ(n + 1 s) Moving the line of integration to: + : gives Var(S n ); inside (0, 1): gives main term in the asymptotics of Var(S n ). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

109 Variance of Size in Symmetric Tries (v) Start with: 1 2πi (3/2) M [ h(z); s] n!γ( s) (1 2 1 s )Γ( s) Γ(n + 1 s) Moving the line of integration to: + : gives Var(S n ); inside (0, 1): gives main term in the asymptotics of Var(S n ). Rice-Noerlund method for variance without cancellations! Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

110 Variance of Size in Symmetric Tries (v) Start with: 1 2πi (3/2) M [ h(z); s] n!γ( s) (1 2 1 s )Γ( s) Γ(n + 1 s) Moving the line of integration to: + : gives Var(S n ); inside (0, 1): gives main term in the asymptotics of Var(S n ). Rice-Noerlund method for variance without cancellations! Longer expansions by moving the line beyond the imaginary axis but for this we need polynomial growth beyond R(s) > 0 Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

111 Variance of Size in Symmetric Tries (v) Start with: 1 2πi (3/2) M [ h(z); s] n!γ( s) (1 2 1 s )Γ( s) Γ(n + 1 s) Moving the line of integration to: + : gives Var(S n ); inside (0, 1): gives main term in the asymptotics of Var(S n ). Rice-Noerlund method for variance without cancellations! Longer expansions by moving the line beyond the imaginary axis but for this we need polynomial growth beyond R(s) > 0 tameness of M [ h(z); s]/γ( s)? (Vallée; 2017) Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

112 Open Problems How to show e.g. Var(S n ) 1 log 2 P 1(n) 1 1 log 2 2 k! P k(n) k 2 via the Rice-Noerlund method? Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

113 Open Problems How to show e.g. Var(S n ) 1 log 2 P 1(n) 1 1 log 2 2 k! P k(n) k 2 via the Rice-Noerlund method? Does the above hold as identity? Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

114 Open Problems How to show e.g. Var(S n ) 1 log 2 P 1(n) 1 1 log 2 2 k! P k(n) k 2 via the Rice-Noerlund method? Does the above hold as identity? For this one would need polynomial growth of M [ h(z); s]/γ( s) on C (when staying uniformly away from the singularities). Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

115 Open Problems How to show e.g. Var(S n ) 1 log 2 P 1(n) 1 1 log 2 2 k! P k(n) k 2 via the Rice-Noerlund method? Does the above hold as identity? For this one would need polynomial growth of M [ h(z); s]/γ( s) on C (when staying uniformly away from the singularities). Conditions on T n X n d = XIn + X n I n + T n such that Var(X n ) can be obtained with Rice-Noerlund method? Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40

116 Open Problems How to show e.g. Var(S n ) 1 log 2 P 1(n) 1 1 log 2 2 k! P k(n) k 2 via the Rice-Noerlund method? Does the above hold as identity? For this one would need polynomial growth of M [ h(z); s]/γ( s) on C (when staying uniformly away from the singularities). Conditions on T n X n d = XIn + X n I n + T n such that Var(X n ) can be obtained with Rice-Noerlund method? B. Vallée has a general framework for mean but how about variance? Michael Fuchs (NCTU) Poissonized Variance June 22nd, / 40