Combinatorial/probabilistic analysis of a class of search-tree functionals

Transcription

1 Combinatorial/probabilistic analysis of a class of search-tree functionals Jim Fill jimfill@jhu.edu fill/ Mathematical Sciences The Johns Hopkins University Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.1

2 This is joint work (in progress) with Philippe Flajolet, INRIA Nevin Kapur, JHU Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.2

3 Goal For additive functionals on random m-ary search trees: derive asymptotic approximations to distributions Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.3

4 Goal For additive functionals on random m-ary search trees: derive asymptotic approximations to distributions obtain rates of convergence (still to do) Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.3

5 Goal For additive functionals on random m-ary search trees: derive asymptotic approximations to distributions obtain rates of convergence (still to do) study relation between input toll sequence and functional Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.3

6 Goal For additive functionals on random m-ary search trees: Tools: derive asymptotic approximations to distributions obtain rates of convergence (still to do) study relation between input toll sequence and functional get complete asymptotic expansions of moments Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.3

7 Goal For additive functionals on random m-ary search trees: Tools: derive asymptotic approximations to distributions obtain rates of convergence (still to do) study relation between input toll sequence and functional get complete asymptotic expansions of moments combinatorial/probabilistic transfer results linking asymptotics of toll sequence to that of functional Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.3

8 Goal For additive functionals on random m-ary search trees: Tools: derive asymptotic approximations to distributions obtain rates of convergence (still to do) study relation between input toll sequence and functional get complete asymptotic expansions of moments combinatorial/probabilistic transfer results linking asymptotics of toll sequence to that of functional singularity analysis (based on complex analysis) Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.3

9 m-ary search trees fundamental data structure in computer science generalizes the concept of a binary search tree Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.4

10 m-ary search trees Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.5

11 m-ary search trees each sequence of n distinct keys can be associated with an m-ary search tree got by inserting successive elements of the sequence into an initially empty tree Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.5

12 Probability models uniform model each tree equally likely (future work) Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.6

13 Probability models uniform model each tree equally likely (future work) random permutation model each permutation on [n] := {1,..., n} equally likely (this talk) Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.6

14 Probability models uniform model each tree equally likely (future work) random permutation model each permutation on [n] := {1,..., n} equally likely (this talk) Q: probability mass function of the distribution induced by associating an m-ary search tree with each of the n! permutations on [n] Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.6

15 Probability models uniform model each tree equally likely (future work) random permutation model each permutation on [n] := {1,..., n} equally likely (this talk) Q: probability mass function of the distribution induced by associating an m-ary search tree with each of the n! permutations on [n] Q is a crude measure of the shape of a tree balanced tree large Q Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.6

16 Probability models uniform model each tree equally likely (future work) random permutation model each permutation on [n] := {1,..., n} equally likely (this talk) Q: probability mass function of the distribution induced by associating an m-ary search tree with each of the n! permutations on [n] Q is a crude measure of the shape of a tree balanced tree large Q (Dobrow & Fill, 1995) Q(T ) = 1/ ( T (x) ) x full m 1 Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.6

17 Probability models uniform model each tree equally likely (future work) random permutation model each permutation on [n] := {1,..., n} equally likely (this talk) Q: probability mass function of the distribution induced by associating an m-ary search tree with each of the n! permutations on [n] Q is a crude measure of the shape of a tree balanced tree large Q (Dobrow & Fill, 1995) Q(T ) = 1/ ( T (x) ) x full m 1 shape functional: Λ(T ) := ln Q(T ) Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.6

18 Notation x T2 T3 T4 T 1 = T(x) T = 15 full nodes contain m 1 keys Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.7

19 Additive-type tree functionals f(t ) = m f(t i ) + c T, T m 1 i=1 (c n ) n m 1 : toll sequence think of toll sequence as input and additive functional as output Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.8

20 Additive-type tree functionals f(t ) = m f(t i ) + c T, T m 1 i=1 (c n ) n m 1 : toll sequence think of toll sequence as input and additive functional as output Examples: 1. c n := 1: space requirement (Mahmoud & Pittel, 1989; Chern & Hwang, 2001) 2. c n := n (m 1): internal path length (Mahmoud, 1992) [analysis of m-ary Quicksort!] 3. c n := ln ( n m 1 ) : shape functional Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.8

21 1. limit laws center the functional using lead order term of the mean Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.9

22 1. limit laws center the functional using lead order term of the mean observe that all moments of the centered functional satisfy a recurrence relation of the same form Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.9

23 1. limit laws center the functional using lead order term of the mean observe that all moments of the centered functional satisfy a recurrence relation of the same form use a general (combinatorial) transfer theorem to get asymptotics for all moments of the centered functional Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.9

24 1. limit laws center the functional using lead order term of the mean observe that all moments of the centered functional satisfy a recurrence relation of the same form use a general (combinatorial) transfer theorem to get asymptotics for all moments of the centered functional conclude convergence in distribution using method of moments (when possible) Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.9

25 1. limit laws center the functional using lead order term of the mean observe that all moments of the centered functional satisfy a recurrence relation of the same form use a general (combinatorial) transfer theorem to get asymptotics for all moments of the centered functional conclude convergence in distribution using method of moments (when possible) alternative method: contraction method (not today) Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.9

26 Details: distn. of subtree size recall f(t ) = m f(t i ) + c T i=1 Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.10

27 Details: distn. of subtree size recall f(t ) = m f(t i ) + c T i=1 for T = n, and all i, j, ( ) /( ) n j 1 n P ( T = j) = m 2 m 1 Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.10

28 Details: distn. of subtree size recall f(t ) = m f(t i ) + c T i=1 for T = n, and all i, j, ( ) /( ) n j 1 n P ( T = j) = m 2 m 1 so, for n m 1, µ n = m ( n m 1) j µ n := Ef(T ) satisfies ( n j 1 m 2 ) µ j + c n Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.10

29 Details: recurrence rels. & diff. eqs. the sequence (a n ) of moments of each specified order satisfy a recurrence relation of the form a n = ( m n m 1) ( ) n j 1 a j + b n m 2 j Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.11

30 Details: recurrence rels. & diff. eqs. the sequence (a n ) of moments of each specified order satisfy a recurrence relation of the form a n = ( m n m 1) ( ) n j 1 a j + b n m 2 j equivalent to linear differential equation with characteristic polynomial ψ(λ) := λ(λ + 1) (λ + m 2) m! with roots 2 = λ 1,..., λ m 1 Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.11

31 Details: explicit solution! for simplicity, consider case b 1 = = b m 2 = 0 Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.12

32 Details: explicit solution! for simplicity, consider case b 1 = = b m 2 = 0 solution easiest to state using generating functions: A(z) = B(z) + m! m 1 j=1 (1 z) λ j ψ (λ j ) z ζ=0 B(ζ)(1 ζ) λ j 1 dζ Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.12

33 Details: explicit solution! for simplicity, consider case b 1 = = b m 2 = 0 solution easiest to state using generating functions: A(z) = B(z) + m! m 1 j=1 (1 z) λ j ψ (λ j ) z ζ=0 B(ζ)(1 ζ) λ j 1 dζ for general initial conditions, get extra term of form m 1 j=1 r j (1 z) λ j Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.12

34 Transfer theorem examples transfer for small input: b n = o(n) and n=0 b j (n + 1)(n + 2) converges transfers to a n = K 1 H m 1 n + o(n), where K 1 := j=0 b j (j + 1)(j + 2). Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.13

35 Transfer theorem examples transfer for small input: b n = o(n) and n=0 b j (n + 1)(n + 2) converges transfers to a n = K 1 H m 1 n + o(n), where K 1 := j=0 b j (j + 1)(j + 2). one refinement: if m 26 and b n = o( n), then a n = K 1 H m 1 n + o( n) Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.13

36 General transfer theorem links the asymptotic behavior of the remainder sequence (b n ) (input) to (a n ) (output) Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.14

37 General transfer theorem links the asymptotic behavior of the remainder sequence (b n ) (input) to (a n ) (output) Input (b n ) Output (a n ) b n = o(n) etc. a n K 1 H m 1 n b n Kn a n K H m 1 n log n b n Kn v, R(v) > 1 a n K 1 m!γ(v+1) Γ(v+m) n v Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.14

38 General transfer theorem links the asymptotic behavior of the remainder sequence (b n ) (input) to (a n ) (output) Input (b n ) Output (a n ) b n = o(n) etc. a n K 1 H m 1 n b n Kn a n K H m 1 n log n b n Kn v, R(v) > 1 a n K 1 m!γ(v+1) Γ(v+m) refined transfers available; require additional conditions n v Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.14

39 Distributional asymptotics & phase changes small toll sequence: c n = O(n 1/2 L(n)): Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.15

40 Distributional asymptotics & phase changes small toll sequence: c n = O(n 1/2 L(n)): asymptotic normality if m 26 Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.15

41 Distributional asymptotics & phase changes small toll sequence: c n = O(n 1/2 L(n)): asymptotic normality if m 26 periodicity (typically) if m 27 Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.15

42 Distributional asymptotics & phase changes small toll sequence: c n = O(n 1/2 L(n)): asymptotic normality if m 26 periodicity (typically) if m 27 moderate toll sequence: c n n γ L(n), 1 2 < γ < 1: Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.15

43 Distributional asymptotics & phase changes small toll sequence: c n = O(n 1/2 L(n)): asymptotic normality if m 26 periodicity (typically) if m 27 moderate toll sequence: c n n γ L(n), 1 2 < γ < 1: convergence to non-normal distributions if m m 0 (where m 0 26) Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.15

44 Distributional asymptotics & phase changes small toll sequence: c n = O(n 1/2 L(n)): asymptotic normality if m 26 periodicity (typically) if m 27 moderate toll sequence: c n n γ L(n), 1 2 < γ < 1: convergence to non-normal distributions if m m 0 (where m 0 26) periodicity (typically) if m m Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.15

45 Distributional asymptotics & phase changes small toll sequence: c n = O(n 1/2 L(n)): asymptotic normality if m 26 periodicity (typically) if m 27 moderate toll sequence: c n n γ L(n), 1 2 < γ < 1: convergence to non-normal distributions if m m 0 (where m 0 26) periodicity (typically) if m m large toll sequence: c n n γ L(n) with γ 1: Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.15

46 Distributional asymptotics & phase changes small toll sequence: c n = O(n 1/2 L(n)): asymptotic normality if m 26 periodicity (typically) if m 27 moderate toll sequence: c n n γ L(n), 1 2 < γ < 1: convergence to non-normal distributions if m m 0 (where m 0 26) periodicity (typically) if m m large toll sequence: c n n γ L(n) with γ 1: convergence to non-normal distributions for all values of m [γ = 1: Quicksort # of comparisons] Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.15

47 2. Moments: full asymptotic expansions recall: moments (a n ) of each specified order satisfy a n = ( m n m 1) ( ) n j 1 a j + b n m 2 j with (b n ) defined in terms of moments of lower orders, and (for vanishing initial conditions) A(z) = B(z) + m! m 1 j=1 (1 z) λ j ψ (λ j ) z ζ=0 B(ζ)(1 ζ) λ j 1 dζ Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.16

48 Moment expansions remember A(z) = B(z) + m! m 1 j=1 (1 z) λ j ψ (λ j ) z ζ=0 B(ζ)(1 ζ) λ j 1 dζ Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.17

49 Moment expansions remember A(z) = B(z) + m! m 1 j=1 (1 z) λ j ψ (λ j ) z ζ=0 B(ζ)(1 ζ) λ j 1 dζ full asymptotic expansion for g.f. of toll sequence (c n ) transfers to one for g.f. of sequence of means Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.17

50 Moment expansions remember A(z) = B(z) + m! m 1 j=1 (1 z) λ j ψ (λ j ) z ζ=0 B(ζ)(1 ζ) λ j 1 dζ full asymptotic expansion for g.f. of toll sequence (c n ) transfers to one for g.f. of sequence of means inductively, get full asymptotic expansion for g.f. of sequence of moments of each order Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.17

51 Moment expansions remember A(z) = B(z) + m! m 1 j=1 (1 z) λ j ψ (λ j ) z ζ=0 B(ζ)(1 ζ) λ j 1 dζ full asymptotic expansion for g.f. of toll sequence (c n ) transfers to one for g.f. of sequence of means inductively, get full asymptotic expansion for g.f. of sequence of moments of each order use singularity analysis [Flajolet & Odlyzko, 1990] to get full asymptotic expansions for moments Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.17

52 Prototype: the shape functional c n := ln ( n m 1), n m 1 Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.18

53 Prototype: the shape functional c n := ln ( n m 1), n m 1 vanishing initial conditions Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.18

54 Prototype: the shape functional c n := ln ( n m 1), n m 1 vanishing initial conditions using Tauberian methods, with C 1 := K 1 /(H m 1), Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.18

55 Prototype: the shape functional c n := ln ( n m 1), n m 1 vanishing initial conditions using Tauberian methods, with C 1 := K 1 /(H m 1), first moments: µ n C 1 n Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.18

56 Prototype: the shape functional c n := ln ( n m 1), n m 1 vanishing initial conditions using Tauberian methods, with C 1 := K 1 /(H m 1), first moments: µ n C 1 n second moments: ν n C 2 1n 2 Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.18

57 Prototype: the shape functional c n := ln ( n m 1), n m 1 vanishing initial conditions using Tauberian methods, with C 1 := K 1 /(H m 1), first moments: µ n C 1 n second moments: ν n C 2 1n 2 can t go beyond lead order, because side conditions can t be established Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.18

58 Prototype: the shape functional c n := ln ( n m 1), n m 1 vanishing initial conditions using Tauberian methods, with C 1 := K 1 /(H m 1), first moments: µ n C 1 n second moments: ν n C 2 1n 2 can t go beyond lead order, because side conditions can t be established in fact, can t get lead order for variance Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.18

59 Singularity Analysis alternative approach: singularity analysis [Flajolet & Odlyzko, 1990] Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.19

60 Singularity Analysis alternative approach: singularity analysis [Flajolet & Odlyzko, 1990] aids in determination of asymptotic order of growth of (Taylor) coefficients of functions analytic at the origin f(z) = O(g(z)) f n = O(g n ) f(z) = o(g(z)) f n = o(g n ) f(z) g(z) f n g n. Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.19

61 Singularity Analysis alternative approach: singularity analysis [Flajolet & Odlyzko, 1990] aids in determination of asymptotic order of growth of (Taylor) coefficients of functions analytic at the origin f(z) = O(g(z)) f n = O(g n ) f(z) = o(g(z)) f n = o(g n ) f(z) g(z) f n g n. based on Cauchy integral formula in indented crown, judicious choices of contours Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.19

62 Singularity Analysis alternative approach: singularity analysis [Flajolet & Odlyzko, 1990] aids in determination of asymptotic order of growth of (Taylor) coefficients of functions analytic at the origin f(z) = O(g(z)) f n = O(g n ) f(z) = o(g(z)) f n = o(g n ) f(z) g(z) f n g n. based on Cauchy integral formula in indented crown, judicious choices of contours no side conditions on sequence required Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.19

63 SA: what we know Flajolet and Odlyzko [1990] give sufficient conditions for validity of these implications when g(z) is of the form (1 z) β (log (1 z)) γ (log log (1 z)) δ Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.20

64 SA: what we know Flajolet and Odlyzko [1990] give sufficient conditions for validity of these implications when g(z) is of the form (1 z) β (log (1 z)) γ (log log (1 z)) δ Flajolet [1998] showed that singularity analysis is applicable to the generalized polylogarithm Li α,r (z) := (ln n) r zn n α n=1 Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.20

65 SA: new extensions (1) Recall A(z) = B(z) + m! m 1 j=1 (1 z) λ j ψ (λ j ) z ζ=0 B(ζ)(1 ζ) λ j 1 dζ where B(z) is the g.f. of a sequence that depends on the toll sequence. We need Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.21

66 SA: new extensions (1) Recall A(z) = B(z) + m! m 1 j=1 (1 z) λ j ψ (λ j ) z ζ=0 B(ζ)(1 ζ) λ j 1 dζ where B(z) is the g.f. of a sequence that depends on the toll sequence. We need (1) closure under indefinite integration Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.21

67 SA: new extensions (1) Recall A(z) = B(z) + m! m 1 j=1 (1 z) λ j ψ (λ j ) z ζ=0 B(ζ)(1 ζ) λ j 1 dζ where B(z) is the g.f. of a sequence that depends on the toll sequence. We need (1) closure under indefinite integration (2) closure under differentiation (used, e.g., for closure under Hadamard product) Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.21

68 SA: new extensions (1) Recall A(z) = B(z) + m! m 1 j=1 (1 z) λ j ψ (λ j ) z ζ=0 B(ζ)(1 ζ) λ j 1 dζ where B(z) is the g.f. of a sequence that depends on the toll sequence. We need (1) closure under indefinite integration (2) closure under differentiation (used, e.g., for closure under Hadamard product) for example, f(z) = O((1 z) β ) in indented crown implies f (z) = O((1 z) β 1 ) in indented crown Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.21

69 SA: new extensions (2) (3) closure under Hadamard product: (f g)(z) := n=0 f n g n z n Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.22

70 SA: new extensions (2) (3) closure under Hadamard product: (f g)(z) := n=0 f n g n z n these three closure results extend SA toolbox for asymptotic expansions Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.22

71 Application: shape functional (1) using the integral representation of the generating function of the mean and singularity analysis we can show (for m even) G 1 (z) m 1 j=1 + Ã j (1 z) λ j + B (1 z) 1 log (1 z) 1 + L 1 (1 z) 1 + r m 1 m 2 r=0 L r (1 z) r [K r (1 z) r log (1 z) 1 + L r (1 z) r ] Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.23

72 Application: shape functional (2) transfer back to get (for m even) µ n m 1 j=1 + Ã j λ j n n! r m 1 H n + L 1 K r r (( 1 n )) = (1 + o(1))c 1 n Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.24

73 Application: shape functional (2) transfer back to get (for m even) µ n m 1 j=1 + Ã j λ j n n! r m 1 H n + L 1 K r r (( 1 n )) = (1 + o(1))c 1 n similar result for m odd except for an extra (1 z) m [log (1 z) 1 ] 2 term in the generating function Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.24

74 Application: second moments alternative form of solution to recurrence relation: for vanishing initial conditions (e.g.), A(z) = m 1 j=1 (1 z) λ j ψ (λ j ) z ζ=0 B (m 1) (ζ)(1 ζ) λ j+m 2 dζ Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.25

75 Application: second moments alternative form of solution to recurrence relation: for vanishing initial conditions (e.g.), A(z) = m 1 j=1 (1 z) λ j ψ (λ j ) z ζ=0 B (m 1) (ζ)(1 ζ) λ j+m 2 dζ when A(z) is the second-moment g.f. G 2 (z), B (m 1) (z) = (m 1)m!(1 z) (m 2) G 2 1(z)+D (m 1) (z), where d n := c n (2µ n c n ), n m 1 Joint Statistical Meetings; Atlanta, GA; August 5, 2001 p.25