Analytic Depoissonization and Its Applications Λ

Transcription

1 Analytic Depoissonization and Its Applications Λ W. Szpankowski y Department of Computer Science Purdue University W. Lafayette, IN May 31, 2003 Λ This research is supported by NSF and NIH. y Joint work with P. Jacquet, INRIA, France.

2 Outline of the Talk 1. Definition of Poisson Transform and Its Properties 2. Motivating Examples Digital Trees ffl Leader Election Problem ffl Probabilistic Counting ffl 3. Algebraic Depoissonization 4. Depoissonization Theorems Simple Depoissonization Result and Its Proof ffl Generalized Depoissonization Results ffl Poissonized Variance vs Bernoulli Variance ffl Diagonal Depoissonization Theorems ffl Exponential Diagonal Depoissonization Theorems ffl 5. Applications DepthinaTrie ffl Size of a a Trie ffl Digital Search Trees ffl ffl Entropy of the Binomial Distribution

3 g n z n n(n 1) (n k) z k+1 1)z) exp((ff e z =(1 z) n! Poisson Transform Let fg n g n2n be a sequence of complex numbers such that jg n j = exp(o(n)): The Poisson transform of fg n g n2n is defined as eg(z) = X n! e z (1) n 0 G(z) e 1 1 g n n z n e 2z ( 1) n ff Basic Problem: How to recover g n from asymptotics of eg(z).

4 gk P ffn g n e G(ffz) gk f n k g n+1 e G(z)e z e G(z) e F (z)e z Some Properties of Poisson Transform Assumption: The Poisson transform G(z) is an entire function. Table 1: Some Properties of Poisson Transform sequence Poisson transform k n k P n e G(z) + e G(z) g n+k Pi k i e Example: Let a n = n p k q n k g k f n k k X with p + q = 1. Then k ea(z) = e F (pz) e G(qz):

5 g n k k! e : Probabilistic vs Analytic Depoissonization Let N be an integer variable with Poisson distribution of parameter : P (N = k) = k Observe that E[gN] = k! e := e G( ) This constitutes the basis for the probabilistic poissonization/depoissonization. n 0 X Thus Poisson transform e G(z) is an analytic continuation of e G( ) to complex plane. Remark: It seems to us that probabilistic poissonization was introduced by M. Kac (1949) (limited to real arguments).

6 Applications: Digital Trees Digital trees are data structures suitable to store data (keys) represented by a sequence of symbols from a finite alphabet. There are three types of digital trees, namely: trie, Patricia trie (PAT), and digital search tree (DST). These structures are recalled below. trie Patricia DST 1 0 S S 2 S 3 S 1 S S 1 S 2 S 3 S 4 S4 S 3 S 4 Figure 1: A trie, Patricia trie and a digital search tree (DST) built from the following four strings S 1 = :::, S 2 = :::, S 3 = :::, S 4 = :::.

7 F (z) = a(z) ρ +? F (zp)ρ +? Generic Example F(z) a(z) contribution of the root zp zq F(zp) F(zq) Figure 2: Generic Situation F (zq) + intitial conditions : ff ff

8 Tries and Their Parameters The parameters of interests are: typical depth D n, i.e., the length of a path from the root to a randomly ffl selected (external) node, height H n, i.e., maximum path from the root to a terminal node, ffl total path length L n, i.e., sum of all paths from the root to (external) ffl nodes, ffl size of the tree S n, i.e., number of nodes.

9 ed(z; u) = 1X D n (u) zn n! e z Recurrences and Functional Equations Average Depth: L n = ne[d n ]: L 0 = 0, L 1 = 1, and L n = n + n p k q n k (L k + L n k ) k X Poisson transform: k el(z) = z + e L(pz) + e L(qz) ze z (2) The bivariate Poisson generating function for the depth D n (u) = E[u Dn ], and and the total path length L n (u) = E[u Ln ] (e.g., n=0 and similar for e L(z; u)). They satisfy the following functional equations: ed(z; u) = u(p e D(zp; u) + q e D(zq; u)) + (1 u)e z ; el(z; u) = e L(zup; u) e L(zuq; u) + z(1 u)e z :

10 s n = 1 + Size of Tries Average Size: s n = E[S n ]: s 0 = s 1 = 1 and n p k q n k (s k + s n k ): k X Poisson transform: k S(z) = 1 + S(pz) + S(qz) 2(1 + z)e z (3) Bivariate Poisson GF: es(z; u) = ue S(zp; u) e S(zq; u) + (1 u)e z :

11 L n+1 (u) = u n Example: Digital Search Trees The exponential generating functions for digital search tree parameters satisfy the following = u(pd(zp; u) + qd(zq; u)) + 1 ; k (z) dh dz = H k 1 (zp)h k 1 (qz) with H 0 (z) = (1 + z). = L(zup; u)l(zuq; u) : Above equations come from corresponding recurrences. n (u) = E[u Ln ] satisfies L For example: nx n p k q n k (L k (u) + L n k (u)): k k=0

12 Leader election algorithm Figure below explains how a randomized leader election algorithm works (Prodinger, DM, 1993) T a b c d e f g H H b d e g a c f T b d e g H T d g b e T H d g d g H The loser -> d g Figure 3: An incomplete trie for the elimination process starting with 7 players. Height H n is the number of steps needed to find a leader (loser).

13 n = H n nx n (u) u k k=1 2 eh(z; u) = u(1 + e z=2 ) e H z 2 ;u Leader Election Poisson Transform H (u) E[u Let ] e Hn n and. P = n 0 H n(z) zn H(z; = u) n! The following functional equations are easy to derive k (u) + uh n(u) H n 2 + e z h (1 + z)(1 u) ue z=2i :

14 Generalized Probabilistic Counting In several algorithms on databases a major determinant of efficiency is the cardinality of the underlying set, saym. Generalized Flajolet & Martin idea works as follows. Consider an empty bitmap string with all positions filled by zeros. ffl Assume that N objects (e.g., data, persons, etc.) can randomly insert ffl (hit) 1 a at any position of the bitmap, however, the probability of hitting j 0 the position is equal 2 to. j 1 Every object can hit only one ffl time. In terms of probabilistic counting, the probability of the occurrence ffl of the 0 pattern 1 like is equal 2 to since 0 and 1 are equally likely. j 1 j ffl We count the number of hits in any position of the bitmap, but we count the number of hits only up to some value d + 1, whered is a given parameter. The bitmap is a d + 1-ary string. Clearly, the bitmap will contain a lot of ffl + 1 s at the beginning of the string. d

15 F n (u) = ef (z; u) = uf d (z=2) e F (z=2;u) + (u 1)(fd (z=2) 1) : Parameter of Interest The cardinality of M can be estimated based on N;d = minfk : BITMAP(k) < d + 1 R and for 0» i < k BITMAP(i) = d + 1g : all Ford = 3 Example: we may bitmap = z } have Then R N;3 = 11. Functional Equations R N;3 Let F n (u) = E[u R n;d ].Then X n d 1 dx n 2 n + u k k=0 n 2 n F k (u) : k and the Poisson transform satisfies k=0 where f d (z) = 1 e d (z)e z and e d (z) = 1 + z1 zd + : + d! 1!

16 g n z n eg(z) = z k n! e z ) g n ο e G(n): Basic Idea of Depoissonization Basic Problem: How to recover g n from asymptotics of eg(z) = X n 0 when e G(z) is only known in the cone as z! 1. Example. From Table 1 we know that for g n = n(n 1) (n k +1) ο n k we have

17 Does Depoissonization Always Work? For g n = ( 1) n the Poisson transform is e G(z) = e 2z g n 6ο e G(n):

18 What about this one? Figure 4: Poisson Transform of sin 2 (log n).

19 g n = n![z n ] n!(e z=m 1) m k Algebraic Depoissonization In the exact or algebraic depoissonization one g extracts n from its Poisson transform, that is, e z e G(z) Example: n Consider balls (items) thrown randomly and uniformly into urns. What is the probability P k (n) that precisely k specified urns are m empty? In the Poisson model we replace stream of balls by a Poisson process with mean z. Each urn receives an independent Poisson processes of mean z=m. : k (z) = e kz=m (1 e z=m ) m k = e z (e z=m 1) m k : P Depoissonizing it we have Prfempty urns = kg = n) = [z n ] = k)! (m n n n m n kψ where denote the Stirling numbers of the second kind. Φ m k o ;

20 Analytic Depoissonization Heuristics We first propose a heuristic derivation. As we observede G(z) = E[g N ], where N is a Poisson with mean z = n. Taylor s expansion around n is g(n) = g(n) + (N n)g 0 (n) g00 (n)(n n) 2 + : Taking the expectation we obtain eg(n) = e G(z)j z=n = E[g(N)] = g(n) g00 (n)n + since E[N n] = 0 and E[N n] 2 = n. Solving the above for g(n) = g n Provided that g n ß e G(n) 1 2 ng00 (n) + = e G(n) + O(ng 00 (n)): we have ng 00 (n) = o(g(n)); g n ο e G(n):

21 Examples Consider the following examples: Let = for which fi e fi G(n) = n + O(n fi 1 ),andg 00 (n) = O(n ), fi 2 ffl n g(n) thus n e 00 G(n) + O(ng (n)) = e fi 1 G(n) + O(n ); g = which is true. Consider g(n) ff now ff > with Thistimee 1. n e G(z) = g, (n) = ffl z(ff 1) = ff, and it 00 is not true that g n ο e ff 2 log n G(n). Nowweassumeg = e n. In this case, it is harder to find the Poisson ffl transform, but one suggests nfi that ο e zfi G(z). We also g have (n) = e 00 O(n 2fi 2 e nfi ). Observe that g n = e G(n) + O(ng 00 (n)) = e G(n) + O(n 2fi 1 e nfi ) and the error is small as long 0 < fi < as 1 2 depoissonization. (the so called exponential

22 n S n w = ne i n - x Basic Depoissonization Theorem Theorem e 1. Let G(z) be an entire function and let S be a complex cone around real < axis with. If the following two conditions hold: 2 ß (I) z 2 S : fi = O(z ) G(z) for some e fi; =2 S (O) e : z = O(e ffjzj ) G(z)e for ff < some z 1, then Even better g n = e G(n) + O(n fi 1=2 ): g n = e G(n) + O(n fi 1 ): y 6 (0;n) C(h)

23 g n = it z=ne = r n 2ß r n 2ß Z I e G(z)e z n+1 z ß Z Sketch of the Proof From the Cauchy and Stirling formulas we have n! dz 2ßi n! eg(ne it ) exp it e nit dt ne ß n n 2ßi = (1 + O(n))(I n (t) + E n (t)) where I n (t) = eg(ne it ;u) exp it 1 it dt e n E n (t) = eg(ne it ;u) exp it 1 it dt e n Z jtj2[ ;ß] Outside the Cone S : n (t) = O( n! E neffn ) = O(e (1 ff)n )! 1 n

24 Saddle Point theta Inside the Cone S (Saddle point method): t 0 =t= p n = n I 1 Z p n p 2ß eg(ne it=p n )exp n n 1 it= p n dt: it=p e p n

25 exp n e it= p n 1 it= p n = e t2 =2 it3 ψ1 t6 t4 + 24n 72n + O ψ 9 log n p n 6 eg(ne it=p n ) = eg(n) + it p n eg 0 (n) + t 2 n(t); eg 0 (z) = O(z fi 1 ) n p n!!

26 eg(z)e z = e jzj for arg(z) = ß : Counterexamples to (I) and (O) Example: (Violation of (O)) Let n ( 1) n e 2z G(z) = e : g = Condition (I) inside the cone is true. But, in this case the condition (O) outside S the cone does not hold because Clearly, g n 6ο e G(n). Example: (Violation of (I)) Take now g n = (1 + t) n e G(z) = e tz : Condition outside (O) the cone S holds for some such that (1+t) cos < 1. But the condition inside (I) the cone S does not hold since e G(z) has not a polynomial growth. Again g n 6ο e G(n).

27 g n = b ij n i e G hji (n) + O(n fi m 1 ) Full Asymptotic Expansion Theorem 2. Under same conditions as in Theorem 1, we have the following full expansion that holds for every fixed m: mx Xi+m i=0 j=0 b where ij are coefficients P of b ijx i y j = exp(x log(1 + y) xy): ij Remark: Gonnet-Munro (84) produced this expansion but without the validity conditions. Proof: 1. Use expansion of e G(z) knowing that e G hki (z) = O(z fi k ); 2. Use dominating convergence argument to prove that there is an expansion of g n as P b ij n i e G hji (n) for some bij. The first few terms of the above expansion are g n = e G(n) 1 2 n e G (2) (n) n e G (3) (n) n2 e G (4) (n) 1 4 n e G (4) (n) 1 6 n2 e G (5) (n) 1 n3 (6) (n) 1 n e (5) G (n) + 13 n2 e e (6) G (n) + 1 n3 e (7) G G (n) + 1 n4 e (8) G + (n) n (6) (n) 11 n2 e (7) G (n) 17 n3 e e (8) G (n) 1 n4 e (9) G G (n) 1 n5 e (10) G (n

28 n = log 2 n + fl g P 0(log 2 n)s log mx kx Example We are interested in asymptotic expansion of g n = 1X (1 2 k ) n : 1 k=0 The Poisson transform of g n is eg(z) = 1X e z2 k ; 1 k=0 and the jth derivative is e G (j) (z) = ( 1) j+1 P 1 k=0 2 jk e z2 k : By Theorem 2 one proves + ( 1) k+i+1 b i;k+i n k P k+i (log 2 n) + O(n m 1 log n) k=1 i=1

29 P 0 (log 2 x) = P j (log 2 x) = 1X (2ßi`= log 2)e 2ßi` log 2 x where 1 X log 2 `6=0 1 (j + 2ßi`= log 2)e 2ßi` log 2 x ; j 1: log 2 `= 1

30 1 2 Exponential Depoissonization Theorem 3. Let f(n) be a sequence whose Poisson transform is e F (z). (i) Assume that for z! 1 the following holds: (I) For z 2 S j e F (z)j» A exp(bjzj ν ) where 0» ν < 1=2, anda; B > 0 are constants. (O) For z =2 S F (z)ez j» A 1 exp(!jzj) je! < 1 for A and > 1 Thenforn! 1 0. = F (n) + O n (1 2ν) exp(bn ν ) : e f(n) (ii) Let conditions (I) and (O) hold again except that ν now in (I) satisfies ν < 1. Then colorblue < log f(n) = log e F (n) + O(n 2ν 1 ): Proof. Use the fact that j e F hki (z)j» Ajzj k(ν 1) exp(bjzj ν ).

31 = ec(z) (1 e z=2 ) e 8 z C ; 2 z Example Let us C study n defined below that appears in the analysis of the height of PATRICIA: nx n 2 n n k + 1 C k; n 2 k k=2 n = 4 C C with = 2 1. ItsPoisson transform satisfies and then F (z) = log z 2 e C(z)=2 has the following asymptotics in S as z! 1 (using Mellin transform) exp[f (z)] ο A p z2 1=12 exp " log 2 (z) 1 + Ψ(log # 2 z) ; 2 log 2 fl(1)+fl where is a fluctuating A = exp 2 =2 ß 2 =12 function and. log Ψ(z) 2 Exponential Depoissonization leads to C n ο A 1 2 n5=2 2 1=12 exp " log 2 (j) 1 + Ψ(log # 2 n) : 2 log 2

32 Poisson Mean and Variance For a random variable X n define by e X(z) and e V (z) the Poisson mean and Poisson variance, respectively. That is, they are the mean and the variance of X N where N is Poisson distributed. How the Bernoulli moments E[X n ] and Var[X n ] related to the corresponding Poisson moments e X(z) and e V (z)?. Theorem 4. Under previous hypotheses we have E[X n ] = e X(n) 1 2 n e X h2i (n) + O(n fi 2 ); Var[X n ] = e V (n) n[ e X 0 (n)] 2 + O(n fi 1 ) Example: Let Z 1 ;:::;Z n be a sequence of independently and identically distributed random variables with generating function P (u) = Eu Z 1 and mean μ and variance v. Then But E[X n ] = nμ; Var[X n ] = nv: 2 (z) (μ + v)z; e V = ] = e thus (n) n[ e0 2 X (n)] = (μ 2 + v)n nμ 2 = n V Var[X vn. ex(z) = zμ;

33 v n = q n (s n ) 2 Application: Variance of Tries Size Consider S the size n of a q trie. = E[S Let 2 n ], and the Poisson variance is n = Q(z) (S(z)) where 2 Q(z) is Poisson transform of q n. V (z) V (z) Observe that satisfies V (z) = V (pz) + V (qz) + (2S(z) 1 + (1 + z)e z )(1 + z)e z One proves (Jacquet-Regnier 87) (z) = C 1 z(1 + P 4 (log z)) + O(1) (p = q = 1=2) V Q(z) = O(z and ).Then 2 = Q(n) n 2 Qh2i (n) (S(n) n 2 Sh2i ) 2 + O(1) = V (n) n 2 (S0 (n)) 2 + O(1): Term n(s 0 (n)) 2 is of the same order as V (n) and cannot be neglected: Poisson variance differs from the Bernoulli variance on the second term asymptotics.

34 g n;k z n n! e z : Limiting Distributions g n;k Depoissonization techniques can also be used to derive limiting distributions. Such extension requires to analyze a double-index sequence ; for example n;k = PrfX n = kg; or g n;k = E[e tx n= sqrtv k ]: g Then Poisson transform is eg(z; u) = Eu XN = 1X n z e z n! 1X g n;k u k ; n=0 k=0 but often it is better to analyze a sequence of Poisson transforms eg k (z) = 1X n=1 How to infer limiting distribution (equivalently g n;k )from eg(z; u) or e Gk (z)?.

35 G n (u) = e G(n; u) + O(n fi 1 ) A Simple Depoissonization Theorem 5. Let e G(z; u) satisfy the hypothesis of previous depoissonization theorems, i.e., for some numbers < ß=2, A, B, ο > 0, fi, andff < 1 (I) and (O) hold for all u in a set U. Then uniformly for u 2 U. Theorem 6. e Suppose (z) = P 1 n=0 Gk n;k zn n! e z,fork belonging to some set K. If the following two conditions hold: g (I) z 2 S : Gk = O(z fi ) for some fi; e (z) =2 S (O) e : (z)e z = O(e ffjzj ) Gk for ff < some z 1, then uniformly in k 2 K g n;k = e Gk (n) + O(n fi 1 ) (4) and the error estimate does not depend on K.

36 ed(z; u) = u(p e D(zp; u) te[d p Var[Dn] E» e td n= p Var[Dn] n]= e Example PATRICIA Depth Example: DepthinPATRICIA. The depth D n in PATRICIA satisfies + q e D(zq; u))(1 u)(p e D(zp; u)e qz + q e D(zq; u)e pz ) : One can prove that conditions (I) and (O) hold (e.g., e D(z; u) = O(z " ) inside a cone). Using Mellin transform as z! 1 we prove that e t e X(z)=ff(z) e D(z; e t=ff(z) ) = e t2 =2 (1 + O(1=ff(z))); for u = e t, t complex, where e X(z) = O(log z) and ff 2 (z) = O(log z) (provided the source is biased). By previous result we can E[D prove ] ο e that n X(n) and ο Var[Dn ] (n). This suffices to establish the next result. 2 ff Theorem 7 (Rais, Jacquet and S., 1993). For complex t = e t2 =2 (1 + O(1= p log n)) ; that is, (D n E[D n ])= p Var[D n ] converges in distribution and in moments to the standard normal distribution.

37 (I) z 2 S, jzj 2 (n Dn; n + Dn): j e Gn (z)j» Bn fi (O) z =2 S, jzj = n: j e Gn (z)e z j» e n nff g n;n = ij i ehji G n (n) + O(nfi m 1 ) n b Diagonal Depoissonization When proving Central Limit Theorem we must deal with g n;k = E[e tx n= p V k]. But often we are only interested in g n;n. Diagonal Depoissonization is useful in this case. Theorem 8. Let e Gm (z) = P g n;m z n be a sequence of Poisson transforms. n! If there exists a ez cone and constants D > 0 B, ff < 1 and such S that Then mx Xi+m i=0 j=0 = G n (n) + O(n fi 1 )

38 j log e Gn (z)j» Bn fi n;n e Gn (n) exp( n g ( e0 G (n) )2 = )(1 + O(n 3fi 2 )): eg(n) 2 Diagonal Exponential Depoissonization Theorem 9. Sequence of Poisson e transforms Gn (z). There exists a S cone log( e where (z)) Gn exists. fi 2 (1=2; 2=3) Let an d B > A, ff < 0, 1. (I) z 2 S, jzj 2 (n Dn; n + Dn): (O) z =2 S, jzj = n: j e Gn (z)e z j» e n Anff, then Even more sophisticated depoissonization theorems can be proved with a combination of algebraic cones, diagonal and exponential depoissonization.

39 eg k (z) = (1 e z ) z=2 k Leader Election Distribution Consider the Poisson transform eg(z; u) = u(1 + e z=2 ) e G z 2 ;u + e z [(1 + z)(1 u) ue z=2 ] : Using Mellin transform we eg(z; u) ο zlog u ((1 u) (1 log u) (1 log u) + P (log u)) and then ln 2 z=2k 1 : e After applying the residue theorem, and P (H depoissonization < j) ο n j (n) we arrive at eg PrfH n» log n + kg = ρ(n) k 2 ρ(n) k ) 1 + O 1 pn exp(2 : where ρ(n) = log n blog nc. Observe that this does not give a limiting distribution.

40 eg(z; u) = uf d (z=2) e G(z=2;u) + (u 1)fd (z=2) Probabilistic Counting Asymptotic Distribution We have the following Poisson transform for the estimate R n;d where f d (z) = 1 e d (z) and e d (z) is truncate exponential function. Then:! ; k = k eg(z; u) ]ψ eg (z) [u u 1 '(z2 k 1 ) = '(z) where '(z) = 1Y f d (z2 j ) = 1Y e d (z2 j )e z2j : 1 This would lead to the following asymptotic distribution if we can prove the depoissonization: j=0 j=0 PrfR n;d» log 2 n + m 1g = 1 ' m ρ(n) + O(n 1=2 ) ; 2 where ρ(n) = log 2 n blog 2 nc.

41 s n = u X k p v n n p k q n k s k s n k k Distribution of Size of a Trie Let s n (u) = E[u Sn ].Thens 0 (u) = s 1 (u) = u, which reads as (Jacquet and Regnier, 1987) S(z; u) = us(pz; u)s(qz; u) (u 2 1)u(1 + z)e z : Mellin log(s(z; e analysis )) = S(z)t + V (z) ), t2 2 + O(zt3 ffl Diagonal depo: t Gn (z) = e ts n= p vn S(z; e ), t=pvn e ffl when z = O(n): ffl e Gn (z) = exp( S(z) s n log V (z) t2 =2 + o(n 1=2 )) = O(n ); 1=2 vn + Diagonal exponential e depo: n= p vn s ts (e t=pvn ) = exp(log e Gn (n) ffl n ((log e n (n)) 0 ) 2 )(1 + O(n Gn )). 1=2 2 Since log e Gn (n) = v(n) 2vn t2 + O(n 1=2 ) and (log e Gn (n)) 0 = s0 (n) p v n t + ffl 1 ) then O(n e (n) v(n) n(s0 (n)) 2 =2 log t 2 Gn =2 + o(n 1=2 t2 ) = = + o(n 1=2 ) 2 n v and the distribution is asymptotically normal.

42 L n+1 (u) = u n X L(z; u) = L(puz; u)l(quz; u) Digital Search Trees Define L n (u) = E[u Ln ].Then n p k q n k (L k (u) + L n k (u) k which leads to particularly differential-function equation with L(z; u) = e L(z; u)e z (exponential PGF). (Ugly nd dirty) asymptotic analysis provides log L(z; e t ) = O(z» (t)) with ffl t )»(t) + (qe t )»(t) = 1. (pe ffl Similarly to tries size but harder analysis that requires polynomial cone (not a linear cone) in order to prove limiting normal distribution. This result was used to find redundancy of Ziv-Lempel compression algorithm (Louchard, S., 1997)