A Multivariate Normal Law for Turing s Formulae

Transcription

1 A Multivaiate Nomal Law fo Tuing s Fomulae Zhiyi Zhang Depatment of Mathematics and Statistics Univesity of Noth Caolina at Chalotte Chalotte, NC Abstact This pape establishes a sufficient condition fo Tuing s fomulae of vaious odes to have asymptotic multivaiate nomality. As an application, a consistent estimato of the tail unde a discete powe tail model is also descibed. 1 Intoduction. Conside the population of all bids in the wold with all of its diffeent species. Suppose a andom sample of n = 2000 is taen fom the population and M = 30 diffeent bid species ae found in the sample. Since the total numbe of bid species in the population must exceed 30, one may be inteested in the total population popotion of bids belonging to the species othe than the 30 obseved in the sample. Could this popotion be easonably estimated? Good 1953) intoduced a nonpaametic estimato, cedited lagely to Alan Tuing, fo just that popotion in question. The fomula is nown as Tuing s fomula. Tuing s fomula suggests that, if among the M = 30 diffeent species in the sample N 1 species include exactly one bid each, say N 1 = 10, then N 1 /n = 10/2000 = is a easonable estimate fo the popotion in question. Tuing s fomula has an intiguing implication: the total popotion associated with species not obseved in the sample can be nonpaametically estimated albeit subject to some eo). This AMS 2000 Subject Classifications. Pimay 62f10, 62F12, 62G05, 62G20; seconday 62F15. Keywods and phases. High ode Tuing s fomulae, asymptotic multivaiate nomality. Reseach patially suppoted by NSF Gants DMS

2 Nomality of Tuing s Fomulae 2 almost-anti-intuitive implication has inspied much eseach both in applications and in theoy. One aea which has seen much eseach elated to Tuing s fomula includes the poblems of estimating population divesity indices. Thee ae seveal diffeent types of divesity indices that ae commonly discussed in the liteatue, the total numbe of population species, Simpson s index and Shannon s entopy. Thee is a lage volume of publications on estimation of total numbe of population species incopoating Tuing s fomula in some way. Notable publications thee include Chao 1984), Chao and Lee 1992), Chao and Bunge 2002), Mao and Lindsay 2005), Wang and Lindsay 2005), and Zhang and Sten 2009) among many othes. Zhang and Zhou 2010) discussed the estimation poblem of Simpson s divesity and its genealized foms. On the estimation of Shannon s entopy, notable eseach includes Chao and Shen 2003), Vu, Yu and Kass 2007), and Zhang 2012). Howeve deviating fom the themes of the above mentioned efeences, the focus of the cuent pape is on the lage sample distibutional chaacteistics of Tuing s fomula and its highe ode elatives. Conside a multinomial distibution with its countably infinite numbe of pescibed categoies indexed by K = {; = 1, } and its categoy pobabilities denoted by {p }, satisfying 0 < p < 1 fo all and p = 1, whee the sum without index is ove all as is obseved in the subsequent text unless othewise stated. Let the categoy counts in an iid sample of size n fom the undelying population be denoted by {X ; 1} and its obseved values by {x ; 1}. Fo a given sample, thee ae at most n non-zeo x s. Let, fo evey intege, 1 n, N = 1 X =], T = ) n n 1 ) 1 N = n + 1 N, and π 1 = p 1 X = 1]. N and π 1 may be thought of as, espectively, the numbe of categoies in the population that ae epesented exactly times in the sample and the total pobability associated with all the categoies that ae epesented exactly 1 times in the sample. T may be thought of as an estimato of π 1. T is also nown as Tuing s fomula of the th ode intoduced by Good 1953). Pehaps the most inteesting case among all Tuing s fomulae of diffeent odes is T 1, nown as

3 Nomality of Tuing s Fomulae 3 just Tuing s fomula, as an estimato of π 0. Given a sample, π 0 epesents the total pobability associated with categoies not obseved in the sample, which is also the pobability that the next obsevation will belong to a categoy peviously unseen. Since the multinomial model is essentially nonpaametic, the fact that something could be said about the total pobability associated with unobseved categoies is somewhat anti-intuitive. The statistical popeties of Tuing s fomula wee lagely unnown until Robbins 1968) gave an intepetation in tems of bias. Anothe fifteen yeas would pass befoe Esty 1983) gave a sufficient condition fo the asymptotic nomality of T 1 π 0. In ecent yeas, eseach on Tuing s fomula has been evitalized. Zhang and Huang 2007) gave anothe intepetation of Tuing s fomula and poposed an impoved vesion of the fomula which essentially eliminated all the bias of Tuing s oiginal fomula. Zhang and Huang 2008) gave a sufficient condition fo the nomality of Tuing s fomula which suppots a non-empty class of fixed distibutions. Zhang and Zhang 2009) gave a necessay and sufficient condition fo the nomality of Tuing s fomula. Lijoi, Mena and Pünste 2007) offeed an discussion in Bayesian pespective. Howeve all the wos thus fa ae on Tuing s fomula of the fist ode. Pio to this pape, the distibutional popeties of high ode Tuing s fomulae ae unnown. The objective of this pape is to establish the asymptotic multivaiate nomality of seveal Tuing s fomulas of vaious odes unde cetain conditions. The main line of the poofs in this pape lagely follows that of Esty 1983) o Zhang and Huang 2008) which involves diect evaluations of the chaacteistic functions of popely nomalized Tuing s fomulas of vaious odes. Seveal ey peliminay esults ae given in Section 2, followed by the univaiate nomality esults fo Tuing s fomulae in Section 3, and followed by the multivaiate nomality esults in Section 4. In Section 5, a bief discussion of how the established esults could be used to deive a consistent estimato fo the tail of a discete pobability distibution with a powe decaying tail. Many of the lengthy poofs fo the esults thoughout the pape ae given in Section 6.

4 Nomality of Tuing s Fomulae 4 2 Peliminay Results. Let K 1 = {1} and K 2 = {2, }. Fo any K = K 1 K 2, let p x = 1, f x) = /n + 1) x =, 0 0 x 2 o x + 1, 2.1) and Z = f X ). The objective is to deive the asymptotic behavio of Zgn), whee gn) is a function of n satisfying gn) = On 1 2δ ) 2.2) fo some δ 0, 1/4), in tems of the limit of its chaacteistic function, EexpisZgn))]. Let Z = Z 1 + Z 2, whee Z 1 = K 1 f X ) and Z 2 = K 2 f X ). Lemma 2.1 below is a well-nown fact and Lemma 2.2 is due to Batlett 1938). Lemma 2.1 Let {X } be the counts of obsevations in categoy, = 1, 2,, in an iid sample unde the multinomial model with pobability distibution {p }. Then P X = x ; = 1, ) = P Y = x ; = 1, Y = n) whee {Y } ae independent Poisson andom vaiables with mean np. Lemma 2.2 Let U, V ) be a two-dimensional andom vecto with U intege valued. Then EexpivV U = n)) = 2πP U = n)) 1 π π EexpiuU n) + ivv )]du. Thus EexpisZgn))) is 1 π 2πP Y = n)) E exp iu )] Y np ) + iszgn) du. π By Stiling s fomula, 2πn) 1/2 P Y = n) 1. Theefoe it suffices to evaluate the limit of H n s) = n 2π π π Eexpiu Y np ) + iszgn))]du,

5 Nomality of Tuing s Fomulae 5 o letting t = un 1/2, H n s) = t <π n] Eexpin) 1/2 t Y np ) + iszgn))]dt. 2.3) 2π Let h n = 1 t < π n]eexpin) 1/2 t Y np ) + iszgn)] h n1 = 1 t < π n]eexpin) 1/2 ty 1 np 1 ) + isz 1 gn)] 2.4) h n2 = 1 t < π n]eexpin) 1/2 t K 2 Y np ) + isz 2 gn)], H n s) = 1 + h n dt = 1 + h n1 h n2 dt. 2.5) 2π 2π The fist tas is to allow the limit opeato to change place with the integal opeato, i.e., to show lim H n s) = 1 2π lim hn dt whee lim = lim n as is obseved elsewhee in the subsequent text. The ey element to suppot this exchange is lim h n1 dt = lim h n1 dt, 2.6) whee h n1 = 1 t π n] {exp itn 1/2 p 1 ) expnp 1 e itn 1/2 1)] +2np 1 ) 1 exp np 1 )/ 1)! + np 1 ) exp np 1 )/!] } is an uppe bound fo h n1 and hence, since h n2 1 implies h n h n1, an uppe bound fo h n. The poof of 2.6) is given by Zhang and Huang 2008) fo a special case of = 1, howeve the poof is also valid fo any 1. By 2.6) and the extended dominated convegence theoem of Patt 1960), the following lemma is established. Lemma 2.3 Let h n and H n be as defined in 2.3) and 2.5) espectively. Then lim H n = 1 2π lim h n dt.

6 Nomality of Tuing s Fomulae 6 Fo each, it can be veified that, letting B = exp itp n 1/2 )expnp expitn 1/2 ) 1))] C = exp itp n 1/2 )expisp gn)) 1) expit 1)n 1/2 ) exp np ) np ) 1 1)! D = exp itp n 1/2 )exp isn + 1) 1 gn)) 1] expitn 1/2 ) exp np ) np )!, and E = C + D, h n = B + E ) fo all t 0 ± π n. The objective is to evaluate lim B + E ). The following two lemmas ae given by Esty 1983) whee is equality in the limit as is obseved elsewhee in the subsequent text. Lemma 2.4 Let {β } and {ϵ } be two sequences of complex numbes, and M n be a sequence of subsets of K, indexed by n. If 1. M n β β, 2. M n ϵ ) ϵ, 3. β 1 unifomly, 4. ϵ 0 unifomly, 5. thee exists a constants, δ 1 such that, M n β 1 δ 1, and 6. thee exists a constants, δ 2 such that, M n ϵ δ 2 ; then whee β and ϵ may also depend on n. M n β + ϵ ) βe ϵ Lemma 2.5 Fo all K, B = exp t 2 /2)p + Ot 3 p n 1/2 )). The next lemma includes thee tivial but useful facts.

7 Nomality of Tuing s Fomulae 7 Lemma Fo any complex numbe x satisfying x < 1, ln1 + x) x 1 x. ) 2. Fo any eal numbe x 0, 1), 1 x exp x 1 x. 3. Fo any eal numbe x 0, 1/2), 1 1 x < 1 + 2x. Poof. 1) By Taylo s fomula, ln1 + x) = j=1 1) j+1 x j /j j=1 x j = x /1 x ). 2) The function y = 1 1+t et is stictly inceasing ove 0, ), and has value 1 at t = 0. Theefoe 1 1+t et 1 fo t 0, ). The desied inequality follows the change of vaiable x = t/1 + t). 3) The poof is tivial. Conside a patition of the index set K = I II whee I = {; p /n 1 δ } and II = {; p > /n 1 δ } whee δ = δ/r + 1) and δ is as in 2.2). Lemma 2.7 a) II E 0; and b) II B + E )/ II B 1. Poof. a) II E 2 II e np np ) 1) / 1)! + e np np ) /!]. Since the deivative of e np np) 1) / 1)! + e np np) /!] with espect to p is negative fo all p /n, 1] and theefoe fo all p /n 1 δ, 1]), e np np ) 1) / 1)! + e np np ) /!] attains its maximum at p = /n 1 δ, fo evey II, with value e nδ On δ ). The total numbe of indices in II is less o equal to n 1 δ /. Theefoe II E 2n 1 δ /]e nδ On δ )] = 2/)e nδ O n 1+ 1)δ ) 0. b) By Lemma 2.5, B is bounded away fom zeo, and by the fact that lim E = 0 and hence lim E / B = 0), and by applying the fist pat of Lemma 2.6 with x = E /B, one has ln II B + E )/ II B ] = ) ) II 1 ln + E B II ln 1 + E B ) E II B E = O II E ) 0.

8 Nomality of Tuing s Fomulae 8 The following is a sufficient condition unde which many of the subsequent esults ae established. Condition 2.1 As n, 1. n 2 g 2 n)p e np c 0, 2. n 1 g 2 n)p +1 e np c +1 0, and 3. c + c +1 > 0. Lemma 2.8 Unde Condition 2.1, all the conditions of Lemma 2.4 ae satisfied with M n = I, β = B, β = B = lim B, ϵ = E, and ϵ = E = lim E. The poof of Lemma 2.8 is given in Appendix. Lemma 2.4 and Lemma 2.8 give immediately the following coollay. Coollay 2.1 Unde Condition 2.1, I B + E ) I B exp I E ). Lemma 2.9 Unde Condition 2.1, B + E ) Be E, whee B = lim B and E = lim E. Poof. B + E ) = I B + E ) II B + E ) I B + E ) II B by Lemma 2.7) I B exp I E ) II B by Lemma 2.8) B exp E ) by Lemma 2.7). 3 Univaiate Nomality. Theoem 3.1 Let gn) be as in 2.2). Unde Condition 2.1, gn)t π 1 ) L N 0, c +1 + c 1)! ).

9 Nomality of Tuing s Fomulae 9 Poof. Since lim B = e t2 2, by a) of Lemma 2.7 and 6.2), s2 lim E = 2 1)! ) 1 lim H n = 2π e t2 2 dt e lim n 1 g 2 n)p +1 e np + lim g 2 n)n p n + 1) 2 e np ] s2 2 1)! lim n 1 g 2 n)p +1 e np g 2 n)n p + lim n +1) 2 e np ], = e s2 2 c+1 1)! + c 1)! ]. Conside the following condition: Condition 3.1 As n, 1. g2 n) EN n 2 ) c! 0, 2. g2 n) EN n 2 +1 ) c +1 +1)! 0, and 3. c + c +1 > 0. Lemma 3.1 Condition 2.1 and Condition 3.1 ae equivalent. The poof of Lemma 3.1 is given in Appendix. Lemma 3.1 allows a e-statement of Theoem 3.1: Theoem 3.2 If thee exists a gn) satisfying 2.2) and Condition 3.1, then nt π 1 ) 2 EN ) + + 1)EN +1 ) L N0, 1). Theoem 3.3 If thee exists a gn) satisfying 2.2) and Condition 3.1, then nt π 1 ) 2 N + + 1)N +1 L N0, 1). The poof of Theoem 3.3 is given in Appendix. It may be of inteest to note that the esults of Theoems 3.2 and 3.3 equie no futhe nowledge of gn), i.e., the nowledge of δ, othe than its existence.

10 Nomality of Tuing s Fomulae 10 4 Multivaiate Nomality. Fo evey K, any two constants a and b, and any two positive integes 1 and 2, let f x) in 2.1) be edefined as f x) = ap x = 1 1, a 1 /n 1 + 1) x = 1 2 1, bp x = 2 1 1, b 2 /n 2 + 1) x = 2, bp a 1 /n 1 + 1) x = 1 = 2 1, 0 elsewhee, 4.1) and Z = f X ). The objective is to evaluate lim H n s) = 2π) 1/2 lim h n dt whee H n s) and h n have the same foms as in 2.4) and 2.5) but with f x) edefined in 4.1). Two sepaate cases ae to be consideed: 1 < 2 1 and 1 = 2 1. Let B = exp itp n 1/2 )expnp expitn 1/2 ) 1))] C = exp itp n 1/2 )expisap gn)) 1) expit 1 1)n 1/2 ) np ) )! e np D = exp itp n 1/2 )exp isa 1 n 1 + 1) 1 gn)) 1] expit 1 n 1/2 ) np ) 1 1! e np F = exp itp n 1/2 )expisbp gn)) 1) expit 2 1)n 1/2 ) np ) )! e np G = exp itp n 1/2 )exp isb 2 n 2 + 1) 1 gn)) 1] expit 1 n 1/2 ) np ) 2 2! e np A = exp itp n 1/2 ){expisgn)bp a n 1 +1 ) 1]} expit 1n 1/2 ) np ) 1 1! e np. If 1 < 2 1, let E = C + D + F + G. If 1 = 2 1, let E = C + A + G. It can be veified that, in eithe case, h n = B + E ) fo all t 0 ± π n. The objective is to evaluate lim B + E ). 1 Condition 4.1 As n, 1. g2 n) n 2 EN 1 ) c 1 1! 0, 2. g2 n) n 2 EN 1 +1) c )! 0,

11 Nomality of Tuing s Fomulae c 1 + c 1 +1 > 0, 4. g2 n) n 2 EN 2 ) c 2 2! 0, 5. g2 n) EN n ) c )! 0, and 6. c 2 + c 2 +1 > 0. Lemma 4.1 Fo any two constants, a and b satisfying a 2 + b 2 > 0, assuming that 1 < 2 1 and that Condition 4.1 holds, then gn)at 1 π 1 1) + bt 2 π 2 1)] L N0, σ 2 ) whee σ 2 = a 2 c c 1 1 1)! + b 2 c c 2 2 1)!. The poof of Lemma 4.1 is staight fowad in light of the agument that led to Theoem 3.1. Lemma 4.2 Fo any two constants, a and b satisfying a 2 + b 2 > 0, assuming that 1 = 2 1 and that Condition 4.1 holds, then gn)at 1 π 1 1) + bt 2 π 2 1)] L N0, σ 2 ) whee σ 2 = a 2 c c 1 1 1)! 2ab c 2 1 1)! + b2 c c 2 2 1)!. The poof of Lemma 4.2 is also staight fowad in light of the agument that led to Theoem 3.1, but with an additional non-vanishing tem in the limit. Let σ 2 = 2 EN ) + + 1)EN +1 ), ρ n) = + 1)EN +1 )/σ σ +1 ), ρ = lim ρ n), ˆσ 2 = 2 N + + 1)N +1, and ˆρ = ˆρ n) = + 1)N +1 / ˆσ 2 ˆσ Coollay 4.1 Assume that 1 < 2 1 and that Condition 4.1 holds, then n T 1 π 1 1)/σ 1, T 2 π 2 1)/σ 2 ] L MV N 0, I 2 2 ).

12 Nomality of Tuing s Fomulae 12 Coollay 4.2 Assume that 1 = 2 1 and that Condition 4.1 holds, then n T 1 π 1 1)/σ 1, T 2 π 2 1)/σ 2 ] L 1 ρ1 MV N 0, ρ 1 1 )). Rema 4.1 Coollaies 4.1 and 4.2 suggest that, in {nt π 1 )/σ ]; = 1,, R}, any two enties ae asymptotically independent unless they ae immediate neighbos in the seies. Coollaies 4.1 and 4.2 immediately give the following theoem. Theoem 4.1 Fo any positive intege R, if Condition 3.1 holds fo evey, 1 R, then n T 1 π 0 )/σ 1,, T R π R 1 )/σ R ] L MV N0, Σ) whee Σ = a i,j ) is a R R covaiance matix with all the diagonal elements being a, = 1 fo = 1,, R, the supe-diagonal and the sub-diagonal elements being a,+1 = a +1, = ρ fo = 1,, R 1, and all the othe off-diagonal elements being zeos. Let ˆΣ be the esulting matix of Σ with ρ eplaced by ˆρ n) fo all. Let ˆΣ 1 denote the invese of ˆΣ and ˆΣ 1/2 denote any R R matix satisfying ˆΣ 1 = ˆΣ 1/2 ˆΣ 1/2. With the consistency of these estimatos aleady established in the poof of Theoem 3.3, Theoem 4.2 below follows Theoem 4.1. Theoem 4.2 Fo any positive intege R, if Condition 3.1 holds fo evey, 1 R, then nˆσ 1/2 T 1 π 0 )/ˆσ 1,, T R π R 1 )/ˆσ R ] L MV N0, I R R ). 5 An Application. An inteesting special case of discete distibution is that of {p } following a discete powe law, as nown as a Paeto law, in the tail, i.e., p = C λ 5.1)

13 Nomality of Tuing s Fomulae 13 fo all 0 whee C > 0 and λ > 1 ae unnown paametes descibing the tail of the pobability distibution at and beyond an unnown positive intege 0, while fo each < 0, p confoms to no paticula paametic foms. This patially paametic pobability model is also nown as one of the powe tail models. Suppose it is of inteest to estimate C and λ. The esults of last section could be used to develop a consistent estimation pocedue as poposed below. Lemma 5.1 Unde the model in 5.1), Condition 4.1 holds. Poof. Letting δ = 4λ) 1 in 2.2), it can be veified that n 2 g 2 n) p e np c > 0 fo evey intege > 0. Coollay 5.1 Unde the model in 5.1), the esults of both Theoems 4.1 and 4.2 hold. Let δ = 4λ) 1 in 2.2). Unde the powe tail model 5.1), fo evey > 0, let evey p in π 1 be eplaced by p = C λ and denote the esulting expession as π 1, i.e., π 1 = C λ 1y = 1]. Then π π = 0 1 =1 1y = 1]p C λ ), and the sum is of finite tems. Since, fo evey, both E1y = 1]) and V a1y = 1]) convege to zeo exponentially, and hence Eg n δ)1y = 1]) and V ag n δ)1y = 1]) convege to zeo, which implies that g n δ)1y = 1] p 0, which in tun implies that π π p 0 fo evey. This agument leads to the following two lemmas: Lemma 5.2 Unde the powe tail model in 5.1), n T 1 π 0)/σ 1,, T R π R 1)/σ R ] L MV N0, Σ) whee Σ is as in Lemma 4.1. Lemma 5.3 Unde the powe tail model in 5.1), nˆσ 1/2 T 1 π 0)/ˆσ 1,, T R π R 1)/ˆσ R ] L MV N0, IR R ). 5.2) whee ˆΣ 1/2 is as in Lemma 4.2.

14 Nomality of Tuing s Fomulae 14 The asymptotic lielihood given by 5.2) depends on the two model paametes, λ and C, only via π 1 = C 1y = 1] λ, = 1,, R. Definition 5.1 If thee exists a pai of values, C = Ĉ and λ = ˆλ, which maximizes the lielihood given by 5.2), then Ĉ, ˆλ) is said to be an asymptotic maximum lielihood estimato AMLE) of C, λ). Given a sample, an AMLE minimizes ll n = ll n C, λ) = T 1 π 0,, T R π R 1 ) ˆΣ 1 R ) T 1 π0,, T R πr 1 whee ˆΣ 1 R 1/ˆσ = nˆσ... 1/ˆσ 2 R and the last displayed R R matix above has non-zeo elements only on the diagonal. Finally the following theoem can be veified. Theoem 5.1 Unde the powe tail model in 5.1), the AMLE of C, λ) is unique fo a sufficiently lage n and is consistent. 6 Appendix. 6.1 Poof of Lemma 2.8. All six conditions in Lemma 2.4 need to be checed. 3) is tue because and p and p / n ae unifomly bounded by Fo 1), since I p 0, B = exp t 2 /2)p ) expot 3 / n)p ))), n and 1 δ nn 1 δ espectively. B = exp t 2 /2) I I p ) expot 3 / n) I p ))) 1.

15 Nomality of Tuing s Fomulae 15 Fo 4), it suffices to show that C and D espectively convege to zeo unifomly. Fist fo all I, exp itp n) 1 unifomly since p n n gn)n δ = On 1/2+δ ) 0 unifomly. Second, expit 1)n 1/2 ) 0 and expitn 1/2 ) 0 unifomly. Thid, exp np ) 1 unifomly. By Taylo s expansion and fo sufficiently lage n, expisp gn)) 1] np ) 1 1)! = isgn)p s2 g 2 n)p 2 2! O s 3 g 3 n)p 3 ) ) np ) 1 1)! = isn 1 gn)p 1)! s2 n 1 g 2 n)p +1 2! 1)! isn 1 gn)p 1)! + s 2 n 1 g 2 n)p +1 2! 1)! O s 3 n 1 g 3 n)p +2 ) O + s 3 n 1 g 3 n)p +2 ) s 1)! n 2δ+ R+1 δ + s2 +1 2! 1)! +1 n 4δ+ R+1 δ +2 6δ+ + O n δ) R+1 0 unifomly. Similaly, it is easily checed that exp isn + 1) 1 gn)) 1 ] np )! = isgn) )) n +1 s2 2 g 2 n) + O s 3 3 g 3 n) np ) 2!n +1) 2 3!n +1) 3! = isgn)n p!n +1) s2 2 g 2 n)n p 2!!n +1) 2 ) s + O 3 g 3 n)n p n +1) 3 ) isgn)n p!n +1) + s2 2 g 2 n)n p + 2!!n +1) 2 O s 3 g 3 n)n p n +1) 3 s n 1)! n +1 n 2δ+ R+1 δ + s2 +2 2!! n 2 n 4δ+ n +1) 2 R+1 δ + O n 3 n 6δ+ n +1) 3 R+1 δ) 0 unifomly. Theefoe E 0 unifomly.

16 Nomality of Tuing s Fomulae 16 Fo 2) and 6), E = e np exp itp n) expit 1)n 1/2 isn ) 1 gn)p )] O s 3 n 1 g 3 n)p +2 1)! s2 n 1 g 2 n)p +1 2! 1)! )] +e np exp itp n) expitn 1/2 ) isgn)n p!n +1) s2 2 g 2 n)n p s + O 3 g 3 n)n p 2!!n +1) 2 n +1) 3 = e np exp itp n) expit 1)n 1/2 isn ) 1 gn)p )] O s 3 n 1 g 3 n)p +2 1)! s2 n 1 g 2 n)p +1 2! 1)! +e np exp itp n) expitn 1/2 ) isgn)n 1 p 1)! is 1)gn)n 1 p 1)!n +1) s2 2 g 2 n)n p 2!!n +1) 2 )] s +O 3 g 3 n)n p n +1) 3 { = e np e itp n e it 1)n 1/2 isn 1 gn)p 1)! s2 n 1 g 2 n)p +1 2! 1)! O s 3 n 1 g 3 n)p +2 ) ) it n t2 it3 2!n O ) isgn)n 1 p 3!n 3/2 1)! is 1)gn)n 1 p 1)!n +1) s2 2 g 2 n)n p 2!!n +1) 2 )]} s +O 3 g 3 n)n p n +1) 3 { = e np e itp n e it 1)n 1/2 is 1)gn)n 1 p 1)!n +1) s2 n 1 g 2 n)p +1 2! 1)! s2 2 g 2 n)n p 2!!n +1) 2 ) ) s +O 3 g 3 n)n p O s n +1) 3 n 1 g 3 n)p +2 3 ) + it n t2 it3 2!n O ) isgn)n 1 p 3!n 3/2 1)! is 1)gn)n 1 p 1)!n +1) s2 2 g 2 n)n p 2!!n +1) 2 6.1) )]} s +O 3 g 3 n)n p. n +1) 3 Noting the unifom convegence of e itp n e it 1)n 1/2 1 and Condition 2.1, it can be checed that all tems in 6.1) vanish unde lim I, except possibly the fist thee tems within

17 Nomality of Tuing s Fomulae 17 the culy bacets, i.e., lim I E = lim I { e np is 1)gn)n 1 p 1)!n +1) s2 n 1 g 2 n)p +1 2! 1)! ]} s2 2 g 2 n)n p 2!!n +1) 2 = is 1) 1)! lim gn)n 1 p I n +1) e np s2 2! 1)! lim I n 1 g 2 n)p +1 e np s2 2 2!! lim g 2 n)n p I e n +1) np. 2 Condition 2.1 guaantees the existence of the second and the thid tems above, and the existence of the thid tem implies that the fist tem is zeo. Theefoe 2) is checed and lim E = s2 lim n 1 g 2 n)p +1 e np + lim g 2 n)n p 2 1)! n + 1) 2 e np. 6.2) I I I The convegence of I E and hence of I E guaantees 6). ) Fo 5), since B = exp t2 2 p + Ot 3 p n 1/2 ) and t2 2 p + Ot 3 p n 1/2 ) 0 unifomly, B 1 t2 2 p + Ot 3 p n 1/2 ) ) t 2 1 t2 2 p + Ot 3 p n 1/2 ) O 2 p + t 3 p n 1/2 and hence ) t 2 B 1 O p ) + t3 p < Ot 2 + t 3 ). 2 I I n I 6.2 Poof of Lemma 3.1. Conside the patition of K = I II. Since pe np has a negative deivative with espect to p on inteval 1/n, 1] and hence on /n 1 δ, 1] fo lage n, pe np attains its maximum at p = /n 1 δ. Theefoe noting that thee ae at most n δ / indices in II, 0 g2 n) n n 2 ) II p 1 p ) n g2 n) n n 2 g2 n) n 2 n ) e II n 1 δ e n n 1 δ ) g2 n) n 2 ) II p e n )p g2 n) n ) n 2 e II p e np n ) e n δ n ) n 1 δ e n 1 δ Thus = g2 n) n n 2 ) e n δ n1 δ e nδ 0. ) lim g2 n) n 2 EN ) = lim g2 n) n n 2 p 1 p ) n 6.3) I

18 Nomality of Tuing s Fomulae 18 and lim n 2 g 2 n) p exp np ) = lim n 2 g 2 n) I p exp np ). 6.4) On the othe hand, g 2 n) n n 2 ) I p 1 p ) n g2 n) n n 2 ) I p e n )p g2 n) n ) n 2 exp supi p ) I p e np. Futhemoe, applying 2) and 3) of Lemma 2.6 in the fist and the thid steps below espectively leads to g 2 n) n n 2 ) I p 1 p ) n g2 n) n ) ) n 2 I p exp n )p 1 p g2 n) n ) n 2 I p exp ) np 1 sup I p g2 n) n ) n 2 I exp 2nsup I p ) 2 )p e np. Noting the fact that lim exp sup I p ) = 1 and lim exp 2nsup I p ) 2 ) = 1 unifomly by the definition of I, lim g2 n) n 2 ) n p 1 p ) n = lim g2 n) n 2 I ) n p e np, I and hence, by 6.3) and 6.4) and by the fact that n ) n /!, the equivalence of the fist pats of Condition 2.1 and Condition 3.1 is established: lim g2 n) n 2 EN 1 ) = 1/!) lim n 2 g 2 n) p exp np ). The equivalence of the second pats can be established similaly. 6.3 Poof of Theoem 3.3. Based on Theoem 3.2, it suffices to show that the vaiances of appoach zeo as n inceases to infinity. ĉ =!g2 n) n 2 N and ĉ +1 = + 1)!g2 n) n 2 N +1 V aĉ ) =!)2 g 4 n) n 4 V an ) =!)2 g 4 n) { n 4 EN 2 ) EN )] 2}. 6.5)

19 Nomality of Tuing s Fomulae 19 EN 2 ) = EN ) + j EN ) 2 = n) p 1 p ) n ] 2 n!!!n 2)! p p j 1 p p j ) n 2 = n) 2 j p p j 1 p ) n 1 p j ) n + n) 2 p2 By the fist pat of Condition 3.1,!)2 g 4 n) n 4 EN ) 0 since g 2 /n 2 0. Theefoe lim!)2 g 4 n) n 4 EN 2 ) EN ) 2] lim g4 n) n 4 = lim g4 n) n 4 j j + lim g4 n) 1 p ) 2n 2. n! n 2)! p p j 1 p p j ) n 2 n!)2 n )!] 2 j p p j 1 p ) n 1 p j ) n ] n! n 2)! p p j 1 p p j ) n 2 n! n 2)! j p p j 1 p ) n 1 p j ) n ] n! n 4 n 2)! The second tem above is bounded by lim g4 n) n! n 4 n 2)! = lim = lim ] n!)2 n )!] 2 j p p j 1 p ) n 1 p j ) n ]. n! n 2)! n! n 2)! = c ) 2! lim ] n!)2 n )!] 2 j p p j 1 p ) n 1 p j ) n ] n!)2 n )!] 2 ] n ) 2 g 2 n) n ) 2 n 2 p 1 p ) n ] 2 ] n ) ] n!)2 2 g 2 n) 2 n )!] 2 EN n 2 ) n! n 2)! ] n ) n!)2 2 n )!] 2 = 0. Noting 1 p j p ) n 2 1 p j p + p j p ) n 2 = 1 p j )1 p )] n 2, and theefoe lim!)2 g 4 n) n 4 EN 2 ) EN ) 2] lim g4 n) n! n 4 n 2)! j p p j 1 p j)1 p )] n 2 j p p j 1 p ) n 1 p j ) n ] { = lim g4 n) n! } n 4 n 2)! j p p j 1 p j)1 p )] n p ) 1 p j ) ] { lim g4 n) n! } n 4 n 2)! j p p j 1 p j)1 p )] n 2 {1 1 p + p j )] }. Noting 1 1 x) 2 1)x fo all x 0, 1], lim!)2 g 4 n) n 4 EN 2 ) EN ) 2] { lim g4 n) n!2 1) n 4 n 2)! j p p j 1 p j)1 p )] n 2 p + p j )} { = 2 lim g4 n) n!2 1) n 4 n 2)! j p+1 p j 1 p j)1 p )] n 2}.

20 Nomality of Tuing s Fomulae 20 On the othe hand, j p+1 p j 1 p ) n 2 1 p j ) n 2 = j,p p j + ) j,p >p j p +1 p j 1 p ) n 2 1 p j ) n 2 j,p p j p p+1 j 1 p ) n 1 p j ) n 3 + j,p >p j p +1 p j 1 p ) n 3 1 p j ) n 2 j p p+1 j 1 p ) n 1 p j ) n 3 2 p 1 p ) n j p+1 j 1 p j ) n 3 = 2 n) 1 EN ) j p+1 j 1 p j ) n 3. Noting that p 1 p) n 3 attains its maximum at p = /n 2) and hence p 1 p) n 3 /n 2), j p+1 p j 1 p ) n 2 1 p j ) n 2 2 n 1 ) n 2) EN ). Finally lim!)2 g 4 n) n 4 EN 2 ) EN ) 2] 4 lim g4 n) n!2 1) n 4 n 2)! n )n 2) EN ) = 4 2 1)c lim g 2 n) n 2 n )! n 2)!n 2) ] = 0. The consistency of ĉ follows. The consistency of ĉ +1 can also be similaly poved. Refeences 1] Batlett, M.S. 1938). The chaacteistic function of a conditional statistic, Jounal of the London Mathematical Society, 13, pp ] Chao, A. 1984). Nonpaametic estimation of the numbe of the classes in a population, Scand. J. Statist., 11, pp ] Chao, A. and Bunge, J. 2002). Estimating the numbe of species in a stochastic abundance model, Biometics, 58, pp ] Chao, A. and Lee, S. 1992). Estimating the numbe of classes via sample covegae, J. Ame. Statist. Assoc., 87, pp

21 Nomality of Tuing s Fomulae 21 5] Chao, A. and Shen, T.J. 2003). Nonpaametic estimation of Shannons index of divesity when thee ae unseen species in sample, Envionmental and Ecological Statistics, 10, pp ] Esty, W.W. 1983). A nomal limit law fo a nonpaametic estimato of the coveage of a andom sample, The Annals of Statist., 11, pp ] Good, I.J.1953), The population fequencies of species and the estimation of population paametes, Biometia, 40, pp ] Lijoi, A., Mena, R.H. and Pünste, I. 2007). Bayesian nonpaametic estimation of the pobability of discoveing new species, Biometia, Vol. 94, No. 4, pp ] Mao, C. X. and Lindsay, B. G. 2002). A Poisson model fo the coveage poblem with a genomic application, Biometia, 89, pp ] Patt, J.W. 1960),On intechanging limits and integals, Ann. Math. Stat. 31, pp ] Robbins, H.E. 1968), Estimating the total pobability of the unobseved outcomes of an expeiment, Annals of Mathematical Statistics, 39, pp ] Vu, V.Q., Yu, B., and Kass, R.E. 2007). Coveage-adjusted entopy estimation, Statist. Med., 26, pp ] Wang, J.P. and Lindsay, B.G. 2005). A penalized nonpaametic maximum lielihood appoach to species ichness estimation, JASA, Vol. 100, No. 471, pp ] Zhang, Z. 2012), Entopy estimation in Tuing s pespective, Neual Computation, Vol. 24, No. 5, pp ] Zhang, Z. and Huang, H. 2007), Tuing s Fomula Revisited, Jounal of Quantitative Linguistics, Vol.4, No.2, pp

22 Nomality of Tuing s Fomulae 22 16] Zhang, Z. and Huang, H. 2008), A sufficient nomality condition fo Tuing s fomula, Jounal of Nonpaametic Statistics, Vol.20, No. 5. pp ] Zhang, H. and Sten, H. 2009). Sample size calculation fo finding unseen species, Bayesian Analysis, Vol. 4, No. 4, pp ] Zhang, C.-H. and Zhang, Z. 2009), Asymptotic nomality of a nonpaametic estimato of sample coveage, The Annals of Statist., Vol. 37, No. 5A, pp ] Zhang, Z. and Zhou, J. 2010). Re-paameteization of multinomial distibutions and divesity indices, Jounal of Statistical Planning and Infeence, Vol. 140, pp