Asymptotic Normality of an Entropy Estimator with Exponentially Decaying Bias
|
|
- Veronica Wheeler
- 6 years ago
- Views:
Transcription
1 Asymptotic Normality of an Entropy Estimator with Exponentially Decaying Bias Zhiyi Zhang Department of Mathematics and Statistics Uniersity of North Carolina at Charlotte Charlotte, NC Abstract This paper establishes the asymptotic normality of an entropy estimator with an exponentially decaying bias on any finite alphabet. Furthermore it is shown that the nonparametric estimator is asymptotically efficient. Introduction. Let {p k } be a probability distribution on a finite alphabet, X {l k ; k K}, where K 2 is a finite integer. Let p X be a random ariable such that P p X p k ) p k. Entropy in the form of H E[ lnp X )] K p k lnp k ), ) was introduced by Shannon 948), and is often referred to as Shannon s entropy. Nonparametric estimation of H has been a subect of much research for many decades. Miller 955) and Basharin 959) were perhaps among the first who studied the intuitie general nonparametric estimator, Ĥ K k ˆp k lnˆp k ) where ˆp k is the sample relatie frequency of the k th letter l k, also known as the plug-in estimator. Others hae inestigated the topic in arious forms and directions oer the AMS 2000 Subect Classifications. Primary 62f0, 62F2, 62G05, 62G20; secondary 62F5. Keywords and phrases. Turing s formula, nonparametric entropy estimation, asymptotic normality. Research partially supported by NSF Grants DMS k
2 Normality of an Entropy Estimator 2 years. Many important references can be found in Antos and Kontoyiannis 200) and Paninski 2003). Among many difficult issues of nonparametric entropy estimation, much research effort in the literature seems to be placed on reducing the bias of the estimators. The main reference point of such discussion is the On ) decaying bias of the plug-in Ĥ whose form may be found in Harris 975). Many bias-adusted nonparametric estimators hae been proposed. All of them hae been shown to reduce bias in certain numerical studies. Howeer the rates of bias decay for most of the bias-adusted estimators are largely unknown, and there is no clear theoretical eidence why any of these proposed estimators should improe the bias decay to a rate faster than On ). Zhang 202) proposed an estimator Ĥz, as gien in 2) below, and showed that the associated bias decays at a rate no slower than On p 0 ) n ) where p 0 min{p k > 0; k,, K}. In addition, Zhang 202) established that a uniform ariance upper bound for the entire class of distributions with finite entropy that decays at a rate of Olnn)/n) compared to O[lnn)] 2 /n) for the plug-in, that in a wide range of subclasses, the ariance of the proposed estimator conerges at a rate of O/n), and that the aforementioned rate of conergence carries oer to the conergence rates in mean squared errors in many subclasses. The computational performances of Ĥ z, and of its ariants, were compared faorably with seeral other commonly known estimators, such as the ackknife estimator by Zahl 977) and Strong, Koberle, de Ruyter an Steeninck and Bialek 998), and the NSB estimator by Nemenman, Shafee and Bialek 2002). Let {y k } be the sequence of obsered counts of letters in the alphabet in an independently and identically distributed iid) sample of size n and {ˆp k y k /n}. The general nonparametric estimator of entropy proposed by Zhang 202) is n n + [n + )]! K Ĥ z n! k ˆp k n) ˆp k. 2) This paper establishes two normal laws of Ĥz as stated in Theorem and Corollary below, and the asymptotic efficiency of Ĥz is gien in Theorem 2. Let H 2) E[ lnp X )] 2 K k p k ln 2 p k ).
3 Normality of an Entropy Estimator 3 Theorem. Let {p k ; k K} be a non-uniform probability distribution on a finite alphabet X and Ĥz be as in 2). Then n Ĥz H) L N0, σ 2 ) where σ 2 V ar [ lnp X )] H 2) H 2. Let Ĥ 2) z n { i ) { n + [n + )]! i i) n! [ K k ˆp k m0 ˆp k m n ) ]}}. 3) Corollary. Let {p k ; k K} be a non-uniform probability distribution on a finite alphabet, Ĥ z be as in 2), and Ĥ2) z be as in 3). Then n Ĥz H Ĥ z 2) Ĥ2 z L N0, ). Theorem 2. Let {p k ; k K} be a non-uniform probability distribution on a finite alphabet X. Then Ĥz is asymptotically efficient. 2 Proofs Ĥ z in 2) may be re-expressed as K n Ĥ z ˆp { n + } [n + )]! n) k ˆp k n! k def K ˆp k ĝ k,n. 4) Of first interest is an asymptotic normal law of ˆp k ĝ k,n. For simplicity, consider first a binomial distribution with parameters n and p 0, ), and functions n { n + } [n + )]! n) g n p) p n! [ n p)+], and h n p) pg n p). Let hp) p lnp). Lemma below is easily proed by induction. k
4 Normality of an Entropy Estimator 4 Lemma. Let a,,, n, be complex numbers satisfying a for eery. n a n a. Then Lemma 2. Let ˆp X/n where X is a binomial random ariable with parameters n and p.. n[h n p) hp)] 0 uniformly in p c, ) for any c, 0 < c <. 2. n h n p) hp) < An) On 3/2 ) uniformly in p [/n, c] for any c, 0 < c < p. 3. P ˆp c) < Bn) On /2 exp{ nc}) where C p c)2 for any c 0, p). Proof of Part. As the notation in g n p) suggests, the range for is from to min{n, n p) + }. For any in that range, let W n,+ n+ [n +)]! n!. Noting 0 /[n p)] /n subect to n p), by Lemma, W n,+ ) p n p) p) ) ) p) n n p) n n ) ) p) + n p) n n ) ) p) n + p) n p) n ) p) n + p) ) p) n + p) p p ) p) n + p) p) n n p) n n n p) 2 n. ) ) n p) + n
5 Normality of an Entropy Estimator 5 For a sufficiently large n, let V n n /8. n hn p) hp) np n p)+ + np n p)+2 p) np V n + np n p)+ V n+ W n,+ W n,+ ) p n p) W n,+ + np n p)+2 p) def np V n n p) n5/8 0. n n /8 2 p n n p)+ V n+ ) p n p) ) p n p) n p) p n[n p)+] np n p)+ V n+ p) n[n p)+] np p) n/8 n[n c)+] nc c) n/ n n p) p) n p)+2 n p) n p)+ n 0. Hence sup p c,) n hn p) hp) 0. Proof of Part 2. The proof is identical to that of Part aboe until the expression where each term is to be ealuated on the interal [/n, c]. It is clear that On 3/8 ). For 2, since n p) + at p /n is n > n, we hae 2 p n min{n, n p)+ } V n+ n p) p n min{n, n /n)+ } V n+ n p) Therefore On 3/2 ). p n n V n + n p) < p nn ) n V n+ p) < nn ) p) V n < nn ) On 3/2 ). 3 p n min{n, n p)+ }+ p) < p n p) < n On /2 ).
6 Normality of an Entropy Estimator 6 Proof of Part 3. Let Z and ϕz) be a standard normal random ariable and its density function respectiely, and let denote asymptotic equality. Since nˆp p) L N0, p p)), P ˆp c) nc p)/ ϕz)dz np c)/ ϕz)dz < np c)/ e z [ np c)/ ]dz np c) [np c)/ ] 2 e x dx { [ np ] } 2 np c) exp c)/ p p) n /2 p c) exp } { np c)2. Proof of Theorem. Without loss of generality, consider the sample proportions of the first two letters of the alphabet ˆp and ˆp 2 in an iid sample of size n. n ˆp p, ˆp 2 p 2 ) L N0, Σ), where Σ σ i ), i,, 2, σ ii p i p i ) and σ i p i p when i. Write n {[hn ˆp ) + h n ˆp 2 )] [ p lnp ) p 2 lnp 2 )]} n {[h n ˆp ) + h n ˆp 2 )] [hˆp ) + hˆp 2 )]} + n {[hˆp ) + hˆp 2 )] [ p lnp ) p 2 lnp 2 )]} n [h n ˆp ) hˆp )] + n [h n ˆp 2 ) hˆp 2 )] + n {[hˆp ) + hˆp 2 )] [ p lnp ) p 2 lnp 2 )]} n [h n ˆp ) hˆp )] [ˆp p /2] + n [h n ˆp 2 ) hˆp 2 )] [ˆp2 p 2 /2] + n [h n ˆp ) hˆp )] [ˆp >p /2] + n [h n ˆp 2 ) hˆp 2 )] [ˆp2 >p 2 /2] + n {[hˆp ) + hˆp 2 )] [ p lnp ) p 2 lnp 2 )]}. The third and fourth terms aboe conerge to zero almost surely by Part of Lemma 2. The last
7 Normality of an Entropy Estimator 7 term, by the delta method, conerges in law to N0, τ 2 ) where after a few algebraic steps τ 2 [lnp ) + ] 2 p p ) + [lnp 2 ) + ] 2 p 2 p 2 ) 2[lnp ) + ][lnp 2 ) + ]p p 2 [lnp ) + ] 2 p + [lnp 2 ) + ] 2 p 2 {[lnp ) + ]p + [lnp ) + ]p } 2. It remains to show that the first term the second term will admit the same argument) conerges to zero in probability. Howeer this fact can be established by the following argument. By Part 2 and then Part 3 of Lemma 2, E{ n h n ˆp ) hˆp ) [ˆp p /2]} An)P ˆp p /2) An)Bn) On 3/2 )On /2 exp{ nc}) 0 for some positie constant C. This fact, noting that n h n ˆp ) hˆp ) 0, gies immediately the desired conergence in probability, that is, n h n ˆp ) hˆp ) [ˆp p /2] the desired weak conergence for n {[h n ˆp ) + h n ˆp 2 )] [ p lnp ) p 2 lnp 2 )]}. By generalization for K terms, nĥz H) ariable that assumes the alue p k when X assumes l k, P 0. In turn, it gies L N0, σ 2 ) where, letting p X denote the random σ 2 K k { [lnp k) + ]} 2 p k { K k { [lnp k) + ]}p k } 2 V ar [ lnp X ) ] V ar [ lnp X )]. Remark. It may be interesting to note that the asymptotic ariance of nĥz H)is identical to that of nĥ H) where Ĥ is the plug-in. Remark 2. When {p k } is a uniform distribution, lnp X ) is constant, V ar [ lnp X )] 0 and therefore nĥz H) asymptotically degenerates. Let ζ, k p k p k ), C i and therefore Ĥ2) z Z, n+ [n +)]! n! n C Z,. i i) for 2 and define C 0), K )] k [ˆp k ˆp k n,
8 Normality of an Entropy Estimator 8 For clarity in proing Corollary, a few notations and two well-known lemmas in U-statistics are first gien. For each i, i n, let X i be a random ariable such that X i l k indicates the eent that the k th letter of the alphabet is obsered and P X i l k ) p k. Let X,, X n be an iid sample, and denote x,, x n as the corresponding sample realization. A U-statistic is an n-ariable function obtained by aeraging the alues of an m-ariable function kernel of degree m, often denoted by ψ) oer all n!/[m!n m)!] possible subsets of m ariables from the set of n ariables. Interested readers may refer to Lee 990) for an introduction. Turing s formula, also known as the Good-Turing estimator, is a nonparametric estimator introduced by Good 953), but largely credited to Alan Turing, as a means of estimate the total probability associated with letters in the alphabet that are not represented in a random sample. In Zhang & Zhou 200), it is shown that Z, is a U-statistic with kernel ψ being Turing s formula with degree m +. Let ψ c x,, x c ) E[ψx,, x c, X c+,, X m )] and σc 2 V ar[ψ c X,, X c )]. Lemmas 3 and 4 below are due to Hoeffding 948). Lemma 3. Let U n be a U-statistic with kernel ψ of degree m. V aru n ) n m ) m c m c ) n m m c ) σ 2 c. Lemma 4. Let U n be a U-statistic with kernel ψ of degree m. For 0 c d m, σ 2 c /c σ 2 d /d. Lemma 5. V arz, ) n ζ, + + n ζ2,. Proof. Let m +. By Lemmas 3, 4, and identity n m V arz, ) ) n m c m c m c ) n m m c ) m c c m c ) n m m c ) m 2 n, ) σ 2 m/m m n σ2 m. 5) Consider σ 2 m V ar[ψx,, X m )] E[ψX,, X m )] 2 [ K k p k p k ) m )] 2. Let y m) k
9 Normality of an Entropy Estimator 9 denote the frequency of the k th letter in the sample of size m. [ σm 2 E[ψX,, X m )] 2 K E m 2 k K )] [y k ]) k [y k ] K E m 2 k [y k ] + 2 ) k<k K [y k ] [yk ] m K k p k p k ) m + 2m ) m k<k K p kp k p k p k ) m 2 m K k p k p k ) m + 2 k<k K p kp k p k p k + p k p k ) m 2 K m k p k p k ) m + 2 [ k<k K pk p k ) m 2 p k p k ) m 2] m K k p k p k ) m + [ K k p k p k ) m 2 ] 2 m ζ,m + ζ 2,m 2. By 5), V arz, ) n ζ, + + n ζ2,. Proof of Corollary. By Zhang & Zhou 200), EZ, ) K k p k p k ) ζ,, and therefore EĤ2) z ) n C K k p k p k ) K k p n k C p k ) K k p k[ lnp k )] 2 K k p k ln 2 p k ). It only remains to show V arĥ2) z ) 0. V arĥ2) z Note C i ) n n [ n C V arz, )] 2. i i) i w C C w coz,, Z,w ) n, ζ, ζ,, ζ 2, ζ,, n w C C w V arz, )V arz,w ) ζ, K k p k p k ) K k p k p 0 ) p 0 ) where p 0 min{p k > 0; k,, K}, and therefore, from Lemma 5 for 2, V arz, ) n + 2)ζ, 2 /2 n p 0 ) )/2.
10 Normality of an Entropy Estimator 0 As n, n C V arz, ) 2 n n /2 p 0 ) 2 n n /4 /2 p 0 ) + 2 n n n /4 + /2 p 0 ) 2 n n /4 n /4) /2 + 2 n n /2 p 0 ) n /4 p 0 2n /8 + 2 p 0 ) n /4 p 0 0, and V arĥ2) z ) 0 follows. Hence Ĥ2) z p H 2). The fact of Ĥz p H is implied by Theorem. Finally the corollary follows Slutsky s Theorem. Proof of Theorem 2. First consider the plug-in estimator Ĥ. It can be erified that nĥ H) N0, σ 2 ) where σ 2 σ 2 {p k }) is as in Theorem. We want to show first that Ĥ is asymptotically efficient in two separate cases: ) when K is known and 2) when K is unknown. If K is known, then the underlying model {p k ; k K} is a K )-parameter multinomial distribution and therefore Ĥ is the maximum likelihood estimator of H which implies that it is asymptotically efficient. Since the estimator Ĥ takes the same alue, gien a sample, regardless whether K is known or not, its asymptotic ariance is the same whether K is known or not. Therefore Ĥ must be asymptotically efficient when K is finite but unknown, or else, it would contradict the fact that Ĥ is asymptotically efficient when K is known. The asymptotic efficiency of Ĥz follows from the fact that nĥz H) and nĥ H) hae identical limiting distribution. References [] Antos, A. and Kontoyiannis, I. 200). Conergence properties of functional estimates for discrete distributions, Random Structures & Algorithms, Vol. 9, pp [2] Basharin, G. 959). On a statistical estimate for the entropy of a sequence of independent random ariables, Theory of Probability and Its Applications, 4, pp
11 Normality of an Entropy Estimator [3] Good, I.J. 953). The population frequencies of species and the estimation of population parameters. Biometrika, 40, pp [4] Harris, B. 975). The statistical estimation of entropy in the non-parametric case, Topics in Information Theory, edited by I. Csiszar, Amsterdam: North-Holland, pp [5] Hoeffding, W. 948). A class of statistics with asymptotically normal distribution, Annals of Mathematical Statistics, Vol. 9, No. 3, pp [6] Lee, A.J. 990). U-Statistics: Theory and Practice, Marcel Dekker, Inc. New York. [7] Miller, G. 955). Note on the bias of information estimates, Information theory in psychology II-B, ed. H. Quastler, Glencoe, IL: Free Press, pp [8] Nemenman, I., Shafee, F. & Bialek, W. 2002). Entropy and inference, reisited. Adances in Neural Information Processing Systems 4, Cambridge, MA, MIT Press. [9] Paninski, L. 2003). Estimation of entropy and mutual information, Neural Comp. 5, pp [0] Shannon, C.E. 948). A Mathematical Theory of Communication, Bell Syst. Tech. J., 27, pp , and pp [] Strong, S.P., Koberle, R., de Ruyter an Steeninck, R.R., & Bialek, W. 998). Entropy and information in neural spike trains. Physical Reiew Letters, 80 ), pp [2] Zahl, S. 977). Jackknifing an index of diersity, Ecology, 58: pp [3] Zhang, Z. 202). Entropy estimation in Turing s perspectie, Neural Computation, Vol. 24, No. 5, pp [4] Zhang, Z. and Zhou, J. 200). Re-parameterization of multinomial distribution and diersity indices, J. of Statistical Planning and Inference, Vol. 40, No. 7, pp
Entropy Estimation in Turing s Perspective
Entropy Estimation in Turing s Perspective Zhiyi Zhang Department of Mathematics and Statistics University of North Carolina at Charlotte Charlotte, NC 28223 Abstract A new nonparametric estimator of Shannon
More informationEntropy and information estimation: An overview
Entropy and information estimation: An overview Ilya Nemenman December 12, 2003 Ilya Nemenman, NIPS 03 Entropy Estimation Workshop, December 12, 2003 1 Workshop schedule: Morning 7:30 7:55 Ilya Nemenman,
More informationEstimation of information-theoretic quantities
Estimation of information-theoretic quantities Liam Paninski Gatsby Computational Neuroscience Unit University College London http://www.gatsby.ucl.ac.uk/ liam liam@gatsby.ucl.ac.uk November 16, 2004 Some
More informationA nonparametric confidence interval for At-Risk-of-Poverty-Rate: an example of application
A nonparametric confidence interal for At-Ris-of-Poerty-Rate: an example of application Wojciech Zielińsi Department of Econometrics and Statistics Warsaw Uniersity of Life Sciences Nowoursynowsa 159,
More informationSUPPLEMENTARY MATERIAL. Authors: Alan A. Stocker (1) and Eero P. Simoncelli (2)
SUPPLEMENTARY MATERIAL Authors: Alan A. Stocker () and Eero P. Simoncelli () Affiliations: () Dept. of Psychology, Uniersity of Pennsylania 34 Walnut Street 33C Philadelphia, PA 94-68 U.S.A. () Howard
More informationOn diamond-free subposets of the Boolean lattice
On diamond-free subposets of the Boolean lattice Lucas Kramer, Ryan R Martin, Michael Young 2 Department of Mathematics, Iowa State Uniersity, Ames, Iowa 500 Abstract The Boolean lattice of dimension two,
More informationCHARACTERIZATIONS OF THE PARETO DISTRIBUTION BY THE INDEPENDENCE OF RECORD VALUES. Se-Kyung Chang* 1. Introduction
JOURNAL OF THE CHUNGCHEONG MATHEMATICAL SOCIETY Volume 20, No., March 2007 CHARACTERIZATIONS OF THE PARETO DISTRIBUTION BY THE INDEPENDENCE OF RECORD VALUES Se-Kyung Chang* Abstract. In this paper, we
More informationarxiv: v1 [stat.ml] 15 Feb 2018
1 : A New Algorithm for Streaming PCA arxi:1802.054471 [stat.ml] 15 Feb 2018 Puyudi Yang, Cho-Jui Hsieh, Jane-Ling Wang Uniersity of California, Dais pydyang, chohsieh, janelwang@ucdais.edu Abstract In
More informationAn Explicit Lower Bound of 5n o(n) for Boolean Circuits
An Eplicit Lower Bound of 5n o(n) for Boolean Circuits Kazuo Iwama, Oded Lachish, Hiroki Morizumi, and Ran Raz Graduate School of Informatics, Kyoto Uniersity, Kyoto, JAPAN {iwama, morizumi}@kuis.kyoto-u.ac.jp
More informationNOTES ON THE REGULAR E-OPTIMAL SPRING BALANCE WEIGHING DESIGNS WITH CORRE- LATED ERRORS
REVSTAT Statistical Journal Volume 3, Number 2, June 205, 9 29 NOTES ON THE REGULAR E-OPTIMAL SPRING BALANCE WEIGHING DESIGNS WITH CORRE- LATED ERRORS Authors: Bronis law Ceranka Department of Mathematical
More informationTHE MINIMUM MATCHING ENERGY OF BICYCLIC GRAPHS WITH GIVEN GIRTH
ROCKY MOUNTAIN JOURNAL OF MATHEMATICS Volume 46, Number 4, 2016 THE MINIMUM MATCHING ENERGY OF BICYCLIC GRAPHS WITH GIVEN GIRTH HONG-HAI LI AND LI ZOU ABSTRACT. The matching energy of a graph was introduced
More informationInformation Measure Estimation and Applications: Boosting the Effective Sample Size from n to n ln n
Information Measure Estimation and Applications: Boosting the Effective Sample Size from n to n ln n Jiantao Jiao (Stanford EE) Joint work with: Kartik Venkat Yanjun Han Tsachy Weissman Stanford EE Tsinghua
More informationBlow up of Solutions for a System of Nonlinear Higher-order Kirchhoff-type Equations
Mathematics Statistics 6: 9-9, 04 DOI: 0.389/ms.04.00604 http://www.hrpub.org Blow up of Solutions for a System of Nonlinear Higher-order Kirchhoff-type Equations Erhan Pişkin Dicle Uniersity, Department
More informationTHE FIFTH DIMENSION EQUATIONS
JP Journal of Mathematical Sciences Volume 7 Issues 1 & 013 Pages 41-46 013 Ishaan Publishing House This paper is aailable online at http://www.iphsci.com THE FIFTH DIMENSION EQUATIONS Niittytie 1B16 03100
More informationHolomorphy of the 9th Symmetric Power L-Functions for Re(s) >1. Henry H. Kim and Freydoon Shahidi
IMRN International Mathematics Research Notices Volume 2006, Article ID 59326, Pages 1 7 Holomorphy of the 9th Symmetric Power L-Functions for Res >1 Henry H. Kim and Freydoon Shahidi We proe the holomorphy
More informationAstrometric Errors Correlated Strongly Across Multiple SIRTF Images
Astrometric Errors Correlated Strongly Across Multiple SIRTF Images John Fowler 28 March 23 The possibility exists that after pointing transfer has been performed for each BCD (i.e. a calibrated image
More informationBayesian estimation of discrete entropy with mixtures of stick-breaking priors
Bayesian estimation of discrete entropy with mixtures of stick-breaking priors Evan Archer 24, Il Memming Park 234, & Jonathan W. Pillow 234. Institute for Computational and Engineering Sciences 2. Center
More informationINTEGERS WITH A DIVISOR IN (y, 2y]
INTEGERS WITH A DIVISOR IN y, 2y] KEVIN FORD Abstract We determine, up to multiplicatie constants, how many integers n x hae a diisor in y, 2y] Introduction Let Hx, y, z) be the number of integers n x
More informationLecture J. 10 Counting subgraphs Kirchhoff s Matrix-Tree Theorem.
Lecture J jacques@ucsd.edu Notation: Let [n] [n] := [n] 2. A weighted digraph is a function W : [n] 2 R. An arborescence is, loosely speaking, a digraph which consists in orienting eery edge of a rooted
More informationFu Yuhua 1. Beijing, China
85 An Example of Guiding Scientific Research with hilosophical rinciples Based on Uniqueness of Truth and Neutrosophy eriing Newton's Second Law and the like Fu Yuhua 1 1 CNOOC Research Institute Beijing,
More informationNEW FRONTIERS IN APPLIED PROBABILITY
J. Appl. Prob. Spec. Vol. 48A, 183 194 (2011) Applied Probability Trust 2011 NEW FRONTIERS IN APPLIED PROBABILITY A Festschrift for SØREN ASMUSSEN Edited by P. GLYNN, T. MIKOSCH and T. ROLSKI Part 4. Simulation
More informationAn Alternative Characterization of Hidden Regular Variation in Joint Tail Modeling
An Alternatie Characterization of Hidden Regular Variation in Joint Tail Modeling Grant B. Weller Daniel S. Cooley Department of Statistics, Colorado State Uniersity, Fort Collins, CO USA May 22, 212 Abstract
More informationHölder-type inequalities and their applications to concentration and correlation bounds
Hölder-type inequalities and their applications to concentration and correlation bounds Christos Pelekis * Jan Ramon Yuyi Wang Noember 29, 2016 Abstract Let Y, V, be real-alued random ariables haing a
More informationSparse Functional Regression
Sparse Functional Regression Junier B. Olia, Barnabás Póczos, Aarti Singh, Jeff Schneider, Timothy Verstynen Machine Learning Department Robotics Institute Psychology Department Carnegie Mellon Uniersity
More informationOPTIMAL RESOLVABLE DESIGNS WITH MINIMUM PV ABERRATION
Statistica Sinica 0 (010), 715-73 OPTIMAL RESOLVABLE DESIGNS WITH MINIMUM PV ABERRATION J. P. Morgan Virginia Tech Abstract: Amongst resolable incomplete block designs, affine resolable designs are optimal
More informationA Regularization Framework for Learning from Graph Data
A Regularization Framework for Learning from Graph Data Dengyong Zhou Max Planck Institute for Biological Cybernetics Spemannstr. 38, 7076 Tuebingen, Germany Bernhard Schölkopf Max Planck Institute for
More informationOBSERVATIONS ON BAGGING
OBSERVATIONS ON BAGGING Andreas Buja and Werner Stuetzle Uniersity of Pennsylania and Uniersity of Washington Abstract: Bagging is a deice intended for reducing the prediction error of learning algorithms.
More informationInsights into Cross-validation
Noname manuscript No. (will be inserted by the editor) Insights into Cross-alidation Amit Dhurandhar Alin Dobra Receied: date / Accepted: date Abstract Cross-alidation is one of the most widely used techniques,
More informationEstimation of Efficiency with the Stochastic Frontier Cost. Function and Heteroscedasticity: A Monte Carlo Study
Estimation of Efficiency ith the Stochastic Frontier Cost Function and Heteroscedasticity: A Monte Carlo Study By Taeyoon Kim Graduate Student Oklahoma State Uniersity Department of Agricultural Economics
More informationTowards Universal Cover Decoding
International Symposium on Information Theory and its Applications, ISITA2008 Auckland, New Zealand, 7-10, December, 2008 Towards Uniersal Coer Decoding Nathan Axig, Deanna Dreher, Katherine Morrison,
More informationPatterns of Non-Simple Continued Fractions
Patterns of Non-Simple Continued Fractions Jesse Schmieg A final report written for the Uniersity of Minnesota Undergraduate Research Opportunities Program Adisor: Professor John Greene March 01 Contents
More informationCollective Risk Models with Dependence Uncertainty
Collectie Risk Models with Dependence Uncertainty Haiyan Liu and Ruodu Wang ebruary 27, 2017 Abstract We bring the recently deeloped framework of dependence uncertainty into collectie risk models, one
More informationOptimized Concatenated LDPC Codes for Joint Source-Channel Coding
Optimized Concatenated LDPC Codes for Joint Source-Channel Coding Maria Fresia, Fernando Pérez-Cruz, H. Vincent Poor Department of Electrical Engineering, Princeton Uniersity, Princeton, New Jersey 08544
More informationRepresentational Power of Restricted Boltzmann Machines and Deep Belief Networks
1 Representational Power of Restricted Boltzmann Machines and Deep Belief Networks Nicolas Le Roux and Yoshua Bengio Dept. IRO, Uniersité de Montréal C.P. 6128, Montreal, Qc, H3C 3J7, Canada {lerouxni,bengioy}@iro.umontreal.ca
More informationarxiv: v1 [math.co] 25 Apr 2016
On the zone complexity of a ertex Shira Zerbib arxi:604.07268 [math.co] 25 Apr 206 April 26, 206 Abstract Let L be a set of n lines in the real projectie plane in general position. We show that there exists
More informationVISUAL PHYSICS ONLINE RECTLINEAR MOTION: UNIFORM ACCELERATION
VISUAL PHYSICS ONLINE RECTLINEAR MOTION: UNIFORM ACCELERATION Predict Obsere Explain Exercise 1 Take an A4 sheet of paper and a heay object (cricket ball, basketball, brick, book, etc). Predict what will
More informationSymmetric Functions and Difference Equations with Asymptotically Period-two Solutions
International Journal of Difference Equations ISSN 0973-532, Volume 4, Number, pp. 43 48 (2009) http://campus.mst.edu/ijde Symmetric Functions Difference Equations with Asymptotically Period-two Solutions
More informationOptimization Problems in Multiple Subtree Graphs
Optimization Problems in Multiple Subtree Graphs Danny Hermelin Dror Rawitz February 28, 2010 Abstract We study arious optimization problems in t-subtree graphs, the intersection graphs of t- subtrees,
More informationSurface Matching Degree
Australian Journal of Basic and Applied Sciences, 5(9): 653-660, 0 ISSN 99-878 Surface Matching Degree Hosein Alizadeh, Hamid Parin, 3 Hamid Alinejad-Rokny, 4 Naser Mozayani Computer Engineering Department,
More informationAlgorithms and Data Structures 2014 Exercises and Solutions Week 14
lgorithms and ata tructures 0 xercises and s Week Linear programming. onsider the following linear program. maximize x y 0 x + y 7 x x 0 y 0 x + 3y Plot the feasible region and identify the optimal solution.
More information(x,y) 4. Calculus I: Differentiation
4. Calculus I: Differentiation 4. The eriatie of a function Suppose we are gien a cure with a point lying on it. If the cure is smooth at then we can fin a unique tangent to the cure at : If the tangent
More informationG 2(X) X SOLVABILITY OF GRAPH INEQUALITIES. 1. A simple graph inequality. Consider the diagram in Figure 1.
SOLVABILITY OF GRAPH INEQUALITIES MARCUS SCHAEFER AND DANIEL ŠTEFANKOVIČ Abstract. We inestigate a new type of graph inequality (in the tradition of Cetkoić and Simić [Contributions to Graph Theory and
More informationSmall area estimation under a two-part random effects model with application to estimation of literacy in developing countries
Surey Methodology, December 008 35 Vol. 34, No., pp. 35-49 Small area estimation under a two-part random effects model with application to estimation of literacy in deeloping countries Danny Pfeffermann,
More informationarxiv: v1 [physics.comp-ph] 17 Jan 2014
An efficient method for soling a correlated multi-item inentory system Chang-Yong Lee and Dongu Lee The Department of Industrial & Systems Engineering, Kongu National Uniersity, Kongu 34-70 South Korea
More informationTHE CAUCHY PROBLEM FOR ONE-DIMENSIONAL FLOW OF A COMPRESSIBLE VISCOUS FLUID: STABILIZATION OF THE SOLUTION
GLASNIK MATEMATIČKI Vol. 4666, 5 3 THE CAUCHY POBLEM FO ONE-DIMENSIONAL FLOW OF A COMPESSIBLE VISCOUS FLUID: STABILIZATION OF THE SOLUTION Nermina Mujakoić and Ian Dražić Uniersity of ijeka, Croatia Abstract.
More informationLESSON 4: INTEGRATION BY PARTS (I) MATH FALL 2018
LESSON 4: INTEGRATION BY PARTS (I) MATH 6 FALL 8 ELLEN WELD. Integration by Parts We introduce another method for ealuating integrals called integration by parts. The key is the following : () u d = u
More informationAn Optimal Split-Plot Design for Performing a Mixture-Process Experiment
Science Journal of Applied Mathematics and Statistics 217; 5(1): 15-23 http://www.sciencepublishinggroup.com/j/sjams doi: 1.11648/j.sjams.21751.13 ISSN: 2376-9491 (Print); ISSN: 2376-9513 (Online) An Optimal
More informationEffect of stray capacitances on single electron tunneling in a turnstile
Effect of stray capacitances on single electron tunneling in a turnstile Young Bong Kang Department of Physics, Cheju National Uniersity, Cheju 690-756, Korea G. Y. Hu and R. F. O Connell a) Department
More informationInformation Theory. Mark van Rossum. January 24, School of Informatics, University of Edinburgh 1 / 35
1 / 35 Information Theory Mark van Rossum School of Informatics, University of Edinburgh January 24, 2018 0 Version: January 24, 2018 Why information theory 2 / 35 Understanding the neural code. Encoding
More informationModeling Highway Traffic Volumes
Modeling Highway Traffic Volumes Tomáš Šingliar1 and Miloš Hauskrecht 1 Computer Science Dept, Uniersity of Pittsburgh, Pittsburgh, PA 15260 {tomas, milos}@cs.pitt.edu Abstract. Most traffic management
More informationRegularity of the density for the stochastic heat equation
Regularity of the density for the stochastic heat equation Carl Mueller 1 Department of Mathematics University of Rochester Rochester, NY 15627 USA email: cmlr@math.rochester.edu David Nualart 2 Department
More informationOn the Linear Threshold Model for Diffusion of Innovations in Multiplex Social Networks
On the Linear Threshold Model for Diffusion of Innoations in Multiplex Social Networks Yaofeng Desmond Zhong 1, Vaibha Sriastaa 2 and Naomi Ehrich Leonard 1 Abstract Diffusion of innoations in social networks
More informationComplex Systems Methods 3. Statistical complexity of temporal sequences
Complex Systems Methods 3. Statistical complexity of temporal sequences Eckehard Olbrich MPI MiS Leipzig Potsdam WS 2007/08 Olbrich (Leipzig) 02.11.2007 1 / 24 Overview 1 Summary Kolmogorov sufficient
More informationDynamics of a Bertrand Game with Heterogeneous Players and Different Delay Structures. 1 Introduction
ISSN 749-889 (print), 749-897 (online) International Journal of Nonlinear Science Vol.5(8) No., pp.9-8 Dynamics of a Bertrand Game with Heterogeneous Players and Different Delay Structures Huanhuan Liu,
More informationA spectral Turán theorem
A spectral Turán theorem Fan Chung Abstract If all nonzero eigenalues of the (normalized) Laplacian of a graph G are close to, then G is t-turán in the sense that any subgraph of G containing no K t+ contains
More informationAssignment 4 (Solutions) NPTEL MOOC (Bayesian/ MMSE Estimation for MIMO/OFDM Wireless Communications)
Assignment 4 Solutions NPTEL MOOC Bayesian/ MMSE Estimation for MIMO/OFDM Wireless Communications The system model can be written as, y hx + The MSE of the MMSE estimate ĥ of the aboe mentioned system
More informationOn computing Gaussian curvature of some well known distribution
Theoretical Mathematics & Applications, ol.3, no.4, 03, 85-04 ISSN: 79-9687 (print), 79-9709 (online) Scienpress Ltd, 03 On computing Gaussian curature of some well known distribution William W.S. Chen
More informationAsymptotic Nonequivalence of Nonparametric Experiments When the Smoothness Index is ½
University of Pennsylvania ScholarlyCommons Statistics Papers Wharton Faculty Research 1998 Asymptotic Nonequivalence of Nonparametric Experiments When the Smoothness Index is ½ Lawrence D. Brown University
More informationPREPERIODIC POINTS OF POLYNOMIALS OVER GLOBAL FIELDS
PREPERIODIC POITS OF POLYOMIALS OVER GLOBAL FIELDS ROBERT L. BEEDETTO Abstract. Gien a global field K and a polynomial φ defined oer K of degree at least two, Morton and Silerman conjectured in 1994 that
More informationBalanced Partitions of Vector Sequences
Balanced Partitions of Vector Sequences Imre Bárány Benjamin Doerr December 20, 2004 Abstract Let d,r N and be any norm on R d. Let B denote the unit ball with respect to this norm. We show that any sequence
More informationGaussians. Andrew W. Moore Professor School of Computer Science Carnegie Mellon University.
Note to other teachers and users of these slides. Andrew would be delighted if you found this source aterial useful in giing your own lectures. Feel free to use these slides erbati, or to odify the to
More informationComputing Laboratory A GAME-BASED ABSTRACTION-REFINEMENT FRAMEWORK FOR MARKOV DECISION PROCESSES
Computing Laboratory A GAME-BASED ABSTRACTION-REFINEMENT FRAMEWORK FOR MARKOV DECISION PROCESSES Mark Kattenbelt Marta Kwiatkowska Gethin Norman Daid Parker CL-RR-08-06 Oxford Uniersity Computing Laboratory
More informationOPTIMIZATION OF FLOWS AND ANALYSIS OF EQUILIBRIA IN TELECOMMUNICATION NETWORKS
OPTIMIZATION OF FLOWS AND ANALYSIS OF EQUILIBRIA IN TELECOMMUNICATION NETWORKS Ciro D Apice (a, Rosanna Manzo (b, Luigi Rarità (c (a, (b, (c Department of Information Engineering and Applied Mathematics,
More informationGeneralization to Unseen Cases
In Y. Weiss, B. Schölkopf, and J. Platt (Eds.), Advances in Neural Information Processing Systems 18 (NIPS-05), pp. 1129 1136, MIT Press, Cambridge, MA, 2006 Generalization to Unseen Cases Teemu Roos Helsinki
More informationOn general error distributions
ProbStat Forum, Volume 6, October 3, Pages 89 95 ISSN 974-335 ProbStat Forum is an e-journal. For details please isit www.probstat.org.in On general error distributions R. Vasudea, J. Vasantha Kumari Department
More informationOptimal Joint Detection and Estimation in Linear Models
Optimal Joint Detection and Estimation in Linear Models Jianshu Chen, Yue Zhao, Andrea Goldsmith, and H. Vincent Poor Abstract The problem of optimal joint detection and estimation in linear models with
More informationNoise constrained least mean absolute third algorithm
Noise constrained least mean absolute third algorithm Sihai GUAN 1 Zhi LI 1 Abstract: he learning speed of an adaptie algorithm can be improed by properly constraining the cost function of the adaptie
More informationReversal in time order of interactive events: Collision of inclined rods
Reersal in time order of interactie eents: Collision of inclined rods Published in The European Journal of Physics Eur. J. Phys. 27 819-824 http://www.iop.org/ej/abstract/0143-0807/27/4/013 Chandru Iyer
More informationDynamic Vehicle Routing with Moving Demands Part II: High speed demands or low arrival rates
ACC 9, Submitted St. Louis, MO Dynamic Vehicle Routing with Moing Demands Part II: High speed demands or low arrial rates Stephen L. Smith Shaunak D. Bopardikar Francesco Bullo João P. Hespanha Abstract
More informationA noncommutative extension of Mahler s theorem on interpolation series 1
A noncommutatie etension of Mahler s theorem on interpolation series 1 Jean-Éric Pin2, Pedro V. Sila 3 Tuesday 20 th December, 2011 Abstract In this paper, we proe an etension of Mahler s theorem on interpolation
More informationSPACE-TIME HOLOMORPHIC TIME-PERIODIC SOLUTIONS OF NAVIER-STOKES EQUATIONS. 1. Introduction We study Navier-Stokes equations in Lagrangean coordinates
Electronic Journal of Differential Equations, Vol. 2013 2013, No. 218, pp. 1 5. ISSN: 1072-6691. URL: http://ejde.math.txstate.edu or http://ejde.math.unt.edu ftp ejde.math.txstate.edu SPACE-TIME HOLOMORPHIC
More informationDensity Estimation: ML, MAP, Bayesian estimation
Density Estimation: ML, MAP, Bayesian estimation CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Introduction Maximum-Likelihood Estimation Maximum
More informationRelativistic Energy Derivation
Relatiistic Energy Deriation Flamenco Chuck Keyser //4 ass Deriation (The ass Creation Equation ρ, ρ as the initial condition, C the mass creation rate, T the time, ρ a density. Let V be a second mass
More informationA New Extended Uniform Distribution
International Journal of Statistical Distriutions and Applications 206; 2(3): 3-4 http://wwwsciencepulishinggroupcom/j/ijsda doi: 0648/jijsd20602032 ISS: 2472-3487 (Print); ISS: 2472-309 (Online) A ew
More informationLecture 1. 1 Overview. 2 Maximum Flow. COMPSCI 532: Design and Analysis of Algorithms August 26, 2015
COMPSCI 532: Design and Analysis of Algorithms August 26, 205 Lecture Lecturer: Debmalya Panigrahi Scribe: Allen Xiao Oeriew In this lecture, we will define the maximum flow problem and discuss two algorithms
More informationPhysics 2A Chapter 3 - Motion in Two Dimensions Fall 2017
These notes are seen pages. A quick summary: Projectile motion is simply horizontal motion at constant elocity with ertical motion at constant acceleration. An object moing in a circular path experiences
More informationarxiv: v2 [math.co] 12 Jul 2009
A Proof of the Molecular Conjecture Naoki Katoh and Shin-ichi Tanigawa arxi:0902.02362 [math.co] 12 Jul 2009 Department of Architecture and Architectural Engineering, Kyoto Uniersity, Kyoto Daigaku Katsura,
More informationPrime and irreducible elements of the ring of integers modulo n
Prime and irreducible elements of the ring of integers modulo n M. H. Jafari and A. R. Madadi Department of Pure Mathematics, Faculty of Mathematical Sciences University of Tabriz, Tabriz, Iran Abstract
More informationSTAT 512 sp 2018 Summary Sheet
STAT 5 sp 08 Summary Sheet Karl B. Gregory Spring 08. Transformations of a random variable Let X be a rv with support X and let g be a function mapping X to Y with inverse mapping g (A = {x X : g(x A}
More informationDynamic Vehicle Routing with Heterogeneous Demands
Dynamic Vehicle Routing with Heterogeneous Demands Stephen L. Smith Marco Paone Francesco Bullo Emilio Frazzoli Abstract In this paper we study a ariation of the Dynamic Traeling Repairperson Problem DTRP
More informationTwo-sided bounds for L p -norms of combinations of products of independent random variables
Two-sided bounds for L p -norms of combinations of products of independent random ariables Ewa Damek (based on the joint work with Rafał Latała, Piotr Nayar and Tomasz Tkocz) Wrocław Uniersity, Uniersity
More informationDynamic Vehicle Routing with Moving Demands Part II: High speed demands or low arrival rates
Dynamic Vehicle Routing with Moing Demands Part II: High speed demands or low arrial rates Stephen L. Smith Shaunak D. Bopardikar Francesco Bullo João P. Hespanha Abstract In the companion paper we introduced
More informationTrajectory Estimation for Tactical Ballistic Missiles in Terminal Phase Using On-line Input Estimator
Proc. Natl. Sci. Counc. ROC(A) Vol. 23, No. 5, 1999. pp. 644-653 Trajectory Estimation for Tactical Ballistic Missiles in Terminal Phase Using On-line Input Estimator SOU-CHEN LEE, YU-CHAO HUANG, AND CHENG-YU
More informationDEVIL PHYSICS THE BADDEST CLASS ON CAMPUS AP PHYSICS
DEVIL PHYSICS THE BADDEST CLASS ON CAMPUS AP PHYSICS LSN 7-4: CONSERVATION OF ENERGY AND MOMENTUM IN COLLISIONS LSN 7-5: ELASTIC COLLISIONS IN ONE DIMENSION LSN 7-6: INELASTIC COLLISIONS Questions From
More informationClassical Mechanics NEWTONIAN SYSTEM OF PARTICLES MISN NEWTONIAN SYSTEM OF PARTICLES by C. P. Frahm
MISN-0-494 NEWTONIAN SYSTEM OF PARTICLES Classical Mechanics NEWTONIAN SYSTEM OF PARTICLES by C. P. Frahm 1. Introduction.............................................. 1 2. Procedures................................................
More informationJournal of Computational and Applied Mathematics. New matrix iterative methods for constraint solutions of the matrix
Journal of Computational and Applied Mathematics 35 (010 76 735 Contents lists aailable at ScienceDirect Journal of Computational and Applied Mathematics journal homepage: www.elseier.com/locate/cam New
More informationOn the Entropy of Sums of Bernoulli Random Variables via the Chen-Stein Method
On the Entropy of Sums of Bernoulli Random Variables via the Chen-Stein Method Igal Sason Department of Electrical Engineering Technion - Israel Institute of Technology Haifa 32000, Israel ETH, Zurich,
More informationOn the lower limits of entropy estimation
On the lower limits of entropy estimation Abraham J. Wyner and Dean Foster Department of Statistics, Wharton School, University of Pennsylvania, Philadelphia, PA e-mail: ajw@wharton.upenn.edu foster@wharton.upenn.edu
More informationOnline Companion to Pricing Services Subject to Congestion: Charge Per-Use Fees or Sell Subscriptions?
Online Companion to Pricing Serices Subject to Congestion: Charge Per-Use Fees or Sell Subscriptions? Gérard P. Cachon Pnina Feldman Operations and Information Management, The Wharton School, Uniersity
More informationNON-PARAMETRIC METHODS IN ANALYSIS OF EXPERIMENTAL DATA
NON-PARAMETRIC METHODS IN ANALYSIS OF EXPERIMENTAL DATA Rajender Parsad I.A.S.R.I., Library Aenue, New Delhi 0 0. Introduction In conentional setup the analysis of experimental data is based on the assumptions
More informationORDER STATISTICS, QUANTILES, AND SAMPLE QUANTILES
ORDER STATISTICS, QUANTILES, AND SAMPLE QUANTILES 1. Order statistics Let X 1,...,X n be n real-valued observations. One can always arrangetheminordertogettheorder statisticsx (1) X (2) X (n). SinceX (k)
More information(a) During the first part of the motion, the displacement is x 1 = 40 km and the time interval is t 1 (30 km / h) (80 km) 40 km/h. t. (2.
Chapter 3. Since the trip consists of two parts, let the displacements during first and second parts of the motion be x and x, and the corresponding time interals be t and t, respectiely. Now, because
More informationOn the Front-Tracking Algorithm
JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS 7, 395404 998 ARTICLE NO. AY97575 On the Front-Tracking Algorithm Paolo Baiti S.I.S.S.A., Via Beirut 4, Trieste 3404, Italy and Helge Kristian Jenssen
More informationMANAGEMENT SCIENCE doi /mnsc ec pp. ec1 ec31
MANAGEMENT SCIENCE doi 10.1287/mnsc.1060.0650ec pp. ec1 ec31 e-companion ONLY AVAILABLE IN ELECTRONIC FORM informs 2007 INFORMS Electronic Companion Temporary and Permanent Buyout Prices in Online Auctions
More informationAsymptotic efficiency of simple decisions for the compound decision problem
Asymptotic efficiency of simple decisions for the compound decision problem Eitan Greenshtein and Ya acov Ritov Department of Statistical Sciences Duke University Durham, NC 27708-0251, USA e-mail: eitan.greenshtein@gmail.com
More informationMaximum union-free subfamilies
Maximum union-free subfamilies Jacob Fox Choongbum Lee Benny Sudakov Abstract An old problem of Moser asks: how large of a union-free subfamily does every family of m sets have? A family of sets is called
More information4. A Physical Model for an Electron with Angular Momentum. An Electron in a Bohr Orbit. The Quantum Magnet Resulting from Orbital Motion.
4. A Physical Model for an Electron with Angular Momentum. An Electron in a Bohr Orbit. The Quantum Magnet Resulting from Orbital Motion. We now hae deeloped a ector model that allows the ready isualization
More informationThe Robustness of Stochastic Switching Networks
The Robustness of Stochastic Switching Networks Po-Ling Loh Department of Mathematics California Institute of Technology Pasadena, CA 95 Email: loh@caltech.edu Hongchao Zhou Department of Electrical Engineering
More informationProbabilistic Engineering Design
Probabilistic Engineering Design Chapter One Introduction 1 Introduction This chapter discusses the basics of probabilistic engineering design. Its tutorial-style is designed to allow the reader to quickly
More informationMath Mathematical Notation
Math 160 - Mathematical Notation Purpose One goal in any course is to properly use the language of that subject. Finite Mathematics is no different and may often seem like a foreign language. These notations
More information