Entropy of Some General Plane Trees

Size: px

Start display at page:

Download "Entropy of Some General Plane Trees"

Victoria Sullivan
5 years ago
Views:

1 Etropy of Some Geeral Plae Trees Zbigiew Gołębiewsi Wroclaw Uiversity of Sciece a Techology Wrocław, Pola zbigiewgolebiewsi@pwreupl Abram Mager Uiv of Illiois at Urbaa-Champaig Urbaa, IL, USA amager@illioiseu Wociech Szpaowsi Purue Uiversity West Lafayette, IN, USA spa@cspurueeu Abstract We cotiue evelopig the iformatio theory of avace ata structures I our previous wor, we itrouce structural etropy of ulabele graphs a esige lossless compressio algorithms for biary trees with structure-correlate vertex ames I this paper, we cosier -ary trees a trees with urestricte egree for which we compute the etropy the first step to esig optimal compressio algorithms It turs out that exteig from biary trees to geeral trees is mathematically quite challegig a leas to ew recurreces that fi ample applicatios i the iformatio theory of structures I INTRODUCTION Avaces i sesig, commuicatio, a storage techologies have create a state of the art i which our ability to collect ata from richly istrumete eviromets has far outpace our ability to process, uersta, a aalyze it i a provably rigorous maer A sigificat compoet of this complexity arises from the multimoal a heterogeeous ature of ata This poses sigificat challeges for theoretical characterizatio of limits of iformatio storage a trasmissio a methos that achieve these limits While a hoc approaches are ofte curretly eploye, critical issues regarig their performace, robustess, a scalability remai These challeges have motivate our recet research program [], [4], [8] a others [], [7], [0] It provies the basis for our effort i evelopig a comprehesive theory of iformatio for multimoal a structure ata, that is, multitype a cotext epeet structures As a start to uersta avace ata structures i a iformatio-theoretic settig, we focuse o graphs [] a trees with vertex ames [8] I [] the etropy a a optimal compressio algorithm up to two leaig terms of the etropy for Erős-Réyi graph structures were presete Furthermore, i [9] a automata approach was use to esig a optimal graph compressio scheme For biary plae-oriete trees, rigorous iformatio-theoretic results were obtaie i [7], complemete by a uiversal grammar-base lossless coig scheme [0] I our recet wor [8] see also [4] we stuy biary trees with structure-correlate vertex ames a esig a optimal compressio scheme base o arithmetic ecoig I this paper, we exte our stuy o etropy of avace ata structures to -ary trees ie, trees with egree a geeral trees without ay restrictio o the egree It turs out that movig from biary trees to -ary geeral trees is mathematically quite challegig First of all, i [8] we prove for biary trees a equivalece betwee two moels: the biary search tree moel a a moel i which leaves are selecte raomly to expa the tree by aig two aitioal oes ew leaves This equivalece allowe us to aalyze the etropy of such trees by solvig a relatively simple recurrece, amely x a x i i for some give a eg, for the etropy, a log, where eotes the umber of iteral oes owever, for -ary trees T o iteral oes the etropy T satisfies T root T p, where root is the etropy of the split probability at the root, a p, is the probability of oe specifie subtree beig of size For the m-ary search tree moel iscusse i Sectio II, this recurrece ca be hale by results from [], [6] I a more iterestig -ary tree moel, we raomly select a leaf a a exactly leaves to it We stuie this moel previously i the special case of, but the aalysis is more complicate whe > After some teious algebra, we prove i Sectio III that the ew type of recurrece we ee to solve to fi the etropy is of the followig form see Lemma : x a α! Γ α Γ α x! where α /, a is give sequece, a Γ is the Euler gamma fuctio The situatio is eve more ivolve whe we cosier geeral trees i Sectio III-C where o restrictios o egrees are impose We preset our mai results i Sectio III We first provie i Corollary the etropy rate for m-ary search trees The we cosier -ary recursive also calle icreasig trees a i Theorem give our expressio for the etropy of such trees We exte it to geeral recursive trees i Theorem II MODELS I this sectio we escribe the cocepts of ulabele plae trees with a without restrictios o the oes out-egrees This will allow us to itrouce three moels of tree geeratio

2 A Ulabele m-ary Search Tree Geeratio Search trees are plae trees built from a set of istict eys tae from some totally orere set, for istace a raom permutatio of the umbers {,,, } A m-ary search tree is a tree i which each oe has at most m chilre; moreover, each oe stores up to m eys We efie the size of a search tree as the umber of eys The costructio of m-ary search tree ca be escribe as follows [5] If 0 the tree is empty If m the tree cosists of a root oly, with all eys store i the root If m we select m eys that are calle pivots The pivots are store i the root The m pivots split the set of remaiig m eys ito m sublists I,, I m : if the pivots are p < p < < p m, the I : p i : p i < p, I : p i : p < p i < p,, I m : p i : p m < p i We the recursively costruct a search tree for each of the sets I i of eys I orer to obtai a ulabele search tree of size we remove the eys from a search tree B Ulabele -ary Recursive Plae Tree Geeratio We cosier the followig geeratio moel of a ulabele -ary recursive plae tree Suppose that the process starts with a empty tree, that is with ust a exteral oe leaf The first step i the growth process is to replace this exteral oe by a iteral oe with successors that are exteral oes see Figure The with probability each, oe of these exteral oes is selecte a agai replace by a iteral oe with successors At the e, we remove the labels which escribe the history of tree evolutio from iteral oes C Ulabele Geeral Recursive Plae Trees Geeratio We cosier the followig geeratio moel of ulabele plae trees Suppose that the process starts with the root oe carryig a label The we a a oe with label to the root The ext step is to attach a oe with label owever, there are three possibilities: either to a it to the root as a left or right siblig of or to the oe with label Similarly oe procees further Now if a oe alreay has out-egree where the esceats are orere, the there are possible ways to a ew oe this time we o ot istiguish betwee exteral a iteral oes ece, if a plae tree alreay has oes the there are precisely possibilities to attach the th oe see Figure More precisely, the probability of choosig a oe of out-egree equals / At the e, we remove the labels from iteral oes of a tree III MAIN RESULTS I this sectio we preset our mai results I particular, we briefly aress the etropy of m-ary search trees The we preset our erivatio of the the etropy of ulabele -ary recursive trees a geeral trees We shoul poit out that i all our moels, the probability of a tree geeratio is o-uiform, a root subtrees are coitioally iepeet give their respective sizes Iee, let T be a raom variable represetig a tree t a c e 4 b Fig : Labele a ulabele -ary trees of size 4 o iteral oes Assume ow that at the root we split t ito subtrees of size,,, respectively, where The the probability P T t of geeratig tree t i all our moels satisfies P T t P,, i P T i t i where P,, is the probability of a split at the root of iteral oes ito subtrees t,, t, respectively This split probability is ifferet for m-ary search trees, -ary trees, a geeral trees, as we shall see i this sectio Throughout we shall use the followig otatio Let,, eote a -imesioal vector of o-egative itegers a m be its L orm Let,,, eote a -imesioal vector with the first cooriate equal to We ofte write istea of A The Etropy of the Ulabele m-ary Search Trees Let U eote a raom ulabele m-ary search tree with eys, geerate accorig to the process escribe earlier We write u for a arbitrary fixe m-ary ulabele search tree with eys We escribe the splittig of eys at the root of the search tree by the raom vector Y m Y,,, Y,m, where Y, I is the umber of eys that go ito the th subtree of the root If m we have Y, Y,m m a P Y m m / m f

3 a c e 4 5 b 4 Fig : Labele a ulabele geeral trees of size 5 Suppose that the tree u has subtrees u,, u m of sizes,, m, the P U u P Y m f m m P U u If 0 we have a empty tree, a U 0 0 Moreover, if m, all eys are store i oe oe, a U 0 For > m, we have U m leaig to U Y m, U Y,,, U Y,m Y m U Y,,, U Y,m Y m m P Y m m Y m m m U P Y, For m a m, the raom variables Y, are ietically istribute, a for 0, P Y, m see [5] ece m log m leaig to the followig recurrece U log m m m m Y m U m The asymptotics of a recurrece lie this oe have bee stuie before; see Theorem 4 i [6] ece, the etropy of the m-ary search tree becomes U c m o where log m c m m, 0 where m is the mth harmoic umber Corollary The etropy rate h m lim U / of the ulabele m-ary trees, is give by log m h m m 0 B The Etropy of the Ulabele -ary Plae Recursive Trees Let F be the set of -ary plae trees with iteral oes, a let G be the set of -ary pla recursive trees with exactly iteral oes By g G we eote the umber of -ary plae recursive trees with iteral oes From [5] we ow that for we have g! Moreover, for > we have Γ g Γ Let G f eote the subset of G of trees that have the same structure as the ulabele tree f F ; that is, G f is the set of labele represetatives of f Moreover, let g f G f be the umber of -ary plae recursive trees that have the same structure as a tree f Observe that P F f gf g Suppose that the tree f has subtrees f,, f of sizes,, The P F f g f 4 g,, g g P F f,, g g g Observe that,, g is the probability that the subtrees of the root are of sizes,, Let us efie a raom vector V : G {0,, } whose th compoet V, eotes the size of the th subtree For we have V, V, a P V g g 5,, g The etropy of ulabele -ary plae recursive trees of size is efie as F f F P F f log P F f If 0 we have a empty tree, a F 0 0 If, we have oe fixe tree a F 0 By 4 for > there is a biectio betwee a tree F a a tuple V, F V,, F V Therefore, for >, we have F V, F V,,, F V, V F,, F P V

4 Sice subtrees F,, F are coitioally iepeet give their sizes, we have F V F P V, For 0,,, let p, be the probability that oe specifie subtree i a -ary recursive tree is of size, that is, p, P V, The we have the followig recurrece F V F p, 6 We ca prove the followig lemma Lemma For 0,, a >, α p, α!γ α!γ α, the I the lemma below we propose a geeral solutio for recurreces of the form 6 Lemma For costat α, x 0 a x, the recurrece x a α! Γ α has the followig solutio for : x a α α α α Γ α x,! 7 x x 0 α a α α Proof Multiply both sies of the recurrece by the ormalizig factor Γα! Defie also ˆx xγα! a â aγα! The ˆx â α ˆx 8 To solve the recurrece 8 we compute ˆx ˆx This leas us to ˆx â â α ˆx, which hols for The after iteratig the above we arrive at ˆx ˆx α â â α 9 The prouct α!γα!γα, a after some staar calculatios we obtai Γ α ˆx â ˆx â Γα! Γ α!! α â Γ α α Goig bac from ˆx a â to x, a, respectively, we obtai a x a α α α α x a α α But x a x x0 α which completes the proof This leas us to out first mai result Theorem The etropy of a ulabele -ary plae tree is V F V αα α α, where α a V 0 P V log P V Furthermore, the etropy rate h lim F / is V h α α α C The Etropy of the Ulabele Geeral Plae Trees Let T be the set all ulabele trees with iteral oes of ay egree, while T be the subset of T cosistig of all trees that cotai exactly oes a have root egree equal to Fially, by R we eote a set of labele plae trees oriete recursive trees with exactly oes Let r R From [5] we ow that there are r!!! ifferet labele plae oriete recursive trees of size As i the case of the -ary plae recursive trees, let R t eote the subset of trees i R that have the same structure as a give ulabele tree t T ie, R t is the set of labele represetatives of t ; moreover, let r t R t be the umber of such trees Observe that P T t rt r Let D eote the raom variable represetig the umber of subtrees of the root Observe that P D r r, where r R is the umber of plae recursive trees with root egree equal to Suppose that the tree t has subtrees t,, t of sizes,, The P T t P D P T t D r r P T t,, r

5 Observe that r r,, r is the probability that the root of a plae recursive tree of size has egree equal to a the root s subtrees are of sizes,, Let W : R {,, }, where its th compoet W, eotes the size of the th subtree whe the root is of egree For we have W, W, a P D P W D D D r r,, r For the etropy of ulabele plae recursive trees of size, usig the coitioal iepeece of T,, T, we have T W P D T P D P D W, For,,, let q, q, be efie as the probability that the root of a plae recursive tree has egree a that oe specifie root subtree is of size The q, P D P W, leaig to T W D P D 4 T q, 5 We ee a expressio for the probability q, which we preset i the ext lemma Lemma For,, we have q, if : q, 0, while for > : q, a The recurrece 5 is aother recurrece that we ee to aalyze Its geeral solutio is presete ext The proof ca be fou i the oural versio of the paper Lemma 4 For costat y a y, the recurrece y b has the followig solutio for > : y b b q, y, > 6 b This leas us to our seco mai result Theorem The etropy of a ulabele geeral plae tree is T D P D 7 where W W D W D log P W D P D, 8 P W D Furthermore, the etropy rate h t lim T / is h t W D P D 9 Fially, we i ot aress how to optimally compress these trees, but it is ot har to see that a irect geeralizatio of the arithmetic ecoig propose i [8] ca be use For example, i the -ary tree cases, we traverse the tree i epth-first orer from left to right, taig avatage of the fact that coitioe o the size of the leftmost root subtree, the rest of the tree is a raom -ary tree of size ACKNOWLEDGMENT This wor was supporte by NSF Ceter for Sciece of Iformatio CSoI Grat CCF-09970, by NSF Grat CCF-54, a NI Grat U0CA Z Gołębiewsi was i aitio supporte by Polish NCN grat 0/09/B/ST6/058 REFERENCES [] D Alous a N Ross, Etropy of Some Moels of Sparse Raom Graphs With Vertex-Names Probability i the Egieerig a Iformatioal Scieces, 04, 8:45-68 [] ua-uai Cher a sie-kuei wag Phase chages i raom m- ary search trees a geeralize quicsort Raom Struct Algorithms, 9-4:6 58, 00 [] Y Choi, W Szpaowsi: Compressio of Graphical Structures: Fuametal Limits, Algorithms, a Experimets IEEE Trasactios o Iformatio Theory, 0, 58:60-68 [4] J Cicho, A Mager, W Szpaowsi, K Turowsi, O symmetries of o-plae trees i a o-uiform moel, ANALCO, Barceloa, 07 [5] M Drmota Raom Trees, A Iterplay betwee Combiatorics a Probability Spriger-Verlag Wie, 009 [6] J Fill a N Kapur Trasfer theorems a asymptotic istributioal results for m-ary search trees Raom Structures & Algorithms, 64:59 9, 005 [7] J C Kieffer, E- Yag, W Szpaowsi, Structural complexity of raom biary trees ISIT 009, pp [8] A Mager, W Szpaowsi, K Turowsi, Lossless Compressio of Biary Trees with Correlate Vertex Names, ISIT 6, Barceloa, 06 [9] M Mohri, M Riley, A T Suresh, Automata a graph compressio ISIT 05, pp [0] J Zhag, E- Yag, J C Kieffer, A Uiversal Grammar-Base Coe for Lossless Compressio of Biary Trees IEEE Trasactios o Iformatio Theory, 04, 60:7-86

Sparsification using Regular and Weighted. Graphs

Sparsification using Regular and Weighted. Graphs Sparsificatio usig Regular a Weighte 1 Graphs Aly El Gamal ECE Departmet a Cooriate Sciece Laboratory Uiversity of Illiois at Urbaa-Champaig Abstract We review the state of the art results o spectral approximatio